Call OnOutboundRouteConfiguration from gateway

This adds missing callbacks for gateway. These are already called from sidecar.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: ostromart
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/pkg/networking/OWNERS [ostromart]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: ostromart, rshriram
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/pkg/networking/OWNERS [ostromart,rshriram]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: kyessenov, ostromart, rshriram
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/pkg/networking/OWNERS [kyessenov,ostromart,rshriram]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ostromart: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e

prow/e2e-bookInfoTests.sh
938d362dc3cd297927862ca709074d4301335118
link
/test e2e-bookInfo

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e

prow/e2e-bookInfoTests.sh
938d362dc3cd297927862ca709074d4301335118
link
/test e2e-bookInfo

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests.sh
938d362dc3cd297927862ca709074d4301335118
link
/test e2e-bookInfo

prow/istio-pilot-e2e.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests.sh
938d362dc3cd297927862ca709074d4301335118
link
/test e2e-bookInfo

prow/istio-pilot-e2e.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e

prow/e2e-bookInfoTests.sh
938d362dc3cd297927862ca709074d4301335118
link
/test e2e-bookInfo

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e

prow/e2e-bookInfoTests.sh
938d362dc3cd297927862ca709074d4301335118
link
/test e2e-bookInfo

prow/istio-pilot-e2e-envoyv2-v1alpha3.sh
938d362dc3cd297927862ca709074d4301335118
link
/test istio-pilot-e2e-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Fix Flaky Tests

Signed-off-by: Aditya Jindal<EMAIL_ADDRESS>Issue #, if available:
Description of changes:
CheckList:
[ ] Commits are signed per the DCO using --signoff
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #319 (707fa1a) into main (5f18a5c) will decrease coverage by 0.07%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main     #319      +/-   ##
============================================
- Coverage     78.59%   78.52%   -0.08%
+ Complexity      218      217       -1
============================================
  Files           173      173
  Lines          6975     6975
  Branches        916      916
============================================
- Hits           5482     5477       -5
- Misses         1005     1007       +2
- Partials        488      491       +3

Impacted Files
Coverage Δ

...ing/destination/client/DestinationEmailClient.java
72.50% <0.00%> (-5.00%)
:arrow_down:

...ing/model/destination/DestinationContextFactory.kt
66.66% <0.00%> (-3.71%)
:arrow_down:

.../kotlin/org/opensearch/alerting/core/JobSweeper.kt
71.50% <0.00%> (-0.50%)
:arrow_down:

...ain/kotlin/org/opensearch/alerting/AlertService.kt
78.53% <0.00%> (-0.49%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f18a5c...707fa1a. Read the comment docs.


timestamp opt (20-25% speed increase)

When we profile an isolated benchmark with http://npm.im/0x (again) we can see that toISOString is hot (10.8% on stack 8.72% on top of stack)

If we alter out timestamps from ISO to epoch, we get the following flamegraph showing  that Date.now() is far less expensive that Date.prototype.toISOString

Removing the bottleneck gives us a 20-25% perf increase:
benchPino10000: 369.998ms
benchPino210000: 278.406ms
benchPinoObj10000: 431.747ms
benchPinoObj210000: 347.675ms
benchPino10000: 344.995ms
benchPino210000: 272.239ms
benchPinoObj10000: 420.123ms
benchPinoObj210000: 340.240ms
We can keep consistent log output formatting via the pino command, by applying the toISOString call at the point of transformation.

oh - also added CLI usage

we should make this configurable via a CLI option.

My contention is we should provide the most optimal approach to logging
We're positioning pino as the fastest logger, so do we really want to add an option that slows it down?

ok after discussion we decided to go with an option with an ominous name "slowtime", a warning in the readme, and the ability to transform json logs with the pino command line tool using the -t flag.


[Azure driver] Allow custom DNS (cloud service) names

Recently the --azure-name parameter got removed (#219) from machine code and azure driver code now creates the Azure Cloud Service (DNS to the VM) in machinename-YYYYMMDDHHmmss.cloudapp.net format:

Each driver will attempt to set the hostname for the VM, except for Azure, which sets it based on the DNS name.

This is done at: https://github.com/ehazlett/machine/commit/861b16dba9020ae6cb3c62e699683db9554300cc#diff-93a074128ed8940ea0dafbd000270978R170
I think this should not be done and user should be allowed to specify the DNS names (as it used to be). In my case, I want to spin up 10 different VMs like mydocker-1, mydocker-2... on Azure and I will be able to easily address them like ssh -p 22 mydocker-1.cloudapp.net. However this change gives me VM names like mydocker-1-20150127152433, mydocker-2-20150127152750 which are not easily readable and manageable.
Another case is, if I want to deploy a VM like ahmetsblog.cloudapp.net, there is no way to do this with docker-machine. I think just limits the flexibility of the user.
I see that the date suffix is added to prevent the chances that the DNS name is already registered. I think the solution here should be giving the --azure-name parameter back to the user so that users can have a better control on what's going on. If the DNS name is already taken, user can simply be alerted "Cloud service name is already taken". (I think it used to be like that before #219) If it's really tedious and redundant, code can allow that parameter to be optional and can still use fmt.Sprintf("%s-%s", driver.MachineName, date).
Also, this decision forces MachineName to comply with Azure Cloud Service Name rules (e.g. 3-25 chars and can contain only chars allowed in DNS). The date suffix already takes up 14 chars, so user has to limit their MachineName to 11 chars max. That sounds wrong.
I'm not sure how the decision is made here but here's my 2c.
cc: @jeffmendoza @ehazlett @sthulb @chanezon
followup to #419.

Thanks for raising this.  Afaik there has never been an alert but I think that's the way to go.   (the names used to be dynamically generated with only the cloud service and DNS being named -- the "name" parameter had no effect on the machine name).  We recently added this functionality to the EC2 driver to check for the same key pair name.  I think we can do the same and query for an existing cloud service and/or DNS lookup (maybe?) to see if the name is taken.

Yes, you can query that if a cloud service name exists, has a bad format or contains reserved/profane words. https://msdn.microsoft.com/en-us/library/azure/jj154116.aspx
I'm not sure how it works with ec2 (or other drivers) in terms of if the machine name also serves as DNS name but in Azure, I can foresee that this would be a big limitation and people would come and ask for this as soon as it's released.

None of the other providers mess with DNS (with the exception of EC2 in which you get the instance IP instead).  Thanks for the link -- I think we can use that in the PreCreate check to make sure there are no collisions.
Thanks!

It will be pretty much the same (checking DNS availability vs getting error when you attempt to create the Cloud Service). Both will return the same error and user of docker-machine will see an human-readable error.
It's done here: https://github.com/MSOpenTech/azure-sdk-for-go/blob/005406ef924d801298dbf035c87272f3b4bd44b5/clients/vmClient/vmClient.go#L67

We should definitely allow for nice hostnames. And obviously @ahmetalpbalkan is right about the API.
I chose to append the date since it was easier at the time.

Unfortunately the azure driver is broken now. New machines are still created, but they are now only available as CHANGE_ME.datacenter.cloudapp.azure.com. Fiddling around with the config.json does not help, the client always adds cloudapp.net to the connection…
see https://github.com/docker/machine/issues/3730
Any tips? :disappointed:

@JohannesHoppe this issue you commented is about the very old azure driver, so your comment is not relevant here.

You are right. Going to delete it, since it's irrelevant!


AWS CLI example on wiki not working for me

I follow the steps shown at https://github.com/andrewgaul/s3proxy/wiki/Client-compatibility-list and I get this:
:.aws $ aws configure
AWS Access Key ID [****************tity]: local-identity
AWS Secret Access Key [****************tial]: local-credential
Default region name [us-east-1]: None
Default output format [None]:
:.aws $ aws s3 ls --endpoint-url http://<IP_ADDRESS>:8080

A client error (InvalidArgument) occurred when calling the ListBuckets operation: Bad Request

:.aws $ aws --version
aws-cli/1.9.15 Python/2.7.6 Darwin/15.0.0 botocore/1.3.15

Am I doing something wrong, or is this a bug?

The above happened with 1.4.0
When I try with latest code, it works

Duplicate of #104 and fixed in master via d8c73bc86d30e885e743eb7afcd41cf0b9e9497b.


Restore native targets for agent

What's done:

Restore native targets for agent

As discussed, it doesn't make sense to compile mingw target, because agent isn't supposed for execution on Windows machine, it will be immediately packed into docker image by orchestrator and executed there.
However, we indeed can fix compilation from windows for linux, because the only thing missing is libcurl.so. I tried a small experiment and was able to succesfully link exeuctable by placing libcurl.so into .konan\dependencies\x86_64-unknown-linux-gnu-gcc-8.3.0-glibc-2.19-kernel-4.9-2\x86_64-unknown-linux-gnu\sysroot\usr\lib. It could be relatively easy automated by gradle by creating a download task (from http://security.ubuntu.com/ubuntu/pool/main/c/curl/libcurl4_7.58.0-2ubuntu3.14_amd64.deb) and extract it using standard Copy task and copy libcurl.so.4.5.0 into the above path.

Also, to fix orchestrator tests on windows we need the following:

Use correct orchestrator.docker.host on windows (tcp://localhost:2375)
Don't use runsc in tests on windows

Unfortunately, gradle supports unpacking only of tar and zip files by tarTree and zipTree correspondingly.
But extension deb is actually ar format, which is not supported by gradle
However, in java it is possible to use commons compress, to unpack such files. java code could be created in buildSrc dir, and called from build.gradle of save-agent


Update samples?

Is this quick start STILL a good example of how to acquire a token to all an API with?
I have a razor app that is secured with ADB2C and want to call an API secured with ADB2C. But this code base seems a bit dated. It still has Global.asax and web.config. This isn't great documentation and/or samples for new developers who are breaking into .NET... these samples should be following .NET Core 2+ best practices.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 976c73ce-6915-43a0-d862-2c05465341f8
Version Independent ID: f4f562c1-8a77-4e36-4815-14533492e52d
Content: Quickstart - Set up sign in for an ASP.NET application using Azure Active Directory B2C
Content Source: articles/active-directory-b2c/active-directory-b2c-quickstarts-web-app.md
Service: active-directory
Sub-service: b2c
GitHub Login: @mmacy
Microsoft Alias: marsma

@aszalacinski  Thank you for your feedback. We will investigate and update the document.

@mmacy
Could this document be checked for updated samples ?


Learn more in Automatic online backup and restore with Azure Cosmos DB - incorrect URL

Currently Learn more in Automatic online backup and restore with Azure Cosmos DB links to
https://docs.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/backup-and-restore
which appears to be incorrect.
URL should probably be https://docs.microsoft.com/en-us/azure/cosmos-db/online-backup-and-restore with additional guidance for long-term retention policies and recovery steps.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6b791922-4cff-59b3-ea11-e13d5c55ffe4
Version Independent ID: d1e2e03c-ad04-f570-b122-c0382ae4bf65
Content: Database security - Azure Cosmos DB
Content Source: articles/cosmos-db/database-security.md
Service: cosmos-db
GitHub Login: @markjbrown
Microsoft Alias: mjbrown

@asears Thank you for bringing this to our attention. I have submitted a pull request to have the public facing instance of this document updated. It should be visible within the next 24hrs.
We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.


"Double" encoded headers

Just wanted to share what I just encountered in the wild (German company, this time ;-)

Let's start with our well-known foobär
Now quoted-printable encode it: =?utf-8?Q?foob=C3=A4r?=
Now add some more text: Re: =?utf-8?Q?foob=C3=A4r?= one two three
Ready for the fun part? base64-encode it!!: =?utf-8?b?UmU6ID0/dXRmLTg/UT9mb29iPUMzPUE0cj89IG9uZSB0d28gdGhyZWU=?=

There you go:
Subject: =?utf-8?b?UmU6ID0/dXRmLTg/UT9mb29iPUMzPUE0cj89IG9uZSB0d28gdGhyZWU=?=

Your $message->getHeader('subject')->getValue() returns:
Re: =?utf-8?Q?foob=C3=A4r?= one two three

But php-mime-mail-parser's $parser->getHeader('subject') just decodes it:
Re: foobär one two three

How does Thunderbird handle it?
php-mime-mail-parser's ability to handle it might be a side-effect of how it's handling headers generally, and may not be intentional.  Anyway, as discussed elsewhere it's not a gauge as its goals are different, etc... etc...

Thunderbird's email window shows the same as you:
Re: =?utf-8?Q?foob=C3=A4r?= one two three

In the overview window (i.e. list of emails) I haven't checked.

Yeah, in addition having "=?utf-8?Q?foobar?= in a subject is valid... I'm allowed to discuss mime headers in an email's subject, aren't I? :tea: :stuck_out_tongue_closed_eyes:

So you're saying that having a character sequence like =?utf-8?.... in the subject is not illegal. But the correct behavior is to just encode the "outer" encoding (and not look for further stuff to decode inside of it)?
I'm closing this, since it's nothing you should change.

Correct :)


Add label for arm64 windows machines

@jashook Is this for build machines or test machines?

Test

It is a pre-req to drop the arm64 internal ci

Safe to ignore failures?

@jashook Yeah.  The "CI Tests" legs only work if you pushed to upstream (security)

That doesnt make sense to me but I can bug you about it later.

Thanks


[OpenEXR] add missing underscore for OpenEXR_IEXMATH_LIBRARY_DEBUG NAMES

fix missing underscore causing the debug library not to be found.
Laurens.

@LaurensVoerman thanks for this PR.
Could you please also update the Version filed in the CONTROL file?
I noticed that this port failed on x64-windows, x86-windows, x64-windows-static trilets due to ogre and alembic.
Could you please take a look and try to fix it ?

I have updated the version number, and added a patch for the cmake error in alembic. The problem with ogre is not changed, this seems to be a conflict between ogre and ogre-next, and not something to do with this PR.
Regards, Laurens.

Thanks for your update and clarify.
About the conflict between ogre and ogre-next, could you update ci.baseline.txt to this PR?
You can add the following lines to ci.baseline.txt file.
# Conflicts with ogre-next
ogre:x64-windows        = skip
ogre:x64-windows-static = skip
ogre:x86-windows        = skip

After a failed search for anything actually using IexMath_2_3.dll I wrote an issue anyway:
https://github.com/microsoft/vcpkg/issues/9283
Maybe the issue can be fixed by not building IexMath_2_3 at all, and updating the cmakefiles in dependent projects to not require it, but the current PR seems safer.
Regards, Laurens.

@LaurensVoerman Thanks for the PR. The ogre-next conflict does not seem related to this port. Currently, we are making changes to the MacOS and Windows CI pipeline so some unrelated failures may happen now.

/azp run

@LaurensVoerman
As some ports depend on ogre, we should set all triplets results of ogre-next instead of ogre to skip in the baseline.
Sorry for my mistake.
Could you please update ci.baseline.txt again?

@NancyLi1013
I have switched to skipping ogre-next in ci.baseline.txt instead of skipping ogre.
searching through all CONTROL files revealed opencv3[ovis] and opencv4[ovis] depend on ogre,
nothing depends on ogre-next.
Laurens.


Resource validation for --reconcile-resource-resync-seconds arguments

As discussed in https://github.com/aws-controllers-k8s/runtime/pull/106#discussion_r1082689001 - we should make code-generator changes to be able to validate the resource names provided by ACK users.

This is a direct follow-up to https://github.com/aws-controllers-k8s/community/issues/1367


Cannot convert langchain's ReadableStream to node ReadableStream

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain.js documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain.js rather than my code.
[ ] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
import { RunnableSequence } from "@langchain/core/runnables";
import { ReadableStream } from "node:stream/web";

let answer_stream = ReadableStream.from(["default", "fallback", "stream"]);
if (req.app.locals.llm_chain) {
  const llm_chain: RunnableSequence<string, string> = req.app.locals.llm_chain;
  answer_stream = await llm_chain.stream("Sample question")
}

Error Message and Stack Trace (if applicable)
Property 'values' is missing in type 'IterableReadableStream<string>' but required in type 'ReadableStream<string>'.ts(2741) web.d.ts(154, 9): 'values' is declared here.
Description
In my business logic, I am trying to extract a stream from langchain, but also have a node ReadableStream as backup in certain cases.
However, it seems langchain's ReadableStream coming from lib.dom.d.ts in typescript/lib is not compatible with node's ReadableStream coming from web.d.ts in node:stream/web . It's missing values method.
Here's the ReadableStream from node:stream/web
 /** This Streams API interface represents a readable stream of byte data. */
interface ReadableStream<R = any> {
        readonly locked: boolean;
        cancel(reason?: any): Promise<void>;
        getReader(): ReadableStreamDefaultReader<R>;
        getReader(options: { mode: "byob" }): ReadableStreamBYOBReader;
        pipeThrough<T>(transform: ReadableWritablePair<T, R>, options?: StreamPipeOptions): ReadableStream<T>;
        pipeTo(destination: WritableStream<R>, options?: StreamPipeOptions): Promise<void>;
        tee(): [ReadableStream<R>, ReadableStream<R>];
        values(options?: { preventCancel?: boolean }): AsyncIterableIterator<R>;
        [Symbol.asyncIterator](): AsyncIterableIterator<R>;
    }

And here's  ReadableStream  from typescript/lib which langchain depends on -
/**
 * This Streams API interface represents a readable stream of byte data. The Fetch API offers a concrete instance of a ReadableStream through the body property of a Response object.
 *
 * [MDN Reference](https://developer.mozilla.org/docs/Web/API/ReadableStream)
 */
interface ReadableStream<R = any> {
    /** [MDN Reference](https://developer.mozilla.org/docs/Web/API/ReadableStream/locked) */
    readonly locked: boolean;
    /** [MDN Reference](https://developer.mozilla.org/docs/Web/API/ReadableStream/cancel) */
    cancel(reason?: any): Promise<void>;
    /** [MDN Reference](https://developer.mozilla.org/docs/Web/API/ReadableStream/getReader) */
    getReader(options: { mode: "byob" }): ReadableStreamBYOBReader;
    getReader(): ReadableStreamDefaultReader<R>;
    getReader(options?: ReadableStreamGetReaderOptions): ReadableStreamReader<R>;
    /** [MDN Reference](https://developer.mozilla.org/docs/Web/API/ReadableStream/pipeThrough) */
    pipeThrough<T>(transform: ReadableWritablePair<T, R>, options?: StreamPipeOptions): ReadableStream<T>;
    /** [MDN Reference](https://developer.mozilla.org/docs/Web/API/ReadableStream/pipeTo) */
    pipeTo(destination: WritableStream<R>, options?: StreamPipeOptions): Promise<void>;
    /** [MDN Reference](https://developer.mozilla.org/docs/Web/API/ReadableStream/tee) */
    tee(): [ReadableStream<R>, ReadableStream<R>];
}

System Info
node -v     v20.12.1
Windows

Thanks for the heads up - will have a look as soon as possible

Having some trouble figuring this out :( does it work if you just type it as any?

Thanks for taking a look.
It's both an issue with type conversion and missing support for .values() method.
See the stream.values({ preventCancel: true }) example in https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream, without values() we can't read the stream in 2 separate for loops. In my use case, I need to read first k characters in a loop, do some processing on it and then continue reading rest of stream. Currently I'm implementing this in a while loop using .read() on a reader instance obtained through getReader().
The type conversion I was able to workaround by typing it as any.


Add broadcast status to DB

This is a breaking change which adds the field sent to the candidates table. Set to False by default, updated to true after the cuts and broadcasting to IPAC.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

3 of 3 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.004%) to 84.65%

Totals

Change from base Build<PHONE_NUMBER>:
0.004%

Covered Lines:
10952

Relevant Lines:
12938

💛  - Coveralls


Need to make "standard" C template

Suppose we have the following C code:
int add(int a, int b){     return a + b; }
Then, on the Haskell end we'll have something like:
cAdd :: Fix CInt -> Fix CInt -> (CInt -> CInt -> CInt)
cAddTemplate CInt "add" [("int", "a"), ("int", "b")] "return a + b;"
But this is not enough to compile/run/call from Haskell. We need something like:
int main(){      int (*add_ptr)(int, int) = add;     send_to_haskell( fun_ptr );     pause_indefinately();     ...
Thus this issue will be solved when an interface generator for C to Haskell is written.

Since pipes are no longer being used (except maybe for compilation alone), I'm not sure how relevant this is.
For one, the Haskell FFI supports returning a function pointer and the (non-recompiled) C code is well-integrated into Haskell.
See Parse.Templates for the current template, which, at the moment, is:
data CFunctionTemplate = CFunTempl {returnType   :: T.Text,                                     functionName :: T.Text,                                     inputVars    :: [(T.Text, T.Text)],                                     functionBody :: T.Text                                    } deriving (Eq, Ord, Show)


Switch Metaphysics query to use POST

To match https://github.com/artsy/emission/issues/455

This is becoming non-trivial – our networking library is URL form-encoding the JSON string, so we get query=JSON_BLOB as an HTTP body instead of {query: JSON_BLOB}. Could use some help here, I don't want to add an additional AFNetworking session manager just to JSON-encode one request. Advice welcome /cc @alloy.

Yay thank you!


Pipeline Editor should inherit assets (files) output from upstream stages without re-defining them

Is your feature request related to a problem? Please describe.
I'm developing a pipeline, some stages acting on a csv, some on image data, so if stage x defines metadata.csv as output, stage x+1 doesn't stage x+2 can't read metadata.csv because it is not explicitly declared so in stage x+1
Describe the solution you'd like
Let's continue the example with metadata.csv: If stage x defines it as output, stage x+1 not, it should be available for stage x+2
Describe alternatives you've considered
Otherwise on has to keep track of each and every file any upstream stage might define as output

Can you please provide an example pipeline. What you are describing should already work.

Used the following pipeline to verify that this works as expected:

download and extract archive
start JL
open pipeline and run on KFP
producer_consumer.tar.gz

Hi @ptitzler thanks for getting back to me. I confirm that your pipeline works as designed, but mine not.
I'm defining metadata.csv as output in the filter stage here:

But in a downstream operator metric-aix360-lime.ipynb I can't access it:

@ptitzler you can try with
https://github.com/romeokienzler/component-library/tree/elyra1407
and https://github.com/romeokienzler/examples/blob/master/pipelines/claimed_covid_ct_trusted_ai/covid19-trustedai.pipeline

I was able to recreate the behavior using the original pipeline and a simplified version.

To recreate:

download and extract pipelines.tar.gz into a work directory
cd into the work directory
launch jupyterlab
in the file browser navigate to pipelines/claimed_covid_ct_trusted_ai/
open simple-covid19-trustedai.pipeline
run pipeline on KFP
the last node should fail with a 404 because metadata.csv was not found

A simpler version ...
pipelines.tar.gz

Should be resolved in 2.1.0rc5

Thanks @ptitzler @lresende I confirm it is resolved in 2.1.0rc5


[:fix] add mobile option, fix useOnClickOutside hook, hover disabled on touchdevices

add mobile option, fix useOnClickOutside hook, hover disabled on touchdevices
Description

No extra tap required to open push overview.
Refactored useOnClickOutside.
Some hover states disabled on mobile

Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Draft PR (breaking/non-breaking change which needs more work for having a proper functionality [Mark this PR as ready to review only when completely ready])
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Fixes/Resolves (Optional)
Resolves https://github.com/WalletConnect/web3inbox/issues/154
Examples/Screenshots (Optional)
Please test on a mobile device:
https://web3inbox-dev-hidden-git-fix-push-tap-walletconnect1.vercel.app/
Checklist:

[x] I have performed a self-review of my own code
[x] My changes generate no new warnings
[x] Any dependent changes have been merged and published in downstream modules

@devceline
Can you please review again?

I recovered the yarn.lock file
Adjusted the clear/unread animation on mobile
Small UX improvements desktop/mobile notifications settings toggle


Provider produces error on crd kupoports

If categories = [], then we can omit the categories parameter out. That won't trigger the error in Kubernetes provider
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to module.ext_cardano_kupo_crds["global"].kubernetes_manifest.customresourcedefinition_kupoports_demeter_run, provider "provider[\"registry.terraform.io/hashicorp/kubernetes\"]" produced an unexpected new value: .object.spec.names.categories: was
│ cty.ListValEmpty(cty.String), but now null.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

@gonzalezzfelipe is it related to this issue #69?

Hey @verbotenj thanks. @gonzalezzfelipe has already solved this problem in issue #69


Coverage report posted to codecov

What does this PR do?
Post nyc code coverage report to https://app.codecov.io/, which can be referenced by perf dashboard.
What issues does this PR fix or reference?

Welcome to Codecov :tada:
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
Thanks for integrating Codecov - We've got you covered :open_umbrella:


Fix clone method to generate a new ID

This PR ensures that a new ID is given to cloned images in order to avoid ID based conflicts.
A new unit test has been added for NVImage as well that will make sure this issue doesn't happen in the future.
List of fixed issues (if they exist):

closes #945

This is a trivial fix, so merging now (without reviewer approval)


Increment versions for scvmm releases

Increment package versions for scvmm releases

API change check
API changes are not detected in this pull request.


arm64 support

I have a couple of arm64 systems (Pi 400, Pi 4, and a server-class Ampere Altra) that I'd be interested in adding to an innernet network. If the project built binaries I'd test them, ditto with make targets.

First attempt to build with cargo build on the Pi 4 yields this error message:
error: failed to run custom build command for `clang-sys v1.1.1`

I installed clang with
apt-get install clang libclang-dev

and got past the clang-sys issue.
Next up is socket2, which says "Socket2 uses 1.46.0 as MSRV." Debian 10 provides Rust 1.41.1, so I need to update that.

Installed a fresh rustc and cargo using rustup, trying cargo build again. socket2 is fine with "rustc 1.53.0-nightly (48691ea6e 2021-03-29)"
The build got pretty far, but error points to a platform issue:
   Compiling wgctrl-sys v1.0.0 (/home/emv/src/github.com/tonarino/innernet/wgctrl-sys)
error: failed to run custom build command for `wgctrl-sys v1.0.0 (/home/emv/src/github.com/tonarino/innernet/wgctrl-sys)`

Caused by:
  process didn't exit successfully: `/home/emv/src/github.com/tonarino/innernet/target/debug/build/wgctrl-sys-bdf69c9f9af8e474/build-script-build` (exit status: 101)
  --- stderr
  /usr/include/aarch64-linux-gnu/sys/types.h:144:10: fatal error: 'stddef.h' file not found
  /usr/include/aarch64-linux-gnu/sys/types.h:144:10: fatal error: 'stddef.h' file not found, err: true
  thread 'main' panicked at 'Unable to generate bindings: ()', wgctrl-sys/build.rs:15:44
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Did I install clang? Because when I installed it again the build went fine.
The docs could use some build instructions, specifically dependencies and minimum versions.
Everything builds! But it fails to link with
  = note: /usr/bin/ld: cannot find -lsqlite3
          collect2: error: ld returned 1 exit status

Install sqliite3 and libsqlite3-dev, and with that everything builds! Now to test.

Sorry for the trouble on this! I agree we should have build instructions and dependencies listed out. I'll try to address that today.

Closing this issue as the problem wasn't arm64 support, but better build instructions which @bschwind has already added to the README.


Clean up superstructure, operator inputs, update intake to use new harwdare setup, set up PID for shooter, and more

This PR does:

Cleaned up un-needed OI methods
Cleaned up the OperatorControl command
Removed redundant states from superstructure and intake
Made intake use only hall sensors for development
Configured a PID controller for our shooter velocity
Made small edits to other systems

It occurred to me that the manual control code that was written is not needed, because we can use LiveWindow to control our motors

@rsninja722 @ExVacuum When testing the intake+hopper+shooter, you must use this branch, not intake.


There is no included safe way to automate

There is no safe way included with acme-tiny to automate the process of renewing certificates.  The suggested way is somewhat unsafe.
I have attached a wrapper script that addresses this.  Now that I am posting this I see I have reinvented the wheel a little, however the script attached here has some benefits:

Can be put in a crontab and run daily, so that on failure it will automatically retry every day.
The script is intended to run as root, which is a benefit for assembling your certificates, but it safely drops root privileges when running acme_tiny.py, so nothing that touches the network runs as root.
The script is aware of lighttpd and its unusual requirement of having the certificate together with the server private key.  The script is capable of safely assembling it.
The script checks the expiry date of your existing certificate and renews a set number of days before it expires.  LetsEncrypt has been suggesting that it wants to rmake certificates valid for an even shorter period.  If that happens, this wrapper will still automatically renew on time.
On success or fail, the contact is emailed with a log of the proceedings to let you know what happened.
The script is self-contained.  It has instructions built-in.

checkcrt.zip
Whether or not any mention or inclusion of this particular script is made into acme-tiny, it would be highly beneficial for this project to adopt some sort of script, or to at least maintain a list of those 3rd party scripts which are available.

The suggested way is somewhat unsafe.

Updated the suggested cronjob script in e9e52678ae05dfa044a137eef5bb1bf6a56d4d92 based on #220 (thanks @FilipeMaia).
Thanks for the package, but I don't think a full management script for a particular setup (lighttpd, etc.) is within the scope of this project. Closing this for now.


Fix single line text input showing scrollbar and accepting Enter

Single line inputs now use input["text"] instead of textarea

@dclaux,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


Publish the js-sys crate

I think we have enough bindings written that it would be useful to publish now. Especially since it is a separate crate from wasm-bindgen now.
Things we need to do first:

[x] Do a once over of the bindings to double check we aren't returning Number objects and all that

[X] Decide on a version. Do we want to match wasm-bindgen or start at 0.1.0?

[x] #[derive(Clone, Debug)] on all the extern types

[x] Make sure we are always passing by-ref

[x] Agree on &str vs &JsString arguments

I think we should probably reset to 0.1.0 since the whole point of a separate crate is the freedom to publish independently and not be super tied together.

Assuming that wouldn't have any sort of negative ergonomic effect on people using these crates, resetting to 0.1.0 makes sense to me, especially if they are separate. That said, my opinion is not strongly held by any means, and would defer to y'all's opinions on it :)

Sounds good to me! (along with 0.1.0)

Some things I've noticed working with this crate:

Strings are somewhat inconsistent, sometimes arguments are &str, others are JsString
It's inconsistent whether arguments are by-ref or by-value
In writing tests I've noticed a few small errors.
Some methods expose all the options while others only expose some.

Strings are somewhat inconsistent, sometimes arguments are &str, others are JsString

I think these should always be &JsString. That way, we can let users choose if/when to make the copy from the JS heap into wasm linear memory. When host bindings are a thing that is actually happening, we can revisit to match what is fastest for them.

It's inconsistent whether arguments are by-ref or by-value

I think we should use by-ref to avoid ref-count noise.

Some methods expose all the options while others only expose some.

I don't think this needs to block publishing 0.1.0, since there are just so many optional arguments to various functions and methods, and it seems we have some design decisions to make about this topic.
We have been choosing whether to expose these arguments on a case by case basis and by applying our own judgement.
As I see it, we have a few choices:

Continue applying our own judgement on a case by case basis
Expose every argument
Expose only the required arguments
Add multiple bindings to the same function, some with optional arguments, some without. Do we do every combination of optional arguments? Always require first N arguments are supplied (so N+1 versions of the function for N optional arguments)?

I'm less certain about using &JsString all over the place, it's actually a lot faster to use &str if you don't reuse the JsString elsewhere because that way you only cross the wasm boundary once instead of N + 1 (where N is the number of strings). I'm not sure how often one is used over the other, but I'm personally partial to a strategy where we support both (through methods like fn parse(&str) and fn parse_js(&JsString)).
I do agree though we should probably take by-ref everywhere!
For optional arguments I think it's fine to basically cut what we have. It's subpar in some places but it can always be worked around locally if necessary or we can have more releases.
One last thing I've realized is we probably want to #[derive(Clone, Debug)] all the types here as well, but that won't be too hard to do!

This is quite trivial thing, but can we resolve the inconsistency whether semblance of "C" should be declared or not, which @alexcrichton pointed out before in #422? I am using rustfmt and it frustrates me to complement semblance (and put some indentations) whenever applying changes to the code.

@toVersus I personally agree we should just switch to using rustfmt for styling, but it definitely isn't a blocker for publishing the js-sys crate.

Will publish tomorrow, if anyone has any last minute concerns, now is your time to voice them!


Add support for fixed effect demeaning

This PR adds supports for fixed effects demeaning for the DuckRegression class.

It adds pyfixest's demean function in demean.py. numba is added to the package requirements.
It reworks the data processing steps and formula deparsing.
It slightly reworks the WLS method. Here I need to verify that it does not break any of the other methods that inherit from the DuckReg base class.

We can then fit models via two-part formula syntax. In contrast to before, f1 and f2 will be treated as categoricals and are projected out prior to fitting with WLS.
%load_ext autoreload
%autoreload 2

import numpy as np
import pandas as pd
from duckreg.estimators import DuckRegression
import duckdb

# Generate sample data
def generate_sample_data(N=10_000_000, seed=12345):
    rng = np.random.default_rng(seed)
    D = rng.choice([0, 1], size=(N, 1))
    X = rng.choice(range(20), (N, 2), True)
    Y = D + X @ np.array([1, 2]).reshape(2, 1) + rng.normal(size=(N, 1))
    Y2 = -1 * D + X @ np.array([1, 2]).reshape(2, 1) + rng.normal(size=(N, 1))
    df = pd.DataFrame(
        np.concatenate([Y, Y2, D, X], axis=1), columns=["Y", "Y2", "D", "f1", "f2"]
    ).assign(rowid=range(N))
    return df

# Function to create and populate DuckDB database
def create_duckdb_database(df, db_name="large_dataset.db", table="data"):
    conn = duckdb.connect(db_name)
    conn.execute(f"DROP TABLE IF EXISTS {table}")
    conn.execute(f"CREATE TABLE {table} AS SELECT * FROM df")
    conn.close()
    print(f"Data loaded into DuckDB database: {db_name}")

# Generate and save data
df = generate_sample_data()
db_name = 'large_dataset.db'
create_duckdb_database(df, db_name)

m = DuckRegression(
    db_name="large_dataset.db",
    table_name="data",
    formula="Y ~ D | f1  + f2",
    cluster_col="f1",
    n_bootstraps=100
)
m.fit()
results = m.summary()

restab = pd.DataFrame(
    np.c_[results["point_estimate"], results["standard_error"]],
    columns=["point_estimate", "standard_error"],
)
restab

# 	point_estimate	standard_error
# 0	0.999347	0.000737

Two additions with #e2b2352

It requires users to specify the seed by hand (vs the 42 default 😅). Highly opinionated by me, happy to take it out =)
Adds support to run the OLS fitting via pyfixest by adding a fitter method to the base duck regression class. So we can now do something as

from duckreg.estimators import DuckRegression

m = DuckRegression(
    db_name="large_dataset.db",
    table_name="data",
    formula="Y ~ D | f1  + f2",
    cluster_col="f1",
    n_bootstraps=100,
    seed = 92,
    fitter = "feols"
)
feols_duckreg = m.fit()
feols_reg = pf.feols("Y ~ D | f1 + f2", data=df)


Replacing a proto equality.

@michael-berlin @sougou another one of these.

LGTM

Alain, I'm merging this now because I'll run an internal import shortly.


chore: Update Dependency sync to sort and exclude comments

Fixes #1232

Diffs look alot quiter now https://github.com/canada-ca/ore-ero/commit/19d5971d1602fa45be3e114ff653faae1fd0df53

OK, looks like admin and project are causing random diffs


[metrics] AUROC Metric can't handle 0 observations of a class with multiclass classifier

I'm attempting to calculate AUROC for a multiclass problem where some classes are very rare, occasionally never seen, and I'm getting the following error: raise ValueError("No positive samples in targets, true positive value should be meaningless")
In the case of 0 observations, I feel the average='weighted' should work, since the contribution to the final AUROC should be 0 regardless. One can think of other scenarios where there are a very high number of classes, some of which will happen to not be seen in some dataset.
Originally posted by @BeyondTheProof in https://github.com/PyTorchLightning/pytorch-lightning/issues/2210#issuecomment-872440776

Hi @BeyondTheProof, I transferred the issue to the torchmetrics repo.
Can I kindly ask if you are calling the forward method or the update method of the metric when trying to update the metric states?

Hi @SkafteNicki, thank you for the response and transferring the issue! I am just calling the update method.

Could you provide an example of how you are using the metric?

In my subclassed lightning module, I have metrics that are calculated at the end of each epoch:
self.auroc = torchmetrics.AUROC(num_classes=15, average="weighted", compute_on_step=False)
self.accuracy = pl.metrics.Accuracy(num_classes=15, average="weighted", compute_on_step=False)
self.metrics = [self.auroc, self.accuracy]

When I calculate all my losses, I also calculate some other metrics (if there are any bugs, it is only because I modified it to simplify as much as possible):
for metric, name in zip(self.metrics, ["AUROC", "ACC"]):
  self.log(
      f"{stage}_{name}",
      metric,
      on_step=False,
      on_epoch=True,
      prog_bar=False,
      logger=True,
  )

I hope this helps. If there's anything else you need, please let me know!

I've found a hack for this by subclassing AUROC:
class FixedAUC(torchmetrics.AUROC):
    def update(self, preds: torch.Tensor, target: torch.Tensor):
        num_classes = preds.shape[1]
        zero_obs_mask = torch.tensor([(target == c).sum() > 0 for c in range(num_classes)])
        preds = preds[:, zero_obs_mask]
        target = target[:, zero_obs_mask]
        self.num_classes = zero_obs_mask.sum().int()
        super().update(torch.softmax(preds, axis=-1).data, target.argmax(axis=-1).int().data)

I will be making a cleaner fix in torch and asking for a PR

In this subclass, there is an error that arises when some batches have only two classes, others have more. I implemented a much cleaner subclass by just subclassing Metrics:
from torchmetrics.functional.classification.auroc import _auroc_compute

class WeightedAUROC(Metric):
    """
    This is used for when the target is not strictly one of K classes, but a probability
    distribution over all K classes. ArgmaxAUC takes the argmax of the classes as truth
    """

    def __init__(
        self,
        num_classes: Optional[int] = None,
        pos_label: Optional[int] = None,
        compute_on_step: bool = False,
        dist_sync_on_step: bool = False,
    ) -> None:
        super().__init__(
            compute_on_step=compute_on_step,
            dist_sync_on_step=dist_sync_on_step,
            process_group=process_group,
        )
        self.num_classes = num_classes
        if self.num_classes > 2:
            self.mode = "multiclass"
        else:
            self.mode = "binary"
        self.pos_label = pos_label
        self.average = "weighted"

        self.add_state("preds", default=[], dist_reduce_fx="cat")
        self.add_state("target", default=[], dist_reduce_fx="cat")

    def update(self, preds: torch.Tensor, target: torch.Tensor):
        assert target.ndim == 2, f"ArgmaxAUC expects a 2D Tensor, got {target.ndim}"
        self.preds.append(torch.softmax(preds, axis=-1).data)
        self.target.append(target.data)

    def compute(self):
        preds = dim_zero_cat(self.preds)
        target = dim_zero_cat(self.target)
        zero_obs_mask = torch.tensor([(target == c).sum() > 0 for c in range(self.num_classes)])
        preds = preds[:, zero_obs_mask]
        target = target[:, zero_obs_mask].int()
        num_classes = zero_obs_mask.sum().int()
        return _auroc_compute(preds, target, self.mode, num_classes=num_classes, average=self.average)```

@BeyondTheProof I thinks that it will be better to add the line process_group: Optional[Any] = None,.

class WeightedAUROC(Metric):
    """
    This is used for when the target is not strictly one of K classes, but a probability
    distribution over all K classes
    """

    def __init__(
        self,
        num_classes: Optional[int] = None,
        pos_label: Optional[int] = None,
        compute_on_step: bool = False,
        dist_sync_on_step: bool = False,
        process_group: Optional[Any] = None, # It seems that this line is missing.
    ) -> None:
        super().__init__(
            compute_on_step=compute_on_step,
            dist_sync_on_step=dist_sync_on_step,
            process_group=process_group,
        )
        self.num_classes = num_classes
        if self.num_classes > 2:
            self.mode = "multiclass"
        else:
            self.mode = "binary"
        self.pos_label = pos_label
        self.average = "weighted"

        self.add_state("preds", default=[], dist_reduce_fx="cat")
        self.add_state("target", default=[], dist_reduce_fx="cat")

    def update(self, preds: torch.Tensor, target: torch.Tensor):
        assert target.ndim == 2, f"WeightedAUC expects a 2D Tensor, got {target.ndim}"
        self.preds.append(torch.softmax(preds, axis=-1).data)
        self.target.append(target.data)

    def compute(self):
        preds = dim_zero_cat(self.preds)
        target = dim_zero_cat(self.target)
        zero_obs_mask = torch.tensor([(target == c).sum() > 0 for c in range(self.num_classes)])
        preds = preds[:, zero_obs_mask]
        target = target[:, zero_obs_mask].int()
        num_classes = zero_obs_mask.sum().int()
        return _auroc_compute(preds, target, self.mode, num_classes=num_classes, average=self.average)

Hi @BeyondTheProof, thanks for getting back to me. It indeed seems to be a problem, even though I would consider this a very corner case. Would you be up for sending a fix to our current AUROC implementation?
IMO the only thing missing from your implementation is that the user should be warned if a class was removed.

Hi @SkafteNicki, good point about the user warning. Also, in the implementation above, the target only allows 2D tensors, i.e., a (obs, classes) binary matrix. I will implement a solution for this as well.

@BeyondTheProof I thinks that it will be better to add the line process_group: Optional[Any] = None,.

Thanks for the catch @KimDaeUng, you're totally right!

@SkafteNicki Submitted a PR here: https://github.com/PyTorchLightning/metrics/pull/376
Thanks!

Hi @SkafteNicki, just following up on this. I have an approval from Borda, but still need one more :)

The tests are failing in the PR though?

Ah, sorry, I didn't catch that. Will fix.


Gold subscription not credited on "Sustainability" page

I appreciate the service that Read The Docs provides, so I signed up and subscribed to donate $5/month.
I had expected that Gold subscribers would have their name and Gravatar to appear on the sustainability page, along with the one-time donors. My name and Gravatar do not appear here.
Is this intentional?

Correct, the donation page is strictly for those that have donated. We could perhaps also display the subscription status of anyone that is giving monthly here though. This sounds like a good addition.

This issue is probably pretty far off of our roadmap, so i'm going to close it as something we likely won't fix


Specify minimum supported Rust version in fuels-rs manifest

This ensures that users are warned in the case that their installed Rust version predates the version required by fuels-rs.
This should be particularly useful for users building forc-generated test harnesses, as I believe we've already had a couple of reports of strange compile errors that just required a Rust version update.
You can find documentation on this field here.

Originally I had planned on exposing this minimum required Rust version from a function in the crate root like so. The idea was that forc could use this function to specify the necessary rust-version field in the generated test harness manifest files.
However after thinking on this a little more, I realised it should be enough to simply specify the minimum required version here as fuels will be a dependency within the generated test harness anyway, meaning the user will be notified either way. As a result, I believe this closes https://github.com/FuelLabs/sway/issues/1239.

I'm just realising we should probably have another CI check that uses the minimum specified version to cargo check the workspace and make sure we don't actually require a newer version?
I'm unsure whether or not specifying the rust-version like this will cause cargo to automatically error on use of language features that became available after the specified version - I suspect that it won't.

@ra0x3 fuel-rs is independent of forc. For example in our faucet app, we use the SDK to send genesis coins to users without interacting with contracts or forc at all.

Put another way, fuels-rs is independent of the particular high level language used.

Thanks for finishing this off / landing this!


Add option to change compilerOptions and provide fix for cd issue

Hi,
first of all thank you guys for this little library makes our life so much easier :+1:
This PR is meant to address these issues:

Workaround for ChangeDetectionStrategy.OnPush bug
Clear unnecessary spacing in html snapshots. link

Modifying component for the purpose of testing it is a bad idea.
I think a simulateInputPush() method somewhere would be nice

I'm 100%  with you, however if i understand this issue correctly there is no such solution only hacky workarounds.

sorry a little late on this.


Fix typo

Fix typo in Connect to the database session

Thanks, thats fixed in the updated documentation.


make wasi-nn backends as separated shared libraries

@tonibofarull please make a review.
BTW, as mentioned in https://github.com/bytecodealliance/wasm-micro-runtime/issues/3493, we're working on more backends.


Merge HiromuHota/webspoon-docker into HiromuHota/pentaho-kettle

I'm merging HiromuHota/webspoon-docker into HiromuHota/pentaho-kettle.
HiromuHota/webspoon-docker was originally experimental, but no any more. It is actually the recommended way of deployment.
Another reason is that contributors might get confused which repository they are supposed to submit an issue to. By merging into a single repository, there will be one single place for issues.
HiromuHota/webspoon-docker will be archived and become read-only, but past issues will remain visible.

+1 for ease of maintenance

+1


LoadEnvironmentVariables.php

Description:
Consider changing Dotenv()->load() to Dotenv()->overload() because when using various plugins with docker, e.g. redis and elasticsearch, the hostnames are defined in .env, but docker defines a global environment variable ELASTICSEARCH_HOST or REDIS_HOST which is in an incompatible format.
Took me 3-4 hours to trace this bug down!
Steps To Reproduce:

Or atleast make this more easily debuggable.  Getting a dump from laravel saying csnt connect to http://elasticsearch:tcp://<IP_ADDRESS>. Is not very helpful for diagnosis.
I did have this problem about six months ago and i think ended up patching some other library as i couldn't work out the cause.

Please move this conversation to internals.


refactor: remove list comprehensions to preserve type-hints

Ref: #299

coverage: 94.063% (-1.2%) from 95.235%
when pulling f698ea1406e9d09090e9e51b7e2d58c92f9e2f14 on 299-remove-list-comprehensions
into 67478a14fc5a8dee94487d181603938227b57e36 on main.


DocsArchive-<IP_ADDRESS>Auto archive content

Auto archive content from https://github.com/OPS-E2E-PPE/docs-archive-test

Docs Build status updates of commit 24a3577:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error]

Cannot sync git repo to specified commit because branch Archive_master_2019-09-25-05-10-52 has been deleted or has been force pushed: fatal: Couldn't find remote ref Archive_master_2019-09-25-05-10-52

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Tvmaze error finding show

Hi I recently upgradet to flexget with enabled tvMaze by default.
running the Series Tasks returns some errors from tvmaze like:

2015-11-26 18:36 WARNING  est_series_tvmaze SERIES Could not found show on TVMaze: u'Orange%20is%20the%20new%20Black not found'

A quick lookup on their webpage showed thes show can be found.
After that I did recive

2015-11-26 18:37 CRITICAL task          SERIES_ARCHIVE BUG: Unhandled error in plugin discover: u'\xfc'
2015-11-26 18:37 CRITICAL manager       SERIES_ARCHIVE An unexpected crash has occurred. Writing crash report to /home/USERNAME/.flexget/crash_report.2<IP_ADDRESS>724001553.log

the chrash log is as follows

Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 438, in __run_plugin
return method(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flexget/event.py", line 22, in call
return self.func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/input/discover.py", line 268, in on_task_input
entries = self.estimated(entries)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/input/discover.py", line 190, in estimated
est_date = estimator.estimate(entry)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/est_released.py", line 29, in estimate
estimate = estimator(entry)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/estimators/est_release_series_tvmaze.py", line 49, in estimate
tvmaze_show = get_show(**kwargs)
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 197, in get_show
show_language, show_country)
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 206, in get_show_by_search
shows = get_show_list(show_name)
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 246, in get_show_list
shows = show_search(show_name)
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 264, in show_search
show = url_quote(show)
File "/usr/lib/python2.7/urllib.py", line 1288, in quote
return ''.join(map(quoter, s))
KeyError: u'\xfc'

Thanks in advance!

Are you sure you're on the latest version of flexget?

yes, did the update 5 minutes before I posted the ticket

$ flexget --version
1.2.395
You are on the latest release.

Ok, thanks, i'll investigate this.

Another question, are you running flexget from pip or git? Can you checked the installed version of pytvmaze please?

Hi i am running from pip.
pytvmaze:

$ pip show pytvmaze

Name: pytvmaze
Version: 1.3.4
Location: /usr/local/lib/python2.7/dist-packages
Requires:

What version of python is this running under out of curiosity?

Is get_show a public method from pytvmaze? If so, passing unicode in sounds correct. It should probably just be encoded first before handing to urllib.

Some more testing is making it seem like this isn't the issue, but just for grins can you try to reproduce the same error with the latest dev of pytvmaze? https://github.com/srob650/pytvmaze/tree/dev

I'm having the same issue with the latest flexget and pytvmaze from pip:

$ flexget execute --now --discover-now --loglevel debug

2015-11-29 20:30 CRITICAL plugin                        Error while registering plugin est_released_movies. A plugin with the same name is already registered
2015-11-29 20:30 CRITICAL plugin                        Error while registering plugin estimate_release. A plugin with the same name is already registered
2015-11-29 20:30 VERBOSE  task_queue                    There are 6 tasks to execute. Shutdown will commence when they have completed.
2015-11-29 20:30 INFO     interval      discover-tv-weekly Ignoring interval because of --now
2015-11-29 20:30 VERBOSE  input_cache   discover-tv-weekly Restored 16 entries from db cache
2015-11-29 20:30 VERBOSE  discover      discover-tv-weekly Discovering 16 titles ...
2015-11-29 20:30 INFO     discover      discover-tv-weekly Ignoring interval because of --discover-now
2015-11-29 20:30 WARNING  est_series_tvmaze discover-tv-weekly Could not found show on TVMaze: u'The%20Good%20Wife not found'
2015-11-29 20:32 WARNING  est_series_tvmaze discover-tv-weekly Could not found show on TVMaze: u'Lost%20Girl not found'
2015-11-29 20:32 WARNING  est_series_tvmaze discover-tv-weekly Could not found show on TVMaze: u'Homeland not found'
2015-11-29 20:33 CRITICAL task          discover-tv-weekly BUG: Unhandled error in plugin discover: u'Maze id 383 not found'
2015-11-29 20:33 CRITICAL manager       discover-tv-weekly An unexpected crash has occurred. Writing crash report to /home/xbmc/.flexget/crash_report.2<IP_ADDRESS>349511639.log
2015-11-29 20:33 WARNING  task          discover-tv-weekly Aborting task (plugin: discover)

The related part from the crashlog:

$ cat .flexget/crash_report.2<IP_ADDRESS>349511639.log

2015-11-29 20:30 WARNING  est_series_tvmaze discover-tv-weekly Could not found show on TVMaze: u'The%20Good%20Wife not found'
2015-11-29 20:32 WARNING  est_series_tvmaze discover-tv-weekly Could not found show on TVMaze: u'Lost%20Girl not found'
2015-11-29 20:32 WARNING  est_series_tvmaze discover-tv-weekly Could not found show on TVMaze: u'Homeland not found'
2015-11-29 20:33 CRITICAL task          discover-tv-weekly BUG: Unhandled error in plugin discover: u'Maze id 383 not found'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 438, in __run_plugin
    return method(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/event.py", line 22, in __call__
    return self.func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/input/discover.py", line 268, in on_task_input
    entries = self.estimated(entries)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/input/discover.py", line 190, in estimated
    est_date = estimator.estimate(entry)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/est_released.py", line 29, in estimate
    estimate = estimator(entry)
  File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/estimators/est_release_series_tvmaze.py", line 49, in estimate
    tvmaze_show = get_show(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 208, in get_show
    show_language, show_country, embed=embed)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 217, in get_show_by_search
    shows = get_show_list(show_name, embed)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 260, in get_show_list
    for show in shows
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 343, in show_main_info
    raise IDNotFound('Maze id ' + str(maze_id) + ' not found')
IDNotFound: u'Maze id 383 not found'

The most weird part is the Maze id 383, because http://www.tvmaze.com/shows/383 points to some show called "Peep Show", which is definitely not in my flexget database:

$ flexget series list

2015-11-29 20:55 CRITICAL plugin                        Error while registering plugin est_released_movies. A plugin with the same name is already registered
2015-11-29 20:55 CRITICAL plugin                        Error while registering plugin estimate_release. A plugin with the same name is already registered
-------------------------------------------------------------------------------
 Name                           Latest     Age        Downloaded
-------------------------------------------------------------------------------
 Angel From Hell                N/A        N/A        N/A
 Ascension                      N/A        N/A        N/A
 Bones                          S11E08     9d 5h      hdtv h264
 Brooklyn Nine-Nine             S03E08 +1  2d 4h      hdtv h264-proper
 Castle                         S08E08 +1  2d 4h      hdtv h264
 Dark Matter                    S01E13     87d 15h    hdtv h264
 Faking It                      S02E15 +2  61d 9h     hdtv h264
 Grandfathered                  S01E08 +1  2d 4h      hdtv h264
 Homeland                       S05E08     2d 4h      hdtv h264
 iZombie                        S02E07     11d 5h     hdtv h264
 Lost Girl                      S05E16 +1  34d 4h     hdtv h264
 Mission Control                N/A        N/A        N/A
 Mr. Robot                      S01E10     87d 15h    hdtv h264
 Orange Is the New Black        S03E06 +1  167d 21h   720p webrip h264 dd5.1
 Rizzoli & Isles                S06E12 +1  87d 15h    hdtv h264
>Rosewood                       S01E01     1h         hdtv h264
 Star Wars Rebels               S02E07 +1  10d 2h     hdtv h264
 The Big Bang Theory            S09E09 +1  9d 5h      hdtv h264
 The Good Wife                  S07E07 +2  12d 3h     hdtv h264
 Veep                           S04E10 +1  165d 9h    hdtv h264
-------------------------------------------------------------------------------

$ pip show flexget

Name: FlexGet
Version: 1.2.395
Location: /usr/local/lib/python2.7/dist-packages
Requires: FeedParser, SQLAlchemy, PyYAML, beautifulsoup4, html5lib, PyRSS2Gen, pynzb, progressbar, rpyc, jinja2, requests, python-dateutil, jsonschema, tmdb3, path.py, guessit, apscheduler, flask, flask-restful, ordereddict, flask-restplus, cherrypy, flask-assets, cssmin, flask-compress, flask-login, pyparsing, pyScss, pytvmaze

$ pip show pytvmaze

Name: pytvmaze
Version: 1.3.5
Location: /usr/local/lib/python2.7/dist-packages
Requires:

Very strange.  All 4 of those shows return fine when using pytvmaze alone with:
s = pytvmaze.get_show(show_name='homeland')
s = pytvmaze.get_show(show_name='the good wife')
s = pytvmaze.get_show(show_name='lost girl')
s = pytvmaze.get_show(show_name='orange is the new black')

Maybe it's not flexget related, because the python console output for me is the following for homeland:

$ python

Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pytvmaze
>>> s = pytvmaze.get_show(show_name='homeland')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 211, in get_show
    show_language, show_country, embed=embed)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 220, in get_show_by_search
    shows = get_show_list(show_name, embed)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 260, in get_show_list
    shows = show_search(show_name)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 284, in show_search
    raise ShowNotFound(str(show) + ' not found')
pytvmaze.exceptions.ShowNotFound: u'homeland not found'

And for veep:
>>> s = pytvmaze.get_show(show_name='veep')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 211, in get_show
    show_language, show_country, embed=embed)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 220, in get_show_by_search
    shows = get_show_list(show_name, embed)
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 263, in get_show_list
    for show in shows
  File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 346, in show_main_info
    raise IDNotFound('Maze id ' + str(maze_id) + ' not found')
pytvmaze.exceptions.IDNotFound: u'Maze id 383 not found'

(So somehow veep became Peep Show with Maze id 393...)
However it seems to be okay with Veep or Homeland as arguments.
>>> s = pytvmaze.get_show(show_name='Veep')
>>> s = pytvmaze.get_show(show_name='Homeland')

Even if pytvmaze were case sensitive (AFAIK it's not), based on my previous log, flexget did pass the arguments with uppercase first letters, which is working fine for me in the console. I'm completely lost now.

I can't figure out why it would be doing that.  Here's the output from my console with the same tests:
>>> s = pytvmaze.get_show(show_name='veep')
>>> s.name
u'Veep'
>>> s.network
{u'country': {u'timezone': u'America/New_York', u'code': u'US', u'name': u'United States'}, u'id': 8, u'name': u'HBO'}

>>> s = pytvmaze.get_show(show_name='homeland')
>>> s.name
u'Homeland'
>>> s.network
{u'country': {u'timezone': u'America/New_York', u'code': u'US', u'name': u'United States'}, u'id': 9, u'name': u'Showtime'}

Maybe try deleting and re-installing pytvmaze? (long shot).
Any reason you're on Python 2.7.6 instead of 2.7.10?  FWIW I'm testing on 2.7.10 as well as 3.5.0

No specific reason for the Python version. This is the newest version in the Ubuntu Trusty and I never had a cause to manually upgrade it. Removing and reinstalling pymaze did nothing (I did it already a few times before your suggestion)
However I ran a quick network diagnostics and my latency was a bit high (around 30-50 ms, it wasn't even noticable for example browsing the web). I restarted the modem, the latency is now <10 ms and pytvmaze (and flexget) works fine. The sofware side of things is the same as before, I didn't even quit the python console. Maybe pytvmaze is this sensitive about network quality?

Hmm, very interesting.  I'm no expert on this kind of thing, but it doesn't seem possible that you could be getting incomplete data from tvmaze.com because of a timeout, I think it's all or nothing.  If you were really getting incomplete data you would probably see other weird errors in pytvmaze like show objects not getting initialized properly.  Someone correct me if this seems wrong

Haven't heard from the OP @kecksk0enig in a while either, was hoping they would try testing with the latest pytvmaze dev build to see if that fixed their problem.

This is very strange, my tests seems to work as well. I'll bet this relates to encoding though, this is the only thing that makes sense.

Maybe this is something else, like tvmaze flood protection or connection issues, and not a functional error.
@srob650 we should add some better response verification of the request to check this.

Awesome, thanks @liiight !

This is however different from the issue that this topic is about so i'll open a new issue.
@kecksk0enig please verify that the problem does not occur with pytvmaze v1.3.5

hi i upgraded pytvmaze to Version: 1.3.5
I still get an exception from pytvmaze but it looks different:

2015-11-30 13:41 CRITICAL task          SERIES_ARCHIVE BUG: Unhandled error in plugin discover: u'Maze id 5 not found'
2015-11-30 13:41 CRITICAL manager       SERIES_ARCHIVE An unexpected crash has occurred. Writing crash report to /home/USERNAME/.flexget/crash_report.2<IP_ADDRESS>136312072.log

The relevant logs from crash_report:

2015-11-30 13:41 CRITICAL task          SERIES_ARCHIVE_DE BUG: Unhandled error in plugin discover: u'Maze id 5 not found'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flexget/task.py", line 438, in __run_plugin
return method(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flexget/event.py", line 22, in call
return self.func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/input/discover.py", line 268, in on_task_input
entries = self.estimated(entries)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/input/discover.py", line 190, in estimated
est_date = estimator.estimate(entry)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/est_released.py", line 29, in estimate
estimate = estimator(entry)
File "/usr/local/lib/python2.7/dist-packages/flexget/plugins/estimators/est_release_series_tvmaze.py", line 49, in estimate
tvmaze_show = get_show(**kwargs)
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 208, in get_show
show_language, show_country, embed=embed)
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 217, in get_show_by_search
shows = get_show_list(show_name, embed)
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 260, in get_show_list
for show in shows
File "/usr/local/lib/python2.7/dist-packages/pytvmaze/tvmaze.py", line 343, in show_main_info
raise IDNotFound('Maze id ' + str(maze_id) + ' not found')
IDNotFound: u'Maze id 5 not found'

Do you get that at the first try? Meaning, at the first episode that need an airdate?

No.
Here is the complete debug output from running the affected task:
http://paste.ubuntu.com/13574284/

I see. I'm pretty sure I know the cause of this, it's a socket error that caused urlib reply to crash. Currently there is a blanket 'except' on the response from urllib but I pushed a pr to pytvmaze that should address this issue. After it's merged, I'll push a pr to flexget to support this and not crash.
Anyways, this relates to a different issue than the one you opened originally: https://github.com/Flexget/Flexget/issues/664
The original issue can be closed IMO, as all Unicode text should now work.

okay I will watch #664.
The commit which fixed the Unicode text  was in pytvmaze or in flexget?
Are the flexget requirements for pytvmaze already fixed?
Should I as Op close the bug or is ther some kind of other review before closing the bug?

You're right, i'll push a pr with just the pytvmaze version update for now.
No need to close this then, it'll closed when the fix will be merged.


fix: Do not encode in mono by default

Encoding in mono feels like it is a waste for most stereo or surround content.  Let's only do stereo and surround by default.

Let's only do stereo and surround by default.

What if the input's channel layout is already mono? We won't be able to up-mix it!
Also, hard coding the layouts like that will cause the streamer to fail if the user provides a bitrate config file that doesn't have the labels stereo or surround:
audio_channel_layouts:
  mono:
    max_channels: 1
    bitrates:
      aac: '64k'
      opus: '32k'
  two: # not stereo
    max_channels: 2
    bitrates:
      aac: '128k'
      opus: '64k'
  six: # not surround
    max_channels: 6
    bitrates:
      aac: '192k'
      opus: '96k'

I think the way it is right now (using all the bitrate config layouts) is the most general and robust. And it's only triggered if we didn't specify channel_layouts in the pipeline config anyways.

I tend to think if the defaults aren't good for me, maybe they aren't good for others... But perhaps I'm overindexing on myself on this one.  Thanks for the feedback!


HHH-12083: Fix test failure in InsertOrderingWithBidirectionalOneToManyFlushProblem on SAP HANA

"TOP" is a reserved key word on HANA, so the name of an entity named "Top" must be quoted.

Applied upstream, thanks


PadeDelay block with small time delay results in unreliable simulation and wrong Bode plot

Reported by otter on 5 Jan 2015 20:59 UTC
Simulating Blocks.Nonlinear.PadeDelay with delayTime=0.001 and n=4 is practically halting the simulation (no progress in simulation time). Linearizing this model give a completely wrong linear model (number of zeros/poles are wrong).

Migrated-From: https://trac.modelica.org/Modelica/ticket/1637

Comment by otter on 5 Jan 2015 21:09 UTC
Fixed in 187f8b39c6a22661082e54807211610e2c5c6117 (in trunk) by a more reliable numerical implementation and adding 2 test models in ModelicaTest:
A new parameter balance=false/true introduced. The default balance=false results in the previous implementation of MSL 3.2.1 (for backwards compatibility). For balance=true, the state space representation is balanced, resulting in a much smaller norm of the system matrix. Simulation of the critical system does now pose not any problems. Also higher order, say n=10, can now be simulated without problems. Furthermore, linearization results in the correct linear system.
In r7840 the Bode plot computation in Modelica_LinearSystems2 was improved to handle systems with large positive zeros in a numerically reliable way. With this change, the Bode plot of the critical system, also with higher n, is now also correct.

Changelog modified by otter on 5 Jan 2015 21:09 UTC
PadeDelay block: More reliable numerical implementation


Weird crashing after long uptime

After 2-3 days of uptime on remote computer very often I find, that the package crashes with this traceback:
[19:31:44] Starting new HTTP connection (1): <IP_ADDRESS>:8080
[19:31:44] Traceback (most recent call last):
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.7/http/client.py", line 1321, in getresponse
    response.begin()
  File "/usr/lib/python3.7/http/client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.7/http/client.py", line 265, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.7/http/client.py", line 1321, in getresponse
    response.begin()
  File "/usr/lib/python3.7/http/client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.7/http/client.py", line 265, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/vlcscheduler.py", line 199, in main
    loop.run_until_complete(main_coro())
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "src/vlcscheduler.py", line 184, in main_coro
    await asyncio.gather(*tasks)
  File "src/vlcscheduler.py", line 59, in player_coro
    player.add(item.path)
  File "/home/monitor/Documents/VLC-Scheduler/src/vlc.py", line 116, in add
    return self._command('in_play', {'input': self._format_uri(uri)})
  File "/home/monitor/Documents/VLC-Scheduler/src/vlc.py", line 106, in _command
    return self._request('requests/status.xml', params=params)
  File "/home/monitor/Documents/VLC-Scheduler/src/vlc.py", line 91, in _request
    resp = self.session.get(urljoin(self.base_url, path), **kwargs)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/requests/sessions.py", line 525, in get
    return self.request('GET', url, **kwargs)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/home/monitor/Documents/VLC-Scheduler/venv/lib/python3.7/site-packages/requests/adapters.py", line 495, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

So periodically I have to check state of its work. Could it be caused by PC's troubles or Python's?

Could it be caused by PC's troubles or Python's?

Please try older or newer version of VLC.

Hm, after start .m3u8-playlist with links to files on remote machine it works stable this week


CFuncs cannot use numba.carray when jitting is disabled

Even when jitting is disabled with the NUMBA_DISABLE_JIT flag, CFuncs still try to compile. This causes issues when the CFuncs are trying to use additional functions provided by Numba, e.g. numba.carray, as those seem not to be available when jitting is disabled.
Could the Numba-provided utilities such as numba.carray be allowed to compile within cfuncs even when jitting is disabled for the rest of the code, or could the error message be clarified?
Minimal example that works with default settings but fails with NUMBA_DISABLE_JIT=1:
import ctypes

import numba
import numpy as np

@numba.cfunc(numba.types.void(
    numba.types.CPointer(numba.types.double)
))
def func(arr: np.ndarray) -> None:
    c_arr = numba.carray(arr, (3,), dtype=np.double)
    c_arr[-1] = 1

my_arr = np.array([1, 2, 3], dtype=np.float_).ctypes.data_as(ctypes.POINTER(ctypes.c_double))
func(my_arr)

Error with NUMBA_DISABLE_JIT=1:
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function carray at 0x7f8805dea5e0>) found for signature:

 >>> carray(float64*, UniTuple(Literal[int](3) x 1), dtype=class(float64))

There are 4 candidate implementations:
  - Of which 4 did not match due to:
  Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba/core/overload_glue.py: Line 131.
    With argument(s): '(float64*, UniTuple(int64 x 1), dtype=class(float64))':
   Rejected as the implementation raised a specific error:
     AttributeError: 'function' object has no attribute 'get_call_template'
  raised from <path_to_project>/venv/lib/python3.9/site-packages/numba/core/types/functions.py:541

During: resolving callee type: Function(<function carray at 0x7f8805dea5e0>)
During: typing of call at <path_to_project>/numba_test.py (11)

File "numba_test.py", line 11:
def func(arr: np.ndarray) -> None:
    c_arr = numba.carray(arr, (3,), dtype=np.double)
    ^

In my use case this error arises, as I have a cfunc that uses numba.carray, and this is passed to the NumbaLSODA library for the integration of a differential equation. It would be useful to be able to compile this CFunc even when jitting is disabled for the rest of the program for debug purposes.

There are two different ideas here:

cfunc doesn't respect NUMBA_DISABLE_JIT=1. That's something we can fix by wrapping the function with ctypes.CFUNCTYPE so the interpreted python function is exposed as a C callable.

It would be useful to be able to compile this CFunc even when jitting is disabled for the rest of the program for debug purposes.

For this, it's probably best to wrap Numba's decorator with custom logic so you can fine tune what is disabled.


stardust per hour calculator plugin

Simple plugin to calculate stardust gained per hour. First runelite plugin

@kevindarbydev Don't delete your branch/open a new PR - just update the existing branch with commit hash of your most recent change.


fix(fulltext-search): prune rows in row group forget to take remainder

I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
Forgot to take the remainder before passing row ids to prune_row_groups_by_rows. Fixed it on this patch.
Checklist

[ ] I have written the necessary rustdoc comments.
[ ] I have added the necessary unit tests and integration tests.
[ ] This PR requires documentation updates.

LGTM. Let's release v0.9.1 @fengjiachun

go go go


Errors after launch. Unsupported JavaFX configuration.

After launching I get a message:
Describe the bug
17:58:50.611 [main] INFO : Recaf-2.21.14
17:58:50.616 [main] INFO : - Java: 17.0.8 (Java HotSpot(TM) 64-Bit Server VM)
17:58:50.826 [main] TRACE: Loading configuration
17:58:51.131 [main] INFO : Discovered plugin 'Resource Cleaner-1.3.0'
17:58:51.132 [main] INFO : Discovered plugin 'Batch Decompile-1.2.1'
17:58:51.132 [main] INFO : Discovered plugin 'Theme Editor-1.0.0'
17:58:51.133 [main] INFO : Discovered plugin 'String Replace-1.0.0'
17:58:51.133 [main] INFO : Discovered plugin 'NO-OP-1.0'
17:58:51.133 [main] INFO : Discovered plugin 'RPC-1.1.2'
17:58:51.133 [main] INFO : Loaded 6 plugins
Dec 01, 2024 5:58:52 PM com.sun.javafx.application.PlatformImpl startup
WARNING: Unsupported JavaFX configuration: classes were loaded from 'unnamed module @3952037b'
Exception in thread "RPC-Callback-Handler" java.lang.ClassCastException: class javafx.scene.control
.ScrollPane cannot be cast to class me.coley.recaf.ui.controls.view.EditorViewport (javafx.scene.co
ntrol.ScrollPane and me.coley.recaf.ui.controls.view.EditorViewport are in unnamed module of loader
'app')
at fr.flowarg.recafrpcplugin.FileType.getCurrentFileType(FileType.java:51)
at fr.flowarg.recafrpcplugin.RPCPlugin.updateRPC(RPCPlugin.java:73)
at fr.flowarg.recafrpcplugin.RPCPlugin.lambda$onStart$1(RPCPlugin.java:50)
at java.base/java.lang.Thread.run(Thread.java:833)
17:58:57.147 [JavaFX Application Thread] INFO : Shutting down
17:58:57.151 [JavaFX Application Thread] TRACE: Saving configuration
17:58:57.173 [JavaFX Application Thread] TRACE: Shutting down thread executors
Screenshots

If applicable, add screenshots to help explain your problem.
OS: Ubuntu Mate 22.04
java version "17.0.8" 2023-07-18 LTS
Java(TM) SE Runtime Environment (build 17.0.8+9-LTS-211)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.8+9-LTS-211, mixed mode, sharing)

Recaf 2 is not supported
Delete all plugins and see if anything changes. The exception is caused by a plugin "RPC-1.1.2"

I removed the plugins in /home/vladimir/.config/Recaf/plugins.
The following error appeared:
14:53:35.670 [main] INFO : Recaf-2.21.14
14:53:35.673 [main] INFO : - Java: 17.0.8 (Java HotSpot(TM) 64-Bit Server VM)
14:53:35.893 [main] TRACE: Loading configuration
Dec 02, 2024 2:53:36 PM com.sun.javafx.application.PlatformImpl startup
WARNING: Unsupported JavaFX configuration: classes were loaded from 'unnamed module @3952037b'
14:53:41.296 [JavaFX Application Thread] INFO : Shutting down
14:53:41.301 [JavaFX Application Thread] TRACE: Saving configuration
14:53:41.319 [JavaFX Application Thread] TRACE: Shutting down thread executors

We will not offer support for Recaf 2.x.
 WARNING: Unsupported JavaFX configuration: classes were loaded from 'unnamed module @3952037b' does not mean anything and could be safely ignored

Go to https://github.com/Col-E/Recaf-Launcher, get the gui jar and enjoy.

Go to https://github.com/Col-E/Recaf-Launcher, get the gui jar and enjoy.

It's works.

This is not a bug, this is a feature.


update fog to v1.28.0

Fog 1.27.0 had a duplicate key in a hash (https://github.com/fog/fog/commit/b8981c1d90d2954aca29fbebd30c789cfb83f08e) which created a warning on Ruby 2.2 every time backup ran.

Thanks! The new version 4.1.8 has been released

Sweet, thanks!


Decouple _render method from _applyTransform.

the _render method (after possibly being renamed) can be simply for calculating sizing (after #66) and transform matrices. A separate function can apply the updated values to DOM or WebGL (eventually the DOMRenderer or WebGLRenderer).
Also related: #70. We want Node to only care about updating size and matrix values, but not about how to pass those values to DOM or WebGL.

This was completed when I re-organized and moved logic into ElementManager.applyTransform. #66 can be done separately.


Git strategy for multiple developers

I like the idea behind this setup and would love to use it, but it's lacking in one area...
Currently, each developer on my team has their own custom config/development.js file which is kept out of version control. Obviously this project won't be able to handle this.
So, do you have a suggested strategy for dealing with this scenario? I feel like it's not an edge case, in which a majority of other teams do this as well.
What are your thoughts on still maintaining a config-development.js file and just performing a deep merge with the main config.js file?

Hi @aesopwolf, a way to handle it in the code level is to merge them (e.g. with default Object.assign):
require('no-config')({
    config: Object.assign(
        {},
        require('./config/config.js'),
        require('./config/development.js')
}).then(
    ...
)

config/development.js in the repo can be:
module.exports = {}

But for some developer:
module.exports = {
    redis: {
        development: {
            host: 'joe.sample.com'
        }
    }
}

If you have a lot of config environments (production, testing, uat, qa, etc.) it still worth to split it to config.js + development.js first (just for one environment). However, in general, this approach replicates the notion from config module: to keep different environments in different files. Which decrease config maintainability.
a majority of other teams do this as well. Well, I'm not sure about a majority, but, a lot of teams require developers to have identical local environments for the same project, e.g. unified host name, ports, db names, etc. This seems like the best approach to me.


Can not process fields with yearmonth and geopoint

Originally comes from https://github.com/datahq/datahub-qa/issues/66 and https://github.com/datahq/datahub-qa/issues/98
We have a problem when pushing data that has types yearmonth and geopoint according to the schema.
Acceptance Criteria

[x] able to push https://raw.githubusercontent.com/frictionlessdata/test-data/master/files/csv/all-schema-types.csv

Tasks

[x] do analysis and find the reason/bug
[x] fix

Analysis
While yearmonth and geopoint are valid types according to tableschema. When I try and push data that looks like this - it fails.
yearmonth,geopoint
2018-01,"-101.473911, 38.704022"
2018-02,"-102.473911, 38.704022"

We have 2 problems here:

If you try and reproduce this (run BB Tests with data above) it will fail with following error messages

| ERROR   :Failed to cast row: Field "yearmonth" can't cast value "[2018, 1]" for type "yearmonth" with format "default"
| ERROR   :Failed to cast row: Field "geopoint" can't cast value "[Decimal('-101.473911'), Decimal('38.704022')]" for type "geopoint" with format "default"

While if you try and push it in the production environment, the processors will not fail, but derived JSON and CSV resources will not be generated. and will be kind of skipped.

Local
Strangely some of the pipelines are executed fine (without casting errors), but some of them (with the same exact data and schema) fails:
./datahub/all-schema-types/1/schema-types_json: FAILURE, processed 0 rows
./datahub/all-schema-types/1/schema-types: SUCCESS, processed 0 rows
./datahub/all-schema-types/1/validation_report: SUCCESS, processed 0 rows
./datahub/all-schema-types/1/schema-types_csv: SUCCESS, processed 2 rows
./datahub/all-schema-types/1/schema-types_csv_preview: FAILURE, processed 0 rows

Also, notice that some of the successful pipelines have processed 0 rows
This is the output of successfully processed CSV
"[2018, 1]","[Decimal('-101.473911'), Decimal('38.704022')]"
"[2018, 1]","[Decimal('-91.254898'), Decimal('35.6087')]"

OK so investigating further we actually have two different errors - 1 for derived/json and another for derived/preview

Error for preview is that it can't cast [2018, 1] as yearmnoth as it is getting data from derived/csv (same with geopoint)
And error for JSON is following: TypeError: Object of type 'Decimal' is not JSON serializable - and laso is correct as it's getting Decimal from somewhere as CSV resource (I guess from tabulator.Stream ??)

So the problem is that somewhere in the processing steps (probably stream_remote_resources) 2018-01 is transformed into -> [2018, 1] if schema says it is the type of yearmonth. And derived CSV now has [2018, 1] instead of original 2018-01. Since that data is later used for derived_preview and tabulator tryies to cast yearmonth type against [2018, 01] it fails.
The same problem with derived/json:  2018-01 is transformed into --> [2108, 1] and geopoint "-102.473911, 38.704022" into --> [Decimal('-91.254898'), Decimal('35.6087')]. This works fine for CSV, but not with JSON as 'Decimal' is not JSON serializable
At this point, I'm not sure where this transformation happens. I tried to reproduce same (transform 2018-01 -> [2018, 1]) with tabulator.Stream, but it works fine
Live
Even though we don't have resources for derived CSV or JSON their pipelines are executed successfully and packages are created as well. Just the files on S3 are empty (processors kind of skipping invalid rows)

CSV https://s3.amazonaws.com/pkgstore.datahub.io/81429cbbddcfb180f54c142fac32f83b/schema/all-schema-types_csv/datapackage.json
JSON https://s3.amazonaws.com/pkgstore.datahub.io/81429cbbddcfb180f54c142fac32f83b/schema/all-schema-types_json/data/d751713988987e9331980363e24189ce/all-schema-types_json.json

validation report says there are 9 casting errors https://pkgstore.datahub.io/81429cbbddcfb180f54c142fac32f83b/schema/validation_report/data/4daf94dcef7e9f9b8acac4b89cb47ff0/validation_report.json

FIXED here https://github.com/frictionlessdata/datapackage-pipelines/commit/f3d83c686f4cc1d142ae25d1f90c99aa1c10363e
See https://testing.datahub.io/adam.kariv/all-schema-types-honest-pig-14


Generalize payload of FormStatus.SubmissionFailed constructor

I'm thinking of making payload of FormStatus.SubmissionFailed constructor generic, i.e. turn this:
module FormStatus = {
  type t('field, 'message) =
    | Editing
    | Submitting
    | Submitted
    | SubmissionFailed(list(('field, 'message)), option('message));
};

into this:
module FormStatus = {
  type t('error) =
    | Editing
    | Submitting
    | Submitted
    | SubmissionFailed('error);
};

I found that I never (literally, zero times) used current payload but do use domain specific error types all the time and put those in component state. With generic version it's still possible to use currently hardcoded type by defining it in user land but it would make it more flexible for other use-cases. I wonder if others use it (if use it at all).

Downside is that it will require addition of a type to a form config... 🤔


Move public class methods out of sections of code with private scope

As part of https://github.com/ManageIQ/manageiq/pull/12041 we found some methods that were declared private, but were called from parts of the code that expected them to be public. This PR moves those offending methods to be public.
Links

Original Issue - Fixup Lint/IneffectiveAccessModifier (ex. private keyword before singleton methods)
PR to tackle issue ^

/cc @jrafanie

Looks good @AndyStabler, thanks.  I only had a minor question about the connection_options.

Closing as I don't have the time to focus on this right now.


Make build-container Dockerfile is missing GCC.

Doesn't seem to yield a working container at this point. It's missing a GCC lib.

Hi Chris - I'm not actively maintaining this code. Do please clone and update as necessary. Thanks

Gotcha. I'm an Elixirlang dev and need to make a pivot to something faster than what I'm usiing. I probably won't be able to contribute much to Golang. I do appreciate your explanation of the parser as I might just want to use the C++ lib for splitting via Erlang port and push Parquet files into S3.
I just want to use your work as a proof of concept. I'll trial building again on my Linux and see if I can update the Dockerfile. Thanks!

error output: /app/mrfparse: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.32' not found (required by /app/mrfparse)`

FROM golang:1.19.12-bullseye AS build

https://github.com/danielchalef/mrfparse/pull/27