Separate projection from query execution

e.g.
query.from(customer).where(customer.firstName.eq("Tom")).list(customer)

into
query.select(customer).from(customer).where(customer.firstName.eq("Tom")).list()

This has the following benefits

queries can be written both with select definition in the beginning as well as the end
subqueries can be defined via normal Query instances
Projectable and SimpleProjectable can be merged into the same class

Disadvantages

more verbose syntax in some cases
generic parameter in Query interface/class

I like this new syntax, this is more "fluent"
Did you also plan to add a method like com.querydsl.sql.ProjectableSQLQuery.getSQL() ?
Otherwise, we will not be able to call either list() or getSQL() for the same query.

getSQL() will be available also in the new version.

I currently waste a lot of time trying to figure out how to convert arbitrary SQL statements into QueryDSL syntax (and I've been using QueryDSL for a while). I like this change because it eases this pain.
I hope you continue moving in this direction. Non-standard expressions are also a pain point (e.g. "SELECT " or "SELECT function(x)" etc... The fact that we have to reach over to classes like com.mysema.query.support.Expressions makes the API harder to learn because it's not clear what are options are at any given point when building the query. Ideally I should be able to figure out what my options are without navigating away from the IDE's code-complete :)

I suggest that rename a few methods from Projectable to have a clear separation between query construction and fetching

list -> fetch
iterate -> fetchIterate ?!?
firstResult -> fetchFirst / fetchOne / fetchAny
uniqueResult -> fetchUnique / fetchOne
listResults -> fetchResults
count -> fetchCount

Otherwise there will be clashes for exists, notExists, count which are useful both in expression form and as fetch results. What do you think?

@cowwoc Expressions contains mostly factory methods for custom expression creation. In normal situations you won't be using this class much.

@timowest I favor introducing an intermediate execute() call and leaving the method names unchanged.
Example: query.select(customer).from(customer).where(customer.firstName.eq("Tom")).execute().list()
where execute() would return some sort of result type, and list() would convert the result into a list.

Regarding Expressions I find myself using it 10-25% of the time. As such, I think it is should be easier to discover/use.

@timowest I favor introducing an intermediate execute() call and leaving the method names unchanged.

Could you elaborate on why you favor this?

where execute() would return some sort of result type, and list() would convert the result into a list.

This has some implications on complexity, since the intermediate 'result' should have to know what state it is in and what results can be expected, (e.g. iteration is something special entirely).
Also, we already have the projection type in the Query, so I think it is best to keep the result processing local to the Query class of the module.

@cowwoc execute() as you describe it is not feasible, since the different projection methods have also different result fetching behaviour

list = eager loading of all results
iterate = lazy iteration
firstResult = only first loaded
uniqueResult = first two loaded
listResults = eager results and total count for paging
count = only count

execute() implies that the query is executed, when the method is called.

@cowwoc Concerning Expressions usage feel free to sketch a more fluent usage pattern for the functionality it provides.

Could you elaborate on why you favor this?

My intent was to:

Visually separate query construction from query execution (fetching the results).
In the existing API (even the one you are proposing) it isn't obvious at which point the query is actually getting fired. With the API I was thinking users could potentially construct a query once and reuse it multiple times, potentially reading the results differently each time (once as a list, once as an iterator, etc).

These points come about from my impression that there is a performance gain to be had if we can reuse PreparedStatements (something that QueryDSL does not currently allow). However, if there is no benefit to this (because, say, connection pools already do this) then I withdraw my suggestion.
@timowest The methods you listed seem to violate the "query construction" vs "query execution" separation of concerns. For example, it sounds as if count() alters the query but the entire point of this issue is to move the query construction components to the beginning of the line.
The methods you listed seem to blur the line a bit between query construction vs execution vs enforcing assertions on the response. I would expect users to select(Expressions.count(customer)).from(customer).where(...).fetchFirst(Integer.class).
How about this syntax (v2)?
select(...columns...).from(...tables...).where(...conditions...).X() where X() is one of:

fetchFirst().as(C) fetches up to one row and casts it to an instance of type C
hasMaxRows(1).fetchFirst().as(C) fetches up to two rows, ensures that there is only one, and casts the result to an instance of type C
fetchList().of(C) fetches all rows and cast them to a List<C>
iterate().as(C) lazy iteration over the rows as a Iterator<C>

Notice that I moved the assertions in front of the fetching operation, because hasMaxRows(1).fetchList().of(C) is legal too (I'd expect this to return a list of zero or one elements).
and this opens up the door to future assertions.
For count() we'd do something along the lines of:
select(Expressions.count(customer)).from(customer).where(...).fetchFirst().as(Integer.class) ... I like how this reads a lot closer to raw SQL/JDBC code (lower learning curve).
As for making Expressions more discoverable, I guess we could start off by using them more frequently in the documentation (show at least 2-3 different examples showcasing things like invoking count(), user functions, advanced expressions, etc)

You are right that the count options are redundant for the querydsl modules
that allow to define a projection.
The queries will be typed, so the as invocations are not necessary. The
type parameter of the query is the element type of the result.
On 13 Apr 2015 07:14, "Gili Tzabari"<EMAIL_ADDRESS>wrote:

Could you elaborate on why you favor this?
My intent was to:

Visually separate query construction from query execution (fetching
the results).
In the existing API (even the one you are proposing) it isn't
obvious at which point the query is actually getting fired. With the API I
was thinking users could potentially construct a query once and reuse it
multiple times, potentially reading the results differently each time (once
as a list, once as an iterator, etc).

These points come about from my impression that there is a performance
gain to be had if we can reuse PreparedStatements (something that QueryDSL
does not currently allow). However, if there is no benefit to this
(because, say, connection pools already do this) then I withdraw my
suggestion.
@timowest https://github.com/timowest The methods you listed seem to
violate the "query construction" vs "query execution" separation of
concerns. For example, it sounds as if count() alters the query but the
entire point of this issue is to move the query construction components to
the beginning of the line.
The methods you listed seem to blur the line a bit between query
construction vs execution vs enforcing assertions on the response. I would
expect users to
select(Expressions.count(customer)).from(customer).where(...).fetchFirst(Integer.class)
.
How about this syntax (v2)?
select(...columns...).from(...tables...).where(...conditions...).X()
where X() is one of:

fetchFirst().as(C) fetches up to one row and casts it to an instance
of type C
hasMaxRows(1).fetchFirst().as(C) fetches up to two rows, ensures
that there is only one, and casts the result to an instance of type C
fetchList().of(C) fetches all rows and cast them to a List
iterate().as(C) lazy iteration over the rows as a Iterator

Notice that I moved the assertions in front of the fetching operation,
because hasMaxRows(1).fetchList().of(C) is legal too (I'd expect this to
return a list of zero or one elements).
and this opens up the door to future assertions.
For count() we'd do something along the lines of:
select(Expressions.count(customer)).from(customer).where(...).fetchFirst().as(Integer.class)
... I like how this reads a lot closer to raw SQL/JDBC code (lower learning
curve).
As for making Expressions more discoverable, I guess we could start off
by using them more frequently in the documentation (show at least 2-3
different examples showcasing things like invoking count(), user
functions, advanced expressions, etc)
—
Reply to this email directly or view it on GitHub
https://github.com/querydsl/querydsl/issues/1120#issuecomment-92199378.

here are the Projectable/Fetchable method names for review, the first option is the current choice and the second an alternative

list -> fetch or fetchAll
iterate -> fetchIterate or iterate
firstResult -> fetchFirst or fetch
uniqueResult -> fetchOne
listResults -> fetchResults
count -> fetchCount

at least fetchIterate feels weird, so maybe iterate is better? what about the others?

I like the second option for all methods.

Correction. I like fetchAll, iterate (second option) but then fetchFirst (first option).

For fetchOne, consider fetchOnlyOne instead. The latter implies there must be only one, whereas the former sounds like "give me any one".

I'd still like to keep the fetch variant as the default, since returning all matching results feels like a reasonable default behaviour, so

list -> fetch
iterate (no change)
firstResult -> fetchFirst
uniqueResult -> fetchOne
listResults -> fetchResults
count -> fetchCount

Sorry for this late comment on that issue, but got the map()-method removed?

We did move the projection logic to Projections.
So getting a Map result could be done like this:
query()
.select(Projections.map(employee.firstname, employee.lastname))
.from(employee)
.fetch()

What about the alternative for a list method that take a parameter?


[Bug]: Polyfill errors when using Angular CLI 15.1.5, Node: 16.16.0 and TypeScript.

Bug expectation
I am using MacOS Ventura 13.1.
Installing puppeteer works just fine via "npm install puppeteer".
I start the app via "ng serve" in Terminal.
I can import it via import "puppeteer from 'puppeteer';", nothing happens yet.
Then when I add the following code
(async () => {      puppeteer.launch();     })
I get the following errors in my Terminal:
`./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserFetcher.js:34:0-25 - Error: Module not found: Error: Can't resolve 'os' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "os": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserFetcher.js:35:0-29 - Error: Module not found: Error: Can't resolve 'path' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserFetcher.js:36:0-27 - Error: Module not found: Error: Can't resolve 'url' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "url": require.resolve("url/") }'
- install 'url'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "url": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserFetcher.js:37:0-29 - Error: Module not found: Error: Can't resolve 'util' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "util": require.resolve("util/") }'
- install 'util'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "util": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserRunner.js:29:0-46 - Error: Module not found: Error: Can't resolve 'child_process' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserRunner.js:30:0-25 - Error: Module not found: Error: Can't resolve 'fs' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserRunner.js:31:0-29 - Error: Module not found: Error: Can't resolve 'path' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserRunner.js:32:0-37 - Error: Module not found: Error: Can't resolve 'readline' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
./node_modules/puppeteer-core/lib/esm/puppeteer/node/BrowserRunner.js:33:0-33 - Error: Module not found: Error: Can't resolve 'util' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "util": require.resolve("util/") }'
- install 'util'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "util": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/ChromeLauncher.js:8:0-32 - Error: Module not found: Error: Can't resolve 'fs' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
./node_modules/puppeteer-core/lib/esm/puppeteer/node/ChromeLauncher.js:9:0-38 - Error: Module not found: Error: Can't resolve 'fs/promises' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
./node_modules/puppeteer-core/lib/esm/puppeteer/node/ChromeLauncher.js:10:0-20 - Error: Module not found: Error: Can't resolve 'os' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "os": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/ChromeLauncher.js:11:0-24 - Error: Module not found: Error: Can't resolve 'path' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/FirefoxLauncher.js:2:0-20 - Error: Module not found: Error: Can't resolve 'fs' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
./node_modules/puppeteer-core/lib/esm/puppeteer/node/FirefoxLauncher.js:3:0-20 - Error: Module not found: Error: Can't resolve 'os' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "os": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/FirefoxLauncher.js:4:0-24 - Error: Module not found: Error: Can't resolve 'path' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/ProductLauncher.js:28:0-32 - Error: Module not found: Error: Can't resolve 'fs' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
./node_modules/puppeteer-core/lib/esm/puppeteer/node/ProductLauncher.js:29:0-32 - Error: Module not found: Error: Can't resolve 'os' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "os": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/ProductLauncher.js:30:0-28 - Error: Module not found: Error: Can't resolve 'path' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }
./node_modules/puppeteer-core/lib/esm/puppeteer/node/PuppeteerNode.js:28:0-28 - Error: Module not found: Error: Can't resolve 'path' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer-core/lib/esm/puppeteer/node'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }
./node_modules/puppeteer/lib/esm/puppeteer/getConfiguration.js:1:0-29 - Error: Module not found: Error: Can't resolve 'os' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer/lib/esm/puppeteer'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }'
- install 'os-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "os": false }
./node_modules/puppeteer/lib/esm/puppeteer/getConfiguration.js:2:0-28 - Error: Module not found: Error: Can't resolve 'path' in '/Users/erikpeeters/Git/compare-site/node_modules/puppeteer/lib/esm/puppeteer'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.
If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }`
Bug behavior

[ ] Flaky
[ ] PDF

Minimal, reproducible example
import puppeteer from 'puppeteer';

(async () => {
    await puppeteer.launch();
})

Error string
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it.
Puppeteer configuration
No response
Puppeteer version
19.7.2
Node version
16.16.0
Package manager
npm
Package manager version
9.2.0
Operating system
macOS

Thanks for the report. From the error message, it looks like you need configure proper polyfills for your webpack version. Also, if you want to run Puppeteer in the Browser you probably don't need to include the node subfolder into the webpack.


Diffie-Hellman exercise: Cannot pass test PrivateKey(2)

Hello,
I am working on the Diffie-Hellman exercise in the Go track. Since the function PrivateKey(p) must generate a private key strictly larger than 1 but also strictly smaller than p, I cannot pass this test.
This test would require me to generate an integer that is both greater than 1 but also smaller than 2, which is impossible. Or is there something I have misunderstood?

My mistake, I think I may have made something wrong with pointers, and misinterpreted the failing message.


module 'comfy.ops' has no attribute 'Linear'

Not sure if this is a workflow-related issue.
Error occurred when executing ADE_AnimateDiffLoaderWithContext:
module 'comfy.ops' has no attribute 'Linear'
The above is generated by the AnimateDiff Loader node when using this workflow:
https://civitai.com/models/297515/a-simple-vid2vid-animatediff-comfyui-workflow
Tried using several different mm models with the same result.
Attempting the code modification proposed on the following page results in breaking the AnimateDiff Loader and Uniform Context Options nodes, preventing them from loading properly (red boxes).
https://github.com/ArtVentureX/comfyui-animatediff/issues/76

Uninstalled/reinstalled AnimateDiff nodes, seems to have resolved the error.


Support test target for desktop msbuild

Description
User should be able to run tests specifying the test target with desktop msbuild. For example
> MSBuild testapp.csproj /t:VSTest

This target should first build the app and then run tests for all the outputs (similar to how dotnet test behaves today).
To get this target, user may need to add a package reference to Microsoft.NET.Test.Sdk. This package will include the target and the related task.
Related customer ask: https://github.com/dotnet/sdk/issues/546

Until this is done, some projects cannot successfully run tests this way. dotnet test cannot execute a test if the csproj contains targets that use a desktop based Task. There are many existing useful build tasks that can be used to generate a netcoreapp project but they must be run using desktop msbuild.

I don't think MSBuild testapp.csproj /t:VSTest should force build on the project. If I wanted to build the project I would have used /t:Build;VSTest.

Also I'd consider calling the target Test instead of VSTest, to be consistent with dotnet test.

Is there a workaround for this? I have a multi-targeted test library that can only be built with desktop msbuild. I want to run tests!

@onovotny Check this one out:
https://github.com/dotnet/roslyn-tools/blob/master/src/RepoToolset/Test.props
https://github.com/dotnet/roslyn-tools/blob/master/src/RepoToolset/Test.targets
You should be able to use these in your project.

(the .props file might not be needed)

Thanks, will check it out. Do you use anything for Code Coverage? I was using opencover but that seems broken lately and def doesn't support ppdb's or embedded ppdb's yet.

@tmat your targets had something that might actually work for me -- after running the build normally with msbuild /t:build, I tried using dotnet test --no-build. I think that actually worked! The issue is that I'm using GitVersionTask in some of my projects and those tasks don't yet support coreclr. So once it's built, it doesn't matter.

Looks similar to issue #1155 I raised. Is there any update when "msbuild /t:VSTest" will be able to test both dncore and regular desktop app ? Thanks!

I have removed the question tag. We will use this item as an input when we take up planning for Q2 of next year.

Bump. Using dotnet test --no-build no longer works for certain scenarios. Really need a way to invoke the targets via an MSBuild target.

I should add that as-of MSBuild 15.6. we can now have SDK's distributed via NuGet, so it should be a lot easier to support adding this as a test sdk out-of-band for both msbuild and dotnet.

Anyone planning to fix this? Building via dotnet msbuild CLI and MSBuild.exe is generally inconsistent.

Bump?

@onovotny We haven't decided when to pick this yet. If you would like to contribute, Let me know I'm happy help you.

Bump.

is there at least a workaround for this? e.g. can I somehow tell msbuild.exe to import the targets explicitly?

Y'all should probably close this issue if the future milestone gets removed without a replacement. As much as i'd like to have the feature i'd rather take a set answer on whether its something y'all want to take on.

@novacole, this is what you're looking at with https://github.com/dotnet/msbuild/pull/9193, right?

@novacole, this is what you're looking at with https://github.com/dotnet/msbuild/pull/9193, right?

Yes, that's correct!

This is a new feature and won't be implemented, we are focusing on adding new features to Testing.Platform instead. https://aka.ms/testingplatform


Intellisense only shows single option on first occurance on new line

VSCode Version: 1.51.1
OS Version: macOS 10.15.7

Steps to Reproduce:

Save and hit enter to begin typing on a new line
Do something like the image seen below (Fig. 1)

Fig.1 - when first typing out the line

Fig. 2 - hitting escape then cmd+space again in the same spot, or just deleting the . and typing it again

Does this issue occur when all extensions are disabled?: Yes/No (it's intellisense)

Seemed to be an issue with the go linter, closing


defaultIgnoreParams and ignoreParams are not working properly

Hi,
Recently, I finished developing a pipeline using nextflow (v24.04.3), but I am having problems with nf-schema (>=2.0.0).
I have 2 conflicting values in the pipeline: pfam_release (Default value: “37.0”) and rnacentral_release (Default value: “24.0”). Both parameters are strings and so I make it known in the nextflow_schema.json putting as type string. However, when I run the pipeline putting as parameter for example --pfam_release 38.0 or --pfam_release “38.0”, it returns an error because it considers it as an integer and in the nextflow_schema.json it is a string.
ERROR ~ Validation of pipeline parameters failed!

 -- Check '.nextflow.log' file for details
The following invalid input values have been detected:

* --pfam_release (38.0): Value is [integer] but should be [string]

Therefore, I decided to use both the defaultIgnoreParams and ignoreParams validation parameters, as shown in this code extracted from my nextflow.config:
validation {
  defaultIgnoreParams = ["pfam_release", "rnacentral_release"]
  help {
    enabled = true
    command = "nextflow run main.nf -profile <docker/singularity/.../institute> --input samplesheet.csv --outdir <OUTDIR>"
  }
}

However, the error remains the same. So, are defaultIgnoreParams and ignoreParams working properly? @nvnieuwk
lenientMode = true is not a solution for me because I don't want to be lax with all parameters. I just want those two not to be evaluated. A temporary solution is to remove from the nextflow_schema.json the line where the type of these two parameters is specified, but if I want to edit it again using nf-core pipelines schema build I get an error precisely because the type lines in those parameters have disappeared.
Thanks in advance
Pascual

Hi @pasviber
Thank you for reporting this. Can you provide me some more information like what version of nf-schema and Nextflow you are using exactly?

The nextflow version is v24.04.3, but in the case of nf-schema I have checked several, for example<EMAIL_ADDRESS><EMAIL_ADDRESS>and<EMAIL_ADDRESS>In all versions I get the same problem. The problem does not seem to be linked to any particular version of nf-schema.

To reproduce the error, using the command line, try entering the value 37.0 in a parameter (defined as a string in the nextflow_schema.json) of one of the pipelines you use. This will obviously return an error during parameter validation because it will find an integer where there should be a string. Then, try putting this parameter in the defaultIgnoreParams list and re-run the pipeline with the same command. If the error persists, something is wrong.
Thanks @nvnieuwk :)

Hi I've found the issue and have posted a fix in PR #78 :) I'll let you know once this is in a released version of the plugin


Odd characters in collapsed arrays in sample json

I expect to see something like this in the sample area (right hand of my screen):

"items":[]

instead i get:

"items":[â€|]

the + sign works to expand the array as expected and it all looks good within.  however, every array in my json collapses to [â€|]
Not sure if this is user error or some kind of issue.

Dup of https://github.com/Rebilly/ReDoc/issues/29

Here is the solution: https://github.com/Rebilly/ReDoc/issues/213#issuecomment-282761418


Did not support on old Mac, for example MacBook 2015 [Toothpick] ...

Extension
https://www.raycast.com/VladCuciureanu/toothpick
Description
When tap Manage Blue Connections，there is nothing found on my MacBook 2015pro
Steps To Reproduce
No response
Current Behaviour
No response
Expected Behaviour
No response

Hey! Could you please share the output of the following command?
system_profiler SPBluetoothDataType -json
Thanks

Hey! Could you please share the output of the following command? system_profiler SPBluetoothDataType -json Thanks

{
  "SPBluetoothDataType" : [
    {
      "controller_properties" : {
        "controller_address" : "AC:BC:32:94:DF:34",
        "controller_chipset" : "BCM_20703A1",
        "controller_discoverable" : "attrib_off",
        "controller_firmwareVersion" : "v169 c4825",
        "controller_productID" : "0x0001",
        "controller_state" : "attrib_on",
        "controller_supportedServices" : "0x382039 < HFP AVRCP A2DP HID Braille AACP GATT Serial >",
        "controller_transport" : "USB",
        "controller_vendorID" : "0x004C (Apple)"
      },
      "devices_list" : [
        {
          "Jabra Elite 65t" : {
            "device_address" : "70:BF:92:24:6A:C0",
            "device_firmwareVersion" : "2.2.2",
            "device_minorType" : "Headset",
            "device_productID" : "0x247E",
            "device_vendorID" : "0x0067"
          }
        },
        {
          "JBL T280TWS" : {
            "device_address" : "F0:A9:68:10:5B:71",
            "device_minorType" : "Headset"
          }
        },
        {
          "shin" : {
            "device_address" : "08:C7:29:2B:BB:15"
          }
        },
        {
          "Beats Solo³-shin" : {
            "device_address" : "DC:D3:A2:BB:E6:A6",
            "device_minorType" : "Headphones",
            "device_productID" : "0x2006",
            "device_vendorID" : "0x004C"
          }
        }
      ]
    }
  ]
}

I'm experiencing the same issue on a 2015 MacBook. Interesting that it used to work a while ago, but after some update it stopped. It just doesn't show any device, even tho there are multiple bluetooth connections.

Thanks for the additional data! I've been experiencing a very tight schedule for the last few weeks so I didn't have the time to look into it but I wrote this bug down and will be in the next update for sure.

@VladCuciureanu @peaceyourself The problem has been fixed after updating to MacOS Ventura 🎉

Glad to hear that! I will investigate the bug when I have some time on my hands though. Wouldn't want older versions to be incompatible.


Highlighting in text area is out of position, and moves moves left when typing more text.

Hi,
When mentioning a user, the highlight appears in the middle of the textarea, and when typing more text the textarea moves with each character to the left. Im thinking I should be adding more style to the highlighted class? Provided a picture.

Hey @Tran-Eric could you share your setup? I see some extra styles. Also it would be interesting which browser you used.

i figured it out. I had css property text-align: center inherited from the parent.


ZeroDivisionError: division by zero

While running any of the training DTU commands i get a dividing by zero error in the neus.py file
# train NeuS on DTU without mask
python launch.py --config configs/neus-dtu.yaml --gpu 0 --train
# train NeuS on DTU with mask
python launch.py --config configs/neus-dtu-wmask.yaml --gpu 0 --train
# train NeuS on DTU with mask using tricks from Neuralangelo (experimental)
python launch.py --config configs/neuralangelo-dtu-wmask.yaml --gpu 0 --train

print statements output for tracing :

this occurs when there are no points being sampled during ray marching. you probably need to investigate training - have u tried visualizing the rendered images during validation?

Hello, I also encountered the same problem, may I ask you to solve it?


Add targets

This PR adds support for targets, which allow a document to know what it's being compiled into.
This is part of the basics that will eventually allow HTML compilation.
Currently missing:

tests (might not be needed? They don't really work with the existing testing pipeline, since they're meta)
clippy. So much clippy (probably)

In my opinion, it should not be PNG but raster or similar. We create a pixel buffer, the file format / compression is incidental. For example, we could also produce a TIFF or JPEG but the target considerations should be the same.

Should I change SVG to vector or similar, then?

I ended up changing Png to Raster and Svg to Vector. Pdf stays the same since it's really the only format that renders... like that.

To be honest, I'm personally somewhat against having such a feature in Typst, at least at the current stage. Because by introducing the possibility to add target-specific code, we can no longer expect all of the output formats to look the same, which in my opinion is an unfortunate side-effect, especially for the web app, since then the outputted PDF might look differently than what was shown in the preview.
I'm aware that since HTML support is planned, at some point there might have to be a way to introduce target-specific code (although I have no clue how this could be done in a nice way), but at least for now ,while we still have the same frame-based representation for each output format (PNG, PDF, SVG), I would personally steer away from allowing users to specify target-specific code.
Just my two cents though. 🙂

We could perhaps delay this change to when HTML export is added, and maybe have a way to differentiate HTML export from the rest. Regarding differentiating PDF / PNG / SVG, that could be postponed until we have a proper discussion on whether that's worth it or not.

I originally thought that there was no harm in supporting this as is already, but it is true that there's still a discussion to be had how fine-grained targets should be. It could be per-format with simple accessors for groups of related formats or it could be that we only expose the group information. We could even store the format internally, but not expose it and that way still have overlapping groups of formats (like is-layouted, is-x, is-y).

I'll close this for now as things will still take a bit. We can see again then. I might also just add this alongside other stuff when I'm working on the relevant parts.


downgrade grafana, because of nil exc in labels

Failed to load target state: failed to generate manifest for source 2 of 2: rpc error: code = Unknown desc = .metadata.labels accessor error: contains non-string key in the map:  is of the type , expected string


Add support for primary public IP

Hi, do you plan add support for the new feature released by Hetzner Cloud API?
Reference: https://docs.hetzner.cloud/#primary-ips

Hey @matteobaldassin,
yes the plan to add support is there, how ever I currently have not that big time to do it in the next few weeks. If you want you can try to contribute it :)

@LKaemmerling I would take care of this during Hacktoberfest and add a basic pull request. I orientate myself on "Image" for the models, but still need support for testing later.

@ayacoo sure feel free to do so :) if you need any help just write me then :)

Was implemented by @ayacoo thank you!


Walkthrough covered by Header and Tabs

I've added a walkthrough to one of my pages, but it's getting covered up by the header and the tabs. If I put in my body, it works fine, but it's a little harder to control from each page controller. Should it be getting covered up?

Actually when I run it on iOS, the overlay screen doesn't show up at all, just the text and arrow. When I refresh the page and reset the is-active flag to true, it never shows up again.

Looks like the maximum spread value I can set on iOS is 1021px. Anything higher and it just disappears.. at least on an iPhone 5S running iOS 9.0.1

Hey @randdusing,
Regarding the first issue, at the current version the element acts as a regular directive thus influenced by the DOM location as any other DOM element, in Ionic's case, not covering header and tabs. In order to deal with this that at least these parts shall not be clickable you can use the 'isBindClickEventToBody' property.
Regarding the second iOS issue, I'll try to get to it in the close days and check it out, update us is anything new comes up regarding this.
Thanks

Hey @randdusing, The new version should have solved the first issue, and might have solved the second as well.
Any chance you check it out and give us feedback?
Thanks

Hi souly1, I'm no longer working on the mobile app that used this, but I'll try it out on fresh Ionic app.

Sounds great, thanks!

I don't remember exactly how I was using ng-walkthrough previously, but first issue is resolved. The other issue with the whole still occurs even using the latest version of Cordova iOS and iOS itself. Most likely an issue with Safari not liking huge box shadow values.

cool @randdusing , Highly appreciate you taking the time to check. I'll close the issue for and if will surface again deal with it then.
Thanks!

Thanks for your work!


Oss-fuzz integration

Description
Integration with oss-fuzz to do fuzz testing
Status
IN DEVELOPMENT
but already consistent, useful
What is left to be done :

better corpuses (for client/server communication)
use of different options for client/server communication

Requires Backporting
No
Migrations
No
Additional comments
Found already one memory leak
Todos

[ ] Tests
[X] Documentation
[X] Changelog updated
[ ] Backported

Steps to test or reproduce
see https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md
Basically :

install docker
Get my oss-fuzz pull request git clone --depth 1 --branch mbedtls https://github.com/catenacyber/oss-fuzz.git oss-fuzz
Run the commands from oss-fuzz directory

python infra/helper.py build_image mbedtls
python infra/helper.py build_fuzzers --sanitizer address mbedtls other sanitizers are available : memory and undefined
python infra/helper.py run_fuzzer mbedtls fuzz_x509crl or using another of the fuzz targets

Wait for the fuzzer to find a bug

I guess we can switch to READY.
There are corpuses for dtls_client, dtls_server, client and server.
These corpuses are just one file with one valid connection and message exchange (built with sample programs using pseudo random instead of safe random)
Last bytes for client and server are used as bit masks for options (use of encrypt then mac extension for example)
I changed some files in library, adding only macros such as #ifdef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION to avoid use of timestamps.
Another solution would be to use the environment variable CFLAGS and undefine MBEDTLS_HAVE_TIME when running make.
I found another bug #1624
Sanitizer memory does not seem useful for client-server communication because there is asm used by that code such as function `mbedtls_aesni_gcm_mult.

Great, I will work on it next week or so

CI is still unhappy. Please restore the commit that adds #include "bignum.h" into x509_crt.h. It uses the type mbedtls_mpi so that type needs to be defined.
That was a preexisting bug: each header is supposed to work on its own. I guess we never discovered it because it only turns up in configurations without X509 support (otherwise MBEDTLS_BIGNUM_C would be defined and so asn1.h would pull in bignum.h), and applications that build without X509 support don't usually include X509 headers.

Please restore the commit that adds #include "bignum.h" into x509_crt.h

done

What remains to be done :

Support building of the fuzz targets with make (and no cmake)
Corpus generators
See if CI is happy and test

The CI was happy except for a build error in config.pl full which is due to what I consider to be a preexisting bug: https://github.com/ARMmbed/mbedtls/issues/2671 . Please bear with us while we decide how to resolve that.

The CI was happy except for a build error in config.pl full which is due to what I consider to be a preexisting bug: #2671 . Please bear with us while we decide how to resolve that.

Ok, cool
Are you ok with my other fixes ?

Manual test looks good to me with https://github.com/catenacyber/oss-fuzz/tree/mbedtlsok
So now only remains corpus generation for client/server fuzz targets.
After that, there will still be work to be done :

check the code coverage by fuzzing
profile the fuzzing performance
make sure the fuzz targets get built with sanitizer support by CI
run them against the corpus during CI
maybe even add structure aware fuzzing for what happens after handshake...

New report from CI (I haven't reviewed your commits from this afternoon yet):

In the same way fuzz_x509*.c are protected by #ifdef MBEDTLS_X509_xxx, please protect fuzz_*server.c by #ifdef MBEDTLS_SSL_SRV_C and fuzz_*client.c by #ifdef MBEDTLS_SSL_CLI_C.
There seems to be something wrong with how build options are passed to tests/fuzz/Makefile or how it uses them. Building with make CFLAGS='-m32' seems to build the fuzzer executables without -m32. To reproduce: ./tests/scripts/all.sh -k component_test_m32_o0

Oh, and please refactor your commits so that there's no temporary modification in the library directory. Please do keep the history otherwise, don't squash everything into a single commit, just squash “Reverting changes to library” into “Fuzz testing”.
Please don't change the base of the PR even if we merge stuff into the target branch in the meantime, to facilitate re-review.

just squash “Reverting changes to library” into “Fuzz testing”

Done.
So, What remains for you :

CI fail from #2671
Choose style #ifdef versus #if defined
Set LDFLAGS for make CFLAGS=-m32
And for me :
better corpus generation for client/server fuzz targets (non blocker)

Ok, I have a way to produce corpus files.
Methodology is the following :

Build mbedtls programs ssl_server2 and ssl_client2 with a patch to make communications reproducible
Run them one against the other while capturing trafic
Extract tcp payloads with tsharktshark -Tfields -e tcp.dstport -e tcp.payload -r test.pcap
Run a dummy python script to output either client or server corpus file like https://github.com/OISF/libhtp/blob/e242fad4205e31b7f6dc6e240013c9541d612e9a/test/pcaptohtp.py

What do you think of this ?
Do you want me to submit the patch adding option reproducible to ssl_server2 and ssl_client2 ?

Do you want me to submit the patch adding option reproducible to ssl_server2 and ssl_client2 ?

I made one of those a while ago… It's very useful for debugging as well as fuzzing. Sadly mine needs a lot of rework to adjust to library changes. If you have one that works against the current state of the library, we'd be happy to take it.
But in a separate PR, please. Let's not delay this one by scope creep.

We may ask you to make further patches or to rebase on top of changes in our development branch.

Ok, I am waiting for your fixes or for you asking me patches

It's very useful for debugging as well as fuzzing. Sadly mine needs a lot of rework to adjust to library changes. If you have one that works against the current state of the library, we'd be happy to take it.

It is now waiting for review here #2681

Ok, I used your 2 commits and am waiting for the rebase
I removed onefile from the gitignor as we should only build fuzz_* for programs

@2697 has been merged. If the conflict in the ChangeLog is resolved, the pr-merge job is expected to run and pass.

So, here is the rebased version with fixed conflict

Should I add the methodology to produce corpus files in README.md (now that #2681 got merged) ?

Also we have a server only build (MBEDTLS_SSL_CLI_C not defined) complaining about unused initialized and cacert variables and a client only build  (MBEDTLS_SSL_SRV_C not defined) complaining about unused initialized, srvcert, pkey variables.

I see the problem with cmake in-tree builds: it's due to the cleanup we do in all.sh, which restores Makefile from git. It isn't a problem with CMakeLists.txt. This should fix it: https://github.com/gilles-peskine-arm/mbedtls/tree/pr_1622-fix_all.sh

About the corpus methodology: by all means, please do. It's always better to document than not to document.

Although #2697 has been merged, we still have a test that tests MBEDTLS_CHECK_PARAMS specifically. test_check_params_functionality unsets MBEDTLS_CHECK_PARAMS_ASSERT and calls make CC=gcc CFLAGS='-Werror -O1' lib test. Since the fuzz directory is in there, the fuzz targets will be compiled as well and fail with undefined reference to mbedtls_param_failed().

@yanesca Good point. Should fuzz be under programs?

I think that the corpus should be regularly tested and I think it makes sense to consider them tests, so leaving them in tests would be a natural choice.
We have regularly tested sample applications in programs (ssl_server2 and ssl_client2) but those feel more like exceptions to me.
I would prefer changing test_check_params_functionality() to exclude fuzz from the build.

@yanesca But we aren't testing the corpuses in every build, and I don't think they count as unit tests. I'm pushing to test programs more (https://github.com/ARMmbed/mbedtls/pull/2698) and we could add (in a subsequent PR) a fuzz_demo.sh script that runs in every CI job.

@gilles-peskine-arm That is true, sounds good to me.

That should be good except for test_check_params_functionality
I am not sure to understand what your conclusion is.
If it is to change test_check_params_functionality to remove fuzz directory from it, I do not see how to exclude it from     make CC=gcc CFLAGS='-Werror -O1' lib test except by setting an environment variable before this line ?

@catenacyber Please move tests/fuzz to programs/fuzz and adjust {tests,programs}/{Makefile,CMakeLists.txt} accordingly. Putting auxiliary programs under programs fits our directory layout better, even if they're programs meant for testing. We have precedents with ssl_{client,server}2 which are test programs far more than sample code.

We need to adapt 1c582c3 too to the change in 48f35f5.

Thank you @yanesca for spotting this

The CI deserves all the credit :wink: . There is one (hopefully) last issue left.

Why is it necessary to set back the crypto submodule to an earlier version?

Sorry, wrong git manipulation

So, do we merge ?

Hi @catenacyber
I was about to merge this today when I noticed it doesn't build with the code coverage test (./tests/scripts/basic-build-test.sh) and it also breaks the Windows build.
Could you look into the issues with ./tests/scripts/basic-build-test.sh please?

Thanks @Patater do you have a pointer for more info ? ie logs of the broken builds

@Patater Isn't it the same problem we've had before with link flags being put in CFLAGS only So tests/scripts/basic-build-test.sh should have
export CFLAGS=' --coverage -g3 -O0 ' LDFLAGS='--coverage'

@gilles-peskine-arm That sounds like the issue. Thanks for having a look.

So, do I need to do something ?

@catenacyber I have raised a PR with the missing line here: https://github.com/ARMmbed/mbedtls/pull/2741
You can merge it into your branch, which will place my commit here, or @Patater may merge my small PR before yours, which will render this PR fine in the coverage scenario.
@Patater, what do you think?

@catenacyber The simplest solution would be if you cherry-pick https://github.com/ARMmbed/mbedtls/pull/2741/commits/ee292a06c68ae2b4cc9ceb14aa2599ee64412caa into this PR.

Snippet from Windows error log:
       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj.metaproj" (Rebuild target) (99) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj" (Rebuild target) (273) ->

       (ClCompile target) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(47): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj.metaproj" (Rebuild target) (100) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj" (Rebuild target) (258) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(47): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj.metaproj" (Rebuild target) (104) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj" (Rebuild target) (256) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(47): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj.metaproj" (Rebuild target) (101) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj" (Rebuild target) (255) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(47): warning C4267: 'return': conversion from 'size_t' to 'int', possible loss of data [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj.metaproj" (Rebuild target) (99) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj" (Rebuild target) (273) ->

       (ClCompile target) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): error C2220: warning treated as error - no 'object' file generated [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_client.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509crl.vcxproj.metaproj" (Rebuild target) (105) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509crl.vcxproj" (Rebuild target) (259) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509crl.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj.metaproj" (Rebuild target) (100) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj" (Rebuild target) (258) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): error C2220: warning treated as error - no 'object' file generated [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsclient.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_pubkey.vcxproj.metaproj" (Rebuild target) (103) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_pubkey.vcxproj" (Rebuild target) (257) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_pubkey.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj.metaproj" (Rebuild target) (104) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj" (Rebuild target) (256) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): error C2220: warning treated as error - no 'object' file generated [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_server.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_privkey.vcxproj.metaproj" (Rebuild target) (102) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_privkey.vcxproj" (Rebuild target) (254) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_privkey.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509crt.vcxproj.metaproj" (Rebuild target) (106) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509crt.vcxproj" (Rebuild target) (274) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509crt.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509csr.vcxproj.metaproj" (Rebuild target) (107) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509csr.vcxproj" (Rebuild target) (275) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_x509csr.vcxproj]

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\mbed TLS.sln" (Rebuild target) (1) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\ALL_BUILD.vcxproj.metaproj" (Rebuild target) (2) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj.metaproj" (Rebuild target) (101) ->

       "C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj" (Rebuild target) (255) ->

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\common.c(29): error C2220: warning treated as error - no 'object' file generated [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj]

         C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\programs\fuzz\onefile.c(2): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\builds\ws\mbedtls-release-new\worktrees\tmpya4kd2jn\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj]

    8 Warning(s)

    13 Error(s)

Time Elapsed 00:00:24.32

Oh, dammit, cmake+MSVC. Why do we even support this? I hope this will fix it. CI run: https://jenkins-internal.mbed.com/job/mbedtls-release-new/336/

Here is another proposal

The CMake+MSVC build is still failing due to two errors.
In onefile.c:
warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

In Mbed TLS, we address this with the following snippet in config.h:
#if defined(_MSC_VER) && !defined(_CRT_SECURE_NO_DEPRECATE)
#define _CRT_SECURE_NO_DEPRECATE 1
#endif

I think the best solution here would be to include "mbedtls/config.h" in onefile.c, even if it doesn't use any mbedtls_xxx identifier. This way it would have any platform-specific definition that people may want to use on “exotic” platforms.
In many files:
fatal error C1083: Cannot open include file: 'stdbool.h': No such file or directory

That's even with Visual Studio 2017, and we currently test with 2010, 2013 and 2015.

Should I rebase for ChangeLog entry ?

Please don't rebase just because there's a conflict in ChangeLog. We have ad hoc merging rules that Github doesn't support.

Sorry, https://github.com/ARMmbed/mbedtls/pull/1622/commits/42a2ce8255e75ea54182e21e9ff611c1c9d19f22 didn't work to get Visual Studio to accept fopen, and I don't understand why since it's working in other source files.
   262>ClCompile:
         C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\CL.exe /c /I"C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\include" /I"C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\crypto\include" /nologo /W3 /WX /O2 /Ob2 /Oy- /D WIN32 /D _WINDOWS /D NDEBUG /D "CMAKE_INTDIR=\"Release\"" /D _MBCS /Gm- /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Fo"fuzz_dtlsserver.dir\Release\\" /Fd"fuzz_dtlsserver.dir\Release\vcWindows7.1SDK.pdb" /Gd /TC /analyze- /errorReport:queue "C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\programs\fuzz\fuzz_dtlsserver.c" "C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\programs\fuzz\common.c" "C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\programs\fuzz\onefile.c"
   262>ClCompile:
         fuzz_dtlsserver.c
   262>ClCompile:
         common.c
   262>ClCompile:
         onefile.c
   262>C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\programs\fuzz\onefile.c(19): error C2220: warning treated as error - no 'object' file generated [C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj]
   262>C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\programs\fuzz\onefile.c(19): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj]
                 C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\stdio.h(234) : see declaration of 'fopen'
         Generating Code...
   262>Done Building Project "C:\builds\ws\mbedtls-release-new\worktrees\tmp341hyvcs\cmake_solution\programs\fuzz\fuzz_dtlsserver.vcxproj" (Rebuild target(s)) -- FAILED.

Thanks for the update. New CI run with full Windows jobs at https://jenkins-internal.mbed.com/job/mbedtls-release-new/343/

The build of 39a615b9d6f54668676701ba9bac3c89e13b3319 is failing in VS2010 through VS2017 with the same errors about a deprecated fopen. I don't understand why.

I propose to cut our losses and skip the compilation of the fuzzing programs on Windows. This commit has passed our CI except for the basic-build-tests.sh job which wasn't on my branch.
@catenacyber Unless you've figured out how to make the Windows build work, please remove the commit “Removes warnings for fopen” and cherry-pick https://github.com/gilles-peskine-arm/mbedtls/tree/pr_1622-no_fuzz_on_windows instead.

Let's cut losses

Running release job at https://jenkins-internal.mbed.com/view/mbed-tls/job/mbedtls-release-new/347/

Release job passed. Yay!

@catenacyber Thanks very much for your contribution and for you patience in landing this!

And thank you all for your help :-)

Now that this is merged, we can take a look at what we could still do better :
Are the fuzz targets built with sanitizer support ?
See for instance https://github.com/aquynh/capstone/blob/master/.travis.yml#L45
If so, are they run for regression testing ?

We are building it with sanitizer support in all.sh, but we are not running it. all.sh already runs in Jenkins, so running the fuzztests on the relevant sanitizer builds should get us there.

Hmm, so how can I do to make, let's say, fuzz_pubkey run on tests/data_files/*.pub ?

I can see two options:

write a function in all.sh that copies the files into a temporary corpus directory, runs the program and checks the return code on the files in the said directory one by one and then cleans up. Call this function on the builds that already use sanitiser.
copy the files into a temporary corpus directory, add a new build to all.sh that builds with sanitisers plus the fuzzer and then runs the targets just on the corpus (I believe it can be done with -runs command line parameter in case of libfuzzer)

(The second feels more straightforward and elegant to me.)

Thank you @yanesca
I think that I still have questions

add a new build to all.sh
Is component_test_default_cmake_gcc_asan what you call a build ?
Can you point me to some lines of code of something similar to what I am trying ?

(I believe it can be done with -runs command line parameter in case of libfuzzer)
I do not need libfuzzer here
I rather use simple onefile.c driver

Call this function on the builds that already use sanitiser
I do not see any setting MBEDTLS_PLATFORM_TIME_ALT which is needed for client and server fuzz targets

Yes, it is. I think component_test_memsan and component_test_default_cmake_gcc_asan are pretty similar to what you are trying to do.
That approach has its advantages as well, because I guess we could build the libfuzzer version only with clang, the onefile.c driver version we can build with anything.
I think the best would be to create a new component_test_corpus_gcc and a component_test_corpus_clang to test the corpus and set  MBEDTLS_PLATFORM_TIME_ALT with scripts/config.pl before building the library.

@yanesca let's continue talking about this in #2748


Added Python 3 support (while continuing Python 2 compatibility)

Updated library to Python 3 whilst preserving Python 2 compatibility.
Leveraged try/except blocks for imports instead of using a compatibility library to keep footprint small. Several of the tests relied on Python 2's dict key order-preservation-as-a-side-effect, but this broke in Python 3, so I updated the relevant methods to use an OrderedDict instead of the default orderless dict. As Python 3 no longer had this order-by-luck functionality, I updated the vcr cassettes to put latitude before longitude and start before end, since the tests were expecting a specific ordering of the query parameters (even though this wouldn't be important in real-world functionality). All tests pass (repeatedly) in both Python 2.7 and Python 3.5 now.
Addresses issues #4, #5, & #6.

I've agreed to the Uber CLA as well.

:+1: :+1: :+1: :+1: :+1: :+1: :+1: :+1: thanks @merrywhether!!!

Fantastic, I also just tested in Python 2 and 3. Thanks @merrywhether - OrderedDict usage looks good to me, too.

@christinekim Thanks for merging! Can a release go out on Pip soon?

Yes @robbiet480 I can get that done today. Will update here when it is complete :)

Thanks @christinekim!

Hi all - the newest distribution is up on the Python Package Index: https://pypi.python.org/pypi/uber_rides/0.2.1

Cool, thanks!

:+1:


Incremental recompilation results in type error

steps
Minimal reproducible example repository with Actions
Namely, an empty Scala 3 project (3.0.0 and upwards) with sbt 1.6.2 and the following source files:
// A.scala

import p.*

type NAME2 = C[?]

// p/p.scala

package p

object M {
  class C[N]()
}

export M.*

Then the following commands are executed in sequence:
sbt compile
sed -i 's/NAME/NAME2/g' src/main/scala/A.scala
sbt compile

In other words the code is compiled a first time, then in the file A.scala the type NAME is renamed to NAME2, and finally the code is recompiled.
problem
The third command fails because of a compilation error:
[error] -- [E043] Type Error: /home/runner/work/sbt-incremental-compilation-bug/sbt-incremental-compilation-bug/src/main/scala/A.scala:3:13
[error] 3 |type NAME2 = C[?]
[error]   |             ^^^^
[error]   |   unreducible application of higher-kinded type p.C to wildcard arguments
[error]   |
[error]   | longer explanation available when compiling with `-explain`

expectation
The updated code should compile normally, and in fact does compile normally when the cache is initially empty (or cleaned before recompilation).
Thus all the commands should succeed.
This leads me to believe that there is an issue with incremental compilation.

The incremental compilation logic for Scala 3 is part of the github.com/lampepfl/dotty/ repository, could you open an issue there instead?

Oh, I see, my bad.

Moved to lampepfl/dotty#14858.


Prefer scheduling registry pod onto infra nodes

This commit specifies .spec.grpcPodConfig.nodeSelector and .spec.grpdPodConfig.tolerations to schedule this operator's registry pod to infra nodes when possible.
OSD-6629

Codecov Report

Merging #767 (0aab164) into master (2f690f8) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #767   +/-   ##
=======================================
  Coverage   42.25%   42.25%
=======================================
  Files          42       42
  Lines        5879     5879
=======================================
  Hits         2484     2484
  Misses       3164     3164
  Partials      231      231

/hold
I'm super excited for this feature now that it's available, however I don't think this applies to AWS Account Operator.


Add image generation with aarch64 runtimes

When using the builder action with aarch64 as arch, the action always install aarch64 runtime.
To avoid this, generating x86_64 images with aarch64 runtime is a solution.
Making images per runtime architecture avoid to make images bigger in size.
Images are tagged with a tag suffixed with the runtime arch, this allow using matrix with containers.
x86_64 runtime images are still also tagged with a non-suffixed tag.
This PR was not tested and two of the checks are failing because images with new tags do not exist.

This probably needs #82 first so you can try and publish these new images to your fork, and then test it there.

This probably needs #82 first so you can try and publish these new images to your fork, and then test it there.

The reason that I'm put this PR as draft is to wait for #82, even if I don't want to mess my registry that I never used.

See also #124


Release 0.8.0

Packaging

Fix URL and Source0 in the SPEC file

=====================================================================
Framework

Fixes

Add workround patch for the multiprocessing library
Exit non-zero on unhandled exceptions and errors
Fix actor library loading, so libs do not have to be imported
in lexicographical order
Fix buf with mode inheritance and list field (#203)
Fix warning about membership check of models when (#503)
Load checkpoint ordered by id instead of timestamp to prevent
issues with changed timezone (#499)
Log on the ERROR level by default instead of DEBUG
Print usage to stderr

Enhancements

Add phase name checking to workflow parameters
Add the fetch_upgrade_report_raw() function to obtain raw report
data
Create a dynamic configuration phase that allows creation
of configuration for repository workflow
Functionalite getting actor and phase names when working with
a workflow

=====================================================================
Leapp

Fixes

Add checks of arguments for cmdline parameters
Load checkpoint ordered by id instead of timestamp to prevent
issues with changed timezone (#499)

Enhancements

Add JSON report generation
Add the preupgrade subcommand for the leapp tool that stops
workflow execution after the Reports phase and display info about
generated report

=====================================================================
Snactor

Fixes

Add a simple logger to snactor discover command to avoid spurious
output about missing logger
Add checks of arguments for cmdline parameters
Exit non-zero on unhandled exceptions and errors
Load checkpoint ordered by id instead of timestamp to prevent
issues with changed timezone (#499)

=====================================================================
stdlib

Fixes

Add checks of arguments for cmdline parameters

Enhancements

Log output of commands to logfile (#520)
stdlib: add option to run() to ignore exit code

=====================================================================
Modifications

introduced conftest.py to utilize fixture mechanism, so fixture
doesn't have to be imported explicitly in tests.
refactoring to satisfy linters

Can one of the admins verify this patch?


Roll engine c43739ba53ce..bf138516d729 (3 commits)

git@github.com:flutter/engine.git/compare/c43739ba53ce...bf138516d729
git log c43739ba53ce..bf138516d729 --no-merges --oneline
2019-08-21<EMAIL_ADDRESS>Do not add null task observers (flutter/engine#11315)
2019-08-21<EMAIL_ADDRESS>When using a custom compositor, ensure the root canvas is flushed. (flutter/engine#11310)
2019-08-20<EMAIL_ADDRESS>[glfw] Send the glfw key data to the framework. (flutter/engine#9386)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (liyuqian@google.com), and stop
the roller if necessary.

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/4707066185252864 , https://cirrus-ci.com/task/5832966092095488

Commit queue failed; closing this roll.


[Feature] Bundle Data Into an R Data Package

Once the data set is more mature, consider creating an R data package to make it easier to share/use with the R community?

Research:
https://rstudio4edu.github.io/rstudio4edu-book/data-pkg.html

So this would serve as a template others could apply to their own, possibly copyright-constrained, literature?

Well, the data package would contain the data pulled from Terra Ignota in a format that makes it easy to ingest and work with in R. So it would make analysis of that specific text possible without forking this repo or downloading the individual csv files and reading the file descriptions.
If I do end up writing some reusable code for the analysis or visualization part of the project, I could release that for those who want to apply to their own texts. Not sure how useful it would be, as the heavy-lifting is the XML markup that's going on in the private repo.

OK, that makes sense. Mentally I pictured the private repo as providing just a character stream of the text that you were parsing. I don't own the book so I should not access that side of things.

Private side:

raw text from books
mark up goes on top, in TEI schema XML, to add machine-readable info we humans find easy,
Xquery pulls various subsets of that markup into data files

Public side:

publish the data files (purged of copyrighted material)
explain the structure of the files for anyone that wants to use them
demo some analysis & visualization with that data

OK, the details of how you split it up make sense now that I see it.


Migrate some JVM language projects to JVM platform

@bot-gradle test this

I've triggered the following builds for you. Click here to see all build failures.

PullRequestFeedback build

@bot-gradle test and merge

I've triggered a build for you. Click here to see all failures if there's any.


Request to add a LICENSE file

Hi @wheatjs! Thanks for creating such beautiful vueuse playground! It made getting started and learning Vue and Vueuse exponentially easier for me!
Would you kindly consider adding a license file for this? I wanted to build upon this project for making a front-end for my custom disassembler and assembler project.

Done https://github.com/wheatjs/vueuse-playground/blob/main/LICENSE
Good luck on your project, I'd love to see it when it is done!

Thanks so much! I'm also excited for the project :)


[DO NOT MERGE] Remove unused Sidekiq config

We now have separate Sidekiq configuration files for our two separate
queues. We no longer need this old configuration file.
Dependencies

[ ] https://github.com/alphagov/govuk-puppet/pull/6521

 Use different concurrency sidekiq policy for GA and for PublishingAPI


Feature: Workerized JS Module Script Loading

The future (and hopefully very near future) way to achieve performative module script loading will likely come in the form of implementing “worklet infrastructure” in core since it does not seem like a userland solution should be reasonably expected.

Worklets are a piece of specification infrastructure which can be used for running scripts independent of the main JavaScript execution environment, while not requiring any particular implementation model.
— https://html.spec.whatwg.org/multipage/worklets.html

This issue should help us track progress in this area, which I find to be of great interest.

This might also be beneficial to syncify some loader hooks. For example it is seeming more and more likely Chromium wants to ship a synchronous import.meta.resolve while Node.js has an experimental async version to support the async resolver. Syncifying resolve would permit alignment on these specs, which is currently at risk.

Synchronous import.meta.resolve() would be great, as it being async is a bit of a pain/gotcha.

Thinking out loud about all the potential use-cases for loaders being discussed:
Do any of the http loader use-cases currently assume the ability to make http requests within resolve()?
It's no problem for resolve() behaviors that synchronously query the local filesystem, just wondering about other use-cases that may be assuming access to async operations.

For example, it is seeming more and more likely Chromium wants to ship a synchronous import.meta.resolve while Node.js has an experimental async version to support the async resolver.

So, all I have for Web stuff is:
https://github.com/DerekNonGeneric/web-context-js
The "specifier" part is "opaque" (defined by "host"), until "import maps" finally ships (did it?), that would be a no.
We cannot use "http" at all currently, (nor do we to my knowledge, that is a Deno-only thing currently).

@DerekNonGeneric looks like no, import maps have not yet landed 😢 #477
For remote resolution (which would suggest needing to check outside knowledge), I would expect a prefix like what @bmeck does in his module mock PR. An import map could suffice, but puts a wee bit more onus on the user (who then needs to be aware of unpkg or whoever's conventions—hardly a showstopper, but could maybe be improved with a hook:
Continuing with unpkg, unpkg:react. The unpkg resolve hook would be aware of its own specifier and URL conventions, so it would pick that up, build the URL to whatever it should be, and then a (probably generic https) load hook would check whether it actually exists, is available (eg if authorisation is a factor). This seems like a proper separation of concerns too—resolution is only concerned with producing the URL, and loading is concerned with getting the resource on the other end of that URL.
This (as well as import-map) also works in scenarios where packages/modules are created during the app's lifetime (such as what Framer.com do constantly): resolve doesn't need to be aware of the new package/module, only how to find it.
If the other side of the URL has a package.json specifying subpaths via exports or something similar, that's still load()'s job (which we already know needs to support async) to follow; resolve() still got us to the front door.
If I understand it correctly, import.meta.resolve() is only async because the resolve() hook is. It seems to me neither resolve()s inherently need to be async. I could see a resolve hook wanting (or maybe needing?) to interact with an API that has only an asynchronous version though, which would be a bummer.
If it's possible to keep import.meta.resolve() and loaders resolve() separate and sync and async respectively, I think that could make sense and allow us time to see the resolve() hook in the wild. But if resolve hooks need to be aware that import.meta.resolve() is synchronous, that would be a mighty pitfall IMO.

@guybedford what would be the reason browsers want a sync import.meta.resolve? Seems not to make sense given the inherent asynchronicity of HTTP, and of ESM in general.

@giltayar with import maps, the initial URL in a browser is synchronously available. The final URL, of course, would be both variable and also async, but I can certainly see the need for a sync API to determine the first network request URL that will be contacted. (no idea if browsers actually want this or not, ofc)

@ljharb I  can see that initially they don't really need an async api, but they would be closing the door on a browser loader api that should be (IMO) async.

Right, I was always personally hopeful for an async resolve hook because of the possibility of dynamic resolution, which could seem important not to rule out. It's at a little bit of an impasse between Node.js and the browser right now (and there is an HTML PR up for a sync import.meta.resolve). Hopefully putting thought to dynamic import maps will shed some light on this question up again for browsers, ideally when firefox implement import maps.

Node and browsers are different; there's really nothing wrong with making a different name - if browsers make a sync resolve, node could provide an async one named something else, and folks who want to write code universally can feature-detect.

@ljharb having two resolves (a sync and an async one) doesn't solve the problem. Because once there is a sync resolve, then the "resolve" loader hook MUST be sync, which will preclude async resolutions (e.g. HTTP).
(and, yes, I know we can now do sync HTTP using workers and atomics, but that means that the loader needs to create a worker for this, which isn't a lightweight option.)
Just putting it here. It's not like there's something that can be done in this context. 😊

I apologize for allowing this issue to get completely derailed from the original goal. We seem to have one global object missing needed to support some sort of “Workerized JS Module Script Loading” according to an unofficial draft specification in the WICG.
I have linked to the issue below — it looks interesting…
https://github.com/nodejs/node/issues/40844
This is somewhat relevant and necessary for interop (with who is not the important part, but perhaps to achieve the goal of having “Worklet Infrastructure” outlined in that specification being the important part).

This issue should help us track progress in this area […]

There seems to be a good amount of momentum behind URLPattern (Chromium is either almost done implementing or needs to write more tests for them to pass) according to the web-platform-tests dashboard. Still, I'd give it some more time before we jump on the bandwagon, though (that's just one implementer).
This API may still not be up to snuff for us as it's currently specced as mentioned in the original issue (need more info). I guess I wouldn't get my hopes up too soon — it looks like something we should keep our eyes on (maybe even potentially review their spec and participate if we know of any ways to optimize to make it more usable on the backend).
Seeing as how non-browser environments would sometimes have a filesystem path rather than an actual URL (as in the case of CommonJS specifiers), I wonder how optimal this may be for us? I almost wish it could handle filesystem paths, too; otherwise, we'd have to normalize some specifiers using pathToFileUrl before using it.

WICG link: https://wicg.github.io/urlpattern/#urlpattern
MDN links:

https://developer.mozilla.org/en-US/docs/Web/API/URLPattern
https://developer.mozilla.org/en-US/docs/Web/API/URL_Pattern_API

WPT links: https://wpt.fyi/results/urlpattern?label=experimental

I still haven't heard too much interest from the loaders team about this yet…

@DerekNonGeneric I at least in the Loaders side of things don't see the need for URLPattern or even the advantage to using it. For example, if we scope things using URLPattern that means a cost prior to doing any operation. Additionally it uses string based semantics and really is best suited for HTTP/HTTPS and not mixing with other URL schemes. Im especially confused on how this relates to putting stuff in workers.


Parse port cat rather than port info

Seems to be a lot quicker

This is also more reliable than port info --index
Having said that, this could be tricky for more complex ports...

Done https://github.com/harens/seaport/commit/28df9da55b7bf7cd155d0e3d7567b424ef3f5875


502 Bad Gateway Connection refused) while connecting to upstream

This works on my dev vm but  stops working on production
Server is ubuntu 16.04.4. Did I missed something?
nginx.1    | 2018/03/06 06:38:39 [error] 52#52: *1 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: one.example.com, request: "GET / HTTP/1.1", upstream: "http://<IP_ADDRESS>:80/", host: "one.example.com"
version: "3"
services:
nginx:
image: jwilder/nginx-proxy
restart: always
networks:
- frontend
ports:
- "443:443"
- "80:80"
- "8080:80"
volumes:
- ./ssl:/etc/nginx/certs
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./nginx/etc/nginx/conf.d:/etc/nginx/conf.d
teampass:
image: teampass/teampass
restart: always
environment:
VIRTUAL_HOST: one.example.com
depends_on:
- db
networks:
- frontend
- backend
volumes:
- ./teampass:/var/www/html
db:
restart: unless-stopped
image: mysql/mysql-server:5.7
expose:
- '3306'
ports:
- '3306:3306'
environment:
MYSQL_ROOT_PASSWORD: mypass
MYSQL_DATABASE: teampass
MYSQL_PASSWORD: teampass
MYSQL_USER: teampass
volumes:
- ./db:/var/lib/mysql
networks:
- backend
networks:
frontend:
driver: bridge
backend:
driver: bridge
~

HOW DID YOU SOLVE THIS?

Up ?

same


Demo site broken

In commit https://github.com/oddbird/css-toggles/commit/ea887275f8a3df5bcf884e67b7f55aecb482f4ed, a few files got moved around. Since then, the demo site is broken, as the linked to https://toggles.oddbird.net/dist/css-toggles.js file does not exist.

Thanks! Demo site is back online


model partitioning algorithm

Hello author,excuse me, I would like to ask where the code of the model partitioning algorithm is, I'd like to have a look，thank you very much

The scheduling code is in the src-native directory.  See also the README.md file at the top level for build dependencies and README_Scheduler.md for additional details on using the scheduler.


2.0-wip: Style progress bar text.

This adds the ability to add a stylized text indicator of the progress bar. This was requested in issue #1076. I've also made updates requested/suggested in #1082.

Tested in:
 Firefox 9.0.1 OS X
 Safari 5.1.2 OS X
 Chrome 16.0.912.75 OS X

Unless you can account for text that hangs out (for say 1% fill), we can't do anything like this.

Would doing it programmatically be a solution here?

Yes, that seems to be the only way to resolve it. If you can, work with @fat to come up with a simple solution. Unsure if we need a full-blown plugin or not, but I'd like to account for this use case.

Seems my width-chaning one-line plugin might come in handy after all ;)

@pokonski For sure :). Let's get @fat to check that out when he has time.
For now, I've added styles for white center-aligned text in the bar. No extra padding though, so use at your own risk and be sure to consider the problems with super low percentages.
Thanks for the feedback everyone!


use /dev/shm for temp storage whilst building cache

Found that using /dev/shm to build cache makes the performance increase significantly, so suggesting using /dev/shm for cache to allow faster building.

zenjabba, do you have your /tmp on an SSD or traditional hard drive? I might try your suggestion but am worried about running out of ram.

At 35T my cache is 18M so I wouldn't be worried about that at all. But more so worried about #64, because if your QPS is too high you may get temporarily banned from going over your API quota.

I wouldn't be to worried about the space, but what I did find is you go over your API quota stupidly fast doing it this way. We need to implement throttling in the cache build part of the script.

You already can use /dev/shm, but as you said I need to implement the throttling. This is already described in issue #64


Add timeout doc

Thanks for contributing to the saucectl project!
A PR well described will help maintainers to quickly review and merge it
Before submitting your PR, please check our contributing
guidelines, applied for this repository. Avoid large PRs, help reviewers by making them as simple
and short as possible.

Description
Add documentation on new saucectl flag
Motivation and Context
Better documentation and onboarding
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[X] I have read the contributing document.
[X] My change requires a change to the documentation.
[X] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

For reference: the corresponding saucectl change: https://github.com/saucelabs/saucectl/pull/24


Add metadata to consumer for postgress

It would be great if we can add additional metadata to the consumer. Kong could generate then an upstream header which contains this metadata.

we have something similar https://getkong.org/plugins/correlation-id/

when using authentication plugins, Kong already sends along some of the IDs, so upstream is able to identify the consumer (custom_id iirc). Relating this to consumer specific meta data is an upstream responsibility imo. Kong only identifies. No?

Just to give you an example I would add a json in this custom metadata with the customer that contains the internal groups that the user belongs to, e.g., canary tester, etc. When the user hits the api gateway the gateway should simply add a header to all downstream requests with X-Custom-Metadata : myJson.
Even better would be if we could store individual key value pairs with each customer and make them selectable in the request-transformer like:
config.add.headers[1]=x-new-header:{customer:mykey}

@schellkenig have you considered configuring the Request Transformer plugin per consumer?
Something like this:
curl -X POST <kong>/<api>/plugins -d "name=request-transformer" -d "consumer_id=<consumer id>" -d "config.add.headers[1]=x-new-header:{customer:mykey}"
It might get a bit messy if you have a large number of consumers, but I think it should work.

I would do this but the request-transfer is not capable of using values from the customer (to the best of my knowledge) and these values are also not present but probably I misunderstood here something?

Closing this as a duplicate of #550.


Function api2

fixes issues with importing styles from #25 . Should make importing as either  a function or component equally predictable.

I can't think of a place where someone might want both ConnectModal and AccountModal individually. FractlModal works better in that situation and handles promises and transition better. This has too much boilerplate, but doesn't even work.  account is a different object from config (so they won't swap, wrapping it all in await doesn't work because checking for connection needs the state and accountData stores destructured, while passing props, needs it to be whole.
{#if account}
	<AccountModal {...account} />
{:else}
	{#await evm then config}
		<ConnectModal {config} state={config.state} {onConnect}/>
	{/await}
{/if}


tests: fix logic to pick up k8s, cilium logs

k8s tests: pick up kubelet, kube-controller-manager, kube-apiserver, cilium logs from both nodes.
runtime tests: pick up cilium logs using journalctl
name each tar.gz file using JOB_BASE_NAME-BUILD_NUMBER to differentiate between tar.gz files downloaded locally.

Signed-off by: Ian Vernon<EMAIL_ADDRESS>

Closing as this was merged with #1169


2404: Convert tools to ESM

Short description
This PR displays in a small scope how the conversion to esm could look like. Feel free to ask questions and make remarks if you are unhappy.
Proposed changes

added linting rules relevant for esm
added file extensions to import
set type to 'module'
update node-fetch to show that other esm packages work fine now :)

Resolved issues

Partially addresses: #2404

Do you have extensions that do the updated importing automatically for us? I foresee a lot of wrong imports in the future...

Do you have extensions that do the updated importing automatically for us? I foresee a lot of wrong imports in the future...

I added a eslint-plugin that checks that. Also, if you have set "type": "module" in the package.json this should done automatically by your IDE