Initialize config global and current from object

Is there any way to initialize JsonConfig global and current using config object instead of a file?
I want to build configuration based on config file, user secrets and environment variables. I am not able to do it directly because of two reasons:

There is no implementation in AtataContextBuilder to handle user secrets
Using environment variables in config files will throw exception when variable is not defined - I would rather have null value if env variable does not exist so it can be set from other source.

To workaround it, I tried to use ConfigurationBuilder to create config object and set is as follows:

However it will only apply values to the context which will not have any custom properties accessible and TestConfig.Global will be null.
I checked the implementation and I am wondering if there is any reason why the below method is so limited:

I would love to have it similar to the one that accepts filePath:

I tried to implement my own extension method in my project but I can't access internal classes like JsonConfigManager and I don't want to duplicate the code and adjust it on framework changes.
For now, I made some workaround to serialize built config to json, save it to file and pass to ApplyJsonConfig(filePath) method but  doing that I have a file stored with secret values so it's not desired approach. Do you have any suggestion how to do it differently?

I created a repository https://github.com/YevgeniyShunevych/atata-samples-json-extended-configuration where I put together a couple of approaches of extended configuration. Please check out SetUpFixture.cs and other files.
The main idea is that you can extend your JsonConfig<T>.Global after you applied it in GlobalSetUp. There you can use any way of getting configuration values.
// Read and set configuration from TestContext:
AtataConfig.Global.Username = TestContext.Parameters.Get("TestUsername", AtataConfig.Global.Username);
AtataConfig.Global.Password = TestContext.Parameters.Get("TestPassword", AtataConfig.Global.Password);

// Read and set configuration from environment variables:
AtataConfig.Global.Username = Environment.GetEnvironmentVariable("TestUsername") ?? AtataConfig.Global.Username;
AtataConfig.Global.Password = Environment.GetEnvironmentVariable("TestPassword") ?? AtataConfig.Global.Password;

// Read and set configuration through ConfigurationBuilder from environment variables, user secrets, etc.:
var configuration = new ConfigurationBuilder()
    .AddEnvironmentVariables()
    .AddUserSecrets<SetUpFixture>()
    .Build();

configuration.Bind(AtataConfig.Global);

// Another alternative approach is to add custom variables:
AtataContext.GlobalConfiguration.AddVariable("key", "val");
// You can get the value in the test this way:
// AtataContext.Current.Variables["key"];

// Additional recommendation is to mask secret strings in log:
AtataContext.GlobalConfiguration.AddSecretStringToMaskInLog(AtataConfig.Global.Password);

But the common and recommended approach for multiple environments is to use different config files, like Atata.local.json and Atata.uat.json. Where in Atata.local.json you can hard-code credentials but for UAT/PreProd/Prod environments you may consider to reference credentials through "{env:...}" variables.
Please let me know if those approaches don't work for you. Maybe I didn't understand your issue completely.

Additionally, I can say that for Atata v3 I want to use ConfigurationBuilder instead of Atata.Configuration.Json. So such things like using environment variables or user secrets will be easier to apply. But it is a far future.

Thanks a lot for the answer. I am able to solve my problem using existing functionalities, maybe not in the purest way but it works :) I am looking forward to Atata v3 so it can be simplified. Nevertheless, great work on the framework


Update documentation regarding preflight use with other middleware

I've run into an issue when using a combination of middleware, particularly ones that modify the headers. As it turns out, ordering is important.
import express from 'express'
import helmet from 'helmet'
import cors from 'cors'
import morgan from 'morgan'
import bodyParser from 'body-parser'
import router from './routes'
import './utils/db'

const app = express()

app.use(cors())
app.use(morgan('combined'))
app.use(bodyParser.json({ type: 'application/json' }))
app.use(helmet())

app.use(router)

let port = 5000

app.listen(port, () => {
  console.log('ready')
})

Stumbled onto this issue which reveals the issue is in this cors middleware.
https://github.com/helmetjs/helmet/issues/157
Digging through the source code for cors, the following line prevents subsequent middleware from being used for the preflight:
https://github.com/expressjs/cors/blob/b5bbc285194568f414afc2390c56f672734d3fac/lib/index.js#L180
I understand that this is required, but it's unclear that the ordering of middleware is important.
I think it's worth adding either a known issues section, or making it clear under usage.

Additional note, Chrome appears to request preflight for all HTTP VERBS (including POST/GET).

I agree. In the preflight section, is this not enough?
You can also enable pre-flight across-the-board like so:
app.options('*', cors()) // include before other routes
Or maybe should the wording be changed? Happy to make whatever improvements you think will help, but need suggestions for what makes sense to you. Would you be willing to put together a pull request with your suggestion?

I think the confusion is where people like me have just enabled it across the board using app.use(cors().
https://github.com/expressjs/cors#simple-usage-enable-all-cors-requests
So might be worth adding a note below that.
I'll have a proper read through it this week and see how best to describe it.

Yep, I understand.  But it's really hard for me to understand why you are getting confused, exactly, and how exactly to improve the readme to help with that is all. I'm not saying your confusion is invalid or anything, just hard for me to fix the readme if I don't see it the same way, haha. So really need to understand what specifically needs to be changed / improved :+1:

Without an actual action item for me to take, leaving the issue open won't really do anything. Take your time to open a pr or reply back with what I can change and will be happy to reopen and make the change :+1:

Happy with the action taken and will potentially follow up with a PR soon.
Just tried this example: https://github.com/expressjs/cors#enable-cors-for-a-single-route

Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

So I think there needs to be more emphasis on enabling preflight and what preflight is.

I'm not sure why you think this is required? It's just the default configuration. Pass preflightContinue: true to the options to not end the response in the middleware.

It's written as a comment in the source code:
// Safari (and potentially other browsers) need content-length 0,
//   for 204 or they just hang waiting for a body

It didn't break Chrome for me, but if that comment still rings true for safari then I'd imagine it is required.

t's written as a comment in the source code:

I think you misunderstood the comment. The comment is about needing to put the content-length header for the 204 code, nothing to do with ending the response.

In short, it is unclear that ordering of middleware is important.
What is unclear is why res.end() is used for preflight and the fact that by default, it blocks subsequent middleware from being used during the preflight request.
It wasn't until I looked at the actual cors code that I could see why subsequent middleware is ignored during preflight.
From my testing, the following doesn't set content-length as expected:
res.setHeader('Content-Length', '0');
next();

However, it allows subsequent middleware to be used. Ending the request will actually set the content-length as expected.
res.setHeader('Content-Length', '0');
res.end();

No where in the documentation does it mention that by default, the preflight request will be ended by this middleware and will not be passed on to the next one... thus making the ordering of middleware important.
As another example, using a global catch all...
app.options('*', cors()) // include before other routes

...will actually prevent:
app.options('/example', example())

...from being used.
Furthermore, in my previous commment I mentioned that the exampled didn't work as expected due to preflight. https://github.com/expressjs/cors#enable-cors-for-a-single-route
var express = require('express')
var cors = require('cors')
var app = express()

app.get('/products/:id', cors(), function (req, res, next) {
  res.json({msg: 'This is CORS-enabled for a Single Route'})
})

app.listen(80, function () {
  console.log('CORS-enabled web server listening on port 80')
})

I actually needed to add preflight:
var express = require('express')
var cors = require('cors')
var app = express()
app.options('/products/:id', cors())
app.get('/products/:id', cors(), function (req, res, next) {
res.json({msg: 'This is CORS-enabled for a Single Route'})
})
app.listen(80, function () {
console.log('CORS-enabled web server listening on port 80')
})

This is unclear until you have read the preflight section.

Sorry, I just wanted to give you as much information as possible to help you understand where my issue was.

Absolutely :+1:

Not certain if related to this, but based on the README, and this part in particular:

Certain CORS requests are considered 'complex' and require an initial OPTIONS request (called the "pre-flight request"). An example of a 'complex' CORS request is one that uses an HTTP verb other than GET/HEAD/POST (such as DELETE) or that uses custom headers. To enable pre-flighting, you must add a new OPTIONS handler for the route you want to support:

I would have assumed OPTIONS requests are not served without app.options handlers, but with only this configuration
app.use(cors({
  origin: 'http://localhost:3000'
  credentials: true
}))

I'm seeing OPTIONS request getting sent and responded with 204 by express. Have I misunderstood something and this is working as intended?

@raine yes, this is part of the confusion.
Your example above is using cors globally, so it enables CORS for everything including options pre-flight.
So there are actually three ways to enable it:
app.use(cors()) // enables cors globally. (every route and verb).

app.options('*', cors()) // enables cors pre-flight for every route

app.options('/products/:id', cors()) // enable pre-flight for a specific route
app.del('/products/:id', cors(), function (req, res, next) {
  res.json({msg: 'This is CORS-enabled for all origins!'})
})

Understood. In that case I would certainly clarify in the README that app.use(cors()) will enable CORS preflight requests. Currently the "Enabling CORS Pre-Flight" section leaves you under the impression that app.options() is the only way.

I think this confusion stems from the middleware incorrectly interpreting all OPTIONS requests as preflight requests. But clearly, some OPTIONS requests are not preflight requests.


groupbar border?

Description
With rounding enabled, the groupbar kinda looks out of place. Maybe something like bar_precedence_over_border similar to how hyprbars has it?

decoration priority iirc, it isn't polished doe

I think the best way would be to put groupbars inside a window's border - make them look like it's a 'part' of the window or smth.

decoration priority iirc, it isn't polished doe

like I said you can already set it in there with group:groupbar:priority, but it isn't polished
mostly because of the issue you can already experience in hyprbars where there are wrong pixels where the boxes touch
additionally this is just a dupe of the groupbar feature request issue

group:groupbar:priority is also poorly documented. Default=3, what 3? What is this even supposed to mean?

see hyprctl decorations <window>

It didn't make it more clear, tbh.
Anyway, if you say - this is bugged, I'll better wait till it gets good.

see hyprctl decorations <window>

It didn't make it more clear, tbh.
Anyway, if you say - this is bugged, I'll better wait till it gets good.

it sets the decoration priority for the groupbar, (using 10001 should make it appear inside the border)
it's not bugged it just doesn't get rounded which likely looks bad if you're not using 0 rounding

I use rounding, so I guess this not for me.
In case rounding gets fixed - I think it'd be better to replace priority with smth like groupbar_inside_border = 0/1

I use rounding, so I guess this not for me.
In case rounding gets fixed - I think it'd be better to replace priority with smth like groupbar_inside_border = 0/1

no because then you wouldn't be able to control it in relation to other decorations
for example if you want groupbar bellow hyprbar bellow the border

#2415


Multi-class segmentation

We get quite a few questions about multi-class segmentation (e.g., https://github.com/Project-MONAI/MONAI/discussions/2302), so it would be good to have a tutorial demonstrating this workflow.

Hi @rijobro ,
I think this Brats tutorial is: https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/brats_segmentation_3d.ipynb.
Thanks.

oh great, thanks!


Create instance key on the client

It's already implemented on the server.  Once we have this deployed on all the clients we can short-circuit the startExperiments call on API Gateway, too.

👍


HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time

JIRA: https://issues.apache.org/jira/browse/HDFS-15808
To monitor how often read/write locks exceed thresholds, we can add two metrics(ReadLockWarning/WriteLockWarning), which are exposed in JMX.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 37s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 52s

trunk passed

+1 :green_heart:
compile
1m 22s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
1m 17s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+1 :green_heart:
checkstyle
1m  2s

trunk passed

+1 :green_heart:
mvnsite
1m 25s

trunk passed

+1 :green_heart:
shadedclient
16m 23s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 58s

trunk passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 26s

trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+0 :ok:
spotbugs
3m 19s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m 17s

trunk passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 11s

the patch passed

+1 :green_heart:
compile
1m 15s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
1m 15s

the patch passed

+1 :green_heart:
compile
1m 12s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+1 :green_heart:
javac
1m 12s

the patch passed

-0 :warning:
checkstyle
1m  1s
/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 165 unchanged - 0 fixed = 173 total (was 165)

+1 :green_heart:
mvnsite
1m 12s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
13m 21s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 48s

the patch passed with JDK Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 27s

the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

+1 :green_heart:
findbugs
3m 18s

the patch passed

_ Other Tests _

-1 :x:
unit
193m 50s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 41s

The patch does not generate ASF License warnings.

281m 31s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2668

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux a123314fa64f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 115623a6ee4

Default Java
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-<IP_ADDRESS>+1-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/testReport/

Max. process+thread count
3065 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/1/console

versions
git=2.25.1 maven=3.6.3 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Failed junit tests
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Sorry. I didn't change those two unit tests, and they worked fine locally.

@xkrogen
Hi Erik Krogen, could you please review the code? Thanks a lot!

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 37s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
33m 16s

trunk passed

+1 :green_heart:
compile
1m 25s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
1m 17s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
1m  6s

trunk passed

+1 :green_heart:
mvnsite
1m 25s

trunk passed

+1 :green_heart:
shadedclient
16m 35s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 54s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 24s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+0 :ok:
spotbugs
3m  8s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m  6s

trunk passed

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 12s

the patch passed

+1 :green_heart:
compile
1m 13s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
1m 13s

the patch passed

+1 :green_heart:
compile
1m  5s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
1m  5s

the patch passed

-0 :warning:
checkstyle
0m 57s
/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 167 unchanged - 0 fixed = 175 total (was 167)

+1 :green_heart:
mvnsite
1m 12s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
13m  7s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 54s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 25s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
findbugs
3m 12s

the patch passed

_ Other Tests _

-1 :x:
unit
192m 53s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 45s

The patch does not generate ASF License warnings.

280m 44s

Reason
Tests

Failed junit tests
hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized

hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2668

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 4e47b5746c6d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f9a073c6c18

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/testReport/

Max. process+thread count
3398 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/2/console

versions
git=2.25.1 maven=3.6.3 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
14m 56s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m  7s

trunk passed

+1 :green_heart:
compile
1m 20s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
1m 14s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
1m  6s

trunk passed

+1 :green_heart:
mvnsite
1m 20s

trunk passed

+1 :green_heart:
shadedclient
15m 32s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 56s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 25s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+0 :ok:
spotbugs
3m  5s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m  3s

trunk passed

-0 :warning:
patch
3m 21s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 10s

the patch passed

+1 :green_heart:
compile
1m 11s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
1m 11s

the patch passed

+1 :green_heart:
compile
1m  6s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
1m  6s

the patch passed

-0 :warning:
checkstyle
0m 57s
/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs-project/hadoop-hdfs: The patch generated 8 new + 167 unchanged - 0 fixed = 175 total (was 167)

+1 :green_heart:
mvnsite
1m 11s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
12m 55s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 48s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 17s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
findbugs
3m  2s

the patch passed

_ Other Tests _

-1 :x:
unit
190m 34s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 43s

The patch does not generate ASF License warnings.

289m 36s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2668

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 15073fe6c019 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / f9a073c6c18

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/testReport/

Max. process+thread count
3120 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/3/console

versions
git=2.25.1 maven=3.6.3 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Failed junit tests  |  hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Sorry. I didn't update this unit tests, and they worked fine locally.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 39s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 39s

trunk passed

+1 :green_heart:
compile
1m 20s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
1m 13s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
1m  3s

trunk passed

+1 :green_heart:
mvnsite
1m 19s

trunk passed

+1 :green_heart:
shadedclient
15m 56s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 55s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 27s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+0 :ok:
spotbugs
3m  6s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m  4s

trunk passed

-0 :warning:
patch
3m 22s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 12s

the patch passed

+1 :green_heart:
compile
1m 16s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
1m 16s

the patch passed

+1 :green_heart:
compile
1m  7s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
1m  7s

the patch passed

+1 :green_heart:
checkstyle
0m 58s

the patch passed

+1 :green_heart:
mvnsite
1m 14s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
13m  4s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 50s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 20s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
findbugs
3m  9s

the patch passed

_ Other Tests _

-1 :x:
unit
194m 59s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 44s

The patch does not generate ASF License warnings.

281m 25s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2668

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 0c1c25c99215 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / c3134ab3a99

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/testReport/

Max. process+thread count
3175 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/4/console

versions
git=2.25.1 maven=3.6.3 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Failed junit tests
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap
Those failed unit tests were unrelated to the change.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
19m 14s

Docker mode activated.

-1 :x:
yetus
0m  6s

Unprocessed flag(s): --findbugs-strict-precheck

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/5/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2668

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/5/console

versions
git=2.25.1

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

This message was automatically generated.

Hi @anuengineer, the failed check looks unrelated to the change. May I ask how I should deal with it?

I found this JIRA in the log. https://issues.apache.org/jira/browse/HADOOP-16870
Hi @aajisaka , Could you please help to take a look at this problem? And how I should deal with this problem? Thank you.
[2021-02-17T01:59:03.362Z] HEAD is now at 4cf35315838 HADOOP-16870.  Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2454) [2021-02-17T01:59:05.444Z]  [2021-02-17T01:59:05.444Z] Testing https://github.com/apache/hadoop/pull/2668 diff on trunk. [2021-02-17T01:59:05.444Z]  [2021-02-17T01:59:05.444Z]  [2021-02-17T01:59:05.444Z] ============================================================================ [2021-02-17T01:59:05.444Z] ============================================================================ [2021-02-17T01:59:05.444Z]                     Re-exec mode detected. Continuing. [2021-02-17T01:59:05.444Z] ============================================================================ [2021-02-17T01:59:05.444Z] ============================================================================ [2021-02-17T01:59:05.444Z]  [2021-02-17T01:59:05.444Z]  [2021-02-17T01:59:05.444Z] ERROR: Unprocessed flag(s): --findbugs-strict-precheck

Just reposting my comment on the jira for visibility.
The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often.
See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful.
I assume you need something like a rate.

Just reposting my comment from the jira for visibility.
The patch looks fine, but I doubt the metric will be useful in its current form. Monotonically increasing counter doesn't tell you much when plotted. Over time it just becomes an incredibly large number, hard to see its fluctuations. And you cannot set alerts if the threshold is exceeded often.
See e.g. ExpiredHeartbeats or LastWrittenTransactionId - not useful.
I assume you need something like a rate.

Hey @shvachko , thank you for your comments and suggestions. I replied to you in JIRA.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
18m 18s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 53s

trunk passed

+1 :green_heart:
compile
1m 20s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
compile
1m 15s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
checkstyle
1m  5s

trunk passed

+1 :green_heart:
mvnsite
1m 22s

trunk passed

+1 :green_heart:
shadedclient
15m 17s

branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 58s

trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 28s

trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+0 :ok:
spotbugs
3m  7s

Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m  5s

trunk passed

-0 :warning:
patch
3m 24s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
1m 16s

the patch passed

+1 :green_heart:
compile
1m 10s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javac
1m 10s

the patch passed

+1 :green_heart:
compile
1m  6s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
javac
1m  6s

the patch passed

+1 :green_heart:
checkstyle
0m 55s

the patch passed

+1 :green_heart:
mvnsite
1m 11s

the patch passed

+1 :green_heart:
whitespace
0m  0s

The patch has no whitespace issues.

+1 :green_heart:
shadedclient
15m 47s

patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
0m 58s

the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04

+1 :green_heart:
javadoc
1m 39s

the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

+1 :green_heart:
findbugs
3m 57s

the patch passed

_ Other Tests _

-1 :x:
unit
204m 49s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 45s

The patch does not generate ASF License warnings.

312m 18s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.ha.TestHAAppend

hadoop.hdfs.qjournal.server.TestJournalNodeSync

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

hadoop.hdfs.server.datanode.TestBlockScanner

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2668

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 6ac55fde21d0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4891e68c2b2

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/testReport/

Max. process+thread count
3046 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2668/6/console

versions
git=2.25.1 maven=3.6.3 findbugs=4.0.6

Powered by
Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Failed junit tests
hadoop.hdfs.server.namenode.ha.TestHAAppend
hadoop.hdfs.qjournal.server.TestJournalNodeSync
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
hadoop.hdfs.server.datanode.TestBlockScanner
Those failed unit tests were unrelated to the change. And they worked fine locally.

Hi @anuengineer , could you please review again? Let's see if there are any other questions. Thank you.

Hi @arp7 , could you take a quick look at this? Let's see if there are any other questions. Thank you.

Hi @shvachko , I uploaded a patch for branch-3.3 in JIRA.

+1 LGTM

Thanks @dineshchitlangia for the review.

Hi @shvachko , I uploaded a patch for branch-3.3 in JIRA, please help to review it. Thank you.

Close this PR because the associated patch has been merged.


Being paid for work

I haven’t received any payment for working with GitHub’s

might have opened the issue in the incorrect repo.


win: include "malloc.h"

As stated in malloc's docs, the malloc.h is required for using the malloc function, and from malloc.h's source code (which can be found under installed VS), malloc is defined as a macro that has different behaviors under different configurations.
So using malloc without including malloc.h is very dangerous, and in Electron when we update to io.js v2.2.1 which uses libuv v1.5.0, we observed crash caused by writing to memory allocated by libuv:
0:000> g
(e08.918): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll!memcpy+0x220:
00007ffc`6d295020 f30f7f40f0      movdqu  xmmword ptr [rax-10h],xmm0 ds:ffffffff`a1ccae8e=????????????????????????????????
0:000> k
Child-SP          RetAddr           Call Site
0000008f`9fdfe0e8 00007ffc`6d227b06 ntdll!memcpy+0x220
0000008f`9fdfe0f0 00007ffc`6d2256d3 ntdll!RtlCopyUnicodeString+0x46
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\KERNELBASE.dll -
0000008f`9fdfe120 00007ffc`6a762571 ntdll!LdrGetDllFullName+0x53
*** WARNING: Unable to verify checksum for C:\cygwin\home\zcbenz\codes\electron\out\D\node.dll
0000008f`9fdfe170 00007ffc`54fd80c6 KERNELBASE!GetModuleFileNameW+0x71
0000008f`9fdfe1b0 00007ffc`54d1c049 node!uv_exepath+0xa6 [c:\cygwin\home\zcbenz\codes\electron\vendor\node\deps\uv\src\win\util.c @ 131]
0000008f`9fdfe220 00007ffc`54d0e7f3 node!node::SetupProcessObject+0x2359 [c:\cygwin\home\zcbenz\codes\electron\vendor\node\src\node.cc @ 2860]
*** WARNING: Unable to verify checksum for electron.exe
0000008f`9fdfef80 00007ff7`5f1c44ac node!node::CreateEnvironment+0x353 [c:\cygwin\home\zcbenz\codes\electron\vendor\node\src\node.cc @ 3898]
0000008f`9fdff0c0 00007ff7`5f186d02 electron!atom::NodeBindings::CreateEnvironment+0x33c [c:\cygwin\home\zcbenz\codes\electron\atom\common\node_bindings.cc @ 160]
*** WARNING: Unable to verify checksum for C:\cygwin\home\zcbenz\codes\electron\out\D\content.dll
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\cygwin\home\zcbenz\codes\electron\out\D\content.dll -
0000008f`9fdff510 00007ffc`4ff3aaf3 electron!atom::AtomBrowserMainParts::PostEarlyInitialization+0xf2 [c:\cygwin\home\zcbenz\codes\electron\atom\browser\atom_browser_main_parts.cc @ 63]
0000008f`9fdff590 00007ffc`4ff3e55a content!ResourceMsg_ReceivedResponse::Read+0x171f71
0000008f`9fdff660 00007ffc`4ff36052 content!ResourceMsg_ReceivedResponse::Read+0x1759d8
0000008f`9fdff750 00007ffc`4fe51d5e content!ResourceMsg_ReceivedResponse::Read+0x16d4d0
0000008f`9fdff7b0 00007ffc`4fe51bd5 content!ResourceMsg_ReceivedResponse::Read+0x891dc
0000008f`9fdff800 00007ffc`4fe4ad50 content!ResourceMsg_ReceivedResponse::Read+0x89053
0000008f`9fdff8a0 00007ff7`5f16003a content!ResourceMsg_ReceivedResponse::Read+0x821ce
0000008f`9fdff8d0 00007ff7`5f3f22a6 electron!wWinMain+0x56a [c:\cygwin\home\zcbenz\codes\electron\atom\app\atom_main.cc @ 159]
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\system32\KERNEL32.DLL -
0000008f`9fdffb90 00007ffc`6aca13d2 electron!__tmainCRTStartup+0x162 [f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c @ 618]
0000008f`9fdffbd0 00007ffc`6d215444 KERNEL32!BaseThreadInitThunk+0x22
0000008f`9fdffc00 00000000`00000000 ntdll!RtlUserThreadStart+0x34

From the stack trace, the crash is caused by memcpy writing to an invalid address, which is allocated by malloc in uv_exepath, and after bring back the include "malloc.h", the crash is gone.
And if it is possible, please consider applying this patch to io.js, since it affects users that using io.js as a library.

The change LGTM, I'll land it and pake a new release, if that helps.

And if it is possible, please consider applying this patch to io.js, since it affects users that using io.js as a library.

Why is this needed though? Now we no longer use malloc throughout the code, we use uv__malloc, which ends up calling malloc, but malloc.h would already be included as per this PR. Can you please elaborate?

Why is this needed though? Now we no longer use malloc throughout the code, we use uv__malloc, which ends up calling malloc, but malloc.h would already be included as per this PR. Can you please elaborate?

I thought io.js is still using libuv v1.5.0, I just took a look and found the master branch is already updated to latest libuv, so I can wait for next release, thanks!

@zcbenz Can you update the commit log to explain why malloc.h is needed?  This is the first time I've heard about that, by the way.

from malloc.h's source code (which can be found under installed VS), malloc is defined as a macro that has different behaviors under different configurations.

@bnoordhuis ^^, Microsoft's CRT does Tricky Things

Right, but I'm more wondering about the specifics.  The standard says that malloc() lives in stdlib.h and AFAIK that works on Windows.  What exactly does including malloc.h, a non-standard header, change?

@bnoordhuis Here's what it looks like on VS 2012 Update 4: https://gist.github.com/paulcbetts/225e20e9d20e7c52b0ec

@bnoordhuis also, FWIW, the MDN website mentions you need both stdlib.h and malloc.h: https://msdn.microsoft.com/en-us/library/6ewkz86d.aspx
At the same time, our tests pass without the include, so what gives?!

I have updated the commit log with the explanations.
The crash should be related to build configurations, Electron links with DLL version of mscrt, while io.js links with the static version, I'm wondering whether it affects the behavior of malloc.

LGTM although I hate landing patches that I don't really understand.

Electron links with DLL version of mscrt, while io.js links with the static version, I'm wondering whether it affects the behavior of malloc.

Is there at any time in your application memory being freed by one that was allocated with the other?  That would explain a lot - mixing memory allocators usually doesn't end well - but I don't think libuv has any APIs where the user frees memory that libuv allocated, or vice versa.

@bnoordhuis

Is there at any time in your application memory being freed by one that was allocated with the other? That would explain a lot - mixing memory allocators usually doesn't end well - but I don't think libuv has any APIs where the user frees memory that libuv allocated, or vice versa.

I think it may be caused by the fact that <malloc.h> was already used elsewhere in libuv (in src/win/process.c), so memory could be freed by the wrong free even within libuv.
Indeed malloc.h may change the allocator, though this only happens in debug mode, which suggest that electron may be using a debug build of libuv?
When MSDN says <stdlib.h> and <malloc.h> they mean "include both".

Landed in c0c26a0, thanks @zcbenz! Also thanks @bnoordhuis and @piscisaureus for the reviews!


Dependent packages download and management

Find the proper ways or tools to solve the problem caused by dependent package on different operating system.
Note:

Priority: Windows > Mac > Linux

On the Window11 system,  install all packages by ’‘requirements .txt ’
win11_install_by_requirements.md

Packaging and uploading in Pypi
Creating and uploading in Read the Docs

Pypi上传package步骤.md

这是来自QQ邮箱的自动回复邮件。您好，我是QYH~您的邮件我已经收到~

在windows 系统下，清屏用到的代码：os.system('cls')

这是来自QQ邮箱的自动回复邮件。您好，我是QYH~您的邮件我已经收到~


Add rails_admin_controller? method

It would be nice if controllers of RailsAdmin would have a method rails_admin_controller? that returns true.
For example Devise has one named devise_controller?.
That would help if one wants to skip authorization for CanCan (and alternatively use another auth solution for RailsAdmin), see https://github.com/ryanb/cancan/wiki/Upgrading-to-1.6 (Conditionally Check Authorization)
It is easy to work around this as one could simply check the class (resp. the module), but having the above described method would be nicer.

Thanks to your work XD


Mutated modules live in memory

All the mutated modules live in memory even though they are not needed.
We need to find a way to remove them as soon as they are not needed.

+1 for having this sooner!

+1 for having this sooner!

This is what I've got recently on CentOS with 2GB of RAM in VirtualBox:
Failed to create watchdog after creating 1012 child processes
Cannot allocate memory
Shutting down

😞


Apple TV Distribution

Describe the solution you'd like
Distribution support of Apple TV apps.
Describe alternatives you've considered
So far we've been using HockeyApp and have not been looking at any other solution besides TestFlight but it doesn't support AdHoc builds.
Additional context
We have been using HockeyApp for tvOS app distribution and I've been expecting support for it to be in AppCenter as well since I was told that you're aiming to support everything HockeyApp has but so far it's not been added, and looking at the platform support matrix it's the only platform without distribution support. Will tvOS be supported before HockeyApp eol?
https://docs.microsoft.com/en-us/appcenter/general/platform-service-matrix
Thanks!

Thanks @MattiasBurstrom for letting us know! We unfortunately have no ETA for this, but will leave this open to keep track of interest. @botatoes fyi ^

Really looking forward to this feature!  We have testers that need to test tvOS builds, so our current solution is manually installing each build for them (not ideal).  Would love a dedicated tvOS app that we can use to install builds (like how TestFlight does it)
Thanks guys~

+1 to reanimate this please

another +1 too


Update the code comment for vocab_overlap_crosstab

s/upper left/upper right

@insop Thanks for this as well!


TODO: Refactor to avoid duplication of this logic.

Looking at PowBlockAssembler.cs the following TODO appears:
        // This matches the calculation in CompareTxMemPoolEntryByAncestorFee,
        // except operating on CTxMemPoolModifiedEntry.
        // TODO: Refactor to avoid duplication of this logic.
        public class CompareModifiedEntry : IComparer<TxMemPoolModifiedEntry>
        {
            public int Compare(TxMemPoolModifiedEntry a, TxMemPoolModifiedEntry b)
            {
                Money f1 = a.ModFeesWithAncestors * b.SizeWithAncestors;
                Money f2 = b.ModFeesWithAncestors * a.SizeWithAncestors;

                if (f1 == f2)
                    return TxMempool.CompareIteratorByHash.InnerCompare(a.iter, b.iter);

                return f1 > f2 ? 1 : -1;
            }
        }

However there are differences in the calculation between the two methods -
CompareModifiedEntry has return f1 > f2 ? 1 : -1 while CompareTxMemPoolEntryByAncestorFee employs:
                    if (f1 > f2)
                        return -1;
                    return 1;

which raises questions about the implementation of the latter. Is it correct?

Thats a good question, can you verify that? it was ported from StratisX.
I think the TODO is wrong.

Ah I see that CompareModifiedEntry is using OrderByDescending. The same should happen with CompareTxMemPoolEntryByAncestorFee which is currently using OrderBy. So CompareTxMemPoolEntryByAncestorFee  should be brought in line with CompareModifiedEntry.

The following may assist in removing the duplication in logic:
        /// <summary>
        /// Default comparator for comparing this object to another TxMemPoolEntry object.
        /// </summary>
        /// <param name="other">Memory pool entry to compare to.</param>
        /// <returns>Result of comparison function.</returns>
        public int CompareTo(object other)
        {
            return uint256.Comparison(this.TransactionHash, (other as TxMempoolEntry).TransactionHash);
        }

        /// <summary>
        /// Used to compare fees on objects supporting the IComparable and ITxMempoolFees interfaces.
        /// </summary>
        /// <typeparam name="T">A type that supports the IComparable and ITxMempoolFees interfaces.</typeparam>
        /// <param name="a">The first object to compare.</param>
        /// <param name="b">The second object to compare.</param>
        /// <returns>Returns -1 if a less than b, 0 if a equals b, and 1 if a greater than b.</returns>
        public static int CompareFees<T>(T a, T b) where T:IComparable,ITxMempoolFees
        {
            // Avoid division by rewriting (a/b > c/d) as (a*d > c*b).
            Money f1 = a.ModFeesWithAncestors * b.SizeWithAncestors;
            Money f2 = b.ModFeesWithAncestors * a.SizeWithAncestors;

            if (f1 == f2)
                return a.CompareTo(b);

            return (f1 > f2) ? 1 : -1;
        }

Yes sounds good, should be quick and easy.

Closed with https://github.com/stratisproject/StratisBitcoinFullNode/pull/1523


Save Model Locally trainer.save_model()

First off, super excited about all this work! Also really appreciate the youtube presentation and the blogpost, super insightful to read through. I went through the existing issues and couldn't find whether this an existing functionality:
Is there a way to save the model locally instead of pushing to the hub?
So in addition to this:
trainer.push_to_hub(f"setfit-myowndataset-multilabel-example")
be able to do this too:
trainer.save_model("file-path/setfit-myowndatase-multilabel-example")

It would be nice to have something like that. Presently that throws an error.

You should be able to do model.save(path_str) as usual.

It can be done by
trainer.model._save_pretrained(save_directory=save_directory).
Here is the link to the solution where it has already been answered - https://github.com/huggingface/setfit/issues/102

Thank you @kgourgou and @kgourgou, that worked!

then how to load it ????

@badi02
from setfit import SetFitModel

model = SetFitModel.from_pretrained("my/model/path")


fix(transaction): fix invalid lock guard in getHash()

Purpose
We had bug (race condition) in getHash() function:  tx hash is cached during first call to the getHash() and after that  this cached value is returned. To see if we already cached it or not, flag is used but we set this flag before we actually assigned cached value itself.
For reviewers

Related Github issues

Changes

Tests

Deployment Considerations
Version Notes
{ major | minor | patch }

NOTE: Include the version notes in the squash commit message

prnet cleaned-up successfully


A problem with some custom UI component plugins

Hi, I am new to svelte-native. Thank you for creating this great project, I love it so much.
I encountered a problem when using some custom UI component plugins like this: https://github.com/nstudio/nativescript-floatingactionbutton
In their usage example, we can use registerElement directly to register the plugin element in Angular and Vue. I don't know if I am not doing it right or other reason, I can't do this in svelte-native.
Currently, I have to create a NativeElementNode for the plugin before using registerElement like this:
import { NativeElementNode, registerElement } from 'svelte-native/dom';

registerElement('Fab', () => new NativeElementNode('Fab', require('@nstudio/nativescript-floatingactionbutton').Fab));

Thanks. You are right, it looks like the other frameworks provide a method that does the NativeElementNode for you. If I were to change the behaviour it would break existing code for svelte-native users.
I will export a new method registerNativeViewElement which does takes a tag name and a method returning a View

This shipped in 0.6.0 which was just released


Add files via upload

Added free function bezier to calculate the bezier coefficient for a given point in an animations cycle to resolve #9

I think it would be good to add some tests for this function (check for expected values).
We'll also need a function needs to get the animation progress based on time using std::time::Duration for both time delta and animation speed.  Then, adding a test that makes sure the beginning and end of the animation are slower than the middle.
Let me know if you have any questions.

I based the function off of the red curve pictured in this graph.
https://en.wikipedia.org/wiki/Bézier_curve#/media/File:Bezier_basis.svg
You could convert a linear animation to this curve by first rasterizing the animation into discrete frames, and then converting as follows:
for frame in frames
param * = bezier(frame/sizeof(frames))
something like that. This is a pretty hacky solution though, and I understand the need for tests. But this was the only solution I could think of that was as simple as taking a single f32 and returning one back.

Yeah, I guess there's a few options:

Calculate all frames
Use DeCastlejau algorithm to split the curve into parts, to only calculate some frames and use linear interpolation
There's also a simplified bezier equation I came up with for the Twang crate - if you set some constants, you can simplify it to a piecewise function.  I just made a graph of it, if you want to try implementing it (I haven't gotten around to implementing it yet for twang): https://www.desmos.com/calculator/wcqr57xzpn

I should mention that the Twang bezier function is less tunable than the CSS bezier animations.  That might be ok for this project though.  Just something to keep in mind.

Actually, I just did some experimenting and it's more or less basically an x² algorithm.

Here's what I'm going to use for bezier curve in twang if you choose to base it on that implementation (where performance is big concern): https://www.desmos.com/calculator/3gf8alz4zy (time from -1 start to +1 end)
A nice benefit is that it's a linear function.

I'm wondering if the way we implement this should depend on how animations are implemented, and therefore we should wait until then to worry about this.

Possibly.  I'll let you make the call if you want to wait until more things are in place to implement this.


i want to open existing stackblitz repository or github repository

Describe the bug
i tried to open my existing project, but it states its not possible
Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-xvslgh
Steps to reproduce

ask : import project from github.com/u007/vue-cms
or
import from https://stackblitz.com/~/github.com/...

Expected behavior
open existing project
Screen Recording / Screenshot
No response
Platform
Browser name = Chrome
Full version = <IP_ADDRESS>
Major version = 129
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
performance.memory = {
  "totalJSHeapSize": 43482859,
  "usedJSHeapSize": 42343187,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Username = u007
Chat ID = d8cb098b651d

Additional context
No response

Hey @u007!
Just to double-check, you want to open a private repository?
We have planned to add support for github repositories in the near future. We'll keep you posted! :raised_hands:

oh right, due to license?

you can open stackblitz projects at https://bolt.new/~/PROJECT_SLUG and repositories at https://bolt.new/~/github.com/owner/repo

Você pode abrir projetos StackBlitz em https://bolt.new/~/PROJECT_SLUG e repositórios em https://bolt.new/~/github.com/owner/repo

I'm trying to open the repository, but it says I can't access it


Error: no repository definition for https://kubernetes-charts.storage.googleapis.com/, https://kubernetes-charts.storage.googleapis.com/. Please add them via 'helm repo add'

I use azure charts as stable, and want to update dependency with local files:
[loki-stack]# helm repo list
NAME  	URL
stable	http://mirror.azure.cn/kubernetes/charts/
local 	http://<IP_ADDRESS>:8879/charts
loki  	https://grafana.github.io/loki/charts
[loki-stack]# helm dependency update
Error: no repository definition for https://kubernetes-charts.storage.googleapis.com/, https://kubernetes-charts.storage.googleapis.com/. Please add them via 'helm repo add'

But helm need  https://kubernetes-charts.storage.googleapis.com/, i do not get it from any docs.
What only can i do is add it:
[loki-stack]# helm repo add stable https://kubernetes-charts.storage.googleapis.com/
"stable" has been added to your repositories
[loki-stack]# helm repo list
NAME  	URL
stable	https://kubernetes-charts.storage.googleapis.com/
local 	http://<IP_ADDRESS>:8879/charts
loki  	https://grafana.github.io/loki/charts

Then update succeed:
[loki-stack]# helm dependency update
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://<IP_ADDRESS>:8879/charts):
	Get http://<IP_ADDRESS>:8879/charts/index.yaml: dial tcp <IP_ADDRESS>:8879: connect: connection refused
...Successfully got an update from the "loki" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 4 charts
Downloading grafana from repo https://kubernetes-charts.storage.googleapis.com/
Downloading prometheus from repo https://kubernetes-charts.storage.googleapis.com/
Deleting outdated charts

I have some question:

why update dependency with local files need to connect network? All i want is to update locally, why no option support this?

why need https://kubernetes-charts.storage.googleapis.com/, and no docs comment it?

Output of helm version:
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}
Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.12", GitCommit:"c757b93cf034d49af3a3b8ecee3b9639a7a11df7", GitTreeState:"clean", BuildDate:"2018-12-19T11:16:52Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.12", GitCommit:"c757b93cf034d49af3a3b8ecee3b9639a7a11df7", GitTreeState:"clean", BuildDate:"2018-12-19T11:04:29Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

duplicate of https://github.com/helm/helm/issues/6005.

@bacongobbler hi, not only this, but question one has no answer.

@bacongobbler any input?

Use below common repository.
k8scka@master:~$ helm3 repo add stable https://charts.helm.sh/stable
"stable" has been added to your repositories
k8scka@master:~$


Release Requests Run Workflow Jobs Multiple Times

The same CI builds are executed multiple times:
https://github.com/open-telemetry/opentelemetry-ruby-contrib/pull/271/checks

A couple of thoughts here:

I think this is caused by having a "push" on branches named releases, which I added because the current toys tooling does not seem to trigger a workflow action using the gh api cli commands
I do not think that we need to run builds on main after we merge a PR and we should instead add build requirements before merging to main
We can add nightly builds on main "just in case" and open issues for failed builds
Release PRs probably do not need to check the status of builds on main since we only ever want to run the canary builds

WDYT @open-telemetry/ruby-contrib-approvers ?


Add ActivityIndicatorView

Project URL
https://github.com/exyte/ActivityIndicatorView
Category
Activity Indicator
Description
A number of preset loading indicators created with SwiftUI.
Why it should be included to awesome-ios (mandatory)
Wide range of indicator types. Also, there are no alternative libraries written in SwiftUI at this point.
Checklist

[x] Has 50 GitHub stargazers or more
[x] Only one project/change is in this pull request
[x] Isn't an archived project
[x] Has more than one contributor
[ ] Has unit tests, integration tests or UI tests
[x] Addition in chronological order (bottom of category)
[x] Supports iOS 9 / tvOS 10 or later
[x] Supports Swift 4 or later
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

@shipinev thanks for contributing 👍


Handle compressed ELF sections

For gimli's dwarfdump to handle compressed ELF sections, support should be added to object.

Actually I guess support needs to be added to both goblin and object.

I suppose we could just handle it in object.
Would you accept addding a method ObjectSection::uncompressed_data that returns Cow<'data, [u8]>? Perhaps also Object::uncompressed_section_data_by_name?

And we'd have to add a flate2 dependency.

SGTM!

Would Object::uncompressed_section_data_by_name automatically load .zdebug_info if .debug_info is requested?

Would Object::uncompressed_section_data_by_name automatically load .zdebug_info if .debug_info is requested?

Note that libbfd will look for .debug_info first and fall back to .zdebug_info if that's not found, independent of the setting of the compression flag.
It would probably make more sense to define yet another function, say uncompressed_debug_section_data, that does the magical .zdebug fallback and "ZLIB" decompression.

Is that what you want?

I'm happy with anything that works, but we need to do the fallback somewhere, and currently gimli consumers only look for the single name provided by Section::section_name. So doing it here would mean no changes needed in gimli.

What about doing that for all .debug_* sections?

Yes it would have to be all sections.

Would it be OK with you if it also automatically decompressed debug sections starting with "ZLIB" like libbfd does?

Yes.

It would probably make more sense to define yet another function, say uncompressed_debug_section_data, that does the magical .zdebug fallback and "ZLIB" decompression.

Do you mean Object::uncompressed_debug_section_data?
I'd rather we only add separate methods if we think there's going to be use cases that require them. So I'd actually be okay with changing Object::section_data_by_name to do all the magic and return Cow<'data, [u8]>, unless you think there's a reason not to.

That makes it sound like we shouldn't decompress sections simply because they start with ZLIB then. So I think Object::section_data_by_name should do the following:

look up the section with the given name
if it exists and has the SHF_COMPRESSED flag, then decompress it
otherwise if the section name starts with .debug_, then look up the .zdebug_ section, verify it starts with ZLIB, and then decompress it

OK I've implemented this. First we need to add support for ELF Chdrs to goblin: https://github.com/m4b/goblin/pull/83

I'm fine with temporarily changing Cargo.toml to specify a git version.

I'm not in a hurry with this so I guess it's less churn to wait for the goblin release.

FYI, goblin 0.0.15 has been released.

Fixed by #50


Render images directly in Jupyter notebooks

When showing an SFrame in Jupyter notebook (without calling explore(), natively in the notebook), it would be great if the output would render images instead of showing their path.

@shreyajain17 I assigned this to you. You should look at https://github.com/apple/turicreate/blob/master/src/unity/python/turicreate/visualization/_plot.py#L255 for an example of how to render things in IPython (Jupyter) Notebook. You can probably write a much simpler version of this function on the tc.Image type in Python though. The documentation says that instead of _repr_javascript, you can instead directly use something like _repr_jpeg: https://ipython.readthedocs.io/en/stable/config/integrating.html#rich-display


New version: RegistryCI v9.0.0

Registering package: RegistryCI
Repository: https://github.com/JuliaRegistries/RegistryCI.jl
Created by: @DilumAluthge
Version: v9.0.0
Commit: 3082b394451a7117be25ac2bf13f7ec1f893576d
Reviewed by: @DilumAluthge
Reference: https://github.com/JuliaRegistries/RegistryCI.jl/commit/3082b394451a7117be25ac2bf13f7ec1f893576d#commitcomment-130947922
Description: Continuous integration (CI) tools for Julia package registries, including registry consistency testing, automatic merging (automerge) of pull requests, and automatic TagBot triggers

[noblock]
This update includes https://github.com/JuliaRegistries/RegistryCI.jl/pull/516.


🔨 config migration|schema and add 🐳 file

Related Tickets

#Ticket_ID

WHAT this PR does?

ex: Change number items completed/total in user page.

Notes Impacted Areas

N/A.

Attachments

N/A.

merged


Better Bouncing

In the Disk.bounce_off_of_disk-method, the impulse of the disk-that's-bounced-off-of is not respected.
Also, I'm not sure what value in the calculations corresponds to the severity of impact that self suffered. Currently, j (the "impulse scalar" (?)) is returned, but that seems quite high. If changed, the damage-calculation for the ship should probably drop its sqrt, as well.
Lastly, the Disk.bounce_off_of_disk more-or-less assumes that no energy is lost (despite causing ship-damage). It used to be the case that the ship was only deflected with about half its original velocity. Is that still desired?

Oh, and I have no idea what the physical interpretation of the if self_vel_along_normal > 0: check is. Is it correct behaviour that the function terminates early in that case, and that it reports that no impact occurred?

Also, in Universe.apply_bounce(), the asteroids try to bounce off of each other. However, the current implementation of Disk.bounce_off_of_disk() only allows a one-sided interaction: It bounces Asteroid A off of Asteroid B, but leaves B unaffected. Because A is then moved outside of B, B never gets a chance at bouncing off of A.
This should probably be dealt with by introducing a second bounce-method that has both objects bounce off of each other. The current method must persist, because e.g. a ship bounces off of a planet, but a planet doesn't bounce off of a ship.

the way the ship stuff bounces against the planets seems fine for now. So creating a new asteroid bounce method seems like the thing to do.


Python documentation is mis-formatted causing it to display oddly

File: themes/default/content/docs/reference/pkg/python/pulumi/_index.md

Fixed via https://github.com/pulumi/docs/pull/10279.


Documentation page offline

Hello, the documentation page below is offline
https://juliaimages.org/OpenCV.jl/latest/

This package is on survival mode only: https://juliaimages.org/OpenCV.jl/dev/ and more documentation is needed

With the link in the About column on the GitHub page referring to the dev documentation, I guess we can close this @ashwani-rathee ?


Update docstring examples and code

Fence/indent/dedent code samples in docstrings with proper language tags so that built-in julia help and Documenter.jl can pick them up.

Yes, I can confirm that the fails here are orthogonal; see https://github.com/JuliaAI/MLJBase.jl/pull/972 .

Closed-reopened to retriever tests.

Thanks @abhro for the rebase. Let's come back to this after:

[ ] Resolve this discussion on julia> prompt.


Help Wanted: Access conversations with deactivated users

Context
The Mattermost Product Documentation doesn't specify whether users can access conversations with deactivated users.
Documentation Request
Clarify in the end user product docs that any direct or group message history you had with a deactivated user remains available unless your system admin disables your ability to do so.

@cwarnermm - Can I help with this one?

@cwarnermm - Is the page you referenced where you want this added? Under Users and teams?

@cwarnermm - I deleted my previous comment, because I realized I didn't see the link under the Documentation Request.
I am seeing that there is also the page about communication with messages and threads so maybe we want to add something there? Could be just a reference?

@TomerPacific - Very open to this approach! Looking forward to your docs PR.

@cwarnermm - Let me know what you think.

@TomerPacific - I wonder if we want to combine both callouts as notes, rather than displaying a tip followed by note. In the combined case, both would be formatted as bullets. Thoughts?

@cwarnermm - Sounds like a good approach as it will lead to a more cohesive UI.


Using TypeScript

I think rewriting pnpm to TypeScript would be beneficial.

I remember several bugs that TypeScript would have caught automatically. For instance, this one: 84d9aa508dcd40d1a396402f68e55c29cb72a8d8.
From a marketing perspective this would be another plus over npm/ied.
For the community it would be easier to contribute to pnpm with type checking, auto-completion, hints and other awesome stuff the TypeScript provides.

Does pnpm have programmatic API to use?

yes, https://github.com/rstacruz/pnpm/blob/master/docs/api.md

than, typescript will not help raw-js consumers to validate their code anyway. and there is not run-time check as far as i know it

also, a lot of people will complain about implementing it in flow instead

I am not really familiar with flow, but I was under the impression that it has the same syntax as TypeScript but better tooling.
It will not change anything for the raw-js users but it would make the life of pnpm maintainers easier.

We can even look at it as a smarter eslint... For instance, like a month ago I changed a method called linkBins to accept only 1 argument. And only recently I noticed that in 1 place I did not update the code which was using it 7ffb692b24650b4da9a2d78258351306c0dc710b. It wasn't a bug, because the additional args were just ignored, but the code looked weird. TypeScript would've caught that

I'm all for static typing, but I'd rather go with Flow as it has more powerful type system and thus much better chances to catch an error.
As a con for both TypeScript and Flow is that build process becomes necessary: you can't run code before transpiring it (well, you can with flow-node for example and I think something like this exists for TypeScript).
I could make a PR with such build process incorporated and some of the modules annotated with Flow to see how it feels.

make the life of pnpm maintainers easier

thats a good point

I will be able to read and edit static-typed js as well, i guess.
So @zkochan, @andreypopp pick a side and go ahead =)

Well, personally I like TypeScript 2 more, but it is in beta yet and it is easier to convert an existing project to Flow. So if @andreypopp wants Flow, I don't resist.
And we can switch to TypeScript later, in case it will become superior.


refactor: fix a react-hooks/exhaustive-deps warning

An ESlint warning of react-hooks/exhaustive-deps is caused by https://github.com/vercel/swr/commit/b18414b901d60fbe63232537f298dc847ae20822 because the rule cannot determine rerender is always the same reference from rerender = useState({})[1]. Previously, this line was [, rerender] = useState({}) and the rule can determine it is always the same reference.
To fix this, I've added rerender into the deps array because I'd like to enable the rule for the setSize function.

Thank you!


Hypershift local hosting addon

Issue: https://github.com/stolostron/backlog/issues/26215

/assign @ray-harris @eemurphy @cameronmwall

/unhold

/lgtm

https://issues.redhat.com/browse/ACM-1754


Zend_Mime_Decode::splitMime error when boundary is '-'

I am having errors parsing mails where the boundary is a dash (-). (Mails are packed with Mpack sources:ftp.andrew.cmu.edu:pub/mpack/ The dash is harcoded in encode.c.)
The error is "Not a valid Mime Message: End Missing."
The code below is searching the boundary but the closing boundary is detected as intermediate boundary and no closing boundary is found after the last loop.
strpos($body, '--' . $boundary . "\n", $start)

I think we should add the newline at the beging of the expression or we need to find the closing boundary first and if $p is greater than $position of the closing boundary we need to stop the loop.

This sems to correct the bug for me.
 public static function splitMime($body, $boundary)
    {
        // TODO: we're ignoring \r for now - is this function fast enough and is it safe to asume noone needs \r?
        $body = str_replace("\r", '', $body);

        $start = 0;
        $res = array();
        // find every mime part limiter and cut out the
        // string before it.
        // the part before the first boundary string is discarded:
        $p = strpos($body, '--' . $boundary . "\n", $start);
        if ($p === false) {
            // no parts found!
            return array();
        }

        // position after first boundary line
        $start = $p + 3 + strlen($boundary);

        //find end boundary
        $p_closeBoundary = strpos($body, '--' . $boundary . '--', $start);
        if ($p_closeBoundary===false) {
            throw new Zend_Exception('Not a valid Mime Message: End Missing');
        }

        //find parts
        while (($p = strpos($body, '--' . $boundary . "\n", $start)) !== false && $p < $p_closeBoundary) {
            $res[] = substr($body, $start, $p-$start);
            $start = $p + 3 + strlen($boundary);
        }

        // the remaining part also needs to be parsed:
        $res[] = substr($body, $start, $p_closeBoundary-$start);
        return $res;
    }

--- Base (BASE)
+++ Locally Modified (Based On LOCAL)
@@ -61,19 +61,20 @@
         // position after first boundary line
         $start = $p + 3 + strlen($boundary);

-        while (($p = strpos($body, '--' . $boundary . "\n", $start)) !== false) {
+        //find end boundary
+        $p_closeBoundary = strpos($body, '--' . $boundary . '--', $start);
+        if ($p_closeBoundary===false) {
+            throw new Zend_Exception('Not a valid Mime Message: End Missing');
+        }
+
+        //find parts
+        while (($p = strpos($body, '--' . $boundary . "\n", $start)) !== false && $p < $p_closeBoundary) {
             $res[] = substr($body, $start, $p-$start);
             $start = $p + 3 + strlen($boundary);
         }

-        // no more parts, find end boundary
-        $p = strpos($body, '--' . $boundary . '--', $start);
-        if ($p===false) {
-            throw new Zend_Exception('Not a valid Mime Message: End Missing');
-        }
-
         // the remaining part also needs to be parsed:
-        $res[] = substr($body, $start, $p-$start);
+        $res[] = substr($body, $start, $p_closeBoundary-$start);
         return $res;
     }

NB : The function Zend_Mime_Message::_disassembleMime seems to contains duplicated code from Zend_Mime_Decode::splitMime.

If you need something to test:
$body = <<<MailMessage

This is a MIME encoded message.  Decode it with "munpack"
or any other MIME reading software.  Mpack/munpack is available
via anonymous FTP in ftp.andrew.cmu.edu:pub/mpack/
---

---
Content-Type: application/octet-stream; name="testFile.txt"
Content-Transfer-Encoding: base64
Content-Disposition: inline; filename="testFile.txt"

dGVzdDEyMzQ=

-----
ignored text line1
ignored text line2

MailMessage;
try{
    $parts = Zend_Mime_Decode::splitMime($body, '-');
    echo 'OK';
}catch (Exception $e){
	echo $e;
}


support aws eks 1.16

does not work on aws eks 1.16
NODE Unable to find kernel headers. Try rebuilding kernel with CONFIG_IKHEADERS=m (module)
[E2] chdir(/lib/modules/4.14.177-139.253.amzn2.x86_64/build): No such file or directory
[E2] Traceback (most recent call last):```

Thanks for the report.
I suspect it will be fixed with https://github.com/kinvolk/inspektor-gadget/pull/110 but I have not tested yet.
Is the package kernel-devel installed on the host?
Is the build directory a symbolic link, and what does it point to?
ls -l /lib/modules/4.14.177-139.253.amzn2.x86_64/build

Yep the kernel-headers are installed.
The directory is a symlink to usr/src/kernels/4.14.177-139.253.amzn2.x86_64.

Hi @aavileli, #110 is merged now. Do you have some time to try it on your eks setup?

Hi @alban
I compiled the latest master it works now.
Thanks


Remove deprecations before release

Remove all deprecations (mostly marked as TODO) just before release, remove update support from ancient versions

Fixed in 25c53a541a


Allow reasding subscribe credentials from NTFY_USER env

This allows running ntfy sub test and having the credentials in an env. Tested locally with my ntfy server.

Thanks. Sorry it took so long.


Reservoir Internal Restart Looping

This allows level pool reservoirs to successfully pass an internal restart by correctly setting each reservoir's water elevation between loops. This also has the ability to toggle a parity test for a reservoir's water elevation.
Some further investigation is needed on Tues (9/14) for the Hybrid and RFC internal restarts because of the timing differences for reading timeslice and time series files. The updates for those two types of reservoirs for internal restarts will follow in another separate PR.
Checklist

[x] PR has an informative and human-readable title
[x] Changes are limited to a single goal (no scope creep)
[x] Code can be automatically merged (no conflicts)
[x] Code follows project standards (link if applicable)
[x] Passes all existing automated tests
[ ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)

Executes all of the following, approving.
python -m nwm_routing -V2  -f ../../test/input/yaml/CustomInput.yaml
python -m nwm_routing -V3  -f ../../test/input/yaml/CustomInput_loop.yaml
python -m nwm_routing -V2  -f ../../test/input/yaml/Croton_NY_levelpool.yaml
python -m nwm_routing -V3  -f ../../test/input/yaml/Croton_NY_levelpool_loop.yaml
python -m nwm_routing -V2  -f ../../test/input/yaml/Croton_NY_rfc.yaml
python -m nwm_routing -V3  -f ../../test/input/yaml/Croton_NY_rfc_loop.yaml
python -m nwm_routing -V2  -f ../../test/input/yaml/Croton_NY_hybrid_usgs.yaml
python -m nwm_routing -V3  -f ../../test/input/yaml/Croton_NY_hybrid_usgs_loop.yaml
python -m nwm_routing -V2  -f ../../test/input/yaml/Florence_Benchmark_diff.yaml
python -m nwm_routing -V2  -f ../../test/input/yaml/Croton_NY_levelpool_diffusive.yaml


[doc] Linux support?

Hey guys,
Could the main README mention whether lumina supports linux?
I assume so, but perhaps there may be some caveats or such.

Thanks for the ticket.  Currently the ReadMe states:

The Lumina Desktop Environment is a lightweight system interface that is designed for use on any Unix-like operating system.

Since Linux is a Unix-like Operating system, we thought this was covered, but I guess we can go in and make it more specific.  The problem is that once we start naming 'some' Unix Like Operating systems, If we don't list all of them then people may believe that Lumina wont run on them.  That's why we chose to write 'Unix Like'.
However, all that being said, we will talk it over and see if we can come up with a better way to explain it.  Thanks for letting us know it's not completely.

Done!
https://github.com/trueos/lumina/commit/1b2959737e8ee8ad79a8b7fed981363aae05b103


Aioredis is now in redis-py 4.2.0rc1!

Aioredis is now in redis-py 4.2.0rc1+
To install, just do pip install redis>=4.2.0rc1. The code is almost the exact same. You will just need to import like so:
from redis import asyncio as aioredis

This way you don't have to change all your code, just the imports.
https://github.com/redis/redis-py/releases/tag/v4.2.0rc1
Now that aioredis is under Redis officially, I hope there will never be an unmaintained, asyncio Redis lib in the Python ecosystem again. I will be helping out maintenance at Redis-py for the foreseeable future just to get some of the asyncio stuff out of the way. There are also some bugs that didn't make it into the PR that I'll be slowly migrating over throughout the next few weeks -- so long as my exams don't kill me beforehand :)
Thank you all so much for your commitment to this repository! Thank you so much to @abrookins @seandstewart @bmerry for all the commits and maintenance. And thank you to everyone here who has been adopting the new code base and squashing bugs. It's been an honor!
Cheers,
Andrew

    async with redis.lock(
AttributeError: __aenter__

locks not implemented

They don't work. We're still speculating how to do the locks properly since there was a debate on whether aredis's lock was correct. But aredis's lock does uhh functionally work so you can take their code. I think I'llmport it over too so it's not a big surprise.

When this project would be reverted to aioredis codebase instead of current one?


Updated SEO options for i18n

Changed example code to conditionally to avoid adding the suffix to the homepage to fix TypeScript error.

Type 'undefined' is not assignable to type 'NextSeoProps'.

Added an example code to conditionally to avoid adding the suffix to the homepage when using i18n.

closed in favor https://github.com/shuding/nextra/pull/1414


How to disable compression

Best practices mention to implement compression in client. https://github.com/Qihoo360/pika/issues/206 mentions the "compression" parameter can be set to "none", but the recent wiki doesn't mention that.
Is it still possible to set compression to none? Does it work with the prebuilt binaries?

you can try to set it none, it should work

Ok, thanks


Catching type errors

From this Tweet by @bennadel - https://twitter.com/BenNadel/status/734399795641655296:

can I enforce type-checking in the TS compiler in the NG2 Plunkr getting started? meaning if I have fn() : string but fn() returns Observable, I’d like to see an error.

@frankwallis do you know if this is possible with this plugin approach at all?

Thanks for creating this - I'm really trying to get on the band-wagon :)

Sure, it really helps to hear these questions to know where polish is needed!

I've updated this plunkr to turn on the type-checking, and add typings configuration for the angular packages and rxjs:
https://plnkr.co/edit/p1herPVQOcSKYq8YWXCd?p=preview

Amazing, thanks! Hopefully we can get the Angular demos to update to this?

@frankwallis @guybedford awesome stuff! After some fiddling, I'm able to get this working in my local demo. One question, though - what does this do in the import statement:
return System.import("plugin-typescript")
    .then(function(pts) {
        return pts.bundle();
    })

The demo seems to work with or without this statement. So, I'm not sure if it's actually needed.

I don't want to overstay my welcome here, but I have a question about the expected behavior of type-checking. I have your demo working locally, but I have code that looks like this (sort of):
public values: Foo[];
// ....
function handleResponse( newValues: Bar[] ) {
    this.values = newValues;
}

Here, you can see I am declaring the values as an array of Foo. But, I get no errors when I try to assign an array of Bar to it. And, I'm wondering if that's because I need to have a custom typings file somewhere? Or, if this just isn't a check that the compiler will look at anyway.
Sorry if these are obvious things, I'm super new to TS.

@bennadel these are both good questions. About the bundle, I'm not sure actually, perhaps that was just @frankwallis testing the behaviours. For the typing, perhaps @frankwallis can suggest further as well, I'm also not super familiar with typescript myself unfortunately.

@guybedford I think that maybe the type-mismatch was working because the value being passed into the function was being returned from another function whose type matched. So, while I had handleResponse( newValues: Bar[] ), TypeScript new that it was actually receiving Foo[] based on something higher up in the call-stack .... so maybe it was just ignoring my method signature.
Only a guess :D

FYI, I shared these results on my blog.
http://www.bennadel.com/blog/3095-better-type-checking-with-in-browser-typescript-transpiling-in-angular-2.htm

FYI, that type-checking problem I was having was due to the fact that I was using method.bind( this ). Apparently .bind() kills type checking.

@bennadel thanks for the awesome blog post, I'm glad I could help get it working.
Regarding the bundle call - this is not strictly necessary, I used it in the plunkr to debug an issue I was having with the typescript definition file not being loaded so you can safely remove it.

@frankwallis ah, good to know, thanks. And thanks again for all the help !


Fsecure interlock

Use FSecure Armory Interlock to fetch keys

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Add  configurable banner

Towards #223

Cc @dcherian, let me know what you think.... For now, this is advertising the user survey.

Once we have our first blog post, i'll look into the latest blog post idea


api: add zookeeper and mysql to v3alpha

Description: Follow-up to #8133, these likely escaped detection since they're versioned "v1alpha1" and the cloning script only searches for the last major version. This should be a one-off to get these caught up so I'm not modifying the tooling to accommodate these.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Derek Argueta<EMAIL_ADDRESS>

@derekargueta I think we do need to have the tools do this, since these APIs might gain new fields before we do the v3 cut. I think it's fairly easy to do, check out tools/type_whisperer/typedb_gen.py and tools/proto_sync.py, look for references to v2alpha.

I didn't want to add one-off logic but since we already handling vNalphaN for v2 in tools/proto_sync.py I feel less bad :) will push an update.

@htuch tooling updated. I believe it's is working as expected. When running with some debug prints, I verified it picks up the v1alpha1 bazel targets but hits this line because (IIUC) there are no deprecated fields in v1alpha1 for these two protos so there's nothing to modify.

@derekargueta ah yeah. So, I think your tool changes make sense, but then we shouldn't have the actual protos in v3alpha. We're only upgrading packages that have either had a breaking change or transitively depend on a package that has been upgraded. We basically have a DAG of versions.


Deprecate ignore field in config and rename to ignoreVulns

the PackageOveride.ignore field is ambiguous as to whether license scanning is still performed. Solution could be to rename ignore to ignoreVulns.

I thought that ignore had it ignored from both license scanning and vulnerabilities?
fwiw, I'd recommend doing something like vulnerabilities.ignore to mirror license.overrides

Huh you are right, I am just very confused at the behavior of our configs. I'll update this issue to be to introduce a just ignore vulns option.


Convert some tests to use expect_correction

This covers the easy to convert tests from metrics, security, and lint. There will be more changes to come.

@rrosenblum The changes look good but the build is failing.

Thanks!


crates.io

I was looking for regex-fancy and was surprised it was not on Cargo. If you want that name grab it.

Done, I just published 0.1.0. It's still rough though, but that's another story.


Test Enzyme on 1.11

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 16.16%. Comparing base (3470e14) to head (4cc62d0).

:exclamation:  There is a different number of reports uploaded between BASE (3470e14) and HEAD (4cc62d0). Click for more details.
HEAD has 39 uploads less than BASE

Flag
BASE (3470e14)
HEAD (4cc62d0)

DI
33
2

DIT
8
0

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #556       +/-   ##
===========================================
- Coverage   98.39%   16.16%   -82.24%
===========================================
  Files         107       74       -33
  Lines        4739     3075     -1664
===========================================
- Hits         4663      497     -4166
- Misses         76     2578     +2502

Flag
Coverage Δ

DI
16.16% <ø> (-82.47%)
:arrow_down:

DIT
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[Feature] HA via Galera

Support for HA via MariaDB Galera following the same approach as this PoC.
A posible solution might be:

When Galera is enabled in MariaDB resources, create the StatefulSet and Pods with the mariadb.mmontes.io/galera: enabled annotation
Create a new GaleraReconciler that will watch StatefulSets and Pods with the mariadb.mmontes.io/galera: enabled annotation
The Pod will have an extra sidecar container, the Galera reloader, which has an API to reload the /etc/mysql/mariadb.conf.d/galera.conf file and gracefully restart the mariadbd process by sending a signal.
Whenever a Pod goes down, the GaleraReconciler queries the StatefulSet to get the total number of pods, and based on their availability makes calls to the Galera reloaders to update the configuration. It will use the StatefulSet FQDN to selectively talk to specific instances (mariadb-0.mariadb.default.svc.cluster.local).
Check the PoC for more details: https://github.com/mmontes11/mariadb-galera-poc

Related to:

https://github.com/mariadb-operator/mariadb-operator/issues/103#issuecomment-1540524903
https://github.com/MariaDB/mariadb-docker/pull/377
https://github.com/MariaDB/mariadb-docker/issues/28
https://jira.mariadb.org/browse/MDEV-29762

Additional resources:

https://mariadb.com/kb/en/galera-cluster/
https://galeracluster.com/library/documentation/
https://groups.google.com/g/codership-team
https://github.com/mariadb-operator/mariadb-ha-poc/tree/main/galera/kubernetes
https://github.com/businessbean/helm-charts/blob/master/common/mariadb-galera/helm/scripts/mariadb-galera/entrypoint-galera.sh
https://github.com/an3l/mariadb-k8s
https://github.com/Pluies/config-reloader-sidecar

Further improvements:

https://github.com/mariadb-operator/mariadb-operator/issues/84
https://github.com/mariadb-operator/mariadb-operator/issues/85

Supported as part of v0.0.16 release:
https://github.com/mariadb-operator/mariadb-operator/releases/tag/v0.0.16


use jakarta Generated

use or allow configuration of annotations to use the jakarta generated annotations. javax annotations are "deprecated" and do not work with jpms. note: I'm not certain if null annotations are added, but these can be jakarta as well, they've not been merged into the same jar.

Just for a quick workaround, you can disable adding @Generated with addGeneratedAnnotation = false (actually false is the default, so you're probably enabling explicitly).
This is a good point though. By switching to jakarta.* it would break users who are still on Spring Boot 2.7, but since generating it is optional and disabled by default, I think I'm ok with that.

Created a PR for this, but need to discuss with @srinivasankavitha as well

I think you might want to define break. Since we're talking about the generated annotation and the Jakarta annotations were available in spring boot 2, they weren't the default yet though. Arguably it should at least be a configuration option.
Yes I enabled it explicitly because I was trying to get one of the other issues I reported to go away. I thought that error prone was in some way configured to ignore generated classes. It might be getting done a different way though... I didn't really bother to dig into it too much further.

I think you might want to define break. Since we're talking about the generated annotation and the Jakarta annotations were available in spring boot 2, they weren't the default yet though. Arguably it should at least be a configuration option.
Yes I enabled it explicitly because I was trying to get one of the other issues I reported to go away. I thought that error prone was in some way configured to ignore generated classes. It might be getting done a different way though... I didn't really bother to dig into it too much further.

Hi @paulbakker,
Related to this topic... In the open source maven plugin port for DGS codegen we had a recent issue opened related to how the @Generated annotations are written. By including the date element we prevent fully reproducible builds. Thoughts on making adding another option to not include the date element of the generatd annotation?

@deweyjose interesting, that's a good point. I wasn't actually aware dates are generated at all.
Also, could I have your thoughts about the javax vs jakarta @Generated? While this should be a simple change, I'm still a bit worried about unforeseen consequences.

Not @deweyjose or OP, but I am the one that created the issue to the Maven plugin about the non-reproducibility caused by the generated dates.
There should be an option to use Java EE for those that need it, but Jakarta EE should be the default since it has been the modern replacement for Java EE since 2019.
For reference, OpenAPI Generator's java generator has a useJakartaEe option. Though I would suggest instead to have a useJavaEe option defaulting to false.

Hi @paulbakker. Have you given any further consideration to adding an option for disabling the generated date?
To me, this is a separate issue than what is being proposed by @xenoterracide. Let me know if you'd prefer a separate issue.

@erichaagdev sorry for the delay, got busy with other things.
For the date generation, I added disableDatesInGeneratedAnnotation as a config setting in the same PR. Merged, and will be part of the next release.

Why did you resolve this issue for that? Turning off dates doesn't fix the fact that the old Java X library isn't supported anymore.

That's great! Thanks, @paulbakker! 🙌
@xenoterracide that's fixed now too.

Why did you resolve this issue for that? Turning off dates doesn't fix the fact that the old Java X library isn't supported anymore.

Yes, it's doing jakarta now, and it has the new option for not generating dates.
Decided against adding another option for a javax fallback. As described in the PR, a Boot 2.7 user can add the jakarta.annotation:jakarta.annotation-api dependency to the build if needed.
Boot 2.7 isn't supported anymore for quite a while now (by Spring), so I don't think it's needed to keep backward compatibility.


Improve SEO

Document doesn't have a  element - https://dequeuniversity.com/rules/axe/4.8/document-title
Document does not have a meta description - https://developer.chrome.com/docs/lighthouse/seo/meta-description/?utm_source=lighthouse&utm_medium=devtools
Incremental static generation - https://nextjs.org/docs/pages/building-your-application/data-fetching/incremental-static-regeneration

https://nextjs.org/docs/app/api-reference/functions/generate-metadata