WIP: typescript sys proxy

Well, I had teasered this in #227 and I guess it's better reviewable if I have an open PR for this branch ;)
So far, this seems to work transparently with useTypescriptIncrementalApi: true and the module resolution is working just fine as well with the latest commit.
I'm currently testing this with .mdx files, but there should not be anything stopping this to work with .vue as well.
I think I'm gonna do some testing with  useTypescriptIncrementalApi: false and get that to work.
From there on, I'm gonna wait for some feedback from you:

How would you structure this? (everything in one wrapTypeScript.ts is not really feasible - would splitting off a wrapperUtils.ts and a wrapCompilerHost.ts be enough?)
Should this replace the .vue code?
Do you want to keep the .vue and .mdx specific code in this repo (now that they're going to be only about 20 lines per extension) or do you still want to have a pluggable approach (which would most likely be more code than the extension-specific handling code itself).
Overall: what do you think of this? Is this even feasible?

You have been busy 😄

How would you structure this? (everything in one wrapTypeScript.ts is not really feasible - would splitting off a wrapperUtils.ts and a wrapCompilerHost.ts be enough?)

Splitting it seems like a good idea.

Should this replace the .vue code?

Possibly - I'd be interested to see how that looks.

Do you want to keep the .vue and .mdx specific code in this repo (now that they're going to be only about 20 lines per extension) or do you still want to have a pluggable approach (which would most likely be more code than the extension-specific handling code itself).

I'm not decided on this yet. My current feeling is that Vue should remain in the box at least for now since it's already there. But I'm unlikely to add anything else - including mdx I'm afraid. I'm still keen on a pluggable solution.

Overall: what do you think of this? Is this even feasible?

I think this is potentially very interesting. I'm afraid I'm occupied with a number of other things at present and so haven't devoted too much thought to it. But I think it has merit.
In terms of sequencing I'd like to see the Vue incremental fixes land first: https://github.com/Realytics/fork-ts-checker-webpack-plugin/pull/220
Essentially I want to get existing users to a good place before looking at changing the underlying architecture and potentially open up the possibility of new users.
I think the work you're doing is very cool though - and I'm grateful for the effort you're putting in 🌻❤😄

Well, now you can take a look at how it looks with .vue ^^
Essentially, all you need is a config like specified here: https://github.com/Realytics/fork-ts-checker-webpack-plugin/blob/12e85371d95c858cf82054948d8c312476a1bd1c/src/wrapTypeScript.ts#L19
That just references this loader: https://github.com/Realytics/fork-ts-checker-webpack-plugin/blob/12e85371d95c858cf82054948d8c312476a1bd1c/src/handleVueContents.ts#L4
But right now, there are some things now working yet and tests are failing.
From what I have gathered so far what goes wrong:

as I just attach .tsx on an early file system level to all vue files without looking at the file contents, every vue file is regarded as .tsx, no special handling for .ts is possible :cry:  Guess I need to figure out a smart way around that sigh
module resolution still has it's kinks (the test with the @-namespace is failing for example)
generally, a lot of .vue-tests are failing. I haven't looked at all of them yet.

Okay, apparently nothing is too bad with my module resolution - it's just that there seems to be an implicit mapping for a virtual "@"-module in vue.
Adding something like
baseUrl: './src',
        paths: {
          '@/*': ['./*']
        }

to the compilerOptions resolves that problem.
Now, I don't know vue myself. Is that something that should implicitly be added?

Now, I don't know vue myself. Is that something that should implicitly be added?

Also not a Vue user I'm afraid; don't know 😥

It seems like that "@"-wildcard was added by @CKGrafico - can you tell us if that should be there implicitly or if that is something a user would have added anyways to their .tsconfig to be able to use it?

Also, ts-loader (as well as apparently babel-loader?)  is going with two configuration options (appendTsSuffixTo, appendTsxSuffixTo), to determine which vue files are ts and which ones are tsx - I think I will go for a similar approach for the time being.
Or is there something actually requiring this auto-detection that is currently in there?

Or is there something actually requiring this auto-detection that is currently in there?

Not to my knowledge. If memory serves @herringtondarkholme came up with this when Vue support was added to ts-loader. See https://github.com/TypeStrong/ts-loader/issues/270

Okay, then I'll go with that for now.

For now, I've added the '@' namespace to the tsconfig of the associated test - I guess that is sensitive.
Also, I've associated .vuex files with .tsx and .vue files with .ts.
This makes all but three tests green: the tests that now complain that example-js.vue, example-jsx.vue and example-nolang.vue now also report errors.
An alternative solution to using file extensions would be to preprend files with their filetype like // @fork-ts-checker-handle-file-as (ts|tsx|js|jsx|json|none).
Adding that in IncrementalChecker would be no problem as the CompilerHost is wrapped there anyways, in ApiIncrementalChecker this would require to start wrapping the CompilerHost. Not the end of the world, but it still looks so prisine - so I'll leave that approach open to discussion :grin:
Aside from that, everything should be working now (?) - it would be cool if an actual vue user could test this, as I only have the test cases ;)

This makes all but three tests green: the tests that now complain that example-js.vue, example-jsx.vue and example-nolang.vue now also report errors.

Might be worth tweaking the tests to get them to get more information....
I don't get why some Vue tests would pass and some would fail. Is there a pattern?

No, that's pretty much expected behaviour at this point. As all .vue files are now handled  as either .tsx or .ts, the three files containing non-ts-javascript now throw type errors that would have been ignored before. So the tests actually do what they are supposed to do ;)
I guess this could only be mitigated by actually handling the files by their contents, not their extension - which is why I suggested that @fork-ts-checker-handle-file-as file header comment to be injected.
But I believe won't get around to implement that this weekend - maybe next weekend :)

I guess this could only be mitigated by actually handling the files by their contents, not their extension - which is why I suggested that @fork-ts-checker-handle-file-as file header comment to be injected.

Maybe that's a good shout. If memory serves @herringtondarkholme had some reservations about the appendTsxSuffixTo approach

I've just tweaked your branch so it runs the Vue tests when useTypescriptIncrementalApi is both true and false - remember to git pull 😄

I just did some reorganization - this way there should be less changes to the original files, everything is a little more transparent.
On the part of filetype-autodetection, I'm having a little bit of a blocker. For the IncrementalChecker, that won't be a problem - I can simply hook into host.getSourceFile and set the correct scriptKind. But I don't see a similar way for the ApiIncrementalChecker - do you have any idea how I could change the scriptKind there? (except changing the file extension dynamically by file content, I'd like to avoid that)

Okay... I could hook into typescript.createEmitAndSemanticDiagnosticsBuilderProgram and wrap the host parameter there once more. Not very elegant but seems to be working. I've gotta give this some though. And if you have a better idea - I'd be happy to hear it :)

Not very elegant but seems to be working. I've gotta give this some thought. And if you have a better idea - I'd be happy to hear it :)

If it's a straight choice between not working and inelegant but working I'll always go for the second option 😄

BTW - is this is what MDX is? https://gatsby-mdx.netlify.com Pretty cool!
Up until now I'd thought it was this: https://en.m.wikipedia.org/wiki/MultiDimensional_eXpressions - without having the first idea how that could be so 😁

Oh, lol, I didn't even know MultiDimensional eXpressions ^^
Yup, that is the mdx I was talking about. Markdown with JSX. Also used in https://github.com/jxnblk/mdx-deck and https://www.docz.site/ - and even currently being implemented to be a part of storybook.

FYI: I just noticed that your commit to run tests with useTypescriptIncrementalApi true|false is pretty pointless: Later in helpers.createVueCompiler, a IncrementalChecker is created manually, so most tests run as if useTypescriptIncrementalApi were false. Would be cool if you could look into this :)
I'm out for now - long day ;)

I just noticed that your commit to run tests with useTypescriptIncrementalApi true|false is pretty pointless: Later in helpers.createVueCompiler, a IncrementalChecker is created manually, so most tests run as if useTypescriptIncrementalApi were false.

I guess you mean this:
https://github.com/Realytics/fork-ts-checker-webpack-plugin/blob/2c03b7e45a9fc023efa3053403cc05ecff9146c0/test/integration/helpers.js#L59
This was added when initial Vue support for the incremental API was being implemented:
https://github.com/Realytics/fork-ts-checker-webpack-plugin/commit/53d690b85c7f0956b5c2909ee323178d2da90adb
@0xorial can you recall what the purpose of this  was please? Presumably we shouldn't need this?

I've looked through the history and creation of IncrementalChecker seems to be there from very beginning. I merely moved it into another file (probably because I had an intention to make 2 separate test sets for incremental=true/false), but I did not pay close attention back then I presume... So, unfortunately, I have no idea why is it there.
It have a feeling that author understood 'integration test' as 'any test that covers more than 1 file/class' while currently they are more like 'test that tests whole plugin as a black box'

Cool - so probably we can drop it.

@johnnyreilly the tests are now running with the real checker from the webpack instance - but, in the same process, not in a forked one.
I guess this is why that IncrementalChecker was added there in the first place.
Problem is: the tests still rely heavily on implementation details of IncrementalChecker that are not met by ApiIncrementalChecker like checker.program and in some cases can't even be met 1:1, like checker.programConfig.fileNames.
I'll try and see what I can do (might need to rewrite the Checkers to match at least partially a common interface), but no promises here.

And most of the tests are actually green now.
But I seem to have some regression regarding #94 - and I'm currently clueless as to why checkSyntacticErrors does not seem to have an effect.

Okay, this took me a LOT of time: with useTypescriptIncrementalApi: true, as soon as a syntactic error is found, it completely stops type-checking the rest. So, if checkSyntacticErrors = false, it just returns zero errors.
I guess this is a limitation of TypescriptIncrementalApi and these two tests can just never test anything meaningful in that mode?

Aha! I think you've just been on the same learning journey @0xorial and went on when support was originally added: https://github.com/Realytics/fork-ts-checker-webpack-plugin/pull/198#issuecomment-453812729
To quote myself:
Actually - I've dug into this myself.  I follow what you meant.  The project context includes a semantic error (index.ts) and a syntactic error (syntacticError.ts).  The test simply checks that there is any error surfaced.  There is different behaviour for useTypescriptIncrementalApi: true and useTypescriptIncrementalApi: false but that is OKAY. 😄
In fact even good.
useTypescriptIncrementalApi: true
The compiler detects errors in both index.ts and syntacticError.ts.  The compiler has found a syntactic error and so doesn't bother to go on to surface the semantic error (for whatever reason - as you say likely performance).  But because we're not reporting syntactic errors we never get to hear about it; and our test fails.
useTypescriptIncrementalApi: false
Same story as with true but there's no short circuiting (that we're aware of) happening in the compiler in this case so we hear about our errors and our test passes.

I think what we should probably do is amend our test pack so we either expect different behaviour depending upon the mode or (perhaps this is better) write tests which should pass consistently regardless of mode.
Essentially, if a single error is found then that's great. We're much more interested in the difference between 0 and 1 errors being found in comparison with say 7 and 8 errors being found.

Essentially, if a single error is found then that's great. We're much more interested in the difference between 0 and 1 errors being found in a test in comparison with say 7 and 8 errors being found.

When I get a moment I'll try and improve the test pack to reflect this.

Just as a heads-up: I've not given up on this or anything. I just have a few turbulent weeks right now and don't have much time to work on this - it'll get better in 2-3 weeks.
But besides fixing up the last tests and maybe moving "fs" to a wrapped dependency injection for IncrementalChecker (so that this line won't be necessary, keeping all the wrapping magic apart from the implementation itself), I think this PR is pretty much approaching a 'ready' state.
In the end, I'd remove the mdx-specific stuff and follow this up with a second PR for plugability.
So if you could @ a few vue-users to maybe give this a real-life test in the meantime, I guess it is already pretty much ready for that.

Just as a heads-up: I've not given up on this or anything. I just have a few turbulent weeks right now and don't have much time to work on this - it'll get better in 2-3 weeks.

Don't worry - life happens and there's no rush.

So if you could @ a few vue-users to maybe give this a real-life test in the meantime, I guess it is already pretty much ready for that.

Happy to do that - recommend you put a branch together which includes the lib folder on GitHub. This will allow people to test easily. We've done this in the past and provided instructions to make it easier for users to test (and hence more likely that it will happen.)
There's an example of what I mean here: https://github.com/TypeStrong/fork-ts-checker-webpack-plugin

@phryneas I've just made change to the vue tests.  Essentially this change starts testing generated errors for the specific types of errors they represent; rather than an arbitrary count.  This is a more targeted testing approach and I think I want to roll this out wider than just the vue tests; but your work gave me a good chance to test the technique out.
Depending on the time I have, I might continue hacking on your branch if that's cool with you?

Yeah, I encountered that one, too.
The thing is - the tests aren't forking any more, everything is executed on the same process to get access to the actual IncrementalChecker used by the plugin. (this wouldn't be possible if it were on another thread, that's why before the IncrementalChecker was created manually, not making it a real integration test).
As a result, the forked processes aren't exiting, and no memory is being freed that way. So they hog up on memory.
But as far as I see, there is no other way of freeing memory, at least fork-ts-checker has no real mechanism for this.
And apparently, mocha is running all tests in the same process, too, so everything hogs up even more memory.
My only ideas to solve this would either be a cleanup method in the service, or to switch to a testing framework like jest that executes every test in it's own process.

The thing is - the tests aren't forking any more, everything is executed on the same process to get access to the actual IncrementalChecker

I didn't quite understand what you meant by this... Could you try and explain some more?

My only ideas to solve this would either be a cleanup method in the service, or to switch to a testing framework like jest that executes every test in it's own process.

I had an idea that we could try and force garbage collection between tests; see here: https://nodesource.com/blog/request-garbage-collection-for-node-js-processes-with-n-solid
Do you reckon that'd work?

The thing is - the tests aren't forking any more, everything is executed on the same process to get access to the actual IncrementalChecker

I didn't quite understand what you meant by this... Could you try and explain some more?

I'll try :)
Take a look at test/integration/helpers.js:
Before my changes, the createVueCompiler function did two things:

create a webpack instance
create an instance of IncrementalChecker
In reality, half of the tests were not using the webpack instance, but the checker instance. Also, that checker instance was created slightly different than the actual checker instance created in service.ts, so this was more of a checker unit test than a real integration test. (and never tested the ApiIncrementalChecker)

Now, what I wanted to do was

create a webpack instance
return the checker instance that is used by that webpack instance
Essentially, that was impossible: as webpack forks a new process for the service, which then generates the IncrementalChecker, the checker is on another process and cannot be accessed from the test.
So, I prevented the plugin from actually forking. In https://github.com/Realytics/fork-ts-checker-webpack-plugin/pull/231/files#diff-b9e343b630cc669a829ec4b8417a5e10 , I fake child_process.fork to execute the code in the current process instead of spawning a new process. Also, I temporarily override process.exit to prevent the service.ts from killing the test itself (which is now the same process as the service).
That way, the test can access the checker that is actually being used by webpack - which is now running on the same process.
Suddenly, the tests also check ApiIncrementalChecker.

The downside: where before, calling process.exit just cleaned up all memory used by the service, it now does nothing. That's why, from test to test, more and more memory is being consumed. There's just no proper cleanup.

My only ideas to solve this would either be a cleanup method in the service, or to switch to a testing framework like jest that executes every test in it's own process.

I had an idea that we could try and force garbage collection between tests; see here: https://nodesource.com/blog/request-garbage-collection-for-node-js-processes-with-n-solid
Do you reckon that'd work?

I guess setting global.checker to undefined and then running garbage collection could work. Would you give it a try?
PS: I really appreciate you fiddling around with this, so yeah, of course you can push to this branch all you want :)

Interesting! So the existing API incremental tests aren't really doing what we hope?  Great that you've found a solution; it's a shame it's not quite a test of what the users will experience as it's going to be in the same process.... But given the existing tests are really false positives this is demonstrably better!

Essentially, that was impossible: as webpack forks a new process for the service, which then generates the IncrementalChecker, the checker is on another process and cannot be accessed from the test.

I wonder if there's a way to tweak the checker to expose the checker purely for the purposes of testing... That could be potentially useful - it wouldn't need to be something we exposed always; we could have a "test" mode or something... Ideas!

I guess setting global.checker to undefined and then running garbage collection could work. Would you give it a try?

Yeah - not sure when but I'll try a few things out and let you know how I get on :smile:

Tried it; no joy alas 😢

Thanks to #246, the tests are working correctly now.
For testers, I published a build of this as https://www.npmjs.com/package/@phryneas/experimental-fork-ts-checker-webpack-plugin

For testers, I published a build of this as https://www.npmjs.com/package/@phryneas/experimental-fork-ts-checker-webpack-plugin

That's awesome. I'd like to publicise this to see if we can get some beta testers on board. I'll tweet about this later and mention a bunch of people in the comments here as well.
Before I do that I just wanted to check:

do Vue users have to use the useTypescriptIncrementalApi flag to activate the incremental API for Vue?

aside from Vue users I guess we're interested to know generally if this has any impact to users generally. Functionality should be unchanged given our tests be passing. We've got no performance tests though. (If anyone ever gets a moment we'd love some to be added. I believe @jantimon has some that could be used for inspiration here: https://github.com/namics/webpack-config-plugins 😁) So a general check from people anecdotally that perf doesn't suffer on their own projects is probably a good start.

* do Vue users have to use the `useTypescriptIncrementalApi` flag to activate the incremental API for Vue?

Yes, along with vue: true - it would also be useful to check with useTypescriptIncrementalApi: false if everything works correctly, because a lot changed there, too.

* aside from Vue users I guess we're interested to know generally if this has any impact to users generally. Functionality should be unchanged given our tests be passing. We've got no performance tests though. (If anyone ever gets a moment we'd love some to be added. I believe @jantimon has some that could be used for inspiration here: https://github.com/namics/webpack-config-plugins grin) So a general check from people anecdotally that perf doesn't suffer on their own projects is probably useful?

I guess that couldn't hurt.

I'll try and give this a spin in the coming days, thanks for your work!

@phryneas I seem to be having problems with the fork-ts-checker-notifier-webpack-plugin which plugs into this https://github.com/johnnyreilly/fork-ts-checker-notifier-webpack-plugin
I think this may mean that the integration hooks are in some way broken.... Alas I think we might be short of tests for that.  You should hopefully be able to reproduce this using this example with your tweaked setup:
https://github.com/TypeStrong/ts-loader/tree/master/examples/fork-ts-checker-webpack-plugin

@johnnyreilly the problems seems to be that fork-ts-checker-notifier-webpack-plugin tries to include the original plugin by name, which is different. As a solution, just
yarn add fork-ts-checker-webpack-plugin<EMAIL_ADDRESS>
oh, and that one is a new build, with all new changes from worker-rpc and master.

Oh, @johnnyreilly could you please give the travis a kick? It seems the tests for this branch aren't running any more, and now that they're fixed I'd like to see the results for all the webpack/node versions :)

I can see Travis running - is that not what you're seeing?

Ah, now Travis is running. And it seems I got some errors due to the merge. Gotta investigate that.
As for your experiences - I currently don't have a big project with fork-ts-checker (we are running babel and tsc --watch in the current project as the vscode integration is more fluent that way - remember, I started all this off with the intent of checking mdx in docz :wink:), just some small test project with about three files (and thus, no tslint).
But it seems I need to set up a big test project. Do you have any open source projects in your mind that I could clone for testing?

Okay, apparently updating typescript from 3.3.3333 to 3.4.2 makes all test fail :(
I've gotta give this some more time, but not today. I've taken free days for the rest of the week though, so I think I'll get some hours in there in the next few days.

But it seems I need to set up a big test project. Do you have any open source projects in your mind that I could clone for testing?

Great question! And no... We should ask the internet?

Great question! And no... We should ask the internet?

If you could ask your twitter, that would be great. I don't have that big of a following -.-

I've gotta give this some more time, but not today. I've taken free days for the rest of the week though, so I think I'll get some hours in there in the next few days.

Don't worry - take your time ❤

can i request that the mdx support be moved to a plugin? according to packagephobia this would add ~8mb to the dep tree. for reference, fork-ts-checker itself is less than half that size

@zackschuster before this gets merged, the mdx part will definitely be moved out again and a second PR with some kind of plugin functionality will be opened. this is currently just in there because I have no vue codebase to test ;)

@phryneas thank you!

Okay, so the recent errors with TS 3.4 are fixed.
@johnnyreilly going forward with this - I would wait for #249 to be merged and as a next step open a new PR to get the corrected vue tests into master, just to reduce the sheer size of this PR as I think this is getting way too big. Is that okay with you?
Then I'd clone myself https://github.com/desktop/desktop and try to reproduce your missing error messages.

@johnnyreilly I did some experimentation and I could not reproduce your missing lints/diagnostics. Could you give me some more information on your setup? Like webpack/tslint/typescript version and plugin config?
I noticed something else, though: when working with typescript 2.5, my modifications did not work. Apparently, some internal APIs I was using have not been in existance for as long as I was hoping. Also, older typescript code seems to do much more internal reference to typescript.sys instead of getting it passed in via DI.
As a consequence, I replaced the functions on the actual imported object instead of wrapping that one - that approach seems to work quite well so far.
I tweaked the travis config so it should now (for this branch) execute the tests for all minor typescript versions from 2.1 to 3.4 - I'm curious what more that will uncover. We'll see.

Could you give me some more information on your setup? Like webpack/tslint/typescript version and plugin config?

My setup is quite like this: https://github.com/TypeStrong/ts-loader/tree/master/examples/fork-ts-checker-webpack-plugin

I noticed something else, though: when working with typescript 2.5, my modifications did not work. Apparently, some internal APIs I was using have not been in existance for as long as I was hoping.

Good luck with your experimentations. I'm open to dropping support for older versions of TypeScript  should it prove useful. ts-loader dropped support for < 2.4 relatively recently which allowed us to simplify the codebase. This may not prove necessary but it's on the table if we need it

@johnnyreilly I got some failures, and as a comparison I am currently running the tests (depending on TS version without useIncrementalApi) against the current master branch. From the looks of it, at least some features are broken on old TS versions :disappointed:
You can check it out here: https://travis-ci.org/phryneas/fork-ts-checker-webpack-plugin/builds/522770455

I'll open an issue for this.

So after thinking about this some time I've put more work into this.
I've removed the code that was faking the typescript.sys object and taken a different approach for this:
Now I'm replacing the fs module with a fake one using mock-require which we had as a devDependency until now anyways. (We could also use something like monkey-fs, but those don't really do something different...)
That fake file system now modifies all paths passed to fs functions if they have specific extensions, just like I was doing it in the wrapped typescript.sys before. (X.vue <=> X.vue.fake.ts etc.)
That way, there is no special handling for this in our code any more, and almost no additional wrapping of TypeScript interna is required.
So far, this looks very promising. I will, however, have to take a look at the watching code, as currently that will emit events with un-faked filenames.
I believe it might be pretty simple to get this to work with TypeScript itself, as according to #256 it internally uses fs.watch and fs.watchFile - but our own code is using chokidar, which handles this differently. On the other hand, if we were to drop the old watch mode, supporting this might not even be necessary.
Just to give you a short report on what's happening here :)

Awesome work @phryneas!

github notified me of this because i was apparently mentioned somewhere, but i can't find the reference.
thanks for the hard work, @phryneas!

So, I think I've got the file watching situation under control, at least for the incrementalApi: true mode. Also, I've added a pretty extensive unit test for the filesystem wrapper.
I've created a new build that can be used with
yarn add fork-ts-checker-webpack-plugin<EMAIL_ADDRESS>
As for me, I'll start dogfooding this version, but as I'm not a vue user myself I won't really see much of a change. It could be great if a few vue users  (@zackschuster ? :smile:  ) could give this a go :)

Awesome work! @zackschuster would you be able to give this a try please?
Likewise I'll start dogfooding when I'm back working on front end stuff again (hopefully in a week 🤞)

@johnnyreilly @phryneas i don't currently have any vuejs projects that use typescript but i'll give it a shot when i'm back on my dev machine.
are there any known concerns i can give special attention to?

Thanks @zackschuster! It would be great to know if experimentalWatchApi: true and experimentalWatchApi: false still work as expected. Also that a production build still works with Vue.
Anything else @phryneas ?

@johnnyreilly I think you mixed the experimentalWatchApi up with ts-loader, in fork-ts-checker-webpack-plugin it's incrementalApi  😋
@zackschuster Thanks a lot!
It would be great if you could test it with incrementalApi: true and incrementalApi: false.
Typechecking in general, but especially if .vue files are being watched.
There's one minor breaking change I can think of: Before, vue files were seen by the typescript compiler as filename.vue.ts, now they're seen as filename.vue.__fake__.ts - this should not change everything if yout tsconfig.json is just including all .ts files, but if it only includes specific files you might need to change that there.

Hi, Sorry that I wasn't able to continue on the vue fix but I gave this a try on our (not open source, sorry) project.
TL;DR

Fastest was this experimental plugin without incrementalApi.
Linter won't be able to show codeframe anymore due to the "fake" name I think.
With incrementalApi enabled there is a HUGE (as in VERY HUGE...6 Minutes compared to 20 Seconds without incrementalApi) performance drop on initial build and also on incremental builds.
With incrementalApi enabled Linter errors are not reported immediatly but instead the compilation is done within milliseconds (I assume it fails silently?) after changing something else the compilation takes longer and finally reports the error.

Config
 new ForkTsCheckerWebpackPlugin({
                tslint: true,
                vue: true,
                useTypescriptIncrementalApi: ....,
                async: false,
                formatter: "codeframe",
                measureCompilationTime: true,
            }),

Tests
"fork-ts-checker-webpack-plugin" (watch)
Configuration: useTypescriptIncrementalApi: false

initial build:Using 1 worker with 2048MB memory limit
webpack is watching the files…
Compilation took: 22557.66 ms.```

incremental build 1: Compilation took: 9706.11 ms.
incremental build 2: Compilation took: 5526.23 ms.
incremental build 3: Compilation took: 5517.79 ms.
syntax error build: Compilation took: 5711.98 ms
ERROR in C:\src\Project\scripts\components\Admin\AdminManager.vue.ts
./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts& (C:/src/Project/node_modules/ts-loader??ref--2-0!C:/src/Project/node_modules/vue-loader/lib??vue-loader-options!./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts&)
[tsl] ERROR in C:\src\Project\scripts\components\Admin\AdminManager.vue.ts(87,29)
     TS1005: ',' expected.
@ ./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts& 1:0-193 1:209-212 1:214-404 1:214-404
@ ./scripts/components/Admin/AdminManager.vue
@ ./scripts/app.ts
@ multi ./scripts/app.ts

lint error build:   Compilation took: 5689.21 ms.
   ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
   ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
   95:1 Consecutive blank lines are forbidden
       93 |     // tslint:enable:object-literal-sort-keys
       94 |
     > 95 |
          | ^
       96 |     private created() {
       97 |         this.loadAll({from: 0, count: 20});
       98 |     }

type error build:Compilation took: 6052.88 ms.
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
95:13 Type '0' is not assignable to type 'string'.
    93 |     // tslint:enable:object-literal-sort-keys
    94 |
  > 95 |     private shouldBeString: string = 0;
       |             ^
    96 |
    97 |     private created() {
    98 |         this.loadAll({from: 0, count: 20});

"@phryneas/experimental-fork-ts-checker-webpack-plugin" (watch)
Configuration: useTypescriptIncrementalApi: false

initial build:
Using 1 worker with 2048MB memory limit
webpack is watching the files…
Compilation took: 19435.08 ms.

incremental build 1: Compilation took: 4081.24 ms

incremental build 2: Compilation took: 3112.74 ms

incremental build 3: Compilation took: 2911.64 ms

syntax error build:
Compilation took: 3098.1 ms.
ERROR in C:\src\Project\scripts\components\Admin\AdminManager.vue.ts
./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts& (C:/src/Project/node_modules/ts-loader??ref--2-0!C:/src/Project/node_modules/vue-loader/lib??vue-loader-options!./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts&)
[tsl] ERROR in C:\src\Project\scripts\components\Admin\AdminManager.vue.ts(87,26)
     TS1005: ',' expected.
@ ./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts& 1:0-193 1:209-212 1:214-404 1:214-404
@ ./scripts/components/Admin/AdminManager.vue
@ ./scripts/app.ts
@ multi ./scripts/app.ts

lint error build:
Compilation took: 2393.22 ms.
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue.__fake__.ts
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue.__fake__.ts
95:1 Consecutive blank lines are forbidden

type error build:
Compilation took: 2815.3 ms
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
95:13 Type '0' is not assignable to type 'string'.
    93 |     // tslint:enable:object-literal-sort-keys
    94 |
  > 95 |     private shouldBestring: string = 0;
       |             ^
    96 |
    97 |     private created() {
    98 |         this.loadAll({from: 0, count: 20});

"@phryneas/experimental-fork-ts-checker-webpack-plugin" (watch)
Configuration: useTypescriptIncrementalApi: true

initial build: Using 1 worker with 2048MB memory limit
 webpack is watching the files…
 Compilation took: 400250.35 ms.

incremental build 1: Compilation took: 11017.46 ms.
incremental build 2: Compilation took: 16267.02 ms.
incremental build 3: Compilation took: 10982.65 ms.
incremental build 4: Compilation took: 10485.2 ms.
incremental build 5: Compilation took: 10634.52 ms.
syntax error build:Compilation took: 3098.1 ms.
ERROR in C:\src\Project\scripts\components\Admin\AdminManager.vue.ts
./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts& (C:/src/Project/node_modules/ts-loader??ref--2-0!C:/src/Project/node_modules/vue-loader/lib??vue-loader-options!./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts&)
[tsl] ERROR in C:\src\Project\scripts\components\Admin\AdminManager.vue.ts(87,29)
     TS1005: ',' expected.
@ ./scripts/components/Admin/AdminManager.vue?vue&type=script&lang=ts& 1:0-193 1:209-212 1:214-404 1:214-404
@ ./scripts/components/Admin/AdminManager.vue
@ ./scripts/app.ts
@ multi ./scripts/app.ts

lint error build: Compilation took: 88.46 ms. (No error reported!)
lint error build (changed something else unrelated):Compilation took: 15772.66 ms.
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue.__fake__.ts
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue.__fake__.ts
95:1 Consecutive blank lines are forbidden

type error build:Compilation took: 11127.61 ms.
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
ERROR in C:/src/Project/scripts/components/Admin/AdminManager.vue
95:13 Type '0' is not assignable to type 'string'.
    93 |     // tslint:enable:object-literal-sort-keys
    94 |
  > 95 |     private shouldBeString: string = 0;
       |             ^
    96 |
    97 |     private created() {
    98 |         this.loadAll({from: 0, count: 20});

@WolfspiritM phew, these results are a bummer :disappointed:
Thank you for testing though, it's a great help! :+1:
One thing I would be interested in: could you give the tests a second round with async: true? There seems to be weird behaviour with different values of async sometimes, so I'm interested if that makes a difference.

Sorry for the late response. While trying to figure out https://github.com/TypeStrong/ts-loader/issues/988 I gave this a try again as it's more complicated to fix ts-loader and maybe ts-loader can use the mocked FS aswell as I was playing around with a similar way to get ts-loader working correctly by replacing ts.sys. But why reimplementing the wheel as you've already done that here.
I think I figured out why it is so slow.
It seems like the initial compile is fast (10 seconds compared to more then 20-30 before) but it's the linter that is very slow. The reason seems to be the call to getProgram() as the program seems to be always invalidated and recreated for every single file. The reason why that happens is that typescript seems to invalidate the program as soon as the host does custom module resolution.
https://github.com/microsoft/TypeScript/blob/01e1b1bb276f48eaa0612a34dd8f1eec5d86bd49/src/compiler/watch.ts#L805
I then removed the "resolveModuleNames" function from the CompilerHost as it didn't seem to do anything special here and everything was done within 10 seconds!
On watch it did compile within 300ms! Awesome!
Here are the changes I made: https://github.com/WolfspiritM/fork-ts-checker-webpack-plugin/commit/bd913cdd1bb6ea02b7da5832c246edf16ce121f0

This is awesome, thank you for taking another look! And for those awesome findings!
I'm currently on vacation but I'll spend some time with this once I'm back home.
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

@johnnyreilly @phryneas fyi i haven't done any testing for vue and i don't plan on using it for future projects, so i won't be following through on testing it. my apologies!

@phryneas Anything new?

@WolfspiritM I'd love to get to this, but seeing that I'd have to essentially re-do the whole PR for a third or fourth time on the current master (this is too far away for a simple rebase), I just haven't found the time for it yet.
I hope to maybe get to it in the holidays after new year's, but that's the earliest I'm seeing for this.
If you want to take a go with it before that, feel free to do so though :)

I've redone the changes on master and pushed everything to a new branch:
https://github.com/WolfspiritM/fork-ts-checker-webpack-plugin/tree/vue-fake-filesystem
I have some problems to get the tests working. Even on master the vue tests take ages to complete for me. And then fail with:
Timeout - Async callback was not invoked within the 60000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 60000ms timeout specified by jest.setTimeout.
Not sure yet why that is the case. Same happens on my branch.
Also I had to remove the precommit hooks. Husky doesn't let me commit anything.
Is it possible that master is having eslint issues?
On a freshly pulled master:
yarn run v1.19.2
$ tslint --project src/tsconfig.json && eslint ./test

ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/FilesRegister.ts:5:27 - Module 'eslint' is not listed as dependency in package.json
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/NormalizedMessageFactories.ts:6:25 - Module 'eslint' is not listed as dependency in package.json
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/index.ts:244:5 - Multiple variable declarations in the same statement are forbidden
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/index.ts:276:5 - Multiple variable declarations in the same statement are forbidden
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/index.ts:305:31 - Module 'eslint' is not listed as dependency in package.json

error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I've merged the settings with the patchTypescript config.
With that branch everything rebuilds within 500ms with one worker...with the current master it takes about >=10 Seconds with multiple workers.
@phryneas Can you maybe help me with the tests?

I've redone the changes on master and pushed everything to a new branch:
https://github.com/WolfspiritM/fork-ts-checker-webpack-plugin/tree/vue-fake-filesystem
I have some problems to get the tests working. Even on master the vue tests take ages to complete for me. And then fail with:
Timeout - Async callback was not invoked within the 60000ms timeout specified by jest.setTimeout.Error: Timeout - Async callback was not invoked within the 60000ms timeout specified by jest.setTimeout.
Not sure yet why that is the case. Same happens on my branch.

Hm. Back then, the vue integration tests were running at 150-200 seconds. Since then, they almost doubled with #355, and they take about 350 seconds on my system with your branch. So while that is a lot of time, it reflects what I would expect, considering those integration tests spawn a webpack instance for almost every test.

Also I had to remove the precommit hooks (locally).
Husky doesn't let me commit anything otherwise.

You can always commit by force with git commit --no-verify.

Is it possible that master is having eslint issues?
On a freshly pulled master:
yarn run v1.19.2
$ tslint --project src/tsconfig.json && eslint ./test

ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/FilesRegister.ts:5:27 - Module 'eslint' is not listed as dependency in package.json
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/NormalizedMessageFactories.ts:6:25 - Module 'eslint' is not listed as dependency in package.json
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/index.ts:244:5 - Multiple variable declarations in the same statement are forbidden
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/index.ts:276:5 - Multiple variable declarations in the same statement are forbidden
ERROR: C:/Work/fork-ts-checker-webpack-plugin2/src/index.ts:305:31 - Module 'eslint' is not listed as dependency in package.json

error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I can confirm that - the affected lines have last been touched in #305. @johnnyreilly, do you remember anything related from that PR?

I've merged the settings with the patchTypescript config.
With that branch everything rebuilds within 500ms with one worker...with the current master it takes about >=10 Seconds with multiple workers.
@phryneas Can you maybe help me with the tests?

I have no timeouts though. Maybe that's a Windows thing?

Good guess! It was a windows thing. Interestingly the tests don't seem to run on windows as webpack is having issues. adding some console.logs allowed me to catch many errors like that:
Module parse failed: Unexpected token (2:0)
    File was processed with these loaders:
     * ../../../../../node_modules/vue-loader/lib/index.js
    You may need an additional loader to handle the result of these loaders.
    |
    > <div class="hello">
    |   <h1>Example</h1>
    | </div>

However...that's another issue. I am now using WSL to test and it works fine. That way I was able to figure out that all that was left was adding corresponding __fake__.ts to the tsconfig includes. I pushed that to my branch and on my end all tests seem to pass now.
I'm wondering if it's okay to have the user specify the suffix in tsconfig (and add it to the readme) or if we need to figure out a way to rewrite the tsconfig aswell (including all "extend"s)

@johnnyreilly, do you remember anything related from that PR?

Could you be more specific? I'm not sure what the question is 😄
Side note: I mean to do this sometime soon: https://github.com/TypeStrong/fork-ts-checker-webpack-plugin/issues/356

@johnnyreilly, do you remember anything related from that PR?

Could you be more specific? I'm not sure what the question is smile
Side note: I mean to do this sometime soon: #356

These eslint warnings all seemed to be "last touched" by that PR. But, also,  taking a second look, currently eslint should only execute on the test' folder, but those errors are inside src` o_O
Anyways, I guess they'll be gone after #356, so lets ignore them for now.

I'm wondering if it's okay to have the user specify the suffix in tsconfig (and add it to the readme) or if we need to figure out a way to rewrite the tsconfig aswell (including all "extend"s)

If you can get that to work reliably, I think that'd be cool!

I will close it as I was able to release v5.0.0-alpha.1 version :)


Implement {f32, f64}::to_int_unchecked

While the diff generally looks okay at first glance, setting aside minor stylistic issues, how do we know what this behavior is definitively correct? I know the behavior will change to saturate in the future, but that's not an issue here as the compiler version is restricted. Is it known that this is the behavior on all compilers going back to 1.31? I presume it is, but it might be worth checking.

I think that's the case - it's a very long standing bug in a compiler. Also, saturating behaviour is valid implementation of this method, UB means the compiler can do anything including saturation.


SHAP for CNN regression model with different image shapes

I'm trying to use SHAP to make an interpretation of CNN regression model. The input images of model are with different shapes. The inconsistent shapes make it hard to find an appropriate background data. So is there any kind suggestion for the application of SHAP on CNN regression model with different input shapes?

@RicardoSun I don't understand the issue.

Could you provide working example that will be used for explanation?
Could you expand on "The inconsistent shapes make it hard to find an appropriate background data." using your example?

Thanks for your attention of this comment. I will try to explain the issue clearly.
As an example, suppose there are three kinds of images with size of 50x50, 100x100, and 200x200. I established a CNN model with global pooling layer to deal with these images and make a regression prediction. I want to use SHAP methd to interpretate the importance of the pixels in each image.
Now, I think this is unsuitable for the SHAP method, as the features numbers are differnet in these three images. The operation of global pooling also makes the unrecoverable loss of information. I wonder if the opinion is right or the requirement can be realized by SHAP method?
Thank your again for your kind reply.

@RicardoSun Now I understand more your concern.

It would be still great if you could prepare an example where it is not working.
You can always upscale or downscale the images to have one size just for the sake of shape evaluation.
Maybe heave a look at the notebooks https://github.com/shap/shap/blob/master/notebooks/image_examples/image_classification/Multi-class ResNet50 on ImageNet (TensorFlow).ipynb and https://github.com/shap/shap/blob/master/notebooks/image_examples/image_classification/Front Page DeepExplainer MNIST Example.ipynb to see how to use shap for CNNs.


Suppress bug using relative lines

Suppress bugs using comments that are not in the previous line.

I think it would be great to support the following scenario:
// suppress commend relative line +2
if ( very large condition
     that spans multiple lines
     and the error is suppressed here)
  ...


locale support

take day names and start day of the week from locale
make variables names human readable
use moment-range for building days range
fix bug with extra week (on screenshot)

is something wrong with this pr?

All things I was missing... Thanks!
lib and distare now git-ignored, so this should be an easy conflict resolution?

Could you clean this pr a little bit?

@alexeyMohnatkin looks like you made the changes in lib instead of src? I think lib and dist get built by webpack?

@nemoDreamer lib was build by webpack

I've moved it few rows below

is it ready to merge?

fixed to weekday

sorry for typo, I've just noticed


new package: sbase

Description of package
Checklist

[x] Latest upstream version.
[x] I agree to maintain this package (required).
[x] I agree to notify KISS if I can no longer maintain this package (required).

this is ready


Add a margin and accent color to the Android notification icon

generate-logos [nfc]: Expand inkscape --export-png for ≥ 1.0 compatibility.

generate-logos [nfc]: Add z-white.svg to the repository.

android assets: Pad notification icon for consistency with other apps.
Add a 10% margin on all sides, so the height of the Z comes out to an integer at all sizes we generate, which are multiples of 12px.

android notif: Set an accent color for notifications.
rgb(100, 146, 253.5) is the sRGB midpoint of our logo’s gradient.

Before:

After:

Before:

After:

Looks great, merged -- thanks @andersk!


Potential fix for overlong parenthesized content

Summary

Test Plan

Current dependencies on/for this PR:

main

PR #7490   👈

This comment was auto-generated by Graphite.


MQTT Issue with ARDUINO_TTGO_LoRa32_v21new

Context

Which example sketch are you using?
BresserWeatherSensorMQTT.ino
and tried also:
BresserWeatherSensorMQTTWiFiMgr.ino

Which changes to the code did you make (if any)?
Added "#define ARDUINO_TTGO_LoRa32_v21new" to WeatherSensorCfg.h @line 103

Also added WIFI info and MQTT infos @sketch line 233:
const char ssid[] = "myWIFI";
const char pass[] = "xyz";
#define HOSTNAME "ESPWeather"
#define APPEND_CHIP_ID
//#define MQTT_PORT 8883 //
#define MQTT_PORT 1883 // note: I first tried port 8883, changed then to 1883 as in MQTT at home assistant
const char MQTT_HOST[] = "<IP_ADDRESS>";
const char MQTT_USER[] = "mqtt-user"; // leave blank if no credentials used
const char MQTT_PASS[] = "mypassword"; // leave blank if no credentials used

Which sensor are you using?
TTGO Lora32 from this link:
https://www.lilygo.cc/products/lora3?variant=42272562282677

Describe the bug
I can compile the sketch successfully with following output:
In file included from C:\Users\x\Downloads\BresserWeatherSensorReceiver-0.28.8\examples\BresserWeatherSensorMQTT\BresserWeatherSensorMQTT.ino:203:
c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver'
  184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver")
      |                                                                                ^
c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'
  549 | #pragma message("Receiver chip: " RECEIVER_CHIP)
      |                                                ^
c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'
  550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )
      |                                                                                ^
Sketch uses 1028561 bytes (78%) of program storage space. Maximum is 1310720 bytes.
Global variables use 48576 bytes (16%) of dynamic memory, leaving 246336 bytes for local variables. Maximum is 294912 bytes.
esptool.py v4.6

To Reproduce
Steps to reproduce the behavior:

Now the device is connected to my WIFI, I can see it in my router and I can ping it.
Then I check my MQTT "Mosquitto broker" at home assistant and see that the device is connected
After a few seconds, the device is suddenly disconnected without sending any data and then reconned error

MQTT Log:
2024-07-13 17:07:22: New client connected from <IP_ADDRESS>:56871 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user').
2024-07-13 17:07:26: Client ESPWeather-A3E100 disconnected.
2024-07-13 17:11:13: New connection from <IP_ADDRESS>:50043 on port 1883.
2024-07-13 17:11:13: New client connected from <IP_ADDRESS>:50043 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user').
2024-07-13 17:11:17: Client ESPWeather-A3E100 disconnected.
2024-07-13 17:14:34: New connection from <IP_ADDRESS>:59625 on port 1883.
2024-07-13 17:14:34: New client connected from <IP_ADDRESS>:59625 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user').
2024-07-13 17:15:00: Client ESPWeather-A3E100 has exceeded timeout, disconnecting.
2024-07-13 17:19:44: New connection from <IP_ADDRESS>:52112 on port 1883.
2024-07-13 17:19:44: New client connected from <IP_ADDRESS>:52112 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user').
2024-07-13 17:19:59: Client ESPWeather-A3E100 disconnected.

Expected behavior
MQTT connection without timeout and receiving data of sensor
Debug Log
The TTGO Lora32 doesn't have a debugger interface
Embedded Device (please complete the following information):

Arduino IDE Version: [2.3.2]

ESP32/ESP8266 Board Support Package Version: [e.g. ESP32 2.0.7]

WiFiManager [2.0.17]

ArduinoJson [7.1.0]

Adafruit BusIO [1.16.1]

Adafruit GFX Library [1.11.9]

Adafruit SSD1306 [2.5.10]

BresserWeatherSensorReceiver [0.28.8]

ESP_DoubleResetDetector [1.3.2]

ESPSoftwareSerial [8.1.0]

MQTT [2.5.2]

Preferences [2.1.0]

RadioLib [6.6.0]

SdFat [2.2.3]

WifiManager [2.0.17]

Your Board:

[brand/model/version] (if unsure, attach a photo)

Selected Board in the Arduino IDE: [TTGO loRa32-OLED]

Hi,
I reformatted your output a little - three backticks to enclose a block of verbatim text.

You don't have to define #define ARDUINO_TTGO_LoRa32_v21new manually. As described here https://github.com/matthias-bs/BresserWeatherSensorReceiver?tab=readme-ov-file#predefined-board-configurations, you just have to select the correct board (if provided)
Your compile output shows #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver") while you selected #define ARDUINO_TTGO_LoRa32_v21new (which matches your photo), so you selected the wrong board variant.
Regarding debug output, please see https://github.com/matthias-bs/BresserWeatherSensorReceiver?tab=readme-ov-file#debug-output-configuration

Due do the wrong board selection (different pinning between ESP32 and radio chip), the configuration of the radio chip will fail, which leads to an endless loop. That's why you encounter an MQTT timeout.

HI matthias,
thanks for your response, ok makes sense when selected wrong board = timeout MQTT.
Ok, so reverted back to your release (undoing all my changes) = original sketch "BresserWeatherSensorMQTT.ino" from your examples.
Then changed the code (removed the # from line 142 @WeatherSensorCfg.h":
// This define is set by selecting "Board: TTGO LoRa32-OLED" / "Board Revision: TTGO LoRa32 V2.1 (1.6.1)" // in the Arduino IDE: #define ARDUINO_TTGO_LoRa32_V21new
And selected (as before) the board "TTGO LoRa32-OLED" in Arduino IDE (as also described in your comment in the code):

So, in my opinion the selected board should be correct?!
In file included from C:\Users\x\Downloads\aaa\BresserWeatherSensorReceiver-0.28.8\examples\BresserWeatherSensorMQTT\BresserWeatherSensorMQTT.ino:203: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver'   184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver")       |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'   549 | #pragma message("Receiver chip: " RECEIVER_CHIP)       |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'   550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )       |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\Lightning.h:72,                  from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\Lightning.cpp:75: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver'   184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver")       |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'   549 | #pragma message("Receiver chip: " RECEIVER_CHIP)       |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'   550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )       |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\RainGauge.cpp:67: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver'   184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver")       |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'   549 | #pragma message("Receiver chip: " RECEIVER_CHIP)       |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'   550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )       |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorDecoders.cpp:58: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver'   184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver")       |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'   549 | #pragma message("Receiver chip: " RECEIVER_CHIP)       |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'   550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )       |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorConfig.cpp:50: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver'   184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver")       |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'   549 | #pragma message("Receiver chip: " RECEIVER_CHIP)       |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'   550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )       |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensor.cpp:110: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver'   184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver")       |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'   549 | #pragma message("Receiver chip: " RECEIVER_CHIP)       |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'   550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )

Please look for the menu entry 'Board variant' and select the appropriate one!Am 14.07.2024 08:51 schrieb m0rph3usX @.***>:
HI matthias,
thanks for your response, ok makes sense when selected wrong board = timeout MQTT.
Ok, so reverted back to your release (undoing all my changes) = original sketch "BresserWeatherSensorMQTT.ino" from your examples.
Then changed the code (removed the # from line 142 @WeatherSensorCfg.h":
// This define is set by selecting "Board: TTGO LoRa32-OLED" / "Board Revision: TTGO LoRa32 V2.1 (1.6.1)" // in the Arduino IDE: #define ARDUINO_TTGO_LoRa32_V21new
And selected (as before) the board "TTGO LoRa32-OLED" in Arduino IDE (as also described in your comment in the code):
grafik.png (view on web)
So, in my opinion the selected board should be correct?!
In file included from C:\Users\x\Downloads\aaa\BresserWeatherSensorReceiver-0.28.8\examples\BresserWeatherSensorMQTT\BresserWeatherSensorMQTT.ino:203: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver' 184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver") |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]' 549 | #pragma message("Receiver chip: " RECEIVER_CHIP) |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33' 550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) ) |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\Lightning.h:72, from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\Lightning.cpp:75: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver' 184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver") |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]' 549 | #pragma message("Receiver chip: " RECEIVER_CHIP) |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33' 550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) ) |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\RainGauge.cpp:67: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver' 184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver") |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]' 549 | #pragma message("Receiver chip: " RECEIVER_CHIP) |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33' 550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) ) |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorDecoders.cpp:58: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver' 184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver") |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]' 549 | #pragma message("Receiver chip: " RECEIVER_CHIP) |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33' 550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) ) |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorConfig.cpp:50: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver' 184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver") |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]' 549 | #pragma message("Receiver chip: " RECEIVER_CHIP) |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33' 550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) ) |                                                                                ^ In file included from c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensor.cpp:110: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:184:81: note: '#pragma message: ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver' 184 |     #pragma message("ARDUINO_TTGO_LoRa32_V1 defined; using on-board transceiver") |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]' 549 | #pragma message("Receiver chip: " RECEIVER_CHIP) |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src\WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->14, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33' 550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

Oh man, thanks for this hint!
Didn't know that Arduino IDE has a submenu for selecting the "Board Revision".
Changed the revision depending my board "TTGO lora32 V.2.1 (1.6.1):

Still getting the pragma message when compiling:
In file included from C:\Users\x\Downloads\BresserWeatherSensorReceiver\examples\BresserWeatherSensorMQTTWifiMgr\BresserWeatherSensorMQTTWifiMgr.ino:225: c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:193:85: note: '#pragma message: ARDUINO_TTGO_LoRa32_V21new defined; using on-board transceiver'   193 |     #pragma message("ARDUINO_TTGO_LoRa32_V21new defined; using on-board transceiver")       |                                                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:549:48: note: '#pragma message: Receiver chip: [SX1276]'   549 | #pragma message("Receiver chip: " RECEIVER_CHIP)       |                                                ^ c:\Users\x\Documents\Arduino\libraries\BresserWeatherSensorReceiver\src/WeatherSensorCfg.h:550:170: note: '#pragma message: Pin config: RST->23, CS->18, GD0/G0/IRQ->26, GDO2/G1/GPIO->33'   550 | #pragma message("Pin config: RST->" STR(PIN_RECEIVER_RST) ", CS->" STR(PIN_RECEIVER_CS) ", GD0/G0/IRQ->" STR(PIN_RECEIVER_IRQ) ", GDO2/G1/GPIO->" STR(PIN_RECEIVER_GPIO) )       |

Yes, this time the pragma message shows that everything is fine.

Hm ok thanks for your help.
So I uploaded the project and the issue remains still the same, MQTT timeout at home assistant:
2024-07-14 13:06:38: Client ESPWeather-A3E100 has exceeded timeout, disconnecting. 2024-07-14 13:11:27: New connection from <IP_ADDRESS>:52673 on port 1883. 2024-07-14 13:11:27: New client connected from <IP_ADDRESS>:52673 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user'). 2024-07-14 13:11:39: Client ESPWeather-A3E100 disconnected. 2024-07-14 13:16:36: New connection from <IP_ADDRESS>:56551 on port 1883. 2024-07-14 13:16:36: New client connected from <IP_ADDRESS>:56551 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user'). 2024-07-14 13:16:47: Client ESPWeather-A3E100 disconnected. 2024-07-14 13:22:37: New connection from <IP_ADDRESS>:57449 on port 1883. 2024-07-14 13:22:37: New client connected from <IP_ADDRESS>:57449 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user'). 2024-07-14 13:22:56: Client ESPWeather-A3E100 has exceeded timeout, disconnecting. 2024-07-14 13:27:31: New connection from <IP_ADDRESS>:58567 on port 1883. 2024-07-14 13:27:31: New client connected from <IP_ADDRESS>:58567 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user'). 2024-07-14 13:27:37: Client ESPWeather-A3E100 disconnected. 2024-07-14 13:32:35: New connection from <IP_ADDRESS>:54564 on port 1883. 2024-07-14 13:32:35: New client connected from <IP_ADDRESS>:54564 as ESPWeather-A3E100 (p2, c1, k10, u'mqtt-user'). 2024-07-14 13:32:37: Client ESPWeather-A3E100 disconnected.
I also enabled
#define _DEBUG_MQTT_
to try to get some dummy data.... but still getting MQTT timeout.
Hm I'll try to modify the code, to get the debug infos via OLED instead of non existing debugging interface of this bord...
BTW it is a Bresser WSX3001.

Would you please kindly follow my advice and read the following regarding debug output via USB?
https://github.com/matthias-bs/BresserWeatherSensorReceiver?tab=readme-ov-file#debug-output-configuration

Ok thanks got it. I already enabled the debug messages, but the "serial monitor" was hidden, instead "gdb-server1" tab was showing with an error... So I thought debugging wouldn't be available.
Here's the log, which seems to get valid data from the sensor, and the timeout @ MQTT is because of the sleep of 300000ms, but that would be ok. I thought the log of MQTT (mosquitto broker) would also show the received data...
`[ 18833][D][WeatherSensorDecoders.cpp:104] findSlot(): sensor[0]: v=0 id=0x00000000 t=0 c=0
[ 18851][D][RainGauge.cpp:194] prefs_load(): lastUpdate        =1720958307
[ 18858][D][RainGauge.cpp:195] prefs_load(): startupPrev       =1
[ 18864][D][RainGauge.cpp:196] prefs_load(): rainPreStartup    =0.000000
[ 18870][D][RainGauge.cpp:197] prefs_load(): tsDayBegin        =0
[ 18876][D][RainGauge.cpp:198] prefs_load(): rainDayBegin      =2023.300049
[ 18883][D][RainGauge.cpp:199] prefs_load(): tsWeekBegin       =6
[ 18889][D][RainGauge.cpp:200] prefs_load(): rainWeekBegin     =144.500000
[ 18896][D][RainGauge.cpp:201] prefs_load(): wdayPrev          =0
[ 18902][D][RainGauge.cpp:202] prefs_load(): tsMonthBegin      =6
[ 18908][D][RainGauge.cpp:203] prefs_load(): rainMonthBegin    =144.500000
[ 18914][D][RainGauge.cpp:204] prefs_load(): rainPrev          =2023.300049
[ 18921][D][RainGauge.cpp:205] prefs_load(): rainAcc           =2023.300049
[ 18928][D][RainGauge.cpp:280] update(): rainDelta: 0.0
[ 18933][D][RainGauge.cpp:325] update(): t_delta:<PHONE_NUMBER>
[ 18939][D][RainGauge.cpp:349] update(): hist[0]=0 (new)
[ 18944][D][RainGauge.cpp:383] update(): hist[]={0, 0, -1, 0, -1, -1, -1, -1, 0, 0, }
[ 18964][I][BresserWeatherSensorMQTTWifiMgr.ino:899] publishWeatherdata(): ESPWeather-A3E100/abbb/data: {"id":43963,"ch":0,"battery_ok":1,"temp_c":24.3,"humidity":46,"wind_gust":2.8,"wind_avg":2.5,"wind_dir":252.0,"uv":10.5,"light_klx":117.7,"rain":0.0,"rain_h":0.0,"rain_d":0.0,"rain_w":1878.8,"rain_m":1878.8}
[ 18997][I][BresserWeatherSensorMQTTWifiMgr.ino:911] publishWeatherdata(): ESPWeather-A3E100/extra: {"wind_dir_txt":"WSW","wind_gust_bft":2,"wind_avg_bft":2,"dewpoint_c":11.9,"perceived_temp_c":25.5}
[ 19016][D][BresserWeatherSensorMQTTWifiMgr.ino:1097] loop(): Data forwarding completed.
[ 19024][I][BresserWeatherSensorMQTTWifiMgr.ino:1100] loop(): Sleeping for 300000 ms
[ 19032][I][BresserWeatherSensorMQTTWifiMgr.ino:1101] loop(): ESPWeather-A3E100/status: offline
`


HDDS-10057. [Snapshot] 'ozone fs -ls' on '.snapshot' dir of a bucket should list only active snapshots

What changes were proposed in this pull request?
Currently when we do ozone fs -ls on .snapshot hidden dir of a bucket, it throws an error message if bucket contains deleted snapshot which are not be garbage collected as mentioned in HDDS-10057.
In this change, snapshot active check is added to make sure it lists only active snapshots and their content.
What is the link to the Apache JIRA
HDDS-10057
How was this patch tested?
Updated the unit test and tested it out on local cluster by increasing the run time of SnapshotDeletingService.

Thanks @xBis7 and @sadanand48 for the review.


Generate executable from flow

I would like to check if there is any way to export/generate/produce an executable file, or even a runnable *.py file from the flow that I've created. I'm wondering if there's any logical way to go about this as I would like to obtain an executable from the flows I've made in Ryven. Thanks!

@Berttwm so you are looking for a way to turn a python script generated from a flow (see #68) into an executable from python itself, i.e. from another python script. There are probably various ways to accomplish that in principal (depends a bit on your system constraints etc.). Finding a suitable way to do that does not depend on the usage of Ryven, it's a typical scripting task.
Once you have a nice way of generating your source code and turning it into an executable, you can simply put this program into a node with a button and an action to trigger it, or something like that. In case your flows are trees (so they have a root, see Unreal Engine's material editor for example), you can directly put this functionality into the root node.


Documentation: identity vs id property

https://github.com/interagent/prmd/blob/master/docs/schemata.md#definitions states that the definition section must contain the definition of an identity property. The following example then defines an id property. This seems like an inconsistency in the document.

Good catch, should now be here: https://github.com/interagent/prmd/commit/56ee715f2301fff0edebfcd2a353d1e7fddf1964


jupiter notebook must be extended to work with unaries

So far, the show-plots-GUI.ipynb detects all the json files with "results-" string in the folder and makes them available for comparison.
However, this script only works for oxides. The results is that errors pops up when a "result-...json" containing unaries set is present in the folder!

Partially resolved in 09c1785.
Now two different notebook are present, one for oxides, one for unaries. The selection of the "results-<>.json" to include in the comparison is based on the detection of the "unaries" string in the file name. Maybe fragile?
Also did not test the export capabilities.

Fixed in #43


lv2 + sratom + lilv: support cross-compilation

Motivation for this change
Support cross-compilation for these packages.
Things done

Built on platform(s)

[x] x86_64-linux (output tree is equal to old tree)
[x] aarch64-linux (cross-compiled)
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Maybe we should think about disabling waf cross flags by default if they are not that useful.

Yeah, thought about that already and I think it'd be a good idea. It'd require some documentation updates and testing of the packages that do require it (they exist). Depending on how many packages need it it may even be more logical to put them in the derivations itself.
It's also unfortunate that the flags introduce qemu to all packages that use it, which can cause a native rebuild of qemu before the package actually builds.

Will look into converting from dontUseWafCross flags, but I'll have to see where that goes.


[ISSUE-780][REFACTOR] Refactor PartitionLocation's methods

What changes were proposed in this pull request?
Refactor PartitionLocation's methods, details and reasons could be found in the #780
toString output before reformatting

toString output after reformatting

Current partition location
Why are the changes needed?
What are the items that need reviewer attention?
Related issues.
#780
Related pull requests.
How was this patch tested?
/cc @AngersZhuuuu
/assign @main-reviewer

Do we need to let more field get involved with overriding equals() and hashCode()?

ping @AngersZhuuuu @FMX

ping @AngersZhuuuu @FMX

Pls resolve conflicts first

ping @AngersZhuuuu @FMX

Pls resolve conflicts first

Done

I think the PbSerdeUtils can be merged into Utils because they have the same functionality.
But considering the purpose of this pull request, I think that can be done through some other pull request.
@AngersZhuuuu @waitinfuture


JOSS review: outdated links in "about" modal

Hi, I am one of the reviewer for the JOSS paper for materialnet. I'll open a few issues as I find surprising things while playing with material net.
On the live demo (https://toyotaresearchinstitute.github.io/materialnet), in the "about" modal, links points to an outdated URL for this repository: https://github.com/arclamp/materialnet/. Github redirect this to https://github.com/ToyotaResearchInstitute/materialnet/, so I guess the repository has been transferred at some point without updating the URLS.

Somehow related, the package.json homepage is set to https://toyotaresearchinstitute.github.io/materialnet, but the JOSS paper refers to https://maps.matr.io. Is any of these the main URL for a live demo/homepage?
If so, the corresponding URL could be added in the readme or the github respository description.

@Luthaf, thank you for bringing these issues up. I have put in a fix for this issue, but it won't be visible on maps.matr.io until Monday, when our devops team has a chance to deploy it.
At that time, if you feel this issue has not been solved, please re-open it.


Include $input-placeholder-font-color variable

The $input-placeholder-font-color: #cccccc !default; is defined in the forms.scss on line 26 and was forgotten in the settings.scss

Thanks! We've removed all the hard coded colors to be variables. This one is $iron but I think we have too many anyways. We'll leave it as is.


"Load From Clipboard" doesn't support UTF-8 encoding

First, thanks for the great effort to make such an amazing Go editor tool.
In Mac OS X, if I copied a SGF, and loaded in Sabaki with "Load From Clipboard", the UTF-8 characters messed up.

I can't reproduce this issue. I tried copying the following text:
(;PB[古力]PW[柯洁]
;B[pd]
;W[dp]C[围棋 is fun])

Loading from clipboard works just fine in Sabaki.

Mac OS 10.12
Sabaki: 1.18.0
Here goes the SGF file I copied to clipboard, and then load from Sabaki:
(;GM[1]FF[4]AP[Sabaki:0.18.0]CA[UTF-8]KM[6.5]SZ[19]AB[cd]C[如何挂角？]PL[W]
(;W[ec]
)(;W[ed]
)(;W[fd]
)(;W[fc]
))
Attached is the screenshot after I loaded.

Please note the text in the comment area (right bottom corner).

Hmm... it seems like if I remove the CA property, it works fine. Which means this can be fixed if we ignore the encoding specified in CA when loading from clipboard.


Error: Packets out of order. Got: 0 Expected: 2

Once in a while I'm seeing this error thrown:
at Parser._trackAndVerifyPacketNumber (node_modules/mysql/lib/protocol/Parser.js:322:15)
at Parser.write (node_modules/mysql/lib/protocol/Parser.js:40:12)
at Protocol.write (node_modules/mysql/lib/protocol/Protocol.js:37:16)
at Socket.<anonymous> (node_modules/mysql/lib/Connection.js:75:28)
at Socket.EventEmitter.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:159:16)
at Socket.Readable.push (_stream_readable.js:126:10)
at TCP.onread (net.js:514:20)

I'm running MySQL 5.5.18 and node v0.11.12. I'm using connection pools.
Oddly, the error is thrown long after my app finishes querying data and responds to the request.

Is that a problem I should be worried about? Incompatibility or maybe just a network error?
Is it possible to catch that error and handle it gracefully? (other than via a domain)

I have still this issue. I try to use a prepared statement to insert data in to a simple table.
The problem seems to have something todo with the string length of one of the fields.
So the everything works fine, but when I try to store a string with the length of 89,764,572 into the Longtext colum I get the error:

Packets out of order. Got: 2 Expected: 7


add items to troubleshooting section

1. student is installing the requirements, but there is no requirements file

2. student is pulling the ds-prep-course repo, but the pull fails because of uncommitted local changes

this needs to go here as well

number 1 above is solved by number 2 so i didn't include it

what should students do when they encounter this situation?

Is this resolved? @buedaswag  If yes please close, if not please create actionable.


Unable to add Conversion configuration without error

I have created rename_column configurations for the following:

BillingCurrencyCode to BillingCurrency
ConsumedService to ServiceName
SubscriptionId to SubAccountId
SubscriptionName to SubAccountName

I am unwilling to submit them in a PR because as-is they don't pass the following test.
Running the following command:
python -m focus_converter.main explain --provider azure
Yields the following error:
ValidationError: 1 validation error for ConversionPlan focus_column   Input should be 'ChargePeriodStart', 'ChargePeriodEnd', 'BillingPeriodStart', 'BillingPeriodEnd', 'Provider', 'Publisher', 'BillingAccountId', 'AmortisedCost', 'BilledCost', 'Region', 'ChargeType' or 'ServiceCategory' [type=enum, input_value='BillingCurrency', input_type=str]
This error also reveals other issues, the following two fields aren't even associated with the FOCUS Spec.

AmortisedCost
BilledCost

Recreated PR and merged #132.


[FLINK-16879] [build] Disable the source-release-assembly execution goal

We should be using the apache-release build profile defined in the Apache Parent POM, with one exclusion: do not run the source-release-assembly execution goal.
We have our own scripts to create the source release, which correctly excludes binaries from distribution tarball. That script can be found under tools/releasing/create_source_release.sh.

Verifying
Build the project using mvn clean install -Papache-release.
Under project-root/target/, you should not find any source release packages, only POM files.

cc @rmetzger
This fixes the problem you found with RC4 of 2.0.0.
This does not require a cancellation of the current ongoing RC, because the artifacts in the new Maven repository for RC4 is still built with the same commit hash.

Thanks a lot for proposing this fix.
I guess we don't have this problem in apache/flink because we are not using the "apache-release" profile for building the releases there?

@rmetzger correct, Flink defines its own release profile, but as far as I can tell, it uses essentially the same plugins used as the apache-release profile, minus the source-release-assembly execution.

Thanks for the review, merging this ...


Implement plain text starting with < in Diet templates.

This is the Jade way to emit conditional comments but works
with all plain text starting with <, too.
See http://jade-lang.com/reference/comments/.

Hm, interesting, did they remove the //if syntax? I'm pretty sure that it came from Jade when I implemented it. Anyway, supporting this shouldn't hurt! Can you add a short unit test around line 1455 to ensure that this stays functional?

Sure!

According to the Jade changelog, version 1.0.0 removed support for conditional comments.

Okay, thanks! Definitely makes sense to try and stay compatible with the 1.0.0 spec.


i added 2 config options to disable the buildin protections

i want to have island towns, managed by towny, that the reason i added it.
you have to edit the version number yourself, i didnt want to mess with that

Thanks for your pull request! 😸


ObserverManager.UnSubscribe has only observers id as argument.

Summary
Fixed parameterns in code sample for Grains Observers
Fixes #Issue_Number (if available)
No Issue available

Internal previews

📄 File
🔗 Preview link

docs/orleans/grains/observers.md
Observers

@dotnet-policy-service agree

Hi @dahermansson - I'm not seeing an overload with two parameters. Have a look here at the official .NET API for the OverserverManager<TIdentity, TObserver>.Unsubscribe. It only defines an id parameter.

Hi @dahermansson - I'm not seeing an overload with two parameters. Have a look here at the official .NET API for the OverserverManager<TIdentity, TObserver>.Unsubscribe. It only defines an id parameter.
Let's ask @ReubenBond to be sure.

The method only takes one parameter. It’s the code example in the documentation that’s show it’s with two parameters. This PR change the example in the documentation.


parsing failure

this test case is fail.
https://github.com/otf/Http-Multipart-Data-Parser/commit/158df5f49daad5050885709cbf68a94d175721e0#diff-f73660006b84fe1ba8d9d28a80166b1fR46

Closed by merging https://github.com/Vodurden/Http-Multipart-Data-Parser/pull/20
Thanks again!