python312Packages.opower: 0.4.5 -> 0.4.6, python312Packages.aiozoneinfo: init at 0.1.0

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 315879 run on x86_64-linux 1

9 packages built:

home-assistant-component-tests.opower
python311Packages.aiozoneinfo
python311Packages.aiozoneinfo.dist
python311Packages.opower
python311Packages.opower.dist
python312Packages.aiozoneinfo
python312Packages.aiozoneinfo.dist
python312Packages.opower
python312Packages.opower.dist


Update droplet to avoid deprecated initializer

Vapor now encourages Droplets to be created using an empty initializer. Middleware and preparations should be supplied after a Droplet has been created.

Tested, works well. Thanks @dan-zheng!
I also noticed there were a few things deprecated here:
Edwards-Work-MacBook:VaporAuth edjiang$ .build/debug/App
[DEPRECATED] Key `server` in `middleware.json` will be removed in a future update. Use key `middleware.server` in `droplet.json`.
[DEPRECATED] Key `server` in `middleware.json` will be removed in a future update. Use key `middleware.server` in `droplet.json`.
[DEPRECATED] Key `server` in `middleware.json` will be removed in a future update. Use key `middleware.server` in `droplet.json`.

Would love to see you submit in another PR =]

I noticed those deprecations. I'll make another PR shortly! 👍


Added search topology resources

Minor bug fixes also included as well as the set up for the documentation generation

@caadam - Another big code review bloke. I'm still testing the search resources on my multi-server setups to confirm they play nice there (they are solid on single server labs) but I thought I would get the PR going and tweak it as I test things. I'll give you a heads up when anything changes.

@camiloborges Fixed the following based on your feedback

Now both search roles and search index use SSA name to look up the SSA
There is a loop count to hang off the starting of the search service app. Hard coded to 10 minutes
Fixed the type of index in the log message

cool. I will try to run few scenarios in a multi server farm. I should have
a reasonably up to date farm in the office.
On Sun, Nov 22, 2015 at 8:41 PM, Brian Farnhill<EMAIL_ADDRESS>wrote:

@camiloborges https://github.com/camiloborges Fixed the following based
on your feedback

Now both search roles and search index use SSA name to look up the
SSA
There is a loop count to hang off the starting of the search service
app. Hard coded to 10 minutes
Fixed the type of index in the log message

—
Reply to this email directly or view it on GitHub
https://github.com/PowerShell/xSharePoint/pull/97#issuecomment-158740303
.

@camiloborges  I'm seeing an issue at the moment where the folder for the search indexes isn't being created on separate remote servers. Say I have sharepoint1 running the search roles and search index partition resources, telling them to create index partitions 0 and 1 on sharepoint3 and sharepoint4. I'll see that SharePoint3 gets a folder created for the indexes locally, but for some reason sharepoint 4 won't. That's the last bug I've found with it, the rest of the topology components are created fine.

@camiloborges @caadam Hey guys, don't bother reviewing the search topology stuff, I'm going to pull it out and push it to the next release. Turns out there was a couple of fatal flaws in my approach so I'm going back to the drawing board on this one. I'll remove the files from this PR later tonight and get the rest of the changes pushed through though


Mutiks&Positioning

Currently the script only works well in the following combination:

positioned left, minimized Layout

Other options also need to work:

positioned left/right
minimized Layout yes/no

since it's pretty simple, maybe it's interesting for you, @Ardenian ? would just be a bit of css-adjusting

would appreciate it if you had those 5 mins to spare, @Ardenian - maybe 10 if you make it beautiful :D


fix message for stale verification link

Closes #102
@keileyo - let me know if you'd like to change this message.

:+1: on the message. Holding until @hkeeler tests the redirect from a stale link.

Looks good.  Merging.


Code Error with latest commit/change

I was noticing that you use + a lot and i knew it wasnt the best solution... unless you have a very specific reason to. Do you mind sharing with me your thoughts on it?

Welp, didn't do a lot with gradle im the past, didn't know it was bad to
use those, never really ran into any issues.
Just thought it was good practice to keep the dependencies always updated
because if players update both and it breaks I can fix it easier in dev.
Anyways, thanks for telling me that, I will look into it.
jmac460<EMAIL_ADDRESS>schrieb am Mo., 19. Feb. 2018, 05:56:

[image: image]
https://user-images.githubusercontent.com/34420502/36362880-f7026234-14fd-11e8-98df-5e983a3d980b.png
I was noticing that you use + a lot and i knew it wasnt the best
solution... unless you have a very specific reason to. Do you mind sharing
with me your thoughts on it?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/BloodWorkXGaming/ExNihiloCreatio/issues/61, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHwpZsH87COOsIOzB5CSPBLWzedhppY3ks5tWP7fgaJpZM4SKDD-
.

Tried my hand at cleaning up the gradle on a new branch using a setup I've seen several other mods use of putting the dependency version numbers in a separate easier to maintain file.

Well, I am pretty sure it doesn't matter and in my opinion it is actually
worse to use a fixed version.
Like do you want to check every maven every day just to see if there are
updates?
SirLyle<EMAIL_ADDRESS>schrieb am Sa., 24. Feb. 2018, 04:14:

Tried my hand at cleaning up the gradle on a new branch
https://github.com/BloodWorkXGaming/ExNihiloCreatio/tree/Gradle using a
setup I've seen several other mods use of putting the dependency version
numbers in a separate easier to maintain file.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/BloodWorkXGaming/ExNihiloCreatio/issues/61#issuecomment-368194408,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHwpZpb0hEgyqNQdJgR_5cHqlgyOKR-Oks5tX36LgaJpZM4SKDD-
.

I partly changed this now, not sure whether it actually is better tho ^^


crawling different domains too if set custom discoverResources function

the crawler crawls different domains too if set custom discoverResouces function
is it bug or intended?
how to prevent not to crawls different domains, only crawls which one provided?
crawler.discoverResources = function(buffer, queueItem) {
    var $ = cheerio.load(buffer.toString("utf8"));

    return $("a[href]").map(function () {
        return $(this).attr("href");
    }).get();
};

the cralwer's options
crawler.maxDepth            = 4;
crawler.downloadUnsupported = false;
crawler.scanSubdomains      = true;
crawler.supportedMimeTypes  = [/^text\/html/i];
crawler.parseHTMLComments   = false;
crawler.parseScriptTags     = false;

any ideas?

Hi — we're changing the API so this won't be forever, but you'll need to call crawler.cleanExpandResources on the returned list of URLs.

thanks,
and so the result will be
crawler.discoverResources = function(buffer, queueItem) {
    var $ = cheerio.load(buffer.toString("utf8"));

    var urls = $("a[href]").map(function () {
        return $(this).attr("href");
    }).get();

   return crawler.cleanExpandResources(urls, queueItem);
};

right?

That's right.

thanks

@cgiffard i tried, but still crawling another domains

Hmm, what domain are you crawling initially and what additional domains does the crawler go to?

e.g.
http://stpaulanimalshelter.com/
debug: starting request of the url
{ url: 'http://stpaulanimalshelter.com/' }
debug: starting request of the url
{ url: 'https://www.hugedomains.com/domain_profile.cfm?d=stpaulanimalshelter&e=com' }
debug: received content of the url
{ url: 'https://www.hugedomains.com/domain_profile.cfm?d=stpaulanimalshelter&e=com' }
debug: starting request of the url
{ url: 'https://www.hugedomains.com/index.cfm' }
debug: starting request of the url
{ url: 'https://www.hugedomains.com/payment-plans.cfm' }
debug: starting request of the url
{ url: 'http://www.impresseo.com/' }
debug: starting request of the url
{ url: 'http://www.walllife.com/' }
debug: starting request of the url
{ url: 'http://www.floridastraits.com/' }
debug: starting request of the url
{ url: 'https://www.hugedomains.com/testimonials.cfm' }
debug: received content of the url
{ url: 'http://www.walllife.com/' }
debug: starting request of the url
{ url: 'https://www.hugedomains.com/shopping_cart.cfm?d=StPaulAnimalShelter&e=com' }
debug: received content of the url
{ url: 'https://www.hugedomains.com/index.cfm' }
debug: starting request of the url
{ url: 'https://www.hugedomains.com/faq.cfm' }
debug: received content of the url
{ url: 'https://www.hugedomains.com/payment-plans.cfm' }
debug: starting request of the url
{ url: 'https://www.hugedomains.com/domain_search.cfm?domain_name=St+Paul+Animal+Shelter' }
debug: received content of the url
{ url: 'http://www.floridastraits.com/' }

any ideas?

Hmm, I've poked around a bit, but I'm unsure. What happens if you remove the scanSubdomains setting?

the code
https://gist.github.com/hbakhtiyor/3d81722f1394c507016486d50c983839
the result
info: starting crawling for the url
{ url: 'http://aclmb.ca' }
error: fetch client error during crawling
{ queueItem:
   { url: 'http:///',
     protocol: 'http',
     host: '',
     port: 80,
     path: '/',
     depth: 1,
     fetched: true,
     status: 'failed',
     stateData: { code: 599 } },
  errorData:
   { code: 'ECONNREFUSED',
     errno: 'ECONNREFUSED',
     syscall: 'connect',
     address: '<IP_ADDRESS>',
     port: 80 } }
info: finished all requests from queue urls
info: 26312 total/1 websites are already crawled
info: starting crawling for the url
{ url: 'http://animaloutreach.ca' }
error: fetch client error during crawling
{ queueItem:
   { url: 'http:///',
     protocol: 'http',
     host: '',
     port: 80,
     path: '/',
     depth: 1,
     fetched: true,
     status: 'failed',
     stateData: { code: 599 } },
  errorData:
   { code: 'ECONNREFUSED',
     errno: 'ECONNREFUSED',
     syscall: 'connect',
     address: '<IP_ADDRESS>',
     port: 80 } }
info: finished all requests from queue urls
info: 26312 total/2 websites are already crawled
info: starting crawling for the url
{ url: 'http://www.accesemployment.ca' }
error: fetch client error during crawling
{ queueItem:
   { url: 'http:///',
     protocol: 'http',
     host: '',
     port: 80,
     path: '/',
     depth: 1,
     fetched: true,
     status: 'failed',
     stateData: { code: 599 } },
  errorData:
   { code: 'ECONNREFUSED',
     errno: 'ECONNREFUSED',
     syscall: 'connect',
     address: '<IP_ADDRESS>',
     port: 80 } }
info: finished all requests from queue urls
info: 26312 total/3 websites are already crawled
info: starting crawling for the url
{ url: 'http://www.capesociety.ca' }
error: fetch client error during crawling
{ queueItem:
   { url: 'http:///',
     protocol: 'http',
     host: '',
     port: 80,
     path: '/',
     depth: 1,
     fetched: true,
     status: 'failed',
     stateData: { code: 599 } },
  errorData:
   { code: 'ECONNREFUSED',
     errno: 'ECONNREFUSED',
     syscall: 'connect',
     address: '<IP_ADDRESS>',
     port: 80 } }
info: finished all requests from queue urls
info: 26312 total/4 websites are already crawled
info: starting crawling for the url
{ url: 'http://www.communitylivingwinnipeg.com' }
error: fetch client error during crawling
{ queueItem:
   { url: 'http:///',
     protocol: 'http',
     host: '',
     port: 80,
     path: '/',
     depth: 1,
     fetched: true,
     status: 'failed',
     stateData: { code: 599 } },
  errorData:
   { code: 'ECONNREFUSED',
     errno: 'ECONNREFUSED',
     syscall: 'connect',
     address: '<IP_ADDRESS>',
     port: 80 } }
info: finished all requests from queue urls
info: 26312 total/5 websites are already crawled

Something looks off there — I wonder whether the URL information in the queueItem being passed in isn't complete. Can you post a complete test case?

what do you mean?
i posted complete

crawler.queueURL(website);
i think here

Ah, my apologies — I completely missed the gist you posted. I'll have a look this evening. :)

any result? ))

@cgiffard did you see what was the wrong?

@cgiffard did you see what was the wrong?

@hbakhtiyor I converted your code from TS to plain JS manually (you can see the result here), and when testing with the latest master on http://aclmb.ca, the crawler ran to completion without any problems (and without starting to crawl other domains).
Could you give the latest master a try as well and see if your problem persists?

Did this issue persist, or were you able to solve it, @hbakhtiyor?

Closing this for now. Let us know if you have any further issues, @hbakhtiyor


Moving settings editor to a new editor group removes the filter.

Issue Type: Bug
After moving the settings editor using drag and drop, the filter will be removed. Bringing back the window to its original group will restore the filter. It happens with all extensions disabled and in the latest version of VS Code Insiders.


Added description for RescueModifier cop explaining reasons for not to use modifier form rescue

fixes #7289
Adds a description to the cop specifying the reason for not using modifier form rescue. The reasons are similar to the ones mentioned in the https://thoughtbot.com/blog/don-t-inline-rescue-in-ruby blog.

Thanks!
Btw, it'd be nice to improve the matching rule in the style guide as well.


Migrate correctness tests to conformance test suites (#2914)

[x] resilience tests
[x] autoscale tests
[x] topoloogy tests
[x] cli tests
[x] circleci test jobs

closes #2914

Things to look for in the review:

that circleci jobs run as before
That there is a new cli tests circleci job (one with and another without resilience)

Otherwise, take a look at the test definitions in testing/conformance/tests/* and feel free to provide feedback.

Generally this looks good.  There appear to be fewer resilience tests running now.  Was that intentional?

Generally this looks good. There appear to be fewer resilience tests running now. Was that intentional?

That's due to the the removal of gensource from the resilience tests


[C++ Comm] Remove a test that binds to IPADDR_ANY.

Commit message has a typo: "IPADDR_ANY"->"INADDR_ANY"
Are we losing something by just removing the test?
Also, what about this comment in cpp/inc/bond/comm/address.h?
// Create socket address, with INADDR_ANY as an ip address and explicit port:
Seems like this might go a bit deeper.

Yeah, upon deeper inspection, uses of the default ctor may need to be purged from the tests as well.

Fixed the typo. Thanks for the catch.
Nothing lost. This test only checks that you can bind INADDR_ANY and connect on <IP_ADDRESS>, which is a test of your networking stack, not of bond.
I can't find any use of that form of the ctor in cpp/test/comm.


First stab at a Webpack based build

This is based on react-component-boilerplate. Following features are provided:

Webpack driven build/development mode with hot loading. Hit npm start to access the dev mode. npm run dist to generate builds
In addition source build is generated for Node through Babel
Linting is provided through ESlint - npm run lint
GitHub Pages support - npm run gh-pages + npm run deploy-gh-pages
npm run version x.x.x for generating versions with dist build etc.
automatically

I didn't touch the test configuration. You will have to tweak the demo and branding to your liking.

Note that things like jscs style can be added easily to the build if you want. Also more linting rules can be enabled at .eslintrc. I disabled a couple of the default ones.

What if we used something like component playground for interactive examples at the demo site? I haven't used that on my projects yet but it seems like it could fit this project well.

Hi,
Thanks for the PR but that seems quite overkill for this kind of project.
The development/release workflow is quite simple as it is and I don't want to add a dependency on Webpack as IMO it's up to the application to handle how it packages the modules, not the library to impose specific implementations.

The implementation doesn't tie possible applications to Webpack in any manner. It is strictly for development.
The Node version remains the same. There is an UMDified version that can be easily consumed from global, CommonJS or AMD environment making it actually easier to consume the library.

Hey, OK I'll have a look when I have time, thanks!

I see you integrated webpack already. It's safe to close this. Note that the boilerplate has evolved somewhat. Perhaps you can pick up a trick or two from there.


Add private and public macro at the cpp-library plugin level to ultimately support exporting transitive macros

Context
Want to get some eyeballs on changes support the exporting of public macros per issue  https://github.com/gradle/gradle-native/issues/755.  To get the ball rolling, I started with creating dsl at the 'cpp-library' plugin level.  The commits in this PR  pass the public/private macros down to the task level CppCompile.  Ultimately, we only the publicMacros to be exported via prefix-header.h.  Wanted to make sure we are on the right path to a solution that would be accepted into master.  This commit touches about 12 source files in order for the macros to be passed through necessary plumbing from CppLibaryPlugin -> CppBinary -> CppCompile.  I verified the macros are passed down the to task which ultimately get passed to the command line args.
library {
    privateMacros.put('MY-PRIVATE-MACRO', null)
    publicMacros.put('MY-PUBLIC-MACRO', null)
}

Contributor Checklist

[ ] Review Contribution Guidelines
[ ] Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
[ ] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[ ] Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[ ] Provide unit tests (under <subproject>/src/test) to verify logic
[ ] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[ ] Ensure that tests pass locally: ./gradlew <changed-subproject>:check

Gradle Core Team Checklist

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation
[ ] Recognize contributor in release notes

Thank you @fletcher-sumglobal for this PR. Unfortunately, macro exportation will most likely be solved differently. I would suggest starting with a native sample https://github.com/gradle/native-sample instead so we can then discuss the use case behind. Let's start with some exploratory questions around this feature:

What are we trying to solve here? Most likely we are talking about targetting variants at the API level.
Is the reason to show/hide APIs when enabling/disabling features?
Is the reason to expose a variation of types to be consumed by the library? For example, a different string library to use incompatible with the standard library.
Is the reason or align the implementation of inline methods available in headers? For example macros within the C++ template implementation.
Is the reason to communicate key-like information like munging or encryption keys/seeds to be compatible with the library data?
Is the reason to assert the consumer is correctly configured with compatible build option? For example, when consuming a pthread-disabled variant you could react to the this by disabling pthread for the current build or you could trigger an exception for incompatible runtime.

The important thing to note here is exporting the macros is how the use case could be solved. Let's identify what we are trying to achieve with clear use cases. Issue https://github.com/gradle/gradle-native/issues/755 was a placeholder to show one use case where exporting macro would be required.

@lacasseio The is only extending the library to support the notion of public/private. It doesn't go any farther than that. Regardless of the use case, there is a need to be able to declare macros as private vs. public. The question should be, is CppLibrary/CppComponent the right place to do it?
Core requirements include ability to define public macros for

different variants (debug, release, etc)
different linkages (static vs. shared)
different platforms (pc, unix, mac, etc)
different architectures (x86, x64, arm, etc)

Use cases -

To drive logging level
To enable/disable features in library
Select plugins (sort of enable/disable plugin but parameterized)
Patch builds (to enable some code only for specific builds targeted for specific consumer)
Other examples includes things like selecting version of OpenGL to use, version of dynamically loaded dlls, etc.
In some cases, files are shared between two independent projects but some small section of the code is enabled for only one but not for other
Enabling/disabling tool specific code when compiling the same module in for two different reasons

Regardless of any of the use cases above, the core functionality of being able to declare and consume these macros is required and that's exactly what this PR is about. Does holding these list of private and public in CppLibrary/CppComponent allows achieving the above core requirements?
If you don't think we are on the right path please advise!

@lacasseio ,
As @amz-shahji pointed out there are many use cases for exporting macro.  You have pointed out several use cases in your comment as well.  The answer to the why is "all of the above".  At the end of the day, the user just wants the macros to appear on the command line under the hood when producing or consuming dependencies.  But not "all" macros, just the public ones.  The current gradle implementation doesn't allow for that.  If the implementation we have proposed is not satisfactory, it would be desirable to come to an agreement on what the DSL should look like (in the build.gradle) to the user when producing a dependency, that way the underlying implementation, (the how) can change all day long just as long as the end result is achieved.  So for producing a lib, my current implementation supports DSL that looks like:
library {
    privateMacros.put('EMPTY_CHECK_PRIVATE', null)
    publicMacros.put('EMPTY_CHECK', null)
}

I don't see how it could get any simpler than this.  My hope is that this is acceptable from an input standpoint and that it could one day be supported.  And on the consume side, only the publicMacros propagate downstream through whatever means necessary as the how.  As discussed before in https://github.com/gradle/gradle-native/issues/755, the how proposed was to implement a prefix header solution which exposes only public macros and it was my understanding that that was an acceptable solution.  If this how is not acceptable, then we would need an alternate proposal from the gradle team that is acceptable.
That said, I hope the DSL above is satisfactory, and would one day be supported.  In the meantime, we could use our custom gradle implementation with the expectation that the DSL would be supported in the future by gradle upstream.
As far as creating an external custom plugin for this, I don't see how this could be accomplished because the underlying gradle api doesn't expose what is needed to export public macros via external means without duplicate configuration and duplication of the native build life cycle.  That is just too much work to build custom plugins every time these edge cases are encountered.

Given the use case exposed, it seems the feature that is required is a better support for variants and most likely some sort of rich information about the variants. This rich information may end up being the exported macros. Given the macros can be exported using a Gradle plugins as shown in this sample, we would most likely concentrate on making such plugin easier to write and wire within a build system. The solution (exporting macros) to the use case seems too liberal and have a lot of possibilities to cause maintenance pain on our part. The use cases exposed are all great ones and would love to detail them a bit more:

To drive logging level

Does this means a library would set his logging level to INFO and all transitive library would need to also follow the upstream logging level?
What happens if a consumer wants to change the logging level on for his library? Does it keep propagating outward with a new logging level?
Can it be done or all logging level need to match?
Are we talking about a logging library that compiles out (remove) any logging level code for finner level? Meaning that if the upstream library compiles with INFO and the consumer compiles with DEBUG it would result in a compilation error because some logging macro resolve to some non-existing code instead of been a no-op?

To enable/disable features in library

Are those macros simply used for better IDE support by hiding the API in the intellisense?
What happens when a feature is miss matched between upstream and downstream (disabled in upstream but enabled in consumer or the opposite)? Compilation errors or linker errors?
Are there features that are dependent on each other? Or mutually exclusive? Or independent? Or a mixture of all of them? Could you give a ballpark figure of how many features are dependent, mutually exclusive, etc. just to give us an idea of what is the most common?

Select plugins (sort of enable/disable plugin but parameterized)

How similar or different is this use case from enabling/disabling features?

Patch builds (to enable some code only for specific builds targeted for specific consumer)

So I understand properly, these patched builds are variant of the common build. What does the patched variant communicate to the consumer via macros?

Other examples includes things like selecting version of OpenGL to use, version of dynamically loaded dlls, etc.

Are the selection simply used for adapting an API between version for the difference?
As for the version of dynamically loaded DLLs, are they compiled directly inside the binary?

In some cases, files are shared between two independent projects but some small section of the code is enabled for only one but not for other

What kind of files are shared between the independent projects? Headers or sources?
Could the sources be compiled as variants and consumed as such by the independent projects?

Enabling/disabling tool specific code when compiling the same module in for two different reasons

Would those difference be considered as variants of the tool?
Why using the tool requires to know the macros exported by the upstream libraries?

In general, when the public headers contain macros that remove or add APIs in them are they always required or only if you include a specific header?
What would be the most widely used case out of all the one specified here?

@lacasseio Very much appreciate the shared sample. This helps a lot. We can use this as an example for more plugins in the future. Thank you!
As you have already inferred, many of these use cases are very much tied to variants. But flexibility is important here. With a team of 300+ engineers, not everyone knows what a variant is, how to create/define one, and how to use it. We need solutions that work out of box for majority of our team members. However, every engineer on the team knows what a macro is, how to define one, how to use one, and what it does. There is something to be said about familiarity and ease of acceptance.

Does this means a library would set his logging level to INFO and all transitive library would need to also follow the upstream logging level?

Yes, but for the most part to enable/disable logging as a whole. Logging is implemented at core level via macros (LogInfo, LogError, etc are all vararg macros). When disabled, these macros are stubbed out, via the command line define. This is generally the case across variants (debug = enabled, release = disabled). However, there are special use cases like disable everything but errors/fatal logs. This is again driven thru' macros on command line and done at core level, and transitively propagated downstream. The logging library can be compiled and linked statically in a dll and configured independently from the rest of the application. This use case is supported by variants of the library itself.

What happens if a consumer wants to change the logging level on for his library? Does it keep propagating outward with a new logging level?
Can it be done or all logging level need to match?

Once defined it doesn't change for the whole binary (dll or exe). Only way to change is to build it as an independent dll and link statically with different logging.

Are we talking about a logging library that compiles out (remove) any logging level code for finner level? Meaning that if the upstream library compiles with INFO and the consumer compiles with DEBUG it would result in a compilation error because some logging macro resolve to some non-existing code instead of been a no-op?

At least our version of it won't result in compile errors because they are stubbed and the linker will drop empty calls.
In many ways, this is like creating multiple variants of the logging library with different levels of logging enabled/disabled. However, the permutations multiply fast if there are more of similar kind of macros. For instance, logging is enabled but with limited details (print only logging site i.e. filename and line number only, and avoid any string formatting), or enable only specific categories for logging (rendering, loading, caching, etc).

Are those macros simply used for better IDE support by hiding the API in the intellisense?

Absolutely not. There are a million different reasons to use macros for features.

Demo builds - builds that are created temporarily for say E3 demos. They have features that are locked down to showcase slice of it
Patches - Temporary fixes that are targeted towards specific group until a proper fix comes along
Platform specific feature support - Whether to enable/disable Apple watch sync with the app on Apple iPhone, for instance.
Hiding future/upcoming features from current release binaries
Intermingled code for multiple tools - a header/source file being compiled for two independent tools with only a single function which is available in one but not the other. Maintaining two independent files is a unnecessary burden.
Support for float vs. double - a library like Lua has the type as a define

What happens when a feature is miss matched between upstream and downstream (disabled in upstream but enabled in consumer or the opposite)? Compilation errors or linker errors?

Yes.

Are there features that are dependent on each other? Or mutually exclusive? Or independent? Or a mixture of all of them? Could you give a ballpark figure of how many features are dependent, mutually exclusive, etc. just to give us an idea of what is the most common?

This very much depends on the scale of your product. For the most part, I would say they are independent. But we do have cases of inter dependency also - for instance, audio could be enabled/disabled and if enabled load Dll_A or Dll_B conditionally.

How similar or different is this use case from enabling/disabling features?

It is different in some cases like my example above in context of audio. When audio is enabled, both Dll_A and Dll_B could support exactly the same features but using different encoding. So, as a feature, nothing really changed but how the feature is "implemented" changed.

So I understand properly, these patched builds are variant of the common build. What does the patched variant communicate to the consumer via macros?

Yes they are. For the consumer it drives their release. For release v1.0.0, the binaries are unpatched, but for release v1.0.1 the patch is applied. However, v1.0.2 can be build off of v1.0.0, without this specific patch applied but some other patch applied. This specific use cases is all about driving releases (while updating only the micro version number) which aren't necessarily off of the same base but converge on major.minor release. Sometimes, these are proprietary patches that can be shared only with specific consumers.

Are the selection simply used for adapting an API between version for the difference?

Yes. Certain features/API might be available in version but not in other.

As for the version of dynamically loaded DLLs, are they compiled directly inside the binary?

No. These DLLs have a "well-known" symbol which is used to initialize, post load.

What kind of files are shared between the independent projects? Headers or sources?

We have same files compiled for the main editor tool and some independent plugins (DLLs). In this specific case, its logic to load global environment settings at runtime. They can be both headers and sources, usually both.

Could the sources be compiled as variants and consumed as such by the independent projects?

Yes, but it adds unnecessary project hierarchy especially when there are too many headers that are required implicitly via #includes. For instance, if ProjectA and ProjectB shared only one source file but that source file needs 20 different headers from ProjectA. Breaking ProjectA into two such that both ProjectA and ProjectB could be dependent on it, is not always possible. At the moment, we are building both ProjectA and ProjectB as part of a composite build.

Would those difference be considered as variants of the tool?

Not always, like my example above with editor and plugin.

Why using the tool requires to know the macros exported by the upstream libraries?

If a DLL was compiled with a specific feature enabled/disabled, the consuming project needs to know that.

In general, when the public headers contain macros that remove or add APIs in them are they always required or only if you include a specific header?

I have to say they are always required. Thought process - externed/globals that can be declared/used in consuming library but are defined in dependent library. The definition itself could have been wrapped with a macro.

What would be the most widely used case out of all the one specified here?
This is hard to answer because we don't look at macros like the way you are describing. For us, its a way to enable/disable lines of code that we don't want someone/everyone else to use. Nothing more nothing less. In even the smallest project we have (just 4 source files), we have nearly 25 macros defined on the command line. Most unused but our build system is configured to add those regardless. These are things like environment being built for, environment being built on, location of specific libraries/dlls, remote server addresses, etc. I don't think our macros use case is be a very good indicator of "most widely used case".

Thought for food - I see where you are going with all these questions and its a noble cause. Unfortunately, for established code bases, like ours, changing/molding to fit these strict/tight requirements wouldn't be possible. It already a uphill battle to even introduce a parallel build system. The usual norms of best software engineering practices are broken thousands of times a day, if not more, in these code bases. The migration is slow, really slow, to show any relevant progress in a timely fashion. What's needed is stop gap solutions that work allowing us to move forward with things as they are and recommended/ideal solution defining how to to do things for best results. We need to start moving forward faster to be able to commit to this HUGE migration. Hope the input helps drive towards a solution!


Open franz from tray

Good day! It would be very cool if Franz, as in the previous version, will open from the tray when calling command. Now he returns that Franz is already running. If, when you enter a command with Franz running, it will be opened from the tray, many Linux users will be grateful =)

Actually I am experiencing the opposite problem. I can only bring up the running instance of Franz from the appidicator icon in Ubuntu Mate. If I click on the icon in the menu or dock (I am currently using plank) nothing happens. Whereas other Electron apps like Mailspring bring up the running instance. I don't know if this happens in the Windows or Mac versions.

Oh, im sorry. Its my mistake. I meant that Franz opens only from the tray. When using a command, or starting an application from the menu, nothing happens. This is my bad English.


Program runs then dies

On a Win7 Ultimate system with 64GB RAM and vast free disk space, the program runs for a few seconds then disappears.  The system has a very large number of programs installed (perhaps 1000); can BCUninstaller handle such a large load?
If you want, I will be a beta tester as I am anxious to use this fantastic-looking software.

Try the test build I posted in #291

Thank you for the response! Unfortunately, the installer for that version won’t even start on my PC. (I do have all the Microsoft redeistributables installed.)

I just realized that the .zip file in #291 downloads to my PC as a tiny file (under 1 MB).  I imagine it's supposed to be a lot bigger -- is there another copy to try?

That's just the launcher .exe, use it to replace your existing bcuninstaller.exe in the root directory (not in the subdirectories with many files). If that doesn't help, try running bcuninstaller.exe directly from the win-x64 folder.

Yeah, that dawned on me when I was away from my computer.  I should have deduced that right away!
Anyway, replacing the .exe from the installer with the one from the .zip results in the same symptom: it runs nicely for a second or two then just quits.
When you say to run it from the win-x64 folder, do you mean the .exe that put there by the installer, or the replacement one?
Thank you very much!

So I tried both ways; using the original exe in the win-x64, the program runs for several seconds longer than before, but ends up quitting before it's done scanning.  The replacement exe won't run at all from that folder (but you probably already knew that).
My offer to be a beta tester still stands.

After it stops working, check the x64 directory for any error report .zip files, send the latest one if you see any. If not, go to system event viewer > application log, and look for errors related to BCUninstaller.exe. Send whatever you can find.

There is nothing in the event logs, and no zip file.  There is a .log file in the x64 directory, but all it has is multiple instances of

01:30:17 p - Application startup
with different times.
I think my system is a good "extreme conditions" testbed, because of the huge numbers of applications installed and the multiple hard drive letters mounted.  I'll keep trying if you will.  THANK YOU!

Try using these settings. If it works, try turning on different detection options and see what causes it to fail.
BCUninstaller.zip

That worked!  I placed the settings file into the same directory as the exe, and things ran as expected.  I will experiment with different options and get back to you.  Thanks so much.

OK, here's the update: it was definitely the Registry scanning that caused the crash.  I have plenty of other evidence that the registry on this PC has issues.  I am hesitant to use any "registry repair"-type utilities, but if you suggest one you're confident in, I'll try it.
  The level of attention to detail and the level of user feedback/control are extraordinary.  I have hundreds of applications installed on my D: drive (to preserve space on my SSD), and this your program located them despite my having disabled registry scanning.
I encountered one crash, which occurred after I hit the down-arrow after changing the sort order of the list of applications.  (I did that because the one I'd had selected was no longer visible and I thought that, like some applications, it was still selected but scrolled out of view and hitting the up or down arrow key would scroll it back into view.)  Your program provided details about the crash and offered to send a report to you -- which I accepted.
As a long-time programmer, I recognize the rare mastery of Windows internals you have.  Would it be possible to contact you offline to see if I can hire you to help resolve some (probably complex) issues that prevent Windows Update and System Restore from running as well as prevent me from upgrading to Windows 10 as an upgrade (or even reinstalling Windows 7)?
Thank you very much,
-Roy E.

Thank you for the follow-up!
If BCU completely dies during registry scan with no errors then there's something very wrong with the registry. You should get at least a crash report box if something goes wrong. If it hard crashes then it's most likely an unrecoverable error originating from outside BCU, indicating some sort of system issue.
You can try this utility, if that doesn't help then try running chkdsk, sfc /scannow, and repair with dism. If none of these help with your system then it's best to just backup your user folder and any important files, and do a clean reinstall of your OS.

Thank you Klocman!  chkdsk is fine and sfc /scannow says there are issues but it can't identify or fix them. (I don't recall what happened last time I tried dism, except that it didn't help, but I may try again.)  The repair program you mentioned is interesting: it found numerous missing and broken reparse points; it offers to repair them but I'm wary because it doesn't say HOW it would do that, nor whether doing that could be unsafe (or unhelpful at best).
I can never do a clean (re)install because I have about 1,000 installed applications, many on D:, many going back years, and many that were only free on the day originally offered.  Also, my PC's USB ports, and NVMe slot aren't recognized by the Windows 7 installer, but installing Windows 10 would mean losing many things, including programs I wrote years ago that won't run under 10.


Fix tdi_pipeline_builder compile errors with gRPC v1.59.1

This CL addresses the issue noticed in tdi_pipeline_builder with compilatio errors of ambiguous function overload with new gRPC (v1.59.1) and protobuf versions.
The change has also been backtested on older version of gRPC and protobuf and compiles ok without issue.

I just had an afterthought.
When you backtested the change, did you do so with C++11, C++14, or both?

I just had an afterthought.
When you backtested the change, did you do so with C++11, C++14, or both?

I compiled the old way (which is NOT using the stratum-deps repo)...so, it would be the C++ 11 method.


zd3689082: Log charge id when making POST requests

We got quite a lot of logs in March like
Connection timed out error for gateway url=...
Gateway returned unexpected status code: 503...
These come from https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/gateway/GatewayClient.java.
It'd be useful to tie the chargeId to these logs so we can have another level
of confirmation what a charge's status should be from the calls that were made.

Most of the time, we should output a representation of the gateway response (see https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/paymentprocessor/service/CardAuthoriseService.java#L91 and https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/paymentprocessor/service/Card3dsResponseAuthService.java#L79 and a few similar lines) but it does require us to know what to look for in the response and I’m not sure this is consistently done for every gateway (I’m not sure Stripe responses get logged like that) and possibly not for Apple Pay and Google Pay. Logging the whole response seems fine but we have to be careful we don’t accidentally log personally-identifiable information.

If you add something to the MDC my understanding is that it will persist for the lifetime of the thread. This presumably means that following requests may inherit a chargeId of a previous request unless we guarantee that every request to connector sets the chargeId key in the MDC
We may be setting ourselves up for a world of pain here - using the power of Splunk we should be able to tie things together by request ID instead?

@rhowe-gds I think you're right about following requests inheriting a chargeId. Looking at https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/paymentprocessor/service/CardCaptureProcess.java#L94 it looks like the requestID is removed. This means I'll have to do a similar thing but it means writing try...finallys everywhere :(

@rhowe-gds I don't think we can use the requestID.... it's already being logged and I can't tell what external requests to PSPs relate to which charge ids.

@rhowe-gds maybe a better solution is to to MDC.put("chargeId", chargeId) in https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/gateway/GatewayClient.java#L56 instead? Then we can remove it in the finally block?

@alexbishop1 you're right. https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/paymentprocessor/service/Card3dsResponseAuthService.java#L85 already logs the raw response from the PSP. i've removed the code i've added to log the response from the server.

@rhowe-gds maybe a better solution is to to MDC.put("chargeId", chargeId) in https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/gateway/GatewayClient.java#L56 instead? Then we can remove it in the finally block?

Seems fair. You only need to use the MDC if you want a shorthand way of passing the chargeId around though - if the logs you want the chargeId to be associated with are logged by that method we can just include it in the log message and the MDC doesn't need to know about it at all.

We may be setting ourselves up for a world of pain here - using the power of Splunk we should be able to tie things together by request ID instead?

I think that's a bit optimistic unfortunately. When you're trying to look at patterns of behaviour, tying things to context via the requestID is challenging. ChargeID is so ubiquitous that I think there's good reason to have it on every log line where it's appropriate.
In the ideal world that would mean:

for HTTP request we should set it at the start of the HTTP request and clear it at the end
for background threads (as is the case here) we should set it when the thread starts processing a charge and clear it when done

As Russell said there the MDC is a thread local so you do need to be careful to clear it, and I think using a try/finally block is the only reliable way to do that.
For the case of this PR, hooking into a try/finally block in the gateway client seems reasonable

@oswaldquek Well, it’s not quite the raw response (it’s the stringification of the Java object we deserialise the response into) but it’s usually pretty good (we’ve improved it a few times to extract more relevant information). For ePDQ payments, it gets the status and the error messages, which is the main thing.

@alexbishop1 oh, I was wrong.  https://github.com/alphagov/pay-connector/blob/master/src/main/java/uk/gov/pay/connector/paymentprocessor/service/Card3dsResponseAuthService.java#L85 DOESN'T log the raw response from the PSP. it's because our error handling code is ridiculously hard to read 💢

AOP sounds good for this type of situation. Sadly it doesn't work out of the box with dropwizard 😢
I'll investigate doing this in a filter.


    release<EMAIL_ADDRESS>

New updates on head repo.
https://github.com/vuejs/pinia/commit/d05970faa23524add6276cc168f2c18de2261c51

The changes on this issues are removed by #23


Dictionary keys are stored by reference

Godot version:
3.0.6 stable
OS/device including version:
Ubuntu Bionic 64bit
Issue description:
var xx = {}

var k1 = [[1,2], [3,4]]
var k2 = [[1,2], [3,4]]
var v1 = [1,2,3]
xx[k1] = v1
print(xx.has(k2))

k1[0][0] = 999
print(xx.has(k2))

print(xx.has(k1))
print(xx.keys())

The result:
True
False
False
[[[999, 2], [3, 4]]]

There are two issues here:

Changing k1 changes the key in the dictionary which implies that the key stored in the dictionary is actually a reference to k1. This might be the design and not a bug, but if it is the design, I am not sure how does it work with the hash of the key which is used to quickly find the value in the internal data structure. Additionally this is counter intuitive and not documented.
After changing k1 there is no more way to retrieve the value. Both k1 and k2 considered not in the dictionary. This is most probably a bug.

Labelling as "bug" for now, I'm not sure if it's actually a bug or by design, but this needs to be reviewed (and if the latter, documented indeed).

take a look at ordered_hash_map.h line ~249 line.
	V &operator[](const K &p_key) {
		Element e = find(p_key);
		if (!e) {
			// consistent with Map behaviour
			e = insert(p_key, V());
		}
		return e.value();
	}

for dictionary, the K template == Variant..
so there needs to be some template specializiation for where K == Variant
in which the k value is copied.

@alkhimey if you were to update k1 so it matches the original value, you would see that you'd be able to retrieve the v1 value? I think it hashes the k value upon insertion, but then keeps a reference to the key value purely for returning the keys list... so theoretically values should still be accessible.

The keys you are using are arrays. Arrays are types that are always passed by reference. That includes when they are used as dictionary values, or dictionary keys. So yeah, it's just by design.
I never really understood the point of using arrays directly as keys hashing by content. I don't know any language doing it, even Python does not allow it and instead asks for a tuple, which is immutable, unlike arrays, probably due to the issue mentionned here.
If we want to keep this, perhaps the only way is to copy the array if used as a dictionary key.

@pgruenbacher The relevant code I believe is here
/* I want to have only one function.. */
	_FORCE_INLINE_ const Element *get_element(const TKey &p_key) const {

		uint32_t hash = Hasher::hash(p_key);
		uint32_t index = hash & ((1 << hash_table_power) - 1);

		Element *e = hash_table[index];

		while (e) {

			/* checking hash first avoids comparing key, which may take longer */
			if (e->hash == hash && Comparator::compare(e->pair.key, p_key)) {

				/* the pair exists in this hashtable, so just update data */
				return e;
			}

			e = e->next;
		}

		return NULL;
}

My conclusion is that after changing r1:
r1 does not have the correct hash anymore.
r2 has the correct hash but it's contents are different from r1 ( Comparator::compare returns false )
I think making a copy of the keys (and values?) is a good solution.
If it is not important enough to change or breaks the overall design, a simple warning in the documentation can be of a great help for other developers.

I think it might be nice to have a warning emitted whenever the key is Variant and is_shared returns true for it.
Alternatively, Dictionary should call Variant.duplicate(deep=true) when inserting the keys.

Probably @reduz should inspect this. Anyways if there will be change it will be a major change, I push it to 3.2 for now

Can we get tuples?

Is this still planned ?

@pwnorbitals We don't know for certain if this is a bug or design decision yet.

The results from modifying one key to match another is not obvious:
var dict = {[1]:'a'}
var key = [1, 2]
dict[key] = 'b'
key.erase(2)
print(dict)
dict[[1]] = 'c'
print(dict)
print(dict.erase([1]))
print(dict)
print(dict.erase([1]))
print(dict)
for k in dict.keys():
	print(k)
dict.clear()
print(dict)

{[1]:a, [1]:a}
{[1]:c, [1]:c}
True
{[1]:Null}
True
{[1]:Null}
[1]
[1]
{}

Modifying the value associated with one key updates the value for both keys.
erase() returns true even if the keys are still in the Dictionary.
Erasing from the Dictionary removes the value, but leaves both keys.
Clearing the Dictionary removes the keys.

I think this issue might be receiving less attention than needed due to the title. I think it needs to be updated to emphasize that dictionary keys are neither stored by value nor by reference. The current behavior is much less intuitive than what the title suggests.

I think it needs to be updated to emphasize that dictionary keys are neither stored by value nor by reference.

If dictionary keys are neither stored by value nor reference, what are they stored as then?

If dictionary keys are neither stored by value nor reference, what are they stored as then?

I intended to mean "not by value only or by reference only" :facepalm: (since it appears to match only if both are same for "shared" built-ins like Array)
Though my claim wasn't completely accurate, it's storing whatever is provided as key.
Since Arrays both "pass by reference" and "compare by value", neither the "reference" nor "value" equality is enough by itself for the Dictionary to consider the keys to match. If Arrays were compared by reference (including their hash calculation), then reference equality would still work (k1). If they were passed by value, then value equality would still work (k2).
While there doesn't seem to be any desire to change the way Array passes or compares, both k1 and k2 not being considered in the dictionary is a bit unexpected... Maybe Dictionary should only accept keys that are passed and compared the same way?

Is this still relevant as of 4.0?

Yes, it is still relevant.
The major point is what @cdemirer said last:

A rule that seems to work: Dictionary keys shouldn't be more than 2 of {Mutable, Passed-by-Reference, Compared-by-Value}.

It can now be somewhat worked around with Array.make_read_only() but that's not recursive, so nested arrays like the example would not benefit directly unless you manually run it recursively.
This does not fix the issue, but allows you to make sure an array key won't be changed.

From the discussion I could not see a preferred solution though. There are a few options:

Forbid mutable reference types from being used as keys (Dictionaries and Arrays, including Packed ones). This is a hard breaking change. Objects need not to be forbidden since they always compare by reference.
Pass keys by copy. Use deep duplication so they can't be changed afterwards. Also not needed for Objects.

This is not 100% foolproof. If you do dict.keys()[0].push_back(0) you can still change the value. Perhaps Dictionary::keys() will also need to make copies before returning.
May not be intuitive to users, though it's better than the current way.
Technically a breaking change too, but the current way is "undefined behavior" and nobody should be relying on it.

Emit a warning in GDScript, perhaps with error by default.

Will only work when types are known at compile time, so cannot be the only solution.

Emit a warning in the console when inserting a key by reference.

Can get annoying fast if you want to actually do it, since there isn't a way to silence it.
Will require an extra check every time a key is inserted.

I do prefer the option 2 since it's less disruptive overall.

@alkhimey if you were to update k1 so it matches the original value, you would see that you'd be able to retrieve the v1 value? I think it hashes the k value upon insertion, but then keeps a reference to the key value purely for returning the keys list... so theoretically values should still be accessible.

Is it worth documenting this behaviour in the Dictionary documentation? I spent a while trying to figure out how I could get tuple like keys in Godot dictionaries, but I couldn't find an answer until I found this comment thread.


Cannot build RTView using nix

I tried to build 0.1.0 and the last commit but both failed:
...
[122 of 270] Compiling Distribution.SPDX.LicenseReference ( libraries/Cabal/Cabal/Distribution/SPDX/LicenseReference.hs, bootstrapping/Distribution/SPDX/LicenseReference.o )
[123 of 270] Compiling Distribution.SPDX.LicenseId ( libraries/Cabal/Cabal/Distribution/SPDX/LicenseId.hs, bootstrapping/Distribution/SPDX/LicenseId.o )
ghc/ghc.mk:107: ghc/stage1/package-data.mk: No such file or directory
make[1]: *** [utils/ghc-cabal/ghc.mk:56: utils/ghc-cabal/dist/build/tmp/ghc-cabal] Killed
make: *** [Makefile:123: all] Error 2
builder for '/nix/store/jp8yc7m6mb273gy6m6sg536kj9fyac4j-ghc-8.6.5.drv' failed with exit code 2
cannot build derivation '/nix/store/y4fr020iv5spkkwfdr8zz7x8kl2yqnwa-ghc-8.10.2-configured-src.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/nqw8bg8n44828rqq3m1han06hb78szyz-ghc-8.10.2.drv': 1 dependencies couldn't be built
building '/nix/store/x5x0bzvdxl5ih3avb05a2z3zp4ck441s-tagged-0.8.6-3.cabal.drv'...
cannot build derivation '/nix/store/0c2rvc6ik4cf0al6m8fb2vhcrj834jsy-Cabal-lib-Cabal-<IP_ADDRESS>-config.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/azclpifm98wgjdc9hbi3p9a9bq21p6x1-Cabal-lib-Cabal-<IP_ADDRESS>-config.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/5bw58sfg09kcgiximbba8a4qklkcr8a0-Cabal-lib-Cabal-<IP_ADDRESS>-ghc-8.10.2-env.drv': 1 dependencies couldn't be built
...
error: while evaluating 'recRecurseIntoAttrs' at /nix/store/7lzfs8kl0s78d84f9q2lz1z095dm6w0n-source/overlays/haskell-nix-extra/default.nix:8:29, called from something-something/cardano-rt-view/default.nix:20:21:
while evaluating anonymous function at something-something/cardano-rt-view/nix/haskell.nix:4:1, called from something-something/cardano-rt-view/nix/pkgs.nix:3:34:
while evaluating 'cabalProject' at /nix/store/dyapy325zdmjq3hcbg7pc9i79qqkk1xb-source/overlays/haskell.nix:551:13, called from something-something/cardano-rt-view/nix/haskell.nix:35:12:
while evaluating the attribute 'hsPkgs' at /nix/store/dyapy325zdmjq3hcbg7pc9i79qqkk1xb-source/overlays/haskell.nix:518:15:
while evaluating the attribute 'hsPkgs' at /nix/store/dyapy325zdmjq3hcbg7pc9i79qqkk1xb-source/overlays/haskell.nix:500:43:
while evaluating 'mkCabalProjectPkgSet' at /nix/store/dyapy325zdmjq3hcbg7pc9i79qqkk1xb-source/overlays/haskell.nix:152:13, called from /nix/store/dyapy325zdmjq3hcbg7pc9i79qqkk1xb-source/overlays/haskell.nix:485:30:
while evaluating the attribute 'haskell-nix.compiler."${(((plan-pkgs).pkgs  hackage)).compiler.nix-name}".version' at /nix/store/acfzlk0y1lmpw05k67a54nws1x8irkzm-source/lib/attrsets.nix:344:7:
while evaluating anonymous function at /nix/store/dyapy325zdmjq3hcbg7pc9i79qqkk1xb-source/lib/import-and-filter-project.nix:5:1, called from /nix/store/dyapy325zdmjq3hcbg7pc9i79qqkk1xb-source/overlays/haskell.nix:487:31:
build of '/nix/store/l227sg85874l75gbkmxlvn86axp7y4ar-cardano-rt-view-src-plan-to-nix-pkgs.drv' failed

Which command are you using?
I've tried 5 minutes ago (commit 7840bff95662ebf809b70667885717619c36818d) - it works for me:
$ nix-build
trace: Not using IOHK default nixpkgs (use 'niv drop nixpkgs' to use default for better sharing)
trace: Using index-state: 2020-07-15T00:00:00Z for cardano-rt-view-src
trace: To make this a fixed-output derivation but not materialized, set `plan-sha256` to the output of /nix/store/3ybccc5vnhhiwb52a2lp56wjf52lljb9-calculateSha
trace: To materialize the output entirely, pass a writable path as the `materialized` argument and pass that path to /nix/store/61q0pzlzsvv4ca54ifb7bnfnv97iq9r6-generateMaterialized
trace: Using index-state: 2020-07-15T00:00:00Z for cardano-rt-view-src
trace: To make this a fixed-output derivation but not materialized, set `plan-sha256` to the output of /nix/store/3ybccc5vnhhiwb52a2lp56wjf52lljb9-calculateSha
trace: To materialize the output entirely, pass a writable path as the `materialized` argument and pass that path to /nix/store/61q0pzlzsvv4ca54ifb7bnfnv97iq9r6-generateMaterialized
/nix/store/nj18vzslbpbhfq8mh6vhb50vm69smy3f-cardano-rt-view-exe-cardano-rt-view-0.1.0
/nix/store/mwg7bbckacc0dsn3k9rj702grp9a3yq3-cardano-rt-view-test-cardano-rt-view-test-0.1.0-check
/nix/store/7n9b6vi5rj2psfa9jwszrc9ilh79p1dl-rt-view-analyzer-exe-rt-view-analyzer-0.1.0
/nix/store/srdz47x0qawxpz44imwz7g6sc0k6nfaa-cardano-rt-view-lib-cardano-rt-view-0.1.0
/nix/store/cr3d8546h5v6ak32k47w8w635a2iv7n5-cardano-rt-view-test-cardano-rt-view-test-0.1.0
/nix/store/xyd72l2r25gjswzd50d3a3g8f47s4hdx-cardano-rt-view-0.1.0-coverage-report
/nix/store/ddzv933idrsh6cv92cp68hfbfdimli2g-default-Setup-setup
/nix/store/hspichkg0ns1ggddhl4vxyfmzcp36xlz-rt-view-analyzer-0.1.0-coverage-report
/nix/store/ddzv933idrsh6cv92cp68hfbfdimli2g-default-Setup-setup
/nix/store/8iw46dd554z50rfmxk3ls5l77h5dif9k-cabal-dev-shell

I also can build Linux package locally:
$ nix-build release.nix -A cardano-rt-view-linux-release
trace: Not using IOHK default nixpkgs (use 'niv drop nixpkgs' to use default for better sharing)
trace: Not using IOHK default nixpkgs (use 'niv drop nixpkgs' to use default for better sharing)
trace: Using index-state: 2020-07-15T00:00:00Z for cardano-rt-view-src
trace: To make this a fixed-output derivation but not materialized, set `plan-sha256` to the output of /nix/store/3ybccc5vnhhiwb52a2lp56wjf52lljb9-calculateSha
trace: To materialize the output entirely, pass a writable path as the `materialized` argument and pass that path to /nix/store/61q0pzlzsvv4ca54ifb7bnfnv97iq9r6-generateMaterialized
...
trace: Using index-state: 2020-07-15T00:00:00Z for cardano-rt-view-src
trace: To make this a fixed-output derivation but not materialized, set `plan-sha256` to the output of /nix/store/rp6yc3q9qkb572pyg0d8mvsh4k9c9vsd-calculateSha
trace: To materialize the output entirely, pass a writable path as the `materialized` argument and pass that path to /nix/store/5b0lafg3x4vg3zxkmh4r35ibgw16jwwy-generateMaterialized
these paths will be fetched (3.54 MiB download, 3.63 MiB unpacked):
  /nix/store/zkqwgzmsq633cqyp790flsyvzfwv52dz-cardano-rt-view-0.1.0-linux-x86_64
copying path '/nix/store/zkqwgzmsq633cqyp790flsyvzfwv52dz-cardano-rt-view-0.1.0-linux-x86_64' from 'https://hydra.iohk.io'...
/nix/store/zkqwgzmsq633cqyp790flsyvzfwv52dz-cardano-rt-view-0.1.0-linux-x86_64

I've tried both commands on Ubuntu 20.04 with the latest Nix installation.

AFAIU, actual error is this one:
ghc/ghc.mk:107: ghc/stage1/package-data.mk: No such file or directory

I see that some users saw the same error (https://forum.cardano.org/t/anyone-succesfully-running-a-node-on-centos-8-or-freebsd-12-1/37780/4). But I never saw it personally, because I only used nix on NixOS...


Add versioning to messaging protocol

Protocol version information should be included in the messages so that order is maintained.

Completed in 0.0.8-alpha release.


Replace func_get_args with params

For CMS 5, find occurrences of func_get_args as replace them with strongly typed parameters
This will improve the IDE experience

@GuySartorelli Provide a list of affected methods so we can better understand the full impact of this. I would be keen to validate with the core committers if that's something that's bugging them.

Many of these situations would probably be well served using a splat operator parameter - but others might be better served by having an explicit type.
These are all in modules in the silverstripe org - I didn't look at other orgs because this list is already long enough to get an idea of the impact. I also didn't bother with anything in tests/

SilverStripe\Admin\CMSMenuItem::getAttributesHTML()*
SilverStripe\Assets\File::join_paths()*
SilverStripe\Assets\ImageManipulation::variantName()*
SilverStripe\BehatExtension\Context\FixtureContext::joinPaths()
SilverStripe\BehatExtension\Context\SilverStripeContext::joinUrlParts()*
SilverStripe\BehatExtension\Utility\TestMailer::findEmails()*
SilverStripe\Control\Controller::join_links()*
SilverStripe\Core\Injector\Injector::create()*
SilverStripe\Forms\CompositeField::__construct()*
SilverStripe\Forms\FieldGroup::__construct()*
SilverStripe\Forms\FieldList::__construct()*
SilverStripe\Forms\FieldList::changeFieldOrder()*
SilverStripe\Forms\LabelField::__construct()*
SilverStripe\Forms\PasswordField::__construct()*
SilverStripe\Forms\RequiredFields::__construct()*
SilverStripe\Forms\Tab::__construct()*
SilverStripe\Forms\TabSet::__construct()*
SilverStripe\Forms\UploadReceiver::setAllowedFileCategories()*
SilverStripe\Forms\GridField\GridFieldConfig::addComponents()*
SilverStripe\Forms\HTMLEditor\TinyMCEConfig::enablePlugins()*
SilverStripe\Forms\HTMLEditor\TinyMCEConfig::disablePlugins()*
SilverStripe\Forms\HTMLEditor\TinyMCEConfig::setButtonsForLine()*
SilverStripe\Forms\HTMLEditor\TinyMCEConfig::addButtonsToLine()*
SilverStripe\Forms\HTMLEditor\TinyMCEConfig::insertButtonsBefore()*
SilverStripe\Forms\HTMLEditor\TinyMCEConfig::insertButtonsAfter()*
SilverStripe\Forms\HTMLEditor\TinyMCEConfig::removeButtons()*
SilverStripe\i18n\i18n::_t()*
_t()*
SilverStripe\ORM\ArrayLib::array_merge_recursive()*
SilverStripe\ORM\ArrayList::sort()
SilverStripe\ORM\ArrayList::filter()
SilverStripe\ORM\ArrayList::filterAny()
SilverStripe\ORM\ArrayList::normaliseFilterArgs()*
SilverStripe\ORM\ArrayList::exclude()
SilverStripe\ORM\DataList::sort()
SilverStripe\ORM\DataList::filter()
SilverStripe\ORM\DataList::filterAny()
SilverStripe\ORM\DataList::exclude()
SilverStripe\ORM\DataList::excludeAny()
SilverStripe\ORM\ListDecorator::sort()
SilverStripe\ORM\ListDecorator::filter()
SilverStripe\ORM\ListDecorator::filterAny()
SilverStripe\ORM\ListDecorator::exclude()
SilverStripe\ORM\FieldType\DBDate::Format()*
SilverStripe\ORM\Queries\SQLConditionalExpression::setWhere()*
SilverStripe\ORM\Queries\SQLConditionalExpression::addWhere()*
SilverStripe\ORM\Queries\SQLConditionalExpression::setWhereAny()*
SilverStripe\ORM\Queries\SQLConditionalExpression::addWhereAny()*
SilverStripe\ORM\Queries\SQLConditionalExpression::normalisePredicates()*
SilverStripe\ORM\Queries\SQLDelete::create()*
SilverStripe\ORM\Queries\SQLInsert::create()*
SilverStripe\ORM\Queries\SQLSelect::create()*
SilverStripe\ORM\Queries\SQLSelect::setSelect()*
SilverStripe\ORM\Queries\SQLSelect::addSelect()*
SilverStripe\ORM\Queries\SQLSelect::setHaving()*
SilverStripe\ORM\Queries\SQLSelect::addHaving()*
SilverStripe\ORM\Queries\SQLUpdate::create()*
SilverStripe\Security\Member_Validator::__construct()
SilverStripe\View\AttributesHTML::getAttributesHTML()*
SilverStripe\Versioned\DataDifferencer::ignoreFields()*

This method has at least one explicit parameter, but then you can just slap anything after it anyway because func_get_args() is used.

@silverstripe/core-team if you have any opinions please comment.

Generally, I don't have a problem with a bigger usage of explicitly typed params or the splat operator instead of func_get_args as long as make sure we're not changing the behaviour of the methods that would need refactoring in project code making the upgrade from 4 to 5 just another bit more involved.
Just had a random look at a few of the methods, out of that majority was doable without functional changes, which is good, though getAttributesHTML has two different behaviours based on what the type of the param is as well, so there might be cases like that, warranting a manual look at every one of them.


cannot embed

on ubuntu, python2, neovim installed using apt.
Doing the simple example from the readme gives nvim = attach('child', argv=['/usr/bin', 'nvim', '--embed'])
File "tk_ui.py", line 577, in <module>
    nvim = attach('child', argv=['/usr/bin', 'nvim', '--embed'])
  File "/home/alfa/Documents/git/python-client/neovim/__init__.py", line 89, in attach
    child_session(argv) if session_type == 'child' else
  File "/home/alfa/Documents/git/python-client/neovim/msgpack_rpc/__init__.py", line 41, in child_session
    return session('child', argv)
  File "/home/alfa/Documents/git/python-client/neovim/msgpack_rpc/__init__.py", line 17, in session
    loop = EventLoop(transport_type, *args, **kwargs)
  File "/home/alfa/Documents/git/python-client/neovim/msgpack_rpc/event_loop/base.py", line 88, in __init__
    getattr(self, '_connect_{0}'.format(transport_type))(*args)
  File "/home/alfa/Documents/git/python-client/neovim/msgpack_rpc/event_loop/uv.py", line 81, in _connect_child
    stdio=(stdin, stdout, stderr,))
TypeError: Required argument 'loop' (pos 1) not found

The first argument should be the program name. Unless you are using a very strange system, "/usr/bin" should be a directory. Just use something like nvim = attach('child', argv=['nvim', '--embed']).(assuming nvim is on PATH, if not just pass the full path of the nvim binary)

which nvim -> /usr/bin/nvim
nvim = attach('child', argv=['nvim', '--embed']) -> same result as above

which nvim -> /usr/bin/nvim

I know, but the stack trace you printed contains nvim = attach('child', argv=['/usr/bin', 'nvim', '--embed']), so I assumed you were passing "/usr/bin" as the first argument.
Unfortunately I can't reproduce your error, I just updated my local repo and did the following in the REPL:
>>> import neovim; print neovim.__file__
neovim/__init__.pyc  # to ensure the module from master branch in current directory is being loaded
>>> nvim = neovim.attach('child', argv=['nvim', '--embed'])
>>> nvim.buffers[0][:] = ['line1', 'line2']
>>> nvim.buffers[0][:]
['line1', 'line2']
>>> nvim.quit()
>>>

Can you paste a similar interactive python session that reproduces the error?

>>> import neovim;neovim.__file__
'neovim/__init__.pyc'
>>> nvim = neovim.attach('child', argv=['nvim', '--embed'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "neovim/__init__.py", line 89, in attach
    child_session(argv) if session_type == 'child' else
  File "neovim/msgpack_rpc/__init__.py", line 41, in child_session
    return session('child', argv)
  File "neovim/msgpack_rpc/__init__.py", line 17, in session
    loop = EventLoop(transport_type, *args, **kwargs)
  File "neovim/msgpack_rpc/event_loop/base.py", line 88, in __init__
    getattr(self, '_connect_{0}'.format(transport_type))(*args)
  File "neovim/msgpack_rpc/event_loop/uv.py", line 81, in _connect_child
    stdio=(stdin, stdout, stderr,))
TypeError: Required argument 'loop' (pos 1) not found

I'm using python 2.7.6, i'm not sure if that plays a role?

i guess it's because my python is version 2.7.6?

I don't think so, my system's python version is 2.7.10. Not to mention the repo is regularly tested in multiple python versions(minimum 2.6): https://travis-ci.org/neovim/python-client/builds/109927964


SparK ResNet and global feature interaction

Hello, thanks for the great paper.
With the ResNet version of SparK using sparse convolution and sparse batch normalization together, the flow and mixing of global semantic information is heavily restricted due to effective masking on the receptive field caused by sparse operations and lack of global channel interaction with batch norm. It seems like this information will struggle to propagate especially in more shallow networks with lower receptive field like ResNet50. In the paper it was empirically shown that ResNet50 benefited the least from SparK, failing to match the performance of supervised ResNet101. I was wonder if the authors or anyone else tried using  sparse group normalization with ResNet so there would be some global interaction of feature channels to better allow the learning of high level features. Masked autoencoder pretraining has shown alot of promise for data limited tasks in medical imaging and ResNet50 is commonly used by practitioners, so understanding how to most effectively use SparK pretraining has big implications for many in the field.

@csvance very insightful thinking. I've also heard before that using a 3D sparse convolutional backbone network can lead to insufficient global information interaction (actually the interaction only occurs within connected components).
Yeah GroupNorm, LayerNorm or some attention-like operator can alleviate this problem. It's a promising direction to explore.

Running some experiments with this on an internal dataset using the Big Transfer ResNetV2 architecture. One of the other reasons I think GroupNorm might be promising was its transfer learning performance as demonstrated in the Big Transfer paper. Even though sparse normalization counteracts some of the distribution shift, there is going to be a higher degree of feature interaction with unmasked input. Group norm could possibly be more robust against this than batch norm for transfer learning. If it shows promise I will do an ImageNet run and post the results here.

Hello, I know the paper says you use a batch size of 4096, but was curious how many GPU that was split between? Having some stability issues and I suspect it has to do with effective batch size for batch norm in the decoder. Previously I was using a batch size of 64 and accumulating gradient 64 times on single RTX 3090 24GB to get 4096. Now I have access to 4x A6000 48GB and am trying batch size 128 + gradient accumulation 8 + sync batch norm to get 4096. Hoping that having a much higher effective batch size for batch norm in decoder will be the key to stop training from diverging.

@csvance Yeah the sync batch norm and a big enough batch size are important for BN stability. We used 32 Tesla A100s, bs=128 per GPU (so total bs will be 4096) in most of the time, and didn't use the gradient accumulation. I think bs=64 is too small for BN, and 4x128=512 can be better.

Yeah I'm definitely seeing a big difference between my new and old setup. There is still some instability with 4*128 effective batch size for sync batch norm, but things converge much better than I have seen before. It looks like BatchNorm + large batch size  is crucial for the decoder here, I have tried decoder with GroupNorm and convergence is significantly worse.
Just as an experiment I'm running with an image size of 128x128 and using a batch size of 512 per GPU giving me 2048 sync BN batch size (accumulate gradient twice to get 4096 for optimizer step). Will be interesting to see if there is still issues with constant gradient explosion. Here is what the divergence looks like in the loss curve, it pretty much always happens when I reach a certain loss around 0.3 MSE or so:

I was able to get SparK to converge with LayerNorm in the decoder instead of BatchNorm! I had forgot to enable decoupled weight decay with the optimizer I was using, which was the source of the divergences (too much weight decay relative to learning rate). Still during training there are some times where the loss spikes a bit, but its not extreme and starts to decrease again to a better minima.
I have no doubt that BatchNorm will converge faster still, but using LayerNorm in decoder could be a good option for those who do not have access to a huge number of GPU.

@csvance Happy to hear that! and thanks for your effort. Substituting BN with LN or GN (groupnorm) is indeed a valuable try, and I guess that BN isn't always essential. We initially adopted BN just because UNet used it, but I believe LN or GN could effectively replace BN without a lot of performance drop, and yes, this could be particularly beneficial for those with limited GPU resources.

For using SparK with backscatter X-ray images, I found it was good to use a larger epsilon for tile normalization and also normalize x_hat tiles. Reason for this is there is many tiles which are mostly background since alot of X-ray are taller than they are wide. This made the learned representation transfer better for downstream problems. Without the large epsilon, training is unstable when normalizing x_hat tiles at the start of training which seems to negatively impact the learned representation. I suspect normalizing x_hat is a useful inductive bias, but I havn't tried any of this with ImageNet yet.
Until now I have been working with a relatively small subset of my dataset, roughly ~100k.  Going to ramp things up several order of magnitude now. Results on downstream tasks are very promising even with such few images. Downstream is already close to ImageNet21K transfer performance.


Client errors when provisioning

When i try to provision my work computer with my GPG key i get the following sequence:

And i am unable to continue.
I am trying to provision this device using my GPG key.

Can you do a keybase log send?

Cc @patrickxb likely this is an issue with not allowing PGP provisioning after a PUK upgrade.

That is behaving as expected.

But why show the passphrase provisioning option if we know it's going to fail?

Desktop needs to make a release.  They are supposed to have that fixed.  Service sends them details about when to show it, when not to.

https://github.com/keybase/client/pull/7767
(it's been in the service for 27 days:  https://github.com/keybase/client/pull/7637)

I was under the impression that i would still be able to use GPG to sign devices/sites... etc...
Are you saying that GPG login is not allowed?

Once you have a device provisioned, you are not allowed to use GPG to provision any additional devices, you need to use one of your existing devices.
Sorry for the inconvenience, but this was a necessary tradeoff we had to make.

Documentation to follow soon.  There's now a per-user key whose secret half is encrypted for all of your device keys, but we were afraid to have clients encrypt for PGP keys, since they are so hard to manage.  This per-user key in term will drive our new teams product.

So, as i understand it, i used my pgp key to sign my machine key, but then when i want to add another machine, i am no longer allowed to use my pgp key to add more machines.
Why block gpg keys, you make a graph of signatures, why not allow gpg keys for advanced users?
I should note that my machine key is my least trusted key, with my gpg key being my most trusted, i want to use it.

This policy simplifies our implementation and seems right for the most Keybase users. As with anything, we can't please everyone unless we make our app hopelessly complex, which will certainly displease users!

Also, PGP continues to be supported, and, heck, you can now have multiple PGP keys on one account. For the truly bold among us. -- https://keybase.io/blog/keybase-new-key-model

I would like to use pgp keys even if it only possible using expert commands.


React Group Project

Hi Reviewer, 👋
In This Project:

Use React documentation, components, props, and Router.
Connect React and Redux
Handle events in a React app
Write unit tests with React Testing Library
Redux: Fetch data and update the Redux store.
Render UI: lists.
Write actions and reducers for booking rockets and joining missions.
Write actions and reducers for canceling rockets and leaving missions
Render UI: conditional components rendering
Render UI: My Profile section
👌🙌

Project Status: INVALIDATED :red_circle:
Hi @ilyassatlassi and @theodoku ,
While you made a great effort in this project, I can't go ahead and review your code.

Invalid Code Review Request

This is because you have submitted a project, that doesn't have an open pull request from the development branch to the main branch with all changes made by your team. :+1:
You can refer to this article https://github.com/microverseinc/curriculum-transversal-skills/blob/main/code-review/articles/code_review_flow_group_projects.md for more clarity :100:

Your Code Review Request will be marked as invalid in your Dashboard, so please submit a new one once you are ready 🙏
Cheers, and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me @NahnahAJ in your question so I can receive the notification.
Please, do not open a new Pull Request for re-reviews. You should use the same Pull Request submitted for the first review, either valid or invalid unless it is requested otherwise.

Invalid Code Review Request does not count into the code reviews limit.


Change layer orders in layergroups failed

I tried to change subLayer orders within a layergroup by call subLayer's bringToBack method, but it seems affects it's order in all map layers.How can I change layer orders only in layergroup?

Leaflet has no built in method for this. You can try to get all layers that are above / below the current layer and then call on each layer bringToBack.


Can't install modules from PSGallery

Steps to reproduce

PS C:\Windows\System32> Install-Module ExchangeOnlineManagement
WARNING: Unable to resolve package source 'https://www.powershellgallery.com/api/v2'.
Install-Package: C:\program files\powershell\7\Modules\PowerShellGet\PSModule.psm1:9711
Line |
9711 |  … talledPackages = PackageManagement\Install-Package @PSBoundParameters
     |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | No match was found for the specified search criteria and module name 'ExchangeOnlineManagement'. Try
     | Get-PSRepository to see all available registered module repositories.

Expected behavior

Actual behavior
PS C:\Windows\System32> Install-Module ExchangeOnlineManagement -Verbose -Debug
DEBUG: 00:00:00.0000001 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0000356 Name: ExchangeOnlineManagement
DEBUG: 00:00:00.0000491 Verbose: True
DEBUG: 00:00:00.0000565 Debug: True
DEBUG: 00:00:00.0026067 INVOKING PowerShell Fn Get-DynamicOptions with args Provider that has length 1
DEBUG: 00:00:00.0032679 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0044567 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.0091190 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0091446 Name: ExchangeOnlineManagement
DEBUG: 00:00:00.0091520 Verbose: True
DEBUG: 00:00:00.0091588 Debug: True
DEBUG: 00:00:00.0098284 INVOKING PowerShell Fn Get-DynamicOptions with args Source that has length 1
DEBUG: 00:00:00.0103915 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0117005 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.0137037 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0137299 Name: ExchangeOnlineManagement
DEBUG: 00:00:00.0137379 Verbose: True
DEBUG: 00:00:00.0137443 Debug: True
DEBUG: 00:00:00.0144086 INVOKING PowerShell Fn Get-DynamicOptions with args Package that has length 1
DEBUG: 00:00:00.0150325 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0173225 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.0195862 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0196125 Name: ExchangeOnlineManagement
DEBUG: 00:00:00.0196199 Verbose: True
DEBUG: 00:00:00.0196268 Debug: True
DEBUG: 00:00:00.0202156 INVOKING PowerShell Fn Get-DynamicOptions with args Install that has length 1
DEBUG: 00:00:00.0207667 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0226469 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.0343107 Calling SearchForPackages. Name='ExchangeOnlineManagement'
VERBOSE: Using the provider 'PowerShellGet' for searching packages.
DEBUG: 00:00:00.0353564 PackageProvider::FindPackage with name ExchangeOnlineManagement
DEBUG: 00:00:00.0363589 Calling SearchForPackages After Select 1
DEBUG: 00:00:00.0378531 Calling New() : MethodName = 'FindPackage'
DEBUG: 00:00:00.0384285 Verbose: True
DEBUG: 00:00:00.0390176 Name: ExchangeOnlineManagement
DEBUG: 00:00:00.0395570 Scope: CurrentUser
DEBUG: 00:00:00.0400839 ProviderName: PowerShellGet
DEBUG: 00:00:00.0406215 Debug: True
DEBUG: 00:00:00.0413142 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString
DEBUG: 00:00:00.0421523 Type: Module
DEBUG: 00:00:00.0584236 INVOKING PowerShell Fn Find-Package with args System.String[], , ,  that has length 4
DEBUG: 00:00:00.0608427 In PowerShellGet Provider - 'Find-Package'.
DEBUG: 00:00:00.0619294 OPTION: Verbose => True
DEBUG: 00:00:00.0625510 OPTION: Name => ExchangeOnlineManagement
DEBUG: 00:00:00.0631710 OPTION: Scope => CurrentUser
DEBUG: 00:00:00.0638907 OPTION: ProviderName => PowerShellGet
DEBUG: 00:00:00.0647056 OPTION: Debug => True
DEBUG: 00:00:00.0654178 OPTION: MessageResolver => Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString
DEBUG: 00:00:00.0659782 OPTION: Type => Module
VERBOSE: The -Repository parameter was not specified.  PowerShellGet will use all of the registered repositories.
VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2' and PackageManagementProvider is 'NuGet'.
DEBUG: 00:00:00.0711990 PackageProvider::FindPackage with name ExchangeOnlineManagement
DEBUG: 00:00:00.0731945 Calling 'NuGet'::'FindPackage' - name='ExchangeOnlineManagement', requiredVersion='',minimumVersion='', maximumVersion='''.
DEBUG: 00:00:00.0739694 Iterating 'ExchangeOnlineManagement'.
DEBUG: 00:00:00.0760626 There are '0' registered sources in 'NuGet' provider.
DEBUG: 00:00:00.0768264 Source 'https://www.powershellgallery.com/api/v2' is not one of the registered sources in 'NuGet' provider.
WARNING: Unable to resolve package source 'https://www.powershellgallery.com/api/v2'.

Confirm
Continue with this operation?
[Y] Yes  [A] Yes to All  [H] Halt Command  [S] Suspend  [?] Help (default is "Y"): Y
VERBOSE: Total package yield:'0' for the specified package 'ExchangeOnlineManagement'.
DEBUG: 00:00:01.3022692 Completed iterating for 'ExchangeOnlineManagement'.
DEBUG: 00:00:01.3033743 Calling 'NuGetRequest'::'FindRegisteredSource', 'https://www.powershellgallery.com/api/v2'.
DEBUG: 00:00:01.3041977 PowerShell Script 'PSModule' Function 'Find-Package' returns null.
DEBUG: 00:00:01.3052509 Done calling powershell «Find-Package» «PSModule»
DEBUG: 00:00:01.3062885 unmatched package name='ExchangeOnlineManagement'
Install-Package: C:\program files\powershell\7\Modules\PowerShellGet\PSModule.psm1:9711
Line |
9711 |  … talledPackages = PackageManagement\Install-Package @PSBoundParameters
     |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | No match was found for the specified search criteria and module name 'ExchangeOnlineManagement'. Try
     | Get-PSRepository to see all available registered module repositories.

Environment data

Might be related to the cert issue? https://twitter.com/Steve_MSFT/status/1400290882515922950?s=20

I was able to install that module. Can we close out this issue?

I was able to install that module. Can we close out this issue?

Yes, happy to close the issue - thanks.


Feature Request: Add Climate as a tracked entity

I'd like to use this integration to track the energy consumption of my heated bathroom floor - I know the wattage, the only thing missing is to be able to use a climate entity attribute (not just the state) to track when the heating is on.
Here are the attributes of the entity, all I need to to track that heat has changed to "ON":
vac_modes: off, heat
min_temp: 5
max_temp: 30
target_temp_step: 1
preset_modes: none, hold, program
current_temperature: 23.5
temperature: 15
preset_mode: hold
child_lock: null
current_heating_setpoint: 15
deadzone_temperature: null
heat: OFF
linkquality: 66
local_temperature: 23.5
max_temperature_limit: 45
preset: hold
sensor: OU
system_mode: heat
friendly_name: Bathroom Floor
icon: mdi:radiator
supported_features: 17
Btw what you've done here really should be integrated directly into the new Energy dashboard and built into HA... (not for you to fix :)

Yes this would not be too complicated to implement. Will have a look somewhere in the next days.
Maybe we can extend the existing states_power property somehow for an additional attribute.
Something like:
states_power:
        heat|OFF: 2
        heat|ON: 8

when the key has an | included it will be assumed it is {attribute}|{value}
or
states_power:
        attribute: head
        OFF: 2
        ON: 8

For the second example I am not sure if it is possible to extend the schema for validation cleanly.

This has been implemented and merged into the master branch. See #60.


docs: add example for applyTo

Thanks for the review @dalefrancis88, I'll clean this up.
I decided to omit larger examples for the sake of terseness. I will defer to your judgement if you think they're a good fit
Additionally, I also have https://gist.github.com/karthikiyengar/b8ce3d2340b327a76c04401f5d904dcf#file-formatter-js and https://gist.github.com/karthikiyengar/33f0c4b53511b84402fc7309f337e8ae#file-normalize-js as examples from an article I wrote a while ago, if that seems helpful.

I think it's a nice example, but it's always nice to have one simple-ish example to set the stage and pre-prime the concept and then a more complicated/real-world scenario, something people can relate to that they've probably implemented themselves. It helps to hopefully reduce the cognitive load.
I like your first example, what do you think?
https://gist.github.com/karthikiyengar/b8ce3d2340b327a76c04401f5d904dcf#file-formatter-js
How was this not your first example? 🤣
autoFormat("YOLO!")

 It seems to have somehow gotten around the protections it should have linted this as part of the pre-commit checks and blocked it

Coverage remained the same at 100.0% when pulling fdcb05223060db0d42c164776e3762df6dc3dfd8 on docs-for-combinators into ea70d14a0e0544abf7a943440bd48dbd01712386 on master.

@evilsoft @dalefrancis88 - I was wondering if you would consider eslint rules to iron out stylistic issues. I get that you're averse to something like prettier, but I'm pretty sure issues like import order and function declaration styles can be solved by tooling.
Don't get me wrong, I am usually the first person to advocate for consistency, but I think that things like these in PR reviews should be left to tooling, and we should be emphasizing on the content. This would also make it easier for contributors, and in general, help us get stuff done quicker.

In particular, I am referring to
https://eslint.org/docs/rules/implicit-arrow-linebreak
https://eslint.org/docs/rules/sort-imports

@karthikiyengar
Was looking at the implicit arrow linebreak, but the sort imports will not work for our specific case.

@karthikiyengar I've been leaving this up to @evilsoft from memory there was investigation done but it was found to not work with what we want like @evilsoft wrote.
I've been meaning to write a contributors.md as i was writing the recent review for this it made me realize it needs to be sooner rather than later. It's only been me and @evilsoft for a little while and I've adopted the style so it doesn't come up as much and so i've been putting off writing it. I'll get it in today, if not this weekend. My intention is to write it and then go back through all of the old PR's and cross check based on review comments then i'll send to @evilsoft for the review


add Create(), Delete() in TfJobClient interface

rename tpr_util.go to tf_job_client.go
same as pr #140 , based on new master branch

/ok-to-test


Fix clone_macro example

I'll then backport this on the pending branch. Fixes #293.
Full explanations: the state2 variable needs to be cloned otherwise it won't exist anymore, which will prevent the closure to be actually "called" (it'll return early).
cc @sdroege @EPashkin

👍

:+1:


[D1] Pricing for writes + index

Which Cloudflare product does this pertain to?
D1
Existing documentation URL(s)

https://developers.cloudflare.com/d1/platform/pricing/#frequently-asked-questions

What changes are you suggesting?
In the docs, it is mentioned that

Can I use an index to reduce the number of rows read by a query?
Yes, you can use an index to reduce the number of rows read by a query. [Creating indexes (https://developers.cloudflare.com/d1/build-databases/use-indexes/) for your most queried tables and filtered columns reduces how much data is scanned and improves query performance at the same time. If you have a read-heavy workload (most common), this can be particularly advantageous. Writing to columns referenced in an index will add at least one (1) additional row written to account for updating the index, but this is typically offset by the reduction in rows read due to the benefits of an index.

But it does not clarify how many writes will be counted if I add rows to a table which contain an indexed column.
For example, if I write 20 new rows in a table, that has a city column (which is indexed), but in my 20 new rows, there are only 5 unique cities. so, how many rows will I be charged for the writes? 20 or 21 or 25?
Another example, if I add more 10 rows to this table but these rows have the city names already present in the existing table. So, will I charged for only 10 rows as no index needs to be updated? Or 11?
Kindly clarify in the docs regarding this.
Additional information
No response

"5. Indexes will add an additional written row when writes include the indexed column, as there are two rows written: one to the table itself, and one to the index. The performance benefit of an index and reduction in rows read will, in nearly all cases, offset this additional write."


If possible, reword questions so they use the same person throughout, i.e., always second--Are you...? Have you...? or always 3rd--Does applicant...? Maybe this is by design (some questions are for preparer, some for applicant), but there were a few places where it felt like it switched back and forth.

I will suggest that users give feedback while using program to see if we can rephrase questions to aid in asking/interpreting.

I asked for feedback. They will provide as needed.


[WIP] [SPARK-11787] [SQL] Improve Parquet scan performance when using…

… flat schemas.
WIP: this still needs to implement more types and some type conversion logic.
This patch adds an alternate to the Parquet RecordReader from the parquet-mr project
that is much faster for flat schemas. Instead of using the general converter mechanism
from parquet-mr, this directly uses the lower level APIs from parquet-columnar and a
customer RecordReader that directly assembles into UnsafeRows.
This is optionally disabled and only used for supported schemas.
Using the benchmark in PerfTest (I'll remove before this is ready to merge) the rate
for the entire query changes from:
For 1 Column:
Before: 11.3M rows/second
After: 18.2M rows/second
For 2 Columns:
Before: 7.2M rows/second
After: 11.2M rows/second
For 5 Columns:
Before: 2.9M rows/second
After: 4.5M rows/second

Test build #46105 has started for PR 9774 at commit 56f86c5.

Test build #46105 has finished for PR 9774 at commit 56f86c5.

This patch fails RAT tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * public abstract class SpecificParquetRecordReaderBase<T> extends RecordReader<Void, T> \n  *  abstract static class IntIterator\n  *  protected static final class ValuesReaderIntIterator extends IntIterator\n  *  protected static final class RLEIntIterator extends IntIterator\n  *  protected static final class NullIntIterator extends IntIterator\n  * public class UnsafeRowParquetRecordReader extends SpecificParquetRecordReaderBase<UnsafeRow> \n

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46105/
Test FAILed.

Test build #46196 has started for PR 9774 at commit b6a3b62.

Did you open two pull requests?

Test build #46196 has finished for PR 9774 at commit b6a3b62.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * public abstract class SpecificParquetRecordReaderBase<T> extends RecordReader<Void, T> \n  *  abstract static class IntIterator\n  *  protected static final class ValuesReaderIntIterator extends IntIterator\n  *  protected static final class RLEIntIterator extends IntIterator\n  *  protected static final class NullIntIterator extends IntIterator\n  * public class UnsafeRowParquetRecordReader extends SpecificParquetRecordReaderBase<UnsafeRow> \n

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46196/
Test FAILed.

Merged build finished. Test FAILed.

Test build #46244 has started for PR 9774 at commit 1afe5dd.

Test build #46244 has finished for PR 9774 at commit 1afe5dd.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * public abstract class SpecificParquetRecordReaderBase<T> extends RecordReader<Void, T> \n  *  abstract static class IntIterator\n  *  protected static final class ValuesReaderIntIterator extends IntIterator\n  *  protected static final class RLEIntIterator extends IntIterator\n  *  protected static final class NullIntIterator extends IntIterator\n  * public class UnsafeRowParquetRecordReader extends SpecificParquetRecordReaderBase<UnsafeRow> \n

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/46244/
Test PASSed.

Merged build finished. Test PASSed.

I'm merging this. Please address feedback in followup pull requests. Thanks!


Assign __version__ of RecordsFromPax

Before you submit this PR: make sure to put all operations-related information in a wiki-note, a PR should be about code and is publicly accessible
What does the code in this PR do / what does it improve?
To fix https://github.com/AxFoundation/strax/actions/runs/10634771050/job/29482863812?pr=882
Can you briefly describe how it works?
Can you give a minimal working example (or illustrate with a figure)?
Please include the following if applicable:

[ ] Update the docstring(s)
[ ] Update the documentation
[ ] Tests to check the (new) code is working as desired.
[ ] Does it solve one of the open issues on github?

Notes on testing

Until the automated tests pass, please mark the PR as a draft.
On the XENONnT fork we test with database access, on private forks there is no database access for security considerations.

All italic comments can be removed from this template.

coverage: 90.307% (-0.04%) from 90.343%
when pulling 1bf14962008144aef41b9388e31d245c79c1ea3f on version_records_from_pax
into ab7f6ddf78eaeaf87725d6027faff9086b75ec8a on master.