added docs for bootstrap password

When contributing to docs, please don't update the content in the v2.x folder.
It's better to update the versioned docs, for example, the v2.5 or v2.6 docs.
This content in v2.x was separated into versioned documentation during the v2.5.8
release. The content relevant to Rancher versions before v2.5 went into the v2.0-v2.4
folder, while the content related to Rancher v2.5 went into the v2.5 folder.
We are trying to get the 2.x content to be removed from Google search results. The only
reason we haven't deleted it is because Google search results would lead to 404
errors if we deleted it.

Sorry, I should have clarified. The changes need to be made in the v2.6 directory, not the 2.x directory.


Test is passing without running when putting ".only" on "it"

We worked with old mocha (1.18.2) and decided to upgrade to the latest.
Then, we saw that some tests are passing without even running.
We found that this regression started in mocha version 2.4.1 (it worked on 2.3.4, stopped working on 2.4.1 and also on latest).
This is a simple tests that reproduce the issue:
it.only('mocha test', function() {
	var addAppBtn = $('[debugId="create-new-app-btn"]');
	addAppBtn.click();
	// You see this console.log on both 2.3.4 and 2.4.1,
	// but on 2.4.1 the test stop right after it, not doing the rest of the actions.
	console.log('HERE');

    var appNameElem = $('[debugId="new-app-name-input"]');
	appNameElem.clear();
	appNameElem.sendKeys('new app');
	var okBtn = $('[debugId="ok0-btn"]');
	okBtn.click();
});

We didn't see any relevant breaking change in the mocha change logs, unless we missed it.
Any idea how this should be resolved / what's the issue here?

Does the console.log('HERE') run in 2.4.1? Do you have a short reproduce case that doesn't involve your app?

The console.log('HERE') does run on 2.4.1..
Actually, if you add console.log('HERE2') as the last line of the test - you see it as well.
But actual test logic doesn't happen.
I don't have a reproduction that doesn't involve my app, but I can come with this shorter one:
it.only('mocha test', function() {
        var addAppBtn = $('[debugId="create-new-app-btn"]');
        addAppBtn.click();

        // The click doesn't happen, test just ends before it.
        console.log('HERE'); // This is shown.
});

But this actually fix the issue:
it.only('mocha test', function(done) {
      var addAppBtn = $('[debugId="create-new-app-btn"]');
      addAppBtn.click().then(function() {
      // Do something
      done();
  }
});

Now the button is clicked, and only then the test finish.
But this fix makes writing tests very annoying - we need to add a callback almost for each statement.
It breaks the whole idea of writing things fluently, which is a strong point of protractor (that we use).

Looks like addAppBtn.click() is a promise (async), and you're running the test in an sync manner. Thankfully, you can also return promises in tests.
I'd suggest the following:
it('example', function() {
  var addAppBtn = $('[debugId="create-new-app-btn"]');
  return addAppBtn.click().then(function() {
    // Add your assertions here
    console.log('INSIDE');
  });
});

Hope that helps!

@danielstjules  you are right - but it worked on Mocha 2.3.4, and stopped working on 2.4.1
I have more then 100 tests, I don't want to re-write them all. We need to understand what has changed.
Can you reopen ?

Writing tests in "sync" manner is the whole idea of protractor.
That's why we wrote tests in the above manner. (similar example can be found on https://angular.github.io/protractor/#/ - no need to put "then" to a button click).
Thanks for reopening. I'll try to share this with protractor team as well.

No problem! Could you write a minimal reproduce case?

The above example is the minimal I can think of - simple app with 1 button, just try to click it (using protractor API).
Can you be more specific of what can I supply you with?

An example that I can copy-paste and run would be perfect. If it's reproducible outside of protractor (no external deps), even better!

Hi, I'm from the protractor team and I'm working to sort out who's bug this is.  I'll let you know if I can reproduce it without protractor :smile:

Thanks.
Actually I made some progress in reproducing the issue in a simple example.
I can share my sample app later. Generally it seems like that something is
wrong when running the test with ".only".
Meaning - put a .only on the "it" and it will not run properly, but put
.only on the "describe" containing it, just works.
Seems this happens only when I run with grunt protractor runner and not
directly with protractor, but not sure yet.
I'll keep pin point the issue and will update.
On Mar 21, 2016 20:13, "Sammy Jelin"<EMAIL_ADDRESS>wrote:

Hi, I'm from the protractor team and I'm working to sort out who's bug
this is. I'll let you know if I can reproduce it without protractor [image:
:smile:]
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/mochajs/mocha/issues/2157#issuecomment-199408760

I pinned down the issue and wrote a simple app that reproduce it:
https://github.com/benbracha/mocha-test-run/blob/master/README.md
Seems like when putting an ".only" on an "it" cause the test to pass without even running.
It doesn't seem related to mocha 2.4.1, it happened also before.

I'm not sure if this is mocha or protractor issue, but I'll continue the discussion on the related protractor issue. So closing this.

I have same issue since 2.4.1;
This commit 08bf74d named "always reference the correct it from it.only" by @cowboyd introduce this issue.
When i use mocha which protractor, selenium webdriver use a Mocha's BDD interface wrapper (index.js):
exports.it = wrapped(global.it);
exports.iit = exports.it.only = wrapped(global.it.only);
Now wrapped "it.only" function not use the wrapped "it" function but the original mocha function and test is passing without running when putting ".only".
@cowboyd : Can you tell us why you did this update?

@noisette44 This commit resolved #1964 wherein beforeEach hooks were not being run by it.only when they were defined with require('mocha').beforeEach and require('mocha').it.
This testcase failed without this change.

@cowboyd : Thanks for reply
In this case, maybe a duplication of code is necessary between it and it.only function.

Thanks for the reply, just to understand - on which mocha version we can now have the it.only() working properly? Because it wasn't working with 2.4.1

in my case (protractor + mocha), the latest version working with it.only is the 2.3.4.

Same here, although I didn't test recent versions (I see there are up to 2.5.3 now..).
@noisette44  did you try them?

Yes, issue is still present with latest version 2.5.3.

So, if I've understood this correctly...

Logically, Mocha's synchronous tests can't be used for asynchronous code.
Selenium WebDriver, used by Protractor among other things, has asynchronous code and handles it by monkey-patching the test interface with a wrapper that looks like Mocha's synchronous tests but acts like the asynchronous version, tied to whatever asynchronous WebDriver commands are called instead of to done. (Tangentially, if a library or framework is going to supply their own behavior, it'd be nice if they didn't make people think it's Mocha. This isn't the first issue to come up with such modifications, and I doubt it will be the last...)
To work around a nasty side effect of the fact that the BDD interface is still initially set up even when another is chosen (such as the require interface, which tries to set up some of the same stuff again), it.only was changed from using the currently exposed it to using the same it that it.only was originally defined along with.
Thus, whereas before it.only made use of Selenium's monkey-patched it, it now makes use of Mocha's original it, which won't work with the asynchronous WebDriver code.

Is that accurate? And does anyone know if a similar issue applies to describe.only?
I'm inclined to say that Selenium WebDriver (or anything that monkey-patches it or describe) should replace only with a version that recreates the functionality using their own it (or describe). (Fortunately, this should be possible since only is really a shortcut for it + grep; if, as most people usually expect, only were actually doing something direct to force the thing only is called on to be the sole suite or test, it might not be possible to recreate externally.) The behavior of most functions in the face of wrapping monkey-patches doesn't seem like part of the external API to me, at least not if it isn't deliberately intended and specifically documented. And even if it were, at this point if Selenium is intended to support arbitrary Mocha versions it'll have to handle these ones where it.only is tamper-proofed (although, if they only want to support arbitrary "minor" versions, I suppose we could publish new patch versions fixing the issue for each affected "minor" version).
(On the other hand, for what it's worth, if it were to rely on only referring to the monkey-patches it, it would not need to wrap only; it may or may not hurt, but it shouldn't be needed in that case.)
That being said, I think we could change the behavior of it.only back if we fixed the underlying issue -- and I think there is an underlying issue that the way the BDD interface is defaulted is wrong. What we should have is that only the specified interface is set up and the CLI specifies BDD if no other is requested, but what we've got, if I'm not mistaken, is that the BDD interface is always set up and then if another interface is specified that other interface is set up too, and then we try to remove effects of the originally set up BDD interface after the fact (not entirely successfully).

Thanks for reply. I totally agree with you.
So, I will open a Selenium WebDriver issue to know their opinions about this.

Thanks for the reply. I agree as well with @ScottFreeCode comment.
@noisette44  can you reference the Selenium WebDriver issue you opened so I can see the progress of it?
Currently, this issue blocks me from upgrading Mocha, and I'm waiting for some new features I want to use..  (like "retry" added in 2.4.1)
As there a reasonable workaround for this issue you can think of that will allow me to upgrade to newer Mocha versions?

Selenium webdriver issue : https://github.com/SeleniumHQ/selenium/issues/2302

My comment has been corrected to reflect new information acquired with the help of the Selenium team (see the Selenium issue).


Support refute transactions

Adds calls to checkpoint and tests pass

I agree we ought to remove it, however phase 1 of our plan with @lalexgap is to get the contract tests passing first and then following that refactoring the protocols to have updated concepts and data structures


Is it possible to use a single operator for multiple cassandra-datacenters across different namespaces?

Can you comment more about the downsides of running one operator per cluster, instead of per-namespace?
In my environment, we will have many independent CassandraDatacenters each in their own kubernetes namespace. Currently, we have 6, 3-node CassandraDatacenters, but could easily scale to 50 or 100 independent CassandraDatacenters in the future, all inside a single kubernetes cluster.
It would be much easier if we could have a single deployment of the operator, maybe with several pods, to manage all the CassandraDatacenters, instead of deploying an operator per namespace.
Thanks for the info.

Hey rltvty, this is definitely a use case we want to support. We're still building out the testing infrastructure to support this strategy at scale. In this scenario, are the CassandraDatacenters each living alone in a namespace, or do their applications deploy in that namespace as well? If the former, how do you plan on exposing those as services?

@eldondevat The applications using the CassandraDataCenters live in the same namespace.
Ideally, nothing outside the namespace would communicate with the CassandraDataCenter, other than the shared cassandra-operator.

I think the downsides are more in theory than in practice, but just not enough testing has been done for this use case to answer definitively. We can get some small tickets on the roadmap though that should improve your experience. Here are some I can think of...

Helm install option for cluster-wide / single-namespace
Better logging to count the reconciles done per CassDC, not just assigning each reconcile loop iteration a uuid.
Generic back-off / cooldown for clusters that just completed a major milestone in reconciling. We have one for starting a Cassandra node, but could use one for updating the statefulset, or completing reconciling with no more work to do. In practice I've seen the eventual consistency cause extra reconcile spam after we just completed getting a DC up and running.

Some more questions: One Cassandra DC per k8s namespace? Are you planning on having k8s workers with multiple Cassandra pods on them, or no?
(some relevant context - https://github.com/datastax/cass-operator/issues/115#issuecomment-663192818)

On the other questions:
Yes, one CassDC per k8s namespace.  And ideally yes we would have k8s workers (nodes) with multiple Cassandra pods on them (from different namespaces).

@rltvty We've merged https://github.com/datastax/cass-operator/pull/182 - would you be up for trying to install / template with that chart? This should satisfy Helm install option for cluster-wide and we've got some tests for it too, although we've not yet tried at the scale you're considering.

@jimdickinson I'm assuming I should run this by itself inside the cluster, and not along-side existing in-namespace installations of the operator?

@jimdickinson I deployed the cass-operator using the cluster-wide option today into our dev cluster.  It is up and running, managing 6 cassandra-datacenters, each in their own namespace.
One note is: I had to make my own build of the docker image from master, to get the deployment to not-crash.  Version 1.3.0 of the image in the chart would not work, and there is no newer version on dockerhub.
Are there any logs I should collect to help you diagnose the performance of the operator in cluster-wide mode?

This is great news! And yes, this is unreleased work on master only right now. 1.4.0 is going out as soon as a couple more small fixes go in, and it will have everything in master right now.
I think it would be interesting to see how everything works for you if you could do something like scale up all the DCs at roughly the same time. The log prints the duration for each iteration of the reconcile loop, and there's some special handling around the case where we are bringing up our first Cassandra node (or while resuming a stopped DC), but it every other case we expect to see the reconcile loops take very little time. Also curious how much logging you see in general from six datacenters... How many nodes in each DC?

@jimdickinson some responses:

I'm hesitant to try a scale-up of each cluster until the operator also supports scale-down.  Even though I'm working in the dev cluster currently, I can't break much of it often like I did today to install the cluster-wide operator.
We currently have 3 nodes in each DC, spread across 3 availability zones in AWS

We have 8 kubernetes worker machines in total, so numerous Cassandra pods are shared on some of the machines

the operator is currently logging about 20,000 lines per hour.

@rltvty That's totally reasonable. I'll set up an environment like yours and run a couple experiments. What's your target scale for dev/test/prod?


Handling trailing whitespace in Heredoc

I extracted #8695 to remove all trailing whitespaces in our own codebase.
This PR fixes Layout/TrailingWhitespace auto-correction in Heredocs to be safe. Instead of removing the trailing whitespace (which is never equivalent), it inserts #{} after it (which evaluates to an empty string) and thus makes the trailing whitespace no longer trailing. Maybe a more explicit form like #{ '' } or #{ :trailing_whitespace!; '' } would be better though?
Given the fact that we can now autocorrect correctly trailing whitespaces in heredoc, and that they can be a pain to deal with for Rubyists like me that set their editor to automatically trim their whitesapce, I feel that the default should be changed in general. I assume that most projects have much less occurrences of trailing whitespace in their heredoc anyways.

Looks like I need to review my PR, some of these trailing whitespaces were actually a mistake. It doesn't invalidate the rest of the discussion (quite the contrary)

So I found 10 useless trailing whitespaces in our own code...
I tweaked the auto-correction to not do anything for static heredocs <<~'RUBY' inside of which there is no interpolation. I imagine these cases are extremely rare.

This PR fixes Layout/TrailingWhitespace auto-correction in Heredocs to be safe. Instead of removing the trailing whitespace (which is never equivalent), it inserts #{} after it (which evaluates to an empty string) and thus makes the trailing whitespace no longer trailing. Maybe a more explicit form like #{ '' } or #{ :trailing_whitespace!; '' } would be better though?

I agree with the general premise, but I'm wondering if that won't be a bit confusing. It would certainly confuse me if I didn't read your explanations in the PR's description. :-)

Given the fact that we can now autocorrect correctly trailing whitespaces in heredoc, and that they can be a pain to deal with for Rubyists like me that set their editor to automatically trim their whitesapce, I feel that the default should be changed in general. I assume that most projects have much less occurrences of trailing whitespace in their heredoc anyways.

Fine by me.

I also think that if we go down this road it'd be nice to expand the cop description with some of the rationale outlined in the PR's description.

Another possibility would be to replace the trailing whitespace(s) by the interpolation:
hello # + 1 space
world         # + 8 spaces
# corrected to:
hello#{ ' ' }
world#{ '        ' }

Is that a better auto-correction?

It seems better/clearer to me.

PR updated. I also realized I had to modify Lint/LiteralInterpolation to accept those without complaining...

Note: I used a NodePattern with a Regexp (I didn't think I'd use those so soon!) so I bumped the rubocop-ast gem to the latest/
Looks like there's a dependency issue though with JRuby... I'll check that tomorrow

This ends up resulting in infinite recursion for us - I'm not sure how to appease the cop, other than just disabling it around heredocs?
Code
Inspecting 105 files
.....W

Offenses:

lib/bolt/bolt_option_parser.rb:366:1: C: [Corrected] Layout/TrailingWhitespace: Trailing whitespace detected.
lib/bolt/bolt_option_parser.rb:366:11: W: [Corrected] Lint/EmptyInterpolation: Empty interpolation detected.
          #{}
          ^^^
lib/bolt/bolt_option_parser.rb:386:1: C: [Corrected] Layout/TrailingWhitespace: Trailing whitespace detected.
lib/bolt/bolt_option_parser.rb:386:2: W: [Corrected] Lint/EmptyInterpolation: Empty interpolation detected.
 #{}
 ^^^
lib/bolt/bolt_option_parser.rb:418:1: C: [Corrected] Layout/TrailingWhitespace: Trailing whitespace detected.
lib/bolt/bolt_option_parser.rb:418:2: W: [Corrected] Lint/EmptyInterpolation: Empty interpolation detected.
 #{}
 ^^^
lib/bolt/bolt_option_parser.rb:482:1: C: [Corrected] Layout/TrailingWhitespace: Trailing whitespace detected.
lib/bolt/bolt_option_parser.rb:482:2: W: [Corrected] Lint/EmptyInterpolation: Empty interpolation detected.
 #{}
 ^^^
lib/bolt/bolt_option_parser.rb:485:1: C: [Corrected] Layout/TrailingWhitespace: Trailing whitespace detected.
lib/bolt/bolt_option_parser.rb:485:2: W: [Corrected] Lint/EmptyInterpolation: Empty interpolation detected.
 #{}
 ^^^

5 files inspected, 10 offenses detected, 10 offenses corrected
Infinite loop detected in /home/lucy/githubs/bolt/lib/bolt/bolt_option_parser.rb and caused by Layout/TrailingWhitespace -> Lint/EmptyInterpolation
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:304:in `check_for_infinite_loop'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:287:in `block in iterate_until_no_changes'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:286:in `loop'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:286:in `iterate_until_no_changes'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:255:in `do_inspection_loop'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:132:in `block in file_offenses'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:157:in `file_offense_cache'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:131:in `file_offenses'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:122:in `process_file'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:101:in `block in each_inspected_file'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:100:in `each'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:100:in `reduce'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:100:in `each_inspected_file'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:86:in `inspect_files'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/runner.rb:47:in `run'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/cli/command/execute_runner.rb:25:in `execute_runner'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/cli/command/execute_runner.rb:17:in `run'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/cli/command.rb:11:in `run'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/cli/environment.rb:18:in `run'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/cli.rb:65:in `run_command'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/cli.rb:72:in `execute_runners'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/lib/rubocop/cli.rb:41:in `run'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/exe/rubocop:13:in `block in <top (required)>'
/home/lucy/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/benchmark.rb:308:in `realtime'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/gems/rubocop-1.0.0/exe/rubocop:12:in `<top (required)>'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/bin/rubocop:23:in `load'
/home/lucy/githubs/bolt/vendor/bundle/ruby/2.6.0/bin/rubocop:23:in `<top (required)>'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:63:in `load'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:63:in `kernel_load'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/cli/exec.rb:28:in `run'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/cli.rb:476:in `exec'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor.rb:399:in `dispatch'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/cli.rb:30:in `dispatch'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/vendor/thor/lib/thor/base.rb:476:in `start'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/cli.rb:24:in `start'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/exe/bundle:46:in `block in <top (required)>'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/lib/bundler/friendly_errors.rb:123:in `with_friendly_errors'
/home/lucy/.rvm/gems/ruby-2.6.3/gems/bundler-2.1.4/exe/bundle:34:in `<top (required)>'
/home/lucy/.rvm/gems/ruby-2.6.3/bin/bundle:23:in `load'
/home/lucy/.rvm/gems/ruby-2.6.3/bin/bundle:23:in `<main>'

I'm sorry @lucywyman, looks like I wrote the right code but somehow later worked from an older version with the empty #{} (probably when switching computer and not updating my local branch).
I'll fix this asap.

#8914 should restore our original intent and fix the issue.
Until the next bugfix release, you can set your config with:
Layout/TrailingWhitespace:
  AllowInHeredoc: true

Thanks for reporting this, and again sorry for the trouble.

No prob! Thanks for addressing it so quickly!


[Feature Request] Open downlaoded file or folder

How about to add a feature to open downloaded folder or file after downloading it.
If user set up server to local paths matching.
For example my NAS is downloading to some path like this:
/media/downloads/aria2
That path mounted on my PC as this (like network drive as an example):
//<IP_ADDRESS>/downloads
If I could add matching server and local paths like:
/media/downloads/aria2=//<IP_ADDRESS>/downloads
So, then AriaNg could open, let's say my downloaded file: file.txt that has server location as /media/downloads/aria2/file.txt as following file: //<IP_ADDRESS>/downloads/file.txt
Transgui has this feature, works really well:
https://github.com/transmission-remote-gui/transgui

AriaNg Native has already supported this feature.

And how about if folder is not exists?


drivers: bluetooth and wifi: added rs9116w

added drivers for the rs9116w supporting both bluetooth and wifi
Signed-off-by: Rick Talbott<EMAIL_ADDRESS>This requires PR:  https://github.com/zephyrproject-rtos/hal_silabs/pull/40

It looks like you have copied the entirety of subsys/bluetooth/host into the drivers folder, this is not how to create a bluetooth driver. Please take a look at the other drivers.

It's unfortunate that that chip does not provide a HCI or L2CAP interface.   So it was necessary to re-use a lot of existing host functions to interface with.

Will break this apart into two different PRs.  Fist will be wifi, next will be bluetooth.


Bump Sentry JavaScript to 7.51+ for Angular 16 support

Environment
Angular 15 (want to move to 16) with Ionic 6
Desired Result
Update peerDependencies and dependencies to at least 7.51.0 to enable compatibility with the current version of Angular 16.
https://github.com/getsentry/sentry-javascript/releases/tag/7.51.0
Actual Result
npm ERR! This version of Sentry Capacitor is incompatible with the following installed packages:
npm ERR! @sentry/node version 7.51.0

Any update here, please? i.e. Angular 16 support.

Hi, it'll be updated to version 7.54.0

Nice Thanks!


[okhttp-tls] HeldCertificate.decode using an RSA key

Hello dear okhttp team!
We recently had to implement a request to receive AWS credentials from our AWS IoT certificate using this process. (https://aws.amazon.com/blogs/security/how-to-eliminate-the-need-for-hardcoded-aws-credentials-in-devices-by-using-the-aws-iot-credentials-provider/)
Since this request requires you to include the certificate in it to authenticate you we found okhttp-tls.
Quickly, we figured out that the HeldCertificate.decode([cerificatePem, privateKey]) API would convert our AWS certificate PEM + Private Key, into something we could hand into the OkHttpClient using the OkHttpClient.Builder#sslSocketFactory function.
However, we noticed a problem with our private key (RSA) as we handed it to the HeldCertificate.decode function it threw an exception at us, because our key had an unexpected type. (https://github.com/square/okhttp/blob/master/okhttp-tls/src/main/kotlin/okhttp3/tls/HeldCertificate.kt#L518)
Our private key starts with -----BEGIN RSA PRIVATE KEY----- and ends with -----END RSA PRIVATE KEY-----
So, our "solution" was to remove the RSA from the start and end of the key and the decoding started to work.
However, now we are not sure if this is:
a) needed
b) a bug(?) (not supported?) in okhttp-tls (I am more than happy to fix it if necessary)
c) vodoo magic
It would be awesome if you could give us feedback on how to proceed from here.
Should we leave this "solution" from above in place or is there a better way to do what we want to do here.
Thank you very much in advance! ✌

These different strings indicate different contents.
BEGIN RSA PRIVATE KEY is PKCS#1
BEGIN PRIVATE KEY is PKCS#8
https://stackoverflow.com/a/20065522/40013
Do you know which kind you have?

...Now that we have our own DER decoder, we should change that function to accept either format. But it’s important to not pass a PKCS#8 value to a function expecting PKCS#8 and vice versa.

We have
BEGIN RSA PRIVATE KEY is PKCS#1

TIL: Good answer here on PKCS#1 vs PKCS#8 https://stackoverflow.com/a/48960291/1542667

@swankjesse What about uplifting this to certifikit and implementing there (currently only exists in tests?).
Then allow certifikit combined with okhttp-tls?

Yep!

So just to understand and summarize a bit:
So PKCS#1 and PKCS#8 are both "hashed" using DER which is why our current solution to just remove RSA from the key works.
Is there anything that needs / can be done from my side or is this solely on you now?
Thank you for the help so far!

I'd stick with a workaround if it's working fine for you.  I'll raise a bug in certifikit and we'll implement and make some example code for using with OkHttp.
OkHttp shouldn't be a kitchen sink. Hence avoiding any PR that adds this directly to this project.

@dhartwich1991 Any thoughts on https://github.com/cashapp/certifikit/pull/53/files and want to try?

So PKCS#1 and PKCS#8 are both "hashed" using DER which is why our current solution to just remove RSA from the key works.

This surprises me, I think they are different entities being encoded.

Hey @yschimke I just reviewed and approved the PR, looks nice!

No action to take on this.


Add Actionscript and Flash

Hi!
Would you mind adding 'Actionscript' and 'Flash' keywords to the list? I'm a veteran Flash developer and am currently living off of Flash-related work. Times are however a bit difficult for someone as passionate and persistent as me. I'm hoping to find a full-time Flash job I can land in Norway, to finally fulfill my dreams and aspirations of climbing the fjords and mountains and code in Actionscript at the summits :)
I regret to confess that my "modern" web technology skills are of questionable nature and I fear that if I fight this battle alone, I'll only cause a mess that is irreversible. I've tried embarking this dangerous quest on my own, but in the end, I had to raise the white flag, put my ego down, and humbly ask for someone else, someone brave and bold enough to take on this task, to do it in my stead!
Salute to the brave one who risks his/her soul for me!
PS: Writing this "issue" alone has drained me of my newfound confidence to navigate the web of tomorrow. Please help me realize my dream before the sand stops flowing!

What in the name of God Almighty has exactly transpired here? I come with humility and my arms up in the air, asking for immediate assistance with a pure heart lest my figurative life perishes. Yet, I am met with premature closure without warning, and no form of explanation, nor redirection to other means of help. All I am faced with is a patronizing sun, seemingly taunting my livelihood for nothing that it's worth, and though it may be true, even I deserve the right to dignity, good sir.
I'd like to reiterate that my needs are of utmost importance as they depend on my survival. So, I kindly ask this "issue" to be reinstated. Thank you!

I regret to inform you that your request to have "actionscript" added to the list has been denied. Have a nice day, ma'am.

I feel as though every fiber of my soul has been tarnished!
Translator's Note: triggered


New version: Web3 v0.2.3

Registering package: Web3
Repository: https://github.com/lambda-mechanics/Web3.jl
Created by: @zot
Version: v0.2.3
Commit: db9612dbe5e4002446b591af63f7c3d3adca7ee5
Reviewed by: @zot
Reference: https://github.com/lambda-mechanics/Web3.jl/issues/4#issuecomment-627830132

Hmm, I (thought I) followed the instructions for CompatHelper and made a .github/workflows/CompatHelper.yml exactly as it said. Am I supposed to plug in some values for GITHUB_TOKEN and COMPATHELPER_PRIV?


ImportError: No module named 'libtiff'

now i'm use python3.5
but i can't find libtiff for py3;only for py2?
anyone else meet this problem? please help me ,thanks!

What happened after you install libtiff with pip3 install libtiff --user?

不用安装这个库了，我将原始tif图分开了,见data文件夹

when I import "from libtiff import TIFF" then i am getting this error on kaggle kernel
ModuleNotFoundError: No module named 'libtiff'
can anyone please help

conda-installed libtff, but when trying from libtiff import TIFF in jupyter notebook, I got the following error:
ModuleNotFoundError: No module named 'libtiff', could anybody help please?

I had this issue as well. Uninstalling then re-ininstalling with pip seemed to fix it.
First, I tried conda uninstall libtiff, which ended up creating a new issue involving the 'brotli' module. I fixed that using this thread.
After that, I reinstalled using `pip install libtiff', and this now seems to be working.

Try with the command
pip install libtiff


hook up with new OOP designed id resolver module

Implemented in: https://github.com/biothings/BioThings_Explorer_TRAPI/commit/04b4150d93636ee2149ad8c967cf08c02110cc1d


Graph is frozen after PC wake-up

Hello,
I honestly don't know if this is an issue from NZXT Cam Web Integration itself, or just in this particular project. What simply happens is that when you wake-up your computer from Windows Sleep mode, the graph is completely frozen at the moment you put your computer to sleep. The only way to make it responsive again is either to close-reopen NZXT Cam or simply swap what you have on the Kraken LCD with something else and then put this graph back.
Would this be fixable in anyway in case it does not depend from Cam?

humm It sounds like a CAM bug, web integration can't handle this kind of behavior. Does it happens with non web integration as well ? Now I'm curious, I will turn on sleep mode to check if happens with me. but it would be better to open this issue at nzxt docs as well : https://github.com/NZXTCorp/web-integrations-docs/issues.

I tested entering sleep mode and hibernate mode and in both case they back normally from frozen state. maybe something with your device config or windows.

humm It sounds like a CAM bug, web integration can't handle this kind of behavior. Does it happens with non web integration as well ? Now I'm curious, I will turn on sleep mode to check if happens with me. but it would be better to open this issue at nzxt docs as well : https://github.com/NZXTCorp/web-integrations-docs/issues.

Thank you so much for the answer and since I haven't told already, thanks for sharing this awesome project. Before using this I was simply using the Dual Infograph with some customized colors I put, but the config was the one directly from NZXT Cam Dual Infograph. That one has always worked, I put my computer everyday to sleep and then get back and never got this issue.
Unfortunately this does seem to happen with this Web Integration. Even if it was my config, thing that it's probably true, I wouldn't honestly know what to touch. As I stated the only way to get it back for me after sleep mode is to put on another config (i.e. Dual Infograph) and then get back to this Web integration.
Shall I notice NZXT about this?

Thanks, I appreciate it. for not getting this error even entering these mods, it's hard to tell the origin, but it would be greate to have nzxt developers consideration, so openning an issue there would be better. I'll be watching the issue to be informed.

Hello, I've opened an issue here https://github.com/NZXTCorp/web-integrations-docs/issues/13

@brunoandradebr has this integration been updated in the meantime? Because it looks like the sleep issue might be fixed!

@TheGITofTeo997 yeah, I have updated with more custom things but nothing related to the bug you mentioned =s. But in the meantime nzxt realsed an update. I will never know hahahaha

But, reinforcing... this bug never happened to me and I think to anyone else =s

I've tested multiple times and either NZXT has been updated or this integration has been updated too, but for now this is fixed for me. I will close the issue!


Update active-directory-certificate-based-authentication-ios.md

Adding "Microsoft Team" as an Office mobile application that has Azure Active Directory certificate-based authentication on iOS

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

@phecdalouie : Thanks for your contribution to the Azure documentation! The author, @MarkusVi, has been notified to review your proposed change.

Hi @phecdalouie, thanks a lot for your contribution.
I have already updated this topic.
Please close this PR.
#please-close


Certificate verify failed SSLError

Calling a google calendar API results in certificate verify failure.
Expected Result
A successful API response, since the API endpoint is running fine
Actual Result
C:\Workspace\Techroom\venvs\lazyhour\lazyhour\Scripts\python.exe C:/Workspace/Techroom/lazyhour-server/main.py
 * Serving Flask app "main" (lazy loading)
 * Environment: development
 * Debug mode: on
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 279-826-945
 * Running on http://localhost:5000/ (Press CTRL+C to quit)
C:\Workspace\Techroom\lazyhour-server\util\service_account.json
<IP_ADDRESS> - - [22/May/2018 15:22:56] "GET /calendar/list HTTP/1.1" 500 -
Traceback (most recent call last):
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask_restful\__init__.py", line 273, in error_router
    return original_handler(e)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\_compat.py", line 34, in reraise
    raise value.with_traceback(tb)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask_restful\__init__.py", line 273, in error_router
    return original_handler(e)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\_compat.py", line 34, in reraise
    raise value.with_traceback(tb)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask_restful\__init__.py", line 480, in wrapper
    resp = resource(*args, **kwargs)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask\views.py", line 88, in view
    return self.dispatch_request(*args, **kwargs)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask_restful\__init__.py", line 595, in dispatch_request
    resp = meth(*args, **kwargs)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\flask_httpauth.py", line 93, in decorated
    return f(*args, **kwargs)
  File "C:\Workspace\Techroom\lazyhour-server\main.py", line 45, in get
    return calendar_service.get_calendar_list()
  File "C:\Workspace\Techroom\lazyhour-server\util\calendar_service.py", line 25, in get_calendar_list
    response = self.authed_session.get(self.service_base_url)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\requests\sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\google\auth\transport\requests.py", line 198, in request
    self._auth_request, method, url, request_headers)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\google\auth\credentials.py", line 121, in before_request
    self.refresh(request)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\google\oauth2\service_account.py", line 322, in refresh
    request, self._token_uri, assertion)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\google\oauth2\_client.py", line 145, in jwt_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\google\oauth2\_client.py", line 106, in _token_endpoint_request
    method='POST', url=token_uri, headers=headers, body=body)
  File "C:\Workspace\Techroom\venvs\lazyhour\lazyhour\lib\site-packages\google\auth\transport\requests.py", line 124, in __call__
    six.raise_from(new_exc, caught_exc)
  File "<string>", line 3, in raise_from
    # Permission is hereby granted, free of charge, to any person obtaining a copy
google.auth.exceptions.TransportError: HTTPSConnectionPool(host='accounts.google.com', port=443): Max retries exceeded with url: /o/oauth2/token (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)'),))

Reproduction Steps
from .auth_service import get_credentials
from google.auth.transport.requests import AuthorizedSession
import json
# Calendar service

class CalendarService:
    def __init__(self, calendar_id=''):
        self.credentials = get_credentials()
        self.authed_session = AuthorizedSession(self.credentials)
        self.service_base_url = 'https://www.googleapis.com/calendar/v3/users/me/calendarList'
        self.calendar_id = calendar_id

    def get_service_url(self):
        return('{base_url}/{calendar_id}'.format(
            base_url=self.service_base_url,
            calendar_id=self.calendar_id))

    def get_calendar(self):
        response = self.authed_session.get(self.get_service_url())
        calendar = response.json()
        return calendar

    def get_calendar_list(self):
        response = self.authed_session.get(self.service_base_url)
        calendar_list = response.json()
        return calendar_list

System Information
The output of pip freeze is:
aniso8601==3.0.0
cachetools==2.1.0
certifi==2018.4.16
chardet==3.0.4
click==6.7
Flask==1.0.2
Flask-Cors==3.0.4
Flask-HTTPAuth==3.2.3
Flask-RESTful==0.3.6
google-auth==1.4.1
idna==2.6
itsdangerous==0.24
Jinja2==2.10
MarkupSafe==1.0
pyasn1==0.4.2
pyasn1-modules==0.2.1
pytz==2018.4
requests==2.18.4
rsa==3.4.2
six==1.11.0
urllib3==1.22
Werkzeug==0.14.1

Python 3.6
Windows 10
This command is only available on Requests v2.16.4 and greater. Otherwise,
please provide some basic information about your system (Python version,
operating system, &c).

I changed the default value for verify argument in sessions.py file. and the error is gone Here is the method that defines the request:
    def request(self, method, url,
            params=None, data=None, headers=None, cookies=None, files=None,
            auth=None, timeout=None, allow_redirects=True, proxies=None,
            hooks=None, stream=None, verify=False, cert=None, json=None):
        """Constructs a :class:`Request <Request>`, prepares it and sends it.
        Returns :class:`Response <Response>` object.

        :param method: method for the new :class:`Request` object.
        :param url: URL for the new :class:`Request` object.
        :param params: (optional) Dictionary or bytes to be sent in the query
            string for the :class:`Request`.
        :param data: (optional) Dictionary, bytes, or file-like object to send
            in the body of the :class:`Request`.
        :param json: (optional) json to send in the body of the
            :class:`Request`.
        :param headers: (optional) Dictionary of HTTP Headers to send with the
            :class:`Request`.
        :param cookies: (optional) Dict or CookieJar object to send with the
            :class:`Request`.
        :param files: (optional) Dictionary of ``'filename': file-like-objects``
            for multipart encoding upload.
        :param auth: (optional) Auth tuple or callable to enable
            Basic/Digest/Custom HTTP Auth.
        :param timeout: (optional) How long to wait for the server to send
            data before giving up, as a float, or a :ref:`(connect timeout,
            read timeout) <timeouts>` tuple.
        :type timeout: float or tuple
        :param allow_redirects: (optional) Set to True by default.
        :type allow_redirects: bool
        :param proxies: (optional) Dictionary mapping protocol or protocol and
            hostname to the URL of the proxy.
        :param stream: (optional) whether to immediately download the response
            content. Defaults to ``False``.
        :param verify: (optional) Either a boolean, in which case it controls whether we verify
            the server's TLS certificate, or a string, in which case it must be a path
            to a CA bundle to use. Defaults to ``True``.
        :param cert: (optional) if String, path to ssl client cert file (.pem).
            If Tuple, ('cert', 'key') pair.
        :rtype: requests.Response
        """
        # Create the Request.
        req = Request(
            method=method.upper(),
            url=url,
            headers=headers,
            files=files,
            data=data or {},
            json=json,
            params=params or {},
            auth=auth,
            cookies=cookies,
            hooks=hooks,
        )
        prep = self.prepare_request(req)

        proxies = proxies or {}

        settings = self.merge_environment_settings(
            prep.url, proxies, stream, verify, cert
        )

        # Send the request.
        send_kwargs = {
            'timeout': timeout,
            'allow_redirects': allow_redirects,
        }
        send_kwargs.update(settings)
        resp = self.send(prep, **send_kwargs)

        return resp

By the way just to save time please do not pass me back to google to resolve this issue, I already submitted an https://github.com/GoogleCloudPlatform/google-auth-library-python/issues/257 on the google-auth github repo, and they have declared you as responsible for the error.

Hi @pmesgari,
It seems like there may be some confusion about what’s going on here. The Python SSL module is rejecting your supplied certificate because it can’t be verified. The comment @theacodes linked in the original ticket has some more info on that.
The traceback you provided uses a large amount of code that isn’t part of Requests that appears to be creating auth objects that may be being handed to Requests. We’ll need a stripped down version of this, with an example of the kind of cert/credentials being provided before we’d be able to start debugging this.
That said, with the current information I don’t believe this is a defect in Requests, but rather bad input. Until we have confirmation there’s actually a bug here, this is something that would likely be better tracked in a StackOverflow question.

Whoa, definitely not blaming requests! (I'd only be blaming myself anyway,
as I'm the maintainer of urllib3). Google auth doesn't do anything in terms
of overriding request's ssl stuff, we just inject an authorization header.
I linked to this issue to show that this is not an issue that is unique to
google-auth. Lots of users run into this and as the comment above mentions
it is due to an outdated ssl version. Your options are to either update
openssl (which generally means updating Python) or switch to a more
insecure set of CA certificates.
On Tue, May 22, 2018, 2:36 PM Payam<EMAIL_ADDRESS>wrote:

@nateprewitt https://github.com/nateprewitt that is the response I was
afraid of google-auth-library-python people are blaming requests and you
are saying it is the code being handed to Requests. That extra code you are
mentioning is generated by google-auth-library-python. So how to go about
this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/requests/requests/issues/4657#issuecomment-391148712,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUc-GgdI6IcwUinBmSgeSp-fq-AWZqks5t1ITcgaJpZM4UJU0F
.

@pmesgari, I’d start with the code you’ve written and work backwards from there. You may consider using something like pdb to see what objects are being handed to the Requests call. Likely the most important piece of information is the SSL certificate being used.
Once you know that, you should be able to create a much simpler repro and we can work from there in a StackOverflow question. Right now responses here are being pushed to 1200 people that are subscribed to this for the purpose on monitoring issues in Requests. Feel free to link the Stackoverflow issue here and I’ll follow up there.
I’d also try this on another machine/on a basic network configuration to verify this isn’t an environmental issues.

@nateprewitt this should be easily reduced by just making a request to "
https://accounts.google.com".
On Tue, May 22, 2018, 2:42 PM Nate Prewitt<EMAIL_ADDRESS>wrote:

@pmesgari https://github.com/pmesgari, I’d start with the code you’ve
written and work backwards from there. You may consider using something
like pdb to see what objects are being handed to the Requests call.
Likely the most important piece of information is the SSL certificate being
used.
Once you know that, you should be able to create a much simpler repro and
we can work from there in a StackOverflow question. Right now responses
here are being pushed to 1200 people that are subscribed to this for the
purpose on monitoring issues in Requests. Feel free to link the
Stackoverflow issue here and I’ll follow up there.
I’d also try this on another machine/on a basic network configuration to
verify this isn’t an environmental issues.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/requests/requests/issues/4657#issuecomment-391150095,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUcz6fwID4js0bMkO75I6zd0m1KAseks5t1IY4gaJpZM4UJU0F
.

@nateprewitt I have already created the StackOverflow question. I will also try to update openssl and get a smaller repo.

@theacodes yep, that’s likely all we need, I haven’t had a chance to look very deep into what all was being passed around. My guess is this is likely an OpenSSL issue or some other system configuration :)
@pmesgari, thanks! We’ll follow up over there.

@pmesgari An easy way to get a newer openssl on your system is by installing the latest pyOpenSSL which requests will use automagically.


name 'twython' is not defined

When I run my script locally it works fine...however, when I transferred to a digital ocean server I am no longer able to run my script and get the following error even though I have the most up to date version of twython installed via pip install twython and also installed sudo apt-get install python3-requests-oauthlib in case that was a dependency
twitter = twython.Twython(api_key, api_secret, access_token, token_secret)
NameError: name 'twython' is not defined

Here's a copy of the script that I am trying to run https://github.com/M0nica/blackgirlmagic/blob/master/blackgirlmagic_rt.py
I am trying to determine if this is a common error/easily fixable, I was not able to find a solution.

Just do Twython(api_key, api_secret, access_token, token_secret), since you're importing it already with this from twython import Twython, TwythonError.

Thank you @ryanmcgrath. That resolved the issue for me.


Contradiction on document top level members

This section (http://jsonapi.org/format/#document-structure-top-level) tell:

A document's top level MAY also have the following members: "meta", "links", "linked"
No other members should be present at the top level of a document.

But the errors node (http://jsonapi.org/format/#errors) is missing from this authorized list:

Error objects SHOULD be returned as a collection keyed by "errors" in the top level of a JSON API document

Also, adding a full example of response with an error could be nice.

@jwhitlock sorry to take so long to reply. Seems that some examples are referenced in PR #281, but I think a new ticket with a list of error examples that you'd like to see would be useful.
Going to close this since #341 addresses the primary focus of this issue.


How the isPending parameter works

I ask this question as I didn't find documentation on this. I see that adding the isPending parameter hides the values from me.
GooglePay supports isPending ?

That sets the PKPaymentSummaryItemType to pending if true, or final if false. It is not supported in Google Pay.


add const generic ENABLE_SANITY_CHECK to control sanity check part in NIMFS prove

closes #12

The Primary and secondary circuit satisfiability checks are added to the sanity check.
The nimfs.v and nifs.v are also added to the sanity check.
Remove all check_relation call(the PCDNode.verify function already contains)


Support Unload statement

Adding UNLOAD as a recognized statement
Redshift -- https://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD.html
Athena -- https://docs.aws.amazon.com/athena/latest/ug/unload.html

Thank you very much @jonathanlehto  for this contribution

I took the liberty of merging the branch up from main to fix a merge conflict, as well as fixing a clippy issue

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

62 of 62 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 87.914%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
20265

Relevant Lines:
23051

💛  - Coveralls

Thanks again @jonathanlehto

No problem, thank you!


Use CUE default values instead of values.cue

Hi,
new to timoni and I just read the documentation. Why is timoni using/recommending a -values.cue file instead of using CUE default values? Instead of
values: {
    resources: requests: {
        cpu:    "100m"
        memory: "128Mi"
    }
    resources: limits: resources.requests
}

you could use this:
resources: {
  requests: {
    cpu:    string | *"100m"
    memory: string | *"128Mi"
  }
  limits: requests
}

To allow various constructs in CUE file, besides values you could have some other fields, schemas, etc that Timoni will ignore. As with values:, there is also bundle: and runtime:, all follow the same convention.

yes but you could skip the whole values: struct which is how I see it borrowed from the Helm design, as you can always set default values for fields inside CUE

The values field from the user supplied CUE file is used for unification with the values schema from inside the module. But regardless of the values file structure, how Timoni would be used is via Bunde files, where values is a property of the instance e.g.
bundle: {
    apiVersion: "v1alpha1"
    name:       "podinfo"
    instances: {
        redis: {
            module: {
                url:     "oci://ghcr.io/stefanprodan/modules/redis"
                version: "7.2.2"
            }
            namespace: "podinfo"
            values: maxmemory: 256
        }
        podinfo: {
            module: url:     "oci://ghcr.io/stefanprodan/modules/podinfo"
            module: version: "6.5.3"
            namespace: "podinfo"
            values: caching: {
                enabled:  true
                redisURL: "tcp://redis:6379"
            }
        }
    }
}

What I mean is that everything the values struct does is provide values for the instance to use. Everything inside the values struct inside the bundle (values that are optional to be set by the user and set to a default) could be removed and replaced by CUE default values. Since CUE allows to override them you don't have to expose everything as value.
I will add another code example with the difference later but currently on the run :D

I sadly dont have time right now to build an example. But what I meant was to replace the whole values structure as its just smth that helm needed. In CUE you could always allow all values to be overrideable and provide a default, which is what I meant. I will reopen this issue as soon as I got time


rustc: Enable profiler on darwin

remove the isRedox condition which was added because stdenv.isLinux uses hostPlatform, after specifying targetPlatform it's no longer necessary
For error[E0463]: can't find crate for 'profiler_builtins'
https://github.com/NixOS/nixpkgs/pull/266570
https://logs.ofborg.org/?key=nixos/nixpkgs.266570&attempt_id=450d0b18-fd6a-4c44-893d-30de5941b73f
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Wouldn't it be better to assume this works on all platforms except ones where we have a reason to think it doesn't, like Redox?  On a platform it doesn't work on, it would be IMO less confusing to get an error about profiler_builtins not building than one about it being missing from the standard library.

LGTM as long as OfBorg is happy.

Ofborg failed with a timeout in llvm.
Tried building on the build box but
rustc> Building bootstrap
rustc> Auto configuration failed
rustc><PHONE_NUMBER>:error:02FFF001:system library:func(4095):Operation not permitted:/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/libressl/libressl-3.3/crypto/bio/bss_file.c:122:fopen('/private/etc/ssl/openssl.cnf', 'rb')
rustc><PHONE_NUMBER>:error:20FFF002:BIO routines:CRYPTO_internal:system lib:/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/libressl/libressl-3.3/crypto/bio/bss_file.c:127:
rustc><PHONE_NUMBER>:error:0EFFF002:configuration file routines:CRYPTO_internal:system lib:/AppleInternal/Library/BuildRoots/91a344b1-f985-11ee-b563-fe8bc7981bff/Library/Caches/com.apple.xbs/Sources/libressl/libressl-3.3/crypto/conf/conf_def.c:202:
rustc> failed to run: /nix/store/d4ih07nvbvvrhgsyxgy4in0b4q220wcl-cargo-bootstrap-1.76.0/bin/cargo build --manifest-path /private/tmp/nix-build-rustc-1.77.2.drv-0/rustc-1.77.2-src/src/bootstrap/Cargo.toml --frozen
rustc> Build completed unsuccessfully in 0:00:00
rustc> make: *** [Makefile:19: all] Error 1

Error happens without this pr too (nix build ".#rustc.unwrapped" --rebuild)
https://github.com/nix-community/infra/issues/1289

Been a few months so retrying.

New errors
Compiling profiler_builtins v0.0.0 (/private/tmp/nix-build-rustc-1.80.1.drv-0/rustc-1.80.1-src/library/profiler_builtins)
The following warnings were emitted during compilation:

warning<EMAIL_ADDRESS>clang-16: warning: argument unused during compilation: '-mmacos-version-min=11.0' [-Wunused-command-line-argument]
warning<EMAIL_ADDRESS>clang-16: warning: argument unused during compilation: '-arch arm64' [-Wunused-command-line-argument]
warning<EMAIL_ADDRESS>In file included from ../../src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c:24:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/errno.h:23:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/errno.h:72:
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/cdefs.h:828:2: error: Unsupported architecture
warning<EMAIL_ADDRESS>#error Unsupported architecture
warning<EMAIL_ADDRESS> ^
warning<EMAIL_ADDRESS>In file included from ../../src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c:25:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/fcntl.h:23:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/fcntl.h:78:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:33:
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/machine/_types.h:36:2: error: architecture not supported
warning<EMAIL_ADDRESS>#error architecture not supported
warning<EMAIL_ADDRESS> ^
warning<EMAIL_ADDRESS>In file included from ../../src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c:25:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/fcntl.h:23:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/fcntl.h:78:
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:55:9: error: unknown type name '__int64_t'; did you mean '__int>
warning<EMAIL_ADDRESS>typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__int128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int>
warning<EMAIL_ADDRESS>typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__int128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int>
warning<EMAIL_ADDRESS>typedef __int32_t       __darwin_dev_t;         /* dev_t */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__int128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__ui>
warning<EMAIL_ADDRESS>typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__uint128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__ui>
warning<EMAIL_ADDRESS>typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__uint128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:62:9: error: unknown type name '__uint64_t'; did you mean '__ui>
warning<EMAIL_ADDRESS>typedef __uint64_t      __darwin_ino64_t;       /* [???] Used for 64 bit inodes */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__uint128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t'
warning<EMAIL_ADDRESS>typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__ui>
warning<EMAIL_ADDRESS>typedef __uint16_t      __darwin_mode_t;        /* [???] Some file attributes */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__uint128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:71:9: error: unknown type name '__int64_t'; did you mean '__int>
warning<EMAIL_ADDRESS>typedef __int64_t       __darwin_off_t;         /* [???] Used for file sizes */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__int128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int>
warning<EMAIL_ADDRESS>typedef __int32_t       __darwin_pid_t;         /* [???] process and group IDs */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__int128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__ui>
warning<EMAIL_ADDRESS>typedef __uint32_t      __darwin_sigset_t;      /* [???] signal set */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__uint128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int>
warning<EMAIL_ADDRESS>typedef __int32_t       __darwin_suseconds_t;   /* [???] microseconds */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__int128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__ui>
warning<EMAIL_ADDRESS>typedef __uint32_t      __darwin_uid_t;         /* [???] user IDs */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__uint128_t' declared here
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__ui>
warning<EMAIL_ADDRESS>typedef __uint32_t      __darwin_useconds_t;    /* [???] microseconds */
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>note: '__uint128_t' declared here
warning<EMAIL_ADDRESS>In file included from ../../src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c:25:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/fcntl.h:23:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/fcntl.h:83:
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types/_size_t.h:31:9: error: unknown type name '__darwin_size_t'
warning<EMAIL_ADDRESS>typedef __darwin_size_t        size_t;
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>In file included from ../../src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c:25:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/fcntl.h:23:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/fcntl.h:358:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types/_timespec.h:31:
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/machine/types.h:39:2: error: architecture not supported
warning<EMAIL_ADDRESS>#error architecture not supported
warning<EMAIL_ADDRESS> ^
warning<EMAIL_ADDRESS>In file included from ../../src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c:25:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/fcntl.h:23:
warning<EMAIL_ADDRESS>In file included from /nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/fcntl.h:358:
warning<EMAIL_ADDRESS>/nix/store/apgxk5abmyyc4b8xydd5ghf4fpb5shkl-libSystem-11.0.0/include/sys/_types/_timespec.h:35:2: error: unknown type name '__darwin_time_t'
warning<EMAIL_ADDRESS>        __darwin_time_t tv_sec;
warning<EMAIL_ADDRESS>        ^
warning<EMAIL_ADDRESS>fatal error: too many errors emitted, stopping now [-ferror-limit=]
warning<EMAIL_ADDRESS>20 errors generated.

error: failed to run custom build command for `profiler_builtins v0.0.0 (/private/tmp/nix-build-rustc-1.80.1.drv-0/rustc-1.80.1-src/library/profiler_builtins)`

Split out the stdenv.targetPlatform.isLinux fix. https://github.com/NixOS/nixpkgs/pull/343059


Category widget aggregation other than count

This PR fixes #11198. It updates dependencies to bring https://github.com/CartoDB/cartodb.js/pull/1546 once it's merged.

Looks nice! Acceptance is 🍏 👏
Instructions for deploy?
I guess we need at least to reset the package.json, but not sure if we need to update any dependencies.

Yes, at least cartodb.js needs to be updated, I'm going to update the files.

🚀


Drastically increase performance when traversing large directories

What does this PR do?
Uses an optimized recursive directory walker, and other minor perf-related changes.
Motivation
Customer report: Large directories (200 GiB+) with many files (10s of thousands) will lock up the agent (including other checks).
Notes
Before: https://travis-ci.org/DataDog/integrations-core/builds/394857200#L646
After: https://travis-ci.org/DataDog/integrations-core/builds/394857766#L646
Due to the unreliable nature of Travis, it only shows a 2x increase, locally however it was 5-7x faster. Now the next slowest thing is our tag normalization, which is being addressed here https://github.com/DataDog/integrations-core/pull/1653.

I've slightly changed the title of the PR thinking about what the resulting changelog would be


Playground chat is broken in staging

Closing this as the proper issue is tracker at https://github.com/airtai/fastagency/issues/74


deps: revert google-java-format to 1.7

Fixes #452 .
(This revert would fix the problem but when RenovateBot create a pull request we'll be forgetting about the problem.)

the CIs failed :
Err:1 http://security.ubuntu.com/ubuntu focal-updates/main amd64 libxml2-utils amd64 2.9.10+dfsg-5ubuntu<IP_ADDRESS>
  404  Not Found [IP: 40.1[19](https://github.com/googleapis/java-shared-config/runs/5723184840?check_suite_focus=true#step:6:19).46.219 80]
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2-utils_2.9.10+dfsg-5ubuntu<IP_ADDRESS>_amd64.deb  404  Not Found [IP: <IP_ADDRESS> 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?


Change TreeNodeList back to struct

Reverts #17
Made a cleaner fix for serialization round-tripping that allows us to keep TreeNodeList as a struct:
All explicit constructors coerce empty and null collections of nodes to null.
Thus, the default struct ctor has the same behavior.

Internal code has to deal with null nodes appropriately.

Manual CR with @jcdickinson


next/release

:tada: This PR is included in version 1.27.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Added information on HoloLens 2 player

Docs Build status updates of commit df7e9ce:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/holographic-remoting-player.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Memory mappable ImmutableRoaring64NavigableBitmap

Is your feature request related to a problem? Please describe.
I would like to use Roaring64NavigableBitmap as an read-only data-structure that may be backed by memory mapped implementations.
Describe the solution you'd like
Add a new implementation in a package *.longlong.buffer immutable as an analogue for the mutable/immutable split of the 32bit options
I am going to try coding this up myself, but would like to know if anyone else is working on this so we could work together.
This might be a time where we can start documenting the on disk format so that it is the same between languages.

I'm quite convinced nobody is working on this yet.

This might be a time where we can start documenting the on disk format so that it is the same between languages.

This has been discussed most recently in https://github.com/RoaringBitmap/RoaringBitmap/issues/366
. There is a PR open in https://github.com/RoaringBitmap/RoaringBitmap/pull/572. I'm not sure of what would be an acceptable strategy (in the vein of https://github.com/RoaringBitmap/RoaringBitmap/issues/351).

Ok. I am going to try taking the current on java disk format as guide. and see how that goes :)

Regarding

I would like to use Roaring64NavigableBitmap as an read-only data-structure that may be backed by memory mapped implementations.

Can this be achieved with org.roaringbitmap.longlong.Roaring64NavigableMap#Roaring64NavigableMap(org.roaringbitmap.BitmapDataProviderSupplier) ?

I would like to use Roaring64NavigableBitmap as an read-only data-structure that may be backed by memory mapped implementations.

Can this be achieved with org.roaringbitmap.longlong.Roaring64NavigableMap#Roaring64NavigableMap(org.roaringbitmap.BitmapDataProviderSupplier) ?

I have been looking into this but I did not see an obvious way to do it. Because we would need to do a lot of hacking in the deserialize method to get round the issues of the fact that getHighToBitmap has default visibility.

Because we would need to do a lot of hacking in the deserialize method to get round the issues of the fact that getHighToBitmap has default visibility.

I do not get your point. I suppose a PR would make things clearer.

as an analogue for the mutable/immutable split of the 32bit options

It looks reasonnable to introduce ImmutableRoaring64NavigableBitmap as parent class of Roaring64NavigableBitmap, to follow the same design.

Because we would need to do a lot of hacking in the deserialize method to get round the issues of the fact that getHighToBitmap has default visibility.
I do not get your point. I suppose a PR would make things clearer.

I get your point after reading more Roaring code :) So using a MutableByteBuffer supplier would allow us to use off-heap buffers. But what I personally need is bytebuffers that are mapped from existing files. I think the only method that we can use to get from a saved on disk status to something accessible from java is the deserialize method that currently exists in Roaring64NavigableMap.
I now have an idea of how this can be hacked in. But a clean supported solution would be off greater benefit :)
(Side note: My problem is I have 37Gb of Roaring64NavigableMaps but only 32Gb of RAM :( so I want much of that the be paged in on demand) Second problem is that at least one of the maps is itself larger than 2gb so I can't just throw in one ByteBuffer :( but have to be a bit smarter about it.

Working on another matter, I feel relevant to add a way to add(high, RoaringBitmap) into ImmutableRoaring64NavigableBitmap. This would relate the a feature to RoaringBitmap get(high)


Error message at 2048

https://play.jsgo.io/github.com/hajimehoshi/ebiten/examples/2048
package github.com/go-gl/gl/v2.1/gl: build constraints exclude all Go files in gopath/src/github.com/go-gl/gl/v2.1/gl

websocket closed but archives not updated

That's odd since Ebiten should not require go-gl on browsers. I'd be happy if you could confirm this. Thank you!

Aaah ok so today I removed the js build tag from jsgo... It broke compatibility with go1.11... i think here:
https://github.com/hajimehoshi/ebiten/blob/063edfcc4a3d0ca5632d84910b3181d3bc81ed88/internal/opengl/context_desktop.go#L16
you need to add:
// +build !gopherjs
or
// +build !jsgo
... I'll take another look to see if I was right to remove js...

To represent GopherJS, // +bulid js && // +build !wasm should work, right?

Hang on - I'm going to have a good look through the code and work out where this is coming from... Bear with me...

OK I think I fixed it... Give this a try...

Yes, this has been fixed. Thank you!


Jumbo authentication is currently broken

Jumbo seems to have changed their API for authenticating or retrieving orders, so currently retrieving your orders is not possible through this wrapper.

I threw a Tweet towards Jumbo's Twitter account, might be worth a shot. If there's no response or nothing of use, I might be able to get hold of another route via someone who owns/runs a Jumbo supermarket.

@MWeesenaar Thank you! I get why they would add bot protection to their API though. The API is only meant to be used in combination with the mobile app.
The only work around I've found so far is to just hardcode your own Jumbo token into the wrapper since the token does not seem to have an expiry date. Although getting the token can still be quite annoying (you have to use something like mitmproxy to sniff the packets and get the token from the request headers).
If they do decide to open up their API some more, I'll see what I can do with incorporating it into the wrapper.

I haven't received a reply on my Tweet, so I have sent a formal question through their website to ask whether they are interested in talking about possible solutions for having an open(er) API available for the community, rather than the community constantly reverse engineering their mechanisms. I guess it would be a win for both ends. :)

Minor update from my side: I received two emails (16-11 and 22-11) that Jumbo had not forgotten me, but they also have not replied me with an answer yet. This morning I asked for an update. To be continued.

Minor update from my side: I received two emails (16-11 and 22-11) that Jumbo had not forgotten me, but they also have not replied me with an answer yet. This morning I asked for an update. To be continued.

Did they reply yet?

Nope still not. Last Friday I sent out a DM via Twitter to please get back to me on that dossier - but no reply there either, yet.
This Friday I might just call their call center.

Another update: I reached out via Twitter again, they cannot find my original letter towards them, so I explained, again, what my/our question is. Hopefully they might be able to respond.

Thanks for the update! With v1.2.0 you can now initialize the Jumbo object with your access token, allowing you to retrieve your orders once again. Getting your access token is not the easiest thing in the world (you'll need to reverse engineer the app), but luckily the token never expires so once you have it you should be good to go.

Great step forward :) I did a quick Google about reversing an Android app (easier than an iPhone app, I assume) and with some proper tooling it 'should be doable' - last famous words... Were you able to get the access token from the app, or by any other means?
In other news; this morning I got a reply from Jumbo, on Twitter, that they are willing to forward my request to those responsible. To be continued again.. 😄

I got my token from the Android app. You probably want to use a tool like mitmproxy to sniff the requests sent by the app and then grab the token from the X-jumbo-token header sent with every request. This will require you to remove the certificate pinning from the app to allow you to use the proxy however. Normally you could decompile the app, remove the pinning and rebuild and sign the app. However, I was only able to find .xapk files for the Jumbo app which makes this process a little more difficult.
In the end, I used a combination of root + Magisk + LSPosed to install a module that removes SSL pinning for apps you specify. However, this will require a rooted Android phone.

Just to let you know that I have been experiencing the same issue in https://github.com/peternijssen/home-assistant-jumbo.
I've trying to see if I could mimic the browser login, but no luck so far. You probably need to scrape the log in page in order to do so.
You can slightly compare it with the PostNL mentioned earlier and the work around here: https://github.com/ToonSoftwareCollective/postnl
Anyway, following the issue with interest, as I have not been able to get a hold of Jumbo myself.

Is there an update for this issue?

I have called with Jumbo three times now, they forwarded my question towards their software department already four times.. So I was thinking of just trying to find another route to get to their IT dept (e.g. via LinkedIn or anything).
I just found that they have job applications for the IT-front, so it might be worth to send out an email to one of their recruiters to try to get via that route. I must say that due to the slowness of Jumbo replying, I tend to forget to post my updates here. Please stalk me if you want me to drop any news.

@helmerzNL ; Nope, no official news. I got someone on the inside too now, it seems that it’s not much of a priority.
Maybe it is good to get more people pushing. I will ask my contact whether there are HA people within Jumbo willing to help.
Sorry that I don’t have any better news.

Let's be honest, they've implemented bot protection for a reason. There aren't going to upon up their "private" APIs to some open source projects unfortunately.

Btw, do the other supermarkets have an (open) API that you know?

I have made similar wrappers for other Dutch supermarkets: Albert Heijn, Coop, Plus and Aldi but none of them have any authentication methods included. I tried to include it in the Albert Heijn one but they are using some form of OAuth which, while not as bad as Jumbo's bot protection, I did not really want to bother with.
Keep in mind that you can still authenticate with the Jumbo API, but you'll need your authorization (JWT) token (which never expires). Getting this token however is a bit more complicated...

Yes, Jumbo moved away from long lived access tokens towards auth tokens that have to be refreshed every 24 hours somewhere around the beginning of October last year. It seems they are now using Auth0 with the standard authorization code flow. It's still possible to get a token using the auth flow but it's a significant amount of work to get it working.
I unfortunately don't have the time to add this functionality to this library, but I'm happy to point others in the right direction.
Currently, Jumbo authentication can work because it does not look like there's any bot protection. However, authenticating through the "Jumbo Extra's" app is not as easy since there's a Captcha at every request.

I haven't had contact with Jumbo on this for quite a while. They promised me to come back to me, which never occurred.
I could have another poke, soon. Will try to update this thread if I have more information.


Disable actors/groups when they're deleted in the IdP

Currently we wait for a user's session to expire before knowing that they aren't allowed to access resources, but one big customer ask for larger orgs was making this more immediate so that terminated employees and removed groups pose less of a security risk to the organization.

Relevant Slack thread

Edge case: What happens to the associated Policy when a Group is removed from IdP?

@jamilbk the group will be deleted if it's removed from IdP but it will stay if it's in IdP but has no members, basically if IdP sends is to us we will sync it.
The Policy will be hidden from UI once the group is deleted (we use inner join with where clause) and it will look like it's also deleted. If it's added to IdP again (with the same ID) it will add it back and Policy will reactivate.

Gotcha, thanks!

Closing as it looks like it already behaves the way we want it.


Create Lamdba function for the demo and deploy the latest code on it

The name of the Lambda function for the demo should be demoTuesdayLambda.
Make sure the code includes the calls to the SageMaker APIs - this is the Lambda from #143

Closes #144
Was able to create the lambda, named demoTuesdayLambda, that would've been used to make the call to the sagemaker API. Due to the sagemaker endpoints not being totally finished, I was not able to call the sage maker API. Screenshotted below is the lambda function: