kubed v0.12.0 image is not available

Looks like latest pushes to the dockerhub removed kubed:v0.12.0 tag
Reproduce:
docker pull appscode/kubed:v0.12.0
v0.12.0: Pulling from appscode/kubed
00411772422b: Pulling fs layer
invalid rootfs in image configuration

I guess they just introduced a breaking change that can break the old running deployment from our side as well.
I just got the failed deployment today, and realized that they changed the metadata of docker image v0.12.0

Hopefully, I did a workaround from my side to use another arch docker image with the same version.
➜  ~ docker pull appscode/kubed:v0.12.0_linux_amd64                                                                                <aws:employmenthero-ops-stg>
v0.12.0_linux_amd64: Pulling from appscode/kubed
9ff2acc3204b: Pull complete
dca1a2f6180a: Pull complete
Digest: sha256:5e9091f069d6fb808d0c229e81ffd65f0283851c859dfe5c48e39b3a6d0f6569
Status: Downloaded newer image for appscode/kubed:v0.12.0_linux_amd64
docker.io/appscode/kubed:v0.12.0_linux_amd64

retagging images is just bad form.

Fortunately, I did a workaround from my side to use another arch docker image with the same version.
➜  ~ docker pull appscode/kubed:v0.12.0_linux_amd64                                                                                <aws:employmenthero-ops-stg>
v0.12.0_linux_amd64: Pulling from appscode/kubed
9ff2acc3204b: Pull complete
dca1a2f6180a: Pull complete
Digest: sha256:5e9091f069d6fb808d0c229e81ffd65f0283851c859dfe5c48e39b3a6d0f6569
Status: Downloaded newer image for appscode/kubed:v0.12.0_linux_amd64
docker.io/appscode/kubed:v0.12.0_linux_amd64

If you have a helm chart it could be fixed by changing operator.tag value from v0.12.0 to v0.12.0@sha256:5e9091f069d6fb808d0c229e81ffd65f0283851c859dfe5c48e39b3a6d0f6569 so it will not be affected if v0.12.0_linux_amd64 will also be retagged by using hash instead

No comment from anyone? the images have been changed (and are no longer same format).
Did the project lose control of their docker credentials?
https://hub.docker.com/u/1gtm
updated all the images https://hub.docker.com/r/appscode/kubed/tags
changing them to type 'helm'.

I concur @donbowman, my response has been to uninstall kubed/config-syncer.  While it is a bit of a pain, I can use kustomize to install it wherever I need it.

This looks very much like a supply chain attack (albeit a clumsy one). Going to have to move away from kubed now, which is vexing.

@tamalsaha sorry to bother you. I'm also an happy user of kubed and I wonder what cause this breaking change. It seems that helm charts have been pushed using the same tags than previous docker images. Do you have details on that? Could it be reverted? Could we help somehow?

i think at this stage this is a pretty severe security breakdown.
Is there any controls so that the developer who builds something cannot push?
is multi-factor used on that end?
the project has no security policy.
e.g. here it might be benign, but its not safe to assume so.
the containers are build not from the CI, so taht is another redflag.

I have published v0.14.0-rc.0 version which used ghcr.io/appscode as the Docker registry by default. Let me know if that fixes the issue.
https://github.com/kubeops/config-syncer/actions/runs/3900336546

Thank you @tamalsaha it's working with the latest stable v0.13.2
By the way, the documentation is still mentioning v0.12.0 as latest version.

0.12.0 should have been re-released and fixed. It's fine to make a mistake and improve process, but now the app is broken for users. This has reputational harm.

I have pushed the v0.12.0 image again. This is the GH workflow run: https://github.com/kubeops/config-syncer/actions/runs/4066827902
Please comment/reopen if there are still remaining issues.


Open iptables by default

All blueprints using centos are failing if they don't specify this option manually

lgtm @aledsage  can you take a look and merge please?

LGTM; merging.


Funct rascal transformation

Using the 'rascal' tool we applied a FUNCT transformation.

I don't know why you'd send a pull request for such a thing, but obviously we can't accept a pull request whose only purpose is to thoroughly mess up and uglify the codebase. Are you doing this for other github projects? Please, pretty please, stop.


panic:Broken pipe (os error 32)

tea does not run after installation
Description
I tried to use tea on MacOS.
I got an error by removing --silent flag on tea.
Environment
MacOS 13.2.1
tea 0.25.1
Reproduction
sh <(curl https://tea.xyz) && source <("$HOME/.tea/tea.xyz/v*/bin/tea" --magic=bash)
Result
$ sh <(curl https://tea.xyz) && source <("$HOME/.tea/tea.xyz/v*/bin/tea" --magic=bash)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13557  100 13557    0     0   138k      0 --:--:-- --:--:-- --:--:--  142k

  hi 👋 let’s set up tea

  • we’ll put it here:  /Users/jwel/.tea
  • everything tea installs goes there
  • (we won’t touch anything else)

  │ docs https://github.com/teaxyz/cli#getting-started

grep: /Users/jwel/.bashrc: No such file or directory

  may we interest you in some magic?

  tea’s shell magic is optional but it’s the way it’s meant to be used.

  │ docs https://github.com/teaxyz/cli#magic

  Added:

   test -d "$HOME/.tea" && source <("$HOME/.tea/tea.xyz/v*/bin/tea" --magic=bash --silent)

  you’re all set!

  try it out:

   exec bash -i   # or open a new tab
   wget -qO- tea.xyz/white-paper | glow -

panic: spilt tea. we’re sorry and we’ll fix it… but you have to report the bug!

    https://github.com/teaxyz/cli/issues/new?title=panic%3ABroken+pipe+(os+error+32)

----------------------------------------------------->> attachment begin
BrokenPipe: Broken pipe (os error 32)
    at async file:///private/var/folders/8b/n_gmhmk57ks85swq3mrdbrwc0000gp/T/7a834af1/src/app.ts:111:13
<<----------------------------------------------------- attachment end

I believe this is the correct error message since literally running:
source <("$HOME/.tea/tea.xyz/v*/bin/tea" --magic=bash --silent)

will fail
source <("$HOME/.tea/tea.xyz/v\*/bin/tea" --magic=bash --silent)

Should succeed.
What are you trying to achieve? Running that command manually is unusual.

k so this syntax doesn’t work with bash…? How’d our tests not pick this up.

K the syntax is ok, it's just that the magic has a syntax error and our magic tests didn't pick it up

@ABevier can you figure out how this is passing and fix it? Same goes for the fish test that also was until recently passing even though it was broken. THanks

It appears that this installation was using bash 3.2 (the default version installed on osx) but tea requires bash 4.0 or higher to run.
In addition to the command_not_found_handler not being added until bash 4.0, bash 3.2 also has a bug that prevents sourcing from a process substitution:   https://lists.gnu.org/archive/html/bug-bash/2006-01/msg00018.html

users cannot upgrade bash on macOS due to System Integrity Protection.
I'm not convinced we require bash 4. Ubuntu has had a command not found handler for more 10 years and uses bash as the default.
macOS has defaulted to zsh for shell for years now and tea requires macOS >=11, so I think we can fix the immediate bug (source failed) and ignore the command not found handler not working until we get a concrete report from another user.
Or @dev-jwel can add more context.

Bash 4 is from 2009. But there might be bash 3 hacks. I remember people being super psyched when bash 4 came out, and I feel like that's when I started seeing command not found logic, but the history of package on Ubuntu shouldn't be hard to find.

@mxcl I'll add more context for future troubleshooting.
I changed default shell to bash before I trying this.
Its version was 3.2.57.
I tried tea again for both bash 5.2.15, zsh 5.8.1 and It works well.
Thanks for everyone!

Yes, it's very odd that Apple ships a version from 2014 of a major version that was superseded in early 2009. I've never understood that choice.
Source: https://ftp.gnu.org/gnu/bash/


Using sequences as default column values doesn't work

Provider Version
0.32.0
Terraform Version
v1.1.8
Describe the bug
Using sequences as default values for columns doesn't work. When adding a snowflake_sequence resource and using that as default value for a column, terraform plan shows the change but when applied the column default setting in the database doesn't change. terraform plan continues to show the change after that.
Expected behavior
The default setting of the column should include the sequence.
Code samples and commands
resource "snowflake_sequence" "sequence" {
  database = var.database_name
  schema   = var.schema_name
  name     = "sequence"
}

resource "snowflake_table" "skills" {
  database            = var.database_name
  schema              = var.schema_name
  name                = "name"

  column {
    name     = "ID"
    type     = "VARCHAR(255)"
    nullable = false

    default {
      sequence = snowflake_sequence.sequence.name
    }
  }

Hey Emanuel, this should work if you pass in snowflake_sequence.sequence.fully_qualified_name

We are closing this issue as part of a cleanup described in announcement. If you believe that the issue is still valid in v0.89.0, please open a new ticket.


feat: add inject_or

This PR removes the required parameter from Injector.inject and introduces inject_or that mimics the behavior of inject(..., required=False). The reasoning for this change is that type checkers take issue with using the result of inject(..., required=True) when the signature says that it could still be None. For example:
storage = context.inject(BaseStorage)
records = await storage.find_all_records(...)

Yields the error "find_all_records" is not a known member of "None" when using pyright, even though we know that the call will have resulted in an error if it was actually None.
This PR is mostly a proposal and does not yet fix the 109 or so instances of inject(..., required=False) that should be switched to inject_or. Open to suggestions or alternatives.

Codecov Report

Merging #1376 (e42bcf0) into main (0d79a0c) will increase coverage by 0.00%.
The diff coverage is 99.28%.

@@           Coverage Diff           @@
##             main    #1376   +/-   ##
=======================================
  Coverage   95.40%   95.40%
=======================================
  Files         476      476
  Lines       28980    29005   +25
=======================================
+ Hits        27648    27672   +24
- Misses       1332     1333    +1

Given no expressed opposition lol, I went ahead and finished the implementation on this to be able to consider these changes holistically.


Migrate beacon spec into beacon-team repository from core schemas repository

Beacon is a demonstration project built on top of the core genomics APIs. In order to distinguish it as such, the Beacon schema should likely live outside the core API schemas. I propose moving the schema here, which will also allow better separation and commentation on Beacon project issues.

It's totally possible to maintain the schema in this repo, and then a remote added to the main GA4GH schemas repo, which can be updated on request. In the Matchmaker Exchange project, we have adopted the gitflow branching methodology for keeping track of work. In the simplest form, the master branch should always be considered "stable" and ready to use. AKA where you have releases.
If you'd like to hear more about this, do feel free to let me know.

Great suggestions @Relequestual. We will put a note in the development guidelines for us to adopt Gitflow.


ncbiblast_lwp.pl suggestions

I added polljob and jobid options.

Added suggestion for psiblast:

optional inputs
specific outputs

Improvements in the clustalo.cwl:

Example of an enumeration for input argument
Optional inputs parameters and additional  inputs
Specific outputs

I am going to make a cleaner branch to merge.


Writeable canned query fails on Cloudrun

I've been banging my head against a wall for a while and would appreciate any pointers...

I have a writeable canned query to update rows in the db.
I'm using the github-oauth plugin for authentication.
I have allow set on the query to accept my GitHub id and a GH organisation.
Authentication seems to work as expected both locally and on Cloudrun -- viewing /-/actor gives the same result in both environments
I can access the 'padlocked' canned query in both environments.

Everything seems to be the same, but the canned query works perfectly when run locally, and fails when I try it on Cloudrun. I'm redirected back to the canned query page and the db is not changed. There's nothing in the Cloudstor logs to indicate an error.
Any clues as to where I should be looking for the problem?

How did you deploy to Cloud Run?
datasette publish cloudrun defaults to running databases there in -i immutable mode, because if you managed to change a file on disk on Cloud Run those changes would be lost the next time your container restarted there.
That's why I upgraded datasette-publish-fly to provide a way of working with their volumes support - they're the best option I know of right now for running Datasette in a container with a persistent volume that can accept writes: https://simonwillison.net/2022/Feb/15/fly-volumes/

If the behaviour you are seeing is because the database is running in immutable mode then that's a bug - you should get a useful error message instead!

Confirmed - this is a bug where immutable databases fail to show a useful error if you write to them with a canned query.
Steps to reproduce:
echo '
databases:
  writable:
    queries:
      add_name:
        sql: insert into names(name) values (:name)
        write: true
' > write-metadata.yml
echo '{"name": "Simon"}' | sqlite-utils insert writable.db names -
datasette writable.db -m write-metadata.yml

Then visit http://<IP_ADDRESS>:8001/writable/add_name - adding names works.
Now do this instead:
datasette -i writable.db -m write-metadata.yml

And I'm getting a broken error:

Ah, that would be it! I have a core set of data which doesn't change to which I want authorised users to be able to submit corrections. I was going to deal with the persistence issue by just grabbing the user corrections at regular intervals and saving to GitHub. I might need to rethink. Thanks!

I've wanted to do stuff like that on Cloud Run too. So far I've assumed that it's not feasible, but recently I've been wondering how hard it would be to have a small (like less than 100KB or so) Datasette instance which persists data to a backing GitHub repository such that when it starts up it can pull the latest copy and any time someone edits it can push their changes.
I'm still not sure it would work well on Cloud Run due to the uncertainty at what would happen if Cloud Run decided to boot up a second instance - but it's still an interesting thought exercise.

A more realistic solution (which I've been using on several of my own projects) is to keep the data itself in GitHub and encourage users to edit it there - using the GitHub web interface to edit YAML files or similar.
Needs your users to be comfortable hand-editing YAML though! You can at least guard against critical errors by having CI run tests against their YAML before deploying.
I have a dream of building a more friendly web forms interface which edits the YAML back on GitHub for the user, but that's just a concept at the moment.
Even more fun would be if a user-friendly for could submit PRs for review without the user having to know what a PR is!

In terms of this bug, there are a few potential fixes:

Detect the write to a immutable database and show the user a proper, meaningful error message in the red error box at the top of the page
Don't allow the user to even submit the form - show a message saying that this canned query is unavailable because the database cannot be written to
Don't even allow Datasette to start running at all - if there's a canned query configured in metadata.yml and the database it refers to is in -i immutable mode throw an error on startup

I'm not keen on that last one because it would be frustrating if you couldn't launch Datasette just because you had an old canned query lying around in your metadata file.
So I'm leaning towards option 2.

I don't think that'd work for this project. The db is very big, and my aim was to have an environment where researchers could be making use of the data, but be easily able to add corrections to the HTR/OCR extracted data when they came across problems. It's in its immutable (!) form here: https://sydney-stock-exchange-xqtkxtd5za-ts.a.run.app/stock_exchange/stocks

Nice custom template!
Yeah, for that I'd recommend hosting elsewhere - on a regular VPS (I use systemd like this: https://docs.datasette.io/en/stable/deploying.html#running-datasette-using-systemd ) or using Fly if you want to tub containers without managing a full server.

Thanks, I'll give it a try!

And in terms of the bug, yep I agree that option 2 would be the most useful and least frustrating.

Decision: I'm going to implement this:

Don't allow the user to even submit the form - show a message saying that this canned query is unavailable because the database cannot be written to

Here's what it looks like:

I put the disabled attribute on the form submission button too. There's no visible difference in how that button is displayed but since this is only a state that should be seen by the developer I think that's OK, especially combined with the warning at the top of the page.


IGNITE-3793: Need to fix dependencies and it's licenses

Fix H2 license listed at LICENSE_FABRIC.
Fix invalid H2 version in examples.
Fix incorrect JCache the license generation after the license changed to Apache 2.0
Enable Apache licenses to be shown at ignite-*-licenses.txt.

Closing PR:
IGNITE-3793 | Closed


[request] Syntax highlighting for IL assembly

It'd be pretty nice to have a syntax highlighter for .NET intermediate language files. While they're not the exactly the most common, they're used in a bunch of libraries that call opcodes not expressible from C# (example). Having syntax highlighting would make a huge difference in readability, and besides, we already have one for LLVM.

@jamesqo - do you know of any grammars (Sublime, Atom or Textmate bundles) that we could use to do this?

@arfon I found this in the MonoDevelop repo - doesn't look like it should be too hard to convert to a .plist or JSON-based grammar, but I'm not too experienced with this stuff so I wouldn't know.

To highlight files, GitHub must first be able to recognize their language using Linguist. Please see the guidelines to add support for a new language to Linguist. If IL assembly meets the requirement on in-the-wild usage, don't hesitate to open a pull request.


[Bug]: Controlled Inputs will not update when defaultValues are flattened key paths.

Which packages are impacted?

[X] remix-validated-form
[ ] @remix-validated-form/with-zod
[ ] @remix-validated-form/with-yup
[ ] zod-form-data

What version of these packages are you using?

"remix-validated-form": "^4.6.8"

Please provide a link to a minimal reproduction of the issue.
https://codesandbox.io/p/sandbox/quizzical-tess-8dgkon
Steps to Reproduce the Bug or Issue
In the linked sandbox, I have three simplified examples of updating a controlled value. When the default values are instantiated from a plain object then the update succeeds.  In the the third case, when the default value is set with flattened keys it fails to update.
Expected behavior
In the the third case, when a default value is declared with a flattened key of a nested path (ex: nested.name.first) then the update does not succeed and reverts back to its default value.  I would expect this to work with the 4.6.7 and onward (https://github.com/airjp73/remix-validated-form/issues/204).
Screenshots or Videos
No response
Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 91.1]

Additional context
No response

This is an interesting issue. It seems like we're ending up in a state where the object we store the values in looks like this:
{
  "name.first": "Default",
  "name.last": "Value",
  name: {
    first: "Foo",
    last: "Bar",
  },
}

And the default value is winning when selecting a value. Fixing this doesn't seem like an easy undertaking. I'll have to think on this a bit.

RVF v6 has been released 🎉
The guidance around how to structure default values has changed in v6 (I know you're already well into using v6 at this point haha). If it's still a problem in that version, please feel free to open a new issue.
You can find the documentation here and the migration guide here.


chore: bump zio, zio-test, zio-test-sbt from 2.0.0-RC2 to 2.0.0-RC6

Updates

dev.zio:zio
dev.zio:zio-test
dev.zio:zio-test-sbt

from 2.0.0-RC2 to 2.0.0-RC6.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "dev.zio" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequest = { frequency = "@monthly" },
  dependency = { groupId = "dev.zio" }
}]

labels: library-update, early-semver-pre-release, semver-spec-pre-release, commit-count:1

Superseded by #216.

Superseded by #226.


each function doesn't work with objects

I was just wandering whether this is a design choice, in that each only works with arrays?

Yes, while doing new unit tests, I also wondered this...
looking at the current unit tests: https://github.com/kenwheeler/cash/blob/master/test/test.js#L133-L139 it seems to be left out... although so does $.each( $("<div>") ); ... but that works... so I'm lost, @kenwheeler it'd be great to know if this is an oversight or design decision? :)

That works as it's an iterable object i.e. array-like

Here's the each function I normally use, which is quite similar to Cash's but does support objects. It wouldn't add much weight to cash. Is that something we want to support?
function each(obj,callback,thisArg) {
  var length = obj.length,
      likeArray = ( length === 0 || ( length > 0 && (length - 1) in obj ) ),
      i = 0;

  if ( likeArray ) {
    for ( ; i < length; i++ ) {
      if ( callback.call( thisArg || obj[i], obj[ i ], i, obj ) === false ) { break; }
    }
  } else {
    for (i in obj) {
      if ( callback.call( thisArg || obj[i], obj[ i ], i, obj ) === false ) { break; }
    }
  }

  return obj;
}

Something else I've noticed... jQuery.fn.each gives callbacks the arguments in the opposite order that we have: ( Integer index, Element element ) ( index first, element/object second )
In the interest of code portability, should we reverse our arguments as well?

In the interest of code portability, should we reverse our arguments as well?

yes, you absolutely should. countless plugins interbally rely on `i´ being arg 0 within jquery's $.fn.each and $.each. this subtle difference manifests itself in unhelpful debug messages rather than "call to undefined method X".
this has proven to be a showstopper for us to cash adoption. we're fine polyfilling missing methods but those that are supported should work like jquery's and support at minimum the most common signatures.

Adding support for objects to $.each seems unnecessary to me, but maybe we could also use it internally and shave some bytes? 🤔 If somebody wants to experiment with this I'll be happy to consider a PR that adds very few bytes to the bundle and basically doesn't affect performance.

In the interest of code portability, should we reverse our arguments as well?

this has proven to be a showstopper for us to cash adoption. we're fine polyfilling missing methods but those that are supported should work like jquery's and support at minimum the most common signatures.

As of v2.1.2 cash's each/map/filter pass arguments in the same order as jQuery's each/map/filter.

Implemented in #347


Is it possible to surf to a given URL (not audio/video)

Hello,
I am searching for a way to navigate with a Google-Nest-Hub to a given URL including a specific port (which is not youtube or tunein...)
The idea is to let the nest-hub open up a Chrome Window (which is available at the nest-hub) opening the URL of my home-automation (iobroker:8000) that I am able to "click" something (no keyboard-entry needed)
Is there any possibility with pychromecast?

yes it is possible but it does not seem to work with arbitrary urls (you have to try).
Example code is here:
https://github.com/robertmuth/mqtt2cast

Hi Robert, thanks for this hint! Did not know about DashCast.
Based on your code, I've made it "display" with this code:
`>>> import pychromecast

import pychromecast.controllers.dashcast as dashcast
cast = pychromecast.Chromecast(host="<IP_ADDRESS>")
cast.wait()
cast.dashcast = dashcast.DashCastController()
dc = cast.dashcast
cast.register_handler(dc)
dc.load_url("http://external_url:8082/linking/back.html#tolocalhost")`

The unexpected downside is, that it is only "displaying" the page and do not accept any interaction (clicking on lamps or other things which I can control from this page...)
Do you know a solution for this behavior ?

Ah, found it by my own^^. You just have to use the "force=True" switch when loading the Url:
dc.load_url("http://external_url:8082/linking/back.html#tolocalhost", force=True)
Now its working like I wanted it to ;-)
Thank you VERY MUCH Robert, you made my day!
Now I will get this in a short script and trigger it via IFTT with a voice command and I can make my home-automation system controllable with my Google Hub :-)


Add setup

We customized workstation-setup for use at my new company and found that we wanted to be able to update older workstations by adding new tools. Seems like this addition would be useful for more than just us.

Break into base_setup and add_setup so that additional items can be installed later
Changed MY_DIR to more descriptive WORKSTATION_SETUP_HOME

We are declaring PR bankruptcy as per #295.
Please review recent changes and direction and resubmit if needed.
Thank you!


max_iter not being passed correctly from Python

Closing this as max_iter is being passed correctly from Python. Here are some more details:
I built the C++ code on an Ubuntu VM running on a Windows laptop and then built the Python with pip install . I then ran a simplified version of test_smaller.py with just the test_small_mnist_known_cases test. I was able to log the max_iter reached by C++ and confirm that it is the same as that passed to the fit method of a KMedoids object in Python.


reorganize dart2js bots in the main waterfall

Here is the current distribution of dart2js tests in our current waterfall:
dart2js: dump-info only
dart2js-d8-hostchecked:
    d8, host-checked, all suites (default, extra, co19, observatory), 2 modes (none, checked)
dart2js-d8-minified:
    d8, host-checked, all suites, most modes (none, checked, fast-startup)
dart2js-jsshell:
  jsshell, all suites, most modes
  dart2js standalone unit tests
dart2js-linux:
  chrome, ff, all suites, most modes
chrome:
  drt linux, all suites, most modes
  drt, cps mode
  chrome in mac
safari: safari in mac, all suites, all modes
ie: ie10 and ie11 in win8, all suites, all modes
dart2js-windows:
  ie10 and ie11 in win7, all suites, all modes
  chrome and ff in win7, all suites, all modes

I propose a few changes:

Move all unit tests to their own bot:

 dart2js-unit:
  * standalone unit tests
  * dump-info and non-dump-info tests

A group of bots for non-browser runtimes like before, but divided by small/large
test suites. Small are "default" and "extra", large is "co19":

dart2js-hostchecked-d8-small
dart2js-d8-minify-small
dart2js-jsshell-small
dart2js-hostchecked-d8-large
dart2js-d8-minify-large
dart2js-jsshell-large

A group of browser bots for each architecture and size:

linux-browsers-small: chrome, ff, drt (includes csp)
mac-browsers-small: chrome, safari
win7-browsers-small: chrome, ff, ie10, ie11
win8-browsers-small: ie10, ie11
linux-browsers-large: chrome, ff, drt (includes csp)
mac-browsers-large: chrome, safari
win7-browsers-large: chrome, ff, ie10, ie11
win8-browsers-large: ie10, ie11

A group of bots for observatory alone:
observatory-linux: chrome, ff
observatory-mac: chrome, safari
observatory-win7: chrome, ie10, ie11
observatory-win8: ie10, ie11

This reorganization also means removing some tests that are not needed or
redundant:

try tests are gone

observatory + d8/jsshell/csp (they might be skipped already)

It would help to organize columns to put all the 'small' suites together, this
will give us a better picture of when failures are mainly co19 failures:

unit || d8-s | jsshell-s | linux-s | mac-s | win7-s | win8-s || d8-L | jsshell-L | linux-L | mac-L | win7-L | win8-L |

@whesse what do you think of these changes?

it might also be nice to subdivide some bots by modes/options given to dart2js (e.g. put fast-check tests on a separate column as well)


utilizing setSwipeItemHorizontalSlideAmount with proportional mode disabled during initial layout doesn't work

When using a SwipeableItemAdapter and setting a swipeItemHorizontalSlideAmount with proportional mode disabled during initial item layout (width & height == 0 of the container), the container is laid out completely beyond the maxLeftSwipeAmount.
expected layout:

actual layout:

What I think is happening is that slideToSpecifiedPositionInternal within ItemSlidingAnimator is incorrectly checking the proportional flag and deferring the initial slide process. the deferred processing then doesn't take the proportional flag into account. I believe the correct fix is to change the if conditional here: https://github.com/h6ah4i/android-advancedrecyclerview/blob/master/library/src/main/java/com/h6ah4i/android/widget/advrecyclerview/swipeable/ItemSlidingAnimator.java#L139 to testing !proportional instead of proportional.

Thanks for the issue report! Definitely, the bug is caused by the incorrect branch condition that you pointed it out.

This bug was brought by the following my commit 😭
Support swipe amount handling in pixels. (#286)

https://github.com/h6ah4i/android-advancedrecyclerview/commit/30cbc39897db4de970d017de6c31c4cb828dd1fd#diff-32f4b4fe94f138860685f0a37dd21440L139

@frett Hi. I have just released v0.10.4 that contains the fix of this bug.

@h6ah4i thanks, I'll check it out when I'm back at work on Monday!


Update collapse.directive.ts

Small API Ref description improvements

Current coverage is 84.14% (diff: 100%)

Merging #1382 into development will not change coverage

@@           development      #1382   diff @@
=============================================
  Files               73         73
  Lines             1961       1961
  Methods             17         17
  Messages             0          0
  Branches           262        262
=============================================
  Hits              1650       1650
  Misses             214        214
  Partials            97         97

Powered by Codecov. Last update a688e6a...3eec5d0


[ADD] Added bank save handling

Hey, not sure if this is the best way to go about bank saving. At some point we will be refreshing the electron app though, so some of the code here may become incompatible. Also there are a few places where configs are being changed that seem like they were versioned unintentionally

I thinks there was a mistake, i didn't mean to merge this, i don't even remember creating this PR.
Sorry for that.


Finder window closes immediately after selecting Open Folder with Fuzzy Finder

I found this when messing with #662...
Release: 0.2.8
OS: Mac Sierra: 10.12.5
Steps to recreate:

assuming issue #662 applies to the user

select directory button (?) in lower-left corner of Oni window
select home directory (i.e. /Users/me or ~/me), click Open

home directory should be visible in the lower-left now

open fuzzy finder <C-P>
select Open Folder (<C-n> then Enter)

Now the finder will open up and immediately close. I'm unsure if this and #662 are related.

Would love to help but unfortunately I don't have a Mac and Linux and Windows works perfect for me.
Does this happen when opening a folder from the menu as well? I think I used the same exact logic with QuickOpen.

Interesting, thanks for logging the issue, @someguynamedmatt ! I'm able to reproduce this too with the step you provided. Very helpful, it was interesting to note the different experiences between clicking the path vs 'Open Folder'.
I also noticed that, in the Command Palette (<C-S-P>) clicking on the 'Open Folder' option directly works, but pressing enter fails as you describe. I'm suspecting that the 'enter' key is actually picked up by OSX's Open File/Folder dialog, and dismisses it immediately. Going to test that out and see if that's the issue...

Would love to help but unfortunately I don't have a Mac and Linux and Windows works perfect for me.

Ah ya, unfortunately this looks a quirk specific to Mac - appreciate the offer though!

@extr0py What files should I check out to see where this issue might be coming from?

Hmm, I haven't been able to dig into this too much - but I suspect what's happening here is that we OSX's platform open-folder modal on the keydown event, and that experience actually listens for a keyup event!
A few options here:

A quick-and-dirty option to validate this is the issue would be to switch the event we listen to from "keydown" to "keyup" - this happens in `Keyboard.ts". This wouldn't be a viable long-term solution, because it would hurt the typing latency when editing, but it could help us at least validate the hypothesis.
A longer-term fix, if that is the issue, would be to delay showing the open folder until we receive a 'keyup' event - the open folder action is registered in Commands.ts, so we could add some logic there to hold off on opening it until we have a keyup event.
An alternative long-term fix,  would be to have some category of commands that we listen to on keyup, and have open folder be one of those commands.

There might be another possibility, like hook the keyup event in Keyboard.ts, and always prevent default/stoppropagation for the enter key in that case.
Hope that helps gives some ideas, at least! Thanks for your help with this :smile:

This was fixed by PR #681 . Thanks for the help and fix, @someguynamedmatt !


Feat/layoutable custom components

PR Checklist

[x] The PR title follows our guidelines: https://github.com/NativeScript/NativeScript/blob/master/CONTRIBUTING.md#commit-messages.
[x] There is an issue for the bug/feature this PR is for. To avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it.
[x] You have signed the CLA.
[ ] All existing tests are passing: https://github.com/NativeScript/nativescript-angular/blob/master/DevelopmentWorkflow.md#running-the-tests
[ ] Tests for the changes are included.

What is the current behavior?

Custom components are created as ProxyViewContainer.
The ProxyViewContainer ignores layout properties like row and col for GridLayout and dock from DockLayout.
What is the new behavior?

Changes custom components to be created as ContentView instead.
This makes custom components layoutable (is that the proper term?) by default.
This should minimize the number of (Stack)Layouts needed in a nativescript-angular app.
Fixes? #1422 .

BREAKING CHANGES:
The switch from ProxyViewContainer to ContentView might break existing apps.
Custom component will become layoutable and can be styled from their parent component.
This can change rendering of existing apps.

test

@VladimirAmiorkov can you share the results of the test or at least tell us if it's a feasible thing but needs more work ?

Hi @danielgek ,
I apologise for the late response. You can view the results of our internal testing here https://s3.amazonaws.com/ns-test-reports/index.html?prefix=reports/nativescript-angular/feat/layoutable-custom-components/188 or by clicking the "Details" by the failed check in the PR.

No problem, haven't noticed that details now go to a page that we can actually see !! Thanks!!

Components use ProxyViewContainer as their root element because it does not actually render a native view. Actually, ProxyViewContainer was created especially for Angular as it is common for angular application have many levels of nesting of components. Having a dedicated view for each component might lead to performance issue.
Unfortunately, this leads to the the "row/col" issue. It is quite easily solvable by adding a registerElement call in your component. You can specify exactly which layout (or ContentView) to be used as a root-view for the component:
registerElement("my-custom-component", () => { return ContentView });

@Component({
  selector: "my-custom-component",
  // ...
})
export class MyCustomComponent { ... }

You can then use the component and set the row/col properties:
<GridLayout rows="* * *" columns="* * *">
	<my-custom-component row="1" col="1"></my-custom-component>
</GridLayout>

Here is a playground showing this approach.

Yes we came to that conclusion(registerElement) on the #1422 but as i stated there, i think this should be handled on the angular package because most of the times, we need to add a outer container not only because of the row/col situation but also because it brings the support for styling the angular host component, this would greatly simplify angular work for beginners
If you feel like it's still not worth the work then this should be well documented !

Hi @danielgek ,
We have already planned on expanding the documentation with the registerElement usages in those cases specifically targeting to improve this behavior. Thank you for bringing this to our attention, we are also discussing a different way in which some of the layouting "attached" properties (like row, col) to be passed from the custom component to its child elements in order to further improve those use cases. If you have any suggestions we area always here for a discussion.
@m-abs Thank you for the work, it brought up some good questions regarding some common uses of custom components and they layouting. We hope to improve this in the future.


Syntax Errors in pandas.core.{frame,series}

mypy complains about syntax errors here:

(Double colon) https://github.com/microsoft/python-type-stubs/blob/509ad0707e5a6fcd1cc43a967a68666e6985ab07/pandas/core/series.pyi#L1391
(Indentation) https://github.com/microsoft/python-type-stubs/blob/509ad0707e5a6fcd1cc43a967a68666e6985ab07/pandas/core/frame.pyi#L955

This was fixed in #156

Alrighty! I guessed so when I saw the big PR, and did not send another PR to avoid conflicts.
Thanks a lot!


Use Rails 5 Attributes API for decimal_or_float?

The motivation for this commit originated in #1342.
@rafaelfranca mentioned that we should be using the new type_for_attribute interface
to determine the precise type of a column instead. At first I used the interface directly like
this:
      def decimal_or_float?
        if object.respond_to?(:type_for_attribute)
          type = object.type_for_attribute(column.name.to_s).type
          type == :float || :decimal
        else
          column.number? && column.type != :integer
        end
      end

I then realized that we're already asking the column object directly for the type, so
why not simple check for the two types we care about and implement the method in a much
more straightforward and backward-compatible way? I ended up with this:
      def decimal_or_float?
        column.type == :float || :decimal
      end

I could be missing something obvious but this is green in my Rails 5.0.0 app. The only
remaining issue I see is how the simple_form specs mock out the now removed number?
ActiveModel method: https://github.com/plataformatec/simple_form/blob/1c389f65594db90a1244d93a00a388b4aea4bcc6/test/support/models.rb#L3-L8
I'm gonna have to remove that most likely.
/cc @lucasmazza

My point is, we should not ask to the column anything anymore. The object that should be injected to the input should be an instance of ActiveModel::Type::Value in the versions that support it.

This should be the method that should be changed https://github.com/plataformatec/simple_form/blob/81b9d7d3439225c7613e570afb44596acfa97a76/lib/simple_form/form_builder.rb#L551-L555

@rafaelfranca Makes sense. I'll update.

@rafaelfranca There's only one major issue though. My current PR is backward-compatible and easily releasable today.
What you're asking for will require some pretty major work to keep things backward-compatible, or will require a major version bump. No?

Is the structure of the object that will be returned by type_for_attribute so different of what we have today? If not we can just call the methods conditionally. If it is we can create an adapter class to make the API the same.

Is the structure of the object that will be returned by type_for_attribute so different of what we have today? If not we can just call the methods conditionally. If it is we can create an adapter class to make the API the same.

I think we only call type and limit on the column object, so using a type object instead should be OK - every implementation of ActiveModel::Type::Value has a limit attribute and a type method on it - but I guess we should upgrade our test suite to use ActiveModel types instead our column-like structs to confirm that.

 I can follow these instructions if you want though and we can review them together to see if they make sense. That said what about backward-compatibility? I think there's definitely more work necessary to create an adapter class as @rafaelfranca mentioned.
To me that sounds a lot like new minor release whereas with a somewhat lazy patch like the one I suggest in this PR's original commit, you have simple_form working with Rails 5 and prior versions today and we can take the time to make the gem compatible with old Rails versions and the new API while updating the tests to account for both.

Only API difference I noticed then playing on the console - column_for_attribute accepts Symbols, type_for_attribute doesn't.

Yep, that originally confused me quite a bit.

Yeah, type_for_attribute intentionally lowers the API surface to only Strings. It's not meant to be called from app code directly, and we don't want HWIA all over our internals. As others have mentioned, Type objects are intentionally designed to have a somewhat similar API to what the columns used to have, but it may vary slightly.


When using Bower, it would be nice with a min version of the js file

P.S. This isn't even my project. How did I end up getting MRs on my fork of someone else's deal?
Nevertheless, whatever works.

What do you mean by this isn't your project? It is your fork... Personally I want to use this instead of the master fork as this is much lighter and I'm only interested in utf8. I would have written an issue/suggestion to you about making it a polyfill, so if the browser doesn't support TextEncoder then this would kick in instead. The result would be that if the browser natively supported TextEncoder then that would be used but if not, then this 'project' would be used instead. In that way one can write code as if all browsers has the TextEncoder implemented. I just think that this is too far from your fork as it would break existing implementations that uses the TextEncoderLite instead of TextEncoder.

Using https://jscompress.com/ the code can be compressed to 1.36 KB. Wow that is really nothing :-)

Ah, I see what I did here. I probably forked his code in the beginning but then ended up going a different direction and never unlinked the fork.
Please feel free to add the minified version. I think I sent you a collab invite.

@coolaj86 thanks for the review and merge :-) I don't see the changes. Do you need to update it there or do something?

Where do you not see the changes? github (I see them)? bower (probably need to push a git tag...)? npm (not updated yet)?

I just pushed a git tag for bower.

Cool, thanks. I see it :-)


RSS usage spikes w/ multiGet usage

Firstly, just wanted to say thanks for maintaining this project :)
Describe the bug
Similar to https://github.com/memcached/memcached/issues/895, we noticed that RSS usage increases quite a bit after enabling multiGet support on the client side. Increased to the point where the memcached pods were exceeding their memory allocations (RAM allocated for item storage + a bit of padding) and the processes were being killed off by the OOM Killer.

(left graph shows memcached_current_bytes & memcached_limit_bytes as gathered from memcached stats, right graph is the actual RSS usage on the server)
To Reproduce
We've been experiencing this across a large number of environments/sites on our infrastructure, and after reading through the code a bit and doing some testing I believe it comes down to these three main factors:

We use persistent connections client-side (PHP client), so each connection will handle a few requests before the php-fpm child worker is recycled. Notably, this means we can't $mc->close() the connections.
There is a high max_connections limit (32768).
There are "large" ascii multigets (the only thing that changed)

My current best guess is that the separate code path hits for "large ascii gets" and memory is allocated differently for the response object, and appears to be stored per request rather than per thread. Since we use persistent connections, conn_close() is called less often (not at all?) and thus rbuf_release() doesn't really get called for the extra allocated memory on the request and it ends up being claimed for a long time. And then I assume there's some connection reclaiming logic that eventually does close out connections once the max starts getting hit. But up until curr_connections reaches  max_connections, the read_buf_bytes is just ever-increasing. Then it hits it's max and stays static.
So as a result, you more or less end up with extra persistent memory usage on the server that is essentially: (size of largest multiGet) * (max_connections). Which doesn't feel like a super great usage of memory 😬.
At first I thought read_buf_mem_limit would be a good solution, but after reading through some of the code I'm not so sure anymore as it seems to handle the response buffers that are on the thread, but not the "malloc'ed ascii multiGet per-request" buffers? Let me know if I'm wrong though!
Details
Here's are some related details from a server we did some testing on:

Before changing max_connections
With max_connections set to 32768, and after waiting a while to exceed that number (total_connections), the read_buf_bytes will hit<PHONE_NUMBER> and remain static from my observations.
["uptime"] => int(58527)
["time"] => int(1686346250)
["version"] => string(6) "1.6.17"
["libevent"] => string(13) "2.1.12-stable"
["pointer_size"] => int(64)
["rusage_user"] => float(3815.643834)
["rusage_system"] => float(10106.782624)

["max_connections"] => int(32768)
["curr_connections"] => int(228)
["total_connections"] => int(46531)
["rejected_connections"] => int(0)
["connection_structures"] => int(520)

["response_obj_oom"] => int(0)
["response_obj_count"] => int(1)
["response_obj_bytes"] => int(98304)
["read_buf_count"] => int(100668)
["read_buf_bytes"] => int(1649344512)
["read_buf_bytes_free"] => int(1649229824)
["read_buf_oom"] => int(0)
["threads"] => int(6)

["bytes"] => int(4206360440)
["curr_items"] => int(7848134)
["total_items"] => int(14157074)

["cmd_get"] => int(298502468)
["cmd_set"] => int(21484210)
["bytes_read"] => int(31594629174)
["bytes_written"] => int(344009993641)
["limit_maxbytes"] => int(6442450944)

After changing max_connections
After halfing max_connections to 16384, and after waiting a while to exceed that number again (total_connections), the read_buf_bytes hits around half of the previous at 788496384 and again remains static from my observations. There's of course a bit of a time gap in these reporting so client-side data is changing but the results remain the same in that read_buf_bytes does hit a "max" and then remains there until the server crashes / is rebooted. And the amount of read buffer bytes definitely goes down alongside max_connections.
  ["uptime"] => int(54060)
  ["rusage_user"] => float(3695.635085)
  ["rusage_system"] => float(10295.950047)

  ["max_connections"] => int(16384)
  ["curr_connections"] => int(247)
  ["total_connections"] => int(40127)
  ["rejected_connections"] => int(0)
  ["connection_structures"] => int(367)

  ["response_obj_oom"] => int(0)
  ["response_obj_count"] => int(1)
  ["response_obj_bytes"] => int(98304)
  ["read_buf_count"] => int(48126)
  ["read_buf_bytes"] => int(788496384)
  ["read_buf_bytes_free"] => int(788381696)
  ["read_buf_oom"] => int(0)
  ["threads"] => int(6)

  ["bytes"] => int(3986392337)
  ["curr_items"] => int(6748437)
  ["total_items"] => int(12232185)

  ["cmd_get"] => int(297327263)
  ["cmd_set"] => int(18400080)
  ["bytes_read"] => int(59914141483)
  ["bytes_written"] => int(765239705287)
  ["limit_maxbytes"] => int(6442450944)

https://github.com/memcached/memcached/blob/e0e415b7b2b43a6ddd01a9c3ad45fb46358d526b/memcached.c#L3099 this is the normal rbuf_release() call, when the client socket goes idle, but for multigets that's not where the memory all goes.
rbuf allocations are internally used for mc_resp response objects, which are ~1k objects that describe the response for each key being fetched. If you multiget 1000 keys it will use 1000 kilobytes of response memory. Since the ascii multiget command cannot begin to return data until it has read all keys from the client, it will buffer memory forever. It does not however suffer from the TCP deadlocking issue I described in my prior comment but the tradeoff is OOM so I've never liked it much.
In the newer meta protocol this syntax doesn't exist at all, and you are required to pipeline simple commands for batch fetches.

To put this another way:

rbuf memory is only allocated from the OS when the existing pool is 100% used.
so if you're allocating enough memory to cause an OOM, you would still OOM.
we can't arbitrarily set an rbuf memory limit for users by default, for various reasons.

I can still audit for bugs, but if you are graphing or storing that information, you should see the "rbuf_bytes_free" metrics hit 0 every time the "rbuf_bytes" metric increases"

Thanks for the replies!
I think that makes sense. So is it true to say that the response buffers even for ascii multigets are supposed to be claimed per thread (not per request)? Or in other words, the "largest multiGet" will claim X memory and remain claimed forever on that thread for the remainder of it's life. But the overall claimed memory for these buffers should never exceed the (largest multiGet) * (# of threads)?
What left me with thinking this might be a bug was following the code a bit and seeing do_cache_free() called when it's time to cleanup, and that's when read_buf_mem_limit would do it's cleanup if set. But the callers don't always appear to account for the rbuf_malloced condition like we see in rbuf_release() (which is only called when connections are closed or sockets go idle). So it kinda felt like the malloced response buffer might just end up sticking around much longer in this case. (Caveat: these are heavily guesses, I don't really know C 😅).

I guess it just seems suspicious that when we decrease max_connections we also see that the max amount for read_buf_bytes decreases (observed on multiple servers). I'll see about adding some graphing to keep an eye on if rbuf_bytes_free is hitting 0.
It also takes quite a while before read_buf_bytes / rbuf_bytes_free hit their "maxes" after a server restart, it starts pretty small and just kinda keeps growing for a few hours (until seemingly total_connections >= max_connections) - I'll see about adding some graphing for that too. The request patterns for each memcached server is pretty static, as they serve web pages that are more-or-less always going to be requesting the same payloads.

It's a bit confusing because of performance requirements. Again, the caches are freed back to the pool when the connection becomes idle, not just when closed.  You can see in your stats that at rest there's no data in use.

in older code, read and write buffers were statically allocated per connection object.
these could grow a bit, but if they grew "too much" (large multiget) they would realloc back down to a limit post-command
which means 32,000 idle connections used 32000 * (sizeof(conn) + read buffer / write buffer)
with this change, an idle connection only holds sizeof(conn) memory, which saves gigabytes of RAM for people with high connection limits.
to avoid a performance loss, each worker thread manages its own pool of read buffers.
when allocating or freeing to these pools, there is no locking overhead and minimal CPU usage
there are no conditions where these pools free back to the OS.

ascii multigets throw a wrench in everything.

read buffers are statically sized at 16kb, which is more than enough memory for any kind of request (set requests with values will read data directly into the memory of the item it's about to store)
multigets must be able to buffer all inbound data, which means we can't use this rbuf pool at all for read buffers
so we switch to a direct malloc, which is then freed after the response is sent (see conn_waiting state)
this temporarily malloced read buffer is not counted in the stats counters (see below)
after a multiget is fully read from the network, it begins allocating mc_resp objects for each key being returned by memcached (this holds the VALUE line, iovec, some other data)

However, again, most of your actual memory usage for dealing with multigets is probably in the resp objects, which draw from the read buffer pool for their memory. There is one object per key that's being stored.
So now we have a temporarily malloc'ed read buffer plus non-temporarily-malloc'ed response objects. Memcached will free (globally) the read buffer when it's done but not the response objects.
I'm still open to there being some sort of bug but I'm not seeing it yet. If there was a bug you would see the read_buf_bytes_free counter staying low and it would continually alloc more memory. I also run benchmark tests with multigets and have many users with multigets and the memory doesn't seem to leak.

also the calculation isn't as simple as largest multiget * etc because if you have multiple clients running multigets at once they'll stack, which is what I assume is killing you and why it tracks with max conns?
I'll read through the code again and see if there's anything obvious that could make this worse than it should be.

Sorry for the spam I'm trying to just fire off things as I think of them :)
You can double check for an rbuf_malloced leak:

memory_malloced from slab stats

hash_bytes for the hash table

read_buf_bytes
.. should be within a few megabytes of the RSS size of the daemon. There's nearly no memory usage outside of those things outside of a couple buffers and thread stacks. If the RSS is off by hundreds of megabytes or gigabytes there could be a leak.

Definitely appreciate all the thoughts here! Up till now, there was very little discussion/info on the internet about this xD
I don't think it's a full leak per-se, since it does appear that read_buf_bytes will hit a (high) max after some time and then happily stay there. (so we are able to adjust/account for this and I think the RSS will then stay steady - I'll report back when we have new data after making adjustments to prevent the crashes). But rather was just thinking that MC might not be cleaning up as often as it could be as the read_buf_bytes does feel to initially grow rather oddly (due to us not closing connections client-side was my guess).
Here are the full stats for 4 servers that service one environment in case there's something else that'd be helpful. These were taken after we dramatically increased the allowed memory (for items) to give us more time before crashes while digging into this.

 But for simple maths and to overestimate a bit, let's assume there is a 25 key multiGet that accounts for 1mb total in server-side data (and all go to the same server for simplicity). If the read buffers you're seeing make sense for that given the average active_conns, then maybe we're all good here.

Thanks for the stats, that is always useful. (stats settings is also nice to have if available)
I think you asked about the per worker thread pools and I didn't quite answer that well enough.
Lets say you have a multiget that comes in via a random connection and fetches 5,000 keys at once:

will require a read buffer > 16kb so we will free ours and malloc a temporary buffer
once read, it will allocate 5,000 mc_resp objects
then it will respond to the client
free the 5,000 mc_resp objects back to the per thread pool
free the temporary read buffer back to the OS

This would result in the per-worker-thread cache using a bit over 5,000 kilobytes of memory after everything is done.
But, you have 6 worker threads per daemon.
So the next time the same query comes in, if it hits a different worker thread, it will allocate another 5,000 kilobytes of memory.
So your total usage will be largest_multiget_memory_requirements * worker_thread_count
which does suck but my hands are kind of tied, and this new memory model still uses less than it used to.
It's tough here because I'd love to share the memory pool (too slow) or free objects over some amount (has pathological use cases), or default the rbuf memory limit to 1% of item memory (pre-existing users could throw errors after upgrading).
It does seem like your multiget batch size is much larger than you're expecting though. I see ~100,000 rbuf objects allocated in one of your examples:
; 100000 * 15 / 6
	250000

That's an over-estimate but either you're doing one multiget with hundreds of thousands of keys, or there are multiple multigets with tens of thousands of keys running at the same time sometimes (big batch fetch from multiple clients?)

That's an over-estimate but either you're doing one multiget with hundreds of thousands of keys, or there are multiple multigets with tens of thousands of keys running at the same time sometimes (big batch fetch from multiple clients?)

I'd be quite surprised to find a multiGet with more than even 200 keys. Most definitely never more than 1000. So especially if curr_connections * (1000 key multiGet) isn't enough to get us to where we are, then I'm definitely suspect.
(For more context, we host thousands of enterprise WordPress applications. They all follow similar underlining DB & memcached patterns that are more or less predicable to the point where I'm quite sure there are not super massive multiGets - especially since this is fairly recently added to our platform and really only utilized from core WP in a few places wherein more than a couple hundred posts being queried at once is not really happening).

If you have 250 active connections and all of them are doing a 1,000 key multiget simultaneously you would arrive at that correct number. However I doubt this.
Most of the times when folks have come across this issue, they had a code bug where they were accidentally fetching the same key 50,000+ times in the same query. I don't know why it's always that class of bug but it's what it is.

So at most, the example site I've been referencing seems to spike at 306 active connections on one of the servers, and hovers around the 250:

And here are the stats again. You can tell by the uptimes how long it's been since the latest crash/restart on each. For example, cache-1 recently restarted and is at ~276mb for the read_buf_count right now. But that will inevitably keep growing up til a certain point.

Fresh stats pull
array(
["cache-0.redacted:11211" ] = array(92) {
  ["pid"] => int(112)
  ["uptime"] => int(13218)
  ["time"] => int(1686604838)
  ["version"] => string(6) "1.6.17"
  ["libevent"] => string(13) "2.1.12-stable"
  ["pointer_size"] => int(64)
  ["rusage_user"] => float(921.488048)
  ["rusage_system"] => float(1727.995956)
  ["max_connections"] => int(16384)
  ["curr_connections"] => int(253)
  ["total_connections"] => int(10999)
  ["rejected_connections"] => int(0)
  ["connection_structures"] => int(310)
  ["response_obj_oom"] => int(0)
  ["response_obj_count"] => int(2)
  ["response_obj_bytes"] => int(98304)
  ["read_buf_count"] => int(47173)
  ["read_buf_bytes"] => int(772882432)
  ["read_buf_bytes_free"] => int(772751360)
  ["read_buf_oom"] => int(0)
  ["reserved_fds"] => int(30)
  ["cmd_get"] => int(97866790)
  ["cmd_set"] => int(3433119)
  ["cmd_flush"] => int(0)
  ["cmd_touch"] => int(0)
  ["cmd_meta"] => int(0)
  ["get_hits"] => int(81244401)
  ["get_misses"] => int(16622389)
  ["get_expired"] => int(9807)
  ["get_flushed"] => int(0)
  ["delete_misses"] => int(302401)
  ["delete_hits"] => int(34862)
  ["incr_misses"] => int(70)
  ["incr_hits"] => int(463956)
  ["decr_misses"] => int(53)
  ["decr_hits"] => int(463619)
  ["cas_misses"] => int(0)
  ["cas_hits"] => int(0)
  ["cas_badval"] => int(0)
  ["touch_hits"] => int(0)
  ["touch_misses"] => int(0)
  ["store_too_large"] => int(14)
  ["store_no_memory"] => int(0)
  ["auth_cmds"] => int(0)
  ["auth_errors"] => int(0)
  ["bytes_read"] => int(9988341023)
  ["bytes_written"] => int(98416955633)
  ["limit_maxbytes"] => int(6442450944)
  ["accepting_conns"] => int(1)
  ["listen_disabled_num"] => int(0)
  ["time_in_listen_disabled_us"] => int(0)
  ["threads"] => int(6)
  ["conn_yields"] => int(0)
  ["hash_power_level"] => int(21)
  ["hash_bytes"] => int(16777216)
  ["hash_is_expanding"] => int(0)
  ["slab_reassign_rescues"] => int(61)
  ["slab_reassign_chunk_rescues"] => int(5)
  ["slab_reassign_evictions_nomem"] => int(0)
  ["slab_reassign_inline_reclaim"] => int(2)
  ["slab_reassign_busy_items"] => int(0)
  ["slab_reassign_busy_deletes"] => int(0)
  ["slab_reassign_running"] => int(0)
  ["slabs_moved"] => int(13)
  ["lru_crawler_running"] => int(0)
  ["lru_crawler_starts"] => int(12093)
  ["lru_maintainer_juggles"] => int(3526968)
  ["malloc_fails"] => int(0)
  ["log_worker_dropped"] => int(0)
  ["log_worker_written"] => int(0)
  ["log_watcher_skipped"] => int(0)
  ["log_watcher_sent"] => int(0)
  ["log_watchers"] => int(0)
  ["unexpected_napi_ids"] => int(0)
  ["round_robin_fallback"] => int(0)
  ["bytes"] => int(1853761569)
  ["curr_items"] => int(2237555)
  ["total_items"] => int(3426300)
  ["slab_global_page_pool"] => int(0)
  ["expired_unfetched"] => int(638452)
  ["evicted_unfetched"] => int(0)
  ["evicted_active"] => int(0)
  ["evictions"] => int(0)
  ["reclaimed"] => int(2809)
  ["crawler_reclaimed"] => int(867282)
  ["crawler_items_checked"] => int(457029946)
  ["lrutail_reflocked"] => int(4949844)
  ["moves_to_cold"] => int(11973571)
  ["moves_to_warm"] => int(10394293)
  ["moves_within_lru"] => int(5255254)
  ["direct_reclaims"] => int(0)
  ["lru_bumps_dropped"] => int(0)
}

["cache-1.redacted:11211" ]= array(92) {
  ["pid"] => int(101)
  ["uptime"] => int(1679)
  ["time"] => int(1686604838)
  ["version"] => string(6) "1.6.17"
  ["libevent"] => string(13) "2.1.12-stable"
  ["pointer_size"] => int(64)
  ["rusage_user"] => float(62.687671)
  ["rusage_system"] => float(305.179734)
  ["max_connections"] => int(16384)
  ["curr_connections"] => int(252)
  ["total_connections"] => int(1524)
  ["rejected_connections"] => int(0)
  ["connection_structures"] => int(257)
  ["response_obj_oom"] => int(0)
  ["response_obj_count"] => int(1)
  ["response_obj_bytes"] => int(98304)
  ["read_buf_count"] => int(16880)
  ["read_buf_bytes"] => int(276561920)
  ["read_buf_bytes_free"] => int(276447232)
  ["read_buf_oom"] => int(0)
  ["reserved_fds"] => int(30)
  ["cmd_get"] => int(8520280)
  ["cmd_set"] => int(714165)
  ["cmd_flush"] => int(0)
  ["cmd_touch"] => int(0)
  ["cmd_meta"] => int(0)
  ["get_hits"] => int(6998791)
  ["get_misses"] => int(1521489)
  ["get_expired"] => int(2645)
  ["get_flushed"] => int(0)
  ["delete_misses"] => int(3878)
  ["delete_hits"] => int(3253)
  ["incr_misses"] => int(2)
  ["incr_hits"] => int(32115)
  ["decr_misses"] => int(0)
  ["decr_hits"] => int(0)
  ["cas_misses"] => int(0)
  ["cas_hits"] => int(0)
  ["cas_badval"] => int(0)
  ["touch_hits"] => int(0)
  ["touch_misses"] => int(0)
  ["store_too_large"] => int(1)
  ["store_no_memory"] => int(0)
  ["auth_cmds"] => int(0)
  ["auth_errors"] => int(0)
  ["bytes_read"] => int(1279519896)
  ["bytes_written"] => int(10285267818)
  ["limit_maxbytes"] => int(6442450944)
  ["accepting_conns"] => int(1)
  ["listen_disabled_num"] => int(0)
  ["time_in_listen_disabled_us"] => int(0)
  ["threads"] => int(6)
  ["conn_yields"] => int(0)
  ["hash_power_level"] => int(19)
  ["hash_bytes"] => int(4194304)
  ["hash_is_expanding"] => int(0)
  ["slab_reassign_rescues"] => int(0)
  ["slab_reassign_chunk_rescues"] => int(0)
  ["slab_reassign_evictions_nomem"] => int(0)
  ["slab_reassign_inline_reclaim"] => int(0)
  ["slab_reassign_busy_items"] => int(0)
  ["slab_reassign_busy_deletes"] => int(0)
  ["slab_reassign_running"] => int(0)
  ["slabs_moved"] => int(0)
  ["lru_crawler_running"] => int(0)
  ["lru_crawler_starts"] => int(1233)
  ["lru_maintainer_juggles"] => int(981046)
  ["malloc_fails"] => int(0)
  ["log_worker_dropped"] => int(0)
  ["log_worker_written"] => int(0)
  ["log_watcher_skipped"] => int(0)
  ["log_watcher_sent"] => int(0)
  ["log_watchers"] => int(0)
  ["unexpected_napi_ids"] => int(0)
  ["round_robin_fallback"] => int(0)
  ["bytes"] => int(485391336)
  ["curr_items"] => int(491082)
  ["total_items"] => int(712253)
  ["slab_global_page_pool"] => int(0)
  ["expired_unfetched"] => int(90615)
  ["evicted_unfetched"] => int(0)
  ["evicted_active"] => int(0)
  ["evictions"] => int(0)
  ["reclaimed"] => int(2162)
  ["crawler_reclaimed"] => int(129997)
  ["crawler_items_checked"] => int(28453868)
  ["lrutail_reflocked"] => int(2966)
  ["moves_to_cold"] => int(463056)
  ["moves_to_warm"] => int(92726)
  ["moves_within_lru"] => int(26961)
  ["direct_reclaims"] => int(0)
  ["lru_bumps_dropped"] => int(0)
}

["cache-2.redacted:11211" ]= array(92) {
  ["pid"] => int(66)
  ["uptime"] => int(106459)
  ["time"] => int(1686604838)
  ["version"] => string(6) "1.6.17"
  ["libevent"] => string(13) "2.1.12-stable"
  ["pointer_size"] => int(64)
  ["rusage_user"] => float(6067.660665)
  ["rusage_system"] => float(13887.490941)
  ["max_connections"] => int(16384)
  ["curr_connections"] => int(253)
  ["total_connections"] => int(80870)
  ["rejected_connections"] => int(0)
  ["connection_structures"] => int(367)
  ["response_obj_oom"] => int(0)
  ["response_obj_count"] => int(1)
  ["response_obj_bytes"] => int(98304)
  ["read_buf_count"] => int(60999)
  ["read_buf_bytes"] => int(999407616)
  ["read_buf_bytes_free"] => int(999292928)
  ["read_buf_oom"] => int(0)
  ["reserved_fds"] => int(30)
  ["cmd_get"] => int(548432272)
  ["cmd_set"] => int(25462593)
  ["cmd_flush"] => int(0)
  ["cmd_touch"] => int(0)
  ["cmd_meta"] => int(0)
  ["get_hits"] => int(500285026)
  ["get_misses"] => int(48147246)
  ["get_expired"] => int(100796)
  ["get_flushed"] => int(0)
  ["delete_misses"] => int(391153)
  ["delete_hits"] => int(133534)
  ["incr_misses"] => int(0)
  ["incr_hits"] => int(1243)
  ["decr_misses"] => int(2)
  ["decr_hits"] => int(0)
  ["cas_misses"] => int(0)
  ["cas_hits"] => int(0)
  ["cas_badval"] => int(0)
  ["touch_hits"] => int(0)
  ["touch_misses"] => int(0)
  ["store_too_large"] => int(19)
  ["store_no_memory"] => int(0)
  ["auth_cmds"] => int(0)
  ["auth_errors"] => int(0)
  ["bytes_read"] => int(51165138854)
  ["bytes_written"] => int(550095389278)
  ["limit_maxbytes"] => int(6442450944)
  ["accepting_conns"] => int(1)
  ["listen_disabled_num"] => int(0)
  ["time_in_listen_disabled_us"] => int(0)
  ["threads"] => int(6)
  ["conn_yields"] => int(0)
  ["hash_power_level"] => int(23)
  ["hash_bytes"] => int(67108864)
  ["hash_is_expanding"] => int(0)
  ["slab_reassign_rescues"] => int(118437)
  ["slab_reassign_chunk_rescues"] => int(133)
  ["slab_reassign_evictions_nomem"] => int(0)
  ["slab_reassign_inline_reclaim"] => int(195)
  ["slab_reassign_busy_items"] => int(903)
  ["slab_reassign_busy_deletes"] => int(0)
  ["slab_reassign_running"] => int(0)
  ["slabs_moved"] => int(157)
  ["lru_crawler_running"] => int(0)
  ["lru_crawler_starts"] => int(93533)
  ["lru_maintainer_juggles"] => int(31136001)
  ["malloc_fails"] => int(0)
  ["log_worker_dropped"] => int(0)
  ["log_worker_written"] => int(0)
  ["log_watcher_skipped"] => int(0)
  ["log_watcher_sent"] => int(0)
  ["log_watchers"] => int(0)
  ["unexpected_napi_ids"] => int(0)
  ["round_robin_fallback"] => int(0)
  ["bytes"] => int(5289368519)
  ["curr_items"] => int(10189139)
  ["total_items"] => int(19930526)
  ["slab_global_page_pool"] => int(0)
  ["expired_unfetched"] => int(4987808)
  ["evicted_unfetched"] => int(0)
  ["evicted_active"] => int(0)
  ["evictions"] => int(0)
  ["reclaimed"] => int(8802)
  ["crawler_reclaimed"] => int(6898590)
  ["crawler_items_checked"] => int(4382047003)
  ["lrutail_reflocked"] => int(10950997)
  ["moves_to_cold"] => int(26000887)
  ["moves_to_warm"] => int(17974237)
  ["moves_within_lru"] => int(13197430)
  ["direct_reclaims"] => int(0)
  ["lru_bumps_dropped"] => int(0)
}

["cache-3.redacted:11211" ]= array(92) {
  ["pid"] => int(83)
  ["uptime"] => int(67712)
  ["time"] => int(1686604838)
  ["version"] => string(6) "1.6.17"
  ["libevent"] => string(13) "2.1.12-stable"
  ["pointer_size"] => int(64)
  ["rusage_user"] => float(5111.148219)
  ["rusage_system"] => float(13650.826982)
  ["max_connections"] => int(16384)
  ["curr_connections"] => int(252)
  ["total_connections"] => int(51213)
  ["rejected_connections"] => int(0)
  ["connection_structures"] => int(367)
  ["response_obj_oom"] => int(0)
  ["response_obj_count"] => int(1)
  ["response_obj_bytes"] => int(98304)
  ["read_buf_count"] => int(74652)
  ["read_buf_bytes"] => int(1223098368)
  ["read_buf_bytes_free"] => int(1222983680)
  ["read_buf_oom"] => int(0)
  ["reserved_fds"] => int(30)
  ["cmd_get"] => int(409976287)
  ["cmd_set"] => int(23456907)
  ["cmd_flush"] => int(0)
  ["cmd_touch"] => int(0)
  ["cmd_meta"] => int(0)
  ["get_hits"] => int(375162555)
  ["get_misses"] => int(34813732)
  ["get_expired"] => int(72338)
  ["get_flushed"] => int(0)
  ["delete_misses"] => int(1329547)
  ["delete_hits"] => int(115884)
  ["incr_misses"] => int(0)
  ["incr_hits"] => int(183473)
  ["decr_misses"] => int(4)
  ["decr_hits"] => int(3013)
  ["cas_misses"] => int(0)
  ["cas_hits"] => int(0)
  ["cas_badval"] => int(0)
  ["touch_hits"] => int(0)
  ["touch_misses"] => int(0)
  ["store_too_large"] => int(21)
  ["store_no_memory"] => int(0)
  ["auth_cmds"] => int(0)
  ["auth_errors"] => int(0)
  ["bytes_read"] => int(39891324498)
  ["bytes_written"] => int(411138931240)
  ["limit_maxbytes"] => int(6442450944)
  ["accepting_conns"] => int(1)
  ["listen_disabled_num"] => int(0)
  ["time_in_listen_disabled_us"] => int(0)
  ["threads"] => int(6)
  ["conn_yields"] => int(0)
  ["hash_power_level"] => int(23)
  ["hash_bytes"] => int(67108864)
  ["hash_is_expanding"] => int(0)
  ["slab_reassign_rescues"] => int(69974)
  ["slab_reassign_chunk_rescues"] => int(98)
  ["slab_reassign_evictions_nomem"] => int(0)
  ["slab_reassign_inline_reclaim"] => int(82)
  ["slab_reassign_busy_items"] => int(1)
  ["slab_reassign_busy_deletes"] => int(0)
  ["slab_reassign_running"] => int(0)
  ["slabs_moved"] => int(107)
  ["lru_crawler_running"] => int(0)
  ["lru_crawler_starts"] => int(61190)
  ["lru_maintainer_juggles"] => int(26403500)
  ["malloc_fails"] => int(0)
  ["log_worker_dropped"] => int(0)
  ["log_worker_written"] => int(0)
  ["log_watcher_skipped"] => int(0)
  ["log_watcher_sent"] => int(0)
  ["log_watchers"] => int(0)
  ["unexpected_napi_ids"] => int(0)
  ["round_robin_fallback"] => int(0)
  ["bytes"] => int(5006559690)
  ["curr_items"] => int(8298205)
  ["total_items"] => int(15405514)
  ["slab_global_page_pool"] => int(0)
  ["expired_unfetched"] => int(3507748)
  ["evicted_unfetched"] => int(0)
  ["evicted_active"] => int(0)
  ["evictions"] => int(0)
  ["reclaimed"] => int(6474)
  ["crawler_reclaimed"] => int(4862539)
  ["crawler_items_checked"] => int(3159741123)
  ["lrutail_reflocked"] => int(20407584)
  ["moves_to_cold"] => int(37410901)
  ["moves_to_warm"] => int(31237139)
  ["moves_within_lru"] => int(7253806)
  ["direct_reclaims"] => int(0)
  ["lru_bumps_dropped"] => int(0)
}
}

And then here are the stats settings:

Stats settings
STAT maxbytes<PHONE_NUMBER>
STAT maxconns 16384
STAT tcpport 11211
STAT udpport 0
STAT inter NULL
STAT verbosity 0
STAT oldest 0
STAT evictions on
STAT domain_socket NULL
STAT umask 700
STAT shutdown_command no
STAT growth_factor 1.25
STAT chunk_size 48
STAT num_threads 6
STAT num_threads_per_udp 6
STAT stat_key_prefix :
STAT detail_enabled no
STAT reqs_per_event 20
STAT cas_enabled yes
STAT tcp_backlog 1024
STAT binding_protocol auto-negotiate
STAT auth_enabled_sasl no
STAT auth_enabled_ascii no
STAT item_size_max 1048576
STAT maxconns_fast yes
STAT hashpower_init 0
STAT slab_reassign yes
STAT slab_automove 1
STAT slab_automove_ratio 0.80
STAT slab_automove_window 30
STAT slab_chunk_max 524288
STAT lru_crawler yes
STAT lru_crawler_sleep 100
STAT lru_crawler_tocrawl 0
STAT tail_repair_time 0
STAT flush_enabled yes
STAT dump_enabled yes
STAT hash_algorithm murmur3
STAT lru_maintainer_thread yes
STAT lru_segmented yes
STAT hot_lru_pct 20
STAT warm_lru_pct 40
STAT hot_max_factor 0.20
STAT warm_max_factor 2.00
STAT temp_lru no
STAT temporary_ttl 61
STAT idle_timeout 0
STAT watcher_logbuf_size 262144
STAT worker_logbuf_size 65536
STAT read_buf_mem_limit 0
STAT track_sizes no
STAT inline_ascii_response no
STAT ext_item_size 512
STAT ext_item_age<PHONE_NUMBER>
STAT ext_low_ttl 0
STAT ext_recache_rate 2000
STAT ext_wbuf_size 4194304
STAT ext_compact_under 0
STAT ext_drop_under 0
STAT ext_max_sleep 1000000
STAT ext_max_frag 0.80
STAT slab_automove_freeratio 0.010
STAT ext_drop_unread no
STAT num_napi_ids (null)
STAT memory_file (null)

Just wrote a script for that :).

To update the thread, so far it does look like there's an accidental megaget going on.
I need a better way of proving this state to end users and will think about it more; it's pretty obvious to me by combining dozens of counters together and knowing how the code works but it's pretty vague when communicated. I can't add a lot of instrumentation to the get path as that is the hottest path in the codebase.

Would maybe be slightly less-hot path to keep a counter for the max response_obj_count seen?

Makes sense!
For anyone that finds this thread in the future, it was indeed some really large multiGets (~50k up to 200k). In summary:

Really large (unbounded 🙃) DB queries would return say 200k rows of IDs.
This causes some multiGet lookups for similar volume to get related objects.
The memcached thread that was lucky enough to get the request would have to significantly allocate some memory for read/response buffers.
Later on the same type of request would happen again and maybe to a new thread this time, so it would have to allocate the same amount of memory.
Eventually all threads got the pleasure of serving the request, and the "max" would be reached which explains why read_buf_bytes would eventually become static.

For this situation, we'll be implementing client-side controls that batch up multiGets into smaller groups to keep the read buffers from growing out of control. (though of course is also ideal to fix unbounded DB queries and multiGets xD)


create buildbot for cygwin

we need a buildbot running on win7 / cygwin for CI

Its broken, re-opening

closing this in favour for feature/windows branch with appveyor :)


Error report should return a clear message

The process implemented here https://github.com/agrobioinfoservices/ClimMob-analysis/commit/e6944e043c4ea4f979bfa0818547223ee299ceaf should work fine. But the 'failed' report should return a clear error message that will help us in debug the problem.

Solved here https://github.com/agrobioinfoservices/ClimMob-analysis/commit/a07aad47dc9ccc708a6cfdd17c7e1c6024baa3f4


ARROW-8841: [C++] Add benchmark and unittest for encoding::PLAIN spaced

Benchmark(1%, 10%, 50% null probabilities) and unittest for future SIMD optimizations.
Signed-off-by: Frank Du<EMAIL_ADDRESS>

Suggestions:

Either Int32 and Int64, or Float and Double, are sufficient; I don't think both at the same time are useful (they should use the same respective code paths)
It may be nice to add tests with null probability 90% and 99%

Suggestions:

Either Int32 and Int64, or Float and Double, are sufficient; I don't think both at the same time are useful (they should use the same respective code paths)
It may be nice to add tests with null probability 90% and 99%

Thanks. Yes the int32/int64 are the same path to float/double, even for SIMD, I removed it. 90% and 99% added also.

I just stumbled on failures on MSVC in these benchmarks (see https://issues.apache.org/jira/browse/ARROW-8892). I'm fixing the immediate issues that I found but the problem of checking that benchmarks compile in CI will have to be done separately
In general, I think we should be less liberal about using auto to help avoid issues like this.

Sorry that using too many autos here, I will notice it for further PRs. Thanks. Adding more case in CI is the fix otherwise I need to setup a MSVC context locally.


Typo "Azure Monitor insights"→"Azure Monitor Insights"

https://learn.microsoft.com/en-us/azure/architecture/example-scenario/data/sql-managed-instance-cmk
https://github.com/MicrosoftDocs/architecture-center/edit/main/docs/example-scenario/data/sql-managed-instance-cmk-content.md
#PingMSFTDocs

Learn Build status updates of commit 9c85b4c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/example-scenario/data/sql-managed-instance-cmk-content.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@EdPrice-MSFT
Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#Sign-Off


update ciimage and nsimage logging

fixes: rdar://problem/41041527
fixes: rdar://problem/56880704

This looks good to me! Thanks for catching and fixing these issue!


Added consuming products on restore - Huawei.

There was a problem when consumable products was not consumed when restoring transaction. User was not able to buy consumed products because of "already owned error". There was no way to consume product. Now consumable products are consumed when restoring. The same behaviour is adopted in google billing.

@FStranieri I did it the same way as you in method onActivityResult() in case OrderStatusCode.ORDER_STATE_SUCCESS. So - these two places needs to change?

I made another commit - https://github.com/Berobasket/gdx-pay/commit/08301616eb1b3bc73e55687590385e680e400cae @FStranieri could you check if it is ok now?

For me It's OK!
Thanks!

Best is if you open another PR


Add draw, now(), immediate() to DummyContext

this creates a new abstract BaseContext that DummyContext and Context
inherit from
All methods available to Context are now stubbed by DummyContext
BaseContext is now used as the type where Context was originally used

Note: npm run test:node still fails at window undefined but node should be able to use the esm build.

@tambien this turned into a lot more churn, but I think it's a lot more robust now.
This seems to be the cleanest way to do this, although I wish there was a way to automatically implement the BaseAudioContextSubset methods in BaseContext.
Let me know what you think.

Thanks for the helpful PR!


500 error trying to use docker spawners

Sorry another issue...
Server is working fine with default configuration.
I tried to follow the docker setup and then when trying to start up server with either the temporary docker spawner or system user spawner, i get a 500 error, and the following in the terminal
 Traceback (most recent call last):
      File "/usr/local/lib/python3.3/site-packages/tornado/web.py", line 1415, in _execute
        result = yield result
      File "/home/gpillemer/jupyterhub/dockerspawner/src/jupyterhub/jupyterhub/handlers/login.py",                                                     line 66, in post
        status = yield user.spawner.poll()
      File "/usr/local/lib/python3.3/site-packages/dockerspawner/dockerspawner.py", line 221, in p                                                    oll
        container = yield self.get_container()
      File "/usr/local/lib/python3.3/site-packages/dockerspawner/dockerspawner.py", line 247, in g                                                    et_container
        'inspect_container', self.container_name
      File "/usr/local/lib/python3.3/concurrent/futures/_base.py", line 394, in result
        return self.__get_result()
      File "/usr/local/lib/python3.3/concurrent/futures/_base.py", line 353, in __get_result
        raise self._exception
      File "/usr/local/lib/python3.3/concurrent/futures/thread.py", line 54, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/usr/local/lib/python3.3/site-packages/dockerspawner/dockerspawner.py", line 209, in _                                                    docker
        return m(*args, **kwargs)
      File "/usr/local/lib/python3.3/site-packages/docker/utils/decorators.py", line 20, in wrappe                                                    d
        return f(self, resource_id, *args, **kwargs)
      File "/usr/local/lib/python3.3/site-packages/docker/client.py", line 474, in inspect_contain                                                    er
        self._get(self._url("/containers/{0}/json".format(container))),
      File "/usr/local/lib/python3.3/site-packages/docker/clientbase.py", line 86, in _get
        return self.get(url, **self._set_request_timeout(kwargs))
      File "/usr/local/lib/python3.3/site-packages/requests/sessions.py", line 477, in get
        return self.request('GET', url, **kwargs)
      File "/usr/local/lib/python3.3/site-packages/requests/sessions.py", line 465, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python3.3/site-packages/requests/sessions.py", line 573, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python3.3/site-packages/requests/adapters.py", line 415, in send
        raise ConnectionError(err, request=request)
    requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such fil                                                    e or directory'))

I suspect there is something missing in the config used to connect to docker. Can you perhaps try a simple docker-py example and see if it works? How are you running docker? Are you using TLS?

Closing this issue as it's been inactive for several months. Please feel free to open a new issue at the jupyter/help repo or reopen this issue. Thanks! :sunflower:


jlq pull request - basic change

comment added to main method

Hi @joselq Can I close this PR? If you want to propose changes to this repo please make a PR to Accenture/adop-doa-materials.

Hi @joselq Can you describe the purpose of this PR please? If it's relevant it needs to go to this repo https://github.com/Accenture/adop-doa-materials


Recreational Fishery Data Length Comp Sample Sizes

OK, for the recreational length comp sample sizes its pretty straight forward:

For the Harvest data, go to workspace BSB.Fishery.Data.For.SS. In that workspace, there is an object called rec.ab1.len. The sample sizes can be summed by year, area, and semester from the column SampleSize.
For the release data, same workspace, but go to the object rec.b2.len. The sample sizes can be summed by year, area, and semester from the column SampleSizeScaled or SampleSize (they are the same).
I am going to update the workspace in a minute, I think Sam made a modification or two since we last uploaded it.
I'll work on the other stuff in the morning.

For some CAL data points, the sample size is 0. Will this be a problem?

This shouldn't be possible, but it suggests that the length data are
imputed somehow. Easiest thing for us right now is to do a filter on the
data frame before writing to the dat file to remove rows where there were
actually no samples, but I can check what is being referenced in the
composition data. Thank you for flagging this!
On Thu, May 4, 2023 at 2:06 AM Lisa Chong @.***> wrote:

For some CAL data points, the sample size is 0. Will this be a problem?
—
Reply to this email directly, view it on GitHub
https://github.com/gavinfay/bsb-ss-2022/issues/17#issuecomment-1534139393,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB5X65E2G5ALVFUGSPGL2ULXENBQBANCNFSM6AAAAAAXNEOKWA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Gavin Fay, Ph.D.
Fisheries & Ecosystem management, Ecological modeling, Stock assessment
@.***
@gavin_fay
www.gavinfay.com

I uploaded the sample sizes for the length comps and updated the data file but there were a few issues I encountered:

I think the sample size for CT (index lengths) are missing
I also think the processing for length comps are being repeated for fleets 9-12 (rec cpue ones)? Lines 309-330 and lines 1015-1026. I commented the 1015-1026 because that was easier, but if I'm wrong, I can uncomment it back out.
I wasn't sure what to do about the sample size for commercial discards and the negative fleets (source = "CFRF")
The ESS data set has something called "LIS" but I wasn't sure what fleet this was referring to?

I will loop back on the other bullets, need to look in to those a bit more, but on your first and last bullets, "LIS" is the CT Long Island Sound trawl survey (a.k.a. fleets 17 and 18), so I think this knocks out two of those questions (bullets 1 and 4).

at some point during sample size calculations, the recreational discard length compositions have been removed from the processing output. @lidach any idea what happened here?
Also, I notice that many of the sample sizes for length comps seem unreasonably large (in the 1000s).

Still struggling to find Recreational Data sample size (or rather what the column 'SampleSize' represents.
For example, the rec.ab1.len$SampleSize are by cm. Are these number of fish sampled for each length (not likely because in some cases they are greater than the N_AB1, which seems infeasible), or are they the number of trips that were sampled that caught fish of these lengths?
If it is the latter (number of samples where fish of these lengths were measured), summing over the lengths within a year/season/area will not be the total number of trips sampled, because individual trips could (probably did) measure fish of more than 1 length - ie it is duplicating trips.
If it is the former (number of fish measured), then we still don't have the sample size information.
@jmcnamee & @lidach can you provide some illumination?
Alternatively, is there just an easy way of downloading the sample size information? (ie it doesn't have to be connected to the data, just have the sample sizes by year/season/area)

@gavinfay I'm not sure how far you got but it seemed like I made some summing mistakes for the length comps. And I'm not sure about the rec.ab1.len, can't find any information on the data Github repository

Hi @gavinfay and @lidach . On this topic (rec length sample sizes), I am communicating with Sam and I think the solution is to get the number of intercepts (closest analog to trips), it currently is number of fish sampled. I'll follow up once I get that data.

@jmcnamee @lidach Was this ever resolved?


Various fixes

This PR introduces two modifications/fixes:

Request::getHost does not include the port, which means that request matching fails for various cassettes that contain hostnames with an explicit port. See the RFC

RequestMatcher::matchHeaders fails to match requests which have the same null headers. But IMO if two requests have Accept: null that denotes equality. This is also causing cassette matching problems

Very nice Jamie, thank you for you contributions!


Fix categorical transform

Issue #, if available:
An error happens when the loaded categorical features include both integers and strings. This may happen when loading features from CSV files.
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

you need to maintain the backward compatibility

you need to maintain the backward compatibility

I have added.


Please delete async when new wallet context in sui-sdk

Steps to Reproduce Issue
The WalletContext::new method does not require any asynchronous functions, thus it does not need to be defined as an asynchronous function.
https://github.com/MystenLabs/sui/blob/d54ac2f4e1fc1cda0a846bc1fea679a9ff45882e/crates/sui-sdk/src/wallet_context.rs#L32
pub async fn new(
    config_path: &Path,
    request_timeout: Option<std::time::Duration>,
    max_concurrent_requests: Option<u64>,
) -> Result<Self, anyhow::Error> {
    let config: SuiClientConfig = PersistedConfig::read(config_path).map_err(|err| {
        anyhow!(
            "Cannot open wallet config file at {:?}. Err: {err}",
            config_path
        )
    })?;

    let config = config.persisted(config_path);
    let context = Self {
        config,
        request_timeout,
        client: Default::default(),
        max_concurrent_requests,
    };
    Ok(context)
}

Expected Result
Delete async when new one wallet context.
pub fn new(
    config_path: &Path,
    request_timeout: Option<std::time::Duration>,
    max_concurrent_requests: Option<u64>,
) -> Result<Self, anyhow::Error> {
    let config: SuiClientConfig = PersistedConfig::read(config_path).map_err(|err| {
        anyhow!(
            "Cannot open wallet config file at {:?}. Err: {err}",
            config_path
        )
    })?;

    let config = config.persisted(config_path);
    let context = Self {
        config,
        request_timeout,
        client: Default::default(),
        max_concurrent_requests,
    };
    Ok(context)
}

@stefan-mysten  how can i push forward this work.

Thanks @ccbond - I am not sure why this was required to be async. The main issue at this moment is that this is used in many parts of the code, which requires a bit of cleanup. I ran the tests on that PR and you can see that there are many failing tests.

Yes,  i do a bit of cleanup in latest version PR, maybe you can help me to review it. Thanks. @stefan-mysten

Nice work, thanks @ccbond !


AVRO-3706: accept space in folder name

What is the purpose of the change
AVRO-3706 : Make IDL accept space in folder name.
Verifying this change
This change added unit tests on compiler module and can be verified by running it.
Documentation

Does this pull request introduce a new feature?  no
If yes, how is the feature documented? not applicable

Thank you, @clesaec !


[WIP]: Add autocomplete functionality

Pull Request Test Coverage Report for Build 548

3 of 49 (6.12%)  changed or added relevant lines in 5 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-1.7%) to 84.347%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/main/java/seedu/address/ui/CommandBox.java
0
7
0.0%

src/main/java/seedu/address/logic/parser/Autocompleter.java
0
39
0.0%

Totals

Change from base Build 544:
-1.7%

Covered Lines:
1956

Relevant Lines:
2319

💛  - Coveralls


lsdsng-import directory recursion

If -f is used by lsdsng-export to put songs in folders, it would be nice to recursively check subfolders for .lsdsng files.

@urbster1 Wait, the title mentions lsdsng-import, but the description mentions lsdsng-export. Which one are we talking about?

both - lsdsng-export places lsdsng files into their own folders with the -f
option. so when calling lsdsng-import, it would be nice to have folder
input as well. otherwise you're stuck manually adding each lsdsng because
each one is in a separate folder. make sense?
On Mon, Nov 9, 2020 at 5:12 AM Stijn Frishert<EMAIL_ADDRESS>wrote:

@urbster1 https://github.com/urbster1 Wait, the title mentions
lsdsng-import, but the description mentions lsdsng-export. Which one are
we talking about?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stijnfrishert/libLSDJ/issues/83#issuecomment-723912662,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABLMCWNEOWTO2T5VJF67CXDSO657FANCNFSM4NKIPU7A
.

actually I think it does have folder input right? but it won't check
subfolders, so if you have one folder as the input but the folder contains
a set of subfolders each with a .lsdsng, none of those songs are added to
the import. that's what the issue is about IIRC
On Tue, Nov 10, 2020 at 12:52 PM Aaron U<EMAIL_ADDRESS>wrote:

both - lsdsng-export places lsdsng files into their own folders with the
-f option. so when calling lsdsng-import, it would be nice to have folder
input as well. otherwise you're stuck manually adding each lsdsng because
each one is in a separate folder. make sense?
On Mon, Nov 9, 2020 at 5:12 AM Stijn Frishert<EMAIL_ADDRESS>wrote:

@urbster1 https://github.com/urbster1 Wait, the title mentions
lsdsng-import, but the description mentions lsdsng-export. Which one are
we talking about?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stijnfrishert/libLSDJ/issues/83#issuecomment-723912662,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABLMCWNEOWTO2T5VJF67CXDSO657FANCNFSM4NKIPU7A
.

Aah right! Yeah, it does have folder support, but doesn't read them recursively.
Except for the fact that I happened to add that yesterday when refactoring some code, so I guess this is solved then 😀


New Feature: PCI config space logging

struct Device impl print_pci_config_space which log!() its hexadecimal dump of the standard part of the configuration space (the first 64 bytes).

Updated the issues


Make AsParameter property not cache Type

Fixes https://github.com/microsoft/typespec/issues/4838

API change check
API changes are not detected in this pull request.


Bugfixes for api version/url and regexps

This pull request fixes two things:

A misunderstanding over the meaning of the ^~ location option in nginx
An api version issue that made CDN-loaded cartodb.js unable to load maps

+1 @redhog I've just experienced the same issue with location regex, thanks!


'-fconcepts' gcc flag in compile_commands.json not ignored in clang10

Using clangd 9.0.1 and a compile_commands.json that states to compile a hello world file with gcc and -std=c++17 -fconcepts with neovim and coc.nvim works perfectly.
Using the same setup as above but with clangd 10.0.0 results in clangd displaying an error:
[clang drv_unknown_argument] [E] Unknown argument: '-fconcepts'
My understanding is this is simply due to gcc having the -fconcepts flag and clang not. As stated,  clangd 9.0.1 ignores this flag and doesn't display this error message which is, in my opinion, definitely the preferred behaviour.
clangd version 10.0.0
NVIM v0.4.3 with coc.nvim v0.0.74
Ubuntu 18.10

it is just an unknown compiler flag and hopefully shouldn't do any harm apart from showing a diagnostic for it. Is it effecting any other feature?
In an ideal world it would be nice to somehow suppress these diags for gcc only flags(or drop them) but i don't think we've got a complete mapping today, maybe we should start creating one.
But not surfacing diags when we can't create a compiler invocation seems like a no go as these are crucial when things go bad.

I appreciate the super fast response.
I had not considered the implications of "ignoring" it as fully as you've stated. My gcc8 manual says:
   -fconcepts
       Enable support for the C++ Extensions for Concepts Technical Specification, ISO 19217 (2015), which allows code like

               template <class T> concept bool Addable = requires (T t) { t + t; };
               template <Addable T> T add (T a, T b) { return a + b; }

I apologise if I don't understand your last sentence correctly, are you saying if clang and therefore clangd can't compile code with a flag it's dangerous to silently ignore the flag in case it changes whether or not the code should compile, or code meaning etc.?
If I understand correctly what you're saying, then as you say, unfortunately this flag most definitely can fundamentally change what constitutes a valid c++ program or not (and said program's behaviour), and so should not be ignored, or especially not silently anyway?

are you saying if clang and therefore clangd can't compile code with a flag it's dangerous to silently ignore the flag in case it changes whether or not the code should compile, or code meaning etc.?

right, it makes clang/clangd perform in a strange mode and tracing it back to compile commands is quite hard in the absence of those diags.

If I understand correctly what you're saying, then as you say, unfortunately this flag most definitely can fundamentally change what constitutes a valid c++ program or not (and said program's behaviour), and so should not be ignored, or especially not silently anyway?

you are totally right.
-fconcepts is special though, in clang they should be turned on by default with -std=c++20.
so i assumed you were just annoyed by the fact that there's a diagnostic that's not going away.

Yeh the diagnostic is just annoying, that's all. Also I had assumed it was a regression since I've never seen it before with clang9. I'll just ignore it.
Thanks for the responses.