Can't process certain cookbooks

Occasionally the cache builder will fail to process cookbooks with invalid characters in their metadata.json:
Processing metadata for 500 cookbooks with 5278 remaining on opscode
2014-04-24_18:38:09.38282 I, [2014-04-24T13:38:00.353518 #8728]  INFO -- : Saving the cache to: /etc/berkshelf/api-server/cerch
2014-04-24_18:38:09.38283 I, [2014-04-24T13:38:00.353977 #8728]  INFO -- : Cache saved!
2014-04-24_18:38:09.38284 E, [2014-04-24T13:38:09.381809 #8728] ERROR -- : Actor crashed!
2014-04-24_18:38:09.38285 Encoding::InvalidByteSequenceError: "\xE2" on US-ASCII
2014-04-24_18:38:09.38285       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/json-1.8.1/lib/json/common.rb:155:in `encode'
2014-04-24_18:38:09.38285       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/json-1.8.1/lib/json/common.rb:155:in `initialize'
2014-04-24_18:38:09.38285       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/json-1.8.1/lib/json/common.rb:155:in `new'
2014-04-24_18:38:09.38285       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/json-1.8.1/lib/json/common.rb:155:in `parse'
2014-04-24_18:38:09.38286       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/ridley-3.1.0/lib/ridley/chef/cookbook/metadata.rb:458:in `from_json'
2014-04-24_18:38:09.38286       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/ridley-3.1.0/lib/ridley/chef/cookbook/metadata.rb:29:in `from_json'
2014-04-24_18:38:09.38286       /opt/berkshelf-api/v1.3.1/lib/berkshelf/api/cache_builder/worker/opscode.rb:59:in `load_metadata'
2014-04-24_18:38:09.38286       /opt/berkshelf-api/v1.3.1/lib/berkshelf/api/cache_builder/worker/opscode.rb:39:in `block in metadata'
2014-04-24_18:38:09.38287       /opt/chef/embedded/lib/ruby/1.9.1/tmpdir.rb:83:in `mktmpdir'
2014-04-24_18:38:09.38287       /opt/berkshelf-api/v1.3.1/lib/berkshelf/api/cache_builder/worker/opscode.rb:37:in `metadata'
2014-04-24_18:38:09.38287       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/calls.rb:26:in `public_send'
2014-04-24_18:38:09.38287       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/calls.rb:26:in `dispatch'
2014-04-24_18:38:09.38288       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/calls.rb:63:in `dispatch'
2014-04-24_18:38:09.38288       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/cell.rb:60:in `block in invoke'
2014-04-24_18:38:09.38288       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/cell.rb:71:in `block in task'
2014-04-24_18:38:09.38288       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/actor.rb:362:in `block in task'
2014-04-24_18:38:09.38289       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/tasks.rb:55:in `block in initialize'
2014-04-24_18:38:09.38289       /opt/berkshelf-api/v1.3.1/vendor/bundle/ruby/1.9.1/gems/celluloid-0.16.0.pre/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'

Perhaps the opscode worker could handle this more gracefully

Not sure if this is the right thread/repo to report this, but it looks like this is the same error message, so I'm continuing the report with details.
Using Ruby 2.2.0, berkshelf-api-client 1.2.1, berkshelf 3.2.3, ridley 4.1.2.
Confirming crash from this cookbook's metadata as well as unicode in the readme being slurped in here.
 But allowing people to use their own name spelling seems like a reasonable allowance, right? :grin:
Stack:
Encoding::InvalidByteSequenceError: "\xE2" on US-ASCII
	$GEM_HOME/gems/json-1.8.2/lib/json/common.rb:155:in `encode'
	$GEM_HOME/gems/json-1.8.2/lib/json/common.rb:155:in `initialize'
	$GEM_HOME/gems/json-1.8.2/lib/json/common.rb:155:in `new'
	$GEM_HOME/gems/json-1.8.2/lib/json/common.rb:155:in `parse'
	$GEM_HOME/gems/ridley-4.1.2/lib/ridley/chef/cookbook/metadata.rb:473:in `from_json'
	$GEM_HOME/gems/ridley-4.1.2/lib/ridley/chef/cookbook/metadata.rb:29:in `from_json'
	$GEM_HOME/gems/ridley-4.1.2/lib/ridley/chef/cookbook.rb:36:in `from_path'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/cached_cookbook.rb:15:in `from_store_path'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/cookbook_store.rb:86:in `cookbook'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/cookbook_store.rb:67:in `import'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/cookbook_store.rb:30:in `import'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/installer.rb:106:in `block in install'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/downloader.rb:38:in `block in download'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/downloader.rb:35:in `each'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/downloader.rb:35:in `download'
	$GEM_HOME/gems/berkshelf-3.2.3/lib/berkshelf/installer.rb:105:in `install'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in `dispatch'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
	$GEM_HOME/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'

I've tried mucking around with the local ENV, combination of these flags:
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
export RUBYOPTS="-E utf-8"

But no luck.

(addendum) We get an older version of the berkshelf-api-client because anything newer is blocked by Berkshelf.

@miketheman what do you mean it's blocked by Berkshelf? The latest version of the API client is 1.2.1 and the constraint allows for ~> 1.2

@miketheman what do you mean it's blocked by Berkshelf? The latest version of the API client is 1.2.1 and the constraint allows for ~> 1.2

My mistake - I was comparing this repo's releases instead of the api client repo - my bad. So please ignore the (addendum).

@miketheman got ya. Yeah, the encoding stuff is still an issue that needs to be addressed.

@reset I've tried the suggested workarounds that involve environment variables - nothing worked. Ideas?

@miketheman I haven't taken the time to familiarize myself with the mess of text encoding and the Ruby VM. That's why encoding issues like this still exist. I'm not sure what the solution is because it seems to vary between versions of Ruby, different VMs, and how the data is being written or read.

Sooo..... I spent a non-trivial amount of time figuring this out.
It's not berkshelf-api's fault, it's not berkshelf's fault, nor erubis, it's not even the JSON gem's fault.
It's not Ruby's fault either.
The solutions provided above, namely export LANG=en_US.UTF-8 is likely to work in most cases.
HOWEVER, if in the event that the en_US.UTF-8 locale is not installed no amount of exporting environment variables will get you there.
Turns out, that this is a semi-known problem with docker images, as they don't always contain much other than the base system and expect you to add whatever you like on top.
Once a locale exists, you can use export to your heart'c content, for my job I only needed LANG and no other environment variables to make this work.
The associated changes to the Dockerfile can be seen here.
I thought about if there's a way to help surface this problem in any other manner, and working through the stacktrace it seems like since ridley's parser for the metadata.rb was the one that caught this.
The only thought I had was if ridley forced an encoding: 'utf-8' in the JSON.parse method, but since the json gem loads all strings into UTF-8 by default, I don't think there's much to be gained there.
I don't think there's any real action to be taken here, other than shout this from the rooftops that if you run locale -a and don't see en_US.UTF-8, you might have a bad day.

Tip for MacOS:
export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8

The rest of Chef has moved on from the JSON gem, and I'm wondering if we should move this from JSON too.  @lamont-granquist, I dimly recall you were involved in that, can you shed some light / do you know who might be able to?

@jkeiser @lamont-granquist which gem is Chef using now?

@jkeiser As I think I figured out a bit above - it's not the JSON gem's faukt - it's the underlying system not having the locale installed.

@reset looks like YAJL FFI: https://github.com/chef/chef/blob/master/chef.gemspec#L25
@miketheman it could be that I've done my search wrong, but I'm not seeing this bug pop up with Chef, which also has to read cookbooks--this makes me wonder if the gem we switched to works around the issue in some way.

@reset the main reason for moving to the YAJL gem was that the error messages were a lot more user-friendly.  @daneilsdeleo was just regaling me with tales of that woe :)

@jkeiser that's entirely possible. I don't recall digging deeper into the Chef codebase for this one - but it boiled down to when running a JSON.parse without having any locales installed in a minimal docker image.
See here: https://github.com/DataDog/docker-jenkins-slave/commit/ab85c65ed231322fe6a20d795ba493cf02cbafb6
It's possible that the same behavior would be exhibited by Chef under the same circumstances - unless it's creating it's own locale settings.

I suspect that @lamont-granquist is likely to have some opinions on how Chef interacts with locales - it appears that there's now some messaging to tell you to go and get one.
https://github.com/chef/chef/pull/2425/files#diff-ce8cfba0deb02929653a78f574664565R649

@miketheman awesome find.    Easiest way is if someone just doesn't like the way ls -la sorts output in the en_US.UTF_8 locale with dotfiles all scattered through the listing alphabetically -- and what people do is export LC_ALL=C or something like that and then they break UTF-8 for everything they run -- which works for 'muricans until they try to parse something with an umlaut in someone's name and then they think the software they ran is buggy (and that is exactly how I ran for years myself as a proud 'murican).  We also ran into people running on windows in Japan with some kind of strange non-UTF-8 charset which they had to have default on the system but which broke knife upload as well.
So, anyway, it turned out that we couldn't really fight the latter battle, and while we could fight the former battle that went nowhere because while locales aren't actually all that hard, its just hard enough that users don't care.  So what Chef 12 does now is force the default external encoding to UTF-8:
https://github.com/chef/chef/blob/68bab0b362c47e5a75492e57094f72ecee1171ae/lib/chef/application.rb#L193-L195
https://github.com/chef/chef/blob/master/lib/chef/config.rb#L698-L705
I still sort of think its philosophically wrong to be doing that, but in practice nobody seems to care and UTF-8 is mostly taking over the world, and by standing ground you just wind up in a mutual firing squad with users on one side and software devs on the other.   Plus if anyone ever does write a cookbook or whatever in Shift-JIS then they can still put a magic comment at the start of it to change the encoding.
If you look further into what Chef does with locales there's also this huge pile of code here which I do not think you need:
https://github.com/chef/chef/blob/master/lib/chef/config.rb#L646-L696
That has to do with the fact that we explicitly need an English locale because in many cases in providers we parse the output of commands and grep for localized strings and if the system is in de_DE.UTF-8 you'll get back German which will break the providers -- so we need to force external commands to English and UTF-8 (except for when service providers are firing up someone's software which executes in the system locale).   I don't think you need that.
So, TL;DR add a line:
Encoding.default_external = Encoding::UTF_8

Make it configurable for bonus points, but chances are nobody will care.

@jkeiser asked me to post this.  It started out as a chef-provisioning issue https://github.com/chef/chef-provisioning/issues/190, where Vagrant very early in its Berks vendoring throws the encoding exception out of the JSON gem -- all components up to date from chefdk 0.4.0, embedded ruby ruby 2.1.4p265,  vagrant 1.7.1.  @jkeiser tracked it down here; per our discussion, I looked at the locales of both my own machine, and the Centos 6.4 box I'm throwing up in Vagrant.  Interesting:
Here’s the host machine (mine, OSX 10.9)
PHUSCA-L24566:cookbooks treffda1$ env | grep LANG
LANG=en_US.UTF-8
PHUSCA-L24566:cookbooks treffda1$ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=
PHUSCA-L24566:cookbooks treffda1$ locale -a | grep en_US.UTF-8
en_US.UTF-8
PHUSCA-L24566
Here’s a vagrant instance on that machine (our vanilla Centos 6.4 image)
[vagrant@localhost ~]$ env | grep LANG
LANG=en_US.UTF-8
[vagrant@localhost ~]$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
[vagrant@localhost ~]$ locale -a | grep en_US.UTF-8
[vagrant@localhost ~]$ locale -a | grep en_US.utf8
en_US.utf8
[vagrant@localhost ~]$
Well well well.  Locale  en_US.UTF-8 isn’t installed; however, en_US.utf8 is.   Haven't yet tried setting the locale in /etc/sysconfig/i18n in the Vagrant instance and re-provisioning, that's next.

I am also getting this error using Chef 12.1.1 and Ruby 2.1.0 while running tests with docker.
I didn't want to clutter the comment but I posted my output here:
http://pastebin.com/W1sqXnK8

HOWEVER, if in the event that the en_US.UTF-8 locale is not installed no amount of exporting environment variables will get you there.
Turns out, that this is a semi-known problem with docker images, as they don't always contain much other than the base system and expect you to add whatever you like on top.

Had the same problem in a docker environment.
export LANG=C.UTF-8

Worked for me as a quick fix.

C.UTF-8 isn't portable across O/Sen.  The only locale that I think is portable is the non-UTF8 'C' locale (kind of the root of this issue).
The ruby code in my TL;DR up there is portable though.

That's why I mentioned it's a quick fix for me (and perhaps for others). Patching the ruby files wasn't an option in my case but I have to agree it is a nice(r) solution.

I had the same error, and fixed it with:
export LANG=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8

$ chef -v
Chef Development Kit Version: 0.6.0
chef-client version: 12.3.0
berks version: 3.2.4
kitchen version: 1.4.0

So this is still an issue and we encounter it when trying to fetch cookbooks using Berkshelf:
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.2/lib/json/common.rb:155:in `encode': "\xE2" on US-ASCII (Encoding::InvalidByteSequenceError)
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.2/lib/json/common.rb:155:in `initialize'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.2/lib/json/common.rb:155:in `new'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.2/lib/json/common.rb:155:in `parse'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.1.2/lib/ridley/chef/cookbook/metadata.rb:473:in `from_json'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.1.2/lib/ridley/chef/cookbook/metadata.rb:29:in `from_json'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.1.2/lib/ridley/chef/cookbook.rb:36:in `from_path'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cached_cookbook.rb:15:in `from_store_path'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cookbook_store.rb:108:in `block in cookbooks'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cookbook_store.rb:98:in `collect'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cookbook_store.rb:98:in `cookbooks'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/resolver/graph.rb:8:in `populate_store'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/resolver.rb:73:in `resolve'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/installer.rb:175:in `install_from_universe'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/installer.rb:39:in `run'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:374:in `install'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:583:in `block in vendor'
        from /opt/chefdk/embedded/lib/ruby/2.1.0/tmpdir.rb:88:in `mktmpdir'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:581:in `vendor'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:387:in `vendor'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in `dispatch'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
        from /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in `execute!'
        from /opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in `<top (required)>'
        from /usr/bin/berks:43:in `load'
        from /usr/bin/berks:43:in `<main>'

Is this an issue with Berkshelf or the JSON gem?  Who do we look to in order to set a default LANG?
When I run through CodeDeploy, LANG is not set.  I'm not a fan of any application that cannot function (with reasonable defaults) without environment variables set as it breaks unattended execution.

@eric-tucker you need to learn how to set the locale of your system.  if you don't learn how to do that this is just the first of many issues you will have.  its like saying "i'm not a fan of any application that needs me to correctly setup DNS in /etc/resolv.conf, why can't it just work right?".  the locale is by definition external config that you have to apply because you're the one that knows the right locale and character set.  and what is right for you is wrong for someone in Europe using ISO-8851-1

@lamont-granquist I agree.  I'm actually just asking for a reasonable default to allow unattended executions to function.  Of course there is a work-around so this isn't exactly a blocking issue.
I just think that with automation and unattended execution becoming more and more common place that relying on variables that are normally only set when a user is logged in is impeding progress from a DevOps standpoint.  Again, just my opinion.

From a systems engineering standpoint, you need to learn how to configure your locale.  Its your responsibility.  Every much your responsibility as configuring DNS, NTP, LDAP, your postgresql settings, etc.  And if CodeDeploy isn't setting LANG based on your system settings then it is the broken piece of software.  We're well into the 21st century now and to configure a server you need to care about localization, and if you can't learn how to do that you're simply not doing your job.
Just my opinion.

@lamont-granquist I'm setting the locale in my CodeDeploy appspec.  This is how I can get the gem to work.
I feel like you're talking personal offence to this or arguing that I'm incompetent for feeling that environment variables should not be required for the gem to function.  Admittedly, I don't understand the deep logic of how the gem works, so a logical explanation why this application needs to have a value set for the locale is acceptable.
I admit I am biased against environment variables.  I'd rather see configuration placed in a file once and be done with it rather than having to set a bunch of environment variables every time I want to execute a program.  Of course there are exceptions.  However, to quote your example regarding /etc/resolv.conf, well that's a file.  Imagine if you had to set an environment variable to configure your DNS server for any application that had to do a DNS lookup.  That sounds crazy (at least to me).
If you feel that this is Amazon's fault for not having the LANG set during CodeDeploy (it actually is set when CloudFormation runs) then I will happily file a ticket with them.  The value is obviously getting set somewhere when CloudFormation runs and not because of a user logging into the system.  So this very well could be a deficiency in CodeDeploy.

Yes, the environment variables are absolutely necessary.  This is like how many apps fail when $HOME is not set correctly.  We now have to add the locale set of variables to that.  You can argue about that all you want but the design of that interface is done and its pretty futile to complain about environment variables being the interface.  They are the interface, and the design of that interface has nothing at all to do with berkshelf.  And yes, this is a bug that needs to get fixed in CodeDeploy and this is just the first of many issues that they'll have by breaking localization variables.

And yes I'm taking personal offense to your statement that "I'm not a fan of any application, [...etc...]" followed up by just blind bias and a demanding tone.  If you think it can be done better, than submit a PR request and show us how to do it.

@lamont-granquist I was going to thank you for the thoughtful interface explanation as I finally see your point.  I was about to agree with you.  Then you had to follow-up with that last statement.
What did I do to provoke you?  Where have I been "demanding"?  Have I demanded that a default be set?  No.  I've asked if it would be possible and who we would look to in order to accomplish that if indeed possible.
You came in here stated that I had to learn how to do things, that I'm not doing my job, and giving me all sorts of sarcastic tone.  All because it is my opinion that things should work (minimally) without environment variables.  If my opinion is incorrect then factually demonstrate why rather than simply stating I need to learn to do things properly for the 21st century and the old line of "if you think you can do better do it yourself."
I'm not going to get into a flame war on GitHub on an issue that remains open for whatever reason.  If the solution is that things are working as intended and that a locale must be set via environment variables then the issue should be closed and discussion should stop and we move on.

@eric-tucker a default being applied along with a warning being issued that the app is falling back to a default that may not be appropriate is what I think would close this issue

On Yosemite, setting LC_ALL is the only variation that did the trick:
export LC_ALL=en_US.UTF-8

In my case changing LANG wasn't an option. Resolve my problem editing Berksfile as follows:
Encoding.default_external = "UTF-8"
source 'https://supermarket.chef.io'
metadata
After this, just executed berks install without any problem

I hit this in docker in an unrelated project and was able to solve it via:
RUN     /usr/sbin/locale-gen en_US.UTF-8
ENV     LANG en_US.UTF-8

Identified and found this issue today. Usage of "export LC_CTYPE=en_US.UTF-8" resolved the issue. Prior to that, "env | Grep LC_CTYPE" was yielding no results.
Thank you all.

@guillaumemaka thanks :)

Weird, berks verify works with LC_CTYPE=en_US.UTF-8 but fails with LC_CTYPE=fr_FR.UTF-8...
In more details:
$ export LC_CTYPE=en_US.UTF-8
$ berks verify
Verifying (40) cookbook(s)...
Verified.
$ export LC_CTYPE=fr_FR.UTF-8
-bash: warning: setlocale: LC_CTYPE: cannot change locale (fr_FR.UTF-8)
$ berks verify
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.3/lib/json/common.rb:155:in `encode': "\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.3/lib/json/common.rb:155:in `initialize'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.3/lib/json/common.rb:155:in `new'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/json-1.8.3/lib/json/common.rb:155:in `parse'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.5.1/lib/ridley/chef/cookbook/metadata.rb:570:in `from_json'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.5.1/lib/ridley/chef/cookbook/metadata.rb:29:in `from_json'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.5.1/lib/ridley/chef/cookbook.rb:36:in `from_path'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cached_cookbook.rb:15:in `from_store_path'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cookbook_store.rb:108:in `block in cookbooks'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cookbook_store.rb:98:in `collect'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cookbook_store.rb:98:in `cookbooks'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cookbook_store.rb:163:in `satisfy'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/dependency.rb:139:in `cached_cookbook'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/lockfile.rb:116:in `block in trusted?'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/lockfile.rb:101:in `each'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/lockfile.rb:101:in `trusted?'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/berksfile.rb:737:in `validate_lockfile_trusted!'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/berksfile.rb:683:in `verify'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cli.rb:404:in `verify'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cli.rb:52:in `dispatch'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/lib/berkshelf/cli.rb:27:in `execute!'
        from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/berkshelf-4.3.5/bin/berks:5:in `<top (required)>'
        from /usr/bin/berks:45:in `load'
        from /usr/bin/berks:45:in `<main>'
$ export LC_CTYPE=en_US.UTF-8
$ berks verify
Verifying (40) cookbook(s)...
Verified.
$ berks --version
4.3.5

Hope this helps

Forget my previous comment. There was an issue with my locale setup.
Sorry for the noide


Refine the storage test cases (Priority=2) on HyperV

Move the Get-DeviceName() to CommonFunctions.psm1
Replace the hard code like $sd = "sdc" with Get-DeviceName()
In STORAGE-DiffDiskGrowth.ps1, there is 50% chance that resource disk is operated as the differencing disk in previous code.

A full storage test is ongoing.

Update the codes to support redhat/centos.

The test results with ubuntu 18.04:
[LISAv2 Test Results Summary]
Test Run On           : 08/14/2019 03:19:33
VHD Under Test        : D:\xhx\vhds\Ubuntu_18.04.1.vhdx
Test Category         : Functional
Test Area             : STORAGE
Initial Kernel Version: 4.15.0-38-generic
Final Kernel Version  : 5.2.5-2519374d2a6b
Total Test Cases      : 41 (39 Passed, 2 Failed, 0 Aborted, 0 Skipped)
Total Time (dd:hh:mm) : 0:3:24

   ID TestArea             TestCaseName                                                                TestResult TestDuration(in minutes)
-------------------------------------------------------------------------------------------------------------------------------------------
    1 STORAGE              STORAGE-TAKE-REVERT-SNAPSHOT                                                      PASS                 3.08
    2 STORAGE              STORAGE-EXPORT-IMPORT-VM                                                          PASS                14.78
    3 STORAGE              STORAGE-VHDX-VERIFY-SECTORSIZE                                                    PASS                 2.37
    4 STORAGE              VHDX-RESIZE-TO-BIGGER512                                                          PASS                 3.37
    5 STORAGE              VHDX-RESIZE-TO-BIGGER4096                                                         PASS                 3.39
    6 STORAGE              VHDX-RESIZE-TO-BIGGER512-FIXED-IDE                                                PASS                 4.03
    7 STORAGE              VHDX-RESIZE-TO-BIGGER512-DYNAMIC-IDE                                              PASS                 4.08
    8 STORAGE              VHDX-RESIZE-TO-SMALLER512                                                         PASS                 3.16
    9 STORAGE              VHDX-RESIZE-TO-SMALLER4096                                                        PASS                 2.96
   10 STORAGE              VHDX-RESIZE-GROWSHRINK-SCSI                                                       PASS                 4.41
   11 STORAGE              VHDX-RESIZE-GROWSHRINK-SCSI-OFFLINE                                               PASS                 4.94
   12 STORAGE              VHDX-RESIZE-OVER2TB-SCSI                                                          PASS                 9.26
   13 STORAGE              VHDX-RESIZE-OVER2TB-IDE                                                           PASS                 9.32
   14 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-SCSI-ONLINE                                        PASS                 9.07
   15 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-SCSI-OFFLINE                                       PASS                10.89
   16 STORAGE              VHDX-RESIZE-GROWSHRINK-IDE                                                        PASS                    5
   17 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-IDE                                                PASS                10.48
   18 STORAGE              STORAGE-VHD-IDE-DYNAMIC                                                           PASS                 3.56
   19 STORAGE              STORAGE-VHD-IDE-FIXED                                                             PASS                  7.2
   20 STORAGE              STORAGE-VHD-ADD-DIFF-DISK-IDE                                                     PASS                 4.35
   21 STORAGE              STORAGE-VHD-ADD-DIFF-DISK-SCSI                                                    PASS                 4.47
   22 STORAGE              STORAGE-VHD-MULTI-DISC-SCSI                                                       PASS                 3.59
   23 STORAGE              STORAGE-VHD-HOT-ADD-MULTI-DYNAMIC-SCSI                                            PASS                  2.7
   24 STORAGE              STORAGE-VHD-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                         PASS                 2.37
   25 STORAGE              STORAGE-SCSI-DYNAMIC-LARGE-DISK                                                   PASS                 3.77
   26 STORAGE              STORAGE-VHDX-IDE-DYNAMIC                                                          PASS                 3.84
   27 STORAGE              STORAGE-VHDX-IDE-FIXED                                                            PASS                 3.72
   28 STORAGE              STORAGE-VHDX-ADD-DIFF-DISK-IDE                                                    PASS                 4.48
   29 STORAGE              STORAGE-VHDX-ADD-DIFF-DISK-SCSI                                                   PASS                 4.48
   30 STORAGE              STORAGE-VHDX-MULTI-DISC-SCSI                                                      PASS                 7.34
   31 STORAGE              STORAGE-VHDX-HOT-ADD-MULTI-DYNAMIC-SCSI                                           PASS                 3.43
   32 STORAGE              STORAGE-VHDX-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                        PASS                  2.4
   33 STORAGE              STORAGE-VHDX-HOT-PLUG-UNPLUG-SCSI                                                 PASS                 3.81
   34 STORAGE              STORAGE-VHDX-4k-MULTI-DYNAMIC-SCSI                                                PASS                 3.41
   35 STORAGE              STORAGE-VHDX-4K-HOT-ADD-MULTI-DYNAMIC-SCSI                                        PASS                 3.66
   36 STORAGE              STORAGE-VHDX-4K-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                     PASS                 2.55
   37 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LARGE-16TB                                              PASS                 3.73
   38 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LARGE-64TB                                              PASS                 4.29
   39 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LOCAL-COPY-FILE                                         PASS                 3.66
   40 STORAGE              VHDX-RESIZE-GROW-FILESYSTEM-512                                                   PASS                 4.44
   41 STORAGE              VHDX-RESIZE-GROW-FILESYSTEM-4096                                                  PASS                 4.82

Test results with centos 7.5:
[LISAv2 Test Results Summary]
Test Run On           : 08/14/2019 06:32:34
VHD Under Test        : C:\Users\v-hoxian\Centos_7.5.vhdx
Test Category         : Functional
Initial Kernel Version: 3.10.0-862.el7.x86_64
Final Kernel Version  : 4.19.52-0d5c4fe9932e
Total Test Cases      : 41 (41 Passed, 0 Failed, 0 Aborted, 0 Skipped)
Total Time (dd:hh:mm) : 0:2:42

   ID TestArea             TestCaseName                                                                TestResult TestDuration(in minutes)
-------------------------------------------------------------------------------------------------------------------------------------------
    1 STORAGE              STORAGE-TAKE-REVERT-SNAPSHOT                                                      PASS                 2.86
    2 STORAGE              STORAGE-EXPORT-IMPORT-VM                                                          PASS                14.62
    3 STORAGE              STORAGE-VHDX-VERIFY-SECTORSIZE                                                    PASS                 2.13
    4 STORAGE              VHDX-RESIZE-TO-BIGGER512                                                          PASS                  2.5
    5 STORAGE              VHDX-RESIZE-TO-BIGGER4096                                                         PASS                  2.5
    6 STORAGE              VHDX-RESIZE-TO-BIGGER512-FIXED-IDE                                                PASS                 3.58
    7 STORAGE              VHDX-RESIZE-TO-BIGGER512-DYNAMIC-IDE                                              PASS                 3.78
    8 STORAGE              VHDX-RESIZE-TO-SMALLER512                                                         PASS                 2.64
    9 STORAGE              VHDX-RESIZE-TO-SMALLER4096                                                        PASS                 2.59
   10 STORAGE              VHDX-RESIZE-GROWSHRINK-SCSI                                                       PASS                 3.73
   11 STORAGE              VHDX-RESIZE-GROWSHRINK-SCSI-OFFLINE                                               PASS                 4.63
   12 STORAGE              VHDX-RESIZE-OVER2TB-SCSI                                                          PASS                 5.11
   13 STORAGE              VHDX-RESIZE-OVER2TB-IDE                                                           PASS                  4.7
   14 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-SCSI-ONLINE                                        PASS                 4.09
   15 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-SCSI-OFFLINE                                       PASS                 5.95
   16 STORAGE              VHDX-RESIZE-GROWSHRINK-IDE                                                        PASS                  4.8
   17 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-IDE                                                PASS                 5.71
   18 STORAGE              STORAGE-VHD-IDE-DYNAMIC                                                           PASS                 3.27
   19 STORAGE              STORAGE-VHD-IDE-FIXED                                                             PASS                 3.17
   20 STORAGE              STORAGE-VHD-ADD-DIFF-DISK-IDE                                                     PASS                 4.13
   21 STORAGE              STORAGE-VHD-ADD-DIFF-DISK-SCSI                                                    PASS                 4.13
   22 STORAGE              STORAGE-VHD-MULTI-DISC-SCSI                                                       PASS                 3.26
   23 STORAGE              STORAGE-VHD-HOT-ADD-MULTI-DYNAMIC-SCSI                                            PASS                 2.41
   24 STORAGE              STORAGE-VHD-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                         PASS                 2.15
   25 STORAGE              STORAGE-SCSI-DYNAMIC-LARGE-DISK                                                   PASS                 3.54
   26 STORAGE              STORAGE-VHDX-IDE-DYNAMIC                                                          PASS                 3.27
   27 STORAGE              STORAGE-VHDX-IDE-FIXED                                                            PASS                 3.27
   28 STORAGE              STORAGE-VHDX-ADD-DIFF-DISK-IDE                                                    PASS                 4.15
   29 STORAGE              STORAGE-VHDX-ADD-DIFF-DISK-SCSI                                                   PASS                 4.14
   30 STORAGE              STORAGE-VHDX-MULTI-DISC-SCSI                                                      PASS                 3.17
   31 STORAGE              STORAGE-VHDX-HOT-ADD-MULTI-DYNAMIC-SCSI                                           PASS                 3.26
   32 STORAGE              STORAGE-VHDX-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                        PASS                 2.14
   33 STORAGE              STORAGE-VHDX-HOT-PLUG-UNPLUG-SCSI                                                 PASS                 3.71
   34 STORAGE              STORAGE-VHDX-4k-MULTI-DYNAMIC-SCSI                                                PASS                 3.28
   35 STORAGE              STORAGE-VHDX-4K-HOT-ADD-MULTI-DYNAMIC-SCSI                                        PASS                 3.31
   36 STORAGE              STORAGE-VHDX-4K-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                     PASS                 2.15
   37 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LARGE-16TB                                              PASS                 3.56
   38 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LARGE-64TB                                              PASS                 3.48
   39 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LOCAL-COPY-FILE                                         PASS                 3.52
   40 STORAGE              VHDX-RESIZE-GROW-FILESYSTEM-512                                                   PASS                  4.1
   41 STORAGE              VHDX-RESIZE-GROW-FILESYSTEM-4096                                                  PASS                  4.1

Test results with ubuntu 18.04:
[LISAv2 Test Results Summary]
Test Run On           : 08/14/2019 03:19:33
VHD Under Test        : D:\xhx\vhds\Ubuntu_18.04.1.vhdx
Test Category         : Functional
Test Area             : STORAGE
Initial Kernel Version: 4.15.0-38-generic
Final Kernel Version  : 5.2.5-2519374d2a6b
Total Test Cases      : 41 (41 Passed, 0 Failed, 0 Aborted, 0 Skipped)
Total Time (dd:hh:mm) : 0:3:24

   ID TestArea             TestCaseName                                                                TestResult TestDuration(in minutes)
-------------------------------------------------------------------------------------------------------------------------------------------
    1 STORAGE              STORAGE-TAKE-REVERT-SNAPSHOT                                                      PASS                 3.08
    2 STORAGE              STORAGE-EXPORT-IMPORT-VM                                                          PASS                14.78
    3 STORAGE              STORAGE-VHDX-VERIFY-SECTORSIZE                                                    PASS                 2.37
    4 STORAGE              VHDX-RESIZE-TO-BIGGER512                                                          PASS                 3.37
    5 STORAGE              VHDX-RESIZE-TO-BIGGER4096                                                         PASS                 3.39
    6 STORAGE              VHDX-RESIZE-TO-BIGGER512-FIXED-IDE                                                PASS                 4.03
    7 STORAGE              VHDX-RESIZE-TO-BIGGER512-DYNAMIC-IDE                                              PASS                 4.08
    8 STORAGE              VHDX-RESIZE-TO-SMALLER512                                                         PASS                 3.16
    9 STORAGE              VHDX-RESIZE-TO-SMALLER4096                                                        PASS                 2.96
   10 STORAGE              VHDX-RESIZE-GROWSHRINK-SCSI                                                       PASS                 4.41
   11 STORAGE              VHDX-RESIZE-GROWSHRINK-SCSI-OFFLINE                                               PASS                 4.94
   12 STORAGE              VHDX-RESIZE-OVER2TB-SCSI                                                          PASS                 9.26
   13 STORAGE              VHDX-RESIZE-OVER2TB-IDE                                                           PASS                 9.32
   14 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-SCSI-ONLINE                                        PASS                 9.07
   15 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-SCSI-OFFLINE                                       PASS                10.89
   16 STORAGE              VHDX-RESIZE-GROWSHRINK-IDE                                                        PASS                    5
   17 STORAGE              VHDX-RESIZE-GROWOVER2TB-SHRINK-IDE                                                PASS                10.48
   18 STORAGE              STORAGE-VHD-IDE-DYNAMIC                                                           PASS                 3.56
   19 STORAGE              STORAGE-VHD-IDE-FIXED                                                             PASS                  7.2
   20 STORAGE              STORAGE-VHD-ADD-DIFF-DISK-IDE                                                     PASS                 4.35
   21 STORAGE              STORAGE-VHD-ADD-DIFF-DISK-SCSI                                                    PASS                 4.47
   22 STORAGE              STORAGE-VHD-MULTI-DISC-SCSI                                                       PASS                 3.59
   23 STORAGE              STORAGE-VHD-HOT-ADD-MULTI-DYNAMIC-SCSI                                            PASS                  2.7
   24 STORAGE              STORAGE-VHD-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                         PASS                 2.37
   25 STORAGE              STORAGE-SCSI-DYNAMIC-LARGE-DISK                                                   PASS                 3.77
   26 STORAGE              STORAGE-VHDX-IDE-DYNAMIC                                                          PASS                 3.84
   27 STORAGE              STORAGE-VHDX-IDE-FIXED                                                            PASS                 3.72
   28 STORAGE              STORAGE-VHDX-ADD-DIFF-DISK-IDE                                                    PASS                 4.48
   29 STORAGE              STORAGE-VHDX-ADD-DIFF-DISK-SCSI                                                   PASS                 4.48
   30 STORAGE              STORAGE-VHDX-MULTI-DISC-SCSI                                                      PASS                 7.34
   31 STORAGE              STORAGE-VHDX-HOT-ADD-MULTI-DYNAMIC-SCSI                                           PASS                 3.43
   32 STORAGE              STORAGE-VHDX-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                        PASS                  2.4
   33 STORAGE              STORAGE-VHDX-HOT-PLUG-UNPLUG-SCSI                                                 PASS                 3.81
   34 STORAGE              STORAGE-VHDX-4k-MULTI-DYNAMIC-SCSI                                                PASS                 3.41
   35 STORAGE              STORAGE-VHDX-4K-HOT-ADD-MULTI-DYNAMIC-SCSI                                        PASS                 3.66
   36 STORAGE              STORAGE-VHDX-4K-HOT-REMOVE-MULTI-DYNAMIC-SCSI                                     PASS                 2.55
   37 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LARGE-16TB                                              PASS                 3.73
   38 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LARGE-64TB                                              PASS                 4.29
   39 STORAGE              STORAGE-VHDX-SCSI-DYNAMIC-LOCAL-COPY-FILE                                         PASS                 3.66
   40 STORAGE              VHDX-RESIZE-GROW-FILESYSTEM-512                                                   PASS                 4.44
   41 STORAGE              VHDX-RESIZE-GROW-FILESYSTEM-4096                                                  PASS                 4.82

@juhlee-microsoft Hongxiong fixed all places.


[GUI] Toggleable switch

Like the ones on iOS and Android.

I assume that you mean to toggle between desktop and touch?

@MrSchism I was thinking more of this:

@MrSchism I've added this gui element in the latest commit. The Settings application use it now:)


Deleting a certificate resource does not delete its secret

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug
What happened:
When you delete a certificate resource, the secret containing the actual certificate will not be deleted.
What you expected to happen:
I think it's safe to assume that deleting a certificate resource means the user does not want to keep the secret around, or wants to force a new certificate to be requested. For instance, when switching a ClusterIssuer from letsencrypt-staging to letsencrypt-prod.
How to reproduce it (as minimally and precisely as possible):

Make sure you have a valid Issuer.
Create a certificate resource or let the ingress-shim create it.
Run kubectl get secrets -o yaml to check if the secret has been created
Delete the certificate
Run kubectl get secrets -o yaml; the secret persists
Recreate the certificate, or wait until the ingress-shim recreates it
Run kubectl get secrets -o yaml and compare its contents; it hasn't been overwritten.

Environment:

Kubernetes version (use kubectl version): 1.8.6
Cloud provider or hardware configuration**: AWS
Install tools: kops

I'm been a big user of kube-cert-manager which deletes Secrets when the Certificate or Ingress that triggered its creation is deleted. I would say though this is a feature, and not a bug. It is a desirable feature and I agree it would nice to add to cert-manager.
It has some wrinkles to implement. When the user deletes a Certificate, all that cert-manager has is the name of the deleted Certificate. So how does it know which Secret to delete? kcm relied on a small database to track that, cert-manager would need to annotate or label Secrets with the name of the Certificate used to create it. Then when a Certificate is deleted, check all Secrets to see if any were created by the deleted Certificate.
From experience, when an Ingress is deleted, it is not always desirable to delete the associated Certificates and Secrets. At least not in a hurry. Often the Ingress is going to be recreated a few seconds later.
@munnerz I would suggest for cert-manager that Secrets have a certmanager.k8s.io/certificate-name: annotation for the Certificate that created it, and a certmanager.k8s.io/delete-with-certificate: true annotation by default. And that Certificate resources have an optonal cascadeDeleteSecrets: true/false field that determines whether the created Secret get true or false for the certmanager.k8s.io/delete-with-certificate: true.
For ingress-shim the Ingress would have a certmanager.k8s.io/cascade-delete-secrets: true/false annotation that would determine if certificates created by it would have true or false for cascadeDeleteSecrets.
The cert-manager should use use deletes on Certificates as well as a (not very frequent) reconciliation timer to trigger a reconciliation of Secrets, checking each Secret. If the Secret has delete-with-certificate: true then check if the named Certificate exists, if it does not, then delete the Secret.

This could be achieved through the use of Kubernetes garbage collection more easily, by simply setting the OwnerReference of the secret to the Certificate resource.
I can see the desire for such a feature, but I can also see why someone may not want it. I think we could make this a field on a Certificate resource to toggle the behaviour, perhaps with a cluster wide default.
Defaulting is something we've touched on before, and I'm still not certain the best way we should handle it. With the new MutatingWebhookConfiguration resource type, we could potentially allow users to do this out-of-tree instead of requiring us to keep adding fields for every default/toggle under the sun...

/kind feature

Also see #87

The problem seems to still be present, can we reopen this issue?

Apparently this has been fixed by #819

Yay! Looks like is has been fixed, but you have to manually enabled the fix by adding the enable-certificate-owner-ref option in the extraArgs setting. (So yeah, not obvious nor documented AFAIK!) This enables clean-up of orphaned Secrets.
I personally think this should be the default behavior as I commented above you year. Let me make two more arguments for that:
The Secret is derivative, generated data, created when you create a Certificate. If I create a Certificate resource, then delete a Certificate resource, the system should be in the same state as when I started. There shouldn't be garbage left behind.
An ACME Secret without a controlling Certificate is just garbage. It won't be renewed because the Certicate is gone. If you are worried about causing outages if someone deletes a Certificate by mistake @munnerz, then consider that if you don't delete the Secret right away, then all you have achieved is setting an outage 'time bomb' for the user in 1-90 days 😄

@whereisaaron it'd be best to open a new issue to track this, as this being closed will not be very discoverable! 😄

So yeah, not obvious nor documented AFAIK!

Not as far as I'm aware 😬

Cheers @munnerz - I don't actually want to resurface the argument, just leaving it here for posterity 👍
It would be a breaking change now we're not alpha. I'm happy I have the option, and now I know how to enable it. Once I have tested it a bit I'll look to patch the chart info or may propose an named chart option for it.

Updating the chart would be great as I'm not really a fan of having to --set "extraArgs={--enable-certificate-owner-ref=true}"
But I can confirm that the fix is working so far.


Density based clustering algorithms

https://en.m.wikipedia.org/wiki/Cluster_analysis#Density-based_clustering

Moved to the projects list of proposals


New-CsMediaConfiguration and Set-CsMediaConfguration Pages missing data

The parameter EnableRtpRtcpMultiplexing has no context on both pages, simply listing as a True/False toggle. No indication on how media is bundled together etc.

There parameter EnableVideoBasedSharing on the Set-CsMediaConfiguration page hasn't been filled in at all. No indication of what switching VBSS on or off would do.

@officedocsbot assign @yogkumgit

Hello @Atreidae James. How are you? 👍 Thanks for reporting this.
I created a pr proposing a description for both parameters, if it gets approved, they will be reflected on the online version of those articles.
Thanks

Oh, I didn't realise that we can submit PR's
Looks like it got merged, I'm cool to close this issue now.
Thanks.


create aria alert service

This creates a new service for triggering aria alerts. It also uses this service for language detection.
This PR fixes #129698

This looks good. Might it be nicer to merge this with the existing accessibilityServce?
Downside is that the accessibilityService seems to live in common and electron-sandbox. So the common implementation would need to be moved to browser.
@TylerLeonhardt I leave this up to you.
Also fyi @sbatten as he added the initial accessibiilityService

💯  for accessibility service! I was actually trying to find a different service we could simply add this into 👍

Sorry I haven't done much service management work across targets... How could I have an electron-browser target and a browser target of the same service? Are they registered differently or perhaps I'm just extending the browser impl with the Electron-browser impl?

Yeah, if you "monkey see, monkey do" from these it should be fine:

src/vs/workbench/services/textfile/browser/textFileService.ts
src/vs/workbench/services/textfile/browser/browserTextFileService.ts
src/vs/workbench/services/textfile/electron-sandbox/nativeTextFileService.ts

Awesome thanks! I'll go off of that.

I'm gonna merge this in @sbatten feel free to give me comments whenever.


customDelete performed very much slow in IE

i am using cutomDelete method for delete resource. its working lightning fast on chrome and firefox but on IE its taking much time.

Closing due to the nature and age of issue. Please consider the updated issue guidelines and reopen if needed.


Proposal: Allow humanizeDuration to work on stringified numbers

When you have a stringified number as a label, it would be nice to be able to use the same humanizeDuration logic on it.
Example: label_name: "interval": "5400" => "1h 30m"

You can't apply a Go function that takes in a float type to a string instead, so there's no way to make that work.
Given that labels are opaque strings I would either suggest changing the label value, or having a time series which has this as both a value and label value and working from there.

What are you trying to achieve?

Hello
We have looked at this during our bug scrub and decided to close this issue.
This is not feasible with go templating, so it is out of scope for prometheus. Please consider the above workaround. Thanks!

Hello
We have looked at this during our bug scrub and decided to close this issue.
This is not feasible with go templating, so it is out of scope for prometheus. Please consider the above workaround. Thanks!

That's not quite true. I have created #8339. This is not to say we have to provide the feature. But technically, it's possible. So we need a "real" decision here.

That's not quite true. I have created #8339. This is not to say we have to provide the feature. But technically, it's possible. So we need a "real" decision here.

Thanks @beorn7 , I did forget that this was indeed custom functions. I am in favour of this, and to extend to other functions. Especially useful alerting on histograms.

Thanks @beorn7 , I did forget that this was indeed custom functions. I am in favour of this, and to extend to other functions. Especially useful alerting on histograms.

I'm not sure it's a good idea to be doing this sort of type magic, it's not very Go and could get weird when you look at the various functions we have and all the potential type conversions. I think any type conversions should be explicit, not implicit magic.
I'd also like to hear more about the original use case, we currently only have one sentence.

I'm not sure it's a good idea to be doing this sort of type magic, it's not very Go and could get weird when you look at the various functions we have and all the potential type conversions. I think any type conversions should be explicit, not implicit magic.
I'd also like to hear more about the original use case, we currently only have one sentence.

So by explicit, you mean we could have a type conversion function?

So by explicit, you mean we could have a type conversion function?

Yeah, though it'd be parsing rather than type conversion as we're not exactly casting here.

Yeah, though it'd be parsing rather than type conversion as we're not exactly casting here.

I'd also like to hear more about the original use case, we currently only have one sentence.

I am alerting when newly encountered objects have a time to live that is too short (e.g. increase(metric_expiration_seconds_bucket{le="54611.0"}[15m] > 0). When alerting for this, it is useful that the alert contains a readable amount of time such as by humanizeDuration: 56411.0s is not readable, 15h10m is.

Would you be open into supporting int as well? We have threshold that we'd like to show in the alerts that might be float or int :
      expr: avg_over_time(aws_rds_freeable_memory_average{dbinstance_identifier=~"${config.db_instance_identifier}.*"}[1d]) < ${config.db_ram_minimum_freeable_memory_bytes_low}
      for: 5m
      annotations:
        description: ${config.db_instance_identifier} is approaching the minimum acceptable threshold for freeable memory. Current freeable memory = {{ $value | humanize}}, threshold = {{ ${config.db_ram_minimum_freeable_memory_bytes_low} | humanize}}.
        summary: Database freeable memory is approaching minimum

Here we have some template coming from terraform but basically ${config.db_ram_minimum_freeable_memory_bytes_low} is sometimes a float, sometimes an int depending on the result of the computation :
db_ram_minimum_freeable_memory_bytes_low = 0.075 * var.instance_max_ram

I could convert everything into a string I guess but if we support float, I can't see why we shouldn't support int as well.
Thanks

Prometheus only knows floats. Even if it is an integral number, it is still represented as a float.

Not sure who's seeing int then but given the error message I got, it's not only floats :
<error expanding template: error executing template __alert_xxx-redacted: template: __alert_xxx-redacted:1:242: executing "__alert_xxx-redacted" at <humanize>: wrong type for value; expected float64; got int>

For others that might end up here, I've solved it by using terraform format function {{ ${format("%.2f, config.your_variable_here)} | humanize}}, that will make any number a float in the yaml.

I see. That number is not coming from Prometheus, but it's inserted into your YAML by terraform. And apparently, a plain integer number in Go templating is parsed as an int, which then collides with humanize only expecting a string ar a float64.
This is probably very niche, but I don't see any reason to not let humanize and humanizeDuration cover that case, too.

Great news thanks, I'll submit a proper issue + PR in the next days, unless you want to do it yourself. In that case, let me know.
Thanks

PRs are always welcome! (@roidelapluie please shout if you don't agree with my assessment here).

It's fine for me :)


fix(curriculum): advanced node express changes for new boilerplate

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Affected Issues:
Closes #38882
#38837 (will not close)
Closes #38064
Closes #36281
Closes #17820
Closes #17781
Closes #17704
Closes #36050
Affected PRs: #38979

These changes solve many other issues, once coupled with the new boilerplate linked: https://github.com/freeCodeCamp/boilerplate-advancednode/pull/11

Also, if you don't mind, I can push a commit to update all the Glitch links in these challenges on Repl.it.

Also, if you don't mind, I can push a commit to update all the Glitch links in these challenges on Repl.it.

@scissorsneedfoodtoo , No, I do not mind. Absolutely, go for it.
We also need to discuss how to update the gists with the code up to this point...

@Sky020, awesome, sounds great. I'll work on updating those links and can try working through the project again to get some of those Gists in place. I'll need to check which account we should have that under.

@RandellDawson, could you take a look at this?

@scissorsneedfoodtoo and @RandellDawson . To help, I have started creating the gists: https://gist.github.com/Sky020
It is just easier this way for you to QA, and work off of. I should have them all finished, soon.

Thanks @Sky020, that's a huge. I started on those last week, but only got to the first few of checkpoints (Set up Passport, and Implement the Serialization of a Passport User).
Are you thinking of having a link to a Gist in each challenge? Either way is totally fine, it would just be a bit more to change if we make modifications in the future.
Either way, if you'd like to continue creating the Gists, I can start adding them to one of our accounts and start updating the links.

@scissorsneedfoodtoo

Are you thinking of having a link to a Gist in each challenge?

I will leave this up to you. I am creating one for each, but I can fully understand some of the lessons are simple/laid-out enough that they do not require one.
Just a heads-up, I am busy changing the Registration of New Users lesson tests, to mimic those of the Exercise Tracker project, so that users do not need to delete the database record, every time they want to run the tests.

@scissorsneedfoodtoo Before I commit these test changes, does this look reasonable?
tests:
  - text: You should register route and display on home.
    testString: 'getUserInput => $.get(getUserInput(''url'')+ ''/_api/server.js'') .then(data => { assert.match(data, /showRegistration:( |)true/gi, ''You should be passing the variable "showRegistration" as true to your render function for the homepage''); assert.match(data, /register[^]*post[^]*findOne[^]*username:( |)req.body.username/gi, ''You should have a route accepted a post request on register that querys the db with findone and the query being "username: req.body.username"''); }, xhr => { throw new Error(xhr.statusText); })'
  - text: Registering should work.
    testString: "async getUserInput => {
      const url = getUserInput('url');
      const user = `freeCodeCampTester${Date.now()}`;
      const xhttp=new XMLHttpRequest();
      xhttp.onreadystatechange = function() {
        if (this.readyState == 4 && this.status == 200) {
          test(this);
        } else {
          throw new Error(`${this.status} ${this.statusText}`);
        }
      };
      xhttp.open('POST', url+'/register', true);
      xhttp.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
      xhttp.send(`username=${user}&password=${user}`);

      function test(xhttpRes) {
        const data = xhttpRes.responseText;
        assert.match(data, /Profile/gi, 'Register should work, and redirect successfully to the profile.');
      }
    }
    "
  - text: Login should work.
    testString: "async getUserInput => {
      const url = getUserInput('url');
      const user = `freeCodeCampTester${Date.now()}`;
      const xhttpReg = new XMLHttpRequest();
      xhttpReg.onreadystatechange = function() {
        if (this.readyState == 4 && this.status == 200) {
          login();
        } else {
          throw new Error(`${this.status} ${this.statusText}`);
        }
      };
      xhttpReg.open('POST', url+'/register', true);
      xhttpReg.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
      xhttpReg.send(`username=${user}&password=${user}`);

      function login() {
        const xhttp=new XMLHttpRequest();
        xhttp.onreadystatechange = function() {
          if (this.readyState == 4 && this.status == 200) {
            test(this);
          } else {
            throw new Error(`${this.status} ${this.statusText}`);
          }
        };
        xhttp.open('POST', url+'/login', true);
        xhttp.setRequestHeader('Content-type', 'application/x-www-form-urlencoded');
        xhttp.send(`username=${user}&password=${user}`);
      }
      function test(xhttpRes) {
        const data = xhttpRes.responseText;
        assert.match(data, /Profile/gi, 'Login should work if previous test was done successfully and redirect successfully to the profile.');
        assert.match(data, new RegExp(user, 'g'), 'The profile should properly display the welcome to the user logged in');
      }
    }
    "

@Sky020, awesome! Looks great. I was also playing around with that yesterday but ran into some CORS issues with fetch. Your changes look good to me. Go ahead and push them up and I'll test that challenge out locally.
About the Gists, it wouldn't hurt to have one for each challenge if you're already in the process. I'm sure anyone who has trouble with a step would appreciate the extra help, and we could always remove some for the very easy challenges later.

@scissorsneedfoodtoo AN IMPORTANT POINT: We need to include somewhere within these lessons:

If a user is using a tool like Repl.it, the authentication steps will not work within the PIP browser. The new window must be used.

I spent the last two days banging my head against the wall, as to why this all-of-a-sudden was not working.
I have pushed my test changes. The issue I am trying to debug is: I am not throwing the correct assertion fail, as the tests pass, but the console says otherwise:

The Gists are done. The issue above still remains.
Here are some final points which might need discussing:

The above mentioned issue...
Socket.io is only set-up to work with the GitHub auth routes. Should we refactor in any way to include the /login and /register routes. NOTE: This is not trivial, as Socket.io is setup to expect the user object from GitHub, which is structured differently to the user object from the app.
One or two of the lessons require quite a lot of code to be added. Should/could this be spread between other lessons?

@Sky020, awesome, thank you. I'll create a fork of your Gist for each challenge under the CamperBot account and push a commit to your PR with a link at the bottom of each description.
For the issue with the PIP browser, do you remember the challenge where you first ran into that issue? It might be enough to include some text in that challenge about having to open up a new window. It might not hurt to include a similar note in the next challenge or two as a reminder.
About the assertion errors, I can look into that.
For the other issues you mentioned, it's probably fine to have Socket.io only work with the GitHub auth for now. That's how the original project was structured, and it would take a bit too long to refactor the project now. While it might be something to consider in the future, you've already done a lot of great work to simplify the project.
I don't think we need to hold up merging this to get Socket.io working for those other routes now. Same goes for spreading the code out between more lessons. That could be tackled in another PR further down the road.

@scissorsneedfoodtoo

For the issue with the PIP browser, do you remember the challenge where you first ran into that issue?

I think it was the Registration of New Users challenge. There is something about the way the routing is done between the server and the in-editor frame which prevents the session from persisting. I do not know much about the topic, but the solution was to not use it from this point forward.

Okay, thanks @Sky020. I can take a crack at adding some text to the challenge, and maybe to the intro page that mentions the PIP browser might have some issues in some challenges, and to open the project up in a separate window. I'll go ahead and fix the merge conflicts before adding that.
Another thing @RandellDawson and I talked about is removing the "As a reminder, this project is being built upon the following starter project" after the first challenge. Our thinking is that, while it's good to have a reminder, if someone takes a break and decides to pick up the project later, they'll just create a new project on Repl.it if they click that link.
Instead, we'll just have those links on the intro page and in the description for the first challenge.

Instead, we'll just have those links on the intro page and in the description for the first challenge.

@scissorsneedfoodtoo That makes sense. Do you want me to go ahead and remove that section, or will you?

Sorry @Sky020, was deep in the command line trying to resolve those merge conflicts. I went ahead and removed all that text from everything but the first challenge. The intro page already has links to start the project on Repl.it and to the boilerplate, so no need to worry about that.
I didn't get a chance to add the text about the PIP browser. It would be great if you have some time to add that. If not, no worries. I'll pick that up in the morning.

Thank you, for sorting out those conflicts, Kris. I have pushed the following not about the PIP browser:

By all means, change the wording and style.

@Sky020, awesome, thanks for adding that. Just took a look and it reads well. My only suggestion would be to move the text to the end of that description section and remove the hr element. A lot of current challenges like Perform New Updates on a Document Using model.findOneAndUpdate() have the note at the end of the description/instructions.
Yes, I did remove most if not all of the existing hr elements. I thought it was a bit jarring that they were used so often in the later challenges. Ideally these challenges would be written in such a way that there wouldn't need to be any extra hr elements to split up in the information. There would just be the description and instructions, which are both separated by an hr.
But with the way a lot of backend challenges are written, one step could require a bunch of code changes, with instructions spread throughout the description. In cases like that, I could see there being just one longer description section.

@RandellDawson, this PR is ready for review when you get a chance.

@moT01

Minor - There's some indentation and spacing issues in a number of the gists - as well as some comments that could probably be removed.

Do you mean this type of comment:
//Change the response to render the Pug template

That is one of the only comments I would remove.

Minor - The sample.env could just be .env so users don't have to copy it? (this one is on the boilerplate)

I was unsure what would happen, if I had .env in .gitignore and wanted to push .env to the repo without changing .gitignore. I think it is best leaving sample.env as is, to prevent eager campers from pushing their secrets everywhere.

Wondering if we should try and lock all the dependencies to a specific version

Similar point of discussion: Would creating a section/file/addition-to-readme with all of the current (right now) versions of the packages used, be beneficial anyway? A simple copy-pasta of the package.json as it is completed.

I was referring to this comment - perhaps it's there for a reason - and yes, the two space indentation to four space indentation you can see in these files - not a big deal - no need to change it, I was just pointing it out.

I was unsure what would happen, if I had .env in .gitignore and wanted to push .env to the repo without changing .gitignore. I think it is best leaving sample.env as is, to prevent eager campers from pushing their secrets everywhere.

Perhaps that's a good idea.

Similar point of discussion: Would creating a section/file/addition-to-readme with all of the current (right now) versions of the packages used, be beneficial anyway? A simple copy-pasta of the package.json as it is completed.

That could be beneficial I suppose - I was thinking to adjust the instructions to tell users to use specific versions. Instead of "install pug" - "install pug version 3.0.0" - but I'm not sure it's worth it to make all those adjustments at this point. I suppose you could possibly just put them all in the package.json file and then all users have to do is require them - that doesn't sound too bad.

@scissorsneedfoodtoo and @RandellDawson I would appreciate you weighing in on the switch to including all necessary packages in the boilerplate package.json? It would involve removing lines in the lessons about "install package x and y"

@scissorsneedfoodtoo and @RandellDawson I would appreciate you weighing in on the switch to including all necessary packages in the boilerplate package.json? It would involve removing lines in the lessons about "install package x and y"

I am still reviewing both this PR and the corresponding boilerplate PR.  Yesterday, I mentioned to you that mongodb should be apart of the boilerplate's package.json file, because if the user just starts the project in development mode, they would get an error right off the bat.  My first thought is that the only packages that need to be present in the boilerplate, are ones that would prevent an error in the terminal when ran locally (npm run dev) the first time.  After that, it would be fine to just have the user install package x and y in the various challenges.

Fixed a merge conflict we inadvertently caused in serialization-of-a-user-object.english.md.

Thanks again for all of the hard work you put into this @Sky020! Looking forward to seeing this live :+1: :+1:
@moT01, thanks for catching the issue with the comments and spacing on the Gists. I can fix those from the Camperbot account.


Feature/initial

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@b587b98). Click here to learn what that means.
The diff coverage is 89.65%.

@@            Coverage Diff            @@
##             master       #1   +/-   ##
=========================================
  Coverage          ?   89.65%
=========================================
  Files             ?        3
  Lines             ?       58
  Branches          ?       14
=========================================
  Hits              ?       52
  Misses            ?        6
  Partials          ?        0

Impacted Files
Coverage Δ

src/components/Alert.ts
100% <100%> (ø)

src/utils/ContainerUtils.ts
69.23% <69.23%> (ø)

src/components/SearchBar.ts
94.73% <94.73%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b587b98...c2ff96b. Read the comment docs.


Generate an rdfs:label from bf:mainTitle in bf:Title

In the LC editor, SVDE & the LC MARC to BF specs (including the comparison viewer), bf:Title includes an rdfs:label in addition to the properties we are using, such as bf:mainTitle. In the "old" LC editor (and I presume the new one), this label is created behind the scenes by duplicating bf:mainTitle; it is not entered through the template and turns out looking like this:

I'd give you an example from the comparison viewer as well, but they are in the midst of redoing the page and the link is broken!
Is there any way that we can replicate this feature? I believe it will help a great deal with interoperability to be in sync with LC on this.

where is the bf:Title label going to be used? is this more of an issue for a consuming application, in which case if the consuming application needs a bf:Title, the consuming application could have a rule to create it from bf:mainTitle, rather than Sinopia creating it

I don't know how it used exactly (how is anything used yet?), but it mirrors how all our MARC records are converted, what the Library of Congress does (in its templates, in its conversion specs, in BF works/instances) and what SVDE does. Actually in the MARC to BF specs, it includes the $b in the MARC as well. In the BF to MARC specs, there is a hint that there may be times the more granular properties, such as bf:mainTitle might not be used and only rdfs:label would. This actually makes sense.

is there an example available yet from the LC comparison viewer?

need to clarify:
@NancyL are you asking for:
(1) automatically filling in the resource's rdfs label field with the resource's title and subtitle?
(2) automatically creating/filling in an rdfs label for the title itself?

for the title itself.

@NancyL it would be helpful to understand why LC is doing this and how. can you get more information from them?

Just received a note from Nate. While they had good reasons for adding the rdfs:label, they are now dropping it, because of rebuild issues in the editor (rebuilding the label when parts of it change...).
This means we can close this issue and for once, I will do so!


[Data Sync] Handle database credentials

Obtain database connection in Oracle Database and Postgres Database using safe stored credentials obtained by a Secret Engine.
Background: Data Sync Engine Needs to obtain connection in databases in run time execution. Credentials must be obtained during runtime in requests to a secrets management engine.
Acceptance Criteria:

Sample code connecting in Postgres and returning a result of a dummy query (like Select CURRENT_TIME a)
Sample code connecting in Oracle and returning a result of a dummy query (like Select NOW() a FROM DUAL)
Documentation available in confluence page.

Handling Jenkins to access Vault to handle credentials.

rolled over from sprint 23

This issue is blocked by #26 and #27 issues, regarding the change of Jenkins Server from Cron to Polaris.

rolled over from Sprint 24

This issue is blocked by #30 (Vault credentials gathering implementation in Jenkins Polaris).

Updates for this task, so far:
I finally could use the Openshift Client inside a Podman instance (at least to run "oc version" command).
I tried:

Use a openshift red hat image, but I couldn't update the machine libraries because it needs a ret hat login credential to access the library/modules repositories.
Use a openshift-client docker image, but it is built in a Linux Alpine base and it was hard (at least for me) to install other things and maintain this image or run this image in parallel with the podman python container acessing it. Besides, it's not an official maintained Red Hat image.
Use a Python Image and install the OpenShift client by wget (using an official Red Hat release). Wget failed to retrieve the URL bevcause the proxy denied the URL access.
Tried to use the Python image with the release Openshift Client already downloaded: it was successful.

Latest error:
https://cd.io.nrs.gov.bc.ca/job/spar/job/DataSyncEngine/116/console
STEP 22/25: RUN ./oc_bin/oc login --token=$oct --server=$ocs
error: context deadline exceeded
Error: building at STEP "RUN ./oc_bin/oc login --token=$oct --server=$ocs": while running runtime: exit status 1
Requesting help to 1Team.

This issue will need to be rolled up to the next sprint.
The Jenkins Polaris approach was put on hold because it was not possible to ensure a Postgres database port expose through portforward command. So, at this point we are trying to use a Github Workflow actions to ensure the deployment of ETL batch instances.

Rolled over from sprint 26

This issue should be closed, as the old repo does not exists anymore (new repo is nr-spar-data).
There are a new structure and new tasks to be included in future sprints.


Apply knockback resistance from custom armor materials

Motivation
See #197. Armor materials other than Netherite don't get their knockback resistance applied in vanilla.
Content
A mixin to let non-Netherite armor materials have knockback resistance, plus a testmod.
Closes #197.

Putting this here so no one forgets (hopefully):
Once this is merged, replace org.quiltmc.qsl.rendering.entity.test.mixin.ArmorItemMixin from #166 with this.

This PR needs to be updated to 1.19.4

rebased onto 1.19.4.

This PR has entered a Final Comment Period of 3 days!


Add benchmark output and GitHub action for monitoring.

Fixes #6

Nice! LGTM other than my naming nit. :)

Benchmark data will be published to https://blinkkcode.github.io/amagaki/dev/bench/

Benchmark data will be published to https://blinkkcode.github.io/amagaki/dev/bench/

How about just https://blinkkcode.github.io/amagaki/benchmark/ ? I kind of feel like this would be one of those things that just gets unnecessarily inserted into "dev" and then it's the only "dev" thing for years. ;)

I agree, the default path isn't very friendly. Updated it to server at benchmark/ instead.


ghost handling

Description

Dev snapshot for qcdb testing. Ok to merge except there's other PRs ahead and needs MolSSI/QCElemental#276
Changelog description

[x] ~includes #315 and #316~
[x] nwchem and cfour ghost atom handling
[x] gamess ghost atom handling can't be done with qcng's present coord=prinaxis strategy
[x] fci testing

Status

[x] Code base linted
[x] Ready to go

It looks good to me, though I can't speak too much on the technical level for ghost atoms.

Thanks. I can well believe that a per-atom custom basis set will show up more problems, but I'm reasonably confident in the current state. Probably Psi4 is the only qcng+ghost path in current use.


[CF] Provide Content Fragment configuration for resource types from Core Components

Added OSGi configuration

(fixes #136)

Codecov Report

Merging #137 into development will not change coverage.
The diff coverage is n/a.

@@              Coverage Diff               @@
##             development     #137   +/-   ##
==============================================
  Coverage          82.49%   82.49%
  Complexity           736      736
==============================================
  Files                 71       71
  Lines               2393     2393
  Branches             427      427
==============================================
  Hits                1974     1974
  Misses               170      170
  Partials             249      249

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9268c8...61860cb. Read the comment docs.

@gknobloch, for these projects we cannot use the provisioning file, since they're not delivered with the Quickstart. Core Components also provide configurations like this.


Error parsing array

In a JSON file, when using array notation (without quotes), only the first value is parsed. When trying to parse:
{   "value" : [3:10]  }
value will evaluate to 3. This problem is related to 'sscanf' function in line 179 from loadjson.m:
[obj, count, errmsg, nextidx]=sscanf(astr,'%f,',[1,inf]);

@toloko, "[3:10]" is not a valid JSON syntax, you can test it on http://jsonlint.com/
according to http://json.org, JSON array only supports comma-separated values.

Thanks for the comment and for your work.
Should it not 'loadjson' throw an error in such a case?
Thanks

interesting, the same line now evaluates as the anticipated result in matlab 2017a, but jsonlab should have thrown an error.
>> dd=loadjson('{ "value" : [3:10] }')

dd =

  struct with fields:

    value: {[3 4 5 6 7 8 9 10]}


person_id inconsistent in final OMOP death table

Hi, I reran the pipeline on MIMIC-III using your latest code (v0.2.1) and I found the person_id final OMOP death table actually contains visit_occurrence_id.
Could you please take a look at this? Thanks!

Thanks again for pointing this out ! I will check that visit_occurrence_ids and patient_ids are properly filled in every table.

In drug_exposure are missing patianet_id s
perhaps this referres to the same issue

Thanks for your input, I'm checking other tables soon !


suggestion: ability to pause the jobcuerrrently running

Using the version from I believe February, or what ever  the latest version is. It would be nice and helpful to pause the running job as sometimes I need to do this if something else needs the network. I can't really go into any more detail than that, but I hope that helps a bit.
Thanks

Technically this is could possible, but the issues is that GUI doesn't have control over how individual files are transferred by rclone. The best I can do is suspend rclone process itself and resume it later. This might not play well when its downloading or uploading files. The cloud services could see that as timeout and terminate connection, thus rclone will probably repeat the upload or download and this leads to wasted bandwidth. Or it will simply terminate connection. So not sure about this. It probably will be very unstable feature.
Ideally rclone should allow to control bandwidth dynamically and allow to set it from different process via some kind of RPC. So this is really a feature request for rclone, not RcloneBrowser.

Ah. Yeah for rclone itself, I just stop the task anyway then just start it again when I'm done, which works. That answer actually does make sense and the staff of rclone have said the same thing. I can understand why and I would not want a ruined  upload/download.Oh well, I tried. Lol!Thanks so much, I'll keep sending feedback as time permits.

Ah. Yeah for rclone itself, I just stop the task anyway then just start it again when I'm done, which works. That answer actually does make sense and the staff of rclone have said the same thing. I can understand why and I would not want a ruined  upload/download.
Oh well, I tried. Lol!
Thanks so much, I'll keep sending feedback as time permits.

One suggestion might be to keep the job in the queue or at least in he jobs list when I cancel the job this way I don't have to fill in all the details again, this could be maybe equivalent to "pause" or add a "stop" command. If that is not there and I missed it, please accept my apologies.
Thanks again.


Argument error

Hello,
After installing -- I followed all the instructions verbatim -- running runtests.jl produces this error:
ERROR: LoadError: ArgumentError: Array dimension must be 0 ≤ dim ≤<PHONE_NUMBER>854775807, got<PHONE_NUMBER>3709551615 for dimension 1
 in pointer_to_array at pointer.jl:41
 in on_body at .julia/v0.4/HttpParser/test/runtests.jl:93
while loading .julia/v0.4/HttpParser/test/runtests.jl, in expression starting on line 129

This is my version info:
julia> versioninfo()
Julia Version 0.4.1
Commit cbe1bee* (2015-11-08 10:33 UTC)
Platform Info:
  System: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i7 CPU       M 620  @ 2.67GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Nehalem)
  LAPACK: libopenblasp.so.0
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

Is it a bug? What am I doing wrong?

It is a bug, you aren't doing anything wrong (probably). What operating system are you using?
Cross-reference https://github.com/JuliaWeb/HttpServer.jl/issues/81

Fedora 23


[bug] configDir and dataDir resolves both to C:\Users\username\AppData\Roaming\ (windows 11)

Describe the bug
configDir and dataDir from @tauri-apps/api/path package resolves both to C:\Users\username\AppData\Roaming\.
This reflects also on appDir which refers to  C:\Users\username\AppData\Roaming\${bundleIdentifier},
Tested with yarn create tauri-app with create-vite-app (react-ts) and create-react-app templates.
Reproduction

create tauri app with yarn create tauri-app with create-react-app template
in App.js create a function test to check the two paths and execute it:

  const test = async () => {
    const config = await configDir();
    const data = await dataDir();
    console.log(config, data);
  };

  test();

check the console in devtools: the two paths are the same

Expected behavior
configDir should refers to C:\Users\username\AppData\Local as described in the doc:

Windows: Resolves to {FOLDERID_LocalAppData}.

Platform and versions
Environment
  › OS: Windows 10.0.22621 X64
  › Webview2: 102.0.1245.44
  › MSVC:
      - Visual Studio Build Tools 2022
      - Visual Studio Build Tools 2019
  › Node.js: 16.15.1
  › npm: 8.13.1
  › pnpm: 6.11.0
  › yarn: 3.2.1
  › rustup: 1.24.3
  › rustc: 1.61.0
  › cargo: 1.61.0
  › Rust toolchain: stable-x86_64-pc-windows-msvc

Packages
  › @tauri-apps/cli [NPM]: 1.0.0
  › @tauri-apps/api [NPM]: 1.0.1
  › tauri [RUST]: 1.0.0,
  › tauri-build [RUST]: 1.0.0,
  › tao [RUST]: 0.11.2,
  › wry [RUST]: 0.18.3,

App
  › build-type: bundle
  › CSP: unset
  › distDir: ../build
  › devPath: http://localhost:3000/
  › framework: React

App directory structure
  ├─ .git
  ├─ .yarn
  ├─ node_modules
  ├─ public
  ├─ src
  └─ src-tauri

Stack trace
No response
Additional context
I don't know why tauri info says I'm on Windows 10 while I am on Windows 11

Roaming is the correct folder. But yes, the docs are incorrect (most likely a copy-paste mistake)

If it's a documentation mistake, why are there two variables with the same value? Isn't it useless?

Because these functions are cross-platform and Windows doesn't have specific folders, unlike Linux: https://github.com/xdg-rs/dirs/tree/master/dirs#features

Got it. Probably a PR is needed for the docs mistake, but the problem itself doesn't exist anymore. Thank you. I'm going to close this issue.


Incorrect platform detection when using MSYS2 MINGW64

Description:
When running click in MSYS2 MINGW64 Python environment, click is trying to import tty. This fails due to termios not being a package supported by Windows.
This happens because of the platform detection in _compat.py. The following code detects the platform as being MSYS2 which then tries to use MSYS2 compatible packages:
MSYS2 = sys.platform.startswith("win") and ("GCC" in sys.version)
This does not work when using python packaged with MINGW64 as it still detects the platform as being MSYS2.
Reproduce:
Open a progress bar in MSYS2 MINGW64 Python3 i.e.
with click.progressbar(length=10, label="Downloading") as pbar:
Traceback (most recent call last):
File "C:\msys64\mingw64\lib\python3.10\site-packages\apio\managers\installer.py", line 200, in install
dlpath = self._download(platform_download_url)
File "C:\msys64\mingw64\lib\python3.10\site-packages\apio\managers\installer.py", line 382, in _download
filed.start()
File "C:\msys64\mingw64\lib\python3.10\site-packages\apio\managers\downloader.py", line 68, in start
with click.progressbar(length=chunks, label="Downloading") as pbar:
File "C:\msys64\mingw64\lib\python3.10\site-packages\click\termui.py", line 394, in progressbar
from ._termui_impl import ProgressBar
File "C:\msys64\mingw64\lib\python3.10\site-packages\click_termui_impl.py", line 626, in
import tty
File "C:\msys64\mingw64\lib\python3.10\tty.py", line 5, in
from termios import *
ModuleNotFoundError: No module named 'termios'
Expected Behaviour:
Expect the platform detection to detect the platform as Windows and not MSYS2.
Environment:

Platform: MSYS2 MINGW64
Python version: 3.10
Click version: 7.1.2

Happy to review a PR. Note that both mingw and msys are not officially supported platforms by us, and are not guaranteed to work.

Thanks, I may have a fix that involves using platform.system() instead. Just need to check it across different Windows platforms.
This comment is what I am basing it off. I did just install python in the MSYS2 environment and sys.platform is returning cygwin for it so that might mean something has changed with how python is packaged in MSYS.

@davidism I see that the change introduced here is now causing this fail. Simply reverting this change seems to work on MSYS2 MINGW64.
I ran the Pytests with the regular Windows python 3.10, the MSYS2 python 3.10 and the MINGW64 python 3.10.
Windows and MINGW64 passed. MSYS2 failed before and after the reverted change however there was no difference what failed.

Thanks for looking at this. However, the previous issue and PR indicate that it was correct at the time because MSYS2 used Bash. Why is removing this correct now? What has changed, and why?

Hi, I'll have to look through the changes in the MSYS2 repo to see if anything has changed.
I'm seeing python report sys.platform on MSYS as being cygwin so maybe the package maintainers changed how the platform appears to python to resolve issues like this?
I don't think checking for the string GCC is the correct way to go about checking for the platform. Both MSYS and MINGW64 compile python with GCC from what I've gathered. MSYS executables use the POSIX emulation layer but I don't think the MINGW ones do. I haven't checked the clang repos either.
I'll get back to you with what I can find.

Ran into this from platformio/platformio#3597 (not sure why they blamed cygwin) and investigated a bit. I think reverting #1393 is the correct solution. I think #1338 was fixed more correctly by #1135, and #1393 was misguided to start with.
There are 2 distinct issues:

which python (this bug)
which terminal (#1338)

For this bug, the question is whether the running python can or should try to load tty/termios libraries. Msys and cygwin provide a compatibility layer that would make that work, so loading them in a python installed from msys/python or from cygwin is valid. Other pythons on windows (in particular mingw64/mingw-w64-x86_64-python from msys) don't have that and will fail as above. #1393 doesn't distinguish between msys/ and mingw64/ and loads termios on both, so if it were kept more tests would have to be done to do this correctly if you really wanted to load termios on msys/python.
For #1338, the question is whether stdout  is a "windows console" as you would get from conhost.exe or a few other terminal programs, or if it is a named pipe as you would get from various ports of unix terminals, in particular the mintty from msys. That is mostly independent of which python you are using, since you can run msys bash in a windows console, and you can run cmd.exe in a msys mintty, or you could run either python directly in either terminal without any shell. Msys users are probably more likely to run unix-like terminals, so there is some correlation, but #1393 still gets that wrong since msys2 also has ucrt64/mingw-w64-ucrt-x86_64-python and clang64/mingw-w64-clang-x86_64-python which it would miss.  #1135 just directly checks for whether we have a console or not, which seems like a better and more general solution.
To complicate it further, on recent win 10 conhost can be made to understand utf8+vt100 instead of using console color API (colorama does this if possible now i think?), and recent mintty uses the ConPTY API to allow it to understand the console APIs. The new windows terminal similarly works with either.  None of that helps with the original reporter using 8.1, but it does make it harder to test things without an actual 8.1 install. Testing in mintty with ConPTY disabled does seem to show that #1338 is still a problem without either #1135 or #1393 though, and that #1135 is enough to fix it.

@3b thanks for the detailed analysis. It sounds like we should revert #1393 and make sure #1135 is still intact.

Testing in mintty with ConPTY disabled

Could you describe how to do that?

I'm not a Windows user or developer, all these additional systems are not familiar to me. From some research, it seems like the modern system is msys2, which uses mingw-w64. I'm not sure how accurate all of the following is, but it's what I found trying to figure out the different ways Python might be available on Windows.

Python's official Windows build is a native Windows program.

py -c 'import sys, os; print(sys.platform, os.name)' outputs win32 nt.
py -c 'import tty' fails.
The Microsoft Store also provides python (and asks to install it if you type python instead of py). It behaves the same.

msys2 provides different environments, which target an architecture and C runtime. It provides packages built for each environment, including Python. It uses mingw-w64 to build those packages.

Install msys2 using scoop install msys2.
Run ucrt64 to get a terminal with the ucrt environment.
Run pacman -S mingw-w64-ucrt-x86_64-python to install the python command in that environment.
python -c 'import sys, os; print(sys.platform, os.name)' outputs win32 nt.
python -c 'import tty' fails.

Git for Windows, and its Git Bash program, use msys2 and mingw-w64.
msys2 also has an "internal" environment based on cygwin.

Run msys2 to get a terminal.
Run pacman -S python to install the python command.
python -c 'import sys, os; print(sys.platform, os.name)' outputs cygwin posix.
python -c 'import tty' does not fail.

msys2 is an updated fork of msys, and mingw-w64 is an updated fork of mingw.

Install msys using scoop install msys.
Run msys to get a terminal.
It's not clear how to install Python with msys and mingw, so I didn't test that.

cygwin is the oldest project. msys and msys2, and the packages they build, are modified from cygwin.

Install cygwin using scoop install cygwin.
From the Start menu, select "Cygwin Setup" to get the package installer. Search for "python3" and select a version to install.
Run cygwin to get a terminal.
python -c 'import sys, os; print(sys.platform, os.name)' outputs cygwin posix.
python -c 'import tty' does not fail.

It's possible to run py from within all these different environments. It still behaves the same as running from a Windows terminal, tty is not available.
It looks like Python built with cygwin sees a Linux-like environment, and Python built with mingw-w64 sees a Windows-like environment.

I've confirmed that click.echo and click.progressbar continue to work after reverting #1393, in both cygwin and win32 environments.

Testing in mintty with ConPTY disabled

Could you describe how to do that?

I think i was using mintty --pcon off,


fix count variables are only valid within resources

Fixes
❯ tf plan

Error: local instance_name: count variables are only valid within resources

Signed-off-by: Denis Iskandarov<EMAIL_ADDRESS>

@antonbabenko This PR looks good, how soon can we merge this?

v1.18.0 has been released.