Fix generated VM names - Round 2

The first PR merged before I relaized I hadn't commited part of the change.
This fixes the worker names per the previous commit message details.

@kiall ready to merge?

@dirkmueller yes, good to go + tested now
(Sorry for the dud first PR - I wasn't expecting review so quickly - I through it up to give to someone to test :D)


reflect: Bytes allows conversion of invariant slices

(This is a very minor issue), but consider the following:
type MyByte uint8
type Bytes1 = []byte
type Bytes2 []byte
type Bytes3 []MyByte

According to the Go specification, Bytes1 and Bytes2 are convertible to each other, but Bytes3 is not convertible to any of the other since the underlying element type differs (even if byte and MyByte are convertible to each other). In other words Bytes3 is not convertible to a []byte.
The Value.Bytes is a convenience method that retrieves the []byte if convertible to such. However, this is possible:
reflect.ValueOf(Bytes3{1, 2, 3}).Bytes()

This allows us to use reflect to convert Bytes3 to Bytes1, when we couldn't do so in the language.
This applies to Value.Bytes, Value.Runes, Value.SetBytes, and Value.SetRunes.

cool finding.
package main

import "fmt"
import "reflect"

type MyByte uint8
type Bytes1 = []byte
type Bytes2 []byte
type Bytes3 []MyByte

func main() {
	var b3 = Bytes3{1, 2, 3}
	b1 := reflect.ValueOf(b3).Bytes()
	b1[1] = 9
	fmt.Println(b3) // [1 9 3]
}

A little related: https://github.com/golang/go/issues/23536

Very related: #19778.

I'm not sure this is a real problem.  It would be a minor problem if the reflect package let you convert a reflect.Value from Bytes3 to Bytes1.  But (as far as I know) it doesn't.    That is a type error, but it's not a very interesting one, as it's a type error when crossing the boundary from a reflect.Value to a real type.  We should consider deprecating this method for Go 2, but there isn't much point to making the method a duplicate of v.Interface().([]byte).

Sure, I'm not surprised that people rely on it.  What I'm trying to say is that I think it's OK to permit this kind of type violation.  It's not the kind of type violation that the reflect package is not supposed to permit.

The hole doesn't do bad, pls don't fix it. :)

One surprising side-effect it is that it allows the program to bypass the usual restrictions on setting unexported fields (and fields of unexported types).
https://play.golang.org/p/7MwAH2x6Uto

@bcmills

..., without even importing unsafe.

This is why this hole is so useful.

This is why this hole is so useful.

This type of sentiment is why this hole is dangerous. Needing to import unsafe is supposed to signal that you are deliberately sidestepping the type system.

I don't see any dangerousness here.
Could you show an example to prove it is dangerous.
I don't think Go type system forbids this conversion for security reason.

And if it is dangerous, the proposal, https://github.com/golang/go/issues/19778, should have been rejected already now.

The problem here is arguably not the conversion at all. Rather, it is that (reflect.Value).Bytes is the only reflect.Value method that returns a mutable value without enforcing CanSet.
If you try to write the equivalent code by copying the slice value out to a different variable, you get panic: reflect: reflect.Value.Set using value obtained using unexported field (https://play.golang.org/p/-jlmAYDY4IE). The same happens with reflect.Copy (https://play.golang.org/p/V7n2EXkRn6Y).
#19778 is not relevant to this problem because ordinary Go code enforces unexported fields and types at compile-time instead of run-time. (It is already impossible to write these examples using ordinary Go code, because outside packages cannot name the types and/or fields involved.)

Could you show an example to prove it is dangerous?

See the numerous Playground examples above. It's not “dangerous” in the “undefined behavior” sense, but rather in the “allows a program to violate API restrictions” sense.

Read-only slices (https://github.com/golang/go/issues/22876) could address this problem: they would allow the Bytes method to return a read-only slice instead of a read-write one.
Programs that need to obtain a read-write slice can already do so today, subject to the usual restrictions on unexported fields and types, by calling (reflect.Value).Interface and type-asserting the result. (And, of course, programs today can also use the (reflect.Value).SetBytes method to set the raw bytes directly.)

@bcmills
https://github.com/golang/go/issues/19778 has nothing related to unexported fields.
As in the 3rd comment you have made, it is very related to this issue.
Agree with you that your play examples is not "dangerous",
and yes, it “allows a program to violate API restrictions”.
But the violation doesn't do any harm.
On the contrary, it has many benefits.
For example, now we can use the functions in bytes standard packages for []MyByte values,
but we can with the help of this violation.

fixed missing a NOT in the last comment.
And I don't think this violation will affect the possible read-only slice feature later.

I'm going to close this because I really don't think there is anything to be done.
I suggest that you write a Go 2 proposal to remove the reflect.Value.Bytes method.

I suggest that you write a Go 2 proposal to remove the reflect.Value.Bytes method.

Done: https://github.com/golang/go/issues/27727.


Automorphic Number in Julia

💥 Proposal
Automorphic Number in Julia
Have you read the Contributing Guidelines on Pull Requests?
YES

/assign


update documentation

We need to update README.md with the new feature(s) such as modal mode.

@vercasson I've added some placeholders here, but if you have a few minutes, it'd be helpful if you could provide some usage examples in README.md.  Once that is complete, we can release v1.0.0

@vercasson @dagellon This issue is the only thing in the way of v1.0.0.  It'd be helpful if somebody could write it.


helm/2517 - Deis on OpenShift

From @kingdonb on August 24, 2017 3:38
I tried to launch Workflow on OpenShift and I have followed from kubernetes/helm#2517 where it was explained how Helm can be used on OpenShift
kingdonb$ helm install --tiller-namespace myproject -n deis --namespace myproject workflow/
Error: release deis failed: User "system:serviceaccount:myproject:default" cannot create daemonsets.extensions in project "myproject"

Is this an issue with OpenShift or with Deis? My reading is that there is a resource kind that does not exist on OpenShift yet, that Deis leveraged on a high version of Kubernetes...
Copied from original issue: deis/workflow#856

From @kingdonb on August 24, 2017 16:49
Here's what I've done:
minishift start
...
oc login -u system:admin
kubectl -n myproject create sa tiller
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=myproject:tiller
helm --tiller-namespace myproject --service-account tiller init
oc policy add-role-to-user admin -z tiller -n myproject
helm install --tiller-namespace myproject -n deis --namespace myproject workflow/

which is the most permissive role I can imagine, still only to get
Error: release deis failed: User "system:serviceaccount:myproject:tiller" cannot create daemonsets.extensions in project "myproject"

I followed this, because I understood that OpenShift role based security has essentially merged with standard kube RBAC, so I guessed that either set of permissions would work.  I don't really want to set the most permissive possible though, I'd love to understand why this just doesn't work.
https://gist.github.com/mgoodness/bd887830cd5d483446cc4cd3cb7db09d

From @kingdonb on August 24, 2017 17:30
I get the same results with https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/
$ helm install -n deis --namespace myproject workflow/
Error: release deis failed: User "system:serviceaccount:kube-system:tiller" cannot create daemonsets.extensions in project "myproject"

From @bacongobbler on August 24, 2017 17:32
Did you grant tiller RBAC permissions to create objects in the "myproject" namespace?
That gist you posted earlier is only relevant if tiller is installed in the kube-system namespace. It looks like from your commands you deployed tiller in the "myproject" namespace.

From @bacongobbler on August 24, 2017 17:45
https://github.com/openshift/origin/pull/8242

From @bacongobbler on August 24, 2017 18:1
Did you make sure to install Workflow with RBAC permissions? You'll need to add --set global.use_rbac=true in your helm install invocation to install Workflow with RBAC permissions. See #812.

From @kingdonb on August 24, 2017 17:56
Oh!
The OpenShift devs helped me out, two things I'm doing wrong...

When a tutorial says to use -z in oc policy, it's a shortcut for system:serviceaccount:[-n]:ARG but this is new in 3.5, and I had an old oc client <IP_ADDRESS> hanging around in my path -- why I thought it would be OK to drop the -z I don't know, but in the end I sometimes got errors telling I couldn't even list the namespace, which I was supposedly granting myself admin role for.  I didn't have the permission applied to the actual serviceaccount properly.

I could see this was the case by looking at oc policy who-can ... but it was not so easy to figure this out in the documentation for OpenShift, unfortunately, and I didn't realize until later I had an old oc binary.

What you just found there... giving the SA an admin role is not enough, because admin role is not allowed to handle daemonsets for some reason.  You can give the cluster-admin role though:

So this didn't work: oc adm policy add-cluster-role-to-user cluster-admin tiller -n myproject
but this did work:
oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:myproject:tiller -n myproject

From @bacongobbler on August 24, 2017 18:2
Not sure about the other issues. We don't officially support Workflow on Openshift, but please continue to post here so others can learn.

From @kingdonb on August 24, 2017 18:2
You have been so helpful.  Thank you!  I did not do that.

From @kingdonb on August 24, 2017 18:7
This big "MISSING" at the bottom looks kind of ominous... those are my RBAC roles, and I'm guessing that Missing means they're using a resource kind that I don't have on my cluster.   It is true, the OpenShift 3.4 security model had not yet started to merge with K8S RBAC because K8S was not fully baked when 3.4 was developed and released.  That doesn't seem to be my problem anymore though.
In 3.5 and 3.6, this should work fine.  I'll test it out and add notes when I find out.  If someone needs to use 3.4, there is probably a way to apply the permissions to the service accounts, but in my case it looks like the only reason I'm having problems is that I have an old oc OpenShift client binary.

From @kingdonb on August 24, 2017 19:0
Almost... there is a bridge for roles (OpenShift has its own role kinds that predate Kubernetes formal implementation of RBAC), but the connector goes the wrong way for what I'm trying to do:
in 3.6, we go openshift -> rbac, *not* rbac to openshift. If you create an RBAC object, it will be stomped away

So if you have OpenShift roles implemented, and you want to access them as Kubernetes roles, you can do it, but if you try the reverse you will know it didn't work because helm very helpfully tells you the state of these resources you created after install is MISSING.  It means just what it says.
There is supposedly a raw and fairly untested tool for doing the conversion of those resources and we are going to try it out against the output of helm --debug

From @kingdonb on August 25, 2017 12:54
these roles/bindings:
https://gist.github.com/enj/078cc916365a7c3db6da581ea4485bd6
and some combination of these scc settings:
# (for tiller in kube-system namespace, cluster-admin because daemonsets)
oc policy add-role-to-user edit system:serviceaccount:kube-system:default
oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:kube-system:default

# for your deis roles, you don't need to use system:authenticated
# but I was having trouble making sure my policies were actually
# getting applied where I thought they were
oc adm policy add-scc-to-group anyuid system:authenticated
oc adm policy add-scc-to-group privileged system:serviceaccounts:NAMESPACE

will get you most of the way there, in the worst possible way... this still didn't quite work for me, most of the containers came up, but a few still didn't.  It's a bit mysterious, but I'll try again today.
Controller for instance, which was complaining about not being able to create a directory /app/data, but why wasn't clear.  (anyuid policy should have fixed that, adding that setting I was able to start pods and see them come up as root via kubectl.  Another security feature of OpenShift is that containers are not allowed to start as root unless you specifically say they are allowed to.)
I'm saying this is the worst possible way because it's basically turning off all of the security, but I'll work on tightening it up for the PR once I figure out what's still missing!

From @kingdonb on August 25, 2017 13:31
Hm.  Today on a clean minishift, it works!  I believe with no more information than I already had posted here.  Waiting for Builder to come up...
NAME                                     READY     STATUS    RESTARTS   AGE
deis-builder-904459970-rwms8             0/1       Running   1          2m
deis-controller-198627737-jqq9m          1/1       Running   3          2m
deis-database-4120584427-kp6l8           1/1       Running   0          2m
deis-logger-2717637750-l9g60             1/1       Running   2          2m
deis-logger-fluentd-zcz1d                1/1       Running   0          2m
deis-logger-redis-1281502559-srbsz       1/1       Running   0          2m
deis-minio-2522384958-kc60z              1/1       Running   0          2m
deis-monitor-grafana-1204143830-q77cx    1/1       Running   0          2m
deis-monitor-influxdb-2842921881-91s7x   1/1       Running   0          2m
deis-monitor-telegraf-dt6w3              1/1       Running   0          2m
deis-nsqd-1225249577-v1lf1               1/1       Running   0          2m
deis-registry-3907297837-msgxm           1/1       Running   2          2m
deis-registry-proxy-74ljd                1/1       Running   0          2m
deis-router-3453227723-x352n             1/1       Running   0          2m
deis-workflow-manager-3757950027-s2d00   1/1       Running   0          2m

$ deis register deis.deis:32423
username: yebyen
password:
password (confirm):
email<EMAIL_ADDRESS>Registered yebyen
Logged in as yebyen
Configuration file written to /Users/kingdonb/.deis/client.json

... there are lots of things to work out after that, but it looks like I can use Deis Workflow on MiniShift now!  That's great news, I love interchangeable parts.

From @kingdonb on August 29, 2017 1:58
I have now deployed Workflow on OpenShift Origin as well as Minishift, and tackled some of the additional challenges that entails.
I think mainly, the struggle came because this time I did not grant any privileges to ambiguous any-user 'system:authenticated' and made it a point to trace down issues individually through each of the logs, so I would know what permissions I was granting, and what wouldn't work until they were granted.  (Most things just worked on minishift as I granted the more permissive settings broadly.)
They were all some subset of these serviceaccounts that required additional 'scc' permissions, which is another OpenShift thing.
... bad example deleted

My search history seems to indicate I also needed
setenforce 0

at some point, or for some reason.
These pods all needed cluster-admin I think mostly because on OpenShift, you can only enumerate namespaces if you have cluster-admin.  (Edit: Perhaps there is a different endpoint that users can query to get all of their own namespaces, or projects as they're sometimes called in OpenShift)
I'm not sure how we can provide CI validation for this for a release,
I have cleaned my rough work up further into almost draft shape for a pull request, I am just running the deployment again onto my OpenShift/CentOS VM now to ensure the chart with rewritten roles, and a new value in values.yaml is still working as expected after I have adjusted it to fit within the template.
Maybe a ten-minute YouTube video that starts from vagrant destroy and skipping up to the part where you install docker onto your CentOS 7.3 machine, will suffice for the purposes of validating the release as "Experimental OpenShift Support" with prominent notice marking it as such.  Not sure if you will want to merge this at all when it's done, actually I foresee probably not...
I predict a fork that is committed to supporting Deis Workflow on OpenShift and tracking maintenance as new Kube releases are tracked against OpenShift :grin: the one thing I have not yet figured out is what will allow telegraf to collect metrics, as I was able to communicate with InfluxDB from the logged-in grafana, and read the basic InfluxDB stats, and the stats from router; the interface worked but no CPU or Memory statistics were gathered, and I gather that is telegraf's job.

From @kingdonb on September 16, 2017 17:44
After a few weeks of upstream development, I went in to review this stuff and try porting it forward to the latest releases of everything (minishift, Deis, OpenShift Origin, all have had releases since 19 days ago...)
I have created a new branch for openshift-deis-v2.18 so you can try release-2.17 and release-2.18, but in 2.17 I'm sure you may need to loosen the permissions model because I hadn't worked out most of the details when I put those changes in 3 weeks ago.
It definitely "worked, mostly."  I've figured out a little more specifically what privileges are needed in the 2.18 branch.

From @kingdonb on August 29, 2017 2:22
I packed the changes up into one repo so they can be reviewed at-once, but
https://github.com/kingdonb/openshift-deis/commits/openshift-deis-v2.17
I will unpack them into PRs on the affected deis/* repos if there's any interest in merging them back.  I'll have to tear down CentOS VM and do from scratch to ensure I got all of the oadm policy things in.  Presumably there is also some kind of declarative way I can set those, though maybe not via helm.
I'll also need to touch base with someone else who has helm working on OpenShift realistically to get some kind of approval or review on this branch, since that is also not especially trivial or well documented.  But it works!  This can deploy Deis Workflow into a scratch project on OpenShift.
I'll try hooking it up to #857 next, since I personally will need both.

From @kingdonb on September 16, 2017 20:4
Yep. The key is to make sure you get the order right.  I've got almost everything coming up now, and I'm closing in on the last of the seemingly new problems I maybe created by upgrading (tl;dr: you'll see if you follow these instructions all the way down to the bottom that they're incomplete, controller and deis-monitor-telegraf both can't seem to come up properly even with all of the permissions I granted and scc rules I thought I relaxed... or did I?)
These instructions "work" on a MacOS machine with Homebrew, installing Docker client v1.12.3 via DVM, with Minishift release v1.5.0 (which currently installs OpenShift v3.6.0 by default, which is roughly equivalent to Kubernetes v1.6.1):
First get the right docker client version (which matches the Docker provided by Minishift) this is important for reasons I do not fully understand but, perhaps take it up with Minishift:
brew install dvm
# ... follow the instructions to add dvm to your bash profile, restart your shell, and then
dvm install 1.12.3
dvm use 1.12.3

Then, start OpenShift via Minishift and plug Docker into it:
git clone -b openshift-deis-v2.18 https://github.com/kingdonb/openshift-deis workflow
minishift start --vm-driver=virtualbox --memory=4GB --cpus=4
eval $(minishift docker-env)

Create a project for Deis to live in, and get Helm up and running:
oc login -u system:admin
oc new-project deis
oc create sa tiller
oc adm policy add-cluster-role-to-user cluster-admin -z tiller

helm init --service-account tiller --tiller-namespace deis
export TILLER_NAMESPACE=deis

Grant a bunch of additional privileges that Deis needs on OpenShift...
Give cluster-admin role to all of these deployments that have their own ServiceAccounts, since Deis will create namespaces, as well as manage and monitor all of the cluster's namespaces (projects):
oc adm policy add-cluster-role-to-user cluster-admin -z deis-router
oc adm policy add-cluster-role-to-user cluster-admin -z deis-builder
oc adm policy add-cluster-role-to-user cluster-admin -z deis-controller
oc adm policy add-cluster-role-to-user cluster-admin -z deis-logger-fluentd
oc adm policy add-cluster-role-to-user cluster-admin -z deis-workflow-manager

Many Deis containers currently depend on being able to start with a root UID (and from what I could tell at first glance, some of them don't appear to have their own ServiceAccounts assigned yet):
oc adm policy add-scc-to-group anyuid system:serviceaccounts:deis
oc adm policy add-scc-to-group privileged system:serviceaccounts:deis

At this stage because you haven't created any system:serviceaccounts:deis yet, you should get no report back from these commands... it was at this moment the Buddha got enlightened. (ed; put a pin in this, I will come back later to make it better)
Router wants to be able to punch a hole to the outside network and open a listening port on the host, and OpenShift does not allow this without another scc... the hostnetwork scc:
oc adm policy add-scc-to-user hostnetwork -z deis-router

Controller seems to need to be able to mount a HostPath volume and get access to /var/run/docker.sock in order to do its job... this sounds like something highly inadvisable, and to my knowledge this can only be allowed on the latest OpenShift release if the global scc restricted rules are modified (is this permissions model quite as complex and variegated as SELinux yet?)
So take care of that, and then grant your developer user some control and visibility into the project:
#oc edit scc restricted   # at this point change allowHostDirVolumePlugin to "true"
oc policy add-role-to-user admin developer -n deis
#oc adm policy add-cluster-role-to-user cluster-admin developer
helm install -n deis --namespace deis workflow/

At this point you'd get an error already if tiller wasn't started, next log into the OpenShift dashboard at the URL you got when minishift start finished.  If you haven't already, login as the developer user (the developer username will accept any password on minishift), and watch as pods come up.
fixme below
There need to be some eyes on all of what I've done above to ensure that it's granting the least required privilege.
I also wasn't able to figure out how to put any of the above into YAML files which we could gate similarly to the modified roles and clusterroles we have already through the use_openshift_rbac flag in values.yaml... not yet anyway...
If you run into problems and want to do helm install again, you'll need to wipe out the secrets that were generated for you first, so the next helm install can generate some more:
kubectl -n deis delete secret builder-key-auth builder-ssh-private-keys database-creds deploy-hook-key django-secret-key logger-redis-creds

Scratch space below...
#oc adm policy add-scc-to-group privileged system:serviceaccount:deis
#oc adm policy add-scc-to-group anyuid system:serviceaccount:deis
#oc adm policy add-scc-to-user anyuid -z deis-database
#oc adm policy add-scc-to-user anyuid -z deis-minio
#oc adm policy add-scc-to-user anyuid -z deis-nsqd
#oc adm policy add-scc-to-user anyuid -z default
#oc adm policy add-scc-to-user anyuid -z deis-router
#oc adm policy add-scc-to-user anyuid -z deis-controller

From @kingdonb on September 16, 2017 20:10
The result is now (running tally of unresolved errors looks like):
Deis Controller deploy logs
system information:
Django Version: 1.11.4
Python 3.5.2
mkdir: cannot create directory '/app/data': Permission denied

Deis Monitor Telegraf daemonset logs
Node Name set (null)
jq: error (at <stdin>:13): Cannot index string with string "hostIP"

Deis Builder
It probably is just failing its readiness check because the controller service is not answering yet...

I'm going to go revisit all of the previous comments I left on this issue and delete any that can be subsumed by what I'm documenting here today.  Then I think I'll go back and try with the previous versions, see if something has changed with how Minishift is handling HostDir volumes (since it looks like maybe it has? More likely I think than Deis added breaking changes in v2.18, but we'll see!)

From @liggitt on September 16, 2017 23:48
FYI, in openshift 3.7, you'll be able to use kube RBAC objects directly. Prior to that, openshift RBAC objects controller authorization policy

From @kingdonb on September 16, 2017 21:10
Rolling the stack back to Minishift v1.4.1 and Deis v2.17, I had much better results:
# the syntax for --memory has changed
minishift start --vm-driver=virtualbox --memory=4096 --cpus=4

#... (everything else the same as above, but against the prior release of Deis Workflow)
cd workflow/
git checkout openshift-deis-v2.17

# Make sure you set platform_domain in the root setting file: workflow/values.yaml
$EDITOR values.yaml
cd ..

#create the Deis project and setup helm, as above
...
#grant the cluster-admin roles, just as shown above
...
#grant the anyuid and/or privileged scc policies to the system:serviceaccounts:deis group as above
...
#grant the hostnetwork scc to the deis-router serviceaccount as above
...
# pull up the dashboard and grant developer user admin on your Deis namespace/project as above
helm install -n deis --namespace deis workflow/
# ... again, just as above

... wait a while, then ...
See that all of the health checks are OK, across the board.
Add a route to deis-router through the GUI so you can reach deis.[basedomain], register a user, create an app... pop into the Service definition and find out what nodeport has been assigned, so you can correct the [deis] remote in .git/config (the nodeport that points to TCP port 2222 is the one that you want)...
Add your keys with deis keys:add
Do git push deis master
Pop over to the Monitoring dashboard of Openshift and watch your build!
Add another route to allow traffic into your app via OpenShift's router and Deis's router (or add a wildcard route for *.[basedomain], or short-circuit any of these if necessary... since OpenShift's router does not support non-HTTP traffic you will be stuck with some short-circuiting) and enjoy!
Unfortunately deis-monitor-telegraf pod is still spewing errors out in its log, although it appears to be passing the health checks I believe it is not collecting any of the host logs, not sure but unlikely if it will get pod logs either.
Otherwise I think everything still worked at this point (with the same stack I used 19 days ago.)  Unfortunately this time I tried bringing up an example app (the ruby-sinatra buildpack example) and noticed that sadly, it does not come up.  (Curious as it worked last time.)
mkdir: cannot create directory '/app/objectstore': Permission denied
/bin/get_object: line 9: /app/objectstore/minio/builder-bucket: No such file or directory
2017/09/16 21:06:12 open /app/objectstore/minio/builder-bucket: no such file or directory

Likely owing to RBAC differences in OpenShift that we discovered a few weeks ago, that I was much more aggressively tearing down and short-circuiting in order to see if I could make it work.  You could easily work around this by disabling all of the nice security features.  It's going to take at least another look or two in order to figure out the "least required privilege" settings and implement it properly.
I don't know who to blame yet, but it obviously can't be Deis as running on OpenShift was never a supported configuration... amirite?  Next step for me is to try the newer Deis release on the older MiniShift and vice versa, and see what else is different.

From @kingdonb on September 18, 2017 16:9
My first problem was that I had omitted use_openshift_rbac setting this time (in my repo's openshift-deis-v2.18 branch, which is where you should start if you're trying to use this), obviously resulted in all manner of things failing and has been corrected in the master branch there now.
Second problem was I wasn't understanding how roles and scc/securityContext settings interact: first the SA needs to have securityContextConstraint or scc relaxed, then the container that is going to get that securityContext needs to have it assigned within the pod container spec.
I added to my deployment and daemonset files for controller, fluentd, monitor-telegraf, and registry-proxy, inside of use_openshift_rbac blocks:
securityContext:
  privileged: true

Then, from the top:
dvm use 1.12.3
git clone -b openshift-deis-v2.18 https://github.com/kingdonb/openshift-deis workflow
minishift start --vm-driver=virtualbox --memory=4GB --cpus=4
eval $(minishift docker-env)

oc login -u system:admin
oc new-project deis

oc create sa tiller
oc adm policy add-cluster-role-to-user cluster-admin -z tiller
helm init --service-account tiller --tiller-namespace deis
export TILLER_NAMESPACE=deis

# I think this part can be done inside of the YAML, but not sure how yet...
oc adm policy add-cluster-role-to-user cluster-admin -z deis-router
oc adm policy add-cluster-role-to-user cluster-admin -z deis-builder
oc adm policy add-cluster-role-to-user cluster-admin -z deis-controller
oc adm policy add-cluster-role-to-user cluster-admin -z deis-logger-fluentd
oc adm policy add-cluster-role-to-user cluster-admin -z deis-workflow-manager

# ...There is no way to assign SCC's through the API yet, per claytonc
oc adm policy add-scc-to-group anyuid system:serviceaccounts:deis
oc adm policy add-scc-to-user hostnetwork -z deis-router
oc adm policy add-scc-to-user privileged -z deis-controller
oc adm policy add-scc-to-user privileged -z deis-logger-fluentd
oc adm policy add-scc-to-user privileged -z deis-monitor-telegraf
oc adm policy add-scc-to-user privileged -z deis-registry-proxy

oc policy add-role-to-user admin developer -n deis
helm install -n deis --namespace deis workflow/

Everything appears to come online now, almost without issue!  You can assign the cluster-admin cluster-role to your developer user if you want to watch your app come up in its own project on the OpenShift dashboard.
I haven't assigned any blanket privileged or cluster-admin permissions, which is better.
I may have granted some privileges unnecessarily (I'll go back and clean it up... I'm not sure)  I am still figuring out RBAC and will harden this up but it should probably not be needed to grant anyuid to every serviceaccount in the system:serviceaccounts:deis group including the default serviceaccount there.

A few problems to still address:

[X] deis-registry-proxy does not have any serviceAccount, so from the oadm policy statements above, one granting privileged scc does not work... adding the scc privilege toserviceaccount deis-registry-proxy is the one that does not work.  I don't know what registry-proxy does but I'm sure it's important somehow. (Edit: fixed this in kingdonb/openshift-deis:openshift-deis-v2.18)
[X] I have not tried deploying anything with this yet, but I am speculating it will fail because At least the first buildpack example I tried (ruby-sinatra-example) expected to be run with UID 0.  I have not found a better way to allow this yet than oc adm policy add-scc-to-group anyuid system:authenticated which you may not want to do, but all of the solutions I can think of either involve changes to the internal permissions model of Deis, or are just as permissive.
[ ] Telegraf pods try unsuccessfully to reach the readonly API for metrics at <IP_ADDRESS>:10255.  The pods for telegraf still pass health checks but they are not collecting metrics from the cluster.  Unfortunately 10255 is not an option in OpenShift and this will probably need to be adjusted on the telegraf side to use a serviceaccount and the kubernetes.default API endpoint instead.
[ ] I think I have hardcoded some "deis" namespaces into my changes for use_openshift_rbac and that's also not right.  I shouldn't assume you're deploying into the deis namespace, I should use whatever namespace that Deis actually gets deployed into, I'm not sure how yet...

For now, make sure helm deploys the workflow chart into the deis namespace if you want it to work, or adjust the changes I've made accordingly for whatever namespace you're deploying into.
The good news is, all of this works with each of the latest stable releases, of minishift and Deis Workflow.  You can start Deis, register a user, deploy an app, and upgrade it.  Yahoo!

From @kingdonb on September 18, 2017 14:27
@liggitt Thanks!  I see there's a new Minishift release an hour ago this A.M. too.
I'd imagine we're always going to need to use OpenShift objects if we want to assign special scc privileges, like anyuid or hostnetwork, unless Kubernetes proper has (or grows) a similar protection that can be mapped to these scc's.
I will go ahead and try with a 3.7.0-alpha build as well, if this can be simplified in newer versions that's great but I'm pretty sure there will remain enough differences that we're going to need more adjustments (and probably will always need a use_openshift_rbac mode) to make everything work.


RHOAIENG-1119: Don't rely on imagestream in image info task

This PR stacks upon #215 as another preparatory commit for RHOAIENG-1119. It will be held until that one is merged, but it can still be reviewed, by just reviewing the new commit(s).
Description
With this change, we now get info for the image from the image metadata using skopeo inspect, rather than from the ImageStream resource in OpenShift.
How Has This Been Tested?

make go-test runs successfully
Example pipelineruns executed manually with success

Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work

/hold until #215 is merged.

/unhold

Going to close and re-open in an attempt to get GitHub to automatically update the PR view (it's out of date and showing a commit that's already on main).

Going to close and re-open in an attempt to get GitHub to automatically update the PR view (it's out of date and showing a commit that's already on main).

Wow, that actually worked. This commit should now be even easier to review

/retest

This /lgtm when test goes green

@Sara4994 thanks for the review! You don't need to wait for the test status check to go green before adding the /lgtm (or approve using the review feature), since it won't automatically merge until it has both the lgtm label and the test job passes. So please just /lgtm :slightly_smiling_face:

/retest

/retest

/retest


/usr/bin/ld: cannot find -lLLVMExtensions

When I tried to build the clam, in the last few steps: it shows the following.
[ 57%] Built target SeaInstCombine
[ 57%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/AnalysisWrappers.cpp.o
[ 59%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/CallGraphPrinterWrapper.cc.o
[ 61%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/GraphPrinters.cpp.o
[ 61%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/BreakpointPrinter.cpp.o
[ 61%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/NewPMDriver.cpp.o
[ 62%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/PassPrinters.cpp.o
[ 62%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/PrintSCC.cpp.o
[ 64%] Building CXX object llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/opt.cpp.o
[ 64%] Linking CXX executable ../../../bin/seaopt
/usr/bin/ld: cannot find -lLLVMExtensions
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/build.make:319: bin/seaopt] Error 1
make[1]: *** [CMakeFiles/Makefile2:2383: llvm-seahorn/tools/opt/CMakeFiles/seaopt.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 64%] Linking CXX static library ../../../lib/libSeaDsaAnalysis.a
[ 64%] Built target SeaDsaAnalysis
make: *** [Makefile:152: all] Error 2
Which part of LLVM did I miss?
I used both from the " cmake --build . --target llvm && cmake .. " and also a local version. Both had the same ld error.
Thank you very much and any suggestions would be much appreciated.
System: ubuntu 20.04

Probably you are using LLVM 10.0.0 and it's buggy. Instead,  use LLVM 10.0.1.

There is also a branch dev11 that works with LLVM 11 in case it's easy for you.

@caballa Thank you so much for the suggestions! You were right LLVM 10.0.0 is buggy and LLVM 11 works smoothly.


Will the configs in Detectron be added?

Hello~The defaults.py has only a few configures in test-time compared with the detectron  version. Will some choice  like softnms be added soon?

Hi,
I'll be adding more options to the config, for example Keypoints and test-time data augmentation.
I was not thinking about softnms in particular, do you use it during training as well?

Hi,
I'll be adding more options to the config, for example Keypoints and test-time data augmentation.
I was not thinking about softnms in particular, do you use it during training as well?
Thanks for the future config improvement. As for the softnms, I just want to use it during testing time to get better performance. I sincerely hope you can  also add the softnms like the detectron do. Thank you!


Add config to restrict token type used in application creation

Related PRs
https://github.com/wso2/carbon-apimgt/pull/8738

Closing this PR as from 3.2.0 onwards the only application token type would be JWT.


Typo correction: spell NTLM correctly (not NTML)

Switch the letter ordering of the word "NTLM" in the Example 2 subheading so it becomes correct:

old spelling "Example 2: NTML relay - Juicy Potato malware variant"
new spelling: "Example 2: NTLM relay - Juicy Potato malware variant"

Closes #8081

Simple 2 letter switch to correct 1 word: NTML => NTLM

#sign-off

It's a pleasure doing these formal, simple tasks. May as well thank @ShaXam-King for pointing out the typo in the first place (I tend to forget to include thanking the originator, now that I have resigned from using my old-school, full-format PR template).


How to generate key and crt files

How to generate the key and crt files? Or where do I get them from?
...
  key: fs.readFileSync('./example/localhost.key'),
  cert: fs.readFileSync('./example/localhost.crt')
...

Example keys are included in the example directory! What error you get and where do you try to run the example server? It should work without any problem with the included cert and key.

I started the server process (node server.js) and then I did curl localhost:8080. No response comes, nor in curl nor in browser.
However, my question is where did you get these files from? How to generate such files?

These are standard PEM key and crt files. See the first few paragraphs of the node TLS docs for example: https://nodejs.org/api/tls.html or this tutorial: http://www.akadia.com/services/ssh_test_certificate.html
When testing in browser or curl, use HTTPS:// since the example server server http2 over tls. Second, try getting the server.js file. There is no index.html to get from the example server.

Thanks! So with openssl we can generate them.
When I provide the https protocol I get an error:
$ curl https://localhost:8080
curl: (60) SSL certificate problem: self signed certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

Yeah, this is a self signed cert, and maybe also expired, so it's time to regenerate it :/ How about curl with the --insecure option? Like this: curl -k https://<IP_ADDRESS>:8080/server.js It works for me, however this falls back to HTTP1 as seen in the server log:
mg-mac:node-http2 mg$ HTTP2_LOG=info node ./example/server.js
09:18:28.547Z  INFO server/http: Creating HTTP/2 server over TLS
09:18:28.557Z  INFO server/http: Listening for incoming connections (on=8080)
09:18:30.959Z  INFO server/http: Falling back to simple HTTPS (client=<IP_ADDRESS>:52118, protocol=false, SNI=false)

My curl is probably not up to date enough to support HTTP2.

@molnarg It works fine with the -k option.
I generated with openssl new key and crt files (replacing the old ones) and I still get the error above. What could be the reason?

Self signed cert, or you generated a CA then a cert? If the former, I'd try the latter, and add the CA to the list of root CAs. But btw, you would need a real, valid cert if you want to use http2 on the web...

a real, valid cert if you want to use http2 on the web - is this paid or...?

Yes. But there are a few alternatives, see https://www.startssl.com/ and https://letsencrypt.org/ . I will close this ticket now. I suggest you to have a valid cert first and try with that. The demo server and cert is only for experimentation.

@molnarg Thanks a lot! :+1:


Headers should be a serie of '#' followed by at least one whitespace

Using Parsedown 1.6.
Currently, this
 #nottitle

will render as:

nottitle

As specified in CommonMark specification:

The opening sequence of # characters must be followed by a space or by the end of line.

Also:

At least one space is required between the # characters and the heading’s contents, unless the heading is empty. Note that many implementations currently do not require the space. However, the space was required by the original ATX implementation

Expected result, same as Github:
#nottitle
Use case: I extended Parsedown to support #hashtags in my software, and hashtags set at the beginning of a new line now generate a header.
EDIT: that's probably not the best way to do it, but I added this to my Parsedown extension to fix it:
protected function blockHeader($line)
{
    if (preg_match('/^#+ /', $line['text'])) {
        return parent::blockHeader($line);
    }
}

This seems like a duplicate of https://github.com/erusev/parsedown/issues/265

Well #265 seems to be the exact opposite (and the issue closed), but yes, that's related.
In this thread, you said that this behaviour is expected, due to existing texts. Why not, but what specification should we refer to when using Parsedown?

There's a long and ongoing discussion about the topic at CommonMark forums - looks like this is a rule that might change in future versions of CommonMark.

@erusev I know that https://github.com/erusev/parsedown/issues/265 happened which previously caused a fix for this issue to be reverted. Given that this occurred over three years ago, I wonder whether given the success of CommonMark in that time, it might be best to behave according to spec by default? Perhaps we could add a setter for those that really don't want to have a space there.

I really need proper support for lines that begin with #hashtags, per the CommonMark and GFM specs. Please consider pull request #589.

Closing as #589 was merged.
The current plan is to add a strict mode in 1.8.0 (as suggested in  https://github.com/erusev/parsedown/issues/265#issuecomment-70016695) to hopefully side step this dichotomy between supporting a commonly used deviation from the spec, and honouring the spec.
Currently this is the only case that strict mode controls, but this opens up the possibility of releasing very breaking (but compliant) changes in the future into strict mode (similar to this case) since the intent of the setter is to be compliant.
I think it is wise to be as compliant as possible even when strict mode is off, but if there are changes where compliance generates major unforeseen breakage in existing texts (like #265) then it is nice to be able to limit that change to those that want strict compliance.


Run the setup playbook if we see that an upgrade has happened

The playbook needs to run on upgrades and ansible tower will refuse to start if it isn't.
https://bugzilla.redhat.com/show_bug.cgi?id=1465590

@simaishi It's likely that this will conflict with https://github.com/ManageIQ/manageiq/pull/15423 so I'll look at a direct-to-fine PR.

@carbonin marking as fine/conflict, please remove the label when you have a PR for Fine branch. Thanks!

Backported to Fine via https://github.com/ManageIQ/manageiq/pull/15487


07/22/2018 20:53 : Create Pull Request for branch master -> prTesting1_2018072220

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestRef1.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Fix #6725][Fix #6540] Mark Style/SymbolProc as unsafe for auto-correct

Marking Style/SymbolProc as unsafe for auto-correct, since there are many cases where this is unsafe.
Some examples: #6725, #6540, #6448, #6052, #5098, #5009, #3071, #2135, #2588

Before submitting the PR make sure the following are checked:

[X] Wrote good commit messages.
[X] Commit message starts with [Fix #issue-number] (if the related issue exists).
[X] Feature branch is up-to-date with master (if not - rebase it).
[X] Squashed related commits together.
[ ] Added tests.
[X] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[X] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[X] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Today I just randomly found out an issue I had forgotten about has been fixed less than a month ago. Thanks for all the work and effort you all put into rubocop!


CVE-2023-39323 - Go version 1.20.8 has a critical vulnerability.

Environmental Info:
K3s Version: 1.28.2

The go version of 1.20.8 has a critical vulnerability - https://nvd.nist.gov/vuln/detail/CVE-2023-39323
k3s need to update go-version to 1.20.9 or 1.21.0 where in the fix is available

We update golang when upstream Kubernetes updates the version of golang that they use to build Kubernetes. I expect that the patch releases coming out this week will use a new version.
Closing as this is handled by our normal release procedures.

Also, this only affects the go compiler, not binaries built with Go, so I don't see how it affects our released artifacts in any way.


Observability documentation duplicates text

Glancing through this page:
https://docs.cilium.io/en/v1.9/concepts/observability/intro/
The second and third paragraphs seem substantially similar, it seems like maybe a merge was mishandled to include multiple copies of the text. This affects the master version of the docs as well. It would be good to review the text and streamline it to avoid duplication.

This page seems to be gone, closing.

Still affects v1.13 or newer on this link: https://docs.cilium.io/en/v1.12/concepts/observability/intro/

Can I take a look at this ?

Based on the commit : https://github.com/vipul-21/cilium/commit/160d6b920e9c8ab0fb1d22c5c488448b690c4aed it does not seem like a merge was.
Might need rephrasing of the two paragraphs i think


Playbook dev HEAD has Gemini HTTP 500 error

If I spin up a fresh Playbook VM using dev branch (755f9d1 (HEAD -> refs/heads/dev, refs/remotes/origin/dev, refs/remotes/origin/HEAD) d9 build (#196)) the system builds and I can log in as admin, but on adding node/1 (Model=Collection) I get "The website encountered an unexpected error. Please try again later." and watchdog shows
GuzzleHttp\Exception\ServerException: Server error: GET http://<IP_ADDRESS>:8000/gemini/35282c65-f955-4cc6-bc03-632127311583resulted in a500 Internal Server Error response: <!DOCTYPE html> <html> <head> <meta charset="UTF-8" /> <meta name="robots" content="noindex,nofollow (truncated...) in GuzzleHttp\Exception\RequestException::create() (line 113 of /var/www/html/drupal/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php).
Also, the playbook now defaults to a Drupal 9.x build, but it would be good to document what's needed to build to 8.9.x instead; is inventory/vagrant/group_vars/webserver/drupal.yml the only place that Drupal core version needs to specified?
^ @dannylamb

Building from b4d6ac6 Update Tomcat groups_vars for Cantaloupe 4+ (#194) seems to work ok.

Building from b4d6ac6 Update Tomcat groups_vars for Cantaloupe 4+ (#194) seems to work ok.

Rebuilt using commit 613f1ca but similar outcome. Looking at gemini.log:
[2021-01-26 16:25:04] app.INFO: Matched route "{route}". {"route":"GET_uuid","route_parameters":{"_controller":"gemini.controller:get","uuid":"876edd9c-8a32-4a8d-a340-3773f7958887","_route":"GET_uuid"},"request_uri":"http://<IP_ADDRESS>:8000/gemini/876edd9c-8a32-4a8d-a340-3773f7958887","method":"GET"} []
[2021-01-26 16:25:04] app.DEBUG: Checking for guard authentication credentials. {"firewall_key":"default","authenticators":1} []
[2021-01-26 16:25:04] app.DEBUG: Checking support on guard authenticator. {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: Calling getCredentials() on guard authenticator. {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] crayfish.syn.jwt_authentication.DEBUG: Token: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE2MTE2OTk3NTYsImV4cCI6MTYxMTcwNjk1NiwiZHJ1cGFsIjp7InVpZCI6IjEifSwid2ViaWQiOiIxIiwiaXNzIjoiaHR0cHM6XC9cL2xvY2FsaG9zdDo4MDAwIiwic3ViIjoiYWRtaW4iLCJyb2xlcyI6WyJhdXRoZW50aWNhdGVkIiwiYWRtaW5pc3RyYXRvciIsImZlZG9yYWFkbWluIl19.HJI_u4wQdFaoi2hE0VAY1YpkDTMvviWTc6whTQ2AThksM8ej2ggIraJ615Y9MvBBsQl9XnbK434MTFekRsZiKY7t7MmvkD2qUabQuyCrNXt2XezjHw8DgFkyRw9L7ZeADkpL46l8oAnuJGRkvcbdJC4CxFRF7bCy10SlZY4JpDq8T6byMQ7wD5ImkWq-gXegCBJJP5eVgxKkqxrF5DTYgF7sMfzyoALgm3QVZTs_ciZRcRy3hWebOkGQ39LwwMPk9I0WkBzW4ZuV5zbEq9pNxGkLj9mKmN-SxK_Tzxg0D3zhiHYNND2lJWaErEQQy9_erx4xdYpd1QISykWJ3Mf2bA [] []
[2021-01-26 16:25:04] app.DEBUG: Passing guard token information to the GuardAuthenticationProvider {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.INFO: Guard authentication successful! {"token":"[object] (Symfony\\Component\\Security\\Guard\\Token\\PostAuthenticationGuardToken: PostAuthenticationGuardToken(user=\"admin\", authenticated=true, roles=\"authenticated, administrator, fedoraadmin\"))","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: Guard authenticator set no success response: request continues. {"authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: Remember me skipped: it is not configured for the firewall. {"authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: > GET /gemini/876edd9c-8a32-4a8d-a340-3773f7958887 [] []
[2021-01-26 16:25:04] app.CRITICAL: Pimple\Exception\UnknownIdentifierException: Identifier "crayfish.drupal_domain" is not defined. (uncaught exception) at /var/www/html/Crayfish/Gemini/vendor/pimple/pimple/src/Pimple/Container.php line 101 {"exception":"[object] (Pimple\\Exception\\UnknownIdentifierException(code: 0): Identifier \"crayfish.drupal_domain\" is not defined. at /var/www/html/Crayfish/Gemini/vendor/pimple/pimple/src/Pimple/Container.php:101)"} []
[2021-01-26 16:25:04] app.DEBUG: < 500 [] []

Rebuilt using commit 613f1ca but similar outcome. Looking at gemini.log:
[2021-01-26 16:25:04] app.INFO: Matched route "{route}". {"route":"GET_uuid","route_parameters":{"_controller":"gemini.controller:get","uuid":"876edd9c-8a32-4a8d-a340-3773f7958887","_route":"GET_uuid"},"request_uri":"http://<IP_ADDRESS>:8000/gemini/876edd9c-8a32-4a8d-a340-3773f7958887","method":"GET"} []
[2021-01-26 16:25:04] app.DEBUG: Checking for guard authentication credentials. {"firewall_key":"default","authenticators":1} []
[2021-01-26 16:25:04] app.DEBUG: Checking support on guard authenticator. {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: Calling getCredentials() on guard authenticator. {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] crayfish.syn.jwt_authentication.DEBUG: Token: eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpYXQiOjE2MTE2OTk3NTYsImV4cCI6MTYxMTcwNjk1NiwiZHJ1cGFsIjp7InVpZCI6IjEifSwid2ViaWQiOiIxIiwiaXNzIjoiaHR0cHM6XC9cL2xvY2FsaG9zdDo4MDAwIiwic3ViIjoiYWRtaW4iLCJyb2xlcyI6WyJhdXRoZW50aWNhdGVkIiwiYWRtaW5pc3RyYXRvciIsImZlZG9yYWFkbWluIl19.HJI_u4wQdFaoi2hE0VAY1YpkDTMvviWTc6whTQ2AThksM8ej2ggIraJ615Y9MvBBsQl9XnbK434MTFekRsZiKY7t7MmvkD2qUabQuyCrNXt2XezjHw8DgFkyRw9L7ZeADkpL46l8oAnuJGRkvcbdJC4CxFRF7bCy10SlZY4JpDq8T6byMQ7wD5ImkWq-gXegCBJJP5eVgxKkqxrF5DTYgF7sMfzyoALgm3QVZTs_ciZRcRy3hWebOkGQ39LwwMPk9I0WkBzW4ZuV5zbEq9pNxGkLj9mKmN-SxK_Tzxg0D3zhiHYNND2lJWaErEQQy9_erx4xdYpd1QISykWJ3Mf2bA [] []
[2021-01-26 16:25:04] app.DEBUG: Passing guard token information to the GuardAuthenticationProvider {"firewall_key":"default","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.INFO: Guard authentication successful! {"token":"[object] (Symfony\\Component\\Security\\Guard\\Token\\PostAuthenticationGuardToken: PostAuthenticationGuardToken(user=\"admin\", authenticated=true, roles=\"authenticated, administrator, fedoraadmin\"))","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: Guard authenticator set no success response: request continues. {"authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: Remember me skipped: it is not configured for the firewall. {"authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
[2021-01-26 16:25:04] app.DEBUG: > GET /gemini/876edd9c-8a32-4a8d-a340-3773f7958887 [] []
[2021-01-26 16:25:04] app.CRITICAL: Pimple\Exception\UnknownIdentifierException: Identifier "crayfish.drupal_domain" is not defined. (uncaught exception) at /var/www/html/Crayfish/Gemini/vendor/pimple/pimple/src/Pimple/Container.php line 101 {"exception":"[object] (Pimple\\Exception\\UnknownIdentifierException(code: 0): Identifier \"crayfish.drupal_domain\" is not defined. at /var/www/html/Crayfish/Gemini/vendor/pimple/pimple/src/Pimple/Container.php:101)"} []
[2021-01-26 16:25:04] app.DEBUG: < 500 [] []

Ah, that' some config that was added to Gemini to make it so you could port your data between domains (e.g. stage.example.org to prod.example.org).
To make it work now, you need to add these two empty variables to your config.yaml
drupal_domain:
fcrepo_domain:

But this is only happening because I was sloppy and am not providing a default value.  I can add a try/catch block a la https://github.com/Islandora/Crayfish/blob/dev/Milliner/src/app.php#L23-L30 and provide those for you so you're not forced to add it to your config.

Ah, that' some config that was added to Gemini to make it so you could port your data between domains (e.g. stage.example.org to prod.example.org).
To make it work now, you need to add these two empty variables to your config.yaml
drupal_domain:
fcrepo_domain:

But this is only happening because I was sloppy and am not providing a default value.  I can add a try/catch block a la https://github.com/Islandora/Crayfish/blob/dev/Milliner/src/app.php#L23-L30 and provide those for you so you're not forced to add it to your config.

Do you remember which config yml file this is exactly?

To make it work now, you need to add these two empty variables to your config.yaml
drupal_domain:
fcrepo_domain:

its in the gemini config at /var/www/html/Crayfish/Gemini/cfg/config.yaml

thank you --- I just found it because I assumed the file extension was a typo so I was searching for "config.yml", but yaml is the same as yml.

Also I think its fedora_domain not fcrepo_domain ?

Hello! We are no longer using Gemini. Please update Islandora and you should no longer experience this issue. If you are still encountering an issue, please feel free to open or resubmit your issue.


html canvas tag support

the htmlunit is not supporting canvas tag in html.

The canvas tag is supported.
Can you please provide some more details and some sample code to  reproduce your problem.
And the version of HtmlUnit you are using.
Any log output?

The canvas tag is supported.
Can you please provide some more details and some sample code to reproduce your problem.
And the version of HtmlUnit you are using.
Any log output?
HtmlUnit maven Dependency that I'm using is

net.sourceforge.htmlunit
htmlunit
2.13

code that im using is
URL url = new File("in1.html").toURI().toURL();
        System.out.println(url);
        WebClient webClient = new WebClient(BrowserVersion.CHROME);
        HtmlPage page = webClient.getPage(url);
        page.save(new File("C:\\Users\\cs3339\\Desktop\\sample13.html"));

        OutputStream os = new FileOutputStream("C:\\Users\\cs3339\\Desktop\\latestDoc17.pdf");

        ITextRenderer renderer = new ITextRenderer();
        renderer.setDocument(page, url.toString());
        renderer.layout();
        renderer.createPDF(os);

the HTML file is

Your browser does not support the HTML5 canvas tag.

Sorry but your version is really really old. Please use the latest version.

Sorry but your version is really really old. Please use the latest version.
See http://htmlunit.sourceforge.net/changes-report.html

net.sourceforge.htmlunit
htmlunit
2.36.0

i even tried using this maven dependency....this is the latest one...sorry it isnt working

Your html looks like the text is always there. You are sore the IText renderer can handle the canvas?

Have a look at the test cases in https://github.com/HtmlUnit/htmlunit/tree/master/src/test/java/com/gargoylesoftware/htmlunit/javascript/host/canvas

INFO: CanvasRenderingContext2D.closePath() not yet implemented im getting this log from htmlunit not from itext

Have added the implementation of the close method. Will inform via twitter if a new snapshot is available.

Have you found some time to test the new snapshot?

No response - will close this. Please open a new issue if you find more problems.


(wip): YJDH-154: add missing component tests

Description :sparkles:
Issues :bug:
Testing :alembic:
Screenshots :camera_flash:
Additional notes :spiral_notepad:

Codecov Report

Merging #177 (37e485b) into develop (b0eed81) will decrease coverage by 5.86%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #177      +/-   ##
===========================================
- Coverage    67.46%   61.59%   -5.87%
===========================================
  Files          272       50     -222
  Lines         5625      651    -4974
  Branches       425       87     -338
===========================================
- Hits          3795      401    -3394
+ Misses        1793      232    -1561
+ Partials        37       18      -19

Impacted Files
Coverage Δ

backend/benefit/companies/tests/factories.py

...ckend/benefit/companies/migrations/0001_initial.py

frontend/shared/src/utils/string.utils.ts

...orms/application/deMinimisAid/DeMinimisAidForm.tsx

frontend/benefit/applicant/.eslintrc.js

backend/benefit/applications/tests/conftest.py

backend/shared/shared/oidc/views/mock_views.py

...enefit/applicant/src/hooks/useApplicationsQuery.ts

backend/shared/shared/azure_adfs/auth.py

...nefit/applicant/src/context/ApplicationContext.tsx

... and 212 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b0eed81...37e485b. Read the comment docs.

sonarcube's security findings are from test files which can be ignored. I wonder how those findings could be disabled


Allow additional markup outside docstrings

It would be nice to be able to structure the documentation by allowing additional markup in a module file, that is not attached to a function. For example:
/// == Utility functions
/// The following functions are utilities.

/// A foo function.
#let foo(a, b) = [#a-#b]

/// == Math functions
/// Some math markup

#let inv(x) = $1/x$

After rendering and parsing, the headings and descriptive text would render between function documentation.
Open question:

How to handle this additional markup with sort-function?

Hi @jneug ,
I would actually suggest putting utility and math functions into two different modules, utilities.typ and math.typ. You can include both from within your main lib file,
// lib.typ
#import "utilities.typ": *
#import "math.typ": *

(so to the outside it makes no difference) and show both modules individually. You then have more control on how everything is styled. In the future, I also plan on adding file descriptions.

The possibility to add doc-strings to variables (e.g. dictionaries) would be very nice, too!

Hi @johannes-wolf yes, this is definitely necessary! You can already put docstrings in front of variables and they will be recognized. Unfortunately, they are recognized as functions 😄. The initial idea was that this way we can document alias functions
#let my-line = line.with(stroke: 2pt)
but I think this behaviour should be changed. The resulting dictionary from the parse-module() step will then contain another list at "definitions" in addition to "functions".

Hi @jneug,
as stated above, I would suggest putting two different "parts" of the code into two modules for your purpose. Otherwise, the order of the function declarations also matters for the documentation which is not ideal, I feel.
However, we can add support for file/module descriptions that can be put at the top of a file. If you have suggestions regarding this, feel free to post them here :)

Closing with merge of #26


Extensions to QC flags in YAML stubs for conventional and AMSU-A observations

Description
This PR adds extensions to quality control flags in YAML stubs for conventional and amsua observations from NOAA-15, 18, and 19, Aqua, and Metop-A and B satellites. The filters used are based on Junmei's newest code: /glade/scratch/jban/pandac/testpr_220126/DA/2018041500/3denvar.yaml. Observation errors for amsua observations are already provided in the observations files as part of the conversion from BURF to IODA-v2 using Jamie's converter. A Gaussian thinning filter is added for the variational data assimilation in which the thinning distance is 200km for the 120km mesh and 145km for finer meshes. A new variable is added to config/experiment.csh to indicate the source of the satellite bias coefficients that are needed for the amsua bias correction. All related satellite bias configuration is moved to satbiasdata.csh.
Issue closed
Closes https://github.com/NCAR/MPAS-Workflow/issues/43
Tests completed

WarmStart

[x] OIE120km 3dvar, 2 cycles
[x] OIE120km 3denvar, 2 cycles
[x] OIE120km eda_3denvar, 2 cycles
[x] O30kmIE60km 3denvar, 2 cycles

ColdStart

[x] OIE120km 3dvar, 2 cycles, PreprocessObs=True, Satbias=GDASFixed

Thank you for submitting the PR @ibanos90.  This will take me a while to review as there are a lot of intricate file changes.  I will look through it a little today to see if there is any high-level feedback I can provide.

Thank you for submitting the PR @ibanos90. This will take me a while to review as there are a lot of intricate file changes. I will look through it a little today to see if there is any high-level feedback I can provide.

Sounds good. Thanks @jjguerrette.

Note that I just merged develop in order to resolve conflicts that I created in #71.  There were only 3 conflicts. One of them I resolved by adding back config/ObsPlugs/hofx/sondes-2018043006.yaml.  If that file was purposefully removed and there is a plan to handle the case for missing specific_humidity, please remove the yaml stub again.

I also created a version of this branch called merge/QCflags_YAMLS/autotest_cases, which includes a quasi-clean merge of feature/autotest_cases.  I did not test it yet, but it is probably a better starting point than feature/QCflags_YAMLS since it utilizes all of the auto-testing features and YAML configuration methods.
I added a lot of in-line comments in merge/QCflags_YAMLS/autotest_cases, especially with regard to the satellite bias correction.

The main point is that we need to be able to turn off bias correction completely in order to run sensitivity tests for new instruments.  In order to keep using the PANDACArchive data (which is pre-bias-corrected), we would like None to be the default bias correction setting and not GDASFixed.  I understand that will make the YAML settings difficult to set up, so let's talk about it when you return.  Possibly we can add a directory like config/ObsPlugs/biasCorrect that includes the yaml stubs with the obs bias section of each instrument.

In this PR, please only add the bias correction to the Variational application (PrepVariational instead of PrepJEDI).  If we want to turn on bias correction for HofX later, we can shift your working components around as needed.

I also created a version of this branch called merge/QCflags_YAMLS/autotest_cases, which includes a quasi-clean merge of feature/autotest_cases. I did not test it yet, but it is probably a better starting point than feature/QCflags_YAMLS since it utilizes all of the auto-testing features and YAML configuration methods.
I added a lot of in-line comments in merge/QCflags_YAMLS/autotest_cases, especially with regard to the satellite bias correction.
1. The main point is that we need to be able to turn off bias correction completely in order to run sensitivity tests for new instruments.  In order to keep using the `PANDACArchive` data (which is pre-bias-corrected), we would like `None` to be the default bias correction setting and not `GDASFixed`.  I understand that will make the YAML settings difficult to set up, so let's talk about it when you return.  Possibly we can add a directory like `config/ObsPlugs/biasCorrect` that includes the yaml stubs with the `obs bias` section of each instrument.

2. In this PR, please only add the bias correction to the `Variational` application (`PrepVariational` instead of `PrepJEDI`).  If we want to turn on bias correction for `HofX` later, we can shift your working components around as needed.

Thanks @jjguerrette for the review and creating a version of this branch with the recent changes of the develop branch. That was very helpful!
I am just wondering if the bias correction coefficients needed for the satellite data assimilation should be added in PrepVariational or keep it in PrepJEDI since in PrepJEDI is where the yaml substitution related to the observations takes place.
I will close this PR for now, but changes based on the suggested modifications and comments will be submitted in a following PR.


Improve Documentation For single_line_if_else_max_width in Configurations.md + Add Tests

The single_line_if_else_max_width  is one of many configuration options that could use some improvements to it's documentation. It would be great to:

Add code snippets to highlight the functionality this option provides
Describe that the behavior of this option changes when version=Two is set.

Additionally, some tests for the options should be added to tests/source/configs/ and tests/target/configs/. See the Create test cases section of the Contributing guide for more details. It would be great to have test cases for both version=One and version=Two.

Hi @ytmimi, I would like to work on this. Could you please add me as the assignee? I am currently in the processes of trying to understand the details of how the single_line_if_else_max_width option works by stepping through the code. I might have some questions.

That would be great! You can actually assign yourself by commenting @rustbot claim. Check out the Mastering @rustbot   Guide.

@rustbot claim

While generally speaking I'm open to adding any relevant clarifying info to the docs that better explain a config option, I think the issue description has too large a scope for what I think belongs in the config doc (also noted on the linked PR).
I don't think we should mention the version=Two behavior here under the docs for this option just because both happen to operate on the construct. I think it might be reasonable to update the docs for the version option to link to the broader issue we used semi recently to catalog all the v2 differences, but I don't think we should start down the path of documenting orthogonal behavior of one option under the section for another

Hey, I'm new to this stuff (and by stuff I mean rustfmt and GitHubs' issues) so sorry if I'm mistaken in any way or addressing my issue in the wrong place.
The first time I was reading the documentation I thought that this configuration will format any if-else expression into a single line if it's not exceeding the specified max width, so that it would look something like this:
// When the `single_line_if_else_max_width` would be set to 100...

// This code...
let foo = if true {
    1
} else {
    2
};

if false {
    println!("Hello");
} else {
    println!("World!");
}

// Would get formatted like this
let foo = if true { 1 } else { 2 };

if false { println!("Hello"); } else { println!("World!"); }

// But actually gets formatted like this
let foo = if true { 1 } else { 2 };

if false {
    println!("Hello");
} else {
    println!("World!");
}

I also thought that this configuration option would handle cases with short if statements such as this:
// Where this code...
if true {
    Err(SomeError::Foo)
}

// Would get formatted like this
if true { Err(SomeError::Foo) }

// But actually gets formatted like this
if true {
    Err(SomeError::Foo)
}

Now I know this option applies only to let statements consisting of if-else but it's not apparent from the first read of the documentation. A better documentation on this configuration or perhaps even a change of name to single_line_let_if_else_max_width or just let_if_else_max_width would be amazing as I'm not the first one to get confused with this.
It would also be great to have a configuration option for collapsing short if statements similar to the example below
// Where this code...
if !is_foo() {
    Err(SomeError::Foo)
}
if !is_bar() {
    Err(SomeError::Bar)
}

// Would get formatted like this
if !is_foo() { Err(SomeError::Foo) }
if !is_bar() { Err(SomeError::Bar) }

I know the last part consisting of a request for such a configuration doesn't belong here, so if you could tell me where to post it I would be really grateful


[question] Way to pipe request1 to request2

I am trying to accomplish something that may not be in scope. I want to basically stream two request where request2 depends on the response of request1.
process.stdin
.pipe(request1)
.pipe(request2)

Where request 2 needs to process request1's response to know how to make it's query. Is this possibly without having to listen to stream events?

What about something like:
process.stdin
.pipe(request1)
.pipe([a through stream that listens to the response from req1 and writes query for req2])
.pipe(request2)

@zacronos I tried that but I had no luck. Where request2 needs a correct argument via the through stream.
process.stdin
.pipe(request1)
.pipe([a through stream that listens to the response from req1 and writes query for req2])// builds arg
.pipe(request2(arg1))

In that case, I think you will have to do some nesting.  You don't necessarily have to listen to stream events if you don't want to though, if you find the right helper lib.  From what you've described, I would do something vaguely like (coffeescript):
concat = require 'concat-stream'
process.stdin
.pipe request1
.pipe concat (response1) ->
  arg = doSomethingWith(response1)
  request2(arg).pipe concat (response2) ->
    # handle response2

If that won't quite do it for you, take a look at some of the other helper libs recommended by @substack: https://github.com/substack/stream-handbook

Nice I'll look into concat-stream.

excellent this is working thanks @zacronos


Simulation fail at start

Hello,
Thank you for your work
I have trouble with simulations on PMB2 robot.
I have Ubuntu 14.04 and ROS Indigo installed. I went through this guide. When I start the simulation, Gazebo spawns with all the environment and robot in it. However, gazebo's node immediately dies after that and robot cannot be controlled.
Gazebo window stay still and working:

I attach the log file of the launch: logs.txt
Could you help me please to solve this issue?
My thoughts:

This could be caused by packages speed_limit and sensor_to_cloud absence (I could not find them on PAL Robotics' Github)
This simulation launches fine in Developer OS provided on USB-stick with real PMB2 robot. It is also Ubuntu 14.04 with ROS Indigo, but it has multiple problems with drivers(Wi-Fi and USB), so I wanted to be able launch simulations in common Ubuntu. Hope this helps.

Will be glad to any help

We've been able to reproduce this as well.
We have updated the rosinstall you have to use, it is now located at https://raw.githubusercontent.com/pal-robotics/pmb2_tutorials/indigo-devel/pmb2_public.rosinstall
Could you please try with this version?

Thank you for fast response!

I deleted previous version of pmb2_public_ws and followed the guide with new rosinstall file.
Then I run catkin_make command and it was almost successful, with errors on "ros_controllers". Catkin logs are here:
logs_catkin_make2.txt
Despite the errors, I tried to launch the simulation. Simulation started, even the blue region of laser scanning was displayed. However, there were several errors in logs and keyboard control was not working. Gazebo logs are here:
logs_gazebo2.txt
I downloaded ros-control package using

apt-get install ros-indigo-ros-control

Built the workspace again using catkin_make. Now it was more successful, with several errors. Catkin logs are here:
logs_catkin_make3.txt
Launched Gazebo simulation and everything worked! Including keyboard control :) Thank you very much!

Could you help to fix build errors please?

You have to uninstall  ros-indigo-ros-control and all ROS Control related packages.
There's some kind of interaction where it will use the ROS Control packages in /opt instead of your workspace.
You can see it also happens here: https://answers.ros.org/question/292578/is-the-tutorial-to-compile-tiago-with-ros-kinetic-working/

Yes, this helped!
More precisely, these commands:
apt-get remove ros-indigo-ros-control
apt-get remove ros-indigo-controller-interface

I pasted them here, maybe it will help someone further.
Thank you very much again!


[Form] Add help text formatting

Maybe we need form help text class in a form? Currently I use "ui pointing above label" for input element.

Moved to #1770


auto scroll enabled in contract editor for dnd

Had to just uncomment the following:
https://github.com/accordproject/web-components/blob/df00e1770b8d1a234f14a9bb0ccdfb1669f1fb38/packages/ui-contract-editor/src/lib/ContractEditor/index.js#L246.
The auto-scroll works but has a very narrow part at the top and the bottom where you'll have to drag the augmented editor to see the auto-scroll.

Closes #

Changes

Flags

Screenshots or Video

Related Issues

Issue #
Pull Request #

Author Checklist

[ ] Vital features and changes captured in unit and/or integration tests
[ ] Commits messages follow AP format
[ ] Extend the documentation, if necessary
[ ] Merging to master from fork:branchname
[ ] Manual accessibility test performed

[ ] Keyboard-only access, including forms
[ ] Contrast at least WCAG Level A
[ ] Appropriate labels, alt text, and instructions

I don't think this change is necessary. The behavior in this PR is the same as the currently deployed version of the Contract Editor here.
Can you explain more about "the very narrow part at the top and the bottom" where you need to drag to auto scroll? What causes that? Can we make it less "narrow"?

There is a very narrow part at the top and bottom around 1rem in size, if you drag the the drag.js component in this area then only you can see the auto scroll effect.
I tried increasing the padding and the border of the body (i assumed that this narrow part would be one among them), but it did not work.
I think we can add some kind of marker in that narrow portion which will be seen only during the dragging state to make it more intuitive.

Closing this PR. I'd recommend opening an issue to discuss what you'd like to see improved and we can figure out the design and plan before doing development work.


Issue: Atom v1.25 does not launch Hydrogen

I updated from Atom v1.23.3 to v1.25 and hydrogen no longer works.
Process: Originally, downloaded windows 64bit zip (v1.23.3), unzipped it and copied all files in c:\tools\atom - all has been well and been using atom for some time. Update followed the same process - download windows 64 bit v1.25 zip, unzip, overwrite all in c:\tools\atom. Atom opened properly, but executing a file with hydrogen, that worked before did not work - even trying to run it from the menu no longer works. No errors. That little window 'hydrogen kernels updated' does not show up. Deleted all v1.25 files and unzipped v1.23.3 from before back - ran it and all works just fine. Not sure what other packages are broken with v1.25.
I noticed that new atom has many more files compared to 1.23 - there are some new dlls in atom.exe directory, etc. Perhaps packing the zip was not done properly?
OS: Windows 10 64bit Home Premium
Atom: 1.23.3 (original working)
Atom 1.25 (broken hydrogen)
Hydrogen 2.3.0

Thanks for contributing!
It appears that this is being caused by the hydrogen package, not Atom itself. It may be that there is still a bug in Atom. If so, we'll need the maintainer of the package that is causing the problem to help track that down. You should file this issue on their repository so they can either fix the problem or tell us where the problem is coming from. The Atom FAQ has information on how to contact the maintainers of any Atom community package or theme.


Added signing_root to ssz_static tests

This PR adds signing_root to the test cases with hash_tree_root if object contains signature field. I think it's good enough for initial validation of clients logic without adding extra complexity with generated structures.
cc: @protolambda

@protolambda merged your fixes and did little cleanup of References in ssz_static/core.md. I think it's done now


Add code coverage tests

I think we might be able to reuse some of the same harness from rust-asn1: https://github.com/alex/rust-asn1/blob/afe141f6df60dee51fd5d78884361beaa46a8122/.github/workflows/ci.yml#L101-L170
Having 100% coverage seems like a good goal to strive for given the complexity of the parsing.

Here what I was using for locally testing coverage:
RUSTFLAGS="-C instrument-coverage"  cargo test --lib
grcov **/*.profraw --source-dir . --binary-path target/debug -t html --branch --ignore-not-existing -o target/coverage

@x-hgg-x do you want to take a stab at wiring this up in CI?

I think we should at least check if the number of missed lines doesn't increase, but with the possibility of bypassing the check when we add a new derive for example.
Here is a working script:
export LLVM_PROFILE_FILE="strftime-%m.profraw"
export RUSTFLAGS="-C instrument-coverage"

# Unstable feature: https://github.com/rust-lang/rust/issues/56925
export RUSTDOCFLAGS="-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctests"

cargo +nightly test --lib
cargo +nightly test --doc

# Produces a folder with a nice html view for manually inspecting coverage
grcov strftime*.profraw --source-dir . --binary-path target/debug -t html --filter covered -o target/coverage

# Produces a machine-readable json
# { "children": {...}, "coveragePercent": 98.48, "linesCovered": 1559, "linesMissed": 24, "linesTotal": 1583, "name": "" }
grcov strftime*.profraw --source-dir . --binary-path target/debug -t covdir --filter covered -o target/coverage.json

Maybe we can do something similary to the gh-pages branch, where a bot will push the last coverage result file to a separate branch after a pull request is merged.

oh that sounds cool, but I think it would require some significant rework. rustdoc.yaml is centrally managed across all of Artichoke's Rust repositories and is currently set to squash and overwrite.
If you'd like, maybe I can set up an S3 bucket and we can publish there?

However I don't know how to check it in CI, since we need to keep the previous coverage results.

Another option is to run the code coverage job against both the PR branch and against trunk in the same workflow and diff those results.

Another option is to run the code coverage job against both the PR branch and against trunk in the same workflow and diff those results.

This solution should work and is easier to implement.

If you'd like, maybe I can set up an S3 bucket and we can publish there?

It would be nice if we could upload the static html somewhere for manual inspection, and have a badge in the readme which has a link for it (like the codecov badge for https://github.com/time-rs/time).

Here is a working script:

This is cool! I played around with it and some bits that were missing:
cargo install grcov
rustup install nightly
rustup +nightly component add llvm-tools-preview

This is the HTML as of the latest trunk: coverage.zip
I'll set up an S3 bucket.

One thing I forgot to mention, the machine readable coverage.json from main can be fetched at https://codecov.artichokeruby.org/strftime-ruby/coverage.json.

This is the json from the html version, which does not contains detailed information with lines.

It looks like cloudfront isn't setting a TTL on the badges which means GitHub's proxy is caching them forever.
I'll take a look at the cloudfront terraform, purge the cloudfront cache, and purge the GitHub cache. I found https://github.com/sbts/github-badge-cache-buster or I can add a query parameter to the image link in the README to bust the cache.

#41 is merged and infra changes are all applied and up at https://github.com/artichoke/project-infrastructure/pull/336.
Also new is the landing page at https://codecov.artichokeruby.org/.

Do we also need to add a link for the coverage.json file for discoverability ?

If you want to PR that at project infra I can apply and merge in the morning. You mean adding a link on the index page of the codecov website right?

Yes, another link next to the Coverage Report link.


Documentation out of date

Would you be able to include detailed installation instructions for 0.50?  Presently it it looks like the installation instructions are for 0.40

Updated the detailed instructions. But it's hard to keep images up-to-date for every release. I'll create a step by step text only installation in coming days and detailed installation will be removed. Thanx for the heads up


fix(toaster): add missing toaster component in Toast

I noticed the cli generation is missing the toaster component for Toast. This PR should fix just that 🙂
And thanks for the awesome library ❤

Noticed too


Author Component

[x] name
[x] headshot (w/fallback)
[x] links to profile
[ ] handles multiple Authors
[ ] accessible

[ ] mobile second

[ ] ability to add guest authors

[ ] name
[ ] optional image (fallback to initials)
[ ] optional link (for website or twitter)


IMPRO-503: Updated argparser

Partially addresses #433
Testing:

[x] Ran tests and they passed OK
[x] Added new tests for the new feature(s)

I'm going to leave my review for now, but I'll take another look, especially at the unit tests, once you've made some of the changes.


Implement an exclusion path list

I'm working on a project that has a series of autogenerated source files that, sadly, do not conform to our desired guidelines. In order to prevent excessive changes to these autogenerated files, I would like to exclude them from being subjected to the codestyle tests.
I have a working patch and can make a pull request if this seems like a desirable feature.

@henry0312 Hey, any chance to get this merged soon? This is the only blocker for us to migrate from pytest-pep8 to pytest-pycodestyle

I've waited a response from @mattwwarren (https://github.com/henry0312/pytest-codestyle/pull/33).

Sorry, I guess I've missed some github notifications. I'll look at this today.


Removing old command-line interface

I confirmed with the other developers and it seems the command-line interface is no longer needed and in fact very much outdated, so let's remove it.

@tomwhite could you have a look?

Coverage increased (+16.8%) to 56.676% when pulling 1ae93253e667ea87f2aa3afb65ca1d5d283a1d1a on AndreSchumacher:remove_cli into 55c4af9d9a52ebdf7f933596ce432b673167944a on HadoopGenomics:master.

+1 from me. Can we remove CmdLineParser too?

I merged this and created a PR to remove CmdLineParser separately (https://github.com/HadoopGenomics/Hadoop-BAM/pull/84).


[Python] Exception performing ICP

In some cases the algorithm is crashing peforming ICP for two similar meshes. The same two point clouds (set of vertices) can converge to a result and sometimes can crash when the mesh have another set of world transforms.
Error: Python: Traceback (most recent call last):
  File "/Users/__USER__/model.blend/Text", line 163, in <module>
  File "/Applications/Blender.app/Contents/Resources/3.2/python/lib/python3.10/site-packages/simpleicp/simpleicp.py", line 211, in run
    distance_residuals_new = optim.estimate_parameters()
  File "/Applications/Blender.app/Contents/Resources/3.2/python/lib/python3.10/site-packages/simpleicp/optimization.py", line 120, in estimate_parameters
    weighted_distance_residuals = self._optim_results.residual[
TypeError: 'float' object is not subscriptable

simpleICP is (currently) for the fine alignment of the point clouds only, i.e. the point clouds must be roughly aligned in its initial state. If that's not the case, the algorithm might crash due to missing correspondences.
Obviously simpleICP currently crashes without proper error handling. Could you send me the two point clouds so that I can debug it?


rust-analyzer-unwrapped: 2022-08-01 -> 2022-08-08

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for rust-analyzer-unwrapped is: "A modular compiler frontend for the Rust language"
meta.homepage for rust-analyzer-unwrapped is: "https://rust-analyzer.github.io"
meta.changelog for rust-analyzer-unwrapped is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2022-08-08 with grep in /nix/store/dn0wa8abbg2z6i8nh4jmz2r0gmc9hcjb-rust-analyzer-unwrapped-2022-08-08
found 2022-08-08 in filename of file in /nix/store/dn0wa8abbg2z6i8nh4jmz2r0gmc9hcjb-rust-analyzer-unwrapped-2022-08-08

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer
vscode-extensions.rust-lang.rust-analyzer

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/dn0wa8abbg2z6i8nh4jmz2r0gmc9hcjb-rust-analyzer-unwrapped-2022-08-08 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A rust-analyzer-unwrapped https://github.com/r-ryantm/nixpkgs/archive/a6d179f81184e7bc389ea53cd449bedbafd17d8c.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/dn0wa8abbg2z6i8nh4jmz2r0gmc9hcjb-rust-analyzer-unwrapped-2022-08-08
ls -la /nix/store/dn0wa8abbg2z6i8nh4jmz2r0gmc9hcjb-rust-analyzer-unwrapped-2022-08-08/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer

Maintainer pings
cc @oxalica for testing.

Result of nixpkgs-review pr 186506 run on x86_64-darwin 1

3 packages built:

rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer (vscode-extensions.rust-lang.rust-analyzer)

A new version is out now.
https://github.com/rust-lang/rust-analyzer/releases/tag/2022-08-15

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for rust-analyzer-unwrapped is: "A modular compiler frontend for the Rust language"
meta.homepage for rust-analyzer-unwrapped is: "https://rust-analyzer.github.io"
meta.changelog for rust-analyzer-unwrapped is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2022-08-29 with grep in /nix/store/adv99z5jdik7qlkzqif93nrp02ywl8l5-rust-analyzer-unwrapped-2022-08-29
found 2022-08-29 in filename of file in /nix/store/adv99z5jdik7qlkzqif93nrp02ywl8l5-rust-analyzer-unwrapped-2022-08-29

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer
vscode-extensions.rust-lang.rust-analyzer

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/adv99z5jdik7qlkzqif93nrp02ywl8l5-rust-analyzer-unwrapped-2022-08-29 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A rust-analyzer-unwrapped https://github.com/r-ryantm/nixpkgs/archive/735859e3c15efd43967a2edea9285ea1ea3f2a80.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/adv99z5jdik7qlkzqif93nrp02ywl8l5-rust-analyzer-unwrapped-2022-08-29
ls -la /nix/store/adv99z5jdik7qlkzqif93nrp02ywl8l5-rust-analyzer-unwrapped-2022-08-29/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer

Maintainer pings
cc @oxalica for testing.

Test passed.

Result of nixpkgs-review pr 186506 run on x86_64-linux 1

3 packages built:

rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer (vscode-extensions.rust-lang.rust-analyzer)

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for rust-analyzer-unwrapped is: "A modular compiler frontend for the Rust language"
meta.homepage for rust-analyzer-unwrapped is: "https://rust-analyzer.github.io"
meta.changelog for rust-analyzer-unwrapped is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2022-09-05 with grep in /nix/store/084q5lx9h5k8a9kni6xr8kjbwh4vm6c5-rust-analyzer-unwrapped-2022-09-05
found 2022-09-05 in filename of file in /nix/store/084q5lx9h5k8a9kni6xr8kjbwh4vm6c5-rust-analyzer-unwrapped-2022-09-05

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer
vscode-extensions.rust-lang.rust-analyzer

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/084q5lx9h5k8a9kni6xr8kjbwh4vm6c5-rust-analyzer-unwrapped-2022-09-05 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A rust-analyzer-unwrapped https://github.com/r-ryantm/nixpkgs/archive/7fc00c1df71e3e693736820ffd8d892504fab675.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/084q5lx9h5k8a9kni6xr8kjbwh4vm6c5-rust-analyzer-unwrapped-2022-09-05
ls -la /nix/store/084q5lx9h5k8a9kni6xr8kjbwh4vm6c5-rust-analyzer-unwrapped-2022-09-05/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer

Maintainer pings
cc @oxalica for testing.

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for rust-analyzer-unwrapped is: "A modular compiler frontend for the Rust language"
meta.homepage for rust-analyzer-unwrapped is: "https://rust-analyzer.github.io"
meta.changelog for rust-analyzer-unwrapped is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2022-09-12 with grep in /nix/store/4fby3sxbsnwbxk3vnm8y8dp1d8a53qg2-rust-analyzer-unwrapped-2022-09-12
found 2022-09-12 in filename of file in /nix/store/4fby3sxbsnwbxk3vnm8y8dp1d8a53qg2-rust-analyzer-unwrapped-2022-09-12

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer
vscode-extensions.rust-lang.rust-analyzer

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/4fby3sxbsnwbxk3vnm8y8dp1d8a53qg2-rust-analyzer-unwrapped-2022-09-12 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A rust-analyzer-unwrapped https://github.com/r-ryantm/nixpkgs/archive/1a255b70264f6355bcd45771ca3f4591d6edb652.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/4fby3sxbsnwbxk3vnm8y8dp1d8a53qg2-rust-analyzer-unwrapped-2022-09-12
ls -la /nix/store/4fby3sxbsnwbxk3vnm8y8dp1d8a53qg2-rust-analyzer-unwrapped-2022-09-12/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

rust-analyzer
rust-analyzer-unwrapped
vscode-extensions.matklad.rust-analyzer

Maintainer pings
cc @oxalica for testing.


[cdc] Add link to nightly CDC build to HW dashboard

This adds a link to the nightly CDC regression results to the HW dashboard.
I still need to add a bunch of common waivers, and I believe that will cut down the number of warnings quite a bit.
But otherwise we can start to dig in - happy fixing ;).
Btwy, in order to understand the errors better, have a look at the CDC report from the tool - it provides additional descriptions of these messages.
Signed-off-by: Michael Schaffner<EMAIL_ADDRESS>

Looking at the output of the tool, I doubt that anyone outside of Google is actually going to be able to fix any of these warnings :-( But it's nice to get a count, at least!

Yeah, not everything can be published, unfortunately.