ACME key seems to not persist

Do you want to request a feature or report a bug?
Bug
What did you do?
I've started with fresh config. I don't have an acme.json and don't use the migration path. I start out with an empty Etcd.
I'm running into a problem where the ACME key specified in storage = ... is not persisted in Etcd. It seems Traefik gets certs every time it starts. I have:
storage = "traefik/acme/account"

but this key does not ever exist:
$ etcdctl get --prefix / --keys-only | grep acme | grep -v '^$'
/traefik/acme/
/traefik/acme/acmelogging
/traefik/acme/caserver
/traefik/acme/delaydontcheckdns
/traefik/acme/dnsprovider
/traefik/acme/domains/0/main
/traefik/acme/email
/traefik/acme/entrypoint
/traefik/acme/httpchallenge/
/traefik/acme/httpchallenge/entrypoint
/traefik/acme/keytype
/traefik/acme/ondemand
/traefik/acme/onhostrule
/traefik/acme/overridecertificates
/traefik/acme/storage

I use storeconfig to update the config in Etcd. This seems to work fine. (Though the storage key isn't created at that point, nor did I expect it to be.)
There's nothing interesting in the logs except things like Datastore sync error: object lock value: expected 05254333-722a-4557-a320-0a12bda7645c, got 686712f1-fe57-4bb0-9438-0f06f3a54db9, retrying in 743.158272ms.
I start Traefik without a config file, and specify the bootstrapping params (--etcd.endpoint=... and so on) on the command line. I don't use a config file once I've bootstrapped Etcd. I only use storeconfig to update it.
When Traefik starts up, it requests certs from Let's Encrypt. However, the key for the ACME state is never saved. The next time Traefik starts up, it has no state, and so requests everything from LE again.
What did you expect to see?
I expected Traefik to persist the ACME account data.
What did you see instead?
No ACME account data persisted to Etcd.
Output of traefik version: (What version of Traefik are you using?)
$ /traefik version
Version:      v1.7.0-rc3
Codename:     maroilles
Go version:   go1.10.3
Built:        2018-08-01_01:37:51PM
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?
etcd (v3 only enabled, v2 disabled):
$ etcdctl version
etcdctl version: 3.3.9
API version: 3.3

Config file (used with traefik storeconfig)

Config file
logLevel = "DEBUG"

defaultEntryPoints = ["http", "https"]

[etcd]
endpoint = "etcd-0.etcd-headless:2379"
watch = true
prefix = "/traefik"
useAPIV3 = true

[entryPoints]
  [entryPoints.http]
  address = ":8080"

  [entryPoints.https]
  address = ":8443"

    [entryPoints.https.tls]

  [entryPoints.admin]
  address = ":8000"

  [entryPoints.api]
  address = ":8001"

[acme]
email = *OMITTED*
entryPoint = "https"
acmeLogging = true
storage = "traefik/acme/account"
overrideCertificates = false

  [acme.httpChallenge]
  entryPoint = "http"

  [[acme.domains]]
  *OMITTED*

[file]

[frontends]
  [frontends.default]
  backend = "glb"
  passHostHeader = true

[backends]
  [backends.glb]
    [backends.glb.servers.glb]
    url = *OMITTED*

[ping]
entryPoint = "api"

[api]
entryPoint = "api"

[traefikLog]
format = "json"

[accessLog]
format = "json"

Traefik command line
traefik \
--etcd.endpoint=etcd-0.etcd-headless:2379 \
--etcd.watch=true \
--etcd.prefix=/traefik \
--etcd.useAPIV3=true

If applicable, please paste the log output in DEBUG level (--logLevel=DEBUG switch)

Debug log
{"level":"info","msg":"Traefik version v1.7.0-rc3 built on 2018-08-01_01:37:51PM","time":"2018-08-09T19:49:00Z"}
{"level":"info","msg":"\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/basics/#collected-data\n","time":"2018-08-09T19:49:00Z"}
{"level":"debug","msg":"Global configuration loaded ","time":"2018-08-09T19:49:00Z"}
{"level":"info","msg":"Preparing server admin \u0026{Address::8000 TLS:\u003cnil\u003e Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:false ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc4207974c0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Preparing server https \u0026{Address::8443 TLS:0xc42074a120 Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:false ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc4207974e0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Starting server on :8000","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Datastore reload","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Preparing server http \u0026{Address::8080 TLS:\u003cnil\u003e Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:false ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc420797440} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Starting server on :8443","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Starting server on :8080","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Preparing server api \u0026{Address::8001 TLS:\u003cnil\u003e Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:false ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc420797460} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Starting provider configuration.ProviderAggregator {}","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Starting provider *etcd.Provider {\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"etcd-0.etcd-headless:2379\",\"Prefix\":\"/traefik\",\"TLS\":null,\"Username\":\"\",\"Password\":\"\",\"UseAPIV3\":true}","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Starting server on :8001","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Node 7c780eb5-c33e-4cec-9927-712ffb1bc373 running for election","time":"2018-08-09T19:49:01Z"}
{"level":"info","msg":"Node 7c780eb5-c33e-4cec-9927-712ffb1bc373 elected worker ♝","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/alias Key not found in store, setting default /traefik","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/alias Key not found in store, setting default /traefik","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/circuitbreaker/expression Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/method Key not found in store, setting default wrr","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/sticky Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/circuitbreaker/expression Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/method Key not found in store, setting default wrr","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/stickiness Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/sticky Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/maxconn/amount Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/stickiness Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/maxconn/amount Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/maxconn/extractorfunc Key not found in store, setting default request.host","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/healthcheck/path Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/maxconn/extractorfunc Key not found in store, setting default request.host","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/backends/glb/buffering/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/healthcheck/path Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/backends/glb/buffering/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/entrypoints Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/entrypoints Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/entrypoints/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/entrypoints/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/basicauth Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/basicauth Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Building ACME client...","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/basicauth/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/basicauth/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/auth/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/auth/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/whitelist/sourcerange Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/whitelist/sourcerange Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/whitelist/sourcerange/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/whitelist/sourcerange/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/permanent Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/permanent Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/entrypoint Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/entrypoint Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/regex Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/regex Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/errors/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/errors/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/ratelimit/extractorfunc Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/ratelimit/extractorfunc Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/customrequestheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/customrequestheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/customresponseheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/customresponseheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/sslproxyheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/sslproxyheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/allowedhosts Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/allowedhosts Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/allowedhosts/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/allowedhosts/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/hostsproxyheaders Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/hostsproxyheaders Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/hostsproxyheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslforcehost Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/hostsproxyheaders/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslredirect Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslforcehost Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslredirect Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/ssltemporaryredirect Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/ssltemporaryredirect Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslhost Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stsseconds Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslhost Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stsincludesubdomains Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stsseconds Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stspreload Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stsincludesubdomains Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stspreload Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/forcestsheader Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/framedeny Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/forcestsheader Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/customframeoptionsvalue Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/framedeny Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/contenttypenosniff Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/customframeoptionsvalue Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/browserxssfilter Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/contenttypenosniff Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/custombrowserxssvalue Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/browserxssfilter Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/contentsecuritypolicy Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/publickey Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/custombrowserxssvalue Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/contentsecuritypolicy Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/referrerpolicy Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/isdevelopment Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/publickey Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/routes/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/referrerpolicy Key not found in store, setting default ","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/tls/\": Key not found in store","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/isdevelopment Key not found in store, setting default false","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Configuration received from provider etcdv3: {\"backends\":{\"glb\":{\"servers\":{\"glb\":{\"url\":\"http://lb-enclosuresites.prod.gcp.t11e.com:80\",\"weight\":0}},\"loadBalancer\":{\"method\":\"wrr\"}}},\"frontends\":{\"default\":{\"entryPoints\":[\"http\",\"https\"],\"backend\":\"glb\",\"passHostHeader\":true,\"priority\":0,\"basicAuth\":null}}}","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot get key /traefik/alias Key not found in store, setting default /traefik","time":"2018-08-09T19:49:01Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/routes/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/tls/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Configuration received from provider etcdv3: {\"backends\":{\"glb\":{\"servers\":{\"glb\":{\"url\":\"http://lb-enclosuresites.prod.gcp.t11e.com:80\",\"weight\":0}},\"loadBalancer\":{\"method\":\"wrr\"}}},\"frontends\":{\"default\":{\"entryPoints\":[\"http\",\"https\"],\"backend\":\"glb\",\"passHostHeader\":true,\"priority\":0,\"basicAuth\":null}}}","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/circuitbreaker/expression Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/method Key not found in store, setting default wrr","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/sticky Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/loadbalancer/stickiness Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/maxconn/amount Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/maxconn/extractorfunc Key not found in store, setting default request.host","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/backends/glb/healthcheck/path Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/backends/glb/buffering/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/entrypoints Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/entrypoints/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/basicauth Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/basicauth/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/auth/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/whitelist/sourcerange Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/whitelist/sourcerange/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/permanent Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/entrypoint Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/redirect/regex Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/errors/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/ratelimit/extractorfunc Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/customrequestheaders/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/customresponseheaders/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/sslproxyheaders/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/allowedhosts Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/allowedhosts/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/hostsproxyheaders Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/headers/hostsproxyheaders/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Using HTTP Challenge provider.","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslforcehost Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslredirect Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/ssltemporaryredirect Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/sslhost Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stsseconds Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stsincludesubdomains Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/stspreload Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/forcestsheader Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/framedeny Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/customframeoptionsvalue Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/contenttypenosniff Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/browserxssfilter Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/custombrowserxssvalue Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/contentsecuritypolicy Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/publickey Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/referrerpolicy Key not found in store, setting default ","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot get key /traefik/frontends/default/headers/isdevelopment Key not found in store, setting default false","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/frontends/default/routes/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Cannot list keys under \"/traefik/tls/\": Key not found in store","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Configuration received from provider etcdv3: {\"backends\":{\"glb\":{\"servers\":{\"glb\":{\"url\":\"http://lb-enclosuresites.prod.gcp.t11e.com:80\",\"weight\":0}},\"loadBalancer\":{\"method\":\"wrr\"}}},\"frontends\":{\"default\":{\"entryPoints\":[\"http\",\"https\"],\"backend\":\"glb\",\"passHostHeader\":true,\"priority\":0,\"basicAuth\":null}}}","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Wiring frontend default to entryPoint http","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Creating backend glb","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Creating load-balancer wrr","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Creating server glb at http://lb-enclosuresites.prod.gcp.t11e.com:80 with weight 0","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Wiring frontend default to entryPoint https","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Creating backend glb","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Creating load-balancer wrr","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Creating server glb at http://lb-enclosuresites.prod.gcp.t11e.com:80 with weight 0","time":"2018-08-09T19:49:02Z"}
{"level":"info","msg":"Server configuration reloaded on :8001","time":"2018-08-09T19:49:02Z"}
{"level":"info","msg":"Server configuration reloaded on :8000","time":"2018-08-09T19:49:02Z"}
{"level":"info","msg":"Server configuration reloaded on :8443","time":"2018-08-09T19:49:02Z"}
{"level":"info","msg":"Server configuration reloaded on :8080","time":"2018-08-09T19:49:02Z"}
{"level":"debug","msg":"Wiring frontend default to entryPoint http","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Creating backend glb","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Creating load-balancer wrr","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Creating server glb at http://lb-enclosuresites.prod.gcp.t11e.com:80 with weight 0","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Wiring frontend default to entryPoint https","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Creating backend glb","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Creating load-balancer wrr","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Creating server glb at http://lb-enclosuresites.prod.gcp.t11e.com:80 with weight 0","time":"2018-08-09T19:49:04Z"}
{"level":"info","msg":"Server configuration reloaded on :8080","time":"2018-08-09T19:49:04Z"}
{"level":"info","msg":"Server configuration reloaded on :8001","time":"2018-08-09T19:49:04Z"}
{"level":"info","msg":"Server configuration reloaded on :8000","time":"2018-08-09T19:49:04Z"}
{"level":"info","msg":"Server configuration reloaded on :8443","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:04Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 483170, duration: 154.375599ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:05Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 79440, duration: 33.79309ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:06Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:07Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 302, Length: 0, duration: 127.548444ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:07Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:14Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 302, Length: 0, duration: 134.830799ms","time":"2018-08-09T19:49:14Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:18Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 142, duration: 5.765189ms","time":"2018-08-09T19:49:18Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:19Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 4298, duration: 8.991574ms","time":"2018-08-09T19:49:19Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 79440, duration: 33.755983ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:21Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 302, Length: 0, duration: 120.641739ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:22Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:26Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 79440, duration: 33.204123ms","time":"2018-08-09T19:49:26Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 4298, duration: 7.134438ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 142, duration: 4.987745ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:27Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:33Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 142, duration: 4.439033ms","time":"2018-08-09T19:49:33Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:33Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:33Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:33Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:33Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:34Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 4297, duration: 12.823533ms","time":"2018-08-09T19:49:34Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:36Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:36Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:36Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:36Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:37Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 302, Length: 0, duration: 196.236599ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:37Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:39Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:39Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 483170, duration: 197.896838ms","time":"2018-08-09T19:49:40Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 371194, duration: 1.45103139s","time":"2018-08-09T19:49:40Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:41Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 79440, duration: 36.360839ms","time":"2018-08-09T19:49:41Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:44Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:44Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:44Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:44Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"http: TLS handshake error from <IP_ADDRESS>:58906: EOF","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"Looking for an existing ACME challenge for stltoday.enclosuresites.com...","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"Looking for provided certificate to validate stltoday.enclosuresites.com...","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"No provided certificate found for domains stltoday.enclosuresites.com, get ACME certificate.","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"ACME got domain cert stltoday.enclosuresites.com","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 79440, duration: 121.630496ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"vulcand/oxy/forward/http: Round trip: http://lb-enclosuresites.prod.gcp.t11e.com:80, code: 200, Length: 483170, duration: 199.388079ms tls:version: 303, tls:resume:false, tls:csuite:c02f, tls:server:stltoday.enclosuresites.com","time":"2018-08-09T19:49:48Z"}
{"level":"debug","msg":"Upstream ResponseWriter of type *pipelining.writerWithoutCloseNotify does not implement http.CloseNotifier. Returning dummy channel.","time":"2018-08-09T19:49:48Z"}

Unfortunately, that's not the same problem, I think. That flag has been renamed to --overrideCertificates (e.g. --acme.overridecertificates=false; see here), but it only controls whether certs are overwritten or not. I have set this flag, but there's no improvement. My problem is that the ACME account is never stored. No certs are stored even during normal operation, outside of storeconfig.

@atombender if you have a chance it would be great to have the large sample blocks progressively revealed under a details element with a summary. It took me a while to get down here.

I found the cause. In Etcd v3, the keyspace is flat; it only has keys, not a hiearchy of "nodes". A consequence of this is that the / character has no intrinsic meaning. In the documentation, you suggest that storage be defined like this:
storage = "traefik/acme/account"

With v2, this creates /traefik/acme/account, but with v3, you get exactly traefik/acme/account. This gets doubly confusing because you'd think v3 etcdctl get --prefix / would be equivalent to v2 etcdctl ls /, but it's not. You have to do etcdctl get --prefix "". This reveals, in my case:
$ etcdctl get --prefix "" --keys-only | grep acme
/traefik/acme/
/traefik/acme/acmelogging
[...]
/traefik/acme/storage
traefik/acme/account/lock
traefik/acme/account/object

This doesn't quite explain why I am apparently seeing Traefik apparently requesting certs from ACME for ones that it has already got certs, but now that I solved this, I'll investigate that and maybe create an issue.
In the mean time, this is a documentation issue. For Etcd v3, the documentation has to be consistent about Etcd prefixes, otherwise this will be confusing.

That said, my testing is uncovering an edge case I don't understand. If I delete the two ACME account files (traefik/acme/account/lock, traefik/acme/account/object), then server startup fails with:
{"level":"fatal","msg":"Error preparing server: error creating TLS config: Key not found in store","time":"2018-08-13T18:57:42Z"}

This is after seeding Etcd with storeconfig. It would seem that the ACME bootstrapping isn't quite working. And since 1.7.0-rc3 no longer has --acme.storageFile, there's no way to bootstrap that key with an empty cert list. Indeed, it also apparently fails to start properly if the lock file is missing. To get things working again I had to extract the two files from an older Etcd snapshot.

I ran just now into this as well:

Edit 3: Actually, I spoke too quickly. 1.6.5 starts up, errors with private key was nil, then seems to continue, says Starting ACME renew job, then Building ACME client runs and everything is OK. But on restart, it fails with private key was nil and nothing happens after that, and ACME is not operational. This happens regardless of whether I use a starting slash or what.

Because of this I ran into quota limitation for creation clients, which lead to manual importing certs into the acme.json and use the import mechanism.
For anyone interested, this works with:
sudo cat /etc/letsencrypt/live/domain.example/fullchain.pem |base64 -w0
and insert it into a new section.

@alwinmarkcf I'm already using that flag, as I said above. I'm not actually using storageFile because I don't have a JSON file to import.

@atombender Hmm... account should then just be empty and there should be the two keys: object and lock.
BTW. I mentioned it in the export issue, but the object value is really just serialized go struct. I guess this happens by accident as the etcd isn't case sensitive and the json field is called just Object.

@alwinmarkcf Well, it's not working.
The fact that it's a serialized Go struct means I can't look into it, or edit its contents, or anything at all. That's a poor design choice.

Yeah, guess it's by accident and without tests nobody ever noticed. I'm trying to provide a PR now as I need this to investigate other issues as well and to have the possibility to backup those certs

@alwinmarkcf I tried again with 1.7.0-rc3 (I was using the head of master earlier), first writing an empty string to the two account keys (/traefik/acme/account/object, /traefik/acme/account/lock), and then starting Traefik. While it initially errors with Cannot unmarshall private key [] (see earlier comments for more detailed message), it now proceeds to register with LE, and if I restart Traefik, it is able to read the state back and continue normally. I don't know what went wrong earlier, but it's possible that master is broken. I'll do some more testing.

@atombender I don't have to write anything manually. All I have to do is run initially with the override parameter the storeconfig job. Tested it now several times and its working with 1.7-rc3 like described. I also updated now several times our cluster with 1.7-rc3 with that override parameter set to false and my certs stayed where they are.
As I monkeypatched my running traefik with #3776 I now also can see that its working in a transparent way.

@alwinmarkcf: Running storeconfig with --acme.overrideCertificates=true is interesting. So that creates empty object and lock files?
The reason I've never done that is that, first of all, it's not documented behaviour. Secondly, it's not expected behaviour. "Overriding" doesn't make sense given that the K/V store is empty and doesn't already contain certs; there are no certs to preserve.
Thirdly, I've been having deploy run storeconfig every time on startup, as part of my Kubernetes deploy. This is because I keep my config in traefik.toml and prefer to update it from the config, not by manipulating Etcd. However, after some testing I realized that I could actually (1) use --configfile=traefik.toml only, (2) not run storeconfig, and (3) let only the ACME stuff live in Etcd, nothing else.
In other words, I don't use storeconfig. In my opinion it shouldn't be required.
You should be able to replicate the behaviour I describe in the earlier comment this way:

Start out with an empty K/V store
Don't run storeconfig at all
Run Traefik with traefik --configfile=traefik.toml and make sure acme.overrideCertificates is false.

Let me know if you want me to create a separate issue for this specific workflow.

Yeah so this is how implemented it into https://github.com/helm/charts/pull/7173.
The reason why it is not documented is that 1.7 is just a release candidate. Maybe when you look into the RC docu you'll find this documented.
Are you sure that the configs are used from configfile while the certs are stored in etcd? Maybe some of the Maintainers should join the discussion whats the best way running Traefik on Kubernetes on their opinion. Currently I'm running storeconfig after each upgrade but only override certs for the initial installation or if I say explicitly I want to import the acme.json.

https://docs.traefik.io/v1.7/

Any updates on this? Planning to use etcd for Traefik HA, as consul had some problems. Any storage where traefik doesn't fails..? ;)

Etcd is not usable in production, you need vodoo sorcery with storeconfig sub command to make it work, I tried 1000 times, never get it works correctly. I have followed all documentation that I found
May be via calling directly the api to create object and lock ... but I give up, it's not ok to have that in prod

I'm thinking to try with boltdb, and they I don't know, probably will change my job, as in docker swarm there's no other way to survive without traefik.

Im doing fine with etcd running on prod,

Are you runnning Docker in Swarm Mode? My problem with etcd is, that I spread three etcd-Instances across my worker nodes and if one node is rebooted for some reason, the etcd-cluster is not able to resync. This is as far as I know I common drawback of running etcd in Docker Swarm as IPs are changed when services restart but etcd is not able to handle this.
If there is a solution, I am happy to continue using etcd. Otherwise, I would really prefer a future redis-option which can be run rock-solid in Docker Swarm Mode.
Greets
-act

@actraiser sorry I will not ever touch docker swarm again if I can avoid it, because in my experience you can not run anything rock solid on swarm (which we tried in my former company two years ago).
Further your problem is not much related to traefik but to etcd and swarm as far as I can see.

It's related in a sense that Traefik supports Docker Swarm but when you require HA, options to get your Traefik-configuration and ACME-certificates into a distributed Key/Value-Store is not optimal. The Traefik-supported K/V-Stores (etcd, consul) do not work well with Swarm plus they are overkill just to store some KB of configuration and certs.
That's why supporting Redis/Sentinel as alternative K/V-Store for storeconfig would be highly appreciated. But this topic is being discussed in another thread anyways.
Greets
-act

Hello,
Many thanks for your interest in the project and all your feedbacks about the use of Træfik with a KV store.
I'll try to give you some answers about the problems described above.
The storeconfig does not create the ACME account key if no ACME json file is providden.
If the object does not exist is the KV store, it is created by Træfik when the first leader starts (using a KV store activate automatically the HA mode).
The lock key is used to manage the concurrency and avoid many Træfik instances to access at the same time to the object key (wich contains the ACME Account information and all the certificates in a gzipped format). A Træfik instance can not access to the ACME account key until the lock does not have the right value in the KV store.
This behavior can take time if many nodes want to access to the object in the same time (when strating many instance of træfik for example).
@atombender , you said you do not want to use your KV as a configuration API, but if you want to store your Træfik certificates (which are part of the ACME account key) in the KV store, Træfik needs to get its static configuration from this KV store.
@actraiser Træfik does not support redis for the moment, the suporrted KV store are supported here (even if Consul and ETCD are the most popular).
@alwinmarkcf

So there is a new config parameter called traefikConfiguration.GlobalConfiguration.ACME.OverrideCertificates which has to be provided in addition.

This parameter is optional, by default, it will return false.

Maybe some of the Maintainers should join the discussion whats the best way running Traefik on Kubernetes on their opinion.

ping @containous/kubernetes
If you want we can dicuss about this feature on our Slack workspace for more community #support.

The storeconfig does not create the ACME account key if no ACME json file is provided.
If the object does not exist is the KV store, it is created by Træfik when the first leader starts (using a KV store activate automatically the HA mode).

In my case, this object is not created at all. I have to create both an empty object and an empty lock file in order for Traefik to successfully start up.
As a sidenote, as @alwinmarkcf points out, the fact that the object is a compressed binary, opaque file — which is practically impossible to read/write by an admin — is problematic for the purposes of diagnostics and recovery. The fact that it's binary and compressed seems to be a workaround for a size limitation in Consul, but no such limitation exists in Etcd.

@atombender , you said you do not want to use your KV as a configuration API, but if you want to store your Træfik certificates (which are part of the ACME account key) in the KV store, Træfik needs to get its static configuration from this KV store.

This seems to be false. I am able to run Traefik with --configfile=... and keep the ACME key in the K/V store.

As a sidenote, as @alwinmarkcf points out, the fact that the object is a compressed binary, opaque file — which is practically impossible to read/write by an admin — is problematic for the purposes of diagnostics and recovery. The fact that it's binary and compressed seems to be a workaround for a size limitation in Consul, but no such limitation exists in Etcd.

This is actually a very interesting catch - it would be awesome to add an additional option to disable compression on etcd like using
etcd.useCompression=false
Greets
Rocco

@actraiser maybe an interesting. The Gzip compression was not implemented in traefik but in staert which is doing this automatically.
IMO the best would be storing certs one by one in a own key as base64 string.

Closed by #3844.

I'm currently on vacation.When I'm back I'll contribute the Issue list with the tasks needed to be done to make traefik Kubernetes and Helm friendly


An open string annotation API

Document Link
https://flutter.dev/go/string-annotation
What problem are you solving?

The ability to add styles to an already styled string is needed for system / 3rd party text service integrations such as https://github.com/flutter/flutter/issues/150068
InlineSpan only supports capabilities defined by its static interface. Custom annotations defined outside the painting library are currently not very well supported.

That's great. Now, if we want to implement features like strikethrough and highlighting, we have to concatenate and transform spans every time, which is super complicated and hard to maintain.


I will add a Jumping square animation project created with HTML and CSS to the web dev projectKart repository.

Project title
(Jumping square animation.)
Project Description
(it is a very simple and useful project for beginners to understand how the keyframes functions work in CSS I was not good at animations but after creating some projects with animation I'm good at it now this project is created with HTML and CSS. also adding the demo below once go through it.)
Stack:
And in order to tick the check box just put x inside them for example - [x] like this. Please delete options that are not relevant.

[x] Html
[x] CSS

https://user-images.githubusercontent.com/95171638/166110377-374263de-5859-4f36-9a14-1ee9a5424362.mp4
@gurjeetsinghvirdee kindly review this issue.

You can't add same project to 3 different repos

Okayyy.
On Sat, Apr 30, 2022, 8:21 PM Gurjeet Singh Virdee @.***>
wrote:

Closed #1086
https://github.com/khushi-purwar/WebDev-ProjectKart/issues/1086.
—
Reply to this email directly, view it on GitHub
https://github.com/khushi-purwar/WebDev-ProjectKart/issues/1086#event-6528151817,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWWDINVHP3226HSZMMSWWBDVHVCGFANCNFSM5UYOM4IA
.
You are receiving this because you were mentioned.Message ID:
@.***
.com>


Authentication with lapin v0.20.0

I was able to connect with a specific username/password (rather than guest/guest) using the Credentials parameter of Client::connect() by say, passing in Credentials::new("bunnyuser", "carrotpasswd"). in v0.19.0.
But in v0.20.0. the Credentials parameter no longer seems to exist.  I can't find how to set the username in the examples or tests or documentation. I am guessing that it would be something like setting something within the ConnectionProperties parameter, but can't quite zero in on what to do.
Would you be able to advise on this? Thanks.
I did try the amqp://username:password@localhost:xxxx trick but the rabbitmq log showed a failure with a wrong default login attempt (using the username 'guest' rather than the user I specified).

username and password need to be url escaped when they contain "special" chars.
It works fine here passing creds in the uri

Can you try "amqp://username:password@localhost:xxxx".parse() and print the output to see if it's the expected one?

Thanks for the quick response.
The username contains only lowercase letters and the password has letters
and numbers only. I ran into some unrelated system trouble today, so
couldn't finish trying your parse() suggestion which I was attempting
nevertheless. I will check tomorrow.
On 28 Jun 2019 1:07 a.m., "Marc-Antoine Perennou"<EMAIL_ADDRESS>wrote:

Can you try "amqp://username:password@localhost:xxxx".parse() and print
the output to see if it's the expected one?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/sozu-proxy/lapin/issues/184?email_source=notifications&email_token=AMOKMAPCYRK7WZUGXAVBA7TP4VI4DA5CNFSM4H4BB2XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYYWI3Y#issuecomment-506553455,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMOKMAPHEAB5UDRQKVJVM7DP4VI4DANCNFSM4H4BB2XA
.

It's working for me today. It could have been my mistake; I was modifying the source file in  lapin/futures/tests/ and executing 'cargo test'. I had different versions of lapin cloned in different folders, and I could have been in the wrong folder.
Maybe it's worth mentioning in the document that putting the username and password in the ampq URI is 'the way' (if indeed it is) to login to the RabbitMQ server?
Apart from that, I think this issue can be closed.


Usage question

    steps:
      - id: "auth"
        name: Login to GCR
        uses: docker/login-action@v1
        ....

      - uses: "docker://us-docker.pkg.dev/gcr-cleaner/gcr-cleaner/gcr-cleaner-cli"
        with:
          args: >-
            -repo=gcr.io/place/my-image-name

ENIED: Failed to read tags for host 'gcr.io', repository
I've tried -repo=gcr.io/place/my-image-name:latest and that didn't work
Why is this happening? Any help is appreciated.

I needed the extra auth step

@hassanshaikley Can you provide clarity on what you mean by "the extra auth step"?
I have configured GitHub Actions as follows and I also encounter the same error message:

failed to list tags for repo gcr.io/webapp-alpha/curl: GET https://gcr.io/v2/project-alpha/curl/tags/list?n=1000: DENIED: Failed to read tags for host 'gcr.io', repository '/v2/project-alpha/curl/tags/list?n=1000'

Here's my action code:
name: 'gcr-cleaner'

on:
  schedule:
    - cron: '0 0 */1 * *'
  push:
  workflow_dispatch:

jobs:
  gcr-cleaner:
    runs-on: ubuntu-latest
    steps:
      - name: Login to gcr.io
        uses: docker/login-action@v2
        with:
          registry: gcr.io
          username: _json_key
          password: ${{ secrets.GCR_CLEANER_SERVICE_ACCOUNT }}

      - name: Purge untagged images
        uses: 'docker://us-docker.pkg.dev/gcr-cleaner/gcr-cleaner/gcr-cleaner-cli'
        env:
          GCRCLEANER_LOG: debug
        with:
          args: >-
            -repo=gcr.io/project-alpha/curl
            -grace=168h
            -keep=10
            -dry-run


Feature/add code coverage to ci

Resolves #24

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@b52721b). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #63   +/-   ##
=======================================
  Coverage        ?   94.74%
=======================================
  Files           ?       31
  Lines           ?      571
  Branches        ?      163
=======================================
  Hits            ?      541
  Misses          ?       29
  Partials        ?        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b52721b...898485c. Read the comment docs.


add closing quotation marks

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

@jh-isw  Thanks!


feature: replace del with weakref.finalize()

Feature addition: replacing del with weakref.finalize() as called method during object garbage collection.
Issue: #3774

renamed __del__ as __finalize, logic stays the same.
added weakref.finalize() in __init__ in instrument.py calling upon '__finalize()' during gc.

@maximilianluc Thanks for this. It looks great but I will need a little bit of time to test this manually to see how it works in practice

@maximilianluc Sorry for the late response. I have spent a bit of time investigating this and unfortunately its a bit more complicated as the finalize method cannot contain a reference to self since that would increase the ref count to the instrument preventing the finalizer to be called. https://docs.python.org/3/library/weakref.html#weakref.finalize
In https://github.com/QCoDeS/Qcodes/pull/5341 I have implemented a partial solution for visa instruments that do not have this issue


Penalty shootout was not correct

See the UEFA Europa League game between ASTANA APOEL:
https://www.fifa.com/live-scores/uefaeuropaleague/matches/match=fc-astana-apoel-nicosia-300453747/index.html
https://data.fifa.com/matches/en/live/info/300453747

That is probably the minute thats wrong


Outdated documentation

It appears the documentation for adding new languages is outdated. With the exception of the first step, it seems everything after doesn't make sense.(I'm referring to the doc/addnewlocale.txt. I was trying to add a language and I'm not sure if adding the locale file in the data directory is sufficient.

Dear Pintert3,
I've updated the documentation here:
https://github.com/Numbertext/libnumbertext/blob/master/doc/addnewlocale.txt
Thanks for your bug report,
László


Plugin fibre section

Add function to export geometry/section to fibre sections.

Quick comment before reviewing in depth - rather than creating a completely new sub-package plugin, I would be happy for this to be a new module under the post sub-package? I think maybe it makes more sense for it to go there as it appears to be post-processing of the analysis results?


bundle check -v => /usr/local/Homebrew/Library/Homebrew/shims/scm/git --version

Hi
When I ...
> cd /tmp
> brew bundle dump
> brew bundle check -v
/usr/local/Homebrew/Library/Homebrew/shims/scm/git --version
The Brewfile's dependencies are satisfied.

Is the output of   /usr/local/Homebrew/Library/Homebrew/shims/scm/git --version  expected behaviour ?
cheers

What do brew doctor and brew config and env | grep HOMEBREW output?

What do brew doctor and brew config and env | grep HOMEBREW output?

❯ brew config
HOMEBREW_VERSION: 2.7.1-111-ge5eb6a2
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: e5eb6a264669afdf6e3def98c967509874501d95
Last commit: 20 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 7c6f18ebdedc74467d5394e733a9f0def7bfed07
Core tap last commit: 10 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: ["--appdir=/Applications"]
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 4
HOMEBREW_NO_EMOJI: set
Homebrew Ruby: 2.6.3 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/bin/ruby
CPU: quad-core 64-bit haswell
Clang: 11.0 build 1100
Git: 2.30.0 => /usr/local/bin/git
Curl: 7.64.1 => /usr/bin/curl
Java: 13.0.2, 1.8.0_265, 1.8.0_101, 1.7.0_45
macOS: 10.15.7-x86_64
CLT: <IP_ADDRESS>
Xcode: N/A

❯ env | grep HOMEBREW
HOMEBREW_CASK_OPTS=--appdir=/Applications
HOMEBREW_NO_EMOJI=1

❯ brew config
HOMEBREW_VERSION: 2.7.1-111-ge5eb6a2
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: e5eb6a264669afdf6e3def98c967509874501d95
Last commit: 20 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 7c6f18ebdedc74467d5394e733a9f0def7bfed07
Core tap last commit: 10 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: ["--appdir=/Applications"]
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 4
HOMEBREW_NO_EMOJI: set
Homebrew Ruby: 2.6.3 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/bin/ruby
CPU: quad-core 64-bit haswell
Clang: 11.0 build 1100
Git: 2.30.0 => /usr/local/bin/git
Curl: 7.64.1 => /usr/bin/curl
Java: 13.0.2, 1.8.0_265, 1.8.0_101, 1.7.0_45
macOS: 10.15.7-x86_64
CLT: <IP_ADDRESS>
Xcode: N/A

❯ env | grep HOMEBREW
HOMEBREW_CASK_OPTS=--appdir=/Applications
HOMEBREW_NO_EMOJI=1

Looks like it has to do with the call to Tap.map(&:to_hash) in tap_dumper.rb.

Looks like it has to do with the call to Tap.map(&:to_hash) in tap_dumper.rb.


Legion Prof buggy with spy data

Summary: changes in legion_spy.py broke legion_proj for cases with spy data
What version: latest stable legion-19.12.0
What happens: "legion_prof.py ./prof_0.gz ./spy_0.log" crashes in multiple places. This seems
to be caused by the following commit that changed a few critical things that prof relied on:
commit 91b65bd67c74a0c8bfa034d3da62bafdf774e0c2
Author: Mike<EMAIL_ADDRESS>Date:   Thu Sep 26 16:26:12 2019 -0700

    tools: updates to legion spy to dump bad equivalence set graphs

I fixed (cludged) it up enough to run locally but I'm pretty sure it's to ugly for a PR. In short API
change in two places (diff output trimmed for brevity):
--- a/tools/legion_prof.py
+++ b/tools/legion_prof.py
@@ -3283,7 +3283,7 @@ class State(object):
-        self.spy_state = legion_spy.State(False, True, True, True)
+        self.spy_state = legion_spy.State("/tmp", False, True, True, True, False)
-        self.spy_state.perform_physical_analysis(False, False, False)
+        self.spy_state.perform_physical_analysis(False, False)

And a bit later it failed to notice the root context so I added explicit check:
@@ -3398,6 +3398,9 @@ class State(object):
         for op in itervalues(self.spy_state.ops):
             if op.context is not None:
+                if str(op.context) == "Root context":
+                    continue

After that the example I ran produced a seemingly ok output directory that rendered in the browser.

This has been fixed in the master branch and will be merged into stable in the next release
On Feb 24, 2020, at 9:24 AM, Peter Kjellstrom <notifications<EMAIL_ADDRESS>wrote:
Summary: changes in legion_spy.py broke legion_proj for cases with spy data
What version: latest stable legion-19.12.0
What happens: "legion_prof.py ./prof_0.gz ./spy_0.log" crashes in multiple places. This seems
to be caused by the following commit that changed a few critical things that prof relied on:
commit 91b65bd67c74a0c8bfa034d3da62bafdf774e0c2
Author: Mike <mebauer<EMAIL_ADDRESS>Date:   Thu Sep 26 16:26:12 2019 -0700
tools: updates to legion spy to dump bad equivalence set graphs

I fixed (cludged) it up enough to run locally but I'm pretty sure it's to ugly for a PR. In short API
change in two places (diff output trimmed for brevity):
--- a/tools/legion_prof.py
+++ b/tools/legion_prof.py
@@ -3283,7 +3283,7 @@ class State(object):

   self.spy_state = legion_spy.State(False, True, True, True)

   self.spy_state = legion_spy.State("/tmp", False, True, True, True, False)

   self.spy_state.perform_physical_analysis(False, False, False)

   self.spy_state.perform_physical_analysis(False, False)

And a bit later it failed to notice the root context so I added explicit check:
@@ -3398,6 +3398,9 @@ class State(object):
for op in itervalues(self.spy_state.ops):
if op.context is not None:

           if str(op.context) == "Root context":

               continue

After that the example I ran produced a seemingly ok output directory that rendered in the browser.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/StanfordLegion/legion/issues/753?email_source=notifications&email_token=AJD6EIWJ2NNT22VOZCVXMP3REP7DPA5CNFSM4K2MUVM2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IPZSLWQ, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJD6EIRG7Y5YDB33EP6FCCTREP7DPANCNFSM4K2MUVMQ.

Dumb of me not to check that first, sorry for the noise, please close.

Closing this since it seems to be fixed in master.