Authorization token including hostname

Description
I'm trying to use the docker registry, portus, and notary together. The registry and notary both are using portus as an authentication provider.
Steps to reproduce

Setup portus
Setup docker registry and notary both with portus as auth token provider
Try to push images

What happens is that the following token is requested:
portus_1            |   Parameters: {"account"=>"mettke", "scope"=>"repository:registry.itmettke.de/test/alpine:push,pull", "service"=>"registry.itmettke.de"}

Expected behavior:

Portus should be able to handle the hostname inside the scope using test and alpine as reference for the repository

Actual behavior:

Portus is parsing the string using this very simple command:
@namespace_name = @resource_name.split("/").first

This leads to the hostname being used as namespace name failing the authorization process.

Initial solution:

This is my first try to solve this problem:
if @resource_name.count("/") == 2
  @tmp, @namespace_name, @tmp2 = @resource_name.split("/", 3)
else
  @namespace_name = @resource_name.split("/").first
end

It works, however it migth not be the best way to solve this

Further considerations:

The hostname might contain a port reference like:
portus_1            |   Parameters: {"account"=>"mettke", "scope"=>"repository:registry.itmettke.de:5000/test/alpine:push,pull", "service"=>"registry.itmettke.de"}

This leads to another problem in this part:
str = scope_string.split(":", 3)
raise ScopeNotHandled, "Wrong format for scope string" if str.length != 3

Instead the code should be able to handle four parts as well

Further reading:

https://github.com/theupdateframework/notary/issues/1023

hi @mettke Ihaveseen other people in the community, complaining about that stale bot, so regardless of that bee :

thank you so much for sharing that issue, it is very valuable because very accurate,
and the subject of the trust management is critical for managing an OCI container image registry

I will address the issue in the next weeks, have a question I want to hear your answer to (will help me a lot to listen to your answer, before doing anything and building my opinion, which I will share and backup with running examples)  :
What If you used a Keycloak Open ID Provider, independently of the running portus, to provide authentication to your notary service : Do you _have_to make portus Auth Service, the notary auth service?
Ok, want you want, is :

when you docker login, you're also logged in the notary
because then, you directly can use docker trust commands to load you private key, add the associated public SSL/TLS Certifcate as identity of a new signer, and sign your images, before you docker push them.

Okay, So in the end, what you want is to be logged  in notary, when you docker login.
I have seen a similar pattern, and :

What About using Keycloak Gatekeeper to drop the notary token into the response to your docker login and...
well ther's some work here, but : do you see more or less, the shape of my idea ?

I 'm definitely on that issue.

hi back @mettke simple question, is this your use case:
export OCI_IMAGE_SIGNER_NOTARY_NAME=jeanbl
export OCI_IMAGE_SIGNER_NOTARY_PASSPHRASE="that1smys3cretpa22phraseyeah"

export OCI_IMAGE_REPOSITORY_NAME=pokus
export NOM_LOCAL_OCI_IMAGE=$OCI_IMAGE_REPOSITORY_NAME/myapp:12.0.2-alpine
export NOM_LOCAL_OCI_IMAGE=$OCI_IMAGE_REPOSITORY_NAME/myapp:12-alpine
export PRIVATE_REGISTRY_HOST=oci-registry.pegasusio.io
export NOM_DE_PUBLICATION_OCI_IMAGE=$PRIVATE_REGISTRY_HOST/$NOM_LOCAL_OCI_IMAGE

docker build $NOM_LOCAL_IMAGE
docker tag $NOM_LOCAL_IMAGE $NOM_COMPLET_AVEC_REGISTRY

# ---
# generate private/public key pair
# You can use the below command (will load the
# key to the local trust store at same time)
# to run it silently,setthe passphrasewith
# export DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE="kwhatever"
# ---
# docker trust key generate $OCI_IMAGE_REPOSITORY_NAME
# ---
# Or you can use :
# https://ruimarinho.gitbooks.io/yubikey-handbook/content/docker-content-trust/delegation-roles/generating-a-delegation-key.html
#
# pre-existing private key :
export OCI_SIGNER_PRIVATE_KEY=./$OCI_IMAGE_SIGNER_NOTARY_NAME-$OCI_IMAGE_REPOSITORY_NAME-oci-signinkey.pem
# pre-existing public key :
export OCI_SIGNER_PUBLIC_KEY=./$OCI_IMAGE_SIGNER_NOTARY_NAME-$OCI_IMAGE_REPOSITORY_NAME-oci-signincert.pem

#
# --- #
# Add the  private key, called the _delegation_ private key, to
# the local Docker trust repository.
# (By default this local trust repository is in  '~/.docker/trust/').
#

# ---
# To run it silently (non-interactively), we can use the
#  DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE
# environment variable to set the passphrase
# ---

export DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE=$OCI_IMAGE_SIGNER_NOTARY_PASSPHRASE
# --
# ok so now let's do this silently
docker trust key load $OCI_SIGNER_PRIVATE_KEY --name $OCI_IMAGE_SIGNER_NOTARY_NAME

# --- #
# register a new signer to the notary server
# the signer is given a name, identified by its
# public key (pubic Certificate), and will be a
# signer only for the specified given
# repo, $NOM_DE_PUBLICATION_OCI_IMAGE, not
# any other.
# --- #
# DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE is
# set, so the signer add command will not
# interactively  askfor the passphrase, but pick
# up the env variable value in the stead.
# ---

docker trust signer add --key cert.pem $OCI_IMAGE_SIGNER_NOTARY_NAME $NOM_DE_PUBLICATION_OCI_IMAGE

# --- #
# DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE is
# set, so the signer add command will not
# interactively  askfor the passphrase, but pick
# up the env variable value in the stead.
# ---

docker trust sign $NOM_DE_PUBLICATION_OCI_IMAGE

# --- #
# setting DOCKER_CONTENT_TRUST to 1 will cause
# the docker client, every time you [docker push] to
# go and ask the notary service :
# Hey, Do you trust me ? Of course if answer is no...
# Can that option be forced from the regsitry ?
# Can we Configure registry, so that no image is
# allowed in, unless it is signed by a notary-trusted ?
# ---
# https://dev.to/idevkamboj/care-about-security-enable-docker-content-trust-4djh
#

# Now, additionally to
# the [DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE]
# env.var., we set DOCKER_CONTENT_TRUST
#
# setting DOCKER_CONTENT_TRUST  to 1 will cause the
# docker client to :
# => check signature of image, every time you docker pull
# => check signature of image, every time you docker run
# => sign the image with $OCI_SIGNER_PRIVATE_KEY, every time you docker push
#
# So, content trust is like CRITICAL, to setting up any app's
# production grade dockerized environment :
#
# Of course, your boss wants you to run **only** trusted image in
# production
#

export DOCKER_CONTENT_TRUST=1

docker push $NOM_DE_PUBLICATION_OCI_IMAGE

?
If any difference with what you want to do,

note :

this : https://github.com/IBM/portieris
is what you can use on production envrionments, to ensure only trusted images roll out.
Called an "Adminission Controller", and you configure it to use the notary server of your choice.

Hey @Jean-Baptiste-Lasselle,
its been a while by now but I will try to remember as much as possible.
What you said so far looks like what I was trying to do except that you are missing the DOCKER_CONTENT_TRUST_SERVER variable which is set to domain of the notary server. Otherwise you would by default use the official docker notary service.
When I remember correctly the problematic token from above is not requested from docker login but from notary. Notary itself can be configured for multiple domains and thus appends the domain to the request.

hi @mettke Thank you so much for this quick feedback !
Excellent :

so I did understand well, and thank you so much about DOCKER_CONTENT_TRUST_SERVER, which I was looking for.  I have just modified my script above, to add this env. var. for docker notary client conf.
I have seriously started working on notary (it's like awesome), a few days ago, along my work on portus. And that especially interested in the capability of notary to act for different, independent registry services. Actually I am especially interested in generalizing notary for trust management on any binary, could be java jar, go executables, or linux packages. Even non executable binaries, like pure internationalization packages. Any dependency, of a given infrastructure, to sum it up.

So yeah, I am l aware this is why notary uses what I call fully qualified namespaces, as defined in my updated script above.
But I am now investigating the whole request response flow involved in the scenario I precisely describe with that script above :

So can I ask you to confirm, that the portus logs below is notary requesting an auth token, to be able to afterwards query the registry ?

portus_1            |   Parameters: {"account"=>"mettke", "scope"=>"repository:registry.itmettke.de/test/alpine:push,pull", "service"=>"registry.itmettke.de"}

Is this line of logs above stating : notary asks portus for an auth token to authenticate to registry  ?

Also, I can't help asking  , did you configure your notary with  :

something like the ROOT_CERT_BUNDLE for notary , with value the path to the portus public certificate e.g. portus.crt ? That, for notary to accept portus token as valid auth token . Here we talk about the docker client's  auth to notary
something like the secret, or Bearer Token :  and here we are talking about notary, authenticating to portus, to then get a registry auth token.  Like when a human user has to log  into portus webui, using his portus credentials, whatever they are, and then create an application token, which can be used to authenticate to the registry, using docker login, or curl if you are dan walsh.

Thank you so much for your help there, because for the time being, it is you, helping me, and I commit to give back :

I will solve this issue, whatever it takes.
Goal :

a proof of concept prototype working out of the box, that fulfills the trust management policy scenario, defined in my script,
and with seamless integration to automated updates solutions, like https://github.com/IBM/portieris or grafeas/kritis. (So I'll have to get one of those up and running, for the proof of concept)

Also, I wanna ask :

Since May 2019, could you enforce a notary service for your docker image registries ?
If yes, did you do that with portus ? I assume not, or you'd have told me if you found a solution to that issue. Nevertheless asking, I know how busy we can get at office.
If yes, did you have to say bye bye to portus, and  use another solution instead, like harbor? (Did it take you to switch to harbor, to get proper trust management ?)

Testing the login process portus logs against DOCKER_CONTENT_TRUST_SERVER
I create the first super admin user with username jbladmin, using the portus webui
Without the DOCKER_CONTENT_TRUST_SERVER env var

Here are the Portus Logs I get, when I docker login, using a wrong password and the first super admin username jbladmin (test login failure) :

background_1                        |   RegistryEvent Load (0.2ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.2ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
registry_1                          | time="2020-02-25T22:05:35Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.7.6 http.request.host=oci-registry.pegasusio.io http.request.id=f43ca8b2-5ca1-491e-acb5-2c4adc7d66f4 http.request.method=GET http.request.remoteaddr=<IP_ADDRESS> http.request.uri="/v2/" http.request.useragent="docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))" instance.id=b87dc65a-4fdf-43ae-8975-c276ec1eaaf0 version=v2.6.2-14-ga66a4c3
registry_1                          | <IP_ADDRESS> - - [25/Feb/2020:22:05:35 +0000] "GET /v2/ HTTP/1.0" 401 87 "" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))"
nginx_1                             | <IP_ADDRESS> - - [25/Feb/2020:22:05:35 +0000] "GET /v2/ HTTP/1.1" 401 87 "-" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \x5C(linux\x5C))"
portuscontainer                     | Started GET "/v2/token?account=jbladmin&client_id=docker&offline_token=true&service=oci-registry.pegasusio.io" for <IP_ADDRESS> at 2020-02-25 22:05:35 +0000
portuscontainer                     | Processing by Api::V2::TokensController#show as JSON
portuscontainer                     |   Parameters: {"account"=>"jbladmin", "client_id"=>"docker", "offline_token"=>"true", "service"=>"oci-registry.pegasusio.io"}
portuscontainer                     |   User Load (0.4ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin' LIMIT 1
portuscontainer                     |   ApplicationToken Load (0.5ms)  SELECT `application_tokens`.* FROM `application_tokens` WHERE `application_tokens`.`user_id` = 2
portuscontainer                     |   User Load (0.5ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin'  ORDER BY `users`.`id` ASC LIMIT 1
portuscontainer                     |    (2.0ms)  BEGIN
portuscontainer                     |   SQL (0.5ms)  UPDATE `users` SET `failed_attempts` = 1, `updated_at` = '2020-02-25 22:05:35' WHERE `users`.`id` = 2
portuscontainer                     |    (0.3ms)  COMMIT
portuscontainer                     | Completed 401 Unauthorized in 106ms (ActiveRecord: 5.5ms)
background_1                        |   RegistryEvent Load (0.3ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.2ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   Tag Exists (0.1ms)  SELECT  1 AS one FROM `tags` WHERE `tags`.`scanned` = 0 LIMIT 1
background_1                        |   RegistryEvent Load (0.2ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.2ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.2ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.2ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000

Here are the Portus Logs I get, when I docker login, using the right password and the first super admin username jbladmin (test login success) :

registry_1                          | <IP_ADDRESS> - - [25/Feb/2020:22:16:07 +0000] "GET /v2/ HTTP/1.0" 401 87 "" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))"
nginx_1                             | <IP_ADDRESS> - - [25/Feb/2020:22:16:07 +0000] "GET /v2/ HTTP/1.1" 401 87 "-" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \x5C(linux\x5C))"
portuscontainer                     | Started GET "/v2/token?account=jbladmin&client_id=docker&offline_token=true&service=oci-registry.pegasusio.io" for <IP_ADDRESS> at 2020-02-25 22:16:07 +0000
portuscontainer                     | Processing by Api::V2::TokensController#show as JSON
portuscontainer                     |   Parameters: {"account"=>"jbladmin", "client_id"=>"docker", "offline_token"=>"true", "service"=>"oci-registry.pegasusio.io"}
portuscontainer                     |   User Load (0.8ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin' LIMIT 1
portuscontainer                     |   ApplicationToken Load (0.4ms)  SELECT `application_tokens`.* FROM `application_tokens` WHERE `application_tokens`.`user_id` = 2
portuscontainer                     |   User Load (0.7ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin'  ORDER BY `users`.`id` ASC LIMIT 1
portuscontainer                     |    (0.5ms)  BEGIN
portuscontainer                     |   SQL (0.5ms)  UPDATE `users` SET `failed_attempts` = 0, `updated_at` = '2020-02-25 22:16:07' WHERE `users`.`id` = 2
portuscontainer                     |    (0.3ms)  COMMIT
portuscontainer                     |    (0.2ms)  BEGIN
portuscontainer                     |   User Exists (0.5ms)  SELECT  1 AS one FROM `users` WHERE (`users`.`username` = BINARY 'jbladmin' AND `users`.`id` != 2) LIMIT 1
portuscontainer                     |   SQL (0.4ms)  UPDATE `users` SET `current_sign_in_at` = '2020-02-25 22:16:07', `current_sign_in_ip` = '<IP_ADDRESS>', `sign_in_count` = 2, `updated_at` = '2020-02-25 22:16:07' WHERE `users`.`id` = 2
portuscontainer                     |    (0.3ms)  COMMIT
portuscontainer                     |   Registry Load (0.3ms)  SELECT  `registries`.* FROM `registries` WHERE `registries`.`hostname` = 'oci-registry.pegasusio.io' LIMIT 1
portuscontainer                     | No hostname matching `oci-registry.pegasusio.io', testing external_hostname
portuscontainer                     |   Registry Load (0.3ms)  SELECT  `registries`.* FROM `registries` WHERE `registries`.`external_hostname` = 'oci-registry.pegasusio.io' LIMIT 1
portuscontainer                     | [jwt_token] [claim] {:iss=>"portus.pegasusio.io", :sub=>"jbladmin", :aud=>"oci-registry.pegasusio.io", :iat=>1582668967, :nbf=>1582668962, :exp=>1582669867, :jti=>"DtiHcBHNzXdynXZkC47cL11ReGddfrnDRFYBw9D8mR"}
portuscontainer                     | Completed 200 OK in 113ms (Views: 0.2ms | ActiveRecord: 7.5ms)
registry_1                          | time="2020-02-25T22:16:07Z" level=info msg="response completed" go.version=go1.7.6 http.request.host=oci-registry.pegasusio.io http.request.id=f519d60f-d072-4fe2-b7c9-2613878f0da3 http.request.method=GET http.request.remoteaddr=<IP_ADDRESS> http.request.uri="/v2/" http.request.useragent="docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.313059ms http.response.status=200 http.response.written=2 instance.id=b87dc65a-4fdf-43ae-8975-c276ec1eaaf0 version=v2.6.2-14-ga66a4c3
registry_1                          | <IP_ADDRESS> - - [25/Feb/2020:22:16:07 +0000] "GET /v2/ HTTP/1.0" 200 2 "" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))"
nginx_1                             | <IP_ADDRESS> - - [25/Feb/2020:22:16:07 +0000] "GET /v2/ HTTP/1.1" 200 2 "-" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \x5C(linux\x5C))"
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   Tag Exists (0.2ms)  SELECT  1 AS one FROM `tags` WHERE `tags`.`scanned` = 0 LIMIT 1
background_1                        |   RegistryEvent Load (0.3ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000

Now with, the DOCKER_CONTENT_TRUST_SERVER env var
first, I docker logout of course, to wipe out base64-encoded credentials from ~/.docker/config.json

Here are the Portus Logs I get, when I docker login, using a wrong password and the first super admin username jbladmin (test login failure) :

background_1                        |   Tag Exists (0.2ms)  SELECT  1 AS one FROM `tags` WHERE `tags`.`scanned` = 0 LIMIT 1
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.3ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
nginx_1                             | <IP_ADDRESS> - - [25/Feb/2020:22:32:36 +0000] "GET /v2/ HTTP/1.1" 401 87 "-" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \x5C(linux\x5C))"
registry_1                          | time="2020-02-25T22:32:36Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.7.6 http.request.host=oci-registry.pegasusio.io http.request.id=847821c9-69ca-4892-9369-330d0986e68e http.request.method=GET http.request.remoteaddr=<IP_ADDRESS> http.request.uri="/v2/" http.request.useragent="docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))" instance.id=b87dc65a-4fdf-43ae-8975-c276ec1eaaf0 version=v2.6.2-14-ga66a4c3
registry_1                          | <IP_ADDRESS> - - [25/Feb/2020:22:32:36 +0000] "GET /v2/ HTTP/1.0" 401 87 "" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))"
portuscontainer                     | Started GET "/v2/token?account=jbladmin&client_id=docker&offline_token=true&service=oci-registry.pegasusio.io" for <IP_ADDRESS> at 2020-02-25 22:32:36 +0000
portuscontainer                     | Processing by Api::V2::TokensController#show as JSON
portuscontainer                     |   Parameters: {"account"=>"jbladmin", "client_id"=>"docker", "offline_token"=>"true", "service"=>"oci-registry.pegasusio.io"}
portuscontainer                     |   User Load (1.0ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin' LIMIT 1
portuscontainer                     |   ApplicationToken Load (0.3ms)  SELECT `application_tokens`.* FROM `application_tokens` WHERE `application_tokens`.`user_id` = 2
portuscontainer                     |   User Load (0.6ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin'  ORDER BY `users`.`id` ASC LIMIT 1
portuscontainer                     |    (0.4ms)  BEGIN
portuscontainer                     |   SQL (0.5ms)  UPDATE `users` SET `failed_attempts` = 1, `updated_at` = '2020-02-25 22:32:36' WHERE `users`.`id` = 2
portuscontainer                     |    (0.4ms)  COMMIT
portuscontainer                     | Completed 401 Unauthorized in 112ms (ActiveRecord: 4.5ms)
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.3ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   Tag Exists (0.3ms)  SELECT  1 AS one FROM `tags` WHERE `tags`.`scanned` = 0 LIMIT 1

Here are the Portus Logs I get, when I docker login, using the right password and the first super admin username jbladmin (test login success) :

background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   Tag Exists (0.2ms)  SELECT  1 AS one FROM `tags` WHERE `tags`.`scanned` = 0 LIMIT 1
registry_1                          | time="2020-02-25T22:33:51Z" level=warning msg="error authorizing context: authorization token required" go.version=go1.7.6 http.request.host=oci-registry.pegasusio.io http.request.id=6edb93ea-ccef-4e1b-9dfc-4fbeaf157004 http.request.method=GET http.request.remoteaddr=<IP_ADDRESS> http.request.uri="/v2/" http.request.useragent="docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))" instance.id=b87dc65a-4fdf-43ae-8975-c276ec1eaaf0 version=v2.6.2-14-ga66a4c3
registry_1                          | <IP_ADDRESS> - - [25/Feb/2020:22:33:51 +0000] "GET /v2/ HTTP/1.0" 401 87 "" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))"
nginx_1                             | <IP_ADDRESS> - - [25/Feb/2020:22:33:51 +0000] "GET /v2/ HTTP/1.1" 401 87 "-" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \x5C(linux\x5C))"
portuscontainer                     | Started GET "/v2/token?account=jbladmin&client_id=docker&offline_token=true&service=oci-registry.pegasusio.io" for <IP_ADDRESS> at 2020-02-25 22:33:51 +0000
portuscontainer                     | Processing by Api::V2::TokensController#show as JSON
portuscontainer                     |   Parameters: {"account"=>"jbladmin", "client_id"=>"docker", "offline_token"=>"true", "service"=>"oci-registry.pegasusio.io"}
portuscontainer                     |   User Load (0.7ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin' LIMIT 1
portuscontainer                     |   ApplicationToken Load (1.2ms)  SELECT `application_tokens`.* FROM `application_tokens` WHERE `application_tokens`.`user_id` = 2
portuscontainer                     |   User Load (0.7ms)  SELECT  `users`.* FROM `users` WHERE `users`.`username` = 'jbladmin'  ORDER BY `users`.`id` ASC LIMIT 1
portuscontainer                     |    (0.4ms)  BEGIN
portuscontainer                     |   SQL (0.5ms)  UPDATE `users` SET `failed_attempts` = 0, `updated_at` = '2020-02-25 22:33:52' WHERE `users`.`id` = 2
portuscontainer                     |    (0.3ms)  COMMIT
portuscontainer                     |    (0.3ms)  BEGIN
portuscontainer                     |   User Exists (0.6ms)  SELECT  1 AS one FROM `users` WHERE (`users`.`username` = BINARY 'jbladmin' AND `users`.`id` != 2) LIMIT 1
portuscontainer                     |   SQL (0.7ms)  UPDATE `users` SET `last_sign_in_at` = '2020-02-25 22:16:07', `current_sign_in_at` = '2020-02-25 22:33:52', `last_sign_in_ip` = '<IP_ADDRESS>', `sign_in_count` = 3, `updated_at` = '2020-02-25 22:33:52' WHERE `users`.`id` = 2
portuscontainer                     |    (0.2ms)  COMMIT
portuscontainer                     |   Registry Load (0.2ms)  SELECT  `registries`.* FROM `registries` WHERE `registries`.`hostname` = 'oci-registry.pegasusio.io' LIMIT 1
portuscontainer                     | No hostname matching `oci-registry.pegasusio.io', testing external_hostname
portuscontainer                     |   Registry Load (0.6ms)  SELECT  `registries`.* FROM `registries` WHERE `registries`.`external_hostname` = 'oci-registry.pegasusio.io' LIMIT 1
portuscontainer                     | [jwt_token] [claim] {:iss=>"portus.pegasusio.io", :sub=>"jbladmin", :aud=>"oci-registry.pegasusio.io", :iat=>1582670032, :nbf=>1582670027, :exp=>1582670932, :jti=>"EQeJNCEC2fgySoTdFpVDPcJvwGT5wX3Jqa2PBGXzow"}
portuscontainer                     | Completed 200 OK in 118ms (Views: 0.2ms | ActiveRecord: 6.3ms)
registry_1                          | time="2020-02-25T22:33:52Z" level=info msg="response completed" go.version=go1.7.6 http.request.host=oci-registry.pegasusio.io http.request.id=4f2071ef-d684-4637-bf9d-2103b7c1e8d7 http.request.method=GET http.request.remoteaddr=<IP_ADDRESS> http.request.uri="/v2/" http.request.useragent="docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.377583ms http.response.status=200 http.response.written=2 instance.id=b87dc65a-4fdf-43ae-8975-c276ec1eaaf0 version=v2.6.2-14-ga66a4c3
registry_1                          | <IP_ADDRESS> - - [25/Feb/2020:22:33:52 +0000] "GET /v2/ HTTP/1.0" 200 2 "" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \\(linux\\))"
nginx_1                             | <IP_ADDRESS> - - [25/Feb/2020:22:33:52 +0000] "GET /v2/ HTTP/1.1" 200 2 "-" "docker/19.03.5 go/go1.12.12 git-commit/633a0ea838 kernel/4.9.0-12-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.5 \x5C(linux\x5C))"
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.4ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000
background_1                        |   RegistryEvent Load (0.3ms)  SELECT  `registry_events`.* FROM `registry_events` WHERE `registry_events`.`status` = 2  ORDER BY `registry_events`.`id` ASC LIMIT 1000

About the registry (Docker Distribution, alright) , and the notary auth. services
I want to point out one thing here :

In the first place, I really wondered, why in the world, could it be required that registry and notary use the same auth token issuer/authentication/authorization service.
And I understood then, that what you wanted is "to use the exact same command as you do with docker hub's public registry'. So authenticate to both notary and registry at the same time, so using the same service.
And My idea about Keycloak, is the following :

ok, portus auth token service is not fit for notary : so let's use another ?
yeah, we then have 2 auth token services, and how do we single sign on 2 different auth services ? Well that is identity federation, and there's Keycloak in the field... isn't there ?

Furthermore, in the docker public registry case, DTR, how / when do you authenticate to notary ? Well, when you dockerlogin, as confirmed in official documentation here, providing the example below :

# -- #
$ docker login dtr.example.com/user/repo
Username: admin
Password:

Login Succeeded

$ docker trust signer add --key cert.pem jeff dtr.example.com/user/repo
Adding signer "jeff" to dtr.example.com/user/repo...
Initializing signed repository for dtr.example.com/user/repo...
Successfully initialized "dtr.example.com/user/repo"
Successfully added signer: jeff to dtr.example.com/user/repo

Right, now last, and worth the note : I found https://forums.docker.com/t/understanding-notary/8684 , and there , Jeff Anderson, from the docker official team, April 2016, says indeed those two auth services can be the same, like on DTR. They can be the same, so they can also not (and be different) :

Notary can be configured to talk to the same token authentication service that the Docker
Registry V2 spec uses. In the case of DTR, DTR provides that component. If my DTR is running
at https://dtr.example.com/ 5, then my auth service is located at https://dtr.example.com/auth/token > 3.
Check out the “auth” section in this example config file: https://github.com/docker/notary/blob/master/docs/reference/server-config.md#overview 25
That should give you at least a starting point.
/Jeff

I definitely think we can federate 2 distincts JWT-token based auth services, using Keycloak, and maybe Keycloak Gatekeeper (A Keycloak Gatekeeper in front of both..?) .


[Release 1v1 [RANK] Jungle Wolf ZH v2] Player is unable to see terrain at all map borders (top and bottom)

Field of view - top and bottom

Now:
https://www.gamereplays.org/community/uploads/post-103811-1616940192.png
https://www.gamereplays.org/community/uploads/post-103811-1616940199.png
v3: https://www.gamereplays.org/community/index.php?act=Attach&type=post&id=861482

@RisingZH please test map and check if your problem is fixed.

Thanks for looking at this but still not fixed.
https://imgur.com/a/RpDE63i (There is a chinook collecting from the pile)
@SpeCoV1k

Thanks for looking at this but still not fixed.
https://imgur.com/a/RpDE63i (There is a chinook collecting from the pile)
@SpeCoV1k

i can just change position of this supply dock, but aften any unit will can drive around up dock (make same with sup dock down player) - ok?

@RisingZH

Hmm perhaps leave this one then. Moving the dock so players can drive round could have a big impact.

Fixed


fix-up: x86: Add bpc suffix to mc functions

Function names for avg were accidentally swapped with w_avg.
There should only have been a suffix change. Fixes #2951.

Coverage increased (+0.2%) to 85.447% when pulling 2bdd0ff4150735660bedb99e96438049a8ba998c on barrbrain:issue-2951 into eebd99d2930f80761a7322cb9a8bbd2dc614885c on xiph:master.


Upgrade Gateway API to v1.1.0

TBD

I wonder if this should be an epic with a few tasks.

would be the task that is already in review: https://github.com/nginxinc/nginx-gateway-fabric/pull/1975
Use the new conformance testing version/framework
support parentRef port field


Avoid overwriting infrastructure roles.

For a role defined in both the infrastructure roles secret and the cluster manifest use the infrastructure role definition and add flags defined in the manifest. #166

@alexeyklyukin Could you please add a small unit test for this case to avoid regressions in the future ?

Coverage decreased (-0.02%) to 8.139% when pulling 6adf447c78e4537acd519b0c34b9ce56711b5136 on merge_infrastructre_robot_roles into 68bb3cd52d2c021dcac28c38fb18c17d11ec8f69 on master.

Coverage decreased (-0.02%) to 8.139% when pulling 8e06c6d60aed3709fcea07ef9a8a6711472cf682 on merge_infrastructre_robot_roles into 68bb3cd52d2c021dcac28c38fb18c17d11ec8f69 on master.

Coverage decreased (-0.05%) to 8.108% when pulling 06436528dc2d7da25113325abd7e96bcf0a34e7a on merge_infrastructre_robot_roles into 68bb3cd52d2c021dcac28c38fb18c17d11ec8f69 on master.

Added tests for the initHumanUser and initRobotUsers

Coverage increased (+2.7%) to 10.888% when pulling 99331203fdad30ad9f3e55bf68be140ed34b2e6b on merge_infrastructre_robot_roles into 68bb3cd52d2c021dcac28c38fb18c17d11ec8f69 on master.

:+1:

👍

Coverage increased (+2.7%) to 10.888% when pulling 5a2a517d0caa6b955fdccc70c5ec33ce3db88646 on merge_infrastructre_robot_roles into 68bb3cd52d2c021dcac28c38fb18c17d11ec8f69 on master.


[issue#7] Includes only necessary methods of lodash

Also, I put "use strict" accordingly to the .jshintrc.
This pull requests solves https://github.com/akiran/react-slick/issues/7

@edygar Thanks for pull requested


feat: add cache max-age of 5 seconds to ?live requests so request collapsing works

We need a short max-age cache on ?live responses so http proxies will collapse long-polling requests.
The time is a bit arbitrary but two considerations:

it's shorter than 20 seconds (our long-polling timeout) — which is necessary as otherwise clients would just poll the cached response over and over until the cache expired.
it's long enough to ensure the vast majority of clients all request within the same five second window. Live clients all get responses at the same time so all request again at the same time. So even accounting for world-wide spread of clients, five seconds should collect pretty much everyone.

There's a very slight chance that a new message could be returned within the 5 second timeout and then someone with ?live gets a cached response. But that's not a big deal as then their next response gets collapsed again with other clients.

I get the live clients connecting at the same time. Because they block on data and then reconnect after it arrives.

There's a very slight chance that a new message could be returned within the 5 second timeout and then someone with ?live gets a cached response.

Trying to wrap my head around the scenario here. New data arrives within 5 seconds, the live request for one (or many) clients returns. A subsequent request gets that response, even if between the initial response and their request additional data has also arrived?
So if you miss being in the collapse gang and data is arriving in sequence you could end up with 5 seconds latency (for weird / hard to debug reasons from a front end POV)?

If I understand correctly the 5s TTL starts when the response is returned (either due to a message or the timeout), and so if it was possible to have a 1ms TTL that would also allow all connections within the 20s long poll to be collapsed.
what's the shortest possible TTL that triggers request collapsing?
this is very cool!

So if you miss being in the collapse gang and data is arriving in sequence you could end up with 5 seconds latency

No because you'd get a cache hit for the initial response and then the client would immediately join the new gang at the new offset and get collapsed into that request. It's possible that they missed several responses within a five second TTL window but that'd just mean they get individual messages for a series of cached http responses so very fast so not the worst thing in the world (in a pretty unlikely edge case).
In general, this is following our normal scheme where we trade off the possibility of needing to do a few http requests in sequence so that we can have longer caches & increase cache hit ratio.

If I understand correctly the 5s TTL starts when the response is returned (either due to a message or the timeout), and so if it was possible to have a 1ms TTL that would also allow all connections within the 20s long poll to be collapsed.

I'm a bit fuzzy on this but I think Fastly, et al. have heuristics around request collapsing so if they've seen similar URLs that have very short caches, they might stop pretty quickly collapsing requests. So e.g. a 1s cache might lead to later requests not being collapsed. Maybe. We could probably do more testing to figure out exactly how this works but it doesn't seem to matter too much as long as it satisfies the two considerations I outlined.

No because you'd get a cache hit for the initial response and then the client would immediately join the new gang at the new offset

I see -- so the idea is that the live response is triggered when there is data, which means it always contains a new offset, so the client then makes a new request with the new offset.
Just to sanity check in case it's a bug that breaks this assumption, do we only set a cache header on a non-empty live request? What happens in the event of a 20s timeout response?

Just to sanity check in case it's a bug that breaks this assumption, do we only set a cache header on a non-empty live request? What happens in the event of a 20s timeout response?

We are but... it's already expired by then 😆 so it doesn't have any effect (other than to tell the CDN that they should keep collapsing requests to the origin)

We are but... it's already expired by then 😆

Trying to twist my brain around that statement. Also reading https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#max-age and https://www.fastly.com/documentation/reference/http/http-headers/Age
Is it that the response has an age added because the CDN monitored the time it took to generate the response? My naive mental model is that the 20s timeout response is our business, generates a response with a max-age and the cdn/client gets that with an age of 0. Thus caches it for 5 seconds.
(On a separate note, I trust you that it's not but if an empty live response were to be cached it would also trigger zero delay polling between the client and the CDN.)

yeah my mental model is that the max-age starts at the time the request is received at the CDN not when the response finishes. Which makes sense as the origin might take 500ms to transmit the data — but the staleness ticking clock starts as soon as the origin starts transmitting the data not when it happens to finish.
Also nothing broke dramatically i.e. clients started furiously looping getting the same cached response. I'll be doing more testing today so we'll see.


Why is "rescore" is not available as a task_choice

In the latest version of peptdeep, global_settings['task_choices'] returns ['train', 'library']. Why is "rescore" not available as a choice? Thanks.

Rescoring is not the task of peptdeep, it should be another package based on peptdeep


Bad Performance related to syntax coloring

Prerequisites

[x] Can you reproduce the problem in safe mode?
[x] Are you running the latest version of Atom?
[x] Did you check the debugging guide?
[x] Did you check the FAQs on Discuss?
[x] Are you reporting to the correct repository?
[x] Did you perform a cursory search to see if your bug or enhancement is already reported?

For more information on how to write a good bug report or enhancement request, see the CONTRIBUTING guide.
Description
When commenting long HTML lines, the editor hangs if the comment is authored in a natural way.  There is a workaround that can avoid the editor hang.
Steps to Reproduce

Create an HTML document with multiple long lines of HTML code.  For example, create a script tag and place the contents of a large file into it.  Then create a second tag with the contents of another file.  NOTE:  you may need more HTML contents to reproduce this issue, if so simply create the code.
Preprend an HTML comment to the first script tag, i.e. "<!--".  The code below will be re-colored as a comment.
Append "-->" to the closing script tag.  The editor will hang while re-coloring the syntax.

Expected behavior: [What you expected to happen]
Just use the previously known coloration when closing the comment.  Why re-process?
Actual behavior: [What actually happened]
Editor hangs while re-coloring.
Workaround
When commenting long lines with an HTML comment, add the closing token first, then add the opening token.  No hanging occurs.

NOTE:  there's no reason to think this doesn't happen with long JavaScript lines commented with /* ... */.  Although unverified there's reason to suspect this could be a general problem for any programming language with multi-line comments in AtomIDE.

Versions
Atom    : 1.9.9
Electron: 0.37.8
Chrome  : 49.0.2623.75
Node    : 5.10.0
apm  1.10.0
npm  2.13.3
node 0.10.40
python 2.7.12
git 2.7.4

Create an HTML document with multiple long lines of HTML code.

Define "long" please?

Thank you for your issue!
We haven't gotten a response to our questions in our comment https://github.com/atom/atom/issues/12784#issuecomment-249675622. With only the information that is currently in the issue, we don't have enough information to take action. I'm going to close this but don't hesitate to reach out if you have or find the answers we need, we'll be happy to reopen the issue.

Re-open this bug, please, dude.  In my humble opinion this kind of support is a disservice to the Atom project.  You already have a definition of "long" in the "steps to reproduce".  Did you read them?  Did you even spend any time trying to reproduce the bug?  The "steps to reproduce" says you can use the contents of a file inside a script tag -- so why not try a JavaScript file that's approx. 1MB?  You don't need me to provide a URL since Atom probably has such files inside itself.

You never defined "long", only saying to "Create an HTML document with multiple long lines of HTML code".  How long is long?  100 characters?  1000?  Neither did you define "large".  Both of those can be extremely subjective.
No, I did not spend any time trying to reproduce this bug as I cannot without knowing how to.  Until you specify what you mean by "long" and "large", the Atom team will not be able to take action.

Do you need me to hold your hand to reproduce the bug after that thorough description and follow up?  Can you please re-open this bug and re-assign it to another community member that may take it more seriously?  When I reported the bug, I was embedding base64 data into HTML and/or CSS, and the Atom IDE was unusable without the workaround I provided.  You don't need to embed Base64 data to reproduce this bug.  I'm very disappointed you didn't even attempt to reproduce the bug because the effort you spent to close the bug is likely greater than that required to reproduce it.

I define long as a file > 500MB.  As stated in the description you may need to embed more than one (1) file.  Just give it a try!  I look forward to re-opening this bug, @50Wliu.

No need to re-open this bug report since it's been re-filed at #13521.


Electrum protocol timeout sometimes too small for a slow Electrum Personal Server

Hi,
I get the following error when trying to connect Electrum wallet to EPS on Lubuntu.

2018-08-08 16:31:07,926835 [  LOG] Electrum connected from ('<IP_ADDRESS>', 39146)
2018-08-08 16:31:40,306907 [  LOG] IOError: BrokenPipeError(32, 'Broken pipe')
2018-08-08 16:40:11,107133 [  LOG] IOError: OSError(0, 'Error')
2018-08-08 16:40:11,710609 [  LOG] IOError: OSError(0, 'Error')
2018-08-08 16:40:11,925471 [  LOG] IOError: OSError(0, 'Error')
2018-08-08 16:40:13,560771 [  LOG] Electrum connected from ('<IP_ADDRESS>', 39464)
2018-08-08 16:41:17,486015 [  LOG] IOError: BrokenPipeError(32, 'Broken pipe')
2018-08-08 16:41:17,708431 [  LOG] IOError: OSError(0, 'Error')
2018-08-08 16:41:17,947199 [  LOG] IOError: OSError(0, 'Error')`

I'm getting this error as well, any luck? I'm running on OS X and do not get any OSErrors, but get the Electrum connected from log and immediate BrokenPipeError. I looked through the debug.log and didn't find anything of note. Happy to help however I can.

I can't think of anything that would cause this.
EPS just displays the error message that it gets from the operating system.
Are you running EPS and Electrum on the same computer? If not, maybe there's something in between that stops the connection.

I'm running both on the same computer and I am connected to a mainnet full node so that can't be it

running electrum in osx and eps on raspibolt I get a disconnection after each block is updated by electrum

regarding raspibolt maybe its a memory issue

@3ntranced That sounds like a bug, check your debug.log as there might be useful information there.
Which Electrum version are you using?

Electrum 3.2.2 on osx running via python. EPS /Satoshi:0.16.1/ Here is some stuff from debug.log I havent spotted anything of interest:
2018-09-03 09:08:18,956379 [  LOG] Electrum connected from ('<IP_ADDRESS>', 54236)
2018-09-03 09:08:18,987160 [DEBUG] => {"method": "server.version", "params": ["3.2.2", "1.2"], "id": 312}
2018-09-03 09:08:18,987878 [DEBUG] <= {"jsonrpc": "2.0", "id": 312, "method": "server.version", "result": ["ElectrumPersonalServer 0.1", "0.1"], "params": ["3.2.2", "1.2"]}
2018-09-03 09:08:18,988171 [DEBUG] => {"method": "blockchain.headers.subscribe", "params": [true], "id": 313}
2018-09-03 09:08:18,994881 [DEBUG] <= {"jsonrpc": "2.0", "id": 313, "method": "blockchain.headers.subscribe", "result": {"hex": "00000020dfa352bb6c354c116dd571b1bcff1f0b1229674694d502000000000000000000bc6b8236f76e924b53e14159a575aaf6b28b0ce1dcb0236c55ac5537d17e32d3b85b8d5b2dd729172dfd11ab", "height": 539809}, "params": [true]}
2018-09-03 09:08:18,995234 [DEBUG] => {"method": "server.banner", "params": [], "id": 314}
2018-09-03 09:08:19,010700 [DEBUG] <= {"jsonrpc": "2.0", "id": 314, "method": "server.banner", "result": "Welcome to Electrum Personal Server\n\nMonitoring 7 deterministic wallets, in total 1400 addresses.\n\nConnected bitcoin node: /Satoshi:0.16.1/\nPeers: 9\nUptime: 21:09:33\nBlocksonly: False\nPruning: False\nDownload: 133.7 MB\nUpload: 60.0 MB\n\nhttps://github.com/chris-belcher/electrum-personal-server\n\nDonate to help make Electrum Personal Server even better:\nbc1q5d8l0w33h65e2l5x7ty6wgnvkvlqcz0wfaslpz\n\n", "params": []}
2018-09-03 09:08:19,011024 [DEBUG] => {"method": "server.donation_address", "params": [], "id": 315}
2018-09-03 09:08:19,011518 [DEBUG] <= {"jsonrpc": "2.0", "id": 315, "method": "server.donation_address", "result": "bc1q5d8l0w33h65e2l5x7ty6wgnvkvlqcz0wfaslpz", "params": []}
2018-09-03 09:08:19,011711 [DEBUG] => {"method": "server.peers.subscribe", "params": [], "id": 316}
2018-09-03 09:08:19,012363 [DEBUG] <= {"jsonrpc": "2.0", "id": 316, "method": "server.peers.subscribe", "result": [], "params": []}
2018-09-03 09:08:19,012545 [DEBUG] => {"method": "mempool.get_fee_histogram", "params": [], "id": 317}
2018-09-03 09:08:30,581692 [DEBUG] <= {"jsonrpc": "2.0", "id": 317, "method": "mempool.get_fee_histogram", "result": [[10.0, 112934], [6.0, 483947], [5.0, 713934], [4.0, 2267678], [3.0, 444389], [2.0, 376929], [1.0, 2684991], [0.0, 1084]], "params": []}
2018-09-03 09:08:30,620212 [DEBUG] => {"method": "blockchain.estimatefee", "params": [25], "id": 318}
2018-09-03 09:08:30,638143 [  LOG] IOError: BrokenPipeError(32, 'Broken pipe')
2018-09-03 09:08:31,001018 [  LOG] Electrum connected from ('<IP_ADDRESS>', 54262)

Hmm, the log above isn't even increasing the Blockchain: blocks in Electrum. Yesterday at least the blocks were slowly increased.

Android electrum client also doesnt get its block height increased and gets constantly disconnected, the debug.log looks like this
2018-09-03 10:14:38,148024 [DEBUG] => {"method": "server.peers.subscribe", "params": [], "id": 84}
2018-09-03 10:14:38,148520 [DEBUG] <= {"id": 84, "method": "server.peers.subscribe", "jsonrpc": "2.0", "params": [], "result": []}
2018-09-03 10:14:38,148770 [DEBUG] => {"method": "mempool.get_fee_histogram", "params": [], "id": 85}
2018-09-03 10:15:07,863515 [DEBUG] <= {"id": 85, "method": "mempool.get_fee_histogram", "jsonrpc": "2.0", "params": [], "result": [[31.0, 100313], [15.0, 109902], [9.0, 129802], [7.0, 188927], [6.0, 1451753], [5.0, 1212650], [4.0, 3727956], [3.0, 563196], [2.0, 523531], [1.0, 2791553], [0.0, 1276]]}
2018-09-03 10:15:07,926704 [DEBUG] => {"method": "blockchain.estimatefee", "params": [25], "id": 86}
2018-09-03 10:15:07,944894 [DEBUG] <= {"id": 86, "method": "blockchain.estimatefee", "jsonrpc": "2.0", "params": [25], "result": 2.008e-05}
2018-09-03 10:15:07,945462 [DEBUG] => {"method": "blockchain.estimatefee", "params": [10], "id": 87}
2018-09-03 10:15:07,963771 [DEBUG] <= {"id": 87, "method": "blockchain.estimatefee", "jsonrpc": "2.0", "params": [10], "result": 3.17e-05}
2018-09-03 10:15:07,964104 [DEBUG] => {"method": "blockchain.estimatefee", "params": [5], "id": 88}
2018-09-03 10:15:07,982929 [DEBUG] <= {"id": 88, "method": "blockchain.estimatefee", "jsonrpc": "2.0", "params": [5], "result": 6.291e-05}
2018-09-03 10:15:07,983260 [DEBUG] => {"method": "blockchain.estimatefee", "params": [2], "id": 89}
2018-09-03 10:15:08,002297 [DEBUG] <= {"id": 89, "method": "blockchain.estimatefee", "jsonrpc": "2.0", "params": [2], "result": 6.942e-05}
2018-09-03 10:15:08,002627 [DEBUG] => {"method": "blockchain.relayfee", "params": [], "id": 90}
2018-09-03 10:15:08,360852 [  LOG] IOError: BrokenPipeError(32, 'Broken pipe')
2018-09-03 10:15:08,634044 [  LOG] IOError: OSError(0, 'Error')
2018-09-03 10:15:08,868799 [  LOG] IOError: OSError(0, 'Error')

Try running electrum with the command line argument -v, then go the Qt console and type network.debug = True. Then you should get lots of debug information on the command line. Hopefully that will provide more clues.

This time its doing one block at a time, then disconnecting and then reconnecting:
[Network] <-- {'id': 78, 'method': 'blockchain.block.get_header', 'jsonrpc': '2.0', 'params': [526231], 'result': {'block_height': 526231, 'merkle_root': 'a6e0063cf1607c8dcbd48e831bbfbee88ab6d2af33394aff6b1ae16498190a1f', 'nonce': 573456290, 'version': 536870912, 'timestamp':<PHONE_NUMBER>, 'prev_block_hash': '0000000000000000002595d7aa312e40a61d86eb7fbf26c517feae3d6a2cbbf9', 'bits': 389609537}}
[Network] <-- {'method': 'blockchain.headers.subscribe', 'jsonrpc': '2.0', 'params': [], 'result': {'height': 539847, 'hex': '00e0ff3f656aaacd2c61797db7aa7d2b4df29b560f0588d860fa1c0000000000000000008a58628897db9987e892df89e67cf272361213aee2e2b6dbd80833411d79203b0db78d5b2dd72917e6cc826c'}}
[<IP_ADDRESS>] blockchain request timed out
[Network] connecting to <IP_ADDRESS>:50002:s as new interface
[<IP_ADDRESS>] connected
[Network] <IP_ADDRESS> --> server.version ['3.2.2', '1.2'] 79
[Network] <IP_ADDRESS> --> blockchain.headers.subscribe [True] 80
[Network] switching to <IP_ADDRESS>:50002:s

is something wrong with my bitcoind config that its getting a timeout when requesting a block? I do have bitcoind and bitcoind-testnet running at the same time on the raspberry pi

If there was something wrong with the bitcoin rpc config then EPS wouldn't even start, because at the beginning it calls getbestblockhash just to see if it works.
I'm going to check if there's a way to increase Electrum's timeout interval. Maybe your network/node/raspberry PI is just a bit too slow.

yes could be

OK this is what you do to increase the timeout.
Edit the Electrum source file. Go to electrum/interface.py on this line: https://github.com/spesmilo/electrum/blob/780b2d067c1ade3575e9511e9e3c71a4a39f50c2/electrum/interface.py#L114
It sets the socket timeout to 2 seconds, change that number to something high (say 10 seconds). Then run Electrum again.
Your problem here reminded me of a similar problem I've had. Sometimes when having my Electrum connect to my ssh tunnel over tor hidden service (as in #36) it would drop the connection. I just tried the same thing with increasing the timeout and it fixed my problem too. Seems to be that tor slowness gave rise to a similar thing.
BTW there's another timeout involved too, on these lines: https://github.com/spesmilo/electrum/blob/780b2d067c1ade3575e9511e9e3c71a4a39f50c2/electrum/interface.py#L324-L325 but they have a timeout of 10 seconds instead of 2, so shouldn't be too bad.
Long term this probably means the Electrum source code should be edited to allow users to configure those timeouts (maybe on the command line)

10 seconds didnt work for me. 20 seconds for all three worked for me. Thank you

@odiyan and @jtomazin are you able to try this thing with increasing the timeouts.

That did it for me thanks! Even 20 seconds didn't work for me, nor did 30, then I went for something arbitrarily high - it's 90 now - and it connected.

That's good, and a bit strange that 90(!) seconds was needed.
Reopening the issue because this is not really fixed. Having users edit source code is not really a solution.

I put 20 seconds in all 3 lines and compiled for android and it worked.

@chris-belcher I didn't try intermediate values - I wouldn't be surprised if it started working at something like 50 seconds. But yes, my system gets incredibly slow when core is running.

If it helps: got the same issue on a node that had continuously been online for a week or so. Restarting core solved it (for the time being).

__t

good information. i'll have to patch electrum for this in my GUI.

A PR which fixes this has been merged into Electrum https://github.com/spesmilo/electrum/pull/4945. So it should be in the next release.

Hi @chris-belcher, I'm experiencing the same loss of connection when tor is activated on my node (works perfectly fine when node connects through clearnet). Would you help me to "run with command-line argument --oneserver and --server [host]:50002:s" so that I can extend timeouts? I'm running OS X. Thank you very much in advance!

Found the solution and it works fine now. For anyone using OS X and experiencing the same issues:
Write a script in Script Editor with the following content (adjust path and IP(host)):
do shell script "open -a '/Users/username/Folder/Electrum.app' --args --oneserver --server [host]:50002:s"
https://superuser.com/questions/16750/how-can-i-run-an-application-with-command-line-arguments-in-mac-os

I ran into same issue recently (raspibolt running over TOR with EPS on it, connecting from Electrum on a Mac laptop). Can confirm that opening Electrum with the line in the Script Editor works. Thanks @ramses2506 !
I'm wondering if it's possible to implement this in a future EPS release, so it works without the workaround in Script editor? It took me quite a bit of time of finding out what the problem might be (wasn't sure if it was with the bitcoin backend, Electrum, EPS or something else), and I think most other newbs like me will struggle, too :)

The configuration needs to be done for Electrum not EPS. So the only way I can think to implement something in a future EPS release is to add a helpful error message, for example if the Electrum wallet is seen to disconnect then write something like "try running with --oneserver and --server <IP_ADDRESS>:50002:s". Also this could be written about in the README.

Opened the config file in the Electrum root directory in notepad. Changed the line oneserver=false to oneserver=true. Worked for me.

Closing this issue as its fixed.


Switch flake8->ruff

Hi @samuelcolvin ,
I noticed you moved from flake8 to ruff on pydantic-core (https://github.com/pydantic/pydantic-core/commit/bfe2d13488edbda0101e96c73cef114999486223), so I thought it would be a good idea too in watchfiles.

Thanks so much.


[BUG] Fix marching cubes not in the right space when CT is not aligned

Fixes issue related to https://mne.discourse.group/t/wrong-ct-and-t1-alignment-shown-in-gui/7325/2

@larsoner would you happen to know why these tests are failing again?
It might be
Collecting git+https://github.com/pyvista/pyvista@main
  Cloning https://github.com/pyvista/pyvista (to revision main) to /tmp/pip-req-build-d27__lhw
  Running command git clone --filter=blob:none --quiet https://github.com/pyvista/pyvista /tmp/pip-req-build-d27__lhw
  Resolved https://github.com/pyvista/pyvista to commit 098a6beabb4ccc924a789ede86002d9fdf56a8aa
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting PyQt6
  Downloading PyQt6-6.6.1-cp38-abi3-manylinux_2_28_x86_64.whl (7.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.9/7.9 MB 77.7 MB/s eta 0:00:00

Collecting PyQt6-Qt6!=6.6.1
  Downloading PyQt6_Qt6-6.6.0-py3-none-manylinux_2_28_x86_64.whl (67.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.4/67.4 MB 38.5 MB/s eta 0:00:00

the PyQt version mismatch here but that's odd that it worked for a bit and then stopped working... I'll try with PyQt6 != 6.6.1

https://output.circle-artifacts.com/output/job/91844d98-d2c7-4156-96da-146da1477aa3/artifacts/0/dev/auto_examples/locate_ieeg_micro.html#sphx-glr-auto-examples-locate-ieeg-micro-py
Looks good to me

Ok, this should handle when images are not LIA now by going through scanner RAS. I also added some very standard boilerplate to the webpage since it was looking very lonely. If there's any issue with that I can split it out. Otherwise, looks good to go.

Well there was actually a larger issue where all the tests were skipped because they didn't have imageio-ffmpeg... there weren't that many failures so it wasn't too hard to fix.

Okay to merge by me

Oh man well I looked again and the 3D rendering of electrodes when the CT is not aligned were off so then I had to think about with no recon-all what space the 3D rendering should be in. On main it's in surface RAS but if it's surface RAS of an unaligned CT this all just seems so unnecessarily complicated so I streamlined the workflow a bit more:

The base image is loaded

If it is aligned to a recon, show the brain and head surfaces in scanner RAS (aligned)
Else just do marching cubes on the image and also show that in scanner RAS

Electrode positions are in head, convert to mri (surface RAS) of base image using trans then convert to scanner RAS

This way everything internally is in scanner RAS and I think it just simplifies a lot all these transforms which caused this bug.

Ok, thanks for the review, the last commit just finishes moving the 3D surfaces to scanner RAS, good that you approve of doing that.
Sounds good, I'll wait a day or so in case I hear back from the people who reported the bug if they have a chance to test this before merging.
Re 1.6, it's odd that it fails on stable still. I guess it's fine just to wait to move to stable until 1.7...

All the examples look the same: https://output.circle-artifacts.com/output/job/aea15c4c-5071-434b-832d-f56b4ea42749/artifacts/0/dev/auto_examples/index.html
The diff is a bit large to review but will merge tomorrow unless I come upon any other issues merging this into the other two WIP PRs.


rustup override remove does not work for windows root directories`

If I'm in C:\
$ rustup override add nightly
$ rustup override remove
info: no override toolchain for 'C:\'

Not works with the root directory, as reported and confirmed.

I was also able to reproduce the error you saw where rustc printed the wrong version for the "C:" override.


CICD Improvements

In light of the errors in build process with the 23.7.1 release, I would like to revisit our CICD process and propose some improvements.

Process Questions

Do we need CI to produce SNAPSHOT versions of jar artifacts? As far as I know these are not re-used elsewhere in build process, and are regularly overwritten by whatever PR was most recently built. Is there enough value here to rebuild and republish these to jfrog as frequently as we are? I suspect not.
Same question with regard to docker images/manifests.

Proposed Changes to CI

Draft PR workflow

Doesn't get any CI run. Draft PRs should be used any time you need to share thoughts or gather feedback from others.
Discussion point would it be helpful to honor PLEASE CI tags in commit messages?

Open PR workflow - no longer in Draft. Asking for deeper thought from peers during review, and should have been evaluated by CI prior to asking peers for that review.

run serially to fail fast:
- gradlew verification - if gradle is compromised, full stop.

compile - if it doesn't compile (reproducibly!), don't run more tests. (Applies to test code too)
dco - if it's not licensed correctly, can't be merged.
spotless - if it hasn't been formatted correctly, fail. Any reasons not to just run spotlessApply when this happens?
javadoc - right now this fails the build at the same time spotless is checked.
repolinter - similar to dco, state of repo must meet HL specs

then run parallel

unit tests - Discussion point - are we ok with these being re-run every time an author adopts a pr suggestion?
CodeQL

Open PRs with at least one approval

all remaining acceptance/priv/integration/reference tests in parallel: Discussion point - do these need to be run earlier? Are they providing more filtering that is important to have done before peer review?

Nightly

run everything above up to unit tests
SonarQube analysis

If SonarQube coverage analysis acceptance/integration test results, then they are worth running as well.

Release

Requires a git tag
Run all the things above except Nightly.
push jar artifacts with release tags to jfrog - Discussion point - jfrog should be configured to not permit updates to the artifact at a GAV coordinate once it is used.
dockerScan
push docker images/manifests/tags to dockerhub
move latest docker tag

If changes adopted, we should also review the default PR comment included for new PRs, which guides users toward crafting a good pr that will likely not fail these later checks.

Do we need CI to produce SNAPSHOT versions of jar artifacts? As far as I know these are not re-used elsewhere in build process, and are regularly overwritten by whatever PR was most recently built. Is there enough value here to rebuild and republish these to jfrog as frequently as we are? I suspect not.
Same question with regard to docker images/manifests.

As recently as today, Linea has used snapshot builds both from jfrog maven and from docker to end-run around release issues.  I think there is value in continuing to build and publish those images, even if they are volatile

As recently as today, Linea has used snapshot builds both from jfrog maven and from docker to end-run around release issues and/or delays. I think there is value in continuing to build and publish those images, even if they are volatile

Would a nightly build have been sufficient, or did they need more recent changes than that?

Would love to hear thoughts from anyone that has developed an opinion on the new(ish) Merge Queue mechanic in github, and how they see it possibly fitting into a new CICD/PR review flow.

Draft PR workflow

Like the idea of a draft PR workflow, but I'd want the option to run the tests before converting to ready for review. I'd say avoiding tests is the exceptional case though, so maybe a manual way to skip tests would be better.
Not a fan of using a comment, would prefer to handle via github if possible.

Would a nightly build have been sufficient, or did they need more recent changes than that?

I think a nightly publish would be sufficient. There's definitely value in having our nightly canaries running with the latest changes. When testing PRs on infra, we also need a way to get that code onto the infra. Maybe a manual publish step from the PR could work. Failing that, there's always just using git/gradle on the box but via CI would be nice :)

Would love to hear thoughts from anyone that has developed an opinion on the new(ish) Merge Queue mechanic in github, and how they see it possibly fitting into a new CICD/PR review flow.

re: Merge Queue, there were a couple of missing features that were essentially blockers for us IIRC, e.g. not being able to edit the description. Doesn't look like it's been added yet but might be worth another look since it's GA now: https://github.com/orgs/community/discussions/46757
I haven't thought about it much, but how do you foresee the Queue helping with the CI problems you've mentioned?
Note, similar discussions are being had here: https://wiki.hyperledger.org/display/BESU/Proposal%3A+Quarterly+releases+from+main+by+default?focusedCommentId=98731846#comment-98731846

Merge queue - I really like the idea in theory - there are times when it would be really handy - but in practice yes it was still a bit green when we tried it.
I have seen with GHA when jobs queue up, that there can be multiple duplicate tasks for the same PR (eg before and after a merge from main update) - but I'm assuming GHA can be configured for newer job requests to cancel older ones.

Do we need CI to produce SNAPSHOT versions of jar artifacts? As far as I know these are not re-used elsewhere in build process, and are regularly overwritten by whatever PR was most recently built. Is there enough value here to rebuild and republish these to jfrog as frequently as we are? I suspect not.
Same question with regard to docker images/manifests.

As recently as today, Linea has used snapshot builds both from jfrog maven and from docker to end-run around release issues and/or delays. I think there is value in continuing to build and publish those images, even if they are volatile

we also use these builds to spin up on-demand nodes to performance test etc for specific PRs - but that's not needed for every single PR

At a higher level, what's the main driver for these improvements - eg do we want to optimise for developer/reviewer time, CI spend, CI time? Something else, or some combination?


Duo Mobile MFA authentication gets triggered 6 times at once

Hello!
With Snowflake we are using the Duo Mobile multi-factor authentication service. It pings my phone with an Accept/Deny message. Acceptance is cached for 6 hours.
When I use the Wizard extension, Duo Mobile is triggered six times at once. This will happen again for me in about 4 hours, since this happened 2 hours ago (and has been happening since I started using the extension a while back). I think this would be prevented if Wizard did one connection to dbt / snowflake, and once that is successful, it does the next five.

Hey @thomasaarholt!
Do you use User / Password + DUO MFA authentication method and specify authenticator: username_password_mfa in your profiles.yml? The issue might be that we don't use the 'authenticator' value in the Snowflake client within the extension. I'm trying to understand the expected value for this field.

I've actually been using authenticator: externalbrowser, but I can actually use that or username_password_mfa. One of our engineers suggested that the multiple number of Duo requests might be due to running dbt on multiple threads?

@thomasaarholt
We use the dbt CLI to run dbt commands. If you don't encounter any issues when running dbt in the terminal, we should also be fine running it from the extension. Additionally, we call the Snowflake API multiple times to fetch source schemas, for example.
I've made a change in the Snowflake API client configuration in v0.33.1 to address this issue. Can you please test it on your end and let me know how it performs?

With ´authenticor: username_password_mfa´ I had perfect behaviour now, only one MFA request!
I'll try again after the token timeout tomorrow morning (europe time here) with the externalbrowser one.
Thank you very much!

Aii, I spoke to soon. I got one message, accepted that one, wrote this message. Then I opened command + Tab'd back into VSCode, and I got another 4 notifications immediately.

I can also add that I'm seeing this message now and then: dbt compile timeout exceededWizard for dbt Core (TM), but its probably unrelated to this. The is only there for a few minutes now and then.

Aii, I spoke to soon. I got one MFA request, accepted that one, wrote this message. Then I opened command + Tab'd back into VSCode, and I got another 4 notifications immediately.

Thank you for checking it. I will continue working on this issue.

Hi @thomasaarholt! In v0.33.2 I prevented concurrent connection attempts to Snowflake. Could you please check it on your side with MFA and let me know if this helps?

Just updated, and it unfortunately still triggers 6 notifications.


Investigate use of emoji in embeds

It doesn't appear that Discord supports emoji in embeds. Let's see if there's an easy workaround.

Closed with 06aba6e.


how to develop skill (api )

Privileged issue

[X] I'm @StreetLamb or he asked me directly to create an issue here.

Issue Content
I am working on the multiagent development, need to add more skill , presales -skill ( product demo, technical discussion, proposal preparing ) , marketing manager （  event arrangement, leads collection ..）, so how to develop the API in tribe？

Hi @jachenms, please take a look at the skills section of the readme. If you are just making an API call, go to the Skills page > click the 'Add Skill' button > define your skill definition. However, for more complex skills, you need to write the code for it.


Why dilated_lstm and normal_lstm are exactly the same?

There is no difference between the two folders. ?

Thank you for your interest! We didn't update our recent version to our master branch.
You should go to dLSTM branch dlstm_a2c folder to check the new one.
However, our code still doesn't work.
We're still on the progress, so please keep interests into our github repo.

Thank you for your interest! We didn't update our recent version to our master branch.
You should go to dLSTM branch dlstm_a2c folder to check the new one.
However, our code still doesn't work.
We're still on the progress, so please keep interests into our github repo.

what do you mean by 'doesn't work', you mean the score doesn't increase after many episodes or the learning curve doesn't converge(ie, policy loss, value loss)? I've designed my own universal HRL algorithm similar to FuN, yet it didn't even converge.


Multiple spaces after namespace keyword

I noticed that PHP-CS-Fixer does not fix multiple spaces after the namespace keyword.
The PHP version you are using ($ php -v):

PHP 7.0.30 (cli) (built: Apr 26 2018 11:49:41) ( NTS )

PHP CS Fixer version you are using ($ php-cs-fixer -V):

PHP CS Fixer 2.11.1 Grey Devil by Fabien Potencier and Dariusz Ruminski

The command you use to run PHP CS Fixer:

vendor/friendsofphp/php-cs-fixer/php-cs-fixer fix

The configuration file you are using, if any:
Using ruleset @PSR1, @PSR2 and @Symfony, otherwise pretty standard.
If applicable, please provide minimum samples of PHP code (as plain text, not screenshots):

before running PHP CS Fixer (no changes):

namespace  My\Name\Space;

with unexpected changes applied when running PHP CS Fixer:

namespace  My\Name\Space;

with the changes you expected instead:

namespace My\Name\Space;

Thanks for reporting @iluuu1994 !
Currently we've no rule for this, maybe we could add this to an existing rule, do you've an idea for a good fit to add this?

@SpacePossum Sure thing, I'll see if I can write or extend a fixer. 👍 I noticed the same thing for return statements.

It is tempting, I recall doing something like that. However there are a few cases left to consider ;)
For example the binary_operator_spaces can be configured to do this:
$a  = 1;
$ab = 1;

than there are cases like;
$abc = 1; // aligned...
$b = 2;   // ... comment

But after all, having a GenericSingleSpaceAfterX - fixer with config options like return, namespace still sounds very tempting to me... yet I'm not sure there will be enough support for such a fixer here..

I would build the fixer the other way around, like;
SingleSpaceAfterFixer
than add configuration options like;
cast (then we deprecate cast_spaces), namespace, visibility, etc., but not other were we have dedicated rules for like binary operator .
Using the white list makes it easier to prevent conflicts and would allow for more refined configuration.
Downside is of course that it takes more time to write the fixer.
As a general rule I would not touch multi line spacing and not spacing before a comment, than we should be on the safe side and still be able to fix a lot :)
WDYT?

I tried the no_multiple_spaces fixer after all and ran it over the Symfony code base:
https://github.com/iluuu1994/symfony/commit/a574c085ea3869947a3ab1e46fdb7ca8d5ca45c4
This works perfectly for our use case. I know this is not exactly what you had in mind but the suggested fixer would leave many things unfixed in our code base. But obviously this breaks intentionally aligned code such as parameters.
Are you interested in this fixer? It could still be useful to people who strictly want to disallow aligning code like we do.


Could you make it so that we can download and run .exe files

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Sorry not possible.

@SoulAFK this is a recreation, not an emulator.


Add AWS Role ARN to access S3

Thank you for contributing to the CF CLI! Please read the following:

Please make sure you have implemented changes in line with the contributing guidelines
We're not allowed to accept any PRs without a signed CLA, no matter how small.
If your contribution falls under a company CLA but your membership is not public, expect delays while we confirm.
All new code requires tests to protect against regressions.
Contributions must be made against the appropriate branch. See the contributing guidelines
Contributions must conform to our style guide. Please reach out to us if you have questions.

Where this PR should be backported?

[ ] main
[ ] v8
[ ] v7

Description of the Change
We must be able to understand the design of your change from this description.
Keep in mind that the maintainer reviewing this PR may not be familiar with or
have worked with the code here recently, so please walk us through the concepts.
Why Is This PR Valuable?
What benefits will be realized by the code change? What users would want this change? What user need is this change addressing?
Applicable Issues
List any applicable GitHub Issues here
How Urgent Is The Change?
Is the change urgent? If so, explain why it is time-sensitive.
Other Relevant Parties
Who else is affected by the change?

Hey all.  I see this was merged 2 hours ago, which is coincidentally around the time I started seeing this error:
  Err:8 https://cf-cli-debian-repo.s3.amazonaws.com/ stable/main amd64 Packages
    403  Forbidden [IP: <IP_ADDRESS> 443]
  Reading package lists...
  E: Failed to fetch https://cf-cli-debian-repo.s3.amazonaws.com/dists/stable/main/binary-amd64/Packages  403  Forbidden [IP: <IP_ADDRESS> 443]
  E: Some index files failed to download. They have been ignored, or old ones used instead.
  The command '/bin/sh -c apt-get update' returned a non-zero code: 100

This occurred using cf-cli-action.v2, and I only started seeing it within the last hour or so.  Seeing cf-cli-debian-repo I decided to check cf-cli issue tracker but didn't see anything, however I did happen to spot this commit/PR going in.
AFAIK an AWS Role ARN wouldn't cause IP whitelist/blacklisting issues, but I'm not 100% sure.  Given the timing coincidence I thought it'd be worth asking here in-case it immediately jumps out at anyone, however it could be completely unrelated, so forgive me if that's the case!  Anyone got any ideas?
Thanks.


[Bug] code generation can't be compiled with v1.49.0 of multi blocks of modules.

But it seems that it can't generate code successfully with the command just:
[SEVERE] : Header C:\WINDOWS\TEMP\.tmpGHyO0z.h: Total errors/warnings: 2.
[SEVERE] :     C:\WINDOWS\TEMP\.tmpGHyO0z.h:20:16: error: redefinition of 'wire_uint_8_list' [Semantic Issue]
[SEVERE] :     C:\WINDOWS\TEMP\.tmpGHyO0z.h:23:3: error: typedef redefinition with different types ('struct (unnamed struct at C:\WINDOWS\TEMP\.tmpGHyO0z.h:20:16)' vs 'struct wire_uint_8_list') [Semantic Issue]

The command indeed finished, but with the log above during generation, I guess these should not appear, right?
Moreover, when checking the generated dart code, say class of RuleApi in gened_rule_api.dart:
class RuleApiImpl implements RuleApi {
  final RuleApiPlatform _platform;
  factory RuleApiImpl(ExternalLibrary dylib) =>
      RuleApiImpl.raw(RuleApiPlatform(dylib));
...
}

It raised an issue on word RuleApi, saying: `"Classes and mixins can only implement other classes and mixins.
Try specifying a class or mixin, or remove the name from the list."  And there are many other issues:

Codegen logs with RUST_LOG=debug environment variable
[SEVERE] : Header C:\WINDOWS\TEMP\.tmpGHyO0z.h: Total errors/warnings: 2.
[SEVERE] :     C:\WINDOWS\TEMP\.tmpGHyO0z.h:20:16: error: redefinition of 'wire_uint_8_list' [Semantic Issue]
[SEVERE] :     C:\WINDOWS\TEMP\.tmpGHyO0z.h:23:3: error: typedef redefinition with different types ('struct (unnamed struct at C:\WINDOWS\TEMP\.tmpGHyO0z.h:20:16)' vs 'struct wire_uint_8_list') [Semantic Issue]

To Reproduce
No response
Expected behavior
No response
Generated binding code
No response
OS
win10
Version of flutter_rust_bridge_codegen
1.49.0
Flutter info
Flutter 3.4.4
Version of clang++
No response
Version of ffigen
6.1.2
Additional context
Since the SEVERE ISSUE is related to wire_uint_8_list, and this is the inner type of rust String.
I hope someone could test the multi_test with API contains String as input parameter, to see if CI test passed or not.

I hope someone could test the multi_test with API contains String as input parameter, to see if CI test passed or not.

Feel free to add a test into the CI, so next time nobody can break the feature you have created :) (I always ensure CI passes before merging any PR.)

As for the bug itself, maybe we can bisect frb so we will know which commit/PR cause the regression

As for the bug itself, maybe we can bisect frb so we will know which commit/PR cause the regression

I think it needed to check if the string parameter is ok with the multiple blocks example, first.  But I have no idea what is wrong here with the CI test. I just added some API with string parameters.


YT Native Controls are not displayed

In case you opt to go with youtube native controls, we are setting the controls param to 0, which hides everything, so you can't scrub, play, save, etc. I think the right default should be to show the controls as normal, so we'd need to set it to 1.
Also, show video info is broken as well.

What we want is this:

But what we have right now is this

I was thinking we could document that if you go with native youtube controls, the views will actually count, albeit as long as you press the play button.

I'm not sure to understand your problem. Native controls are wroking fine. What is the steps to reproduce and what is the expected behavior?

I made this quite a while ago, and I was working on an earlier version. Right now it seems like everything is working well. Thanks for all your hard work!


Nonexistent bimmer_connected version specified

https://github.com/home-assistant/home-assistant/pull/26012 bumps bimmer_connected version to 0.5.6, but there is no such version on PyPI, so things are failing.
https://pypi.org/project/bimmer-connected/#history

Seems this got fixed in 34f7bb5b8c06d4dd96c36f3f84d09fdb9a48679b


[Fern's Documentation] SDK generator options missing from documentation

What can be improved
Previously it was possible to see the available options for any given SDK generator on Fern's website. This is no longer possible it seems. The list of SDKs don't link to any SDK specific documentation: https://buildwithfern.com/learn/sdks/introduction/overview
I also weren't able to find this information anywhere else on the site.

We're iterating on our documentation and expect it to get to parity with our previous docs over time. For now, you can use this link to access our previous docs site which has the info I think you're looking for! https://docs.buildwithfern.com/generate-sd-ks/generators/type-script-node-js-sdk

That link is unfortunately wildly out of date, listing the latest version of the TS generator as 0.9.5. Could this ticket be reopened until such time the docs are updated?

Any updates on this?

Will this documentation make a return at some point?

@mstade this isn't something we've got on the near term roadmap. We're prioritizing our time based on what our customers are asking for.
Here's a link to the TypeScript SDK generator custom config (without full documentation): https://github.com/fern-api/fern/blob/main/generators/typescript/sdk/cli/src/custom-config/SdkCustomConfig.ts


Can't find package "yaml"

Describe the bug
When I used vscode to read the source code, an error showd that "could not import github.com/falcosecurity/event-generator/events/k8saudit/yaml (no required module provides package "github.com/falcosecurity/event-generator/events/k8saudit/yaml")"
Screenshots

How can I solve  it?

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/close


Use Haskell instead of bash

@begriffs: Interested in converting usage of bash to Haskell?  This commit implements the vast majority of functionality:  https://github.com/jship/haskell-vim-now/commit/d608a71ae84cf22c57891b12f0d7432c5e4256a2
My main motivation for doing this was to bring haskell-vim-now to Windows and there are other advantages besides Windows support:

Type safety
Installer speed - helper binaries are installed much quicker via dependency filtering, way easier to do in Haskell than in bash
Access to all packages in Stackage snapshots - this was nice for stache, ansi-terminal, mtl, etc.

This would definitely take additional iterations and community testing to ensure it is feature-complete with the bash scripts.  Also have a handful of TODOs that should be resolved for it to be as rock-solid as can be.  Maybe the Haskell version could be mentioned as "in beta" in the readme?
I saw an old issue (#131) where @SX91 was interested in this.  Since then, Stack 1.4.0 has made Haskell scripting much nicer, and the Turtle lib is great to work with.

Been meaning to comment on this, sorry for the delay. Pretty cool that you got it rewritten like that. I feel kind of like a heretic, but have to admit that I'm not sure the Haskell version looks easier to understand than bash. Maybe if the main logic happened at the start and then functions came later it would orient me better to the code.

I agree that using bash is more readable as there's no layer between the script and the shell commands.  Turtle does a great job of making scripting nice in Haskell, but there is still a noticeable divide.  :)
The code is set up to parallel the existing bash scripts, so I kept most function names the same just with Haskell's casing conventions.  The big difference is that it's all in one script instead of split across multiple files.  Wrote this very much in a "bang out the code" mentality,  so there are places where it could be refactored for readability/cleanliness.
Supporting Windows is what led me to do it in Haskell and am open to any language really.

Agree that the windows support would be nice. By the way, does the rest of the vim configuration work OK on windows once the installer is done? Have you given it a try?

Things work from my testing - ghc-mod, codex, autocomplete, compile check, hlint, git stuff, ctrlp, nerdtree, comment selection, etc.  I haven't done an exhaustive test of every config though.
There are some caveats/TODOs I've marked via comments in the script.  Would want to resolve those, and might be something the community can help with?
Plugin install is fine, but there is some weirdness around upgrading plugins.  vim-plug seems to be able to download new plugin versions and it might still work, but it throws an error on accessing a temp directory.
I'll need to set up my fork to run the install on Travis.  Haven't tried the script on non-Windows platforms yet.

I might be radio silent on this for a week or so.  ZuriHac's next week - woo!

Hope you're having fun at ZuriHac.
So I've been thinking more about the Haskell installer. Another argument against it is that fewer people will be able to easily contribute fixes. I'm imagining people who want to learn Haskell installing haskell-vim-now and being able to contribute to the Bash script if something goes wrong but having more trouble contributing to a Haskell installer.
On the other hand, there was that recent bug in the installer due to the treatment of booleans which was totally a type error that would never happen in Haskell. Maybe we want to practice what we preach and use Haskell... I think the code should be consciously written in a very approachable style though.
How about opening a pull request with your work in progress so we can discuss in greater detail?

Will do, thanks!

I think the code should be consciously written in a very approachable style though.

Completely agree.  I used a MTL-style for this and am fine changing this up if it would make it more approachable.  Most functions have MonadIO as a constraint anyways, so we could just as well use IO directly or use plain transformers with no MTL typeclass constraints.

Closing this as we want to gradually introduce Haskell and see how it goes along the way.  For more on this discussion, see #253.


Tokens: Opacity

⚠ On hold until https://github.com/ouds-sandbox/ouds-android/issues/34

Description
The aim of this issue is to study and implement the opacity tokens taking into account the cohesive multi-platform approach, the customization by libraries inheriting OUDS Android, but also what we're using for OUDS Android: Jetpack Compose, Material 3, etc.

Define the possibilities
Define the constraints
Define the impacts
Implement the tokens (with a future Style Dictionary usage in mind)

Reminder: there will be in the end 3 layers of tokens:

Raw primitive values: Insertion of brand foundations.

These tokens will be customizable by libraries inheriting OUDS Android.
These tokens cannot be used by developers using OUDS Android.

Semantic applications: Depends on raw primitive values.

Mapping between these tokens and the raw primitive values will be customizable by libraries inheriting OUDS Android.
These tokens can be used by developers using OUDS Android.

Component applications: Depends on semantic applications.:

Mapping between these tokens and the semantic tokens should rarely be customized by libraries inheriting OUDS Android.
These tokens can be used by developers using OUDS Android.

Tokens: Opacity
Raw primitive values

Token name
Token value

opacity-0
0

opacity-100
0.04

opacity-200
0.08

opacity-300
0.16

opacity-400
0.24

opacity-500
0.32

opacity-600
0.48

opacity-700
0.64

opacity-800
0.88

opacity-900
1

Semantic applications

Token name
Token value

opacity-transparent
opacity-0

opacity-weaker
opacity-100

opacity-weak
opacity-300

opacity-medium
opacity-500

opacity-emphasis
opacity-700

opacity-opaque
opacity-900

Component applications
⚠ This is just an example, it doesn't need to be implemented for now, it will be when the modals will be tackled

Token name
Token value

overlays-modal-backdrop-opacity
opacity-emphasis

Study

[x] Study the architecture of these tokens, and define the technical details

Technical details

[ ] Add opacity semantic tokens to implement in the OudsCustomTheme interface of the "theme-contract" module
[ ] Add raw primitive values for opacities in the "theme-orange" module
[ ] Implement the opacity semantic tokens in "theme-orange" module with the help of opacity raw values

Isn't it weird to be able to rewire an opaque token or a transparent token? Opaque will always be 1 and transparent 0, no ?

Thinking to a theme not extending the Orange theme, does it have to rewire all the opacity tokens or we should define default values to avoid that?


Error when deploying system contract

Most recent versions for eosio, eosio.cdt, and eosio.contracts.
Build the system contract using the build script.
Reading WASM from /Users/jaechung/eosio.contracts/build/contracts/eosio.system/eosio.system.wasm...
Publishing contract...
Error 3080006: Transaction took too long
Error Details:
deadline exceeded
pending console output:

just try again


Unable to download Facebook reel

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
US
Provide a description that is worded well enough to be understood
I am unable to download a Facebook reel. This is the first time I've tried to download from Facebook, and it appears to be a supported site, but I don't know if I need to do something about login/authentication. I have updated to the nightly build.
It seems to be having a problem with the file or directory name, and I have to wonder if it's the non-ASCII characters there. In the output below, one of them shows up as the 🏕 emoji after "Dystopian campfire song", but in the command line window (see attached), I see a couple others - just before and after the first "Reina del Cid".

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', 'https://www.facebook.com/reel/9489207347819100']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[bbeacff7f] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 5.1.2-full_build-www.gyan.dev (setts), ffprobe 5.1.2-full_build-www.gyan.dev, rtmpdump 2.3
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.07.22, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1857 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: 9d17e631df35ea8cbe0c91b7ba8e34c2602b95c9c7ae6065be68ed6adfe385eb
yt-dlp is up to date<EMAIL_ADDRESS>[facebook:reel] Extracting URL: https://www.facebook.com/reel/9489207347819100
[facebook] Extracting URL: https://m.facebook.com/watch/?v=9489207347819100&_rdr
[facebook]<PHONE_NUMBER>819100: Downloading webpage
[debug] Sort order given by extractor: res, quality
[debug] Formats sorted by: hasvid, ie_pref, res, quality, lang, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info]<PHONE_NUMBER>819100: Downloading 1 format(s):<PHONE_NUMBER>43873v+1006454377379056a
[debug] Invoking http downloader on "https://video.fagc1-1.fna.fbcdn.net/v/t66.30100-2/10000000_993824931645300_5360112277497998341_n.mp4?_nc_cat=102&ccb=1-7&_nc_sid=9c5c06&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfdnA5ZXZlXzEwODBwX3YxIn0=&_nc_ohc=NI6naKqCcBkAX9mScd2&_nc_ht=video.fagc1-1.fna&oh=00_AfBuaRS4xmNyNdNmkZNR4qjxy7ZynE1CBfIir3s2Jqla9g&oe=64C96BD6"
ERROR: unable to open for writing: [Errno 2] No such file or directory: 'Dystopian campfire song 🏕 #paro dy #americanpie #ai #artificialintelligence #machinelearning #robot #asimov #hal9000 #terminator #dystopian #apocalypse ｜ Reina del Cid ｜ Reina del Cid · Original audio [9489207347819100].f673967434743873v.mp4.part'
Traceback (most recent call last):
  File "yt_dlp\utils\_utils.py", line 593, in sanitize_open
yt_dlp.utils.LockingUnsupportedError: File locking is not supported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\downloader\http.py", line 260, in download
  File "yt_dlp\downloader\common.py", line 245, in wrapper
  File "yt_dlp\downloader\common.py", line 231, in error_callback
  File "yt_dlp\utils\_utils.py", line 5117, in report_retry
  File "yt_dlp\downloader\common.py", line 240, in wrapper
  File "yt_dlp\downloader\common.py", line 251, in sanitize_open
  File "yt_dlp\utils\_utils.py", line 596, in sanitize_open
FileNotFoundError: [Errno 2] No such file or directory: 'Dystopian campfire song 🏕 #parody #americ anpie #ai #artificialintelligence #machinelearning #robot #asimov #hal9000 #terminator #dystopian #apocalypse ｜ Reina del Cid ｜ Reina del Cid · Original audio [9489207347819100].f673967434743873v.mp4.part'

[debug] Invoking http downloader on "https://video.fagc1-2.fna.fbcdn.net/v/t66.30100-2/42100359_1508588236613793_2013930831616496268_n.mp4?_nc_cat=109&ccb=1-7&_nc_sid=9c5c06&efg=eyJ2ZW5jb2RlX3RhZyI6ImRhc2hfbG5fYXVkaW9fdjEifQ==&_nc_ohc=dK1uwdXjmBoAX9glIKJ&_nc_ht=video.fagc1-2.fna&oh=00_AfAInzEKR6BIpBhm78jEafddKz-2L2KrASwyi8mvA66GGQ&oe=64C89EB1"
ERROR: unable to open for writing: [Errno 2] No such file or directory: 'Dystopian campfire song 🏕 #paro dy #americanpie #ai #artificialintelligence #machinelearning #robot #asimov #hal9000 #terminator #dystopian #apocalypse ｜ Reina del Cid ｜ Reina del Cid · Original audio [9489207347819100].f1006454377379056a.m4a.part'
Traceback (most recent call last):
  File "yt_dlp\utils\_utils.py", line 593, in sanitize_open
yt_dlp.utils.LockingUnsupportedError: File locking is not supported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\downloader\http.py", line 260, in download
  File "yt_dlp\downloader\common.py", line 245, in wrapper
  File "yt_dlp\downloader\common.py", line 231, in error_callback
  File "yt_dlp\utils\_utils.py", line 5117, in report_retry
  File "yt_dlp\downloader\common.py", line 240, in wrapper
  File "yt_dlp\downloader\common.py", line 251, in sanitize_open
  File "yt_dlp\utils\_utils.py", line 596, in sanitize_open
FileNotFoundError: [Errno 2] No such file or directory: 'Dystopian campfire song 🏕 #parody #americ anpie #ai #artificialintelligence #machinelearning #robot #asimov #hal9000 #terminator #dystopian #apocalypse ｜ Reina del Cid ｜ Reina del Cid · Original audio [9489207347819100].f1006454377379056a.m4a.part'

It's erroring because the filename is too long
You can mitigate this with an output template, e.g.: -o "%(title).150B [%(id)s].%(ext)s"
(or decrease to .100B etc if that still doesn't work)

Thank you, that worked! I take it that's not normally necessary? Of course, I had to stumble across the one and only obscenely long FB reel filename, right?
Thanks again!

I take it that's not normally necessary?

Usually only if the pathname of the downloaded file is > ~255 chars and your OS is windows