Can we avoid manual steps around nginx and let's encrypt SSL?

I was wondering if it would be possible for this project to publish a docker-compose file that would take care of let's encrypt certs creation and renewal automatically? For those people using letsencrypt.
And for everyone, can we use something that manages adding sites to nginx  reverse proxy automatically without need for manual steps?
Something like this: https://github.com/jwilder/nginx-proxy and this: https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion ?

It's possible yes but I don't think that's the purpose of this docker image, unlike mailinabox for example, hardware/mailserver isn't a mailserver ready out-of-box with automatic reverse proxy, dns and ssl configuration. Everyone is free to use what is needed beside this docker image. Postfixadmin, Rainloop, NSD, Nginx, letsencrypt...etc all of this components are all optional.

My Ansible playbooks add the letsencrypt SSL creation on top of this awesome project.

Or use ksylvan's awesome ansible playbooks :p

@AndrewSav https://github.com/jwilder/nginx-proxy is a very interesting approach.
In my use cases I also have hosts that serve normal nginx setups... but I guess that can be handled by a normal nginx:alpine container (so that I can still serve up the www.foo.bar/some_file.txt and www.foo.bar/~user/foo.txt requests).

@ksylvan so are you running ansible in a cron job or something, to keep the certs up to date? Intuitively it does not feel that ansible is the right tool for this job, it is traditionally runs when configuration changes not regularly. But then again, I might be thinking about it all wrong.

@hardware I understand your answer. You are not interested in making improvements here yourself but you will be happy to review a PR if any. I guess this answers the question. Thanks.

No. The playbooks simply deploy a server with a working web and mail stack. Running Ansible inside a cron job would be crazy. :-)
In the process, they create the initial LetsEncrypt SSL certs and also set up an auto-renewal mechanism. Specifically, your running server will run a cron job to auto-renew your SSL certs when they are near expiry.

@ksylvan oh, that's neat! Did not realize that!

@AndrewSav specifically, all I do is set up a secure server (set up firewall, fix ssh so no password logins, no root login, etc.), install Docker CE on it, and then set up this docker-based stack.
On your deployed server, even certbot and docker-compose are run as scripts that utilize underlying docker images.
 You can even run a wordpress install as your web server if you want (and @outbackdingo is adding the ability to run Opencart as store.yourdomain.tld).

@ksylvan - I expanded a bit my previous message and there is a question there for your opinion on how your playbook fits my scenario ;) Thanks!

So the actual problem, I'm trying to solve is ability to easily migrate my setup somewhere else
when I need it it. (At the same time being able to host other apps behind the same proxy,
being able to migrate them too and have auto renewals for all of them) In this regards,
the initial configuration is not that important for me, because it is already set up ;)

So for me the most promising avenue seems to be a docker-compose file that I can use to
down on the old installation copy the volumes to the other one, and up in the new place.
Do you think your playbook still might be more convenient in this scenario?

Yes. Easily. Here is how I would do it.

Instantiate a new VPS instance (Debian 9, etc.)

Bring down docker on the old server, copy the entire /mnt/docker directory to new server.

On my control host (where Ansible is run), hack /etc/hosts file so that the server name is pointing to the new IP address.

Run my playbooks against the new server, this will ensure that docker and the software stack, firewall settings, etc. are fixed. You should have your new stack completely running, with all the modifications you made over time preserved.

Fix the DNS settings to point to the new IP address. Also will have to fix the SPF record (I think that's the only one you'd need to fix, since the DKIM keys are saved in /mnt/docker hierarchy).

The only caveat is that if you add other apps behind that nginx proxy, you just want to make sure that all the persistent data ends up in /mnt/docker in some logical fashion so that you don't have to grab files/directories scattered around your system.
One final note: I never change the docker-compose.yml on the deployed system. I let that be generated and installed by Ansible.

https://traefik.io

I might be too late in the party here, but I am using https://github.com/jwilder/nginx-proxy in the mail server that hardware has here.  The nginx-proxy provides me the certificate that I feed in the mail server container.  Also, the webmail and the postfix containers using the SSL from the nginx-proxy container and the proxy container handles the traffic between the postfix admin and the webmail.
I put everything together in a docker-composer file.  If you need an example happy to share.

please do share quite interesting stuff here... i used the
docker-mailserver ansible runbook worked like a charm for me
On Sat, Nov 11, 2017 at 4:07 PM, Istvan Szabo<EMAIL_ADDRESS>wrote:

I might be too late in the party here, but I am using
https://github.com/jwilder/nginx-proxy in the mail server that hardware
has here. The nginx-proxy provides me the certificate that I feed in the
mail server container. Also, the webmail and the postfix containers using
the SSL from the nginx-proxy container and the proxy container handles the
traffic between the postfix admin and the webmail.
I put everything together in a docker-composer file. If you need an
example happy to share.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hardware/mailserver/issues/188#issuecomment-343671355,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABXFTlhXtS8U4R5ob-53lRjOXfyaAz7hks5s1bg4gaJpZM4QYkGR
.

Here is my docker-compose yaml file:
services:
  mailserver:
    image: hardware/mailserver:1.1-stable
    container_name: mailserver
    domainname: <FQDN>
    hostname: mail
    restart: always
    networks:
      - mailnet
    links:
      - "mariadb:mariadb"
    ports:
      - "25:25"     # SMTP                - Required
      - "143:143"   # IMAP       STARTTLS - Optional - For webmails/desktop clients
      - "587:587"   # Submission STARTTLS - Optional - For webmails/desktop clients
      - "993:993"   # IMAPS      SSL/TLS  - Optional - For webmails/desktop clients
      - "11334:11334" # RSPAMD WEBINTERFACE
    environment:
      - DBPASS=<DBPASS>
      - RSPAMD_PASSWORD=<RSPAMDPWD>
      - ADD_DOMAINS=<IHAVEOTHERDOMAINSHERE, for example: example.com,newword.com>
    volumes:
      - /mnt/docker/mail:/var/mail
      - /mnt/docker/mail/opendkim:/etc/opendkim/keys
      - /usr/local/etc/proxy/certs/:/etc/letsencrypt
    depends_on:
      - mariadb
      - redis
 # Redis
  redis:
    image: redis:3.2-alpine
    container_name: redis
    command: redis-server --appendonly yes
    networks:
      - mailnet
    volumes:
      - /mnt/docker/redis/db:/data
 # Administration interface
  postfixadmin:
    image: hardware/postfix:latest
    container_name: postfixadmin
    domainname: <FQDN>
    hostname: mail
    restart: always
    networks:
      - proxynet
      - mailnet
    links:
      - mariadb:mariadb
    environment:
      DBHOST: mariadb
      DBUSER: <DBUSER>
      DBNAME: <DBNAME>
      DBPASS: <DBPWD>
      VIRTUAL_HOST: postfixadmin.<FQDN>
      VIRTUAL_PORT: 8081
      LETSENCRYPT_EMAIL: <YOURVALIDEMAIL ADDRESS>
      LETSENCRYPT_HOST: "<HOSTLIST THAT YOU NEED FOR SSL"
      HTTPS_METHOD: redirect
      CERT_NAME: mail.<FQDN>
    depends_on:
      - mailserver
      - mariadb
 # Webmail (Optional)
  rainloop:
    image: hardware/rainloop
    container_name: rainloop
    restart: always
    links:
      - mailserver:mailserver
    environment:
      VIRTUAL_HOST: mail.<FQDN>
      VIRTUAL_PORT: 8888
      LETSENCRYPT_EMAIL: <ANOTHER VALID MAIL ADDRESS THAT YOU USE>
      LETSENCRYPT_HOST: "mail.<FQDN>,postfixadmin.<FQDN>"
      HTTPS_METHOD: redirect
      CERT_NAME: mail.<FQDN>
    networks:
      - proxynet
      - mailnet
    volumes:
      - /mnt/docker/rainloop:/rainloop/data
    depends_on:
      - mailserver
      - mariadb
  > # Database
  mariadb:
    image: mariadb:10.1
    container_name: mariadb
    restart: always
    networks:
      - mailnet
    environment:
      - MYSQL_ROOT_PASSWORD=<ROOTPWD>
      - MYSQL_DATABASE=<POSTFIXDB>
      - MYSQL_USER=<POSTFIXUSER>
      - MYSQL_PASSWORD=<POSTFIXUSERPWD>
    volumes:
      - /mnt/docker/mysql/db:/var/lib/mysql
networks:
  proxynet:
    external:
      name: proxy_proxynet
  mailnet:
    driver: bridge

I modified the postfix image with using a different http port than 8888.  And here is my nginx-proxy yaml file:
# NGINX server

version: '2'

services:
  nginx-proxy:
    image: jwilder/nginx-proxy:alpine
    container_name: nginx-proxy
    networks:
      - proxynet
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - /usr/local/etc/proxy/certs/:/etc/nginx/certs:ro
      - /etc/nginx/vhost.d
      - /usr/share/nginx/html

  letsencrypt-nginx-proxy:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: letsencrypt-nginx-proxy
    networks:
      - proxynet
    volumes_from:
      - nginx-proxy
    volumes:
      - /usr/local/etc/proxy/certs/:/etc/nginx/certs
      - /var/run/docker.sock:/var/run/docker.sock:ro
    depends_on:
      - nginx-proxy

networks:
  proxynet:
    driver: bridge

I hope this example help others to using nginx-proxy with hardware-mail* solution.

can you possible give a breif detail on why nginx-proxy is maybe better
then the one being used for us ?
On Mon, Nov 13, 2017 at 2:37 PM, Istvan Szabo<EMAIL_ADDRESS>wrote:

Here is my docker-compose yaml file:
services:
mailserver:
image: hardware/mailserver:1.1-stable
container_name: mailserver
domainname:
hostname: mail
restart: always
networks:
- mailnet
links:
- "mariadb:mariadb"
ports:
- "25:25"     # SMTP                - Required
- "143:143"   # IMAP       STARTTLS - Optional - For webmails/desktop clients
- "587:587"   # Submission STARTTLS - Optional - For webmails/desktop clients
- "993:993"   # IMAPS      SSL/TLS  - Optional - For webmails/desktop clients
- "11334:11334" # RSPAMD WEBINTERFACE
environment:
- DBPASS=
- RSPAMD_PASSWORD=
- ADD_DOMAINS=<IHAVEOTHERDOMAINSHERE, for example: example.com,newword.com>
volumes:
- /mnt/docker/mail:/var/mail
- /mnt/docker/mail/opendkim:/etc/opendkim/keys
- /usr/local/etc/proxy/certs/:/etc/letsencrypt
depends_on:
- mariadb
- redis
Redis
redis:
image: redis:3.2-alpine
container_name: redis
command: redis-server --appendonly yes
networks:
- mailnet
volumes:
- /mnt/docker/redis/db:/data
Administration interface
postfixadmin:
image: hardware/postfix:latest
container_name: postfixadmin
domainname:
hostname: mail
restart: always
networks:
- proxynet
- mailnet
links:
- mariadb:mariadb
environment:
DBHOST: mariadb
DBUSER:
DBNAME:
DBPASS:
VIRTUAL_HOST: postfixadmin.
VIRTUAL_PORT: 8081
LETSENCRYPT_EMAIL:
LETSENCRYPT_HOST: "<HOSTLIST THAT YOU NEED FOR SSL"
HTTPS_METHOD: redirect
CERT_NAME: mail.
depends_on:
- mailserver
- mariadb
Webmail (Optional)
rainloop:
image: hardware/rainloop
container_name: rainloop
restart: always
links:
- mailserver:mailserver
environment:
VIRTUAL_HOST: mail.
VIRTUAL_PORT: 8888
LETSENCRYPT_EMAIL:
LETSENCRYPT_HOST: "mail.,postfixadmin."
HTTPS_METHOD: redirect
CERT_NAME: mail.
networks:
- proxynet
- mailnet
volumes:
- /mnt/docker/rainloop:/rainloop/data
depends_on:
- mailserver
- mariadb

Database

mariadb:
image: mariadb:10.1
container_name: mariadb
restart: always
networks:
- mailnet
environment:
- MYSQL_ROOT_PASSWORD=
- MYSQL_DATABASE=
- MYSQL_USER=
- MYSQL_PASSWORD=
volumes:
- /mnt/docker/mysql/db:/var/lib/mysql
networks:
proxynet:
external:
name: proxy_proxynet
mailnet:
driver: bridge
I modified the postfix image with using a different http port than 8888.
And here is my nginx-proxy yaml file:
NGINX server
version: '2'
services:
nginx-proxy:
image: jwilder/nginx-proxy:alpine
container_name: nginx-proxy
networks:
- proxynet
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- /usr/local/etc/proxy/certs/:/etc/nginx/certs:ro
- /etc/nginx/vhost.d
- /usr/share/nginx/html
letsencrypt-nginx-proxy:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: letsencrypt-nginx-proxy
networks:
- proxynet
volumes_from:
- nginx-proxy
volumes:
- /usr/local/etc/proxy/certs/:/etc/nginx/certs
- /var/run/docker.sock:/var/run/docker.sock:ro
depends_on:
- nginx-proxy
networks:
proxynet:
driver: bridge
I hope this example help others to using nginx-proxy with hardware-mail*
solution.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/hardware/mailserver/issues/188#issuecomment-343920961,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABXFTnrZiMRIjIv_ulXuH4lh34RZDbIkks5s2EYxgaJpZM4QYkGR
.

I am using the nginx-proxy on my web hosting service and for me, it was easy to setup/reuse the same container here since the cert support is available in the mail server container.  Also the nginx-proxy with the companion handle the renewal process based on the docker-compose file.  I hope this answer your question.
For the record: I did not try to use the let's encrypt feature in the mailserver.

@sknight80 is this more convenient to split it over two docker-compose files like this?

For me yes, because one yml file handles the proxy-related events and the second one is focusing on the mail-server related events.  You can combine this two of course but for start and stop command in docker-compose will be a little bit longer and you might stop the mail server when you don't want too.  I hope this answer your question.

Hi,
Here is my confiuration 100% working using this mailserver and traefik for proxy and letsencrypt certs :
First, the docker-compose.yml for traefik :
`version: '2'
services:
traefik:
restart: unless-stopped
image: traefik
container_name: proxy_traefik
hostname: traefik
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./files/traefik.toml:/traefik.toml
- ./files/acme:/etc/traefik/acme
networks:
- traefik_proxy
networks:
traefik_proxy:
external:
name: traefik_proxy`
And the appropriate traefik.toml file :
`#debug = true
defaultEntryPoints = ["http", "https"]	# defaultEntryPoints must be at the top because it should not be in any table below
[web]
Port for the status page
address = ":8080"
#[web.auth.basic]
users = ["traefik:generate-with-htpasswd"]
Entrypoints, http and https
[entryPoints]
[entryPoints.http]					# http should be redirected to https
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"				# https is the default
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
Enable ACME (Let's Encrypt): automatic SSL
[acme]
caServer = "https://acme-staging.api.letsencrypt.org/directory"
email =<EMAIL_ADDRESS>storage = "/etc/traefik/acme/acme.json" 				# or "traefik/acme/account" if using KV store
entryPoint = "https"
onDemand = false
OnHostRule = true
[[acme.domains]]
main = "smtp.domain.tld"
[[acme.domains]]
main = "imap.domain.tld"
[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "domain.tld"
watch = true
exposedbydefault = false				# tells Træfik to only include containers with the label traefik.enable=true`
Second, my docker-compose.yml file for mailserver :
(Please note that I don't use rainloop through this package. I am using it through nextcloud)
`version: '2.1'
services:
mailserver:
image: hardware/mailserver:1.1-stable
container_name: mail-server
hostname: mail
domainname: domain.tld
depends_on:
- mariadb
- redis
ports:
- "25:25"     # SMTP                - Required
- "143:143"   # IMAP       STARTTLS - For webmails/desktop clients
- "465:465"   # SMTPS      SSL/TLS  - Enabled for compatibility reason, otherwise disabled
- "587:587"   # Submission STARTTLS - For webmails/desktop clients
- "993:993"   # IMAPS      SSL/TLS  - For webmails/desktop clients
- "4190:4190" # SIEVE      STARTTLS - Optional, but recommended for mail filtering
volumes:
- ./files/mail/data:/var/mail
- ./files/mail/opendkim:/etc/opendkim/keys
environment:
- VMAILUID=1069
- VMAILGID=1069
- OPENDKIM_KEY_LENGTH=4096
- DBPASS=xxxxxxx
- RSPAMD_PASSWORD=xxxxxxx
- ADD_DOMAINS=domain2.tld,domain3.tld
networks:
- mail_back
Databases
mariadb:
image: mariadb:latest
container_name: mail-server_db
environment:
- MYSQL_ROOT_PASSWORD=xxxxxxx
- MYSQL_DATABASE=postfix
- MYSQL_USER=postfix
- MYSQL_PASSWORD=xxxxxxx
volumes:
- ./files/mail/db:/var/lib/mysql
networks:
- mail_back
Control panel
postfixadmin:
image: hardware/postfixadmin
container_name: mail-postfixadm
domainname: gardelle.net
hostname: mail
labels:
- traefik.enable=true
- traefik.frontend.rule=Host:mailadm.domain.tld
- traefik.backend.port=8888
- traefik.docker.network=traefik_proxy
depends_on:
- mailserver
- mariadb
environment:
- UID=1069
- GID=1069
- DBPASS=xxxxxxx
networks:
- mail_back
- traefik_proxy
Cache
redis:
image: redis:4.0-alpine
container_name: mail-cache_redis
# restart: always
command: redis-server --appendonly yes
volumes:
- ./files/redis/db:/data
networks:
- mail_back
volumes:
files:
driver: local
networks:
traefik_proxy:
external:
name: traefik_proxy
mail_back:`
Regards,
z.

That's quite neat. Thanks for sharing that,

@sknight80 I noticed that for postifxadmin and rainloop your environment block does not have dashes preceding each environment variable. Is that because you did not copy your actual config here and edited out private information, but instead just typed in something from memory, hoping that it might work?

The docker compose environment block can be written in two ways:
environment:
  RACK_ENV: development
  SHOW: 'true'
  SESSION_SECRET:

Or:
environment:
  - RACK_ENV=development
  - SHOW=true
  - SESSION_SECRET

And since we are on a config sharing spree, here is what I ended up with:
version: '2.1'

services:
  nginx-proxy:
    image: jwilder/nginx-proxy:alpine
    container_name: nginx-proxy
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - /mnt/docker/certs:/etc/nginx/certs:ro
      - /mnt/docker/nginx/vhost.d:/etc/nginx/vhost.d
      - /mnt/docker/nginx/html:/usr/share/nginx/html
    depends_on:
      - postfixadmin
      - rainloop
      - mailserver

  letsencrypt-nginx-proxy:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: letsencrypt-nginx-proxy
    restart: always
    environment:
      # Unfortunately every time TOS changes this breaks. There is a PR to fix it here
      # https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion/issues/90
      # Eventually it will make it to the official container
      ACME_TOS_HASH: cc88d8d9517f490191401e7b54e9ffd12a2b9082ec7a1d4cec6101f9f1647e7b
    volumes_from:
      - nginx-proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /mnt/docker/certs:/etc/nginx/certs
    depends_on:
      - nginx-proxy

  mailserver:
    image: hardware/mailserver:1.1-stable
    container_name: mailserver
    domainname: <domain.name>                  # Mail server A/MX/FQDN & reverse PTR = mail.domain.tld.
    hostname: mail
    restart: always
    # extra_hosts:                          - Required for external database (on other server or for local database on host)
    #  - "mariadb:xx.xx.xx.xx"              - Replace with IP address of MariaDB server
    ports:
      - "25:25"       # SMTP                - Required
    # - "110:110"     # POP3       STARTTLS - Optional - For webmails/desktop clients
      - "143:143"     # IMAP       STARTTLS - Optional - For webmails/desktop clients
    # - "465:465"     # SMTPS      SSL/TLS  - Optional - Enabled for compatibility reason, otherwise disabled
      - "587:587"     # Submission STARTTLS - Optional - For webmails/desktop clients
      - "993:993"     # IMAPS      SSL/TLS  - Optional - For webmails/desktop clients
    # - "995:995"     # POP3S      SSL/TLS  - Optional - For webmails/desktop clients
      - "4190:4190"   # SIEVE      STARTTLS - Optional - Recommended for mail filtering
    # - "11334:11334" # HTTP                - Optional - Rspamd WebUI
    environment:
      # MariaDB database password (required)
      - DBPASS=<dbpass>
      # Rspamd WebUI password (required)
      - RSPAMD_PASSWORD=<rspamdpass>
      # Add additional domains separated by commas (needed for dkim keys etc.)
      - ADD_DOMAINS=<additional domaings>
      - VIRTUAL_HOST=<spam.FQDN,mail.FQDN>
      - VIRTUAL_PORT=11334
      - HTTPS_METHOD=redirect
    # - ENABLE_POP3=true                    # Enable POP3 protocol
    # - ENABLE_FETCHMAIL=true               # Enable fetchmail forwarding
    # - DISABLE_CLAMAV=true                 # Disable virus scanning
    # - DISABLE_SIGNING=true                # Disable DKIM/ARC signing
    # - DISABLE_GREYLISTING=true            # Disable greylisting policy
    # - DISABLE_RATELIMITING=true           # Disable ratelimiting policy
    #
    # Full list : https://github.com/hardware/mailserver#environment-variables
    #
    volumes:
      - /mnt/docker/mail:/var/mail
      - /mnt/docker/certs:/etc/letsencrypt
    depends_on:
      - mariadb
      - redis

  # Administration interface
  # https://github.com/hardware/postfixadmin
  # http://postfixadmin.sourceforge.net/
  # Configuration : https://github.com/hardware/mailserver/wiki/Postfixadmin-initial-configuration
  postfixadmin:
    image: hardware/postfixadmin
    container_name: postfixadmin
    domainname: <domain.name>
    hostname: postfixadmin
    restart: always
    environment:
      DBPASS: <dbpass>
      VIRTUAL_HOST: postfixadmin.FQDN
      HTTPS_METHOD: redirect
    depends_on:
      - mailserver
      - mariadb

  # Webmail (Optional)
  # https://github.com/hardware/rainloop
  # https://www.rainloop.net/
  # Configuration : https://github.com/hardware/mailserver/wiki/Rainloop-initial-configuration
  rainloop:
    image: hardware/rainloop
    container_name: rainloop
    restart: always
    environment:
      VIRTUAL_HOST: webmail.FQDN
      LETSENCRYPT_EMAIL: <let's encrypt email>
      LETSENCRYPT_HOST: "mail.FQDN,webmail.FQDN,postfixadmin.FQDN,spam.FQDN,other hosts behind this nginx-proxy"
      HTTPS_METHOD: redirect
    volumes:
      - /mnt/docker/rainloop:/rainloop/data
    depends_on:
      - mailserver
      - mariadb

  # Database
  # https://github.com/docker-library/mariadb
  # https://mariadb.org/
  mariadb:
    image: mariadb:10.1
    container_name: mariadb
    restart: always
    # Info : These variables are ignored when the volume already exists (databases created before).
    environment:
      - MYSQL_ROOT_PASSWORD=<mysqlrootpassword>
      - MYSQL_DATABASE=postfix
      - MYSQL_USER=postfix
      - MYSQL_PASSWORD=<mysqlpassword>
    volumes:
      - /mnt/docker/mysql/db:/var/lib/mysql

  # Cache Database
  # https://github.com/docker-library/redis.
  # https://redis.io/
  redis:
    image: redis:3.2-alpine
    container_name: redis
    restart: always
    command: redis-server --appendonly yes
    sysctls:
      - net.core.somaxconn=1024
    volumes:
      - /mnt/docker/redis/db/:/data

# other containers that can be exposed by the same proxy
  <somecontainer>:
    image: <some image>
    container_name: <some name>
    restart: always
    environment:
      VIRTUAL_HOST: <some.FQDN>
      HTTPS_METHOD: redirect
    ...

Notable differences from @sknight80 script:

It looks like let's encrypt TOS changed lately and the container did not keep up, so ACME_TOS_HASH is added
I used a single cert for all domains in the config, could not get it working otherwise, wrong cert was picked up
Consequently, I did not need any of the CERT_NAME because single cert
Since I wanted everything in a single docker-compose.yml I could dispense with network configuration
VIRTUAL_PORT seemed to be on a wrong port for postfixadmin, but since just a single port is exposed it was not needed. For rspamd on the opposite it was added since the container exposes multiple ports.

So as I said before, no one size fits all, but that's what worked for me.

That's the way I used to do it (with jwilder build and Jrsc's lestencrypt), but it seems Jcrs does not maintain he's container anymore.
And since traefik was developped FOR containers and is efficient and simple, I migrated everything through it.

@zerpex good to know, I'll have a look. So are you saying that traefic can do everything that jwilder / Jrsc can?

@AndrewSav sorry copy paste error + markdown.

@AndrewSav nginx is a great proxy/web server ( I was a great fan :) ), but traefik was design specifically for containers and manages LE certificates generation and renewal nativelly.
So you need only one container to get proxy + certs management.
One more argument is that traefik is maintained by it's developpers. I appreciate Jwilder and Jrcs for their contribution to the community, but what if they stop maintening their containers (as Jrcs seems to !) ?
From my tests, I can tell that traefik is more efficient and optimized.
nginx and Apache are still good, but more as webservers from inside containers :)

Hello,
Here is my setup with traefik on one docker-compose.yml file :
From my previous post, I added rainloop and access to rspamd webui.
This setup is better using @hardware 's architecture with domains and sub-domains.
version: '2.1'

services:
   traefik:
    restart: unless-stopped
    image: traefik
    container_name: proxy_traefik
    hostname: traefik
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik.toml:/traefik.toml
      - ./files/acme:/etc/traefik/acme
    networks:
       - traefik_proxy

   mailserver:
    image: hardware/mailserver:1.1-stable
    container_name: mail-server
    hostname: mail
    domainname: domain.tld
    labels:
    - traefik.enable=true
    - traefik.frontend.rule=Host:spam.domain.tld
    - traefik.backend.port=11334
    - traefik.docker.network=traefik_proxy
    depends_on:
      - mariadb
      - redis
    ports:
     - "25:25"     # SMTP                - Required
     - "143:143"   # IMAP       STARTTLS - For webmails/desktop clients
     - "465:465"   # SMTPS      SSL/TLS  - Enabled for compatibility reason, otherwise disabled
     - "587:587"   # Submission STARTTLS - For webmails/desktop clients
     - "993:993"   # IMAPS      SSL/TLS  - For webmails/desktop clients
     - "4190:4190" # SIEVE      STARTTLS - Optional, but recommended for mail filtering
    volumes:
     - ./files/mail/data:/var/mail
     - ./files/mail/opendkim:/etc/opendkim/keys
    environment:
     - VMAILUID=1069
     - VMAILGID=1069
     - OPENDKIM_KEY_LENGTH=4096
     - DBPASS=xxxxxxx
     - RSPAMD_PASSWORD=xxxxxxx
    networks:
      - mail_back
      - traefik_proxy

   rainloop:
    image: hardware/rainloop
    container_name: rainloop
    # restart: always
    labels:
    - traefik.enable=true
    - traefik.frontend.rule=Host:webmail.domain.tld
    - traefik.backend.port=80
    - traefik.docker.network=traefik_proxy
    volumes:
      - ./files/rainloop:/rainloop/data
    depends_on:
      - mailserver
      - mariadb
    networks:
      - mail_back
      - traefik_proxy

# Databases
   mariadb:
    image: mariadb:latest
    container_name: mail-server_db
    environment:
     - MYSQL_ROOT_PASSWORD=xxxxxxx
     - MYSQL_DATABASE=postfix
     - MYSQL_USER=postfix
     - MYSQL_PASSWORD=xxxxxxx
    volumes:
     - ./files/mail/db:/var/lib/mysql
    networks:
      - mail_back

# Control panel
   postfixadmin:
    image: hardware/postfixadmin
    container_name: mail-postfixadm
    domainname: gardelle.net
    hostname: mail
    labels:
    - traefik.enable=true
    - traefik.frontend.rule=Host:mailadm.domain.tld
    - traefik.backend.port=8888
    - traefik.docker.network=traefik_proxy
    depends_on:
      - mailserver
      - mariadb
    environment:
     - UID=1069
     - GID=1069
     - DBPASS=xxxxxxx
    networks:
      - mail_back
      - traefik_proxy

# Cache
   redis:
    image: redis:4.0-alpine
    container_name: mail-cache_redis
    # restart: always
    command: redis-server --appendonly yes
    volumes:
      - ./files/redis/db:/data
    networks:
      - mail_back

networks:
  traefik_proxy:
    external:
      name: traefik_proxy
  mail_back:

And the associated traefik.toml file :
#debug = true

defaultEntryPoints = ["http", "https"]		# defaultEntryPoints must be at the top because it should not be in any table below

[web]
  # Port for the status page
  address = ":8080"
  #[web.auth.basic]
  #  users = ["traefik:generate-with-htpasswd"]

# Entrypoints, http and https
[entryPoints]
  [entryPoints.http]						# http should be redirected to https
    address = ":80"
    [entryPoints.http.redirect]
      entryPoint = "https"					# https is the default
  [entryPoints.https]
    address = ":443"
    [entryPoints.https.tls]

# Enable ACME (Let's Encrypt): automatic SSL
[acme]
  # caServer = "https://acme-staging.api.letsencrypt.org/directory"
  email =<EMAIL_ADDRESS>  storage = "/etc/traefik/acme/acme.json" 	# or "traefik/acme/account" if using KV store
  entryPoint = "https"
  onDemand = false
  OnHostRule = true

[[acme.domains]]
  main = "domain.tld"
  sans = ["mail.domain.tld"]

[docker]
  endpoint = "unix:///var/run/docker.sock"
  domain = "domain.tld"
  watch = true
  exposedbydefault = false					# tells Træfik to only include containers with the label traefik.enable=true

Actually, this should work only with replacing "domain.tld" by yours in both files (and the RSPAMD_PASSWORD, of course :D).
Regards,
z.

@zerpex I will try your yaml file.  Thanks for sharing it!

@sknight80 @hardware
There seem to be an issue with certificates:
mailserver:
    volumes:
      - /usr/local/etc/proxy/certs/:/etc/letsencrypt

Here https://github.com/hardware/mailserver/blob/master/rootfs/usr/local/bin/run.sh#L89
/live/ is added at the end of certs path, but there is no /live/ in jwilder / Jrsc  setup, so the mailserver does not get the right cert.
How do I fix it?

Hm...
changed to  - /usr/local/etc/proxy/certs/:/etc/letsencrypt/live
seems to work. Is it alright?

@zerpex in your setup above if you need to add more web apps beside mailserver to traefic, where do you add them to?

seems to work. Is it alright?

Yes

I wonder why most of the clients do not care...

Because most of the mail clients are bad and do not want to bother users with messages they do not understand. You're fine with a self-signed certificate if you have a way to check the fingerprint. Thunderbird and K-9 Mail warn the user when the fingerprint change.

@AndrewSav 2 things have to be changed on your docker-compose files :

Add the labels (see ranloop for instance) and adapt them (URL and port)
Add your containers to be proxy_traefik network.

@zerpex
I tried your yml and toml by replacing my domain and DB password. I am seeing error while running docker-compose for mail server  as below. Can you help me how it can be fixed?
Status: Downloaded newer image for hardware/postfixadmin:latest
Creating mail-cache_redis ...
Creating proxy_traefik ...
Creating mail-server_db ...
Creating mail-cache_redis
Creating proxy_traefik
Creating mail-server_db ... done
Creating mail-server ...
ERROR: for mail-server  dictionary update sequence element #0 has length 19; 2 is required
ERROR: for mailserver  dictionary update sequence element #0 has length 19; 2 is required
Traceback (most recent call last):
File "bin/docker-compose", line 6, in
File "compose/cli/main.py", line 68, in main
File "compose/cli/main.py", line 121, in perform_command
File "compose/cli/main.py", line 952, in up
File "compose/project.py", line 455, in up
File "compose/parallel.py", line 70, in parallel_execute
ValueError: dictionary update sequence element #0 has length 19; 2 is required
Failed to execute script docker-compose

Also tried below config by @AndrewSav  but not able to setup the db
Error: Can't connect to database
Please edit the $CONF['database_*'] parameters in config.local.php.
DEBUG INFORMATION:
Connect: php_network_getaddresses: getaddrinfo failed: Name does not resolve
version: '2.1'
services:
nginx-proxy:
image: jwilder/nginx-proxy:alpine
container_name: nginx-proxy
restart: always
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- /mnt/docker/certs:/etc/nginx/certs:ro
- /mnt/docker/nginx/vhost.d:/etc/nginx/vhost.d
- /mnt/docker/nginx/html:/usr/share/nginx/html
depends_on:
- postfixadmin
- rainloop
- mailserver
letsencrypt-nginx-proxy:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: letsencrypt-nginx-proxy
restart: always
environment:
# Unfortunately every time TOS changes this breaks. There is a PR to fix it here
# https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion/issues/90
# Eventually it will make it to the official container
ACME_TOS_HASH: cc88d8d9517f490191401e7b54e9ffd12a2b9082ec7a1d4cec6101f9f1647e7b
volumes_from:
- nginx-proxy
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /mnt/docker/certs:/etc/nginx/certs
depends_on:
- nginx-proxy
mailserver:
image: hardware/mailserver:1.1-stable
container_name: mailserver
domainname: mydomian.com                  # Mail server A/MX/FQDN & reverse PTR = mail.domain.tld.
hostname: mail
restart: always
# extra_hosts:                          - Required for external database (on other server or for local database on host)
#  - "mariadb:xx.xx.xx.xx"              - Replace with IP address of MariaDB server
ports:
- "25:25"       # SMTP                - Required
# - "110:110"     # POP3       STARTTLS - Optional - For webmails/desktop clients
- "143:143"     # IMAP       STARTTLS - Optional - For webmails/desktop clients
# - "465:465"     # SMTPS      SSL/TLS  - Optional - Enabled for compatibility reason, otherwise disabled
- "587:587"     # Submission STARTTLS - Optional - For webmails/desktop clients
- "993:993"     # IMAPS      SSL/TLS  - Optional - For webmails/desktop clients
# - "995:995"     # POP3S      SSL/TLS  - Optional - For webmails/desktop clients
- "4190:4190"   # SIEVE      STARTTLS - Optional - Recommended for mail filtering
# - "11334:11334" # HTTP                - Optional - Rspamd WebUI
environment:
# MariaDB database password (required)
- DBPASS=db2345XXNy!
# Rspamd WebUI password (required)
- RSPAMD_PASSWORD=db2345XXNy!
# Add additional domains separated by commas (needed for dkim keys etc.)
- ADD_DOMAINS=mydomian.com
- VIRTUAL_HOST=spam.mydomian.com,mail.mydomian.com
- VIRTUAL_PORT=11334
- HTTPS_METHOD=redirect
# - ENABLE_POP3=true                    # Enable POP3 protocol
# - ENABLE_FETCHMAIL=true               # Enable fetchmail forwarding
# - DISABLE_CLAMAV=true                 # Disable virus scanning
# - DISABLE_SIGNING=true                # Disable DKIM/ARC signing
# - DISABLE_GREYLISTING=true            # Disable greylisting policy
# - DISABLE_RATELIMITING=true           # Disable ratelimiting policy
#
# Full list : https://github.com/hardware/mailserver#environment-variables
#
volumes:
- /mnt/docker/mail:/var/mail
- /mnt/docker/certs:/etc/letsencrypt/live
depends_on:
- mariadb
- redis
Administration interface
https://github.com/hardware/postfixadmin
http://postfixadmin.sourceforge.net/
Configuration : https://github.com/hardware/mailserver/wiki/Postfixadmin-initial-configuration
postfixadmin:
image: hardware/postfixadmin
container_name: postfixadmin
domainname: mydomian.com
hostname: postfixadmin
restart: always
environment:
DBPASS: db2345XXNy!
DBHOST: mariadb
VIRTUAL_HOST: postfixadmin.mydomian.com
HTTPS_METHOD: redirect
depends_on:
- mailserver
- mariadb
Webmail (Optional)
https://github.com/hardware/rainloop
https://www.rainloop.net/
Configuration : https://github.com/hardware/mailserver/wiki/Rainloop-initial-configuration
rainloop:
image: hardware/rainloop
container_name: rainloop
restart: always
environment:
VIRTUAL_HOST: webmail.mydomian.com
LETSENCRYPT_EMAIL<EMAIL_ADDRESS>LETSENCRYPT_HOST: "mail.mydomian.com,webmail.mydomian.com,postfixadmin.mydomian.com,spam.mydomian.com"
HTTPS_METHOD: redirect
volumes:
- /mnt/docker/rainloop:/rainloop/data
depends_on:
- mailserver
- mariadb
Database
https://github.com/docker-library/mariadb
https://mariadb.org/
mariadb:
image: mariadb:10.1
container_name: mariadb
restart: always
# Info : These variables are ignored when the volume already exists (databases created before).
environment:
- MYSQL_ROOT_PASSWORD=db2345XXNy!
- MYSQL_DATABASE=postfix
- MYSQL_USER=postfix11
- MYSQL_PASSWORD=db2345XXNy!
volumes:
- /mnt/docker/mysql/db:/var/lib/mysql
Cache Database
https://github.com/docker-library/redis.
https://redis.io/
redis:
image: redis:3.2-alpine
container_name: redis
restart: always
command: redis-server --appendonly yes
sysctls:
- net.core.somaxconn=1024
volumes:
- /mnt/docker/redis/db/:/data

@cloudtestsoftware where you able to get this working following the original instructions of README.MD in the repository? If not, I'd suggest you to start there. Here we mostly building up on what is already done over there, and it would be good to confirm that you got the basics right.

@cloudtestsoftware  got the same error and fixed it with updating docker-compose
@zerpex I tried to use your yml file but I would think that imaps and smtp tls would need the let's encrypt certificate. How do they get passed from traefik to mailserver or is it useless ?

@cloudtestsoftware  got the same error and fixed it with updating docker-compose
@zerpex I tried to use your yml file but I would think that imaps and smtp tls would need the let's encrypt certificate. How do they get passed from traefik to mailserver or is it useless ?

All wonderfall's docker images are no longer maintained. I will update hardware/mailserver with Træfik reverse proxy instead of boring-nginx before the end of the year. There should be no breaking changes with the current installations.

I'm working on integrating use of traefik in my setup. This mostly seems like a confiuration/documentation change, in the base image (differences in the docker-compose.yml mostly).

Running the other way:

lsPHP [5|7]* API & HTTP(s) web server postadmin/rainloop/etc without php-fpm

https://hub.docker.com/r/litespeedtech/openlitespeed/

HAProxy 1.8+ (TLS 1.3) https://hub.docker.com/_/haproxy/

Hi there,
Here is a full docker-compose.yml file with rainloop.
Just tested it on fresh installed server and it's working.
How to use :
1- Configure this entries on your DNS :

mail.domain.tld => webmail
mailadm.domain.tld => postfixadmin
smtp.domain.tld => smtp :)
imap.domain.tld => imap :)

2- Create docker's proxy network :
docker network create traefik_proxy
3- Copy this in the file ./files/traefik.toml and replace<EMAIL_ADDRESS>with your mail :
#debug = true

defaultEntryPoints = ["http", "https"]	# defaultEntryPoints must be at the top because it should not be in any table below

[web]
  # Port for the status page
  address = ":8080"
  #[web.auth.basic]
  #  users = ["traefik:generate-with-htpasswd"]

# Entrypoints, http and https
[entryPoints]
  [entryPoints.http]					# http should be redirected to https
    address = ":80"
    [entryPoints.http.redirect]
      entryPoint = "https"				# https is the default
  [entryPoints.https]
    address = ":443"
    [entryPoints.https.tls]

# Enable ACME (Let's Encrypt): automatic SSL
[acme]
  # caServer = "https://acme-staging.api.letsencrypt.org/directory"
  email =<EMAIL_ADDRESS>  storage = "/etc/traefik/acme/acme.json" 				# or "traefik/acme/account" if using KV store
  entryPoint = "https"
  onDemand = false
  OnHostRule = true

[docker]
  endpoint = "unix:///var/run/docker.sock"
  domain = "toolz.pm"
  watch = true
  exposedbydefault = false				# tells Træfik to only include containers with the label traefik.enable=true

4- Copy this in the file docker-compose.yml and replace domain.tld with your domain :
version: '2.1'

services:
   traefik:
    restart: unless-stopped
    image: traefik
    container_name: proxy_traefik
    hostname: traefik
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./files/traefik.toml:/traefik.toml
      - ./files/acme:/etc/traefik/acme
    networks:
       - traefik_proxy

   mailserver:
    image: hardware/mailserver:1.1-stable
    restart: always
    container_name: mail-server
    hostname: mail
    domainname: gardelle.net
    labels:
      - traefik.enable=true
      - traefik.smtp.backend.port=587
      - traefik.smtp.frontend.rule=Host:smtp.domail.tld
      - traefik.imap.backend.port=993
      - traefik.imap.frontend.rule=Host:imap.domail.tld
      - traefik.sieve.backend.port=4190
      - traefik.sieve.frontend.rule=Host:sieve.domail.tld
      - traefik.docker.network=traefik_proxy
    depends_on:
      - mariadb
      - redis
    ports:
     - "25:25"     # SMTP                - Required
     - "587:587"   # Submission STARTTLS - For webmails/desktop clients
     - "993:993"   # IMAPS      SSL/TLS  - For webmails/desktop clients
     - "4190:4190" # SIEVE      STARTTLS - Optional, but recommended for mail filtering
    volumes:
     - ./files/mail/data:/var/mail
     - ./files/mail/opendkim:/etc/opendkim/keys
    environment:
     - VMAILUID=1069
     - VMAILGID=1069
     - OPENDKIM_KEY_LENGTH=4096
     - DBPASS=xxxxxxx
     - RSPAMD_PASSWORD=xxxxxxx
     - ADD_DOMAINS=domail.tld
    networks:
      - mail_back

# Databases
   mariadb:
    image: mariadb:latest
    restart: always
    container_name: mail-server_db
    environment:
     - MYSQL_ROOT_PASSWORD=xxxxxxx
     - MYSQL_DATABASE=postfix
     - MYSQL_USER=postfix
     - MYSQL_PASSWORD=xxxxxxx
    volumes:
     - ./files/mail/db:/var/lib/mysql
    networks:
      - mail_back

# Control panel
   postfixadmin:
    image: hardware/postfixadmin
    restart: always
    container_name: mail-postfixadm
    domainname: gardelle.net
    hostname: mail
    labels:
    - traefik.enable=true
    - traefik.frontend.rule=Host:mailadm.domail.tld
    - traefik.backend.port=8888
    - traefik.docker.network=traefik_proxy
    depends_on:
      - mailserver
      - mariadb
    links:
      - mariadb
    environment:
     - UID=1069
     - GID=1069
     - DBPASS=xxxxxxx
    networks:
      - mail_back
      - traefik_proxy

# Webmail
   rainloop:
    image: hardware/rainloop
    container_name: mail-webmail_rainloop
    restart: always
    labels:
      - traefik.enable=true
      - traefik.backend.port=8888
      - traefik.frontend.rule=Host:mail.domail.tld
      - traefik.docker.network=traefik_proxy
    volumes:
      - ./files/mail/rainloop:/rainloop/data
    depends_on:
      - mailserver
      - mariadb
    networks:
      - traefik_proxy

# Cache
   redis:
    image: redis:4.0-alpine
    restart: always
    container_name: mail-cache_redis
    # restart: always
    command: redis-server --appendonly yes
    volumes:
      - ./files/redis/db:/data
    networks:
      - mail_back

volumes:
  files:
    driver: local

networks:
  traefik_proxy:
    external:
      name: traefik_proxy
  mail_back:

5- execute docker-compose up -d
6- Set postfixadmin & rainloop configs
7- Start spamming the world :)
Depending on DNS update time, you can have a full https mailserver in less than 10mn.
Regards,
z.

@zerpex, do you happen to know if the acme.domains section is required at all if you already specified OnHostRule = true? By reading the doco I got the impression that with latter domains come from the frontend host rules automagically...


New version: DynamicQuantities v1.2.0

Registering package: DynamicQuantities
Repository: https://github.com/SymbolicML/DynamicQuantities.jl
Created by: @MilesCranmer
Version: v1.2.0
Commit: 5ad02cc0c4fa1e002d20448e1f80fc87d1fae1d2
Reviewed by: @MilesCranmer
Reference: https://github.com/SymbolicML/DynamicQuantities.jl/commit/5ad02cc0c4fa1e002d20448e1f80fc87d1fae1d2#commitcomment-148626849
Description: Efficient and type-stable physical quantities in Julia

block


[Enhancement] Remove connection_id and current_catalog from syntax file

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #17604
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto backported to target branch

[x] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

[FE PR Coverage Check]
:heart_eyes: pass : 3 / 3 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
2
2
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
1
1
100.00%
[]

@Mergifyio backport branch-3.0


“Poșaga de Jos", "Corțești", " Incești", "Lunca", “Orăști”, "Poșaga de Sus", "Săgagea" are villages in "Poșaga" common, "ALBA" county

Description : When user is typing "Poșaga de Jos"/"Corțești"/"Incești"/"Lunca"/“Orăști”/ "Poșaga de Sus"/ "Săgagea",  in the search bar, the locations should appear in the dropdown menu correctly spelled and with localities information (as in Law nr. 2 din 16 februarie 1968)
Preconditions : website is up and runing

Steps 1:  Click on the search bar and write "Poșaga de Jos"

Expected result: In the drop down menu you find ""Poșaga de Jos, ALBA( Poșaga)"

Steps 2 : Press and hold the Back key to delete the text.

Expected result: The search bar is empty and the cursor active

Steps 3:  Click on the search bar and write "Corțești"

Expected result: In the drop down menu you find ""Corțești, ALBA( Poșaga)"

Steps 4 : Press and hold the Back key to delete the text.

Expected result: The search bar is empty and the cursor active

Steps 7:  Click on the search bar and write "Lunca"

Expected result: In the drop down menu you find ""Lunca, ALBA( Poșaga)"

Steps 8 : Press and hold the Back key to delete the text.

Expected result: The search bar is empty and the cursor active

Steps 9:  Click on the search bar and write "Orăști"

Expected result: In the drop down menu you find ""Orăști, ALBA( Poșaga)"

Steps 11:  Click on the search bar and write "Poșaga de Sus""

Expected result: In the drop down menu you find "Poșaga de Sus", ALBA( Poșaga)"

Steps 12: Press and hold the Back key to delete the text.

Expected result: The search bar is empty and the cursor active

Steps 13:  Click on the search bar and write "Săgagea"

Expected result: In the drop down menu you find "Săgagea", ALBA( Poșaga)"


Make the default placeholder views center, but take into account the keyboard frame.

Also upped to iOS 11+, etc.

Needs a bump in CocoaPods to fix the swift_version DSL issue. Also probably a bump in the Xcode image used in .travis.yaml.


fix: add timeout on transaction submission

This adds a 5 minute timeout on all transaction submissions. Since I am not sure of subxt/jsonrpsee deals well with cancelled subscriptions, and because a timeout might indicate an invalid state in those dependencies, I made all timeouts fire the shutdown signal so a fresh connection will get established upon restart. Without this explicit shutdown signal, non-essential calls (such as bitcoin block relay) would not cause a restart.
The reason for the long timeout time is so that it's less likely the stuck transaction will get included into the parachain after we timeout.

Caused by upstream issues
jsonrpsee: https://github.com/paritytech/jsonrpsee/issues/714
substrate: https://github.com/paritytech/substrate/issues/10910


删除FAQ和FAQ_CN中的多余符号

Summary
删除FAQ和FAQ_CN中的多余符号

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Main rules notes

As discussed via video, a place to list items we need to look at in the core rules.
Orders and conditions > Immobilised condition
This is currently listed as a detachment condition that prevents orders and movement.
Issue: It's a unit-level condition rather than a detachment-level condition, and the original GW rules for Immobilised and War Engine orders lead to weird interactions like using March orders, getting Immobilised in the first part of the Shooting phase, and then being able to fire at full effect in the second part because you're no longer applying the March order!
Suggestion: Rewrite as a unit-level rule that prevents movement and affects being shot at, but doesn't affect orders. Presumably move to the War Engine rules, maybe as a partial so that it can be used in the movement, critical damage, and shooting at war engines page.

It'd definitely good to start collating these things. However, we really ought to have a separate issue for each thing to keep things easier to manage (imagine if we have five or ten items in this thread and then we need to discuss some of them in depth...that will get unwieldy in this thread, fast). As per GitHub documentation:

Breaking a large issue into smaller issues makes the work more manageable and enables team members to work in parallel. It also leads to smaller pull requests, which are easier to review.

To track how smaller issues fit into the larger goal, use task lists, milestones, or labels. For more information, see "About task lists", "About milestones", and "Managing labels."

I'm not 100% sure which method is best in this case — but it seems like this is sort of a 'parent issue' under which we'd like to collate/discuss/track sub-issues, and it sounds like the task lists approach would facilitate that. (See the link above.) So I suggest we try that.
You don't have to create the issue first — you can create your list in your post at the head of this issue, and then convert list items into related issues as required: Creating an issue from a task list item

Suggestion: Rewrite as a unit-level rule that prevents movement and affects being shot at, but doesn't affect orders. Presumably move to the War Engine rules, maybe as a partial so that it can be used in the movement, critical damage, and shooting at war engines page.

I agree in principle, but an idea for a slightly different take: What if we just tweak the existing 'Conditions' section a bit? I mean, move the substance of the first paragraph:

When a detachment becomes subject to a condition then any orders that were in effect are no longer valid — remove the order marker (if any) and place the appropriate condition marker.

...into the explanation for the Broken condition.
And then make it plain that the Immobilised condition only applies to individual units. The "remove the order marker" part will no longer apply here because we'll have moved that instruction so that it is specific to the Broken condition. (Of course, we can make it plain that existing orders are still in force and that Immobilised is additional for the affect unit(s), to be crystal clear.)

As per the video discussion, this was planned as a place to list of things we need to check in the future. So a task list sounds good.
On the specific issue, I have a counterpoint. Immobilised is not a detachment level rule, it doesn't affect* orders, and it's specific to War Engines. Why would it be in the Main rules section instead of the War Engine section?
*Other than making it pointless to use movement-based orders.

On the specific issue, I have a counterpoint. Immobilised is not a detachment level rule, it doesn't affect* orders, and it's specific to War Engines. Why would it be in the Main rules section instead of the War Engine section?
*Other than making it pointless to use movement-based orders.

Note that this topic is in summary form with a consistent structure for each item — I thought it was nice to summarise the effects that relate to all six faces of the order die, all on the same page. Where there is more to say, detailed explanations exist in pertinent topics elsewhere (as in 'Broken detachments'). The bit that makes it odd is that for some orders/conditions there really isn't much else to say outside of the summary...
I see what you mean about Immobilised though. So perhaps so other arrangement would be better. My idea above was just an idea. Go with your idea if you think it's better than having it all on one page. Or, I have a feeling that a third way might emerge when we play around with it.

Also, an aside in relation to conditions: I think we could actually view Flak and Preparatory Bombardment as unit-specific conditions, rather than 'orders' in their own right, somewhat like the Immobilised condition. I mean, you declare one set of Special orders which can combine both Flak and Preparatory Bombardment, to apply to all appropriate individual units in the detachment — and we treat the rest of the detachment as though they are on Overwatch orders. One order in force with a single marker, with a 'treat as Overwatch' order effect, and two conditions in force for specific individual units as appropriate while the Special orders are in effect.
I'm not sure whether or not this might actually be a useful distinction though — it's just a half-baked thought at the moment.


Can I use this qr code generator to store images?

You just need a method to save UIImage instances on disk (see https://stackoverflow.com/a/44911381/5082444). Then you can simply pass qrCode.image to that method, e.g.:
let qrCode = QRCode(URL(string: "https://example.com")!)!
saveImage(image: qrCode.image!)


New version: JuliaPetra v0.2.0

Registering package: JuliaPetra
Repository: https://github.com/Collegeville/JuliaPetra.jl
Created by: @katmendy
Version: v0.2.0
Commit: 4c61dbba206a5ce5e6db74ec0cb21368406f03e5
Git reference: master
Release notes:

Bug fixes, performance improvements, API clean up, and more.

Can you also add
[compat]
julia = "1"

to your project file and retrigger the bot?

@JuliaRegistrator register

You either have to comment that on the relevant new commit, or go to the web interface again, and make a new release from there in order to update this PR.

Whoops! Thanks for letting me know.

Sorry to bother you again, but it looks like you added it to docs/Project.toml instead of Project.toml.

No problem! Thank you for your help.


CLI can select target arch

mopro-ffi:app_config::ios::build and mopro-ffi:app_config::android::build now require target_arch as an input instead of being hardcoded value inside the function.
The build scripts in cli/src/template/init/bin/* and test-e2e/src/bin/* read the target_arch value from the environment variable. If no value is provided, all available architectures are selected by default.
Did not add mopro-ffi crate in the cli for only reading *_ARCHS variables. It's too heavy to compile for simple.

This PR will be replace another new PR with https://github.com/zkmopro/mopro/tree/selectable_target_arch

Replaced #271


Polyfills not included

#442 stated that VA doesn't work for IE11. I tested and it reproduced.
I included polyfills to hotfix the issue.
import 'es6-promise/auto'
import 'babel-polyfill'

But in fact vue-cli3 should include polyfills by itself! See docs.
We have to investigate why the inclusion doesn't happen.
You can check by simply opening main page in IE11. Wihout polyfills IE should complain about Object.assign and the page should stay blank.

We can get some hints from this article: https://sgom.es/posts/2019-03-06-supporting-old-browsers-without-hurting-everyone/

I checked PR and it's all nice. We need to resolve conflicts, then merge?

Fixed in issue #684


Split up run()

What?
Refactor of run() no functionality changes.
Also solves SC-586
Why?
SC-557, SC-562 and others are difficult to do since _runtask() and run() are huge function in which changes could easy break something important.
How?
We split up run() to make editing and testing the steps of the flow easier.
Test
So far just the existing test cases.
I could add further ones if necessary.
Successful run on remote server

@troibe looks good, just need to resolve the conflicts

@troibe looks good, just need to resolve the conflicts

Done now! Merging this stuff is horrible though.

I think the refactor will help deal with the merging, now that things are broken up, we can move it out of the core when we have time

New run on remote server


Close connection to unavailable orderers

Currently, if an orderer goes down, the stale connection to it still remains cached in the gateway registry.  This commit implements similar logic as used by connections to external endorsing peers whereby a stale connection is closed and flushed from the cache allowing future invocations to attempt to reconnect.
Signed-off-by: andrew-coleman<EMAIL_ADDRESS>

I don't think this is a good idea. Why do we want to do book-keeping of the connections ourselves?
gRPC has a mode where it reconnects to the remote node on its own when it is unreachable, via exponential backoff.
You can activate it by using the AsyncConnect flag in the ClientConfig.
I think the right approach would be to let gRPC do its probing for us, and we just probe its connectivity state similarly to what's done in the orderer cluster package.
Suppose an orderer is down for a day. Every time a client tries to do a broadcast  - are you going to try and use this orderer and connect to it synchronously? This would slow down the client, no?
Besides, looking at the current code (not the code you have in the PR) don't we have a problem that if an orderer is down and the gateway is contacted by a lot of clients concurrently (suppose, by 20 clients), then you might connect to the orderer 20 times and then close 19 connections and use the first one?


add support for service plan

renamed compared to the old app_service_plan in newest terraform providers.

/label azure
/label "new resource"

Thanks for the issue!

@JochemBoersma these are live in v0.10.19


trainer.test() with given checkpoint logs last epoch instead of checkpoint epoch

Bug description
Testing from a given checkpoint leads to logging the epoch number of the last checkpoint instead of the checkpoint specified:
trainer = Trainer(..., max_epochs=10)
lightning_module = MyLightningModule(...)
datamodule = MyDatamodule()

trainer.fit(lightning_module , datamodule=datamodule)

trainer.test(lightning_module , datamodule=datamodule, ckpt_path="last")     # <-- ok: logs correct epoch and step
ckpt_path="/.../checkpoints/epoch=2-step=396.ckpt"
trainer.test(lightning_module , datamodule=datamodule, ckpt_path=ckpt_path)  # <-- incorrect: logs last epoch and step

The second test logs epoch 10 instead of epoch 2. Similarly, the step number of the second test is incorrect.
What version are you seeing the problem on?
v2.2.1

I guess this could be caused by the same as https://github.com/Lightning-AI/pytorch-lightning/issues/18060. The checkpoint callback is not the last callback called, and thus some loop counter are not updated. Have a look at the fields mentioned in https://github.com/Lightning-AI/pytorch-lightning/issues/18060#issuecomment-2080180970 and see if this explains the behavior you notice, it might also offer you a workaround.


Notifications badge text color

The badge text color should be colorGray400 (#fff)

Is this one closed? @mziemer21

@AlbertCarreras yes


Unit Testing Pipeline

We have several pipelines in our request.  I'm writing unit test for each pipeline.  Each pipeline writes to our requestBag that then passes to the next pipeline step.  I need to find a way to capture the requestBag as if it has passed to the next pipeline step.
Handle(RequesBag requestBag, CancellationToken, RequestHandlerDelegate<> next)
{
requestBag.SomeId = "1234"  ///the next step may need requestBag.SomeId
}
I need to see requestBagSomeId value in my Unit Test.

What do you have so far? Can you show some code from your tests?

        [TestMethod]
        public async Task Pipeline_TransformPayloadToXDocumentHandler_SuccessTransform_Test()
        {
            //arrange
            var autoMocker = new AutoMocker();
            //var mockStringToXmlTranslator = new Mock<IStringToXmlTranslator>();
            //mockStringToXmlTranslator.Setup(m => m.Transform(_oemLeadsRequest.Payload)).Returns(_xDocument).Verifiable();

            var mockRequestPayloadRepository = new Mock<IRequestPayloadRepository>();

            var mockPipelineBehaviourDelegate = new Mock<RequestHandlerDelegate<ILeadProcessorResponse>>();
            mockPipelineBehaviourDelegate.Setup(m => m()).ReturnsAsync(_leadProcessorResponse).Verifiable();

            //autoMocker.Use(mockStringToXmlTranslator);
            autoMocker.Use(mockPipelineBehaviourDelegate);
            var handler = autoMocker.CreateInstance<TransformPayloadToXDocumentHandler>();

            //act
            var response = await handler.Handle(OemLeadsRequestBag.Empty.WithRequest(_oemLeadsRequest)
                , default(CancellationToken), mockPipelineBehaviourDelegate.Object);

            //HOW CAN I GET ACCESS TO THE TO OemLeadsRequestBag.Empty.WithRequest(_oemLeadsRequest) FROM INSIDE THE PIPELINE
            //assert
            autoMocker.VerifyAll();
            Assert.IsNotNull(response);
            Assert.IsTrue(response.Leads.Count() == 1);
            Assert.IsTrue(response.Leads.Any(f=>f.LeadTrackingID == _leadTrackingID));
        }

    [PipelineBehaviorStep("OEM Payload Validation Handler")]
    public class TransformPayloadToXDocumentHandler : ITransformPayloadToXDocumentHandler
    {
        private readonly IStringToXmlTranslator _stringToXmlTranslator;
        private readonly IRequestPayloadRepository _requestPayloadRepository;
        public TransformPayloadToXDocumentHandler(IStringToXmlTranslator stringToXmlTranslator
            , IRequestPayloadRepository requestPayloadRepository)
        {
            _stringToXmlTranslator = stringToXmlTranslator;
            _requestPayloadRepository = requestPayloadRepository;
        }
        public async Task<ILeadProcessorResponse> Handle(OemLeadsRequestBag requestBag, CancellationToken cancellationToken, RequestHandlerDelegate<ILeadProcessorResponse> next)
        {
            try
            {
                requestBag.PayloadXDoc = _stringToXmlTranslator.Transform(requestBag.Request.Payload);
                requestBag.LeadID = _stringToXmlTranslator.Transform(requestBag.Request);
            }
            catch (Exception ex)
            {
                if (Enum.TryParse(requestBag.Request.Source, out LeadOemSource source))
                    await _requestPayloadRepository.SaveBadPayload(source, requestBag?.Request?.Payload).ConfigureAwait(false);
                throw new BadPayloadException("Bad Payload - Can't read XML.", ex);
            }
            return await next().ConfigureAwait(false);
        }
    }

Ideally I want to access requestBag.LeadID to determine if the value was assigned by the Translator

does this not work?
var requestBag = OemLeadsRequestBag.Empty.WithRequest(_oemLeadsRequest)
var response = await handler.Handle(requestBag, default(CancellationToken), mockPipelineBehaviourDelegate.Object);

Assert.That(requestBag.LeadID, Is.EqualTo(123))

does this not work?
var requestBag = OemLeadsRequestBag.Empty.WithRequest(_oemLeadsRequest)
var response = await handler.Handle(requestBag, default(CancellationToken), mockPipelineBehaviourDelegate.Object);

Assert.That(requestBag.LeadID, Is.EqualTo(123))

I will try.  I was off Friday and just now getting back to it.


Cross-validation using a list of k input sets and a corresponding list of k output sets

Hi I wonder if a single Scan execution could perform k-fold cross validation using a list of k input sets and a corresponding list of k output sets to obtain the average performance metric. Specifically, my classification problem uses time series data containing episodes recorded at different times and durations. Ideally I wish to pass those episodes as a list of training sets into the Scan function, and that each set would take turn to be treated as a held out fold while the other sets are being used together as a training set.

Do you mean that you would like to use k-fold approach for each permutation in the experiment?

@mikkokotila basically yes but with my own pre-splitted folds stored in a list.


Compiling ES on Udoo (Udoobuntu 1.0)  - Question/Problem

Hi,
more a question than an issue:
I'm trying to get ES running on my Udoo Quad (running Udoobuntu 1.0), pretty much trying to transfer the functionality of RetroPie on an Udoo.
I'm a software developer (primarily PHP, Java and databases, mostly web based stuff), but I have pretty much no experience with C++.
I tried to adapt the RasPi installation instructions (http://emulationstation.org/gettingstarted.html#install_rpi_standalone) as close as possible, here's what I did:
sudo apt-get install -y libudev-dev libasound2-dev libdbus-1-dev libboost-system-dev libboost-filesystem-dev libboost-date-time-dev libfreeimage-dev libfreetype6-dev libeigen3-dev libcurl4-openssl-dev libasound2-dev cmake libxext-dev libgl1-mesa-dev git
g++-4.7 is not in the official repositories, so I added ubuntu-toolchain to install it from there:
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install g++-4.7
Similar to Raspbian libsdl2-dev isn't in Udoobuntu's repositories either, so I went on compiling it:
wget http://libsdl.org/release/SDL2-2.0.3.tar.gz
tar xvfz SDL2-2.0.3.tar.gz
rm SDL2-2.0.3.tar.gz
pushd SDL2-2.0.3
./configure
make
sudo make install
popd
rm -rf SDL2-2.0.3
I substituted 2.0.1 with 2.0.3  -  2.0.1 wouldn't compile with an compiler error message that looked like a syntax error in the sources (whitespace after a negotiation exclamation mark), so I looked for the most recent version, which seems to compile just fine.
git clone https://github.com/Aloshi/EmulationStation
cd EmulationStation
git checkout unstable
cmake -D CMAKE_CXX_COMPILER=g++-4.7 .
Up to this point everything worked fine,
make
starts, but stops after about a minute:
/usr/include/eigen3/Eigen/src/Core/DenseBase.h:103:10: warning: enumeral mismatch in conditional expression: 'Eigen::internal::size_at_compile_time<4, 4>::<anonymous enum>' vs 'Eigen::internal::traits<Eigen::Matrix<float, 4, 4> >::<anonymous enum>' [-Wenum-compare]
'
Internal compiler error: Error reporting routines re-entered.
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.7/README.Bugs> for instructions.
Preprocessed source stored into /tmp/ccaHSkKn.out file, please attach this to your bugreport.
make[2]: *** [es-core/CMakeFiles/es-core.dir/src/GuiComponent.cpp.o] Error 1
make[1]: *** [es-core/CMakeFiles/es-core.dir/all] Error 2
make: *** [all] Error 2
As said: I have pretty much no clue of C++, so I don't really know what to make of that. It's a problem with the compiler itself, isn't it? Not some bug in ES's sources? Or could it have something to do with substituting SDL 2.0.1 with 2.0.3?

sudo apt-get install libgl1-mesa-glx --reinstall

I am having the same issue but "sudo apt-get install libgl1-mesa-glx" doesnt help nether "sudo apt-get install libgl1-mesa-glx --reinstall
@Aloshi ?


🐛 [Android] - Recording error (Error retrieving camcorder profile params)

What's happening?
When trying to record a video, onRecordingError throw an error. The error is:
{"cause": {"message": "Error retrieving camcorder profile params", "stacktrace": "java.lang.RuntimeException: Error retrieving camcorder profile params

I did try to reproduce this error in the example app, but wasn't able to, although my usage is pretty much the base minimum usage of the package.
Reproduceable Code
const cameraBack = useCameraDevice("back");
const cameraFront = useCameraDevice("front");
const [device, setDevice] = useState(cameraBack ?? cameraFront);

...

const toggleCamera = () => {
  if (device && device.position === "back" && cameraFront) {
    setDevice(cameraFront);
  } else if (device && device.position === "front" && cameraBack) {
    setDevice(cameraBack);
  }
};

const takeRecording = async () => {
  cameraRef?.current.startRecording({
    flash: flashMode,
    onRecordingFinished: (video) => {
      if (onCaptureVideo) onCaptureVideo(video);
      if (onClose) onClose();
    },
    onRecordingError: (err) => {
      logger("onRecordingError: ", err);
    }
  });
};

...
<Camera
  ref={cameraRef}
  className="flex-1"
  device={device}
  isActive={visible}
  photo
  video={true}
  audio={true}
/>

Relevant log output
onRecordingError:  {"cause": {"message": "Error retrieving camcorder profile params", "stacktrace": "java.lang.RuntimeException: Error retrieving camcorder profile params
	at android.media.CamcorderProfile.native_get_camcorder_profiles(Native Method)
	at android.media.CamcorderProfile.getAll(CamcorderProfile.java:605)
	at com.mrousavy.camera.extensions.RecordingSession_getRecommendedBitRateKt.getRecommendedBitRate(RecordingSession+getRecommendedBitRate.kt:32)
	at com.mrousavy.camera.core.RecordingSession.getBitRate(RecordingSession.kt:132)
	at com.mrousavy.camera.core.RecordingSession.<init>(RecordingSession.kt:37)
	at com.mrousavy.camera.core.CameraSession.startRecording(CameraSession.kt:532)
	at com.mrousavy.camera.CameraView_RecordVideoKt.startRecording(CameraView+RecordVideo.kt:44)
	at com.mrousavy.camera.CameraViewModule$startRecording$1.invokeSuspend(CameraViewModule.kt:91)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
"}, "code": "capture/unknown", "message": "An unknown error occurred while trying to start a video recording! Error retrieving camcorder profile params", "userInfo": null}

Camera Device
{
  "hardwareLevel": "limited",
  "minExposure": -300,
  "neutralZoom": 1,
  "supportsFocus": true,
  "supportsLowLightBoost": true,
  "hasTorch": true,
  "supportsRawCapture": false,
  "sensorOrientation": "landscape-right",
  "maxZoom": 4,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "isMultiCam": false,
  "id": "0",
  "position": "back",
  "minZoom": 1,
  "hasFlash": true,
  "maxExposure": 300,
  "name": "BACK (0)"
}

Device
Samsung Galaxy A03 Core
VisionCamera Version
3.6.12
Can you reproduce this issue in the VisionCamera Example app?
No, I cannot reproduce the issue in the Example app
Additional information

[ ] I am using Expo
[ ] I have enabled Frame Processors (react-native-worklets-core)
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

The error is happening inside .../react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/RecordingSession+getRecommendedBitRate.kt. In this part of the code:
fun RecordingSession.getRecommendedBitRate(fps: Int, codec: VideoCodec, hdr: Boolean): Int {
  try {
    val targetResolution = size
    val encoder = codec.toVideoEncoder()
    val bitDepth = if (hdr) 10 else 8
    val quality = findClosestCamcorderProfileQuality(targetResolution)
    Log.i("CamcorderProfile", "Closest matching CamcorderProfile: $quality")

    var recommendedProfile: RecommendedProfile? = null

    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
      val profiles = CamcorderProfile.getAll(cameraId, quality)
      if (profiles != null) {
        val best = profiles.videoProfiles.minBy { abs(it.width * it.height - targetResolution.width * targetResolution.height) }

        recommendedProfile = RecommendedProfile(
          best.bitrate,
          best.codec,
          if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) best.bitDepth else 8,
          best.frameRate
        )
      }
    }

    if (recommendedProfile == null) {
      val cameraIdInt = cameraId.toIntOrNull()
      val profile = if (cameraIdInt != null) {
        CamcorderProfile.get(cameraIdInt, quality)
      } else {
        CamcorderProfile.get(quality)
      }
      recommendedProfile = RecommendedProfile(
        profile.videoBitRate,
        profile.videoCodec,
        8,
        profile.videoFrameRate
      )
    }

    var bitRate = recommendedProfile.bitRate.toDouble()
    // the target bit-rate is for e.g. 30 FPS, but we use 60 FPS. up-scale it
    bitRate = bitRate / recommendedProfile.fps * fps
    // the target bit-rate might be in 8-bit SDR, but we record in 10-bit HDR. up-scale it
    bitRate = bitRate / recommendedProfile.bitDepth * bitDepth
    if (recommendedProfile.codec == VideoEncoder.H264 && encoder == VideoEncoder.HEVC) {
      // the target bit-rate is for H.264, but we use H.265, which is 20% smaller
      bitRate *= 0.8
    } else if (recommendedProfile.codec == VideoEncoder.HEVC && encoder == VideoEncoder.H264) {
      // the target bit-rate is for H.265, but we use H.264, which is 20% larger
      bitRate *= 1.2
    }
    return bitRate.toInt()
  } catch (e: Throwable) {
    // some Samsung phones throws an error here, returns 8 by default in these cases
    return 8;
  }
}

The try/catch was added by me as temporary workaround just to get it working, but it's not a good solution to run in production since when it catches an error it will always return a bitrate of 8, which will probably compromise the picture/video quality.
The problem seems to be anytime it tries to run either CamcorderProfile.getAll(...) or CamcorderProfile.get(...).

I believe this is being tracked and worked on in #2208 . Been following closely as i've been running into this also.

#2208

Oh true, I didn't knew that issue had been opened again with a different error than the original one posted.
I'll close this one and keep track on that one, thanks.

The error is happening inside .../react-native-vision-camera/android/src/main/java/com/mrousavy/camera/extensions/RecordingSession+getRecommendedBitRate.kt. In this part of the code:
fun RecordingSession.getRecommendedBitRate(fps: Int, codec: VideoCodec, hdr: Boolean): Int {
  try {
    val targetResolution = size
    val encoder = codec.toVideoEncoder()
    val bitDepth = if (hdr) 10 else 8
    val quality = findClosestCamcorderProfileQuality(targetResolution)
    Log.i("CamcorderProfile", "Closest matching CamcorderProfile: $quality")

    var recommendedProfile: RecommendedProfile? = null

    if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
      val profiles = CamcorderProfile.getAll(cameraId, quality)
      if (profiles != null) {
        val best = profiles.videoProfiles.minBy { abs(it.width * it.height - targetResolution.width * targetResolution.height) }

        recommendedProfile = RecommendedProfile(
          best.bitrate,
          best.codec,
          if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) best.bitDepth else 8,
          best.frameRate
        )
      }
    }

    if (recommendedProfile == null) {
      val cameraIdInt = cameraId.toIntOrNull()
      val profile = if (cameraIdInt != null) {
        CamcorderProfile.get(cameraIdInt, quality)
      } else {
        CamcorderProfile.get(quality)
      }
      recommendedProfile = RecommendedProfile(
        profile.videoBitRate,
        profile.videoCodec,
        8,
        profile.videoFrameRate
      )
    }

    var bitRate = recommendedProfile.bitRate.toDouble()
    // the target bit-rate is for e.g. 30 FPS, but we use 60 FPS. up-scale it
    bitRate = bitRate / recommendedProfile.fps * fps
    // the target bit-rate might be in 8-bit SDR, but we record in 10-bit HDR. up-scale it
    bitRate = bitRate / recommendedProfile.bitDepth * bitDepth
    if (recommendedProfile.codec == VideoEncoder.H264 && encoder == VideoEncoder.HEVC) {
      // the target bit-rate is for H.264, but we use H.265, which is 20% smaller
      bitRate *= 0.8
    } else if (recommendedProfile.codec == VideoEncoder.HEVC && encoder == VideoEncoder.H264) {
      // the target bit-rate is for H.265, but we use H.264, which is 20% larger
      bitRate *= 1.2
    }
    return bitRate.toInt()
  } catch (e: Throwable) {
    // some Samsung phones throws an error here, returns 8 by default in these cases
    return 8;
  }
}

The try/catch was added by me as temporary workaround just to get it working, but it's not a good solution to run in production since when it catches an error it will always return a bitrate of 8, which will probably compromise the picture/video quality.
The problem seems to be anytime it tries to run either CamcorderProfile.getAll(...) or CamcorderProfile.get(...).

worked

Use such settings with low resolution and frame rate, it will work:
const device = useCameraDevice(cameraType);
const format = useCameraFormat(device, [
{ videoResolution: { width: 1280, height: 720 } },
{ fps: 30 }
])

const format = useCameraFormat(device, [
{ videoResolution: { width: 1280, height: 720 } },
{ fps: 30 }
])

Crash on stopRecording

Use such settings with low resolution and frame rate, it will work:
const device = useCameraDevice(cameraType);
const format = useCameraFormat(device, [ { videoResolution: { width: 1280, height: 720 } }, { fps: 30 } ])

Not specifying the format led to this error 'Error retrieving camcorder profile params'. However, after applying the settings below (also specifying pixelFormat), it is working now.
const device = useCameraDevice('back')
const format = useCameraFormat(device, [
{ videoResolution: { width: 1280, height: 720 },
pixelFormat: 'native' },
{ fps: 30 }
])
If vid resolution or pixel format won't be compatible then you might get this error:
ERROR  {"cause": {"message": "prepare failed.", "stacktrace": "java.io.IOException: prepare failed.
at android.media.MediaRecorder._prepare(Native Method)
at android.media.MediaRecorder.prepare(MediaRecorder.java:1320)
at com.mrousavy.camera.core.RecordingSession.start(RecordingSession.kt:91)
at com.mrousavy.camera.core.CameraSession.startRecording(CameraSession.kt:533)
at com.mrousavy.camera.CameraView_RecordVideoKt.startRecording(CameraView+RecordVideo.kt:34)
at com.mrousavy.camera.CameraViewModule$startRecording$1.invokeSuspend(CameraViewModule.kt:91)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
"}, "code": "capture/unknown", "message": "An unknown error occurred while trying to start a video recording! prepare failed.", "userInfo": null}

Fixed in https://github.com/mrousavy/react-native-vision-camera/pull/2389! :)


Objects as keys

I have an use-case for objects as keys, where I'd use a "blockstate object" as a key.
A "blockstate object" would be an object representing one or more Minecraft blockstates.

I'm not sure I understand what you mean by this. What would the config file look like?

{key object} = {value object}?

My guess is that's a pretty large patch to parser, API, tests, and docs. Probably doesn't carry its weight. But I will leave the issue open in case people want to discuss or explore.

Or in my use-case,
"minecraft:stone" = {
  {
    variant = [ andesite, diorite ]
  } = {
    sound = "minecraft:note.pling"
    volume = 3.0
  }
  {
    variant = [ granite ]
  } = {
    sound = "minecraft.note.harp"
    volume = 3.0
  }
}

It's much less verbose/much more terse than having {blockname = "minecraft:stone", variants = [{variant = {}, instrument = {}}]}

Oh and also note there will be less indentation, and thus things won't go past the screen width as often.


integrate more translators

I'm opening this issue for new comers who want to contribute to the project.
The goal is to integrate more translators in this tool. For example linguee, pons or deepl.
If anyone is interested, feel free to write me or leave a comment

I would like to work on this issue. Can you brief me on what to do I will get it done? @nidhaloff

I am also interested on the issue .Can you breif me on what to do  and I will get it done ?? @nidhaloff

Basically as stated in the description, there are other translators that can be integrated like linguee, pons and deepl. If you know python, you can take a look at this repo https://github.com/nidhaloff/deep-translator

DeepL translator requires an account to use their API. I want to begin looking over the code and work on integrating it, but how would I manage the permissions and account requirements?

@Sowatee Users should handle that. The package is just a high-level API to use multiple translators. Unfortunately, not all translators are available for public free use. Therefore, in some cases (like deepl), the user would need to go to the deepl website and get a private api_key.
Example of using deepl (this is just a suggestion):
deepl = DeepL("some_api_key", "en", "de").translate("anything")


fix typos and num of features

Fixed some typos in the README.md and updated the number count of how many features there are in the README.md
as well!

Thank you for your small contribution towards this repository!


Is there any plan for Python gRPC nameresolver interface?

Hi, i have used the python gRPC to build services, but there isn't an elegant desgin for service load balance. After reading grpc-load-balancing and grpc-nameresolver,
i got the architecture about this, but now only dns nameresolver is available. Meanwhile in Python's implemention, there isn't any interfaces about this.

This is part of our long-term project plans but it's not on any near-term road map.

Is there a walk around solution for python to do load balancing based on name resolving currently?

@nathanielmanistaatgoogle any updates on when this will be picked up?
We are currently using custom resolvers in Java and Golang and would like to offer the same thing for our Python users.

@jhaals: @srini100 is more likely than I to have up-to-date information.

@jhaals, there is no immediate plan to add name resolver plugin in gRPC Python. There are some discussions going on around making the Core API public but that's in very early stage. Will update when there is more clarity on this.

@jhaals, what's your time frame to require this feature?

I can't speak in time frames but we have multiple Python only teams that expressed an interest in adopting gRPC. Service to service name resolution is entirely DNS(SRV) based for discovering ports and endpoints.

@srini100 Hi, what are the required steps for adding this API?
Does it require separate proposal steps or can I send pull request to grpc-core ~now and we will iterate in it?

As @srini100 mentioned above, we do have work going on now to make our current internal resolver API public, but that is part of a larger project and will probably take at least a couple of quarters.  However, I think we could provide a Python resolver API without having to wait for those changes.  We could do this in two steps:

Define a C-core surface API for resolver plugins.  This is different from our current internal API (which is actually C++); it would be implemented as a shim that plugs into our existing C++ API but delegates its functionality to a resolver written against the new C-core surface API.

Define a Python resolver API.  This would be implemented as part of the gRPC Python layer that sits on top of C-core; it would basically convert between the C-core API described in the previous step and the Python API.

These would probably best be done as two different projects.  I don't think we have any cycles to actually do either of them right now, but if someone wants to contribute, we can provide guidance and reviews from our end.  I can be the contact point for the first part.  @srini100 can find someone to help with the second part.
Please let me know if you have any questions about any of this.

@markdroth Thanks for the elaborate answer. I've linked the pull request for grpc-core part and would welcome any feedback.

This feature is still on-demand.

Again, this feature is still on-demand.

Any updates?

Want NameResolver in Python as well. This feature is still on-demand.

NameResolver in python feature is still on-demand.

Thanks for bring this up, we understand that this is still needed by our users, but there are some internal changes (in grpc core) needs to be made before the change can start at all (It's called EventEngine). Once the eventEngine changes merged, we'll still need extra hands to make this happen, if someone is willing to help, they'll need write a gRFC first and then implement it. We're more than happy to provide more information with video call :)

This is part of our long-term project plans but it's not on any near-term road map.

How long term are we talking?

How long term are we talking?

Given that the comment you quoted is from 2018, pretty long term.
On a more serious note, this is one of the more important long-standing gaps in the Python API as I see it. The issue is just technical complexity. This will require making many currently private Core APIs public. The Core maintainers have been hesitant to do that for the past several years because we're in the middle of a major overhaul of how Core works (Event Engine). Luckily, that effort is starting to wrap up, so the time might be right to jump on this soon.

Just thought I would check on the status of this since it's been over a year since the last update.  This is a pretty desirable feature for us.


Migration to new Concourse instance

@spikymonkey ready to merge?

@royclarkson oops, sorry I missed this, yes


Remove "2525" From All_ID_Mapping_X.Standard Entries

To make these comparisons a little more efficient (now that frame queries are no longer unique on this table) - I would like these strings changed in the Standard field to something more succint:
Old -> New
2525C -> C
2525Bc2 -> B2
https://github.com/Esri/joint-military-symbology-xml/blob/master/samples/legacy_support/All_ID_Mapping_Latest.csv
https://github.com/Esri/joint-military-symbology-xml/blob/master/samples/legacy_support/All_ID_Mapping_Original.csv

Fixed in the dev branch in #300
@csmoore , please verify and close as appropriate.

Verified in latest style submitted for testing


Update and rename New-SecOpsOverrideRule.md to New-ExoSecOpsOverrideR…

…ule.md
updating to new cmdlet

Full ramifications of the cmdlet replacement taken care of in PR #11572.
So, closing this PR.