Docker + Docker Compose

Docker + Docker Compose
The project deployment needs to be simplified with a docker and docker-compose structure. We need a pull request to address adding this as a first class support model.

I'm looking to use this as an "on-demand" platform for provisioning live video streams between students and teachers. It would be nice if we can get a terraform deployment together as well.
https://github.com/scorpion/lms/issues/50

I'm looking to use this as an "on-demand" platform for provisioning live video streams between students and teachers. It would be nice if we can get a terraform deployment together as well.
https://github.com/scorpion/lms/issues/50

I agree, a Docker Compose stack would greatly simplify the setup for the project.
If this was converted into a mono repo (Taking all 3 projects (Rust, Go & React) and putting them into one repository) would mean you could run a command as simple as the one below to get started:
git clone https://github.com/GRVYDEV/Project-Lightspeed.git && \
cd Project-Lightspeed && \
docker-compose up -d

The docker compose file can then just be placed in the project root and link to the Dockerfiles located in each projects directory. EG:
version: '3'

services:
    ingest:
        build:
            context: ./Lightspeed-ingest
            dockerfile: Dockerfile
        image: GRVYDEV/lightspeed_ingest:latest
        container_name: lightspeed_ingest
        ports:
            - 8084:8084
     webrtc:
        build:
            context: ./Lightspeed-webrtc
            dockerfile: Dockerfile
        image: GRVYDEV/lightspeed_webrtc:latest
        container_name: lightspeed_webrtc
        ports:
            - 8080:8080
     react:
        build:
            context: ./Lightspeed-react
            dockerfile: Dockerfile
        image: GRVYDEV/lightspeed_react:latest
        container_name: lightspeed_react
        ports:
            - 80:80

I agree, a Docker Compose stack would greatly simplify the setup for the project.
If this was converted into a mono repo (Taking all 3 projects (Rust, Go & React) and putting them into one repository) would mean you could run a command as simple as the one below to get started:
git clone https://github.com/GRVYDEV/Project-Lightspeed.git && \
cd Project-Lightspeed && \
docker-compose up -d

The docker compose file can then just be placed in the project root and link to the Dockerfiles located in each projects directory. EG:
version: '3'

services:
    ingest:
        build:
            context: ./Lightspeed-ingest
            dockerfile: Dockerfile
        image: GRVYDEV/lightspeed_ingest:latest
        container_name: lightspeed_ingest
        ports:
            - 8084:8084
     webrtc:
        build:
            context: ./Lightspeed-webrtc
            dockerfile: Dockerfile
        image: GRVYDEV/lightspeed_webrtc:latest
        container_name: lightspeed_webrtc
        ports:
            - 8080:8080
     react:
        build:
            context: ./Lightspeed-react
            dockerfile: Dockerfile
        image: GRVYDEV/lightspeed_react:latest
        container_name: lightspeed_react
        ports:
            - 80:80

@NuroDev Yup, we can use this as the "builder repo" or we could merge the projects into one repo.
If we keep them separate like they are now, all we would need to do is compile the binary and pull a docker hub image. I opened this issue to address - GRVYDEV/Lightspeed-ingest/issues/15

@NuroDev Yup, we can use this as the "builder repo" or we could merge the projects into one repo.
If we keep them separate like they are now, all we would need to do is compile the binary and pull a docker hub image. I opened this issue to address - GRVYDEV/Lightspeed-ingest/issues/15

Here are the ports per - GRVYDEV/Lightspeed-ingest/issues/15
Ingest  - 8084:8084
WebRTC - 65535:8080
Lightspeed / React -  80:80

Here are the ports per - GRVYDEV/Lightspeed-ingest/issues/15
Ingest  - 8084:8084
WebRTC - 65535:8080
Lightspeed / React -  80:80

As I put in the example docker-compose.yml file above, alot of the stuff in it is ready to go. Just need to create the individual Dockerfile's for each project.
But it should be noted we will need a basic web server for React frontend UI. Also I think it would be better to set the webtrc address via an environment variable rather than a command.

As I put in the example docker-compose.yml file above, alot of the stuff in it is ready to go. Just need to create the individual Dockerfile's for each project.
But it should be noted we will need a basic web server for React frontend UI. Also I think it would be better to set the webtrc address via an environment variable rather than a command.

The golang service is not particularly docker friendly. You have to specify listen IP for it but it expects external IP. By passing internal IP, that IP is then sent to the client (web browser), and obviously browser can't access some random docker internal IP over internet.
This is a bit rough but it launches and accepts streams, just delivering them to the browser is problematic, but that should be an easy fix.
docker-compose.yml:
version: '3.3'
services:
    lightspeed:
      build: .
      command: /lightspeed/run.sh
      restart: always
      environment:
        - EXTERNAL_HOSTNAME=example.com:8080
      ports:
        - "80:80"
        - "8084:8084"
        - "8080:8080"
        - "65535:65535/udp"

Dockerfile:
FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
RUN apt-get update && \
  apt-get install -y \
  golang \
  rustc \
  nodejs \
  npm \
  git
RUN mkdir -p /lightspeed && \
  cd /lightspeed && \
  git clone https://github.com/GRVYDEV/Lightspeed-ingest.git && \
  cd Lightspeed-ingest && \
  cargo build --release
RUN mkdir -p /lightspeed && \
  cd /lightspeed && \
  git clone https://github.com/GRVYDEV/Lightspeed-webrtc.git && \
  cd Lightspeed-webrtc && \
  GO111MODULE=on go build
RUN mkdir -p /lightspeed && \
  cd /lightspeed && \
  git clone https://github.com/GRVYDEV/Lightspeed-react.git && \
  cd Lightspeed-react && \
  npm install && \
  npm install -g serve && \
  sed -i "s|export default|export|g" src/wsUrl.js && \
  sed -i -e '$a export default url;' src/wsUrl.js && \
  npm run-script build
COPY run.sh /lightspeed/run.sh

run.sh:
#!/bin/bash
find /lightspeed/Lightspeed-react/build/ -name "main*.js" -exec sed -i "s|stream.gud.software:8080|$EXTERNAL_HOSTNAME|g" {} \;

/lightspeed/Lightspeed-ingest/target/release/lightspeed-ingest &
/lightspeed/Lightspeed-webrtc/lightspeed-webrtc --addr=$(awk 'END{print $1}' /etc/hosts) &
cd /lightspeed/Lightspeed-react/ && serve -s build -l 80

You can use it for some inspiration maybe. :smile:

The golang service is not particularly docker friendly. You have to specify listen IP for it but it expects external IP. By passing internal IP, that IP is then sent to the client (web browser), and obviously browser can't access some random docker internal IP over internet.
This is a bit rough but it launches and accepts streams, just delivering them to the browser is problematic, but that should be an easy fix.
docker-compose.yml:
version: '3.3'
services:
    lightspeed:
      build: .
      command: /lightspeed/run.sh
      restart: always
      environment:
        - EXTERNAL_HOSTNAME=example.com:8080
      ports:
        - "80:80"
        - "8084:8084"
        - "8080:8080"
        - "65535:65535/udp"

Dockerfile:
FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=UTC
RUN apt-get update && \
  apt-get install -y \
  golang \
  rustc \
  nodejs \
  npm \
  git
RUN mkdir -p /lightspeed && \
  cd /lightspeed && \
  git clone https://github.com/GRVYDEV/Lightspeed-ingest.git && \
  cd Lightspeed-ingest && \
  cargo build --release
RUN mkdir -p /lightspeed && \
  cd /lightspeed && \
  git clone https://github.com/GRVYDEV/Lightspeed-webrtc.git && \
  cd Lightspeed-webrtc && \
  GO111MODULE=on go build
RUN mkdir -p /lightspeed && \
  cd /lightspeed && \
  git clone https://github.com/GRVYDEV/Lightspeed-react.git && \
  cd Lightspeed-react && \
  npm install && \
  npm install -g serve && \
  sed -i "s|export default|export|g" src/wsUrl.js && \
  sed -i -e '$a export default url;' src/wsUrl.js && \
  npm run-script build
COPY run.sh /lightspeed/run.sh

run.sh:
#!/bin/bash
find /lightspeed/Lightspeed-react/build/ -name "main*.js" -exec sed -i "s|stream.gud.software:8080|$EXTERNAL_HOSTNAME|g" {} \;

/lightspeed/Lightspeed-ingest/target/release/lightspeed-ingest &
/lightspeed/Lightspeed-webrtc/lightspeed-webrtc --addr=$(awk 'END{print $1}' /etc/hosts) &
cd /lightspeed/Lightspeed-react/ && serve -s build -l 80

You can use it for some inspiration maybe. :smile:

This is what I meant:

This is what I meant:

Here are the ports per - GRVYDEV/Lightspeed-ingest/issues/15
Ingest - 8084:8084
WebRTC - 65535:8080
Lightspeed / React - 80:80

Ingest - 8084:8084
WebRTC - 65535:65535 && 8080:8080
React 80:80

Here are the ports per - GRVYDEV/Lightspeed-ingest/issues/15
Ingest - 8084:8084
WebRTC - 65535:8080
Lightspeed / React - 80:80

Ingest - 8084:8084
WebRTC - 65535:65535 && 8080:8080
React 80:80

If this was converted into a mono repo...

You could also use submodules if you prefer to keep issues, commits, versions, etc separate.

If this was converted into a mono repo...

You could also use submodules if you prefer to keep issues, commits, versions, etc separate.

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

Yeah so if we are behind we will need to use a TURN server for WebRTC

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

Yeah so if we are behind we will need to use a TURN server for WebRTC

Something like https://github.com/coturn/coturn?

Something like https://github.com/coturn/coturn?

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

What about something like this?

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

What about something like this?

Yeah, I was about to

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

What about something like this?

That would work actually. I was playing around with that mode for Consul and Nomad. The networking parts are complicated enough that I am just going to install the binary on the servers. But for an app like this it would be perfect.

Yeah, I was about to

I thought that adding a quick workaround by replacing the *addr in ListenAndServe and ListenUDP calls to hard coded "<IP_ADDRESS>" would solve it but then I noticed it still sent internal addresses, so I guess it gets them dynamically somewhere. As long as those don't get rewritten to external address, it won't work in Docker or behind any other NAT.

What about something like this?

That would work actually. I was playing around with that mode for Consul and Nomad. The networking parts are complicated enough that I am just going to install the binary on the servers. But for an app like this it would be perfect.

I am running each service in its own docker, and using a docker-compose for them. Ingest seems to accept input from OBS and OBS is happy. I can access the react site on localhost, but the webRTC is not able to connect to webRTC as I assume it uses internal docker ips, instead of host.
EDIT: I changed it to use network_mode: host in the compose, and that makes everything work on localhost. Although not ideal for a deployment scenario. My branch: https://github.com/Crowdedlight/Project-Lightspeed/tree/feature/docker

I might try to deploy the dockers I have made on my own VPS when I got time again. And ideally, I want to incorporate the scripts to add custom streamkeys etc. But the dockerfiles and composer can be used as inspiration.
I do not know how easily this carries over to public outfacing ips, but at least it all works on localhost with docker and composes.
I agree that adding the three service-repos as submodules in this top repo would probably make it easier for CI/CD as you can use the submodules directly in the docker files without having to git clone it.

I am running each service in its own docker, and using a docker-compose for them. Ingest seems to accept input from OBS and OBS is happy. I can access the react site on localhost, but the webRTC is not able to connect to webRTC as I assume it uses internal docker ips, instead of host.
EDIT: I changed it to use network_mode: host in the compose, and that makes everything work on localhost. Although not ideal for a deployment scenario. My branch: https://github.com/Crowdedlight/Project-Lightspeed/tree/feature/docker

I might try to deploy the dockers I have made on my own VPS when I got time again. And ideally, I want to incorporate the scripts to add custom streamkeys etc. But the dockerfiles and composer can be used as inspiration.
I do not know how easily this carries over to public outfacing ips, but at least it all works on localhost with docker and composes.
I agree that adding the three service-repos as submodules in this top repo would probably make it easier for CI/CD as you can use the submodules directly in the docker files without having to git clone it.

Great idea, Docker would help launch adoption dramatically I think.
Adding my grain of salt, you can use buildx to build it for multiple CPU architectures (:eyes: arm) using a Github workflow similar to this

Great idea, Docker would help launch adoption dramatically I think.
Adding my grain of salt, you can use buildx to build it for multiple CPU architectures (:eyes: arm) using a Github workflow similar to this

I am running each service in its own docker, and using a docker-compose for them. Ingest seems to accept input from OBS and OBS is happy. I can access the react site on localhost, but the webRTC is not able to connect to webRTC as I assume it uses internal docker ips, instead of host.
EDIT: I changed it to use network_mode: host in the compose, and that makes everything work on localhost. Although not ideal for a deployment scenario. My branch: https://github.com/Crowdedlight/Project-Lightspeed/tree/feature/docker

I might try to deploy the dockers I have made on my own VPS when I got time again. And ideally, I want to incorporate the scripts to add custom streamkeys etc. But the dockerfiles and composer can be used as inspiration. (Ideally they should all also just use the binaries or multi-stage build. Only react uses multistage build atm)
I do not know how easily this carries over to public outfacing ips, but at least it all works on localhost with docker and composes.
I agree that adding the three service-repos as submodules in this top repo would probably make it easier for CI/CD as you can use the submodules directly in the docker files without having to git clone it.

Due to the nature of WebRTC I think if we are going to use docker it will have to be --net=host otherwise we would have to deploy a turn server which would greatly complicate the deployment process and thus render the point of docker kind of useless

I am running each service in its own docker, and using a docker-compose for them. Ingest seems to accept input from OBS and OBS is happy. I can access the react site on localhost, but the webRTC is not able to connect to webRTC as I assume it uses internal docker ips, instead of host.
EDIT: I changed it to use network_mode: host in the compose, and that makes everything work on localhost. Although not ideal for a deployment scenario. My branch: https://github.com/Crowdedlight/Project-Lightspeed/tree/feature/docker

I might try to deploy the dockers I have made on my own VPS when I got time again. And ideally, I want to incorporate the scripts to add custom streamkeys etc. But the dockerfiles and composer can be used as inspiration. (Ideally they should all also just use the binaries or multi-stage build. Only react uses multistage build atm)
I do not know how easily this carries over to public outfacing ips, but at least it all works on localhost with docker and composes.
I agree that adding the three service-repos as submodules in this top repo would probably make it easier for CI/CD as you can use the submodules directly in the docker files without having to git clone it.

Due to the nature of WebRTC I think if we are going to use docker it will have to be --net=host otherwise we would have to deploy a turn server which would greatly complicate the deployment process and thus render the point of docker kind of useless

I am running each service in its own docker, and using a docker-compose for them. Ingest seems to accept input from OBS and OBS is happy. I can access the react site on localhost, but the webRTC is not able to connect to webRTC as I assume it uses internal docker ips, instead of host.
EDIT: I changed it to use network_mode: host in the compose, and that makes everything work on localhost. Although not ideal for a deployment scenario. My branch: https://github.com/Crowdedlight/Project-Lightspeed/tree/feature/docker

I might try to deploy the dockers I have made on my own VPS when I got time again. And ideally, I want to incorporate the scripts to add custom streamkeys etc. But the dockerfiles and composer can be used as inspiration. (Ideally they should all also just use the binaries or multi-stage build. Only react uses multistage build atm)
I do not know how easily this carries over to public outfacing ips, but at least it all works on localhost with docker and composes.
I agree that adding the three service-repos as submodules in this top repo would probably make it easier for CI/CD as you can use the submodules directly in the docker files without having to git clone it.

In regards to stream keys I will implement a better system for them. Basically the ingest will make a cfg file that houses the stream key, If you want to reset it you will just run a command. Could you add some deployment instructions to your docker repo? I want to give it a try on my VPS and if it works would like to get it to master!

I am running each service in its own docker, and using a docker-compose for them. Ingest seems to accept input from OBS and OBS is happy. I can access the react site on localhost, but the webRTC is not able to connect to webRTC as I assume it uses internal docker ips, instead of host.
EDIT: I changed it to use network_mode: host in the compose, and that makes everything work on localhost. Although not ideal for a deployment scenario. My branch: https://github.com/Crowdedlight/Project-Lightspeed/tree/feature/docker

I might try to deploy the dockers I have made on my own VPS when I got time again. And ideally, I want to incorporate the scripts to add custom streamkeys etc. But the dockerfiles and composer can be used as inspiration. (Ideally they should all also just use the binaries or multi-stage build. Only react uses multistage build atm)
I do not know how easily this carries over to public outfacing ips, but at least it all works on localhost with docker and composes.
I agree that adding the three service-repos as submodules in this top repo would probably make it easier for CI/CD as you can use the submodules directly in the docker files without having to git clone it.

In regards to stream keys I will implement a better system for them. Basically the ingest will make a cfg file that houses the stream key, If you want to reset it you will just run a command. Could you add some deployment instructions to your docker repo? I want to give it a try on my VPS and if it works would like to get it to master!

Due to the nature of WebRTC I think if we are going to use docker it will have to be --net=host otherwise we would have to deploy a turn server which would greatly complicate the deployment process and thus render the point of docker kind of useless

I think that is an fair assumption. I don't know the inner workings of webRTC enough to judge how and when you would need a turn server. :)
Probably best just to go with --host for time being. That will however make it more important that individual ports on all network services can be configured. As many servers will already have webservers or services running on port 80 or 8080 etc. So it would be a requirement to configure those, to avoid collisions now docker can't map it to custom ports.

In regards to stream keys I will implement a better system for them. Basically the ingest will make a cfg file that houses the stream key, If you want to reset it you will just run a command. Could you add some deployment instructions to your docker repo? I want to give it a try on my VPS and if it works would like to get it to master!

That make sense. I will add some information in the readme and do a push in a bit.

Due to the nature of WebRTC I think if we are going to use docker it will have to be --net=host otherwise we would have to deploy a turn server which would greatly complicate the deployment process and thus render the point of docker kind of useless

I think that is an fair assumption. I don't know the inner workings of webRTC enough to judge how and when you would need a turn server. :)
Probably best just to go with --host for time being. That will however make it more important that individual ports on all network services can be configured. As many servers will already have webservers or services running on port 80 or 8080 etc. So it would be a requirement to configure those, to avoid collisions now docker can't map it to custom ports.

In regards to stream keys I will implement a better system for them. Basically the ingest will make a cfg file that houses the stream key, If you want to reset it you will just run a command. Could you add some deployment instructions to your docker repo? I want to give it a try on my VPS and if it works would like to get it to master!

That make sense. I will add some information in the readme and do a push in a bit.

Due to the nature of WebRTC I think if we are going to use docker it will have to be --net=host otherwise we would have to deploy a turn server which would greatly complicate the deployment process and thus render the point of docker kind of useless

I think that is an fair assumption. I don't know the inner workings of webRTC enough to judge how and when you would need a turn server. :)
Probably best just to go with --host for time being. That will however make it more important that individual ports on all network services can be configured. As many servers will already have webservers or services running on port 80 or 8080 etc. So it would be a requirement to configure those, to avoid collisions now docker can't map it to custom ports.

In regards to stream keys I will implement a better system for them. Basically the ingest will make a cfg file that houses the stream key, If you want to reset it you will just run a command. Could you add some deployment instructions to your docker repo? I want to give it a try on my VPS and if it works would like to get it to master!

That make sense. I will add some information in the readme and do a push in a bit.

For those that are already running webservers something like NGINX could be used to route a certain subdomain to a different webport. For example gud.software could be my main site then stream.gud.software could route to the react port

Due to the nature of WebRTC I think if we are going to use docker it will have to be --net=host otherwise we would have to deploy a turn server which would greatly complicate the deployment process and thus render the point of docker kind of useless

I think that is an fair assumption. I don't know the inner workings of webRTC enough to judge how and when you would need a turn server. :)
Probably best just to go with --host for time being. That will however make it more important that individual ports on all network services can be configured. As many servers will already have webservers or services running on port 80 or 8080 etc. So it would be a requirement to configure those, to avoid collisions now docker can't map it to custom ports.

In regards to stream keys I will implement a better system for them. Basically the ingest will make a cfg file that houses the stream key, If you want to reset it you will just run a command. Could you add some deployment instructions to your docker repo? I want to give it a try on my VPS and if it works would like to get it to master!

That make sense. I will add some information in the readme and do a push in a bit.

For those that are already running webservers something like NGINX could be used to route a certain subdomain to a different webport. For example gud.software could be my main site then stream.gud.software could route to the react port

For those that are already running webservers something like NGINX could be used to route a certain subdomain to a different webport. For example gud.software could be my main site then stream.gud.software could route to the react port

I might be wrong here. But I saw the hosts bind port: <IP_ADDRESS>:80 and <IP_ADDRESS>:8080 which means that you bind all addresses and can't use the same port between different services?

For those that are already running webservers something like NGINX could be used to route a certain subdomain to a different webport. For example gud.software could be my main site then stream.gud.software could route to the react port

I might be wrong here. But I saw the hosts bind port: <IP_ADDRESS>:80 and <IP_ADDRESS>:8080 which means that you bind all addresses and can't use the same port between different services?

I added a readme to my branch. I hope it clears some stuff up. It also showcases the pitfalls of the current docker setup in terms of setting the right IPs etc. ;-)
https://github.com/Crowdedlight/Project-Lightspeed/blob/feature/docker/docker_README.md

I added a readme to my branch. I hope it clears some stuff up. It also showcases the pitfalls of the current docker setup in terms of setting the right IPs etc. ;-)
https://github.com/Crowdedlight/Project-Lightspeed/blob/feature/docker/docker_README.md

You can run TURN server in Docker as well (for example coturn). I don't really know the WebRTC protocol and all the signaling bits but I thought that it is just a matter of adding turn in the mix.

You can run TURN server in Docker as well (for example coturn). I don't really know the WebRTC protocol and all the signaling bits but I thought that it is just a matter of adding turn in the mix.

I added a readme to my branch. I hope it clears some stuff up. It also showcases the pitfalls of the current docker setup in terms of setting the right IPs etc. ;-)
https://github.com/Crowdedlight/Project-Lightspeed/blob/feature/docker/docker_README.md

Going to give it a try now!

I added a readme to my branch. I hope it clears some stuff up. It also showcases the pitfalls of the current docker setup in terms of setting the right IPs etc. ;-)
https://github.com/Crowdedlight/Project-Lightspeed/blob/feature/docker/docker_README.md

Going to give it a try now!

I added a readme to my branch. I hope it clears some stuff up. It also showcases the pitfalls of the current docker setup in terms of setting the right IPs etc. ;-)
https://github.com/Crowdedlight/Project-Lightspeed/blob/feature/docker/docker_README.md

absolutely fantastic work! I was able to get it up and running really easily! I would like to merge this into master and then turn the respective folders into submodules

I added a readme to my branch. I hope it clears some stuff up. It also showcases the pitfalls of the current docker setup in terms of setting the right IPs etc. ;-)
https://github.com/Crowdedlight/Project-Lightspeed/blob/feature/docker/docker_README.md

absolutely fantastic work! I was able to get it up and running really easily! I would like to merge this into master and then turn the respective folders into submodules

absolutely fantastic work! I was able to get it up and running really easily! I would like to merge this into master and then turn the respective folders into submodules

I haven't played with submodules and docker that much, but it should be possible to add the dockerfile to the root of each repo and then add each service repo as submodule in this repo. Then have the composer file in root of this repo, and have it reference the submodules. Might be some methodologys that works better if you release binarys from each service instead etc.  I believe some examples was given further up in this thread.
And a better method of setting the wsUrl.js that doesn't require modification to the dockerfile itself. :)

absolutely fantastic work! I was able to get it up and running really easily! I would like to merge this into master and then turn the respective folders into submodules

I haven't played with submodules and docker that much, but it should be possible to add the dockerfile to the root of each repo and then add each service repo as submodule in this repo. Then have the composer file in root of this repo, and have it reference the submodules. Might be some methodologys that works better if you release binarys from each service instead etc.  I believe some examples was given further up in this thread.
And a better method of setting the wsUrl.js that doesn't require modification to the dockerfile itself. :)

absolutely fantastic work! I was able to get it up and running really easily! I would like to merge this into master and then turn the respective folders into submodules

I haven't played with submodules and docker that much, but it should be possible to add the dockerfile to the root of each repo and then add each service repo as submodule in this repo. Then have the composer file in root of this repo, and have it reference the submodules. Might be some methodologys that works better if you release binarys from each service instead etc. I believe some examples was given further up in this thread.
And a better method of setting the wsUrl.js that doesn't require modification to the dockerfile itself. :)

I am working on the submodules right now on /feature/submodules as far as react goes we could just grab an ENV var and default to localhost?

absolutely fantastic work! I was able to get it up and running really easily! I would like to merge this into master and then turn the respective folders into submodules

I haven't played with submodules and docker that much, but it should be possible to add the dockerfile to the root of each repo and then add each service repo as submodule in this repo. Then have the composer file in root of this repo, and have it reference the submodules. Might be some methodologys that works better if you release binarys from each service instead etc. I believe some examples was given further up in this thread.
And a better method of setting the wsUrl.js that doesn't require modification to the dockerfile itself. :)

I am working on the submodules right now on /feature/submodules as far as react goes we could just grab an ENV var and default to localhost?

Would be great if you would update the value during startup so you don't have to recompile it every time you want to adjust some parameters.

Would be great if you would update the value during startup so you don't have to recompile it every time you want to adjust some parameters.

Could you all review https://github.com/GRVYDEV/Project-Lightspeed/tree/feature/submodules and let me know what you think?

Could you all review https://github.com/GRVYDEV/Project-Lightspeed/tree/feature/submodules and let me know what you think?

We decided in the last few weeks at American Airlines not to use submodules. After an internal review, it caused more confusion and commit issues than it was worth.

We decided in the last few weeks at American Airlines not to use submodules. After an internal review, it caused more confusion and commit issues than it was worth.

We decided in the last few weeks at American Airlines not to use submodules. After an internal review, it caused more confusion and commit issues than it was worth.
Edit to say that the docker files and docker compose look good. With the submodules it adds some management overhead. I also have a concern about how this can get deployed with something like K8's / Nomad if host networking mode is enabled. We need an MVP, so that is more like a 0.2.0 concern.

Yeah we can look into a TURN implementation if we want to get away from host networking

We decided in the last few weeks at American Airlines not to use submodules. After an internal review, it caused more confusion and commit issues than it was worth.
Edit to say that the docker files and docker compose look good. With the submodules it adds some management overhead. I also have a concern about how this can get deployed with something like K8's / Nomad if host networking mode is enabled. We need an MVP, so that is more like a 0.2.0 concern.

Yeah we can look into a TURN implementation if we want to get away from host networking

If each repo has it's own Dockerfile and builds + deploys the images somewhere (Dockerhub?) then a compose file in this repo can just reference the latest images no? Rather than using submodules.

If each repo has it's own Dockerfile and builds + deploys the images somewhere (Dockerhub?) then a compose file in this repo can just reference the latest images no? Rather than using submodules.

Yes it could! Ideally I would like to keep the submodules in here just for the sake of having them however if we switched it to that system then you wouldn't need to worry about downloading them!

Yes it could! Ideally I would like to keep the submodules in here just for the sake of having them however if we switched it to that system then you wouldn't need to worry about downloading them!

It would definitely be my preference to have the docker images built and pushed per repo.

It would definitely be my preference to have the docker images built and pushed per repo.

It would definitely be my preference to have the docker images built and pushed per repo.

Yeah I definitely would like to do this

It would definitely be my preference to have the docker images built and pushed per repo.

Yeah I definitely would like to do this

It would definitely be my preference to have the docker images built and pushed per repo.
You would only have to download this repo and run a docker-compose up -d in the root.

My question to this would be how will we handle the URL and address configuration?

It would definitely be my preference to have the docker images built and pushed per repo.
You would only have to download this repo and run a docker-compose up -d in the root.

My question to this would be how will we handle the URL and address configuration?

It would definitely be my preference to have the docker images built and pushed per repo.
You would only have to download this repo and run a docker-compose up -d in the root.

My question to this would be how will we handle the URL and address configuration?

I'm thinking you have an "entrypoint.sh" file in the root and have docker bootstrap on startup. Do a delay until ready and show back the connection IP/Port in the deployment. Mine is not ready, but in practice it should work fine. You would need to proxy the traffic to the docker with NGINX or Caddy I think?

https://github.com/scorpion/uvdesk - Clever way of doing this, check the dockerfile.
https://github.com/multani/docker-nomad
https://github.com/scorpion/lms/blob/main/entrypoint.sh

It's something I would have to think/play with.

It would definitely be my preference to have the docker images built and pushed per repo.
You would only have to download this repo and run a docker-compose up -d in the root.

My question to this would be how will we handle the URL and address configuration?

I'm thinking you have an "entrypoint.sh" file in the root and have docker bootstrap on startup. Do a delay until ready and show back the connection IP/Port in the deployment. Mine is not ready, but in practice it should work fine. You would need to proxy the traffic to the docker with NGINX or Caddy I think?

https://github.com/scorpion/uvdesk - Clever way of doing this, check the dockerfile.
https://github.com/multani/docker-nomad
https://github.com/scorpion/lms/blob/main/entrypoint.sh

It's something I would have to think/play with.

I would also suggest serving static files from the same golang or rust service. Why have additional background process?
And ideally the golang and rust service could be merged in one. I suppose you chose two different ecosystems because of library availability?

I would also suggest serving static files from the same golang or rust service. Why have additional background process?
And ideally the golang and rust service could be merged in one. I suppose you chose two different ecosystems because of library availability?

I would also suggest serving static files from the same golang or rust service. Why have additional background process?
And ideally the golang and rust service could be merged in one. I suppose you chose two different ecosystems because of library availability?

So I like the idea of decoupling the ingest from the broadcast since it makes it more modular but yeah this whole project could be one codebase.

I would also suggest serving static files from the same golang or rust service. Why have additional background process?
And ideally the golang and rust service could be merged in one. I suppose you chose two different ecosystems because of library availability?

So I like the idea of decoupling the ingest from the broadcast since it makes it more modular but yeah this whole project could be one codebase.

I would prefer this be focused into one Rust project personally. It would make maintenance a lot easier and I will be able to eventually throw resources at Rust projects where I would not with Elixar and Go. Those are just not in our wheelhouse of developer skills.
I am advocating within AA a push to learn Rust.

I would prefer this be focused into one Rust project personally. It would make maintenance a lot easier and I will be able to eventually throw resources at Rust projects where I would not with Elixar and Go. Those are just not in our wheelhouse of developer skills.
I am advocating within AA a push to learn Rust.

Here is how Apollo does their Rust development for the Stargate package-

https://github.com/apollographql/federation/tree/main/stargate
https://crates.io/crates/stargate

Definitely a good approach.

Here is how Apollo does their Rust development for the Stargate package-

https://github.com/apollographql/federation/tree/main/stargate
https://crates.io/crates/stargate

Definitely a good approach.

I would prefer this be focused into one Rust project personally. It would make maintenance a lot easier and I'll be able to throw resources at Rust projects eventually where I would not with Elixar and Go. Those are just not in our wheelhouse of developer skills. Go may be in the future though...
I am advocating within AA a push to learn Rust.

The issue with this is that the WebRTC support isnt great in Rust ATM (however someone is rewriting Pion in Rust). I like the overall modular structure of this project though and the goal will be to flesh out the Rust ingest to relay the packets to the loopback interface so the WebRTC service does not need to listen on an external port. Modularity in this case is good IMO because you could easily write another ingest that accepts lets say RTMP for example and as long as you relay RTP packets on localhost it will be plug and play with the WebRTC service. In traditional streaming architecture it is broken up into 3 main parts. Ingest, Broadcast and the Website and I plan to always have Lightspeed mirror that architecture.

I would prefer this be focused into one Rust project personally. It would make maintenance a lot easier and I'll be able to throw resources at Rust projects eventually where I would not with Elixar and Go. Those are just not in our wheelhouse of developer skills. Go may be in the future though...
I am advocating within AA a push to learn Rust.

The issue with this is that the WebRTC support isnt great in Rust ATM (however someone is rewriting Pion in Rust). I like the overall modular structure of this project though and the goal will be to flesh out the Rust ingest to relay the packets to the loopback interface so the WebRTC service does not need to listen on an external port. Modularity in this case is good IMO because you could easily write another ingest that accepts lets say RTMP for example and as long as you relay RTP packets on localhost it will be plug and play with the WebRTC service. In traditional streaming architecture it is broken up into 3 main parts. Ingest, Broadcast and the Website and I plan to always have Lightspeed mirror that architecture.

I think a monorepo and an official Docker Image would both be great

I think a monorepo and an official Docker Image would both be great

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.

I have not used Docker hub before so I need to go through and do all of that.

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.

I have not used Docker hub before so I need to go through and do all of that.

It takes a few weeks - https://www.docker.com/community/open-source
You are welcome to use the scorpion namespace if desired, or we can just build the images until finalized. I will be building and publishing my own scorpion versions anyways. :)

It takes a few weeks - https://www.docker.com/community/open-source
You are welcome to use the scorpion namespace if desired, or we can just build the images until finalized. I will be building and publishing my own scorpion versions anyways. :)

It takes a few weeks - https://www.docker.com/community/open-source
You are welcome to use the scorpion namespace if desired, or we can just build the images until finalized. I will be building and publishing my own scorpion versions anyways. :)

That would be great! I will kickstart that process right now and then I am going to work on the documentation per Paul Irish's issue first then move to docker images afterwards!

It takes a few weeks - https://www.docker.com/community/open-source
You are welcome to use the scorpion namespace if desired, or we can just build the images until finalized. I will be building and publishing my own scorpion versions anyways. :)

That would be great! I will kickstart that process right now and then I am going to work on the documentation per Paul Irish's issue first then move to docker images afterwards!

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.

Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage

I think a monorepo and an official Docker Image would both be great

So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!

I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.

Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage

Couldn’t you use flairs/tags?

Le 5 janv. 2021 à 14:07, Garrett GRVY Graves<EMAIL_ADDRESS>a écrit :

I think a monorepo and an official Docker Image would both be great
So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!
I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.
Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Couldn’t you use flairs/tags?

Le 5 janv. 2021 à 14:07, Garrett GRVY Graves<EMAIL_ADDRESS>a écrit :

I think a monorepo and an official Docker Image would both be great
So I think the route I am going to take is auto-generation of docker images on tagged commits to each respective repository and then I would like to include the other repos as submodules in this one. Only issue with that is i need to figure out how to keep the submodules updated to the heads of the other repos. Checkout the branch feature/submodules to see where I am at on this!
I would not use submodules and instead just use this repo as the monorepo. I would prefer the code be in one repo, but I really just need a way to have the docker compose pull the compiled docker images. Do you have a location on docker hub to push docker images? There is a pull limit if you have not gone through the OSS org steps.
Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage

100% okay with this. I would still not use submodules and just use this as the build repository for docker and documentation. The docker-compose.yml will pull the images from docker hub and those images are triggered from each repository using GitHub Actions.
I'm just trying to convey that you will 100% have issues with submodules no matter how perfectly maintained they are. It's just near impossible since users are not very familiar with them as they are not used very often.

Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage

100% okay with this. I would still not use submodules and just use this as the build repository for docker and documentation. The docker-compose.yml will pull the images from docker hub and those images are triggered from each repository using GitHub Actions.
I'm just trying to convey that you will 100% have issues with submodules no matter how perfectly maintained they are. It's just near impossible since users are not very familiar with them as they are not used very often.

Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage

100% okay with this. I would still not use submodules and just use this as the build repository for docker and documentation. The docker-compose.yml will pull the images from docker hub and those images are triggered from each repository using GitHub Actions.
I'm just trying to convey that you will 100% have issues with submodules no matter how perfectly maintained they are. It's just near impossible since users are not very familiar with them as they are not used very often.

Understood and will stay away from them!

Another note, the reason I do not want a monorepo is that I would like to keep issue tracking separate since it makes it easier to manage

100% okay with this. I would still not use submodules and just use this as the build repository for docker and documentation. The docker-compose.yml will pull the images from docker hub and those images are triggered from each repository using GitHub Actions.
I'm just trying to convey that you will 100% have issues with submodules no matter how perfectly maintained they are. It's just near impossible since users are not very familiar with them as they are not used very often.

Understood and will stay away from them!

Couldn’t you use flairs/tags?

Absolutely but I dont think this is what I want for this project

Couldn’t you use flairs/tags?

Absolutely but I dont think this is what I want for this project

@brettwilcox Okay so here is an issue. Right now you need to edit a dockerfile for the react website in order to make sure your websocket url is correct. How would I remedy this with a premade image? Is there a way I can use a flag or env var?

@brettwilcox Okay so here is an issue. Right now you need to edit a dockerfile for the react website in order to make sure your websocket url is correct. How would I remedy this with a premade image? Is there a way I can use a flag or env var?

Below is an example pattern I use.
.env
IMAGE=myimage
TAG=1.2.3

docker-compose.yml
services:
  my-app:
    env_file:
      - ".env"
    labels:
      - "IMAGE=${IMAGE}"
      - "TAG=${TAG}"
    build:
      context: ./docker/my-app
      args:
        - "IMAGE=${IMAGE}"
        - "TAG=${TAG}"
    restart: always

./docker/my-app/Dockerfile
ARG IMAGE =${IMAGE}
ARG TAG=${TAG}

# Image and version to pull
FROM ${IMAGE}:${TAG}

Below is an example pattern I use.
.env
IMAGE=myimage
TAG=1.2.3

docker-compose.yml
services:
  my-app:
    env_file:
      - ".env"
    labels:
      - "IMAGE=${IMAGE}"
      - "TAG=${TAG}"
    build:
      context: ./docker/my-app
      args:
        - "IMAGE=${IMAGE}"
        - "TAG=${TAG}"
    restart: always

./docker/my-app/Dockerfile
ARG IMAGE =${IMAGE}
ARG TAG=${TAG}

# Image and version to pull
FROM ${IMAGE}:${TAG}

I've slept since I did this...

https://github.com/scorpion/php
https://www.docker.com/blog/docker-github-actions/

I've slept since I did this...

https://github.com/scorpion/php
https://www.docker.com/blog/docker-github-actions/

You would have a .env.example file in the root with the documented environment variables.

You would have a .env.example file in the root with the documented environment variables.

How would I remedy this with a premade image? Is there a way I can use a flag or env var?

IIRC ARG in dockerfiles is used for build-time variables - if we want to pre-build a docker image, and have websocket URL as a run-time variable, we'd want ENV, so the Dockerfile would include stuff like:
ENV WS_URL=localhost
CMD sed -i "s|stream.gud.software|$WS_URL|g" public/config.json && serve -s build -l 80

(Then you can override WS_URL using whatever mechanism (docker-compose.yml, .env, docker -e, ...) and it'll be inserted into the config file at runtime)

How would I remedy this with a premade image? Is there a way I can use a flag or env var?

IIRC ARG in dockerfiles is used for build-time variables - if we want to pre-build a docker image, and have websocket URL as a run-time variable, we'd want ENV, so the Dockerfile would include stuff like:
ENV WS_URL=localhost
CMD sed -i "s|stream.gud.software|$WS_URL|g" public/config.json && serve -s build -l 80

(Then you can override WS_URL using whatever mechanism (docker-compose.yml, .env, docker -e, ...) and it'll be inserted into the config file at runtime)

I've made a few PRs to each of the repos. Right now, in hopes that this is useful information, I have everything running from these commands and this docker-compose.yml file:
(to build the docker images locally)
mkdir lightspeed
cd lightspeed
git clone https://github.com/GRVYDEV/Lightspeed-ingest/
cd Lightspeed-ingest
docker build . -t lightspeed-ingest-test
cd ..
git clone https://github.com/GRVYDEV/Lightspeed-webrtc/
cd Lightspeed-webrtc
docker build . -t lightspeed-webrtc-test
cd ..
git clone https://github.com/GRVYDEV/Lightspeed-react/
cd Lightspeed-react
sed -i "s|stream.gud.software:8080|<my public domain>:8085|g" src/wsUrl.js
docker build . -t lightspeed-react-test
cd ..

(docker-compose.yml)
version: '3.7'
services:
    lightspeed_ingest:
        image: lightspeed-ingest-test
        container_name: lightspeed_ingest
        restart: unless-stopped
        ports:
            - 8084:8084

    lightspeed_webrtc:
        image: lightspeed-webrtc-test
        container_name: lightspeed_webrtc
        restart: unless-stopped
        entrypoint: lightspeed-webrtc
        command: --addr=<IP_ADDRESS> --ip=<my public IP> --ports=19050-19100
        ports:
            - 8085:8080 # WebRTC
            - 65535:65535/udp # RTP
            - 19050-19100:19050-19100 # WebRTC PeerConnection
            - 19050-19100:19050-19100/udp # WebRTC PeerConnection UDP

    lightspeed_react:
        image: lightspeed-react-test
        container_name: lightspeed_react
        restart: unless-stopped
        ports:
            - 8086:80

Now from this setup, I have the react web server running on port 8086, the websocket server on port 8085, and the FTL port on 8084. I also have port 65535 forwarded for RTP and ports 19050-19100 forwarded for WebRTC ICE connections.
Once these images exist on docker hub, I'm sure this setup will become much simpler.
Hope that's helpful!

Awesome work! Thank you so much! I am going to work on getting the images on docker hub ASAP along with github actions for building them

I've made a few PRs to each of the repos. Right now, in hopes that this is useful information, I have everything running from these commands and this docker-compose.yml file:
(to build the docker images locally)
mkdir lightspeed
cd lightspeed
git clone https://github.com/GRVYDEV/Lightspeed-ingest/
cd Lightspeed-ingest
docker build . -t lightspeed-ingest-test
cd ..
git clone https://github.com/GRVYDEV/Lightspeed-webrtc/
cd Lightspeed-webrtc
docker build . -t lightspeed-webrtc-test
cd ..
git clone https://github.com/GRVYDEV/Lightspeed-react/
cd Lightspeed-react
sed -i "s|stream.gud.software:8080|<my public domain>:8085|g" src/wsUrl.js
docker build . -t lightspeed-react-test
cd ..

(docker-compose.yml)
version: '3.7'
services:
    lightspeed_ingest:
        image: lightspeed-ingest-test
        container_name: lightspeed_ingest
        restart: unless-stopped
        ports:
            - 8084:8084

    lightspeed_webrtc:
        image: lightspeed-webrtc-test
        container_name: lightspeed_webrtc
        restart: unless-stopped
        entrypoint: lightspeed-webrtc
        command: --addr=<IP_ADDRESS> --ip=<my public IP> --ports=19050-19100
        ports:
            - 8085:8080 # WebRTC
            - 65535:65535/udp # RTP
            - 19050-19100:19050-19100 # WebRTC PeerConnection
            - 19050-19100:19050-19100/udp # WebRTC PeerConnection UDP

    lightspeed_react:
        image: lightspeed-react-test
        container_name: lightspeed_react
        restart: unless-stopped
        ports:
            - 8086:80

Now from this setup, I have the react web server running on port 8086, the websocket server on port 8085, and the FTL port on 8084. I also have port 65535 forwarded for RTP and ports 19050-19100 forwarded for WebRTC ICE connections.
Once these images exist on docker hub, I'm sure this setup will become much simpler.
Hope that's helpful!

Awesome work! Thank you so much! I am going to work on getting the images on docker hub ASAP along with github actions for building them

@k3d3 @brettwilcox Is there a way I could setup a stage in docker-compose file that would tell the lightspeed-react image to run the sed -i command with a provided value? What I mean is if the main repo contained a docker compose and a .env file is there a way someone could edit the .env file and then the compose file would edit the proper values in each image? Im trying to figure out a solution that is basically clone the main repo, change a couple lines in a .env file and boom itll work

@k3d3 @brettwilcox Is there a way I could setup a stage in docker-compose file that would tell the lightspeed-react image to run the sed -i command with a provided value? What I mean is if the main repo contained a docker compose and a .env file is there a way someone could edit the .env file and then the compose file would edit the proper values in each image? Im trying to figure out a solution that is basically clone the main repo, change a couple lines in a .env file and boom itll work

You could technically put a variable in an entrypoint: sed ${VARIABLE} && /entrypoint or command:  block, which could be read from a .env file.
As a side note, I wrote reactserv which reads static react files from disk, and serves them from memory as well as modify them to set the root url in the react code. You could use a similar approach to serve your react code as well as replace values at start through an env variable. That way you can even have the final image based on scratch (no OS).

You could technically put a variable in an entrypoint: sed ${VARIABLE} && /entrypoint or command:  block, which could be read from a .env file.
As a side note, I wrote reactserv which reads static react files from disk, and serves them from memory as well as modify them to set the root url in the react code. You could use a similar approach to serve your react code as well as replace values at start through an env variable. That way you can even have the final image based on scratch (no OS).

You could technically put a variable in an entrypoint: sed ${VARIABLE} && /entrypoint or command:  block, which could be read from a .env file.
As a side note, I wrote reactserv which reads static react files from disk, and serves them from memory as well as modify them to set the root url in the react code. You could use a similar approach to serve your react code as well as replace values at start through an env variable. That way you can even have the final image based on scratch (no OS) as long as you use go/rust and compile statically.

So theoretically in the docker compose file I could do command: sed -i "s|stream.gud.software|ENV_VAR|g" build/config.json && serve -s build -l 80

You could technically put a variable in an entrypoint: sed ${VARIABLE} && /entrypoint or command:  block, which could be read from a .env file.
As a side note, I wrote reactserv which reads static react files from disk, and serves them from memory as well as modify them to set the root url in the react code. You could use a similar approach to serve your react code as well as replace values at start through an env variable. That way you can even have the final image based on scratch (no OS) as long as you use go/rust and compile statically.

So theoretically in the docker compose file I could do command: sed -i "s|stream.gud.software|ENV_VAR|g" build/config.json && serve -s build -l 80

A simple solution to overriding the websocket url in a running container is to just document how to provide your own config.json file using docker mounts.

A simple solution to overriding the websocket url in a running container is to just document how to provide your own config.json file using docker mounts.

While I'm not sure of the best solution, I would also like some way to change the port number as well, since I'm using port 8085 in my setup.

While I'm not sure of the best solution, I would also like some way to change the port number as well, since I'm using port 8085 in my setup.

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

While I'm not sure of the best solution, I would also like some way to change the port number as well, since I'm using port 8085 in my setup.

Yup the plan is to have the config change the entire url including port. It sounds like we may mount a custom config and then point react at that

While I'm not sure of the best solution, I would also like some way to change the port number as well, since I'm using port 8085 in my setup.

Yup the plan is to have the config change the entire url including port. It sounds like we may mount a custom config and then point react at that

While I'm not sure of the best solution, I would also like some way to change the port number as well, since I'm using port 8085 in my setup.

Yup the plan is to have the config change the entire url including port. It sounds like we may mount a custom config and then point react at that

also I will add support for changing the websocket host URL as well in Lightspeed-webrtc once I get this docker stuff finished

While I'm not sure of the best solution, I would also like some way to change the port number as well, since I'm using port 8085 in my setup.

Yup the plan is to have the config change the entire url including port. It sounds like we may mount a custom config and then point react at that

also I will add support for changing the websocket host URL as well in Lightspeed-webrtc once I get this docker stuff finished

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

I think it's a fairly common pattern - mounting config files into docker containers. While it's just a url/port change now it also allows further config easily later :)

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

I think it's a fairly common pattern - mounting config files into docker containers. While it's just a url/port change now it also allows further config easily later :)

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

It's probably not a bad idea to have a config file for this project in general. There does seem to be enough state to require some way of configuring the project (other than using sed) and a single config (volumed read-only into each container) would solve this problem.
Alternatively, as someone mentioned above ENV variables are not a bad idea either. There are a lot of containers which use this.
This is fairly common, for example the official NGINX image uses:
  environment:
   - NGINX_HOST=foobar.com
   - NGINX_PORT=80

although I think this might actually just be a feature of NGINX and not something handles by docker...
If having configuration within the code isn't desired, the entrypoint of each container is also a good place to handle this (obviously, as long as these are runtime configurations. I haven't dug deep enough to see if this is the case).

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

It's probably not a bad idea to have a config file for this project in general. There does seem to be enough state to require some way of configuring the project (other than using sed) and a single config (volumed read-only into each container) would solve this problem.
Alternatively, as someone mentioned above ENV variables are not a bad idea either. There are a lot of containers which use this.
This is fairly common, for example the official NGINX image uses:
  environment:
   - NGINX_HOST=foobar.com
   - NGINX_PORT=80

although I think this might actually just be a feature of NGINX and not something handles by docker...
If having configuration within the code isn't desired, the entrypoint of each container is also a good place to handle this (obviously, as long as these are runtime configurations. I haven't dug deep enough to see if this is the case).

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

I think it's a fairly common pattern - mounting config files into docker containers. While it's just a url/port change now it also allows further config easily later :)

Specifically in regards to the react site I know for SURE there will be more config in the future. Especially in respect to monetization when we get to that point (think stripe api key etc). In order to keep this project as general purpose as possible I want to make sure we make smart configuration decisions. Ideally 1 file for the whole project would be GREAT however I am very new to dockerization so I am all ears.

I do like that idea too - providing a config.json via docker volume - but that also seems pretty heavyweight for just a url change.

I think it's a fairly common pattern - mounting config files into docker containers. While it's just a url/port change now it also allows further config easily later :)

Specifically in regards to the react site I know for SURE there will be more config in the future. Especially in respect to monetization when we get to that point (think stripe api key etc). In order to keep this project as general purpose as possible I want to make sure we make smart configuration decisions. Ideally 1 file for the whole project would be GREAT however I am very new to dockerization so I am all ears.

Both "one config.json for the project, mounted into each container" and "config stored in .env, which docker-compose passes into the containers via environment variables" sound good to me (though the first gives you all the JSON data types and structure, while the later only does key/value string pairs, so maybe the first is significantly better...?)
(I'm another person hoping for configurable port numbers as I'm already using 8084 :P)

Both "one config.json for the project, mounted into each container" and "config stored in .env, which docker-compose passes into the containers via environment variables" sound good to me (though the first gives you all the JSON data types and structure, while the later only does key/value string pairs, so maybe the first is significantly better...?)
(I'm another person hoping for configurable port numbers as I'm already using 8084 :P)

okay, so I think this actually just goes deeper in general.
In the short term it likely makes sense to just choose something. Mentioned above:

entrypoint: sed ${VARIABLE} && /entrypoint

This will work and is very easy to configure. It allows you to pass in the variables via an .env file and also set them by passing them in when running docker-compose
here are some references:

https://stackoverflow.com/questions/47451060/multiple-docker-build-args-from-docker-compose-env-file
https://docs.docker.com/compose/environment-variables/

essentially, these variables end up being used as "build-args" which can modify the the container during build time.

Now, let's ignore docker for a second because it can basically be thought of as any deployment environment.
So, for your future config needs, I would ignore docker and focus on doing what is right for the project elements. A config file is probably the best idea. But really, spend some time thinking about an extensible method that fits your needs.

okay, so I think this actually just goes deeper in general.
In the short term it likely makes sense to just choose something. Mentioned above:

entrypoint: sed ${VARIABLE} && /entrypoint

This will work and is very easy to configure. It allows you to pass in the variables via an .env file and also set them by passing them in when running docker-compose
here are some references:

https://stackoverflow.com/questions/47451060/multiple-docker-build-args-from-docker-compose-env-file
https://docs.docker.com/compose/environment-variables/

essentially, these variables end up being used as "build-args" which can modify the the container during build time.

Now, let's ignore docker for a second because it can basically be thought of as any deployment environment.
So, for your future config needs, I would ignore docker and focus on doing what is right for the project elements. A config file is probably the best idea. But really, spend some time thinking about an extensible method that fits your needs.

Hmm, after thinking about it, I definitely agree that a mounted config file is the way to go. Especially with docker-compose, those are very simple to make.
Within docker-compose.yml, you'd just add another section under lightspeed_react:
        volumes:
          - ./local/path/to/config.json:/internal/path/to/config.json:ro

The ro keeps it read-only within the container, and can be omitted to make it read-write (though we probably don't want that here).

Hmm, after thinking about it, I definitely agree that a mounted config file is the way to go. Especially with docker-compose, those are very simple to make.
Within docker-compose.yml, you'd just add another section under lightspeed_react:
        volumes:
          - ./local/path/to/config.json:/internal/path/to/config.json:ro

The ro keeps it read-only within the container, and can be omitted to make it read-write (though we probably don't want that here).

@GRVYDEV there are some things you'll probably want to keep as docker configurable.
For example, the port mappings probably make the most sense being configured (via ENV or .env file) and then keeping the project with default port values.
if this is unclear, basically, in the docker-compose.yml you would change the port number before the : to be:
lightspeed-ingest:
    restart: on-failure
    build: ingest/
    network_mode: host
    ports:
        - "${LIGHTSPEED_PORT}:8084"

oh, additionally, you can also modify the CMD for a container in the docker-compose.yml so, for the WebRTC container, you could do:
command: ["lightspeed-webrtc", "--addr=${IP_ADDRESS}"]

@GRVYDEV there are some things you'll probably want to keep as docker configurable.
For example, the port mappings probably make the most sense being configured (via ENV or .env file) and then keeping the project with default port values.
if this is unclear, basically, in the docker-compose.yml you would change the port number before the : to be:
lightspeed-ingest:
    restart: on-failure
    build: ingest/
    network_mode: host
    ports:
        - "${LIGHTSPEED_PORT}:8084"

oh, additionally, you can also modify the CMD for a container in the docker-compose.yml so, for the WebRTC container, you could do:
command: ["lightspeed-webrtc", "--addr=${IP_ADDRESS}"]

Hmmmmm, looking deeper,  looks like everything could be setup as a runtime variable configured in docker-compose. The only thing standing out is: wsUrl.js
but this file could have an expression which checks for an ENV, and if that isn't set, sets to default

Hmmmmm, looking deeper,  looks like everything could be setup as a runtime variable configured in docker-compose. The only thing standing out is: wsUrl.js
but this file could have an expression which checks for an ENV, and if that isn't set, sets to default

Hmmmmm, looking deeper, looks like everything could be setup as a runtime variable configured in docker-compose. The only thing standing out is: wsUrl.js
but this file could have an expression which checks for an ENV, and if that isn't set, sets to default

wsURL.js is no longer used. Instead there is a config.json file in the build folder

Hmmmmm, looking deeper, looks like everything could be setup as a runtime variable configured in docker-compose. The only thing standing out is: wsUrl.js
but this file could have an expression which checks for an ENV, and if that isn't set, sets to default

wsURL.js is no longer used. Instead there is a config.json file in the build folder

How about a shell entrypoint. Just make it sed things from env variables and make it call serve...  at the end. That solves a lot of it I think and is extensible.
Ideally a statically compiled program to do the shell script role AND the HTTP serving (that is, unless you do server side rendering) would be best. But you can do that later.

How about a shell entrypoint. Just make it sed things from env variables and make it call serve...  at the end. That solves a lot of it I think and is extensible.
Ideally a statically compiled program to do the shell script role AND the HTTP serving (that is, unless you do server side rendering) would be best. But you can do that later.

How about a shell entrypoint. Just make it sed things from env variables and make it call serve...  at the end. That solves a lot of it I think and is extensible.
Ideally a statically compiled program to do the shell script role AND the HTTP serving (that is, unless you do server side rendering) would be best. But you can do that later.

ENTRYPOINT would likely be the best place for this script to run from.
Checkout the NGINX entrypoint.
The exec "$@" at the end is particularly useful because it allows the CMD to be run after the entrypoint script has finished.

How about a shell entrypoint. Just make it sed things from env variables and make it call serve...  at the end. That solves a lot of it I think and is extensible.
Ideally a statically compiled program to do the shell script role AND the HTTP serving (that is, unless you do server side rendering) would be best. But you can do that later.

ENTRYPOINT would likely be the best place for this script to run from.
Checkout the NGINX entrypoint.
The exec "$@" at the end is particularly useful because it allows the CMD to be run after the entrypoint script has finished.

How about a shell entrypoint. Just make it sed things from env variables and make it call serve...  at the end. That solves a lot of it I think and is extensible.
Ideally a statically compiled program to do the shell script role AND the HTTP serving (that is, unless you do server side rendering) would be best. But you can do that later.

While that is a quick fix we will be dealing with a lot of configuration values in the future so I want to be future oriented with this decision

How about a shell entrypoint. Just make it sed things from env variables and make it call serve...  at the end. That solves a lot of it I think and is extensible.
Ideally a statically compiled program to do the shell script role AND the HTTP serving (that is, unless you do server side rendering) would be best. But you can do that later.

While that is a quick fix we will be dealing with a lot of configuration values in the future so I want to be future oriented with this decision

ENTRYPOINT would likely be the best place for this script to run from.

Yes plus you could convert environment variables to flags to serve as well for convenience.

While that is a quick fix we will be dealing with a lot of configuration values in the future so I want to be future oriented with this decision

A shell entrypoint is quick to do but would be the best way I'd say, wether it's common practice or for ease of use for end users. The entrypoint can always be changed later to something more robust like rust or go without breaking changes.

ENTRYPOINT would likely be the best place for this script to run from.

Yes plus you could convert environment variables to flags to serve as well for convenience.

While that is a quick fix we will be dealing with a lot of configuration values in the future so I want to be future oriented with this decision

A shell entrypoint is quick to do but would be the best way I'd say, wether it's common practice or for ease of use for end users. The entrypoint can always be changed later to something more robust like rust or go without breaking changes.

I opened https://github.com/GRVYDEV/Lightspeed-react/pull/10 with such entrypoint and a few tiny Dockerfile improvements. There is still a few questions to iron out and documentation to add though. Anyone, feel free to criticize 😉

I opened https://github.com/GRVYDEV/Lightspeed-react/pull/10 with such entrypoint and a few tiny Dockerfile improvements. There is still a few questions to iron out and documentation to add though. Anyone, feel free to criticize 😉

I opened GRVYDEV/Lightspeed-react#10 with such entrypoint and a few tiny Dockerfile improvements. There is still a few questions to iron out and documentation to add though. Anyone, feel free to criticize wink

I am a Docker noob so I may be missing the point of the entrypoint script but this looks like I should be able to pass it a WEBSOCKET_HOST env var and that will automatically replace it?

I opened GRVYDEV/Lightspeed-react#10 with such entrypoint and a few tiny Dockerfile improvements. There is still a few questions to iron out and documentation to add though. Anyone, feel free to criticize wink

I am a Docker noob so I may be missing the point of the entrypoint script but this looks like I should be able to pass it a WEBSOCKET_HOST env var and that will automatically replace it?

You need to replace the value at runtime, not build time. That way the user doesn't have to build the image himself. To replace it you need a server side entrypoint of some sort, like a shell script. Although for now there is no HTTP server in the image, I think it should be part of the image and entrypoint.

After having prepared Docker images we should also think of a K8S HELM package for Kubernetes (K8S).

For the config file, can we consider TOML? It's so much easier to read and work with.

https://toml.io/

For the config file, can we consider TOML? It's so much easier to read and work with.

https://toml.io/

For the config file, can we consider TOML

Agreed, but the configuration is in JSON for the React project (native format). And I'd think environment variables are easier to use than a configuration file at least for Docker (except for secrets where we should use files).

For the config file, can we consider TOML

Agreed, but the configuration is in JSON for the React project (native format). And I'd think environment variables are easier to use than a configuration file at least for Docker (except for secrets where we should use files).

Closed with #34


[Deque][NFC] Remove unused code

Replace this paragraph with a description of your changes and rationale. Provide links to an existing issue or external references/discussions, if appropriate.
Checklist

[X] I've read the Contribution Guidelines
[X] My contributions are licensed under the Swift license.
[X] I've followed the coding style of the rest of the project.
[ ] I've added tests covering all new code paths my change adds to the project (if appropriate).
[ ] I've added benchmarks covering new functionality (if appropriate).
[X] I've verified that my change does not break any existing tests or introduce unexplained benchmark regressions.
[ ] I've updated the documentation if necessary.

@swift-ci test


Added function to partition an IEnumerable based on a given predicate.

Please let me know if you think this is useful and fits with the purpose of the library.

Awesome; thank you :)


Fluor not working in CLion

Fluor not working in CLion: even if I activate the "FN" mode both as default or app-specific, I will always have to press FN + F6, for example. If it can help, there's a detail I want to include: when CMake focus is active, and I activate the FN mode, the keys won't work as Apple keys, but neither as FN keys as I said above. Hope it helps!

Same on Mojave 10.14.6


feat(debezium): rebuild dbz_setup in restart scripts to pick up config changes automatically

@FB-ping @pjhale This should make it easier to deploy changes to debezium and elasticsearch configuration files

Adding @azurebrd and @sweng66 as reviewers


About dialog version update check shows current version

Windows Terminal version
1.18.1421.0
Windows build number
10.0.19045.3031
Other Software
n/a
Steps to reproduce
Launch WT
Click v to open menu, choose About ?
If check for update is showing, click it
Expected Behavior
I would have expected:
Windows Terminal Preview
Version: 1.18.1421.0
An update is available.
Version: 1.18.1462
Actual Behavior
The current version is shown as the update version:
Windows Terminal Preview
Version: 1.18.1421.0
An update is available.
Version: 1.18.1421

Ahhh, heck. I forgot to service this to the 1.18 branch. Thanks :)

So, I figured out what happened! This change made it into the 1.18 servicing pipeline, but got marked as "already cherry picked" (somehow(?)). The servicing script rightly ignored it. I wonder if we can do any validation on those cards...
I wonder. Is this a change that I cherry picked on one of my dev machines and didn't push? What else have I missed?


Removed log print from create issue

I noticed some output in my own script and traced it back to this log.Printf which isn't being used anywhere else in the project. (Great project btw, saved me a lot of time! :))

Thanks @Joukehofman! LGTM


resolve unnormalised coordinate names

Change 'x' and 'y' to unnorm_coord_names['x1'] and unnorm_coord_names['x2'] throughout model.py so code is agnostic to the original coordinate names.

Thanks @MartinSJRogers - can I confirm please, does this PR do the same job as @nilsleh's most recent commit on the patchwise_train branch? 8da48c1
Should we revert that commit and replace it with the commits from this branch?

Yeah Martin and I fixed that separately and now realized. Feel free to follow through with your proposed solution @davidwilby

Hey @nilsleh, please merge this PR if happy. I will revert to orig_x1_name etc in due course. I will then send a seperate commit to resolve the monotonic coordinates error next week.


Support for maximum number of inflight workflow executions

Cadence client can provide support for steadily make progress on workflows in order when large number of such workflows are started.
The scenario is to start 1 million workflows and have max number of active workflows (like 1000) per worker.  This means a worker picks up decision task for 1000 workflows and stops polling on the global worker tasklist until it has space for more after completion of any of the 1000 sticky workflows.
This would give the semantics of processing workflows in order to churn through the backlog of such large number of workflows.

@yycptt is this part of your current work with sessions?

No, I don't think so. I can limit the number of current sessions, but that's on session level, not workflow level.


Low-level takeEvery blocks other sagas

Hi,
While implementing CRUD operations I noticed that this code works perfectly:
function* rootSaga() {
  yield takeEvery(INSERT_DATA, insertDataSaga)
  yield takeEvery(FETCH_DATA, fetchDataSaga)
  yield takeEvery(UPDATE_DATA, updateDataSaga)
  yield takeEvery(REMOVE_DATA, removeDataSaga)
}

but when I try to replace the second takeEvery effect with my own implementation, taken from the docs:
function* myOwnTakeEvery(pattern, saga, ...args) {
  const task = yield fork(function*() {
    while (true) {
      const action = yield take(pattern)
      yield fork(saga, ...args.concat(action))
    }
  })
  return task
}

function* rootSaga() {
  yield takeEvery(INSERT_DATA, insertDataSaga)
  yield myOwnTakeEvery(FETCH_DATA, fetchDataSaga)
  yield takeEvery(UPDATE_DATA, updateDataSaga)
  yield takeEvery(REMOVE_DATA, removeDataSaga)
}

then the first two sagas are called as always but the second two aren't anymore.
Am I wrongly using the low level example? I'm using<EMAIL_ADDRESS>Antony

Actually takeEvery implementation is closer to:
const takeEvery = (pattern, saga, ...args) => fork(function*() {
	while (true) {
		const action = yield take(pattern)
		yield fork(saga, ...args.concat(action))
	}
})

You're absolutely right, this works as expected! Maybe we should update the docs accordingly, would you like me to do it?

Sure, please do, would be greatly appreciated. Please check also if there are such low-level examples of throttle and takeLatest and update them too if neccessary :)

Incoming PR ;)

@Andarist Fixes to the docs should be pushed in the gh-pages branch to be effectively pubilshed, is there a best practice we should follow?

I just rely on myself to publish them manually because not everything which gets to the master needs to be published asap, sometimes i merge new features etc and cannot publish updated docs for such cases. Feel free to suggest better workflow - would be great to have this automated reliably

By having a look in the package.json I found a docs:publish script. If that's the way you use to publish docs then I have no better suggestions :)

Yeah, task is in place, just need to be ran manually. Gonna do this later tomorrow

For those of you who came here because you didn't realize take every wasn't blocking, here is my quick and dirty implementation of a blocking one.
// this is hard to type, so were just stealing take every types
const blockingTakeEvery: typeof takeEvery = (
  patternOrChannel: any,
  saga: any,
  ...args: any[]
) =>
  call(function* () {
    while (true) {
      const action = yield take(patternOrChannel)
      yield fork(saga, ...args.concat(action))
    }
  }) as any

It would be really convenient to have an official equivalent.

It doesnt seem to make much sense to have a blocking takeEvery - it wont ever unblock, so what's the point? If you don't want to execute any lines following takeEvery then just make it the last effect within your saga

@Andarist
I'll write you a simplified example for how I am using it. In my case, I am using peerjs and channels.
function *peerComSaga(myPeerId: string) {
   // peer will manage multiple connections from multiple peers.
   const peer = new Peer(myPeerId)
   // emits connections whenever there is an incoming connection.
   const connectionChannel = call(createConnectionChannel, peer)

  // ... a few other forks and channels with peer
  try {
    // it's nice that this blocks so that when this saga is cancelled, the cleanup happens. Otherwise we need to do a while true loop
    // with a take and fork.
    yield blockingTakeEvery(connectionChannel, peerDataConnectionSaga)
  } finally {
    peer.destroy()
  }
}


[WIP] Python3.8 support

It's been about a year since the end of support for Pytnon 3.6, so why not change the support version to 3.8?
https://endoflife.date/python

[ ] requirements.txt and setup.py (jira> = 2.0.0 → jira> = 3.0.0)

It seems that you have to remove jira from requirements.txt and setup.py and execute the following command separately
pip install -U git +<EMAIL_ADDRESS>
Once jira 3.0.0 is officially released, you will need to modify requirements.txt and setup.py (jira> = 2.0.0 → jira> = 3.0.0).
Due to bug fixes for Python 3.8
https://pypi.org/project/jira/
https://github.com/pycontribs/jira/releases/tag/3.0

Add support for Python 3.8 (#895)

Once jira 3.0.0 is officially released, you will need to modify requirements.txt and setup.py (jira> = 2.0.0 → jira> = 3.0.0).
https://pypi.org/project/jira/
https://github.com/pycontribs/jira/releases/tag/3.0

Add support for Python 3.8 (#895) @japz

When using Python3.9, blist does not work properly.
It seems that the following pull request needs to be captured
Python 3.9 compatibility: Use PyObject_GC_IsTracked() #91

configparser
Also supports Python 3.9
configparser>=3.5.0
↓
configparser>=5.0.1
https://github.com/jaraco/configparser/blob/master/CHANGES.rst

I want you to capture the following pull request first
Sync requirements.txt and setup.py & update py-zabbix #2818
Pin elasticsearch to 7.0.0 in requirements.txt #2684
Fixing jira version in python3.7 #2673

[ ] requirements.txt and setup.py (jira> = 2.0.0 → jira> = 3.0.0)

Any idea when will 3.0 be officially out?


Add the psp reference as the response code if it doesn't exist.

In a previous PR (#64) we fixed an issue where the redirect didn't have a psp_reference, so couldn't combine with the notification properly. However the PR only fixed the scenario where the redirect was first. This ties up the loose ends when the notification is first, and adds a test for the issue when psp_reference isn't included in the redirect response.

:+1:


feat: add requirements.txt

solved issue #20

Thank you for your contribution!

add level labels @thestuti