daemon https access using jc21/nginx-proxy-manager web ui

Hi, I setup mcsmanager on my home server (Ubuntu Server) and added an entry in nginx proxy manager (docker jc21/nginx-proxy-manager) web ui. someone provided nginx configuration settings on the mcsmanager website, but it's all text based and I can't figure out how to set this up. Can someone help me with this? I created a sub domain for the daemon thinking that would be my work around, but that didn't work. I do have authentik enforced for this web page since it is meant to face the public internet. See my attached pictures, Thanks!

Sorry, I have not used nginx proxy manager (docker jc21/nginx-proxy-manager) before, and I have not found other MCSM users who have used it. I probably can't tell you how to set it up using this web ui. It is recommended that you ask the community help of this image for how to set it up accordingly.

for port 24444, use ws scheme.

I can only use http and https scheme. nginx web ui doesn't support anything else. Maybe I should set it up using self signed certs in my own network, then work from there.

okay, I found a way to make it work. It's not working the way you might expect, but I don't care. I visited mcsmanager website, clicked on https, then reverse proxy. I installed nginx proxy manager on my ubuntu server host and I copy and pasted the code into my nginx conf file. I changed 8081 to 80 and 1082 to 81, then pointed my web ui nginx to that host on port 80. It works now. I visit the website on the internet with the domain name configured and SSL works. I left the daemon at localhost 24444 because I only want to encrypt the web panel.

i have to remind you that you can only connect via wss protocol on https pages.

i have to remind you that you can only connect via wss protocol on https pages.

i have to remind you that you can only connect via wss protocol on https pages.

Yeah, I was able to access the Web panel and checked the Daemon status and it appeared to be working until I clicked on my Minecraft server and realized it isn't working. At this point I think I'll copy my wildcard certificate and host this on a different port using the settings provided by mcsmanager website. I already setup nginx on the host, so I'm half way there.

since nginx-proxy-web-ui doesnt support websocket, i suggest configure nginx yourself with config file.

since nginx-proxy-web-ui doesnt support websocket, i suggest configure nginx yourself with config file.

I noticed the WebSocket setting in the screenshot, it seems to have a WebSocket switch.
MCSM can set HTTPS for the web, and use HTTP for the daemon, and it will work normally.

okay, I found a way to make it work. It's not working the way you might expect, but I don't care. I visited mcsmanager website, clicked on https, then reverse proxy. I installed nginx proxy manager on my ubuntu server host and I copy and pasted the code into my nginx conf file. I changed 8081 to 80 and 1082 to 81, then pointed my web ui nginx to that host on port 80. It works now. I visit the website on the internet with the domain name configured and SSL works. I left the daemon at localhost 24444 because I only want to encrypt the web panel.

The user's browser will exchange data with MCSM-daemon. If HTTPS is used, I personally recommend setting the daemon as well. In addition, after the daemon turns on https, you need to enter wss://example.com:24444 on the web to connect to the backend.
Also, I don’t know how compatible this nginx-proxy-web-ui is. It is recommended to try opening the "terminal" after connecting to the daemon on the web to test whether the browser can link to the daemon.

since nginx-proxy-web-ui doesnt support websocket, i suggest configure nginx yourself with config file.

I noticed the WebSocket setting in the screenshot, it seems to have a WebSocket switch.
MCSM can set HTTPS for the web, and use HTTP for the daemon, and it will work normally.

okay, I found a way to make it work. It's not working the way you might expect, but I don't care. I visited mcsmanager website, clicked on https, then reverse proxy. I installed nginx proxy manager on my ubuntu server host and I copy and pasted the code into my nginx conf file. I changed 8081 to 80 and 1082 to 81, then pointed my web ui nginx to that host on port 80. It works now. I visit the website on the internet with the domain name configured and SSL works. I left the daemon at localhost 24444 because I only want to encrypt the web panel.

The user's browser will exchange data with MCSM-daemon. If HTTPS is used, I personally recommend setting the daemon as well. In addition, after the daemon turns on https, you need to enter wss://example.com:24444 on the web to connect to the backend. Also, I don’t know how compatible this nginx-proxy-web-ui is. It is recommended to try opening the "terminal" after connecting to the daemon on the web to test whether the browser can link to the daemon.

I setup a self signed certificate and used the config file provided as an example on mcsmanager's website. I didn't get good results. I changed the daemon to wss://localhost:24444 and nginx said it's an invalid url prefix. I have moved to nginx on the host machine, this is cli only.
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
}
http {
# Limit file upload size to 100G
client_max_body_size 100g;
server {
    # Web panel public access port
    listen 8081 ssl; # Add ssl option here

    ssl_certificate /etc/nginx/self-signed.crt; # Path to your SSL certificate
    ssl_certificate_key /etc/nginx/self-signed.key; # Path to your SSL private key

    location / {
        # Local address of web panel
        proxy_pass http://localhost:23333/;
        root html;
        index index.html index.htm;
        # Some necessary HTTP Header settings
        proxy_set_header Host localhost;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        # Required for Websocket support
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        add_header X-Cache $upstream_cache_status;
        add_header Cache-Control no-cache;
        expires -1;
    }
}

server {
    # Daemon public access port
    listen 8082 ssl; # Add ssl option here

    ssl_certificate /etc/nginx/self-signed.crt; # Path to your SSL certificate
    ssl_certificate_key /etc/nginx/self-signed.key; # Path to your SSL private key

    location / {
        # Local address for Daemon(s)(the configuration is the same as that of the web side)
        # You can repeat this block for different address & port for multiple daemons.
        proxy_pass wsl://localhost:24444/;
        root html;
        index index.html index.htm;
        proxy_set_header Host localhost;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        add_header X-Cache $upstream_cache_status;
        add_header Cache-Control no-cache;
        expires -1;
    }
}

}

proxy_pass http://localhost:24444/;

plus, you can easily get ssl certificate with acme. i do not support using self-signed certificate.
plus plus, https://proxyformcsm.bddjr.com/english/

proxy_pass http://localhost:24444/;

plus, you can easily get ssl certificate with acme. i do not support using self-signed certificate.
plus plus, https://proxyformcsm.bddjr.com/english/

I switched to a self signed certificate for testing purposes. I use let's encrypt certificates when it's hosted publicly on the Internet with Authentik as my MFA provider. I should try Cloudflare also and see if the tunnel works. I have my doubts. I'll read the link you provided later today after work. Thanks!

proxy_pass http://localhost:24444/;

plus, you can easily get ssl certificate with acme. i do not support using self-signed certificate.
plus plus, https://proxyformcsm.bddjr.com/english/

Thanks for the help, but I still seem to be missing something. I can get the Web p panel, but can't get the Daemon to work.

can your panel connect daemon?
can you connect daemon via http(s) (directly visit it in browser)?
what is your config now?

proxy_pass http://localhost:24444/;

plus, you can easily get ssl certificate with acme. i do not support using self-signed certificate.
plus plus, https://proxyformcsm.bddjr.com/english/

Thanks for the help, but I still seem to be missing something. I can get the Web p panel, but can't get the Daemon to work.

nope, /etc/nginx/nginx.conf is right.

Change this server_name
# If the domain name used cannot match in other servers {}, it will
# match here.
server_name _ mcsm.mc16.duckdns.org, *.mc16.duckdns.org;

to
# If the domain name used cannot match in other servers {}, it will
# match here.
server_name _ ;

This address may not be the directory where the nginx configuration file resides
/etc/nginx/nginx.conf

Where is your nginx configuration directory?

unfortunately that didn't work. I was able to sign into mcsmanager, but I couldn't get the Daemon to load. Now that I am on my laptop, I will paste my config here.
user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
The following http blocks are the ones that need to be understood and
modified, please make changes according to your needs and running
environment. Assumptions:
The port that Daemon truly listens to: 24444 The port that the web
panel truly listens to: 23333 Proxy port: 12333 SSL certificate
directory: /etc/nginx/ssl/domain.com.crt SSL private key directory:
/etc/nginx/ssl/domain.com_ECC.key You need to allow access to the
primary domain domain.com and any of its subdomains
http {
# Configure SSL certificate. The SSL ports monitored below will default
# to using this certificate.
#SSL-START
ssl_certificate "/etc/nginx/ssl/npm-3/cert.pem";
ssl_certificate_key "/etc/nginx/ssl/npm-3/privkey.pem";
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 10m;
ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3; #TLSv1.0
# Allow connections to be established using these encryption methods
ssl_verify_client off;
# Do not verify the client's certificate
#SSL-END Enable gzip compression by default during transmission
gzip on;
gzip_types text/plain text/css application/javascript application/xml application/json;
gzip_proxied any;
# The compression level during transmission, the higher the level, the
# more CPU is consumed by compression, with a maximum of 9 levels.
# Usually, 5 levels are sufficient
gzip_comp_level 5;
# Compression is required only when the transmission size reaches 1k.
# It is meaningless to compress small content
gzip_min_length 1k;
# The server in the response header returns only nginx, not the version
# number.
server_tokens off;
# There is no limit on the file size uploaded by the client
client_max_body_size 0;
server {
# This is used to prevent cross domain access. Proxy port (multiple
# listens can be used to listen to multiple ports)
listen 12333 ssl ;
listen [::]:12333 ssl ; #IPv6
# If the domain name used cannot match in other servers {}, it will
# match here.
server_name _ ;        # should not delete this block, which is more likely to lead to
# certificate leakage, attackers scan the IP address directly bound
# to the source IP and domain name together.
ssl_reject_handshake on;
    # When using HTTP access, disconnect.
    error_page 497 =200 /; location / { return 444;
    }
}
server {
    # After the Daemon proxy, the localhost accesses the HTTP protocol
    # port (multiple listens can be used to listen to multiple ports)
    listen <IP_ADDRESS>:12333 ;
    listen [::1]:12333 ; #IPv6
    # Local loopback domain name
    server_name localhost ;

    # The local loopback address does not take up the bandwidth, so
    # compression is not required.
    gzip off;
    # Start reverse proxy Proxy Daemon
    location / {
        # Enter the port number that the Daemon listens to
        proxy_pass http://localhost:24444 ;
        # Some request headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        # Necessary request headers for WebSocket
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        # Add response header
        add_header X-Cache $upstream_cache_status;
    }
}
server {
    # Proxy public HTTPS port (multiple listens can be used to listen
    # to multiple ports)
    listen 12333 ssl ;
    listen [::]:12333 ssl ; #IPv6
    # The domain name you use when you visit (wildcards are supported,
    # but wildcards cannot be used for root domain names) If the link
    # you are accessing uses a public IP address directly, enter the
    # public IP address here.
    server_name mcsm.mc16.duckdns.org *.mc16.duckdns.org ;
    # HTTP redirect to HTTPS
    error_page 497 https://$host:$server_port$request_uri;
    # Start reverse proxy Proxy Daemon
    location ~ (^/socket.io/)|(^/upload/)|(^/download/) {
        # Fill in the port number that the Daemon end truly listens to,
        # and do not add a slash after it!
        proxy_pass http://localhost:24444 ;
        # Some request headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        # Necessary request headers for WebSocket
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        # Add response header
        add_header X-Cache $upstream_cache_status;
    }
    # Proxy Web Side
    location / {
        # Enter the number of the port monitored by the Web panel
        proxy_pass http://localhost:23333 ;
        # Some request headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        # Necessary request headers for WebSocket
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        # Add response header
        add_header X-Cache $upstream_cache_status;
        # Only allow clients to send cookies using HTTPS

proxy_cookie_flags ~ secure;
        # Within 1 year after client access, HTTP will automatically
        # redirect to HTTPS (invalid after clearing browser cache)
        add_header Strict-Transport-Security "max-age=31536000";
    }
}

}

ouch the weird format,,, you can pase in ubuntu pastebin
i'll check it as soon i get it.
plus, use this to test the config:
nginx -t

plus, in markdown paste your code in ``` block.

ubuntu usually requires nginx to be compiled and installed, so /etc/nginx may not be the real directory

he made nginx work for panel. so i thought it's right. maybe.

Change this server_name
# If the domain name used cannot match in other servers {}, it will
# match here.
server_name _ mcsm.mc16.duckdns.org, *.mc16.duckdns.org;

to
# If the domain name used cannot match in other servers {}, it will
# match here.
server_name _ ;

This address may not be the directory where the nginx configuration file resides
/etc/nginx/nginx.conf

Where is your nginx configuration directory?

unfortunately that didn't work. I was able to sign into mcsmanager, but I couldn't get the Daemon to load. Now that I am on my laptop, I will paste my config here.
user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
The following http blocks are the ones that need to be understood and
modified, please make changes according to your needs and running
environment. Assumptions:
The port that Daemon truly listens to: 24444 The port that the web
panel truly listens to: 23333 Proxy port: 12333 SSL certificate
directory: /etc/nginx/ssl/domain.com.crt SSL private key directory:
/etc/nginx/ssl/domain.com_ECC.key You need to allow access to the
primary domain domain.com and any of its subdomains
http {
# Configure SSL certificate. The SSL ports monitored below will default
# to using this certificate.
#SSL-START
ssl_certificate "/etc/nginx/ssl/npm-3/cert.pem";
ssl_certificate_key "/etc/nginx/ssl/npm-3/privkey.pem";
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 10m;
ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3; #TLSv1.0
# Allow connections to be established using these encryption methods
ssl_verify_client off;
# Do not verify the client's certificate
#SSL-END Enable gzip compression by default during transmission
gzip on;
gzip_types text/plain text/css application/javascript application/xml application/json;
gzip_proxied any;
# The compression level during transmission, the higher the level, the
# more CPU is consumed by compression, with a maximum of 9 levels.
# Usually, 5 levels are sufficient
gzip_comp_level 5;
# Compression is required only when the transmission size reaches 1k.
# It is meaningless to compress small content
gzip_min_length 1k;
# The server in the response header returns only nginx, not the version
# number.
server_tokens off;
# There is no limit on the file size uploaded by the client
client_max_body_size 0;
server {
# This is used to prevent cross domain access. Proxy port (multiple
# listens can be used to listen to multiple ports)
listen 12333 ssl ;
listen [::]:12333 ssl ; #IPv6
# If the domain name used cannot match in other servers {}, it will
# match here.
server_name _ ;        # should not delete this block, which is more likely to lead to
# certificate leakage, attackers scan the IP address directly bound
# to the source IP and domain name together.
ssl_reject_handshake on;
    # When using HTTP access, disconnect.
    error_page 497 =200 /; location / { return 444;
    }
}
server {
    # After the Daemon proxy, the localhost accesses the HTTP protocol
    # port (multiple listens can be used to listen to multiple ports)
    listen <IP_ADDRESS>:12333 ;
    listen [::1]:12333 ; #IPv6
    # Local loopback domain name
    server_name localhost ;

    # The local loopback address does not take up the bandwidth, so
    # compression is not required.
    gzip off;
    # Start reverse proxy Proxy Daemon
    location / {
        # Enter the port number that the Daemon listens to
        proxy_pass http://localhost:24444 ;
        # Some request headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        # Necessary request headers for WebSocket
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        # Add response header
        add_header X-Cache $upstream_cache_status;
    }
}
server {
    # Proxy public HTTPS port (multiple listens can be used to listen
    # to multiple ports)
    listen 12333 ssl ;
    listen [::]:12333 ssl ; #IPv6
    # The domain name you use when you visit (wildcards are supported,
    # but wildcards cannot be used for root domain names) If the link
    # you are accessing uses a public IP address directly, enter the
    # public IP address here.
    server_name mcsm.mc16.duckdns.org *.mc16.duckdns.org ;
    # HTTP redirect to HTTPS
    error_page 497 https://$host:$server_port$request_uri;
    # Start reverse proxy Proxy Daemon
    location ~ (^/socket.io/)|(^/upload/)|(^/download/) {
        # Fill in the port number that the Daemon end truly listens to,
        # and do not add a slash after it!
        proxy_pass http://localhost:24444 ;
        # Some request headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        # Necessary request headers for WebSocket
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        # Add response header
        add_header X-Cache $upstream_cache_status;
    }
    # Proxy Web Side
    location / {
        # Enter the number of the port monitored by the Web panel
        proxy_pass http://localhost:23333 ;
        # Some request headers
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header REMOTE-HOST $remote_addr;
        # Necessary request headers for WebSocket
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        # Add response header
        add_header X-Cache $upstream_cache_status;
        # Only allow clients to send cookies using HTTPS

proxy_cookie_flags ~ secure;
        # Within 1 year after client access, HTTP will automatically
        # redirect to HTTPS (invalid after clearing browser cache)
        add_header Strict-Transport-Security "max-age=31536000";
    }
}

}

ouch the weird format,,, you can pase in ubuntu pastebin
i'll check it as soon i get it.
plus, use this to test the config:
nginx -t

plus, in markdown paste your code in ``` block.

I pasted the text and I had to type nginx -t to get this config file to work. nginx failed a few times, so I commented out the lines.

i just wanna see what nginx -t output.

my timezone (same for bddjr) UTC+8, 9 a.m. to 11 p.m.. better to select a time both ok for us all, to fix your issue quicker.

here is the output for sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
here is the link for pastebin I made it last for a week
https://paste.ubuntu.com/p/sMVbZhJRp6/

what do you see if you visit it(daemon)?

my advice: remove non-ssl server block, and try again. better to remove all but the one with domain.


Feature Request: disable HSTS preloading/Subdomaining

https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/
specifically, add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

I've added this feature!
Thanks!


readFrom() return BadData for 4-byte buffer

In [0] there is the check of the variable byte integer buffer length
uint32 readFrom(const uint8 * buffer, uint32 bufLength)
{
    for (size = 0; size < 4;)
    {
        if ((uint32)(size+1) > bufLength) return NotEnoughData;
        value[size] = buffer[size];
        if (value[size++] < 0x80) break;
    }
    return size < 4 ? size : (uint32)BadData;
}

for buffer = {0x8c, 0xda, 0xc4, 0x09} and bufLength = 4 this will result in BadData but it should be a valid input.
Is the return condition correct?
[0] https://github.com/X-Ryl669/eMQTT5/blob/2f261615d3c2adc5e0c273876cfb35a0c7304b95/lib/include/Protocol/MQTT/MQTT.hpp#L692C31-L692C31

You're right, the condition is too restricted.
I believe the last commit should have fixed it. Please confirm it's ok for you.

yes, now it seems to be ok. thanks


Adds new resources to ActiveAdmin for Scholarship and ScholarshipApplication

Feature
Why is this feature being added?

Gives admins the ability to conduct CRUD actions on the Scholarship and ScholarshipApplication models.  Currently this has to be done through prod's rails console.
What should your feature do?
Here are the ActiveAdmin docs:  https://activeadmin.info/documentation.html
Here is a sample implementation of a Resource in our codebase:  https://github.com/OperationCode/operationcode_backend/blob/master/app/admin/team_member.rb

[ ] Adds a Resource for Scholarship to ActiveAdmin
[ ] Adds a Resource for ScholarshipApplication to ActiveAdmin
[ ] Updates ability.rb with the proper permissions - confirm with @hpjaj
[ ] These two models have a has_many/belongs_to relationship with one another.  Be sure to take that into account.  We have an existing implementation for CodeSchool and Location that can be used for reference.
[ ] Create three AdminUsers. One with each role: super admin, admin, and board member. And then login and out as each, testing access. This is all done in your local test env (i.e. localhost). The seeds file can help with this.

Can I do this?

@nagano564 - It's yours!


Reset button on advanced filters doesn't work

Describe the bug
DSpace version: 6.x
Theme: Mirage2
When advanced filters are set and user wants to restore filters form, reset button doesn't execute an action.
To Reproduce
Steps to reproduce the behavior:

Go to /discover.
Set an advance filter like title (at least one filter).
Click on "Apply" button.
After results has been shown, click "Reset" button. You will notice that this button has no effect.

Expected behavior
After results are shown, clicking "Reset" button should restore advanced filters form to its original empty state.
Related work

It is possible also affects DSpace 5.x. I've tried on 5.10 and has same issue.

Just confirmed this issue on our DSpace 6.3 instance as well as demo.dspace.org.


Array different behavior between godot2 and godot3

Operating system or device, Godot version, GPU Model and driver (if graphics related):
Linux, Godot 2.1 / Godot 3 alpha2
Issue description:

I was looking at a project for path finding written with godot 2.1 and getting it to work in my project in godot 3. I was banging my head on the wall until I realized the problem was because of Array.
I encountered this here: https://github.com/ttux/GodotIsoCustomPathfinder3
See https://github.com/Lohan120/GodotIsoCustomPathfinder for the original godot 2 project.
Issue occurs in https://github.com/ttux/GodotIsoCustomPathfinder3/blob/master/scripts/pathfinder.gd at Line 36
In godot 2 if you add the following, this will print nothing (after doing a right click on one of the tile):
var priority = Array()
for p in priority:
print(p)
In godot 3, if you do the same thing it will print 0. Something fishy here?
var priority = [] in godot 3 will solve the problem.

So I was told that "in Godot 3.0, arrays are by default references, if you define an array like Array() inside a function that gets called multiple times or in an iteration, it doesn't reset the array like in 2.1.4"
c.f. https://github.com/Lohan120/GodotIsoCustomPathfinder/issues/2


CheckNxModuleBoundaries target fails on project root with a space in it

Describe the bug
building a project with a space in the project root path fails because when a --project-root argument is passed to check-module-boundaries.js with a space the process.argv array has only the first part of the path.  See https://github.com/nx-dotnet/nx-dotnet/blob/master/packages/core/src/tasks/check-module-boundaries.ts#L91
To Reproduce
Steps to reproduce the behavior:

make a project with a space in the path
call npx nx run project:build

Expected behavior
The task runs without problems
Screenshots
Environment:
nx 15.0.0
@nx-dotnet/core: 1.15.0
windows 11
node 16.17.1

I made a pr that avoids the issue https://github.com/nx-dotnet/nx-dotnet/pull/551 not sure if it should not be handled in check-modules-boundaries.js as well.

I think this should be fixed by #551, I'm going to go ahead and close this out.


Update css bootstrap links in examples

That's good, i would prefer it if you kept the // instead of https:// as getting the files can take up to 5x as long, and that's not necessary when surfing without SSL. If you disagree, please do tell.

@Dervisevic Thank you for the remark! I've updated the PR. Protocol-related urls is good solution, but if you open file locally, browser would prefix it with "file://" protocol. As I understand these examples were created mostly for use on github pages, but at the same time I haven't found custom-validators page there?

@Minisai It looks like the custom-validators page was just created within the past week. I'm sure @davidlgj or @Dervisevic will get it up on the website soon.
I agree for protocol-less URLs :+1:

@Minisai we should probably add some server to the gulpfile to make it easier to test out the examples. I usually use http-server or puer. The examples also play a second role to ease manual testing.


Integrate with PagerDuty for Alerts

Integrate with PagerDuty for Alerts

The PagerDuty trial account has expired, so if we want to use it we'll have to re-up. New Relic has basic alerting in the form of e-mail and we could use an e-mail to SMS gateway if desired but it's not as sophisticated as PD.

done


Update Benchmarks

This PR also adds "overhead" on top of the jet cost.  This overhead isn't included in the CMR commitment, but is added on during the total cost calculation.
rust-simplicity needs to be updated with this change in overhead computation.

I didn't test this against the Rust code -- in fact we know that it will break a bunch of our unit tests because the fixed merkle root targets will have changed.


api endpoints for credentials

#1444 is going to need several new API endpoints that implement CRUD for credentials.
Given #1444's high priority, this is as well.

https://github.com/akuity/kargo/issues/1531 will need to use this too

This was closed by #1566


Why does Uptime Kuma always connect to SQLite instead of MariaDB?

⚠ Please verify that this question has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

📝 Describe your problem
Hi,
I’m encountering an issue with the Uptime Kuma setup. Even though I’ve correctly configured the .env file to use MariaDB as the database, it always connects to SQLite instead of MariaDB, which I’ve specified.
📝 Error Message(s) or Log
Details of the issue:
Operating system: Ubuntu on a virtual machine (VM).
Uptime Kuma version: [specify the version if known].
Configured database: MariaDB.
.env file configuration:
env
Copiar código
DB_TYPE=mariadb
DB_HOST=localhost
DB_PORT=3306
DB_USER=uptimekuma
DB_PASSWORD=Qp1*z-Y.m_1#z8M.+g4H6
DB_NAME=uptimekuma
Logs: Every time I start Uptime Kuma, the logs show that it is connecting to SQLite instead of MariaDB:
shell
Copiar código
2024-10-08 18:23:45 [Info] : Starting Uptime Kuma with database type: sqlite
2024-10-08 18:23:45 [Info] : Database file: /app/data/data.sqlite
What I’ve tried:
Verify that MariaDB is running properly on the virtual machine.
Ensure that the .env file has the correct settings for MariaDB.
Restart the application multiple times, but it still keeps connecting to SQLite.
Check logs and configuration, but couldn’t find any indication of the issue.
My question is: Is there any reason why Uptime Kuma always uses SQLite instead of MariaDB, even after correctly configuring the .env file?
🐻 Uptime-Kuma Version
1.23.14-782-g59e70cb7
💻 Operating System and Arch
ubunto
🌐 Browser
google
🖥 Deployment Environment

Runtime:
Database:
Filesystem used to store the database on:
number of monitors:

@juanpablocaro7monroy please see #4500 Mariadb is a v2.0 feature


cephfs: refactor cephfs core functions

This commits refactors the cephfs core functions with interfaces.
Signed-off-by: Madhu Rajanna<EMAIL_ADDRESS>

Quite a large PR, but most things look trivial. Reviewing and adding unit-tests would be easier when it is split in a few commits.

@nixpanic I thought to make multiple commits. as most of the things are interconnected I kept it as a single commit for easy rebase and cherry-picking if required. Hope should be fine.

@ceph/ceph-csi-contributors anyone else who can help with review/approval?

@mergifyio rebase

@Mergifyio rebase

@mergifyio rebase

/retest all

/retest all

CI seems stuck restarting all tests


Heads up: User just edited the Wiki home page to link to malware

When I accidentally clicked on the link that @FrankFeng added in this commit, Firefox warned me that it was malware. I've reported this user to GitHub.

I did undo the change.

GitHub appears to have banned the user. I'll close this, but consider this a reminder to keep an eye on wiki edits. It's too bad GitHub wikis don't support pull requests.

Thank you very much!!

I've reported the user and reverted the change.

Worst case scenario we could restrict wiki editing to collaborators only for now but unfortunately I think only the repo owner can do this, not collaborators.

@TooTallNate could you please change this? Thx

Not much I can do here. Closing issue for now


Show tooltip immediately if the tooltip trigger is manual

I think tooltips should display immediately for trigger: 'manual' (currently, setting that trigger type has no effect).
The Bootstrap tooltip has a trigger option that's click | hover | focus | manual http://getbootstrap.com/javascript/#tooltips-options For 'manual' you need to call show() on the tooltip to get it to display, but the current Atom tooltip API doesn't expose that method. This makes it hard to get a tooltip to show immediately (e.g. you'd have to set trigger: 'click' and dispatch a click event on the target element).

Closed via https://github.com/atom/atom/pull/11063.


Turn Stdlib methods into static methods

Attempt 2 because rebasing the squashed commit didn't work

The CI job found a couple of places where stdlib was still lower case; I pushed a quick fix for those. Fingers crossed.

Fortune favors the bold!


Remove create agent via console option

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

is this still relevant @mttanke ?

is this still relevant @mttanke ?

yes, this addresses the issues we are having with the agents lab at the moment


Concept redirects

Removed REST concept topics for webhooks and recurrence and redirected to Graph site.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

docs/rest/recurrence-rules.md
:white_check_mark:Succeeded

docs/rest/toc.yml
:white_check_mark:Succeeded

docs/rest/webhooks.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[Question/Request] More Low-level acces to tensorflow

For my research i need to read out the gradients during backpropagation. However most classes (Sequential) can not be inherited and most  variables are internal or private.
Would it be possible to expose more of the internal api to get optional more low level access to the api and allow for easier extension?
Similar to:
https://github.com/Dongdongshe/neuzz/blob/2c7179557a491266ca1478e5f8c431d0b69d3e3a/nn.py#L267
And:
https://www.tensorflow.org/overview

Hi @stheid, thank you for your question.
At this moment we have plans to make the hierarchy of classes open with the ability to make possible the inheritance for the main building blocks of the KotlinDL like layers, initializers, activation functions, callbacks, metrics, and loss functions, probably we should do it and for Sequential and Functional models (need to think about it)
But due to usage under the hood, the TF runtime 1.15 and limitation of Java API for the TF 1.15 and TF 2.x developed the Eager Mode and Gradient Tape are not supported and you will not have access to the gradients like in Python
Hope it will be possible in the future (and some works are in progress, for example, registration of the custom gradients) and KotlinDL will be useful for scientific research in the future.

Thanks for the informative reply. I think that means that KotlinDL is of the table for now, but i will keep a close eye on the development and maybe use it in the future ;)


ttyd not working with terminal running in luci window.

Hi I cant seem to get ttyd to work in openwrt when using https.
If i setup luci with plain http the terminal in web page works perfectly.
do you have any pointers ?

please report it to the luci repository.


[* DateTimeV2] Duration patterns wrongly normalized/resolved when number is missing

A pattern like "within the next day and 5 hours" returns a duration of PT5H  which does not take into account the "next day" part of the expression.
The pattern should return a duration of P1DT5H as it happens when the number of days is explicitly given as in "within the next 1 day and 5 hours".

working on this


How to generate hover pseduo-classes?

Suppose 'sprite' has two icons: normal.png and hover.png.
I want to generate a class .sprite-normal which uses normal.png as background in normal status, but uses hover.png as background when hovered.
Right now I can achieve this by renaming hover.png to normal:hover.png. Is this the intended way to support hover?  It feels like a hack.

That is not possible with the style processors that currently available.

If you want to use inline styles with react you can automate the process pretty easily.  Check out this example from sprity-js https://github.com/gregthebusker/sprity-js#programmatic-checking

Also you can check this solution https://github.com/sprity/sprity/issues/65


Microsoft states that the CQRS pattern isn't recommended in situations where the domain or the business rules are simple

I am not sure if that app is the best real-world example of a ASP.NET Core app.
Microsoft states that this pattern isn't recommended in situations where the domain or the business rules are simple, a simple CRUD-style user interface and the related data access operations are sufficient and/or for implementation across the whole system.

There are specific components of an overall data management scenario where CQRS can be useful, but it can add considerable and unnecessary complexity when it isn't required."

https://docs.microsoft.com/en-us/azure/architecture/patterns/cqrs

Microsoft guidance isn't the gospel truth.  There's a wide community of .NET developers that explore things outside of the Microsoft box.  CQRS doesn't mean anything other than having separate pathways for commands vs queries.  There is usually a big distinction in what they do.
If you're looking for a Repository pattern with horizontal layers and unnecessary duplication then this isn't it.  The vertical slices for operations allows for a more concise code base.  If complexity grows then refactoring can group functionality and common code together.
The references on the README try to explain further.

I have spent the last few days pouring over all articles on the README, articles related to Onion architecture, building my own prototype apps to learn some nuances of each and I can see both having their own place.

horizontal layers and unnecessary duplication

The repository pattern allows you nice isolation of your business logic in your services for test-ability. Anyone who has ever had to talk to SQL, Hbase, and Dynamo will tell you that they are probably thankful all they had to do was re-write their repository layer. All that to say I think each pattern really depends on your use-case and the complexity of your application.
I think this RealWorld repository is inherently flawed in the fact that it says "this is the best way to solve all problems". I like the showcase of things like adding Swagger, using cors, what libraries to use and how to abstract certain things away. Also popular libraries like these tend to get some experienced eyes on them which always makes me feel better when I read and use examples from code that I know has been proofed before. Even though I have recently added an issue about a lack of comments in this code base.
My original comment was just to warn people that this library uses a very opinionated pattern of problem solving and that its not the hammer for all nails.

The repository pattern allows you nice isolation of your business logic in your services for test-ability. Anyone who has ever had to talk to SQL, Hbase, and Dynamo will tell you that they are probably thankful all they had to do was re-write their repository layer. All that to say I think each pattern really depends on your use-case and the complexity of your application.

Repostory doesn't give you anything but an unneeded layer of abstraction when what you want is something specific like the Query classes do.  A quick google for repository anti pattern finds a lot: https://www.infoworld.com/article/3117713/application-development/design-patterns-that-i-often-avoid-repository-pattern.html
This definitely does not say "best way to solve all problems" but it is a demo of more modern practices than things like the Repository pattern.  It is definitely not a one-size-fits-all.
Every strategy is opinionated and tries to guide users down a path.  Everything should be examined.  As I've said, this is a demo of where a lot of the mindshare of some things that the community does outside of the Microsoft sanctioned practices.

I created a fork without MediatR, just in case someone wants to compare it to the original.


Custom GF graphics become a black box or crash to desktop and picking Restart Song also causes a crash to desktop

Describe your bug here.
When using the build that is current as of me typing this (0.5.1), when changing the GF's graphics and XML files in "mods/images/characters" and then picking a song, she will spawn as a black box in the game, but if you exit back to the menu and pick another song, the game will just crash to desktop after a split second of freezing and saying "(Not Responding)". Also, when choosing "Restart Song" from the pause menu of a song, the game just immediately crashes to desktop of a little bit of the freezing mentioned previously.
Command Prompt/Terminal logs (if existing)
No response
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows
Did you edit anything in this build? If so, mention or summarize your changes.
Only made a custom GF graphic and xml that were placed into "mods/images/characters"- nothing else was modified.

Show the character files before reporting, this sounds like a PC issue anyways

Again
Show the files

she will spawn as a black

I'm pretty sure that like frantastic said, this is an issue with your PC, the game probably needs more allocated RAM.
Happens to me too sometimes.


Swift 4 supported ?

Hello, Is it available for Swift 4 version. Can you please update to latese xcode version.

Yes Swift 4 is supported as it is stated in README.


Notebook on Stereoisomer and Tautomer generation

Discussed in https://github.com/PatWalters/practical_cheminformatics_tutorials/discussions/9

Originally posted by sitanshubhunia October 26, 2023
Thanks for your helpful resources @PatWalters . It's helped me a lot to learn. Requesting you to make a useful notebook on how to generate  Stereoisomer and Tautomer . That would be very helpful to me.

@PatWalters Thanks for your time and kind attention. Eagerly waiting for the Notebook.

Added a new tutorial.
https://colab.research.google.com/github/PatWalters/practical_cheminformatics_tutorials/blob/main/fundamentals/stereo_and_tautomers.ipynb


test.python.visualization.test_visualization.TestLatexSourceGenerator should test against references

The tests in test.python.visualization.test_visualization.TestLatexSourceGenerator should check against a file reference.

Does something like this fix the issue? If so I will work on replicating something similar for all the other LatexSource tests. Also, would it be better if I defined the expected content in an external file rather than a variable in every test, or is this fine?
    def test_tiny_circuit(self):
        """Test draw tiny circuit."""
        expected_content = "\\begin{equation*}\\Qcircuit@C=1.0em@R=0.0em@!R{\\lstick{{q}_{0}:\\ket{0}}&\\gate{H}&\\qw&\\qw\\\\}\\end{equation*}\\end{document}"
        ...
        try:
            ...
            with open(filename,'r') as f:
                output = f.read()
                exported_content = re.sub(r'\s+', '', output.split("\\begin{document}")[1])
                self.assertEqual(exported_content, expected_content)

I think I (kinda accidentally) closed this one with #5062.


Move from Scheduled Queries to Materialized Views

Requirement:

All dashboard tables stored in Materialized Views
Scripts managed in version control
Use terraform to create instead of bash

There's a branch for this: bigquery_views, but it's blocked on #155

We're actually going with regular (non-materialized) views... this change is included in PR #160


Theme inheritance (nesting) not working for Typography but working fine for color palette

Steps to reproduce
Link to live example: https://codesandbox.io/p/sandbox/y8crhq?file=%2Fsrc%2Fcomponents%2FChild.tsx%3A20%2C50
Steps:

Install the latest MUI in a basic React project.
Set up a global (App-wide) Mui theme using ThemeProvider.
Now somewhere in a child component overwrite the theme using:

<ThemeProvider<Theme> theme={(parentTheme) => ({...parentTheme})}>
...
</ThemeProvider>

Current behavior
When we modify the inherited theme, the palette colors work but none of the Typography changes work.
Expected behavior
Typography fontFamily and fontSize etc should also be updated for the child components.
I don’t want to call createTheme() again but instead the <ThemeProvider<Theme> theme={(parentTheme) => ({...parentTheme})}> method should work properly.
Context
App.tsx
    <ThemeProvider
      theme={createTheme({
        palette: {
          primary: {
            main: teal[500],
          },
        },
        typography: {
          // fontSize: 20,
          fontFamily: "Sans",
        },
      })}
    >
      <CssBaseline />
      <Parent />
    </ThemeProvider>

Parent.tsx
    <>
      <Typography>Parent Typography (Normal Sans font)</Typography>
      <Button variant="contained" color="primary">
        Parent Primary
      </Button>
      <Button variant="contained" color="secondary">
        Parent Secondary
      </Button>
      <Child />
    </>

Child.tsx
    <ThemeProvider<Theme>
      theme={(parentTheme) => ({
        ...parentTheme,
        palette: {
          ...parentTheme.palette,
          primary: {
            ...parentTheme.palette.primary,
            main: pink[500], // This is working 👏
          },
        },
        typography: {
          ...parentTheme.typography,
          fontSize: 20, // This is not working!!!
          fontFamily: "Arial", // This is not working either!
        },
      })}
    >
      <Typography>Child Typography (Should be bigger, Arial font)</Typography>
      <Button variant="contained" color="primary">
        Child Primary
      </Button>
      <Button variant="contained" color="secondary">
        Child Secondary
      </Button>
    </ThemeProvider>

Some references from the docs:
https://mui.com/material-ui/customization/theming/#nesting-the-theme
https://mui.com/system/styles/advanced/#theme-nesting
Your environment

npx @mui/envinfo
  System:
    OS: macOS 15.0.1
  Binaries:
    Node: 22.6.0 - /usr/local/bin/node
    npm: 9.5.0 - /usr/local/bin/npm
    pnpm: Not Found
  Browsers:
    Chrome: 129.0.6668.90
    Edge: Not Found
    Safari: 18.0.1

Search keywords: theme-inheritance theme-nesting

Merging the themes as in your code-snippet only replaces some values in the theme, however, if you want the new theme to be processed, you need to invoke the createTheme util. When invoking the theme inside createTheme it will make sure that the value you provide in fontSize is going to be applied for all variants. You can also use the deepmerge util to make sure the two themes are merged correctly, check https://codesandbox.io/p/sandbox/mui-nested-theme-forked-8p4kv7?file=%2Fsrc%2Fcomponents%2FChild.tsx%3A16%2C20&workspaceId=02ae2942-9632-4b06-9a20-6ccadc8dcb97


Exclude all spec blocks from Metrics/BlockLength

This was something that was originally discussed in https://github.com/rubocop/rubocop-rspec/issues/988
I'll probably need some more support from @pirj to just confirm I've done everything right (I think I may have missed some things)
It's likely some other maintainers may want to discuss this (As I couldn't spot any other examples of non-rspec cops in the default.yml

Before submitting the PR make sure the following are checked:

[x] Feature branch is up-to-date with master (if not - rebase it).
[ ] Squashed related commits together.
[-] Added tests.
[-] Updated documentation.
[x] Added an entry to the CHANGELOG.md if the new code introduces user-observable changes.
[x] The build (bundle exec rake) passes (be sure to run this locally, since it may produce updated documentation that you will need to commit).

As I couldn't spot any other examples of non-rspec cops in the default.yml

rubocop-rails does a similar thing.

Hi there. I've not been monitoring this as I've been off for a while.
Thanks for jumping on this. Evidently it was a bit bigger haha :D

This I believe is now rtm @pirj - Thanks for all your help fixing up this!

The build is green after merging in a fix for a couple of internal affairs offences.
@bquorning @Darhazer I believe this small change is such a relief for our users who previously had to exclude /spec/* from Metrics/BlockLength in their .rubocop.yml on every new project.

@luke-hill Thanks!


Change the data source of the GeoPage to be a database class

Judging from yesterday's experience on CodeNight, it does not make sense to use something that is NOT a database class as the primary data source for a page in the KitchenSink.
Therefore I think GeoPage needs to be changed. Maybe something else too.

@mnosek can you do this? @aloscha can help

I've created a pull request, please review it.

Pull request accepted and merged


Phone auth with backend verification

As mentioned here: https://github.com/kreait/firebase-php/issues/293#issuecomment-635334717,
it is useful in some cases to have phone auth verification to the backend.

Firebase doesn't provide the means to do phone auth verification on the backend. If there's a solution, it is out of the scope of this SDK.
I've written a lengthy comment on this exact topic on StackOverflow - it's from March 2019, but I still stand by it :)
https://stackoverflow.com/questions/55121166/phone-authentication-validation-from-server-side-firebase-php/55171266#55171266
As soon as one of the official Firebase Admin SDKs provide such a solution, I might reconsider :)


Enable configuration of callback_time periodicity.

I'm missing availabilities because the servers apparently get ordered (perhaps programmatically) in less than the 15 seconds it takes for the crawler to respawn.
I know there's a rate limit set up by OVH, but I believe it's less than 15sec (I think it's 500 calls per hour-long slot).

Great, thanks! By the way I managed to get the server I wanted :)

Cool, happy to hear that!


How to include unobserved confounders in the model?

Hi, thank you for a great package!
I was wondering if there's a possibility to include unobserved confounders in CausalModel.
Under certain conditions back-door and front-door criteria can provide us with correct causal estimands even in face of unobserved confounding (e.g. Pearl, Glymour & Jewell, 2016).
I tried including unobserved confounders by adding these variables to the graph but not including them in the data.
It seems that the model does not return correct estimates in such a case (although the model should theoretically be fully identified).

@AlxndrMlk DoWhy does support unobserved confounders. You are adding them correctly, by not including those variables in the dataframe. The default behavior is that the identification algorithm considers the full graph, but outputs estimands only in terms of the observed variables.
Can you share an example where you are obtaining an incorrect output?

Hi @amit-sharma,
thank you for a quick reply. I created a self-contained example that demonstrates the behavior I described:
from dowhy.causal_model import CausalModel
from sklearn.linear_model import LinearRegression

# Create the graph describing the causal structure
graph = """
graph [
    directed 1

    node [
        id "X"
        label "X"
    ]
    node [
        id "Z"
        label "Z"
    ]
    node [
        id "Y"
        label "Y"
    ]
    node [
        id "U"
        label "U"
    ]

    edge [
        source "X"
        target "Z"
    ]

    edge [
        source "Z"
        target "Y"
    ]

    edge [
        source "U"
        target "Y"
    ]

    edge [
        source "U"
        target "X"
    ]
]
""".replace('\n', '')

N_SAMPLES = 10000

# Generate the data
U = np.random.randn(N_SAMPLES)
X = np.random.randn(N_SAMPLES) + 0.3*U
Z = 0.7*X + 0.3*np.random.randn(N_SAMPLES)
Y = 0.65*Z + 0.2*U

# Data to df
df = pd.DataFrame(np.vstack([X, Z, Y]).T, columns=['X', 'Z', 'Y'])

# Create a model
model = CausalModel(
    data=df,
    treatment='X',
    outcome='Y',
    graph=graph
)

# Get the estimand
estimand = model.identify_effect(proceed_when_unidentifiable=True)
print(estimand)

# Estimand type: nonparametric-ate

# ### Estimand : 1
# Estimand name: backdoor
# Estimand expression:
#  d
# ────(E[Y])
# d[X]
# Estimand assumption 1, Unconfoundedness: If U→{X} and U→Y then P(Y|X,,U) = P(Y|X,)

# ### Estimand : 2
# Estimand name: iv
# No such variable found!

# ### Estimand : 3
# Estimand name: frontdoor
# Estimand expression:
#  ⎡ d       d       ⎤
# E⎢────(Y)⋅────([Z])⎥
#  ⎣d[Z]    d[X]     ⎦
# Estimand assumption 1, Full-mediation: Z intercepts (blocks) all directed paths from X to Y.
# Estimand assumption 2, First-stage-unconfoundedness: If U→{X} and U→{Z} then P(Z|X,U) = P(Z|X)
# Estimand assumption 3, Second-stage-unconfoundedness: If U→{Z} and U→Y then P(Y|Z, X, U) = P(Y|Z, X)

# Estimate the effect with front-door
estimate = model.estimate_effect(
    identified_estimand=estimand,
    method_name='frontdoor.linear_regression'
)

estimate.value
# Out[12] 0.511009176530488

# Compute expected output

# Model P(Z|X)
lr_zx = LinearRegression()
lr_zx.fit(
    X=df['X'].values.reshape(-1, 1),
    y=df['Z']
)

# Model P(Y|X, Z)P(X)
lr_yz = LinearRegression()
lr_yz.fit(
    X=df[['Z', 'X']],
    y=df['Y']
)

# Compute the expected causal effect
lr_zx.coef_ * lr_yz.coef_[0]

# Out[13] array([0.45161212])

# Sanity check -> compute naive estimate
lr_naive = LinearRegression()
lr_naive.fit(
    X=df['X'].values.reshape(-1, 1),
    y=df['Y']
)

lr_naive.coef_

# Out[14] array([0.51100918])

I might be missing something, but there are a couple of things that drew my attention:

I don't understand why back-door criterion appears as a valid criterion for this graph; it seems to me that backdoor cannot help us in this case
It seems that model.estimate_effect(...) returns precisely the naive estimate (vide # Sanity check...) rather than the front-door adjusted estimate

Am I using the correct model for this graph (frontdoor.linear_regression)?
What are your thoughts?
Environment details:
Windows 11
Python 3.8
dowhy==0.6

Ah, you are using an older version of dowhy. Can you update to the latest version v0.8?

When I ran the code, it does not show any valid backdoor variables.
You need to use the two-stage regression estimator for frontdoor. method_name="frontdoor.two_stage_regression".

Estimand type: nonparametric-ate

### Estimand : 1
Estimand name: backdoor
No such variable(s) found!

### Estimand : 2
Estimand name: iv
No such variable(s) found!

### Estimand : 3
Estimand name: frontdoor
Estimand expression:
Expectation(Derivative(Y, [Z])*Derivative([Z], [X]))
Estimand assumption 1, Full-mediation: Z intercepts (blocks) all directed paths from X to Y.
Estimand assumption 2, First-stage-unconfoundedness: If U→{X} and U→{Z} then P(Z|X,U) = P(Z|X)
Estimand assumption 3, Second-stage-unconfoundedness: If U→{Z} and U→Y then P(Y|Z, X, U) = P(Y|Z, X)

### Estimand : 1
Estimand name: frontdoor
Estimand expression:
Expectation(Derivative(Y, [Z])*Derivative([Z], [X]))
Estimand assumption 1, Full-mediation: Z intercepts (blocks) all directed paths from X to Y.
Estimand assumption 2, First-stage-unconfoundedness: If U→{X} and U→{Z} then P(Z|X,U) = P(Z|X)
Estimand assumption 3, Second-stage-unconfoundedness: If U→{Z} and U→Y then P(Y|Z, X, U) = P(Y|Z, X)

## Realized estimand
(b: Z~X)*(b: Y~Z+X)
Target units: ate

## Estimate
Mean value: 0.4599358419804275

Thank you @amit-sharma, I updated to 0.8 and front-door works smoothly!

@amit-sharma points out that in dowhy version 0.8.0, the unobserved confounder - a variable included in the graph but not in the dataset - is properly considered when performing frontdoor.two_stage_regression, and demonstrated that he could obtain the estimate value of 0.46 nearly identical to the actual value of 0.455.
However, when running the same example proposed by @AlxndrMlk myself with the corrected method frontdoor.two_stage_regression  in dowhy version 0.8.0. Once again the estimate.value came out to be 0.51, identical to the naive regression coefficient (which is incorrect) with the following warnings.

WARNING:dowhy.causal_model:The graph defines 4 variables. 3 were found in the dataset and will be analyzed as observed variables. 1 were not found in the dataset and will be analyzed as unobserved variables. The observed variables are: '['X', 'Y', 'Z']'. The unobserved variables are: '['U']'. If this matches your expectations for observations, please continue. If you expected any of the unobserved variables to be in the dataframe, please check for typos.

Therefore, contrary to @amit-sharma's explanation, it seems there is no easy method to include the unobserved confounder in the model. This appears to be more true in the following message as well.

Estimand assumption 1, Full-mediation: Z intercepts (blocks) all directed paths from X to Y.
Estimand assumption 2, First-stage-unconfoundedness: If U→{X} and U→{Z} then P(Z|X,U) = P(Z|X)
Estimand assumption 3, Second-stage-unconfoundedness: If U→{Z} and U→Y then P(Y|Z, X, U) = P(Y|Z, X)

Doesn't the second and third unconfoundedness assumptions imply that the influence of the unobserved confounder, U, has been ignored?
This result is also replicated in dowhy version 0.10.0

Also the output of the following statement is not the same as in @amit-sharma
estimate = model.estimate_effect(
    identified_estimand=estimand,

    method_name="frontdoor.two_stage_regression"
print(estimate)
)

Estimand : 1
Estimand name: frontdoor
Estimand expression:
 ⎡ d       d       ⎤
E⎢────(Y)⋅────([Z])⎥
 ⎣d[Z]    d[X]     ⎦
Estimand assumption 1, Full-mediation: Z intercepts (blocks) all directed paths from X to Y.
Estimand assumption 2, First-stage-unconfoundedness: If U→{X} and U→{Z} then P(Z|X,U) = P(Z|X)
Estimand assumption 3, Second-stage-unconfoundedness: If U→{Z} and U→Y then P(Y|Z, X, U) = P(Y|Z, X)

Realized estimand
(b: Z~X)*(b: Y~Z)
Target units: ate
Estimate
Mean value: 0.5004912411156469

One of the readers of "Causal Inference & Discovery in Python" reported that they had a similar issue with front-door in DoWhy 0.10.0:
Hello Aleksander.
While running chapter 7 notebook, in the front-door case I got -0.33 (naive one) instead of -0.42 (real causal) using dowhy method as you suggest. This does not match with what is written in the book (that I believe is the correct figure -0.42)
But it worked for them correctly in 0.8:
Good morning Aleksander. With version 0.8 it works properly.
Have you tried to replicate the issue over a multiple runs (and datasets), @AnselmJeong?

Thank you @AlxndrMlk and @AnselmJeong for resurfacing this issue. Unfortunately the error creeped up again in 0.10, while it works fine in v0.8.
I have now added a fix through PR #1060 . I have also included @AlxndrMlk example as a test in the library so we never see this bug again in future versions of DoWhy.


prediction result is unconsistent

Description
using pytorch as the backend model, the same input, we run 100 time, but get different result?
in the pytorch model, contain Linear layer and BatchNorm Layer, Does BatchNorm Layer need more processing?

Please let us know if you have any script to reproduce. We will take a look


adding support for multiple client options on connect

Description of your changes
Adding the ability to set multiple ClientOptions as part of the Provider/ProviderConfig CRD.
Adding endpoint and withoutAuthentication options to be able to connect to other APIs except the default one and without authentication.
Renamed GetAuthInfo method to GetConnectionInfo since it will represent better the functionality after that change.
Fixes #432
I have:

[ x] Read and followed Crossplane's contribution process.
[ x] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested
Tested locally using kind - Crossplane and PubSub emulator were installed.
Then I ran provider-gcp locally (make run)  with port forwarding to the cluster API and to PubSub emulator.
Created new Topic resources and saw that the resource created in the emulator (verified by REST request to emulator's API) and saw in the logs.

@negz any thing we can do to get this merged?

@turkenh - updated, thanks, missed that  🙏


Image for TMDB API

Added image to the TMDB api

Thanks a ton! Would love if you could add images to other API's without images too.

Ok No problem!


[improve][test] Fix flaky test SimpleProducerConsumerStatTest#testMsgRateExpired

Fixes #20615
Motivation
SimpleProducerConsumerStatTest#testMsgRateExpired seems flakey, so I'll try to improve it.
Modifications
Before this fix, we checked if the messages expired by referring to the msgRateExpired value in the subscription stats. However, this value resets to 0 over time, so we instead refer to the totalMsgExpired value, which never resets.
Additionally, we get the stats directly from the instances in the broker server and not from the admin API.
Verifying this change

[x] Make sure that the change passes the CI checks.

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Codecov Report

Merging #20629 (04261c8) into master (2b01f83) will increase coverage by 39.48%.
The diff coverage is 92.85%.

@@              Coverage Diff              @@
##             master   #20629       +/-   ##
=============================================
+ Coverage     33.58%   73.07%   +39.48%
+ Complexity    12127     3701     -8426
=============================================
  Files          1613     1867      +254
  Lines        126241   138684    +12443
  Branches      13770    15240     +1470
=============================================
+ Hits          42396   101338    +58942
+ Misses        78331    29310    -49021
- Partials       5514     8036     +2522

Flag
Coverage Δ

inttests
24.15% <50.00%> (-0.04%)
:arrow_down:

systests
24.97% <50.00%> (?)

unittests
72.36% <92.85%> (+40.32%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...he/pulsar/client/impl/MultiTopicsConsumerImpl.java
78.40% <92.85%> (+46.04%)
:arrow_up:

... and 1520 files with indirect coverage changes


onRowClick is triggered when clicking a button in a row

What the title says, the problem started happening in version 3.0.14

Thank you for this! Per our guidelines you will need to create a sandbox for this issue if you want it addressed, otherwise please submit a PR that resolves this issue.

Here's a reproducible demo:
https://codesandbox.io/s/material-table-starter-template-forked-3uzt9?file=/src/index.js

Did you try https://developer.mozilla.org/en-US/docs/Web/API/Event/stopPropagation?

Yes, you can see it in my sample.
On Wed, 28 Jul 2021 at 16:14 Domino987 @.***> wrote:

Did you try
https://developer.mozilla.org/en-US/docs/Web/API/Event/stopPropagation?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/material-table-core/core/issues/277#issuecomment-888344608,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJLWQXSTRL6C4X23XTEALTT2AGFDANCNFSM5BCH2MJQ
.

--
Roman Alifanov - MCAD - MCTS - MCP - MCT
http://www.animaonline.com

I saw you used prevent Default.

tried it out and does indeed work, thanks!


Do you think we can support a list of application here (splitted via comma):

Do you think we can support a list of application here (splitted via comma):
--applications app2,app5
because with next application it will become complexe to add all combination (I have submitted to support TSP 2121-1, TSP 2121-4 can come this year and App4 too).
So internally we can have an array of ApplicationCompositionType.
What do you think ?
Originally posted by @MarcAntoine-Arnaud in https://github.com/Netflix/photon/pull/238

What would be the expected behaviour when you specify "--applications app2,app5"?

Photon will check the compliancy of any package compatible with App2 or App5 (no App2e here).
--applications app2,app2e,app5 will check for App2, App2e, and App5.
It will just change these line of code:
https://github.com/wruppelx/photon/blob/master/src/main/java/com/netflix/imflibrary/app/IMPAnalyzer.java#L644

The default behaviour is to check against the application specified in ApplicationIdentification of a given CPL.
Please note that the --application option proposed by #238 is optional.
--application is meant to force Photon to test against a specific application.
The list of permitted parameters for --application can be extended as more IMF Applications are supported.
I have modified the list of parameters to be app2, app2E or app5.
Does that make sense?

--applications app2,app5

I do not see a clear use case for this since app2 and app5 are mutually exclusive, unless the validation against each app is applied successively, in which, why not simply run the validator multiple times.
There is probably a case for a --plugins plugin1-uri, plugin2-uri, since multiple plugins can be applied to a single application.

Note that this issue is related to #218 and should be taken into account.


tetra: Implement getevents stdin mode

Import the code that was used to build the initial tetragon CLI, and
update it to reflect all the changes in the gRPC APIs. The initial CLI
has stopped working because the gRPC APIs have changed.
Ref: https://github.com/cilium/tetragon/releases/tag/tetragon-cli
Fixes: #438
Signed-off-by: Michi Mutsuzaki<EMAIL_ADDRESS>

you can test the CLI locally with:
make tetra
kubectl logs -n kube-system ds/tetragon -c export-stdout -f | ./tetra getevents -o compact

once this pr gets merge, i will:

update README.md file.
update the release process to include the CLI.
work with the next release manager to get the CLI released 🚀


How about adding to struct account
	enum vidmode vmode;          /**< Video mode for incoming calls      */

and account_set_vmode() API function?
Once done, sipsess_conn_handler() ua_call_alloc call could look like this:
err = ua_call_alloc(&call, ua, ua->acc->vmode, msg, NULL, to_uri, true);

Best solution would be if vmode param could be added to ua_answer API function:
int  ua_answer(struct ua *ua, struct call *call, enum vidmode vmode);

same way as it is in ua_connect().  Then user could decide on the fly if he/she wants to have video stream in the call.

here is a suggestion:
int call_answer(struct call *call, uint16_t scode, enum vidmode vidmode)
{
    if (vidmode == VIDMODE_OFF)
        call->video = mem_deref(call->video);
}

adding a vidmode parameter to the API should be a fine solution, I think ..

Looks good.  For apps that do not call call_answer() directly, but via ua_answer(), would it be OK to also include vidmode param to ua_answer:
int ua_answer(struct ua *ua, struct call *call, enum vidmode vidmode)
{
	if (!ua)
		return EINVAL;

	if (!call) {
		call = ua_call(ua);
		if (!call)
			return ENOENT;
	}

	return call_answer(call, 200, vidmode);
}

yes, ua_answer and call answer should have a similar signature.

Done by https://github.com/alfredh/baresip/commit/b01f649c06e34cf7a3f6a6072796b96e182f3088.


[Bug] Remove duplicated contributions

Is there an existing issue for this?

[X] I have searched the existing issues

What product(s) are you seeing the problem on?
OpenChallenges
Current behavior
In the examples below, IBM and Mount Sinai are listed twice in lists of challenge contributors.
Task:

[ ] Check for duplicated contributions in OC Data (@vpchung )
[ ] Review frontend implementation (@rrchai )

Expected behavior
No response
Anything else?
No response
Commit ID
770a5cc0958a66eb246729ea7f1dae0736c0befb
Are you developing inside the dev container?

[X] I am using the dev container

Code of Conduct

[X] I agree to follow this project's Code of Conduct

This is a duplicate of #1947


Add messages for Transform Feedback examples when running under WebGL1

For #2235

Background
To Do List

[x] Add label and assign to milestone
[ ] Coding

We update the title to have WebGL2, but we should update demos so message is displayed so it is less confusing.

This should be luma.gl issue, opened https://github.com/uber/luma.gl/issues/743
closing this one.


How are supabase edge functions meant to be deployed?

I noticed there's no CI/CD flow to deploy supabase edge functions, nor is there a specific script in apps/api/package.json. Is the assumption that the user just deploys them manually with supabase functions deploy or am I missing something?

Also, are the npm imports for deno deploys working for anyone? Meaning imports like
import { WelcomeEmail } from "@v1/emails/welcome";
import React from "react";
import { render } from "react-email/components";
import { Resend } from "resend";
import { Webhook } from "standardwebhooks";

in apps/api/supabase/functions/send-email/index.ts
seems to not working when deployed with supabase functions deploy :
Bundling Function: send-email
Error: failed to create the graph

Caused by:
    Relative import path "@v1/emails/welcome" not prefixed with / or ./ or ../ and not in import map from "file:///Users/username/git/v1/apps/api/supabase/functions/send-email/index.ts"
        at file:///Users/username/git/v1/apps/api/supabase/functions/send-email/index.ts:2:26

and same on VS code:
Relative import path "@v1/emails/waitlisted" not prefixed with / or ./ or ../
  hint: If you want to use a JSR or npm package, try running


Improve gRPC support

Middleware
Share middleware with http versions?
Update documentation

Actually, I think we can minimally support normal tower middleware by adding a template parameter for the middleware stack. That only helps with the first item, though

Pushing this out of any specific milestone for now. If any consumer wants/needs more control over their grpc initialization, they can use the FunctionService added in https://github.com/roadster-rs/roadster/issues/189

Actually, I think we can minimally support normal tower middleware by adding a template parameter for the middleware stack. That only helps with the first item, though

I was having trouble getting this to work with how the tonic/tower generics work.


AsyncCanDriver is not Sync

AsyncCanDriver is not Sync because TaskHandle_t is not Sync. AsyncUartDriver has also a field with type TaskHandle_t but Sync is implemented for it.

Why do you need it to be sync in the first place?

@ivmarkov I was doing some testing with LocalExecutor and Executor from the edge_executor crate with an esp32-s3. The best solution is probably to have one LocalExecutor per core, but that's not the point. As far as I know the twai driver is thread safe and if this is the case I don't understand why Sync should not be implemented. There seems to be a lot of inconsistency about the drivers being Sync and/or Send throughout the crate. But surely I am wrong and there is an explanation for each one of them. In this case I would suggest adding comments explaining the reason for these choices.

@ivmarkov I was doing some testing with LocalExecutor and Executor from the edge_executor crate with an esp32-s3. The best solution is probably to have one LocalExecutor per core, but that's not the point. As far as I know the twai driver is thread safe and if this is the case I don't understand why Sync should not be implemented. There seems to be a lot of inconsistency about the drivers being Sync and/or Send throughout the crate. But surely I am wrong and there is an explanation for each one of them. In this case I would suggest adding comments explaining the reason for these choices.

Look at the reply I gave you for the issue where one of the SPI drivers is not Send. I hope it sums up the status quo pretty nicely. But maybe again:

ALL drivers should be Send. If you find a driver which is not Send, this is a bug, needs to be fixed.
Some drivers might be Sync but so far we have never chased formally exposing the thread safety of the ESP IDF drivers/services, as it is often very poorly documented. Sometimes it is mentioned with 5 words like "this API is thread safe", sometimes we don't even have that. As an example, right now I'm implementing the GATT/BLE bindings for ESP IDF's Bluedroid and there is. no. single. statement. w.r.t. the thread safety of this complex API. Just one very misleading comment in the Esp32 forums. And unlike for the "regular" drivers' case, here I really really do need to know if I can make expose a thread safe variant of the API, due to how multiple higher layer BT drivers are stacked on top of the base BtDriver and work concurrently with each other. So I had to examine the C code in details so as to realize, that while Bluedriod itself is unlikely to be thread-safe, the ESP IDF wrappers isolate it to a dedicated task and the whole ESP IDF API works by posting to this task. So everything except setting the callbacks for event notifications is actually thread safe, it seems.

So you undersatand, in this situation, the safer bet is always to expose something as not thread-safe, rather than as thread-safe when it turns out later it is not (and you have to make backwards-incompatible changes; going the other way, from non-thread safe to thread safe is backwards compatible).
When I was mentioning that folks often stumble on the need to have something Sync when they (most likely) wrongly try to use work-stealing executor (like edge_executor::Executor) I meant exactly that - having the requirement your Futures to be Send (which means the driver should be Sync and all &mut self methods should become &self) is a heavy requirement, yet for embedded - at least for now - to me it always had a questionable value. Perhaps you can share your thoughts / view on the subject?
With that said... if you or other folks feel Send futures are important, and actually help me - driver by driver - with PRs - to expose a Sync variant of those drivers which are thread safe in ESP IDF - I'm all for that! But I really do need help here, as most of this work is on my shoulders only, few people understand it, and I often end just with issues by folks complaining and with no or very low quality PRs as folks simply do not understand these things, as they are above a certain complexity margin.

Thanks for the reply, yes you are right if we don't have confidence that a driver is thread-safe we shouldn't expose it as such. What bothered me is the fact that some drivers are Sync and I couldn't find an explanation why. For example LedcDriver and the TxRmtDriver are Sync even though they have methods that take &mut self (and now that I checked they also expose methods that are not thread-safe). For the CanDriver since there is no information I thought all the methods were threrad-safe but, as you rightly said, we would have to analyze the C code to be sure.
I don't think it is a priority to have send futures but I think it is ok to have them if it is possible to implement them, the choice of the Executor to use should be of the user not imposed by the library.
Unfortunately I don't think I fall among those few people and I don't want to waste your time with more low quality PRs, keep up the good work!

Thanks for the reply, yes you are right if we don't have confidence that a driver is thread-safe we shouldn't expose it as such. What bothered me is the fact that some drivers are Sync and I couldn't find an explanation why. For example LedcDriver and the TxRmtDriver are Sync even though they have methods that take &mut self (and now that I checked they also expose methods that are not thread-safe).

These drivers were contributions by other folks, and marking these as Sync was probably an omission on the side of the contributors, and of course also my omission as I did not catch this.

For the CanDriver since there is no information I thought all the methods were threrad-safe but, as you rightly said, we would have to analyze the C code to be sure.

ESP IDF generally advertises itself as "most APIs are thread-safe unless stated otherwise". How much that statement can be trusted without carefully examining the C source code... I think we agree we should be careful here. Yet I think I'm warming up to the idea of - over time - exposing as thread safe APIs these which we have checked in C to be really thread safe.

I don't think it is a priority to have send futures but I think it is ok to have them if it is possible to implement them, the choice of the Executor to use should be of the user not imposed by the library.

Well it is to some extent also a library decision. As it is the library which decides whether to expose Send futures or not, and as per above, exposing Send futures might mean a lot of work for diminishing results, so that's why - at least for me - it had been second priority so far. Also let's not forget that this is embedded. Very little RAM. Every additional thread means extra stack you need to care about as in sizing. I'm therefore in the camp that believes - unless you have really good reasons to spin extra threads (as in - let's say - the need to preemptively execute a very high priority task), app threads should be kept to a minimum. Ideally, just one thread which executes all your tasks. But if you do this, then you don't need your futures to be Send as you don't schedule them across multiple threads in the first place. Not to mention that you can have multiple local whose tasks communicate across thread boundaries using synchronization primitives which are thread safe.

Unfortunately I don't think I fall among those few people and I don't want to waste your time with more low quality PRs, keep up the good work!

 It is just the reality a bit is that we have these, and to get these through, there is a need to do a lot of hand holding (and persistence on the contributor side!).
I think the actual problem is, there are very few active contributors willing to dedicate a sizeable amount of their time to learn it all on these crates. :)

Well it is to some extent also a library decision. As it is the library which decides whether to expose Send futures or not, and as per above, exposing Send futures might mean a lot of work for diminishing results, so that's why - at least for me - it had been second priority so far.

Yes I agree with you. I was referring to all those drivers that are thread safe, if we don't need to introduce some primitive for synchronization I don't think we should omit the Sync implementation. If, on the other hand, they are not thread-safe I think the opposite is true.

Also let's not forget that this is embedded. Very little RAM. Every additional thread means extra stack you need to care about as in sizing. I'm therefore in the camp that believes - unless you have really good reasons to spin extra threads (as in - let's say - the need to preemptively execute a very high priority task), app threads should be kept to a minimum. Ideally, just one thread which executes all your tasks. But if you do this, then you don't need your futures to be Send as you don't schedule them across multiple threads in the first place. Not to mention that you can have multiple local executors whose tasks communicate across thread boundaries using synchronization primitives which are thread safe.

Yes most of the time I don't think send futures are necessary but on some occasions they might be useful. With two cores, as you said, you can use two local executors but it is not always easy to figure out how best to distribute tasks between the two. Maybe it may even be impossible if the amount of work of each task varies a lot over time.
But all this maybe is a bit off-topic for this issue :)


Water shader not working properly in Game view

The URP Water Shader does not work properly in Game view when using Perspective Camera. It does however work in Scene view or when using Orthographic Camera.
This is in Game view:

And this in Scene view:

Hi,

What version of Unity and Flat Kit are you using?
Are there any errors in Console?
Do the demo scenes ('Ocean' or 'Pool') work correctly? If yes, how to reproduce your issue?

Unity 2019.4.19f1 and the latest version of Flat Kit.
No errors in the console
The demo scenes do work correctly. To reproduce the issue I create a new scene, create a new material with the water shader on it and assign it to a plane in the scene.

Updating my project to Unity 2020.2 fixed my issue


Add Logging and PubSub documentation

This change is

Thanks!

@iantalarico Sorry I closed the PR before you commented. I opened another PR to address the comment here https://github.com/GoogleCloudPlatform/google-cloud-powershell/pull/448.


Error secret not found

Resolves issue https://github.com/cyberark/conjur-credentials-plugin/issues/18


Nix Hour How to: Package multiple Shell scripts and handle "command not found"

Dear Tweag team and Silvan @infinisil
Could you please discuss on Nix Hour how to package a single Bash script (using writeScript) or multiple (probably using mkDerivation) into a single package?
I would be specifically interested to know how to package multiple scripts into a single package and how to deal with commands in those Bash scripts that work fine on other Linux distros but that return command not found on Nix.
Let's say the script runs fdisk so I assume I need to specify pkgs.fdisk as buildInput and then somehow change fdisk inside the Bash script to /nix/store/rfil5rhr8dw5rdqk4cyndcm2jc4fr695-fdisk/bin/fdisk... But I don't know ;-)
Thank you.

Good question! Usually we'd look at questions with the most :+1:'s, but since I don't always know the answer to those, I opted to just look at your question in todays Nix Hour!
Unfortunately the live streaming failed, but I did upload the recording afterwards: Nix Hour #28
And here are the resulting files: https://github.com/tweag/nix-hour/tree/master/28
Hope that helps :smiley:

I've just watched it. Thank you @infinisil very much.


Add support for tracing

This adds support for tracing via Datadog (wrapped in Opentracing so in the future people can integrate their own tracing services if needed).
Why?
As we build out larger CICD agent pools/queues, we want to be able to see failures and metrics at a holistic level while also being able to zoom into specific jobs to see causes of failures or performance regressions (eg. Where was the failure/latency? Was it the specific agent? Something about the hooks?). The current metrics don't give us this kind of granularity.
Changes
This is for the most part broken out self-contained commits, with the first one being a quick POC we did to prove out the idea.

Tracing is reporting if the BUILDKITE_TRACING_DATADOG_ADDR env var or the tracing-datadog-addr config is set.
If neither are, the tracing code still runs, but uses a no-op tracer so that things aren't reported.
This will also ensure distributed tracing works between any dynamic jobs that might be created by encoding tracing context data into an env var and propagating that to the new jobs.
Includes tags to traces such as agent, queue, pipeline IDs, etc. This will help make aggregation in reporting easier.
Spans are wrapped around the major phases of a job and its hooks (pre-commit, etc).
Errors propagate back up to the root span so the trace itself will be marked erroneous if something failed.
Service name is  buildkite-agent, operation name is job.run, resource names follow an org / pipeline / job label pattern

Examples

This shows one main job spawning off five more jobs, which are all linked to the first.

Zoom into a specific failing job (in this case the failing coming from a command)

This is amazing! 😳 Thanks so much for your time and effort @goodspark, we'll review and come back to you with feedback.

This looks really useful. I have a few questions:

You mentioned it uses OpenTracing - would you mind elaborating on how that can be supplied over to other products like Jaeger?

If so, and if all it needs is an address, could the tracing datadog addr be named something less vendor-specific?

How're you noticing when hooks start/stop? That was a shortcoming of our own implementation that we used agent-level hooks to work around changing the agent itself.
Does the trace context propagate over

trigger'd pipelines? It looks like it...?
parallel jobs that are sharded across agents via command.parallelism?

Opentracing: So to plug in other tracing libraries, people would need to make code changes. Specifically in

Bootstrap.startTracing (pass opentracing.SetGlobalTracer an object that meets the opentracing.Tracer interface) and
~(if you want distributed tracing working) define a propagator that's compatible with the TextMap format, accepts a map[string]string as the carrier, and configure your specific tracer to use that propagator (as seen in the Datadog-specific setup code in Bootstrap.startTracing).~ This bit is no longer necessary!

In fact, if you look at the first commit, you can see that's how we started! But we're moving to use Datadog more so the implementation was changed.
We can definitely make the tracing host less vendor-specific, but we'll need another flag to indicate the tracing implementation so that the specific tracing library can be pulled. Very similar to the metrics Datadog env vars I suppose.

I'd very much like to be able to plug in opentracing to publish into jaeger. We currently (still!) do this by writing out JSON blobs when we see particular patterns in the build log, which we put there via agent-level hooks (to capture the coarse-grained step and hook-level spans fleet-wide) and via build-automation (the user-level finer-grained spans). We then pair those start/end blobs up at the end of the step and write them into elasticsearch, which jaeger-UI is configured to be reading from (we don't currently run the rest of jaeger, since this approach seemed simpler and we weren't sure people were going to use it at the time - they definitely do though, especially people with long-running or unstable builds!).

Hook start/stop: This is caught directly in the agent code itself. To make the code clearer, I pulled out each hook phase's lookup and execution code into a function. Then it's easier to setup spans and error tracking. This is the code that runs the global, agent, and (if applicable) plugin hooks for a phase.

Ah, nice :)

Trace propagation: Yup! This is why we built the custom EnvVarPropagator (which unfortunately is Datadog specific at the moment) and configured Datadog's tracer to use that instead of its defaults. ~I have not tested with command.parallelism but I can do that now. Will update this comment on results.~ Yup this also works with parallelism steps! However, there's no data in the spans or tags that indicate which parallel job run a span belongs to.

Is there no data in the spans/tags because the BUILDKITE_PARALLEL_JOB isn't noticed, or some other reason? We don't at the moment, but will soon do the same thing with our tracing as we do with our logging - add pretty much all the BUILDKITE_* environment variables onto the trace context key/value bucket.
That makes me think - it would be nice to see the step's depends_on reflected into there. The tracing UI that jaeger offers is a standard time-based waterfall, not a DAG, and so one has to flip between the BK /dag and jaeger to trace the critical path; that's a bit of a rough edge. If the dependency information can be reflected into tracing and then used, that would be really nice.

Hi Buildkite team,
Is there anything I can do to help get this reviewed?
Thanks!

@goodspark this is amazing, thank you for all the work you have put into this! I have shared this with the team cc/ @chloeruka and @pda for a review! 👌

This is very cool. I'm gonna aim to get this merged in next week! 👍🏻

@goodspark first of all, awesome work! Thanks so much, we're really excited to test this and roll it out. I did have a question you might be able to answer; how hard would it be to migrate this to OpenTelemetry? Is it something that can be done soon or is there still some missing piece that we'd be waiting on? It sounds like OpenTelemetry is still pretty new, and I'm wondering how the APIs are coming along on the go/Datadog side.

I can definitely look into it. @samschlegel and I were talking about that a while ago too. I'll spend a day on it and if it's too much, I'll report back here.

AFAICT there's still no DataDog Trace exporter, which is what blocked us from using it in the first place :(

Yeah I've been looking around for code or examples. Looks like Datadog only has Ruby, Python, and JS support. But once they have Golang support, will be happy to help code/review the move to OpenTelemetry.

Hey @goodspark  - I was chatting about this PR in the #agent-dev channel on the buildkite slack with @chloeruka and it was mentioned that it'd be great if the merge conflicts could be fixed, as they should be able to start testing this out.
let me know if you want a hand with this as I'm pretty eager for this

Hey @goodspark  - I was chatting about this PR in the #agent-dev channel on the buildkite slack with @chloeruka and it was mentioned that it'd be great if the merge conflicts could be fixed, as they should be able to start testing this out.
let me know if you want a hand with this as I'm pretty eager for this

Sure. Done.
@chrisfowles @chloeruka
Let me know if there's any way I can help the review process.

Sure. Done.
@chrisfowles @chloeruka
Let me know if there's any way I can help the review process.

I also added a commit to add a URL to each job in the tags, since I found that would've been helpful when debugged some pipelines with several jobs in the past.

I also added a commit to add a URL to each job in the tags, since I found that would've been helpful when debugged some pipelines with several jobs in the past.

For watchers/interested folks, I'll circle back with how to opt in to this via the elastic stack when this is available on edge.

Sorry about the delay; had to shave a few yaks. This is now available as an experimental edge release, you can opt into using it by setting the BuildkiteAgentRelease in the Elastic CI stack for AWS to "edge"
OR manually install by downloading the v3.27.0 experimental release:

https://download.buildkite.com/agent/experimental/<IP_ADDRESS>15/buildkite-agent-linux-386
https://download.buildkite.com/agent/experimental/<IP_ADDRESS>15/buildkite-agent-linux-amd64
https://download.buildkite.com/agent/experimental/<IP_ADDRESS>15/buildkite-agent-linux-arm
https://download.buildkite.com/agent/experimental/<IP_ADDRESS>15/buildkite-agent-linux-arm64

#1491
Bung


Future work: more complete search of "bad lines"

When dealing with missing keys in dict-like objects, the "bad line" might be of the form
raise KeyError(...)

If the initial user code was started by calling the dict or dict-like object directly, we can currently find it as the "program_stopped_bad_line".  However, if this was called indirectly, say by importing a file, we would miss it.
For such cases, it might be useful to retrieve the entire chain of "bad lines" and perform a search, rather than working with only both extremities (program stopped, and error raised).
This is a low priority item.
Edit: the easiest way of getting this information is most likely by using stack_data.

This has been taken care of for code in Python's standard library. For the rest, it might be useful to ignore this issue.


Feature/ampersand node hierarchy

Loading SceneNodes into an ampersand model, and traversing the scene node tree.
Not yet clear whether this is the right way to do it, so initiating a PR for the review process.

:+1: merge ok from my side.


fix: pin bitsandbytes version

PR Type

[x] Bugfix
[ ] Feature
[ ] CI related changes
[ ] Documentation content changes
[ ] Tests

https://trello.com/c/BBFL1Uk5/113-fix-mac-poetry-installation
Proposed Changes
This PR pins the bitsandbytes version to fix the issue of the library installation on Mac.
Screenshots
N/A

server, now works


Automatically Generate Mochawesome HTML

Currently the project only generates the JSON needed by the Mochawesome Report Generator.
Have the reporter generate the JSON as well as create the HTML report

Note to self: This would require ensuring the correct version of MARGE is installed.


Remove warning supression from prod

Rails emits a lot of warnings under Ruby 2.7. Yesterday it merged PRs into Rails 6 to silence them.
Prod puma config + cron jobs currently include RUBYOPT='-W:no-deprecated' until we're on 5.1 and need to bump to whatever future release of 6 includes these. Filing this ticket as a reminder to myself to remove those.

These are been fixed for a while.


Removing email address from display in groups

For data protection reasons Ally, Lisa and Steve have decided to remove email addresses from being displayed in groups.

Fixed in 11354c9289d3384df166ca3e77284f3751fd6811


support treeshaking to reduce bundle size for web project ?

it seems otel grows bigger and bigger, perhaps we can add an es entry like module in package.json

[x] This only affects the JavaScript OpenTelemetry library
[ ] This may affect other libraries, but I would like to get opinions here first

I'd like to second this request. I'm profiling the impact on our production build and noticing that adding a number of opentelemetry packages is adding roughly 60kb~ to the final build, which is really large. It will have a negative impact on frontend performance...

@jessebond2 Could you please provide the list of otel dependencies you are using ? If you can also have the details of which package is the heaviest, that would help a lot !
For information: there are work done to ship ESM version (https://github.com/open-telemetry/opentelemetry-js/pull/2112) so components are better treeshaked (at least we hope)

Hey @vmarchaud, sorry for the delayed response. Here's the list of packages I tried using and the incremental size diffs each package added. This was from the output of our production webpack build.


Fix package.json

Before this change npm said the package was invalid.

👍

👍

👮

@k-vyn you need to approve in the GitHub UI. (The emoji commands aren't enabled on public repos.)


Add IntoIterator<Item = &Pk> for various types

Related to https://github.com/rust-bitcoin/rust-miniscript/issues/223. Might make ForEachKey obsolete. However, these impls use the heap with Boxes.

I feel quite neutral towards this particular implementation, it's probably good enough. But I'm wondering how hard it would be to, instead of essentially re-building the miniscript tree as an interator tree, to build a custom iterator that can traverse the miniscript tree. I'll take a more thorough look once the fuzz test runs through.

I never really understand the fuzzer error outputs.. Anyone that does? https://github.com/rust-bitcoin/rust-miniscript/runs/3076686285
About your comment, @sgeisler I thought about a custom iterator struct that takes keeps a descriptor reference and some "incrementor metadata" to see where it is. You'd kinda need inspection into the descriptor, but I think it'd be possible. Not really sure how the different Terminal vs Policy vs Miniscript vs Descriptor implementations would interact, though.

@stevenroose, the fuzzer bug is not a bug in miniscript, but a bug in rust-secp-zkp handwritten crypto for fuzzing.

0x515B0941aFc4Db1F2D20f2379323a44Edb3c4460

@mmm0171 what are you trying to tell us?

First I thought it was an hash of a CVE
Then I recognized the Ethereum address mixed case style

0xa36fD8fF0524f287422337d99bABdf05923f581B

So I added fixup commits that would change the implementation to yield an iterator of ForEach elements so that you can also get easy access to key hashes.
However the ForEach type is named quite specifically for the for_each method use-case, so perhaps it makes sense to find a more general-purpose name for it like KeyRef or KeyId or something?

I don't think it makes sense to have the iterator return only public keys. Miniscripts are not public key containers, they contain different kinds of data. It might make sense to iterate over all the sub-miniscripts, or over all the terminals, or over all the pk/hash/timelocks.

Hmm, I mean keys are the most meaningful things miniscripts contain. Other than keys there's timelocks and hashes.
I'd be ok trying to make this into a more general iterator over interesting things, MiniscriptIterator or so that iterators over MiniscriptItem::{PublicKey, Hash256, Sha256, Hash160, Timelock} or something. If we have a newtype on it, we can even define filters on them fn keys(self) -> impl Interator<Item = &PublicKey>.

I like the idea of having newtypes that can wrap (references to) things! Would be cool if we could genericize this so it worked on all the objects in the crate.

Just as a reminder: we already have iterators for miniscript items: https://github.com/rust-bitcoin/rust-miniscript/blob/master/src/miniscript/iter.rs


Correct compute to config in recommended gcloud config command

gcloud compute set project is not the correct command. It is gcloud **config** set project

Whooops this is my handwork =p
Thanks for the fix!

Whooops this is my handiwork =p
Thanks for the fix!

Oh no worries. A very small and quick change! Happy to help


One more card added

New Card added

I have added the card named Aman Chaudhary.

your pr is conflicted plz make a new pr I will accept that.


See Google Cal Schedule

Adds functions to see google cal schedule for user
Fixes #20

@joshuan98 need help to see if schedule is used as intended in habit_command.py


Cache last sent msgs

Mainly useful for seeing what your last replies/posts were, especially when using @@!!.
|21:17 <hloeung> lastsent
|21:17 <mattermost> [@@jz6umt46w38spqcjppox6kk7ky] user1: my first message
|21:17 <mattermost> [@@ndziyrp7pig7zrsf643qiwe97a] #haw-test: Testing
|21:17 <mattermost> [@@zajoh97ms7f8jyyq1j3iu4nudr] user1: ok (re @user1: Here …)

👍