Unable to extract mp3Form form for InfoQ video

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log
$ youtube-dl  --verbose  'https://www.infoq.com/presentations/problems-async-arch/'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.infoq.com/presentations/problems-async-arch/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.5 (CPython) - macOS-12.5-x86_64-i386-64bit
[debug] exe versions: ffmpeg 5.1, ffprobe 5.1
[debug] Proxy map: {}
[InfoQ] problems-async-arch: Downloading webpage
ERROR: Unable to extract mp3Form form; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/infoq.py", line 128, in _real_extract
    + self._extract_http_audio(webpage, video_id))
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/infoq.py", line 93, in _extract_http_audio
    fields = self._form_hidden_inputs('mp3Form', webpage)
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 1367, in _form_hidden_inputs
    form = self._search_regex(
  File "/usr/local/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 1012, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract mp3Form form; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description
I can watch the video in a browser when I'm logged in or not. When downloading the video with youtube-dl, I get the error, "Unable to extract mp3Form form".

The page doesn't have the <form...> with id="mp3Form" that the extractor expects. This shouldn't be a crashing error. The http_video format has audio.

I was able to work around this problem by modifying the _real_extract() method in youtube_dl/extractor/infoq.py and commenting out the self._extract_http_audio(webpage, video_id) call in line 128.

That would work in this case.
More generally, let's assume that the targeted form is sometimes present. We can either

trap the failing code inside _extract_http_audio() with a try:, or
extend the __form_hidden_inputs() method signature with **kwargs passed into its _search_regex() call so that fatal=False can be passed down.

The first option occurred to me to minimize the number of things touched (infoq.py vs infoq.py and common.py) and just deal with a smaller blast radius.
The second option would better if other extractors might need to deal with a similar situation. This would expose more of the facilities that _search_regex() provides.
Thanks for looking into this!

I tried extending the _form_hidden_inputs() method to pass fatal=False and it does download the video from InfoQ but with WARNING: unable to extract mp3Form form.
$ youtube-dl -v https://www.infoq.com/presentations/problems-async-arch/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.infoq.com/presentations/problems-async-arch/']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.5 (CPython) - macOS-12.5-x86_64-i386-64bit
[debug] exe versions: ffmpeg 5.1, ffprobe 5.1
[debug] Proxy map: {}
[InfoQ] problems-async-arch: Downloading webpage
WARNING: unable to extract mp3Form form; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://videoh.infoq.com/presentations/21-nov-unblockeddesign.mp4?Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6IioiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2NTk0OTQ0MDZ9LCJJcEFkZHJlc3MiOnsiQVdTOlNvdXJjZUlwIjoiMC4wLjAuMC8wIn19fV19&Signature=KUA9-Ak19p9ZPMBvwk1KTESuNeZXi~nLl8HSlcyOEUmRP6CRa~5LXQsBxOqSRgyHeKDMd9OFidkSyysMkwbav3msMuV6nfR8P1KdbUKSfX-c980~KvPQn51X15IxLQpYrPjfoU-TMiGp232JwL3i5vizxcX-8MN3KLuIHmqY1RQ_&Key-Pair-Id=APKAIMZVI7QH4C5YKH6Q'
[download] Destination: Unblocked by Design-problems-async-arch.mp4
[download]   1.8% of 217.90MiB at  7.89MiB/s ETA 00:27^C
ERROR: Interrupted by user

This is what I did:
$ diff -u extractor/infoq.py-orig extractor/infoq.py
--- extractor/infoq.py-orig	2021-12-16 09:02:01.000000000 -1000
+++ extractor/infoq.py	2022-08-02 14:31:51.000000000 -1000
@@ -90,7 +90,7 @@
         }]

     def _extract_http_audio(self, webpage, video_id):
-        fields = self._form_hidden_inputs('mp3Form', webpage)
+        fields = self._form_hidden_inputs('mp3Form', webpage, fatal=False)
         http_audio_url = fields.get('filename')
         if not http_audio_url:
             return []

$ diff -u extractor/common.py-orig extractor/common.py
--- extractor/common.py-orig	2021-12-16 09:02:01.000000000 -1000
+++ extractor/common.py	2022-08-02 14:33:30.000000000 -1000
@@ -1363,10 +1363,12 @@
                 hidden_inputs[name] = value
         return hidden_inputs

-    def _form_hidden_inputs(self, form_id, html):
+    def _form_hidden_inputs(self, form_id, html, fatal=True):
         form = self._search_regex(
             r'(?is)<form[^>]+?id=(["\'])%s\1[^>]*>(?P<form>.+?)</form>' % form_id,
-            html, '%s form' % form_id, group='form')
+            html, '%s form' % form_id, group='form', fatal=fatal)
+        if not form:
+            form = ''
         return self._hidden_inputs(form)

     def _sort_formats(self, formats, field_preference=None):

The warning comes from the _search_regex() method in common.py.  When the fatal parameter is passed as False and no match is found, things end up in else clause  and the warning message is printed with the bug_reports_message() text. No exception is raised.
To have the warning message not printed when passing fatal=False to the _form_hidden_inputs() method, a check for no match object and not fatal is needed in the else clause of the _search_regex() method.
$ diff -u extractor/common.py-orig extractor/common.py
--- extractor/common.py-orig	2021-12-16 09:02:01.000000000 -1000
+++ extractor/common.py	2022-08-02 14:51:27.000000000 -1000
@@ -1011,6 +1011,8 @@
         elif fatal:
             raise RegexNotFoundError('Unable to extract %s' % _name)
         else:
+            if not mobj and not fatal:
+                return None
             self._downloader.report_warning('unable to extract %s' % _name + bug_reports_message())
             return None

@@ -1363,10 +1365,12 @@
                 hidden_inputs[name] = value
         return hidden_inputs

-    def _form_hidden_inputs(self, form_id, html):
+    def _form_hidden_inputs(self, form_id, html, fatal=True):
         form = self._search_regex(
             r'(?is)<form[^>]+?id=(["\'])%s\1[^>]*>(?P<form>.+?)</form>' % form_id,
-            html, '%s form' % form_id, group='form')
+            html, '%s form' % form_id, group='form', fatal=fatal)
+        if not form:
+            form = ''
         return self._hidden_inputs(form)

     def _sort_formats(self, formats, field_preference=None):

You can use default=None (no warning log message) or fatal=False (warning message) depending on whether the missing form is expected or its absence indicates that more work is needed on the extractor.  There isn't, though there could be, an expected parameter along with fatal that would suppress the bug_reports_message() in the warning when True, if the user should be able to see why no separate audio was extracted.
So I would (and did, just like your diff above) use fatal=False initially. Then, depending on tests:

if the form is never there in current pages, I'd just remove the _extract_http_audio() method;
if the form is still served in old pages but not in new ones, I'd switch to default=None;

Having a look at yt-dlp's extractor, the case is already handled and ignored there. Equivalent yt-dl code would be:
     def _extract_http_audio(self, webpage, video_id):
+        try:
-        fields = self._form_hidden_inputs('mp3Form', webpage)
+            fields = self._form_hidden_inputs('mp3Form', webpage)
+        except ExtractorError:
+            fields = {}
         http_audio_url = fields.get('filename')
...

This code assumes that any kind of ExtractorError should be ignored  here, not just RegexNotFoundError. For now I'll probably just push this for compatibility.

Awesome! Thanks, again!

Thanks for the commit!


Feature Request: Ability to bump versions

Hello, I would like to request the ability to bump versions by using a command similar to npm patch
For example

Package.json
Android
iOS

1.3.4
1.0.5
1.1.0

npx capacitor-set-version --bump:all

Package.json
Android
iOS

1.3.5
1.0.6
1.1.1

You can use release-it for this!
https://github.com/aeharding/voyager/blob/main/.release-it.json
then run
BUILD=123 release-it and it will do the thing.


Deadline for the project (question by Maxime Depoorter)

When is defined the mandatory deadline? is there any flexibility? eg Mips forces all user to upgrade their LIS before implementation.

Deadlines will be available in the communication (circulaire) by RIZIV/INAMI


🐛 [BUG] - Intermittent issue freezing when choosing shard in retail

Description
Every once in a while I experience a freeze when choosing "Atlantic" shard on retail server.  Seems to be a websocket connection error.  Attached is a screenshot from the console.

Reproduction steps
1. Go to https://play.classicuo.org/
2. Click on 'Play Now' for Ultima Online Retail
3. Scroll down to 'Atlantic' (only one I've tested with)
4. Once in a while experience a freeze with a websocket connection error in the console

Screenshots
No response
Logs or Error messages
No response
Browsers
Chrome
OS
Mac

@johnpwrs could you attach the screenshot again please looks like it never came through correctly 😄

Hi @michaelwhelehan
Hope this one works:


The CART tree is not binary tree

the CART tree is not binary tree. Why?

The algorithm covered in the library is mentioned in this tutorial: https://sefiks.com/2018/08/27/a-step-by-step-cart-decision-tree-example/


[4.0] EditorInspector doesn't reset scroll when a different object is inspected

Godot version
v4.0.alpha13.official [59dcddc45]
System information
Windows 11 x64, Lenovo 81WH
Issue description
If you select an object that has a lot of properties, scrolls a little and then select another object, EditorInspector will keep the scroll value.
https://user-images.githubusercontent.com/7025991/184065062-78832db9-1d8c-4760-8610-27e14af0a27d.mp4
This doesn't happens in 3.x
Steps to reproduce

Select an object (node, resource) to inspect it in editor.
Select an object in scene (node) or a resource. It doesn't matter, just try to inspect something that has a lot of values.
Scroll in that inspector.
Now select object described in 1

Minimal reproduction project
No response

Duplicate of #63334, which has been fixed in alpha14.


⚠ Auth has degraded performance

In 9e71965, Auth (https://auth.k2lin.com) experienced degraded performance:

HTTP code: 200
Response time: 7999 ms

Resolved: Auth performance has improved in 9145aae.


TLS termination not working with wildcard certs

I have many web apps each running on a subdomain and I opted for a wilcard cert (*.example.com) to cover them all, but it seems liked TLS termination is not working with a SANs url in the config!
Here's my json (let's call this Case 1):
{
    "logging": {
        "logs": {
            "default": {
                "level": "DEBUG"
            }
        }
    },
    "apps": {
        "layer4": {
            "servers": {
                "proxy": {
                    "listen": [
                        ":443"
                    ],
                    "routes": [
                        {
                            "match": [
                                {
                                    "tls": {
                                        "sni": [
                                            "sub1.example.com"
                                        ]
                                    }
                                }
                            ],
                            "handle": [
                                {
                                    "handler": "tls"
                                },
                                {
                                    "handler": "subroute",
                                    "routes": [
                                        {
                                            "match": [
                                                {
                                                    "http": [
                                                        {
                                                            "path": [
                                                                "/api"
                                                            ]
                                                        }
                                                    ]
                                                }
                                            ],
                                            "handle": [
                                                {
                                                    "handler": "proxy",
                                                    "upstreams": [
                                                        {
                                                            "dial": [
                                                                "xray:3445"
                                                            ]
                                                        }
                                                    ]
                                                }
                                            ]
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            }
        },
        "tls": {
            "certificates": {
                "automate": [
                    "*.example.com",
                    "example.com"
                ]
            },
            "automation": {
                "policies": [
                    {
                        "subjects": [
                            "*.example.com",
                            "example.com"
                        ],
                        "issuers": [
                            {
                                "challenges": {
                                    "dns": {
                                        "provider": {
                                            "api_token": "CF_TOKEN",
                                            "name": "cloudflare"
                                        }
                                    }
                                },
                                "email"<EMAIL_ADDRESS>                                "module": "acme"
                            },
                            {
                                "challenges": {
                                    "dns": {
                                        "provider": {
                                            "api_token": "CF_TOKEN",
                                            "name": "cloudflare"
                                        }
                                    }
                                },
                                "email"<EMAIL_ADDRESS>                                "module": "zerossl"
                            }
                        ]
                    }
                ]
            }
        }
    }
}

Caddy logs:
caddy  | {"level":"debug","ts":1668285360.007823,"logger":"events","msg":"event","name":"tls_get_certificate","id":"c64370e8-8001-425d-ba76-9213fcdf2550","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"sub1.example.com","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"Conn":{}}}}
caddy  | {"level":"debug","ts":1668285360.0085123,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"sub1.example.com"}
caddy  | {"level":"debug","ts":1668285360.0088234,"logger":"tls.handshake","msg":"choosing certificate","identifier":"*.example.com","num_choices":1}
caddy  | {"level":"debug","ts":1668285360.009071,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"*.example.com","subjects":["*.example.com"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"3a276470a02cfd24f62c7b8834d9cb69a10f7a2295ba511a49f166e53183637d"}
caddy  | {"level":"debug","ts":1668285360.0092616,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"x.x.x.x","remote_port":"47922","subjects":["*.example.com"],"managed":true,"expiration":1675999223,"hash":"3a276470a02cfd24f62c7b8834d9cb69a10f7a2295ba511a49f166e53183637d"}

Case 2 with the subdomain added:
"tls": {
            "certificates": {
                "automate": [
                    "*.example.com",
                    "example.com",
                    "sub1.example.com"
                ]
            },
            "automation": {
                "policies": [
                    {
                        "subjects": [
                            "*.example.com",
                            "example.com",
                            "sub1.example.com"
                        ],
                        "issuers": [
                            {
                                "challenges": {
                                    "dns": {
                                        "provider": {
                                            "api_token": "CF_TOKEN",
                                            "name": "cloudflare"
                                        }
                                    }
                                },
                                "email"<EMAIL_ADDRESS>                                "module": "acme"
                            },
                            {
                                "challenges": {
                                    "dns": {
                                        "provider": {
                                            "api_token": "CF_TOKEN",
                                            "name": "cloudflare"
                                        }
                                    }
                                },
                                "email"<EMAIL_ADDRESS>                                "module": "zerossl"
                            }
                        ]
                    }
                ]
            }
        }
    }

Caddy logs:
caddy  | {"level":"debug","ts":1668286665.5286486,"logger":"events","msg":"event","name":"tls_get_certificate","id":"6d5c061a-0407-48e2-8bc3-9c1dbb98e2ae","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"sub1.example.com","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"Conn":{}}}}
caddy  | {"level":"debug","ts":1668286665.5296712,"logger":"tls.handshake","msg":"choosing certificate","identifier":"sub1.example.com","num_choices":1}
caddy  | {"level":"debug","ts":1668286665.5325434,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"sub1.example.com","subjects":["sub1.example.com"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"700af100c686af79ab90cf9c506c56b9af694f6e5abdda75ec519b24db5e0148"}
caddy  | {"level":"debug","ts":1668286665.532762,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"x.x.x.x","remote_port":"48126","subjects":["sub1.example.com"],"managed":true,"expiration":1676052278,"hash":"700af100c686af79ab90cf9c506c56b9af694f6e5abdda75ec519b24db5e0148"}
caddy  | {"level":"debug","ts":1668286686.210998,"logger":"layer4","msg":"connection stats","remote":"x.x.x.x:47312","read":1710,"written":7136,"duration":25.669313421}

Case 3 same as above with SANs url removed:
"tls": {
            "certificates": {
                "automate": [
                    "example.com",
                    "sub1.example.com"
                ]
            },
            "automation": {
                "policies": [
                    {
                        "subjects": [
                            "example.com",
                            "sub1.example.com"
                        ],
                        "issuers": [
                            {
                                "challenges": {
                                    "dns": {
                                        "provider": {
                                            "api_token": "CF_TOKEN",
                                            "name": "cloudflare"
                                        }
                                    }
                                },
                                "email"<EMAIL_ADDRESS>                                "module": "acme"
                            },
                            {
                                "challenges": {
                                    "dns": {
                                        "provider": {
                                            "api_token": "CF_TOKEN",
                                            "name": "cloudflare"
                                        }
                                    }
                                },
                                "email"<EMAIL_ADDRESS>                                "module": "zerossl"
                            }
                        ]
                    }
                ]
            }
        }
    }

Caddy logs:

caddy  | {"level":"debug","ts":1668286893.8412163,"logger":"layer4.matchers.tls","msg":"matched","remote":"x.x.x.x:51926","server_name":"sub1.example.com"}
caddy  | {"level":"debug","ts":1668286893.875634,"logger":"events","msg":"event","name":"tls_get_certificate","id":"06b0e899-d535-425d-a372-242f5e80858d","origin":"tls","data":{"client_hello":{"CipherSuites":[4865,4867,4866,49195,49199,52393,52392,49196,49200,49162,49161,49171,49172,156,157,47,53],"ServerName":"sub1.example.com","SupportedCurves":[29,23,24,25,256,257],"SupportedPoints":"AA==","SignatureSchemes":[1027,1283,1539,2052,2053,2054,1025,1281,1537,515,513],"SupportedProtos":["h2","http/1.1"],"SupportedVersions":[772,771],"Conn":{"Conn":{},"Context":{"Context":{"Context":0}}}}}}
caddy  | {"level":"debug","ts":1668286893.886206,"logger":"tls.handshake","msg":"choosing certificate","identifier":"sub1.example.com","num_choices":1}
caddy  | {"level":"debug","ts":1668286893.889086,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"sub1.example.com","subjects":["sub1.example.com"],"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"700af100c686af79ab90cf9c506c56b9af694f6e5abdda75ec519b24db5e0148"}
caddy  | {"level":"debug","ts":1668286893.889988,"logger":"tls.handshake","msg":"matched certificate in cache","remote_ip":"x.x.x.x","remote_port":"51926","subjects":["sub1.example.com"],"managed":true,"expiration":1676052278,"hash":"700af100c686af79ab90cf9c506c56b9af694f6e5abdda75ec519b24db5e0148"}
caddy  | {"level":"debug","ts":1668286894.016458,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"x.x.x.x:51926","server_name":"sub1.example.com"}
caddy  | {"level":"debug","ts":1668286894.0264504,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"x.x.x.x:51926","upstream":"xray:3445"}
caddy  | {"level":"debug","ts":1668286894.1843724,"logger":"layer4","msg":"connection stats","remote":"x.x.x.x:47530","read":494,"written":4496,"duration":10.683719456}

The only working setup is Case 3.

Not sure if it is relevant, but wildcart certs work for me using acme-dns plugin. It took some time(maybe 2 hours) to get the DNS01 challenge answered but now it works and renews them automatically.
Here is an example:
logging:
  logs:
    default:
      level: INFO
storage:
  "module": "file_system"
  "root": "/data"
apps:
  tls:
    certificates:
      automate:
        - *.myhost.com
    automation:
      policies:
        - issuers:
            - module: acme
              challenges:
                dns:
                  provider:
                    name: acmedns
                    config_file_path: /etc/acme-dns-cred.json
  layer4:
    servers:
      main:
        listen: [":443"]
        routes:
          - match:
            - tls:
                sni:
                  - "livekit-turn.myhost.com"
            handle:
              - handler: tls
              - handler: proxy
                upstreams:
                  - dial: ["localhost:5349"]
          - match:
              - tls:
                  sni:
                    - "livekit.myhost.com"
            handle:
              - handler: tls
                connection_policies:
                  - alpn: ["http/1.1"]
              - handler: proxy
                upstreams:
                  - dial: ["localhost:7880"]
          - match:
              - tls:
                  sni:
                    - "livekit-whip.myhost.com"
            handle:
              - handler: tls
                connection_policies:
                  - alpn: ["http/1.1"]
              - handler: proxy
                upstreams:
                  - dial: ["localhost:8080"]

@kyochikuto Sorry I dropped this on the floor. Haven't had a chance to look into it :see_no_evil:
Can you try upgrading everything to the latest version and try again?

Sorry I dropped this on the floor. Haven't had a chance to look into it 🙈

Hey no problem at all! ;)

Can you try upgrading everything to the latest version and try again?

It is indeed fixed somehow, I even noticed caddy reformatted my json to merge multiple subdomains of the same domain to wildcard one!
Thanks for all the hard work!

Phew, awesome! I love it when we fix bugs by accident 😁

I even noticed caddy reformatted my json to merge multiple subdomains of the same domain to a wildcard one!

Huh? We don't do any such manipulations of JSON config. I don't understand.

I assume he's referring to a Caddyfile to JSON translation somewhere.

I even noticed caddy reformatted my json to merge multiple subdomains of the same domain to a wildcard one!

Huh? We don't do any such manipulations of JSON config. I don't understand.
You're right, my bad! It must have been rewritten by a script I used! Well that's a nice feature in your backlog now! :smile:


Wrong comment in Waypoint - Modify Array Data With Indexes

// ourArray[1] now equals [1,3,3].
should be
// ourArray now equals [1,3,3].

Working on it.


Migrate ci.adoptopenjdk.net to ci.adoptium.net

I want to kick off the process of migrating the Jenkins server to the Adoptium DNS space. I've listed a few things that will likely need to be changed:

[ ] 1. Create ci.adoptium.net A record pointing to <IP_ADDRESS>
[ ] 2. Create ci.adoptium.net AAAA record pointing to 2a01:4f8:c0c:1804::2
[ ] 3. Update let's encrypt config file on Jenkins to use new DNS
[ ] 4. Update any hosts that don't connect via SSH.
[ ] 5. Update Jenkins URL at https://ci.adoptopenjdk.net/configure
[ ] 6. Update Jenkins logo to use Adoptium logo
[ ] 7. Update any references to ci.adoptopenjdk.net in Adoptium repos
[ ] 8. Ensure that ci.adoptopenjdk.net still redirects to prevent breaking existing bookmarks

Request for DNS records to be created https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/2670

I suspect we will have a long tail of internal and external scripts referencing the old URL.  Probably need to add a few tasks for that.

do we have issue to update https://trss.adoptopenjdk.net to something similar?

@zdtsw I think TRSS is already running at https://trss.adoptium.net. There seems to be a redirect for me at least.

@zdtsw I think TRSS is already running at https://trss.adoptium.net. There seems to be a redirect for me at least.

ok, nice! probably just need to update our job config

ok, nice! probably just need to update our job config

Which job are you referring to @zdtsw?

I presume we will still redirect from ci.adoptopenjdk.net to ci.adoptium.net ... otherwise, I will also need to update which pipelines we monitor in TRSS.

ok, nice! probably just need to update our job config

Which job are you referring to @zdtsw?

it is https://ci.adoptium.net/job/nightlyBuildAndTestStats_temurin/, we have job not created by scripts. i've updated the config for both URl:s in this job.

Things seem good here, no issues observed so I'm going to mark this as closed.


[Jetcaster] Migrate from M2 to M3.

Upgrading Jetcaster from M2 to M3. This migration changes one of the original purposes of the sample which was to demonstrate dynamic theming. We would still like to demonstrate dynamic theming but using a more modern approach using Material's material-color-utilities.

The status bar color doesn't look correct in your screenshot, should be white.

The status bar color doesn't look correct in your screenshot, should be white.

That change is intentional — though the status bar should match surface color. Updating that

Does the contrast of the text on the status bar color look correct now? Hard to tell from the code

Does the contrast of the text on the status bar color look correct now? Hard to tell from the code

Ahh good catch. I didn't notice the status bar text contrast. Looks like there was a bug in JetcasterTheme.


Swipe for android for stack layout

In Android swipe for stack layout is from left side but i want that swipe from right side like ios,what can i do for it?

@RutujaSoftKoash did you found solluttion?

Any news on this?


Add support for notifying the delegate when changes are detected in parts list, server list, device type list, etc in the descriptor cluster

Any changes in the above attributes are treated as device configuration changes and should be notified to the delegate.

Duplicate of #32353


Feature: Validate BagIt bags

When Enduro receives a BagIt Bag as a SIP or from pre-processing, it currently converts the bag to an Archivematica Standard Transfer and sends this to preservation as the PIP (preservation information package).  If the bag has been corrupted in delivery to Enduro then the PIP will still be sent to preservation, at the cost of unnecessary system resource usage and network traffic.
Describe the solution you'd like
It would be better to validate Bags in Enduro to catch errors early (e.g. file corruption) and avoid unnecessary system resource or network usage. The bags should be validated just before they are zipped and delivered to preservation (see diagram below) to validate bags after possible pre-processing changes.  If bag validation fails, halt the Enduro workflow and send the SIP to a "failed SIP" location.

Describe alternatives you've considered

Validate bags in preservation. As noted, this wastes system and network resource unnecessarily.
Validate SIPs that are bags upon receipt. This would require a separate bag validation step for any bags produced by pre-processing.

Additional context
See issue #805 for more discussion of the proposed Enduro Bagit workflow.

In Enduro, we can see that the bag is being created and then validated. It's hard to look at the bag at this point, since it moves rapidly into Archivematica's processing queue, but looking at the Archivematica tasks we can see that Archivematica treats the transfer as a bag, unzips it, and verifies it. This two-part check gives me enough confidence to call this closed.


Add CRTP information to "Contributing" section of docs

PR Summary
Adds information about the CRTP to the "Contributing" section of the sphinx documentation
I also added a placeholder for somebody to talk about mpark::variant and the Variant class.
Fixes https://github.com/lanl/singularity-eos/issues/93
PR Checklist

[N/A] Adds a test for any bugs fixed. Adds tests for new features.
[N/A] Format your changes by using the make format command after configuring with cmake.
[x] Document any new features, update documentation for changes made.
[x] Make sure the copyright notice on any files you modified is up to date.

@Yurlungur and @dholladay00 let me know what you think. Just a reminder again, I haven't been programming in C++ for long so I may not understand all of the subtleties here. Feel free to suggest minor changes or just push commits yourself for more major changes

@jhp-lanl we've kind of let this MR slide. Do you want to pull it through? I can also take a stab at making the suggested revisions if you want.

@jhp-lanl we've kind of let this MR slide. Do you want to pull it through? I can also take a stab at making the suggested revisions if you want.

If you have the cycles, feel free! I'm focusing on the Gruneisen stuff right now, but I can get to this after I've tackled that.

Haha ok. I will see if I have a spare few minutes or not. I think you already got us 90% of the way there.


eroshare ripper now grabs album name

Some test albums
https://eroshare.com/dz8rx7mt
https://eroshare.com/63uw1r2x

Looks like the first one has an album name and the second one doesn't. Both look good. Thanks!


feat(linter/jsx-a11y): add fixer for aria-unsupported-elements

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#4855
#4854  👈
#4853
#4852
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @DonIsaac and the rest of your teammates on  Graphite


Remove unused type alias 'value_type' in xt::mean()

Remove unused type alias in xt::mean(). (Now unused after #435.)
Eliminates a warning from clang.

Thanks!


Ramda documentation references<EMAIL_ADDRESS>

Documentation https://ramdajs.com/docs references in html following javascript file: https://ramdajs.com/docs/dist/ramda.js. Unfortunately this file is<EMAIL_ADDRESS>bundle instead of<EMAIL_ADDRESS>

Yes, I noticed this too.  I will try to look at why tomorrow.

I believe this bundle should be updated to latest one as some examples may fail in console e.g. R.adjust because of changes in argument ordering.

Wow, I dropped that.  Just getting back into looking at Ramda stuff.  I'll see what I can do tonight.

Seems like issue #2803 is a duplicate of this one?

@customcommander yes, #2803 is just another manifestation of the issue. I've just checked and the issue is still present.

@char0n We just published an update to the documentation. This should be fixed now. Could you please verify and close the issue if you're happy with the resolution?

I can confirm the latest version 0.28.0 is now served inside documentation:
dist/ramda.js:
//  Ramda v0.28.0
//  https://github.com/ramda/ramda
//  (c) 2013-2022 Scott Sauyet, Michael Hurley, and David Chambers
//  Ramda may be freely distributed under the MIT license.


[et] Update execuion trace dep to support new format

What does this PR do?
Updates to new Chakra schema carried forward in Param repo
Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)

[x] N/A

[x] Did you write any new necessary tests?

[ ] N/A

[ ] Did you make sure to update the docs?

[x] N/A

[x] Did you update the changelog?

[ ] N/A

Codecov Report
All modified lines are covered by tests :white_check_mark:

Comparison is base (66e3c54) 0.00% compared to head (950a906) 91.06%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##           main      #72       +/-   ##
=========================================
+ Coverage      0   91.06%   +91.06%
=========================================
  Files         0       30       +30
  Lines         0     2562     +2562
=========================================
+ Hits          0     2333     +2333
- Misses        0      229      +229

Files
Coverage Δ

tests/test_execution_trace.py
100.00% <100.00%> (ø)

... and 29 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


docs: Fix nav clutter on installation pages

Actually, it makes more sense for this to be the default toc level


Convert add-custom-domain.md to yml

…als/add-custom-domain-old.md
new file:   docs/fundamentals/add-custom-domain.yml

@mattbriggs @rwestMSFT Please review

@Taojunshen - please re-open this in the private repo entra-docs-pr thanks


create ready to use bundle

I would like to be able to use element-js via cdn (unpkg i.E.) for rapid prototyping.

This is not as easy as I thought...
All the known CDNs only serve from the normal npm registry and don't have a mechanism to change the registry. Since we are hosting our packages on the https://npm.pkg.github.com/ registry we currently cannot simply use those tools.
I have requested feedback for unpkg and jsdelivr:
https://github.com/mjackson/unpkg/issues/277
https://github.com/jsdelivr/jsdelivr/issues/18261
another option could be to host unpkg ourselfs (https://github.com/mjackson/unpkg/pull/216) - but I'd rather not..

A solution that we could use for the time being is manually generating the /dist folder locally and pushing it up. If the /dist folder was inside the git repository we could fetch the files via jsDelivr like so:
<EMAIL_ADDRESS>But I'm pretty sure that this would be quite error-prone...

As mentioned in the comment above, I have a "solution" in https://github.com/webtides/element-js/compare/feature/dist-bundle
The GitHub action will run the "npm run build" script and push it back to the main branch. With this in place we would be able to read the /dist bundles via<EMAIL_ADDRESS>The only problem with this is that we will only be able to read from @latest and not from specific tags. This is because the action will always run after the tagged commit...
But it would work in theory! See: https://codepen.io/eddyloewen/full/abZdNaj

The GitHub action will run the "npm run build" script and push it back to the main branch. With this in place we would be able to read the /dist bundles via<EMAIL_ADDRESS>
If you go this way, I recommend tagging sources as v0.4.0-source on the main branch and then pushing dist to a separate branch, and tagging that as v0.4.0.

If you go this way, I recommend tagging sources as v0.4.0-source on the main branch and then pushing dist to a separate branch, and tagging that as v0.4.0.

Yeah! That is a good idea 👍 But I think I would then prefer to not use the GitHub package registry and instead use the NPM registry.

But I think I would then prefer to not use the GitHub package registry and instead use the NPM registry.

Yes, that sounds better.


FP fix: Update link_microsoft_low_reputation.yml

Adding more MS domain to linked domains, and requiring cred_theft of med/high confidence. Was firing on cred_theft low, on images with only a single word "Microsoft"

These regressions do not contains any microsoft specific content, the rule explicitly looks for a MS logo, and the failed regressions are not Microsoft themed at all. These are acceptable regressions.


cairo: fix build on darwin

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

@ofborg eval

This worked for me.  Updated nixpkgs to master this afternoon, cairo did not build without this and did build with this, on Mac OS.


Can't manage to specify images per page for http://meme.booru.org/

App crashes if images changed from sources tab. Changed manually in the .ini file, doesn't seem to have an effect on the images per page on the download tab.
Can you add meme.booru.org? Seems to be a derivative of  Gelbooru Beta 0.1.11  but slightly different.

I couldn't get a crash when using the source.
Browsing and downloading worked as expected:

About changing the images per page for this source, it seems to just not be supported by Gelbooru 0.1.11 on which meme.booru.org is based on.


Collada texture images

Additions

The COLLADA/KML/glTF export now supports exporting textures to binary glTF files .glb.

The COLLADA/KML/glTF export now supports exporting textures of .gltf files in Draco compression.

Changes

The option to embed textures in .gltf files is now set to true by default to reduce network calls while visualizing. If not selected, the texture images shall be moved to the folder Appearances to reduce redundancies.

The option to embed textures will be automatically activated once the option to export in .glb is selected.

Optimized COLLADA/KML/glTF export to prevent redundant texture as well as visualization files of implicit geometries from being exported repeatedly leading to file redundancy. This includes the following changes:

Exported textures are now stored in the folder Appearances.  Each texture has a unique name, i.e. no texture shall be overwritten.

COLLADA .dae and .gltf files of implicit geometries shall be exported in the folder ImplicitObjects. Similarly, they all have unique name to prevent overwriting. The .kml file in each tile e.g. X/Y shall point to these files, so that the 3DCityDB-Web-Map-Client can visualize them as usual. Empty folders such as X/Y/<gmlid> that used to contain the now relocated .dae and .gltf files shall be removed.

I've made some tests with Exporter and Webclient. looks good!
Two remarks:

Maybe the folder ImplicitGeometry should be renamed to e.g. ImplicitObject(s) or LibraryObject(s), because the dae and gltf objects also contain non-geometry information.
Wenn activating the Collada/glTF display rendering option Crop texture images, a number of non-used empty JPEG images are created. Maybe a tiny bug.

maybe it is better to rename the folder name "appearance" to "Apperances", so that the folder names "Appearances", "Tiles", and "ImplicitObjects" have a consistent form?

What is the status of this PR?

Son still has to look into an issue where the textureAtlas export is not correct. Once the issue has been fixed by Son, I'll resolve the git branch conflicts, and merge the PR.

The fix for texture atlas in implicit objects has been committed. I did some tests using the railways dataset and so far so good. @yaozhihang please review.

@Son-HNguyen thanks for the fix. The export of texture atlas works well now.
However, there is a further bug related to the export of implicit objects. When you export the vegetation objects in KMZ format, the Google Earth throws the following error message while loading the KMZ data:
"..../ImplicitObject/38.dae" Fehler: Datei konnte nicht gelesen werden

@yaozhihang Thanks for the info. I'm working on it.

This branch shall be split into two smaller branches for better management. The first branch is glb-draco-support containing the support for export of glb and with Draco compression. The second branch will contain the changes to how we deal with implicit objects.

For the second branch to handle implicit objects, after a discussion with @yaozhihang, we are reconsidering whether:

We should store texture images of all objects in one folder Appearances and implicit objects (.dae, .gltf and .glb) in the folder ImplicitObjects (as implemented so far in this PR). This removes most of the file redundancies but may cause some performance issues when the number of texture files becomes very large (since all textures are stored in one single folder).
OR we should store all implicit objects (.dae, .gltf and .glb) and their texture images in one folder ImplicitObjects. Other objects that are not implicit shall be exported in the same way as in master, i.e. to export them (.dae, .gltf and .glb) together with their texture images in the respective tiles folders. This could improve the visualization performance, since not all textures are stored in the same folder, but might contain some redundancies.

@Son-HNguyen, @yaozhihang: what is the status of this PR? Are you still working on this?

This PR was split into two halves, the first half has already been merged to master. The second half, which is about restructuring the export directories (see my comment above https://github.com/3dcitydb/importer-exporter/pull/112#issuecomment-697285653), has not been further worked on, since, if I understood him correctly, @yaozhihang prefers the existing structure of the export directories. For archiving purposes, we'd suggest still keeping this PR online (and marking it as a draft as you already did).

@Son-HNguyen yes, I preferred the second option, because putting a large number of all textures images into a single folder may cause performance problem in Windows OS. Anyway, the implementation of the 1. or 2. option is quite easy.
The difficult thing is how to handle the archived exports (KMZ). The key question is how to reference the external implicit object (e.g. .dae) from the archived KML file. I think @Son-HNguyen is working hard on this currently.

Closing this PR as there is no obvious development going on. Please start a new issue or a new PR for parts of this PR if they make sense and can be completed.


angular meteor not working

I've been using this package in regular angular apps with no problem at all
But I've tried using it with angular-meteor and when I add the directive ui-router-styles it crashes my app.
Errors prevented startup:

While processing files with angular-templates (for target web.browser):
client/index.html:18: Parse error in tag

While processing files with angular-templates (for target web.cordova):
client/index.html:18: Parse error in tag

Your application has errors. Waiting for file change.

<body ng-controller="IndexCtrl as ic" ui-router-styles>

Still have the issue?


carsh "Failed to initialize filesystem: no error" / crash on windows 10 1809

reaseon:
library physfs on window 1809 will error when "__PHYSFS_platformCalcUserDir".\
todo :
love2d team has solve the problem, need to update love source and re-compile.
file path : love\src\libraries\physfs\physfs_platform_windows.c
before:

after:

version 11.0.12 solve this problem :
https://www.nuget.org/packages/Love2dCS/11.0.12


The ElementName scope in Binding differs between TreeView and ListBox

This Content binding works:
<TabControl Classes="default">
    <TabItem Header="Shapes" Classes="property">
        <TreeView Name="shapes" Items="{Binding Path=Shapes}" Classes="default">
            <TreeView.DataTemplates>
                <DataTemplate DataType="core:BaseShape">
                    <TextBlock Text="{Binding Mode=OneWay, Converter={Static converters:ObjectToTypeStringConverter.Instance}}" Classes="default"/>
                </DataTemplate>
            </TreeView.DataTemplates>
        </TreeView>
    </TabItem>
    <TabItem Header="Selected" Classes="property">
        <ContentControl Content="{Binding Path=#shapes.SelectedItem, Mode=TwoWay}"/>
    </TabItem>
</TabControl>

and this Content binding does not work:
<TabControl Classes="default">
    <TabItem Header="Shapes" Classes="property">
        <ListBox Name="shapes" Items="{Binding Path=Shapes}" Classes="default">
            <ListBox.DataTemplates>
                <DataTemplate DataType="core:BaseShape">
                    <TextBlock Text="{Binding Mode=OneWay, Converter={Static converters:ObjectToTypeStringConverter.Instance}}" Classes="default"/>
                </DataTemplate>
            </ListBox.DataTemplates>
        </ListBox>
    </TabItem>
    <TabItem Header="Selected" Classes="property">
        <ContentControl Content="{Binding Path=#shapes.SelectedItem, Mode=TwoWay}"/>
    </TabItem>
</TabControl>

The ListBox version is actually the correct behavior: when a tab goes unselected, its container is dematerialized meaning the controls in it no longer exist, however there was a bug which caused the TreeView's removal to go unrecognised.

The WPF keeps controls alive while TabControl is visible.

Yeah, you've just made me realize that the control's lifetime is currently scoped to its existence on the visual tree, whereas it should be scoped to its existence on the logical tree. I've opened a new issue: #340.


Allow defaultSorted to use data properties, not just row properties

I'm using react-table v6.8.6.
What I'd like to do is have the table sorted by default using a property in my data that I don't want to show in the table. Specifically in this case createdAt, though I have another use case where I'll be doing something similar with viewOrder.
Adding defaultSorted={[ { id: 'createdAt', desc: true } ]} only works when I have a column defined with an ID of createdAt. If I remove the column definition, the sort function is passed undefined values. From what I gather, the sort id is the name of the property from the row, not the original data.
If there was a way to reference the original data row in the sort expression, that would be ideal.
For instance, something like defaultSorted={[ { id: 'original.createdAt', desc: true } ]}. To avoid conflict with existing properties, the original property could be prefixed: .original.createdAt or @original.createdAt, for example.
Or an additional property could be added to the sort options that would specify the data property instead of the row property: defaultSorted={[ { dataId: 'createdAt', desc: true } ]}.
For now, I'm pre-sorting the data before I pass it into react-table, but this splits the logic of where my sorting is occurring, and makes it more confusing what's going on in the code.

I agree. The sortMethod is called with formatted/accessed values, not the original row, which can make sorting really difficult. For example, if the table shows localized values, I would have to un-localize the parameters and then sort them.
A work-around is to have the accessor return a JSX.Element and then use the props property in the sortMethod. This is clearly not ideal.

This feature/issue has been tagged as one that will likely be fixed or improved in the next major release of React-Table. This release is expected to be completed in the next 6 - 8 months. Please see #1143 for more detail. If you believe this feature can be easily fixed, we invite you to fork the project, attempt the fix, and use it until the latest version is available. If the fix is simple enough, we will also consider it as a PR to the current version.


Double notifications on  IOS only Foreground State, Background  and kill Working fine

I am getting 2 Notifications in foreground state. Same In Kill and Background working fine
Version : - "@react-native-community/push-notification-ios": "^1.8.0",
react-native: 0.62.3
Notification Type : APNS Alert
We received 2 different  NotificationId same time like :-

&&&&& App:: onNotif called {"action": undefined, "badge": 1, "data": {"notificationId": "5D37FB9D-A6BE-418D-913A-9F6C62957CA6", "remote": true}, "finish": [Function finish], "foreground": true, "id": undefined, "message": "You have missed call from \"1008\"", "reply_text": undefined, "soundName": "default", "subtitle": undefined, "title": "Missed Call", "userInteraction": false}

2: - &&&&& App:: onNotif called {"action": undefined, "badge": 1, "data": {"notificationId": "C1010B25-2DD1-4086-B6EA-0FABB3F3A91A", "remote": true}, "finish": [Function finish], "foreground": true, "id": undefined, "message": "You have missed call from "1008"", "reply_text": undefined, "soundName": "default", "subtitle": undefined, "title": "Missed Call", "userInteraction": false}
Received  2 Different Notification Id :
"notificationId": "5D37FB9D-A6BE-418D-913A-9F6C62957CA6",
"notificationId": "C1010B25-2DD1-4086-B6EA-0FABB3F3A91A",
All Notification Received onNotif event
async onNotif(notif) {
console.log("&&&&& App:: onNotif called", notif);
Note :  Same Case Working in Background and Kill State but foreground case received 2 Notification. And Also we received 2 event only foreground state
Please help me if you have any idea for this issues.

I have found the solution
In App delegate we have added 2 time
didReceiveRemoteNotification
Now it working fine


single header contains local includes in some CI's only

wanted to sort out local includes inside "simdjson" with "simdjson/" in front .. then I noticed some CI's are complaining ... see the image below with the apparent cause...

The singleheader version is generated by make amalgamate, which you can run locally. I changed it recently to find headers more dynamically, and perhaps that broke something?
I see both #include "simdjson/x.h" and #include "x.h" in the simdjson.h headers. You're right that we should really pick one.

In any case single header simdjson.h  including "simdjson/x.h " is not a good idea, I might be sure you would agree :)

I see both #include "simdjson/x.h" and #include "x.h" in the simdjson.h headers. You're right that we should really pick one.

If "x.h" and "y.h" are in the same folder, why not just include "x.h" :)
In the repo right now, single header simdjson.h does not contain local includes.  only those very few CI scripts have a problem ...

@DBJDBJ @jkeiser
I am going to close this PR without merging. It fails our tests. If there is an issue with the headers, please submit a patch that passes our tests. Currently, I cannot reproduce any error with the amalgamation script. If there is an error, please suggest a test that exposes the bug.

@lemire what happens if you do this same simple change on your side?

The amalgamation script works and is part of the CI. If I make your proposed change then it breaks the build as you report. So we won’t break the build.


Rename TupleExprSyntax.elementList to  TupleExprSyntax.elements

Resolves #1713
This is to be more consistent in naming

@swift-ci please test


implement choice of delimiter for seed files

resolves #1119
Problem
CSV seed work only with ',' delimiter.
Solution
This PR solves the problems by adding parameter of delimiter to the macro which build the table from the csv file
Checklist

[x] I have read the contributing guide and understand what's expected of me
[x] I have run this code in development and it appears to resolve the stated issue
[x] This PR includes tests, or tests are not required/relevant for this PR

Looking good so far @salimmoulouel 👍
A key thing we'll need is a functional test to make sure the change works.
Thankfully, we already have a relevant test defined here -- we just need to inherit and run it within dbt-bigquery!
Here's how to do it:

Add the following import to the top of this file:

from dbt.tests.adapter.simple_seed.test_seed import BaseSeedWithUniqueDelimiter

Add this new code right under here:

class TestBigQuerySeedWithUniqueDelimiter(BaseSeedWithUniqueDelimiter):
    pass

Here's instructions how to run the test locally to make sure it breaks before your change but works once it is included.

Looking good so far @salimmoulouel 👍
A key thing we'll need is a functional test to make sure the change works.
Thankfully, we already have a relevant test defined here -- we just need to inherit and run it within dbt-bigquery!
Here's how to do it:
1. Add the following import to the top of [this file](https://github.com/dbt-labs/dbt-bigquery/blob/5501cd34b12965e0380952bdbb65fd52f49b49f5/tests/functional/adapter/test_simple_seed.py):

from dbt.tests.adapter.simple_seed.test_seed import BaseSeedWithUniqueDelimiter

2.

        This method will poll for operation status periodically, blocking if
        necessary. If you just want to make sure that this method does not block
        for more than X seconds and you do not care about the nitty-gritty of
        how this method operates, just call it with ``result(timeout=X)``. The
        other parameters are for advanced use only.

        Every call to this method is controlled by the following three
        parameters, each of which has a specific, distinct role, even though all three
        may look very similar: ``timeout``, ``retry`` and ``polling``. In most
        cases users do not need to specify any custom values for any of these
        parameters and may simply rely on default ones instead.

        If you choose to specify custom parameters, please make sure you've
        read the documentation below carefully.

        First, please check :class:`google.api_core.retry.Retry`
        class documentation for the proper definition of timeout and deadline
        terms and for the definition the three different types of timeouts.
        This class operates in terms of Retry Timeout and Polling Timeout. It
        does not let customizing RPC timeout and the user is expected to rely on
        default behavior for it.

        The roles of each argument of this method are as follows:

        ``timeout`` (int): (Optional) The Polling Timeout as defined in
        :class:`google.api_core.retry.Retry`. If the operation does not complete
        within this timeout an exception will be thrown. This parameter affects
        neither Retry Timeout nor RPC Timeout.

        ``retry`` (google.api_core.retry.Retry): (Optional) How to retry the
        polling RPC. The ``retry.timeout`` property of this parameter is the
        Retry Timeout as defined in :class:`google.api_core.retry.Retry`.
        This parameter defines ONLY how the polling RPC call is retried
        (i.e. what to do if the RPC we used for polling returned an error). It
        does NOT define how the polling is done (i.e. how frequently and for
        how long to call the polling RPC); use the ``polling`` parameter for that.
        If a polling RPC throws and error and retrying it fails, the whole
        future fails with the corresponding exception. If you want to tune which
        server response error codes are not fatal for operation polling, use this
        parameter to control that (``retry.predicate`` in particular).

        ``polling`` (google.api_core.retry.Retry): (Optional) How often and
        for how long to call the polling RPC periodically (i.e. what to do if
        a polling rpc returned successfully but its returned result indicates
        that the long running operation is not completed yet, so we need to
        check it again at some point in future). This parameter does NOT define
        how to retry each individual polling RPC in case of an error; use the
        ``retry`` parameter for that. The ``polling.timeout`` of this parameter
        is Polling Timeout as defined in as defined in
        :class:`google.api_core.retry.Retry`.

        For each of the arguments, there are also default values in place, which
        will be used if a user does not specify their own. The default values
        for the three parameters are not to be confused with the default values
        for the corresponding arguments in this method (those serve as "not set"
        markers for the resolution logic).

        If ``timeout`` is provided (i.e.``timeout is not _DEFAULT VALUE``; note
        the ``None`` value means "infinite timeout"), it will be used to control
        the actual Polling Timeout. Otherwise, the ``polling.timeout`` value
        will be used instead (see below for how the ``polling`` config itself
        gets resolved). In other words, this parameter  effectively overrides
        the ``polling.timeout`` value if specified. This is so to preserve
        backward compatibility.

        If ``retry`` is provided (i.e. ``retry is not None``) it will be used to
        control retry behavior for the polling RPC and the ``retry.timeout``
        will determine the Retry Timeout. If not provided, the
        polling RPC will be called with whichever default retry config was
        specified for the polling RPC at the moment of the construction of the
        polling RPC's client. For example, if the polling RPC is
        ``operations_client.get_operation()``, the ``retry`` parameter will be
        controlling its retry behavior (not polling  behavior) and, if not
        specified, that specific method (``operations_client.get_operation()``)
        will be retried according to the default retry config provided during
        creation of ``operations_client`` client instead. This argument exists
        mainly for backward compatibility; users are very unlikely to ever need
        to set this parameter explicitly.

        If ``polling`` is provided (i.e. ``polling is not None``), it will be used
        to control the overall polling behavior and ``polling.timeout`` will
        control Polling Timeout unless it is overridden by ``timeout`` parameter
        as described above. If not provided, the``polling`` parameter specified
        during construction of this future (the ``polling`` argument in the
        constructor) will be used instead. Note: since the ``timeout`` argument may
        override ``polling.timeout`` value, this parameter should be viewed as
        coupled with the ``timeout`` parameter as described above.

        Args:
            timeout (int): (Optional) How long (in seconds) to wait for the
                operation to complete. If None, wait indefinitely.
            retry (google.api_core.retry.Retry): (Optional) How to retry the
                polling RPC. This defines ONLY how the polling RPC call is
                retried (i.e. what to do if the RPC we used for polling returned
                an error). It does  NOT define how the polling is done (i.e. how
                frequently and for how long to call the polling RPC).
            polling (google.api_core.retry.Retry): (Optional) How often and
                for how long to call polling RPC periodically. This parameter
                does NOT define how to retry each individual polling RPC call
                (use the ``retry`` parameter for that).

        Returns:
            google.protobuf.Message: The Operation's result.

         You might receive a "quota exceeded" or "API not enabled" error. See the following page for troubleshooting: https://cloud.google.com/docs/authentication/adc-troubleshooting/user-creds.
    warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=================================================================== short test summary info ===================================================================
ERROR tests/functional/adapter/test_simple_seed.py::TestBigQuerySeedWithUniqueDelimiter::test_seed_with_unique_delimiter - dbt_common.exceptions.base.DbtDatabaseError: Database Error
================================================================ 2 warnings, 1 error in 8.00s =================================================================

to get arround this problem i inherited the TestSimpleSeedConfigs and i modified the delimiter of the csv used in the seed, i get an error before applying change and it disappears after the changes

@dbeatty10  i didn't get any news since my last commit, is the pull request okay ?

Hello, this merge request is becoming crucial for my project, i'm working for big french company. Could you please confirm if it's acceptable?
Thanks a lot :)

Hope you're doing well! I wanted to chat about some merge requests I've submitted to dbt. It's been over Two months since I sent them in, and I haven't heard a peep back. Any chance you could give me the lowdown on what's up with them?
I get that running an open-source project can be hectic, and I totally respect the hustle. But it's a bit frustrating not knowing where my contributions stand. I've put some real effort into these requests and would love to see them get some love.
If there's anything I need to tweak or if my requests aren't quite hitting the mark, I'm all ears. Just looking for some clarity so I can get back in the game!
Thanks a bunch for your time, and looking forward to hearing from you soon.

@salimmoulouel Sorry for the delay LGTM, can you just update the code formatting? (just need to run pre-commit )


gpt3

I implemented the gpt3 model request. I only added the davinci modell to the enum, because it is the normal gpt3 model. All the other models could be used with custom or be added to the enum. The responses are in a enum right now. Its not optimal to work with, so i am open for suggestions how it should be refactored.

Thank you for the pull request. I currently do not see the need for GPT-3 support in this crate as it was originally designed for ChatGPT models (so GPT-3.5 and later). Integrating GPT-3 there would be pretty hard. Plus, GPT-3.5-turbo is cheaper to use than GPT-3 (assuming you want to use Davinci), with 0.002$/1k tokens vs the 0.02$/1k tokens of the Davinci model. If you're interested in working with GPT-3, You should take a look at other OpenAI crates, such as https://github.com/deontologician/openai-api-rust

its integrated pretty well in my opinion. The output has the same format and the only difference is the creation. The benefits of gpt-3 are that its more accurate than 3.5turbo. gpt 4 isnt publicly available.

Alright, I'll have a look at how to integrate it as a crate feature tomorrow!

Thanks for the consideration

I have looked more into integrating it, and found out that a lot of logic will have to be rewritten for it to work properly as a feature. I'm afraid that I will have to close this PR.


fix unreadable font on light theme

Hi! Awesome page.
When browsing in light mode OS settings card text is black on black:

after this fix:

@bgalek @max-programming already done #165

Fixed already. Thanks anyways!


Ubuntu 24.04 support

Ubuntu 24.04 is already in its feature freeze. We need time to test, report and fix the bugs to be able to get them fixed right before the release, rather than long after.

Related to https://github.com/ROCm/ROCm/issues/2939

needs to be handled by AMD per https://bugs.launchpad.net/ubuntu/+bug/2060391

This seems rather critical since Ubuntu 22.0.4 has a known issue with
[ 7813.309408] workqueue: ttm_bo_delayed_delete [amdttm] hogged CPU for >10000us 4 times, consider switching to WQ_UNBOUND
Which is fixed in 23.0.4 (and thus 24.0.4)
https://gitlab.freedesktop.org/drm/amd/-/issues/2282#note_1901512
Right now it's pretty easy to crash any apps using the GPU doing any tensorflow or onyx on ubuntu because of this

Please it would be important for tools like pytorch, hashcat, blender, etc which needs gpu

With amdgpu-install_6.1.60101-1_all.deb on Ubuntu 24.04 (6.9.2 kernel):
$ amdgpu-install --usecase=rocm
[…]
The following packages have unmet dependencies:
 amd-smi-lib : Depends: python3-pip but it is not installable
               Recommends: python3-argcomplete but it is not installable
 hipsolver : Depends: libcholmod3 but it is not installable
             Depends: libsuitesparseconfig5 but it is not installable
 rocm-gdb : Depends: libtinfo5 but it is not installable
            Depends: libncurses5 but it is not installable
            Depends: libpython3.10 but it is not installable or
                     libpython3.8 but it is not installable
 rocm-llvm : Depends: libstdc++-5-dev but it is not installable or
                      libstdc++-7-dev but it is not installable or
                      libstdc++-11-dev but it is not installable
             Depends: libgcc-5-dev but it is not installable or
                      libgcc-7-dev but it is not installable or
                      libgcc-11-dev but it is not installable
             Recommends: gcc-multilib but it is not going to be installed
             Recommends: g++-multilib but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 hipsolver : Depends: libcholmod3 but it is not installable
             Depends: libsuitesparseconfig5 but it is not installable
 rocm-gdb : Depends: libtinfo5 but it is not installable
            Depends: libncurses5 but it is not installable
            Depends: libpython3.10 but it is not installable or
                     libpython3.8 but it is not installable
E: Unable to correct problems, you have held broken packages.

Same :(

Try https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/native-install/ubuntu.html
or https://github.com/nktice/AMD-AI

Try https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/native-install/ubuntu.html
or https://github.com/nktice/AMD-AI

This worked for me thank you so much !

Try https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/native-install/ubuntu.html or https://github.com/nktice/AMD-AI

This also helped get me to the next step. I now have rocm installed (thank you), but it doesn't seem to have library files for my older AMD GPU:
rocBLAS error: Cannot read /opt/rocm/lib/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1032
 List of available TensileLibrary Files :
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1101.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx940.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1100.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1030.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx900.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx908.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx90a.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx1102.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx941.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx906.dat"
"/opt/rocm/lib/rocblas/library/TensileLibrary_lazy_gfx942.dat"

I'm trying to compile llama.cpp if that makes a difference.

@mikeperalta1
List your GPU here with rocminfo or clinfo so that I could see what is going on here.
If there is no luck with RoCM, I guess you have to try OpenCL route based on clBLAST, which is also a project I helped to tune for speed on llama.cpp.
By the way, I did encounter some hanging behaviour with my 7900XTX Ubuntu 24.04 while selecting any word in the editor of GNU Octave when the .m file is large (I don't know if this is the driver problem). In the meantime, I encountered a WebGPU problem with 7900XTX while no problem on 7800XT (two cards are in two different PCs with different motherboards x570/550 and the same CPU 5700G)

While it might work for some use cases to add the old PPA's, that is also a great way to brick your system and should not become an advised option at all. There's a good reason these PPA's aren't included as a fallback by default. Please take great caution before undertaking any of the sort.

@mikeperalta1 List your GPU here with rocminfo or clinfo so that I could see what is going on here. If there is no luck with RoCM, I guess you have to try OpenCL route based on clBLAST, which is also a project I helped to tune for speed on llama.cpp.
By the way, I did encounter some hanging behaviour with my 7900XTX Ubuntu 24.04 while selecting any word in the editor of GNU Octave when the .m file is large (I don't know if this is the driver problem). In the meantime, I encountered a WebGPU problem with 7900XTX while no problem on 7800XT (two cards are in two different PCs with different motherboards x570/550 and the same CPU 5700G)

I ended up using an ENV trick found elsewhere to get the program to run. Basically a var to force an earlier version that supports my card:
HSA_OVERRIDE_GFX_VERSION=10.3.0

But here's the info in case anyone is interested
*******
Agent 2
*******
  Name:                    gfx1032
  Uuid:                    GPU-XX
  Marketing Name:          AMD Radeon RX 6600 XT
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  Profile:                 BASE_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        128(0x80)
  Queue Min Size:          64(0x40)
  Queue Max Size:          131072(0x20000)
  Queue Type:              MULTI
  Node:                    1
  Device Type:             GPU
  Cache Info:
    L1:                      16(0x10) KB
    L2:                      2048(0x800) KB
    L3:                      32768(0x8000) KB
  Chip ID:                 29695(0x73ff)
  ASIC Revision:           0(0x0)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   2900
  BDFID:                   768
  Internal Node ID:        1
  Compute Unit:            32
  SIMDs per CU:            2
  Shader Engines:          2
  Shader Arrs. per Eng.:   2
  WatchPts on Addr. Ranges:4
  Coherent Host Access:    FALSE
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      TRUE
  Wavefront Size:          32(0x20)
  Workgroup Max Size:      1024(0x400)
  Workgroup Max Size per Dimension:
    x                        1024(0x400)
    y                        1024(0x400)
    z                        1024(0x400)
  Max Waves Per CU:        32(0x20)
  Max Work-item Per CU:    1024(0x400)
  Grid Max Size:<PHONE_NUMBER>(0xffffffff)
  Grid Max Size per Dimension:
    x<PHONE_NUMBER>(0xffffffff)
    y<PHONE_NUMBER>(0xffffffff)
    z<PHONE_NUMBER>(0xffffffff)
  Max fbarriers/Workgrp:   32
  Packet Processor uCode:: 118
  SDMA engine uCode::      76
  IOMMU Support::          None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    8372224(0x7fc000) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Recommended Granule:2048KB
      Alloc Alignment:         4KB
      Accessible by all:       FALSE
    Pool 2
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    8372224(0x7fc000) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Recommended Granule:2048KB
      Alloc Alignment:         4KB
      Accessible by all:       FALSE
    Pool 3
      Segment:                 GROUP
      Size:                    64(0x40) KB
      Allocatable:             FALSE
      Alloc Granule:           0KB
      Alloc Recommended Granule:0KB
      Alloc Alignment:         0KB
      Accessible by all:       FALSE
  ISA Info:
    ISA 1
      Name:                    amdgcn-amd-amdhsa--gfx1032
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Size:      1024(0x400)
      Workgroup Max Size per Dimension:
        x                        1024(0x400)
        y                        1024(0x400)
        z                        1024(0x400)
      Grid Max Size:<PHONE_NUMBER>(0xffffffff)
      Grid Max Size per Dimension:
        x<PHONE_NUMBER>(0xffffffff)
        y<PHONE_NUMBER>(0xffffffff)
        z<PHONE_NUMBER>(0xffffffff)
      FBarrier Max Size:       32

@erkinalp Is this ticket still relevant? If not, please close the ticket. Thanks!

Yes, still cannot install.

@erkinalp Did you try https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/native-install/ubuntu.html

@ppanchad-amd 24.04 not listed in that page, and the APT repo also lacks the release definitions for 24.04.

@erkinalp Sorry, 24.04 is not officially supported yet in the lastest ROCm 6.1.2. Thanks!

Curious if there is any update.  Ubuntu 22.04 is long in tooth for hardware - we're having to run custom build kernels which break ROCm. Would love to just be modern with 24.04

Is there a timeline on this?

Documentation for just-released ROCm 6.2.0 lists official support for Ubuntu 24.04. I've yet to confirm this, though, especially since support for gfx906 is officially dropped.

Install instructions here

Sorry for the wait. Ubuntu 24.04 is now officially supported with the release of ROCm 6.2.0!

When the latest ROCm 6.2.0 has been installed, almost all GUI applications cannot be opened, here's from the journalctl:
Aug 07 07:11:34 dev-ubuntu dbus-daemon[2309]: [session uid=1000 pid=2309] Failed to activate service 'org.freedesktop.portal.Desktop': timed out (service_start_timeout=120000ms)
Aug 07 07:11:34 dev-ubuntu dbus-daemon[2309]: [session uid=1000 pid=2309] Failed to activate service 'org.freedesktop.impl.portal.desktop.gnome': timed out (service_start_timeout=120000ms)
Aug 07 07:12:15 dev-ubuntu dbus-daemon[2309]: [session uid=1000 pid=2309] Activating service name='org.gnome.Nautilus' requested by ':1.85' (uid=1000 pid=3559 comm="gjs /usr/share/gnome-shell/extensions/ding@rasters" label="unconfined")
Aug 07 07:12:15 dev-ubuntu nautilus[10742]: Connecting to org.freedesktop.Tracker3.Miner.Files
Aug 07 07:12:15 dev-ubuntu dbus-daemon[2309]: [session uid=1000 pid=2309] Successfully activated service 'org.gnome.Nautilus'
Aug 07 07:12:15 dev-ubuntu org.gnome.Nautilus[10742]: libEGL warning: DRI2: failed to authenticate
Aug 07 07:12:15 dev-ubuntu org.gnome.Nautilus[10742]: libEGL fatal: DRI driver not from this Mesa build ('24.2.0-devel' vs '24.0.9-0ubuntu0.1')

The Ubuntu 24.04 quick-start install instructions here simply don't work, yielding:
The following packages have unmet dependencies:
 hipsolver : Depends: libcholmod3 but it is not installable
             Depends: libsuitesparseconfig5 but it is not installable
 rocm-gdb : Depends: libpython3.10 (>= 3.10.0) but it is not installable

Is it too much to ask that documenters test the steps they're writing?

@chris-hatton, did some more digging and was able to reproduce these exact dependency errors when using the Ubuntu 22.04/jammy amdgpu-install on Ubuntu 24.04/noble. See the difference in the paths for the two installers below.
wget https://repo.radeon.com/amdgpu-install/6.2.4/ubuntu/jammy/amdgpu-install_6.2.60204-1_all.deb

wget https://repo.radeon.com/amdgpu-install/6.2.4/ubuntu/noble/amdgpu-install_6.2.60204-1_all.deb

Please be sure to use the correct installer according to the OS that you have installed. As for next steps, you can remove the existing amdgpu-install with the commands from my previous comment and install the 24.04/noble packages with
wget https://repo.radeon.com/amdgpu-install/6.2.4/ubuntu/noble/amdgpu-install_6.2.60204-1_all.deb
sudo apt install ./amdgpu-install_6.2.60204-1_all.deb
sudo apt update
sudo apt install amdgpu-dkms rocm


Test suite with load tests

Prepare test suite with load tests
#123

To be done with https://github.com/Nitrokey/nitrokey-3-tests.
To do:

[ ] add link to readme


Session timer

Hey, I haven't had time to look closely at this, and I just wanted to thank you for the PR and for being patient with us on merging it.  New features like this will have to go through a little more consideration and likely some tuning before being merged into the core library.

Hi @vassjozsef,
Has this been tested at all? I looked this over with @joseph-onsip offline for a few minutes, and he explained that his initial commits were scratch work and needed testing.
An example of this code working in production would be good, but some tests added to our jasmine testing set would be better. Both of those together would be ideal.
Thanks,
James

Yes, it is has been tested, it is running in our production. I will add tests.

@vassjozsef, have you made any progress with the tests?

This work has been added to and is now being tracked here:
https://github.com/onsip/SIP.js/tree/session-timers
The current status of the branch is very much a work in progress, but it is a bit more fleshed out to cover more session timer features. I will be closing this, as the PR itself is no longer most up to date, but we will be tracking progress from the original issue:
https://github.com/onsip/SIP.js/issues/18
Thanks,
James


Deposit token at starting application doesn't work?

I don't know the meaning of the deposit token when creating a new application, cause even i set a big amount of it it seems to be no difference in the next stages. I thought those tokens is used to limit the challengers as described here https://camo.githubusercontent.com/f16e12e98f8f8c5d79e7318bff1889b2c78363b9/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f656c61737469636265616e7374616c6b2d75732d656173742d312d3136373631313735323837342f64657461696c65642d776f726b666c6f772e706e67.
Sorry if i misunderstand something here.

I meant what is the difference when i start an application with 10 deposit token and 100 deposit token, cause it seems to be no difference to me

if you deposit with 10, and the minDeposit changes to 11, your application will be at risk to be "touch-and-removed" - if someone calls challenge on that application, it's immediately removed without going to voting. see link above for more info.

oh, so the deposit at starting application stage only to deal with "touch-and-removed". At first when reading this: https://camo.githubusercontent.com/f16e12e98f8f8c5d79e7318bff1889b2c78363b9/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f656c61737469636265616e7374616c6b2d75732d656173742d312d3136373631313735323837342f64657461696c65642d776f726b666c6f772e706e67 , i thought it defined the number of tokens that a challenger need to challenge the application. Correct me if i misunderstand

yes, the applicant's "staking deposit" is equivalent to the minDeposit because that's the only amount that's actually at risk. in the code, it's actually termed unstakedDeposit because of this reason.

so the motivation of applicants to put the "deposit token" high when starting a new one is to avoid the "touch-and-removed" edge case ?

yup, exactly


Refac state remove connections

removed connections from state (connections are now only retrieved from the ones within needs)
TODO:
connections-actions method getEvents -> eventContainer must be retrieved without that "hardcoded" connuri+"/events"

^ we should either remove the color highlight or -- preferably -- not show the number when it's 0.
sign-in: when entering a wrong password, the popup still closes, thus hiding the error message

^ notification counter for hint's failed

^ this should read: "This is a need automatically created by the DebugBot." (description) and "
Greetings! I am the DebugBot...." (message)
{wip}

Imo not that many regression bugs, given a refactoring of this pervasiveness and the low refactoring support javascript allows.

issue 2 is a bug that was present before refactoring as well (issue #945)
issue 4 and issue 6: are having the same ( we need to search through the events and store the connect messages within need-reducer, after storing the connection)
issue 8: is not reproducible for me (no line of code found within our code that uses length instead of size)
issue 9: is probably just a lag/data retrieval issue
please recheck the pull request and test the fixed issues,and make bug entries for the still existing issues, as they are not really blocking in my opinion


vinyl-fs dependency

Hi everyone,
I just wanted to report that gulp uses an old version of vinyl-fs (@0.3.0 is used, @2.4.2 is the newest).
The problem is that<EMAIL_ADDRESS>depends on<EMAIL_ADDRESS>which is deprecated and marked as non working on the most recent versions of node.
dvitali ~ npm info<EMAIL_ADDRESS>dependencies
{ 'glob-stream': '^3.1.5',
  'glob-watcher': '^0.0.6',
  'graceful-fs': '^3.0.0',
  'lodash.defaults': '^2.4.1',
  'map-stream': '^0.1.0',
  mkdirp: '^0.5.0',
  'strip-bom': '^0.3.0',
  through2: '^0.5.1',
  'through2-map': '^1.3.0',
  vinyl: '^0.2.0' }

The following warning is thrown
npm WARN deprecated<EMAIL_ADDRESS>graceful-fs version 3 and before will fail on newer node releases. Please update to<EMAIL_ADDRESS>as soon as possible.

Hi, denysvitali
I came across the same issue yesterday, but found #1571. To  me it seemed obvious that it couldn't be due to a devDependency as phated claims, because, well, I get the deprecation warning on a regular npm install, and that shouldn't install devDependencies ... I didn't dare do as you did now, starting a new issue, since #1571 says in bold letters we are not allowed to. So. Kudos to you.

Same think I thought... Closing as per #1571

Better to be safe and sorry! :-)
I also see another glpjs dev says to update to gulp 4 in #1537. But, as far as I can see, you are totally correct. The warning is because of a dependency to an old version of vinyl-fs, while phated claims the log warning comes from a devDependency. Plus he closes the possibility to comment on the issue, so we are not allowed to correct him ...

The graceful-fs in the vinyl dependency won't be upgraded due to it being a breaking version and vinyl-fs in gulp 3.x won't be upgrade, again, because there are breaking changes.  Use gulp 4 or ignore the deprecation warning.

Thanks for the info @phated.

So, basically, should I gulp => npm due to this? gulp 4 isn't primary release yet, and yet gulp 3's dependency chain is broken on new node. This issue has been pending for about two years without progress or resolution and frankly that's a bit weird.


db pull adds redundant comment about ignored table

Bug description
In my schema, one of the models has a @@ignore tag. When doing db pull the following comment is added on top of the model definition
/// The underlying table does not contain a valid unique identifier and can therefore currently not be handled by the Prisma Client.

If I do db pull again, another (identical) comment is added on top of the model, again.
In other words, after doing db pull three times, this is the result:
/// The underlying table does not contain a valid unique identifier and can therefore currently not be handled by the Prisma Client.
/// The underlying table does not contain a valid unique identifier and can therefore currently not be handled by the Prisma Client.
/// The underlying table does not contain a valid unique identifier and can therefore currently not be handled by the Prisma Client.
model MyTable {
    time        DateTime @db.Timestamptz(6)
    someColumn  String

    @@ignore
}

How to reproduce

Create a table with "no valid unique identifier". E.g. in postgres create table "SomeTable"(t timestamp);
Run prisma db pull on the database
In the generated schema.prisma file, observe how the comment about "... table does not contain a valid unique identifier ..." appears on top of the model.
Run prisma db pull again
In the schema.prisma, observe how another (redundant) comment was added

Expected behavior
The comment should not appear more than once per "model".
Prisma information
(The issue is regarding the schema, no queries)
Environment & setup

OS: Linux Mint 20.3 Cinnamon
Database: PostgreSQL (with timescale extension)
Node.js version: v16.18.1

Prisma Version
❯ npx prisma -v
prisma                  : 4.7.1
@prisma/client          : 4.7.1
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine 272861e07ab64f234d3ffc4094e32bd61775599c (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli 272861e07ab64f234d3ffc4094e32bd61775599c (at<EMAIL_ADDRESS>Introspection Engine    : introspection-core 272861e07ab64f234d3ffc4094e32bd61775599c (at<EMAIL_ADDRESS>Format Binary           : prisma-fmt 272861e07ab64f234d3ffc4094e32bd61775599c (at<EMAIL_ADDRESS>Format Wasm             : @prisma/prisma-fmt-wasm 4.7.1-1.272861e07ab64f234d3ffc4094e32bd61775599c
Default Engines Hash    : 272861e07ab64f234d3ffc4094e32bd61775599c
Studio                  : 0.477.0

Hmm, I actually can't reproduce the bug outside of my project. But it's definitely something added in 4.7.x. If I downgrade to 4.6.x then the comment is not added multiple times.
Issue could be removed I guess, perhaps someone else will run into it and give a better explanation about how to reproduce...

Anything special about the table/model this is happening for?
Or its position in your file?
Have you looked at it with an editor that highlights unvisible characters like line endings? That might be interesting.

I'm also experiencing this since 4.7.x. Nothing special in the schema or models. Each db pull adds another comment to any ignored models.

Does this also happen when you create a new project and point it towards the same database, then run db pull a few times?

Does this also happen when you create a new project and point it towards the same database, then run db pull a few times?

Just tested. Yes, it does. Prisma 4.8.1.

yup

Still occuring for me, in prisma version 4.10.1

Then I would guess you have a partitioned table, and that this one is currently ignored in your schema (which it does not have to).
Can you help us reproduce this problem again?

This is a new issue, introduced by the new feature. So when we reintrospect, we should not add the comment if the model is already in the PSL.

Then I would guess you have a partitioned table, and that this one is currently ignored in your schema (which it does not have to).
Can you help us reproduce this problem again?

FWIW my database has timescaledb.
The models I am getting the redundant info comment for, are "timescale hypertables".
The tables are created as with any psql table, and then I ran SELECT create_hypertable('sometable', 'time');.

I figured since these issues may not be related, it warrants an issue of its own. See: https://github.com/prisma/prisma/issues/18103


feat(concurrency): implement execute_chunk

This change is

crates/blockifier/src/blockifier/transaction_executor.rs line 57 at r2 (raw file):
Previously, Yoni-Starkware (Yoni) wrote…
Is this necessary? (same re stateful validator)
If so, please move execute_chunk to a separate impl block

It is necessary, If you want another impl block we need to also move execute_txs that calls to execute_chunk, so we would get one block with execute_txs and execute_chunk and another block with the rest, WDYT?

crates/blockifier/src/blockifier/transaction_executor.rs line 200 at r3 (raw file):
Previously, Yoni-Starkware (Yoni) wrote…
Why not?
Sounds like you should use replace if you care about the old value. In our case, it must always be None.

It's not exactly the same thing. replace (see docs) puts the value inside the existing Some wrapper without dropping it. Your suggestion will drop the old Some and create a new wrapper. I'm not sure it is significant, but the reciprocal operation of take is replace, so I think what barak did is the correct way to do it.

crates/blockifier/src/blockifier/transaction_executor.rs line 199 at r3 (raw file):

                )
            })
            .commit_chunk_and_recover_block_state(commit_index);

This is incorrect and confusing. See my suggestion for the correct code.
Note:

The array tx_execution_results should be of length n_committed_txs. The current code makes it of length n_committed_txs - 1.
I think we should use a variable named something like last_committed_tx_index to make it clear what the argument provided to commit_chunk_and_recover_block_state needs to be. Just putting - 1 in the input is unclear.

Suggestion:
        let n_committed_txs = worker_executor_factory.scheduler.get_n_committed_txs();
        let tx_execution_results = worker_executor_factory
            .execution_outputs
            .iter()
            .fold_while(Vec::new(), |mut results, execution_output| {
                if results.len() > n_committed_txs {
                    Done(results)
                } else {
                    let locked_execution_output = execution_output
                        .lock()
                        .expect("Failed to lock execution output.")
                        .take()
                        .expect("Output must be ready.");
                    results.push(
                        locked_execution_output.result.map_err(TransactionExecutorError::from),
                    );
                    Continue(results)
                }
            })
            .into_inner();

        let last_committed_tx_index = n_committed_txs - 1
        let block_state_after_commit = Arc::try_unwrap(worker_executor_factory)
            .unwrap_or_else(|_| {
                panic!(
                    "To consume the block state, you must have only one strong reference to the \
                     worker executor factory. Consider dropping objects that hold a reference to \
                     it."
                )
            })
            .commit_chunk_and_recover_block_state(last_committed_tx_index);

crates/blockifier/src/blockifier/transaction_executor.rs line 262 at r5 (raw file):
Previously, noaov1 (Noa Oved) wrote…
No?

You are correct Noa

crates/blockifier/src/blockifier/transaction_executor.rs line 283 at r5 (raw file):

            .into_inner();

        let last_committed_tx_index = n_committed_txs - 1;

Maybe put it inside commit_chunk_and_recover_block_state (condition and variable)?

Codecov Report
Attention: Patch coverage is 33.33333% with 16 lines in your changes missing coverage. Please review.

Project coverage is 78.17%. Comparing base (12572b9) to head (61ad4ab).

Files
Patch %
Lines

.../blockifier/src/blockifier/transaction_executor.rs
33.33%
16 Missing :warning:

Additional details and impacted files
@@                                Coverage Diff                                @@
##           barak/execute_chunk/5_default_concurrency_config    #1953   +/-   ##
=================================================================================
  Coverage                                             78.17%   78.17%
=================================================================================
  Files                                                    63       63
  Lines                                                  8851     8851
  Branches                                               8851     8851
=================================================================================
  Hits                                                   6919     6919
  Misses                                                 1492     1492
  Partials                                                440      440

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

crates/blockifier/src/blockifier/transaction_executor.rs line 262 at r10 (raw file):

            concurrent_block_context,
            Mutex::new(&mut self.bouncer),
        ));

As I said offline, I don't think "factory" fits here. You have a single WorkerExecutor instance and each thread gets a pointer to it. The word factory implies that the object generates other objects and it does not.
Nonblocking
Suggestion:
        let worker_executor = Arc::new(WorkerExecutor::initialize(
            block_state,
            chunk,
            concurrent_block_context,
            Mutex::new(&mut self.bouncer),
        ));

crates/blockifier/src/blockifier/transaction_executor.rs line 306 at r10 (raw file):

                     it."
                )
            })

Suggestion:
            .except(
                "To consume the block state, you must have only one strong reference to the \
                worker executor factory. Consider dropping objects that hold a reference to \
                it."
            )

crates/blockifier/src/blockifier/transaction_executor.rs line 306 at r10 (raw file):
Previously, barak-b-starkware wrote…
The except method requires deriving Debug for the WorkerExecutor, the ThreadSafeVersionedState, the PapyrusReader, and the StorageReader.
The last one is from the papyrus_storage crate and doesn't derive Debug there.
It also requires + std::fmt::Debug in the second impl block of the TransactionExecutor.
What would you like to do?

Keep it as it was. I thought it is a small change.

crates/blockifier/src/blockifier/transaction_executor.rs line 253 at r10 (raw file):
Previously, noaov1 (Noa Oved) wrote…
Where there any tests that did not do it?
I think that yes. If concurrency mode is not enabled, then we will not want to execute the transactions concurrently, no?

I'm not sure.
The "clone and field changing" that I've added here is not the best solution for what we want, but it's the shorter one.
I thought you might want to merge it today, so I chose the shorter one with a TODO for the "block_context passing" issue.
I'll open a new PR with a nicer solution, and I'll rebase this one on top of it (or the opposite). I'll ping you once it's ready.

crates/blockifier/src/blockifier/transaction_executor.rs line 253 at r10 (raw file):
Previously, barak-b-starkware wrote…
I'm not sure.
The "clone and field changing" that I've added here is not the best solution for what we want, but it's the shorter one.
I thought you might want to merge it today, so I chose the shorter one with a TODO for the "block_context passing" issue.
I'll open a new PR with a nicer solution, and I'll rebase this one on top of it (or the opposite). I'll ping you once it's ready.

New PR to eliminate the comment and the clone.