Daily Wire Not Working TypeError: 'NoneType' object is not iterable

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2<IP_ADDRESS> (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
United States
Description
I'm excited about the new Daily Wire Extractor but I have not been able to get it to work. I tried both without and with cookies and I get a TypeError: 'NoneType' object is not iterable.  Let me know if I'm doing something wrong.  Thank you!
Verbose log
PS C:\Users\jcsch> yt-dlp https://www.dailywire.com/episode/ep-1520-the-return-of-religious-freedom-bonus-hour-2 --verbose
[debug] Command-line config: ['https://www.dailywire.com/episode/ep-1520-the-return-of-religious-freedom-bonus-hour-2', '--verbose']
[debug] Portable config "G:\yt-dlp\yt-dlp.conf": ['-o', 'G:/YoutubeDL/%(title)s.%(ext)s', '--external-downloader', 'aria2c']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2<IP_ADDRESS> [a86e01e] (win32_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2022-03-24-git-28d011516b-essentials_build-www.gyan.dev (setts), ffprobe 2022-03-24-git-28d011516b-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] [DailyWire] Extracting URL: https://www.dailywire.com/episode/ep-1520-the-return-of-religious-freedom-bonus-hour-2
[DailyWire] ep-1520-the-return-of-religious-freedom-bonus-hour-2: Downloading webpage
ERROR: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1427, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1497, in __extract_info
  File "yt_dlp\extractor\common.py", line 647, in extract
  File "yt_dlp\extractor\dailywire.py", line 63, in _real_extract
TypeError: 'NoneType' object is not iterable

PS C:\Users\jcsch> yt-dlp https://www.dailywire.com/episode/ep-1520-the-return-of-religious-freedom-bonus-hour-2 --cookies G:\yt-dlp\dw.txt --verbose
[debug] Command-line config: ['https://www.dailywire.com/episode/ep-1520-the-return-of-religious-freedom-bonus-hour-2', '--cookies', 'G:\\yt-dlp\\dw.txt', '--verbose']
[debug] Portable config "G:\yt-dlp\yt-dlp.conf": ['-o', 'G:/YoutubeDL/%(title)s.%(ext)s', '--external-downloader', 'aria2c']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2<IP_ADDRESS> [a86e01e] (win32_exe)
[debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.22000-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2022-03-24-git-28d011516b-essentials_build-www.gyan.dev (setts), ffprobe 2022-03-24-git-28d011516b-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.14.1, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] [DailyWire] Extracting URL: https://www.dailywire.com/episode/ep-1520-the-return-of-religious-freedom-bonus-hour-2
[DailyWire] ep-1520-the-return-of-religious-freedom-bonus-hour-2: Downloading webpage
ERROR: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "yt_dlp\YoutubeDL.py", line 1427, in wrapper
  File "yt_dlp\YoutubeDL.py", line 1497, in __extract_info
  File "yt_dlp\extractor\common.py", line 647, in extract
  File "yt_dlp\extractor\dailywire.py", line 63, in _real_extract
TypeError: 'NoneType' object is not iterable

Those videos appear to need login. Pass cookies
@HobbyistDev Can we add a better error message when login is needed?

As I mentioned I tried it with and without cookies.  Neither worked and I received the same error both times.
Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: pukkandan @.>
Sent: Thursday, June 23, 2022 6:35:05 PM
To: yt-dlp/yt-dlp @.>
Cc: Josh @.>; Author @.>
Subject: Re: [yt-dlp/yt-dlp] Daily Wire Not Working TypeError: 'NoneType' object is not iterable (Issue #4154)
Those videos appear to need login. Pass cookies
@HobbyistDevhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FHobbyistDev&data=05|01||0d206387902c42ec1b2808da557100b0|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637916241078242099|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=5vk0pjXieOWsx36OJcIpr1IKtwCsYQL6MgDDbkuE7ZQ%3D&reserved=0 Can we add a better error message when login is needed?
—
Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fyt-dlp%2Fyt-dlp%2Fissues%2F4154%23issuecomment-1165012136&data=05|01||0d206387902c42ec1b2808da557100b0|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637916241078242099|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=W6WdXWO14zV2zghkDlHvsui371voKwdIIVYvw0SwgOU%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAF7AVKSI2W3V7WEMOI7IVZLVQTYCTANCNFSM5ZUXRNZQ&data=05|01||0d206387902c42ec1b2808da557100b0|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637916241078242099|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=B%2BztnlaRcUvnVlu2zN88fSgGvXD32SR8K7DrzsLlo6M%3D&reserved=0.
You are receiving this because you authored the thread.Message ID: @.***>

@HobbyistDev Can we add a better error message when login is needed?
For simple fix, i think this is sufficient

        for url in urls or []:

But, this fix didn't actually detect login, just throw error if the media is not found

As I mentioned I tried it with and without cookies.  Neither worked and I received the same error both times.

ah, sorry. I didn't notice cookies in the second log

Can you share your crendential in Discord (HobbyistDev#3257) or in email<EMAIL_ADDRESS>?

Note to self:
It seems whole extractor can be rewrite to use graphql json. Login function add authorization header with Bearer token. The return json looks identical with __NEXT_ID__ in webpage

I have implemented grapql api in DailyWire (not pushed yet) but have some issue for login.
DailyWire use Oauth2 for authorization, here's some steps that i monitored in dev tools:

The login page is in 'https://authorize.dailywire.com/usernamepassword/login' with POST method and have request_data like this

{
            'client_id': '<client_id>',
            'redirect_uri': 'https://www.dailywire.com/callback',
            'tenant': 'dailywire',
            'response_type': 'code',
            'scope': 'openid profile email',
            'state': <state>,
            'nonce': '<nonce>',
            'connection': 'Username-Password-Authentication',
            'username': f'{username}',
            'password': f"{password}",
            'popup_options': {},
            'sso': True,
            'response_mode': 'query',
            '_intstate': 'deprecated',
            '_csrf': <_csrf>, # always change
            'audience': 'https://api.dailywire.com/',
            'code_challenge_method': 'S256',
            'code_challenge':'<code_challenge>',
            'auth0Client': '<auth0Client>',
            "protocol":"oauth2"
        }

The return of this requests is html for successful login and json for failed login. If this request return html they contain something like result and then the url request to something like <redirect_url>?code= with HTTP code 302.

The code from  is requested to https://authorize.dailywire.com/oauth/token with POST method and have request_data like this:

{
            'client_id': <client_id>,
            'code_verifier': <code_verifier>, # always change
            'grant_type': 'authorization_code',
            'code': , # always change
            'redirect_uri': 'https://www.dailywire.com/callback',
            }

and then from this we can get the token from access_token and set Authorization Header to something like this:
Authorization: <token_type> <access_token>
The issues are :

The return of this requests is html for successful login and json for failed login.

How to handle the return like this ? should i set something like self._parse_json(..., fatal=False) ?

then the url request to something like <redirect_url>?code= with HTTP code 302

How to get this url as the  is not found in api call or in html ? can't this redirected automatically ?

Anyway, i'm not too familiar too Oauth2, please correct me if i'm wrong

The return of this requests is html for successful login and json for failed login.

How to handle the return like this ? should i set something like self._parse_json(..., fatal=False) ?

try:
    webpage = _download_webpage(url, video_id)
except ExtractorError as e:
    if not isinstance(e.cause, urllib.error.HTTPError):
        raise
    error = self._parse_json(e.cause.read(), video_id)

then the url request to something like <redirect_url>?code= with HTTP code 302

How to get this url as the  is not found in api call or in html ? can't this redirected automatically ?

I believe we currently have no mechanism to get the intermediate URL of the redirect 🤔. Any ideas @coletdjnz

Thanks for solution (1)

The return of this requests is html for successful login and json for failed login.

How to handle the return like this ? should i set something like self._parse_json(..., fatal=False) ?

try:
    webpage = _download_webpage(url, video_id)
except ExtractorError as e:
    if not isinstance(e.cause, urllib.error.HTTPError):
        raise
    error = self._parse_json(e.cause.read(), video_id)

then the url request to something like <redirect_url>?code= with HTTP code 302

How to get this url as the  is not found in api call or in html ? can't this redirected automatically ?

I believe we currently have no mechanism to get the intermediate URL of the redirect thinking. Any ideas @coletdjnz

I don't quite understand what is going on here.
Is this doing something like:
POST /login
This returns a 302 response with a Location header for the redirect, but this response also contains JSON data we need to extract?
or are we doing POST /login, which returns a 200 OK response with a JSON data. the JSON data contains a "redirect url" which we are to request?
If we really need to it should be possible to disable redirects, but might require #2861

I don't quite understand what is going on here.
Is this doing something like: POST /login This returns a 302 response with a Location header for the redirect, but this response also contains JSON data we need to extract?
or are we doing POST /login, which returns a 200 OK response with a JSON data. the JSON data contains a "redirect url" which we are to request?

We want to get the Location header in the direct response of the POST. We can't use urlh.geturl() since it redirects multiple times

The login page is requesting like this:
Request Header:
POST /usernamepassword/login HTTP/1.1
Host: authorize.dailywire.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:101.0) Gecko/20100101 Firefox/101.0
Accept: */*
Accept-Language: id,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate, br
Referer: https://authorize.dailywire.com/login?state=<state>&client=<client_id>&protocol=oauth2&audience=https%3A%2F%2Fapi.dailywire.com%2F&redirect_uri=https%3A%2F%2Fwww.dailywire.com%2Fcallback&scope=openid%20profile%20email&response_type=code&response_mode=query&nonce=<nonce>&code_challenge=<code_challenge>&code_challenge_method=S256&auth0Client=<auth0Client>
Content-Type: application/json
Auth0-Client: <auth0Client>
Content-Length: 890
Origin: https://authorize.dailywire.com
DNT: 1
Connection: keep-alive
Cookie: _csrf=<_csrf>
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Sec-GPC: 1

Response Header:
HTTP/3 200 OK
date: Mon, 27 Jun 2022 00:39:41 GMT
content-type: text/html; charset=utf-8
cf-ray: 721a20823e0a879d-SIN
cache-control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0, no-transform
etag: W/"7dc-FnmExTq+m3Ik6DzPpJEIW86OawM"
strict-transport-security: max-age=31536000
vary: Accept-Encoding
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
ot-baggage-auth0-request-id: 721a20823e0a879d
ot-tracer-sampled: true
ot-tracer-spanid: 764161b849a62105
ot-tracer-traceid: 14319fd741f13af0
traceparent: 00-764161b849a62105-000000000000000014319fd741f13af0-01
tracestate: auth0-request-id=721a20823e0a879d
x-auth0-requestid: af7851cd0f4a80613ba7
x-content-type-options: nosniff
x-ratelimit-limit: 100
x-ratelimit-remaining: 99
x-ratelimit-reset:<PHONE_NUMBER>
server: cloudflare
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

Response:
<form method="post" name="hiddenform" action="https://authorize.dailywire.com/login/callback">
    <input type="hidden" name="wa" value="wsignin1.0">
    <input type="hidden"
           name="wresult"
           value=<wresult>>
    <input type="hidden" name="wctx" value=<wctx>

            Script is disabled. Click Submit to continue.

<input type="submit" value="Submit">

</form>

The login page is requesting like this:
Request Header:
POST /usernamepassword/login HTTP/1.1
Host: authorize.dailywire.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:101.0) Gecko/20100101 Firefox/101.0
Accept: */*
Accept-Language: id,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate, br
Referer: https://authorize.dailywire.com/login?state=<state>&client=<client_id>&protocol=oauth2&audience=https%3A%2F%2Fapi.dailywire.com%2F&redirect_uri=https%3A%2F%2Fwww.dailywire.com%2Fcallback&scope=openid%20profile%20email&response_type=code&response_mode=query&nonce=<nonce>&code_challenge=<code_challenge>&code_challenge_method=S256&auth0Client=<auth0Client>
Content-Type: application/json
Auth0-Client: <auth0Client>
Content-Length: 890
Origin: https://authorize.dailywire.com
DNT: 1
Connection: keep-alive
Cookie: _csrf=<_csrf>
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Sec-GPC: 1

Response Header:
HTTP/3 200 OK
date: Mon, 27 Jun 2022 00:39:41 GMT
content-type: text/html; charset=utf-8
cf-ray: 721a20823e0a879d-SIN
cache-control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0, no-transform
etag: W/"7dc-FnmExTq+m3Ik6DzPpJEIW86OawM"
strict-transport-security: max-age=31536000
vary: Accept-Encoding
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
ot-baggage-auth0-request-id: 721a20823e0a879d
ot-tracer-sampled: true
ot-tracer-spanid: 764161b849a62105
ot-tracer-traceid: 14319fd741f13af0
traceparent: 00-764161b849a62105-000000000000000014319fd741f13af0-01
tracestate: auth0-request-id=721a20823e0a879d
x-auth0-requestid: af7851cd0f4a80613ba7
x-content-type-options: nosniff
x-ratelimit-limit: 100
x-ratelimit-remaining: 99
x-ratelimit-reset:<PHONE_NUMBER>
server: cloudflare
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400

Response:
<form method="post" name="hiddenform" action="https://authorize.dailywire.com/login/callback">
    <input type="hidden" name="wa" value="wsignin1.0">
    <input type="hidden"
           name="wresult"
           value=<wresult>>
    <input type="hidden" name="wctx" value=<wctx>

            Script is disabled. Click Submit to continue.

<input type="submit" value="Submit">

</form>

that isn't a redirect :confused:
you'd need to handle that callback url yourself in the extractor

The requests flow itself quite similar to this (https://developer.okta.com/blog/2017/06/21/what-the-heck-is-oauth#oauth-tokens)

The return of this requests is html for successful login and json for failed login.

How to handle the return like this ? should i set something like self._parse_json(..., fatal=False) ?

try:
    webpage = _download_webpage(url, video_id)
except ExtractorError as e:
    if not isinstance(e.cause, urllib.error.HTTPError):
        raise
    error = self._parse_json(e.cause.read(), video_id)

then the url request to something like <redirect_url>?code= with HTTP code 302

How to get this url as the  is not found in api call or in html ? can't this redirected automatically ?

I believe we currently have no mechanism to get the intermediate URL of the redirect thinking. Any ideas @coletdjnz

I don't think this has anything to do with getting the intermediate URL between redirects.

then the url request to something like <redirect_url>?code= with HTTP code 302

How to get this url as the  is not found in api call or in html ? can't this redirected automatically ?

Use self._request_webpage which gives you the http response instance (not just the webpage data), which you can do .geturl() on. This will give you the final url in the HTTP redirect chain.

👍 I thought they were asking for https://github.com/yt-dlp/yt-dlp/issues/4154#issuecomment-1166700181

Use self._request_webpage which gives you the http response instance (not just the webpage data), which you can do .geturl() on. This will give you the final url in the HTTP redirect chain.

Thank you for your suggestion

Is there any way to get response cookie from _request_webpage  ?

self._get_cookies(url).get(cookie_name)

Sorry to bother you again. I want to get a key-value of cookie to be set on header, is there anyway to do this ?

I don't understand

i mean the url/<slug> request cookie like this:
Cookie: _dd_s=logs=<dd_logs>&created=<created>; spotim_visitId=<spotim_visitId>; _legacy_auth0.<state>.is.authenticated=<bool>; auth0.<client_id>.is.authenticated=<bool>; accessToken=<access_token>
As we can see, the accessToken is inside the cookie.
However, in the next request after that, accessToken is set in header. It's looks like this:
Authorization: Bearer <access_token>

token = self._get_cookies(url).get('access_token') doesn't work?

token = self._get_cookies(url).get('access_token') doesn't work?

No, it doesn't work, it always return None. I mean the cookies not neccessary from the url as the cookies can 'supplied' from user too. I think _get_cookies supposed to return response cookie not the request cookie, am i right ?

It returns all the cookies currently associated with the url. Are you sure you are using the correct URL? If you can push your code to GH, it would be much easier than sharing snippets back and forth

Sorry for slow response, here's my branch for this fix https://github.com/HobbyistDev/yt-dlp/tree/dailywire-fix

Hi, is this still being worked? I have a login to test.

Sorry for slow response, here's my branch for this fix HobbyistDev/yt-dlp@dailywire-fix. Should i make draft PR to this repo ?

orry, it seems I completely missed this message. What do you get when you do elf._get_cookies('https://www.dailywire.com). It should have the accessToken if it is being set correctly. Can you see it being set in --print-trafic?


ZMQ freezes when receiving data

Send and recv is currently handled by
function sendExpression(omc, expr)
   ZMQ.send(omc.socket, expr)
   message=ZMQ.recv(omc.socket)
   return Parser.parseOM(unsafe_string(message))
end

The issue here is that I have managed to freeze it a couple of times (The reason being is that we have ZMQ communication but we do not have any timeout or polling to make the decision to throw a communication error and terminate).
This permanently blocks OMJulia and you need to quit the application. It is possible to use polling or timeout techniques to prevent this behavior. If someone else thinks that could be a good addition I could implement it :)

@JKRT  adding a timeout is a good option , OMPython also uses timeout mechanism to connect with omc, but not in the sendExpression(), i did not face any such freeze situation in OMJulia,  but we can add a timeout parameter in the sendExpression() method, but hopefully timeout value should not be very short

@arun3688 I noticed it when I tried some things. Basically, I put a lot of print statements for debugging in the Compiler and that broke OMJulia.. See if I can replicate the issue. However, as you said from just reading the code one can imagine the issue even if it is rare..
Do you know anything about usage patterns for OMJulia? I guess certain things can indeed take a very long time hmm.. Some kind of polling option would be the best I guess?

I currently do not have the time to do work on this. It is not really critical but I leave it open should either of us have the time

@JKRT , I will start looking into it once i finished with OMMatlab Linearization API

It is not really a big issue. Just do it if there is nothing else to do. As we discussed we should look into the same thing for OMPython as well, and maybe even OMMatlab since the same issue might occur there

sure

Seems alright. My only comment is that we maybe should do import ZMQ instead of using ZMQ so we can see which parts that belongs to different modules


Query without Limit throws exception when too many results

Hi,
I'm using Datastore with gcloud 0.30.3 and when executing a simple Query to find all entities in a Kind that has 1.000 entities, it fails if the query isn't provided a Limit value:
Without Limit
{"namespace":null,"kinds":["postdatastore"],"filters":[],"orders":[],"groupByVal":[],"selectVal":[],"autoPaginateVal":true,"startVal":null,"endVal":null,"limitVal":-1,"offsetVal":-1}

Output
/Users/jpdiazvaz/Documents/github/loopback-connector-gcloud-metrics/node_modules/gcloud/lib/datastore/request.js:475
      var limit = reqOpts.query.limit.value;
                                     ^
TypeError: Cannot read property 'value' of undefined
    at onResponse (/Users/jpdiazvaz/Documents/github/loopback-connector-gcloud-metrics/node_modules/gcloud/lib/datastore/request.js:475:38)

With Limit
{"namespace":null,"kinds":["postdatastore"],"filters":[],"orders":[],"groupByVal":[],"selectVal":[],"autoPaginateVal":true,"startVal":null,"endVal":null,"limitVal":100000,"offsetVal":-1}

Output
[{"key":{"id":4503908059709440,"kind":"postdatastore","path":["postdatastore",4503908059709440]},"data":{"body":"This is the body of a test POST","title":"This is a test POST"}},...]

@jpdiazvaz this should be fixed in v0.31.0, thanks again for reporting this and please let us know if you experience any more issues!


Automatic Version bump tool for Python

Increment after release following guidelines at: https://github.com/Azure/azure-sdk/blob/master/docs/policies/releases.md#python
This process should fit into the unified pipeline model. We should be able to relate a released artifact back to the project that produced it.
Will need to normalize how versions are specified across projects in the repo

This is almost ready for PR. I'm meeting w/ @lmazuel to ensure I am approaching standardizing how we handle versions correctly.

There are some additional changes required for this from changes we are taking today :)


Outline can't be hidden through the СanvasGroup

How to reproduce:

Create an SDF Quad inside a Сanvas group
Change the alpha of the Canvas group

https://github.com/user-attachments/assets/e7c7b491-f6de-4e27-be8c-3c59a6e60e54
I need to be able to control the color of both fill and stroke together throw the vertex color. The color property in the image affects the colors of the vertexes and is modified by the canvas group.
I suggest the following:

Add a color property for the fill. And stop using the property from the picture for it.
Make the stroke and fill colors multiply by the vertex color.

Thanks for the suggestion. I modified sahder to override alpha with vertex alpha. The latest update fixed this problem.

// i.color <--- vertex color
shadowAlpha *= pow(shadowAlpha, _ShadowPower) * _ShadowColor.a * i.color.a;
outlineAlpha *= _OutlineColor.a * i.color.a;
graphicAlpha *= color.a;

Current Version broken
Uploading bandicam 2024-07-18 17-06-56-018.mp4…

Sorry for the inconvenience of my destructive bug. It's because I replaced the maskable graphics with my own implementation and then unexpectedly changed unity's default material and this caused a problem. Now a fixed version has been released and please update the package.

Outline now is not working as expected.
Now it's a 2nd filled layer not just a line. Because of this it's not possible to use fill with transparency.
Also, the stroke works as outside, which is not convenient when switching from the previous version. I think you should add a property of the line placement (Inside, Center, Outside)
And make the default mode Inside. This will simplify the migration and will work like figma.
Translated with DeepL.com (free version)


dask.dataframe read_csv, to_csv crashes when '#' in path

$ pip freeze | grep -E 'dask|pandas'
dask==2.0.0
pandas==0.24.2
import dask.dataframe as ddf
import pandas as pd
from os import mkdir
from os.path import join
folder='Fdcm #31H~Vell 1_Out'
mkdir(folder)
df = pd.DataFrame({'A': [1, 1], 'B': [3, 3]})
df.to_csv(join(folder,'fil.csv'))
df2=ddf.from_pandas(df,npartitions=1)
df2.to_csv(join(folder,'fil2*.csv'))

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\dataframe\core.py", line 1167, in to_csv
    return to_csv(self, filename, **kwargs)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\dataframe\io\csv.py", line 661, in to_csv
    delayed(values).compute(scheduler=scheduler)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\base.py", line 156, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\base.py", line 398, in compute
    results = schedule(dsk, keys, **kwargs)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\threaded.py", line 76, in get
    pack_exception=pack_exception, **kwargs)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\local.py", line 462, in get_async
    raise_exception(exc, tb)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\compatibility.py", line 112, in reraise
    raise exc
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\local.py", line 230, in execute_task
    result = _execute_task(task, data)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\core.py", line 119, in _execute_task
    return func(*args2)
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\dataframe\io\csv.py", line 508, in _write_csv
    with fil as f:
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\bytes\core.py", line 183, in __enter__
    f = SeekableFile(self.fs.open(self.path, mode=mode))
  File "%USERPROFILE%\Miniconda3\lib\site-packages\dask\bytes\local.py", line 61, in open
    return open(self._normalize_path(path), mode=mode)
FileNotFoundError: [Errno 2] No such file or directory: '%USERPROFILE%\\Projects\\Fdcm \\0.part'

cc @martindurant
In infer_storage_options, we have
(Pdb) ll
329     def infer_options(urlpath):
330         if hasattr(urlpath, "name"):
331             # deal with pathlib.Path objects - must be local
332             urlpath = str(urlpath)
333             ispath = True
334         else:
335             ispath = False
336
337         options = infer_storage_options(urlpath)
338  ->     protocol = options.pop("protocol")
339         urlpath = options.pop("path")
340
341         if ispath and protocol != "file":
342             raise ValueError("Only use pathlib.Path with local files.")
343
344         return urlpath, protocol, options
(Pdb) pp options
{'path': 'Fdcm ',
 'protocol': 'file',
 'url_fragment': '31H~Vell 1_Out/fil2*.csv'}

I'm not sure what the expected behavior with # is here.

urllib's urlsplit apparently does <scheme>://<netloc>/<path>?<query>#<fragment>
(because URLs can include anchors?)
So we could just join the fragment back on? That would be in infer_storage_options. I can do it tomorrow.

Seems reasonable. I don't imagine the current behavior is preferable to that.

Closing as the original failing example is passing on the current master branch of dask (0ecb05ffb0eda4d85a332c5d3c09bd37228ee4b2) and fsspec (https://github.com/intake/filesystem_spec/commit/3e2abdc5ce1617c3faa4946833050f684cf4bb66). @schnekke feel free to re-open if you're still experiencing issues.


[Feature-2925][server] Init TaskLogger in TaskExecuteProcessor (#2925)

What is the purpose of the pull request
Init task logger before init TaskExecuteThread and pass the taskLogger in to TaskExecuteThread init method as parameter, so we can record the user and work dir creating log to task instance log, and user can read this log in front-end WebUI. #3014
Use common ShellExecutor in FileUtils. #2925
Brief change log

Add taskLogger parameter  to TaskExecuteThread.
Add three log utils to LoggerUtils.
Init task logger before create initing TaskExecuteThread, and init task logger process to TaskExecuteProcessor.
Log all detail when create user and working dir in FileUtils.

Verify this pull request
This pull request is code cleanup without any test coverage.

Codecov Report

Merging #2965 into dev will increase coverage by 0.05%.
The diff coverage is 33.33%.

@@             Coverage Diff              @@
##                dev    #2965      +/-   ##
============================================
+ Coverage     39.11%   39.17%   +0.05%
- Complexity     2678     2682       +4
============================================
  Files           435      435
  Lines         20119    20161      +42
  Branches       2454     2459       +5
============================================
+ Hits           7870     7898      +28
- Misses        11514    11525      +11
- Partials        735      738       +3

Impacted Files
Coverage Δ
Complexity Δ

...che/dolphinscheduler/server/log/TaskLogFilter.java
25.00% <0.00%> (-8.34%)
2.00 <1.00> (ø)

.../server/worker/processor/TaskExecuteProcessor.java
8.95% <0.00%> (-1.22%)
1.00 <0.00> (ø)

...eduler/server/worker/runner/TaskExecuteThread.java
0.00% <0.00%> (ø)
0.00 <0.00> (ø)

.../apache/dolphinscheduler/common/utils/OSUtils.java
47.36% <33.33%> (-0.96%)
23.00 <3.00> (ø)

...pache/dolphinscheduler/common/utils/FileUtils.java
30.57% <47.36%> (+1.60%)
9.00 <0.00> (ø)

...che/dolphinscheduler/common/utils/LoggerUtils.java
66.66% <50.00%> (-20.00%)
6.00 <2.00> (+2.00)
:arrow_down:

.../apache/dolphinscheduler/common/utils/IOUtils.java
0.00% <0.00%> (-50.00%)
0.00% <0.00%> (-1.00%)

...he/dolphinscheduler/api/service/LoggerService.java
82.60% <0.00%> (-8.70%)
9.00% <0.00%> (-1.00%)

...e/dolphinscheduler/remote/NettyRemotingClient.java
51.07% <0.00%> (-2.88%)
9.00% <0.00%> (-2.00%)

...inscheduler/common/task/sqoop/SqoopParameters.java
74.00% <0.00%> (ø)
25.00% <0.00%> (ø%)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a7fd72...f36af53. Read the comment docs.

Hi @gabrywu @dailidong , Can you help to review this PR? Thx a lot.

@yangyichao-mango What's purpose of this PR? Why are there so many irrelevant files to change ?

@yangyichao-mango What's purpose of this PR? Why are there so many irrelevant files to change ?

Thx a lot for your review @gabrywu .
This PR is essentially to solve the abnormal exit caused by the error (exceptions caused by permissions, etc.) when the worker starts to create the working directory
But after my change, I found that if the worker failed to create the working directory, the exception it threw would be caught, only recorded in the log of the worker node, not in the log of a single task instance. Although the task will fail at the end of execution, the exception log is very few and it is difficult to locate the exception (permission exception, etc.) directly.
In my opinion, I understand that this error should fail fast, or at least recorded by logs. So in order to match the change of my exitval judgment work, I put the initialization change of tasklogger to  the beginning of TaskExecuteProcessor. And I understand that the task logger should record all the logs related to the task, not only the task runtime logs.
If I have any misunderstanding, please give me suggestions.

非常感谢你的review。
这个pr本质上是为了解决在worker执行task时创建工作目录时的错误（由于权限导致的问题等）导致的异常退出。
但是在改动完成后，我发现如果worker创作工作目录失败其抛出的异常会被catch住，异常信息只会在worker节点的日志中被记录下来，不会在单个task的日志中记录下来，这相当于我一开始的工作完全不会被外显。虽然最后task执行的时候会失败，但是异常日志非常少，很难直接定位到工作目录中的异常（权限异常）。
我理解这种错误应该fail-fast，或者至少有日志来记录，所以为了配合我这次针对exitval判断的改动，我就把taskLogger的初始化改动到了最前面，用来记录涉及到这个task的所有日志。并且我理解taskLogger本身就应该记录所有涉及到这个task的日志，不仅仅是task运行时的日志。
如果我有理解不正确的地方，敬请指出。

@yangyichao-mango What's purpose of this PR? Why are there so many irrelevant files to change ?

Thx a lot for your review @gabrywu .
This PR is essentially to get the abnormal exit value caused by the error (exceptions caused by permissions, etc.) when the worker starts to create the working directory, to judge if the user info and work dir is created successfully.
But after my change, I found that if the worker failed to create the working directory, the exception it threw would be caught, only recorded in the log of the worker node, not in the log of a single task instance. Although the task will fail at the end of execution, the exception log is very few and it is difficult to locate the exception (permission exception, etc.) directly.
In my opinion, I understand that this error should fail fast, or at least recorded by logs. So in order to match the change of my exitval judgment work, I put the initialization change of tasklogger to the beginning of TaskExecuteProcessor. And I understand that the task logger should record all the logs related to the task, not only the task runtime logs.
If I have any misunderstanding, please give me suggestions.
非常感谢你的review。
这个pr本质上是为了获取在worker执行task时创建工作目录时的错误（由于权限导致的问题等）值，来判断工作目录，用户权限是否创建成功。
但是在改动完成后，我发现如果worker创作工作目录失败其抛出的异常会被catch住，异常信息只会在worker节点的日志中被记录下来，不会在单个task的日志中记录下来，这相当于创建工作目录相关的所有错误信息完全不会被直接外显给用户。虽然最后task执行的时候会失败，但是异常日志非常少，很难直接定位到工作目录中的异常（权限异常等）。
我理解这种错误应该fail-fast，或者至少有日志来记录，所以为了配合我这次针对exitval判断的改动，我就把taskLogger的初始化改动到了最前面，用来记录涉及到这个task的所有日志。并且我理解taskLogger的原本目的就是用来记录所有涉及到这个task的日志，不仅仅是task运行时的日志。
如果我有理解不正确的地方，敬请指出。

You'd better give a suitable title to this issue, otherwise others may have a misunderstanding

@yangyichao-mango What's purpose of this PR? Why are there so many irrelevant files to change ?

Thx a lot for your review @gabrywu .
This PR is essentially to get the abnormal exit value caused by the error (exceptions caused by permissions, etc.) when the worker starts to create the working directory, to judge if the user info and work dir is created successfully.
But after my change, I found that if the worker failed to create the working directory, the exception it threw would be caught, only recorded in the log of the worker node, not in the log of a single task instance. Although the task will fail at the end of execution, the exception log is very few and it is difficult to locate the exception (permission exception, etc.) directly.
In my opinion, I understand that this error should fail fast, or at least recorded by logs. So in order to match the change of my exitval judgment work, I put the initialization change of tasklogger to the beginning of TaskExecuteProcessor. And I understand that the task logger should record all the logs related to the task, not only the task runtime logs.
If I have any misunderstanding, please give me suggestions.
非常感谢你的review。
这个pr本质上是为了获取在worker执行task时创建工作目录时的错误（由于权限导致的问题等）值，来判断工作目录，用户权限是否创建成功。
但是在改动完成后，我发现如果worker创作工作目录失败其抛出的异常会被catch住，异常信息只会在worker节点的日志中被记录下来，不会在单个task的日志中记录下来，这相当于创建工作目录相关的所有错误信息完全不会被直接外显给用户。虽然最后task执行的时候会失败，但是异常日志非常少，很难直接定位到工作目录中的异常（权限异常等）。
我理解这种错误应该fail-fast，或者至少有日志来记录，所以为了配合我这次针对exitval判断的改动，我就把taskLogger的初始化改动到了最前面，用来记录涉及到这个task的所有日志。并且我理解taskLogger的原本目的就是用来记录所有涉及到这个task的日志，不仅仅是task运行时的日志。
如果我有理解不正确的地方，敬请指出。

You'd better give a suitable title to this issue, otherwise others may have a misunderstanding. If you want to fix different issue, different PRs should be created

Thx a lot for your review @gabrywu .
I've change the description of this issue and PR and connect the [FEATURE] #3014 and [BUG] #2925.

What is the function of this ThreadLocal? Is it necessary? I think we changed our log format and implemented it with loggerutils

What is the function of this ThreadLocal? Is it necessary? I think we changed our log format and implemented it with loggerutils

Thx a lot for your review @dailidong .
In my opinion, it's better to set the tasklogger threadlocal value to log the task detail but not to pass the tasklogger through the function's input params in the utils class of FileUtils and OSUtils.

What is the function of this ThreadLocal? Is it necessary? I think we changed our log format and implemented it with loggerutils

Thx a lot for your review and work of merging dev branch to current branch @dailidong .
In my opinion, although my branch is out-of-date with the base branch, maybe it's better to rebase the current branch. If it's needed, I will rebase my branch after your approval this PR and reply me.

I think no need to rebase if no confilct files

I think no need to rebase if no confilct files

Thx a lot for your review. @dailidong

Hello, there is still a problem under the modified logic:
If you copy a resource file and there is no exception (the file is in the resource list, but not in HDFS), you will still not see the relevant error message in the task log of the Web UI, making it difficult to troubleshoot the problem
@yangyichao-mango
您好，在现在修改后的逻辑下，还存在一个问题是：
如果在copy资源文件时发生文件不存在异常（文件在资源列表中，但是在hdfs上不存在的特殊情况下），那么在web UI的task日志中还是看不到相关错误信息，不便于排查问题

Hello, there is still a problem under the modified logic:
If you copy a resource file and there is no exception (the file is in the resource list, but not in HDFS), you will still not see the relevant error message in the task log of the Web UI, making it difficult to troubleshoot the problem
@yangyichao-mango
您好，在现在修改后的逻辑下，还存在一个问题是：
如果在copy资源文件时发生文件不存在异常（文件在资源列表中，但是在hdfs上不存在的特殊情况下），那么在web UI的task日志中还是看不到相关错误信息，不便于排查问题


Director API Metrics broken

Describe the bug
The API Metrics endpoint is failing, missing the metrics.
To Reproduce
Steps to reproduce the behavior (example):

Deploy a bosh director on AWS, Azure or Alicloud
configure your monitoring system to observe the Metrics API Endpoint (Prometheus). You can also curl to that enpoint with the correspondig certs with curl --cacert 'some-ca'.cert --cert 'some-cert'.cert --key 'some-key'.cert  https://:9091/api_metrics

Expected behavior
The API Metrics Endpoint should provide metrics to monitor the director behaviour.
Logs
Puma caught this error: undefined method unpack' for nil:NilClass                 value, timestamp = @f.read(16).unpack('dd')                                               ^^^^^^^ (NoMethodError) /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:261:in block (2 levels) in all_values'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:259:in each' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:259:in map'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:259:in block in all_values' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:295:in with_file_lock'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:258:in all_values' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:136:in block in all_values'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:133:in each' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:133:in all_values'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/metric.rb:65:in values' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/formats/text.rb:30:in block in marshal'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/formats/text.rb:26:in each' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/formats/text.rb:26:in marshal'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/exporter.rb:69:in respond_with' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/exporter.rb:30:in call'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:32:in block in call' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:61:in block in trace'
/var/vcap/data/packages/ruby-3.1/8b0d198f2a4c28350a677dc3db638d96b3fbe750/lib/ruby/3.1.0/benchmark.rb:311:in realtime' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:61:in trace'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:32:in call' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/rack-<IP_ADDRESS>/lib/rack/common_logger.rb:38:in call'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/sinatra-2.2.0/lib/sinatra/base.rb:255:in call' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/configuration.rb:252:in call'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/request.rb:77:in block in handle_request' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/thread_pool.rb:340:in with_force_shutdown'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/request.rb:76:in handle_request' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/server.rb:441:in process_client'
/var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/thread_pool.rb:147:in block in spawn_thread' /var/vcap/data/packages/director/4c00823f3ea2802c7eaa9dde6f4197481e9b9ce6/gem_home/ruby/3.1.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context'prometheus2/888c639b-3248-4454-b841-415f11dbc2df:/var/vcap/jobs/prometheus2/certs
Versions (please complete the following information):

Infrastructure: tested on AWS, Azure and Alicloud
BOSH version 272.6.0
BOSH CLI version 6.4.17
Stemcell version ubuntu-bionic/1.90
bosh-deployment: https://github.com/cloudfoundry/bosh-deployment/tree/0abc4c270277343ad28a3844172da43dfb8f5e79

Deployment info:
For deploying the directors, we use the bosh-deployment repo. To get some metrics for the monitoring system, we use the following ops-file:
-o $bosh_deployment/experimental/enable-metrics.yml \

Hello,
Similar problem with 273.0.0 :
bosh/9aec1ed1-52b6-4683-8fdd-f0843cd6c14e:~$ curl -X GET http://localhost:25556/metrics
Puma caught this error: undefined method `unpack' for nil:NilClass

                value, timestamp = @f.read(16).unpack('dd')
                                              ^^^^^^^ (NoMethodError)
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:261:in `block (2 levels) in all_values'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:259:in `each'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:259:in `map'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:259:in `block in all_values'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:295:in `with_file_lock'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:258:in `all_values'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:136:in `block in all_values'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:133:in `each'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/data_stores/direct_file_store.rb:133:in `all_values'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/metric.rb:65:in `values'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/formats/text.rb:30:in `block in marshal'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/formats/text.rb:26:in `each'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/client/formats/text.rb:26:in `marshal'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/exporter.rb:69:in `respond_with'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/exporter.rb:30:in `call'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:32:in `block in call'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:61:in `block in trace'
/var/vcap/data/packages/ruby-3.1/8d4a28b822d63493496db6b6cf885b8b3d881fd1/lib/ruby/3.1.0/benchmark.rb:311:in `realtime'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:61:in `trace'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/prometheus-client-2.1.0/lib/prometheus/middleware/collector.rb:32:in `call'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/rack-<IP_ADDRESS>/lib/rack/common_logger.rb:38:in `call'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/sinatra-2.2.0/lib/sinatra/base.rb:255:in `call'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/configuration.rb:252:in `call'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/request.rb:77:in `block in handle_request'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/request.rb:76:in `handle_request'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/server.rb:441:in `process_client'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/puma-5.6.4/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
/var/vcap/data/packages/director/f9a42818ceba1e4a5425cc5bae46f862ccb8659e/gem_home/ruby/3.1.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'bosh/9aec1ed

Thanks and regards,
Jean-Christophe.

Similar issue was previously reported into https://github.com/cloudfoundry/bosh/issues/2332 with a workaround to discarded the suspected corrupted buffer file.

i tried several deployments but i cannot seem to replicate it on a clean enviorment.
does this always happen after an upgrade maby?

Hello,
Yes. It happens after an upgrade.
I have successfully applied the workaround described there : https://github.com/cloudfoundry/bosh/issues/2332#issuecomment-979452338.
Regards,
Jean-Christophe.

Hi,
thank you very much @JCL38-ORANGE & @gberche-orange! This workaround from issue #2322  worked like a charm.
Regards
Fabian


omni completion source does not work with coc-omni

Describe the bug
I haven't found the time to migrate over the nvim's built in lsp & nvim-compe and so I'm trying to use coc-omni with orgmode.nvim. However it doesn't work. Is the problem with this plugin or on coc-omni's side?
Steps to reproduce

Have orgmode.nvim & coc.nvim w/ coc-omni installed
Try to trigger completion from within an org file

Expected behavior
Omni func is callable & works?
Emacs functionality
No response
Screenshots and recordings

OS / Distro
Manjaro
Neovim version/commit
0.5
Additional context
No response

Does omni without the coc work fine for you? If yes, it's most likely issue with coc.

Omni works fine without coc, so maybe the problem is on coc's end. Still I find it odd that there's an unknown function error.

I pushed a small change. Can you give it a test?

function! org#complete(findstart, base) abort
  let l:cmd = printf("orgmode.omni(%s, '%s')", a:findstart, a:base)
  return luaeval(l:cmd)
endfunction

setlocal omnifunc=org#complete

^ This workaround works perfectly with coc-omni. Vimscript is not my language, but I'd be happy to clean & test things and submit a PR.
Looking into coc-omni the it seems to be an upstream problem with coc.nvim's workspace not picking up on v:lua variables.
I think I'll also open up an issue on the coc.nvim repo to see if this is the case.
// coc-sources/packages/omni/index.js
    doComplete: async function (opt) {
      let {nvim} = workspace
      let func = await nvim.eval('&omnifunc') // here
      if (!func) return null
      let {line, colnr, col} = opt
      let startcol = col
      try {
        startcol = await nvim.call(func, [1, ''])

The above wrapper function would enable orgmode.nvim to work with coc.nvim out of the box, which is important to people like me who use coc.nvim for it's maturity & relative feature richness.
However I understand that wrapping it in a viml function would go against the "write vim plugins in lua and keep it that way" philosophy, and that the above solution is really just a janky patch

I did something similar in last commit https://github.com/kristijanhusak/orgmode.nvim/commit/9b412b3a8356d15423f953f90d56c38b8917f290, isn't it working for you?

Oh I didn't see that, let me try

Yup, works great. Thanks!


Filter with null value in cell can't be reset

Not sure if this is an issue of aurelia-talbe:
I have a filter for a column with some null values.
I'm able to init the filter with null to see all entries but after I typed something in the filter input and removed it again the filter is set to an empty string.
This results in not showing all rows having null in this column.

Yes, I think it's an issue, I faced out the same error.
When you init the filter value to null, au-table doesn't filter the row, so you will see all the entries.
And every time you programmatically reset the filter value to null, you will see all the entries.
But when you bind a filter value to an html input value, things go in a different way, because Aurelia.io framework accepts to change a model value to an empty string '' from null in the case that a user enters a value and then subsequently backs out that value.
This behaviour is explained in the Aurelia.io framework issue #288 and #308, and they suggest to use a NullToEmptyStringValueConverter.
I report here the cause (from #288 and #308):
The primary issue is the input element's value property only stores strings. The element converts null/undefined to empty string.
This behaviour is caused by html elements not supporting null values, the value is altered to empty-string. Aurelia supports this in its binding processing. When changing a model property from "some text" to null, this triggers:
1: Aurelia binding ValueAttributeObserver coerces null newValue to empty-string
2: Value detects element.value not equal newValue, so updates the element with newValue and calls notify
3: Notify calls element value subscribers with coerced empty-string
4: The Binding subscriber compares newValue empty-string with evaluated source expression which is still null ...
5: and so updates source model with newValue empty-string.


[13258] Release notes v0.4.0

Signed-off-by: JLBuenoLopez-eProsima<EMAIL_ADDRESS>

CI is failing due to an issue which is explained and solved in #140. This PR should be rebased after that PR is merged.

CI failures are unrelated


Short solution needed: "How to expire key" (lua-redis)

Please help us write most modern and shortest code solution for this issue:
How to expire key (technology: lua-redis)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

How to set expiration for key


Update document to work with new OHDSI github location

Updated links etc so that they point to OHDSI GitHub location rather than personal GitHub location

Temporary fix by simply removing the workflows .yml file. If someone comes across this in the future and knows how to resolve the issue, please go ahead! The last commit with this file present is 56a5961

(issue reference: #1 )


Update Transaction_Date__c.field-meta.xml

changes Transaction Date field from Date type to Date/Time type, since API response includes time

Hi @rob-alexa, thank you for trying to contribute. For date to datetime update, it requires some code changes as well since the code is particularly converting it to a date value. I remember from my faded memory that we initially had some issue with the data from the paypal and that's why particularly kept/convert it to date. If you still need to try and do it as date then I would recommend you to do it on your salesforce environment. Thanks Again


[MM-48772] Fix Drafts Save to Wrong Channel

Summary

Fixes an issue where drafts would save to the wrong channel when switching teams or channels.
Ticket Link

https://mattermost.atlassian.net/browse/MM-48781
https://mattermost.atlassian.net/browse/MM-48780
https://mattermost.atlassian.net/browse/MM-48772

Release Note

NONE

/e2e-test

Successfully triggered E2E testing!
GitLab pipeline | Test dashboard

E2E tests not automatically triggered, because the PR is not in a mergeable state. Please update the branch with the base branch and resolve outstanding conflicts.

Thanks @mylonsuren
Tested, looks good to merge.
Verified drafts are no longer saving to wrong channel and can be deleted without re-appearing on channel switch.
E2E report looks good, no PR related failures.

/cherry-pick cloud

Cherry pick is scheduled.


OCM-2864 | feat: Added default push request template

Added a new template for merge request's on the ROSA repo.
OCM-2864
example of what it will look like :
Describe your changes
JIRA link

Please include a link to the JIRA
Design document

Please include a link to the design document if there is one
associated with these changes
What's new?

Please describe the changes in this PR
How to test?

Please describe how to test these changes
Checklist

Before asking for review please confirm you self reviewed the changes

[ ] I have performed a self-review of my code

Please confirm you have tested these changes locally, if possible include
verification steps

[ ] I have tested these changes locally

Confirm if these changes have test coverage, if there is no test coverage
please explain why

[ ] Changes have test coverage

Confirm the changes have been deployed to indicate these changes
are safe to be included in a release

[ ] API changes have been deployed

LGTM but could I suggest something like:
* [ ]  has X tests

i.e. unit tests
I think it would be useful but maybe this is what you were envisioning with the How to test? section 🙂

I think a step by step list off bullet points pointing out how to achieve the result of the changes would be helpful especially for new team members, this of course wouldn't be applicable to every change but for major changes that have prerequisites it would be nice, in regards to referencing which tests were ran , that's something i can add

I think a step by step list off bullet points pointing out how to achieve the result of the changes would be helpful

+1

@den-rgb Thanks for adding this 👍
I personally find the self-review checklist item unnecessary and would trust that people are opening PRs when they think they're good for review.
I'm also wondering if we can make the checklist a bit easier to action/process and help to enforce standards, something like:
PR Author Check List

[ ] All commit messages adhere to the project standard (link to CONTRIBUTING.md to show our standards)
[ ] All commits are squashed into a single commit (link to CONTRIBUTING.md to show how to do this)
[ ] All code changes have unit test coverage (missing coverage is explained)
[ ] All changes have been tested locally and the steps documented in the How to Test? section
[ ] All required API changes are deployed

On the point of deployment, is the requirement that API changes are deployed to production before we can merge changes into this repo?
cheers,
Rob

@den-rgb Thanks for adding this 👍
I personally find the self-review checklist item unnecessary and would trust that people are opening PRs when they think they're good for review.
I'm also wondering if we can make the checklist a bit easier to action/process and help to enforce standards, something like:
PR Author Check List
* [ ]  All commit messages adhere to the project standard (link to CONTRIBUTING.md to show our standards)

* [ ]  All commits are squashed into a single commit (link to CONTRIBUTING.md to show how to do this)

* [ ]  All code changes have unit test coverage (missing coverage is explained)

* [ ]  All changes have been tested locally and the steps documented in the `How to Test?` section

* [ ]  All required API changes are deployed

On the point of deployment, is the requirement that API changes are deployed to production before we can merge changes into this repo?
cheers,
Rob

I'll implement that suggestion it seems much better then the above, and yes

@den-rgb What do you think about making the section on the commit message part of the CONTRIBUTING.md a header so that it can be linked to and is easier to find when reading the Contributing guide?

@den-rgb What do you think about making the section on the commit message part of the CONTRIBUTING.md a header so that it can be linked to and is easier to find when reading the Contributing guide?

The commit message part is the first thing you see in the contribution guide anyway unless I misunderstand what you're referring to?


Resubscription on the proposal_open_contract send does not work

If the connection is closed before the proposal_open_contract is sent live api is unable to resubscribe.
The reason is streamId is required to save the contracts: https://github.com/binary-com/binary-live-api/blob/master/src/ApiState.js#L40
@borisyankov I think it's worth fixing if we can make things a little bit better in the code, otherwise this is a very rare condition and is not of high priority.

This was not true. I made more experiments and the open contract proposal is made again on resubscribe function.


[GOBBLIN-2173] Avoid Adhoc flow spec addition for non leasable entity

Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[ ] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"

https://issues.apache.org/jira/browse/GOBBLIN-XXX

Description

[ ] Here are some details about my PR, including screenshots (if applicable):
https://issues.apache.org/jira/browse/GOBBLIN-2173

Before launching the flows we store adhoc flows temporarily in our flowspec DB. In case of concurrent execution of the flows within the epsilon time defined the first execution is launched successfully, but for the second execution flow spec entry is created but launching is not successful since lease cannot be acquired, for this scenario we should prevent creation of flowspec entry in the spec store, this PR contains code change to address this.
Tests

[ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:

[ ] Tests added to ensure onAddSpec in Orchestrator is successful only if lease can be acquired on entity(call to function leasableEntryExists), if it does not then throw LeaseNotAvailable Exception

[ ] Tests in MysqlMultiActiveLeaseArbiter to ensure existsLeasableEntry returns false, if tried to call this function within epsilon time of acquiring the lease else return true.

[ ] existing tests pass

Commits

[ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 41.32%. Comparing base (edaf474) to head (7d2b49f).
Report is 2 commits behind head on master.

:exclamation:  There is a different number of reports uploaded between BASE (edaf474) and HEAD (7d2b49f). Click for more details.
HEAD has 1 upload less than BASE

Flag
BASE (edaf474)
HEAD (7d2b49f)

2
1

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #4076       +/-   ##
=============================================
- Coverage     51.47%   41.32%   -10.15%
+ Complexity     7546     2237     -5309
=============================================
  Files          1386      486      -900
  Lines         52105    20614    -31491
  Branches       5727     2385     -3342
=============================================
- Hits          26821     8519    -18302
+ Misses        22990    11195    -11795
+ Partials       2294      900     -1394

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles


Custom training loss with custom gradients

I am trying to write a custom loss in Tensorflow v2, for simplicity let's say that I'm using Mean Squared Error loss as follows,
loss_object = tf.keras.losses.MeanSquaredError()

def loss(model, x, y, training):
  # training=training is needed only if there are layers with different
  # behavior during training versus inference (e.g. Dropout).
  y_ = model(x, training=training)
  return loss_object(y_true=y, y_pred=y_)

Now I know that Tensorflow does automatic differentiation.
But I want to specify my custom gradient, in the BackPropagation algorithm,
if we use MSE, we have to do the following

Is it possible in Keras to replace   with  where p is a tensor that is passed during training before applying gradients.

@gadagashwini can you take a look at this.

@rohanmohapatra Could you please provide with an implementable stand alone code so we could help you resolve the issue.

@Saduf2019 Hi, what I'm interested is how is the gradient of the loss w.r.t to the pred_y calculated? I want to replace the that gradient with my custom gradient value.

@rohanmohapatra, Tensorflow provides two ways  to add custom loss with Keras. Please refer this doc. Thanks!

@gadagashwini Can I also change the value of the backprop gradient for the loss? I do not want to explicitly use the AutoGrad.

I have the same issue - is there a way to define a custom gradient when using keras API?

I would like to improve it, Do anyone please help to fix it

@rohanmohapatra,
Sorry for the delayed response. Can you please look into the Documentation of Custom Gradients and let us know if it helps? Thanks!


How to use AriaViewer with Python?

Hi!
I am trying to visualize sequences with AriaViewer in Python but I get that there is no main.py file.
Am I doing something wrong with my python setup? Thank you!

Hello @maria-t ,
I assume you are trying to use this documentation to run our python visualizers: https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/visualization/visualization_python
We are using short names that refers to main in python files packaged with projectaria_tools, you can find the mapping here https://github.com/facebookresearch/projectaria_tools/blob/main/setup.py#L201
If you want to use the C++ binaries (referred here https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/visualization/visualization_cpp#run-aria-viewer), we are not packaging them but you can build them following the instruction shared here https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/installation/installation_cpp

Hi again!
No, I am using the documentation here, since I am trying to visualize sequences from the Pilot dataset.
In the meantime, I managed to run the visualization tool in C++, but then I have the problem that the eye-tracking red cross, trajectory and speechtotext paths are not working.

@maria-t the dataset has been updated and shared here
https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_everyday_activities_dataset
You can also use the new visualizer developed for it https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_everyday_activities_dataset/aea_visualizers#python-visualizer

Hello @maria-t, please do find the official new re-release of the dataset here https://www.projectaria.com/datasets/aea/

Great, thank you!


Add a gRPC call to request 2^16 subtree roots

In order to use shardtree efficiently in a mobile client, lightwalletd needs to be able to supply it with information about roots of 216 subtrees within the commitment tree.
The usage pattern for this data is likely to be:

Light client starts up.
Light client requests latest height

lightwalletd returns it.

Light client requests all 216 roots since height X (<= where they are synced up to).

lightwalletd returns a stream of:

Index of the subtree
Root of the subtree
Height of block that completed this subtree

The light client can use this position : height map to perform a reverse lookup from nullifiers -> block heights -> 216 subtree ranges, in order to learn what to query.

When a newly-created note is detected (either by decrypting a received tx or nullifier lookups to find txs to trial decrypt), the light client queries for all note commitments within the 216 subtree that the new note falls within. It does this by making a getblocks request filtered to note commitments, for a range of blocks that spans the 216 subtree (or whatever fraction thereof it needs, modulo privacy), which it can determine from the height:position map.

lightwalletd responds with a stream of:

Block height
Block hash
Starting block position
List of:

List of:

Note commitment

(Note: we don't include any transaction metadata like txindex, we just use its implicit tx_index within the block).

The latter information can be produced by lightwalletd filtering down the CompactBlocks it already obtains. So all we need is a new gRPC method that takes a starting height, and returns a stream of objects containing the per-subtree data we need.
https://github.com/zcash/zcash/issues/6336 is the issue for exposing the necessary information in a zcashd RPC method, which lightwalletd can then re-export.

I could start working on this toward the end of next week if that would be helpful.


[request] make the sanitize function public

Hi there,
I have a potential use case for this library to be optionally incorporate into Og and two possible ways of doing it.

The ideal approach would be where sanitize/1 functions are made public and docless perhaps in your library and then I can invoke them where I please in Og.

The less desirable approach is to apply the inspect function from SecureLogFormatter directly - because i was hoping to sanitize and then apply an option of different formatters.

This might not suit you though & I'll totally understand if it's :-1: .. I can probably work with option 2. if needs be.
Let me know what you think.

Howdy @stephenmoloney and thanks for your interest!  I'd be open to doing this, let me take some time today to look over the possible changes and get back to you 😀

@doomspork
This is how it will look no my side...
https://github.com/stephenmoloney/og/tree/sanitize#configuration-options
I just add an option to sanitize and it will apply all the goodness from your lib. 😄

Hot diggity!  If you're happy with this change I'll go ahead and merge this and push to hex.

closed by #3
Many thanks... I'll be integrating this into Og

@doomspork
This is how it will look no my side...
https://github.com/stephenmoloney/og/tree/sanitize#configuration-options
I just add an option to sanitize and it will apply all the goodness from your lib. 😄

Many thanks... I'll integrate this into Og

@doomspork
That looks perfect... wow, what a fast response time.. hard to beat.
💯


Cannot create new translation language

I've registered to the weblate instance for helping translate Lagrange into Turkish, but it seems I don't have the necessary permissions to add another language. Is there anything I need to do from my side?
Thanks in advance!

I've now added Turkish (tr): https://weblate.skyjake.fi/projects/lagrange/ui/tr/
Only admins (i.e., me) are able to add new languages. I've given you Translator access rights so you can submit strings. Thanks for translating!
The language selection menu items have recently been removed from Weblate and are now hardcoded (#391). Please let me know what should the menu item be for Turkish (i.e., the Turkish word for the language itself).

Thanks for adding!
For the hardcoded string, it will be Turkish -> Türkçe.

I'd like to translate Lagrange to Dutch ("Nederlands - nl"). I've created an account on Weblate with the same username.

@jobukkit I've updated your account's access rights. Here you go: https://weblate.skyjake.fi/projects/lagrange/ui/nl/

@skyjake Could you please add Swedish as well?


chore: update package dependencies for primary detail

Updates package.json to include devDependencies:

pfe-sass: Used in compiling the CSS assets
pfe-styles: Used on the demo page
pfe-cta: Used on the demo page

This update is urgent because without pfe-sass as a dependency, lerna is not handling the timing of compilation correctly and it's running without the compiled assets available and throwing Sass errors.
Ready-for-merge Checklist

[x] Expected files: all files in this pull request are related to one request or issue (no stragglers or scope-creep).
[ ] Repository compiles and tests pass.

Merging
Please squash when merging and ensure your commit message uses conventional commit formatting.
Be sure to share your updates with the<EMAIL_ADDRESS>mailing list!

Selenium is down - joy of joys.  Unrelated to this PR.


"SocketTimeoutException: Read timed out" during creating/rebuilding index on large database

OrientDB Version: VERSION 3.0.0RC2
Java Version: 1.8.0_161
OS: Ubuntu 16.04.4 LTS (Xenial Xerus)
Expected behavior
Creating index...
Created index successfully in 0.702000 sec(s).
Index created successfully
Actual behavior
Creating index...
Error: com.orientechnologies.common.io.OIOException: Read timed out
Error: java.net.SocketTimeoutException: Read timed out
(After a while index is created anyway)
Steps to reproduce

Create graph database
Load 10 milion records
Create index on two fields in one class (eg. CREATE INDEX Person.givenName_familyName ON Person(givenName, familyName) NOTUNIQUE)
Get the error:
Error: com.orientechnologies.common.io.OIOException: Read timed out
Error: java.net.SocketTimeoutException: Read timed out
After a while index is created anyway but not prompted, behaviour looks like an error

Settings
java -server -Xms30G -Xmx30G -Djna.nosys=true
-XX:+HeapDumpOnOutOfMemoryError
-Djava.awt.headless=true -Dfile.encoding=UTF8
-Drhino.opt.level=9 -Denvironment.dumpCfgAtStartup=true
-Djava.util.logging.manager=com.orientechnologies.common.log.ShutdownLogManager
-Djava.util.logging.config.file=/opt/orientdb/config/orientdb-server-log.properties
-Dorientdb.config.file=/opt/orientdb/config/orientdb-server-config.xml
-Dorientdb.www.path=/opt/orientdb/www -Dorientdb.build.number=develop@rf833ecfdbc293875a429e9831d0ff6a467db076d

Hi @jedrzej999
How are you creating the index? Are you doing it from Java API? Connecting from plocal or remote?
Thanks
Luigi

Hi,
I am creating index via SQL statement in through console.sh
CREATE INDEX Person.givenName_familyName ON Person(givenName, familyName)
NOTUNIQUE
2018-04-12 11:53 GMT+02:00 luigidellaquila<EMAIL_ADDRESS>
Hi @jedrzej999 https://github.com/jedrzej999
How are you creating the index? Are you doing it from Java API? Connecting
from plocal or remote?
Thanks
Luigi
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/orientechnologies/orientdb/issues/8208#issuecomment-380745598,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AI-514SMSMygYXN-1wL0QlLJFOSBuwk0ks5tnyQFgaJpZM4TRc7e
.

Ok, thank you, I'll try to reproduce
Thanks
Luigi

Exception in thread "main" com.orientechnologies.common.io.OIOException: Read timed out
at com.orientechnologies.orient.client.remote.OStorageRemote.baseNetworkOperation(OStorageRemote.java:327)
at com.orientechnologies.orient.client.remote.OStorageRemote.networkOperationRetryTimeout(OStorageRemote.java:214)
at com.orientechnologies.orient.client.remote.OStorageRemote.networkOperationNoRetry(OStorageRemote.java:239)
at com.orientechnologies.orient.client.remote.OStorageRemote.execute(OStorageRemote.java:960)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentRemote.execute(ODatabaseDocumentRemote.java:425)
at com.bt.srims.shared.framework.graph.execution.engine.GraphExecutionEngineImpl.executeGremlin(GraphExecutionEngineImpl.java:51)
at com.bt.srims.shared.framework.graph.execution.engine.test.GremlinExecutionEngineTest.main(GremlinExecutionEngineTest.java:32)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at java.io.DataInputStream.readInt(Unknown Source)
at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.readBytes(OChannelBinary.java:159)
at com.orientechnologies.orient.core.serialization.serializer.result.binary.OResultSerializerNetwork.fromStream(OResultSerializerNetwork.java:655)
at com.orientechnologies.orient.client.remote.message.OMessageHelper.readProjection(OMessageHelper.java:503)
at com.orientechnologies.orient.client.remote.message.OMessageHelper.readResult(OMessageHelper.java:471)
at com.orientechnologies.orient.client.remote.message.OQueryResponse.read(OQueryResponse.java:73)
at com.orientechnologies.orient.client.remote.OStorageRemote.lambda$networkOperationRetryTimeout$2(OStorageRemote.java:227)
at com.orientechnologies.orient.client.remote.OStorageRemote.baseNetworkOperation(OStorageRemote.java:285)
... 6 more
This thing i got when i am querying from db  query is big enough at the end of i am adding "map{it.get().values()}"
then i got it, If i am not using map{} it is working fine can u help me out with this.
Thanks.


Remove imake

Imake is an obsolete build system, which was removed from X11 as of X11R7.0 (in December 2005). The only package in homebrew that requires imake is homebrew-science/transfig. When transfig is updated to version 3.2.6a (https://github.com/Homebrew/homebrew-science/pull/4871), which doesn't use imake anymore, we should remove imake from the repo.

transfig has been updated, thus imake should removed or moved to the boneyard.

Let's wait and see what the analytics usage numbers are like. It's possible that some people still need this for legacy build systems even if we don't need it for other formulae.

This was migrated to homebrew/core because it's still used (for now).


consolidate buttons on review screen

Updating review screen

In this PR, I have:

[x] Manually tested to ensure my PR is working
[ ] Ensured issues that this PR closes will be automatically closed
[x] Requested review from Mia or Quinten

I didn't add any conditional logic inside ${ }, I added backslashes at the end of the text to fix the problem.

Ok! That helps then. I still dont think this is exactly what @nonprofittechy was asking.

From @tobyfey

I saw in Pfeiffer a review block that seems to be like Quinten is talking about. I don't know if that will work with the factors that are asked in different spots (which we were using edit above before).

Toby-- what do you mean by "the factors that are asked in different spots"?

[ ] Get rid of horizontal lines
[ ] Use plain language headings for defense language
[ ] An icon makes sense to use, but explain what it means (maybe choose a different heading), fa icon feature?
[ ] Use consistent headings (Defense:, Defense:, etc), step down in size
[ ] Consider adding collapsable review/edit screens

Maybe a clearer information hierarchy that uses heading level 2, 3, and 4 with a little longer text like this:

I tested this on mobile. While we wait on accordion-type edit screens, what are your thoughts on putting an additional "Continue" type button at the top of the review page screen (in addition to having one at the bottom)?

See #376

Updated and ready for a new review, Mia and Toby testing on mobile, Toby testing again for 4.
@nonprofittechy can you check this out:

There is something buggy going on with the footer. I was in a Chrome browser on Mac and captured it in two screenshots, but note I scrolled to the side to grab it. It is only on the review screen / screen with accordion that this happens.

I don't see the footer issue on Android.

I can't figure out how to recreate the problem with the footer.

I tried Chrome and Firefox and the responsive design mode on each.


feature:  Agent 配置重载功能优化 (closed #1769)

PR
CheckList（PR Assigners）
确认已完成以下操作：

[ ] 已将分支与当前合入分支同步
[ ] 已在关联 Issue 中补充「功能自测」部分内容
[ ] 已更新关联 Issue 的状态 backlog -> todo -> for test -> tested（需通过测试同学测试） -> done
[ ] 将 PR 添加上 pr/reviewable

CheckList（PR Reviewers）
确认已完成以下操作：

[ ] 检查关联 Issue 「功能自测」部分内容
[ ] 必要时打回 pr/reviewable，要求重新修改
[ ] 及时 Approve

Codecov Report

Merging #1861 (842cade) into v2.4.2-dev (1c02d93) will decrease coverage by 0.02%.
The diff coverage is 92.30%.

@@              Coverage Diff               @@
##           v2.4.2-dev    #1861      +/-   ##
==============================================
- Coverage       74.38%   74.37%   -0.02%
==============================================
  Files             398      398
  Lines           27134    27144      +10
==============================================
+ Hits            20184    20188       +4
- Misses           6950     6956       +6

Files
Coverage Δ

...ollections/agent_new/render_and_push_gse_config.py
100.00% <100.00%> (ø)

apps/backend/constants.py
98.00% <100.00%> (+0.12%)
:arrow_up:

...ackend/subscription/steps/agent_adapter/adapter.py
91.78% <80.00%> (-0.87%)
:arrow_down:

... and 3 files with indirect coverage changes


Module build failed: visitors[i].run is not a function

the main.less file is empty, and i'm getting:
ERROR in ./styles/main.less
Module build failed: visitors[i].run is not a function
 @ /Users/bnaya/dev/opensource/hasadna/open-budget-frontend/open-budget/app/styles/main.less (line null, column -1)
 near lines:

 @ ./scripts/main.coffee 3:2-29

less 1.7.5 installed
my webpack config:
loaders:
...
          {
            test: /\.less$/,
            loader: 'less-loader'
          }

js code:
      require("styles/main.less");

Had old less version for some reason


Unexpected tags Value in Path's Operations

Given the following Swagger v1.2 Doc:

api-docs (parent)
/v1/analysis

$cat api-docs
{
    "apiVersion": "1.0.0",
    "apis": [
        {
            "description": "...",
            "path": "/v1/analysis"
        }
    ],
    "swaggerVersion": "1.2"
}

$cat /v1/analysis
{
    "apiVersion": "1.0.0",
    "apis": [
        {
            "operations": [
                {
                    "items": {
                        "$ref": "UnhandledColumnsInfo"
                    },
                    "method": "GET",
                    "nickname": "unhandledColumnsCount",
                    "notes": "",
                    "parameters": [],
                    "produces": [
                        "application/json"
                    ],
                    "summary": "Unhandled Columns",
                    "type": "array"
                }
            ],
            "path": "/v1/analysis/unhandledColumnsCount"
        },
        {
            "operations": [
                {
                    "method": "GET",
                    "nickname": "getPublishedSampleInfo",
                    "notes": "",
                    "parameters": [],
                    "produces": [
                        "application/json"
                    ],
                    "summary": "Get published foobar sample information",
                    "type": "FeatureCollection"
                }
            ],
            "path": "/v1/analysis/published"
        }
    ],
    "basePath": "/spatialfoobar/rest",
    "models": {
        "Feature": {
            "id": "Feature",
            "properties": {
                "geometry": {
                    "$ref": "JsObject"
                },
                "properties": {
                    "$ref": "JsObject"
                },
                "type": {
                    "type": "string"
                }
            }
        },
        "FeatureCollection": {
            "id": "FeatureCollection",
            "properties": {
                "features": {
                    "items": {
                        "$ref": "Feature"
                    },
                    "type": "array"
                },
                "type": {
                    "type": "string"
                }
            }
        },
        "JsObject": {
            "id": "JsObject",
            "properties": {
                "fields": {
                    "$ref": "Map[string,JsValue]"
                }
            }
        },
        "UnhandledColumnsInfo": {
            "id": "UnhandledColumnsInfo",
            "properties": {
                "count": {
                    "format": "int32",
                    "type": "integer"
                },
                "unhandledColumnName": {
                    "type": "string"
                },
                "userId": {
                    "type": "string"
                }
            }
        }
    },
    "produces": [
        "application/json"
    ],
    "resourcePath": "/v1/analysis",
    "swaggerVersion": "1.2"
}

Yet, running new io.swagger.parser.SwaggerParser().read("...api-docs") resulted in the following v2 doc:
{
  "swagger" : "2.0",
  "info" : {
    "version" : "1.0.0"
  },
  "basePath" : "/spatialfoobar/rest",
  "schemes" : [ "http" ],
  "paths" : {
    "/v1/analysis/published" : {
      "get" : {
        "tags" : [ "v1analysis" ],
        "summary" : "Get published foobar sample information",
        "description" : "",
        "operationId" : "getPublishedSampleInfo",
        "produces" : [ "application/json" ],
        "parameters" : [ ],
        "responses" : {
          "default" : {
            "description" : "success",
            "schema" : {
              "$ref" : "#/definitions/FeatureCollection"
            }
          }
        }
      }
    },
    "/v1/analysis/unhandledColumnsCount" : {
      "get" : {
        "tags" : [ "v1analysis" ],
        "summary" : "Unhandled Columns",
        "description" : "",
        "operationId" : "unhandledColumnsCount",
        "produces" : [ "application/json" ],
        "parameters" : [ ],
        "responses" : {
          "default" : {
            "description" : "success",
            "schema" : {
              "type" : "array",
              "items" : {
                "$ref" : "#/definitions/UnhandledColumnsInfo"
              }
            }
          }
        }
      }
    }
  },
  "definitions" : {
    "JsObject" : {
      "properties" : {
        "fields" : {
          "$ref" : "#/definitions/Map[string,JsValue]"
        }
      }
    },
    "UnhandledColumnsInfo" : {
      "properties" : {
        "count" : {
          "type" : "integer",
          "format" : "int32"
        },
        "unhandledColumnName" : {
          "type" : "string"
        },
        "userId" : {
          "type" : "string"
        }
      }
    },
    "FeatureCollection" : {
      "properties" : {
        "features" : {
          "type" : "array",
          "items" : {
            "$ref" : "#/definitions/Feature"
          }
        },
        "type" : {
          "type" : "string"
        }
      }
    },
    "Feature" : {
      "properties" : {
        "geometry" : {
          "$ref" : "#/definitions/JsObject"
        },
        "properties" : {
          "$ref" : "#/definitions/JsObject"
        },
        "type" : {
          "type" : "string"
        }
      }
    }
  }
}

Why does each the tags of each path's get operation have a value of "tags" : [ "v1analysis" ]?
I believe that this code is responsible.

Hi, in 1.2 there was no notion of tags--it was really just the path from api-docs (i.e. /v1/analysis).  The 2.0 spec has a grouping mechanism of tags which are computed from the 1.2 path value.

Hey fehguy. Is the tags added (per a v1.2 doc) due to spec or an implementation decision?
It seems to me that, in the below snippet taken from the above example, what's the value in providing the tags of v1analysis?
  "paths" : {
    "/v1/analysis/published" : {
      "get" : {
        "tags" : [ "v1analysis" ],

Since grouping of operations is now done with tags, we had two choices for 1.2 spec conversions, since tags did not exist in that version.

Lump them all under the default tag group
Programmatically generate a tag for them based on the path

So in your case, the tag is a little ugly, since your resource listing had a multi-part api declaration path (/v1/analysis).  Most apis have a more simple path to the api declaration, so something like analysis would appear just fine.
Obviously the best thing is if your code is upgraded to swagger-spec 2.0.  You can, of course, override the tag generation logic in the codegen as well.


Create run-tests-draft.md - no-merge (WIP)

Does this mean that there will be 2 ways to build source build:

Build sources only. This step will not download prebuilts
Build sources and/or run tests. This step may download prebuilts and run tests?

Does this mean that there will be 2 ways to build source build:

Build sources only. This step will not download prebuilts
Build sources and/or run tests. This step may download prebuilts. All tests from all projects will be run (including end-to-end cli tests).

Building and running test will work with toolsets built from sources, no prebuilts.

@NikolaMilosavljevic want to merge this or should we close it?

@NikolaMilosavljevic want to merge this or should we close it?

This can be closed - it was meant to be used for design discussion. It's also outdated, I believe.
Test documentation can be added to main document. @adaggarwal, would you like to add instructions for running tests to main source-build documentation?

I think Aditya already merged that one, it was https://github.com/dotnet/source-build/pull/1263.  Thanks!

Test documentation can be added to main document. @adaggarwal Aditya Aggarwal FTE, would you like to add instructions for running tests to main source-build documentation?

@NikolaMilosavljevic Yep it was added. Feel free to suggest changes if you'd like see some.


Unable to disable quote after using it for a section of text.

Informations
When using FireFox. If you use quote, the editor keeps using it and it is not possible to turn if off. Tested with Chromium and the error is not present.
Browser version: FireFox 58.0.1
OS: Linux KDE Neon
Screen resolution: 1920*1080
How to reproduce the bug?
I confirmed the issue using the demo at https://alex-d.github.io/Trumbowyg/.
If you look at the html code, it keeps moving the  to be at the end of all content.
I have tested various ways to write and press the buttons, the result is the same. Write text, select it, press quote. Text is made into a quote block. Write some more text, or hit return a couple of times, and then write text and select it and click paragraph or H1 and the text remains in quote block.
I have tested without using any FireFox addons and the result was the same.
I have tested on my own site by changing the trumbowyg options, but has not made a difference.

Hey!
I know this bug, but Idk how to fix it :/

setup up full debug on jquery,the it's easy to track the bug after what i see i can say one thing there must be a loop bug or last element append bug try look in to this direction if had free time i could debug and find it but now bussy on projects.

Duplicates #532
@Faks you can continue on the other issue :)


Replace unrooted with relative path

I have never seen the word unrooted used to refer to relative paths. I
believe that line is referring to relative path so let's use the proper
term.

Agree, thanks!

Agree, thanks!


Correctly set Int16 or Int64 DataFormat

Inside TraitsToDataFormat, the last if (isInteger) condition will
overwrite other integer datatypes (like Int16 and Int64) to Int32.
This patch fixes the issue.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: wheatdog sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


[data mgt] MISCONF Redis is configured to save RDB snapshots

What were you trying to achieve?
Automate the discard of cache data after one day since storage in memory
What are the expected results?
Discard the cache data from memory after the expiry period is reached and then continue to use Redis
What are the received results?
After the time specified in the eviction policy for Redis, we are no longer able to use Redis and the following error comes up:
MISCONF Redis is configured to save RDB snapshots
More details: https://stackoverflow.com/questions/19581059/misconf-redis-is-configured-to-save-rdb-snapshots
What are the steps to reproduce the issue?
Just wait for the Redis expiry/eviction period (one day) to reach - then Redis will stop to function on all endpoints
In what environment did you encounter the issue?
Node v12.18.1
Additional context
N/A

Was addressed by @danielogen but we are still looking for that custom Redis config file......


Problems trying to install it on a .venv environment.

When running pip install textpack it returns an error. This might likely be to the name change of Scikit-learn install pip.

Any news on this issues?


How do you implement your dropout method in unsupervised setting? And do you apply any dropout in the supervised setting?

I didn't find how you apply dropout in the unsupervised setting on the glance at the code. Would you mind directly navigating me to the implementation code on this?
Besides, did you apply dropout when doing supervised contrastive learning using the NLI dataset?
Thanks a lot.

Hi,
Our model simply uses standard dropout from the transformer model, which is implemented in the transformers package. We also use dropout in the NLI dataset, as dropout is part of the standard training procedure.


Always download

hi ，nice work
When i use this package,  BertModel.from_pretrained('bert-base-uncased'), it's always download when program restart, Can I just download it once？

@carry-xz may be you could replace bert-base-uncased with the download path ~/.pytorch_pretrained_bert/
You could refer https://github.com/Meelfy/pytorch_pretrained_BERT#loading-google-ais-pre-trained-weigths-and-pytorch-dump.

@carry-xz may be you could replace bert-base-uncased with the download path ~/.pytorch_pretrained_bert/
You could refer https://github.com/Meelfy/pytorch_pretrained_BERT#loading-google-ais-pre-trained-weigths-and-pytorch-dump.


WideResNet-28-10

Hi there, this repo is very useful, but could you add WideResNet-28-10? It is highlighted in the official repo and is the best-performing model on CIFAR-10 in RobustBench, so I believe many people will find it helpful.

Will look into it, thanks for bring it to my attention! Would also be open to PRs if you are able.

I did some research and opened an issue on Flax. I would love to contribute, but the problem is I don't really understand how WideResNet-28-10 is defined.


rewrite readme

not sure if you like this, but should make the readme slightly more useful and nicer to read - feel free to change / reject as you wish

Surly better than now.


Infinite sums in PDFs

http://www.math.wm.edu/~leemis/chart/UDR/UDR.html
Some of the distributionds from the above link have PDFs that include infinite sums. How could these be implemented in Hakaru?
Examples are: Power series, Zeta, Doubly noncentral F, Noncentral beta, Noncentral F, and, von Mises

summate can do infinite sums.  You can use the PDF as-is, certainly for noncentral beta, but I think for most of them.

@JacquesCarette
I get the error bellow when I execute this code (summate i from 1 to ∞: (1/i)**3) . Is this the right way to do it?
hakaru: Can not evaluate infinity for natural numbers CallStack (from HasCallStack):   error, called at haskell/Language/Hakaru/Sample.hs:339:27 in hakaru-0.6.0-AbnzUW5EnqjBuQmLn3LJbU:Language.Hakaru.Sample

I do believe that that is the right way to do it. But you might not be able to ever Sample it.  But that's ok. Being able to represent them is already quite useful.

The main question is, does it work in Simplify?

That is enough!