Cannot add Ytube Music to Music Assistant

What version of Music Assistant has the issue?
2.3.2
What version of the Home Assistant Integration have you got installed?
2024.11.1
Have you tried everything in the Troubleshooting FAQ and reviewed the Open and Closed Issues and Discussions to resolve this yourself?

[X] Yes

The problem
When authenticating Youtube music and saving the token generated, the following error was produced
ERROR: [youtube+yt_access_token] dQw4w9WgXcQ: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>)
How to reproduce

Authenticate ytube music
Save

Music Providers
Ytube Music
Player Providers
Google Cast
Full log output
music-assistant.log
Additional information
No response
What version of Home Assistant Core are your running
2024.10.4
What type of installation are you running?
Home Assistant OS
On what type of hardware are you running?
Raspberry Pi

yeah mine has stopped working after yesterdays update - I have not tried to trouble shoot or reauthenticate or anything, just acknowledging that it stopped.

People are finding a solution is to create another brand account. https://sproutsocial.com/glossary/youtube-brand-account/

People are finding a solution is to create another brand account. https://sproutsocial.com/glossary/youtube-brand-account/

ah that does seem to work, although now my playlists need to be moved over, bit annoying LOL, I understand YT Music is best efforts, so just happy to have it working again! thanks.

Our best guess is YouTube is doing something deliberate to deny access.

Our best guess is YouTube is doing something deliberate to deny access.

all the more reason to work towards not being dependent on them, and google in general - that's my home lab driver at the moment..

I'm having the same issue, e.g. ERROR: [youtube+yt_access_token]...
I can't connect my Youtube (personal) account with Music Assistance any more, that was previously working.

@Dion-HA read this issue for the workaround….

@Dion-HA read this issue for the workaround….

@OzGav  read what for a workaround?
I looked at the Youtube brand accounts, we should not need to create a new Youtube brand account, just to link Youtube Music with Music Assistant, it was working previously.
I tried linking Spotify and that worked without any issues, but I don't use Spotify, I ony use Youtube Music :)

@Dion-HA Do you understand this is Google's doing? Please contact them with your frsutrations.

Sigh... looks like I have to go back to ytubemediaplayer

@cherryocola So the workaround is not working for you?

@cherryocola So the workaround is not working for you?

Apologies, I havent tried it I am sure it works, I just didnt want another google account. thanks.

@cherryocola It isn't another Google account. It is a "brand" within your Google account.

@cherryocola Not at this time. You are the first person to say they tried creating a brand account and it didn't work...

@cherryocola Not at this time. You are the first person to say they tried creating a brand account and it didn't work...
edit: Keep an eye on things as sometimes Google reverts these changes and everything just starts working again...

maybe because I always had a channel and I only added a branding? I probably need another account.

Mine just stopped working with the same error

yt-dlp is having the same issue: https://github.com/yt-dlp/yt-dlp/issues/11503

I just tried the workaround but I get the same error. The workaround seems not to work anymore. I get the same HTTP400 error with the new "brand account"

Any other solutions other than brand account?

From the yt-dlp thread it seems like the real fix is to store a cookie to use for Auth. Unfortunately this will require rewriting the music-assistant YouTube auth code and a much more complicated user flow involving extracting cookies from a browser

I already had a brand account.  But I created another one to try the solution above.  And I got the same error.

It appears that OAUTH support has been removed. An alternate method of logging in is being investigated. This will take some time.

My Youtube Music is still working with Music Assistant since I set in up yesterday using the Youtube Brand account work around with my existing personal Google account, I'm reluctant to remove this connection and set it up again as it's still working for me now :)

I doubt it adds any value to resolving this issue but I'm curious if anyone else is experiencing numerous Home Assistant failed login attempts from Google datacenters since the YouTube Music issue started in Music Assistant. Here is an example:
Login attempt failed
Login attempt or request with invalid authentication from <IP_ADDRESS>.bc.googleusercontent.com (<IP_ADDRESS>). See the log for details.
What surprises me is that this has continued even after I removed YouTube Music as a provider in Music Assistant. Oh well.

You can contact google about this and give them your thoughts here https://support.google.com/youtube/gethelp

Sigh... looks like I have to go back to ytubemediaplayer

that doesn't work for me either :(

Same issue here. Adding a Brand account won't work.
ERROR: [youtube+yt_access_token] dQw4w9WgXcQ: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>)

So, everything was working perfectly fine for me until last night when I updated the HA Integration (not the add-on). Suddenly, both MA AND my YT Music app on my phone started throwing a ton of errors. On the YTM app, it would just keep skipping through tracks as unavailable until hitting the end of the queue. During this period, I saw various errors on MA about unavailable/invalid tokens, API availability, etc.
Hours later, YTM app slowly regained the ability to start playing tracks again, albeit with some random track skips, until it finally regained full function this morning. YTM now can play music again, but even after fully removing and trying to re-add YTM to MA, I get the following errors:
WARNING: [youtube+yt_access_token] Incomplete data received in embedded initial data; re-fetching using API.
WARNING: [youtube+yt_access_token] YouTube said: ERROR - Request contains an invalid argument.
WARNING: [youtube+yt_access_token] HTTP Error 400: Bad Request. Retrying (1/3)...
WARNING: [youtube+yt_access_token] YouTube said: ERROR - Request contains an invalid argument.
WARNING: [youtube+yt_access_token] HTTP Error 400: Bad Request. Retrying (2/3)...
WARNING: [youtube+yt_access_token] YouTube said: ERROR - Request contains an invalid argument.
WARNING: [youtube+yt_access_token] HTTP Error 400: Bad Request. Retrying (3/3)...
WARNING: [youtube+yt_access_token] YouTube said: ERROR - Request contains an invalid argument.
ERROR: [youtube+yt_access_token] dQw4w9WgXcQ: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>)
2024-11-14 11:58:16.256 ERROR (MainThread) [music_assistant.webserver] Error handling message: config/providers/save: ERROR: [youtube+yt_access_token] dQw4w9WgXcQ: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>)

To be clear, I already have multiple brand accounts on my main account and none of them work. So, I have two theories at this time:

YTM implemented some new rate limiting and MA's syncing is now hitting this limit (which is why a new brand account worked for some; because a sync on a new clean brand would not hit the rate limit on initial sync)
YTM has decided to break or change some bit that our current YTM sync engine relies on; perhaps accidentally, perhaps intentionally (MA has massively gained in popularity in recent months and Google may have finally noticed the uptick in our traffic)

Let me know if there is any testing I can help perform from my end.

I'm not sure if it helps, or if this wouldn't work with MA, but the HA-Google-Home uses Google Authentication Tokens for access to Google accounts, and that add-on is still working for me.
There are tons of forum posts about folks having issues with using the tokens, so they can be a bit fiddly, but once I got it working for the HA-Google-Home add-on, it's been solid for months now.

Same issue at me :(

I'm not sure if it helps, or if this wouldn't work with MA, but the HA-Google-Home uses Google Authentication Tokens for access to Google accounts, and that add-on is still working for me.
There are tons of forum posts about folks having issues with using the tokens, so they can be a bit fiddly, but once I got it working for the HA-Google-Home add-on, it's been solid for months now.

I don't think it's the tokens themselves in this case. I think the change is in how the API is parsing requests against YTM. Perhaps a slow server-side rollout across all YTM accounts, which would explain why it has seemingly affected us in waves over the last week or so. Mine worked flawlessly until last night while some have been having the issue for a week or more.

Brand account alternative solution failed to resolve the issue here.

Brand account failed for me too

Brand account failed for me as well

Just for clarity for anyone that created a brand account as a solution, are you supposed to somehow authenticate with that account? I just created the brand account but authenticated with my normal account.

I created a new brand account on the old account, even created a completely new google account. But it seems nothing has changed. Still ERROR 400 when Add Youtube Music

I'm not sure if it helps, or if this wouldn't work with MA, but the HA-Google-Home uses Google Authentication Tokens for access to Google accounts, and that add-on is still working for me.
There are tons of forum posts about folks having issues with using the tokens, so they can be a bit fiddly, but once I got it working for the HA-Google-Home add-on, it's been solid for months now.

I don't think it's the tokens themselves in this case. I think the change is in how the API is parsing requests against YTM. Perhaps a slow server-side rollout across all YTM accounts, which would explain why it has seemingly affected us in waves over the last week or so. Mine worked flawlessly until last night while some have been having the issue for a week or more.

I'm now having the same issue using the YT Brand account to authenticate with MA, e.g.
ERROR: [youtube+yt_access_token] dQw4w9WgXcQ: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>)

I'm now having the same issue using the YT Brand account to authenticate with MA, e.g. ERROR: [youtube+yt_access_token] dQw4w9WgXcQ: Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>)

It's almost certainly an API change and this means we'd have to wait for the maintainer of the Python library MA is depending on (yt-dlp) to find and correct accordingly. This will take time. I'm lowkey sad that every time I get used to using YTM again, it breaks again.

Youtube dropped support for OAuth as described in yt-dlp comment here. Music assistant needs to rework the login to intercept user cookies instead, which is painful, but is the only way forward.
Yt-dlp already works with cookies.

Progress has been made. Expect more info in the next couple of days,

The documentation page provides instructions for heading down the "cookie path", yet the UI does not provide the option to paste the cookie info. I can only take the "token path" using the UI.
Is there a manual way to enter the cookie data?

Thanks for the UI update!
I can play music now, though uploaded music is not working.

Try 2.4.0b5 which has the cookie solution

So I did. Everything seems to be working now.
Perhaps the sync was partial when I first tried.

Try 2.4.0b5 which has the cookie solution

Cookie solution is working for me. It also seems Google is locking down on multi-device playback because YTM stopped on my phone with a "you're using this on another device" error when MA began syncing YTM for the first time. I've previously been able to play different YTM streams (directy, not via MA) on 6+ devices at the same time in my house (all one external IP) without issue, so this is definitely a new change. I wonder if upgrading to a YT Family Plan loosens this restriction back to the old limits.
I was able to immediately resume my music on the phone while MA is syncing (my initial library sync takes over 10 minutes due to amount of content). But I can already see that YTMs new restrictions are targeted at folks like us. I doubt this will be the last change coming to make our lives difficult. With all the pricing changes lately, they may be preparing to launch a new higher Premium tier for YT/YTM.

Using the cookie method, the interface sends a popup that claims I do not have YouTube Music Premium. This is technically correct, since I have YouTube premium, which includes all the benefits of YouTube Music Premium, which evidently somewhere deep within their system makes some sort of difference. I have no way through their spaghetti to make any kind of distinction between the two products. Is this something that can even be handled on the Music Assistant side?

Cookie login method works perfectly fine for me with branded account.
Thanks to everybody involved in the Music Assistant project for your hard work and dedication. Much appreciated!
Respectfully,
JR

Try 2.4.0b5 which has the cookie solution

Tested and working for me so far. Thank you!

Cookie solution is working for me. It also seems Google is locking down on multi-device playback because YTM stopped on my phone with a "you're using this on another device" error when MA began syncing YTM for the first time. I've previously been able to play different YTM streams (directy, not via MA) on 6+ devices at the same time in my house (all one external IP) without issue, so this is definitely a new change. I wonder if upgrading to a YT Family Plan loosens this restriction back to the old limits.

I've tested, and am able to playback different streams on multiple devices right now. I'm only testing 2 concurrent, but no issues so far. I do use a separate account I made for HA stuff that is connected to my family plan though.

Using the cookie method, the interface sends a popup that claims I do not have YouTube Music Premium. This is technically correct, since I have YouTube premium, which includes all the benefits of YouTube Music Premium, which evidently somewhere deep within their system makes some sort of difference. I have no way through their spaghetti to make any kind of distinction between the two products. Is this something that can even be handled on the Music Assistant side?

I have YTP vs YTM as well. It's a family plan, not sure if that matters for this. Maybe try using a private window to generate a cookie, and see if that one give better success?

Beware!
The Cookie method worked for me, but not with a cookie copied out of Firefox. Needed to be chromium like Brave.

Using the cookie method, the interface sends a popup that claims I do not have YouTube Music Premium. This is technically correct, since I have YouTube premium, which includes all the benefits of YouTube Music Premium, which evidently somewhere deep within their system makes some sort of difference. I have no way through their spaghetti to make any kind of distinction between the two products. Is this something that can even be handled on the Music Assistant side?

Did you pull the cookie from youtube.com or music.youtube.com?

I had to use a private window to log in directly to music.youtube.com and pull that new cookie, worked great. You guys are wizards.

Try 2.4.0b5 which has the cookie solution

Tested and working for me so far. Thank you!

Cookie solution is working for me. It also seems Google is locking down on multi-device playback because YTM stopped on my phone with a "you're using this on another device" error when MA began syncing YTM for the first time. I've previously been able to play different YTM streams (directy, not via MA) on 6+ devices at the same time in my house (all one external IP) without issue, so this is definitely a new change. I wonder if upgrading to a YT Family Plan loosens this restriction back to the old limits.

I've tested, and am able to playback different streams on multiple devices right now. I'm only testing 2 concurrent, but no issues so far. I do use a separate account I made for HA stuff that is connected to my family plan though.

Using the cookie method, the interface sends a popup that claims I do not have YouTube Music Premium. This is technically correct, since I have YouTube premium, which includes all the benefits of YouTube Music Premium, which evidently somewhere deep within their system makes some sort of difference. I have no way through their spaghetti to make any kind of distinction between the two products. Is this something that can even be handled on the Music Assistant side?

I have YTP vs YTM as well. It's a family plan, not sure if that matters for this. Maybe try using a private window to generate a cookie, and see if that one give better success?

A family plan would absolutely have to grant more simultaneous streams than an individual plan. I'm on an individual plan. It appears that while I can play streams simultaneously on the app on my phone and via MA, the MY YTM sync task seems to be what causes a simultaneous account error on any other YouTube app that is playing music at the time the sync is started. At least, that's been my observations thus far. Or perhaps it's only during the initial account setup and sync in MA that the interruption occurs. I'm not entirely certain; I'll have to do more testing to see if the issue reoccurs outside of initial setup.

The Cookie method worked for me, but not with a cookie copied out of Firefox. Needed to be chromium like Brave.
I was able to use a cookie from FF without issue. It's been working for ~1H now, but I'll report back if it stops.

Whenever a try to enter a cookie in the Login-Cookie field my browser opens a popup with the message:
"__Secure-3PAPISID"
Any idea how to get the correct one / fix this error?

Whenever a try to enter a cookie in the Login-Cookie field my browser opens a popup with the message: "__Secure-3PAPISID"
Any idea how to get the correct one / fix this error?

Make sure you are copying the full Request Headers > Cookie value. It starts with VISITOR_INFO1_LIVE= and is very long. If you miss any of it, or include any additional text beyond the actual value, you will get random errors as MA tries to parse the input for what it needs to extract from it.
The best way to ensure you get the whole value properly is to maximize your browser, open the Dev tools (F12), then drag the divider bar (between page and Dev tools) to make the Dev tools panel wider so the Cookie value is all showing on the screen at once. Don't try to select while scrolling, as this increases the likelihood of errors.

If it helps, Firefox has a "Copy Value" option when you right-click on the cookie key value:

This broke for me too -- confirming that cookies worked for some browsers. For me chrome did not work on Windows 10 or Windows 11 but IE did. Huge props to the team supporting this for coming up with a solution! <3

The cookie-based identification works, and the cookie can be easily extracted through the Dev Tools (I did it using Edge). However, when I restart the MA Addon, I have to reconfigure the YouTube Music provider among the Music Providers and extract a new cookie from the browser again to input it. This is a bit of a hassle, but at least it works...

Actually.... I spoke too soon.  Music Assistant accepts my cookie text.. but it's not showing me my playlists or favorited music.  Like I don't see anything from my profile.
I have a personal and business profile on YouTube.  My content is all on the business profile.  I've tried putting the exact name in "username" as well as adding an "@" to the beginning of it.  Nada.
I also noticed that I could type anything under "username" and it accepts it along with the cookie text.  It doesn't work, but it doesn't log an error.
What am I doing wrong?

Running into issues with the cookie. On edge I get "user does not have a youtube music premium account" error, chrome I get "ERROR: [youtube] dQw4w9WgXcQ: Requested format is not available. Use --list-formats for a list of available formats", and on Brave it will accept the cookie, but all content from youtube music is greyed out. I am logged in on each browser and obtaining the cookie from the playlist page

@grainsoflight look here https://github.com/orgs/music-assistant/discussions/1551#discussioncomment-11323331 and in the docs for more info that may help.

@grainsoflight look here https://github.com/orgs/music-assistant/discussions/1551#discussioncomment-11323331 and in the docs for more info that may help.

I am using neither a brand account nor a family account.

@grainsoflight make sure you are running the latest beta of the server

@OzGav Updated to latest beta, did not resolve the problem

@grainsoflight where did you get that log excerpt you show above? It doesn’t look like something from music assistant?

@OzGav from the logs section of the add on backend

@MarvinSchenkel any idea about the 400s above?

Refreshing individual items makes the album art come back and the content selectable, but it still wont play.
Edit: nothing from my Plex or Local media appears to be playing either, so Im not sure what the issue is. In addition to nothing playing on any speaker, youtube is still logging:
WARNING: [youtube] YouTube said: ERROR - Request contains an invalid argument. WARNING: [youtube] HTTP Error 400: Bad Request. Retrying (1/3)... 2024-11-27 12:01:02.871 INFO (MainThread) [music_assistant.players] Player registered: 455bae44-cb9a-17be-93e8-f790a8ae57ae/Living Room speaker WARNING: [youtube] YouTube said: ERROR - Request contains an invalid argument. WARNING: [youtube] HTTP Error 400: Bad Request. Retrying (2/3)... WARNING: [youtube] YouTube said: ERROR - Request contains an invalid argument. WARNING: [youtube] HTTP Error 400: Bad Request. Retrying (3/3)... WARNING: [youtube] YouTube said: ERROR - Request contains an invalid argument. WARNING: [youtube] Unable to download API page: HTTP Error 400: Bad Request (caused by <HTTPError 400: Bad Request>)

Just double checking. Did you completely remove the YTM integration and logged in again using the cookie method using the latest beta?

@MarvinSchenkel I am not sure what that topic is referring to as far as exporting a cookie after closing the youtube tab. I cant find a way to export a cookie from the blank tab.

First thing that Marvin asked is have you completely removed the YouTube provider since the change to the cookie method?

@OzGav yes, i have

Reinstalling the Cast integration in HA fixed my speaker issue, and after that Youtube Music started playing nicely again, both correctly authorizing and (at least at this point) not crashing.

We recommend you do not use the cast integration. If MA has a native player you should be using that.

@OzGav I'm not sure what you mean by that

@OzGav its back to not playing anything from any source at all

You said

Reinstalling the Cast integration in HA

but that should not be installed. You should have added the cast provider in Music Assistant and be using the players that are discovered by that.

@OzGav Ive completely removed all cast devices from HA, uninstalled and removed data from MA, restarted, reinstalled MA, added the chromecast source provider, and it still doesnt play media from any source. It establishes a connection to the speaker, but will not play.

@OzGav it is occasionally playing now, but regardless of whether it plays or not, HA logs an out of memory error. Is there a specific amount of memory MA needs?

https://music-assistant.io/installation/?h=ram#server-notes

@OzGav increasing RAM to 8GB from 2 seems to have fixed all the issues I was having, will monitor

https://music-assistant.io/installation/?h=ram#server-notes

@OzGav increasing RAM to 8GB from 2 seems to have fixed all the issues I was having, will monitor

To be clear, that 2GB requirement on RAM is just for MA, itself. If you are running MA as an Add-on within HA, you'll need at least 4GB for a small HA install and much more if you are running a large HA deployment with lots of other add-ons, etc.
Each Add-on is a docker container, so the RAM requirements of each should be tallied up to help determine an appropriate amount for your entire HA server/VM. Of course, all add-ons aren't going to use the max RAM all the time, but any one of them could demand it...and OOM panic if it doesn't get it.
Further still, some add-ons could require a lot more than the recommended RAM under some circumstances. A good example of this is the ESPHome Device Compiler. It could gobble up as much as 8GB RAM if you are compiling devices like the Atom Echo, which compiles tensorflow components, while only needing sub-2GB RAM for compiling most other ESPHome devices.
My HA VM has MA, ESPHome, Whisper, OpenWakeWord and I had to allocate 32GB RAM to my VM to ensure everything had enough headroom during peak usage spikes (like transcribing the audio of HA Assist requests in multiple rooms at once or compiling ESPHome firmware). Sure, I'm a bit of a superuser, but as one continues to build out the perfect HA smart home, resource requirements are going to climb. I started my HA VM with just 4GB RAM back in 2022. As I write this, my HA VM is using 3GB RAM at idle.

The cookie method is working and additional information has been added to the docs.

getting the same error again actually

getting the same error again actually

Same here

cannot add YT music
getting this error
User does not have Youtube Music Premium

cannot add YT music
getting this error
User does not have Youtube Music Premium

Free accounts are not supported anymore. You need to have a YouTube music premium subscription.

I DO have a premium account but I am getting this message for some reason

cannot add YT music
getting this error
User does not have Youtube Music Premium

Free accounts are not supported anymore. You need to have a YouTube music premium subscription.

Premium as well, not working.

I see that @MarvinSchenkel has already committed a fix to master; but meanwhile a workaround I have been using for a few days is to simply remove the cookiefile param from ytdl by commenting out this line in my installation.

I see that @MarvinSchenkel has already committed a fix to master; but meanwhile a workaround I have been using for a few days is to simply remove the cookiefile param from ytdl by commenting out this line in my installation.

Hi Jules, where may I find this file?

I see that @MarvinSchenkel has already committed a fix to master; but meanwhile a workaround I have been using for a few days is to simply remove the cookiefile param from ytdl by commenting out this line in my installation.

Interesting.. Two questions from my side about this

Did you apply this on the current main branch (eg on top of my fix) or on the current stable version? The latter still has the quick ios client that can quickly resolve steam  urls
I expect you won't be able to retrieve high quality streams without the cookie (ie logging in). Didn't you run into the error "User does not have YT Music premium" error after removing the cookie line?

I think Google changed it so you cant play your personal playlists


Expandable detail-pane tables in UI

UX part of #322
The UI will know what rows to expand based on a new optional row key: expandable
Sample table:
{
    "title": "Connections",
    "numeric": false,
    "rows": [
        {
            "key": "TCP connections",
            "value_major": "2"
        },
        {
            "key": "Client",
            "value_major": "Server",
            "expandable": true
        },
        {
            "key": "<IP_ADDRESS>:3445",
            "value_major": "<IP_ADDRESS>:80",
            "expandable": true
        },
        {
            "key": "<IP_ADDRESS>:5647",
            "value_major": "<IP_ADDRESS>:560",
            "expandable": true
        }
    ]
}

Can we make the details pane moveable instead of fixing it to the default right hand side?

@miztiik any specific reason other than aesthetics?

I found the details pane sometimes blocking the view of the underlying container, I intuitively was looking to move it to see the underlying container connections information and also the details pane.

@miztiik you can drag the container view. But thanks for pointing out your intuitive use. The blocking of the view is certainly a shortcoming.


How to use variables in migration sql scripts

I have a migration that creates a table.  I would like to grant permissions.  The trouble is the user may differ between environments, so I need to pass a variable to the script to specify the user:
grant select on my_new_table to :reader;
grant select, insert, update on my_new_table to :writer;

How can I pass variables to SQL migration scripts?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/36790191-how-to-use-variables-in-migration-sql-scripts?utm_campaign=plugin&utm_content=tracker%2F73887&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F73887&utm_medium=issues&utm_source=github).

Hey first of all, permissions is something not really supported by db-migrate and I'm not sure if permissions really are something that should be set via migrations, if you have use case that does make sense just let me know, may be it could be  a feature for the future and I'm open to opinions and ideas :). But  you can naturally just use runSql.
 One option would be thus to pass this variable in via your environment.

Ok, I'll dig in and look.
But, just to address the philosophical point of whether permissions belong in migrations...
Maybe I'm using Postgres incorrectly, but when I last looked, there's nothing like permissions inheritance.  For example, the following script only applies to tables that exist at the time it is run:
grant select on all tables in schema public to app_reader;

So, when I add a table later, the app_reader user doesn't automatically have permission on that table.  A grant statement is required.  Maybe there's a way to get postgres to allow permissions to be inherited or automatically created when a new table is created without the explicit grant, but I'm not aware of any.
With that in mind, I don't see how the permissions step wouldn't be part of db-migrate.  It's a mandatory part of creating tables (unless your app is connecting as the user that created the table, which is a bad idea).

With that in mind, I don't see how the permissions step wouldn't be part of db-migrate. It's a mandatory part of creating tables (unless your app is connecting as the user that created the table, which is a bad idea).

Ok with that statement given, it is not part of db-migrate in this case, b/c this is pretty much directed to postgresql,  I can't mind similar problems on other DBs. So this would be a driver related topic more than for db-migrate itself. I'm not quite sure though if postgres is configureable to make your use case work. https://www.postgresql.org/docs/9.0/static/sql-grant.html
But I think that ON DATABASE and ON SCHEMA should apply also on newly created tables in the database or schema.

But I think that ON DATABASE and ON SCHEMA should apply also on newly created tables in the database or schema.

I know, right?  It totally should!  Sadly, SELECT permissions cannot be granted directly to databases or schemas.  Attempting to do so results in error.  The closest is ALL TABLES IN SCHEMA, which only includes the currently existing set of tables, and not tables created in the future.
I only added db-migrate to my project after my database had been built.  So, I have separate scripts outside of db-migrate for creating the database, users, and granting permissions.  It seems like it would be cleaner if I had used db-migrate from the beginning, including using it to create the database.  I've thought about going back and converting those scripts to work with db-migrate for consistency.  But, if I did so, how would I manage permissions according to db-migrate best practices?  A second script for managing permissions, to be run externally from db-migrate?  It begs the question, why not just put everything in that second script?

As already said, this is not a topic for db-migrate itself anymore, but for the driver in this case: https://github.com/db-migrate/pg
I'm not an active user of postgres tbh, so I really would need to reproduce what you say, but it sounds like the permission system of pg is a bit broken, I guess I need to give that a more intensive look though. However, your use case is very much about pg, thus I wont consider adding permission stuff globally as part of db-migrate yet, but possibly for the pg driver.


Faulty tag suggestions

On making a typo with a tag, any additional text that was sent along with it is also sent in the "Did you mean..." embed.

The fuzzy matching also seems a bit odd here, with pste being matched to site instead of paste.

As to why it happens, https://github.com/python-discord/bot/blob/master/bot/exts/backend/error_handler.py#L208 shows that we simply use .replace on user input, we probably could simply just put the suggested command name instead of parsing whole user input and replacing it with suggestion. Would you like to work on it?

I'm busy currently; anyone is free to work on this.

I'm open to working on this. But to make sure I'm not misunderstanding this, you want to only use the first word and not the whole statement, correct?

The fuzzy matching also seems a bit odd here, with pste being matched to site instead of paste.
Yeah, I guess difflib.get_close_matches finds multiple matches but we only chose the first one. With testing I found out that if the cutoff is 0.8, then it only matches paste and not site.

It also seems that the error handler doesn't try to make suggestions for tags, and !paste is a tag.


Serializing EntityType properties causing Getter error

The following code:
public IActionResult GetTableInfo(string table)
{
    // GetEntity just calls this.Model.FindEntityType(table)
    var entity = ProductionContext.GetEntity(table);
    var result = new
    {
        Entity = entity,
        Properties = entity.GetProperties(),
        Navigations = entity.GetNavigations(),
        Keys = entity.GetKeys(),
        Indexes = entity.GetIndexes(),
        ForeignKeys = entity.GetForeignKeys(),
        Annotations = entity.GetAnnotations()
    };

    return new ObjectResult(result);
}

Causes an error when ObjectResult attempts to serialize the result. The error is:
Newtonsoft.Json.JsonSerializationException: Error getting value from 'Getter' on 'Microsoft.Data.Entity.Metadata.Internal.Property'. ---> System.NullReferenceException: Object reference not set to an instance of an object.
This error can be removed by removing all properties except for Annotations and Entity.
My current work around is to do the following:
public IActionResult GetTableInfo(string table)
{
    // GetEntity just calls this.Model.FindEntityType(table)
    var entity = ProductionContext.GetEntity(table);
    var result = new
    {
        Entity = entity,
        Properties = entity.GetProperties().Select(q => new { Name = q.Name, Type = TypeHelper.GetBaseTypeName(q.ClrType), Nullable = TypeHelper.IsNullableType(q.ClrType) }),
        Navigations = entity.GetNavigations().Select(q => q.Name),
        Keys = entity.GetKeys().Select(q => q.SqlServer().Name),
        Indexes = entity.GetIndexes().Select(q => q.SqlServer().Name),
        ForeignKeys = entity.GetForeignKeys().Select(q => q.SqlServer().Name),
        Annotations = entity.GetAnnotations().Select(q => q.Name)
    };

    return new ObjectResult(result);
}

I am not 100% sure if this an error or not, but it would seem like something that people might want to be able to do.
This was using ASP.Net 5, MVC 6 and Entity Framework 7 all at RC1 and an MSSQL 12.0 database.

These types are not designed to be serializable, projecting out to a purpose built serializable object is the right thing to do.


Link to "Frontier at a glance" on the docs is broken

The Frontier at a glance link here
http://frontera.readthedocs.org/en/latest/topics/scrapy-integration.html#spider-logic
is broken.

Fixed!


AttributeError: 'Process' object has no attribute 'cpu_affinity'

Has anyone else had this error with psutil?
Traceback (most recent call last):
  File "examples/atari_dqn_async_cpu.py", line 70, in <module>
    run_ID=args.run_ID,
  File "examples/atari_dqn_async_cpu.py", line 59, in build_and_train
    runner.train()
  File "/Users/billtubbs/rlpyt/rlpyt/runners/async_rl.py", line 40, in train
    throttle_itr, delta_throttle_itr = self.startup()
  File "/Users/billtubbs/rlpyt/rlpyt/runners/async_rl.py", line 108, in startup
    throttle_itr, delta_throttle_itr = self.optim_startup()
  File "/Users/billtubbs/rlpyt/rlpyt/runners/async_rl.py", line 115, in optim_startup
    p.cpu_affinity(main_affinity["cpus"])
AttributeError: 'Process' object has no attribute 'cpu_affinity'

Seems to be a problem with my psutil version on Mac OS.  My current version is 5.6.3.
This illustrates the root of the problem:
(rlpyt) Bills-Mac-mini-2:rlpyt billtubbs$ python
Python 3.7.4 (default, Aug 13 2019, 15:17:50)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.__version__
'5.6.3'
>>> import psutil
>>> p = psutil.Process()
>>> p.cpu_affinity
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Process' object has no attribute 'cpu_affinity'

Full installation and execution details:
(base) Bills-Mac-mini-2:rlpyt billtubbs$ ls
LICENSE			linux_cpu.yml		rlpyt.egg-info
README.md		linux_cuda10.yml	scratch
data			linux_cuda9.yml		setup.py
examples		macos_cpu.yml
images			rlpyt
(base) Bills-Mac-mini-2:rlpyt billtubbs$ conda env create -f macos_cpu.yml
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
pytorch-1.3.0        | 32.6 MB   | ################################### | 100%
setuptools-41.4.0    | 644 KB    | ################################### | 100%
wheel-0.33.6         | 40 KB     | ################################### | 100%
sqlite-3.30.0        | 2.4 MB    | ################################### | 100%
numpy-base-1.17.2    | 5.0 MB    | ################################### | 100%
numpy-1.17.2         | 4 KB      | ################################### | 100%
pip-19.2.3           | 1.9 MB    | ################################### | 100%
mkl_random-1.1.0     | 320 KB    | ################################### | 100%
mkl-service-2.3.0    | 187 KB    | ################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Ran pip subprocess with arguments:
['/anaconda3/envs/rlpyt/bin/python', '-m', 'pip', 'install', '-U', '-r', '/Users/billtubbs/rlpyt/condaenv.m7_60k9g.requirements.txt']
Pip subprocess output:
Collecting atari-py (from -r /Users/billtubbs/rlpyt/condaenv.m7_60k9g.requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/f9/bb/b01e780ac5abc41f41b37faeaa005fcdabffe37713d943f86190305c8294/atari_py-0.2.6-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting pyprind (from -r /Users/billtubbs/rlpyt/condaenv.m7_60k9g.requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/1e/30/e76fb0c45da8aef49ea8d2a90d4e7a6877b45894c25f12fb961f009a891e/PyPrind-2.11.2-py3-none-any.whl
Requirement already satisfied, skipping upgrade: numpy in /anaconda3/envs/rlpyt/lib/python3.7/site-packages (from atari-py->-r /Users/billtubbs/rlpyt/condaenv.m7_60k9g.requirements.txt (line 1)) (1.17.2)
Requirement already satisfied, skipping upgrade: six in /anaconda3/envs/rlpyt/lib/python3.7/site-packages (from atari-py->-r /Users/billtubbs/rlpyt/condaenv.m7_60k9g.requirements.txt (line 1)) (1.12.0)
Installing collected packages: atari-py, pyprind
Successfully installed atari-py-0.2.6 pyprind-2.11.2

#
# To activate this environment, use
#
#     $ conda activate rlpyt
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) Bills-Mac-mini-2:rlpyt billtubbs$ source activate rlpyt
(rlpyt) Bills-Mac-mini-2:rlpyt billtubbs$ python examples/atari_dqn_async_cpu.py
Traceback (most recent call last):
  File "examples/atari_dqn_async_cpu.py", line 7, in <module>
    from rlpyt.utils.launching.affinity import make_affinity
ModuleNotFoundError: No module named 'rlpyt'
(rlpyt) Bills-Mac-mini-2:rlpyt billtubbs$ pip install -e .
Obtaining file:///Users/billtubbs/rlpyt
Installing collected packages: rlpyt
  Running setup.py develop for rlpyt
Successfully installed rlpyt
(rlpyt) Bills-Mac-mini-2:rlpyt billtubbs$ python examples/atari_dqn_async_cpu.py
cat: /proc/cpuinfo: No such file or directory
logger_context received log_dir outside of /Users/billtubbs/rlpyt/data: prepending by /Users/billtubbs/rlpyt/data/local/<yyyymmdd>/
using seed 9387
2019-10-12 11:07:28.712698  | async_dqn_pong_0 Running 50000 sampler iterations.
2019-10-12 11:07:29.073120  | async_dqn_pong_0 Frame-based buffer using 4-frame sequences.
2019-10-12 11:07:46.956154  | async_dqn_pong_0 Agent setting min/max epsilon itrs: 250, 25000
Traceback (most recent call last):
  File "examples/atari_dqn_async_cpu.py", line 70, in <module>
    run_ID=args.run_ID,
  File "examples/atari_dqn_async_cpu.py", line 59, in build_and_train
    runner.train()
  File "/Users/billtubbs/rlpyt/rlpyt/runners/async_rl.py", line 40, in train
    throttle_itr, delta_throttle_itr = self.startup()
  File "/Users/billtubbs/rlpyt/rlpyt/runners/async_rl.py", line 108, in startup
    throttle_itr, delta_throttle_itr = self.optim_startup()
  File "/Users/billtubbs/rlpyt/rlpyt/runners/async_rl.py", line 115, in optim_startup
    p.cpu_affinity(main_affinity["cpus"])
AttributeError: 'Process' object has no attribute 'cpu_affinity'

Or, maybe I shouldn't be trying to run this example on Mac OS with CPUs?

I found this project that had this issue and resolved it:

[crossbar](https://github.com › crossbario › crossbar): https://github.com/crossbario/crossbar/issues/1333

In the psutil documentation for cpu_affinity it says:

Availability: Linux, Windows, FreeBSD

Hi, thanks for the question!
Short answer: to run on MacOS, comment out the lines with cpu_affinity.
Longer answer:
Yes, my understanding is that the cpu_affinity functionality is not available in MacOS.  Some places in the code (like the sampler parallel worker), there's a try statement to protect against this.  I must've left this out of the async code (developed a little later).  Also the async code only really makes sense with GPUs, which I figured would be on Linux and not MacOS  (but maybe so?).  In general in MacOS I don't think you can control the cpu affinity of processes.
Why affinity:
In some experiments I found sampling to run faster when assigning the CPU of each processes, so they stay on their own hardware lanes and the system doesn't get bogged down with the OS's cpu scheduler shifting processes all around.  But it's not a hard and fast rule, just something to try for your own program. :)

Thanks.  This makes sense.  Maybe I shouldn't be trying to run this example.  What's a good example to script to get me started on Mac OS X with CPUs?  (I'm just trying out this package for the first time and will transition to a linux-GPU machine once I figured it out).

Looks like examples/example_1.py seems to be the best place to start.  This is working now.  Thanks.


[Extension Bug]

Extension – Title

Author:

Description
Steps To Reproduce

The current behavior
The expected behavior
Raycast version
Version:

Could you fill up the template?


migrate to viewer 7.0 and Pushpin 7.0

Please add a branch of v6 (previous version before merge the PR..

seems like the mainViewer is never used: https://github.com/Autodesk-Forge/bim360-csharp-issues/pull/8/files#diff-e56c5c4b9e74d416f3e7f561db1f6b37R19
maybe use the same naming as learnforge sample? https://github.com/Autodesk-Forge/learn.forge.viewhubmodels/blob/master/js/ForgeViewer.js

thanks for spotting this! I tried a couple of codes of migrating to v7 until the current codes can work without any error in console, but forgot to check the mismatch of the variable. fixed.


Manage Existing Short URLs

Manage existing short URLs.

This is currently not possible, do to the fact that the Polr API does not allow for managing existing shortened URLs.
https://docs.polr.me/en/latest/developer-guide/api/#api-endpoints

I created an issue on Polr that is the first step to adding this functionality. cydrobolt/polr #221


Impossible to partially merge semantics

Imagine a contact list where you can click on a list entry to call the contact or you can click on a trailing i icon to bring up more information about that contact. Essentially, a tile of that contact list would look like this:
----------------------------------
|                     --------   |
| Max Mustermann      |   i  |   |
|                     --------   |
----------------------------------

Only the big and the little box should be a11y focusable. When the big box is focused, it should read out "Max Mustermann". Essentially, we would want the SemanticsTree to look like this:
SemanticsNode(label: "Max Mustermann", actions: [SemanticsAction.tap])
   - SemanticsNode(label: "Details", actions: [SemanticsAction.tap])

With our current implementation, that SemanticsTree is very hard (impossible?) to achieve because we don't allow partial merging.
Out of the box, the SemanticsTree would currently look like this:
SemanticsNode(actions: [SemanticsAction.tap])
   - SemanticsNode(label: "Max Mustermann")
   - SemanticsNode(label: "Details", actions: [SemanticsAction.tap])

All three Nodes are focusable ==> sad.
All you can currently really do about it is throw in a MergeSemantics, which yields the following SemanticsTree:
SemanticsNode(label: "Max Mustermann\nDetails", actions: [SemanticsAction.tap])

It all gets munched into one node, you can't focus on the little i anymore and you can only perform one action (the call action probably wins) ==> double sad.
What we really need is some way to partially merge semantics. Ideally, the system should figure out automatically what needs to be merged without the developer telling flutter.

The new a11y algorithm has solved this.


[FLINK-20625][pubsub,e2e] Add Google Cloud Pub/Sub source connector in accordance with FLIP-27

What is the purpose of the change
Adds a source connector implementation following FLIP-27 to consume messages from Google Cloud Pub/Sub.
Brief change log

Add basic Pub/Sub source using new interface
Add checkpointing and do some refactorings
Allow multiple records inside single Pub/Sub message for deserialization
Add Javadocs, README and clean up

Verifying this change
A new end-to-end test EmulatedPubSubNewSourceTest.java was added to verify these changes. However, currently the test never finishes and instead keeps waiting for new input records forever. The end-to-end test for the old source (EmulatedPubSubSourceTest.java) uses a stop marker mechanism to finish the test. I tried out different things to apply it to the new source end-to-end test but without success. Maybe you have an idea?
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (don't know)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (yes)
If yes, how is the feature documented? (JavaDocs)

@fapaul @AHeise after last week's final presentation in the seminar we had another look at our benchmarking and figured out that our data generator was a big bottleneck. Thus we were able to increase the throughput a lot in comparison. Here are the new results: https://docs.google.com/document/d/1235-Cc-fHXvsPzYiA_ISbhQBnfWU-RyqpMoQB4jKrUU/edit?usp=sharing

@fapaul @AHeise after last week's final presentation in the seminar we had another look at our benchmarking and figured out that our data generator was a big bottleneck. Thus we were able to increase the throughput a lot in comparison. Here are the new results: https://docs.google.com/document/d/1235-Cc-fHXvsPzYiA_ISbhQBnfWU-RyqpMoQB4jKrUU/edit?usp=sharing

@jakob-ed are those results still against the emulator?

@fapaul @AHeise after last week's final presentation in the seminar we had another look at our benchmarking and figured out that our data generator was a big bottleneck. Thus we were able to increase the throughput a lot in comparison. Here are the new results: https://docs.google.com/document/d/1235-Cc-fHXvsPzYiA_ISbhQBnfWU-RyqpMoQB4jKrUU/edit?usp=sharing

@jakob-ed are those results still against the emulator?

Yes, they are still against the emulator so far.

@jakob-ed What do you think is the status of this PR?

Hello!  I'm interested in continuing this work -- I don't want to step on any toes!  I've created a new PR (in draft for now) that rebases, fixes merge conflicts and the never-ending end-to-end test.  I kept your authorship since all of the code is the same.
I'll be adding some additional tests while master is frozen for 1.15; don't hesitate to let me know if you'd prefer I proceed differently!

Much appreciated @RyanSkraba

For info: this PR has been migrated with the connector and can be closed.  The new PR is from my repo but the commits retain @JakobEdding 's authorship.


Whilelist verify jenkins server

What
Whitelist verify jenkins IP so that they can deploy onto CF. User for this jenkins has been already created.
How to review
Double-check the ip address is one mentioned in #1050 support ticket in deskpro.
Who can review
not @mtekel

:+1:


should update on navigation or the subsequent updatefound event be delayed until document DOM is loaded?

Recently we got a bug report for the following demo:
https://adriftwith.me/sandbox/service_worker_reload/go.html
When you refresh in chrome you always get the "new ServiceWorker installed" text.  Usually it seems after some delay.
In firefox, however, you usually don't get the text, but sometimes you do.
I believe this is happening because we trigger the update immediately after the fetch event resolves.  This means the update can complete before the document script has a chance to attach to the registration objects.  So it never sees the updatefound events.
I agree with the developers that it would be nice to reliably get updatefound events if you are updating on navigation.
Should the spec be changed to delay navigation updates or the navigation updatefound event until the target document has completed loading?

In this gecko bug I am modifying our code to delay update until a controlled document reaches DOMContentLoaded.
https://bugzilla.mozilla.org/show_bug.cgi?id=1226443

Presumably chrome is delaying for such a long time so that it the update does not regression load times.

Yes Chrome triggers update after a one second delay after load completes, so it doesn't contend with the network for loading the page.

@annevk  what'd be a good way to spec this requirement? Setting a fixed amount of delay in the spec doesn't seem like a good idea. Also, the target resource can be a worker, a shared worker as well as a document, so no unified concept can be reference to get the right timing. Any idea?

Depends on the semantics you want I suppose. (Note that workers cannot be navigated so I'm not sure how the same problem applies to them.)

Soft Update is triggered for all the non-subresource requests (whose destination is not "serviceworker" of course).

@mattto What does chrome do if the client has been closed when the update timer triggers?  Do you go ahead with the update or cancel the update?

We continue with the update, it's scheduled on the SW entity itself rather than attached to the client.
For that matter, opening lots of tabs really quickly will only trigger one update in Chrome.

I'm not against delaying the update a bit for performance reasons (or at least suggest it can be low priority), but I don't think it fixes the original issue.
Imagine:

Functional event lands in SW
More than 24hrs since update, so update triggered, updated SW found, becomes installing
User navigates to in-scope page
"updatefound" listener added
Update triggered, SW is byte identical so it's ignored
SW moves from "installing" to "installed"

Even with the delay, the listener misses the new SW. The only safe way know when a SW becomes installed is to listen for "updatefound" but also track any current reg.installing. Eg https://github.com/jakearchibald/wittr/blob/cache-avatars/public/js/main/IndexController.js#L45.

I think that case is a race between two different operations.  I'm not sure devs would expect to duplicate those conditions often.
The update on navigate, though, is a very repeatable use case.
FWIW, in gecko we have implemented a 1 second delay after document load event.  For non-navigation events that trigger an update we fire them 1 second after the event.  One an update check is scheduled, any further update checks are coalesced into that currently scheduled timer.  We don't push back the timer, though, so that we are guaranteed to fire the update regardless of how many events the SW is receiving.
I think this is close to what chrome is doing, but not 100% sure.

Chrome's implementation tries to schedule the updates after the worker stops:

Schedule an update 1 second after page load for a navigation in a SW's scope.
Whenever the active worker stops, if it hasn't attempted an update in 24 hours, and no update is scheduled, schedule an update.
If the active worker has been running without stopping for 5 minutes, if hasn't attempted an update in 24 hours, and no update is scheduled, schedule an update.

Scheduling an update pushes back the timer.
I implemented this before the spec required Soft Updates at the end of each event. I wanted to wait for the worker to stop to avoid situations where the worker triggers an update and skipWaiting() kills it while it's in middle of an event (Chrome doesn't wait for the active worker to be finish in-progress requests... that's a bug). But as it's approximately the same as the spec (the rough result is a Soft Update approx every 24 hours), I probably wouldn't expend much effort to get it to exactly match unless there's a real benefit for devs.

F2F resolution: Add a note to the spec to suggest delaying triggering an update until after "DOMContentLoaded" of the associated page. If it's a worker, don't update until the after the worker script has run.


HBASE-27485 HBaseTestingUtility minicluster requires log4j2

@ndimiduk I just removed the setLogLevel call in HBTU, and then added some configurations in log4j2.properties to set the log level. I think for downstreamer users, since they could use different logging framework, it is their duty to change the log level if they want.
Thanks.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 39s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
hbaseanti
0m  0s
Patch does not have any anti-patterns.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 13s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
2m 57s
master passed

+1 :green_heart:
compile
2m 58s
master passed

+1 :green_heart:
checkstyle
0m 49s
master passed

+1 :green_heart:
spotless
0m 43s
branch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
2m  8s
master passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 10s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 34s
the patch passed

+1 :green_heart:
compile
2m 56s
the patch passed

+1 :green_heart:
javac
2m 56s
the patch passed

+1 :green_heart:
checkstyle
0m 43s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
hadoopcheck
9m 45s
Patch does not cause any errors with Hadoop 3.2.4 3.3.4.

+1 :green_heart:
spotless
0m 43s
patch has no errors when running spotless:check.

+1 :green_heart:
spotbugs
2m 32s
the patch passed

_ Other Tests _

+1 :green_heart:
asflicense
0m 22s
The patch does not generate ASF License warnings.

36m 49s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4941

Optional Tests
dupname asflicense spotless javac spotbugs hadoopcheck hbaseanti checkstyle compile

uname
Linux d6b8e6ad9f03 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 7ef63b65f0

Default Java
Eclipse Adoptium-11.0.17+8

Max. process+thread count
80 (vs. ulimit of 30000)

modules
C: hbase-logging hbase-server hbase-testing-util U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/console

versions
git=2.34.1 maven=3.8.6 spotbugs=4.7.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 37s
Docker mode activated.

-0 :warning:
yetus
0m  3s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 12s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
2m 23s
master passed

+1 :green_heart:
compile
0m 55s
master passed

+1 :green_heart:
shadedjars
4m  4s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 40s
master passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 10s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m  4s
the patch passed

+1 :green_heart:
compile
0m 55s
the patch passed

+1 :green_heart:
javac
0m 55s
the patch passed

+1 :green_heart:
shadedjars
4m  5s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 38s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
0m  8s
hbase-logging in the patch passed.

+1 :green_heart:
unit
208m  6s
hbase-server in the patch passed.

+1 :green_heart:
unit
2m 16s
hbase-testing-util in the patch passed.

231m 56s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4941

Optional Tests
unit javac javadoc shadedjars compile

uname
Linux ffd0d9261b96 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 7ef63b65f0

Default Java
Temurin-1.8.0_352-b08

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/testReport/

Max. process+thread count
2641 (vs. ulimit of 30000)

modules
C: hbase-logging hbase-server hbase-testing-util U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 54s
Docker mode activated.

-0 :warning:
yetus
0m  4s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ master Compile Tests _

+0 :ok:
mvndep
0m 12s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
2m 48s
master passed

+1 :green_heart:
compile
1m 11s
master passed

+1 :green_heart:
shadedjars
3m 47s
branch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 52s
master passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 13s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 31s
the patch passed

+1 :green_heart:
compile
1m 10s
the patch passed

+1 :green_heart:
javac
1m 10s
the patch passed

+1 :green_heart:
shadedjars
3m 45s
patch has no errors when building our shaded downstream artifacts.

+1 :green_heart:
javadoc
0m 50s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
0m 11s
hbase-logging in the patch passed.

+1 :green_heart:
unit
215m 18s
hbase-server in the patch passed.

+1 :green_heart:
unit
2m 14s
hbase-testing-util in the patch passed.

240m 31s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/4941

Optional Tests
unit javac javadoc shadedjars compile

uname
Linux 2be03a2d6a17 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 7ef63b65f0

Default Java
Eclipse Adoptium-11.0.17+8

Test Results
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/testReport/

Max. process+thread count
2792 (vs. ulimit of 30000)

modules
C: hbase-logging hbase-server hbase-testing-util U: .

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4941/1/console

versions
git=2.34.1 maven=3.8.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


Expose Siftdescriptor

Hi, I need to estimate similarity score which is available by vl_feat as follow:
[fa, da] = vl_sift(Image_a) ;
[fb, db] = vl_sift(Image_b) ;
[matches, scores] = vl_ubcmatch(da, db) ;
has Vl_ubcmatch been implemented by cython ? or not yet ?


Typo

Description of the change:Fixing a typo
Reason for the change:
Link to original source:

Closes #

Hello @beccaweisssg,Thanks again for the PR! It's HACKTOBERFEST! We want to show our appreciation by sending                                              you some special Hacktoberfest swag. If you have not already, could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX


Upgrade Notebook with codeflare-sdk version 0.21.1

Description

Upgrade Notebook with codeflare-sdk version 0.21.1
Fixes: #719
As the github action failed: https://github.com/project-codeflare/codeflare-sdk/actions/runs/11134262380
How Has This Been Tested?

NA
Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work

/lgtm

/retest-required

@harshad16, are these skipped intentionally? Not touching any intel ones?
jupyter/intel/ml/ubi9-python-3.11/Pipfile:codeflare-sdk = "~=0.19.1"
jupyter/intel/pytorch/ubi9-python-3.11/Pipfile.cpu:codeflare-sdk = "~=0.13.0"
jupyter/intel/pytorch/ubi9-python-3.11/Pipfile.gpu:codeflare-sdk = "~=0.13.0"
jupyter/intel/tensorflow/ubi9-python-3.11/Pipfile.cpu:codeflare-sdk = "~=0.13.0"
jupyter/intel/tensorflow/ubi9-python-3.11/Pipfile.gpu:codeflare-sdk = "~=0.13.0"
intel/runtimes/ml/ubi9-python-3.11/Pipfile:codeflare-sdk = "~=0.19.1"
intel/runtimes/pytorch/ubi9-python-3.11/Pipfile.cpu:codeflare-sdk = "~=0.13.0"
intel/runtimes/pytorch/ubi9-python-3.11/Pipfile.gpu:codeflare-sdk = "~=0.13.0"
intel/runtimes/tensorflow/ubi9-python-3.11/Pipfile.cpu:codeflare-sdk = "~=0.13.0"
intel/runtimes/tensorflow/ubi9-python-3.11/Pipfile.gpu:codeflare-sdk = "~=0.13.0"

are these skipped intentionally? Not touching any intel ones?

yes @jstourac
currently it isn't part of our OOTB
I m leaving it out of the custom PR, lets see it later on.

are these skipped intentionally? Not touching any intel ones?

yes @jstourac currently it isn't part of our OOTB I m leaving it out of the custom PR, lets see it later on.

Great, thanks for confirmation, Harshad.
In that case /lgtm for this one.
As such, should we ignore python 3.9 and intel based images in the GHA from the codeflare sdk repository?

As such, should we ignore python 3.9 and intel based images in the GHA from the codeflare sdk repository?

i m sure on python3.9
about intel, will confirm after some discussions.

Thanks for all the review
/approve
/override ci/prow/images
/override ci/prow/notebook-rocm-jupyter-pyt-ubi9-python-3-11-pr-image-mirror
/override ci/prow/notebooks-ubi9-e2e-tests
/override ci/prow/rocm-notebooks-e2e-tests
/override ci/prow/rocm-runtimes-ubi9-e2e-tests
/override ci/prow/runtimes-ubi9-e2e-tests

@harshad16 Also can you include this codeflare-sdk version 0.21.1 changes to the notebook for RHOAI 2.14 release branch

@Srihari1192 all the work would be moved today to release branch.


How to handle sentinel client failover?

Hey guys, how can I deal with sentinel instance fail-over, in which circumstance GetMaster always returns a ClientError.
Do I have to close that sentinel and try another one manually? Any better solutions?

Do I have to close that sentinel and try another one manually?

Unfortunately yes, I think calling Close and remaking the client again is the only solution this package provides.
I would encourage you in general to check out the radix.v2 package. It's actively supported and developed and in many ways is much easier to use than this one, as well as being faster. Unfortunately it probably won't make this particular problem any easier to solve, as the sentinel package remained largely the same =/

Thank you very much!


Feature - Opening Up Request

There are a couple of changes here.
Removal of respondsToSelector
The first was to remove the respondsToSelector override in the Manager class. The only reason I could come up with as to why they were there in the first place was that the original SessionDelegate override closures were internal. Having this override would allow clients to "peak" into whether or not those internal closures were set. Since the override closures are now public themselves, this logic no longer seems necessary.

If anyone can think of a good reason as to why these shouldn't be removed...please comment on this PR!

Opening the TaskDelegate Queue
I originally set out to open up the TaskDelegate APIs completely. After a deep investigation of the way the task delegate override closures are implemented and abstracted, I think it's best to leave them as they are. They are extremely well designed and provide an amazing amount of abstraction that can be overridden by the SessionDelegate closures. If a user really needs to customize the default behaviors, the SessionDelegate override closures should be used instead. Opening the TaskDelegate override closures would not only be redundant, but I'm afraid it would open up an abundance of confusion to the general user.
With that said, there is still an opportunity to allow users to implement custom behaviors on the TaskDelegate queue. At the moment, you cannot customize any Request operation logic until one of the response serializers are called. This is problematic if you need to run any other operations prior to this. For example, if you wanted to run some pre and post operations around validation, you couldn't do so.
Because of this, I opened up the TaskDelegate and queue to allow for customization. I also added some unit tests around this behavior to ensure it works as expected. This should give users ultimate flexibility without exposing hardly any of the internal API. Win win win. I'm excited to see what users will end up doing with this new found freedom! 😁

cc @kylef and @kcharwood

The removal of respondsToSelector seems to have broken NTLM (and maybe other authentification methods?) support. I've opened a separate issue #721 for this.


Updated FOC files

Description
Updated FOC files for NRT-Streamflow, Bridge, and MR.

Due to Air Force deadlines, I approved my own pull request.
breaking the law, breaking the law
breaking the law, breaking the law


bug - IOS 15. buttons (and other elements) are not pressed at the top (in the area of 50-60 pixels from the top)

bug - IOS 15. buttons (and other elements) are not pressed at the top (in the area of 50-60 pixels from the top)

Hello, please provide full reproducible SwiftUI view code or project and specify exact iOS version where the problem occurs.

Hello, any empty project. iphone 7 ios 15.0


Simplecov doesnt generating report with apartment gem

Simplecov doesnt generating report with apartment gem when i put the model at config.excluded_models .. I think the gem do a constantize at the model...
How can i include to generate the report?

In order to help, we need to be able to reproduce your issue. Please read https://github.com/colszowka/simplecov/blob/master/CONTRIBUTING.md#reporting-issues

Please reopen this issue with more details if you’re still unable to get it working.


os: File.Seek on Windows errors on offsets like 4G-1, 8G-1, 16G-1

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
1.8.1
Does this issue reproduce with the latest release?
I didn't retry, but by looking at the relevant code on the master branch it looks like the bug is still there.
What operating system and processor architecture are you using (go env)?
Windows 64 bit
What did you do?
Create a 4GB sparse file by moving the file pointer to before the last byte and then write a \x00
Code at:
https://play.golang.org/p/kzMJvpa_eJ
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
What did you expect to see?
It should create the file and returns with success
What did you see instead?
Instead it returns an invalid argument error
This is because SetFilePointer did not handle the return value correctly.  According to Microsoft's doc:

Note  Because INVALID_SET_FILE_POINTER is a valid value for the low-order DWORD of the new file pointer, you must check both the return value of the function and the error code returned by GetLastError to determine whether or not an error has occurred. If an error has occurred, the return value of SetFilePointer is INVALID_SET_FILE_POINTER and GetLastError returns a value other than NO_ERROR. For a code example that demonstrates how to check for failure, see the Remarks section in this topic.

SetFilePointer in Go always returns the error if you pass 4G - 1, 8G - 1, 16G - 1, etc to the Seek function, even if the operation does return successfully.

Seems like the fix plus a test would be rather simple - would you like to give it a try? See https://golang.org/doc/contribute.html

@mvdan I can try, but it may take a few days.

SetFilePointerEx seems to be the more appropriate function here, it's even suggested in the docs of SetFilePointer for 64 bit seeks.

This function stores the file pointer in two LONG values. To work with file pointers that are larger than a single LONG value, it is easier to use the SetFilePointerEx function.

This is fixed by https://github.com/gilbertchen/duplicacy/commit/197d20f0e07284c9c8a35d9d0800b366d53b2f18

Gentle ping, how's it going @gilbertchen and @mvdan?

I tried the trivial change, like:
diff --git a/src/syscall/syscall_windows.go b/src/syscall/syscall_windows.go
index 21d5ecf..e6ceb9c 100644
--- a/src/syscall/syscall_windows.go
+++ b/src/syscall/syscall_windows.go
@@ -9,6 +9,7 @@ package syscall
 import (
        errorspkg "errors"
        "internal/race"
+       "runtime"
        "sync"
        "unicode/utf16"
        "unsafe"
@@ -332,6 +333,16 @@ func Write(fd Handle, p []byte) (n int, err error) {

 var ioSync int64

+var procSetFilePointerEx = modkernel32.NewProc("SetFilePointerEx")
+
+func setFilePointerEx(handle Handle, distToMove int64, newFilePointer *int64, whence uint32) error {
+       _, _, e1 := Syscall6(procSetFilePointerEx.Addr(), 4, uintptr(distToMove), uintptr(unsafe.Pointer(newFilePointer)), uintptr(whence), 0, 0, 0)
+       if e1 != 0 {
+               return errnoErr(e1)
+       }
+       return nil
+}
+
 func Seek(fd Handle, offset int64, whence int) (newoffset int64, err error) {
        var w uint32
        switch whence {
@@ -342,13 +353,17 @@ func Seek(fd Handle, offset int64, whence int) (newoffset int64, err error) {
        case 2:
                w = FILE_END
        }
-       hi := int32(offset >> 32)
-       lo := int32(offset)
        // use GetFileType to check pipe, pipe can't do seek
        ft, _ := GetFileType(fd)
        if ft == FILE_TYPE_PIPE {
                return 0, ESPIPE
        }
+       if runtime.GOARCH == "amd64" {
+               err = setFilePointerEx(fd, offset, &newoffset, w)
+               return
+       }
+       hi := int32(offset >> 32)
+       lo := int32(offset)
        rlo, e := SetFilePointer(fd, lo, &hi, w)
        if e != nil {
                return 0, e

But that doesn't work. Looks like SetFilePointerEx comes with a bunch of warnings about multi-threaded programs and overlapped I/O. Not sure whether those apply.
Also, that GOARCH == "amd64" check is bogus. This should also work on 386 but I don't know how to shove a LARGE_INTEGER through syscall.Syscall6 on Windows.
Maybe @alexbrainman is interested in looking into this?

This is an existing issue.  We can look at a CL, but otherwise kicking it down the road.

This is the how SetFilePointerEx is called in Rust, maybe you can copy that (except for the actual calling of the syscall): https://github.com/rust-lang/rust/blob/833785b090c30d4a359d901fb41bfafbe1607ce9/src/libstd/sys/windows/fs.rs#L337-L352.
I think the remarks about overlapped IO and multithreading are irrelevant for the implementation of this function, it's clear that two threads trying to set the file pointer and read a bit from the file must synchronize somehow, that's no difference between SetFilePointer and SetFilePointerEx.

I think the remarks about overlapped IO and multithreading are irrelevant for the implementation

Yeah, true.
I just don't know the calling convention for GOARCH=386 passing a LARGE_INTEGER to Windows via Go's Syscall6.
But then again, my change also didn't work on amd64, so not sure what I screwed up.

_, _, e1 := Syscall6(procSetFilePointerEx.Addr(), 4, uintptr(distToMove), uintptr(unsafe.Pointer(newFilePointer)), uintptr(whence), 0, 0, 0)
@bradfitz
The handle is passed in to setFilePointerEx, but it isn't given as an argument to the actuall syscall.Syscall6.

@as, hah. Whoops. Thanks!

Ah, here's what LARGE_INTEGER means: https://msdn.microsoft.com/en-us/library/windows/desktop/aa383713(v=vs.85).aspx

And I can repro that my new test fails on 386 and passes with my new fix on amd64:
bradfitz@gdev:~/go/src/os$ gomote run user-bradfitz-windows-amd64-2016-0 go/bin/go test -short -v -run=Seek os
=== RUN   TestSeek
--- PASS: TestSeek (0.01s)
=== RUN   TestSeekError
--- PASS: TestSeekError (0.00s)
PASS
ok      os      0.045s
bradfitz@gdev:~/go/src/os$ gomote run -e GOARCH=386  user-bradfitz-windows-amd64-2016-0 go/bin/go test -short -v -run=Seek os
=== RUN   TestSeek
--- FAIL: TestSeek (0.01s)
        os_test.go:1382: #8: Seek(4294967295, 0) = 0, seek C:\Users\gopher\AppData\Local\Temp\1\_Go_TestSeek195906635: invalid argument want<PHONE_NUMBER>, nil
        os_test.go:1382: #9: Seek(0, 1) = 0, seek C:\Users\gopher\AppData\Local\Temp\1\_Go_TestSeek195906635: invalid argument want<PHONE_NUMBER>, nil
        os_test.go:1382: #10: Seek(8589934591, 0) = 0, seek C:\Users\gopher\AppData\Local\Temp\1\_Go_TestSeek195906635: invalid argument want<PHONE_NUMBER>, nil
        os_test.go:1382: #11: Seek(0, 1) = 0, seek C:\Users\gopher\AppData\Local\Temp\1\_Go_TestSeek195906635: invalid argument want<PHONE_NUMBER>, nil
=== RUN   TestSeekError
--- PASS: TestSeekError (0.00s)
FAIL
FAIL    os      0.137s
Error running run: exit status 1

Fixing on 386 now.


Operator sometimes gets stuck

I have only seen this in CI or dev deployments so far.
Sometimes the operator just gets stuck and even though the jupyterserver resources can be created the operator does not act to create the child resources required.
In all cases simply deleting the operator pod fixes this. But we should try to catch or avoid this behaviour all together.

As of yesteday this started occurring on production deployments. Namely on renkulab there were several sessions where the status was not updating because the operator was stuck. Users could not use these sessions because the ui and notebook service depend on the status of the jypyterserver resource to judge when the sessions are ready. The operator seems to have been stuck so the status of every new session was not being updated.
Deleting the operator pod and letting it get recreated fixes this. But it should not occur in the first place.
Have not been able to replicate.

It seems that this is related to these series of events (extracted from the log of the operator):

A GET to https://<IP_ADDRESS>:443/apis/amalthea.dev/v1alpha1/namespaces/renku/jupyterservers?watch=true&resourceVersion=23516041 fails 1/9
A repeat of the GET request fails 2/9
After this amalthea starts shutting down its watches on any resources it is supposed to control Stopping the watch-stream for jupyterservers.v1alpha1.amalthea.dev in 'renku'
Then we get the errors like: Watcher for jupyterservers.v1alpha1.amalthea.dev@renku has failed
I think at this point the controller is fully unresponsive.

I tried to replicate a scenario like this by deploying Amalthea, then editing its role to prevent it from having any rights to touch Jupyterservers. During this time I create a new session. As expected this new session is not created. But when the roles are fixed Amalthea still does not respond to the new session. It only creates the new session after the operator pod is fully deleted and restarted.
So this shows the same outcome as the issue described here. But the error messages in the log are the same as what we saw in production when this happens "naturally". I will attempt to trigger this once more by adding a network policy that will block all egress for the operator pod.

Another attempt to replicate this issue has been made. In this case by using a k8s networkpolicy that blocks all egress for the amalthea operator.
This gets close to the original error where I can see this kind of messages:
"Request attempt #3/9 failed; will retry: GET https://<IP_ADDRESS>:443/api/v1/namespaces/tasko/pods?watch=true&resourceVersion=8314911 -> ClientConnectorError(ConnectionKey(host='<IP_ADDRESS>'

However the difference between this and the original error is that the resource watches are never shut down by amalthea. I think this is because when I apply the network policy this always results in a 5XX - like error. And kopf treats these as eventually recoverable. So it will keep trying.
Based on the logs of the original error the k8s api eventually returns a 403:
{"message": "Watcher for jupyterservers.v1alpha1.amalthea.dev@renku has failed: ('jupyterservers.amalthea.dev is forbidden: User \"system:serviceaccount:renku:renku-amalthea\" cannot watch resource \"jupyterservers\" in API group \"amalthea.dev\" in the namespace \"renku\": RBAC: role.rbac.authorization.k8s.io \"renku-amalthea\" not found', {'kind': 'Status', 'apiVersion': 'v1', 'metadata': {}, 'status': 'Failure', 'message': 'jupyterservers.amalthea.dev is forbidden: User \"system:serviceaccount:renku:renku-amalthea\" cannot watch resource \"jupyterservers\" in API group \"amalthea.dev\" in the namespace \"renku\": RBAC: role.rbac.authorization.k8s.io \"renku-amalthea\" not found', 'reason': 'Forbidden', 'details': {'group': 'amalthea.dev', 'kind': 'jupyterservers'}, 'code': 403})", "exc_info": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/errors.py\", line 148, in check_response\n    response.raise_for_status()\n  File \"/usr/local/lib/python3.8/site-packages/aiohttp/client_reqrep.py\", line 1000, in raise_for_status\n    raise ClientResponseError(\naiohttp.client_exceptions.ClientResponseError: 403, message='Forbidden', url=URL('https://<IP_ADDRESS>:443/apis/amalthea.dev/v1alpha1/namespaces/renku/jupyterservers?watch=true&resourceVersion=23516041')\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/aiokits/aiotasks.py\", line 69, in guard\n    await coro\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_core/reactor/queueing.py\", line 184, in watcher\n    async for raw_event in stream:\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/watching.py\", line 82, in infinite_watch\n    async for raw_event in stream:\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/watching.py\", line 182, in continuous_watch\n    async for raw_input in stream:\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/watching.py\", line 247, in watch_objs\n    async for raw_input in api.stream(\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/api.py\", line 200, in stream\n    response = await request(\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/auth.py\", line 45, in wrapper\n    return await fn(*args, **kwargs, context=context)\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/api.py\", line 85, in request\n    await errors.check_response(response)  # but do not parse it!\n  File \"/usr/local/lib/python3.8/site-packages/kopf/_cogs/clients/errors.py\", line 150, in check_response\n    raise cls(payload, status=response.status) from e\nkopf._cogs.clients.errors.APIForbiddenError: ('jupyterservers.amalthea.dev is forbidden: User \"system:serviceaccount:renku:renku-amalthea\" cannot watch resource \"jupyterservers\" in API group \"amalthea.dev\" in the namespace \"renku\": RBAC: role.rbac.authorization.k8s.io \"renku-amalthea\" not found', {'kind': 'Status', 'apiVersion': 'v1', 'metadata': {}, 'status': 'Failure', 'message': 'jupyterservers.amalthea.dev is forbidden: User \"system:serviceaccount:renku:renku-amalthea\" cannot watch resource \"jupyterservers\" in API group \"amalthea.dev\" in the namespace \"renku\": RBAC: role.rbac.authorization.k8s.io \"renku-amalthea\" not found', 'reason': 'Forbidden', 'details': {'group': 'amalthea.dev', 'kind': 'jupyterservers'}, 'code': 403})", "timestamp": "2022-02-07T16:58:55.870799+00:00", "severity": "error"}
--

I can confirm that using a networkpolicy will temporarily block amalthea from reacting to new resources. But after the deny all ingress network policy is deleted then amalthea can recover without needing a full pod delete/restart. So to replicate this we need to get a real error response from the k8s api.


How to set jQuery-Momentum-Scroll as default smooth scrolling for Chrome browser?

Hello guys! I need to set these effects as my default Chrome settings. Is it possible?

I've made an article showing you how to setup a smooth scroll based on iOS system on Chrome browser: https://medium.com/@renandiett/deixando-seu-navegador-chrome-com-o-melhor-scroll-suave-e698117cf046
Note: This article was scripted on my native language (Portuguese). See the translated version below.
Google Chrome: Setup the best smooth scroll as default
Firstly we need to disable the default smooth scroll from Chrome browser at chrome://flags/#smooth-scrolling.
Secondly we need to install the SmoothScroll extension. You can find it on Chrome Web Store.
You're done! You can setup websites that you do not want to run the extension.
Enjoy!
Official extension website: https://www.smoothscroll.net


Why not continue with PlugsWork?

PlugsWork seems great, why not just continue with it? isnt it the same with JamAuth? :o
Tags: Question
Hopefully won't mark as spam

Yeah, plugswork was awesome, why not continue?

And Plugswork was suspended for no reason... on a $49/month host.

Explained here:
https://forums.pocketmine.net/threads/link-mcpe-server-to-forum-websites.18554/#post-173414

Okay, Well. Great to see you back. Happy coding, deot.

Something similar to this would be awesome for the @ImagicalCorp to work on... Never got around to doing big projects after ImagicalMine though


[tango.me] Support for live streaming website tango.me

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting a new site support request
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that none of provided URLs violate any copyrights or contain any DRM to the best of my knowledge
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and am willing to share it if required

Region
Canada
Example URLs

Single video: https://tango.me/stream/NbZIryrYlvvAC01RCV8dcw
Single video: https://tango.me/immm11

Provide a description that is worded well enough to be understood
It says unsupoprted URL because it's redirecting to tango.me/stream/noscript.html. I even tried with cookies too.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp --cookies-from-browser firefox https://tango.me/stream/NbZIryrYlvvAC01RCV8dcw -vU

[debug] Command-line config: ['--cookies-from-browser', 'firefox', 'https://tango.me/stream/NbZIryrYlvvAC01RCV8dcw', '-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [4d9231208] (pip)
[debug] Python 3.12.7 (CPython x86_64 64bit) - Linux-6.11.5-arch1-1-x86_64-with-glibc2.40 (OpenSSL 3.4.0 22 Oct 2024, glibc 2.40)
[debug] exe versions: ffmpeg 7.0.2 (setts), ffprobe 7.0.2
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, mutagen-1.47.0, requests-2.32.3, sqlite3-3.46.1, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
Extracting cookies from firefox
[debug] Extracting cookies from: "/home/$USER/.mozilla/firefox/cl8j9z4o.default-release/cookies.sqlite"
Extracted 3071 cookies from firefox
[debug] Request Handlers: urllib, requests, websockets
[debug] Loaded 1830 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
[debug] Downloading _update_spec from https://github.com/yt-dlp/yt-dlp/releases/latest/download/_update_spec
Current version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
ERROR: You installed yt-dlp with pip or using the wheel from PyPi; Use that to update
[generic] Extracting URL: https://tango.me/stream/NbZIryrYlvvAC01RCV8dcw
[generic] NbZIryrYlvvAC01RCV8dcw: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] NbZIryrYlvvAC01RCV8dcw: Extracting information
[debug] Looking for embeds
[redirect] Following redirect to https://tango.me/stream/noscript.html
[generic] Extracting URL: https://tango.me/stream/noscript.html
[generic] noscript: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] noscript: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://tango.me/stream/noscript.html
Traceback (most recent call last):
  File "/home/$USER/.local/share/pipx/venvs/yt-dlp/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 1626, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/$USER/.local/share/pipx/venvs/yt-dlp/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 1761, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/home/$USER/.local/share/pipx/venvs/yt-dlp/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 740, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/$USER/.local/share/pipx/venvs/yt-dlp/lib/python3.12/site-packages/yt_dlp/extractor/generic.py", line 2526, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://tango.me/stream/noscript.html

I even tried with cookies too.

does the site require you to be logged in to watch streams?

I even tried with cookies too.

does the site require you to be logged in to watch streams?

Yes it does. The login options are either with google account, apple account or phone number.
I have used google account hence I tried it with using cookies.

Hi, could someone help? Will this require changes in the code or is there a way I can find a way to download using yt-dlp without changes to yt-dlp itself?

I don't have the time to work on this right now. Everything is done on a volunteer basis. Since you log in via Google account, you are likely not willing to share your account with a developer (and understandably so), making this issue difficult to work on. You'll have to be patient and wait for a developer who has their own account (or who has the time and motivation to create their own account) wants to work on this.
In the meantime, you could try sniffing out a m3u8/mpd url from your browser's dev tools and passing that to yt-dlp

I don't have the time to work on this right now. Everything is done on a volunteer basis. Since you log in via Google account, you are likely not willing to share your account with a developer (and understandably so), making this issue difficult to work on. You'll have to be patient and wait for a developer who has their own account (or who has the time and motivation to create their own account) and wants to work on this.
In the meantime, you could try sniffing out a m3u8/mpd url from your browser's dev tools and passing that to yt-dlp

I understand. I truly appreciate your efforts too!
I guess I don't mind sharing my google credentials if you're okay with it.
You can use that account whenever or however you feel like it - except deleting / closing it.
I don't mind even if you take few months to a couple of year for it. I completely understand if you decide on not doing it as well.


BAP - cannot install/update italian translations

When trying to update italian translations we get:
Could not download translations, error: Error parsing YAML, invalid file "/home/crm.mekit.it/mekitcrm/app/cache/dev/translations/it_IT/OroEmailBundle/translations/messages.it_IT.yml
To me these keys result invalid:
[Default]: '[Default]'
%subject% - Email: '%subject% - Email'
%subject% - My Emails - %username%: '%subject% - Mie emails - %username%'

Hi, @adamjakab, thank you for your feedback. According to http://www.yaml.org/refcard.html, '[]' is collection indicators, so you shouldn't use [Default] as key. As a workaround you can surround this key with quotes, e.g:
'[Default]': '[Default]'
%subject% - Email: '%subject% - Email'
%subject% - My Emails - %username%: '%subject% - Mie emails - %username%'

hi @hugeval - the translations come from the Crowdin translation platform! I agree with you that [] should not be used as key - I can only translate/ignore it - someone who manages the keys should check this out!

@adamjakab, thanks for response. I've just downloaded Italian translations for OroEmailBundle, [] are in quotes already.

Issues with true/false, with leading [ and % were fixed in crowdin and we will rebuild translations as soon as possible

Hello, now install/update translations works correct.

Hi, there are still some issues! The previouse error is now gone and I get another one "Could not download translations, error: unknown" - when updating italian translations.
I am able to see some of the translations but given that the italian translations are at 99% as far as the platform is concerned I am sure that there are some translations that are not working.

Hello, it seems that it is your local issue. What version of application do you use?
At our latest public version everything is ok.

I am on a fresh oro/platform 1.6.1 and Italian language pack, when I click download gives the error "Could not download translations, error: unknown" - however when I refresh the page I can see it is installed, I can enable it and it works.


Use Github hosted assets in emails sent by Fleet

This change allows the images in Fleet emails to load properly from any
device with connectivity to github.com. Previously, emails might try to
load resources from a Kolide server not accessible from the email
client.
The asset URL will be based on the most recent git tag to accomodate
backwards-compatibility if the assets in the repo change.
Closes #1471

IMO this doesn't need to be configurable. The new behavior should work well for almost all cases, and in cases it fails (no internet connectivity) the existing behavior wouldn't work either.

There are certainly a handful of places that would not want to expose their use of fleet to github. But, I imagine we'll hear a feature request when/if it matters to them. 👍


depressor anguli oris muscle

abnormal depressor anguli oris muscle morphology
DEF: any structural anomaly of the facial muscle that originates from the mandible and inserts into the angle of the mouth; it is innervated by the marginal mandibular branch of the facial nerve and receives its blood supply from the facial artery; the muscle depresses the corners of the mouth which is associated with frowning
SYN: abnormal triangularis muscle morphology
SOURCE: https://en.wikipedia.org/wiki/Depressor_anguli_oris_muscle
Annotations in J:36235

abnormal depressor anguli oris muscle morphology MP:0030152


⚠ Adacta-fintech.com has degraded performance

In c6508be, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 1621 ms

Resolved: Adacta-fintech.com performance has improved in 79382c5 after 9 minutes.


EZBSystems,Inc.UltraISO version <IP_ADDRESS>12

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Yeeee boiiiiii


Build 2.1.0 for GPU, osx-arm64 only

Other feedstock changes reviewed in a previous PR.

Linter check found the following problems:
The following problems have been found:
ERROR: clone/recipe/build_pytorch.sh:197: pip_install_args: output "pytorch": pip install should be run with --no-deps and --no-build-isolation.
ERROR: clone/recipe/meta.yaml:1: missing_tests: output "pytorch-gpu": No tests were found.
ERROR: clone/recipe/meta.yaml:33: patch_unnecessary: patch should not be in build when source/patches is not set
WARNING: clone/recipe/meta.yaml:132: build_tools_must_be_in_build: output "pytorch": The build tool pkg-config is not in the build section.
WARNING: clone/recipe/meta.yaml:158: build_tools_must_be_in_build: output "pytorch": The build tool ninja is not in the build section.
Errors were found


stream api for uWS

Is there a stream API like https://github.com/maxogden/websocket-stream for ws available for uWS?

I don't know, Node.js questions are in the bindings repo (but I don't know there either).


Reorganize interop articles

Status
Several problems with interop topics in .NET docs.

Scattered and repetitive. There are identical/similar contents in .NET Guide and .NET Framework Guide.
Incomplete. For more information, see #4649.

Rationale

P/Invoke is still widely adopted.
CoreCLR has enabled us to embed .NET Core into native code.
The behaviors apply to both .NET Framework and .NET Core.
The behaviors apply to both P/Invoke method and calli instruction.
The behaviors apply to both calling from managed code to native code and the reverse.

Proposal

Migrate archived interop topics back to docs. #4649
Integrate and reorganize topics in .NET Guide and .NET Framework Guide. dotnet/dotnet-api-docs#1199 and #6629.
New topics about calling from native code to managed code. .NET Core hosting
Keep consistent with C++ docs if possible.

@NextTurn, thanks for opening this issue. I also added a link to #6629 to the Proposal section. We'd welcome your contribution in improving the interop documentation, and your proposal looks really good. Before actually migrating archived content, it's a good idea to agree on the topics that are to be migrated.

Change mentions of Unicode to UTF-16. https://github.com/dotnet/dotnet-api-docs/pull/4238#discussion_r443747196