Spirc: Replace Mecury with Dealer

As suggested from @roderickvd a draft PR so i can have some community feedback and give now and then updates on the progress.
The replacement is in a working state, so the very basic do work again. There is still a lot of cleanup and re-implementing to do.
But so far what is working:

transferring the playback so that we are the player

funnily enough, transferring back isn't working yet

play, pause, resume, prev, next, seek
just general communication of our state to other clients

Known Problems:

[x] on the initial connection librespot doesn't appear, but if for example the volume is slightly updated, we should appear

this might be a web-player specific problem, in the client the dealer just pops up

this seems to be to some degree fine? if the devices are reopend it seems to be always there, it just isn't dynamically updated.

[x] if the session or spirc/dealer dies, we lose our complete ConnectState

it would be nice if that is preserved so we could start from that again

[x] sometimes other client's forget that we are the current player

~might be that that is already resolved, but i think i still saw that behavior not so long ago~
update: seems to be resolved, was probably just an early development problem

What i still have to do:

[x] handling the remaining request commands

[x] queue handling (set_queue, add_to_queue)
[x] repeat handling (set_repeating_track, set_repeating_context)

i didn't see these in the wild yet, so it might that they do not exists

[x] context handling (update_context)
[x] options handling (set_options)

should be related to shuffle and repeat

[x] transferring the state to a different player again (stop playback if we are not active anymore)
[x] resolving the context from a cold state

currently it works really well if the playback starts from a different device like the web-player
but from a cold start it's still a bit weird and not fully sorted out
we don't copy the player state from a running instance anymore,
so that we actually use the the cold state resolving/handling/transfering

[x] shuffling has to be re-implemented

we can set restrictions for the clients getting our state
so for autoplay we should ignore shuffeling and just disable repeat and shuffle

[x] autoplay has to be fixed
[x] i added the option for track repeat, but it's not handled yet
[x] i would like to remove the remaining mercury calls

but for that i will have to see if the dealer sendes equivalent messages/requests

I tried to keep my code clean, but if there are points that i could improve and such, i would like to here them^^

oh the branch might not compile because there are still some todo!() here and there which result in unreachable statements that clippy, understandably, doesn't like.

Super job! I know you're not there yet, so let's all contribute here. Stuff like this almost makes me sign up for a Spotify plan again :wink:
I'm gonna have a more thorough look later. Wanted to ping back on your comment over at #1349:

The hm://pusher/v1/connections/ message is send after connecting to set up some connect stuff initially. Can be safely ignored as far as i know. It just falls under one of the many unknown subscriptions category that are logged.

If you look at librespot-java, you'll see that it listens to this message to extract the Spotify-Connection-Id from the headers:
https://github.com/librespot-org/librespot-java/blob/d0ff31b4f476590c9c17401aac01a6762e4ba908/player/src/main/java/xyz/gianlu/librespot/player/state/DeviceStateHandler.java#L162
I think if you follow the code flow, updating the connection ID then triggers the ready event in the deviceStateListener, which in turn gets the volume. And that seems to gel with you saying that:

on the initial connection librespot doesn't appear, but if for example the volume is slightly updated, we should appear

I did recently go through old issues to see what this PR might fix. Following issues might we fix after merging these changes:

#19

repeat track is an own state in the connect state context
and i will probably adjust the repeat handling to behave like the web-player dealer

#434, #442

this should also be fixed, but that's related to resolving the context and i currently push that back because it will require a bit more time to figure out how we want to resolve that correctly

#676

sometimes i have the behavior that we don't disappear, but usually, when the web socket is closed we also disappear in the ui
might still have to investigate why we sometimes don't disappear tho

#701, #861

i just implemented the set_queue command, which does exactly resolve this problem
set_queue is also used to remove all queued tracks from the queue
but for adding an item to the queue we receive the add_to_queue command

Thanks for sifting though the issue list. I agree, with this new infrastructure in place we'll stand a real chance of fixing those and perhaps more issues. A lot were related to stuff missing on spirc.
I said it before, but mega job man. This is going to be awesome.

So... if you don't use repeat, play unavailable tracks, or have any other edge cases, this should be a quite stable state, in terms of no hard bugs and crashes (except maybe an integer underflow that might happen if you request skip_prev too much).
The current collection handling is the same as in all playlist. But it seems like spotify handles the collection differently and the index of the current track seems to be always at 10... will see if i change that because that handling seems kinda unintuitive.
While testing i also still had some weird behaviors where i had offsets where there shouldn't be any. Did happen in my liked songs. where at around 5k i had an offset of 25, around 100 3 and at the beginning none. I think it might have to do something with unavailable songs... But technically we have all songs locally loaded as context... Will investigate that probably next time.
Next up will probably be repeat, a bit more stabilization and then autoplay. The might actually still work, but i would like to adjust them to the new logic, or at least test them and fix probelem^^;

Some updates for stability. Mostly reconnecting should now work (it actually doesn't reconnect, it just picks up the transfer-state that is given to use from the first connect-state update) and updating the volume now happens with a delay of 2 seconds. Additionally the context is back to being resolved asynchronously because for a smooth transfer it was a really bad idea to resolve the context synchronously.
There still seems to be some problems here and there. Sometimes completely losing our playback on the transfer (seems to be sporadic... yay) and a weird offset problem with the position of the track, or sometimes playing the prev track if the playback is transferred around the track transition.

Curious: when shuffling is enabled, does Spotify provide the shuffled list of tracks? Or do we need to shuffle ourselves, "broadcast" the shuffled queue, and potentially un-shuffle again when requested?
I remember libzespot-java had this shuffle implementation with a reversible randomiser.

I remember librespot-java had this shuffle implementation with a reversible randomiser.

Yeah librespot-java does it that way. I would probably stay at the current solution, unless reversible shuffle is preferred or a better solution,  and just figure out how to only hold the refs in the shuffle context. By that we only have to shuffle once every time it gets enabled.

I remember librespot-java had this shuffle implementation with a reversible randomiser.

Yeah librespot-java does it that way. I would probably stay at the current solution, unless reversible shuffle is preferred or a better solution,  and just figure out how to only hold the refs in the shuffle context. By that we only have to shuffle once every time it gets enabled.

No preference. If this works, all good to me!

good god... autoplay and all the possible states is something else... I really have to revisit most of the code, because i pretty much lost the overview what is where in place. So far it does work quite decently tho.
There is only one weird thing that i couldn't solve for now. After playing all 50 autoplay tracks (or just skipping), the next autoplayed context that we retrieve, seems to be nearly identical. So there might be a missing understanding from my side what the endpoint want's from me. Will probably look into it again the next days.

So, now we also handle repeat context and track. The logic is based on how spotify does things. I will probably have to document all the new state handling way more then now. And i would also like to go over the code and split it up into more files, currently we just have two ~1k+ files, which is fine in itself, but i find it a bit overwhelming if for example one only wants to adjust a smal section.
So yeah, the next steps will be stabilizing, testing, documenting and a whole lot of cleanup :D

So you're saying it's feature complete? 👀 🤩

I would think so? When i cleanup, i will probably check again. But the main features, in addition to track repeat, should be there and working.
There are probably still some cases that are not handled yet. For example, i only tested with an android device, the web-player and an linux desktop client. As android did send some other commands then desktop and web, there might still be some other commands that aren't covered yet.

So i found a small road blocker, some requests require a uid to be available. But the resolve-context endpoint (that i now use to resolve the context) doesn't seem to send the uid when requesting for album or the collection (playlists work). The uid doesn't seem to be much of relevance for desktop or web. But mobile sends for the play command a uid to skip to. Because the "new" context endpoint doesn't send the uid we can't skip to a specified uid... if anyone has some info around the uid i would be happy to hear how we can retrieve it maybe form the default id.
I already tried it with apollo and that seems to send the uid, in addition to more metadata. But apollo resolves more then the requested context and seems to be limited at 50 tracks. Currently i like the resolve-context endpoint, because it does only resolve the requested context, and resolves it completely (it doesn't seem to have a limit, or at least it does resolve a context of 5k tracks). I will probably also look into some other requests that the official clients do, so that we could maybe use a different endpoint to resolve it on another way.

Unfortunately I have no experience with that. Best advice I can offer is to read through the librespot-java or maybe librespot-go source code to see how it's done there.
I agree that the Apollo station is too limiting.

So, just added the update of the context and playlist modification to the logic. It should work fine, but i had some cases where it behaved a bit weird. So might adjust some stuff the following days.
Besides that i only have three todo left in the code:

two in regards to replacing the mercury subscriptions for:  spotify:user:attributes:mutated and spotify:user:attributes:update

if anyone knows how to trigger them, I would much appreciate a info on that, otherwise i will probably just remove these todos

and one for the reconnecting of the player state

Info's about the reconnecting todo:
Currently when the session or dealer dies, everything is restarted. This usually isn't much of a problem, but when playing a shuffled context, it feels weird because the queue is also reshuffled. So our previous tracks we saw in the queue, are just being replaced out of nowhere. To fix that behavior i just have to handle the send context when connecting, it's easier to just ignore it, because it is usually isn't the entire context, but for this specific case it might be a good idea to handle it.
I use the current version quite regularly and for my use case it seems to work very well. Because I use bluethooth headphones it's a bit of a pain, because librespot stops working with the headphones when disconnecting and reconnecting. But the dealer aspect with desktop, web and mobile seems to work fine.

Give us a ping when you feel you're ready to go for code review.

Hmm, okay. Will try out the new changes for about a week and look into replacing the last mercury events the following days. So around next weekend i might put the pr out of the draft state. Might also make some final cleanups to the connect state because there are still some methods that without context probably are hard to distinguish what they actually do.
And thanks for the fast response and hint on the mercury subscription. I will update here again once I'm finished to a degree where we could merge it^^

Just found a part of my code that handled the updated of the context incorrect, which turned out to break the playback of artists. I put a quick fix on it, which just resolves the problem for now. But i will have to revisited a portion of the code and adjust it accordingly to handle multiple pages before falling back to autoplay or stopping the playback.

So, this might be a state that is worthwhile reviewing.
In a previous version i already replaced the attribute mutation and updates that were still related to mercury. So mercury is now completely replace with the dealer for the Spirc.
The latest updates just revisited the initial take-over handling. I did also look into the transfer error in regards to multiple librespot instances mentioned by @YutongGu in  #1394. But that seemed to be fixed? Or at least i couldn't reproduce it anymore. Besides that there was an annoying bug where a mobile device just stole the playback, which should also be fixed.
I left a single todo (the before mentioned reconnect/shuffle todo) and a fixme where it technically works, but is just a workaround cause of missing knowledge what exactly the web-player requires to behave correctly.
I might still make some changes to the branch over the course of the week if i find some bugs that can be easily fixed.
Already massive thanks for anyone reviewing the code changes in advance, because they kinda escalated just a bit ^^;

Hey @photovoltex, I tested out your PR and I must say that it looks awesome, so many bugs fixed. I (currently) only found two bugs:

search for a song using the normal search while connected to librespot. tap on the song to play. it does not play and the following error is logged:

[2024-11-24T18:05:56Z ERROR librespot_connect::spirc] couldn't handle connect state command: Invalid state { could not find track None in context of 20 }

try to move a song in the Next from: ... queue. if you do so, it will get moved but the old position will get replaced with the moved song. E. g.: Next from queue is A, B, C, D. Drag C to the beginning. Now the next from queue should be C, A, B, D. But it actually is C, B, C, D.

Hi, I also tested your PR in the Context of the MuPiBox project. There we use the Spotify web api to play on a librespot connect device. If I select an album or a playlist through the web api I get the following errors:
WARN  librespot_connect::spirc] unknown request command: {"context":{"uri":"spotify:album:1Wstoinsm4CGJ4ybpWg8iH","url":"context://spotify:album:1Wstoinsm4CGJ4ybpWg8iH"},"endpoint":"play","logging_params":{"device_identifier":"webapi-xxxx"},"options":{"seek_to":0,"skip_to":{"track_index":0}},"play_origin":{"device_identifier":"webapi-xxxx"}}
WARN  librespot_connect::spirc] unknown request command: {"context":{"uri":"spotify:playlist:57ta9SFJHUNCemuhm2ncEM","url":"context://spotify:playlist:57ta9SFJHUNCemuhm2ncEM"},"endpoint":"play","logging_params":{"device_identifier":"webapi-xxx"},"options":{"seek_to":0,"skip_to":{"track_index":0}},"play_origin":{"device_identifier":"webapi-xxx"}}

First of all, thanks for testing the branch :)

search for a song using the normal search while connected to librespot. tap on the song to play. it does not play and the following error is logged:

I can't reproduce the bug anymore. Tho, I know that there was a bug previously which blocked playing single tracks. This was fixed in https://github.com/librespot-org/librespot/pull/1356/commits/ac10e63163b3cf1f50bb00f50ad865b0a22df23e. So maybe just update your branch and the problem might go away.
If not, could you please provide a more detail explanation how you produced the incorrect state? The device (desktop, web, mobile) make a big difference (platform is probably unrelated). Especially mobile does things quite differently then web or desktop.

try to move a song in the Next from: ... queue. if you do so, it will get moved but the old position will get replaced with the moved song. E. g.: Next from queue is A, B, C, D. Drag C to the beginning. Now the next from queue should be C, A, B, D. But it actually is C, B, C, D.

~That I'm aware of, sadly. One of the pain points that are out of our control after we published our connect-state. If you try that example in the desktop or web-player it works without problems, but mobile does something else with it, which results in the messed up state of the queue. I already tried to figure out why it behave like it does, but couldn't find a solution for it, yet.
The problem with every around that is, that it is updated via the set_queue command. So we just apply what the command sends us. So there is something not quite correct with the state for the mobile client i presume.~
Scrap that, I think i just forgot to add the "queue-uid" on the set_queue command. Because it works, when the items are added via the web-player, so that might actually be an easy fix.

First of all, thanks for testing the branch :)

There we use the Spotify web api to play on a librespot connect device.

Hui... that I didn't test at all so far. In the provided case, their seems to be a missing field in the our request command defined of the play endpoint. I will try all player endpoints and add the remaining fields, so that the commands are de-serialized correctly.
Thanks for the reminder that the API exists^^. I totally forgot that there is another "client" besides web, desktop and mobile

The add from mobile should now work correctly and afterwards moving these tracks inside the queue shouldn't remove/overwrite tracks anymore.
I checked all web-api calls for the player and it seems that only the "Start/Resume Playback" had a problem with the PlayCommand model. In addition the seek_to field should now also be applied (seems to be web-api specific).

First of all, thanks for testing the branch :)

search for a song using the normal search while connected to librespot. tap on the song to play. it does not play and the following error is logged:

I can't reproduce the bug anymore. Tho, I know that there was a bug previously which blocked playing single tracks. This was fixed in https://github.com/librespot-org/librespot/pull/1356/commits/ac10e63163b3cf1f50bb00f50ad865b0a22df23e. So maybe just update your branch and the problem might go away.
If not, could you please provide a more detail explanation how you produced the incorrect state? The device (desktop, web, mobile) make a big difference (platform is probably unrelated). Especially mobile does things quite differently then web or desktop.

try to move a song in the Next from: ... queue. if you do so, it will get moved but the old position will get replaced with the moved song. E. g.: Next from queue is A, B, C, D. Drag C to the beginning. Now the next from queue should be C, A, B, D. But it actually is C, B, C, D.

~That I'm aware of, sadly. One of the pain points that are out of our control after we published our connect-state. If you try that example in the desktop or web-player it works without problems, but mobile does something else with it, which results in the messed up state of the queue. I already tried to figure out why it behave like it does, but couldn't find a solution for it, yet.
The problem with every around that is, that it is updated via the set_queue command. So we just apply what the command sends us. So there is something not quite correct with the state for the mobile client i presume.~
Scrap that, I think i just forgot to add the "queue-uid" on the set_queue command. Because it works, when the items are added via the web-player, so that might actually be an easy fix.

$ target/release/librespot --help
librespot 0.5.0 dab0948 (Built on 2024-11-24, Build ID: C54gdSQ6, Profile: release)

I already was on the latest commit when I wrote that, unfortunately. I updated to the now latest commit, both issues still exist for me. I tested both using the Android Spotify app. The queue still gets mixed up and I still get the same error message... I can also try a different client tomorrow just to verify whether it is just a problem with the Android app, if that helps.

Now it also works with the web api 👍
But I see a lot ERROR librespot_connect::spirc] could not dispatch connect state update: Resource has been exhausted { Response status code: 429 Too Many Requests }.

@PocketMiner82

Note that re-sorting a "manually" created queue works for some reason. Only re-sorting the tracks under Next from: ... (e.g. Liked songs) produces that weird behavior.

That was a very good hint. With that i could reproduce the bug. I adjusted some stuff in regards to that. Maybe try playing a different context and try then to reproduce the bug again. I couldn't reproduce it anymore so it might be fixed now.

I can also try a different client tomorrow just to verify whether it is just a problem with the Android app, if that helps.

That might solve your problem. But probably doesn't solve the underlying issue. So just use it as a workaround not a solution^^

@b-nobis

But I see a lot ERROR librespot_connect::spirc] could not dispatch connect state update: Resource has been exhausted { Response status code: 429 Too Many Requests }

Usually this shouldn't happen anymore. So could i ask what you request via the web-api? This might happen if you update the state of the player a lot. Like skipping many times or loading multiple contexts in a short amount of time.

But I see a lot ERROR librespot_connect::spirc] could not dispatch connect state update: Resource has been exhausted { Response status code: 429 Too Many Requests }

Usually this shouldn't happen anymore. So could i ask what you request via the web-api? This might happen if you update the state of the player a lot. Like skipping many times or loading multiple contexts in a short amount of time.

Just checked it again, and now I don‘t see them anymore. I was when I selected a specifc playlist in our ui. Then just „play“ is called via the api and the current player state is requested every second. But maybe it was because I tested some other stuff before with two boxes running. The ui also requests a lot of playlist information via the api to show it in the menus. Maybe I just hit a request limit in general. Also wanted to reduce the calls in the ui anyway. I will monitor the behaviour further, but currently it looks good.

the current player state is requested every second

Ah i see, so perhaps its a combination out of your and librespots requests. Do you use the token retrieved by the login5 auth_token method? Because thats the token that is also used internally to update the state and retrieve additional information.
Later down the line I plan to add an option to listen to the remote and local state changes (will be another pr after this is merged). So when you have access to an spirc instance, you could just listen to the changes instead of polling the state :D

@PocketMiner82

Note that re-sorting a "manually" created queue works for some reason. Only re-sorting the tracks under Next from: ... (e.g. Liked songs) produces that weird behavior.

That was a very good hint. With that i could reproduce the bug. I adjusted some stuff in regards to that. Maybe try playing a different context and try then to reproduce the bug again. I couldn't reproduce it anymore so it might be fixed now.

I can also try a different client tomorrow just to verify whether it is just a problem with the Android app, if that helps.

That might solve your problem. But probably doesn't solve the underlying issue. So just use it as a workaround not a solution^^

@photovoltex I pulled the latest changes and now tested different clients.
Android app:
I can only reproduce the re-sorting bug in my "Liked songs" playlist. All others I tested work. Resorting an artist playlist also works. I also noticed that sometimes a "No internet connection available." popup pops up only when trying to play the "Liked songs" playlist. That only happens when connected to librespot. In fact, I've never seen that popup before. I'm still unable to select a song to play from the search: I clicked on the search tab at the bottom and then just searched for e.g. "never gonna". then i selected "Never gonna give you up" that spotify showed in the results. the song does not play and said error is logged.
Web player (https://open.spotify.com):
re-sorting does not work, no matter the playlist I tried. Resorting an artist playlist or the "manual" queue works. Picking a song manually works. Autoplay also works, but only if enabled via command line tag (can't find a toggle in the web player UI to turn autoplay on or off).
Linux app:
re-sorting never works. not even the green bar that appears when you drag a song is shown. re-sorting only works if I'm not connected to librespot. Picking a song manually works. Autoplay only works if enabled via command line tag. The UI toggle is ignored. I tried to reconnect after changing the toggle and even restarted librespot.
Windows app (Microsoft store):
re-sorting only works for artist playlists and the "manual" queue. Picking a song manually works. Autoplay only works if enabled via command line tag. The UI toggle is ignored. I tried to reconnect after changing the toggle and even restarted librespot.

we all really like it, if every single client produces a different behavior, don't we? xD

the current player state is requested every second

Ah i see, so perhaps its a combination out of your and librespots requests. Do you use the token retrieved by the login5 auth_token method? Because thats the token that is also used internally to update the state and retrieve additional information.

We have our own OAuth access-token retrieval in our app and pass it to the librespot binary with the -k option. Not sure how this is then internally used in librespot.

Later down the line I plan to add an option to listen to the remote and local state changes (will be another pr after this is merged). So when you have access to an spirc instance, you could just listen to the changes instead of polling the state :D

Also thought about something like this, when I saw the open MPRIS PR. With this we could retrieve the information directly from librespot instead of the Spotify API.

So... resorting should hopefully work on all platforms with these changes. I did test it on my side and it seemed to work fine. It's a semi optimal solution in my opinion, but as close as we get without using the pathfinder endpoint. Unless someone else finds a way to retrieve the actual uid of an item.

And with that the search result we resolve, should now expect the result from the search in each client. Additionally i could cleanup a workaround for the single track and autoplay context :D
Thx again for reporting the issues^^

Also, I do believe that when playing a song from search, that it continues playing other results from the search (which can result in the same song being played multiple times, e. g. from different albums or dozens of remixes) instead of creating an autoplay based on the selected song.

I would actually say the behavior of librespot is not wrong. But yeah web-player and android seem to not actually play from the search context. Just checked what happens If we would run into autoplay for the search and it seems to fail. So I will rewrite it so that we always just play the track provided by a search. Seems easier anyways after implementing the other solution xD.
But mobile really is a special client for the dealer implementation...

I disabled logout support for now, because there is no proper logic in place to logout from a session. We could solve that afterwards, but for now it should probably stay disabled so it doesn't get in the way of merging these changes^^

very nice work @photovoltex! playing from recent searches works now. this PR also seems to fix #1205 then :D
just noticed one small bug introduced with the latest changes: if you play a song from recent searches or search, then everything looks fine, you can click the queue, shuffle and repeat buttons:

but when you skip to the next song, then the queue, shuffle and repeat buttons get grayed out and are no longer clickable but it is still possible to skip to the next or previous songs:

That's an intended behavior because you switched to the autoplay context.
We could probably argue if it makes sense to disabled the queue (don't even know exactly why that happens). But I would argue shuffle and repeat don't make much sense when you are playing an infinite context.

That's an intended behavior because you switched to the autoplay context.
Ah, I didn't knew that.

We could probably argue if it makes sense to disabled the queue (don't even know exactly why that happens). But I would argue shuffle and repeat don't make much sense when you are playing an infinite context.
I personally do think that it makes sense to not disable the queue button because you could/should be able to re-sort the autoplay queue.
The only reason I could come up with why repeat could be useful is that you would be able to repeat a single song that is in the queue.
I'd also argue that shuffle isn't needed for a (already random) context.

Also, the normal behavior of the Spotify app when not connected to librespot is to disable the repeat and shuffle buttons but leave the queue button enabled.
Now the question is: Do we want librespot to try to mimic the exact behavior of the official clients or do we want the additional feature to enable repeating of a song in the queue (if it is even possible to only enable the repeat button)? I'd argue it's better to try to mimic the exact behavior of the official clients because of consistency. What do you think @photovoltex?

I adjusted the handling yet again on that part. It might now hopefully behave similar to the other clients.
Additionally i fixed the weird behavior when going to the previous track, without there being a prev track. There is a restriction set, that technically forbids the client to do it anyways... but mobile does seem to ignore it :D

@photovoltex The behavior seems to be the same as on the official clients now - the buttons get grayed out except for the queue button. Very nice!
I can also say now that the UI autoplay switch is also ignored when using the Android app. autoplay only works when forcing via --autoplay on. log when toggling the autoplay switch off and on again in the Android app:
[2024-11-30T08:50:35Z TRACE librespot_core::mercury] mercury response <spotify:user:attributes:mutated> is handled by dealer
[2024-11-30T08:50:35Z TRACE librespot_connect::spirc] Received attribute mutation for autoplay but key was not found!
[2024-11-30T08:50:36Z TRACE librespot_core::mercury] mercury response <spotify:user:attributes:mutated> is handled by dealer
[2024-11-30T08:50:36Z TRACE librespot_connect::spirc] Received attribute mutation for autoplay but key was not found!

Also, when autoplay is turned off, I see these errors when selecting a song from search:
[2024-11-30T09:01:41Z INFO  librespot_playback::player] <We Are The People> (267354 ms) loaded
[2024-11-30T09:01:42Z ERROR librespot_connect::spirc] failed to resolve context '': Requested entity was not found { Response status code: 404 Not Found }
[2024-11-30T09:01:42Z ERROR librespot_connect::spirc] ContextError: Invalid state { context is not available. shuffle: Default }

skipping to the next song then causes these errors (i think skip should be disabled when there are no songs left in the queue):
[2024-11-30T09:04:55Z INFO  librespot_connect::spirc] Not playing next track because there are no more tracks left in queue.
[2024-11-30T09:04:55Z ERROR librespot_connect::spirc] failed to handle request: Invalid state { context is not available. shuffle: Default }

it also seems like that the official app loads the autoplay recommendations not when clicking on the search result (as librespot does) but after skipping to the next song. so if autoplay is on and you already selected a song before connecting to librespot and did not skip to the next song and then connect to librespot, you get these errors after connecting:
[2024-11-30T09:06:31Z INFO  librespot_playback::player] <We Are The Champions - Remastered 2011> (179200 ms) loaded
[2024-11-30T09:06:32Z ERROR librespot_connect::spirc] failed to resolve context '': Requested entity was not found { Response status code: 404 Not Found }
[2024-11-30T09:06:32Z ERROR librespot_connect::spirc] ContextError: Invalid state { context is not available. shuffle: Default }

skipping to the next song also does not work (because there is no queue). this error gets logged:
[2024-11-30T09:07:55Z INFO  librespot_connect::spirc] Not playing next track because there are no more tracks left in queue.
[2024-11-30T09:07:55Z ERROR librespot_connect::spirc] failed to handle request: Invalid state { context is not available. shuffle: Default }
[2024-11-30T09:07:55Z ERROR librespot_connect::spirc] failed resolving context <context_uri: , resolve_uri: Some(""), autoplay: true, update: true>: Client specified an invalid argument { Response status code: 400 Bad Request }
[2024-11-30T09:07:55Z ERROR librespot_connect::spirc] ContextError: Invalid state { context is not available. shuffle: Default }

I can also say now that the UI autoplay switch is also ignored when using the Android app. autoplay only works when forcing via --autoplay on. log when toggling the autoplay switch off and on again in the Android app:

The info is still retrieved via mecury. In the future we probably would want to switch to the ucs info retrieval. I also saw that behavior before, while testing but currently it does seem to send the data as expected.
Funnily enough, for myself autoplay sometimes is enable and sometimes not, even tho I didn't change if autoplay was enabled or not.

Also, when autoplay is turned off, I see these errors when selecting a song from search:

Ugh, that's not so good. Already got to reproduce it. Don't know why that didn't occur when i testing. Will look into it.

i think skip should be disabled when there are no songs left in the queue

Before these changes it was disabled, but it did feel a bit wrong to not be able to skip to the next track. I will adjust the logging messages accordingly.

it also seems like that the Android app loads the autoplay recommendations differently than librespot. The app loads the recommended songs after skipping to the next song. Librespot directly loads them when clicking on a search result.

Huh... I looked into it what the mobile client actually send... For the first played track from the search it sends 40 tracks all delimited by delimiter-tracks. So in total 80 tracks. Anyhow meaning we probably just need to add a delimiter and the mobile client should display our content as expected. I also tested if the mobile client without autoplay would maybe play the search-context, but no. It just stops after skipping to the next tracks.

@photovoltex now the recommendations load exactly the same as in the Android app. That means #1205 can now definitely also be closed when merging this PR :D
Unfortunately I noticed a new bug since the last update (which was not there before I updated):

When clicking on a song from recent searches and then trying to transfer playback back to the Android app it does not work. When you then select another song, it suddenly transfers the playback to the Android app. If you don't do that, you are no longer able to control librespot (e. g. pause) until you restart the Android app.
Log after selecting another song (after trying to transfer playback to the Android app):

[2024-12-01T08:23:36Z INFO  librespot_playback::player] Loading <Live Is Life (Live)> with Spotify URI <spotify:track:7fCjVnzKfFecNfJ745u3MH>
[2024-12-01T08:23:37Z INFO  librespot_playback::player] <Live Is Life (Live)> (465833 ms) loaded
[2024-12-01T08:25:49Z INFO  librespot_connect::spirc] device became inactive
[2024-12-01T08:25:49Z WARN  librespot_core::dealer] No subscriber for msg.uri: social-connect/v2/broadcast_status_update
[2024-12-01T08:25:49Z WARN  librespot_connect::spirc] failed filling up next_track during stopping: context is not available. type: Default
[2024-12-01T08:25:51Z WARN  librespot_core::dealer] No subscriber for msg.uri: social-connect/v2/broadcast_status_update

When you pause before trying to transfer playback back, it enters (I think) the same weird state but this time you cannot resume. When you then select another song, it suddenly transfers the playback to the Android app.
Log after selecting another song (after trying to transfer playback to the Android app):

[2024-12-01T08:30:19Z INFO  librespot_playback::player] Loading <We Will Rock You - Remastered 2011> with Spotify URI <spotify:track:4pbJqGIASGPr0ZpGpnWkDn>
[2024-12-01T08:30:19Z INFO  librespot_playback::player] <We Will Rock You - Remastered 2011> (122066 ms) loaded
[2024-12-01T08:30:40Z INFO  librespot_connect::spirc] device became inactive
[2024-12-01T08:30:40Z WARN  librespot_core::dealer] No subscriber for msg.uri: social-connect/v2/broadcast_status_update
[2024-12-01T08:30:40Z WARN  librespot_connect::spirc] failed filling up next_track during stopping: context is not available. type: Default
[2024-12-01T08:30:41Z WARN  librespot_core::dealer] No subscriber for msg.uri: social-connect/v2/broadcast_status_update

This seems to only happen when playing a song from recent searches, I could not reproduce it otherwhere (not even in the normal search).

Funnily, that is one of the issues I encountered myself, but ignored it with the reasoning of "it's probably just a development state that will not happen in the wild :^)"
I tried a bit around, fixed some other incorrect states and behavior but didn't find a way to mitigate the issue regarding transferring the playback to the mobile app. I would suspect that it has to do with some incorrect state. Will try to find a solution the next days.
Big thanks again for all the testing and additional issue reports^^

Awesome work here! I've been playing around with this and I've found that that there's issues playing podcasts. This was tested by playing an episode from 'The Daily' and it would just continuously skip tracks. Here's my output. Any ideas?
[2024-12-02T19:48:09Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:09Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:0QCBV9G9U7zcMpQQlQ1svF")>: ()
[2024-12-02T19:48:09Z WARN  librespot_core::dealer] No subscriber for msg.uri: social-connect/v2/broadcast_status_update
[2024-12-02T19:48:09Z INFO  librespot_connect::spirc] session update: <Ok(NEW_SESSION)> for self, current session_id 91f8087d952e4fccbb439dd437ce5cf0, new session_id 01c353cbe58934afca6af2e7dcf1a641
[2024-12-02T19:48:09Z INFO  librespot_connect::spirc] Not playing next track because there are no more tracks left in queue.
[2024-12-02T19:48:09Z ERROR librespot_connect::spirc] could not dispatch player event: Invalid state { context is not available. type: Default }
[2024-12-02T19:48:10Z ERROR librespot_connect::spirc] could not parse session_update: Invalid state { Unknown enum variant name: `WIFI_BROADCAST_CHANGED` at 1:11 }
[2024-12-02T19:48:10Z ERROR librespot_connect::spirc] failed resolving context <resolve_uri: <Some("spotify:show:3IM0lmZxpFAY7CwMuv9H4g")>, context_uri: <spotify:show:3IM0lmZxpFAY7CwMuv9H4g>, autoplay: <true>, update: <true>>: Service unavailable { tried resolving not allowed context: "spotify:show:3IM0lmZxpFAY7CwMuv9H4g" }
[2024-12-02T19:48:10Z WARN  librespot_connect::spirc] failed filling up next_track during stopping: context is not available. type: Default
[2024-12-02T19:48:11Z WARN  librespot_connect::state::context] current track was removed, setting pos to last known index
[2024-12-02T19:48:14Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:14Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:0QCBV9G9U7zcMpQQlQ1svF")>: ()
[2024-12-02T19:48:14Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:14Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:02UL6q2ZSd2GVRcEof0OxR")>: ()
[2024-12-02T19:48:15Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:15Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:3YBgeJsIBZk7GPRIaAXXvs")>: ()
[2024-12-02T19:48:15Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:15Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:3FUMELhLmENjNH4DLuXnDa")>: ()
[2024-12-02T19:48:16Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:16Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:6dRm8iM1hNMBH93XsiaMbb")>: ()
[2024-12-02T19:48:17Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:17Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:2Y0LwuZTk5pDp2A5AJZFu7")>: ()
[2024-12-02T19:48:18Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:18Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:6L0UavBmtlggPSq5WKwjp7")>: ()
[2024-12-02T19:48:19Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:19Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:5Esudj52STTftmOIP9Hlb1")>: ()
[2024-12-02T19:48:19Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unknown, error: StatusCode(502) }
[2024-12-02T19:48:19Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:episode:1XEFMcqlZOdYxNnFw6GI6A")>: ()

@YutongGu Thanks for reporting^^
Oh yeah podcasts/episodes don't work currently. I thought to have tested it against dev, but tested it again just now and it seems to work on dev... Soooo i suppose that's an open point that did overlook or tested wrong back then.
I will take it into the open points/issues

I did some playing around and I narrowed the cause to the the request function in spclient.rs. The issue was that podcasts/episodes were unable to get metadata from spotify's servers. After making the following changes, podcasts would start working.
             // `vodafone-uk` but we've yet to discover how we can find that value.
             // For the sake of documentation you could also do "product=free" but
             // we only support premium anyway.
-            if options.metrics {
-                let _ = write!(
-                    url,
-                    "{}product=0&country={}",
-                    separator,
-                    self.session().country()
-                );
-            }
+
+            let _ = write!(
+                url,
+                "{}product=0&country={}",
+                separator,
+                self.session().country()
+            );
+

             // Defeat caches. Spotify-generated URLs already contain this.
-            if options.salt && !url.contains("salt=") {
-                let _ = write!(url, "{separator}salt={}", rand::thread_rng().next_u32());
+            if !url.contains("salt=") {
+                let _ = write!(url, "&salt={}", rand::thread_rng().next_u32());
             }

             let mut request = Request::builder()

These changes undo some of the work in 85a68c8. I understand the full context for these changes but it seems like there may be a bug in the implementation.

I've said it before and I'll say it again: mega job guys. First of all towards @photovoltex but also to everyone contributing with feedback and ideas.
It feels like we've past a first shakedown period. How about we merge this to dev and get the second round of shakedown going, when more users will get to use it on the bleeding edge?

@photovoltex I'd like to contact you 1-on-1 but can't find your address. Would you send me a message?

I think that's a good idea to merge this in! It would be good to have more people try it out, leave feedback, and implement fixes of their own.
Separately, I wanted to bring attention to another issue. I'm not sure if it only affects me but when I pause the song while connected to librespot, the app on my phone will show it as paused but the seek/playback bar will keep seeking forward as if the song was still playing. It seems like the app is correctly reporting the is_paused, is_playing, and is_buffering states when you pause, perhaps something else intuitively controls whether the seek bar is continuously moving forward.

I've said it before and I'll say it again: mega job guys. First of all towards @photovoltex but also to everyone contributing with feedback and ideas.
It feels like we're past a first shakedown period. How about we merge this to dev and get the second round of shakedown going, when more users will get to use it on the bleeding edge?

@roderickvd (i messaged you and my email is now also publicly available)
If everyone is fine with the current state being merged in to dev, I would have nothing against it. Tho I would like to do a final cleanup of some clones that are not required. So that we could maybe merge it the next days?

So, did a final refactoring of a section I wasn't so happy with, might be worse, might be better, will probably see with time. The previous version did try to reduce cpu usage and with refactoring it should now use less memory, but might use the cpu more when for example, skipping a lot of tracks in the queue.
I additionally improved the context handling again. And for myself, the issue @PocketMiner82 described around the transferring of the recent search seems to be resolved by that :D
That will probably be the last changes for now, unless I find some major points. But the initial impl is probably now in a complete state^^

I additionally improved the context handling again. > And for myself, the issue @PocketMiner82 described around the transferring of the recent search seems to be resolved by that :D

@photovoltex yes, it is indeed working now :)

Awesome so let's get it in dev. Would you consider squashing it?

Hmmm... I think I found a new issue, happened to me 3 times today (but for some reason never before): I was listening to the liked songs playlists for some time and suddenly the music stops playing in the middle of a song. the following errors got logged:
Dec 06 10:39:52 rpi-3a librespot[1607]: [2024-12-06T09:39:52Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:00 rpi-3a librespot[1607]: [2024-12-06T09:40:00Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:08 rpi-3a librespot[1607]: [2024-12-06T09:40:08Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:16 rpi-3a librespot[1607]: [2024-12-06T09:40:16Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:24 rpi-3a librespot[1607]: [2024-12-06T09:40:24Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:32 rpi-3a librespot[1607]: [2024-12-06T09:40:32Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:40 rpi-3a librespot[1607]: [2024-12-06T09:40:40Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:48 rpi-3a librespot[1607]: [2024-12-06T09:40:48Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:56 rpi-3a librespot[1607]: [2024-12-06T09:40:56Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:41:04 rpi-3a librespot[1607]: [2024-12-06T09:41:04Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")

Note that the wifi of this device (Raspberry Pi 3 A+) is a bit unstable for me, sometimes it doesn't respond for more than 20 seconds... Maybe that's why the wait timeout exceeded. I thought I'll report it regardless, maybe some timeout can be increased 🤷

Awesome so let's get it in dev. Would you consider squashing it?

Sure thing. But did anyone look over the code or do we just merge it? I could understand why, the changes are a bit too drastic/large to review. But just wanted to hear what the stance in regards to that is^^

In an ideal world I'd like to run through the code, but I don't have a lot of time for it in the coming weeks. So I thought that it's better to get it in and we can still place code comments on GitHub after the merge.

Sounds reasonable. Feel free to merge whenever fits best :)

Hmmm... I think I found a new issue, happened to me 3 times today (but for some reason never before): I was listening to the liked songs playlists for some time and suddenly the music stops playing in the middle of a song. the following errors got logged:
Dec 06 10:39:52 rpi-3a librespot[1607]: [2024-12-06T09:39:52Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:00 rpi-3a librespot[1607]: [2024-12-06T09:40:00Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:08 rpi-3a librespot[1607]: [2024-12-06T09:40:08Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:16 rpi-3a librespot[1607]: [2024-12-06T09:40:16Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:24 rpi-3a librespot[1607]: [2024-12-06T09:40:24Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:32 rpi-3a librespot[1607]: [2024-12-06T09:40:32Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:40 rpi-3a librespot[1607]: [2024-12-06T09:40:40Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:48 rpi-3a librespot[1607]: [2024-12-06T09:40:48Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:40:56 rpi-3a librespot[1607]: [2024-12-06T09:40:56Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")
Dec 06 10:41:04 rpi-3a librespot[1607]: [2024-12-06T09:41:04Z ERROR librespot_playback::player] Skipping to next track, unable to get next packet for track <SpotifyId("spotify:track:4GewRLEiI4BDTQsoH8BRVV")>: SymphoniaDecoder("Deadline expired before operation could complete { wait timeout exceeded }")

Note that the wifi of this device (Raspberry Pi 3 A+) is a bit unstable for me, sometimes it doesn't respond for ~15 seconds... Maybe that's why the wait timeout exceeded. I thought I'll report it regardless, maybe some timeout can be increased 🤷

That should probably be unrelated to these changes. But if they appear again we can open an issue then^^

Hmmm... I think I found a new issue, happened to me 3 times today (but for some reason never before): I was listening to the liked songs playlists for some time and suddenly the music stops playing in the middle of a song. the following errors got logged...

I can report that this issue occurred for me as well:
[2024-12-06T21:34:29Z INFO  librespot_playback::player] Loading <Guy For That (Feat. Luke Combs)> with Spotify URI <spotify:track:6StwwqB84sJeLr7tZDTxEX>
[2024-12-06T21:34:32Z ERROR librespot_core::audio_key] Audio key response timeout
Rate limited 1 logs
[2024-12-06T21:34:37Z ERROR librespot_core::session] Connection to server closed.
[2024-12-06T21:35:01Z WARN  librespot_core::dealer] Websocket peer does not respond.
[2024-12-06T21:35:06Z ERROR librespot_playback::player] Unable to read audio file: Symphonia Decoder Error: Deadline expired before operation could complete { wait timeout exceeded }
[2024-12-06T21:35:08Z WARN  librespot] Spirc shut down unexpectedly
[2024-12-06T21:35:08Z ERROR librespot_core::session] Transport endpoint is not connected (os error 107)
[2024-12-06T21:35:18Z INFO  librespot_core::session] Connecting to AP "ap-guc3.spotify.com:4070"
[2024-12-06T21:35:23Z INFO  librespot_core::session] Authenticated as '1219828745' !
Rate limited 1 logs
[2024-12-06T21:35:31Z INFO  librespot_core::spclient] Resolved "guc3-spclient.spotify.com:443" as spclient access point
[2024-12-06T21:35:39Z INFO  librespot_connect::spirc] active device is <> with session <0d5cfc65538b301a627f3c9d7dce54b2>
[2024-12-06T21:38:11Z WARN  librespot_connect::state::context] switching to active context had issues: Invalid state { context is not available. type: Default }
---- 2024-12-06 21:38:14.953302: Now starting Spotify ----

Hui that's a lot of errors and warning in one spot. Seems all be related to sort of timeout or unresponsive network connection. Can both of you maybe try to reproduce these issues on the dev branch?
I will look into it if there might be some changes that unintentionally affect the player. But from the errors in the log it sounds more like network issues.

I did also find a weird behavior from the context-resolver... if you request a playlist that contains tracks and episodes, only the tracks will be resolved. The response seems to not contain any info about the episodes... I will probably not solve it anymore in context of this PR, so might be moved out into a new issue afterwards.

There is also an oversight with the current impl in relation to shuffle and a context with multiple pages. For example playing an artist with shuffle doesn't work yet as one would expect. That probably needs a revisit at some point in the future. Adjusting the current impl would probably add another 100+ lines to the current PR, which i would like to prevent for now^^

Looked again at this comment (https://github.com/librespot-org/librespot/pull/1356#issuecomment-2427617758), and it might be a better solution in the end, when we play a shuffled context we usually don't need the default context. I had hopped we could just keep refs to the original tracks in a vec, but that collides with every state modification you still want to do.
That might be a nice next task to do after this initial state is merged and we iterate on improving performance and stability :D

Continuing development here (https://github.com/photovoltex/librespot/tree/improve-dealer) so that this PR is only the basic integration (with some new features provided by the dealer)

Drumroll for merging, coming up in 3... 2....

Well, and now we wait for issues that we didn't spotted during testing/development


issue with height and width

Hey,
As you know about my application, what i am trying to do now is that if the height of the image is larger than its width then I am rotating the image by 90 Degree and then applying cropbox on it. But the thing is the image isn't contained the parent div. I am attaching the link and I am also attaching the image you should upload to see the problem I am talking about.
Hope to get an insight on it.
Regards,
Mohsin

pprdev.onjumpstarter.io/resapp/#/selectImage

This is out of scope for this plugin.


⚠ Bofrost.fr has degraded performance

In 7d14eb7, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1586 ms

Resolved: Bofrost.fr performance has improved in 704e94b after 6 minutes.


VueJS 3, Wavesurfer.js + microphone

Hello!
I tried to create an app based on VueJS3 and Wavesurfer.js that would visualize microphone input. Unfortunately, microphone plugin does not work.
I can load audio using this.waveRecorder.load('https://wavesurfer-js.org/example/media/demo.wav'), and in this case, wavesurfer will work. But, if I use microphone plugin, I can't see visualization of microphone input.
I have codepen.
And there is another thing.
If I load audio using this.waveRecorder.load('https://wavesurfer-js.org/example/media/demo.wav'), and change stop_recording method to
stop_recording() {
      this.isRecording = false;
      this.waveRecorder.playPause();
}

microphone plugin will work, after the second attempt to turn on (The loaded file will reset). And this also will reduce canvas refresh time.

There is no stopRecording method in wavesurfer. First try reproducing the issue without vue.js.

There is no stopRecording method in wavesurfer. First try reproducing the issue without vue.js.

It was my method of onClick event of VueJS.
Ok, I reproduced the issue [without vuejs] (https://codepen.io/markitantov/pen/WNpXmLP).
The microphone plugin doesn't work anyway.
And as I said if I will load file (waveRecorder.load('https://wavesurfer-js.org/example/media/demo.wav');) after initialization, and in this pen change waveRecorder.microphone.stop(); to waveRecorder.playPause();, the microphone plugin will work after the second attempt to start (of course, after stop method).

The microphone plugin doesn't work anyway.

That's a bold statement, it works fine for me.

change waveRecorder.microphone.stop();

I have no idea what waveRecorder is or looks like.

The microphone plugin doesn't work anyway.

That's a bold statement, it works fine for me.

change waveRecorder.microphone.stop();

I have no idea what waveRecorder is or looks like.

I uploaded code to codepen in the post upper. If you don't see this, I will attach code here.
var MicrophonePlugin = window.WaveSurfer.microphone;

var startBtn = document.querySelector('.start');
var stopBtn = document.querySelector('.stop');

var waveRecorder = WaveSurfer.create({
  container: '#waveform',
  waveColor: '#000000',
  barWidth: 2,
  barHeight: 1,
  barGap: null,
  plugins: [MicrophonePlugin.create()]
});

startBtn.addEventListener("click", function() {
  waveRecorder.microphone.start();
});

stopBtn.addEventListener("click", function() {
  waveRecorder.microphone.stop();
});

I use this code and I don't know, why microphone plugin doesn't work.

I figured out, microphone plugin works well, but here is the question.
Why is it initialized only in the onclick method?
I used code from documentation (https://wavesurfer-js.org/plugins/microphone.js) and it works well,
but when I change it to this one (intialized wavesurfer on the DOMContentLoaded), the microphone plugin doesn't work:(
Here the not working code:
var wavesurfer;

// Init & load
document.addEventListener('DOMContentLoaded', function() {
    var startBtn = document.querySelector('.start');

    wavesurfer = WaveSurfer.create({
      container: '#waveform',
      waveColor: 'black',
      interact: false,
      cursorWidth: 0,
      plugins: [WaveSurfer.microphone.create()]
    });

    startBtn.addEventListener("click", function() {
      // start/stop mic on button click
      if (wavesurfer.microphone.active) {
        wavesurfer.microphone.stop();
      } else {
        wavesurfer.microphone.start();
      }
    });
});

Why is it initialized only in the onclick method?

It's an example. Use whatever works for you?

It works in the example, when wavesurfer initialized in onclick method.
But it does not work, when I want to initialize wavesurfer and microphone plugin on page load method.

Perhaps a browser restriction that requires user input before accessing the microphone is cascading down into the way the plugin captures the audio, and thus preventing it from capturing unless the code executes in the context of a click.


core: prefer flag value over env value

resolves https://github.com/textileio/go-textile/pull/726#discussion_r280688442

We should document this change in a changelog, as users who where using both, will have different behaviour once this merges.

@carsonfarmer this needs to be reviewed again, due to the conflict resolution


Feature Request: add camera input for new GPT4-vision model via API

Hi @AndraxDev !
Maybe you've heard but OpenAI now added Vision support via API
The API is referenced here.
It would be great if SpeakGPT supported vision in a similar way as you can send an MMS to someone in the original messaging apps etc. Either from the library or from the camera app.
Cheers!

Released in SpeakGPT 3.13


[FLINK-34712][release] Generate reference data for state migration tests based on release-1.19.0

The data was generated based on release-1.19.0.

@masteryhx Thanks for looking into it and the tips! I'll try regrenerating data.

@masteryhx I've tried manually regenerate the specified test mvn clean package -Pgenerate-migration-test-data -Dgenerate.classes=StatefulJobSnapshotMigrationITCase -Dgenerate.version=1.19 -nsu -Dfast -DskipTests
a strange log says the test class does not exist:
main:
     [java] 743  [main] INFO  org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Start test data generating for module /Users/lilin/work/git/flink/flink-tests and version 1.19
     [java] 749  [main] WARN  org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Class StatefulJobSnapshotMigrationITCase does not exist.
[INFO] Executed tasks
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:42 min
[INFO] Finished at: 2024-03-20T23:35:58+08:00

Do you know what's wrong here? (If you know this, I haven't had time to dive into it yet)

The generate.classes should be configured as its complete path, e.g. org.apache.flink.test.checkpointing.StatefulJobSnapshotMigrationITCase
I just also tried to generate it but seems no result. I took a quick look and haven't found the reason. Maybe there are some issues for the generator logic. You could also debug this.

@masteryhx thank you!
Another input:  there was no snapshot file generated for this case related data directory in previous releases, so I'm a bit skeptical that it's due to other changes.

@snuyanzin sorry to ping you here, do you still remember this data generation during 1.18 release ( https://github.com/apache/flink/pull/23710)?
Is there any exception when do generate-migration-test-data via maven?
I debugged generating org.apache.flink.test.checkpointing.StatefulJobSnapshotMigrationITCase locally via MigrationTestsSnapshotGenerator with following args:
--dir
/Users/lilin/work/git/flink/flink-tests
--version
1.19
--classes
org.apache.flink.test.checkpointing.StatefulJobSnapshotMigrationITCase

but will fail to load the specified class...
Before diving into each line of the generator code, try asking if you have encountered similar issues before, thanks

After some debugging, seems MigrationTestsSnapshotGenerator cannot work for the four failure tests ( https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58514&view=logs&j=5c8e7682-d68f-54d1-16a2-a09310218a49&t=86f654fa-ab48-5c1a-25f4-7e7f6afb9bba&l=9218 ) even it logged generating succeed:
main:
     [java] 1296 [main] INFO  org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Start test data generating for module /Users/lilin/work/git/flink/flink-tests and version 1.19
     [java] 1345 [main] INFO  org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Start test data generating for org.apache.flink.test.checkpointing.StatefulJobWBroadcastStateMigrationITCase
     [java] 1867 [main] INFO  org.apache.flink.test.migration.MigrationTestsSnapshotGenerator [] - Finish test data generating for org.apache.flink.test.checkpointing.StatefulJobWBroadcastStateMigrationITCase

So manually generate the corresponding savepoint/snapshot files by modifying the 4 cases that failed to generate automatically, let's complete this updating work first. (The problem on MigrationTestsSnapshotGenerator tracked by https://issues.apache.org/jira/browse/FLINK-34972)

@masteryhx Could you take a look again when you have time?

@masteryhx Thanks for reviewing this!

Rebased latest 1.19 and triggered a new ci run before merging...


Remove mainline patches (April 14, 2022)

This has been merged into mainline:
https://git.kernel.org/linus/6d4a6b515c39f1f8763093e0f828959b2fbc2f45

Presubmit should pass now: https://github.com/ClangBuiltLinux/continuous-integration2/pull/354/commits/e095eac0e5d5d52e6a94cd8ce382ae6820155503
Guess check-patches.sh has never been run when there are zero patch files in the repository?


Implement stringutils.Ellipsis()

Append an ellipsis (...)  when truncating strings in output; came up in https://github.com/docker/docker/pull/25556#discussion_r74174528, so I thought adding it in some places may be nice and consistent :smile:

@thaJeztah 16:55:30 pkg/stringutils/stringutils.go:35:1: comment on exported function Ellipsis should be of the form "Ellipsis ..." :angel:

Other than that, LGTM 👼

oh! hah thought I'd do a quick change :smile: let me fix

genuine failure, I'll look into that as well;
15:10:27 --- FAIL: TestContainerPsContext (0.00s)
15:10:27 	container_test.go:94: Expected 733908409c91817, was 733908409c91...
15:10:27 FAIL

ping @justincormack @vdemeester I updated my PR to make these utilities work with multibyte characters, and fixed the TestContainerPsContext test

LGTM

LGTM 👼


Bug fixes

Summary
MM-8435 - LHS headings fix
MM-9697 - Displaying help text on tablets
Ticket Link
https://mattermost.atlassian.net/browse/MM-8435
https://mattermost.atlassian.net/browse/MM-9697
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] Ran make check-style to check for style errors (required for all pull requests)
[ ] Ran make test to ensure unit and component tests passed
[ ] Added or updated unit tests (required for all new features)
[ ] Has server changes (please link)
[ ] Has redux changes (please link)
[x] Has UI changes
[ ] Includes text changes and localization file (.../i18n/en.json) updates
[ ] Touches critical sections of the codebase (auth, posting, etc.)

Line followed by a bullet

Bullet, followed by an empty line

Line


Run cargo-sort on the whole workspace

@robert3005 and I talked, trying to automate some minor chores

@a10y is there a way to test that I  didn't break any of the new release-related changes?


Add the KataMshvVmIsolation option to --workload-runtime

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
The precondition is to put your code inside this repository and upgrade the version in the pull request but do not modify src/index.json.

AKS


Too Many Requests

I use Paperless Document API.
When I run it in production, it reports an error.
{   "response": {     "errors": [       {         "code": "10429",         "message": "Too Many Requests"       }     ]   } }
The API documentation does not mention the need to limit the request frequency.How should I solve this problem?

I guess it is duplicate for https://github.com/UPS-API/api-documentation/issues/8

Hi, thank you for your comment. When you are receiving this response, how many requests were made? Further, similarly to what nightcoding2021 mentioned, are you requesting a new access token per call?


what's version for Microsoft Excel can run CPU.xlsx?

After compiled bouncingBall.s , I use WPS or office2016 Microsoft Excel open CPU.xlsx and change "read ROM" , and use F9 key,  but CPU.xlsx can not change Secreen area.   what's wrong?

at least excel 2019,  which support switch function

at least excel 2019, which support switch function

it's useful, thank you very much! brother~


Add documention for Capybara::Node::Actions#attach_file options hash

This PR adds some missing documentation for the attach_file method, which takes an options hash but didn't previously document what parameters it accepted. Please let me know if the descriptions make sense.
This is a fix for #1659

I've added a couple of comments to clarify things --  Also when adding documentation only PRs - if you add [ci skip] to end of your commit message it will prevent travis from wasting time doing builds

@twalpole, thanks for the feedback (and the tip about [ci skip]). I've updated the PR based on your feedback.
One thing I wasn't entirely sure about was what the default value for multiple is. From what I understand about how the Selector is loaded, it appears to be true, but let me know if I've got this wrong.

@alextaylor000 From https://github.com/jnicklas/capybara/blob/master/lib/capybara/selector.rb#L287 you can see the multiple option doesn't have a default, so it's ignored unless passed in.  A default option would be passed in if there was a default value (like the disabled filter above) --  boolean: true is just a syntactic sugar instead of passing valid_values: [true,false] to the filter

Thanks for the notes, I think I was looking too far into file to figure out those default values. Updated the PR.

Thanks!


doc: kconfig: Explain the ' ' shorthand

The board porting guide assumed that people were already familiar with
the Kconfig 'bool "foo"' shorthand for giving a symbol's type and prompt
at the same time. I got an email pointing out that it isn't obvious.
Explain the shorthand, and also mention that it's the preferred style in
Zephyr.

CC @dbkinder

Codecov Report

Merging #10293 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #10293   +/-   ##
=======================================
  Coverage   53.05%   53.05%
=======================================
  Files         214      214
  Lines       26201    26201
  Branches     5645     5645
=======================================
  Hits        13901    13901
  Misses      10028    10028
  Partials     2272     2272

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7bf9de...52685c6. Read the comment docs.


Missing sdist tarball on pypi

I'm maintaining oauthlib for Void Linux and it would be great if you could publish a sdist tarball for the 1.0.2 release.

1.0.2 was released today. Travis has some problems with deploying to PyPi at the moment so I released manually.
Sorry about that.


Special Characters in Field Values

Trying to allow user to filter on some fields, and one field's values contains an apostrophe and ampersand. How do we allow for the values to contain special characters?
PreFilterLink =
var base = "https://app.powerbi.com/groups/me/"
var appID = ""
var reportid = ""
var ctid = ""
var ReportSection = "ReportSection__"
var __ResponsibleGroup = if(ISFILTERED(QualifiedEventList[ResponsibleGroup])
," and QualifiedEventList/ResponsibleGroup in ('" & CONCATENATEX( filters(QualifiedEventList[ResponsibleGroup]),QualifiedEventList[ResponsibleGroup],"','") &"')","")
var __LegacyArea = if(ISFILTERED(ServiceCenterHierarchy[Legacy Area])
," and ServiceCenterHierarchy/Legacy_x0020_Area in ('" & CONCATENATEX( filters(ServiceCenterHierarchy[Legacy Area]),ServiceCenterHierarchy[Legacy Area],"','") &"')","")
...
var __Normalization = if(ISFILTERED(QualifiedEventList[Normalization])
," and QualifiedEventList/Normalization in ('" & CONCATENATEX( filters(QualifiedEventList[Normalization]),QualifiedEventList[Normalization],"','") &"')","")
var __filters = __ResponsibleGroup & __LegacyArea ... & __Normalization
var __filtersR = if( len(__filters) < 5
,""
,right(__filters,len(__filters)-5)
)
return
if(__appID = ""
,__base & "reports/" & __reportid & "/" & __ReportSection & "?filter=" & __filtersR
,__base & "apps/" & __appID & "/reports/" & __reportid & "/" & __ReportSection & "?filter=" & __filtersR & "&ctid="&__ctid
)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 70b360f3-21a7-adc2-896a-b1dfd5d9e384
Version Independent ID: 15e8c09f-46ad-68f9-75e6-c3d5c2c214e5
Content: Filter a report using query string parameters in the URL - Power BI
Content Source: powerbi-docs/service-url-filters.md
Service: powerbi
Sub-service: powerbi-service
GitHub Login: @maggiesMSFT
Microsoft Alias: maggies

Thanks for your comment, @soldstatic This looks like product feedback or a product question, rather than documentation feedback. You can submit product feedback or feature ideas by going to Power BI Ideas. You can also engage with the Power BI engineering team by being active in the Power BI Community.

Hi, I disagree. I believe this is a part of the filtering functionality that simply needs to be documented. I'm sure there's an easy way to escape the characters, I'm just not sure what that way is. Thanks though!

Thanks for your comment, @soldstatic  I've assigned this issue to the doc author for review.

Hi, @soldstatic -- have you tried using the Unicode for your special characters? Here's the place in the doc that describes how to use Unicode:
https://docs.microsoft.com/en-us/power-bi/service-url-filters#special-characters-in-url-filters
In this Unicode table, https://unicode-table.com/en/#control-character, looks like
& = 0026
and a straight apostrophe:
' = 0027
Does that answer your question?

Hey y'all! Thanks so much for your help. I've tried that with the x?, but it appears to just treat it as part of the string when it's in the variable (which is a string).
For instance, trying to pass the following:
?filter=ServiceCenterHierarchy/Service_x0020_Center in ('F&M','Lee's Summit')&ctid=__
Among many others, I've tried this unicode slant:
?filter=ServiceCenterHierarchy/Service_x0020_Center in ('F_x0026_M','Lee_x0027_s Summit')&ctid=__

Ah, it's in the value, not in the table or field name. Hmmm. I'll have to get back to you on that.

Hi all,
just a quick solution that I found.
In the value, for '&', you can actually use %26 instead of x0026
I'm not sure whether this issue was fixed minutes ago or what.. as I was trying to put # in the value

@soldstatic
as for your example,
?filter=ServiceCenterHierarchy/Service_x0020_Center in ('F&M','Lee's Summit')&ctid=__
you can write like this,
?filter=ServiceCenterHierarchy/Service_x0020_Center in ('F%26M','Lee%27s Summit')&ctid=__
Let me know if that solves your problem.

Hey that works great for the &! But it doesn't work for the '. I tried to escape the ' by doing '' as well, but still no dice.
After the page loads this is what's in the URL bar (so it's automatically converting all ' to %27), but I need to escape it somehow.
?filter=ServiceCenterHierarchy~2FService_x0020_Center%20in%20(%27F%26M%27,%27Lee%27s%20Summit%27)&ctid=__

I did some try and error but still unable to escape the apostrophe.
Since I was able to use %22 (double quotes) within the value parameter, I guess the problem might be the two single quotes surrounding the value.
My suggestion would be update on this by using two double quotes instead of single quotes. Although it may restrict to escape the double quotes but come to think of it, how common is it to have values with double quotes? But in the end, it would be best if every symbols are able to be escaped.
I have another solution for this but it may provide quite an impact to the query efficiency. So let's just forget it.
p.s. I might be talking big since I'm new to web development and am of course have no idea on how PowerBI datasets work in the backend.

I tried the below too but it doesn't work:
?filter=ServiceCenterHierarchy/Service_x0020_Center in ("Lee's Summit")&ctid=____

Anyone able to get this to work?... all of my sales people named o'grady and o'conner are getting screwed here..
The irish mafia is coming after me.. help!
I can confirm soldstatics experience
double quotes DONT work.. ever,
and that replacing the ' with an %27 or _x0027 doesn't either, because when the link actually goes...it does replace..but not escape the special characters... you just end up with a slighlty less legible link that also doesn%27t work.
Slaint!

Anyone able to get this to work?... all of my sales people named o'grady and o'conner are getting screwed here..
The irish mafia is coming after me.. help!
I can confirm soldstatics experience
double quotes DONT work.. ever,
and that replacing the ' with an %27 or _x0027 doesn't either, because when the link actually goes...it does replace..but not escape the special characters... you just end up with a slighlty less legible link that also doesn%27t work.
Slaint!

No @gebberry, I couldn't get it to work. Sounds like you better pay up for your protection!
@maggiesMSFT any luck looking into this?

Hi, @gebberry and @soldstatic --happy to report a fix is coming soon, around mid-March. You'll be able to use 2 single quotes in the filter string to return 1 single quote in the value. So tell the Irish mafia to back off!

Hi, @maggiesMSFT, is this fix coming soon?
Thanks.

Any update on this?

@dschlabach @soldstatic @gebberry @vincentys99 we have a fix checked in that allows escaping single quotes in OData values. We follow the OData standard which escapes a single quote by using 2 consecutive single quotes.
For example:

?filter=Table/Name eq 'O''Brien'
Becomes:

?filter=Table/Name eq 'Lee''s Summit'
Becomes:

And of course the "in" operator supports this escaping as well:

?filter=Table/Name in ('Lee''s Summit', 'O''Brien')
Becomes:

Notes:

This is for values only. For identifiers we use the x0000 escape format described in the documentation: https://docs.microsoft.com/en-us/power-bi/service-url-filters#special-characters-in-url-filters
Other special characters are already supported in values. Now single quotes are supported by using ''

Thanks to all of you, @dschlabach @soldstatic @gebberry @vincentys99, and thanks to @Mdevlin4 for the fix! I've updated the article. Update should be live Monday May 4, 2020.


chore: split up make clean target

Description
Splitting up some of the make targets for cleaning up artifacts, cache, etc. This will help when the intent is only delete build artifacts or models, without also hitting .env or ruff_cache.
BREAKING CHANGES
None
CHANGES

Replaced make clean with an include for mk-clean.mk
Split make clean functionality into clean-artifacts, clean-cache, clean-env, clean-models, and clean-logs
Added make clean-all to replicate previous functionality.

Related Issue
None, just an problem spotted in the course of rebuilding.
Checklist before merging

[x] Tests, documentation, ADR added or updated as needed
[x] Followed the Contributor Guide Steps

One nitpick that technically doesn't need to be here - can we include a UDS clean too?
This works on Debian based distros, but the temp files may be located elsewhere in Mac and Windows. The temp files do take up a TON of space if your OS clean-up job isn't often or thorough.
uds zarf tools clear-cache && rm -rf ~/.uds-cache && rm -rf /tmp/zarf-*

@justinthelaw I don't disagree that this is a valuable alias. My only concern with adding it to this PR is that it causes our make clean to exceed the scope of LeapfrogAI. By limiting our cleanup to files within this directory we avoid that. We have uds-clean documented as an alias in DEVELOPMENT.md, do you think that is sufficient to satisfy your need?


oauth2: cannot fetch token error preventing get method to succeed

Error: Action Failed get_task: Task zzz result: Applying: Applying job consul_agent: Applying package adapter for job consul_agent: Fetching package blob: Getting blob from inner blobstore: Getting blob from inner blobstore: Shelling out to bosh-blobstore-gcs cli: Running command: 'bosh-blobstore-gcs -c /var/vcap/bosh/etc/blobstore-gcs.json get xxx yyy', stdout: '', stderr: '2018/03/02 18:42:14 Get https://www.googleapis.com/storage/v1/b/pcflabs-boshblobs?alt=json&projection=full: oauth2: cannot fetch token: Post https://accounts.google.com/o/oauth2/token: dial tcp <IP_ADDRESS>:443: i/o timeout
': exit status 1

added this issue for now for tracking. i believe agent currently does some form of retrying, though i dont see in the error message.
similar issue in cpi: https://github.com/cloudfoundry-incubator/bosh-google-cpi-release/issues/203

I don't believe bosh-agent does retries... https://github.com/cloudfoundry/bosh-utils/blob/01cfe283d17a7b5aa18f03c1a1d8eb78cf25e249/blobstore/external_blobstore.go#L51
I don't believe the gcscli retries gets either... https://github.com/cloudfoundry/bosh-gcscli/blob/2aef468b5b8561560918014b4fbce7549de2283d/client/client.go#L93

Any update on this? I seem to be having the same issue after moving from using S3 compatible to GCS.

/cc @evandbrown

Looking into this:
The HTTP/GRPC stuff that deals with  should be fine because the google-cloud-go library is supposed to handle retries as long as the context.Context is valid and it includes code for exponential backoff. See: https://cloud.google.com/storage/docs/exponential-backoff for more info.
There is a difference between this and https://github.com/cloudfoundry/bosh-google-cpi-release/issues/203 because that application is using the deprecated Google generated API: https://godoc.org/google.golang.org/api/storage/v1 whereas this one is using the current: https://godoc.org/cloud.google.com/go/storage, however behind the scenes it looks like they both boil down to using the same code to construct new clients.
For the modern lib it goes through here: https://github.com/GoogleCloudPlatform/google-cloud-go/blob/master/storage/storage.go#L88 to get to here: https://github.com/googleapis/google-api-go-client/blob/master/transport/http/dial.go#L34:6
The problem looks like that code will always call this function which sets up the credential fetching OAUTH with go's default HTTP transport, meaning we have to modify the default to include a retrying roundtripper and/or a longer timeotu (unless I'm missing something).
I'd recommend basing the default roundtripper off this code so we can retry indefinitely, or to the end of a context based on wall-clock time that a user would find acceptable rather than an arbitrary number of times. Without a full stack-trace it's difficult to determine if a dial timeout error is a net error that has the retryable flag set, but based on some of the code I found it looks like it might be so we could try using that code out of the box.

@evandbrown, I added a proposed fix, it's based on what I saw when digging down the API client rabbit-hole. Not super clean, and I'm looking for ideas about how to test it correctly without dedicating a significant chunk of work towards this single issue.

Is this related then?
Do I have to wait for a fix or can I do anything to prevent it from running into the timeout?
Task 34 | 10:31:59 | Update stemcell: Uploading stemcell bosh-google-kvm-ubuntu-xenial-go_agent/97.28 to the cloud (cpi: 437bd5f14b2c13913774) (00:00:30)
               L Error: CPI error 'Bosh::Clouds::CloudError' with message 'Creating stemcell: Creating Google Image from URL: Failed to create Google Image: Post https://www.googleapis.com/compute/v1/projects/pcf-test-1/global/images?alt=json: oauth2: cannot fetch token: Post https://accounts.google.com/o/oauth2/token: dial tcp <IP_ADDRESS>:443: i/o timeout' in 'create_stemcell' CPI method (CPI request ID: 'cpi-729640')

Task 34 | 10:32:29 | Error: CPI error 'Bosh::Clouds::CloudError' with message 'Creating stemcell: Creating Google Image from URL: Failed to create Google Image: Post https://www.googleapis.com/compute/v1/projects/pcf-test-1/global/images?alt=json: oauth2: cannot fetch token: Post https://accounts.google.com/o/oauth2/token: dial tcp <IP_ADDRESS>:443: i/o timeout' in 'create_stemcell' CPI method (CPI request ID: 'cpi-729640')


CUPS prints error messages when more than 1 page

Hi everybody from Paris,
I have CUPS 1.7.5 installed on my RPi B+, with a 8Gb SD Cad on it, running Jessie Raspbian (january 2017), connected through USB to a Laser Postscript Lexmark printer (352dn), and through wifi to my network, using the AirPrint (Avahi) feature on my Mac.
When I print small text or pdf docs, printing only 1 page at a time, everything is fine. I generally have to resize printing to 90% of the actual size. If I do otherwise, I get a blank page with hermetical code in it, like :
7oM*o%:D37/I$S.]\mgscphMRIG413                                                              (Z=Q;")5F8)Qr"=NfCOoQ)ibuDCCLE1_#S2=*J!5rCj3Lr
So I tend to think that it's either a SD card size issue, or a memory issue (the latter being IMHO the most plausible explanation).
The problem is that when I print directly from my Mac, without RPi and CUPS, no problem : therefore the printer itself does have enough memory to print a poor 2 pages PDF bill...
I tried 3 different PPD files already : the official Lexmark for Linux 4, the one I got already installed on the Mac, and the Gutenprint one... The generic Postscript one works faster, but otherwise, nothing changes.
Does anyone has had the same problem, or have an idea ?
Thanks a lot everybody
i-Moi

Not sure what is going wrong here; you might try using the network interface on the printer to eliminate the USB port as the problem...


Add rmul for ECKey and ECPubkey

Fixes #105

Thanks. I've pushed a couple of fixes and nits.

lol sorry. I'm way too used to rust's return style


ValueError while executing act_and_train in TRPO

Traceback (most recent call last):
File "TRPO.py", line 169, in
action = agent.act_and_train(obs, reward)
File "C:\Anaconda3\envs\osim-rl2\lib\site-packages\chainerrl\agents\trpo.py", line 680, in act_and_train
b_state = self.obs_normalizer(b_state, update=False)
File "C:\Anaconda3\envs\osim-rl2\lib\site-packages\chainerrl\links\empirical_normalization.py", line 93, in call
mean = xp.broadcast_to(self._mean, x.shape)
File "<array_function internals>", line 6, in broadcast_to
File "C:\Anaconda3\envs\osim-rl2\lib\site-packages\numpy\lib\stride_tricks.py", line 180, in broadcast_to
return _broadcast_to(array, shape, subok=subok, readonly=True)
File "C:\Anaconda3\envs\osim-rl2\lib\site-packages\numpy\lib\stride_tricks.py", line 125, in _broadcast_to
op_flags=['readonly'], itershape=shape, order='C')
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (1,158) and requested shape (1,160)
please help.. any hints are welcome..
length of obs returned me 160, so i assume req shape is (1,160)
I see that
obs_space = env.observation_space
returns a space size of (158)

any hints, how can i fix this while google colab for this code.


set up purl.org domain for cg elements

I started redirecting from purl.org/cg/terms to this repo.
Need to do it for all the cg specific elements. First, we need to move on issue #31
exemple of working purl: cg.hasMetadata

Oh that's cool! I didn't know you could do that with purl.org.
I can make a pull request with changes required for #31 if you like.

Oh that's cool! I didn't know you could do that with purl.org.
I can make a pull request with changes required for #31 if you like.

@alanorth That would be great, thanks

@alanorth That would be great, thanks

FYI @ayabowork

FYI @ayabowork

All the PURLs should be working now.
@alanorth yCoord, xCoord and code don't link to the correct sections. I cannot find why. Could you please take a look into that issue. Thanks

All the PURLs should be working now.
@alanorth yCoord, xCoord and code don't link to the correct sections. I cannot find why. Could you please take a look into that issue. Thanks

@alanorth yCoord, xCoord and code don't link to the correct sections. I cannot find why. Could you please take a look into that issue. Thanks

Sure I can fix that. The issue is that you should only use "#" in the link, not the target id itself. I will prepare a pull request to clean these up, as I see they are also duplicated:

We only need the first one (without the "#" of course).

@alanorth yCoord, xCoord and code don't link to the correct sections. I cannot find why. Could you please take a look into that issue. Thanks

Sure I can fix that. The issue is that you should only use "#" in the link, not the target id itself. I will prepare a pull request to clean these up, as I see they are also duplicated:

We only need the first one (without the "#" of course).

I will fix the Dublin Core links as well

I will fix the Dublin Core links as well


[Bug]: Regular towers not spawning

Is there an existing issue for this?

[X] I have searched the existing issues

Description
Regular towers do not spawn.
To Reproduce

Launch modpack with towers of the wild datapack.
Use /locate to find regular towers.
Command gives off some coords, but no tower spawns.

Expected behavior
Regular towers spawn.
Mod Version
v4.2.0
Mod Variation
Datapack - Waystones
Minecraft Version
1.20.1
Java Version
Java 17
Fabric Version (only if below v3.0.0)

Fabric: 0.15.1
Fabric API: 0.91.0+1.20.1

Relevant log output
[16:49:14] [Render thread/ERROR]: Error when terminating data generation for section {6*-61,-101}
[16:49:14] [Render thread/ERROR]: Error when terminating data generation for section {6*-47,0}
[16:49:14] [Render thread/ERROR]: Error when terminating data generation for section {6*-48,1}
[16:49:14] [Render thread/ERROR]: Error when terminating data generation for section {6*-47,1}
[16:49:14] [Render thread/ERROR]: Error when terminating data generation for section {6*-48,0}

Not sure if the above is relevant to this issue, please refer to the full log instead.

Anything else?
I'm using the v4.2.1 of the datapack, which is not on the scrollmenu above.
Full log (includes mod list):
https://mclo.gs/H8ACPRk

Changing "location": "totw_reworked:regular/tower_bottom" to "location": "totw_reworked:regular/regular_tower_bottom" in data/totw_reworked/worldgen/template_pool/regular/bottom.json seems to fix this issue.

Is this still a problem? I'm wondering if this is afflicting older versions of MC, too.


re-add ipip and nat-outgoing to calico command

Looks good. I will investigate changing the Mesos drivers for Calico as well.


clean library folder

fix: removed unused .md pages

Pull Request Test Coverage Report for Build 528

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 77.284%

Totals

Change from base Build 521:
0.0%

Covered Lines:
1243

Relevant Lines:
1539

💛  - Coveralls