regex-syntax Hir should merge alternation of Class & Literal into Class

What version of regex are you using?
0.7.2
Describe the bug at a high level.
When you have a HirKind::Class and a HirKind::Literal and you join the two, I expect it to be a Class to reduce the syntax tree layers.
What are the steps to reproduce the behavior?
  #[test]
    fn test_merge() {
        let let_dig = regex_syntax::parse("[a-zA-Z0-9]").unwrap();
        let hyp = regex_syntax::parse("-").unwrap();
        let let_dig_hyp = Hir::alternation(vec![let_dig, hyp]);

        // Expected: (?:[0-9A-Za-z-])
        // Got: (?:[0-9A-Za-z]|\-)
        println!("{}", let_dig_hyp.to_string());
    }

What is the actual behavior?
It's added into a whole new HirKind::Alternation.
What is the expected behavior?
We already have the optimization of simplifying a|b|c into [abc], but I would like to see (?:(?:a|b)|c) to also be simplified to [abc].
Context
I am writing a "composable regex" library to allow users to combine pieces of regexes with |, +, *, ? to make regexes more maintainable. When I write test cases I realized the output Hir's are not optimal.

When I write test cases I realized the output Hir's are not optimal.

They never will be. What you're asking for here is an optimization. As long as it can be done cheaply, I'm open to PRs.
Note that it can't be done for every literal and class combination. The literal needs to be at most a single codepoint.


[BUG] non-integer frame rates are truncated

Describe the bug
Non-integer framerates like 23.976 are truncated down to 23, etc. This occurs because there is a wrong calculation in the "should send frame" function. It casts self.fps to u64 before performing the division:
1_000_000_000u64 / fps as u64
should be
(1_000_000_000f64 / fps) as u64

fixed with 14058ea5b20d885ca1958f8cedddcef6b7810802


Suddenly unable to play any tracks

I am using LibreSpot with the JACK Audio backend, and it worked perfectly, but since a few days I can't play any tracks. I didn't change any configurations on my side, so I assume there is a problem with spotify like it already occured here: #510
Setup
Custom compiled with --no-default-features and --features jackaudio-backend running as a user service on a Raspberry Pi.
Used Arguments:

backend: jackaudio
bitrate: 320
no-audio-cache & no-credential-cache

LibreSpot allows connections from any device just fine, but can't play any song, no matter of its length. Mots of the time it just skips every song after a few seconds without any audio played, but sometimes it also crashes completely.
Logs:
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z INFO  librespot_playback::player] Loading <Lightning Over Heaven> with Spotify URI <spotify:track:0S0zgiheqNBkRjEMo7pnig>
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z WARN  librespot_playback::player] Skipping to next track, unable to load track <SpotifyId { id:<PHONE_NUMBER>6839703698416089418933192304, audio_type: Track }>: ()
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z INFO  librespot_playback::player] Loading <Lightning Over Heaven> with Spotify URI <spotify:track:0S0zgiheqNBkRjEMo7pnig>
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z WARN  librespot_playback::player] Skipping to next track, unable to load track <SpotifyId { id:<PHONE_NUMBER>6839703698416089418933192304, audio_type: Track }>: ()
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z INFO  librespot_playback::player] Loading <Lightning Over Heaven> with Spotify URI <spotify:track:0S0zgiheqNBkRjEMo7pnig>
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_core::channel] channel error: 2 1
Mar 02 10:48:40 raspberrypi librespot[714]: [2022-03-02T15:48:40Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Mar 02 10:48:40 raspberrypi librespot[714]: thread '<unnamed>' panicked at 'Map must not be polled after it returned `Poll::Ready`', /home/pi/.cargo/registry/src/github.com-1285ae84e5963aae/futures-util-0.3.17/src/future/future/map.rs:62:17
Mar 02 10:48:40 raspberrypi librespot[714]: stack backtrace:
Mar 02 10:48:41 raspberrypi librespot[714]:    0:   0x8febfc - std::backtrace_rs::backtrace::libunwind::trace::h532c701585cd392d
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
Mar 02 10:48:41 raspberrypi librespot[714]:    1:   0x8febfc - std::backtrace_rs::backtrace::trace_unsynchronized::h15e96deae50ea7f1
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
Mar 02 10:48:41 raspberrypi librespot[714]:    2:   0x8febfc - std::sys_common::backtrace::_print_fmt::h2de65accbd58b3e4
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/sys_common/backtrace.rs:67:5
Mar 02 10:48:41 raspberrypi librespot[714]:    3:   0x8febfc - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h85671ae76efc2464
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/sys_common/backtrace.rs:46:22
Mar 02 10:48:41 raspberrypi librespot[714]:    5:   0x8f709c - std::io::Write::write_fmt::hcccb69ae7e0e5712
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/io/mod.rs:1697:15
Mar 02 10:48:41 raspberrypi librespot[714]:    6:   0x900c68 - std::sys_common::backtrace::_print::hedd1975fac8ef971
panicking.rs:211:50pberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/sys_common/backtrace.rs:49:5
Mar 02 10:48:41 raspberrypi librespot[714]:    9:   0x900710 - std::panicking::default_hook::h76fa136e1f70a214
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/panicking.rs:228:9
Mar 02 10:48:41 raspberrypi librespot[714]:   14:   0x61fd54 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h6cdb6d970c4ac361
Mar 02 10:48:41 raspberrypi librespot[714]:   15:   0x60d7d8 - <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll::hfe2e84411f277e84
Mar 02 10:48:41 raspberrypi librespot[714]:   16:   0x5fbd10 - <librespot_playback::player::PlayerInternal as core::future::future::Future>::poll::h28a4cde177e46443
Mar 02 10:48:41 raspberrypi librespot[714]:   17:   0x4c7774 - std::thread::local::LocalKey<T>::with::h85f1e7d59454df33
Mar 02 10:48:41 raspberrypi librespot[714]:   18:   0x4bbad4 - futures_executor::local_pool::block_on::hefeadbfdb9a235f9
Mar 02 10:48:41 raspberrypi librespot[714]:   19:   0x567ff4 - std::sys_common::backtrace::__rust_begin_short_backtrace::ha9233c878f46eaea
Mar 02 10:48:41 raspberrypi librespot[714]:   20:   0x5287ac - core::ops::function::FnOnce::call_once{{vtable.shim}}::he863134c8af815b9
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/alloc/src/boxed.rs:1694:9
Mar 02 10:48:41 raspberrypi librespot[714]:   22:   0x9061e4 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h3a9e9491476f36e5
Mar 02 10:48:41 raspberrypi librespot[714]:                        at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/alloc/src/boxed.rs:1694:9
Mar 02 10:48:41 raspberrypi librespot[714]: [2022-03-02T15:48:41Z ERROR librespot_playback::player] Player Commands Error: channel closed
Mar 02 10:48:41 raspberrypi librespot[714]: [2022-03-02T15:48:41Z ERROR librespot_playback::player] Player Commands Error: channel closed
Mar 02 10:48:41 raspberrypi librespot[714]: [2022-03-02T15:48:41Z ERROR librespot_playback::player] Player Commands Error: channel closed
Mar 02 10:48:42 raspberrypi librespot[714]: [2022-03-02T15:48:42Z ERROR librespot_playback::player] Player Commands Error: channel closed

Just in case you are gathering stats/data:
Same issue for me (in Denmark).

Same here (via Spotty integration in Logitech Media Server) (also Denmark)

@jesperll - you're not JesperDB who's been reporting this similar issue to me, are you?

I am from Austria @michaelherger

Also saw similar channel errors from DK..

@jesperll - you're not JesperDB who's been reporting this similar issue to me, are you?

No, that's me :)

FWIW: A similar issue seem to happen if I try using new-api branch, except the spotify client/UI on my PC/phone does not repeatedly skip fwd after ~5 secs [still no playback, though]
Console log with -v from new-api branch:
[...]
[2022-03-03T10:43:35Z TRACE librespot_connect::spirc] Frame has 48 tracks
[2022-03-03T10:43:35Z DEBUG librespot_playback::player] command=SetAutoNormaliseAsAlbum(false)
[2022-03-03T10:43:35Z DEBUG librespot_playback::player] command=Load(SpotifyId("spotify:track:2C0iEdLXanscPYGpwO2oSm"), true, 115962)
[2022-03-03T10:43:35Z TRACE librespot_connect::spirc] Sending status to server: [kPlayStatusPlay]
[2022-03-03T10:43:35Z DEBUG librespot::component] new SpClient
[2022-03-03T10:43:35Z INFO  librespot_core::spclient] Resolved "gew1-spclient.spotify.com:443" as spclient access point
[2022-03-03T10:43:35Z DEBUG librespot::component] new TokenProvider
[2022-03-03T10:43:35Z TRACE librespot_core::token] Requested token in scopes "playlist-read" unavailable or expired, requesting new token.
[2022-03-03T10:43:35Z TRACE librespot_connect::spirc] ==> kPlayStatusLoading
[2022-03-03T10:43:35Z TRACE librespot_connect::spirc] Sending status to server: [kPlayStatusLoading]
[2022-03-03T10:43:35Z ERROR librespot_core::mercury] error 403 for uri hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae
[2022-03-03T10:43:35Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unavailable, error: Response(MercuryResponse { uri: "hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae", status_code: 403, payload: [[123, 34, 99, 111, 100, 101, 34, 58, 52, 44, 34, 101, 114, 114, 111, 114, 68, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 34, 58, 34, 73, 110, 118, 97, 108, 105, 100, 32, 114, 101, 113, 117, 101, 115, 116, 34, 125]] }) }
[2022-03-03T10:43:35Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId("spotify:track:2C0iEdLXanscPYGpwO2oSm")>: ()
[2022-03-03T10:43:35Z ERROR librespot_core::session] could not dispatch command: Service unavailable { error handling Mercury response: MercuryResponse { uri: "hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae", status_code: 403, payload: [[123, 34, 99, 111, 100, 101, 34, 58, 52, 44, 34, 101, 114, 114, 111, 114, 68, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 34, 58, 34, 73, 110, 118, 97, 108, 105, 100, 32, 114, 101, 113, 117, 101, 115, 116, 34, 125]] } }
[2022-03-03T10:43:35Z DEBUG librespot_connect::spirc] Marked <gid: "U\332\267\0046uOl\202YV\344\261\037\241\332" context: "NonPlayable"> at 1 as NonPlayable
[2022-03-03T10:43:35Z DEBUG librespot_playback::player] command=Preload(SpotifyId("spotify:track:40NRy5xeT32HCLae48DVov"))
[2022-03-03T10:43:35Z DEBUG librespot_playback::player] Preloading track
[2022-03-03T10:43:35Z TRACE librespot_core::token] Requested token in scopes "playlist-read" unavailable or expired, requesting new token.
[2022-03-03T10:43:36Z ERROR librespot_core::mercury] error 403 for uri hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae
[2022-03-03T10:43:36Z ERROR librespot_core::session] could not dispatch command: Service unavailable { error handling Mercury response: MercuryResponse { uri: "hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae", status_code: 403, payload: [[123, 34, 99, 111, 100, 101, 34, 58, 52, 44, 34, 101, 114, 114, 111, 114, 68, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 34, 58, 34, 73, 110, 118, 97, 108, 105, 100, 32, 114, 101, 113, 117, 101, 115, 116, 34, 125]] } }
[2022-03-03T10:43:36Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unavailable, error: Response(MercuryResponse { uri: "hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae", status_code: 403, payload: [[123, 34, 99, 111, 100, 101, 34, 58, 52, 44, 34, 101, 114, 114, 111, 114, 68, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 34, 58, 34, 73, 110, 118, 97, 108, 105, 100, 32, 114, 101, 113, 117, 101, 115, 116, 34, 125]] }) }
[2022-03-03T10:43:36Z DEBUG librespot_playback::player] Unable to preload SpotifyId("spotify:track:40NRy5xeT32HCLae48DVov")
[...]

However, if I start librespot with -u <username> -p <password> with same build from new-api branch, I am able to play back the audio.
Maybe there were some recent changes to how the authentication is done with zeroconf?

Put simply, we can't do anything about channel errors. They sometimes hit the Spotify infrastructure and then go away after some time.
The panic is due to trying to reload a track that couldn't be loaded the first time. This is fixed in new-api meaning that it won't panic anymore, but still won't be able to play the track.
There is no difference in authentication to the Spotify infrastructure between zeroconf and command line, all that matters is the way it receives the credentials (either from the command line or over the LAN). From there it authenticates against Spotify in the same way.
It's more likely that the time you logged in with command line credentials, you connected to an access point that didn't suffer from channel errors. You can check the access point in your (verbose) logs when librespot first connects.
I'm closing this as this is not something we can do anything about.

I do agree in general. But we could in theory provide an API to control what access point is used rather than always taking the first (or last, whatever it is, I find the comment there confusing). I am not saying that should then also be exposed to librespot application users through yet another program option (but it could).

Just sharing info, in case it's helpful:
In DK.
2 official spotify clients working (1 not updated macOS, 1 latest releas iOS).
They play just fine, except for trying to use Spotify Connect, with anything derived from or using librespot to enable Spotify Connect.
Problem: Causes all tracks to skip.

i am seeing same thing here. also in denmark. skips all tracks in playlist

Would everybody mind sharing

what ISP you're using
the result of http://apresolve.spotify.com/

Though there have been reports from Germany and Austria, too, I strongly believe this is either a geographical thing, or some network having issues or blocking things.

Of course! :)
Here you go:
ISP: Hiper A/S
http://apresolve.spotify.com/ gives:
{"ap_list":["ap-gew4.spotify.com:4070","ap-gew4.spotify.com:443","ap-gew4.spotify.com:80","ap-guc3.spotify.com:4070","ap-gae2.spotify.com:443","ap-gew1.spotify.com:80"]}

I just did the test again and the client still does the skipping.

You will also want to lookup those host names and report their IP. Same names could result in different IPs depending on geography, I’m not sure.

ap-gew4.spotify.com: <IP_ADDRESS>
ap-guc3.spotify.com: <IP_ADDRESS>
ap-gae2.spotify.com: <IP_ADDRESS>
ap-gew1.spotify.com: <IP_ADDRESS>

From The Netherlands (Ziggo) I get the same IP addresses for those hosts. My apresolve answer looks like this and I can play tracks fine:
{"ap_list":["ap-gew1.spotify.com:4070","ap-gew1.spotify.com:443","ap-gew1.spotify.com:80","ap-gew1.spotify.com:4070","ap-guc3.spotify.com:443","ap-gae2.spotify.com:80"]}

librespot picks the first access point from the list, in my case ap-gew1.spotify.com:4070. Again you can see this in your verbose logs.

Interesting, mine chooses the ap-gew4:
[2022-03-04T11:10:10Z INFO  librespot_core::session] Connecting to AP "ap-gew4.spotify.com:4070"

And then starts skipping with these in the log:
[2022-03-04T11:10:12Z DEBUG librespot_connect::spirc] At track 4 of 30 <"spotify:playlist:37i9dQZEVXcEqdyS2qRIO4"> update [false]
[2022-03-04T11:10:12Z TRACE librespot_connect::spirc] Sending status to server: [kPlayStatusPlay]
[2022-03-04T11:10:12Z DEBUG librespot_playback::player] command=Load(SpotifyId { id:<PHONE_NUMBER>87612460604504651615889269027, audio_type: Track }, true, 0)
[2022-03-04T11:10:12Z TRACE librespot_connect::spirc] Sending status to server: [kPlayStatusPlay]
[2022-03-04T11:10:12Z INFO  librespot_playback::player] Loading <Uku> with Spotify URI <spotify:track:5tvSDMa9x46Rdd4ndtJBBh>
[2022-03-04T11:10:12Z DEBUG librespot_audio::fetch] Downloading file 3f9ee5484ef61d6d72cfccd30446eb01e2dee49a
[2022-03-04T11:10:12Z ERROR librespot_core::channel] channel error: 2 1
[2022-03-04T11:10:12Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
[2022-03-04T11:10:12Z WARN  librespot_playback::player] Unable to load <SpotifyId { id:<PHONE_NUMBER>87612460604504651615889269027, audio_type: Track }>
    Skipping to next track

I just tried to force the ap-gew4.spotify.com domain to point to the IP of ap-gew1.spotify.com, by setting this in /etc/hosts:
<IP_ADDRESS>  ap-gew4.spotify.com

That works!! Sooo.... Is Spotify having an issue? Or are Spotify changing something?

Ha, you beat me to it! I wanted to test the opposite (using ap-gew4 instead of what I receive).
Yes, I believe there's a problem with that one host.

That's annoying... Have you experienced this before? In that case: how long does this normally last? I'd rather not do this trick on my music streamer, but if it's normally a problem that takes a long time to fix, I might need to :)
Thanks for your time guys!
And thanks for this project! I love to have easy ability to get clean 320 bit directly into my DAC board :)

I don't think it's a common problem. But it's probably still worth trying to report to Spotify?
And should librespot be smarter and not only rely on the first element in the list?

I was thinking the same thing. Maybe there's some indicator that can used to drop a host and try the other.
I guess in this case it could be difficult, since the server and API is responding on some calls.

Channel errors are not new. If you use GitHub search on this project you will find a number of reports them. How long they'll last, only Spotify could say.
Programmatically detecting channel errors is not difficult, you literally see channel error: 2 1 in the log so we could act on that. But architecturally it's more difficult to let librespot reconnect to another access point. In new-api it's a bit easier to implement but still needs work.

Nice that there is something to detect it by :)
But yeah... I can imagine how many little details has to be thought through in order for a total re-connect to work seamlessly.
I wish I had learned Rust by now, so that I could try to help out.

i know it is not a beautiful solution but i can confirm that i works. as morphar suggested. :) thanks for the tip.
But instead of doing in all the pi's  /etc/hosts we have implemented the redirect in our dns - <IP_ADDRESS> ponting to ap-gew4.spotify.com

@roderickvd Would you accept a PR with a "fix" where the host is picked at random instead of picking the first one returned?
If the bad host is picked uniformly at random, natural trouble-shooting behaviour like restarting librespot would likely fix the issue.
For example, if one out of five hosts are bad, the probability of picking the bad host twice in a row is just 4%. Though, I should say, for me apresolve.spotify.com returns three out of five bad hosts (same host on different ports), so it may be less effective in practice...

My reading of the host names is that the other two would be US Central and Asia East (Taiwan). Such choices could come at a cost in latency.

They do seem like regional names; I agree that it's not a desirable solution then.

{"ap_list":["ap-gew4.spotify.com:4070","ap-gew4.spotify.com:443","ap-gew4.spotify.com:80","ap-gae2.spotify.com:4070","ap-gew1.spotify.com:443","ap-guc3.spotify.com:80"]}
Denmark here, and not working.

See @morphar's comment further up this report:

I just tried to force the ap-gew4.spotify.com domain to point to the IP of ap-gew1.spotify.com, by setting this in /etc/hosts:
<IP_ADDRESS>  ap-gew4.spotify.com

That works!! Sooo.... Is Spotify having an issue? Or are Spotify changing something?

Hello, I just want to say thank you for fixing my Spotify Connent issues on Volumio.

@kingosticks's suggestion to offer an ABI to manually set the access point isn't a bad idea in this particular case. I am however hesitant to offer it as a command-line option because I don't want to get it misused.
What you can also try is blocking apresolve.spotify.com altogether, which will cause librespot to default to pick ap.spotify.com as a fallback access point. I assume that this is geobalanced and might offer some other, in working order, access point.
I consider all of these as band-aids, ideally librespot would reconnect and drop the erroneous access point. Like I said I can investigate this as part of new-api. It's not on top of my list -- I'd rather have Spotify to fix their infrastructure -- but I should get around to it sometime. In the meantime PR's are more than welcome of course.

One way to enforce this behavior actually already is built in: set ap-port to some random, invalid port. If librespot fails to connect to that port, it would use the fallback.

Ha I never thought about that, talk about emergent behaviour 😆

I didn't know about ap-port until I started looking into how an override could be implemented 😄.

I got reports that things started working again. Can anybody confirm? Remove workaround and try again?

It doesn't work for me.
Maybe the reports are false positives? Maybe they can stream, but it's because they're not connecting to ap-gew4.spotify.com for some reason?

Yes. It was a misunderstanding. The person was already using a workaround I implemented in my Spotty plugin (forcing fallback by providing an invalid ap-port value). That seems to be working for people with the failing ap-gew4.spotify.com.

Confirming that I also had this skipping issue here in Denmark for at least a week before finding the time to search for solutions.
Apresolve points to ap-gew4.spotify.com - but through a VPN to Sweden/Stockholm I get ap-gew1.spotify.com so this could be a country specific Spotify issue..
Can confirm that using the workaround @michaelherger supplied with spotty version 4.8.1 solved my issues.
-for now at least. 🤔
Thanks Michael!

I've also updated to Spotty 4.8.1 but that didn't solve my issues. ;(
I've tried multiple ways of tricking my installation to resolve gew4 to a different IP but so far without success.
(Running LMS with Spotty as addon under home assistant)

Did you check the new "fallback" option in the Spotty settings? No need to fiddle with DNS.

I have two things working against me, Not were good at programming and I'm from Denmark. Spottify Connect still doesn't work here ;(.
I have logged into my rasp4 with a ssh. OS is HiFiBerryOS. How do I get from here, to implementing :
"by setting this in /etc/hosts:
<IP_ADDRESS>  ap-gew4.spotify.com"
??

Hi Mads80,
I think I got it, omitted the "sudo" though. Is there any known and easy cmd that can confirm. Spotify connect went from skipping songs after 1 sec to stopping music after 1 min. It's an improvement ;)

This problem started for me a few days ago in Norway.
After finding this thread, I blocked appresolve.spotify.com, and not librespot works OK again. For now.

Yeah, got reports from Norwegian users in the past two days, too...

+1 here from Norway

+1 here from Norway.

+1 from here in Denmark 🇩🇰

+1 from here in Denmark denmark
Thank you for your advice

+1 from Norway. Fixed hosts in Arch and Win11 and working fine for now.(ncspot)

+1 from Norway, glad I found this thread, and the hosts entry did the job.

+1 from Switzerland
With "<IP_ADDRESS>  ap-gew4.spotify.com" in the /etc/host file it works, thanks!
I'm still not sure where the problem is and how to fix it correctly.

+1 from Norway
Adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts did solve the problem. Have had this issue for a while and suddenly came across this thread.
Anyone got any answers from Spotify? This is lazy work at their end.

+1 from Sweden
Used the "fallback ap hack". Set --ap-port=[some random port]. Log then said Using fallback "ap.spotify.com:443" and it works!

+1 from Finland
Adding a line containing <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts solved the issue.

I've now had reports from Austria and Switzerland, too... are they shutting down old infrastructure?

I know its not quite AT & CH but I can confirm in Germany I have had this issue for quite some time.

+1 from The Netherlands (using ncspot under macos).
Fixed by adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts :)

+1 from Sweden
Adding a line containing <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts solved the issue.

+1 from Denmark 🙏🙏🙏

+1 from The Netherlands.
Fixed by adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts

+1 from 🇳🇱, the above fix helped me too

I am on Balena OS using balena sound to play tracks from Spotify.
I added <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts.
But it is non persistant, since it is running in a container.
I tried adding it to pi.hole local DNS records. I can ping ap-gew4.spotify.com from Spotify container and Host OS with reply.
With no luck tracking down the real issue.
Log-snippet:
<spotify> [2022-07-17T12:58:45Z INFO  librespot_playback::player] Loading <Guilty of Love> with Spotify URI <spotify:track:6SskWYTJwI77yhvNJscIiu>
<spotify> [2022-07-17T12:58:45Z ERROR librespot_core::channel] channel error: 2 1
<spotify> [2022-07-17T12:58:45Z ERROR librespot_playback::player] Unable to load encrypted file.
<spotify> [2022-07-17T12:58:45Z WARN  librespot_playback::player] Unable to load <SpotifyId { id:<PHONE_NUMBER>78731222655173408678663613098, audio_type: Track }>

@CaptainAweesome Maybe try extra_hosts in docker compose? See https://docs.docker.com/compose/compose-file/compose-file-v3/#extra_hosts
Or if possible modify https://github.com/balenalabs/balena-sound/blob/master/plugins/spotify/start.sh and add command line arg --ap-port=[some random port]

I tried adding it to pi.hole local DNS records. I can ping ap-gew4.spotify.com from Spotify container and Host OS with reply. With no luck tracking down the real issue.

Try blocking apresolve.spotify.com in pihole, that worked for me.

Try blocking apresolve.spotify.com in pihole, that worked for me.

This actually worked. I am astounded. Thank you very much!
@agramner I did not find any recommended ways to add parameters to containers in Balena. Since Balena-sound is a custom build of Balena OS with many containers, without any excessive customizability.
None the less, I thank you for your reply!

This actually worked. I am astounded. Thank you very much!

I'm now back to the same errors as before. Unable to load tracks from Spotify.
apresolve.spotify.com is blocked in pi.hole and nslookup apresolve.spotify.com gives the following address: <IP_ADDRESS>
Even though I can access http://apresolve.spotify.com/ from my browser all of a sudden.

Had this problem today in Finland. Blocking apresolve.spotify.com on pihole was a successful workaround.

@CaptainAweesome how do you precisely run librespot from a container? Do you use docker or docker-compose or something else?

@ajarmoniuk

I am on Balena OS using balena sound to play tracks from Spotify.

It's a way of deploying docker containers to devices and managing said devices remotely.
Here is the balena-sound startup for Spotify: https://github.com/balenalabs/balena-sound/blob/master/plugins/spotify/start.sh
I have now instead of adding apresolve.spotify.com to Blacklist, added apresolve.spotify.com <IP_ADDRESS> to Local DNS Records inside pi.hole. And strangely it works now, again. Thank you!

+1 from NL. Adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts fixed it. Also blocked apresolve.spotify.com in pihole to be on the safe side ;-).

Hello, I am having the same problem with Moodeaudio version 8.1.2. with raspberry pi4.
I can't listen to Spotify. I am in Argentina, can someone help me with the dns of this area, to add: xxx.xxx.xxx.xxx ap-gew4.spotify.com to /etc/hosts? Thank you!

sucama@sucama-B250M-D3H:~$ ping ap-gew4.spotify.com
PING ap-gew4.spotify.com (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>.bc.googleusercontent.com (<IP_ADDRESS>): icmp_seq=1 ttl=101 time=246 ms
64 bytes from <IP_ADDRESS>.bc.googleusercontent.com (<IP_ADDRESS>): icmp_seq=2 ttl=101 time=245 ms
64 bytes from <IP_ADDRESS>.bc.googleusercontent.com (<IP_ADDRESS>): icmp_seq=3 ttl=101 time=246 ms
64 bytes from <IP_ADDRESS>.bc.googleusercontent.com (<IP_ADDRESS>): icmp_seq=4 ttl=101 time=246 ms

Hi Sucama, I've just solved the issue by changing /etc/hosts as follows:

<IP_ADDRESS>  ap-gue1.spotify.com
Basically, ap-gue1.spotify.com is the host volumio/spotify try to resolve from Argentina but it doesn't work, so you need to point it to ap-gew4.spotify.com by adding its IP address (<IP_ADDRESS>) in the /etc/hosts file.
Make sure after the change to restart the network interfaces and restart volumio. You can also restart the whole server to make it easier.
Hope it helps.

Hi Sucama, I'm from Argentina too and I had been having this problem since a week ago. Yesterday I edited hosts file with:
<IP_ADDRESS> apresolve.spotify.com
and now works fine. Tested spotify all day without problems.
I'm using HiFiBerry OS with Raspberry 3+.
Just for information, along all the trials I performed with no luck (updating Spotify in mac and iOS devices, clean install of HiFiBerry OS on the raspy, downgrading HiFiBerry OS to prior version) I installed spotify app in my LG TV and it worked fine. I could connect from spotify desktop and control the TV but not the raspberry. So I'm not sure if this is a DNS problem alone.
Hope it helps.

I had the same issue, I'm from Italy 🤌. I'm using a Raspberry Pi 3B with Raspberry Pi OS and Raspotify.
I added the following line to /etc/hosts and now everything works fine:
<IP_ADDRESS>		apresolve.spotify.com

I had the same issue, I'm from Italy 🤌. I'm using a Raspberry Pi 3B with Raspberry Pi OS and Raspotify.
I added the following line to /etc/hosts and now everything works fine:
<IP_ADDRESS>		apresolve.spotify.com

Thank you @EdoardoLaGreca :) Same country same fix ;)

+1 from NL. Adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts fixed it for me

+1 from NL. Adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts fixed it for me

Same for me :) I was struggling for 3 days tinkering with my devices to find this thread and learn it's from Spotify's side. Thanks, guys, you are the best!

Awesome workarounds! However, I am a bit curious what the difference is between defining
<IP_ADDRESS>		apresolve.spotify.com

and
<IP_ADDRESS> ap-gew4.spotify.com

Both seem to fix the problem for me. But is either preferable for some reason?

Hi , Thanks To  programamapache for this:
<IP_ADDRESS>		apresolve.spotify.com
Work for me in Argentina.. Moodeaudio versio 8.1.2 , Raspberry pi4
Thanks Very much!

Awesome workarounds! However, I am a bit curious what the difference is between defining

Have you tried opening apresolve.spotify.com in your browser?
If you do that, you will get something like:
{"ap_list":["ap-gew1.spotify.com:4070","ap-gew1.spotify.com:443","ap-gew1.spotify.com:80","ap-gew1.spotify.com:4070","ap-guc3.spotify.com:443","ap-gae2.spotify.com:80"]}
which is a list of Spotify content delivery servers most suitable for your location (as determined from Spotify's side)
As stated previously in this thread, it looks like Spotify has started rolling out some new method of serving content through some of these servers. That new method works with the official Spotify client, but not with librespot. (Which I don't consider "lazy" as stated in a previous comment - why should they care about unofficial clients?)
With apresolve.spotify.com blocked, librespot defaults to some server that doesn't have that new method enabled (yet).
If you just define a single server, there is a chance that the response from apresolve could change at some point in time, and then you would need to enter a new combination of IP address + URL.
As I'm not experienced with the inner workings of librespot or Spotify, I'm just going with my basic knowledge and logic, so what I've just written may be wrong in some way. If that is so, somebody please correct me :)

@kirenida:

why should they care about unofficial clients?

Because they deprecated the official libspotify so there isn't an "official client" anymore.

Sorry to be pedantic but libspotify was deprecated in May 2015, they've not given a hoot for 7 years.

Sorry to be pedantic but libspotify was deprecated in May 2015, they've not given a hoot for 7 years.
Edit:
And it is lazy since they promised a replacement back and then and didn't deliver. Useless? Lazy? Pick one!
Anyway, grumbling aside, what's interesting is there is another potential breakage coming in September 2022. This might be a deadline for getting new api branch merged (sorry, I know deadlines are not helpful @roderickvd and it's on everyone to step up and help). https://developer.spotify.com/community/news/2022/07/15/mobile-streaming-sdks-update/

I was hoping to make some time to do a quick update release to blacklist this “broken” (to us) access point, but chances of me completely upgrading to the HTTP-based API before September are next to zero:

the current state of new-api is functional  but needs bugs ironed out and dev merged in. I don’t realistically have time for all of that before September. I have called multiple times for people to take on that chore but no one has answered the call.

new-api now retrieves files over audio files over HTTP but not encryption keys or commands.

Now I don’t know what Spotify will be deprecating and what not, but:

the encryption keys over HTTP have not been reversed engineered yet (with decompilation attempts from some having been taken down).

the HTTP-based network commands (websockets) require implementing the dealer and this will take a lot of work as well as some advanced Rust skills and knowledge of librespot internals. librespot-java has the reverse engineering mostly down but it’s much more than just a matter of Java-2-Rust as the code architecture has diverged a lot.

Again I don’t know what that Spotify SDK deprecation may bring but if this project wants to stay up to date then we really need more contributors.

Good summary. I have a few Python bits to finish in my downstream project this weekend, after which I'm going to focus on getting gstspotifysrc where I need it to be. I'll switch it to the new-api (and worry about the release implications of that later) and then be some help to you here. I don't need all of librespot's features for my stuff but I'll do what I can.
However, I don't profess to having advanced Rust skills.

+1 from NL. Adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts fixed it for me

Same for me :) I was struggling for 3 days tinkering with my devices to find this thread and learn it's from Spotify's side. Thanks, guys, you are the best!

Can confirm from the Netherlands that adding <IP_ADDRESS> ap-gew4.spotify.com to /storage/.config/hosts.conf did the trick as well!
Thanks!

Just a little heads up. As outlined before there's no need to fiddle with DNS, hosts files, IP addresses etc. You can work around this issue using a librespot parameter:

Used the "fallback ap hack". Set --ap-port=[some random port]. Log then said Using fallback "ap.spotify.com:443" and it works!

the encryption keys over HTTP have not been reversed engineered yet (with decompilation attempts from some having been taken down).

As a fallback - we can use widevine, but it will probably need a lot of re/writing code because it differs a lot from the current implementation.

What do you mean by that @SuisChan? Widevine is a (commercial?) closed source library, isnt it? Reverse engineering that software would be like poking a (small, angry) bear.

What do you mean by that @SuisChan? Widevine is a (commercial?) closed source library, isnt it? Reverse engineering that software would be like poking a (small, angry) bear.

Everything is not quite right, since we can get a private key, we can use it accordingly, and the algorithms have been public for a long time (leaks and not only), besides, I have experience working with it, so I can confirm that it works.

+1 from NL. Adding <IP_ADDRESS> ap-gew4.spotify.com to /etc/hosts fixed it for me

Please take #1026 for a spin. If this works (it does for me -- I also get ap-gew4 in the Netherlands usually) I can do a point release the coming days.

Please take #1026 for a spin. If this works (it does for me -- I also get ap-gew4 in the Netherlands usually) I can do a point release the coming days.

For me adding ap-gew4 (manually) did not work for 2 out of 3 devices. Adding "<IP_ADDRESS>		apresolve.spotify.com" however did work on all 3 devices. Also Netherlands here, so that's a bit weird.
If I understand correctly then adding ap-gew4 is rerouting the stream while adding <IP_ADDRESS> is blocking some resolve api? Anyways, for now everything works here (faster than usual btw).

Yes blocking apresolve blocks the resolving mechanism altogether.
It would be helpful if you could list those other access points besides ap-gew4 on those 2/3 devices that did not work for you?

Fix worked for me aswell! Thank you so much <3
User from Switzerland here.
and just to be clear the fix for me was: add this line to etc/hosts
<IP_ADDRESS>		apresolve.spotify.com

Yes blocking apresolve blocks the resolving mechanism altogether.
It would be helpful if you could list those other access points besides ap-gew4 on those 2/3 devices that did not work for you?

Unfortunately I can't extract logs from my spotify implementation in Hifiberry (or don't know how). But, I just randomly tried adding other servers and apparently my 1 Hifiberry device uses "ap-guc3.spotify.com". Rerouting this to <IP_ADDRESS> did the trick. Seems completely random as I have 3 Hifiberry devices here and 1 uses the ap-gew4, the other ap-guc3 and the third I haven't checked yet. My ISP is KPN (glasvezel).
My (foolproof) suggestion would be to add multiple servers to the hosts file, pointing to <IP_ADDRESS> or blocking apresolve.

Hmm. You’re the first to report defunct servers other than ap-gew4 but this may be because so many others are blocking the resolver completely.
I’d like some confirmation on this because it matters for the PR I linked to. I don’t like blocking the resolver completely but if access point compatibility is dropping like flies then we might have to. Note that the GeoIP-based fallback might still direct you to a broken server so it’s not entirely foolproof.
HiFiBerryOS runs Spotifyd right? That’s a good crew but know that it’s still running on librespot 0.2 so I am unsure how fast they will pick up a librespot point release with a small fix.

I have just made a test by using each gateway's IP address for all hosts in /etc/hosts and it turns out only <IP_ADDRESS> (which is ap-gew4) is using the new interface so far.

librespot also did stop working for me since this morning due to the problem described in this issue. I am located in Bavaria, Germany.
my apresolve.spotify.com also contains the  ap-gew4 as first entry.
root@al:~# curl apresolve.spotify.com
{"ap_list":["ap-gew4.spotify.com:4070","ap-gew4.spotify.com:443","ap-gew4.spotify.com:80","ap-gew1.spotify.com:4070","ap-guc3.spotify.com:443","ap-gae2.spotify.com:80"]}

Can confirm that adding
<IP_ADDRESS>                apresolve.spotify.com

OR
# IP of ap-guc3.spotify.com
<IP_ADDRESS>         ap-gew4.spotify.com

to /etc/hosts both work for me.

Hello, noob here – I added following line to /etc/hosts of Spotify service (via dashboard -> Terminal -> Spotify target -> then editedd with vi, saved with :x), but the changes are not kept after rebooting.
Could you please explain step by step how to make this fix?
Context: unable to play songs since this morning, I'm also located in Germany.

Hello, noob here – I added following line to /etc/hosts of Spotify service (via Balena dashboard -> Terminal -> Spotify target -> then editedd with vi, saved with :x), but the changes are not kept after rebooting.
Could you please explain step by step how to make this fix?
Context: unable to play songs since this morning, I'm also located in Germany. I'm using BalenaSound, not Librespot, but this issue has been seen across many platforms (also Volumio afaic)

It seems like you're running in a container. You have to make this change in the container's config or network wide (typically in your router). It very much depends on your particulars: ask the community support for your container image or check how to do it with your particular router model.

@piotrzieba since you're likely using balena-sound, the fastest way you can achieve this is by going to your Pihole configuration and blocking apresolve.spotify.com
Another way would be to modify your balena-sound project (the one you downloaded and put on Raspberry Pi using balena push) by modifying the file docker-compose.yml.
In the file, locate the spotify section and add extra-hosts, so that the whole section looks like this:
  spotify:
    build: ./plugins/spotify
    restart: on-failure
    privileged: true
    network_mode: host
    extra-hosts:
      - "apresolve.spotify.com:<IP_ADDRESS>"
    volumes:
      - spotifycache:/var/cache/raspotify

Be careful, the space indenting is important.
After which you need to push your modified project again to your Raspberry Pi:
balena push <ip>
where the  is the IP number of your Rapsberry Pi from the Balena console.

I merged #1026 with a fix for ap-gew4. If nothing else pops up I will do a 0.4.2 release the coming days.

@roderickvd also having the same problem with ap-gue1

@roderickvd also having the same problem with ap-gue1

Here in Brazil ap-gue1 is the problem.
I had to redirect dns from ap-gue1 to ap-gew1 on my router and now it's working.

Hello, noob here – I added following line to /etc/hosts of Spotify service (via Balena dashboard -> Terminal -> Spotify target -> then editedd with vi, saved with :x), but the changes are not kept after rebooting.
Could you please explain step by step how to make this fix?
Context: unable to play songs since this morning, I'm also located in Germany. I'm using BalenaSound, not Librespot, but this issue has been seen across many platforms (also Volumio afaic)

I just temporary fixed the issue for BalenaSound by adding a command in the start.sh file of the Spotify Docker container:

locate the file in plugins/spotify/start.sh
Add the following line before the "set" command for librespot (almost last command): <IP_ADDRESS> ap-gew4.spotify.com
Push the new code to BalenaCloud via balena push or git

Thanks @miguelprates and @marciogranzotto, I will add ap-gue1 to the blacklist and do a release. Hope this will cover most of the cases and otherwise people can use any of the workarounds mentioned in this issue.
Will thereafter focus on development of new-api which is where the real fix is -- that new API doesn't run into trouble with any of these access points because it downloads files over HTTP instead of Mercury.

v0.4.2 has been released: https://github.com/librespot-org/librespot/releases/tag/v0.4.2

Just to help others that come here with the same error: I have a setup with a dnsmasq in my internal network and multiple librespot devices in that. network. So if you add
<IP_ADDRESS> apresolve.spotify.com
to your dnsmasq config (e.g. /etc/hosts) on your (wlan) router, all librespot devices profit immediately from that fix. Of course it's better to upgrade :smile:

Just to help others that come here with the same error: I have a setup with a dnsmasq in my internal network and multiple librespot devices in that. network. So if you add
<IP_ADDRESS> apresolve.spotify.com
to your dnsmasq config (e.g. /etc/hosts) on your (wlan) router, all librespot devices profit immediately from that fix. Of course it's better to upgrade 😄

May i ask what you mean by, of corse it's better to update? is there any newer client to stream spotify to a pi, or what do you mean? :)

v0.4.2 works around the issue for two known bad access points.
You can also compile 0.5.0-dev from source which fixes the issue entirely by using another API. It’s in actively development so expect other bugs (and we’d welcome the feedback).

v0.4.2 works around the issue for two known bad access points.
You can also compile 0.5.0-dev from source which fixes the issue entirely by using another API. It’s in actively development so expect other bugs (and we’d welcome the feedback).

Not so much. I'm getting 403 errors for hm://keymaster/token/authenticated?scope=playlist-read on 0.5.0-dev I have no problems with v0.4.2 though.
It looks like they're shutting down or at least scaling back that interface.
...
[2022-08-25T16:30:20Z ERROR librespot_core::mercury] error 403 for uri hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae
...

v0.4.2 works around the issue for two known bad access points.
You can also compile 0.5.0-dev from source which fixes the issue entirely by using another API. It’s in actively development so expect other bugs (and we’d welcome the feedback).

Not so much. I'm getting 403 errors for hm://keymaster/token/authenticated?scope=playlist-read on 0.5.0-dev I have no problems with v0.4.2 though.

Which commit of 0.5.0-dev is this, the latest?

It looks like they're shutting down or at least scaling back that interface.

I don't know, 0.4.2 uses the old keymaster ID only, so if it works on that version then it would be evidence to the contrary...

...
[2022-08-25T16:30:20Z ERROR librespot_core::mercury] error 403 for uri hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae
...

I'll try and fiddle with this tonight. With @SuisChan's code to solve the hashcash challenge, I can revert-and-update https://github.com/librespot-org/librespot/commit/cdf84925ad0bf3b872a13922a0e9ee9523e17f26 and pass the real client ID everywhere. This should be the way forward, I think.

Yep. I started having issues last night. My PR branch and the latest dev have the same problem (which stands to reason since I didn't mess with those bits).

I don't know, 0.4.2 uses the old keymaster ID only, so if it works on that version then it would be evidence to the contrary...

I don't know either? I haven't really dug that deep in the network code?

From the top of my head it might not send the device_id and now that we do, some servers may not like when that device_id does not correspond to some authenticated client_id, I don't know. No matter, let's focus on solving the challenge and then we can test that.
I can't reproduce your issue here from The Netherlands. Let me see if I can get it working tonight, so you can still test on a "misbehaving" access point.

From the top of my head it might not send the device_id and now that we do, some servers may not like when that device_id does not correspond to some authenticated client_id, I don't know. No matter, let's focus on solving the challenge and then we can test that.
I can't reproduce your issue here from The Netherlands. Let me see if I can get it working tonight, so you can still test on a "misbehaving" access point.

Sounds good. I pretty keen on providing reasonably full metadata over events. I think that would make users and upstream pretty happy to not rely on some other separate mechanism to retrieve it.
But again I think that should be a separate PR after my current one is merged.

Working on it now. For posterity, the hashcash challenge is not presented when you say you're a Mac, but it is when you say you're on Linux 😕

Working on it now. For posterity, the hashcash challenge is not presented when you say you're a Mac, but it is when you say you're on Linux

Can we just lie?

Working on it now. For posterity, the hashcash challenge is not presented when you say you're a Mac, but it is when you say you're on Linux

I just tried it again after the revert. No go. Still not working.

Yes I see it too on Linux (and not on macOS). It's not even that we're presented a hash cash challenge, it just fails outright with a HTTP/400 when requesting the client token. The Spotify servers are really picky about the header values that you send, it must be that we're sending some they don't like all of the sudden.
I tried "lying" by changing more than a few instances of Linux -> macOS but I haven't found it yet, argh...

Yes I see it too on Linux (and not on macOS). It's not even that we're presented a hash cash challenge, it just fails outright with a HTTP/400 when requesting the client token. The Spotify servers are really picky about the header values that you send, it must be that we're sending some they don't like all of the sudden.
I tried "lying" by changing more than a few instances of Linux -> macOS but I haven't found it yet, argh...

It's very strange that I have no problem with v0.4.2?

This concerns the HTTP client token in spclient, v0.4.2 doesn't have that at all.

This concerns the HTTP client token in spclient, v0.4.2 doesn't have that at all.

See I told you I know basically nothing about the network stuff,lol!!! Kinda on that note I'm wondering if we can get the user's display-name I saw it somewhere in the web API docs, can't remember where but it would be nice to add that to the Connected and Disconnected events. Currently what we call the "username" is really the user-id and that's not super useful for the event for display purposes. It would be nice to have the "pretty" name.

My idea for the events is to make it trivial for upstream to be able to display the user's display-name along with the client device's name and all the relevant metadata. I'm sure moode and and such would like that. Last I checked they don't display anything when using librespot?

With all that info you could write a pretty basic Python script to push to the framebuffer on a Pi for example and not even need an event loop. It could just be driven purely by the events.

You may want to try the latest commit, it works on Linux and macOS here. If it doesn't for you, please let me know your access point so I can manually try to connect to that.
It sure looks like Spotify is changing things on their end. Linux is now no longer presented a hash cash challenge. It was some weeks ago... to make Linux get a client ID again we still have to use the keymaster ID.
In case any hash cash challenge is thrown at us, @SuisChan's excellent example over at https://github.com/SuisChan/ctk-example shows how it is done for Android. I have ported this code, but not tested it, so at this point consider it experimental.
I was mixing up the client token (which I was working on) and the Mercury 403 error for the "normal" token you posted above. Again, let me know which access point is troubling you and I can try some stuff. Basically this is trial and error swapping between the keymaster ID and real client ID, and having the device ID there, removed, or truncated to 16 characters.
At this point it's clear that Spotify isn't consistent cross-platform and we should take care to test stuff like this on the major OSs.
I have not had the time to look at your event code but let's keep the discussion with that PR.

Still no go. The access point is.
...
[2022-08-26T00:07:41Z INFO  librespot_core::session] Connecting to AP "ap-guc3.spotify.com:4070"
...
[2022-08-26T00:07:42Z INFO  librespot_core::spclient] Resolved "guc3-spclient.spotify.com:443" as spclient access point
...
[2022-08-26T00:07:42Z ERROR librespot_core::mercury] error 403 for uri hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae
[2022-08-26T00:07:42Z ERROR librespot_playback::player] Unable to load audio item: Error { kind: Unavailable, error: Response(MercuryResponse { uri: "hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480ea92a077233ca87bd&device_id=ce8d71004f9597141d4b5940bd1bb2dc52a35dae", status_code: 403, payload: [[123, 34, 99, 111, 100, 101, 34, 58, 52, 44, 34, 101, 114, 114, 111, 114, 68, 101, 115, 99, 114, 105, 112, 116, 105, 111, 110, 34, 58, 34, 73, 110, 118, 97, 108, 105, 100, 32, 114, 101, 113, 117, 101, 115, 116, 34, 125]] }) }
...

Totally weird. I just connected to that AP manually and no problems here. But off to bed now...

Totally weird. I just connected to that AP manually and no problems here. But off to bed now...

Yep, I don't know what to say?

I have 2 other librespot devices both running v0.4.2 one wired and one on wifi and they both work fine. I even build v0.4.2 on my desktop and it works fine so it's not a network issue as far as I can tell?

I don't know either. I tried both on Linux and macOS, both successful. Can you try v0.5.0 on those other devices, see if that matters or if it sticks to a certain OS?

Can you try v0.5.0 on those other devices, see if that matters or if it sticks to a certain OS?

I already did. It didn't matter as they are all Linux, my desktop runs Fedora, my server runs Debian and and my Pi Zero v2 runs RaspberryPiOS.
It must be some kind of geo location blocking thing? I'm looking for a free proxy that's compatible with librespot, as in is http(s)://Host:Port and not IP:Port. Librespot errors out if you try to use a IP:Port proxy.

I would consider not being able to use IP:Port style proxy a bug or at least a missing feature if you already have http(s)://Host:Port support. I will look into that and see what I can do.

As a temporary workaround here, can you use an entry in /etc/hosts to do this?

As a temporary workaround here, can you use an entry in /etc/hosts to do this?

Tried that already also. No dice.
I added this to my /etc/hosts:
<IP_ADDRESS> ap-guc3.spotify.com
2600:1901:1:5ca:: guc3-spclient.spotify.com

Where <IP_ADDRESS> is the IP to ap-gew1.spotify.com and 2600:1901:1:5ca:: is the IP to gew1-spclient.spotify.com.

I even tried connecting to my phone via a hotshot so my IP was different.

Sorry, maybe I misunderstood what you were trying to do, but I meant you could add an entry for the free hostname-only proxy you are trying to use in order to avoid the geo-restiction.

Sorry, maybe I misunderstood what you were trying to do, but I meant you could add an entry for the free hostname-only proxy you are trying to use in order to avoid the geo-restiction.

I tried setting up a proxy via GNOME's network settings but none of them actually worked that I could find.
I have never really had to mess with proxies and I just googled setting a proxy in /etc/hosts and everything I read says that you can redirect to specific ports in /etc/hosts so that's not very useful.

I was talking about a workaround for

Librespot errors out if you try to use a IP:Port proxy.

/etc/hosts
some.proxy.you.only.have.an.ip.for foo.com

Then use proxy http://foo.com:Port with librespot, which it should like, but you'll actually have a connection to some.proxy.you.only.have.an.ip.for.
Years ago when I tried random free proxies on the net, they never worked. Alternatively there are some free VPN services these days, that should Just Work.

Alternatively there are some free VPN services these days, that should Just Work.

The only free VPN's I can find are IPV4 only and *-spclient.spotify.com seems to be IPV6 only.
I even tried to power cycle my modem to get a different IP from my ISP which worked in the sense that I got a different IP, before my IP said I was in New York City, New York no I'm apparently now I'm in Hamilton Missouri. I ofc are in neither of those places,lol!!! But that was no good either.

Well as a workaround I just tried to use my Spotify creds and that worked without a hitch.

Still don't have a clue what's going on. Maybe you can sprinkle some traces around the Mercury sending code, to see the difference in request we're making depending on whether you're using zeroconf or command-line authentication?

...spot_core::mercury] error 403 for uri hm://keymaster/token/authenticated?scope=playlist-read&client_id=65b708073fc0480...

Hey! Are you having trouble getting an access token via hm://keymaster... or something else?
Why not (as an experiment) try to take it from login5 endpoint? (since it returns noy only the login credentials but also the token, but don't know what scopes it has since no information provided).

Well as a workaround I just tried to use my Spotify creds and that worked without a hitch.

What if I just have a Facebook login? 🥲

This backend is known to panic on several platforms.
Consider using some other backend, or better yet, contributing a fix.
Country: "US"
The application panicked (crashed).
Message:  could not find device
Location: /Users/brew/Library/Caches/Homebrew/cargo_cache/registry/src/github.com-1ecc6299db9ec823/librespot-playback-0.2.0/src/audio_backend/portaudio.rs:62

what is workaround?

This backend is known to panic on several platforms.
Consider using some other backend, or better yet, contributing a fix.
Country: "US"
The application panicked (crashed).
Message:  could not find device
Location: /Users/brew/Library/Caches/Homebrew/cargo_cache/registry/src/github.com-1ecc6299db9ec823/librespot-playback-0.2.0/src/audio_backend/portaudio.rs:62

what is workaround?  macOS 11.6

I don't think this is related to the subject of this issue.
But on Mac, just use Rodio instead of PortAudio. I don't know about this Homebrew recipe but Rodio should be the default backend anyway. Or specify --device with whatever PortAudio needs, because that's what it's complaining about.

Weird. Have you tried blacklisting ap-gue1.spotify.com in /etc/hosts? See if another access point will work?

FWIW I'm having the same issue, though I'm unable to modify /etc/hosts due to some details of my setup. Did you happen to find a fix?

I ran into this on a Raspberry Pi running raspotify. I've been able to use it to play music from Spotify without problems, but all of the sudden I started getting this error when trying to play tracks - and tracks just skipped ahead immediately all the time:
Aug 30 12:56:08 Grottan librespot[558]: [2024-08-30T10:56:08Z ERROR librespot_core::channel] channel error: 2 0
Aug 30 12:56:08 Grottan librespot[558]: [2024-08-30T10:56:08Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError
Aug 30 12:56:08 Grottan librespot[558]: [2024-08-30T10:56:08Z ERROR librespot_playback::player] Skipping to next track, unable to load track <SpotifyId { id:<PHONE_NUMBER>10060129689800402008964857369, audio_type: Track }>: ()
Aug 30 12:56:08 Grottan librespot[558]: [2024-08-30T10:56:08Z ERROR librespot_core::channel] channel error: 2 0
Aug 30 12:56:08 Grottan librespot[558]: [2024-08-30T10:56:08Z ERROR librespot_playback::player] Unable to load encrypted file: ChannelError

The output of http://apresolve.spotify.com/ for me is:
{"ap_list":["ap2-gew4.spotify.com:4070","ap2-gew4.spotify.com:443","ap2-gew4.spotify.com:80","ap2-guc3.spotify.com:4070","ap2-gue1.spotify.com:443","ap2-gew1.spotify.com:80"]}

If I block apresolve.spotify.com, i.e. add the following to /etc/hosts: <IP_ADDRESS> apresolve.spotify.com, raspotify on my Raspberry Pi starts working again because it falls back to ap.spotify.com:443:
Aug 30 13:29:01 Grottan librespot[2870]: [2024-08-30T11:29:01Z INFO  librespot] librespot 0.4.2 a6e1258 (Built on 2023-06-21, Build ID: g9ifMDjd, Profile: release)
Aug 30 13:29:15 Grottan librespot[2870]: [2024-08-30T11:29:15Z WARN  librespot_core::apresolve] Failed to resolve Access Point: error trying to connect: tcp connect error: Connection refused (os error 111)
Aug 30 13:29:15 Grottan librespot[2870]: [2024-08-30T11:29:15Z WARN  librespot_core::apresolve] Using fallback "ap.spotify.com:443"
Aug 30 13:29:15 Grottan librespot[2870]: [2024-08-30T11:29:15Z INFO  librespot_core::session] Connecting to AP "ap.spotify.com:443"

Yay!

Had the same problem, can verify that @bjorgvino solution worked. Thanks!

@bjorgvino 's solution worked for me! Many thanks!

@bjorgvino 's solution worked for me also! Thanks a lot

Are you all using zeroconf to authenticate ?

Yeah I'm using the latest release, 0.4.2.
No authentication. It's just that the first access point doesn't seem to work for me (and others it seems), perhaps because of the port, or something completely different, and the fallback logic doesn't seem to kick in... i.e. it doesn't try the other access points.
Btw what I posted was not in any way "my solution", I just tried stuff already suggested in this thread and reported back how it went..

Understood. Perhaps a new release is in order then?

Blocking apresolve.spotify.com on my AdGuard Home server worked for me too.

It worked for me since this morning. I'm getting the same error with the ap.spotify.com fallback

Then you'll have to switch to the real fix. which is to compile using the dev branch.

https://github.com/librespot-org/librespot/issues/972#issuecomment-2320943137 workaround does not work for me:
$ cat /etc/hosts
<IP_ADDRESS> localhost
<IP_ADDRESS> s4ap
# spotify_player workaround, see https://github.com/librespot-org/librespot/issues/972#issuecomment-2320943137
<IP_ADDRESS> apresolve.spotify.com
...

restart spotify_player and it keeps skipping as I show in

https://github.com/aome510/spotify-player/issues/557#issuecomment-2328710437

Then you'll have to switch to the real fix. which is to compile using the dev branch.

I did this two days ago because of the zeroconf thing but as of this morning it stopped working again. Overriding the DNS entry also didn't work this time.
But blocking apresolve.spotify.com did the trick!


ISSUE-119 Add OceanWorks API's

This PR will eventually address #119
It is not ready to merge yet as the core functionality still needs to be implemented. This will consist of introducing a new Class called NexusCLI()... we will then essentially wrap all of the nexuscli calls making them available from within Podaacpy.

@lewismc
This is looking great!


[SPARK-27221][SQL]Improve the assert error message in TreeNode.parseToJson

What changes were proposed in this pull request?
When TreeNode.parseToJson may throw an assert error without any error message when a TreeNode is not implemented properly, and it's hard to find the bad TreeNode implementation.
This PR adds the assert message to improve the error, like what TreeNode.jsonFields does.
How was this patch tested?
Jenkins

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9116/
Test PASSed.

Test build #103735 has started for PR 24159 at commit 294da92.

Test build #103735 has finished for PR 24159 at commit 294da92.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #103741 has started for PR 24159 at commit ed3eea4.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9122/
Test PASSed.

Test build #103741 has finished for PR 24159 at commit ed3eea4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103741/
Test PASSed.

Merged to master.


HDDS-2026. Overlapping chunk region cannot be read concurrently

What changes were proposed in this pull request?
Only allow a single read/write operation for the same path in ChunkUtils, to avoid OverlappingFileLockException due to concurrent reads.  This allows concurrent reads/writes of separate files (as opposed to simply synchronizing the methods).  It might be improved later by storing and reusing the file lock.
Use plain FileChannel instead of AsynchronousFileChannel for reading, too, since it was used in synchronous fashion (by calling .get()) anyway.
https://issues.apache.org/jira/browse/HDDS-2026
How was this patch tested?
Added unit test.
Used improved Freon tool from #1341 to perform read of same key from multiple threads (which revealed the bug in the first place).
$ ozone freon ockg -n 1 -p asdf
$ ozone sh key list vol1/bucket1
[ {
  "version" : 0,
  "size" : 10240,
  "keyName" : "asdf/0"
  ...
} ]

$ ozone freon ocokr -k 'asdf/0'
...
         mean rate = 164.39 calls/second
                  ...
              mean = 53.75 milliseconds
            stddev = 42.11 milliseconds
            median = 44.05 milliseconds
...
Total execution time (sec): 6
Failures: 0
Successful executions: 1000

/label ozone

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
48
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
1160
trunk passed

+1
compile
441
trunk passed

+1
checkstyle
88
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
1183
branch has no errors when building and testing our client artifacts.

+1
javadoc
205
trunk passed

0
spotbugs
528
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
774
trunk passed

_ Patch Compile Tests _

+1
mvninstall
673
the patch passed

+1
compile
454
the patch passed

+1
javac
454
the patch passed

+1
checkstyle
86
the patch passed

+1
mvnsite
0
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
785
patch has no errors when building and testing our client artifacts.

+1
javadoc
185
the patch passed

+1
findbugs
738
the patch passed

_ Other Tests _

+1
unit
327
hadoop-hdds in the patch passed.

-1
unit
1785
hadoop-ozone in the patch failed.

+1
asflicense
56
The patch does not generate ASF License warnings.

9172

Reason
Tests

Failed junit tests
hadoop.ozone.container.server.TestSecureContainerServer

Subsystem
Report/Notes

Docker
Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1349

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux cbe1e1f743ac 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 23e532d

Default Java
1.8.0_222

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/1/artifact/out/patch-unit-hadoop-ozone.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/1/testReport/

Max. process+thread count
5398 (vs. ulimit of 5500)

modules
C: hadoop-hdds/container-service U: hadoop-hdds/container-service

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/1/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

/retest

@anuengineer @bshashikant @elek please review

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
77
Docker mode activated.

_ Prechecks _

+1
dupname
0
No case conflicting files found.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 1 new or modified test files.

_ trunk Compile Tests _

+1
mvninstall
641
trunk passed

+1
compile
378
trunk passed

+1
checkstyle
72
trunk passed

+1
mvnsite
0
trunk passed

+1
shadedclient
934
branch has no errors when building and testing our client artifacts.

+1
javadoc
182
trunk passed

0
spotbugs
477
Used deprecated FindBugs config; considering switching to SpotBugs.

+1
findbugs
711
trunk passed

_ Patch Compile Tests _

+1
mvninstall
602
the patch passed

+1
compile
407
the patch passed

+1
javac
407
the patch passed

+1
checkstyle
74
the patch passed

+1
mvnsite
0
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedclient
730
patch has no errors when building and testing our client artifacts.

+1
javadoc
199
the patch passed

+1
findbugs
755
the patch passed

_ Other Tests _

+1
unit
355
hadoop-hdds in the patch passed.

-1
unit
2792
hadoop-ozone in the patch failed.

+1
asflicense
52
The patch does not generate ASF License warnings.

9119

Reason
Tests

Failed junit tests
hadoop.ozone.container.server.TestSecureContainerServer

hadoop.ozone.client.rpc.TestWatchForCommit

hadoop.ozone.client.rpc.Test2WayCommitInRatis

hadoop.ozone.client.rpc.TestOzoneRpcClientForAclAuditLog

Subsystem
Report/Notes

Docker
Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1349

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle

uname
Linux 9d517b259197 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 4379370

Default Java
1.8.0_222

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/2/artifact/out/patch-unit-hadoop-ozone.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/2/testReport/

Max. process+thread count
5234 (vs. ulimit of 5500)

modules
C: hadoop-hdds/container-service U: hadoop-hdds/container-service

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1349/2/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Thanks @adoroszlai for working on this. The locking logic seems good to me.
But in Ozone world, a chunk once written is immutable , and hence we might not need the lock at all
while reading a chunk.
@anuengineer @elek @mukul1987 what do you think?
There is also a race condition which exists in the system where a chunk might get deleted by BlockDeleting service in Datanode while readChunk is happening which is where i think we need to have synchronization on a container level (not precisely at the chunk level) for closed containers.

Thanks @anuengineer and @bshashikant for reviews, and @anuengineer for committing it.


Updating computing related stat on the main homepage

Description
The women in tech stats on the home page require updating. It is currently at 25%, but has since changed since the original stat from 2011.
Ask
There are more current statistics from the Bureau of Labor Department.
To Do

[ ] Update stats to reflect more recent numbers.

Additional Info
The stats on the page are from 2011 there are more recent stats from 2013.
http://www.bls.gov/cps/cpsaat11.htm

I posted the second screen shots so the column headings on the page. Here's the link to the Bureau of Labor website http://www.bls.gov/cps/cpsaat11.htm

Computer Programmers 23.0%
Software Developers, applications & systems software 19.7%
Web Developers 39.5%

@staceyturnage I think we need to be a bit more specific on the numbers you think need to be changed on the homepage. I am going to list the numbers below and would appreciate you specifying what exactly needs to be swapped out where - thanks!

Women hold only 25% of the 3,816,000
Women obtain 58% of bachelor’s degrees in the U.S, but less than 18% of those degrees are in computer science. (Compared to 1984, when women earned 37%)
Tech companies with more women in management have a 34% higher return of investment.

@juliaelman Ok I confirmed the following number should be:
Women hold only 35.2% of web development jobs.
I couldn't find a total number of jobs.

I got the information from the Bureau of Labor Stats. The link is above.

Here's where the current information is located, so you maybe able to have the all up number remain the same but the percentage changed...http://www.ncwit.org/blog/did-you-know-demographics-technical-women

@staceyturnage hrmmm... I doubt that the number of jobs has stayed the same and don't want to assume anything. We need to find another source with the official numbers that we can post on the homepage.
@DBNess would love you input on this and what is a good reliable source to find these numbers (when you get a chance)!

Baffles me why the Bureau of Labor stats doesn't have an total number and maybe it does and I may have missed it but that was my point earlier if the percentage has changed surely the number of jobs has changed as well they go hand in hand.

@corinnepw would love you help on this one to check on updated numbers for the homepage stats. The one's we currently have are for 2013.

Hi all! Got the latest stats from our research partner, National Center for Women & IT:
http://www.ncwit.org/sites/default/files/resources/btn_02282014web.pdf & http://www.ncwit.org/sites/default/files/resources/impactgenderdiversitytechbusinessperformance_print.pdf & http://www.ncwit.org/sites/default/files/resources/ncwitfactsheet.pdf
I'm comfortable using the following updated stats:

Women hold only 26% of the approximately four million computing related occupations in the US.
Women obtain 57% of bachelor’s degrees in the U.S, but fewer than 18% of those degrees are in computer science. (Compared to 1984, when women earned 37%) - I think the displayed percentage number here should be the 18% though, not 57%
Tech companies with more women in management have a 34% higher return of investment. This one remains the same

@corinnepw sold! I vote sendinf to NCWIT resources, but am not sure we need to cite. Going to submit a PR for updating the stats for now until I hear further from you and/or @DBNess!

@juliaelman how about we cite (in a smaller size) and link the citation to NCWIT.
I know we're not using Bootstrap, but this is what I'm thinking scroll down to small headings. Does that make sense?

Women hold only 26% of the approximately four million computing related occupations in the US. U.S. Department of Labor, Bureau of Labor Statistics, 2013 (Occupational Category: 15–0000)
Women obtain 57% of bachelor’s degrees in the U.S, but fewer than 18% of those degrees are in computer science. (Compared to 1984, when women earned 37%) National Center for Education Statistics (NCES), 2012 (CIP 11)
Tech companies with more women in management have a 34% higher return of investment. National Center for Women and IT Fact Sheet

@corinnepw that works for me! Thanks for the detailed docs to clarify the linkage and how you would like it to work. Super helpful!

:raised_hands: looks great!! Thanks, ladies!


Add raw_initial, scaled_initial and raw_invalid to Signal

Closes #534
@andlaus Could you take a look at the failing test? I cannot find the initial value in the arxml file 🤔

You're too fast, it's not ready for review. But thanks for all the hints 😄
Could you help me with this test? Where is this 0 in the arxml file?
https://github.com/cantools/cantools/blob/df21fe3db7144e7e72e9882a661c5d4ea8c1dce7/tests/test_database.py#L4540

You're too fast, it's not ready for review. But thanks for all the hints

don't let yourself be stressed by me ;)

Could you help me with this test? Where is this 0 in the arxml file?

this should be specified as the initial dynamic part alternative for the multiplexer (hopefully):
https://github.com/cantools/cantools/blob/master/tests/files/arxml/system-3.2.3.arxml#L361-L365

@andlaus I'm not sure anymore. Maybe it is indeed better to add raw_invalid and raw_initial, and make initial, invalid, minimum and maximum all scaled without a scaled_ prefix. It will break user code of dbc users, but if we bump the major version number and highlight it in the changelog, it should be acceptable.

Maybe it is indeed better to add raw_invalid and raw_initial, and make initial, invalid, minimum and maximum all scaled without a scaled_ prefix.

+1

It will break user code of dbc users, but if we bump the major version number and highlight it in the changelog, it should be acceptable.

I would support that. If we get too many complaints (ehem), we can add switch over to the prefixes...

I'm happy with this. Do I see this correctly: signal.{minimum,maximum} are not yet handled and the non-ARXML and non-DBC file formats are not adapted yet?

The tests pass 😄
No, i didn't look at other formats to be honest. I will do that tomorrow.
Also, i did not really plan on changing minimum and maximum. The corresponding raw values would only depend on the bit count and data types. Is that really useful?

No, i didn't look at other formats to be honest. I will do that tomorrow.

+1. if they use positional arguments to construct Signal objects it might even work as-is. (we should check before merging.)

Also, i did not really plan on changing minimum and maximum. The corresponding raw values would only depend on the bit count and data types. Is that really useful?

useful: IDK, but for consistency's sake it would IMO be a good idea not to treat these quantities any different from initial and invalid. That said, I'm okay with merging the PR as-is and I'll follow up with a separate PR for minimum and maximum. (I can do this for you if you don't feel like it.)

I don't see any other usage of initial or invalid.

The corresponding raw values would only depend on the bit count and data types.

ah, btw: that's not necessarily true. it is possible to specify a narrower range than the scaled minimum and maximum of the raw value. e.g., if the necessary raw value does not align to a power of two...

The corresponding raw values would only depend on the bit count and data types.

ah, btw: that's not necessarily true. it is possible to specify a narrower range than the scaled minimum and maximum of the raw value. e.g., if the necessary raw value does not align to a power of two...

There's also the question: how do you define the raw min and max? The invalid and initial values are often not included in the physical min and max values. So, do the raw min/max values include intial and invalid or not?

yep, it makes sense to assign these quantities to a named value representing e.g. "Not Available". But as I see it, these are all reasons why signal.{minimum,maximum} should be handled the same as signal.{invalid,initial}?!

So, do the raw min/max values include intial and invalid or not?

I just checked: For the files I have, the invalid value is never included, while for the initial one it depends: sometimes it is the same as invalid, sometimes it is a numeric value...

So, do the raw min/max values include intial and invalid or not?

I just checked: For the files I have, the invalid value is never included, while for the initial one it depends: sometimes it is the same as invalid, sometimes it is a numeric value...

And sometimes it is a separate named init value.
For me, a raw min/max is too ambiguous. As a user, it might not be clear, whether it is a simple conversion of the physical min/max values or if it includes values outside the physical range.

And sometimes it is a separate named init value.

in this case the non-postfixed member of Signal should be set to None while the corresponding `_raw' member should be set to the specified value. (I'd argue, though, that this is a bug of the respective database file, and should produce a warning...)

For me, a raw min/max is too ambiguous. As a user, it might not be clear, whether it is a simple conversion of the physical min/max values or if it includes values outside the physical range.

As far as I understood this, the specified min/max values must always be in the physical range with the corresponding raw values. In some low-level cases -- e.g., writing database files -- one needs to use the raw values, so it would be more convenient if they were directly exposed. The potential confusion should IMO be fixed using the documentation.
On a different note: What I noticed while having a closer look at my files is that many signals are integers in the real world (sequence counters, checksums, etc.). maybe we should allow signal.{scale,offset} to be None and return an integer as the scaled type in this case. Stuff for another PR...

On a different note: What I noticed while having a closer look at my files is that many signals are integers in the real world (sequence counters, checksums, etc.). maybe we should allow signal.{scale,offset} to be None and return an integer as the scaled type in this case. Stuff for another PR...

That should already be covered by this:
        if self.offset == 0 and self.scale == 1:
            # treat special case to avoid introduction of unnecessary rounding error
            return raw

That is the "no scaling" special case 😄

That should already be covered by this:

yeah, but IMO this strikes me as a bit inelegant ("duck typing"). Anyway, let's go this route for now.
I'd like to get this PR merged soonish: did you check the SYM and KCD parsers? if no, shall I help on that front?

I'd like to get this PR merged soonish: did you check the SYM and KCD parsers? if no, shall I help on that front?

They do not use initial or invalid as far as i can see.

They do not use initial or invalid as far as i can see.

okay. feel free to merge, then!


While running the model, facing the error: exception: access violation writing 0x000002B6F404B000

I am trying to run llama2 gguf on windows 11 Version 22H2. I have python 3.11 installed on my local machine.
Below is the code:
import gradio as gr
from ctransformers import AutoModelForCausalLM

def load_llm():
    print("loading")
    try:
        llm = AutoModelForCausalLM.from_pretrained(
            "./llama-2-7b.Q4_0.gguf",
            model_type="llama",
            max_new_tokens=512,
            repetition_penalty=1.13,
            temperature=0.1
            )
    except Exception as e:
        print(e)
        print("failed to load")
        raise e
    else:
        print("loaded")
        return llm

def llm_function(message, chat_history):
    print("message: ", message)
    llm = load_llm()
    return llm(message)

title = "Codellama"
description = "Codellama is a chatbot developed by IsolatedFalcon. Ask me anything!"

# gr.ChatInterface(
#     fn=llm_function,
#     title=title
# ).launch()

llm = load_llm()

# while(True):
#     query = input("You: ")
#     print("Codellama: " + llm(query))
print(llm("What is the capital of India? Tell me something about that."))

and while running the file, i am getting this output

loading
exception: access violation writing 0x0000019A7D97B000
failed to load
Traceback (most recent call last):
  File "D:\llama2\main.py", line 36, in <module>
    llm = load_llm()
          ^^^^^^^^^^
  File "D:\llama2\main.py", line 17, in load_llm
    raise e
  File "D:\llama2\main.py", line 7, in load_llm
    llm = AutoModelForCausalLM.from_pretrained(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama2\llama\Lib\site-packages\ctransformers\hub.py", line 173, in from_pretrained

Kindly let me know if i am missing something

The other time I ran the program, got this output
loading
exception: access violation writing 0x000002BA6BBF2000
failed to load


Improve Subgraph Query & Valuable Data - Get full DT name and other useful information to ape

[Poolinfo Data]
Right now the DT_symbol is coming in without it's code-number at the end.
There is other data that we're currently receiving that is just not useful/friendly for DF decision-making (datatoken/pool/basetoken addresses, etc...)
[Improving Data]

We want to reduce information that is not usesful
We want to increase engagement/performance data that is useful/friendly (below is just a sample list... KISS + focus on low-hanging fruits)

Full DT_symbol
Total Staked
Number of LPers
Number of Token Holders
Dataset Human-Readable Name

DoD:

Add code-number at end of DT_symbol
Add other basic stats: NumLP, NumTokenHolders, TotalOceanStaked
This has been implemented in df-py + df-sql

We need to revisit this as part of the new table.

Obsolete in the face of ve, I believe. Reopen if you see otherwise


Unable to build using Carthage

command: carthage update --platform ios --no-use-binaries
I am attempting to install the current version (4.1.7), but have also tried with the version I was previously using (4.0.4) and the Carthage install is failing. I noticed you just updated the library to remove support for iOS 8, and the error when trying to install is pointing to the DKPhotoGallery project
CompileSwift normal arm64 (in target 'DKPhotoGallery' from project 'DKPhotoGallery')
    cd /Users/<project path>/Carthage/Checkouts/DKPhotoGallery
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -emit-bc /Users/<project path>/Carthage/Checkouts/DKPhotoGallery/DKPhotoGallery/Preview/D$
/Users/j<project path>/Carthage/Checkouts/DKPhotoGallery/DKPhotoGallery/Preview/DKPhotoBasePreviewVC.swift:38:38: error: 'UIPreviewActionItem' is only available in iOS 9.0 or newer
    func defaultPreviewActions() -> [UIPreviewActionItem]
                                     ^
/Users/<project path>/Carthage/Checkouts/DKPhotoGallery/DKPhotoGallery/Preview/DKPhotoBasePreviewVC.swift:38:10: note: add @available attribute to enclosing instance method
    func defaultPreviewActions() -> [UIPreviewActionItem]
         ^
/Users/<project path>/Carthage/Checkouts/DKPhotoGallery/DKPhotoGallery/Preview/DKPhotoBasePreviewVC.swift:18:17: note: add @available attribute to enclosing protocol
public protocol DKPhotoBasePreviewDataSource : NSObjectProtocol {
                ^
/Users/<project path>/Carthage/Checkouts/DKPhotoGallery/DKPhotoGallery/Preview/DKPhotoBasePreviewVC.swift:97:41: error: 'UIPreviewActionItem' is only available in iOS 9.0 or newer
    private var _customPreviewActions: [UIPreviewActionItem]? {
                                        ^
/Users/<project path>/Carthage/Checkouts/DKPhotoGallery/DKPhotoGallery/Preview/DKPhotoBasePreviewVC.swift:97:17: note: add @available attribute to enclosing property
    private var _customPreviewActions: [UIPreviewActionItem]? {
                ^
/Users/<project path>/Carthage/Checkouts/DKPhotoGallery/DKPhotoGallery/Preview/DKPhotoBasePreviewVC.swift:85:12: note: add @available attribute to enclosing class
open class DKPhotoBasePreviewVC: UIViewController, UIScrollViewDelegate, DKPhotoBasePreviewDataSource {

It goes on like that for a little longer, but it looks like the DKPhotoGallery project did not get the same update that the main project did.
I also tried downloading your full project from GitHub and trying to run it by building all of the dependencies locally on my machine, but I get the same error.

Sorry for that 😢 Fixed in DKPhotoGallery 0.0.16.

i upated it but still have problem with DKPhotoGallery
dyld: Library not loaded: @rpath/SDWebImageFLPlugin.framework/SDWebImageFLPlugin
Referenced from: /private/var/containers/Bundle/Application/61/test.app/Frameworks/DKPhotoGallery.framework/DKPhotoGallery
Reason: image not found
(lldb)


Fix missing closing quotes in Local dependencies part of language-reference doc

Fix missing closing quotes in 'Local Dependencies' section of index.adoc to ensure syntax correctness.

Thank you!


refactor(controllers): set conditions on top level

@byashimov yes, you were right! :laughing:

@byashimov yes, you were right! 😆

Hey. What do you mean? :)

@byashimov yes, but i only see the issue for KafkaACL.
We're only using kafka-related stuff though. What works is ServiceUser and KafkaTopic.

Ah i see, i was talking "Status Conditions" so different thing but the symptom seems the same.


For some reason the plugin not executed on startup

And I don't know why. I just doesn't run. I have orphans to clean but it doesn't clean them on startup, regardless if I select with or without confirmation.
Tested a few times, created a note,  added a PDF, then deleted note but left PDF untouched, then run the plugin manually, it offers to delete them.  But if I want the plugin to find the orphans and delete them on start of Obsidian, it doesn't.
Not sure what is going on.

Hi, thanks for raising the issue, I might have an idea as to what it could, currently the plugin uses an arbitrary wait time of 1 second from being loaded until it runs the cleanup, this might be affected how long it takes for Obsidian to start up.
Could you try to close down the vault, then re-open it.
After it has finished indexing, open up the Developer Console by pressing (Control + Shift + I / Command + Shift + I on Mac)
You should see see something along the lines of the following

If you do not see the lines, could you try to run the cleanup process while having the console open, this should generate the output, once you see it, copy the line saying File Cleaner Redux: Finished indexing X files and Y folders in Zms.


Track withdrawn ZETA

Part of https://github.com/zeta-chain/node/issues/643
Track and control ZETA withdrawal for security and check that the ZETA deposited back doesn't exceed withdrawn ZETA.
Solutions:

Add a store entry to track withdrawn ZETA per chainID
(Personally preferred) define Escrow addresses per chaindID and move withdrawn ZETA into this escrow instead of burning the token (this is the solution used by IBC transfer https://github.com/cosmos/ibc-go/blob/2ac55069ad479838f93d9f4000f526a944196236/modules/apps/transfer/keeper/relay.go#L108)

Implementation:

Add the tracker system
Add check when depositing ZETA
Would be interesting: track the total withdrawn value and implement an invariant that checks ∑(withdrawn) == totalWithdrawn

@brewmaster012 to get a confirmation, is it something that can be considered as part of the invariance checking tasks for defensive programming? https://github.com/zeta-chain/node/issues/840


E2E testing and tooltip hover

Provide a general summary of the issue here
Hello,
I'm using tooltip from @nextui-org/react that uses @react-aria/tooltip internally (specifically useTooltipTrigger).
While trying to verify that tooltip opens in my tests (using playwright) I stumbled upon tooltip not getting opened on hover while testing.
The diff that is happening is that the tooltip state always has prop open: false and never shows up in playwright's browser.
It is happening because of the useTooltipTriggers onHoverStart function that checks interactionModality and doesn't open tooltip if it's not pointer.
Real browser sets interactionModality to pointer whenever hover is happening on a document, but testing software doesn't happen to do that.
When I add additional clicking in the browser before first hover - it works as intended.
🤔 Expected Behavior?
I expect playwright's browser hover make tooltip open - without additional clicking.
😯 Current Behavior
Tooltip is not opened if no click (move?) was made before in playwright.
💁 Possible Solution
No response
🔦 Context
No response
🖥 Steps to Reproduce

Add a tooltip on a page
(in playwright test) Hover the tooltip
(in playwright test) Verify tooltip exists

Version
3.20.0
What browsers are you seeing the problem on?
Chrome
If other, please specify.
No response
What operating system are you using?
MacOS
🧢 Your Company/Team
No response
🕷 Tracking Issue
No response

When I add additional clicking in the browser before first hover - it works as intended.

I think this is working as expected, since this is how it works in a real browser currently. (i.e. open https://react-spectrum.adobe.com/react-aria/Tooltip.html and click on some other window or your desktop, then try hovering the tooltip).
If the issue is mainly excess verbosity in tests, I would suggest putting this as setup inside test.beforeEach/test.beforeAll.
I think changing this behavior could potentially have unintended consequences in places where we are relying on checking for pointer modality.

Correct, we cannot know all possible events and data that test utilities fire. Especially ones that are an abstraction to be close to user events. We work around userEvent in the same way you are describing.
https://github.com/adobe/react-spectrum/blob/cd91f8a55f4f3ab21011fffb21408c2b0ea98b0f/packages/%40react-spectrum/tooltip/test/TooltipTrigger.test.js#L446
https://github.com/adobe/react-spectrum/blob/cd91f8a55f4f3ab21011fffb21408c2b0ea98b0f/packages/%40react-spectrum/tooltip/test/TooltipTrigger.test.js#L500
I'm going to close this as working as intended, though it might be something useful to add to the discussion in https://github.com/adobe/react-spectrum/pull/5508


PASSPORT WITH SERVERLESS FUNCTIONS [VERCEL]

Is it just me!?
I cant get passport with its local & jwt strategies to work with serverless functions.
Previously, when working traditionally with MEVN stack I had no issues.
PLIZ ASSIST
The best way to assist is to provide a simple working example of a serverless function with passport authentication

The best way to assist is to provide a simple working example of a serverless function with passport authentication

The best way to assist is to provide a simple working example of a serverless function with passport authentication

+1 for this - @jaredhanson
Best I could find was Implementing an Authentication Flow with Passport and Netlify Functions, where the solution is to spin up Express and adapt to the function with serverless-http.
Why not factor out the Oauth functionality and provide by default, but optionally, ExpressJS middleware adapter?  Is there something else that would better fit this use case?


Refactor schema access into separate class

This PR implements a big refactor that's been in our backlog for a while - adding a Schema() class that encapsulates basic manipulation of the config dictionary.
Rationale
The primary value I see in this change is to give us an alternative for the cfg keyword arguments that many of the schema manipulation functions had. These were being used to manipulate config dictionaries other than the current Chip's (for example, when merging in a dictionary from a manifest). However, these keyword arguments go against our object-oriented model, and handling them incorrectly can result in subtle bugs like  #1122.
This refactor also allows for low-level config manipulation without accessing the dictionary directly. We do this a lot in the docs generation and the server-side code, but this is a bit clunky, and abstracting with an API will enable us to make changes to the underlying dictionary representation (which will let us fix other issues down the road, such as #1146).
Another benefit is this helps organize our codebase a bit and keep down the size of our bloated core.py. I also see this as a chance to push for improved coding style, and I'm hoping to revisit an incremental approach to linting in CI starting with this new file.
Despite citing this as a key thing in my original proposal, I don't see design name bootstrapping as a key reason for this change (although this will help us clean some hacks in the doc generator). Rather, I think my proposal in #1169 will cover that well.
API changes
One goal of this change was to make it a pure refactor with minimal-to-no API changes. The one exception is the removal of the 'cfg` keyword argument from all functions that used it. However, I did take a few other opportunities to clean up the API, here are the changes:

valid()

Get rid of quiet keyword argument
Schema() doesn't have access to logger, making this tricky to implement, but the non-default value was only used in one place

write_manifest()

Get rid of job keyword argument - this was unused

record_history()

Moved fully into Schema(), since it seems to only be used internally at the end of run()
Open to reconsidering this

Follow-up PRs

Clean up docs gen to take advantage of Schema()
Clean up server code to take advantage of Schema()
Add linting to CI for schema_obj.py
Refactor fundamental schema functions to clean up, fix latent bugs (tuple handling), improve performance (valid), and improve error messages
Fix default value issue(s) (#1146,  #1150)

Yep!
On Thu, Dec 1, 2022 at 9:31 PM Andreas Olofsson @.***>
wrote:

Is this ready for review?
—
Reply to this email directly, view it on GitHub
https://github.com/siliconcompiler/siliconcompiler/pull/1147#issuecomment-1334686747,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBVIK4U2ETCY6O2AEE7PYTWLFNPHANCNFSM6AAAAAAQPXNXDE
.
You are receiving this because you authored the thread.Message ID:
@.***>


fix(content): dispatch load event if wallet address is valid

Context

Closes #365.
Changes proposed in this pull request

Additional changes:

small refactoring for the wallet address check
update the checkWalletAddressUrlFormat function since we do not need to check for SPSP endpoints

Extension builds preview

Name
Link

Latest commit
1b47918ca607a6141c63003e7657262a8f51b2c0

Latest job logs
Run #9661173418

DownloadDownload