Add a bounded SPSC queue

Inspired by the Rust 2019: Rust Audio thread, I decided to add a high-performance wait-free bounded SPSC queue. This is the most common queue type in audio programming and it's very important it's wait-free and as fast as possible.
cc @engid @raphlinus Pinging you if interested.
Simple benchmarks (ArrayQueue vs SegQueue vs spsc, smaller is better):
bounded_seq               Rust ArrayQueue   0.176 sec
bounded_spsc              Rust ArrayQueue   0.104 sec

unbounded_seq             Rust SegQueue     0.236 sec
unbounded_spsc            Rust SegQueue     0.141 sec

bounded_seq               Rust spsc         0.031 sec
bounded_spsc              Rust spsc         0.029 sec

I cannot say thank you enough, but I'll try: Thank you!!!

This is fantastic. Thanks!

Thanks, that would be a way to go now :tada:

Great work! What's the status on this?

The two SPSC ringbuffer implementations I know have a way to read/write multiple "items" at a time (and a way of checking how much space is there to read/write). Would it be feasible to add such a feature to this implementation?
The main use case would be if you have a ringbuffer of audio samples (e.g. T = f32) you can read/write blocks of different lengths from/to the ringbuffer (and you don't need a separate pop()/push() operation for each single audio sample).
Here's two examples:
The JACK ringbuffer (docs, code) has jack_ringbuffer_read_space() and jack_ringbuffer_write_space() to check the available size and jack_ringbuffer_get_read_vector() and jack_ringbuffer_get_write_vector() to get pointers to memory where data can be read from or written into.
The PortAudio ringbuffer (docs, code) has PaUtil_GetRingBufferReadAvailable(), PaUtil_GetRingBufferWriteAvailable(), PaUtil_GetRingBufferReadRegions() and PaUtil_GetRingBufferWriteRegions() with pretty much the same functionality.
I guess instead of (unsafe) raw pointers it would make more sense to provide (safe) &[T] slices for reading and &mut [T] for writing.

@alexbool The PR is blocked mostly by the fact that I'm not sure what changes to anticipate in the future and how to structure the modules.
Right now I'm thinking we should perhaps move the SPSC queue (types Producer and Consumer) into a submodule array::spsc. Then, later in the future, we might add SPMC and MPSC versions of array-based and segment-based queues too.
So the idea is to structure the crate like this:
mod array {
    mod spsc {
        struct Producer<T>;
        struct Consumer<T>;
        fn new(cap: usize) -> (Producer<T>, Consumer<T>);
    }

    mod mpsc {
        // ...
    }

    mod spmc {
        // ...
    }

    struct ArrayQueue<T>; // mpmc version
}

mod seg {
    mod spsc {
        struct Producer<T>;
        struct Consumer<T>;
        fn new() -> (Producer<T>, Consumer<T>);
    }

    mod mpsc {
        // ...
    }

    mod spmc {
        // ...
    }

    struct SegQueue<T>; // mpmc version
}

use array::ArrayQueue;
use seg::SegQueue;

How does that look?

@mgeier Interesting!
I wonder how should we expose a safe interface for that in Rust. Note that we can't just return &[T] and &mut [T] for reading and writing - there are multiple problems with that:

Reading and writing from/into uninitialized memory is possible.
There is a risk of data races because we could read from &[T] while the producer thread is writing into the slice.

It's a bit tricky, but we can probably figure out a reasonable interface...
Another option is to expose methods for batched reads and writes, e.g.:
impl<T> Consumer<T> {
    fn read_batch(&self, dest: &mut Vec<T>);
}

impl<T> Producer<T> {
    fn write_batch(&self, src: &mut Vec<T>);
}

@stjepang I'm a total Rust beginner and I'm not at all an expert on lock-free data structures. Please bear that in mind!
I didn't think about uninitialized memory. The implementations I was mentioning just operate on plain bytes of memory, I guess that's not a serious approach for Rust, right?
Would it be possible to somehow default-initialize the whole underlying Vec<T>?
I guess it would be fine if this feature were only available for a somewhat "trivial" subset of types T. Mostly types like f32 would be interesting.

There is a risk of data races because we could read from &[T] while the producer thread is writing into the slice.

If a user requests a "read slice", this memory area must of course be "blocked" for writing (and for other readers as well, but it is a "single reader" queue anyway).
In the JACK example, the function jack_ringbuffer_get_read_vector() provides access to all "items" that can currently be read from the ringbuffer. But until jack_ringbuffer_read_advance() is called, no writes are possible to this area of memory (but there may still be some other memory left to write to).
In the PortAudio example, callers of the function PaUtil_GetRingBufferReadRegions() can specify how many "items" they want to be reading (but they may actually get less). Those "items" are not available for writing until PaUtil_AdvanceRingBufferReadIndex() is called.
I would hope that with the lifetimes of Rust slices, it would not be necessary to call a separate function when finished accessing the data. But of course I have no idea if this would actually work.

Another option is to expose methods for batched reads and writes

Do read_batch() and write_batch() really need Vec in their API?
Wouldn't that also works with slices?
Anyway, this may force an unnecessary copy of data in some cases, because you need the data to live in some contiguous memory before/after the write/read operation.
If you have e.g. strided data (e.g. interleaved audio channels), you'll have to make an additional copy.
OTOH, if you get a &mut [T], you could directly write into it from your strided values.

@mgeier Here's another try - how about the following interface?
impl<T> Consumer<T> {
    fn pop_batch(&mut self) -> PopBatch<'_, T>;
}

impl<T> Producer<T> {
    fn push_batch(&mut self) -> PushBatch<'_, T>;
}

struct PopBatch<'a, T>;

impl<T> PopBatch<'_, T> {
    fn len(&self) -> usize;
    fn pop(&self) -> Result<T, PopError>;
}

struct PushBatch<'a, T>;

impl<T> PushBatch<'_, T> {
    fn len(&self) -> usize;
    fn push(&self) -> Result<(), PushError<T>>;
}

The idea is that we start a batch by calling pop_batch() or push_batch(), then do a number of pop/push operations, and finally drop the PopBatch/PushBatch type to update the atomic indices inside the queue.
This way each individual pop/push operation in a batch will only do a single memcpy to transfer the value, whereas each regular non-batched pop/push operation updates the atomic indices every single time.
Would this work for you? Or is it really important that you're able to directly access contiguous slices of memory inside the queue?

We could then also add ability for the batches to access direct slices of memory when T is a trivial type (i.e. it implements Copy):
impl<T: Copy> PopBatch<'_, T> {
    fn as_slice(&self) -> &[T];
    fn advance(&mut self, n: usize);
}

impl<T: Copy> PushBatch<'_, T> {
    fn as_slice(&mut self) -> &mut [T];
    fn advance(&mut self, n: usize);
}

For example, in order to push a bunch of elements, we would call push_batch(), grab a slice into the queue with as_slice(), then write data into the slice, and call advance(n) to move the write index by n positions forward.

Thanks @stjepang, this looks very promising!
But we would still need two slices, right?
I guess something like (probably using the plural as_slices()?):
fn as_slice(&mut self) -> (&mut [T], &mut [T]);

I don't think that the additional push() and pop() methods of PushBatch and PopBatch objects are really needed. At least I cannot see a real use case. I think such a thing should only be provided if there is need for it in the community.
If anything, I think some kind of a "peek" functionality would be more interesting. But for the cases where the "slices" functionality is available, a separate "peek" method is not necessary, because we can simply "peek" into the slices.

and finally drop the PopBatch/PushBatch type to update the atomic indices inside the queue.

Assuming that len(), push() and pop() are not really needed, do we actually need those separate types?
What if advance() directly updates the atomic indices?
I guess it wouldn't make too much sense to call advance() multiple times (without updating the indices) and then update the indices when dropping PopBatch/PushBatch. Also, calling advance() (without updating the indices) and afterwards as_slice() again doesn't make much sense, either, right?
When I hear advance(), I expect some atomic operations to happen ...
If this works:
{
    let mut mybatch = myqueue.push_batch();
    let (slice1, slice2) = mybatch.as_slice();
    // write to slices
    mybatch.advance(some_amount);
}

... shouldn't something like this also work?
let (slice1, slice2) = myqueue.write_slices();
// write to slices
myqueue.advance_write_index(some_amount);

(I probably missed a mut somewhere)

Or is it really important that you're able to directly access contiguous slices of memory inside the queue?

Well I don't know if it would be that important for me personally. But if we want to convince low-level audio programmers to switch to Rust, we should try to provide library constructs with as little overhead as possible. Having to make unnecessary copies of audio data is not a good selling point!

@mgeier Thanks for your patience, I really appreciate it!
After thinking a bit more about it, I believe we could implement the following API:
impl<T: Copy + Default> Producer<T> {
    fn write_slices(&mut self) -> (&mut [T], &mut [T]);
    fn advance(&self, n: usize);
}

impl<T: Copy> Consumer<T> {
    fn read_slices(&self) -> (&[T], &[T]);
    fn advance(&self, n: usize);
}

Two tricks that are necessary for ensuring safety:

The two advance() functions will check whether n is valid, i.e. they will make sure the index doesn't go too far (or else panic occurs).

Function write_slices() will initialize the slices with T::default() values if they would otherwise hold uninitialized data. Note that if they hold some data that has been written and subsequently read, we don't have to do anything.

Neither of these two should really be a performance hit since they can be optimized away very well.
Would you be happy with this API?

Alternative 1: write_slices() could return slices of type &mut [MaybeUninit<T>], which would totally sidestep the problem of uninitialized memory. But then users of the API would have to deal with MaybeUninit, which could be a little annoying.
Alternative 2: It would also be possible for write_slices() to return ((*mut T, usize), (*mut T, usize)), but I suppose you'd rather not have the users of the API be forced to write unsafe code.

Would you be happy with this API?

Yes, thanks, that looks very nice!
Please excuse my ignorance, but I have several more questions:
One open question for me is whether there should be additional convenience functions. Getting those pairs of slices is the most important functionality because it is the lowest-level interface. But probably there should also be convenience functions like the read_batch() and write_batch() functions you suggested above?
I'm definitely biased, because the JACK and PortAudio ringbuffers have such functions as well.
Probably it's not necessary to add more functions until somebody requests them?
As I said, I'm a Rust beginner, so I cannot really tell which interface makes sense and which doesn't.
Would it be feasible to use iterators on top of the suggested API?
Something like creating a mutable iterator for each of the returned slices, chaining the two and then assigning values via iteration?
Or should there be a separate API for iterators?
I'm also not sure if the functions read_available() and write_available() are missing or not.
Of course, one could call read_slices() and check their combined length, but probably a separate function for that would simplify things?
I guess it is a common use case to regularly check if the ringbuffer has enough data available and only get the slices when there's enough data.
Another open question is the T::default() thing you described above.
It seems strange to me that write_slices() would sometimes call T::default() and sometimes it wouldn't.
I'd like to believe you when you say that this can be optimized away, but it makes me feel a bit uneasy.
Wouldn't it be possible to initialize the whole memory in the constructor?
I guess if only push() is used, the memory doesn't have to be initialized and default-initialization would cause unwarranted overhead in that case, right?
TBH, I didn't know MaybeUninit and still don't really understand it. But I don't think it would simplify the usage, right? And it must have a performance penalty, right?
Finally, I'm wondering whether the push()/pop() and the write_slices()/read_slices() APIs will ever be used on the same object. It seems to me that I would want to use either one or the other API, depending on whether I want to use chunks of (audio) data or some higher-level user-defined structs.
Would it make sense to have two different spsc types (which would share a large part of the implementation)?

Would it be feasible to use iterators on top of the suggested API?
Something like creating a mutable iterator for each of the returned slices, chaining the two and then assigning values via iteration?

You could do that with the standard iterator API:
let (s1, s2) = p.write_slices();
let iter = s1.iter_mut().chain(s2.iter_mut());

for slot in iter {
    // do something
}

p.advance(/* some number */);

Or with a slightly different API, similar to one of those I proposed before:
let mut batch = p.batch();

for v in my_values_to_push {
    batch.push(v);
}

drop(batch); // advance the write_index by the number of pushed items

I'm also not sure if the functions read_available() and write_available() are missing or not.
Of course, one could call read_slices() and check their combined length, but probably a separate function for that would simplify things?

I just assumed those functions are not needed because you can compute them from .len() and .capacity(). But then again, it doesn't hurt to also have .read_available() and .write_available(). So maybe we should add them, too. Good idea!

It seems strange to me that write_slices() would sometimes call T::default() and sometimes it wouldn't.
I'd like to believe you when you say that this can be optimized away, but it makes me feel a bit uneasy.
Wouldn't it be possible to initialize the whole memory in the constructor?

We would zero-initialize the whole buffer inside the constructor. Note that we can't initialize with T::default() because the constructor doesn't require T: Default. In order to be able to initialize with the default value, we'd need a special constructor that does require T: Default.
Fortunately, almost every piece of data that implements Copy is zero-initializable. We can check whether T::default() really does return a zeroed value of type T, and if so, write_slices() doesn't have to do anything because the whole buffer was zeroed in the constructor. This check will happen during compile time because the compiler is sufficiently smart.
If T::default() doesn't return a zeroed value (and we don't have the special constructor that requires T: Default), then it gets tricky. We only need to set the slots in slices (returned by write_slices()) to T::default() if they've never been written to before. You can think of this as lazy initialization of the buffer - instead of doing it during construction, we do it lazily on write_slices(). Also note that if you call write_slices() twice in succession, only the first call initializes memory.
So yeah, it's really just a bunch of boring and pedantic safety stuff. But the overall effect on performance is zero in 99% cases and minimal in the remaining 1% cases. So no big deal.

TBH, I didn't know MaybeUninit and still don't really understand it. But I don't think it would simplify the usage, right? And it must have a performance penalty, right?

MaybeUninit<T> has literally zero cost and is just a simple wrapper type indicating that the piece of T may be uninitialized. So if you want to read from it, you must open an unsafe {} block and call .read() on it, promising you're 100% sure the data inside is initialized.
Fortunately, writing into an MaybeUninit<T> is perfectly safe. And that's great because we don't want to read anything from [MaybeUninit<T>] slices returned by write_slices() anyway. We only want to write into them!
Again, more boring pedantic stuff. We're just making sure that if the user really attempts to do something silly like read from slices returned by write_slices(), nothing explodes. :)

Finally, I'm wondering whether the push()/pop() and the write_slices()/read_slices() APIs will ever be used on the same object. It seems to me that I would want to use either one or the other API, depending on whether I want to use chunks of (audio) data or some higher-level user-defined structs.
Would it make sense to have two different spsc types (which would share a large part of the implementation)?

It's probably true that one would either use single-value or batched operations. But does it hurt if we support both? So if we support both, push()/pop()will always be available, whilewrite_slices()/read_slices()will be available only whenT: Copy`.
My feeling is that it makes sense to have different types only if supporting both types of operations simultaneously somehow hurts performance or increases the risk of bugs. But that doesn't seem true to me. Is my thinking correct here?

So writing to MaybeUninit<T> is safe and without overhead, but users still have to use the .write() method all the time, right?

Correct.

But I think the bigger problem is that we cannot force users to .write() to all the items before they advance(), so we would still have to check for "unwritten" items and call T::default() before being able to read from them, right?
If I understand correctly, we therefore don't really gain anything from MaybeUninit<T> (except a worse user experience).

Oh, I forgot about that! You're totally right.

Why exactly does the Consumer need Copy?

Because otherwise we can't "take" items from the slices returned by read_slices(). Suppose you have a &[T] and T is not Copy. You can't do let x = slice[0] to take a value from the slot, that won't compile. You might instead clone the value from the slot with let x = slice[0].clone() if T is Clone. But sometimes cloning is expensive.
An alternative to that would be to put every T inside an Option so that read_slices() returns slices of type &mut [Option<T>]. Then you can do let x = slice[0].take().unwrap() to take a value from a slot. But using options like that incurs time and space overhead, which you probably don't want.

But Producer::is_empty() and Consumer::is_full() are superfluous, right?

They exist in case someone needs them. shrug Lots of collection types in Rust have methods .len(), .capacity(), .is_empty(), and .is_full() so I included them here, too. It's just a common idiom.

I think it would be perfectly safe to mix the two APIs, I just think there would be no real use case for that.

I'm thinking maybe it's good to have push() and pop() because they're a bit easier to use. Like, one might want to use them to quickly code up a prototype and then later on switch to batched reads/writes for that extra performance.

You can't do let x = slice[0] to take a value from the slot, that won't compile.

Sure, but isn't that the user's problem for trying to move a non-Copy thing out of a (borrowed) slice?

You might instead clone the value from the slot with let x = slice[0].clone() if T is Clone.

Sure, but what I would be more interested in (given some non-trivial user-defined type) is taking a reference let x = &slice[0], which should work for unconstrained T, right?
This way I could do something like:
let x = &slice[0]
if x.some_attribute > 42 {
    let x = consumer.pop();
    // do something with x
} else {
    // don't pop()
}
// don't call consumer.advance()!

AFAICT, getting a reference to a non-Copy element in the slice should work. Is there another reason for requiring Copy?
BTW, this would be an example where the two APIs would be mixed. And I think this would be a somewhat reasonable scenario.

But using options like that incurs time and space overhead, which you probably don't want.

No, I would definitely not want to be forced to use Option<T> in this situation.

Lots of collection types in Rust have methods .len(), .capacity(), .is_empty(), and .is_full() so I included them here, too.

As I said, I can see how .is_empty(), and .is_full() might be useful even if they are not strictly necessary.

It's just a common idiom.

Yeah, but the spsc queue isn't really a "typical" collection type.
It's more akin to a std::sync::mpsc::channel (which has none of those methods), isn't it?
I would go so far as to say that providing .capacity() might be reasonable (because we might have forgotten what we specified in the constructor), but .len() really doesn't make any sense (given that we have "read available" and "write available")!
BTW, calling the constructor argument cap is inconsistent with Vec::with_capacity(capacity: usize)!
But it looks like this mistake has been made before in crossbeam::queue::ArrayQueue and crossbeam::channel::bounded. So I guess it has to be cap to at least have internal consistency.

Like, one might want to use them to quickly code up a prototype and then later on switch to batched reads/writes for that extra performance.

You are right, this could happen.

I forgot something: I still think Producer::is_empty() and Consumer::is_full() are superfluous. At best they are useless, but they may be actually confusing and are definitely a waste of space and time for everyone.
I've seen that crossbeam::channel::Sender::is_empty() and crossbeam::channel::Receiver::is_full() also exist. I also think it was a mistake to create those functions, but I understand if you want to have those for spsc for internal consistency, however useless they might be.

Sure, but isn't that the user's problem for trying to move a non-Copy thing out of a (borrowed) slice?

Yeah, just wanted to point out that we'd either have to support only Copy types or always wrap T inside Option. :)

AFAICT, getting a reference to a non-Copy element in the slice should work. Is there another reason for requiring Copy?

Oh, I see now. So you just want to read values by reference and not really take them out (i.e. "consume" them by value). I just kinda assumed you'd always want to consume the read values.

It's more akin to a std::sync::mpsc::channel (which has none of those methods), isn't it?

Heh, it does actually :) See Sender and Receiver.

But it looks like this mistake has been made before in crossbeam::queue::ArrayQueue and crossbeam::channel::bounded. So I guess it has to be cap to at least have internal consistency.

We should just rename those arguments then. The names only appear in the generated docs and are not really a part of the API anyway.

I forgot something: I still think Producer::is_empty() and Consumer::is_full() are superfluous. At best they are useless, but they may be actually confusing and are definitely a waste of space and time for everyone.

That's fair enough! I guess we could remove them then as they might be a net negative overall.

Oh, I see now. So you just want to read values by reference and not really take them out (i.e. "consume" them by value). I just kinda assumed you'd always want to consume the read values.

Well, I want to consume them eventually.
But I might want to "peek" at some of the items beforehand to check if I want to consume them already or probably wait for more data.
I haven't actually used such a functionality, but I might at some point. The use case would be a message queue where I have special messages "start bundle" and "stop bundle". All other messages between those should happen at the same time (= in the same audio block). Whenever I "peek" a "start bundle", I check if I also find a "stop bundle" in the queue before I handle all messages. If there is no "stop bundle", I just wait until the next audio block. I'm not sure if such a thing would work reliably, for now it's just an idea ...
Having said all this, isn't it generally a good idea to keep the trait bounds to a minimum?
Is there a reason to add T: Copy if it's not actually necessary?

Heh, it does actually :)

No it doesn't!
You are talking about crossbeam_channel:
https://docs.rs/crossbeam-channel/0.3.8/crossbeam_channel/struct.Sender.html
https://docs.rs/crossbeam-channel/0.3.8/crossbeam_channel/struct.Receiver.html
I'm talking about std::sync::mpsc::channel:
https://doc.rust-lang.org/std/sync/mpsc/struct.Sender.html
https://doc.rust-lang.org/std/sync/mpsc/struct.Receiver.html

We should just rename those arguments then.

OK, that sounds good.
Interestingly, the names aren't even consistent in Rust itself: https://github.com/rust-lang/rust/issues/60271

I guess we could remove them then as they might be a net negative overall.

If at a later point somebody actually needs them (which I doubt will ever happen), it should be easy to add them. Removing them later is probably not that easy.

I haven't actually used such a functionality, but I might at some point. The use case would be a message queue where I have special messages "start bundle" and "stop bundle".

Very interesting! I'll think about this use case some more - but pretty sure we can devise something to support it well.
Here's an idea. We could have two methods:

read_slices() that returns (&[T], &[T]) as usual.
pop_iter() that returns PopIter<T>, which implements Iterator<Item = T>. You can use it to take elements out one by one. The read index is finally updated once you stop iterating (i.e. drop PopIter<T>) so that only one atomic operation actually ocurrs under the hood.

No it doesn't!
You are talking about crossbeam_channel:

Sorry, my bad! FWIW, .len() and .capacity() are useful in crossbeam-channel because it allows Servo to estimate how much memory the channel is currently using.

Your PopIter idea sounds really interesting! It looks like a more ergonomic version of your PopBatch::pop() idea above (https://github.com/crossbeam-rs/crossbeam/pull/338#issuecomment-485367921).
It may seem a bit asymmetric to have only Consumer::pop_iter() and nothing on the Producer side.
OTOH, iterating over the Producer can easily be implemented on top of the write_slices() function, as you have shown in https://github.com/crossbeam-rs/crossbeam/pull/338#issuecomment-485952222. It would also be easy to create a "peek" iterator (with Item = &T) in user code. So I don't think it's really worth adding more iterators to the API.

FWIW, .len() and .capacity() are useful in crossbeam-channel because it allows Servo to estimate how much memory the channel is currently using.

As I said above, I think .capacity() might be OK (but I guess it will not be heavily used). But Servo doesn't need .len(), right?

Okay, to summarize, the Consumer side has (we can discuss final naming choices later):
impl<T> Consumer<T> {
    fn read_available(&self) -> usize;
    fn read_slices(&self) -> (&[T], &[T]);

    fn pop(&mut self) -> Result<T, PopError>;
    fn pop_batch(&mut self) -> PopBatch<'_, T>;
}

struct PopBatch<'a, T>;

impl<T> PopBatch<'_, T> {
    fn read_available(&self) -> usize;
    fn pop(&mut self) -> Result<T, PopError>;
}

impl<T> Iterator for PopBatch<'_, T> {
    type Output = T;
}

The producer side is analogous, except we don't implement Iterator for PushBatch and don't have write_slices() due to problems with uninitialized memory:
impl<T> Producer<T> {
    fn write_available(&self) -> usize;

    fn push(&mut self, val: T) -> Result<(), PushError<T>>;
    fn push_batch(&mut self) -> PushBatch<'_, T>;
}

struct PushBatch<'a, T>;

impl<T> PushBatch<'_, T> {
    fn write_available(&self) -> usize;
    fn push(&mut self, val: T) -> Result<(), PushError<T>>;
}

we [...] don't have write_slices() due to problems with uninitialized memory

Based on your comments above, I thought those problems could be solved!
I was expecting something like this (which you suggested in https://github.com/crossbeam-rs/crossbeam/pull/338#issuecomment-485746068):
impl<T: Copy + Default> Producer<T> {
    fn write_slices(&mut self) -> (&mut [T], &mut [T]);
    fn advance(&self, n: usize);
}

And I'm missing advance() in the Consumer:
impl<T: Copy> Consumer<T> {
    fn advance(&self, n: usize);
}

The big (and I guess only) difference to your original suggestion is that Consumer::read_slices() doesn't need T: Copy.
I have no opinion on that, but your push()/pop() signatures seem to have changed from &self to &mut self, see also https://github.com/crossbeam-rs/crossbeam/pull/338/files#r264038883.
I was a bit confused by the seemingly unnecessary duplication of Consumer::read_available() and PopBatch::read_available(), but I think I understand it now.
Just to make sure my understanding is correct:

Between calling Consumer::pop_batch() and dropping the returned PopBatch object, it is not possible to call Consumer::read_available(). Therefore, we need an additional PopBatch::read_available().
For each successful call to PopBatch::pop(), the value returned from PopBatch::read_available() decreases, even though the atomic read index hasn't been changed yet.
When Consumer::pop_batch() is called, the atomic write index is cached and not checked anymore for subsequent PopBatch::pop() calls.
Having a single PopBatch object, I can switch between calling .pop() and iterating over it and calling .pop() again. The value returned from PopBatch::read_available() decreases each time.
In summary, a sequence of statements like this would work (and the commented ones wouldn't compile), assuming there are enough items available for reading:let n1 = consumer.read_available();
let (slice1, slice2) = consumer.read_slices();
// read some values from the slices (or not)
// don't call consumer.advance()!
let batch = consumer.read_batch();
let n2 = batch.read_available();  // n1 == n2, unless someone has written to the Producer
// wait a bit
let n3 = batch.read_available();  // n2 == n3, even if someone has written to the Producer
let x = batch.pop();
let n4 = batch.read_available();  // n4 == (n3 - 1), even if someone has written to the Producer
let x = batch.next();
let n5 = batch.read_available();  // n5 == (n4 - 1), even if someone has written to the Producer

//let n = consumer.read_available();  // Compiler error
//let (slice1, slice2) = consumer.read_slices();  // Compiler error
//let x = consumer.pop();  // Compiler error

Finally, I think your Iterator implementation above is a bit off. If I'm not mistaken, it should be something like this:
impl<T> Iterator for PopBatch<'_, T> {
    type Item = T;
    fn next(&mut self) -> Option<Self::Item>;
}

Based on your comments above, I thought those problems could be solved!

Ugh, sorry! Yeah, with Copy and Default we can have write_slices().
I've just created a new document where we can write the code together and add comments. Might be easier than going back and forth on Github.
In this PR, I'll just add the minimal possible interface (push/pop only) and then we can follow up with a new PR that adds more features to it.

Finally, I think your Iterator implementation above is a bit off.

I'm just omitting fn next for brevity because it doesn't add any information ;)

I have no opinion on that, but your push()/pop() signatures seem to have changed from &self to &mut self, see also https://github.com/crossbeam-rs/crossbeam/pull/338/files#r264038883.

If we don't make pop() take &mut self, then one could do the following:

Grab slices using read_slices().
Call pop() a couple times.
Read invalid data from the slices retrieved in the first step.

But if we make pop() take a &mut self, the borrow checker will complain.
If method read_slices() didn't exist, then it'd be totally okay to take a &self. Alternatively, we can make read_slices() take a &mut self so that pop() doesn't have to. But I find the inverse a cleaner solution.

Between calling Consumer::pop_batch() and dropping the returned PopBatch object, it is not possible to call Consumer::read_available(). Therefore, we need an additional PopBatch::read_available().

I agree.

For each successful call to PopBatch::pop(), the value returned from PopBatch::read_available() decreases, even though the atomic read index hasn't been changed yet.

Correct.

When Consumer::pop_batch() is called, the atomic write index is cached and not checked anymore for subsequent PopBatch::pop() calls.

Yes.

Having a single PopBatch object, I can switch between calling .pop() and iterating over it and calling .pop() again. The value returned from PopBatch::read_available() decreases each time.

Yes. By the way, it's worth mentioning that PopBatch mutably borrows Consumer so you can't call pop() while there's an active PopBatch.

I've just created a new document

Thanks, it looks great!
I can't wait to try out this API!

In this PR, I'll just add the minimal possible interface (push/pop only) and then we can follow up with a new PR that adds more features to it.

OK, cool.
And thanks for the explanation about &mut self, that makes sense!

All right, I have slimmed down the API and will leave the PR open so that everyone can take one final look.
After that I'll follow up with several PRs introducing the new methods we have discussed.
@mgeier By the way, in case you live in Berlin, I'll be tomorrow in co.up coworking space in Adalbertstraße. :)

Sorry if that's a naive question: Is it possible to make the ring buffer UnwindSafe (or RefUnwindSafe?)?
I would like to use it in a FFI function within catch_unwind() and I get this error:
error[E0277]: the type `std::cell::UnsafeCell<usize>` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary

An extension of this work enhanced with batch operations on slices can be found here:
https://github.com/uklotzde/crossbeam/tree/spsc2
Some background: I'm a developer of the Mixxx team. Recently I came up with a new SPSC queue implementation to replace our naive C++ implementation and the error-prone PortAudio C code. The enthusiasm for my initiative was limited, so I decided to abandon the C++ work. Since I didn't find a versatile Rust implementation I ported and extended my code.
Publishing another implementation in a separate crate would further fragment the ecosystem and crossbeam seems to be the right place where those concurrency tools should reside.
I've tried to closely follow the existing design in this PR with some notable exceptions:

The names of operations where an error result is not considered exceptional behavior start with try_, i.e. try_push() and try_pop(). This is consistent with the mpsc facilities in std.
The error results should explicitly express their intention by using an enum with corresponding (tuple) variants. Even if only a single variant exists.
Completely hiding mutation between behind interior mutability is disputable. Modifying the contents of the queue should require exclusive access, i.e. &mut self.
The idea to add cache pads to the buffer upon allocation is borrowed from here: https://github.com/rigtorp/SPSCQueue. Possible extension: Instead of only requiring a single empty slot between writer and reader one could reserve multiple slots occupying a whole cache line. This would prevent false sharing in situations when the queue is almost full. But it cannot prevent false sharing in the opposite situation when the queue is almost empty. Might not be worth the effort.
Both an allocating constructor with_capacity() (renamed) as well as a generic constructor new() is provided. The latter works for no_std and allows to provide a static or externally allocated buffer that is managed by the owner of the queue.
The raw batch operations in the generic producer/consumer are declared as unsafe. Imposing additional trait bounds on T as suggested in the preceding comments removes this burden.
I'm a strong advocate of extensively using debug assertions in code that could not be tested exhaustively. For detecting violations of invariants or for catching wrong implicit assumptions early and confidently during development.
This more versatile implementation is slightly slower when running the benchmarks, expect +10% total time. The benefits of the batch operations on slices don't count yet.

I didn't want to open a competing PR and instead are asking for your feedback here. It is still work in progress.

Yeah I really want this in. Updates would be nice.

Looks like it'll be really useful for audio work. I'm happy to help if there's anything still required to get it in

I'd also be happy to help out here, what needs to be done?

@stjepang are you still planning to land this PR?

@stjepang I'm happy to help as well.
This design is very close to two other super useful designs, and the wrappers could all share the same datastructure:

A single-producer broadcast queue. Each write will be seen by multiple viewers. You can't move out of the queue but can operate on const references to Sync data in the queue.
A queue where the writer can overwrite slow readers. In this, the writer will never block and instead will invalidate readers that it passes, while the readers detect this failure. This is useful for ensuring that slow readers will fail instead of blocking others, but it prevents operations that operate on a view of the queue.

I'm happy to help with both of these features.

Also interested in seeing this land in crossbeam and can give time to make it happen...

I just discovered an unrelated (AFAIK) implementation of a lock-free SPSC ringbuffer: https://crates.io/crates/ringbuf
Any opinions on that?

OK, since there doesn't seem to be much movement here, I decided to take things into my own hands ...
With the permission of @stjepang I grabbed the code from this PR and shoved it into a new repo: https://github.com/mgeier/rtrb.
I then implemented some of the things discussed in the comments above, some of the things discussed in a few DropBox Papers documents, and I came up with a few additional API tweaks.
I was trying to not cause any performance regressions compared to the code in this PR, I hope I was successful.
If you are interested, please have a look: https://github.com/mgeier/rtrb.
The code in this PR is of course very good, I hope I didn't butcher it too much, but I've certainly introduced some bugs.
I'm very much open for further suggestions regarding API and/or implementation. Please feel free to create issues and PRs on my aforementioned repo.
If there are not too many objections, I will publish this in the next few days as a package on https://crates.io.

It took me a few more days than expected, but finally I've released my new ring buffer (based on this very PR):
https://crates.io/crates/rtrb
The API documentation, as usual, is available at:
https://docs.rs/rtrb
I'd love to hear some feedback, but please use https://github.com/mgeier/rtrb in order to not spam the comments of this PR.

Hi, is there any reason why this has stalled?

Crossbeam is largely stalled.

Crossbeam is largely stalled.

That is unfortunate. Are there any competing or forked implementations to be aware of?

Hi, is there any reason why this has stalled?

@likeabbas , the work in this PR lives in a separate crate: rtrb, see this comment above. See also this discussion about some upcoming changes.


jdk_net and jdk_nio fails on linux with java8

No issues in JBS:
java/net/InetAddress/BadDottedIPAddress.java failed with 'UnknownHostException was not thrown for: 999.999.999.999'
Stacktrace

Execution failed: `main' threw exception: java.lang.Exception: UnknownHostException was not thrown for: 999.999.999.999

Standard Output

Standard Error

java.lang.Exception: UnknownHostException was not thrown for: 999.999.999.999
	at BadDottedIPAddress.main(BadDottedIPAddress.java:49)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
	at java.lang.Thread.run(Thread.java:748)

https://ci.adoptopenjdk.net/view/Test_openjdk/job/openjdk8_hs_openjdktest_x86-64_linux/306/testReport/junit/java_net_InetAddress_BadDottedIPAddress/java/BadDottedIPAddress/
With issue opened in JBS:
java/net/InetAddress/CachedUnknownHostName.java failed with 'java.lang.Exception: Missing java.net.UnknownHostException    '
https://ci.adoptopenjdk.net/view/Test_openjdk/job/openjdk8_hs_openjdktest_x86-64_linux/306/testReport/junit/java_net_InetAddress_CachedUnknownHostName/java/CachedUnknownHostName/
Standard Error

Missing java.net.UnknownHostException for host bogusHostName
java.lang.Exception: Missing java.net.UnknownHostException
	at CachedUnknownHostName.main(CachedUnknownHostName.java:41)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
	at java.lang.Thread.run(Thread.java:748)

https://bugs.openjdk.java.net/browse/JDK-8056988 opened for window, should update with linux too.

java/net/ipv6tests/B6521014.java
java/net/MulticastSocket/SetGetNetworkInterfaceTest.java on linuxgot 'Cannot assign requested address 'Test failed: cannot create socket'
Standard Error

java.lang.RuntimeException: Test failed: cannot create socket.
	at B6521014.main(B6521014.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
	at java.lang.Thread.run(Thread.java:813)
Caused by: java.net.BindException: Cannot assign requested address (Bind failed)
	at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
	at java.net.Socket.bind(Socket.java:644)
	at B6521014.test2(B6521014.java:103)
	at B6521014.main(B6521014.java:121)
	... 6 more

JBS: https://bugs.openjdk.java.net/browse/JDK-8006560
java/net/Inet6Address/B6206527.java 'Cannot assign requested address (Bind failed) '
java.net.BindException: Cannot assign requested address (Bind failed)
	at java.net.PlainSocketImpl.socketBind(Native Method)
	at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
	at java.net.ServerSocket.bind(ServerSocket.java:375)
	at java.net.ServerSocket.bind(ServerSocket.java:329)
	at B6206527.main(B6206527.java:53)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
	at java.lang.Thread.run(Thread.java:813)

https://ci.adoptopenjdk.net/view/Test_openjdk/job/openjdk8_j9_openjdktest_x86-64_linux/433/testReport/junit/java_net_Inet6Address_B6206527/java/B6206527/
JBS: https://bugs.openjdk.java.net/browse/JDK-6670327

com/sun/net/httpserver/Test1.java get timeout error Program /home/jenkins/workspace/openjdk8_hs_openjdktest_ppc64le_linux/openjdkbinary/j2sdk-image/jre/bin/../../bin/java' timed out (timeout set to 960000ms, elapsed time including timeout handling was 960563ms)' could be a net work issue?
https://ci.adoptopenjdk.net/view/Test_openjdk/job/openjdk8_hs_openjdktest_ppc64le_linux/265/console

Intermittent failure on Macosx as timeout
java/net/Inet6Address/B6558853.java same behavior as java/net/ipv6tests/UdpTest.java

This issue is pretty stale.  Are there more current or specific issues that cover the failures?  What actions need to be taken to address and close this one?

Those tests failed with hotspot. Some of them are recorded in JBS with issue opened or resolved ( resolved, but still failed in AdoptOpenJDK).
Most of them are still excluded by ProblemList. I can reenable to see if issues still exist.

Doing a review of old issues, @sophia-guo is this an issue that needs to remain?  It is somewhat general and does not include some of information that we would hope to capture and there are also no PRs linked to this issue, though I have not looked in ProblemLists if there is mention of it.  There are 3 other issues that have referenced it.

All tests mentioned here are referenced by ProblemLists. The exclusion may be done combined with others so no PR is linked.

All tests excluded by https://github.com/adoptium/aqa-tests/issues/593.

excluded tests with this issue jdk8
java/net/InetAddress/BadDottedIPAddress.java https://github.com/adoptium/aqa-tests/issues/827 linux-all
java/net/InetAddress/CachedUnknownHostName.java https://github.com/adoptium/aqa-tests/issues/827 linux-all
java/net/InetAddress/IPv4Formats.java https://github.com/adoptium/aqa-tests/issues/827 linux-all
java/net/SocketPermission/Wildcard.java https://github.com/adoptium/aqa-tests/issues/827 linux-all
java/net/Socks/SocksV4Test.java https://github.com/adoptium/aqa-tests/issues/827 linux-all
com/sun/net/httpserver/Test1.java https://github.com/adoptium/aqa-tests/issues/827 linux-ppc64le
java/nio/channels/DatagramChannel/SendToUnresolved.java https://github.com/adoptium/aqa-tests/issues/827 linux-all
java/net/Inet6Address/B6558853.java https://github.com/adoptium/aqa-tests/issues/827 macosx-all
grinder
linux-x64 https://ci.adoptium.net/view/Test_grinder/job/Grinder/11350/console
mac https://ci.adoptium.net/view/Test_grinder/job/Grinder/11351/console
ppc64le https://ci.adoptium.net/view/Test_grinder/job/Grinder/11352/console
jdk21
java/net/Socket/asyncClose/AsyncClose.java https://github.com/adoptium/aqa-tests/issues/827 aix-ppc64, linux-ppc64le
aix https://ci.adoptium.net/view/Test_grinder/job/Grinder/11353/
ppc64le https://ci.adoptium.net/view/Test_grinder/job/Grinder/11354/


Overridden Functions Not Called if Invoked from a Typed Variable

Godot version
v4.0.dev.calinou.de83ee57e
System information
Windows 10 x64
Issue description
Let's say you want to override a built-in function like queue_free(). That's easy and straightforward and normally when you call object.queue_free() the overridden implementation takes precedent.
But if object is a typed variable reference and it's typed to something higher up in the hierarchy than your overridden implementation (or so I speculate), then the custom implementation is not called.
Steps to reproduce
Have some object override queue_free(). Make a variable reference to it that is typed (var typed_reference : Node = $Path). Call typed_reference.queue_free() and notice that the custom code you wrote was ignored.
Meanwhile if you reference was var untyped_reference = $Path then calling untyped_reference.queue_free() would have the expected behavior.
Minimal reproduction project
QueueFree.zip
There are two instances of the same class which has an overridden queue_free(). I instantiate them both in the same manner and treat them almost identically except one reference is held in a typed variable and the other is held in an untyped variable.
When I call queue_free() on both of them, the custom queue_free() implementation is only called on the untyped variable reference.
Object-oriented inheritance should not be broken just based on the type assigned to a variable. (This bug took a real long time to isolate.)

Also, let's say I have a script with this queue_free override:
func queue_free():
	print("emit")
	super.queue_free()

Inside this script, if I call queue_free() nothing will be printed, which is wrong to me (?). I need to call self.queue_free() to have the expected result.
I'm not sure it's linked so tell me if I need to open a new issue for this.

Overriding non-virtual native methods is not supported. See #72608.


Have a flag or function to produce only unique states

Currently the generated state function does not care if the generated state is unique with respect to previously generated states. It would be nice to ensure the generated states are unique.

In a network of dimension n, the number of possible states (with respect to a state and it's inverse being equal) is 2^(n-1). For a network of adequate size, this is good enough to say repeated states should not be an issue with large enough values of n or with repeated trials.


Isinders: workspace.findFiles doesn´t return any matches when  "searchRipgrep.enable": true

Create a test workspace with a folder that contains a file package.json

use yo code to create an extension

Replace the extension.ts with the code below.

Run the extension and execute the ´Hello World' command

--> the call workspace.findFiles returns 0 matches instead of 1.
This works in the latest stable. When toggling searchRipgrep the the correct number of matches is returned.
The code for extension.ts
'use strict';
// The module 'vscode' contains the VS Code extensibility API
// Import the module and reference it with the alias vscode in your code below
import * as vscode from 'vscode';

// this method is called when your extension is activated
// your extension is activated the very first time the command is executed
export function activate(context: vscode.ExtensionContext) {

    // Use the console to output diagnostic information (console.log) and errors (console.error)
    // This line of code will only be executed once when your extension is activated
    console.log('Congratulations, your extension "test-ts" is now active!');

    // The command has been defined in the package.json file
    // Now provide the implementation of the command with  registerCommand
    // The commandId parameter must match the command field in package.json
    let disposable = vscode.commands.registerCommand('extension.sayHello', () => {
        // The code you place here will be executed every time your command is executed

        // Display a message box to the user
        const workspace = vscode.workspace.workspaceFolders[0];
        if (workspace) {
            vscode.workspace.findFiles(new vscode.RelativePattern(workspace, "**/package.json"), "**/node_modules/**").then(results => {
                console.log('results: ', results);
            });
        }
    });

    context.subscriptions.push(disposable);
}

// this method is called when your extension is deactivated
export function deactivate() {
}

// @joaomoreno fyi, could this be related to the recent TPromise refactorings?

@egamma Doesn't look like it.


fatal error: Cannot create ConcatV2. Missing or invalid attribute axis.

Hello, just update to your new version, seems to work but i got this error when loading my pb model.
guard let axisInt32 = axisNode.nodeDef.attr["value"]?.tensor.intVal.first, let axis = LayerSizeAxis.fromTF(index: Int(axisInt32)) else {
fatalError("Cannot create (Constants.Ops.Concat). Missing or invalid attribute axis.")
}
here axistInt32 = 3 , but cannot create concatv2 when axis = LayerSizeAxis.fromTF(index: Int(axisInt32))
Any idea?
I am basically trying to convert my pb model which is one derived from https://github.com/Joker316701882/Salience-Object-Detection
and then froze.
Also some warning
"Unknown input: pool4_1:1"
"Unknown input: pool4:1"
"Unknown input: pool3:1"
"Unknown input: pool2:1"
"Unknown input: pool1:1"
thanks a lot

my input tensor ( 1,320,320,3)
output (1,320,320,1)
any idea on how I can tweak bender a bit to make it work?

Is there a way maybe in tensor flow to reshape input/output tensor to 3d from 4d ?

Can you try setting the first dimension as None in python? And see if that works.


Exception sass

I've tried to run in my local computer but I got this error. Has anyone already gotten this same log?
[info] Sass compiling on 1 source(s) [info] Packaging /Users/danielgoncalves/devel/scala-exercises/compiler/target/scala-2.11/exercise-compiler_2.11-0.4.0-SNAPSHOT.jar ... [info] Done packaging. java.lang.Exception:  Sass compiler: Unknown error, try running sass directly File: /Users/danielgoncalves/devel/scala-exercises/server/app/assets/stylesheets/main.scss Line: 0 Col: 0 	at sass.SassCompiler$.compile(SassCompiler.scala:29) 	at sass.SbtSass$$anonfun$5$$anonfun$6$$anonfun$7.apply(SbtSass.scala:55) 	at sass.SbtSass$$anonfun$5$$anonfun$6$$anonfun$7.apply(SbtSass.scala:43) 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) 	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) 	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) 	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) 	at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) 	at scala.collection.AbstractTraversable.map(Traversable.scala:105) 	at sass.SbtSass$$anonfun$5$$anonfun$6.apply(SbtSass.scala:43) 	at sass.SbtSass$$anonfun$5$$anonfun$6.apply(SbtSass.scala:39) 	at com.typesafe.sbt.web.incremental.package$.syncIncremental(package.scala:228) 	at sass.SbtSass$$anonfun$5.apply(SbtSass.scala:38) 	at sass.SbtSass$$anonfun$5.apply(SbtSass.scala:33) 	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47) 	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40) 	at sbt.std.Transform$$anon$4.work(System.scala:63) 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) 	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17) 	at sbt.Execute.work(Execute.scala:237) 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) 	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159) 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28) 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 	at java.lang.Thread.run(Thread.java:748) [error] (server/web-assets:sass) java.lang.Exception:  [error] Sass compiler: Unknown error, try running sass directly [error] File: /Users/danielgoncalves/devel/scala-exercises/server/app/assets/stylesheets/main.scss [error] Line: 0 Col: 0 [error] Total time: 150 s, completed May 5, 2018 11:33:39 PM

I have no clue. Have you tried running sass directly? Which commit did you check out? Does it also happen for other commits / tags?

Yes, I’ve already checked sass in terminal and I got ok. I’m gonna try another tag.

is there a fix for this?  I am getting the same error now

Thanks @jmjava. Is there a piece of documentation which should get updated to make this more obvious?

Frank,  I think the link takes you to a page that is focused on the node version.  Maybe link to the Ruby version directly?
Get Outlook for Android
On Sun, Jul 22, 2018 at 10:57 AM -0400, "Frank Rosner"<EMAIL_ADDRESS>wrote:
Thanks @jmjava. Is there a piece of documentation which should get updated to make this more obvious?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


Ability to access aot bin file with ffi dynamiclibloader method

Since we already have a dart2aot or dart2native compiler if we able to load those binary  files with ffi package like DynamicLibLoader Function we can produce packaged libraries in pub repos without exposing proprietary code
or
if we able to compile code to c compatible binaries we can able to access those directly from c lang which gives  cross lang compatibility

@canewsin can you be more specific on your enhancement request, maybe with some simple examples on how this requested feature would be used.

At this stage of SDK we are able to compile dart file to aot binary file using this tool
what i was proposing is :
if we able to load this binary file with some method similar to ffi method like libraryloader without exposing code to the user but exposing method prototype to the user like c/c++ which exposes header file with compiled files.
Use case :
proprietary can be distributed without exposing method implementation.

second request is compile dart code to binary code which can be accessed with in c code without any dependencies

remember that this is not about single executable binary, i want to compile dart code as library and distribute without disclosing implementation like c/c++ does(something like .dll or .so files)

ref from kotlin native : https://kotlinlang.org/docs/reference/native-overview.html
Interoperability
Kotlin/Native supports two-way interoperability with the Native world. On the one hand, the compiler creates:
an executable for many platforms
a static library or dynamic library with C headers for C/C++ projects
an Apple framework for Swift and Objective-C projects
On the other hand, Kotlin/Native supports interoperability to use existing libraries directly from Kotlin/Native:
static or dynamic C Libraries
C, Swift, and Objective-C frameworks


Inconsistent fee amounts allowed for the same class

Expected Behavior:
The system should enforce a consistent fee amount for all students within the same class, or provide a prompt if there’s a valid reason for exceptions.
Actual Behavior:
The system allows different fee amounts to be recorded for students in the same class without any warnings, leading to inconsistencies.

Realistically, eg, for private tutors, long time students could have a cheaper rate than newer students.


The report objects currently nested under jobs

Should be nested in own section, without jobs in path

@fqrious should be nested under reports


Add OGGraphInvalidate and OGGraphSetXXCallback API

Codecov Report
Attention: Patch coverage is 46.15385% with 14 lines in your changes are missing coverage. Please review.

Project coverage is 20.23%. Comparing base (a9f350f) to head (992148e).

Files
Patch %
Lines

Sources/_OpenGraph/Graph/OGGraph.cpp
35.29%
11 Missing :warning:

Sources/_OpenGraph/Data/ClosureFunction.hpp
66.66%
1 Missing :warning:

Sources/_OpenGraph/Graph/Graph.cpp
0.00%
1 Missing :warning:

Sources/_OpenGraph/Graph/GraphContext.cpp
0.00%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #49      +/-   ##
==========================================
+ Coverage   19.74%   20.23%   +0.49%
==========================================
  Files          48       49       +1
  Lines        1079     1102      +23
==========================================
+ Hits          213      223      +10
- Misses        866      879      +13

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Validate devices in bulk uploader

Description
The universal pipeline currently does not validate data the same way that the COVID pipeline does (e.g., validating against the LIVD table). We should validate devices on our end so we're sending them valid data, and so that users can see errors real-time.
Acceptance criteria

Add flu LOINC codes to our validation
Ensure that equipment_model_name andtest_performed_code combination matches a value in our database (check against devices in our database)
If no, validate that the LOINC code is one of the flu LOINCs
Document validation logic in SR wiki

Implementation notes

Validate these devices against our device table
Not adding flu to device table for now, just hardcoding flu LOINCs for validation

@zdeveloper if we implement the disease selection dropdown in the UI for the bulk uploader, would it make sense to tweak the validation to be specific to flu or covid, or would the RS validation handle that? e.g., if someone tried to submit COVID results after selecting "flu," should they get an error back that those results didn't go through? That's my main concern with the dropdown option (that a user will think results are going through that aren't).


Release v0.34.0

[x] Create release in Github

[x] Bump Version
[x] Create Draft Github Release
[x] Create Release Notes
[x] Generate Binaries and Checksum
[x] Upload Binaries and Checksum
[x] Publish the Draft Release
[x] Close any GitHub issues that have been delivered.
[x] Add a link to the release on the issue.

[x] Update the packaging repositories

[x] Update Homebrew
[x] Update Website Installation Script
[x] Update Chocolatey
[x] Update Github Action

[x] Update Documentation by generating a new docs version

Push any artifacts to a registry

[x] Communicate in Slack

[x] Add to "Announcements" in Next Community Meeting Agenda

Release checklist from here.

Done. https://github.com/vmware-tanzu/carvel-imgpkg/releases/tag/v0.34.0


some demand subclasses need to get demand suffix or be moved

Description of the issue
We have the following subclasses of demand subclasses:

office, retail, warehouse: CommercialDemand
Appliance: ResidentialDemand
Aviation, BatteryElectric, Combustion, HydrogenVehicle, Rail, V2Grid: TransportDemand

that all dont have definitions and are not just demands but often objects. So either we:

delete them
move them
rename them to -demands

this depends on what was intended when they were added and if we need them as demand or are they even demands?
To clarify this is especially important to not give them wrong definitions in #175
Ideas of solution
Workflow checklist

[ ] I discussed the issue with someone else than me before working on a solution
[ ] I already read the latest version of the workflow for this repository
[ ] The goal of this ontology is clear to me

I am aware that

[ ] every entry in the ontology should have an annotation
[ ] classes should arise from concepts rather than from words
[ ] class or property names should follow the UpperCamelCase

delete all demand, quantity etc for first release

When I compare it to the TIMES energy system model, all the existing subclasses for commercial demand could be deleted. We just have the energy service demands in our model. So in the commercial sector we have the following demands:
Space heating
Space cooling
Water heating
Lighting
Cooking
Refrigeration
Electric equipment
Other energy uses
In my model, the demands in the residential sector are quite similiar to the ones in the commercial sector. But appliance or electrical appliance is a good demand for the residential sector (amongst others).
In the transport sector we have those demand groups in the model:
aviation
bus
car
freight transport
motorcycle
naviation
train freight
train passenger
From the existing demands I would delete BatteryElectric, Combustion, HydrogenVehicle and V2grid.

As far as I can see, all demands have been removed.
I like the list of commercial demands from @Vera-IER and I think it can also be used fot the residential sector. I would suggest to implement those with a demand suffix.

I don't understand this. What would be a lighting demand or a car demand? Do you mean energy demand for lighting and energy demand of cars? I don't think we need explicit subclasses for this but we can link concepts like a concept of energy demand with a concept of lighting or a concept of car. (By the way, we already have a class vehicle).

A lighting demand would be the demand for light, measured in lumen. The model can choose, if this demand is supplied by LED lights or or older technologies and the energy demand for light is calculated accordingly. Same thing for the car demand (which could be also named as individual transport demand). There is a certain demand for indivial transport by cars in person-kilometers and the model can choose if this demand is supplied by electric cars, gasoline cars etc.

Okay, it seems that we need a lot of completely new concepts here and not only some subclasses of demand...

I think it's not a priority to have a very detailed demand description with all the subclasses in the OEO now. I think these very detailled demand subclasses might be specific for my model and not necessary for other energy models. So my suggestion would be to close this issue now, since the classes of the description are already deleted?


ModuleNotFoundError: No module named 'songbird.q2'

Hi,
I installed your software in a conda environment where qiime2 was already installed following the 2/3 commands. Then when I type qiime dev refresh-cache I got this error
Traceback (most recent call last):
  File "/Users/michoug/miniconda3/envs/qiime2/bin/qiime", line 11, in <module>
    sys.exit(qiime())
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/q2cli/dev.py", line 27, in refresh_cache
    import q2cli.cache
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/q2cli/cache.py", line 301, in <module>
    CACHE = DeploymentCache()
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/q2cli/cache.py", line 61, in __init__
    self._state = self._get_cached_state(refresh=refresh)
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/q2cli/cache.py", line 107, in _get_cached_state
    self._cache_current_state(current_requirements)
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/q2cli/cache.py", line 200, in _cache_current_state
    state = self._get_current_state()
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/q2cli/cache.py", line 238, in _get_current_state
    plugin_manager = qiime2.sdk.PluginManager()
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/qiime2/sdk/plugin_manager.py", line 44, in __new__
    self._init()
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/qiime2/sdk/plugin_manager.py", line 59, in _init
    plugin = entry_point.load()
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2346, in load
    return self.resolve()
  File "/Users/michoug/miniconda3/envs/qiime2/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2352, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'songbird.q2'

Any ideas ?

Hi @michoug thank you for reporting.  This error is popping up likely because there is no conda recipe for this at the moment.  That is currently happening on conda-forge here
The workaround to this is
git clone https://github.com/biocore/songbird.git
cd songbird
pip install -e .
qiime dev refresh-cache

Will update once recipe is in.

Thanks a lot @mortonjt
It worked

@michoug , thanks for raising this issue.  Now that there is a conda recipe in, I'm going to resolve this issue.  Feel free to open if this problem pops up again.


Allow title to be empty if specifying path for the identifier

Describe the bug
Confluence of assumptions:

In backend.js an identifier is required even if {{slug}} is not used in the config.yml::collections:slug

IDENTIFIER_FIELDS includes title and path but searches for title first. So if you specify both title and path as fields, title is always preferred.

For a microblog, the title needs to be an empty string in the rss.

In my configuration, I set title to an empty string, and path to an identifier, but I always get Collection must have a field name that is a valid entry identifier error because title is empty.
I'm not sure what the right solution is given how the logic is distributed, but ideally it would support title as an empty string with path specified as the identifier. Or more basically, only require the identifier field if {{slug}} is in the slug template string

Once #1543 merges, you'll be able to explicitly set "path" (or any other field name) as the identifier.

Fixed in #1543.


Bind Satisfactory http query server to <IP_ADDRESS>

Description

simple fix to fix the "Failed to Connect to the Server API" error
caused because the http query server binds to localhost of the container (seen in the start up logs)
the extra startup param binds it to <IP_ADDRESS>

Checklist for all submissions

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

#78 includes changes for Satisfactory 1.0, but not this fix

[x] Have you tested and reviewed your changes with confidence that everything works?
[ ] Did you branch your changes and PR from that branch and not from your master branch?

If not, why?:

[x] You verify that the start command applied does not use a shell script

[ ] If some script is needed then it is part of a current yolk or a PR to add one

[ ] The egg was exported from the panel

since the patch yesterday -multihome should work again

btw: https://github.com/pelican-eggs/games-steamcmd/pull/78


fix: passing null as first param to find() no longer throws

Fixes a bug in which calling pull.find(null, ...) would throw

thanks!

merged into 3.0.1


Read the Spring Active Profile from karate.env @ karate-config.js

Is there a way to read the active spring profile from the karate.env @ karate-config.js? I'm creating a QA project for testing based on Spring Cloud Contracts and I need that: for a specific spring profile the Spring Cloud Contract StubRunner will be disabled and the karate-config should inject a url property pointing for the staging environment; otherwise this property should point to localhost wire mock server created by Spring Cloud Contracts StubRunner.

you can read system properties by using karate.properties['my.prop.name'] so if I remember right, the spring profile is available as one. if not in your runner, set a system property from Java so that it can be used from Karate.

Thanks @ptrthomas for your help. I'll post my solution: Created two different maven profiles and the activation of one is when the karate-env is present (and it'll look for localhost stubs) and the other will be active when the karate-env is not present (and it'll look for staging url)


tifficc: relative intents disregarded in soft-proof mode

Lately I have been experiencing some problems with tifficc: in softproof mode, it looks like changing the intent by using relative intents always produce the same output (i.e., there is no difference in output among relative colorimetric, perceptual, an saturation), not only visually, files are the same byte by byte; visible changes are there when the absolute colorimetric intent is selected. I have tried different images that are out of gamut w.r.t. the target profile. I may add that, instead, applying the actual transformation correctly produces a different results.
For your reference, I use the following commands for soft proofing:
tifficc -i sRGB.icc -p paperprofile.icc -c2 -m0 -b infile.tif perc.tif
tifficc -i sRGB.icc -p paperprofile.icc -c2 -m1 -b infile.tif rel.tif
In the past, I could see visible differences among the different intents when using the soft proofing mode, but I am unable to tell when this stopped, as I am not using tifficc often for this purpose.

I have checked tifficc by using  the ICC probe profile and found it works ok.
But in the way you are using command line options,, for sure it will give same result.  -m intent is the proofing intent That is the intent used to translate the simulation to the output device, the screen probably.  It is not the intent to simulate. Since you have not specified  output profile with -o, it reverts to built-in sRGB which only have relative colorimetric, so -m0 and -m1 (also -m2) does exactly the same, instructs to translate the simulation to relative colorimetric sRGB)
Probably what you want is (this is with the probe profile)
tifficc -t0 -p Probev1_ICCv4.icc -c2 -m1 -b  infile.tif perc.tif
tifficc -t1 -p Probev1_ICCv4.icc -c2 -m1 -b  infile.tif rel.tif

Note that the simulated intent is set by using -t. not by -m

Thanks a lot Marti for spotting my mistake. I tried to use -t as you suggested and I confirm that everything works as expected.

Just a quick suggestion: maybe it would help to update the manpage of tifficc. For example, for -m it reads
-m <0,1,2,3>
SoftProof intent.
Adding an example on softproofing would also be helpful.

Done as suggested. I'm closing the issue now.

@mm2 May I ask some question about semantic? So "soft-proof intent" for you is the intent made to replace the normally used display rendering intent (which would then be discarded).
So first: is this wording from Little-CMS only or is it a widely recognized name that when we say "soft-proof intent", we actually mean the intent for the simulated space (e.g. some CMYK space for printing) to output space (e.g. a screen) transform?
In this case, we can consider there are 3 intent concepts, not 2. You simply use your -t intent for 2 of these concepts (depending or not we are in soft-proofing mode). How would you name these differently? A display rendering intent and a re-purposing rendering intent maybe? (I saw the wording "re-purposing" and "re-targeting" to name the 2 steps in some ICC white paper)
Finally I have a last question about lcms soft-proofing code:
    hArray[0]  = InputProfile; hArray[1] = ProofingProfile; hArray[2]  = ProofingProfile;               hArray[3] = OutputProfile;
    Intents[0] = nIntent;      Intents[1] = nIntent;        Intents[2] = INTENT_RELATIVE_COLORIMETRIC;  Intents[3] = ProofingIntent;

So for soft-proofing, you do 3 transform: from image space (e.g. a RGB image) to simulated space (e.g. CMYK printing profile) using the rendering intent (-t), then from the simulated space to itself in relative colorimetric, and finally from the simulated space to output space (e.g. the profile of a calibrated screen).
What I don't understand is this intermediary transform. Why transforming from simulated space to itself in relative colorimetric? Won't it be a no-op anyway?

I heard the term "soft proofing intent" in ICC meetings and probably is jargon. It means to me the way you translate colors from the simulation to the device that is used as proofer. Usually the display, but may be a printer. Epson Canon and HP have such printers. I worked in making one when I was at HP.  So if you emulate, for example, how an AppleRGB image would show on AdobeRGB, and your display is sRGB only, then you cannot see all the emulated colors so some gamut mapping should be done. For proofer devices the bigger the gamut the better but sometimes it is not feasible at all.
The process is, you convert from the source color space to the simulated color space, in our example from AppleRGB to AdobeRGB. Then you have the device values of colors in the simulated space, RGB in this case, so you need to measure to Lab to get colorimetric information. To do so you perform a relative colorimetric measurement device->Lab. This Lab have to be converted to proofer by means of output profile and proofing intent. You probably want relative colorimetric or absolute colorimetric here, but in the case you are using a printer proofer the device gamut is small and if you only want to capture the overall appearance,  to spot artifacts for example, in this case perceptual would be useful.

I heard the term "soft proofing intent" in ICC meetings

OK. For sure, you have more experience than I in the ICC/color industry field, so I'll take this jargon then. We'll name the first intent (from image space to simulated space) differently.

Then you have the device values of colors in the simulated space, RGB in this case, so you need to measure to Lab to get colorimetric information. To do so you perform a relative colorimetric measurement device->Lab.

It looks to me you are answering about the intermediary proofingProfile to proofingProfile transform in relative colorimetric, but I don't get it. Sure you'll convert to Lab but that would be done internally anyway. How is doing an intermediate transform in-between the 2 expected transforms (so 3 transforms happening in the end) helping here?
Basically what I'm asking is why your code (instead of being what I wrote above) is not just:
    hArray[0]  = InputProfile; hArray[1] = ProofingProfile; hArray[2]  = OutputProfile;
    Intents[0] = nIntent;        Intents[1] = nIntent;             Intents[2] = ProofingIntent;

t looks to me you are answering about the intermediary proofingProfile to proofingProfile transform in relative colorimetric, but I don't get it. Sure you'll convert to Lab but that would be done internally anyway.
Well, to convert from device to Lab you need a profile. Please note that profiles can go Lab->Device but also Device ->Lab, this is mostly intended to measure values OR do proofing.

The transform goes
Image  -> Lab -> Device to simulate -> Lab -> Render to proofer
The Image->Lab->Device to simulate is a normal transform to the profile being simulated. The used intent may move colors
The  Device to simulate -> Lab measures colors from the device to simulate to Lab. It needs the profile and relative colorimetric because relative colorimetric does not move in gamut colors. Since the direction is device->Lab all colors are in-gamut
Finally you need to represent those Lab values so that's Lab ->Render to proofer.
In total 3 transforms and 2 profiles

The normal transform that you would do to the simulated device
a measurement transform from simulated device to Lab
a conversion from measured lab to output proofer device

This is a multiprofile transform using the simulated profile twice, but with different intents.

Basically what I'm asking is why your code (instead of being what I wrote above) is not just:

Because output from proofing profile are device values, and you need Lab. You have to measure those values to get Lab, the color engine knows nothing on how to do this conversion. Is up to the profile.

The Image->Lab->Device to simulate is a normal transform to the profile being simulated. The used intent may move colors
The Device to simulate -> Lab measures colors from the device to simulate to Lab. It needs the profile and relative colorimetric because relative colorimetric does not move in gamut colors. Since the direction is device->Lab all colors are in-gamut
Finally you need to represent those Lab values so that's Lab ->Render to proofer.

Sure. That's all clear and understandable.
OK. I see. So I guess that I was confused by this part:

hArray[1] = ProofingProfile; hArray[2]  = ProofingProfile;

Considering what you say, what it does is rather something like:

hArray[1] = ProofingProfile; hArray[2]  = WhateverRepresentLabSpace;

Your code must surely handle this properly down the way (ignoring that it was ProofingProfile repeated twice) if I were to follow it.

You have to measure those values to get Lab, the color engine knows nothing on how to do this conversion. Is up to the profile.

Sure. But I was assuming that your engine would do this intermediate Lab conversion (using the profile indeed!) automatically. As an implied intermediate transform between 2 profiles/different spaces.
Same as when you said the first transform is Image->Lab->Device to simulate. Here you count it as a single transform, yet also write it with 2 arrows, implying an additional/intermediate passage through Lab. So in this sense, it would be even 4 transforms in total.
After all, what is the difference between a transform from Image to Device to simulate, and another from Device to simulate to Render to proofer? In the end, it's just transforms between 2 ICC profile spaces (or I'm missing something).
Anyway I think I got it. I was just confused by how it was written in your code, but that's probably just internal logic. The logic you explain here makes sense and it's what I would expect indeed.

You almost got it.
Ok, so let's say there are two transforms. One that is the transform that we want to simulate, goes image -> Lab->Device to simulate. It internally uses Lab, but as you will see this lab is not the same as measured lab.  Let's take as example AppleRGB, SWOP CMYK and sRGB
I have a value of RGB=(120, 20, 130) in apple rgb.
transicc.exe -t0 -i AppleRGB.icc -o *Lab
This is Lab (L*=36.7445 a*=53.0873 b*=-36.2030)
Now I want to simulate a SWOP press. Let's see which CMYK I get on perceptual
transicc.exe -t0 -i AppleRGB.icc -o USWebCoatedSWOP.icc
I convert this to SWOP in perceptual intent, gives me (C=56.0555 M=99.8657 Y=5.6336 K=0.1892)
Ok, now I want to know the Lab of this CMYK. Note that is not the original Lab of (36, 53, -36) which cannot be rendered on a SWOP printer, let's see what has done the SWOP profile. Using SWOP in the reverse direction and with relative intent I measure the color. It is important to use relative colorimetric otherwise it will "undo" the perceptual and this is not what we want.
transicc.exe -t1 -i USWebCoatedSWOP.icc -o*Lab
L*=(38.8067 a*=43.6602 b*=-29.5742)
Which is a quite different Lab value. This is the measured color we get at the simulation. So now lets represent it in sRGB
transicc.exe -t1 -i*Lab -o*sRGB
R=(140.1642 G=58.5783 B=140.6331)
As you can see a quite different color form original one.

Yes. It's all quite clear. But I mostly wondered (and still do) why you needed to explicit this CMYK -> Lab transform, and why you say:

It internally uses Lab, but as you will see this lab is not the same as measured lab.

Here is the computation from CMYK to sRGB:
$ transicc -t1 -i USWebCoatedSWOP.icc' -o*sRGB
LittleCMS ColorSpace conversion calculator - 5.0 [LittleCMS 2.12]
Copyright (c) 1998-2020 Marti Maria Saguer. See COPYING file for details.

Enter values, 'q' to quit
C? 56.0555
M? 99.8657
Y? 5.6336
K? 0.1892

R=140.1641 G=58.5783 B=140.6331

I got exactly the same value, and I didn't need to do the Lab transform explicitly. For sure, it's without a doubt still implicitly done as an intermediary step by LittleCMS. Just like it's also done in the first RGB→CMYK step too. But why do you make a difference here because the first and last transform? This is what I don't understand.
I perfectly understand (I think but it's complicated, so who knows! 🥲) all your explanation and I completely agree. I just don't understand why you single out this CMYK→Lab transform by making it explicit (as if it were different from the RGB→Lab transform which is done implicitly). The numbers with your own example seem to show that it would not make a difference to leave this transform implicit (as I get the same numbers as you).

P.S.: by the way, thanks a lot for all the patient explaining!

There is no such thing as implicit lab. When you create a transform you chain two profiles so you use first one from device to lab and the second from lab to device. In normal transforms both profiles are using same intent
What makes this second proofing transform special is that the intent is not same on both profiles. First profile uses relative colorimetric and second one uses what you give with -m
You could  think in just one multiprofile transform going device-lab-device-lab-device, which is how it is implemented.
BTW, yes you  need not to go to lab in the example. I used it to demonstrate color is different.


hide postcss-preset-env stage information

if i run nuxt dev it will show the following information:

postcss-preset-env stage is set to 1 for supporting advanced css features

is it possible to hide this? I think our developers don't need to see this information every time.


Content Overflow

Is there an existing issue raised for this?

[X] I have searched the existing issues

Description:
There is content overflow in register and login page of both the user and customer at 100% default zoom value.
Reason for Change:
The register button and some contents are not visible.
Current Styling:
Register

Login

Proposed Styling:
There are two ways we can tackle this design problem, these are as follows:

By increasing the parent size so that the screen can be scrolled and the content can be appear
By reducing the form field sizes.
Let me know which way to use to solve this problem.

Screenshots or Design Mockups (optional):
No response
Tick the appropriate boxes as per your choice

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'23 contributor
[X] I want to work on this issue

go for it @AlfiyaSiddique


Documentation: Google People API Python quick start example produces invalid scope

Regarding documentation page: https://developers.google.com/people/quickstart/python
In the quick start example, the line
SCOPES = 'https://www.googleapis.com/auth/contact.readonly'
suggest changing to:
SCOPES = 'https://www.googleapis.com/auth/contacts.readonly'
Without that change, it was giving an invalid scope error.
Error: invalid_scope

Some requested scopes were invalid. {invalid=[https://www.googleapis.com/auth/contact.readonly]}

Python 2.7
Ubuntu 16.04 LTS
Firefox 48.0

This fix, made internally at Google, is now live at https://developers.google.com/people/quickstart/python.
Thank you for reporting the problem!


Exception on latest versions of Xamarin.iOS, Xcode, etc

Xamarin.Auth Issue
Version

[ ] nuget version = <IP_ADDRESS>35
[ ] component version = <IP_ADDRESS>
[ ] Xamarin.iOS Version: <IP_ADDRESS> (Visual Studio Professional)
[ ] Apple Developer Tools: Xcode 10.2.1 (14490.122)

Steps to reproduce
Simply attempt to use the latest version of Xamarin.Auth on the latest versions of Xcode, VS4Mac, Xamarin.iOS, etc.
This exception occurs regardless of whether the app was previously installed on the iOS device or not.
I've attempted to completely uninstall and reinstall Xamarin.Auth from both my common project, as well as my iOS project.
Platform:

[] .NET version = Runtime Versions: 2.1.9, 2.1.8, 2.1.2, 2.1.1
[] mono version = <IP_ADDRESS> (2018-08/fdb26b0a445) (64-bit)

Expected behaviour
Preferably not throw an exception.
Actual behaviour
I'm receiving the following exception when attempting to FindAccountsForService:
"Search/Find FindAccountsForServiceAsync Object reference not set to an instance of an object"

"  at Xamarin.Auth.KeyChainAccountStore.FindAccountsForServiceAsync (System.String serviceId) [0x0007f] in <85539ee049364f3e8d64880a4a42b589>:0 \n  at Xamarin.Auth.KeyChainAccountStore.FindAccountsForService (System.String serviceId) [0x00000] in <85539ee049364f3e8d64880a4a42b589>:0 \n  at MetaFarmsSow.Model.Utilities.CryptographicUtility.RetrieveKey (System.String propertyName, System.Int32 length) [0x00011] in /Users/jangles/TFS/SowApp/V4/Model/Utilities/CryptographicUtility.cs:31 "

Inner exception:
"Object reference not set to an instance of an object"

"  at Xamarin.Auth.KeyChainAccountStore+<>c.<.cctor>b__12_2 () [0x00042] in <85539ee049364f3e8d64880a4a42b589>:0 \n  at System.Lazy`1[T].ViaFactory (System.Threading.LazyThreadSafetyMode mode) [0x0001c] in <50aa8c853ba54c99b7836ba4b45a5828>:0 \n--- End of stack trace from previous location where exception was thrown ---\n\n  at System.LazyHelper.ThrowException () [0x00000] in /Library/Frameworks/Xamarin.iOS.framework/Versions/<IP_ADDRESS>/src/Xamarin.iOS/external/corefx/src/Common/src/CoreLib/System/Lazy.cs:97 \n  at System.Lazy`1[T].CreateValue () [0x0007e] in <50aa8c853ba54c99b7836ba4b45a5828>:0 \n  at System.Lazy`1[T].get_Value () [0x0000a] in <50aa8c853ba54c99b7836ba4b45a5828>:0 \n  at Xamarin.Auth.KeyChainAccountStore.FindAccountsForServiceAsync (System.String serviceId) [0x00030] in <85539ee049364f3e8d64880a4a42b589>:0 "

My invocation:
        public static byte[] RetrieveKey(string propertyName, int length)
        {
            Account account = null;

            try
            {
                account = AccountStore.Create().FindAccountsForService(nameof(PersistanceUtility)).FirstOrDefault((Account arg) => arg.Username == propertyName);
            }
            catch (Exception ex)
            {
                System.Diagnostics.Debug.WriteLine(ex);
            }

            ...
        }

Hi @Janglinator , please check this out:
https://github.com/xamarin/Xamarin.Auth/issues/370/

Thank you!  I searched around for an open issue, didn't think to look in closed issues.
To be fair, issues shouldn't be closed until a release fixes the issue  😉

Any idea on when the fix will be released?  Would be beneficial for me as well

^ Ditto.
Currently attempting to build the project myself, but running into:
Error occurred when compiling build script: /Users/jangles/Downloads/Xamarin.Auth-master/build.cake(2051,55): error CS0012: The type 'Object' is defined in an assembly that is not referenced. You must add a reference to assembly 'netstandard, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'.

@kleindev

Any idea on when the fix will be released?

i'm working on this when I have time -  when I'm not on other projects. Sorry.

We too are struggling with this issue. We can't migrate to Xamarin.Essentials SecureStore because of this crash.

@mlyrstad best I can find so far is to try out these manually built dlls: https://github.com/xamarin/Xamarin.Auth/issues/370#issuecomment-489140679
Disclaimer - I haven't done myself yet.  I'm planning to use the dlls and also migrate to Xamarin Essentials (need a working Xamarin Auth for migration path to work)

1.7.0-alpha01 released.

with fix for this issue
a lot of older platforms removed
cake build script needs complete rewrite
VS2019 not supported completely yet
build are on CI Azure DevOps

Can confirm that this works for me - I'm using Visual Studio 2019.  Thank you

@kleindev
Thanks for reporting. I will create release <IP_ADDRESS> and publish ASAP (today or tomorow).
After that I would like to ask you to close this issue please.

duplicate of https://github.com/xamarin/Xamarin.Auth/issues/370
closing

@moljac 1.7.0-alpha01 released.
with fix for this issue
What branch is the fix on? Is it in a particular commit? Thanks!


ApplicationInsights does not track the blob storage dependency

As per the documentation blob storage interaction should be tracked, however application map in Application insights does not for me.
I have added custom setup as below which also does not work.
import * as appInsights from 'applicationinsights';

        appInsights.setup(mykeyValue)
            .setAutoDependencyCorrelation(true)
            .setAutoCollectRequests(true)
            .setAutoCollectPerformance(true, true)
            .setAutoCollectExceptions(true)
            .setAutoCollectDependencies(true)
            .setAutoCollectConsole(true, false)
            .setUseDiskRetryCaching(true)
            .setAutoCollectPreAggregatedMetrics(true)
            .setSendLiveMetrics(false)
            .setInternalLogging(false, true)
            .setDistributedTracingMode(appInsights.DistributedTracingModes.AI_AND_W3C)
            .start();

Is there any option to setup manually tracking of dependencies?

I was able to see the interaction with Azure blob storage on the Application Map and in the Application Insights logs. Can I confirm that you're on the latest version of Application Insights and the Azure SDK?

I was able to see the interaction with Azure blob storage on the Application Map and in the Application Insights logs. Can I confirm that you're on the latest version of Application Insights and the Azure SDK?

Hi @JacksonWeber,  I have below versions. When manually tracked dependency, it does populate the application Map with blob storage entries. It would be good if that happens automatically as azure sdk is used for blob storage interaction.

Dependency versions:
"@azure/storage-blob": "^12.5.0"
"applicationinsights": "^2.3.1"

Manual tracking:
appInsights.TelemetryClient.trackDependency()

Blob storage accessed as :

private getContainerClient(containerName): ContainerClient {
        const blobClientService = BlobServiceClient.fromConnectionString(this.azureConnection);
        const containerClient = blobClientService.getContainerClient(containerName);
        return containerClient;
    }
const containerClient = getContainerClient(containerName);
containerClient.listBlobsFlat({ includeTags: true });
containerClient.getBlockBlobClient(blobName);
containerClient.deleteBlob(blobName);

@mdv27 I was able to get the Application Map to populate the dependencies without manual tracking just using the standard appInsights.setup() statement. What happens for you without adding the manual tracking statements?
If nothing is being displayed when the manual tracking statement is removed in the application map, could you update the applicationinsights and azure blob storage dependencies.


the frida require python3.x , and house is require python2.x

Hi, I have one question is that   the frida require python3.x , and house is require python2.x ，
if i use python3.x ，and run  python3.x  app.py 8888 , an error appeared ,
File "app.py", line 127
"""
^
SyntaxError: invalid syntax

Hey, thanks for letting me know, I did not know Frida recommends using python 3; I will def. update House a little bit in the future to make it compatible with that. But at this point, Frida also runs well with python 2 as I am using it w/o issues; so I would suggest keep on it for now, also I would suggest using virtualenv for different python envs to get it work.

I am gonna close this for now as it is not a deadly issue at this point, but thanks for the feed back.


Bazel Javascript rules

Bazel rules now work for the Javascript, but the closure compiler throws an internal compiler error on the definition of SecurityException in javascript/exception/securityexception.js.
More work is needed, but this is a starting point, and I want to make sure there are no regressions in the rest of Tink.
#189 advice was followed to rearrange the order of WORKSPACE, but perhaps more can be done.
More advice is appreciated.

I have successfully run bazel build ... and all the targets build.  bazel test ... still fails (only 24/47 tests pass), but I have some ideas for fixing the tests.
I need my https://github.com/bazelbuild/rules_closure/pull/339 PR merged before I'd be able to use an unmodified upstream rules_closure.  When that happens, I will update the WORKSPACE accordingly.

Hi Michael, sorry for the delay on this PR.  I'd be happy to proceed with review, but could you please resolve conflicts before that?

I've been away from this so long that I don't remember what changes are necessary to fix the build errors introduced by upstream.  The current branch has no more merge conflicts, but some Closure typing is still necessary to get javascript/clearset_keyset_handle.js to build.
Help would be appreciated... I'm not sure when I'll be able to resume work on Tink JS.

Here are the errors:
javascript/cleartext_keyset_handle.js:55: ERROR - could not determine the type of this expression
    return new KeysetHandle(PbKeyset.deserialize(keysetJspbString));
                            ^
  ProTip: "JSC_UNKNOWN_EXPR_TYPE" or "missingSourcesWarnings" or "reportUnknownTypes" can be added to the `suppress` attribute of:
  //javascript:cleartext_keyset_handle
  Alternatively /** @suppress {reportUnknownTypes} */ can be added to the source file.

javascript/cleartext_keyset_handle.js:55: ERROR - proto.google.crypto.tink.Keyset.deserialize is never defined
    return new KeysetHandle(PbKeyset.deserialize(keysetJspbString));
                            ^
  ProTip: "JSC_UNDEFINED_NAME" or "missingSourcesWarnings" or "undefinedNames" can be added to the `suppress` attribute of:
  //javascript:cleartext_keyset_handle
  Alternatively /** @suppress {undefinedNames} */ can be added to the source file.

javascript/cleartext_keyset_handle.js:55: ERROR - Property deserialize never defined on proto.google.crypto.tink.Keyset
    return new KeysetHandle(PbKeyset.deserialize(keysetJspbString));
                                     ^
  ProTip: "JSC_INEXISTENT_PROPERTY" or "checkTypes" or "missingProperties" or "missingSourcesWarnings" can be added to the `suppress` attribute of:
  //javascript:cleartext_keyset_handle
  Alternatively /** @suppress {missingProperties} */ can be added to the source file.

javascript/cleartext_keyset_handle.js:68: ERROR - could not determine the type of this expression
    return keysetHandle.getKeyset().serialize();
           ^
  ProTip: "JSC_UNKNOWN_EXPR_TYPE" or "missingSourcesWarnings" or "reportUnknownTypes" can be added to the `suppress` attribute of:
  //javascript:cleartext_keyset_handle
  Alternatively /** @suppress {reportUnknownTypes} */ can be added to the source file.

javascript/cleartext_keyset_handle.js:68: ERROR - Property serialize never defined on proto.google.crypto.tink.Keyset
    return keysetHandle.getKeyset().serialize();
                                    ^
  ProTip: "JSC_INEXISTENT_PROPERTY" or "checkTypes" or "missingProperties" or "missingSourcesWarnings" can be added to the `suppress` attribute of:
  //javascript:cleartext_keyset_handle
  Alternatively /** @suppress {missingProperties} */ can be added to the source file.

5 error(s), 0 warning(s), 97.4% typed

Thanks a lot of working on this.
Based on this PR, we've come up with a new design that will allow us to build a NPM package. We're actively implementing this design and will open source what we've got soon.
Again, thanks for working on this!

FYI: @taymonbeal has helped us publish a package, check out https://www.npmjs.com/package/tink-crypto.