Portable packed SIMD vector types

This RFC extends Rust with portable packed SIMD vector types.
Rendered.

Acknowledgments
It would not have happened without the invaluable contributions of @alexcrichton and @BurntSushi , the excellent feedback given by the many reviewers of the initial drafts: @sunfish, @nagisa, @eddyb, and @hsivonen, amongst others, all the people using stdsimd, filling issues, and submitting PRs, and @huonw's work on the rust-lang-nursery/simd crate.

Minor request: Please define the m types in the guide explanation.  I think I know what they are, and [iuf] are obvious, but since there's no m16 type in rust, it would be good to describe what it is and why it exists and why there isn't just boolx4.
What are the layout guarantees of these types?  Are they repr(C)?  Could they offer .as_array(&self) -> &[T;N] (or even Deref to arrays)?

@rkruppe and @hsivonen I think this commit incorporates all of your feedback, but it would be great if you could review the changes in case I missed or misinterpreted something.
@scottmcm I've added more documentation about vector masks to the guide-level explanation in this commit. Let me know what you think.

@scottmcm

What are the layout guarantees of these types? Are they repr(C)? Could they offer .as_array(&self) -> &[T;N] (or even Deref to arrays)?

Those are really good questions. The short answer is that, as the ABI section mentions, their layout is unspecified.
The ABI section does however explicitly forbids their use on extern functions, which means that right now they are definitely not repr(C).
Also, neither this RFC nor the std::arch RFC defines what the behavior of this is:
union A {
    data: [f32; 4],
    vec: f32x4,
}
let x: [f32; 4] = unsafe { A { vec: f32x4::splat(0.) }.data };

so for all practical purposes this is also currently unspecified behavior.
The only thing this RFC (and IIRC the std::arch RFC) specifies is the behavior of mem::transmute from one vector type to another vector type. For everything else one needs to use the memory load/store API ({load,store}_{aligned,unaligned}{,_unchecked}).
A future RFC specifying the layout and ABI of these types will certainly have to properly answer all these questions. Right now, there are a just a couple of things that could be done differently ABI-wise and that might produce a better ABI than what we currently have. So the purpose of leaving the ABI unspecified in these initial RFCs is to not limit those efforts in any way.

I'd personally expect such union-using behavior to work and operate as intended, I think like the non-portable SIMD types we'll want to at least define how large these are for unsafe operations. (aka is it safe to transmute f32x4 to __m128?

I think there are a couple of issues.
The first one is whether one can transmute a vector type into an equally-sized non-vector type like an array of a tuple. The std::arch RFC specified that the vector types are Sized and their size, and this RFC specifies this as well. So to me this answer is a clear yes.
The other issue is whether these types are layout compatible with arrays and tuples. That is, whether we guarantee the following assert to pass everywhere:
union A { arr: [f32; 4],  vec: f32x4, tup: (f32,f32,f32,f32) }
let x: [f32; 4] = unsafe { A { vec: f32x4::new(0., 1., 2., 3.) }.arr };
assert_eq!(x[2], 2.);  // OK?
let y: (f32,f32,f32,f32) = unsafe { A { vec: f32x4::new(0., 1., 2., 3.) }.tup };
assert_eq!(y.2, 2.);  // OK?

These two cases currently work on all platforms, but this is incidental. For example, the layout of tuples is unspecified, so we can't guarantee this for tuples AFAICT.
The layout of arrays is specified so we could do this for arrays. For that, we probably need to write down that a vector type has the same layout as an array or something like this. This is a bit weird to me, because when two things are layout compatible, I intuitively expect transmutes between them to have zero-overhead, that is, to just change the type of my data. However, a transmute between a vector and an array needs to perform an aligned/unaligned memory store. Since transmute is defined as semantically being like a memcpy, this might be ok. I still find it weird though.

Curious if this RFC might provide enough of a justification for adding f16 to the language. I know that the half crate exists, but it's still harder to use considering how f16 literals can't actually be written.

API question: should Wrapping<u32x4> and similar be allowed?

Curious if this RFC might provide enough of a justification for adding f16 to the language. I know that the half crate exists, but it's still harder to use considering how f16 literals can't actually be written.

This RFC forwards that option to future RFCs proposing to add more std::arch intrinsics. Note, however, that while many architectures (x86 included) have intrinsics for working on vectors of f16s, most of them do not support operating on single f16 values.

API question: should Wrapping and similar be allowed?

I think so. Given the change towards "safety" by default, allowing Wrapping<{i,u}{w}x{N}> makes sense.

If wrapping types seem reasonable, I'd recommend suggesting the same inherent methods (excluding the wrapping_-prefixed ones), similar to how rust#32463 did it.

@clarcharr

If wrapping types seem reasonable, I'd recommend suggesting the same inherent methods (excluding the wrapping_-prefixed ones), similar to how rust-lang/rust#32463 did it.

There are many methods there that would need to be disambiguated on whether they are horizontal or vertical operations, and it also unclear to me whether some of these methods are efficiently implementable or they would need to fall back to scalar code in most or all cases.
I think it would be pretty much uncontroversial to add these behind a feature gate to stdsimd, and once they become "worth stabilizing", just submitting either an RFC about them or doing a mini-FCP, depending on how trivial their design turns out to be.
Feel free to open a stdsimd issue about these if you want to discuss them and/or send PRs, but I don't think the core parts of portable vector types proposed in this RFC should be blocked on Wrapping<T> implementations being available.

So I've pushed a commit that:

specifies the portable vector types to be layout-compatible with the architecture-specific vector types, and with arrays of the same element type and number of elements (this defines the behavior of using an union to type-pun between arrays and vector types)
renames hmin/hmax to min_element/max_element
adds saturating arithmetic

The implementation in std::simd is pretty much 1:1 with the updated RFC (might land on nightly one of these days when it gets synchronized with rust-lang/rust).
The only things missing in std::simd are:

portable shuffles: there is a PR with a working conforming implementation, but this might need to be moved to rust-lang/rust
wrapping and saturating arithmetic methods: needs some tiny rustc changes
checked arithmetic by default: needs some tiny rustc changes

I've got some feedback that it might be worth to also add portable floating-point operations like sqrt and friends. I've got a rustc branch that adds these, and they will probably land in std::simd at some point for experimentation purposes but these are explicitly not being proposed here.
Also, most LLVM bugs related to the implementation are being worked on, so at least right now it seems that code generation for everything being proposed here might look pretty good in LLVM7.

If I understand correctly, there's nothing stopping us from backwards-compatibly replacing these types with typedefs later if/when we can implement a more generic solution, right? (E.g., type f32x4 = Simd<f32, 4> or type f32x4 = Simd<[f32; 4]>.)

If I understand correctly, there's nothing stopping us from backwards-compatibly replacing these types with typedefs later if/when we can implement a more generic solution, right?

Yes, backwards compatible changes do not break backwards compatibility.

I've pushed a couple of commits:

hopefully fixing the bug in the example that @jesskfullwood reported,
adding the From conversions from vector types to array types of the same element type and number of lanes (and vice-versa) - @Centril mentioned these
adding some widely available floating-point vertical operations (fused-multiply-add, square root, and reciprocal square root estimate).

An exact vertical square root is available on some architectures, but fused-multiply-add and reciprocal square-root estimate are controversial because their precision is in general, architecture dependent.
I haven't specified the precision of fma and rsqrte in the RFC, but what I had in mind was to state that:

fma is at least as precise as a * b + c, which is true for all architectures that offer it (the faster intrinsic has typically has more precision). This allows us to use the faster intrinsic when available (which is the whole point), and to fall back to doing a * b + c for portability otherwise
rsqrte should come with an error-bound. On architectures with an estimate, like x86, ARM, and PowerPC, we should be able to either use the estimate, or use the estimate + a couple of newton raphson iterations (an operation that most architectures support as well for reciprocal square root) to satisfy the error bound, where the exact error bound should be resolved before stabilization.

For background on rsqrte:

x86 SSE supports rsqrte with an error bound 12 bits = 1/4096 = 2.4 * 10^(-4) for 32-bit floats, no 64-bit support. AVX-512 supports rsqrte for 32-bit floats with 14bit precision, and for 64-bit floats with 26bit precision.
ARM NEON supports rsqrte for 32-bit and 64-bit floats. I haven't been able to find the error-bound, just that it is higher than the one on x86, but this is not an issue because NEON provides an instruction for doing Newton-Raphson iterations here (vrsqrts: reciprocal square-root step), so worst case we can do an rsqrte + vrsqrts + vrsqrt + ....
PowerPC altivec supports rsqrte for 32-bit floats with 12-bit precision (same as x86). I haven't been able to find if altivec or vsx supports 64-bit floats (@lu-zero ?).
MIPS MSA supports it for both 32-bit and 64-bit floats with a precision of 2 ULPs: "The approximation differs from the compliant reciprocal of the square root representation by no more than two units in the least significant place"

In any case, all these architectures provide an exact vector sqrt operation, so we can always fall back to that followed by a reciprocal.
@sunfish I think you mentioned interest in some other operations: maybe reciprocal and/or reciprocal estimate? I'd have to test if f32x4::splat(1.) / a lowers down to the appropriate exact reciprocal intrinsic on LLVM, and if not, why not. But I think that we could also add a reciprocal estimate to the set of operations provided by floating-point vectors following a similar argument to that of rsqrt if people agree.

It is called xvrsqrtedp
#ifdef __VSX__
static __inline__ vector double __ATTRS_o_ai vec_rsqrte(vector double __a) {
  return __builtin_vsx_xvrsqrtedp(__a);
}
#endif

(to be noted: vec_rsqrte for f32x4 would emit different instructions between altivec and vsx)

Two open questions:

Should the vector load/store (unaligned, ...) methods be renamed to read/write for consistency with std::ptr methods ?

In the WASM ISA repo it was discussed that for WASM it might make sense for the floating-point estimate functions to take the desired maximum error as an immediate mode argument. Would something like that make sense for the portable vector types?
I am unsure. I think that the "estimate" functions (sqrte, rsqrte, ...) should have a minimum reasonable precision on all architectures, and if someone wants to have different implementations for different precisions they can extend the std::simd types using an extension trait on top of std::arch, and that we can always reconsider this in the future depending on what we end up doing with respect to fast math support, and also depending on whether LLVM/Cretonne end up exposing similar functionality portably.

Given all the open questions around fast math and precision of estimates, I think it's prudent to omit it from the RFC for now.
I'm even still skeptical of having approximate rsqrt at all: we don't have any precedent for strict documented accuracy bounds, and it seems a speculative feature to add (the more usual approach, AFAIK, is to offer some kind of rsqrt and have it interact with -ffast-math).

the more usual approach, AFAIK, is to offer some kind of rsqrt and have it interact with -ffast-math

Is there even an usual approach to portable floating-point vector intrinsics? I've taken a look at the following compiler/libraries/documents and everybody does something different:

Dart SIMD does not provide any of the estimate functions.
SIMD.js makes this "implementation" dependent, which might mean it depends on which arch the javascript implementation is running on.
C++'s P0214r9, P0820r3, P0918r0, ... allow using <cmath> on simd vector types. This could mean, in practice, that precision could be controlled via global fast math flags, but the std doesn't know about this. The prototype implementation [@https://github.com/google/dimsum (link to one of the estimate intrinsics)](https://github.com/google/dimsum/blob/f60cc12fdd3103170ba443672af135052d5b1ff5/operations.h#L256] does add estimate functions to the API with target dependent precision.
The Intel C compiler provides multiple versions of the approximative intrinsics with different accuracy (..._A11, ..._A21, ...)  and also let's users set the accuracy of each intrinsic via a compiler flag globally.
ISPC makes the approximation error target dependent.
LLVM has the afn (approximate function) flag and unsafe-math attributes but does not have any way to specify a precision.

Given all the open questions around fast math and precision of estimates, I think it's prudent to omit it from the RFC for now.

This makes sense. I am going to move the approximate/estimate versions to the unresolved questions of the RFC, so that we can at least try to resolve these issues before stabilization. If don't make it, we can always just leave them out then.

If anybody has any thoughts on this please chime in. My current thoughts on the estimate functions are the following:

the most common estimate intrinsics (sqrte, rsqrte, recpe) are often much faster than the exact ones (we are talking here about O(5-10x): ~10-20ns to ~100ns).
for these to be truly portable, the minimum precision must be architecture independent
the "best" default precision for these intrinsics is controversial (and application dependent)
changing the precision globally can be useful, but changing it on a per crate and per intrinsic basis might still be a pretty big hammer. Changing the "fast math" settings for an expression, block, module, crate, etc. might be the ideal solution, but we won't probably have a solution for this in the mid term.
offering different versions of the intrinsics like Intel does is not a bad solution

I don't really know what the best way to solve this, but I think that at least we should try to get versions with a "default" precision working at some point. Even if they don't fit every use case, we can always add a way to configure the precision later on via fast-math flags or via other methods that allow the user to request a minimum precision.

I have pushed two commits. The first one renamed the load/store methods to read/write and the second one added portable gather/scatter operations.

@rkruppe I found an "issue" with the portable vector shuffles, or better said, something that we might want to improve.
When using shuffles, very very often you only care about creating a vector with some elements of the inputs, but some elements of the resulting vectors do not matter. For example:
// add and reduce
let x = i32x4::new(0, 1, 2, 3);
let y = shuffle!(x, [2, 3, undef, undef]); // i32x4(2, 3, undef, undef)
let a = x + y;  // i32x4(2, 4, undef, undef)
let b = shuffle!(x, [1, undef, undef, undef]);
let c = a + b; // i32x4(6, undef, undef, undef);
c.extract(0)

In the wild, I see a lot of people use 0 instead of undef but I wondered whether that severely constraints which code LLVM can generate, so I gave it a try:
define i32 @add_and_reduce(<4 x i32> %a) {
  %b =  shufflevector <4 x i32> %a, <4 x i32> %a, <4 x i32> <i32 2, i32 3, i32 0, i32 0>
  %sum = add <4 x i32> %a, %b
  %c = shufflevector <4 x i32> %sum, <4 x i32> %sum, <4 x i32> <i32 1, i32 0, i32 0, i32 0>
  %d = add <4 x i32> %sum, %c
  %r = extractelement <4 x i32> %d, i32 0
  ret i32 %r
}

emits with llc -O3 -mattr=avx2 this assembly (godbolt):
add_and_reduce: # @add_and_reduce
  vpshufd xmm1, xmm0, 14 # xmm1 = xmm0[2,3,0,0]
  vpaddd xmm0, xmm0, xmm1
  vpshufd xmm1, xmm0, 1 # xmm1 = xmm0[1,0,0,0]
  vpaddd xmm0, xmm0, xmm1
  vmovd eax, xmm0
  ret

and if I use undef instead of the 0s, it emits (godbolt):
add_and_reduce: # @add_and_reduce
  vpshufd xmm1, xmm0, 78 # xmm1 = xmm0[2,3,0,1]
  vpaddd xmm0, xmm0, xmm1
  vphaddd xmm0, xmm0, xmm0
  vmovd eax, xmm0
  ret

which is slightly better: a vpshufd+vpaddd is replaced with a single vphaddd. This might well be an llvm bug (I've reported it here: ), but if it is not, I think it might be worth it to allow passing undef as shuffle indices to indicate "i am not going to use this value afterwards in any meaningful way". The troubling thing though is that the subsequent operations do use the value (e.g. adding i32 to undef, undef to undef, etc. So I have no idea how we could make this work.
In this particular case, because of how this algorithm works, we can shuffle! into smaller vectors to avoid the undef:
// add and reduce
let x = i32x4::new(0, 1, 2, 3);
let y: i32x2 = shuffle!(x, [2, 3]); // i32x2(2, 3)
let x: i32x2 = shuffle!(x, [0, 1]); // i32x2(0, 1)
let a = x + y;  // i32x4(2, 4)
a.extract(0) + a.extract(1); // 6

but that generates even weirder code (godbolt):
define i32 @add_and_reduce(<4 x i32> %a) {
  %b =  shufflevector <4 x i32> %a, <4 x i32> %a, <2 x i32> <i32 2, i32 3>
  %as = shufflevector <4 x i32> %a, <4 x i32> %a, <2 x i32> <i32 0, i32 1>
  %sum = add <2 x i32> %as, %b
  %c = extractelement <2 x i32> %sum, i32 0
  %d = extractelement <2 x i32> %sum, i32 1
  %r = add i32 %c, %d
  ret i32 %r
}

emits this assembly (godbolt):
add_and_reduce: # @add_and_reduce
  vpshufd xmm1, xmm0, 250 # xmm1 = xmm0[2,2,3,3]
  vpmovzxdq xmm0, xmm0 # xmm0 = xmm0[0],zero,xmm0[1],zero
  vpaddq xmm0, xmm0, xmm1
  vmovd ecx, xmm0
  vpextrd eax, xmm0, 2
  add eax, ecx
  ret

cc @RalfJung @eddyb

From an unsafe code guidelines perspective, having undef in a u32 may be violating the invariants of that type ("may" because like most everything this isn't decided yet). One interesting question here is what the rules are for u32x4. If these were arrays, I'd suggest you use [MaybeUninit<u32>; 4] to express that every element, individually, could be uninitialized -- but with these fixed inflexible SIMD types, that doesn't seem like an option. It also seems strange to have different invariants for the elements of a u32x4, and u32.
So, I don't think I have an answer yet for how I think this should be done, but this is certainly an interesting datapoint. Thanks!

If these were arrays, I'd suggest you use [MaybeUninit; 4] to express that every element, individually, could be uninitialized

So if I understand that part correctly, this would allow us to, for example, add two arrays element-wise (vertically), by specifying the semantics of the addition to propagate whether the elements are uninitialized or not. That is, two initialized elements would be added, but if one or both are uninitialized, then the result is always uninitialized. Does this make sense?
If so, this would make the current signature of some of the vector methods, like vec.extract(idx), unsafe, because they return T and not MaybeUninit<T>. We could then either make them unsafe, or make them return MaybeUninit<T> just like the Index trait for a [MaybeUninit<T>; N] would do.

-- but with these fixed inflexible SIMD types, that doesn't seem like an option.

I have a branch of stdsimd that makes the fixed inflexible types type aliases into a Simd<[T; N]> internally (this new generic type is not initially exposed). Once that is finished, I'll open a PR on stdsimd for review and link that here, so that we can see how that looks like, and sketch whether this RFC should be "re-drafted" or not to expose Simd<[T; N]> on its API. The new API is backwards compatible with everything in this RFC, except maybe with the memory gather/scatters, so we don't really need to add it before it is merged, but one thing it "could" allow is using Simd<[MaybeUninit<T>; N]>, so not all hope is lost here.

That is, two initialized elements would be added, but if one or both are uninitialized, then the result is always uninitialized. Does this make sense?

If any input to an arithmetic operation is uninitialized, the output is uninitialized. I view these vector additions as being equivalent to pointwise additions, so uninitializedness propagates within a vector element but not across vector elements -- similar to arrays, and doing the arithmetic by looping over the array. Is that what you are asking?

If so, this would make the current signature of some of the vector methods, like vec.extract(idx), unsafe, because they return T and not MaybeUninit<T>

If u32x4 is equivalent to [MaybeUninit<u32>; 4] then this is correct. I assumed there would still be an equivalent for [u32; 4] where extracting an element is safe, but that could be hard to achieve.

Is that what you are asking?

Yes that was it, thanks.

but that could be hard to achieve.

I am leaning to think that we just need both.

Disregarding any details of how we could fit "garbage lanes"1 into safe or unsafe Rust, I think there's an extremely high barrier to bringing that baggage in the APIs. Questions like whether vectors can be uninitialized in some lanes and not others need settling either way (cc https://github.com/nikomatsakis/rust-memory-model/issues/48), but producing and consuming partially-garbage vectors as part of the standard API surface many programmers are expected to use is something we should try really hard to avoid. I say this not just because it exposes huge and subtle (IME most programmers are quite care-free when reasoning about uninitialized memory) footguns prominently, but also because I am concerned that it's motivated by some current quirks in LLVM's X86 backend rather than anything inherent to the problem domain.
It's true that putting undef into some lanes of your vector values gives instruction selection more liberties, but at least conceptually, using a shorter vector type gives the same liberties -- the upper lanes don't even exist in the source program, so the compiler could put whatever you want in them. Admittedly, there are subtleties with that (as an analogy, when putting an i1 into a 32 bit register, a backend may choose to keep it zero extended at all times even though it doesn't have to for correctness -- it's a trade off). But still, it seems quite likely to me that a differently-written instruction selector would generate equivalent or better code for the "shuffle into smaller vectors" strategy.
In fact, I expect the good codegen for the "shuffle with undefs" variant to be partially because of hand-written isel patterns that specifically recognize this sequence of shuffles and adds (because some compiler engineer thought it important, or because some important piece of software did that operation in hot code). Such special-casing peephole optimizations are important for good codegen on natural code, but they're somewhat arbitrary and subject to the whims of said natural code. When the patterns don't fire on code you care about, the right fix is to extend the patterns (or add canonicalizations in the optimizer), not to contort the source code.
I'd also be interested in hearing whether there is a similar discrepancy (& apparent motivation for shuffle-with-undef) for (1) other targets or subtargets, and (2) other operations, e.g. things that aren't reductions, or reductions where there isn't a suitable "horizontal" instruction to exploit. It seems quite premature to throw the portable shuffle API in disarray just because one weird way to write a sum (for which there's already a built-in function in the RFC!) hits a codegen bug on AVX2.
1IMO uninitialized is a rather misleading term here, even if there's good reasons to want the same mechanism for it as for uninitialized memory
PS: one clarifying questions @RalfJung:

If any input to an arithmetic operation is uninitialized, the output is uninitialized.

In past discussion it was a plausible alternative for this to be instant UB rather than producing an undef/poison value. I believe that's even the current miri behavior. As you say, nothing is decided yet, but has this particular approach been ruled out while I wasn't looking?

In past discussion it was a plausible alternative for this to be instant UB rather than producing an undef/poison value. I believe that's even the current miri behavior. As you say, nothing is decided yet, but has this particular approach been ruled out while I wasn't looking?

No, you are absolutely right. I keep forgetting to switch between "compiler IR" and "surface language" when talking about uninitialized data.
For a surface language, immediate UB is perfectly reasonable.

So I was looking with @rkruppe into the code-gen issues yesterday and the problem was me: I was doing it wrong, calling llc directly instead of piping the output through opt first. YIL.
The issue with the smaller vector types is a real LLVM bug, but that is not something that we have to worry about here. Sorry for the noise.

I've opened an issue in https://github.com/rust-lang-nursery/packed_simd/issues/48 about removing the PartialOrd implementation of the vector types. Please chim in there if you want to discuss that. I'll update the RFC and the library with the outcome of that discussion.

When starting to work with an early version I ran into the problem of loading many f32s from a slice.
A detailed description of the problem can be found here.
In essence:

To load multiple vector elements I either have to call from_slice_aligned / from_slice_unaligned, which are safe but have some overhead for each vector loaded, or I have to load them via from_slice_aligned_unchecked / from_slice_unaligned_unchecked, which are fast, but unsafe
Performance difference between these two can be noticeable
I think there should be a way to convert a "slice of elements" of length x*N into a "slice of N-sized vectors" of length x, that is both safe and fast
Basic idea, do all the checks from_slice_aligned does (but only once) then re-interpret the rest of all elements
It's probably fine to have same assertions / error behavior as from_slice_aligned, e.g., panic if alignment is bad

Possible solutions:

Add a set of functions, e.g., try_as_f32x4s(my_slice)
Add a trait / generic method to do that conversion, e.g., f32x4::multiple_from_slice_aligned(my_slice)
Add simple iterators, e.g., f32x4::iterator(my_slice)

My recommendation in that issue was to just write this helper:
fn foo(x: &mut [f32]) -> Option<&mut [f32x4]> {
    // This operation succeeds only when it is actually safe to perform,
    // and no elements are lost:
    //  * the length of the slice is divisible by the number of lanes in the vector
    //  * the slice is aligned to an appropriate alignment boundary
    unsafe {
        if x.len() % f32x4::lanes() != 0 ||
        x.as_ptr().align_offset(mem::align_of::<f32x4>()) != 0 {
            None
        } else {
            Some(
                slice::from_raw_parts_mut(x.as_ptr_mut() as *mut f32x4,
                x.len() / f32x4::lanes())
            )
        }
    }
}

As @ralfbiedert mentions, for convenience, this RFC could provide this helper as an inherent vector method, e.g.,
impl f32x4 {
    fn foo(&[f32]) -> Option<&[Self]>;
    fn foo_mut(&mut [f32]) -> Option<&mut [Self]>;
}

I think these two methods would be safe as long as the appropriate alignment for the vector types is respected and we don't create slices with elements that are out-of-bounds (thus creating references that point to uninitialized memory), given that the RFC guarantees that f32x4 is layout compatible with [f32;4] which is layout compatible with an [f32] with 4 elements.
My only concern is how useful will these methods be. Vector types can have alignment requirements that on common platforms like x86 range from 16 (f32x4) to 64 bytes (f32x16). This means that  these methods will fail unless users obtain properly aligned memory.
AFAICT, the easiest way on stable Rust to obtain memory that would be properly aligned would be to:

use GlobalAllocator with a specially crafted Layout

use a repr(align) wrapper, e.g., #[repr(align(64)] struct F32(f32); let x: Vec<F32>;

both approaches currently still require unsafe code anyways: e.g. to go from *mut u8 to a &[f32], or to go from a &[F32] to a &[f32].
Without a reliable way to obtain properly aligned memory, any function that goes from &[f32] to &[f32x4] is probably going to fail very very often.

@gnzlbg good point of not having a safe and easy way to allocate such a slice predictably today.
I agree, in that case the cleaner approach would be to use Vec<f32x4>, and have a safe way to borrow them as a flat &mut [f32] when loading.
However, then there is no good way to do a runtime algorithm / vector width selection anymore. Instead I would have to "typedef" at compile time whether I want a Vec<f32x4>, Vec<f32x8> or Vec<f32x16>, and either waste cycles, or have wide vectors wider than what I might need (assuming simd_add and friends automatically map to the best CPU instruction available per vector width and instruction availability).

What about using align_to::<f32x4>? That should do exactly what you need, right?

@RalfJung the method looks really interesting for use inside of faster.  From a user perspective using std::simd I'd hope I can avoid unsafe for "basic stuff" (such as adding two slices).

Quick update about the issue of safe but fast vector loading: I now experimented a bit (see crate simd_aligned) and realized it is not as bad as I thought:

Going from &[f32x4] to &[f32] is much nicer than trying the other way (in retrospect kind of obvious) and feels like "the right way" of doing it if I look at it from an application level.
It only needs two relatively isolated unsafe calls to create a number of safe abstractions on top.
These abstractions can easily be put in a thin 3rd party crate.

While there could still be a minimal packed_as_flat(data: &[Simd]) -> &[Simd::Element] convenience function in std::simd, I now wouldn't mind if there is not.

Just keep in mind that the alignment for f32x4 differs from f32x8 which differs from f32x16.
So &[f32x4] -> &[f32] -> &[f32x8] won't work unless you use unaligned loads.
One day we will be able to just Vec<T, AlignedAlloc<Global, U32>> and get a vector that uses the Global allocator, but rounds up the Layout's align of T to a 32-byte boundary or whatever you need, but until then one will just need to be careful.

So I just released version 0.2 of packed_simd with some major changes that will need to be updated in this RFC:

[ ] a single Simd<[T; N]> type - all types exposed in this RFC are still available, but are just type aliases, e.g., f32x4 is an alias for Simd<[f32; 4]>.
[ ] vectors of {i,u,m}size - these are required to handle operations on packed vectors of pointers (e.g. like offset on all elements of a vector). Type aliases for these are provided, but are not very readable, e.g., isizex8.
[ ] support for pointer types, e.g, Simd<[*const *mut f32; 4]> - the vectors of pointers support common operations like offset, add, etc. as well as (masked) gather and scatter operations.
[ ] some renames here and there (::from_slice_unaligned, ::write_to_slice_unaligned_unchecked, shuffle! does not accept a single vector anymore, a permute! macro that accepts a single vector has been added)
[ ] other new features (rotates, permute_dyn, more mathematical functions, ...)
[ ] many more examples: all examples from the old simd crate have been ported to packed_simd, and two ISPC examples have also been ported
[ ] packed vectors do not implement PartialOrd/Ord anymore - instead, there is a method that returns a wrapper type that implements a lexicographical order, and implements PartialOrd/Ord.
[ ] From<[T; N]> for Simd<[T; N]> implementations for easily converting between vectors and arrays

The new release also comes with:

more examples: all examples from the old simd crate have been ported to packed_simd, and two ISPC examples have been ported as well, showing how to use the library in combination with rayon
the floating-point vector support in LLVM turned to deliver very bad performance (all calls were turned into scalar calls). The library now supports using a short vector math library called sleef for this which significantly improves performance for some workloads (up to ~10x). Ideally it will support using Intel SVML, libmvec, and similar libraries in the future. There is also a project to re-implement sleef in Rust and avoid the "pain" of having to link a C-library.

Missing RFC features in packed_simd:

saturating arithmetic,
checked arithmetic,
default arithmetic being checked - currently wrapping arithmetic is the default,
using as for numeric casts - currently a .cast() method is used for performing lane-wise numeric casts (equivalent to as per vector lane)

Open questions:

Currently, packed vector of pointer types are only provided for the lengths 2, 4, and 8. The largest size for a vector of 8 packed pointers is 512-bit. The library exposes up to 512-bit wide long vectors with up to 64 elements, e.g., i8x64 (512-bit wide long). A vector of pointers with 64 pointer elements would, however, be 4096-bit wide long. It feels weird to not support vectors of pointers up to the maximal number of elements that the other vectors of the library expose, but at the same time, a 4096-bit wide vector is a very big vector.
Implementing as for vector types might not be a great idea. The .cast() method is a solution, but the RFC XXX discusses better finer-grained solutions that we could implement. I'd prefer to move the as part of the RFC to the unresolved question section, until that RFC is resolved, and then re-evaluate.

@gnzlbg

permute!

permute_dyn

These aren't permutations, a permutation can't duplicate or drop elements.

These aren't permutations, a permutation can't duplicate or drop elements.

Yeah, naming is hard. Permutations with repeating elements are called n-tuples, where n is the length of the tuple, and they allow dropping elements (https://en.wikipedia.org/wiki/Permutation#Permutations_with_repetition).
The only advantage of the permute names is that they sound familiar to those that know the vperm... family of Intel instructions. I'd like to find better names for these. Combinations with multiple elements are called multicombinations, so maybe we could use multipermutations here. I don't know. Feedback is welcome. Worst case we can just go back to using shuffle! for both cases, and call the dynamic variants shuffle_dyn(indices) and shuffle2_dyn(vec2, indices).

I've always known that operation as swizzle

I've created a thread to discuss the naming issue: https://github.com/rust-lang-nursery/packed_simd/issues/102
Please submit a proposal as a comment, or vote on the existing ones (you can also discuss the names there).

@gnzlbg thanks for the update! In terms of stabilizibility I think it'd be easiest to not stabilize Simd<T>, and just wanted to confirm but is it possible with the crate currently to stabilize f32x4 and such without stabilizing Simd<T>?

In terms of stabilizibility I think it'd be easiest to not stabilize Simd, and just wanted to confirm but is it possible with the crate currently to stabilize f32x4 and such without stabilizing Simd?

I don't know the exact rules for visibility, but maybe we could do something like:
mod __rustc_private {
    pub struct Simd<T>...;
}
#[stable(...)]
pub type f32x4 = Simd<[f32; 4]>;

#[unstable(packed_simd_generic)]
pub type Simd<T> = __rustc_private::Simd<T>;

What we couldn't easily do without significant effort (and maybe not even in a backwards compatible way), is to use newtypes for f32x4 and the other type aliases to completely hide Simd<T> because the dereferencing vector of pointers generically kind of rely on all of these types being the same generic type: Simd<[*mut *mut f32; 4]> => Simd<[*mut f32; 4]> => Simd<[f32; 4]>.

Do the stable and unstable attributes work on impls ? If so, what we could do is limit which arrays Simd<T> accepts, so that if we wanted to e.g. stabilize f32x4 but not f32x8 we can stabilize the impl of Simd<[f32; 4]> but not stabilize the impl of Simd<[f32; 8]>.

Ah unfortunately a type annotation can subvert stability (users would still have somewhat of access to Simd<T>. Additionally unstable attributes don't work on trait impls, but they work on inherent methods of normal impls.

@gnzlbg ok cool! I wonder, would it be possible to use #[repr(transparent)] to make them newtype wrappers? If that's the case then we could get a rock-solid guarantee about the stable API

ok cool! I wonder, would it be possible to use #[repr(transparent)] to make them newtype wrappers? If that's the case then we could get a rock-solid guarantee about the stable API

The problem is that if we make them newtypes then f32x4(Simd<[f32; 4]>) is not Simd<[f32; 4]>. This can be problematic, for example, if you have Simd<[*mut f32; 4]>and load it from memory you get aSimd<[f32; 4]>which is a different type thanf32x4(which is a new type aroundSimd`).

@gnzlbg ah ok. I do suspect though that trying to work with Simd<[T; N]> will likely effectively block any stabilization until const generics is stabilized at least, and it itself will likely be pretty contentious.

I do suspect though that trying to work with Simd<[T; N]> will likely effectively block any stabilization until const generics is stabilized at least, and it itself will likely be pretty contentious

I understand that such a type definition could be contentious, but it is very common in the ecosystem (e.g. SmallVec<[T; N]>, ArrayVec<[T; N]>, GenericArray<[T; N]>. Also, I don't think const generics will improve the API nor the implementation of Simd<[T; N]> much... if at all...  in the same way that one doesn't need const generics to use SmallVec.

@gnzlbg and I had some discussion on IRC (pasted below), but unfortunately not a ton of conclusions :(

11:10 < gnzlbg> acrichto: the only improvement I can think of that cost generics
                would add to the ppv RFC is that instead of Simd<[T; N]> we will be
                able to write Simd<T, N>
11:11 < gnzlbg> but that will need a lot beyond just woboats const generics proposal
11:11 <@acrichto> gnzlbg: right yeah, but it's also sort of unknown waters for simd
                  and languages, I'm not sure if we'd be able to put it in libstd
11:11 <@acrichto> all I remember is like dozens of threads about this
11:11 <@acrichto> and would have to scour them again for arguments one way or another
11:11 < gnzlbg> my best proposal here is to expose Simd<[T; N]> in nightly only via
                a ::__rustc_Simd<T>
11:12 < gnzlbg> the second best is to use new types to hide that
11:12 <@acrichto> I don't think the way we treat typedfes can hide the generic param
11:12 <@acrichto> in that it's just the same type to rustc
11:12 <@acrichto> and can affect things like inference and method resolution and such
11:12 < gnzlbg> yes, the error will say ::__rustc_Simd<...>
11:13 < gnzlbg> typedefs go away, so essentially we would be stabilizing a
                __rustc_Simd<T> type, but making it illegal to use directly
11:13 <@acrichto> yeah but I'm not sure we could even do that
11:13 <@acrichto> like you could probably still use it somehow
11:13 <@acrichto> in a stable fashion
11:14 < gnzlbg> i mean, we can stabiliza Simd<T>
11:14 <@acrichto> in that our stability never was rock solid really, it's always
                  best effort
11:14 < gnzlbg> its just a type
11:14 < gnzlbg> without methods
11:14 < gnzlbg> like Simd<T> it cannot do anything
11:14 <@acrichto> oh sure, but that's not what I'm worried about
11:14 <@acrichto> I'm worried about the interactions with the tyep arameter
11:14 <@acrichto> across blocks and such
11:14 < gnzlbg> the type is actually Simd<T: __rustc_Simd_array>
11:14 <@acrichto> I don't really have a great articulation
11:14 < gnzlbg> where the __rustc_Simd_array is a sealed trait
11:14 < gnzlbg> so its not exposed
11:14 <@acrichto> I just know that not having generics is super simple
11:14 <@acrichto> if verbose
11:14 < gnzlbg> ah yes
11:14 <@acrichto> and is much easier to stomach stabilizing
11:14 < gnzlbg> yeah
11:15 < gnzlbg> I agree
11:15 < gnzlbg> i think that ideally we would only stabilize the 128-bit wide types
11:15 < gnzlbg> and see how that goes first
11:16 <@acrichto> we could try?
11:16 < gnzlbg> the 8-64 bit wide, and 256-512 bit wide types could be stabilized
                later, but with 128-bit portable simd one can do a lot already
11:16 <@acrichto> I can't really personally render a judgement on whether a type
                  parameter will work
11:16 <@acrichto> I'm just trying to act as a proxy for all the folks who commented
                  on historical threads
11:16 <@acrichto> where I thought there was enough pushback to generics to not do it
11:16 <@acrichto> I could be misremembering though
11:16 < gnzlbg> i recall something like that
11:17 < gnzlbg> for this, the element type and length need to be part of the type if
                we ever want to support packed vectors of pointers
11:17 < gnzlbg> but if we are not stabilizing that, then its not necessary
11:17 < gnzlbg> and I'd try to avoid that initially
11:18 <@acrichto> I could see that yeah
11:22 < gnzlbg> acrichto: I'll try to make the current type aliases newtypes. It
                might not be that much work - but it would be nice if the libs team
                could discuss this
11:22 < gnzlbg> there are a couple of libraries in the ecosystem that do this
                (SmallVec, ArrayVec, etc. come to mind)
11:22 < gnzlbg> and just because Simd<T> is generic, it does not mean that it
                accepts every T
11:23 < gnzlbg> Simd<T: sealed_trait> lets us control exactly which Ts it can
                accept, and stabilize these independently of one another
11:23 < gnzlbg> So Simd<[f32; 4]> could be stabilized without stabilizing Simd<[f32;
                8]> etc.
11:24 < gnzlbg> I don't know how that would interact with type inference though
11:24 < gnzlbg> and I see that it could be problematic
11:35 <@acrichto> gnzlbg: from an api-design perspective in a vacuum I think we'd
                  all agree that generics are the way to go
11:35 <@acrichto> (no way we'd ever say Vec<T> shouldn't exist)
11:35 <@acrichto> but SIMD I thought was a special case
11:36 <@acrichto> in the sense that the libs team probably wouldn't have many
                  thoughts on this
11:36 <@acrichto> other than "let's consult those using lots of SIMD"
11:36 < gnzlbg> i'm open to better suggestions
11:36 <@acrichto> and it may be the case now that with the platform apis the genrics
                  are fine
11:37 < gnzlbg> as in, we didn't use generics at first in std::simd, it just turned
                out that I didn't figure out a way to implement some features
                without them
11:37 < gnzlbg> and I still don't know how to implement them without them
11:37 < gnzlbg> but maybe others do / will
11:37 < gnzlbg> or in other words, the library doesn't use generics because it makes
                the api nicer, but because afaik it currently cannot be implemented
                without them
11:39 < gnzlbg> i wish i did not had to use generics actually, because that required
                re-implementing the whole library :/
11:39 < gnzlbg> this is why i would be fine with hiding generics in the API
11:45 < rkruppe> gnzlbg: i think we talked about it before but remind me, what were
                 these features that you couldn't figure out how to do without
                 generics?
11:47 < gnzlbg> rkruppe: packed vectors of pointers
11:48 < rkruppe> going from <n x T*> to <n x T> or what
11:48 < gnzlbg> the other way around
11:48 < gnzlbg> ah no yeah, that way
11:48 < gnzlbg> <n x T*> to <n x T>, <n x T**> to <n x T*>, etc.
11:49 < rkruppe> that screams "associated type" at me
11:49 < gnzlbg> when going from <n x T**> to <n x T*> what do you put in the
                associated type ?
11:50 < rkruppe> `<VectorType::Element as Deref>::Target`. well not literally Deref
                 because raw pointers, but like Deref
11:51 < rkruppe> i.e. a helper trait implemented on pointers such that `<*const
                 T>::Pointee = T`
11:51 < gnzlbg> so yes, kind of, since the result is also a vector
11:52 < gnzlbg> that's pretty much how it is currently implemented
11:52 < rkruppe> to be clear we're talking about the signature of gather?
11:52 < gnzlbg> yes, that type is used in that signature
11:53 < rkruppe> oh hm i think i see the problem
11:53 < gnzlbg> the thing is now you need to implement this trait for some concrete
                types
11:53 < gnzlbg> *concrete vector types
11:53 < gnzlbg> and you have *mut f32 and *const f32 which deref into f32x2, f32x4,
                ... depending on the vector length
11:54 < gnzlbg> so all of this are different types and can have different impls, so
                far so good
11:54 < gnzlbg> the leafs are concrete types, so a pf32x4 will dereference into a
                f32x4, which is nice
11:55 < gnzlbg> but now we wanted to also support dereferencing a ppf32x4 into a
                pf32x4
11:55 < gnzlbg> (I think sunfish eddyb you and maybe some others talked about this,
                that supporting vectors of pointers to pointers would be nice)
11:56 < gnzlbg> and that gives you an infinite amount of vector types that you have
                to support
11:56 < rkruppe> `ppf32x4` is `Simd<[*const *const f32; 4]>`?
11:56 < gnzlbg> for example
11:56 < gnzlbg> but we can have `Simd<[*mut *const f32; 4]>` (all of these
                combinations work
11:56 < rkruppe> i was actually thinking `ptrx4<T>` where T can be for example `f32`
                 or `*mut *const f32`
11:57 < rkruppe> well maybe two variants of that type if you want both mut and const
                 at the outer layer
11:57 < gnzlbg> i'd have to check the implementation but I think what I ended up
                doing is having a x4<T> vector type where T can be anything
11:57 < gnzlbg> so you can have x4<*mut *const *mut *const f32>
11:58 < gnzlbg> this is all an internal implementation detail anyways, none of this
                can be observed from outside
11:59 < rkruppe> yes that is how i understood the current solution
11:59 < gnzlbg> https://github.com/rust-lang-nursery/packed_simd/blob/master/src/vPtr.rs#L5
11:59 < gnzlbg> those types are actually exposed
11:59 < rkruppe> i am just trying to understand why it was necessary
12:00 < gnzlbg> and build on the x4 types (which are internal)
12:00 < gnzlbg> yeah, rkruppe so it was necessary to implement vector of pointers
                for all pointers
12:00 < gnzlbg> not only for pointers to a non-pointer value, but also for pointers
                to pointers
12:02 < rkruppe> the impls you linked seem to be generic over the pointee type tho?
                 as in, are there really different impls for e.g. `x4<*const T>` and
                 `x4<*const *const >`?
12:02 < rkruppe> [...] and `x4<*const *const T>`?
12:02 < gnzlbg> hmm no, they are only generic over the first *mut/*const T
12:02 < gnzlbg> depending on the first *mut/*const you can only read, or read/write
12:03 < gnzlbg> so the api of Simd<[*mut T; N]> is different than that of
                Simd<[*const T; N]>
12:03 < gnzlbg> whether T is an f32, or another pointer (e.g. *mut f32) is irrelevant
12:04 < rkruppe> so then i don't understand why you can't have one concrete type
                 (generic over `T`) corresponding to `Simd<[*const T; 4]>` and
                 another corresponding to `Simd<[*mut T; 4]>`
12:05 < gnzlbg> rkruppe: we have, those are the types exposed
12:05 < gnzlbg> type cptrx2<T> = Simd<[*const T; 2]>
12:05 < rkruppe> i mean something like `struct mutptr4<T>(...);` and `struct
                 constptr4<T>(...);`
12:06 < gnzlbg> yep, those are cptrx4 and mptrx4
12:06 < rkruppe> again i want to understand why you moved to having x4<T> /
                 Simd<ARRAY> at all
12:06 < rkruppe> i fear the signature of gather/scatter would involve an associated
                 type from hell but it still seems like it should be *possible*
12:07 < gnzlbg> rkruppe: ah because the generic impl that dereferences into the
                target type is easier if mutptrx4 and f32x4 are the same type
12:07 < rkruppe> `fn gather(self) -> <Self as GatherResult>::It` where `It` is
                 either another pointer vector or a vector of ints/floats
12:08 < gnzlbg> the impls for the pointer types are generic, there is only one
                impl<T> for cptrx4<T> { ... } and the target type is just an
                associated type Target = Simd<[T; 4]>
12:08 < rkruppe> yes, that is in many ways much nicer, but i don't see why it's
                 *necessary*
12:08 < rkruppe> oh well
12:08 < gnzlbg> if T is just f32 at the leaf, then Target == f32x4 automatically,
                while if T is *mut T, then Target is Simd<[*mut T; 4]>
12:08 < rkruppe> doesn't matter
12:08 < gnzlbg> it isn't necessary if you add all the impls automatically
12:08 < gnzlbg> *manually
12:09 < gnzlbg> the problem is, if you add a generic impl for all pointer types, and
                you add an impl for the leaf primitive types
12:09 < gnzlbg> they will conflict
12:09 < rkruppe> no point rewriting the library once again and having the length as
                 an explicit parameter is better for the future anyway
12:10 < gnzlbg> one might be able to work around this using a separate trait, but
                then you want to have an impl for all T's that are pointers, and one
                impl for all Ts that are not pointer
12:10 < rkruppe> gnzlbg: if you're talking the impl that provides gather, there
                 would just be one`impl<T> cptr4<T>` and the selection between
                 gather returning another vector pointer or a vector of leaf types
                 would be made with associated types dispatched on T
12:11 < gnzlbg> i am talking about the impls for the trait that is used to find the
                associated type
12:11 < rkruppe> that trait is implemented for `*const T`, `*mut T`, and the
                 float/int types?
12:13 < gnzlbg> that might work I think
12:14 < gnzlbg> looking at the impl of gather currently it dispatches to a concrete
                type
12:16 < gnzlbg> but i guess one can also add a trait with an associated type and
                implement it for the vectors of pointers
12:18 < gnzlbg> rkruppe so the problem is that particular trait
12:18 < gnzlbg> lets call it trait VecDeref { type Target; }
12:19 < gnzlbg> I can add  an impl<T> VecDeref for cptrx2<*mut T> { type Target =
                cptrx2<T>; }
12:19 < gnzlbg> and then I can add the manual impls for the concrete types, impl
                VecDeref for cptrx2<f32> { type Target = f32x2; }
12:20 < gnzlbg> so no, I don't think there is a problem, that should just work
12:21 < gnzlbg> I guess that can be retroffitted without many changes
12:21 < gnzlbg> so maybe we don't need to expose the Simd<[T; N]> type at all

oh also gisted a bit with @rkruppe which I didn't get a chance to follow over the weekend!

FWIW my recollection of the internals thread(s) I participated in about this is that people were concerned about it being premature to speculate about Simd<[T; N]>, it wasn't clear whether or how well it could work, it'd be more conservative and prudent to just stabilize f32x4 et al first, unknown unknowns, and that sort of thing. Process oriented and meta-level arguments. I don't recall there having been specific technical objections to Simd<[T; N]>. "I don't remember that there were" is not the same as "I remember that there weren't", of course.
If @gnzlbg has implemented it and it turned out that it not only works but works better than the alternatives, then that answers some of those questions, I think?

@glaebhoerl My main worry is that I don't know the answer to the following questions:

Can we put impl Foo { } blocks behind feature flags ? (to only stabilize some?)
Can we put impl Trait for Foo {} impls behind distinct feature flags ?  (to only stabilize some?)

so I don't know whether it is possible to stabilize some part of the implementation without stabilizing all of it (and instantaneously stabilizing all future additions).
If the answer to both questions is that yes, those things are possible, and trait impls behind feature flags do not affect type inference on stable, and on nightly when that particular feature flag is not active, then I think all my concerns would be resolved (cc @eddyb ?) .
@alexcrichton mentioned that similar discussions about generics have been had, and that other points about these issues have been raised. It might be worth it to dig those up: could someone share a link?

@gnzlbg yes impl Foo { ... } blocks (and methods) can be separately stabilized. We cannot, however, selectively stabilize impl Trait for Foo { .. }.

We cannot, however, selectively stabilize impl Trait for Foo { .. }.

So this might be the killer. To explain the problem, Simd<T> does not accept every T, it does not accept every array. For example Simd<[f32; 4]> is ok, but Simd<[f32; 3]> (and many others) is not.
The way that's implemented in the library is just using a trait to constrain T in the Simd implementation: Simd<T: sealed::SimdArray>. The private SimdArray trait is implemented in the library for some array times, like [f32; 4], but also some generic blanket impls like impl<T> SimdArray for [*const T; 4] where ... and similar.
This is problematic, because there are parts of the library that we don't want to stabilize at first, like 512-bit wide vectors, maybe vectors of pointers, etc. but we still want to be able to use and improve on nightly.
AFAICT const generics won't help. If we had a Simd<T, const N: usize> type we still have similar problems (e.g. restricting the Ns allowed, restricting T to not allow pointers at first, etc. ).
The only idea I have is that we could use conditional compilation to only put in core the pieces that are "ready" for stabilization, and that people would need to use the packed_simd crate for the "unstable" pieces. But that would mean that every time that we move something new to core it could become pretty much instastable, skipping the release train, which is a no go because the release train has detected many issues in the past when this was in std::simd.

@gnzlbg
Sorry about the slow response time. Looks great! Thank you. (I wish we could get safe wrappers around endianness-exposing bitcasts of vectors of the same bit width but different lane configuration sooner than later, though.)

PartialOrd<Self>: compares two vectors lexicographically.

Is it endianness-dependent? (Asking out of curiosity, not out of complaint, because endianness-exposing bitcasts were excluded from the RFC but endianness-dependent extract, new, etc. were included.)
@ralfbiedert

/// # Panics
///
/// If `slice.len() != Self::lanes()` or `&slice[0]` is not
/// aligned to an `align_of::<Self>()` boundary.
pub fn write_aligned(self, slice: &mut [element_type]);

Out of curiosity, have you checked if the compiler is smart enough to hoist the panic condition checks out of the loop when used with exact_chunks?

Is it endianness-dependent?

No. AFAIK one cannot create endian-dependent behavior with packed_simd unless one writes unsafe code to e.g. transmute an u64 into an [u32; 2] or similar. These types of issues also affect primitive types like u64->[u32; 2] though.

Out of curiosity, have you checked if the compiler is smart enough to hoist the panic condition checks out of the loop when used with exact_chunks?

Most of the recent work in the packed_simd examples and benchmarks has gone into beating ISPC in all benchmarks and manually removing these checks has played a non-negligible role in that. So what we have ended up doing is using the unsafe _unchecked methods pretty much in all kernels, but changing their implementation to include debug_assert!s so that they are checked in debug builds. That is, a method like vec.write_to_slice_aligned_unchecked(&[...]) does check that the target slice is large enough to hold vec and also suitably aligned if debug-assertions = true.
Having said this, there have been many changes that have improved performance in the recent weeks, so when all benchmark-related PRs have been merged it might be worth it to test this again by just enabling debug-assertions in release mode and running all benchmarks.

I'm closing this for now:

packed_simd has been updated with many new APIs since, and the RFC would need updating with those.

packed_simd is being updated to use const generics, which results in a couple of minor stylistic changes to the API (e.g. Simd<T, N> instead of Simd<[T; N]>). That work is not done, and it is unclear whether all currently-implemented and proposed APIs play well with const-generics.

This is kind of blocked on how much time I've got, and I don't think we should advance this RFC until both issues are resolved, so I'm closing this for now to reduce the RFC repo queue. Once that work is done, I'll update the RFC and re-open it or re-submit it if the changes are significant (I hope not).


Fetch import depends on non-package files

Importing fetch requires reading filetypes.dat which is installed external to the package.  Better to lazy-load any required files and bundle them in the package installation instead of $SKA/data/eng_archive.
This was a problem for building skare on a clean system because xija imported fetch, which failed.

Closed in 6bc041c5017ad4c4b6c998c2a7cec96cd23df475.


Fix a network policy typo

When syncing network policy, it reports "Spurious object" error.
This patch fix the correct way to get network policies.

@shettyg @rajatchopra This is a simple fix of syncNetworkPolicies. Could you please have a look at it?


Please give Github Copilot chat VISION 👀

Please can you give GitHub copilot eyes using OpenAI's GPT4 vision API?
As you can imagine, the use cases are endless and not limited to:

Referencing matplotlib charts to help recreate new charts
Analysing screenshots of websites or apps to create code to recreate those screenshots
Provided suggestions via code to help improve designs
etc ...

Here is the link to the API: https://platform.openai.com/docs/guides/vision
Thanks,
Babs

Yes, I want to pop up this feature.  With an image uploaded to Copilot, we can do someting like , for example:

For AI code: "Look at the red boxed part in the image, could you tell me how to recog this kind of stuff using OpenCV in python/C++"?
For UI design："I want to make a nice lookin g tab bar just like what I show to you, how to do it using flutter(it is ok if need 3rd party package, just tell me what and how to do)"

It looks like the cursor has already done this  https://twitter.com/amanrsanger/status/1729943964000883186 , hoping to see similar functionality in Copilot

what woould be awesome is if you could connect with figma and it would make the UI layout

Parsing and implementing Figma drafts would be 💪

Frankly speaking, now I am using Cursor not Vscode anymore, though I've already subscribed 1year for ms copilot 2 month ago before I moved to Cursor.
But I feel that be it be tech limitation or not paying enough attention,   I just feel that ms copilot is trying to add some features useless, like voice understanding(besides this voice understanding is really poor), github commit-msg auto generation.   For the real user-acquired feature: like the vision/image understanding and more  project context understanding , or doc and compser feature(from the Cursor), I see no roadmaps from them.
And even after the MS Copilot updated to OpenAi o1 mini model,  I still usually feel that the AI is just telling me superfical guidance like:  "You should check this, debug that ", and not really point out the core issues---it seems like the model is my boss, not the contrary way.

Frankly speaking, now I am using Cursor not Vscode anymore, though I've already subscribed 1year for ms copilot 2 month ago before I moved to Cursor.
For the Copilot team, be it tech limitation or not paying enough attention, I feel that the MS Copilot is trying to add some useless features, like voice understanding(besides, this voice understanding is really poor) and GitHub commit-msg auto-generation. For the real user-acquired features, like the vision/image understanding and more project context understanding, acquire updated msg from online search, or the doc and composer features(from the Cursor), I see no roadmap for them.
Even after the MS Copilot updated to OpenAi o1 mini model, I still usually feel that the AI is just giving me superficial guidance like: "You should check this, debug that...", and not point out the core issues---it seems like the model is my boss, not the contrary way.

I do not see commit-msg auto-generation as a useless feature. If you switch to small commits, the generated messages are pretty good. And lots of good commit messages result in marvelous, good pull request descriptions!

Frankly speaking, now I am using Cursor not Vscode anymore, though I've already subscribed 1year for ms copilot 2 month ago before I moved to Cursor.
For the Copilot team, be it tech limitation or not paying enough attention, I feel that the MS Copilot is trying to add some useless features, like voice understanding(besides, this voice understanding is really poor) and GitHub commit-msg auto-generation. For the real user-acquired features, like the vision/image understanding and more project context understanding, acquire updated msg from online search, or the doc and composer features(from the Cursor), I see no roadmap for them.
Even after the MS Copilot updated to OpenAi o1 mini model, I still usually feel that the AI is just giving me superficial guidance like: "You should check this, debug that...", and not point out the core issues---it seems like the model is my boss, not the contrary way.

I do not see commit-msg auto-generation as a useless feature. If you switch to small commits, the generated messages are pretty good. And lots of good commit messages result in marvelous, good pull request descriptions!

Maybe I put it too arbitrary. Anyway, we human have to do the monitor job CHECK anything the AI generated, which led me think the auto-short commit msg generation seems to be not quite useful. But you are right, it does can decorate the commit msg better, AS LONG AS we does know what all modified codes are, and we does at least have a rough idea on what is a good commit msg.

Frankly speaking, now I am using Cursor not Vscode anymore, though I've already subscribed 1year for ms copilot 2 month ago before I moved to Cursor.
For the Copilot team, be it tech limitation or not paying enough attention, I feel that the MS Copilot is trying to add some useless features, like voice understanding(besides, this voice understanding is really poor) and GitHub commit-msg auto-generation. For the real user-acquired features, like the vision/image understanding and more project context understanding, acquire updated msg from online search, or the doc and composer features(from the Cursor), I see no roadmap for them.
Even after the MS Copilot updated to OpenAi o1 mini model, I still usually feel that the AI is just giving me superficial guidance like: "You should check this, debug that...", and not point out the core issues---it seems like the model is my boss, not the contrary way.

I do not see commit-msg auto-generation as a useless feature. If you switch to small commits, the generated messages are pretty good. And lots of good commit messages result in marvelous, good pull request descriptions!

Maybe I put it too arbitrary. Anyway, we human have to do the monitor job: CHECK EVERYTHING the AI generated, which led me think the auto-short commit msg generation seems to be not quite useful. But you are right, it does can decorate the commit msg better, AS LONG AS we do know what all modified codes are, and we do at least have a rough idea on what is a good commit msg.

one more thing I want to share with you 😎 https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-copilot-vision

Found something that I want to share: https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-copilot-vision

@Khalizo Found something that I want to share: https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-copilot-vision … Somehow I think it closes that issue

I now had the chance to test the extension. It has one big disadvantage. You have to bring your own model and key. That would be ok for me, but the main disadvantage is that it does not support Models from Azure Open AI ... which is what I am using

Thanks for sharing this! I have moved across to Cursor now for a while now.
Really like the Claude 3.5 sonnet model for coding.
It's a shame Copilot can't iterate as fast...
On Sat, 2 Nov 2024, 07:51 Alexander Kastil, @.***>
wrote:

@Khalizo https://github.com/Khalizo Found something that I want to
share:
https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-copilot-vision
… Somehow I think it closes that issue
I now had the chance to test the extension. It has one big disadvantage.
You have to bring your own model and key. That would be ok for me, but the
main disadvantage is that it does not support Models from Azure Open AI ...
which is what I am using
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode-copilot-release/issues/554#issuecomment-2452910965,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFMRRLP67DJNTINA3PKFJVLZ6SAA7AVCNFSM6AAAAAA7DB762SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJSHEYTAOJWGU
.
You are receiving this because you were mentioned.Message ID:
@.***>

@Khalizo Found something that I want to share: https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-copilot-vision … Somehow I think it closes that issue

this ext is not good.   it need API key.....   but corsor is built-in for image


Review Notes

As promised, I went through the code, and here are my findings:
Kontent.Statiq

https://github.com/alanta/Kontent.Statiq/blob/main/Kontent.Statiq/Kontent.cs#L46

why not just return await Task.WhenAll(documentTasks);

https://github.com/alanta/Kontent.Statiq/blob/main/Kontent.Statiq/SortOrder.cs

this class seems to be completely unnecessary

https://github.com/alanta/Kontent.Statiq/blob/main/Kontent.Statiq/Kontent.cs#L36

There is no need to wrap the client in Lazy<>. Once an instance is injected, it's just there and there is no overhead accessing it.

Kentico.Kontent.Delivery

It seems to me that Kentico.Kontent.Delivery.Urls.QueryParameters should be part of the .Abstractions library

It's a common logic that creates an unnecessary dependence on the Delivery SDK and even worse, on its dependencies (such as AngleSharp). Would you agree?

https://github.com/alanta/Kontent.Statiq/blob/main/Kontent.Statiq/Kontent.cs#L46
why not just return await Task.WhenAll(documentTasks);

Good catch

https://github.com/alanta/Kontent.Statiq/blob/main/Kontent.Statiq/SortOrder.cs
this class seems to be completely unnecessary

Agreed

https://github.com/alanta/Kontent.Statiq/blob/main/Kontent.Statiq/Kontent.cs#L36
There is no need to wrap the client in Lazy<>. Once an instance is injected, it's just there and there is no overhead accessing it.

Looks like a left-over from the earlier versions where the client was instantiated by the plugin. Removed it.

It seems to me that Kentico.Kontent.Delivery.Urls.QueryParameters should be part of the .Abstractions library
It's a common logic that creates an unnecessary dependence on the Delivery SDK and even worse, on its dependencies (such as AngleSharp). Would you agree?

Yes, that makes sense.

URL generation was made public and together with the filters is part of the Kentico.Kontent.Urls package now.
https://github.com/Kentico/kontent-delivery-sdk-net/releases/tag/15.0.0-alpha4

not 100% sure if it's already in alpha4 but it'll certainly be in alpha5...sometime today :)

I'll check it out asap


added toString() to models

Status
READY/IN DEVELOPMENT/HOLD
Description

Type of Change

[ ] ✨ New feature (non-breaking change which adds functionality)
[ ] 🛠 Bug fix (non-breaking change which fixes an issue)
[x] ❌ Breaking change (fix or feature that would cause existing functionality to change)
[x] 🧹 Code refactor
[ ] ✅ Build configuration change
[ ] 📝 Documentation
[ ] 🗑 Chore

@sgehrman Thank you for your contribution! Please add a proper description to the PR and enlist the things that change.

"Please add a proper description"  I just did this as a favor to you.  I'm not going to do that.  I'll just use my fork if you don't want to merge it.  What's the point in insisting on this?

If you don't want to adhere to open-source contribution standards, don't contribute 🤷♂. You're not doing this as a favour to me or anyone else, but out of self-interest. Which is fine, but don't have a tantrum when you're told to do things the right way by the people whose software you're using.
But feel free to use your fork, man! God bless you, and have a nice day 🙂


Update to arrow 26

Which issue does this PR close?

Closes #.
Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

I'm getting failures in test_sort_fetch_memory_calculation which appears to have been added as part of #3593 by @isidentical. I'm having a hard time working out why it is failing, perhaps someone might be able help out?

I'm getting failures in test_sort_fetch_memory_calculation which appears to have been added as part of #3593 by @isidentical. I'm having a hard time working out why it is failing, perhaps someone might be able help out?

Somehow the implementation spilled memory, because it believes memory usage exceeded the limit. Maybe some change happened in arrow 26 not shrinking the buffer smaller somewhere?

@tustvold @Dandandan it seems like the limit is never applied for multi-column sorts when lexsort_to_indices is called. Probably due to the changes in https://github.com/apache/arrow-rs/pull/2929, where we previously created an array of (&value_indices)[0..len] but now just create it from value_indices directly without any limit (the single-column case is handled separately; and it still respects the limit).

@tustvold @Dandandan it seems like the limit is never applied for multi-column sorts when lexsort_to_indices is called. Probably due to the changes in apache/arrow-rs#2929, where we previously created an array of (&value_indices)[0..len] but now just create it from value_indices directly without any limit (the single-column case is handled separately; and it still respects the limit).

Ah, great find. @tustvold might be worth to block the RC to fix this issue?

Aah nice find will workaround.
Given it can easily be worked around by slicing the returned array I don't think we need to block the RC

First thing I recommend we do is to file an arrow-rs ticket explaining the issue and then we can discuss a new RC based on that issue. Doing that now

Filed https://github.com/apache/arrow-rs/issues/2990 -- will work on that shortly

Proposed fix https://github.com/apache/arrow-rs/pull/2991

I created a new release candidate for arrow

Arrow 26 has been released -- https://github.com/apache/arrow-rs/issues/2953

I plan on creating DataFusion 14.0.0 rc1 once this is merged.

I am working on updating this PR and polishing it for merge

sorry @alamb, I merged a PR that created a small conflict here

sorry @alamb, I merged a PR that created a small conflict here

No worries -- I just fixed it

🚀  FYI @Ted-Jiang


Add shellcheck, yamllint and boilerplate

What type of PR is this?
/kind feature

What this PR does / why we need it:
Add lint yaml, shellcheck, boilerplate
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
We migrate to another region and test is fix only in https://github.com/outscale-dev/cluster-api-provider-outscale/pull/105
Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.
TODOs:

[ ] squashed commits
[ ] includes documentation
[ ] adds unit tests

close https://github.com/outscale-dev/cluster-api-provider-outscale/issues/46

Closes https://github.com/outscale-dev/cluster-api-provider-outscale/issues/46


V5 update

PR summary

Capabilities support
Added additional Connection String types (MySQL, MongoDB EE, Datastax Enterprise)

Fixes:

Point In Time Restore Data response parsing

PR Checklist
Please make sure that your PR fulfills the following requirements:

[ ] The commit message follows the Angular Commit Message Guidelines.
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] New tests
[ ] Build/CI related changes
[ ] Documentation content changes
[ ] Other (please describe)

Does this PR introduce a breaking change?

[x] Yes
[ ] No

Other information

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Alex Hemard seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

The restarted PR build was clean, but for some reason the status is not reflected here in the PR:
https://app.travis-ci.com/github/IBM/cloud-databases-go-sdk/builds/246512721

Passing integration, example tests
(⎈ |:) cloud-databases-go-sdk (v5-update) $ make test-int
go test `go list ./...` -tags=integration
ok  	github.com/IBM/cloud-databases-go-sdk/clouddatabasesv5	196.379s
ok  	github.com/IBM/cloud-databases-go-sdk/common	(cached)

(⎈ |:) cloud-databases-go-sdk (v5-update) $ go test `go list ./...` -tags=examples
ok  	github.com/IBM/cloud-databases-go-sdk/clouddatabasesv5	204.069s
ok  	github.com/IBM/cloud-databases-go-sdk/common	(cached)

:tada: This PR is included in version 0.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


LazyVim

Hi,
Is it possible to use this plugin with LazyVim? I see you have 3 methods listed in the readme but noticed it's lacking LazyVim which is what I and many others use for plugin management. I had a quick try at getting it to work with no luck.

try this with the default LazyVim plugin setup style e.g. in lua/plugins/aider.lua
return {
	"joshuavial/aider.nvim",

	opts = {
		-- auto_manage_context = false,
		default_bindings = false,
	},

	keys = function()
		return {
			{
				"<leader>ao",
				"<cmd>lua AiderOpen()<cr>i",
				desc = "Aider Open",
			},
			{
				"<leader>ab",
				"<cmd>lua AiderBackground()<cr>",
				desc = "Aider Background",
				-- defaults to message "Complete as many todo items as you can and remove the comment for any item you complete."
			},
		}
	end,
}

Great thank you, that works. However when I run <leader>ao and Aider opens up, it weirdly creates 3 empty files: [1000], neo-tree and filesystem. Is that normal?

no, I don't get those other files.

It appears that this no longer works as intended. I've manged to put together:
return {
"joshuavial/aider.nvim",
config = function()
require("aider").setup({
auto_manage_context = true,
default_bindings = true,
})
end,
keys = {
{
"oa",
function()
require("aider").AiderOpen()
end,
desc = "Aider Open",
},
{
"ob",
function()
require("aider").AiderBackground()
end,
desc = "Aider Background",
},
},
}
after some troubleshooting under aider.lua in the plugins folder (Lazyvim the distro, mind you, not lazy.nvim.) The behavior I get from this (after adding the auto_manage_context and default_bindings fields to the .setup block, originally i didn't error out but i didn't get any behavior) is that the window flashes open in a split to the right for just a moment, the gitignore and neoconf get created, but immediately closes with error code 127 when i run in the background. Any ideas about what's going on? I'm relatively new to vim/Lazyvim/lua but pretty familiar with aider otherwise.

no idea, sry, currently running the same config as I posted above with latest LazyVim distro without problems.

return {
"joshuavial/aider.nvim",
config = function()
require("aider").setup({
auto_manage_context = true,
default_bindings = true,
})
end,
keys = {
{
"oa",
function()
require("aider").AiderOpen()
end,
desc = "Aider Open",
},
{
"ob",
function()
require("aider").AiderBackground()
end,
desc = "Aider Background",
},
},
}

I confirm this is working fine for me to install on lazy vim

Great thank you, that works. However when I run <leader>ao and Aider opens up, it weirdly creates 3 empty files: [1000], neo-tree and filesystem. Is that normal?

Did you manage to fix this issue?

the window flashes open in a split to the right for just a moment, the gitignore and neoconf get created, but immediately closes with error code 127 when i run in the background

I have the same issue. Did you ever get it solved @twilwa?

Lazy is using "old" which-key mappings, I've ended splitting the keys =  into:
config = function()
    requiere("which-key").add({.....})  -- instead of .register which is lazy defaults
end

the window flashes open in a split to the right for just a moment, the gitignore and neoconf get created, but immediately closes with error code 127 when i run in the background

I have the same issue. Did you ever get it solved @twilwa?

It's been a while, but I beleive the issue was something to do with my API key not being set as an env var, so verify that (as well as making sure that the venv your aider is starting from has the same env var) first.


Latest docker image no longer found

Which service(blob, file, queue, table) does this issue concern?
All?
Which version of the Azurite was used?
latest
Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
DockerHub
What's the Node.js version?
Unknown
What problem was encountered?
Trying to pull the image from DockerHub, but the latest tag was not found.
My exact use-case is with docker-compose, where one of the services uses mcr.microsoft.com/azure-storage/azureite as aan image.
docker pull mcr.microsoft.com/azure-storage/azureite
Using default tag: latest
Error response from daemon: manifest for mcr.microsoft.com/azure-storage/azureite:latest not found: manifest unknown: manifest tagged by "latest" is not found

Steps to reproduce the issue?
docker pull mcr.microsoft.com/azure-storage/azureite
Have you found a mitigation/solution?
No

Typo in docker name. Should "azurite" instead of "azureite".

That solved the issue for me.
Though I thought I had the same problem, after copy/paste the command in the documentation, but that too (now) works as expected.


Add basic documentation or example

Present simple example of interpreter usage. Maybe single _example_test.go.
Right know now I don't know where to start work with project.

Also thank you for your work and VK Go Conference presentation. Still looking forward to see go in Jupyter :)

Hi.
It's not supposed to be used yet. I will add docs and examples when it hits alpha release.


Cannot edit car owner cid

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
So when changing owner cid in the panel to transfer vehicles from user to other the new cid wont be saved any the vehicle is present on the original owner of the vehicles editing and saving new cid wont apply the new owner's cid
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
New cid should be saved and the vehicle should be transferred
Additional context
Add any other context or screenshots about the feature request here.
Replicated on c2 after having this issue on c8
https://i.itsguardian.xyz/u/yWZKJAwpqqdSzoLa75aJOte970Swe20V.gif

should be fixed now


Set kysely-bun-worker as an optional peer dep

Currently kysely-bun-worker, a package that provides a kysely sqlite dialect for bun, is set as optional: false. This causes unmet peer dependency warnings when installing packages if using a different dialect.
Example: yarn output
➤ YN0002: │ --@workspace:. doesn't provide kysely-bun-worker (pdcbd9), requested by kysely-codegen.

Perfect - merged into 0.14.0!


SSL server sockets that haven't completed handshake lead to log spew on loop close

This is very similar in root cause to issue #483 - SSL handshakes that have been initiated but not completed.   This code in sslproto.py:
if compat.PY34:
    def __del__(self):
        if not self._closed:
            warnings.warn("unclosed transport %r" % self, ResourceWarning)
            self.close()

leads to annoying log spew that is out of the control of the server developer.  The cause is wrapped SSL sockets that have not completed their handshake and that therefore the transport has not been provided to the protocol's connection_made() callback.  As server developer I therefore have no way to cleanly close down these transports.  So when the server admin shuts down the server, which does cleanly close all the transports it is aware of then does a loop.close() and exits, he is left with many messages like this sent to the logs.  This is unsightly and frustrating as I am unable to prevent it.
Perhaps the above code in __del__() should not be executed if the loop has been closed?
017-01-20 19:32:16.331209500 Exception ignored in: <bound method _SSLProtocolTransport.del of <asyncio.sslproto._SSLProtocolTransport object at 0x80a8e5e10>>
2017-01-20 19:32:16.331215500 Traceback (most recent call last):
2017-01-20 19:32:16.331218500   File "/usr/local/lib/python3.5/asyncio/sslproto.py", line 328, in del
2017-01-20 19:32:16.331220500     self.close()
2017-01-20 19:32:16.331222500   File "/usr/local/lib/python3.5/asyncio/sslproto.py", line 319, in close
2017-01-20 19:32:16.331224500     self._ssl_protocol._start_shutdown()
2017-01-20 19:32:16.331340500   File "/usr/local/lib/python3.5/asyncio/sslproto.py", line 542, in _start_shutdown
2017-01-20 19:32:16.331343500     self._write_appdata(b'')
2017-01-20 19:32:16.331345500   File "/usr/local/lib/python3.5/asyncio/sslproto.py", line 547, in _write_appdata
2017-01-20 19:32:16.331347500     self._process_write_backlog()
2017-01-20 19:32:16.331349500   File "/usr/local/lib/python3.5/asyncio/sslproto.py", line 649, in _process_write_backlog
2017-01-20 19:32:16.331359500     self._on_handshake_complete(exc)
2017-01-20 19:32:16.331361500   File "/usr/local/lib/python3.5/asyncio/sslproto.py", line 587, in _on_handshake_complete
2017-01-20 19:32:16.331364500     self._wakeup_waiter(exc)
2017-01-20 19:32:16.331366500   File "/usr/local/lib/python3.5/asyncio/sslproto.py", line 445, in _wakeup_waiter
2017-01-20 19:32:16.331368500     self._waiter.set_exception(exc)
2017-01-20 19:32:16.331378500   File "/usr/local/lib/python3.5/asyncio/futures.py", line 349, in set_exception
2017-01-20 19:32:16.331393500     self._schedule_callbacks()
2017-01-20 19:32:16.331395500   File "/usr/local/lib/python3.5/asyncio/futures.py", line 242, in _schedule_callbacks
2017-01-20 19:32:16.331398500     self._loop.call_soon(callback, self)
2017-01-20 19:32:16.331400500   File "/usr/local/lib/python3.5/asyncio/base_events.py", line 497, in call_soon
2017-01-20 19:32:16.331403500     handle = self._call_soon(callback, args)
2017-01-20 19:32:16.331412500   File "/usr/local/lib/python3.5/asyncio/base_events.py", line 506, in _call_soon
2017-01-20 19:32:16.331414500     self._check_closed()
2017-01-20 19:32:16.331416500   File "/usr/local/lib/python3.5/asyncio/base_events.py", line 334, in _check_closed
2017-01-20 19:32:16.331419500     raise RuntimeError('Event loop is closed')
2017-01-20 19:32:16.331421500 RuntimeError: Event loop is closed
20

@fafhrd91 This one also seems to be important. There was an open bug for it somewhere on bpo, I can't find it now.

this is actually another leak I want to work on. here is another issue KeepSafe/aiohttp#1679

Yep, it seems the bug in aiohttp is caused by this.
Speaking of sslproto.py I don't like its design. I think it's possible to simplify it a great deal, which should also improve performance. If, while working on sslproto, you have any ideas how we can rewrite it, let's discuss.

sure


Feature: exclude columns

This is a:

[ ] bug fix PR with no breaking changes
[x] new functionality
[ ] a breaking change

Description & motivation
dmt already supports the ability to compare only a subset of columns, and this PR supports the inverse.
I require this because there's an array column in my data that dmt struggles to mock, so I want to exclude it. Currently, I have to list about 15 columns to include, which isn't ideal if someone adds a new column and forgets to update the mock input/output.
Checklist

[x] I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)

[x] BigQuery
[ ] Postgres
[ ] Redshift
[ ] Snowflake

[x] I have updated the README.md (if applicable)
[x] I have added tests & descriptions to my macros (and models if applicable)

@mjirv do you have an eta of when you'll be able to review this one? 🙏


Add auto assign annotation

Create button which toggles auto annotate allowing new regions to be automatically assigned a annotation
What is needed
Add button to toggle the mode
create popup/dropdown to select the label the user wants to automatically put

Demo ready: https://github.com/UCSD-E4E/Pyrenote/tree/apply-previous-annotations

Carries over previous annotation from last selected region


configs: fix GPU's default number of HW barrier/CU

AMD GCN3 and Vega GPUs assume a max of 16 WG/CU.  Any GPU WG with more
than 1 WF requires a hardware barrier to allow WFs in the WG to synchronize locally.
However, currently the default gem5 GPU configuration assumes only 4 barriers per
CU, which artificially prevents applications with > 4 WG/CU that could run
simultaneously from running simultaneously.
This fix resolves this by updating the default number of hardware barriers per CU
to 16, which mimics the support described in slide 39 here:
https://www.olcf.ornl.gov/wp-content/uploads/2019/10/
ORNL_Application_Readiness_Workshop-AMD_GPU_Basics.pdf
Change-Id: Ib7636a13359d998e676c1790f436a83ce88cbfc0

@mattsinc I created a new pull request to fix the issues.

@KaiBatley please go ahead and merge this -- congrats on your first patch submission!


Enable lifecycles in buckets

We should enable and deploy lifecycles in S3 buckets that we use for backups so that we don't keep backups forever and waste lots of money.
This adds the feature. Further details in commits.
/cc @rjw1 for lifecycle lengths in https://github.com/alphagov/govuk-terraform-provisioning/commit/ac105ab0a5ae38fbf69167e072f8970a4a61c3e8

they look like numbers :)

but are they the right numbers?!

Merge away.


ClusterLoader - Adding test name to config

In future there will be added option to run multiple tests in single clusterloader execution. Cause files will be created (for example: measurement summaries) for every test, there is a need to include test name in file name to prevent name collisions.

/assign @shyamjvs
/cc @wojtek-t

/lgtm

/retest


Bump react dependency to 0.14.6

I'm having some issues when I shrinkwrap my package with formsy-react... Can we bump the dependency to the latest react?

Sorry for the long pause. Formsy-react updated, check new version.


maximize command not found in examples - changed to maximize_utility

The examples were not working. Looks like the maximize command was changed to maximize_utility at some point during dev. The commands are changed in the examples in this fork, where appropriate.

@taylor-curley, thanks for catching that!


The bottom of truncated headings text is not visible

When I use a bootstrap heading with text-truncate, the bottom of text is not visible.

jsfiddle: https://jsfiddle.net/qx1b4sLu/
I know how to fix it (make line-height equal or larger than font-size). However, I think that it will be very useful if it is defined by default when heading and text-truncate are used together.

Yeah we should increase the line-height. Hoping 1.25 or similar works?

@mdo
In my case, I overwrote line-heght to 1.3 like:
h1, h2, ..., .h6 {
 &.text-truncate {
    line-height: 1.3;
  }
}

However, top and bottom space of these headings are slightly different from other headings that doesn't have text-truncate class (= line-height: 1.1). Unfortunately I have no idea to make them the same height.