Alloca for Rust

(rendered)

Using a function for this would be a mistake IMO. It's not a true function in C either, for various reasons.
&[x; n] producing &[T] for runtime values of n is more appealing from an implementation perspective.
That is, LLVM's stack variables (its alloca instruction, which is poorly named but whatever) have an optional "dynamic arity", which can be considered to be 1 for regular variables.
This model is closer to VLAs than alloca, and it could work well with box [x; n] too (which would evaluate n first, allocate, then initialize the slice with copies of x, and produce Box<[T]>, Rc<[T]> etc).

I do know some people are very wary of alloca/vlas in rust, but I can't remember who....

It's worth being careful to avoid accidentally considering any constant n as runtime at the same time though, so fixing issues like https://github.com/rust-lang/rust/issues/34344 As otherwise Foo<&[x; n]> becomes messy and fixing it risks breakage.

@burdges We already have an analysis which treats rust-lang/rust#34344 correctly - the problem there is the evaluator (which is AST-based and pre-typeck, this will change next year hopefully), not the qualifier.
A simpler and far more problematic example would be something like if false { 0 } else { 1 } - and I have to agree here, as much as I'd like box [x; n] for runtime n (even w/o VLAs).

@steveklabnik That might've been @cmr.

Could we get an example of how one might use this reserve function in the RFC? Because I can't tell just from its signature. It basically returns a &[T] so you can't mutate it? (unless it's supposed to only be used with Cell types?). And it seems to me that function lets you read unitialized memory (mem::reserve::<Droppable>(2)[0]) but it's not marked as unsafe?

@burdges Oh I remember now, it's just that it's been a while since I thought about this.
The solution is to only do this if the expected type is &(mut) [T] to begin with and still require constants otherwise. This is one of the rarer cases where type ascription makes a noticeable difference, I suppose.

SmallVecs work well enough and have the added benefit of limiting stack usage.

As I have already said in an issue somewhere, the assembly generated by SmallVec is very disappointing and saying that it "works well enough" is not really true.

As you can see this is really suboptimal. Instead a solution that uses alloca would just do a sub esp followed with four mov and you're done. Five instructions.

SmallVecs work well enough and have the added benefit of limiting stack usage.

I am unconvinced this is a benefit. When properly implemented, alloca is safe, extremely cheap, has nice locality and aliasing guarantees and even avoids an extern crate alloc;. In functional-ish languages you usually have a generational GC where the young heap has most of these nice properties but I do not really expect to see one in Rust any time soon, much less an efficient implementation.

Interesting @eddyb so fn g() { let mut x : &[T] = &[T::new(); f()]; ... } should not compile without the : &[T] if the ... only uses x for scratch space and does not return it or anything?

Thanks everyone for the great comments! I'm going to follow @oli-obk 's advice and change to the closure based version. Need some time to do it, though... :smile:

@llogiq I'm trying to tell you that anything using functions is unnecessarily suboptimal from an implementation perspective. The feature LLVM has is effectively VLAs. clang implements alloca(n) as something like &((char[n]) {})[0] (not sure if that literally works in C. char a[n]; in any case).

if the ... only uses x for scratch space and does not return it or anything

@burdges I'm not sure what you mean here, you can never return a stack local and rvalue promotion is backwards compatible in that expanding the set of constants only allows more code to compile.

@llogiq Apart from @eddyb's concerns, which I also support, using a closure will make this feature significantly less useful. For example then you could not call alloca in a loop while accumulating the results, which I would like to rely on for writing zero-heap-allocation parsers.
Also, in general I am opposed to deliberately decreasing usability in arbitrary ways because of your (or anyone else's) subjective opinion of whether it is "considered bad". You are not making a tradeoff with some other feature, as it usually goes in language design, you are simply making my life harder for no reason at all.

Now, a constructive suggestion: it looks like there is no existing way to reflect the lifetime of alloca exactly, and I agree with other commenters that introducing VLAs seems like an excessively invasive change for a fairly niche use case. How about making it a built-in macro?
let n = 10; for i in 1..n { let ary: &[u8] = alloca![0; n]; }

This is nicely similar to the existing case of vec![] and I believe it could return a value with any lifetime it wants, especially given that in this case the lifetime is simply fully lexical (the entire containing function).

@whitequark While that could work, doing it with an intrinsic is more problematic than having it first-class.
Hmm, we can make the macro encapsulate the unsafety by using a helper function that takes a reference to some other stack local and equates the lifetimes, only problem being that I'm not sure how to create something like that which lives long enough to be useful. May need to abuse some of the rvalue lifetime rules to get it.

I suppose there's always the possibility of making the intrinsic generic over a lifetime that the compiler restricts artificially.

Yes, that's what I was thinking about. The intrinsic would merely translate to alloca of the right type directly, and the lifetime will be restricted early (you could do it right after you introduce lifetimes into the AST, really). I think this is a very nonintrusive solution.

@eddyb  Ok, so it would need to be a compiler-internal macro. That's a neat solution to both the lifetime problem and the problem of extending the compiler with as little fuss as possible.
One thing I should probably add – because this is low-level, performance-über-alles, we'd likely not want to initialize the memory, so we could expand to mem::uninitialized(), rigging the type as necessary, but the whole thing would be unsafe and should be marked as such. I'm not too deep into macros; do we have unsafe macros?

One thing I should probably add – because this is low-level, performance-über-alles, we'd likely not want to initialize the memory, so we could expand to mem::uninitialized(), rigging the type as necessary, but the whole thing would be unsafe and should be marked as such. I'm not too deep into macros; do we have unsafe macros?

No, this isn't necessarily the case. There are different motivations for using alloca: performance, avoiding using the heap (because there might not be one), maybe others. I want to use this alloca facility but I still zero the memory first and in particular I don't want to have to use unsafe for it as avoiding unsafe is a key selling point for what I'm buliding.
I'd suggest instead that the feature should allow the user to somehow specify the value to fill the memory with. Then the user could pass unsafe { mem::uninitialized() } if it doesn't want the memory to be zeroed and if it is OK with using unsafe, or it could use [0u32] or whatever the element type is, if it wants it to be zeroed.

I agree with @briansmith on the safety aspect. I think that  a safe interface should be provided as main interface with a additional  way to have it uninitialized.
Through passing in mem::uninitalized() might not work depending on how it is normally initialized. (initializing through a closure would call mem::uninitalized() ntimes and copy the result to the place in the stack (should get optimized away or ??) and initializing with Clone would try to clone uninitialized memory, lastly initializing with Copy might be to restrictive...)
So maybe alloc![...] and unsafe alloc_unitialized![...].

Ideally the initializing version would take an iterator so it could be initialized with something other than copies of the same value.

Ideally the initializing version would take an iterator so it could be initialized with something other than copies of the same value.

Ideally that would be true for let x: [T; N] = <iterator>; too, but it isn't, and we get by. If you specify it as an iterator then you have to deal with the case where the iterator iterators over a different number of elements than what is requested, which seems like too much complexity for this.

@comex That would be collecting into a "VLArrayVec" (which needs to handle partial drop in case of panic, on top of what @briansmith already mentioned).

I think the main stopper for initializing [T; N] with something else than Copy is (not) having numbers as generic arguments, else you could enable <[Foo; 10]>::from(|| Foo::new()) and <[Foo; 10]>::try_from(&mut iter) for iterators (where the type of err represents a partial initialized slice containing informations about how many elements where initialized and a unsafe method to retrieve the partial initialized array, continue initialization etc.).
But implementing this for alloca/reserve would not work, as it is not really a method, so you would have to implement it with a multiple macro's (or a "multi branch" macro). Probably slightly broken e.g.: alloca![0u32; 10], alloca_from_fn![|| Foo::new(); 10], alloca_from_iter![ iter; 10].
I'm not sure but I think there was some work to properly name space macros,  so it would be possible  to have something like stack::reserve::by_value![v; n], stack::reserve::from_fn![...] etc.
Note that the above mentioned try_from/from implementations would have to use catch_unwind to prevent a partial initialized slice from being dropped fully, I'm not sure what the overhead of this would be after optimizations.

Is there any need to initializing an alloca this way?  About the only reason I can imagine is because you want an immutable array of either variable or fixed length.  In that case, you can always initialize it mutably and then borrow it immutably under the desired name.  I suppose the syntax would be roughly
let mut arr_mut = [Default::default(); n];
for i in iter { arr_mut[i] = ...; }
let arr : &[u8] = &arr_mut;

or
let arr_mut : &mut [T] = alloca![Default::default(); n];
for i in iter { arr_mut[i] = ...; }
let arr : &[u8] = arr_mut;

@briansmith @comex @dathinab I agree that we may have multiple motivations to use this prospective feature, each with its own cost-benefit tradeoffs.
I think we have a different situation to initializing a Vec, because external code has no way of accessing the slice before we're done with it, so we actually have three options of dealing with it:

We can simply mem::forget() what we have so far and propagate the panic to the caller. This should be a noop, also the program is broken at that point anyway, no need for clean up.
1.1 We can also drop() what we have so far and then propagate the panic. However, this would swallow the original panic in case the drop() implementation panics, or we'd have to swallow panics from the drop. Both options are suboptimal.
Return a Result<&mut [T], Box<Any + Send + 'static>> (which is pretty much #1 + a plain catch_unwind + try!/?)
Return a (&mut [T], Result<(), Box<Any + Send + 'static>) containing the slice filled so far and the optional error. This would have the benefit of allowing the caller to use what has been initialized so far while still providing error information. This could be implemented with the uninitialized_stack_slice!(ty, num) macro, catch_unwind(..) and a bit of machinery. Note that this version is strictly more useful than #2.

I'm personally in favor of option 1, mostly because I don't want to complicate the implementation (and the return type), also catching panics is an antipattern anyway. With an iterator we can still restrict the returned slice to the actually initialized memory.
Regarding initialization, the following benchmark:
#![feature(test)]
extern crate test;

use test::Bencher;
use std::mem;

#[bench]
fn once(b: &mut Bencher) {
    b.iter(|| {
        let x : &[usize; 4096] = unsafe { mem::uninitialized() };
        x
    });
}

#[bench]
fn each(b: &mut Bencher) {
    b.iter(|| {
        let mut x : [usize; 4096] = unsafe { mem::uninitialized() };
        for mut e in (&mut x[..]).iter_mut() {
            *e = unsafe { mem::uninitialized() };
        }
        x
    });
}

shows 0ns for both once and each when compiled with optimizations (though there may still be a flaw in the benchmark). So writing an iterator that simply returns mem::uninitialized() is probably just as fast and would allow us to do the right thing with only one macro. Even so, I'd personally be more happy with one unsafe-fast and one safe-still-fast version of the macro.
@burdges reading uninitialized memory provokes undefined behavior.

Threw this function together.... There is a good question as to whether rustc will compile it to anything as efficient as simply subtracting from the stack pointer (I suspect no) but it seems to solve some of the complexity issues.
macro_rules! match_arms {
    ($n: ident, $f: ident, $t: ty; $($ns : expr),*) => {
        match $n {
            $($ns => (|| $f(unsafe {&mut ::std::mem::uninitialized::<[$t; $ns]>()}))(),)*
            _ => panic!("n was too large in alloca"),
        }
    }
}

pub fn alloca<T, F, R>(n: usize, f: F) -> R
where F: FnOnce(&mut [T]) -> R
{
    match_arms!(n, f, T; 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
                17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,
                33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48,
                49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64)
}

You can obviously go up to a way higher number, I have it working up to 1024. There is also a possiblility of using it with a T: Default bound (and avoid the unsafe), but unfortunately you can only go up to 32 in this case because of rust limitations.
You would use it like so:
#[test]
fn allocate_and_sum() {
    for i in 1..64 {
        let x: usize = alloca(i, |arr| {
            for (j, elem) in arr.iter_mut().enumerate() {
                *elem = j;
            }
            arr.iter().cloned().sum()
        });
        assert_eq!(x, i * (i - 1) / 2);
    }
}

@djzin Sadly, it doesn't optimize (select Release and then either LLVM IR or ASM).
In fact, it uses the largest array size as the static stack size, so there's no gain over ArrayVec.

@eddyb
Shame that it doesn't optimize - at least this pushes the problem to one of optimization rather than adding a new language construct.

In fact, it uses the largest array size as the static stack size, so there's no gain over ArrayVec.

Is this correct? In my testing it was not doing this; that's why I added the closure invocation (|| f(...))() - I do get stack overflow sometimes, but only when actually passing in a large number into alloca with the type T being [usize; 1024] or some such.

Shame that it doesn't optimize - at least this pushes the problem to one of optimization rather than adding a new language construct.

It doesn't as this is less powerful than a proper alloca; the lifetimes are restricted to just the closure body whereas in reality they are the same as the entire calling function's frame.

@whitequark surely any function defined as such:
fn use_alloca(args: Args) -> Output {
    let x = alloca(n);
    function_body
}

can be rewritten like so
fn use_alloca_closure(args: Args) -> Output {
   alloca(n, |x| function_body)
}

or am I missing something?

@djzin You're essentially suggesting to rewrite all my code in continuation passing style. That would be bad enough if it worked, but it doesn't. Consider what happens when you are trying to allocate in a loop. You would have to rewrite the loop as a recursive closure invocation, but this isn't actually possible in Rust AFAIU:
fn main() {
    let mut closure = || {
        closure();
    };
    closure()
}

error[E0425]: unresolved name `closure`
 --> <anon>:3:9
  |
3 |         closure();
  |         ^^^^^^^ unresolved name

@djzin It's probably due to constants. You can obscure values with black_box from the test crate.

@whitequark you could probably trampoline, but the ergonomics would be abysmal.

@whitequark ah I see it now, you want to write something like this
fn loop_and_allocate(n: usize) {
    let mut x = alloca::<&mut [i32]>(n);
    for i in 0..n {
        x[i] = alloca::<i32>(i);
    }
    // do stuff with x
}

which indeed requires some trickery, silly me :)

I rest my case. :smile:

@eddyb I finally see what you are seeing now. I made an interactive version of this - when run in debug mode it allows me to allocate up to 15 elements, and will crash if I type in 16. When run in release mode the program immediately segfaults without typing anything. This means that, if my program could possibly segfault, a valid "optimization" by LLVM is to compile my entire program into a segfault. So I concur with the need for this RFC ^^

@whitequark could you tell me a bit more about your use case? This could be something for the Rust patterns repo once this feature is live & stable (Note to self: Add TODO to "Howto teach" section).

@llogiq Sure. First, let me make a small digression with some context, to try and explain as to what brought me to this evidently unusual solution.
So, ARTIQ is a system that lets physicists control high-speed I/O ports in realtime using a Python-like language. Since it is Python-like, people naturally expect to be able to use features requiring allocation, such as lists or list comprehensions, and expect to not need to manually manage memory or watch out for use-after-frees. Since it requires hard realtime, anything resembling a traditional allocator was out of question.
I've solved this in the ARTIQ Python dialect by dedicating the entire 4G of FPGA's onboard memory to the stack, allocating everything using alloca, and implementing a fairly rudimentary region checker. This actually worked remarkably well, and has so far caused much fewer complaints than I expected.
ARTIQ is an asymmetric multiprocessor system: one CPU runs ARTIQ Python, and another CPU runs the TCP/IP stack and management software (written in Rust). After reimplementing the management software in Rust, I've discovered that Rust often forces you to allocate, especially when working with strings or byte buffers or recursive data structures. Many of such allocations live no longer than the stack frame of the current function--so alloca would be perfect! If it existed.
Here's the rough overview of the use cases I wish to get from alloca:

allocating a dynamically sized array of bytes;
allocating a dynamically sized array of arrays of bytes;
allocating an arbitrarily deep AST, with caller doing the allocation;
allocating a String or concatenating a few Strings;
all of the above, but additionally, stack-overflow-free.

Now, some of these are easier than others. Allocating an array of bytes can obviously be done with any of the above proposals. The ones below are more complex though.
Here's how I think it all could mesh together:

allocating a dynamically sized array of bytes: the alloca![0; n] macro, as proposed above;
allocating a dynamically sized array of arrays of bytes: the same proposed alloca! macro, for which lifetimes permit this, unlike the closure proposal;
allocating an arbitrarily deep AST: the AST would use borrowed pointers, and the caller would explicitly use alloca! as opposed to box and place the pointer inside; a box with an embedded lifetime would be ideal for genericism but possibly unrealistic;
allocating a String or concatenating a few Strings: assuming we have a String that can allocate its buffer in an arena (whether in libstd or elsewhere), this means creating an arena beforehand, populating it using alloca, and then using it when creating a String;
all of the above, but additionally, stack-overflow-free: simply use the arena approach above, and then let it either panic or return Err depending on your taste.

Especially when using the arena, this is a proposal that is not inherently harder to control than, say, traversing arbitrary data structures recursively. I think that the most sketchy part here is whether box could ever be used together with alloca, but even if not the proposal still has a lot of value.

@whitequark Hmm... isn't this likely to run into ergonomic issues?  It essentially blocks abstraction by forcing you to write how much you want to allocate up front, rather than delegating it to a callee function.  You say you can create an arena populated with alloca, but that's an extra layer of indirection and an opportunity to crash at runtime if you run out of space.  I guess there isn't any clearly better option, but still... ideally heap allocation and freeing should be a handful of instructions each, or you could have an 'alloca stack' separate from the call stack (in which case you can ask a callee function to allocate in your alloca-stack frame, though this only works if it doesn't do its own allocation there).
anyway, I'm not against supporting alloca as an option, just spitballing.

Hmm... isn't this likely to run into ergonomic issues?

Depends. There are several options here:

The "I just need fifteen things at worst" option: you aren't any more likely to get a crash than e.g. trying to recursively process a tree, so this is fine.
The "I allocate a lot and all of my RAM of stack" option: don't bother with an arena, just allocate. Right now, if you're trying to allocate in Rust and get an OOM you will crash, so this isn't any worse, at least.
The "I allocate a lot and I'm running in an OS thread" option: yep, running into ergonomic issues here. If you're bumping into allocator slowness and your workload gives you a nice way to estimate a bound on memory consumption then you will easily use an arena, otherwise you should just stay away from alloca.

ideally heap allocation and freeing should be a handful of instructions each

Indeed, but this is empirically not true and I don't see how this will change any time soon.

an 'alloca stack' separate from the call stack (in which case you can ask a callee function to allocate in your alloca-stack frame, though this only works if it doesn't do its own allocation there)

I don't see any obvious reason why you would manage to avoid both the (i) heap allocation problems, e.g. say you mmap it and what happens when you bump into the next mapping? (ii) stack allocation problems, e.g. say you use a fixed contiguous area and what happens when it is exhausted?
It does have the benefit that the use of space on the alloca stack would perhaps be fully predictable, with the knowledge of the allocator, but I'm not sure how much that will help you in practice, especially looking as data structures don't usually divulge the sizes of interior allocations.

@whitequark Maybe I'm misunderstanding something, but the problems you're describing sound more like arguments for custom allocators in general than alloca specifically, especially since you used the term "arena" and referred to many data structures besides byte arrays. And we already have an accepted RFC for custom allocators (https://github.com/rust-lang/rust/issues/32838), it just needs to be implemented.
On the other hand, I'm not sure how a custom StackAllocator could be implemented without some kind of alloca/mem::reserve operation exposed by std, so maybe we do still need a new primitive. I'm definitely not familiar enough with the subject to know either way, but I feel the relationship with allocators in general should be addressed in the text of the RFC.

I'm not sure how much that will help you in practice, especially looking as data structures don't usually divulge the sizes of interior allocations.

I think this is exactly the problem that parameterizing data structures by allocators is supposed to solve.

Maybe I'm misunderstanding something, but the problems you're describing sound more like arguments for custom allocators in general than alloca specifically

They are both. However, you can replace libcollections with whatever you want right now, but you cannot write alloca yourself, so I am far more concerned about alloca.

@whitequark very interesting. Thank you for your in-depth reply.
Regarding ergonomics, I'm thinking about if we could get away with one macro that returns a newtype. Then we could implement the functions to init or unsafely get the slice as methods on that newtype, e.g.
let x = stack!(Frazzle, number_of_frazzles).of(copyable_frazzle);
let y = stack!(usize, 42).from_iter(0..42);
//etc.

@llogiq Is the primary motivation here the fact that alloca would not provide an equivalent of Vec::collect()?

@whitequark Exactly. Since we need the macro to construct the alloca'd value and we don't have an expressible type for it anyway, collect() is a non-starter here.

@llogiq I feel a bit iffy about introducing such a high-level wrapper for alloca, but on the other hand, the problem you mention does exist, and I can't easily think of a better replacement.

(Bikeshed: keep alloca![elem; count], add alloca![for iter; count] or something like that, which is maybe a little more consistent with vec!?)

@whitequark we don't even need the count in the for iter version if we can get it from the iterator.

@llogiq Thanks, I am much happier about this design. A few comments:

Should the iterator panic, the macro will forget(_) all values inserted so far and re-raise the panic.

Shouldn't this be rather drop(_)? Why not?

On unices, the stack can usually be extended at runtime, whereas on Windows stack size is set at link time (default to 1MB).

The stack for the initial thread. On all OSes when creating a thread you may specify the required stack space explicitly, and Rust exposes this in a platform-independent way with std::thread::Builder::stack_size.

With this functionality, we lose the ability to statically reason about stack space.

This is not true. In reality, we have 'lost' the ability to statically reason about stack space the moment we have added virtual calls (closures, trait objects, etc). If you really are trying to statically predict stack space usage, you are confining yourself to a much smaller subset of the language; see for example the MISRA-C guidelines. This means:

never using alloca (duh) or VLAs
never calling functions recursively*
never calling functions through a function pointer
never calling functions from dynamic libraries

I am going to argue that anyone who cares about being able to reliably predict stack usage will not use std, further, they will not use core as core relies on trait objects for e.g. formatting, and fairly heavily relies on closures, and without cross-crate inlining you will get virtual calls. (This is something I had to investigate just recently...) alloca itself does not create a regression here.

Adding this will increase implementation complexity and require support from possible alternative implementations / backends (e.g. Cretonne, WebASM).

This is a good point, although I believe that all such implementations will be able to use a "shadow stack", and they will also face a motivation to provide a community-agreed-upon way to implement alloca for portability reasons.

Overflowing the stack space leads to segfaults at best and undefined behavior at worst.

This is definitely troublesome, and I believe that until Rust gains proper portable stack probes, the alloca feature should not be stabilized.

Bikeshedding: Can we find a better name?

I still think alloca! is better but I am not going to argue with anyone about this unimportant detail.

@llogiq In light of the above I propose to rewrite the doc comment as follows:
/// **Warning:** the Rust runtime currently does not reliably check for
/// stack overflows. Use of this feature, even in safe code, may result in
/// undefined behavior and exploitable bugs. Until the Rust runtime is fixed,
/// do not use this feature unless you understand the implications extremely
/// well.
///
/// The `stack!` macro works much like an unboxed array, except the size
/// is determined at runtime. The allocated memory resides on the stack;
/// when allocating, be careful not to exceed the size of the stack, or
/// the *entire process* will crash. The stack size of the main thread
/// is operating system dependent, and stack size of newly spawned threads
/// can be set using `std::thread::Builder::stack_size`.
///
/// The `stack!` macro is primarily useful on embedded systems where heap
/// allocation is either impossible or too costly, where it can be used
/// to obtain scratch space for algorithms, e.g. in sorting, traversal,
/// parsing, etc.

@whitequark
Usually, the reasoning is that forget(_) is safer than drop(_) because it cannot panic!(..) and leave the memory in an observably inconsistent state (see http://cglab.ca/~abeinges/blah/everyone-poops). I don't think this applies here though, because there's no way to get at the allocated region anyway – only the slice filled should be dropped by the end of the function.
This means we could safely drop(_) all elements returned so far on iterator panic without any problems. If there is a discrepancy between the size of the memory region and the actually returned slice, we need to forget(_) the rest of the region anyway, lest uninitialized memory become observable.
Also since people might use this to store e.g. file descriptors, dropping values on panics would be a net improvement in applicability.
On an unrelated note, I think that leaving the feature unstable is not the correct response here – if it's unsafe as of yet, let's mark it accordingly. We can remove the unsafe during the course of stabilization.
I like your suggestion regarding the docs; will copy into the RFC shortly.

On an unrelated note, I think that leaving the feature unstable is not the correct response here – if it's unsafe as of yet, let's mark it accordingly. We can remove the unsafe during the course of stabilization.

This has no practical difference as sometimes notionally safe unstable Rust features let you bypass the type system and memory safety anyhow, so I have no strong opinion on this choice. It does seems like less work to keep it safe but unstable, and once it stabilizes less external code will need to get updated, though.

However, it can be argued that static stack reservations (e.g. let _ = [0u64;<PHONE_NUMBER>]; already suffices to do this. Perhaps someone should write a lint against this.

Um, no. Not a lint. As Rust exists right now, safe Rust code (without alloca or anything) is simply not actually memory-safe on all platforms without stack probes, and it should be exploitable although in somewhat restricted conditions. The only correct solution is to implement stack probes for every platform.

There is one slight hiccup with regards to dropping on panic: The drop could panic, so we'd have to make the tough choice whether to raise the original panic or the panic from the drop.

@llogiq What tough choice? If a drop panics during a panic, then Rust simply aborts.

Also re naming: As low-level programmers are the most likely audience for this feature, keeping the "alloca" name will make for better discoverability.

Ok, now we drop everything on panic. Thanks for setting me straight, @retep998

@llogiq a few minor clarifications:

more of these algorithms could run at full speed – and would be available on systems without an allocator, e.g. embedded, soft-real-time systems

I would put hard-real-time here, since it is not immensely complicated to get soft-real-time behavior by careful usage of an allocator and a GC (see: Erlang), but hard-real-time is the real challenge here, and alloca is not only perfect for tackling this but it is also how I use it right now, as discussed in https://github.com/rust-lang/rfcs/pull/1808#issuecomment-266225999.

/// The stack! macro

The doc comment should be updated to say alloca!.

With this functionality, we lose the ability to statically reason about stack space.

I would put this as: "With this functionality, trying to statically reason about stack usage, even in an approximate way, gains a new degree of complexity, as maximum stack depth now depends not only on control flow alone, which can sometimes be predictable, but also on arbitrary computations." This avoids the issue of large arrays allocated on stack (which aren't really relevant here as there's nothing special about an array, you can do the same with a deeply nested structure), and also elaborates on the specifics of what is meant by "lose the ability".

@whitequark thanks. I copied your suggestion verbatim.

@llogiq Now to get onto something more substantial:

Is the feature as defined above ergonomic? Should it be?

In my opinion it is quite ergonomic; I can see nothing more to wish for (and actually I wouldn't have come up with the iterator suggestion myself). Regarding a question raised earlier about whether it should be available in libstd, my argument is as follows:

alloca! is supposed to be memory-safe, and thus it is not less deserving of being exported in libstd than many wildly unsafe things already exported there, like std::mem::transmute. It is also not implementable outside of libstd, which I believe is the policy for inclusion. Finally, including it in libcore but omitting it from libstd would, to my knowledge, will make it the first feature to ever get this treatment.
Right now alloca is not memory-safe, but this is a symptom of a far more sinister underlying issue that is not even introduced with alloca, so I don't think right now it makes sense to restrict alloca specifically in that regard (especially while it's unstable anyway).

How do we deal with the current lack of stack probes?

 (That, in my opinion, should be fixed at LLVM level, but it's an A-an-interesting-project level thing.) I consider them a problem that should be urgently fixed on its own regard.

@llogiq Thanks. I have no more suggestions on this RFC, except the suggestion to merge it ;D

stack overflows (in today's safe Rust) are a persistent and extremely nasty problem on my system right now

For some context on how bad it is, depending on the exact access and allocation patterns and luck, a stack overflow will:

corrupt heap metadata;
corrupt the state of the network stack;
corrupt vtables in .rodata;
all of the above.

You can guess how fun it is to debug any of the above.

I feel you.

Maybe we can implement alloca only on platforms that have stack probes?

@retep998 That would leave out essentially every platform where alloca is wanted (since the current architecture of LLVM can only provide stack probes on hosted platforms by design).

Maybe we can implement alloca only on platforms that have stack probes?

I'd be OK with this only if we also forbid stack-consuming recursion on those platforms. As it's a change to ensure safety, I believe it would technically be allowed even though it would be breaking.
:)

I would like to add that the drawbacks that this can't be supported in all cases for stackless coroutines.

@Zoxc As any other form of stack usage. Do you have a specific example I could cite? Perhaps there's a way to identify and make alloca![..] report a helpful error in those cases?
@eddyb @sfackler is one of you willing to shepherd this RFC?
Also note to self: Propose extending vec![..] with vec![for $iter:expr] variant that expands to $iter.into_iter().collect::<Vec<_>>()`

I would really prefer not to open another memory safety hole. How about having mandatory size checks until we fix stack probes?

@pcwalton how do you propose implementing size checks in a way that's significantly easier than just adding stack probes?

@pcwalton one could argue that this is only another way to exercise the same problem with potentially large stacks we already have.
We need stack probes, alloca! or not.

If the lack of stack probes is the only issue blocking this, we could add "improvised" stack probes just for alloca! by performing a read_volatile at every 4KB interval in the newly allocated memory to check for stack overflows.
These would be replaced by proper compiler-based stack probes once support is implemented in LLVM.

@Amanieu Just don't add those "improvised" stack probes on platforms with actual stack probes.

Yes please. Also add them for static initialization >4k

Stack probing doesn't affect this proposal. We either have them or we don't.

Has somebody confirmed that values of stack-allocated-dynamically-sized types are unreasonable to implement in a reasonable timeframe? For most applications of this, they are clearly the preferred solution because they compose with other proposed features, such as alignment control and future proposals such as must-zero-when-drop or no-optimizing-away-drop.
Otherwise, could you add a guarantee to the RFC that the first byte of the alloca will be aligned to a cache line boundary or at least a SIMD alignment boundary?

Last time I checked, dynamically-sized stack types would require type numerals, which are wanted for various things but notorically complex and hard to get right, whereas this proposal can be implemented with comparatively little effort.
There's an orthogonal concern: The function-bound lifetime allows for a number of use cases that would be impossible with a normal block scope restricted lifetime.

Otherwise, could you add a guarantee to the RFC that the first byte of the alloca will be aligned to a cache line boundary or at least a SIMD alignment boundary?

Such a guarantee would be very nice to have, but I'm not sure how to do that portably.

Note: I understand that stack probes are implemented on -msvc Windows targets because they are part of the Windows ABI or similar. The consequence of this is that the implementation of alloca must implement the probing logic on -msvc targets. OpenSSL had a bug where they were doing a custom alloca in their modular exponentiation assembly language code without _chkstk, which would cause segment faults when large RSA and DH keys were used. They fixed this just this year.

Otherwise, could you add a guarantee to the RFC that the first byte of the alloca will be aligned to a cache line boundary or at least a SIMD alignment boundary?
Such a guarantee would be very nice to have, but I'm not sure how to do that portably.

LLVM's alloca has an align attribute. I don't think it makes sense to do this for every allocation (imagine allocating u8s) but certainly, when a type with a high alignment is allocated, that alignment must be respected--this is required for memory safety anyhow.

@llogiq

Last time I checked, dynamically-sized stack types would require type numerals, which are wanted for various things but notorically complex and hard to get right, whereas this proposal can be implemented with comparatively little effort.

I think @briansmith was referring to @eddyb's idea, where the result type, being a dynamically-sized type (as is the whole point) such as &[T], would explicitly not involve any type-level numbers. I also think that's a cool idea and would be curious about its prospects. (I don't have any personal stake in which feature we get or when, just interested.)

I think @briansmith was referring to @eddyb's idea, where the result type, being a dynamically-sized type (as is the whole point) such as &[T], would explicitly not involve any type-level numbers. I also think that's a cool idea and would be curious about its prospects. (I don't have any personal stake in which feature we get or when, just interested.)

Right. My understanding is that it is already possible for a struct to be dynamically-sized and the struct's length is not encoded in the type. Therefore, it should be possible to create an dynamicaly-sized-array type where the length isn't encoded as a type parameter. In fact, for my own code, I absolutely wouldn't want the length to be encoded as a parameter of the type because I don't want my code to be instantiated N type for N different lengths during monomorphization.
Also, I suspect @whitequark's intended usage is actually exceptional in wanting to have a stack-allocated arena. In most cases, I think people would prefer to be able to use alloca() for non-array types. For example, in my case I have types Elem { limbs: [Limb] } and Scalar { limb: [Limb] } where I want to declare several variables, x, y, z to be values of Elem and Scalar for some dynamically-chosen number of limbs. Being familiar with OpenSSL, I know they use alloca() for similar purposes, but only because they can't use VLAs or alignas due to restricting themselves to C89, but they'd be better off using the Elem/Scalar type interface if they didn't restrict themselves to C89.
Put another way, VLAs seem like a much better solution in all ways except maybe it might be politically easier to add an alloca() in a reasonable time.

Also, I suspect @whitequark's intended usage is actually exceptional in wanting to have a stack-allocated arena. In most cases, I think people would prefer to be able to use dynamic stack allocation for non-array types.

We are in agreement here; I also want to allocate non-array types. In fact one of my motivations for working on this is a case where I needed to parse some metadata (statically bounded in depth and always well under a hundred entries) into an AST.

Put another way, VLAs seem like a much better solution in all ways except maybe it might be politically easier to add an alloca() in a reasonable time.

We've discussed lifetime ascription with @eddyb on Twitter, let me summarize some points.
One of the possible benefits of lifetime ascription (that you would need to use with VLAs to get alloca-like functionality) is that VLAs plus lifetime ascription could actually be more generic than alloca. (By "lifetime ascription" here I mean "explicitly specifying the lifetime of a pointer of a literal that is taken, placing the literal on the stack".) Consider this example:
fn doit() {
  loop {
    let len = receive_len();
    let mut data = alloca![0; len];
    receive_data(&mut data);
    process_data(&data);
  }
}

In this case, unless the content of the loop is moved into another function, the stack will eventually overflow. If we could use lifetime ascription with arbitrary pointers to literals, then we could rewrite it as:
fn doit() {
  loop {
    let len = receive_len();
    'a: {
      let mut data = &'a [0; len];
      receive_data(&mut data);
      process_data(&data);
    }
  }
}

This could be implemented using LLVM's stacksave/stackrestore feature around the nested scope.
Drawbacks of this include:

Needs new syntax: 'a { ... } is currently only permitted for loops.
Not obvious how the new feature will interact with the rest of the type system. Where can you put such a bound lifetime variable?
There are patterns that are impossible to actually assemble on stack while guaranteeing that the memory will be freed (@eddyb, I think you've missed this part in your suggestion):

fn foo() {
  'a: {
    'b: {
      let x = &'b 42;
      loop {
        let y = &'a 10;
      }
    }
  }
}

These can all be statically checked for, but it's still work for the compiler, and it makes the feature more confusing.

I like the ideas of unsized arrays and lifetime ascription. I consider them vastly superior to this amateurish hack RFC. They are more general and more orthogonal in every respect. I heartily support any RFC for one of those features. Is anyone writing them already?
I particularly like lifetime ascription because it uses two existing syntactic features (lifetimes and block labels), and unifies them to create something more powerful than each in isolation. Also it allows us to make lifetimes visible in the code. IDEs could use the syntax to show existing lifetimes. Apart from making the type system more powerful, this is a great teaching device.
Unsized arrays are also a crazily cool feature; and looking at my proposal, at least the HIR part would be essentially the same. Especially the possibility of having a bunch of methods implemented on arrays of any size is sure tempting. There are however some interactions that would have to be carefully designed – especially casting (can a sized array be un-sized? What about vice versa? What about slices?) and type compatibility, especially with regards to
specialization.
One problem with unsized arrays is creation – and this is where this RFC could still be valuable. Even if we have an unsized array on the stack, we will want to safely initialize it (or unsafely keep it uninitialized for later use). The herein suggested macro delivers so good ergonomics that I've made a note to suggest extending the vec![..] macro to match it. Perhaps there's even a
better way to do this that I'm not seeing yet.
So I'm going to list unsized arrays and lifetime ascription as extensions under the Alternatives section.

I also like the idea of lifetime ascription, but I have no idea how to do that for non-lexical lifetimes, and I'm not convinced any of the benefits of the feature really hold up if we have to restrict it to only lexical or near-lexical lifetimes. In particular, how would SEME (single entry multiple exit) lifetimes be ascribed?
(if anyone's unfamiliar with why SEME lifetimes are useful, see http://smallcultfollowing.com/babysteps/blog/2016/05/09/non-lexical-lifetimes-adding-the-outlives-relation/)

Once you can name lexical lifetimes you can use generics to define other lifetimes outliving them (however, there's no guarantee the borrow checker will accept them.

Once you can name lexical lifetimes you can use generics to define other lifetimes outliving them (however, there's no guarantee the borrow checker will accept them.

This is basically the reason I am so enthusiastic about this RFC; it is less generic but it is far more clear how it works and integrates with the rest of the language.

Should we introduce more macros which cannot expand to syntax? The new merged procedural macros require such an expansion to exist.
I think a better idea is to introduce an intrinsic which allocates bytes and build the alloca! macro on top. The proposed macro also seems quite ambiguous.
Now the intrinsic would however give terrible errors for generators/coroutines, so I think some actual syntax is needed here.

@Zoxc We can check intrinsics just as well as direct syntax. I'm expecting the intrinsic to take both the element to repeat and the number of elements in order to be forward-compatible with VLA [x; n].

Posting here as well to get more eyeballs.
As much as "lifetime ascription" could be an interesting feature, I don't believe we need it for alloca. What we need is an unsafe intrinsic, re-exported through core::ptr, like this:
// libcore/ptr.rs
#[inline(always)]
pub fn stack_reserve<T>(count: usize) -> *mut T {
    core::intrinsics::stack_reserve(count)
}

...that returns uninitialized data. That would be good to have anyway; since alloca will be used in performance critical code, so having something with minimal overhead seems useful.
We can then build a safe abstraction around that, by relying on existing borrowck and a macro to make the borrowed variable inaccessible. I have a proof of concept implementation of that here, feel free to try to poke holes in it.
With just an alloca intrinsic, we can let people build their own safe (or unsafe) abstractions around it for a while, then if we have one that's good enough we can select it for inclusion into std.

#[inline(always)]
pub fn stack_reserve<T>(count: usize) -> *mut T {
    core::intrinsics::stack_reserve(count)
}

The principle is good, but I was wondering if something like this is actually possible?
The stack space is supposed to be "free'd" when the function returns, so the stack_reserve function here would reserve some stack memory, then unreserve this memory and return a pointer to it.
You're going to say "but the pointer remains valid, so who cares?", but the problem is that if I call stack_reserve multiple times I don't want the pointers returned by the various calls to overlap.
Even if there's some dark magic that makes it possible to make this work, I think this could be confusing from the user's point of view.
Maybe it's actually better (both for the rustc implementation and the users that will use alloc) to have a stack_reserve macro instead? Since all macros are directly embedded in the code, at least it is very clear that we're doing the alloca for the current function.

I guess the #[inline(always)] is intended to work around that problem. However, AFAIK, we leave always-inline-handling to LLVM, i.e. the function above will generate a shim LLVM function (for each T) that looks like %ptr = alloca %bytes; ret %ptr, and then run an LLVM pass which inlines that function. By LLVM's rules, the alloca return value is only valid for the call to that shim, so LLVM would be within its rights (and the normal inliner does that AFAIK) to insert lifetime markers, i.e. after inlining the call the alloca would still be as short-lived as the call to the shim and actually using it will be UB.
If my analysis is correct, a shim function is right out, unless rustc is changed to implement inline(always) in MIR before generating LLVM IR --- but this may have other repercussions.

so LLVM would be within its rights (and the normal inliner does that AFAIK) to insert lifetime markers

It's even easier, it is legal for LLVM to convert any %a = alloca; ret %a sequence into unreachable.

I'm in favour of alloca and think it would meaningfully improve Rust. However, I have doubts about how it fits in with the 2017 roadmap, and thus whether it should be prioritised.
I think alloca has to be a first-class language feature (a VLA, I guess) with its own syntax. I really don't like the magic macro approach in this RFC - as I see it, a built-in macro has no advantage over new syntax, except that it doesn't look like new syntax. And it has the disadvantage that real syntax would probably be more ergonomic.
Some of the initialisation problems from the comment thread feel like they should be solved by placement box/the <- operator.

Not sure if you intended this with your comment, but I don't think concerns about implementation priority or roadmaps should affect the RFC process. It is okay if an RFC lands and is not implemented by anyone because of priority. If folks really want it they can implement it themselves. Similarly, I don't see what the roadmap has to do with this -- we didn't plan for allocate, but if people want it and there's general consensus about it we shouldn't make "roadmap" an issue.
I can understand if there is an interaction with future planned features that we feel can't be resolved until that future feature gets rfcd. This happens pretty often and is okay, if annoying. Barring that I don't see how the roadmap affects things :|
(Mostly agree with the rest of your points about syntax and placement new)

However, I have doubts about how it fits in with the 2017 roadmap, and thus whether it should be prioritised.

To condense some discussion on Twitter:
The alloca feature is primarily useful on embedded, and right now and for foreseeable feature that will require using feature flags anyway. Therefore, an unstable alloca! macro that gets scrapped in a couple of years and replaced with VLAs+asscription would be a perfectly fine solution for almost everyone who will want to use alloca.

The alloca feature is primarily useful on embedded, and right now and for foreseeable feature that will require using feature flags anyway. Therefore, an unstable alloca! macro that gets scrapped in a couple of years and replaced with VLAs+asscription would be a perfectly fine solution for almost everyone who will want to use alloca.

Given this (which seems a reasonable position to me), I'd be interested in exploring what the max-min solution here is (with the intention that it is not a long-term solution). E.g., could we do this as an unsafe intrinsic in a simple manner? ( I realise that this is not good enough for @briansmith 's use case, but perhaps it is for most other cases? And presumably could be wrapped in safe code (using closures, or whatever).

@tomaka (and others)

The principle is good, but I was wondering if something like this is actually possible?
Maybe it's better (both for the rustc implementation and the users that will use alloca) to have an opaque stack_reserve! macro instead

Okay, I stand corrected w r t what LLVM is currently allowed and not allowed to do with functions that are inline(always). We can make a built-in stack_reserve! macro instead, if that's easier than changing how Rust/LLVM does inline(always).
I do hope that such a macro will stabilize in the not too distant future, though. If nothing else so for the reason that we don't want "one more thing you can only do on nightly".

(fn is a keyword of course so I don't think that's necessary)

Good point.

And presumably could be wrapped in safe code (using closures, or whatever).

No. The ergonomics of using closures to reintroduce control flow are prohibitive and it would be no better than not having alloca at all. See this example.
In my opinion the minimally viable solution is the alloca![uninitialized $ty; n] macro returning an uninitialized slice, although once that's in, building a convenient safe interface (allowing to initialize the values) would be a few lines of macro-by-example code, to provide alloca![$expr; n] and alloca![$($expr),*].
@llogiq's suggestion of initialization from iterator is rather elegant but by no means necessary.

building a convenient safe interface (allowing to initialize the values) would be a few lines of macro-by-example code, to provide alloca![$expr; n] and alloca![$($expr),*].

There's a good argument for delegating this to an external crate, namely that once VLAs and lifetime ascription is in, the crate could transparently provide the alloca! syntax as a shim to those, so I came agree with @nrc in that core/std should only provide the minimal abstraction (which I still believe is alloca_uninitialized! special macro).

I'm fine with that and am in the process of rewriting the RFC. Again. As they say, the best writing is rewriting. :smile:

@llogiq Which way do you go? VLAs or alloca_uninitialized!?

I think we may as well go the VLA route.

@llogiq With lifetime ascription? What's your strategy for handling impossible constraints of lifetimes?

As I wrote before, I'd suggest allowing for an 'fn lifetime. This is a fairly minimal form of aacription that nonetheless gives us a lot of bang for little buck.

Sounds good to me.

This has been quite the thread. I wonder if someone could take the time to try to summarize all the of the back-and-forth and the various proposals? I've started 2 or 3 comments by now only with various ideas, only to find that someone had already proposed some variant of what I was going to say, so I definitely think it's time. =)

Some thoughts:
On a 'fn lifetime
I am mildly reluctant to add some sort of special 'fn lifetime here. I agree that this is a potential use-case for such a feature, and I think we should keep that in mind -- and probably add it to the list of a few others -- but it seems like it causes this feature's "complexity scope" to grow considerably. I think that I'd prefer some sort of primitive based on *mut and/or special macros, things like that which don't affect the "surface area" of the language on the whole. I guess the bottom line is that 'fn still feels insufficiently "fundamental" to me, and I'd rather hold out on adding new special lifetimes until we have a better feeling for our complete needs -- if ever.
On initialization
I'd like to drill into the arguments around uninitialized memory a little more. Dealing with uninitialized memory in unsafe code is a very problematic point -- consider unwinding, for example, which can cause reads you might not have considered -- and I think the impact of zeroing etc is sometimes overestimated. In any case, I agree with @briansmith's intution that initialization or lack thereof feels like something we should try to make orthogonal.
On an intrinsic
I agree with @eddyb that packing alloca as a function call is a bit...odd, but it is something with precedent. I just want to point out that the problems around #[inline(always)] are basically an artifact of how we do intrinsics right now, which is something I would like to change anyway. The model I've been kind of quietly promoting -- but haven't properly written up yet -- is that we just generalize the concept of a "lang item" to "some definition well known to the compiler" (this is already the case in many ways). The idea then would be that alloca is a function which, when invoked directly, has the special meaning of allocating memory in its callers function that is freed when the caller returns. It returns a *mut; it is up to the caller to manage it.
In other words, the function in libstd would be declared something like:
#[lang(alloca)]
pub unsafe fn alloca<T>(n: usize) -> *mut T {
    panic!("alloca() cannot be invoked indirectly")
}

Note that the body is a panic! -- this is because, if you invoke alloca(22) directly, you never call the body at all. The body is only used if you were to invoke it indirectly, like let f: fn(usize) -> *mut u8 = alloca; f(22);
(This is a twist on the usual pattern for "lang-item intrinsics", which is to have the body call itself, so that indirect calls "just work" the way one might expect without the compiler needing special code. Since indirect calls don't make sense in this context, that's fine.)

I think that I'd prefer some sort of primitive based on *mut and/or special macros, things like that which don't affect the "surface area" of the language on the whole.

I don't see how it is possible to make a safe abstraction without something equivalent to a 'fn lifetime, whether reified in syntax or just as an implementation detail of an intrinsic/macro.

On initialization

I understand your concerns. Personally, I don't really intend to alloca anything with a destructor (or even use unwinding much...) so I have not gave it much thought. This is something that does warrant further investigation.

On an intrinsic

This works. I thought this could only be done with a macro, but if a function works (and simplifies compiler internals), sure.

I suggest that the VLA stuff and the lifetime stuff be split into separate RFCs. They are orthogonal. There are lots of uses for VLAs that don't need the lifetime stuff.
To see why I say they are orthogonal, consider again the example psuedo-code above:
fn loop_and_allocate(n: usize) {
    let mut x = alloca::<&mut [i32]>(n);
    for i in 0..n {
        x[i] = alloca::<i32>(i);
    }
    // do stuff with x
}

Here's roughly what it would look like using VLAs instead of alloca, using made-up-:
fn loop_and_allocate(n: usize) {
    let mut x = [unsafe { mem::uninitialized() }; n];
    for i in 0..n {
        x[i] = &[0; i];
    }
    // do stuff with x
}

And here's an analogous example of the same issue, which doesn't use alloca or VLAs at all:
fn loop_and_allocate(n: usize) {
    let mut x = Vec::new();
    for i in 0..n {
        let item = 7;
        x.push(&item);
    }
    // do stuff with x
}

To clarify, I just wanted to keep the 'fn lifetime because it's semantically what LLVM already does with alloca. I'm fine with splitting the RFC in two, one for VLAs and one for an 'fn lifetime. In my mind, the two are orthogonal anyway (which is exactly what I like about them). Still, VLAs are useful even with default lifetimes, so this is what I'll tackle first.
Regarding suspicions that 'fn isn't fundamental, consider that it allows building a linked list on the stack:
struct Node<T> {
    t: T,
    next: Option<&'fn Node<T>>
}

fn link_list(len: usize) {
     let mut head : Node<usize> + 'fn = Node { t: 0, next: None };
     for i in 1..len {
         head = Node { t: i, next: &'fn head };
     }
     // wow. So much linked. Very list.
}

To clarify, I just wanted to keep the 'fn lifetime because it's semantically what LLVM already does with alloca.

This is not quite the case. The 'fn lifetime corresponds to the default behavior of LLVM, but it is possible to manage the stack of a function explicitly using the llvm.stacksave/llvm.stackrestore intrinsics.

Sorry I was being unclear here: I meant the default behavior of alloca(..) as implemented by LLVM, clang and gcc.

@llogiq ah, I was confused about which 'fn you are referring to, actually. If you mean giving a name to some lifetimes within the current fn, I feel mildly differently. I still don't think it's fundamental, though, and I'd prefer to do it in conjunction with having the ability to give a name to the lifetime of a block, e.g., 'a: { /* 'a is in scope here */ }, which has long been "a plan". I want that primarily because I think it would help with teaching, but it could sometimes be useful for documentation and when writing unsafe code too (so you can write types manually when inference is insufficient). It interacts with NLL in a sort of odd but harmless way, I think.

@nikomatsakis that's lifetime ascription as I understand it, and I agree that it's something we probably want. Even then, an implicit 'fn lifetime for the function body would improve ergonomics (as opposed to writing out the function block's lifetime):
fn explicit(..) -> Blue + 'a
'fn: { .. }

seems to me awfully hard to parse, especially for a human.

@llogiq yes, agreed.

Btw, I want break 'fn ... to be the same as return .... This naturally meshes very well with @nikomatsakis's desire to stage/replace 'fn with allowing 'a " { .. } for all blocks.

I'll actually have two separate RFCs in addition to this one. I just need more time to write them down. Next up will be 'fn, then (unless someone beats me to it) lifetime ascription.

@llogiq if you want, ping me with a draft and TODOs, and I'll PR your branch accordingly. I'd find that more enjoyable and dividing-and-conquering along RFC boundaries.

I'll be glad to.

So, I'm in favor of the idea of moving towards [T; n] sorts of expressions. However, I think the RFC as written leaves a lot unspecified. Basically, for a chance of this magnitude, it's way too short. =)
For example, consider this fragment:
fn foo(n: usize) {
    let x = [0; n];
    ...
}

Are we lifting the restriction that locals must have sized types? If so, what about locals like let x: Trait? I actually want this to work =) but I think it'll take some thought to work through the details, and clearly the RFC says nothing about it.
Another option might be to say that [0; n] is only legal as part of a borrow (i.e., let x = &mut [0; n]). This would preserve the invariant that all variables are sized, but I didn't see any syntactic restriction of that kind in the RFC. We should also then discuss what lifetime it gets (I ... presume it would follow the usual temporary rules, and hence get the enclosing block.)

Thoughts?

@nikomatsakis see https://github.com/rust-lang/rfcs/pull/1808#pullrequestreview-14984160

It's not completely clear to me from the RFC and that review link alone, so just to make sure: @glaebhoerl you're saying it's already the intent of the RFC that [T; (some runtime expr)] will be "only legal as part of a borrow" and therefore does not in any way "lift the restriction that locals must have sized types"?
If so I think the RFC should be much more explicit about that, since that completely resolves all the qualms I had about this proposal.

Are we lifting the restriction that locals must have sized types? If so, what about locals like let x: Trait? I actually want this to work =) but I think it'll take some thought to work through the details, and clearly the RFC says nothing about it.

I think the intent is to introduce the first case where a local has a dynamically-sized type, specifically for [T; n]. I think going beyond that, e.g. x: Trait is out of scope and much harder.

Another option might be to say that [0; n] is only legal as part of a borrow (i.e., let x = &mut [0; n]).

That might be a reasonable last resort, but it would be able to do things like this:
struct Elem {
    value: [0; n]
};

fn foo(elem_width: usize, num_elems: usize) {
    let mut lookup_table: [Elem; num_elems] = [Elem { value: [0; elem_width] }, num_elems];
    …
}

The above is how I'd like to do the RSA math code in ring, in particular.

you're saying it's already the intent of the RFC that [T; (some runtime expr)] will be "only legal as part of a borrow" and therefore does not in any way "lift the restriction that locals must have sized types"?

I... actually think these are orthogonal questions now that you mention it (I was reacting more to @nikomatsakis's code snippet than to his words):

Is an unsized expected-type context required to be able to use runtime n in [val; n]? My opinion is "definitely yes". (This is what the review comment was about.)

Is [val; n] where n is runtime required to be behind a borrow? I haven't formed an opinion and I'd have to re-read the RFC to find out if it says anything about it. (Given that the lang team tends to prefer to evolve the language in the smallest steps possible, though, I expect that the eventual answer will also be "yes".)

@briansmith For your example, is it important that lookup_table is of type [Elem; num_elems] rather than &[Elem; num_elems]? I agree that "allowing nesting" in alloca'd rvalues is desirable, but I don't think that requires or even benefits from an unsized local variable.

For your example, is it important that lookup_table is of type [Elem; num_elems] rather than &[Elem; num_elems]?

No. But, conversely, is there any concrete benefit by forcing people to write the &? It seems like requiring the & might allow punting on dynamically-sized local variables for a while, I guess, but I'm not sure how that concretely helps anything.

you can't put the runtime num_elems in a type without full-on runtime-dependent typing (e.g. Idris) and Rust is just not getting that for this feature alone, you really want a slice there for the time being
it used to be the case that doing this for borrows only was much much easier but now with MIR doing for borrows without having two kinds of borrows would be hard
then again, you have to somehow assign the length - one might consider the initial assignment to do the allocation, which is pretty weird but it might just work, or you could modify StorageLive

you can't put the runtime num_elems in a type without full-on runtime-dependent typing (e.g. Idris) and Rust is just not getting that for this feature alone, you really want a slice there for the time being

“Full-on” is greatly overstating things. Regardless, here's a modified version of my example that doesn't put the length in the type:
type Limb = usize;

struct Elem {
    value: [Limb] // dynamically-sized.
};

fn foo(elem_width: usize, num_elems: usize) {
    let mut lookup_table: [Elem] = [Elem { value: [0; elem_width] }, num_elems];
    …
}

In the modified example, the length isn't a property of the type. But, in order for bounds checking to work, the compiler needs to keep track of the length of the length of each variable-length array somehow.

Here's a counterexample of the claim that VLAs require heavy dependent type machinery, in case people still aren't convinced. This is from GCC's documentation regarding variable-length arrays in C:

The length of an array is computed once when the storage is allocated and is remembered for the scope of the array in case you access it with sizeof.

Transliterating into Rust terms, this means that GCC's C compiler can do the equivalent of this:
fn foo(n: usize) -> usize {
    let mut my_vla = [0; n];
    …
    my_vla.len(); // GCC knows how to compute `len()` like this.
}

And, very interestingly, GCC also supports types declarations where the length is part of the type—i.e. `[T; n]`—including in particular multi-dimensional arrays of variable dimension size, as GCC's example shows:

```c
struct entry
     tester (int len; char data[len][len], int len)
     {
       /* ... */
     }

C is much weaker as it effectively forgets the length of the array for the purposes of type equality, and its dependent typing there for making sizeof work has to do with lexical scoping, so you can model as if you bound a runtime expression to a name and sizeof goes and fetches that name.
When the scope the VLA was declared in is exited, the length expression doesn't need to exist anymore.
But C doesn't have slice types, anything like DST or fat pointers so we can do better than "typing" it.
C's lexical hack is pretty unacceptable in Rust unless it was based on some generative typing which hasn't been attempted yet and it's less likely to be randomly accepted than "just making [T; runtime] a slice".

C is much weaker as it effectively forgets the length of the array for the purposes of type equality,

For a first iteration of this, I think Rust could do the same.

and its dependent typing there for making sizeof work has to do with lexical scoping, so you can model as if you bound a runtime expression to a name and sizeof goes and fetches that name.
When the scope the VLA was declared in is exited, the length expression doesn't need to exist anymore.

I admit I don't know as much about Rust's internals as you, but it seems reasonable to do the same in Rust.

But C doesn't have slice types, anything like DST or fat pointers so we can do better than "typing" it.

Slices and fat pointers and whatnot seem straightforward to implement for dynamically-sized arrays.

C's lexical hack is pretty unacceptable in Rust unless it was based on some generative typing which hasn't been attempted yet

Why would it be unacceptable?

and it's less likely to be randomly accepted than "just making [T; runtime] a slice".

Nobody is asking for random behavior, for sure.

C is much weaker as it effectively forgets the length of the array for the purposes of type equality,
For a first iteration of this, I think Rust could do the same.

In fact, it is critical that for the purposes of type equality, all [T] are considered equal. Also, it is critical for this feature that the length not be considered part of the type, but rather a dynamic property of values of type [T], because don't want generated code to be specialized for the length.
At some point, it would also probably be useful to have [T; n] types, where n is constant, which would require type-level integers, but that's a totally different feature than what is implemented here.

@briansmith
allocas are orthogonal to doubly-unsized types (aka [[T]] aka [Elem]).

allocas are orthogonal to doubly-unsized types (aka [[T]] aka [Elem]).

It's not orthogonal because it's important to think ahead enough to make sure that the syntax and semantics of local variable-length arrays would be consistent with the most-likely syntax and semantics of VLAs inside of other types. But, other than that, I do think that it local VLAs could be implemented earlier than a more general VLA feature.

I'm not sure unsized type can work without being behind a &.
How would the following code behave for instance :
fn foo(n: usize, m: usize) {
  let mut x : [u8] = [0u8; n];
  if random() {
    x = [0u8; m];
  }
}

It would need reallocating the stack slot, which is impossible if any variable has been created since.

@plietar Two possible semantics: it could keep growing down or not allow reassignment (only initialization).

but it seems reasonable to do the same in Rust

Right now Rust has a type/trait-system based on pure logic (that blog post shows one path for formalizing and extending it) and introducing runtime values into types, without a formalization focused around dependent and/or generative typing, would require ad-hoc handling at every level of the compiler just to avoid those types leaking locations that can't reason about them.
C gets away with what it does because it barely has a typesystem, and what is has is lexically scoped, as if you recreated the types at runtime, every time you went through that code path, and they can't escape.
You can even interpret C by allocating type descriptors when reaching their definition and deallocating them when the lexical scope that defined them exits, as nothing can possibly continue to refer to them.
IOW C is weakly dependent in the same way Rust is dependent on "runtime" lifetimes, and to do the same in Rust we need a third system (to supplement the HM-like typesystem and the lifetime subtyping), and you really really don't want it to be ad-hoc because experience shows us we screw it up the first 2-3 times.
It's so much easier to deal with unsized stack slots, and it's not like [T; runtime] could be represented any differently from [T] anyway so the exact same machinery is required to supply the runtime value.

@plietar Two possible semantics: it could keep growing down or not allow reassignment (only initialization).

If we were to add heap-allocated VLAs and/or VLAs within other types in the future, "not allow reassignment" (with a different length) seems to be the the thing that would make sense. In particular, trying to allow assignment with a larger size for a heap-allocated VLA would require a (silent) reallocation, which (1) could fail, and (2) wouldn't be a zero-cost abstraction.

@briansmith Yeah, I think I am strongly shifting towards "initialization computes storage layout".
That means let x: T = y; can always work for unsized types T even if it's literally x = y in the MIR.

Right now Rust has a type/trait-system based on pure logic (that blog post shows one path for formalizing and extending it) and introducing runtime values into types, without a formalization focused around dependent and/or generative typing, would require ad-hoc handling at every level of the compiler just to avoid those types leaking locations that can't reason about them.

Again, this RFC isn't (shouldn't be) about adding runtime values into types, and in fact that's exactly what I don't want. There's only one type, [T]. Values of type [T] have a fn len() -> usize property, which is stored/computed at runtime in some opaque way. In other words, much like a Vec<T> except the storage for a VLA is alongside the length instead of off in the heap.

@briansmith Yes, that's what you get by having unsized locals. Lvalues in Rust can already be unsized, there's no difference between an lvalue and a pointer as far as the representation is concerned.
That is, ptr and *ptr have to be represented the same, since dereferences only give you a semantic "view" of the contents "at the address", there's never any operation going on there.
And if one kind of lvalue can have extra information because it came from a "fat pointer", all lvalues necessarily do if they're treated uniformly (and MIR requires that uniform treatment).

Essentially,
let x: [T] = [1, 2, 3, 4];
println!("{}", x.len());

using the existing fn len(&self) -> usize method on slice will auto-ref the x: [T] fat lvalue to the &x: &[T] fat-pointer and the compiler will know how to provide the length metadata.

During the compiler sprint, we came to believe that we can just allow unsized rvalues "everywhere" - except for function returns and assignment RHS.

I'd suggest that allocation of VLAs in loops, if possible at all, is at least linted against (warn- or even deny-by-default) as it seems to be a nonobvious stack overflow footgun in most cases.

I'll try to write up a more detailed report, but we had a long discussion in the compiler team, and we came roughly the following design. This is a very brief outline more designed to help me remember than to clearly explain to others who did not participate in the discussion, but I'll post it here anyhow:
https://public.etherpad-mozilla.org/p/rust-compiler-design-sprint-paris-2017-unsized

@tanriol It's not possible to have a non-static number of  allocas in a function without &'block rvalue doing dynamic allocas in 'block's scope, with a loop in between the current scope and 'block.
And to clean up that stack space or at least drop the values we'd need a linked list through them.

@tanriol

I'd suggest that allocation of VLAs in loops, if possible at all, is at least linted against (warn- or even deny-by-default) as it seems to be a nonobvious stack overflow footgun in most cases.

As discussed above, stack allocation being possible in Rust on heap-less systems is one of the end goals of this RFC. I'm sure clippy could run with it or something.

@rfcbot fcp close
I propose that we close this RFC because I would prefer to solve this problem as part of a more general push towards unsized values. The outline of such a proposal is here (as previously linked):
https://public.etherpad-mozilla.org/p/rust-compiler-design-sprint-paris-2017-unsized
I should note that the rest of the lang team is not necessarily on board with this idea, though I know that we have discussed it in the past. =) They can weigh in themselves as they comment on my proposal to merge if they choose (or on whatever RFC results from this).
The most important high-level bits of the proposal are that we will generalize to allowing unsized values to be used in more contexts. In particular:
- support `let n: usize = ...; let data: [u8] = [0; n];` (VLAs; roughly this RFC)
- support `fn foo(self)` in trait objects naturally
- support `fn foo(x: FnOnce())` as a nice way to pass a closure object

As long as we get VLAs, I'm fine with closing this.

It seems bad to close this without an RFC to replace it and without any clear description of what's wrong with moving ahead with it as an incremental step towards more general support for dynamically-sized types.
Also, is the RFC actually proposing [T] as unsized types? I think they're just dynamically sized. Maybe now in Rust there's no difference between unsized and dynamically-sized types, but I think maybe there should be: dynamically-sized types are types with values whose sizes aren't known statically at compile-time and which aren't equal (different values of a given dynamically-sized type may have different sizes), where unsized types would be types where there is no way—statically or at runtime—to learn the size, e.g. c_void and any other extern types.
In particular, I think that dynamically-sized types could and should be Clone and even Copy, but unsized types never could be.

@briansmith A long time ago Rust started using "unsized type" to mean "a type which has values obtained from values sized types, after applying unsizing" where "unsizing" is "to make the information required to compute alignment and size, dynamic".
I agree it's not the best naming scheme, but it's the one all uses of "unsized" (AFAIK) makes sense with (and it happens to also make "truly unsized" an oxymoron).
We could change to "dynamically sized" but we don't have a good verb to go with it. "reify"?

@briansmith: To expand on @eddyb's answer, the term unsized mainly arose because the people naming the feature were, well, operating from the POV of the compiler. And for the compiler, any information that's only available at runtime might as well not exist at all. IIRC, @nikomatsakis noted this in one of his posts.
Admittedly, this is suboptimal in some respects, much like how the GNU cross-compilation terminology is misleading when applied to anything but compilers.
It may, however, be early enough yet to address this. In particular, RFC 982 - which would enshrine these terms in the type system - is not yet stable. It specifies the traits Unsize<T: ?Sized>, whose implementors can unsize to a T, and CoerceUnsized<Target>, for smart pointers which coerce "around" a DST.
DST-ifying a type honestly has a very close relationship to type erasure, and there have been concerns that Unsize<T> "reads the wrong way around" in the past anyway. Thus, let me suggest:

EraseTo<T: ?Sized>
CoerceErased<Target>

?Sized is already out of the bag, but it may be possible to do better by the word "Unsized".

FWIW I prefer to generalize CoerceUnsized and thus have a single Coerce trait with a bunch (some builtin?) of impls. As for Unsize, we could invert the relationship and have U: FromSized<T>, perhaps.

Nominating for @rust-lang/lang discussion.

Note: there's now an RFC to subsume this one.

@aturon Worth mentioning that these RFCs are not equivalent. Unsized rvalues aren't equivalent to alloca and aren't useful for the embedded use cases.

@whitequark Can you elaborate on cases that this would solve but VLAs wouldn't?

@joshtriplett Allocating in a loop.

unnsized rvalues aren't equivalent to alloca and aren't useful for the embedded use cases

This is overstating things a bit. There's one particular thing that you want to do, which that proposal (nor the current form of this one, IIUC), doesn't support. It's not clear that that particular thing really needs to be supported, and that thing doesn't comprise all or even most “embedded” use cases.
However, I think there is a more serious problem with #1909, and maybe also this RFC: The proposed syntax for VLAs hint that they can be used intuitively like arrays [T; n] and vectors Vec<T>, but the restrictions on their usage are quite severe and counterintuitive. This RFC (#1808), reduced to supporting only VLAs as local variables (i.e. not embedded in structs like struct { x: [T] }) and giving the variables slice types, e.g. x: &[i32] = &[0i32; dynamic n]; makes more sense to me than #1909 does.

reduced to supporting only VLAs as local variables (i.e. not embedded in structs like struct { x: [T] })

This is already supported AFAIK. The last field of a struct can be a DST, and is itself thus a DST.

reduced to supporting only VLAs as local variables (i.e. not embedded in structs like struct Foo { x: [T] })
This is already supported AFAIK. The last field of a struct can be a DST, and is itself thus a DST.

I understand that. But, adding support for stack-allocating values of such structs is, IMO, counterproductive, because of the severe limitations mentioned in #1909: They can't be passed by value (copied/moved) nor returned from functions. Given those limitations, is stack allocation of these types widely useful? Those limitations are dealbreakers for my code. Maybe other people have other use cases for which they'd work, in which case it would be useful to see them.

@whitequark Hmmm.  You could certainly allocate a VLA in a loop.  But as @briansmith notes, you can't copy/move them, which means you can't accumulate them in a loop.
Fair point.  That seems worth considering in the discussions between this issue and #1909.

@whitequark
Accumulating alloca (aka 'fn locals) is quite separate in its problems, uses and implications from VLAs (aka this RFC and #1909). Please move that discussion to #1847.  If we adopt the "unsized ADT expressions" option, you could also put unsized values in structs. If we adopt the Copy: ?Sized option, you could also copy rvalues.

With #1909, you can definitely move rvalues (e.g., you should be able to call Box::new on a local). If we adopt the "unsized ADT expressions" option, you could also put unsized values in structs. If we adopt the Copy: ?Sized option, you could also copy rvalues.

That sounds better than I understood from the discussion. Could we return them from functions? In particular, given #[derive(Clone, Copy)] struct Elem { value: [usize] }, can we follow idiomatic patterns like this?:
`#[derive(Clone, Copy)]
struct Elem {
    value: [usize]
}

impl Elem {
    pub fn new(...) -> Self { ... }
}

pub fn elem_product(x: &Elem, y: &Elem, m: &Modulus) -> Elem { ... }

pub fn elem_pow(base: Elem, e: int, m: &Modulus) -> Elem {
    let limbs = ...;
    let mut table = [Elem { value: [0; dynamic limbs] }; 5];
    ...
}

If VLAs look like arrays then people will expect all of that kind of stuff to work, especially since it works fine for arrays. If that stuff doesn't work then a syntax and semantics that looks and feels more like "alloca(), but safe and Rustic" makes more sense, IMO—especially if the reason for deferring returning VLAs from functions or having arrays of VLA-containing structures are implementation concerns (difficulty modifying LLVM to support them).

There's also a possible way of returning them from functions, but the ABI implications are... quite interesting in a possibly non-portable way (either the called function must allocate stack on the caller's stack, or the caller needs to access memory after the called function returned). #[derive] is probably not going to work because of that, but if your types are Copy you can implement them yourself.

It's nice to see Rust take more care of stack allocation. A VLA is usable as brick to build more complex stack-allocated data structures.

It is not yet clear to me whether #1909 or this RFC is the way forward, as both seem incomplete to me.

Ain't clear if you can avoid some incompleteness here since it'd be nice to future-proof against some rather far out stuff.
As an extreme example, [x; n] could make sense with n being defined at runtime by using dependent types ala https://github.com/rust-lang/rfcs/issues/1930#issuecomment-283855743 In essence, our runtime value n could itself be viewed as type level information, but n viewed as an immutable value must outlive n viewed as a type, and n viewed as a type must outlive any type involving it, like [T; n].  If you have fn foo<n: usize>() -> [u8; n] then n gets promoted to a type in the caller which creates the space with alloca and passes the buffer to foo along with n as an implicit argument.
We clearly want stuff like raw-ish alloca well before anything like this.  And I think the syntax let x : &[T] = &[T::default(); n]; is actually forwards compatible with "promote n to a type, make a [T; n], and cast its reference to a slice".  I thought this made a nice example though.

@budges yeah that's roughly what we eventually want in principle, but there's so many subtleties in the design I'd be extremely wary about declaring something forward comparable now.

Per the FCP, I am going to close this RFC, in favor of unsized rvalues. Thanks @llogiq for the work you put into it. =)

@burdges: If a dependency then impls Trait on an unsized type, then that's a semver break. This would basically make unsized types implicitly #[fundamental], as adding an impl may now force semver breaks on dependents.
"If none do X, then Y" - negative reasoning - is a thorny thicket to dive into.

Oops, I misspoke, but you read into my mistake more sense than existed.  ;) There is no way the compiler could statically determine all types T satisfying FnOnce()!
It requires way more hackery to make this make sense:
fn foo(a: bool) -> impl FnOnce() {
    let f = || { ... };
    let g = || { ... };
    trait MyFn : FnOnce() {}
    impl MyFn for type_of<f> {}
    impl MyFn for type_of<g> {}
    if a {
        f as dyn MyFn
    } else {
        g as dyn MyFn
    }
}

In this code, we know all T satiating MyFn because no other crate can impl MyFn, so the compiler really could declare dyn MyFn : Sized, but..
I can now answer my own question:  We lack a convenient enough way to impl traits for anonymous types to make this an ergonomic solution for any problems under consideration.  If the calling convention ever allows returning VLAs then a much better feature falls out naturally without the hackery.  If that calling convention incurs any runtime costs, then the compiler could compile appropriate cases to sized returns under the hood.

@burdges: It's more that I then applied the same reasoning to dyn Trait in argument position, which I failed to adequately state. That issue applies even to your expansion. Sizedness is a matter of API, and implementing a Foo on struct Bar { a: [u8] } would thus be a "take-back" of all impls of Foo being sized. If Foo and Bar are in different crates, this can get very messy.

A trait exported from any crate could never benefit.  I read you point two ways:  First, adding impls can now break things within the same crate.  Second, even if Trait is not exported, we might still impl Trait for Foreign where Foreign comes from another crate, and hence Foreign might stop being sized in future.  Yes, you'd need an explicit bound, ala
trait MyFn : FnOnce()+Sized {}


Fix cts DecodeGlAccuracyTest fail

Previously the dataspace data for YUV is hardcoded. Since gralloc4 can retrieve the dataspace data now, we just set a default value and try to get real value from gralloc4.
Tracked-On: OAM-119899

Dependent pr program name is not compatable this pr's program name, for more details please check tracked_on

Android CI has started Engineering Build for this issue ,Please check the linked Tracked-On issue/Android CI Web  for more details.

Android CI has started Engineering Build for this issue ,Please check the linked Tracked-On issue/Android CI Web  for more details.

SUCCESS: Android CI has completed Engineering Build for this issue.Please check the linked Tracked-On issue/Android CI Web  for more details.

SUCCESS: Android CI has completed Engineering Build for this issue.Please check the linked Tracked-On issue/Android CI Web  for more details.

Android CI has started MERGE Build for this pr ,Please check the linked Tracked-On issue/Android CI Web  for more details.

Android CI has completed MERGE Build for this pr, build is SUCCESS. Please check the linked Tracked-On issue/Android CI Web  for more details. For Binaries: /cactus-absp-or-local/celadon_smr0_master-merge/1012


Azure Devops Fails with CLI to remove firewall rules when resource group is locked for deletion

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
We are using Sql Azure in a resource group , the resource group also has a delete lock to avoid accidental deletion of resources, which is according to us is a standard practice. The sql azure database is updated using code first pattern using Deploy Azure Database task in Azure DevOps . The details in this documentation.
https://docs.microsoft.com/en-us/azure/devops/pipelines/targets/azure-sqldb?view=azure-devops&tabs=yaml
We also found, this task internally adds and removes a firewall rules to sql azure server to allow agent to access the sql serve.
However If resource group is locked for deletion, this scenario fails as pipeline is unable to delete a firewall rule.
This seems a catch 22 situation as we want resource group locked for deletion but also want pipeline to be able to delete firewall rules to access server to deploy database scripts.
To Reproduce

Create SQL Azure Server and Database

Setup with code first deployment

Lock resource group for deletion

Try to deploy database using azure devops pipeline.

pipeline fails

Expected behavior

The resource group lock should have an exception for a service principle which is able to bypass this resource lock.

route to CXP team

Adding the Service Attention label so that the Github Devops Service team can look into this issue.

I am also having this issue.


Updates in resources in Groovy Maven project are not transferred to Agent

Hello,
I have a Groovy Maven project which has a resources folder. Inside the resources folder there is a properties file. I update the properties file and commit it on SVN. Although, the agent still receives the old version of the properties file. Would you please tell how to make my agent to always get the fresh files from the repository? I have tried the system.conf properties without luck. At the current moment I'm investigating whether the issue is related to EHCache.
Thanks,
Tolis

Hello,
It has nothing to do with EHCache.
The agent is always supposed to receive new resources every time the test is run.
Try saving the test file again,
It may be necessary to check the resources downloaded by the agent.
See the figure below for confirmation.

If you are using the latest version of ngrinder, I think agent restart seems to be good.

@songeunwoo Thank you for your answer.
The test file (MyTest.groovy) is always updated on the Agent. The myproject/resources/conf/my.properties of my Groovy Maven project is not updated.
I have tried everything. Installed NGrinder 3.3, installed NGrinder 3.4.1, deleted the SVN repository and created a new one with different name. Still I get a cached old version of my resources folder.

I found the root cause. NGrinder works correctly.
Thank you @songeunwoo and team for the great product :)


[CH-293] Upgrade clickhouse to <IP_ADDRESS>-stable

Changelog category (leave one):

New Feature

Information about CI checks: https://clickhouse.tech/docs/en/development/continuous-integration/

Can one of the admins verify this patch?

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957

test this please with 957


join: [...] syntax

Currently joins work when I write join: on each line.
Should join: [...] work?

yes
join: [ oneThing on oneKey, twoThing on twoKey ]

should totally work

I have no tests, so I have no idea if it does work.

I tested it and it's not working for me currently, here's what I tried. Using the ecommerce model:
 join: [
    users on user_id,
    ii on inventory_item_id,
    user_order_facts on user_id
  ]

I get an extraneous input error on the bracket. It does actually seem to be accepting this, and queries requiring the join are running properly:
join: {
    users on user_id
    ii on inventory_item_id
    user_order_facts on user_id
}

doh ... yeah it should be bracket. fixing that now.
Fixed in 67712100e9ac15d90638909254809d2fa3d63b0a


Don't Merge! PR real time test for source branch full build

Docs Build status updates of commit c1146f7:
:clock10: Full build: the average full build time is 5 min(s) 56 sec(s), this is based on the last 724 build(s) for this repository.

Docs Build status updates of commit c1146f7:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Cursor graphic glitches out

Normally, the cursor displays fine. After pressing ctrl-alt-shift-<= (or =>) to switch back to crosh and then back to Ubuntu, the cursor graphic glitches out. It becomes a distorted image repeated 4 times horizontally. It still functions perfectly. Typing anything into an Ubuntu textbox, like terminal, fixes the issue.
Using acer-c740
crouton: version 1-20151104200003~master:2a1fc9da
release: trusty
architecture: amd64
xmethod: xorg
targets: unity
host: version 7520.62.0 (Official Build) stable-channel auron_paine
kernel: Linux localhost 3.14.0 #1 SMP Tue Dec 8 02:00:18 PST 2015 x86_64 x86_64 x86_64 GNU/Linux
freon: yes

The easiest way is to press "esc" or any other key in chromeOS before you switch to chroot.
For some cpu series it's possible to install display drivers to fix, but that's dangerous.

I wonder if it's a problem crouton can solve or just inevitable

This has been fixed (a while ago)


Vagrantfile: Add vagrant inventory file in any directory to .gitignore

Follow-on fix for #2654
Signed-off-by: Ganesh Maharaj Mahalingam<EMAIL_ADDRESS>

Apologies for missing this the first time around.

@Atoms Might i bother you to take a quick look at this. I missed it the first round. Apologies about that.


Sudo command is missing in the latest version of docker container for sfpowerscripts

Describe the bug
Running scripts using the sudo command shows (sudo: command not found).
To Reproduce
Run any script prefaced by sudo will cause it to fail with error sudo: command not found.
Expected behavior
The container would run the script prefaced by sudo.
Screenshots

Platform Details (please complete the following information):

OS: Ubuntu 20.04
Docker Version: [release-apr22,latest]
CI Platform: Azure Pipelines

Additional context
Workaround is to use previous version of the docker container as the issue is not present there.

Fixed in Apr 22 - 2