Unified coroutines a.k.a. Generator resume arguments

//Edit after a long time: For everyone, I suggest you read https://github.com/rust-lang/lang-team/issues/49, it summarizes state of things, and explains ideas explored in this RFC and subsequent discussion.
This RFC outlines a way to unify existing implementation of the Generator feature with the Fn* family of traits and closures in general. Integrating these 2 concepts allows us to simplify the language, making the generators 'just pinned closures', and allows implementation of new patterns due to addtional functionality of generators accepting resume arguments.
Generator resume arguments are a sought after feature, since in their absence the implementation of async-await was forced to utilize thread-local storage, making it no_std.
Rendered
The RFC builds upon original coroutines eRFC
Main contention points:

Syntax of arguments changing between yields (explicit vs implicit) and the interaction with lifetimes.
Use of tuples and connection to closures, and the interaction with yield being an expression which resolves to arguments passed to resume.

Examples of new patterns designed with proposed design of the generator trait & Feature:

Re-implementation of Sink in terms of generators by @Nemo157 :
https://play.rust-lang.org/?version=beta&mode=debug&edition=2018&gist=c1750a50fbeff78200537d6f133ecb6e

Another view of generators is that they are a more general form of Iterator. With this in mind, under future possibilities could we add combinators that can pass the output of one generator/iterator to he input of another generator? Basically baking the input arguments.

@KrishnaSannasi Yes, I think. But until we have GAT, and streaming iterators, the arguments would have to be 'static
Take an some combinator, which wraps one generator output with an Some and yields it back.
The ultimate implementation would something similar to this:
fn some<Arg, G: FnGen<Args>>(g: G)
        -> impl FnGen<Arg, Yield=Option<G::Yield>, Return=Option<G::Return>>
{
    |arg| {
        match gen(arg) {
            GeneratorState::Yielded(v) => {
                yield Some(v);
            }
            GeneratorState::Retur(v) => {
                return Some(v);
            }
        }
    }
}

@semtexzv
I was thinking something along the lines of, this

Similar to core::future::Future experimenting with combinators outside the standard library as part of futures I believe it would make sense to leave Generator combinators up to an external library to start (and by the time Generator might be approaching stabilisation we'll hopefully have some examples of moving combinators into the standard library with Future).

I feel that the RFC as currently written spends a lot of “weirdness budget” in a way that is not necessary. In “normal” closures, the bindings introduced for arguments (between the two | pipes) exist from the very start of the closure’s body, which matches their location in source code. This RFC proposes the same location for introducing names that are not available until the first yield, and then can change value at each yield. (This leads to questions like what if the previous value was borrowed?)
How about this alternative?

Like in the current RFC, the Generator trait gains a new type parameter. Except I’ll call it ResumeArg (singular)

ResumeArg can be any type, not only a tuple. (Although it might default to (), and using a tuple would be an idiomatic way to pass multiple values.)

yield is an expression of type ResumeArg. That’s it. It is not concerned at all with name bindings. You can use that expression in a let $pattern = $expr;, but that’s not necessary. It could be passed as a function argument, or anything that accepts an expression.

There is no way to specify the ResumeArg type as part of generator literal syntax. (Nothing is allowed between the pipes when yield is used.) This RFC and its proposed alternatives seem to try hard to find such a way, but that may not be necessary: it can be a type variable left for inference to resolve (based both on what the generator’s body does with its yield expression, and how the generator is used).
There is nearby precedent for this: generator literals don’t have syntax to specify the Generator::Yield associated type either. And the -> $type syntax that specifies Generator::Return is optional.

@SimonSapin You raise an important point. But, like closures, the arguments are available from the start of the generator, since generator is created suspended at the start, like closures.
Instead of a generator as it stands today, think about a pinned FnMut closure.

Hmm I may have been mislead by this part of the RFC:

Notice that the argument to first resume call was unused,

I took this to mean that the proposed language semantics are that the argument to the first resume call is always dropped, because that call doesn’t have a corresponding yield expression. But maybe instead you only meant that this particular example generator does nothing with the argument of the first resume call?
If we consider that this first value passed to resume doesn’t need to be dropped, then indeed the “closure argument” syntax would be the appropriate way to make them available to the generator’s body.
So I think preference is closest to what is currently listed as alternative # 2. I’ll quote it below and comment inline:

Creating a new binding upon each yield

I’d still phrase this as: yield is an expression, whose value may or may not be bound to a name with let.

let gen = |name :&'static str| {

    let (name, ) = yield "hello";
    let (name, ) = yield name;
}

We are creating a new binding upon each yield point, and therefore are shadowing earlier bindings.

Yes about shadowing. But again, creating a binding doesn’t have to be mandatory.

This would mean that by default, the generator stores all of the arguments passed into it through the resumes.

Not necessarily, even with a binding. The value won’t be stored in the generator if it has been moved or dropped before the next yield. Like any other value manipulated on the stack of a generator.

Another issue with these approaches is, that they require progmmer, to write additional code to get the default behavior. In other words: What happens when user does not perform the required assignment ? Simply said, this code is permitted, but nonsensical:
let gen = |name: &static str| {
    yield "hello";
    let (name, ) = yield name;
}

This is not nonsensical at all. Like any $expr; statement, the first yield simply drops its resume value.

Another issue is, how does this work with loops ? What is the value assigned to third in following example ?
let gen = |a| {
    loop {
        println!("a : {:?}", a);
        let (a,) = yield 1;
        println!("b : {:?}", a);
        let (a,) = yield 2;
    }
}
let first = gen.resume(("0"));
let sec = gen.resume(("1"));
let third = gen.resume(("2"));

third is the integer 1. But I think the intended question was: what line is printed third? The answer to that is a : "0". The let bindings shadowed a for the rest of the lexical scope (which is the rest of the loop). When the loop does its next iteration, control goes back out of the scope of shadowing a’s and the initial a is visible again. The same would happen with a similar loop outside of a generator.

In short, I feel strongly that yield should be a stand-alone expression, not necessarily tied to name bindings.

Separately from the above, I feel there is a choice between:

(As in the current RFC), the value passed to resume has to be a tuple. For the first resume call, this maps very nicely to closure argument syntax with any number of argument. However for subsequent calls, in the case of a 1-tuple we’d need somewhat-awkward unpacking like let (name,) = yield; foo(name).

Or, the value passed to resume can be any type, and yield expressions have that type. let name = yield; foo(name) or even foo(yield) looks much nicer without tuple unpacking. However this forces the “closure argument” syntax to only have one argument. || {…} (zero argument)  could be syntactic sugar for |_: ()| {…} (one argument of type unit-tuple). (Only for generators of course, not normal closures.) It’s for the initial bindings of multiple values as one tuple argument that this becomes awkward: |(foo, bar): (u32, &str)| {…} instead of |foo: u32, bar: &str| {…}

Trying to reconcile both nice-to-haves creates inconsistencies or discontinuities that are undesirable IMO. For example, does |foo: u32| { yield } implement Generator<u32> or Generator<(u32,)>? Why?

Well,  the argument HAS to be a tuple, and it HAS to be unpacked inside the argument list of the generator, since it is the behavior of closures, and deviating from this behavior would most certainly be a mistake.  I againt, point out the Fn* family of traits.
As for the third point, the argument would most certainly have to be a tuple. (Interaction of Fn traits and closures).
I accept syntactic inconvenience, but I think, the deviation from concepts introduced in closures is a huge mistage.

Not necessarily, even with a binding. The value won’t be stored in the generator if it has been moved or dropped before the next yield. Like any other value manipulated on the stack of a generator.

This would imply you need to scope every yield call (even moving the value out is not enough, see https://github.com/rust-lang/rust/issues/57478), generators are like normal code and drop their values at the end of scope. (Although there are optimizations applied to drop values early if that is not observable).

@semtexzv

HAS to be

Yes, the goal of making generators and closures as close to each other as possible leads to arguments being a tuple. But I’m personally not very attached to that goal in the first place.
Closures have a whole family of traits with Fn, FnMut, and FnOnce. They are useful for reasons that doesn’t apply to generators. Fn taking &self doesn’t work for generators since resume always mutates (at least to track initial state v.s. each yield point v.s. returned). FnOnce defeats the point of having a generator in the first place.
@Nemo157 https://github.com/rust-lang/rust/issues/57478 is an implementation bug to be fixed, right?

Even if https://github.com/rust-lang/rust/issues/57478 is fixed I would expect there to be a lot of generators that just shadow their existing bindings without moving out of them, having those all be kept alive to be dropped at the end would not be good, e.g.
|arg: String| {
    let arg = yield;
    let arg = yield;
    let arg = yield;
}

would have to keep all 4 strings live until dropped when the generator completes.

@SimonSapin Yes, this zealotry for theoretical purity has drawbacks. But I believe this is the right choice in this case. The connection to FnOnce provides a way we could extend current compiler mechanics which apply to closures. And the RFC mentions possible FnPin trait which would extend the function trait hierarchy to connect it fully to generator trait.
This has to do with the future plans part of the RFC, the ultimate goal being the generator callable as a closure is.  To ensure this is possible in the future, we have to design the generator trait in this way, and it introduces friction between :

yield being and expression and returning a tuple of the same type as the argument list written out between the closure syntax ( the 'pipes' )
yield being a statement, and the arguments changing values between yield points.

Simply, the theoretically correct, while not pragmatically correct choice now, will have implications down the road, that will allow us to make more pragmatic choices.

@Nemo157 Yes, and? That preserving destruction order can cause unexpected generator size increase is not at all unique to values that yield evaluates to. For example:
|| {
    let foo = String::from(…);
    yield;
    let foo = String::from(…);
    yield;
    let foo = String::from(…);
    yield;
}

So giving yield weird semantics is not the way to fix this IMO. For this particular issue I feel that there is no substitute for education and reminding people that shadowing does not drop. Your example is easy enough to fix:
|mut arg: String| {
    arg = yield;
    arg = yield;
    arg = yield;
}

@semtexzv

The connection to FnOnce provides a way we could extend current compiler mechanics which apply to closures.

Sorry, I don’t understand what that means.

I meant the interaction between Fn* traits, and how are closures invoked. There have to be some mechanics for packing/unpacking the arguments passed into the closure

@SimonSapin I think it'd be much more preferable if one wasn't forced to make things mutable (or to wrap the yield in braces), just to prevent size increase of the generator.

I came into this expecting I wasn't going to like the argument rebinding. But I think that the parallel with regular FnMut is the correct direction to take, at least if generator syntax continues to look like closures.
Sure, in the case of one argument, you can use mut and rebind, but what about with two or more?
|mut one: String, mut two: String| {
    (one, two) = yield; // ???
}

I think a view of generator syntax as "just" sugar over "FnPin" is the correct place to aim for. Consider the following:
|one: String, two: String| {
    let both = one.clone() + &two;
    loop {
        dbg!((&both, &one, &two));
        yield;
    }
}

"Desugared" to a "FnPin"-like syntax:
pin |one: String, two: String| -> GeneratorState<Return=(), Yield=()> {
    match self.state {
        0 => {
            self.both = one.clone() + &two;
            dbg!(&self.both, &one, &two);
            self.state = 1;
            Yield(())
        }
        1 => {
            dbg!(&self.both, &one, &two);
            self.state = 1;
            Yield(())
        }
    }
}

In a yield-value syntax:
|mut one: String, mut two: String| {
    let both = one.clone() + &two;
    loop {
        dbg!(&both, &one, &two);
        let resume = yield;
        one = resume.0;
        two = resume.1;
    }
}

Rebinding the arguments' names on yield may seem foreign at first, but it makes sense if you think of a generator as a resumable coroutine; the arguments are whatever they were to that specific resumption call. (Though to be fair, it only really makes sense if we eventually get resume spelled as gen(one, two), not gen.resume((one, two)). Mimicking extern "rust-call"'s API here makes sense to me.)
I think it's easier to teach that if you want a previous resumption's arguments you need to bind them locally than that you need to explicitly drop this resumption's arguments if you don't want them increasing your generator size. (Explicitly store them, implicitly drop, rather than implicitly store them, explicitly drop.)
Re-issuing the arguments solves the problems of how to handle multiple arguments with drop glue nicely. It also gives us a sensible way to handle passing a borrow to resume, I believe:
Add a new specially treated lifetime, 'yield. A lifetime passed to a generator literal has '_ = 'yield; 'yield doesn't need to exist in the surface language, just in MIR. To the body of the generator, &'yield is always valid, but copying it out of the argument bindings makes the other binding not live over yield points. Lifetime checking is done by the MIR borrowck phase on the CFG of the generator, which gets a new &'yield binding by the same name at every yield point.
I can write up a more formal RFC-style of this interpretation if people think it might be useful.

Falling from this is an obvious question: what about generator free fn, not closures? And I think the answer should be that we don't have them.
In free function position, how would you interact with a generator?
fn foo(one: String, two: String) -> ! yield () {
    let both = one.clone() + &two;
    loop {
        dbg!(&both, &one, &two);
        yield;
    }
}

fn main() {
    foo("1".to_string(), "2".to_string()); // ???
    foo("3".to_string(), "4".to_string()); // ??????
}

The pinning and state requirement makes me think that always using closure syntax makes more sense:
fn foo() -> impl Generator<Return=!, Yield=()> {
    |one: String, two: String| {
        // blah blah blah yield
    }
}

(Honestly, an extension of this train of logic gets me thinking that maybe not having async fn wouldn't be so bad, and just have async closures. But that doesn't matter anymore.)
While the transform for async fn makes sense (async fn(Args) -> Ret => fn(Args) -> impl Future<Item=Ret> + '_), the transform for "yield fn" would be a lot more questionable (yield fn(Args) -> Ret yield Yield => fn() -> impl Generator<Return=Ret, Yield=Yield>? const impl Generator<Return=Ret, Yield=Yield>?).
Because of this, we don't have to worry about what rebinding acts like in free fn in a rebinding world. It's just the closure syntax that exists, and my argument is that creating new 'yield bindings accessed with the same name declared in the arguments list at each yield point, while foreign, is easier to teach on its own merit than yield returning a value on resume and generator size ballooning unless you know this One Weird Trick To Optimize Your Generators.

@CAD97 Thank you, I'd be thrilled if you explained how could the type checking work, since i'm not that qualified in this area. I'm going to reformat the RFC in a way that presents the proposed approach in full before examining other approaches, and more information about how would the type checking work would be invaluable.

One thing to keep in mind is that the generator argument used by async will be a reference- this means it has none of problems of drop order or generator size. (Though of course its use by the async desugaring wouldn't be problematic either way because we could simply define away the problem in the desguaring.)
Someone directly writing an equivalent generator won't even be able to accidentally keep one around, because its lifetime is restricted to a single resume.

On another note, I think it is much more important for generators to unify with Rust's various "effects," than it is to unify with closures.
First, like SimonSapin mentions, we gain very little by sharing the Fn traits- only FnMut really matches, and writing that mapping explicitly doesn't cost much.
Second, like the earlier question of whether to unify generators with async, unification prevents us from treating the two as orthogonal, combinable language features. Specifically, what if we want generator functions (and closures), similar to async functions (and closures)?
That is, we should track generator-ness like we track async-ness, and forbid programs from accidentally using generators as non-generators. This is a more useful consistency that facilitates composition with other "effects," and potentially leaves the door open for effect polymorphism.

FWIW I'm also in favour of yield simply being an expression, and the points regarding shadowing being easy to misuse don't bother me for reasons already discussed; the rest of the language consistently works in the same way, and moved/non-Drop values should presumably be optimized out of the generated state machine storage even if they aren't today. I do feel like the larger concern here is how to handle lifetimes that do not cross yield points, and a way to express that lifetime may guide which syntax is most appropriate? It does match what you'd expect from a for<'a> FnPin(&'a Context), just feels awkward to explicitly convey this somehow at the generator definition... for<'a> |&'a Context| { ... } I guess?
Has there also been discussion yet regarding the way the design necessetates that the final resume passes an argument which cannot be received by the generator? That kind of indicates to me that distinction between Yield and Return being separate types maybe isn't the right design here? Before resume args were a thing a no-op resume to check if the generator is done wasn't that weird, but now you have to explicitly pass it useless data to check?

Also as a side note, what if generators are just closures that produce a generator, and thus can have execution before the first resume? The argument list would just be a way to move initial state into the generator I guess, and I'm not sure if this is a more or less weird of an approach to avoid unifying argument list and resume arg type:
fn x(start: usize) -> impl Generator<(&'static str), Yield=bool, Return=u32> {
    // Possibly `FnOnce` instead if it captures something non-Copy?
    let x: impl Fn(usize) -> impl Generator = |start| {
        println!("running before first yield with {}", start);
        println!("first resume passed me {}", yield start == 5);
        5u32
    };
    x(start)
}

// or in a sugared item form...
yield fn x(i: usize) yield(&'static str) bool -> u32 {
    println!("running before first yield with {}", start);
    println!("first resume passed me {}", yield start == 5);
    5u32
}

let y = x(5);
// first println happens above!
let GeneratorState::Yield(true) = y.resume("hello");
let GeneratorState::Complete(5) = y.resume("hi"); // isn't it a bit strange how this resume arg gets ignored/dropped?
// etc...

I agree on the comments that the changing bindings being too much magic.
Isn't yield returning resume args already used in other languages and therefore a somewhat reasonable solution?
Another idea:
In order to mark the temporary nature of resume args, why not explicitly pass them through accessor functions, which would return most recent passed resume arg to the generator:
let gen = |name: &dyn Fn() -> 'static str| {
    yield "Hello";
    yield name();
    return name();
}

That would be in-line with any other Rust code.
Non Fn parameters could be either rejected, or those would be the parameters that are only passed when the generator is started (are moved once inside the generator), and not passed on every resume.

Isn't yield returning resume args already used in other languages and therefore a somewhat reasonable solution?

In python, but given that python is dynamically typed it may work better for them than it would in Rust.

I was also thinking about the same issue that already came up in the discussion between @Nemo157 and @SimonSapin: The scoping/lifetimes with yield as expression is quite "uncommon".
Going back to the example and modifying it slightly the following questions arise:
let gen = |name :&'static str| {
    let (name1, ) = yield "hello";
    // Is name here still valid?
    let (name2, ) = yield name1;
    // Is name2 here still valid?
}

I think the answer is no. Otherwise the requirements regarding lifetimes for callers of resume() would be extremely hard to describe. Everything which is passed to resume() at one point of time would need to be marked as borrowed for the lifetime of the generator. Which is most likely not what we want. Rather we want things to be only borrowed for the duration of the resume call.
In order to make that work, one would need to explain to the borrow-checker that lifetimes of bindings which are produced by a yield end at the next yield.
Or it could be made a bit more obvious by enforcing scoping in the generator. E.g. by making yield taking a closure which gets passed the resume arg:
let gen = |name :&'static str| {
    yield("hello", |name1: &'static str| {
        yield(name1, |name2: &'static str| {})
    });
}

Now this isn't super nice either - and most of all it also doesn't solve the issue that name1 could be utilized after the next yield returned. Maybe it can work out if an extra rule is introduced that yield is only allowed as the last statement inside a yield closure, so that the parameter can't be utilized anymore after it.
Whether that's a great solution -> Don't know.

Going back to the example and modifying it slightly the following questions arise:
let gen = |name :&'static str| {
    let (name1, ) = yield "hello";
    // Is name here still valid?
    let (name2, ) = yield name1;
    // Is name2 here still valid?
}

I think the answer is no. Otherwise the requirements regarding lifetimes for callers of resume() would be extremely hard to describe. Everything which is passed to resume() at one point of time would need to be marked as borrowed for the lifetime of the generator. Which is most likely not what we want. Rather we want things to be only borrowed for the duration of the resume call.

Yes, in this example all variables should be available until the end of the scope. Why wouldn’t they be? And yes, this means that the generator type needs to have a lifetime parameter and borrow the resume arguments. (If they have lifetimes other than 'static, unlike this example.) A way to enforce that would be to make ResumeArg an associated type rather than a type parameter of the trait:
impl Generator for MyGenerator {
    type ResumeArg = &'a str;
    // …
}

The above cannot use 'a because it is not in scope, so it needs to be added to the impl block:
impl<'a> Generator for MyGenerator {
    type ResumeArg = &'a str;
    // …
}

This is also now allowed because the 'a has an unused parameter. If the Generator trait doesn’t accept parameters, the only way to make that parameter be used is therefore to parameterize the type:
enum MyGenerator<'a> {
    Initial,
    Yield1 { name: &'a str },
    Yield2 { name: &'a str, name1: &'a str },
    Returned,
}

impl<'a> Generator for MyGenerator<'a> {
    type ResumeArg = &'a str;
    // …
}

For multiple different yield poins you'd have to add multiple lifetime arguments, so I don't think this is the right choice. Previous attempts went this way, and met the same issues, ultimately resolving to using the same hacks to solve the lack of GATs.

Yes, in this example all variables should be available until the end of the scope. Why wouldn’t they be?

I think this doesn't work great for a variety of reasons:
1. I don't see how this would look the caller, and how it would interact with the borrow-checker.
All arguments that are passed to resume() need to be permanently consumed - which is not something that is a known concept. And things like the following code wouldn't be valid:
let mut data: String = "hello".to_owned();
gen.resume(&data); // Now gen internally captures a reference to the original string
data += " world"; // This invalidates the reference
gen.resume(&data); // Now gen would have an invalidated and a new reference

2. It's unclear how this would interact with control flow in generators.
E.g. something along
let gen = |name :&'static str| {
    let (name, done) = yield "hello";
    while !done  {
        let (name, done) = yield name;
    }
}

If this follows the rule to capture everything, then the generators size would be infinite.
If the binding is the same for all iterations of the loop so that only 2 variants are captured, then the behavior on the resumer side is unclear. For some resumes the argument is only borrowed for the duration of the resume, for others for a few resume calls until it is released again, and yet for others it is the whole lifetime of the generator.
This also makes any change inside the generator non API compatible anymore, since introducing any control flow would change the lifetimes.
3. It increases generator size, even if applications don't require parameters across yield points
Lots of applications for resume args, e.g. passing Context for Futures, don't require any parameters across yield points. Passing them again is totally fine - or even expected, since things might have changed. Storing more - for the sake that some applications might require it - seems to be the wrong default. If some generators need data again they could just pass it again into the next resume call.
We could argue that if things are not used across yield points they are optimized away by the compiler. However in practice that seems to be tricky. For async fn there were already lots of questions when parameters are allowed to be dropped, in order to be semantically compatible with non async code. Here the latest changes were to keep the parameters as long as possible, which increases the Future sizes.

The "yield evaluates to the resume argument" approach doesn't have any of those problems. And it doesn't have them precisely because it matches the behavior of normal Rust code more closely!

And things like the following code wouldn't be valid:
let mut data: String = "hello".to_owned();
gen.resume(&data); // Now gen internally captures a reference to the original string
data += " world"; // This invalidates the reference
gen.resume(&data); // Now gen would have an invalidated and a new reference

You can already write this in stable, non-generator Rust, and it handle both the cases of the generator keeping the argument and the generator dropping the argument. This simply needs to be encoded into the type of resume:

If the type of resume is fn resume<'a>(&mut Self<'a>, &'a str) then the the borrow checker forbids the program above: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5287660952409cf4b44386edf0a55ffa
If the type of resume is fn resume<'a>(&mut Self<'a>, &str) then the borrow checker allows the program above: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0f6660c3450cfcb0b3b9c3f48a646b08

let gen = |name :&'static str| {
    let (name, done) = yield "hello";
    while !done  {
        let (name, done) = yield name;
    }
}

If this follows the rule to capture everything, then the generators size would be infinite.

No, the bindings created in the loop body are dropped at the end of the iteration, and thus don't live across any yield point. (This also doesn't do what you want, for the same reason.)

If the binding is the same for all iterations of the loop so that only 2 variants are captured, then the behavior on the resumer side is unclear. ...
This also makes any change inside the generator non API compatible anymore, since introducing any control flow would change the lifetimes.

This is already the case for async/await, and is why Pin exists. This is just a fundamental part of how structs-with-references work in Rust today, you can't get around that. Even with the "magic mutating arguments" approach, the generator author has the same decision to make, because they can move the value into a local that does live across yield points.
Fortunately, as above, the resumer-side behavior can be determined by looking at the type of resume in the particular generator impl.

It increases generator size, even if applications don't require parameters across yield points

It does not, please stop making this claim. The counterexamples you bring up all have to do with Drop, which simply does not apply to reference arguments. And when a generator argument does implement Drop, matching sync code is good and a large part of why this design makes sense. If size is a problem here the author can simply drop the argument manually, just like they do in sync code.

Thanks for the clarifications. I wasn't aware of this kind of borrowing, and that it's intended to configure the borrowing mode via explicit type annotations.
I think more complete examples how that would look like could be helpful. I still can't fully follow where those things should go to (e.g. the example of @SimonSapin had a MyGenerator struct, but I imagined the actual struct to be compiler-generated).

If the type of resume is fn resume<'a>(&mut Self<'a>, &'a str) then the the borrow checker forbids the program above: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5287660952409cf4b44386edf0a55ffa
If the type of resume is fn resume<'a>(&mut Self<'a>, &str) then the borrow checker allows the program above: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=0f6660c3450cfcb0b3b9c3f48a646b08

More accurately these signatures wouldn't be on the resume function as that's specified by the trait, but instead on the impl; impl<'a> Generator<(&'a str,)> for Self<'a> and impl<'a, 'b> Generator<(&'b str,)> for Self<'a>.
Problematically the former is saying that the argument must live for the same lifetime as the generator, there is no way I know of to say "the argument must be valid from now until the generator is dropped". For example this fails even though it could be valid, there's just (AFAIK) no way to write a set of signatures that satisfies the borrow checker.
This is all very straightforward from the caller side (it's identical to how the Fn* traits are used today), and works the same with all suggested syntaxes as they use the same trait definition. One issue with all the syntaxes is they don't have any way to actually indicate these lifetimes, if we start with the running example:
let gen = |name: &'static str| {
    let (name1,) = yield "hello";
    let (name2,) = yield name1;
};

this is easily inferring the type impl Generator<(&'static str,), Yield = &'static str, Return = ()>.
We want to also be able to write a generator matching impl for<'a> Generator<(&'a str,), Yield = &'a str, Return = ()> (i.e. a generator that does not capture its input value) along with for<'a> impl Generator<(&'a str,), Yield = &'a str, Return = ()> + 'a (i.e. a generator that captures its input value, and so cannot outlive it). The syntax with an implicit lifetime could easily refer to either of these:
let gen = |name: &str| {
    let (name1,) = yield "hello";
    let (name2,) = yield name1;
};

I think this could be valid under either interpretation, depending on whether you can propagate a lifetime between the previous yield-return and the current yield-argument. But, trying to store a value across a yield:
let gen = |name: &str| {
    let (_,) = yield "hello";
    let (_,) = yield name;
};

This could only be valid under the second interpretation, where implicit lifetimes are captured into the generator, rather than being only valid for the single resume call.

For multiple different yield poins you'd have to add multiple lifetime arguments

Why?

For example this fails even though it could be valid, there's just (AFAIK) no way to write a set of signatures that satisfies the borrow checker.

To me it’s obvious that this example could not be valid for exactly the reason given by rustc’s error message. data is dropped at the end of foo, but still borrowed by gen.

To me it’s obvious that this example could not be valid for exactly the reason given by rustc’s error message. data is dropped at the end of foo, but still borrowed by gen.

But gen is also dropped at the end of foo. If you remove foo from the program, the exact same construct works in the body of main. The problem is that, as a parameter to foo, there is no lifetime you can give it to make it behave like a local binding.

Hmm I see. This seems to be a general issue with language expressivity of lifetimes in function signatures, not at all related to generators. Here is the same with Vec: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=262406bc79c51a44aeda0d90b27b9b0e
I don’t think this means that generators should not be allowed to borrow from their resume arguments.

Simply adding a rebinding changing the parameter into a local causes both example to compile, proving that is just a problem with lifetimes declared in function signatures.

I don’t think this means that generators should not be allowed to borrow from their resume arguments.

Neither do I, the usecases I care about only need transient per-resume borrows (G: for<'a> Generator<&'a FOO>. It is just something that seems much more likely to be useful as part of a generator, and if it were somehow possible to specify at the user/trait interface it might affect the syntax for declaring argument lifetimes.

Simply adding a rebinding changing the parameter into a local causes both example to compile, proving that is just a problem with lifetimes declared in function signatures.

So it does. I have absolutely no idea why this works, I guess it's somehow related to Gen being (co/contra?)variant over the lifetime, but why does that only apply when it's rebound to a local variable instead of as an argument?

I also don't like how this proposal spends the weirdness budget and I prefer the @SimonSapin's approach.
But I would like to take a step back and ask the following question: do we really want to reuse the closure syntax for generators? Honestly I find it really strange and worrying that we have to look into a "closure" body to understand if we have a true closure or generator on our hands.
I would strongly prefer to have a dedicated syntax for generators which would allow to immediately see that we create a generator only by looking at a given "signature" and which possibly would allow to specify argument, yield  and return types. Something like this (a very-very rough draft, feel free to propose a better alternatives):
// `generator` here is a keyword, so we will have to wait for the next edition

// a simple generator, return and yield types are inferred
// if not provided explicitly, `ResumeArg` is always set to `()`
// Yield=u8, Return=u32, ResumeArg=()
let gen1 = generator {
    yield 1u8;
    42u32
};
// specifies all generator types explicitly
// ResumeArg=&str, Yield=u8, Return=u32
let gen2 = generator[&str -> u8] -> u32 { ... };
// specifies only some types
// Yield=u8, Return=u32, ResumeArg=()
let gen3 = generator[u8] -> u32 { ... };
// yield type is inferred, ResumeArg=()
let gen4 = generator -> u32 { ... };
// return type is inferred, ResumeArg=&str, Yield=u8
let gen5 = generator[&str -> u8] { ... };
// return and yield types are inferred, ResumeArg=(u8, &str)
let gen6 = generator[(u8, &str) -> _ ] { ... };

We may even add something like generator fn:
generator fn foo(a: u8, b: u8) [&str -> u8] -> u32 {
    let s1 = yield 0;
    let s2 = match s1 {
        "ok" => yield a,
        _ => yield b,
    };
    42
}
let gen = foo(1, 2);
let y1 = gen.resume("foo");
let y2 = gen.resume("ok");
let r = gen.resume("bar");

But it's debatable whether there is a sufficient motivation for it.

I think I've found a good way to generalize coroutines in Rust that could be helpful here. Check out my post! I also included some comparison of the different proposals that people have made. I hope it can give us a common language to make this debate a little easier. For example,
generator fn foo(a: u8, b: u8) [&str -> u8] -> u32 {
    let s1 = yield 0;
    let s2 = match s1 {
        "ok" => yield a,
        _ => yield b,
    };
    42
}

is the same as
fn foo(a: u8, b: u8) -> impl FnPinMut(u8) -> GeneratorState<u32> {
    coroutine {
        let s1 = Yielded(0).yield;
        let s2 = match s1 {
            "ok" => Yielded(a).yield,
            _ => Yielded(b).yield,
        };
        Returned(42).yield;
        panic!()
    }
}

and
let gen = |name: &str| {
    let (_,) = yield "hello";
    let (_,) = yield name;
};

is the same as
let gen = coroutine {
    let (name,) = ().yield;
    let (_,) = Yielded("hello").yield;
    let (_,) = Yielded(name).yield;
    Returned(()).yield;
    panic!()
};

This can also describe odd kinds of coroutines that haven't come up yet. For example,
let future = ...;
let fused_future = coroutine {
    let ctx: &mut Context = ().yield;
    let value = await!(future, ctx);
    yield Ready(value);
    loop { yield Pending; }
};

or
let sum_every_other: impl FnPinMut(u32) -> u32 = coroutine {
    ().yield;
    let mut sum = 0;
    loop {
        sum += sum.yield;
        sum.yield;
    }
};

After getting a bunch of feedback on by post, I realized we can extend the magic mutation syntax to be completely general. We don't go as far as requiring "no return for generators" but we do require return and yield to take the same type. return x can then be desugared into yield x; unreachable!(). The user has to decide how to signal return themselves but I think giving the user that choice will have a lot of benefits:

a generator is just a pinned closure
generators can act like iterators by returning None
generators can act like futures by returning Poll::Ready
generators which use ? can signal return with Result::Err
generators can use .await so long as they take &mut Context and return Poll
an async { ... } block can desugar as |&mut Context| Poll::Ready(...)

The other option is to make every generator loop so that return desugars as yield x; continue. I think this is too weird to take seriously but it does have the unusual benefit that closure syntax and generator syntax become one and the same so long as pinning is only required once the generator creates an internal reference. That is, something like |x| x + 1 has the same exact meaning whether interpreted as a closure or a generator.

but we do require return and yield to take the same type.

I think there's a lot of value in allowing a generator to return a different type than it yields. This seems to be a very common pattern in Haskell streaming libraries, for leftovers, eg.

that's just an enum situation it seems like.

GeneratorState is not any other enum. When GeneratorState::Complete is returned, this signals at the language level that it is an error to call resume again.

@SimonSapin The same could be said of Option in the context of iterators or Poll in the context of futures.  I could conceivably create a fuse implementation for generators that produce GeneratorState just as easily as for iterators or futures and it could be just as useful.

In other words, GeneratorState::Complete signals at the library level that it is a logical error to poll the generator again, just like Option::None does for Iterator. Nothing in the language prevents you from polling again, or even providing an implementation of the trait that promises behavior after that yield (such as fuse).
That library convention is baked into the language via desugaring of await/for, but it's still "just" a trait around a function that returns an enum.

The same could be said of Option in the context of iterators

Yes, exactly. And it’s important that Iterator::next returns Option<Self::Item>, not an arbitrary Self::Item type.

Yah, that's because iterators are really high level. They have a semantic meaning of their own. I'm saying that people don't care if something is a "generator". They care if something is an iterator, stream, sink, future, reader, writer, etc. A generator is only valuable as a way of implementing one of those. In short, we have an opportunity here to make generators a lower level syntax which is useful for building up higher level objects! It makes me really really excited! Why create yet another convention when we can use all the ones that already exist?

Why create yet another convention when we can use all the ones that already exist?

Because people need to be able to compose them. This is the reason async/await is still a distinct language feature from generators- for example, you should eventually be able to write a single function that is both async and an iterator, or in other words a stream.
The same logic applies to the iterator and closure traits- they may have the same "shape," but they are used for different purposes and can be composed, so keeping them separate is valuable not only for readability but for expressiveness.

It's important to detach the "coroutine" and "generator" ideals.
"coroutine" is the "FnPin", where you yield some item on every poll.
"generator" is "just" a "coroutine that returns GeneratorState"; it yields an item on each poll then returns some final value; continuing after that is a logical error.
"async" is "just" a "coroutine taking an async context that yields blockers and returns an item".
So the generalized approach @samsartor is arguing for is a coroutine, but it would still behoove the language to have an idea of and integrate with generators even while they're built on top of stackless coroutines.
Approximately,
trait FnPinMut<Args> { // generalized stackless coroutine
    type Output;
    extern "rust-call" fn call_pin(self: Pin<&mut Self>, args: Args) -> Self::Output;
}

type Generator<Args, Yield, Return> = FnPinMut<Args, Item=GeneratorState<Yield, Return>>;
type Future<Output> = FnPinMut<&mut Context, Item=Poll<Self::Output>>;
type AsyncGenerator<Yield, Return> = FnPinMut<&mut Context, Item=Poll<GeneratorState<Yield, Return>>>; // ignore resume args for simplicity
}

Once we agree on terminology, then we can have a productive conversation. It's not exactly useful when one party wants to build the generalized approach and another wants the more specialized, while thinking they're discussing the same thing.

It isn't that I disagree with the Generator trait in principle. Its just that I think Rust would benefit from a common way of building coroutines. If people later want a less explicit way of building generators specifically, I'm cool with that. However, @CAD97 is right that serving "I want an escape hatch to make more complex iterators/closures/futures/streams." vs serving "I want a generator syntax specifically." is really a separate debate. This proposal is just so close to serving the first that I figured I might as well suggest so.

you should eventually be able to write a single function that is both async and an iterator, or in other words a stream.

https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-referential-structs/:
    Iterators: A generator which yields T and returns () can be treated as an Iterator of T.
    Futures: A generator which yields () and returns Result<T, E> can be treated as a Future of T and E.

Combining with: an async { ... } block can desugar as |&mut Context| Poll::Ready(...)
You get:

Iterator<T>: A generator which yields T and returns () can be treated as an Iterator of T.
Future<T>: A generator which takes &mut Context yields () and returns T can be treated as a Future of T
Stream<T>: A generator which takes &mut Context yields T and returns () can be treated as a Stream of T
Future<Iterator<T>>: A generator which takes &mut Context yields () and returns a generator-Iterator
Iterator<Future<T>>: A generator which yields a generator-Future and returns ()
...

So the question is:
Do we add some generator creation syntax plus utilities to convert between generators and iterator, future, stream, closure, etc. or do we add some coroutine creation syntax which can produce any of those types (including generators) directly? I see pros and cons to each but neither totally eliminates any possibilities.

you should eventually be able to write a single function that is both async and an iterator, or in other words a stream.

https://boats.gitlab.io/blog/post/2018-01-25-async-i-self-referential-structs/:
    Iterators: A generator which yields T and returns () can be treated as an Iterator of T.
    Futures: A generator which yields () and returns Result<T, E> can be treated as a Future of T and E.

Combining with: an async { ... } block can desugar as |&mut Context| Poll::Ready(...)
You get:
* `Iterator<T>`: A generator which yields T and returns ()

* `Future<T>`: A generator which takes `&mut Context` yields () and returns T

* `Stream<T>`: A generator which takes `&mut Context` yields T and returns ()

* `Future<Iterator<T>>`: A generator which takes `&mut Context` yields () and returns a generator-Iterator

* `Iterator<Future<T>>`: A generator which yields a generator-Future and returns ()

* ...

Looks promising.

Currently, the only way to declare generator\coroutine is by using closure syntax. We could make generators have no initial arguments, only resumes and yields, in such case generator declaration would look like:
fn makeG<RT,YT>(args...)-> impl Generator<Resume=RT,Yield=YT>{
    //do something with args
   move |a: RT| {
      //code with yields
   }
}

There, arguments of fabric fuction are moved inside of generator.
The logical finish of work of coroutine should be denoted inside yield type.
So:
Iterator<T> becomes Generator<Resume=(),Yield=T>;
Future<T> and Stream<T> become Generator<Resume=&mut Context,Yield=Poll<T>>;
AsyncGenerator<RT,YT> become `Generator<Resume=(&mut Context,RT),Yield=Poll>;
(Future should panic if polled after completion)
Actually it is a "stateful function object".

Also, we might want to be able to say "for now we don't take resume arguments, but in future we will take them again...".
Consider async generator which is waiting for future to complete, it will yield Poll::Pending, but I'm not sure if this enough, what will happen to data passed in resume which returned Poll::Pending? (it will be lost).
First argument problem is also the case.
Maybe we want typestate pattern?

JFYI: https://github.com/rust-lang/rust/pull/68524

Also, this allow efficient and natural actor system, where actor receiving message is just resume with the value, and send operation is yield certain variant of some enum. Combined with macroses this allow us to have easy to use actor system, which is also parallel and efficient due to being coroutines, not threads.
P.S. Smalltalk DSL?

I'm surprised I'm not seeing any mention of Generator<Yield = !>, because that should be pretty close to a closure, including the fact that GeneratorState<!, R> should be the same size as R.
Well, a FnPinMut closure, which doesn't really exist today.

I'm surprised I'm not seeing any mention of Generator<Yield = !>, because that should be pretty close to a closure, including the fact that GeneratorState<!, R> should be the same size as R.
Well, a FnPinMut closure, which doesn't really exist today.

Generator<Yield=!> is a closure in terms of generators. In terms of proposed FnPinMut, closure is coroutine with one resume point and one yield point, it have dedicated storage for captured values and thus it separated via object boundary. fn(RT)->YT is the same except absence of object boundary ( fn doesn't capture anything from non-'static enviroment and thus don't need to be instanciated before use)
Now, we really have opportunity to generalize fn-family with generators (and all their kinds)

We have also question: how does the ? operator work inside of coroutine?

I'd like to help the discussion along by summarizing the syntax discussion. I won't really address anything surrounding the implementation with trait definitions and whatnot. It seems the main issues of contention for syntax are the following:
0. Coroutine vs Generator
For some, both terms refer to the same syntactical and behavioral concept.. For others, coroutine refers to a generic behavior, and generator refers to the syntax allowing for easy definition of a specific kind of coroutine.
The original RFC (2033) refers to them interchangeably. I will use the term "Rust coroutines" to refer to the syntax and associated behavior that we are discussing. See this comment on one of the other pre-RFCs for some backup that coroutines is the preferable word.
1. Just a Closure with yield?
The original RFC introduced closures-with-yield as the syntax for defining Rust coroutines. Some think that perhaps using the same syntax is a mistake and causes too many other issues due to limitations of closure syntax. Others ask if any given closure is also a valid Rust coroutines, or if that only applies to closures without arguments, or not at all. Would it not be better to have a different syntax entirely if Rust coroutine syntax would just look similar but behave differently from normal closures?
My Thoughts
I think that the re-use of "just" closure syntax and the behavior we want out of Rust coroutines are impossible to reconcile. Especially if we want to be able to specify types within Rust coroutine declarations. Issues with using "just closure" syntax:

no way to force an arbitrary closure-like declaration to be a Rust coroutine
no way to fully specify input and output types at declaration
confusing difference in behavior between normal closures and Rust coroutines
magic of inferring a Rust coroutine when yield is present

I like the fully-qualified syntax |let (a, b): (A, B) = yield<Y>| -> R, with |let (a, b) = yield| as the shorthand with type inference, and just yield to ignore resume args (at least for the first resume).

(A, B) example tuple return argument type
Y yield output type
R return output type

Unlike other previous suggestions, it is unambiguous from <variable> <logical or> <code block>. It also echos later yield usage, making it very clear that yield is bound to those variables at the beginning of the scope.
2. Resume Types, Yield Types, Resume Types
Should we use the |...| field to specify the resume argument type? Should we use the return type field -> T to specify the yield output type? Should there be separate yield and return types, or should they be the same? Or should there be no return type at all ()? Some think that these types shouldn't even be specifiable, and should always rely on type inference.
My Thoughts
There are use cases for separate return and yield types, and this already exists in nightly. I don't think it makes much sense to artificially limit this capability. I do think there absolutely needs to be some way of manually specifying all of these types, which the current syntax does not provide.
I don't see why resume types should necessarily have to be tuples, either. There's no contractual requirement in the Fn* traits (that I can see) that requires that Args be a tuple.
3. Binding of Resume Arguments
Seems like the main question here is whether to allow arbitrary binding of the yield arguments, or whether to always require that they use the same names and are cannot be carried across yield points without explicitly doing so.
My Thoughts
I'm in agreement with @SimonSapin and yields should act like a normal expression which can be destructured, matched, etc any way people see fit. This seems to make the most sense and looks the best to me.
However, a big downside is that with current behaviors, those variables could be held until the Rust coroutine returns, which would be a lot of bloat. We'd need to introduce a differing behavior in Rust coroutines to avoid this issue, which could cause confusion when normal fns and closures work differently.
We'll have to carefully weights the pros and cons of both the binding-reuse of this pre-RFC and SimonSapin's more flexible suggestion.

Another thing to consider is the existing implementation, which works much like @SimonSapin has proposed:
let mut g = |mut one_two: &str| -> &str {
    println!("{}", one_two);
    one_two = yield 0; // rebind a mut variable / argument
    println!("{}", one_two);
    let three = yield a + b; // bind to a new variable
    three
};

println!("first resume: {:?}", Pin::new(&mut g).resume("one"));
println!("second resume: {:?}", Pin::new(&mut g).resume("two"));
println!("third resume: {:?}", Pin::new(&mut g).resume("three"));

Given this syntax will work going forward (not necessarily true), that resolves the issues of defining types for resume args, binding resume args, and having separate types for yield and return. I think the idea of having only a single argument works well in this case because we can only pass a single value into or out of the generator at a time. It avoids issues of always needing to destructure a tuple and converting a list of arguments into a tuple.
Remaining Issue: Do we provide a way to define the yield value type in the generator definition syntax? If so, how?
I think we must provide a way to do so. Some options:

|arg: ResumeType = yield YieldType| -> ReturnType {
|yield YieldType -> arg: ResumeType| -> ReturnType {
|arg: ResumeType| -> ReturnType yield YieldType {
|arg: ResumeType| yield YieldType -> ReturnType {
yield YieldType |arg: ResumeType| -> ReturnType {

Also consider variations wrapping YieldType or yield YieldType with <> or using the word yields instead of yield. My favorite option is the first, as it mirrors the actual assignment syntax of resume args and uses the already-reserved keyword yield. It also doesn't interrupt the return type area, and can be easily left out without causing much confusion when one wants to allow for type inference.
Any of these would solve the following two issues:

ability to force an arbitrary closure-like declaration to be a generator
ability to fully specify input and output types at declaration

And could solve the following two as well if we make the yield part mandatory to define a generator:

confusing difference in behavior between normal closures and generator
magic of inferring a generator when yield is present

I've burned a lot of brain cells working on corountine craziness since last November. Both on my own and with @CAD97 & @pcpthm in a draft RFC. I'd like to say I have something concrete to show for it but, thanks to my own distractability and lack of time, it has mostly resulted in a few out-of-date rustc branches, a massive WIP blog post, etc. But I do think I have a good response to @pitaj's argument that "that the re-use of 'just' closure syntax and the behavior we want out of generators are impossible to reconcile".
Those things can totally be reconciled! I can even explain such a proposal in two sentences:

The statement yield x; is allowed in closures. It behaves the same as return x; except the next time the closure is called, execution resumes immediately following the statement instead of at the top of the block.

To get "yield closures" in Rust you don't need any fancy Generator trait, wild type declaration syntax, or even a new syntax to distinguish between the yield/return types. All you need is a new FnPin trait for the occasional closure that requires address stability because of live borrows across yield statements (plus the MIR transform and some type checking shenanigans). In fact, there is no need even for the "magic of inferring a generator when yield is present". yield could be simply a feature that mutable closures have. There would be no effect to adding, for example, if false { yield unreachable!(); } at the beginning of every existing closure.
I really really want to find time to evangelize this more. Talk about how it makes "magic mutation" kinda obvious and very sane, how it makes async streams practically trivial, how you might integrate await into the mix to do coroutine delegation, how it makes implementing new Future combinators really easy, how much rustc could be simplified by truly unifying "generators" and "closures", even about the remaining challenges with those semantics and how I think I've solved them. But me shouting "hey everyone! I love how yield could be totally ordinary" isn't really the point.
The point is that any kind of new generator syntax (now available anywhere Rust is sold!) makes the a lot of sense as a sugar. @pitaj's proposed fully qualified syntax could be implemented with something like:
macro_rules! gyield {
    ($x:expr)  => { {
        yield GeneratorState::Yielded($x);
        __coarg
    } }
};

macro_rules! generator {
    (|let $ap:pat: $at:ty = yield<$yt:ty>| -> $rt:ty $x:expr)  => { |__coarg: $at| {
        let $ap = __coarg;
        yield GeneratorState::Completed($x);
        panic!("generator resumed after completion")
    } }
};

And there are cases where that sugar is quite useful! Having your different return/yield types automatically packed into an enum is pretty darn ergonomic. People want GeneratorState-wrapping for the same reason they want Ok-wrapping. But corountines don't really need to be their own enormous feature with all these kinds of syntatic considerations. Unification is possible!

@samsartor I don't have any problem with the idea of closures and generators actually being the same thing. That sounds like a novel and interesting idea, and one that would reconcile them.
What I was saying isn't reconcilable is having completely different behavior depending on whether a closure has yield or not. And it seems we agree on that point.
However I don't see how you could have different yield and return types without also having a new syntax for specifying the yield output type. (Or you could just force everyone to use type inference for it, which I don't agree with).
That's a minor issue and one easily solved. In fact, I'd say it makes sense that if the initial arguments into a generator are the same as the resume args, then it makes sense that the yield output type would be the same as the return type.
I like your idea though. Essentially the idea is that any given closure is not necessarily complete when it returns a value. The result of calling it the next time is dependent on whether that return value was provided by a yield or by a return (implicit or explicit).
Them you can reuse .call(...) for generators and use the same x(args...) syntax for generators as well.
This could also prevent some overhead when used for async functions, since it can return the future variants directly without needing wrapped in a generator variant.
It's more flexible, re-uses existing syntax, and is implemented as a fairly easy to understand extension to existing behavior.

I finally got round to revisiting my attempt at working out how to shim an argument accepting generator into a Sink now that we have resume arguments in nightly, though it's still blocked on https://github.com/rust-lang/rust/issues/68923.
One thing I have realised is that the "yield returns value" approach makes this macro-rules attempt impossible, since because of the changing lifetime on each yield they would need new bindings to store into. It also means that fixing rust#68923 wouldn't allow the async/await transform to be a trivial safe proc-macro level transform. Even a simple case like this I can't work out a possible expansion without needing to involve the existing lifetime erasure (which should be just to workaround rust#68923)
async {
  loop {
    foo().await;
  }
}

The "single binding updated by yield" approach mentioned in this RFC makes a macro-rules attempt of the core Future async/await syntax possible (though still not nice) and a proc-macro transform trivial.

However I don't see how you could have different yield and return types without also having a new syntax for specifying the yield output type. (Or you could just force everyone to use type inference for it, which I don't agree with).

I find it quite similar to  ... -> Result<T,E> vs ... -> T throws E. Your proposed syntax reminded me checked exceptions syntax. We could do just better naming for GeneratorState<Y,R> type, like Yield<Y,R>; we can note absence of yield or return types by doing Yield<Y,!> and Yield<!,R>, the last coroutine output type is actually a plain return without a yield.
Also, interaction with ?, error propagation operator, is totally unclear, where we want the error type to go, to return, yield, or entirely different branch? We could look for type which implements Try trait, but if both Yield and Return types implement Try? This is area of effect systems.

I have a bunch of ideas, but they involve more than generators feature.
So, I propose following:

Rename GeneratorState<Y,R> to Yield<Y,R> to align well with Result<T,E> and reduce typing overhead.
Add trait to allow piping iterator output to coroutine input, aka resume arguments. Propose name Fuser.
? operator in coroutines: Make it always do termination. For possibility of propagating error to yield type, there two problems: 1) Which resume point will be selected by next resume call after such propagation? 2) How maintain correctness considering 1) ? 3) Even if we go with magic mutation approach, and resume directly on the propagation site, there no way to resume after propagation, because it would require correct value to recover error, which we can't provide in general case.
Add dedicated Stream and IntoStream traits. This is because Stream is not Iterator<Item=impl Future> nor Future<Output=impl Iterator>. Symmetrically to Iter and IntoIter this should be consumed by for-loops.
Add second form of for loop, async one; Precise syntax is for x.await in stream {...} and for x.await? in y.stream() {...}: first one simply process the stream, second additionally propagates any error up to the caller.
A bit off topic, but we could do for x? in iter in sync, if iter's Item is Try.

For working with non-terminating errors there try_block feature, and lot of other stuff. The reason for "Fuser" trait is that we might want to iterate over coroutines, at least partially, here we have problem with leftovers: what if data which was streamed into some coroutine was not sufficient for it to complete? In this case we want a way to get the coroutine back for further processing and get its return value when it's ready.

A bit off topic, but we could do for x? in iter in sync, if iter's Item is Try.

This is unnecessary, you can do that propagation in the loop body. Similarly with .await?, it's not needed, do it in the body.

I came across this RFC today, and, after reading through the discussion, the thing I feel most strongly about is that simply introducing yield shouldn't magically change the return type or behavior from the callers perspective, and we shouldn't try to cram additional types into the current closure syntax. I do think there are reasonable solutions on either side of that, though.
Option 1: Closure-like, creates regular FnMut or FnPinMut:
On the near side, I liked @samsartor's suggestion in https://github.com/rust-lang/rfcs/pull/2781#issuecomment-639199953 of making yield a small extension to the existing closure syntax: you still get an FnMut (or an FnPinMut where necessary), and the only difference between yield and return is where execution resumes on subsequent calls (after the yield versus back at the top). In this case, yield is just making it nicer to do something you can already do with FnMut. One appealing thing about this approach is because it's a relatively small addition, several questions that have come up have fairly obvious answers.

Should generators return values of a special type (GeneratorState)? Can yield and return provide different types?
No. yield and return must both provide an expression matching the closure's return type, which is the same type the caller receives when invoking it. However, this can be an enumeration (Yield<Y, R>, Option<T> for an iterator-like coroutine, et cetera), with yield and return providing different variants.

Should starting a coroutine be different from resuming it, with different parameters?
No. Both the initial and subsequent invocations use the same call operation and the same parameters (but potentially different values), just like invoking an FnMut multiple times, today.

What about inputs that need to be provided once up front and held by the coroutine?
They should be captured by the closure, rather than passed as arguments.

What are the lifetime requirements for values passed in when resuming?
Exactly the same as calling a standard closure with the same signature. This mean if the coroutine wants to store arguments across yield points, it needs to specify an appropriately long lifetime for the respective parameters.

How does a coroutine receive the arguments passed when called after a yield?
This one could probably still be debated, but given the mental model of this syntax, and with the answer to (4) above, having the closure parameters reflect arguments passed to the current invocation makes the most sense, regardless of whether execution is starting from the top or after a yield. I admit this can be a little weird, in some cases:
let coroutine = |x: i32| {
  dbg!(x);  // 1
  {
    let x = x;  // Copy and shadow x
    yield;
    dbg!(x);  // 1 (prints shadowing copy)
  }
  dbg!(x);  // 2 (parameter no longer shadowed, has value from current call)
}
coroutine(1);
coroutine(2);

However, given the suggested mental model, I don't think it's that weird, and actually makes sense once you understand what's going on. I also think, given (4), the alternative (yield returning the new arguments) could lead to frustrating borrow check errors if the parameters contain lifetimes and the user, by storing the result in the obvious way, causes the arguments to remain live across a subsequent yield.

Can you yield from an async closure? How do you use it to implement a Stream?
This seems to be one of the less obvious questions to answer for any of the proposals (including this one) where every invocation of the coroutine (including the first) takes the same parameters. This is rather different than Futures and Streams, where the initial invocation serves more or less as a constructor, with the passed arguments getting stored for later, and subsequent calls to poll or poll_next don't provide any additional input to the function.
One solution might be to create a ResumableFuture trait that gets automatically implemented for any async closures that can be invoked multiple times (i.e., closures that don't consume their captures) regardless of whether they use yield or not. (Another option might be to put a resume method directly on Future and have the associated ResumeArgs type be ! for FnOnce-style closures.) To turn this into a stream, ResumeArgs would have to be (), which means any needed state (e.g., a TcpSocket to listen on) would need to be captured by the closure.
Aside: I originally thought it might be possible to handle async+yield by invoking the coroutine multiple times, which each invocation returning a different future. Unfortunately, this runs into two problems. First, the future generated by async takes ownership of the needed state, rather than borrowing the originating closure. As such, driving the future can't update state in the closure. Second, even if that were changed, there's nothing to stop the user from driving the future only half way to completion, dropping it, and calling the closure again, which could result in the coroutine state being inconsistent.

Option 2: async-like, generates impl Generator
On the far side, I like the idea of having a distinct keyword, analogous to today's async, and attaching any additional types to that. Of course, this leaves all of the other questions open for debate. If we go this route, I think we should try to be as similar to async as possible to reduce the novelty of the feature, leading to the following constraints (names of traits and types are placeholders):

There should be a keyword (which I'll refer to as generator for now) that can be applied to functions, closures, and blocks.
When a generator function or closure is invoked, it returns an impl Generator capturing the arguments and state. No code from the body is executed until the first resume call.
A generator block immediately evaluates to an impl Generator.
Calling resume takes a value of the associated ResumeArg type, and returns a GenerateState enum containing either Yield(YieldType) or a Return(ReturnType).
It is an error to call resume again after receiving Return.

I think (2) is the trickiest of the requirements, and indeed I have no solution for it. The requirement is definitely necessary, as, like a Future, a Generator needs to be moveable until the first resume call. On the other hand, it's unclear what should happen to the value passed to the first call to resume. Dropping it on the floor definitely seems wrong, but providing some bespoke method to obtain it (unlike future resumes, which can just return it from yield) feels gross.
Setting that aside for the moment, I think such a syntax could look something like the following, partially inspired by @newpavlov's suggestion in https://github.com/rust-lang/rfcs/pull/2781#issuecomment-544373078):
// Free function. Initial call takes two &str args, takes an i32 for each resume,
// yields usize, and finally returns a String.
generator[i32 -> usize] fn(arg1: &str, arg2: &str) ->  String {
  // …
}

// Closure.
// Yield type and return type can be omitted and inferred.
generator[i32] |arg1: &str, arg2: &str| {
  //…
}

// Async block. Immediately evaluates to an `impl Generator`.
generator[i32] {
  //…
}

My opinion
If we can find an elegant solution for passing the initial resume argument for option 2, I think I'd lean toward that option, as it would make both compiler state-machine transforms very similar, and it would be very easy to understand one after having learned the other.
However, if this turns out not to be possible, or even just requires consequential divergence from the way async functions work, then I very strongly favor option 1, as it basically just provides a nicer way to write something that FnMut can already do today, with FnMutPin being a very straightforward extension to allow closure state to contain internal references. While I admit having the arguments be different after a yield is a little weird at first glance, I find it makes sense after I think about what's actually going on. My closure has been called again with different arguments, it's just that I've opted (by using yield instead of return) for execution to jump down to after the yield instead of starting at the top. In any case, I find the cognitive load added by this weirdness to be significantly less than that of having a third construct that acts neither like a closure nor like async.

@rkjnsn Wow, this is a really great breakdown of these two options, and tracks pretty well with my understanding of the subject!
The only correction I'd like to make is on "shadowing resume arguments" vs "reassigning resume arguments". Under MCP-49 (Option 1 above), the previous resume arguments are not shadowed. Instead they are completely dropped when entering a yield and the new arguments take their places when exiting it. This does prevent users from naively holding references to prior arguments, but the user <--> compiler feedback loop in that case looks fairly nice:
=> |x| {
    let y = &x;
    yield;
    dbg!(y, x);
}

error[E0506]: cannot assign `x` on yield because it is borrowed
 --> src/lib.rs:3:4
  |
2 |     let y = &x;
  |             -- borrow of `x` occurs here
3 |     yield;
  |     ^^^^^ assignment to borrowed `x` occurs here
4 |     dbg!(y, x);
  |          - borrow later used here
  |
  = help: consider moving `x` into a new binding before borrowing

=> |x| {
    let a = x;
    let y = &a;
    yield;
    dbg!(y, x);
}

As you mentioned, shadowing previous resume arguments can be quite confusing. But more importantly, it makes implementing poll functions needlessly difficult by introducing context reuse bugs in code like the following:
std::future::from_fn(|ctx| {
  if is_blocked() {
    register_waker(ctx);
    yield Pending;
  }

  while let Pending = task.poll(ctx) { .. }
})

In my experience, it is fairly rare that corountine authors want to reuse past arguments. And when they do, creating a new binding is a clear and obvious workaround. Hence, I'm pretty solidly in the reassignment camp.
Can you yield from an async closure?
In my mind, no. @CAD97, @pitaj, @pcpthm, and myself had quite a vigorous debate around this when drafting MCP-49 and basically came to the conclusion that there was no obvious way to accomplish it, at least for the time being.
I think the reasoning is simple: coroutines are all about giving the user control over per-resume input but Futures (and async blocks in general) already know what per-resume input they want, an std::task::Context. Trying to satisfy both the desires of the async keyword and the user simultaneously is a loosing battle. It sounds like you quite thoroughly explored the consequences of that!
How do you use it to implement a Stream?
Even without much syntactic sugar, it isn't too hard under MCP-49. A Stream::then implementation looks roughly like:
std::stream::from_fn(|ctx| {
  while let Some(item) = await_with!(inner.next(), ctx) {
    yield Ready(Some(await_with!(func(item), ctx)));
  }
  Ready(None)
})

Although we can all agree that a generator sugar makes this look nicer!
async gen {
  while let Some(item) = inner.next().await {
    yield func(item).await;
  }
}

So generators or yield closures?
Por que no los dos?
This is just my opinion, but I think these are really two completely different features with two different goals. Trying to pack both into one syntax just creates a mess that is worse for both cases.
Generators want only one thing: easily implement Iterator and Stream.  Once you realize that, the syntax becomes pretty trival.
If you want to hear my thoughts on generalized coroutines, how they might support generators (as either a language feature or proc-macro crate), and what else they might have to offer (yield closures + iterator combinators = unlimited power), check out the design notes I wrote for the language team.

The only correction I'd like to make is on "shadowing resume arguments" vs "reassigning resume arguments"

To clarify my point on shadowing, it was indeed my understanding with option 1 that the existing arguments would dropped on yield, with the parameters getting set to the new arguments on resume. Indeed, I think that's one of the key benefits of this approach over some other suggestions for generalized coroutines discussed in this thread. My point about shadowing was that if user explicitly shadowed a parameter (the let x = x; line in my example), then the parameter would still be shadowed upon resumption, even though the previous argument would have been dropped and a new one passed. Once executions leaves the scope with the shadowing binding, however, the parameter (now holding the new argument) would be visible, again.
I was merely pointing out that this could seem a little weird at first ("how is the parameter reassigned when it isn't even visible?"), but that (to me, at least) it made sense after thinking about it for a bit. So, basically I was agreeing with you. :slightly_smiling_face:

Can you yield from an async closure? In my mind, no.

This seems reasonable. While I think you could probably get it to do something (basically, you'd have poll and resume as separate operations, and you'd have to poll to ready before you could resume), it wouldn't exactly be nice (it's basically two separate, commingled coroutines with a weird interface at that point), and the from_fn+await_with! seems much better. Especially considering…

So generators or yield closures? Por que no los dos?

Agreed. I was evaluating both options from a perspective of "assuming we want generalized coroutines, which syntax makes sense?"
Given that resume arguments are where async-like syntax falls short, and given that iterators and streams don't care about that functionality, I certainly have no objection to providing async-like syntax for those use cases, either as a stop gap before general coroutines land, or as a user-friendly facade over them for the common case.

My point about shadowing was that if user explicitly shadowed a parameter, then the parameter would still be shadowed upon resumption.

Ah, I totally misread your code example. It's actually kind of an interesting case I hadn't thought about, thanks for sharing! ThankfuIy, I think it is behavior that's kind of hard to get in the first place, and that users familiar with Rust's shadowing rules shouldn't be too surprised by it.
It sounds like we're pretty much on the same page! 😁

Generators want only one thing: easily implement Iterator and Stream.  Once you realize that, the syntax becomes pretty trival.

I disagree with this, I would really like generator based Sink/AsyncRead/AsyncWrite implementations too to replace the state machine contortions you need to deal with them currently.

So generators or yield closures? Por que no los dos?

Agreed. I was evaluating both options from a perspective of "assuming we want generalized coroutines, which syntax makes sense?"
Given that resume arguments are where async-like syntax falls short, and given that iterators and streams don't care about that functionality, I certainly have no objection to providing async-like syntax for those use cases, either as a stop gap before general coroutines land, or as a user-friendly facade over them for the common case.

The first option is more general. It helps writing any state machine, including the same as produced by async, at the cost of requiring spelling out the state enum. The second option, on the other hand, is just a different special case similar to async.
In that light, I would:

say that the first option fits the purpose of specific RFC better,
vote for doing the first option first, and
creating a separate RFC to bikeshed the second (the first one is fairly minimal, so there fortunately isn't much to bikeshed).

I would really like generator based Sink/AsyncRead/AsyncWrite implementations too to replace the state machine contortions you need to deal with them currently.

We all want a nicer syntax for those traits! But I would say you are looking for a generalized coroutine syntax. For example, here is a sketch of using MCP-49's yield closures to decode a base64 stream:
let mut decoder = |sextet: u8, octets: &mut ReadBuf| {
    let a = sextet; // witness a, b, and c sextets for later use
    yield;
    let b = sextet;
    octets.append_one(a << 2 | b >> 4); // aaaaaabb
    yield;
    let c = sextet;
    octets.append_one((b & 0b1111) << 4 | c >> 2); // bbbbcccc
    yield;
    octets.append_one((c & 0b11) << 6 | sextet) // ccdddddd
};

io::read_from_fn(move |ctx, octet_buffer| {
     // do some prep work during the first call to poll_read
    pin!(inner);
    let mut sextet_buffer = ReadBuf::new([MaybeUninit::uninit(); 1024]);

    'read loop {
        // wait for the inner reader to provide some bytes
        await_with!(AsyncRead::poll_read, &mut inner, ctx, &mut sextet_buffer)?;

        for byte in sextet_buffer.filled() {
            while octet_buffer.remaining() == 0 {
                // the given buffer is filled, the poll_ready function should
                // return Ready
                yield Ready(Ok(()));
                // pick up where we left off
            }

            // pass a byte of input to our decoder
            decoder(match byte {
                b'A'..=b'Z' => byte - b'A' + 0,
                b'a'..=b'z' => byte - b'a' + 26,
                b'0'..=b'9' => byte - b'0' + 52,
                b'+' | b'-' => 62,
                b'/' | b',' | b'_' => 63,
                b'=' => return Ready(Ok(())),
                e => yield Ready(Err(InvalidChar(e).into())),
            }, octet_buffer);
        }
    }
})

Unlike yield closure implementations of Iterator and Stream, which benefit from some sugar on top of this, this AsyncRead impl is pretty hard to improve upon. You could eliminate a little verbosity by finding some way to make the ctx resume arg implicit, and get back the .await syntax, but it is a fairly small gain for quite a lot of magic.

And here's an example using async-io-macros (full example):
async_io_macros::async_read! {
    futures::pin_mut!(input);

    loop {
        let mut bytes = [0; 4];
        let len = input.read(&mut bytes).await?;
        if len == 0 {
            break;
        }
        input.read_exact(&mut bytes[len..]).await?;
        for byte in &mut bytes {
            *byte = match *byte {
                b'A'..=b'Z' => *byte - b'A',
                b'a'..=b'z' => *byte - b'a' + 26,
                b'0'..=b'9' => *byte - b'0' + 52,
                b'+' | b'-' => 62,
                b'/' | b',' | b'_' => 63,
                b'=' => b'=',
                _ => return Err(std::io::Error::new(std::io::ErrorKind::Other, "invalid char")),
            }
        }
        let out = [
            bytes[0] << 2 | bytes[1] >> 4,
            bytes[1] << 4 | bytes[2] >> 2,
            bytes[2] << 6 | bytes[3],
        ];
        let mut out = if bytes[2] == b'=' { &out[..1] } else if bytes[3] == b'=' { &out[..2] } else { &out[..] };
        while !out.is_empty() {
            yield |buffer| {
                let len = buffer.len().min(out.len());
                buffer[..len].copy_from_slice(&out[..len]);
                let (_, tail) = out.split_at(len);
                out = tail;
                Ok(len)
            };
        }
    }

    Ok(())
}

I'm not 100% happy with the looped yielded closure for output, but I haven't had time to try and come up with a better syntax.
Ah, I think I see the disconnect here, when you say generator you're specifically talking about something used for iterator/stream-like usecases? While I see generator (in Rust land) as being generalized coroutine syntax, since that's what it is currently on nightly.

Ah, I think I see the disconnect here, when you say generator you're specifically talking about something used for iterator/stream-like usecases? While I see generator (in Rust land) as being generalized coroutine syntax, since that's what it is currently on nightly.

Yep. The generalized coroutine design notes go over this in detail, but the term "generator" is a bit overloaded in the Rust language design space right now. I tend to use it more to describe the generators in RFC-2996 (gen fn) than the ones in RFC-2033 (the current Generator trait). Neither is strictly wrong, but since I was making a point about including both yield closures (which are fairly similar to generators today) and gen functions (which are more like the async-stream or propane macros), the distinction was important.

What if we make yield to have syntax: receiver.yield expr, meaning that the receiver mutable binding will be assigned with the next resume argument?
This makes things explicit about where the next resume argument will be stored, so that the magic mutation is no longer too confusing.
UI with borrows is likely to be improved, as if there is a borrow of some binding, then users cannot assign to it, neither with plain assign nor by yielding to it (hence the requirement for receiver to be a mutable binding).
And if desired, we could easily allow yield expr to be also an expression - this proposal doesn't intervenes in any way.

I don’t see what foo.yield bar brings over foo = yield bar where yield bar is an expression.

Another idea about the topic is that we can simply say that the resume argument isn't initialized before the first yield, that behavior would be surprising yet correct.

After a lot of time and reading design notes many times, I got a feeling that there are two entirely distinct level of abstractions involved:

First is what people often want for implementing iterators and streams: co-called gen fns from RFC 2996. These are high level concepts useful in a lot of places (for example, proposed async for);
Second one is level of protocol state machines and other coroutines.

I think that instead of trying to fit one feature on two uses, we have to design distinct UIs for the feature:
The high-level use I imagine is more of that in RFC 2996, here we allow functions to have only yield and produce impls of either Stream or Iterator. When exhausted we default to restarting unless we have a move capture  - in this case coroutine becomes poisoned.
The low-level use and API is described in MCP-49. On behavioural part we don't do implicit restarting and instead always poison coroutine when final state is reached (we mention this in the docs, and ask users to explicitly loop internals of their coroutines if needed). return is allowed and can have type other than yield; in case of both being used in same closure, the resulting type implements Generator and has GeneratorState wrapping. If a closure has only yields, it implements only the most basic FnPinMut trait.
We need to somehow distingush all these kinds, I think that more common (and one having less types involved) high level use and thus it deserves its top-level own syntax as of async.  My bet is gen fn syntax for high level case and just reusing closures for low level case.
Traits are following:

Iterator<Item=Item> ~= FnPinMut(())->Option<Item> + Unpin this is case for gen fn and gen closures;
Stream<Item=Item> ~= FnPinMut(&mut Context) -> Option<Item> - async gen  closures and fns;
Generator<R,Yield=Item,Return=Return> ~= FnPinMut(R) ->GeneratorState<Item,Return> - closures with both yield and return;
AsyncGenerator<R,Yield=Item,Return=Return> ~= FnPinMut(R,&mut Context)->impl Future<Output = GeneratorState<Item,Return>> - async generators.

Today, only FnPinMut and AsyncGenerator are absent. Also, we can implement these traits for FnPinMut types with right bounds and make state machine transform to just produce impl FnPinMut types.

i wonder: why not just do what async/await did and return the state machine after calling the generator?
consider the following:
let g = |stuff| {
     yield stuff;
    /* ... */
};

currently, this g will contain the state machine. however, what if g was simply a closure that returns a state machine after being called with the necessary arguments, like async fns? that would allow us to relax our requirements so that we can have generator functions, as well as being more consistent (we call .resume() to get the next value, which is similiar to .poll(ctx) in async/await)

The high-level case I imagine is more of that in RFC 2996, here we allow functions to have only yield and produce impls of either Stream or Iterator. When exhausted we default to restarting unless we have a move capture - in this case coroutine becomes poisoned.
The low-level case and API is described in MCP-49. On behavioural part we don't do implicit restarting and instead always poison coroutine when final state is reached (we mention this in the docs, and ask users to explicitly loop internals of their coroutines if needed). return is allowed and can have type other than yield; in case of both being used in same closure, the resulting type implements Generator and has GeneratorState wrapping. If a closure has only yields, it implements only the most basic FnPinMut trait.

I think this is backwards.  There is no need for different yield and return types or poisoning to be built in to the lower level feature, which should do nothing but add yield to closures and make them FnMut or FnPinMut as needed.  Those aspects can be added on top by some syntax sugar, similarly to how async/await is supposed to be syntax sugar over generators/coroutines.  In the case of async/await, the higher level feature is actually more complex, with its context and wakers, although this complexity is nicely hidden away by the sugar and only relevant when you're working with the Future trait directly to implement a primitive or an executor.
let mut store = Some(consume_once);
let mut coroutine = move || {
  let consume_once = store.take().expect("poisoned!");
  yield Yielded(42);
  drop(consume_once);
  Complete("hello")
};
assert_eq!(coroutine.resume(()), Yielded(42));
assert_eq!(coroutine.resume(()), Complete("hello"));
coroutine.resume(()); // poisoned!


Can't build with my Webpack/Babel bundled React app

My React app fails to build whenever I import mssql. It's important that I do this on the client side, since I won't be running on an Node/Express server. It seems that executing in Webpack browser bundle requires additional configuration.  Some kind of Polyfill maybe? I can't find any help on this anywhere  -- I could really use some advice.
The build errors and warning are:
    WARNING in ./~/mssql/lib/tds-fix.js
    Module not found: Error: Cannot resolve module 'tds/package.json' in /Users/hills/upload-covers/node_modules/mssql/lib
     @ ./~/mssql/lib/tds-fix.js 6:8-35

    WARNING in ./~/mssql/lib/tds-fix.js
    Module not found: Error: Cannot resolve module 'tds/lib/tds-constants.js' in /Users/hills/upload-covers/node_modules/mssql/lib
     @ ./~/mssql/lib/tds-fix.js 13:4-39

    WARNING in ./~/mssql/lib/tds-fix.js
    Module not found: Error: Cannot resolve module 'tds' in /Users/hills/upload-covers/node_modules/mssql/lib
     @ ./~/mssql/lib/tds-fix.js 14:4-18

    ERROR in ./~/mssql/lib/msnodesql.js
    Module not found: Error: Cannot resolve module 'msnodesql' in /Users/hills/upload-covers/node_modules/mssql/lib
     @ ./~/mssql/lib/msnodesql.js 9:14-34

    ERROR in ./~/mssql/lib/msnodesqlv8.js
    Module not found: Error: Cannot resolve module 'msnodesqlv8' in /Users/hills/upload-covers/node_modules/mssql/lib
     @ ./~/mssql/lib/msnodesqlv8.js 9:14-36

    ERROR in ./~/mssql/lib/tds.js
    Module not found: Error: Cannot resolve module 'tds' in /Users/hills/upload-covers/node_modules/mssql/lib
     @ ./~/mssql/lib/tds.js 9:8-22

    ERROR in ./~/tedious/lib/connection.js
    Module not found: Error: Cannot resolve module 'net' in /Users/hills/upload-covers/node_modules/tedious/lib
     @ ./~/tedious/lib/connection.js 25:13-27

    ERROR in ./~/tedious/lib/instance-lookup.js
    Module not found: Error: Cannot resolve module 'dgram' in /Users/hills/upload-covers/node_modules/tedious/lib
     @ ./~/tedious/lib/instance-lookup.js 3:12-28

    ERROR in ./~/tedious/lib/message-io.js
    Module not found: Error: Cannot resolve module 'tls' in /Users/hills/upload-covers/node_modules/tedious/lib
     @ ./~/tedious/lib/message-io.js 11:10-24

Here is my webpack config:
    var path = require('path');
    var webpack = require('webpack');
    var HtmlWebpackPlugin = require('html-webpack-plugin');

    module.exports = {
      devtool: 'eval-source-map',
      entry: [
        'webpack-hot-middleware/client?reload=true',
        path.join(__dirname, 'app/main.js')
      ],
      output: {
        path: path.join(__dirname, '/dist/'),
        filename: '[name].js',
        publicPath: '/'
      },
      plugins: [
        new HtmlWebpackPlugin({
          template: 'app/index.tpl.html',
          inject: 'body',
          filename: 'index.html'
        }),
        new webpack.optimize.OccurenceOrderPlugin(),
        new webpack.HotModuleReplacementPlugin(),
        new webpack.NoErrorsPlugin(),
        new webpack.DefinePlugin({
          'process.env.NODE_ENV': JSON.stringify('development')
        })
      ],
      module: {
        loaders: [{
          test: /\.js?$/,
          exclude: /node_modules/,
          loader: 'babel',
          query: {
            "presets": ["react", "es2015", "stage-0", "react-hmre"]
          }
        }, {
          test: /\.json?$/,
          loader: 'json'
        }, {
          test: /\.css$/,
          loader: 'style!css?modules&localIdentName=[name]---[local]---[hash:base64:5]'
        }]
      }
    };

Thanks.

@hillscottc were you ever able to resolve this issue? I just encountered the same problem.

Made a pull request: https://github.com/patriksimek/node-mssql/pull/325

Umn, I'm pretty sure you still won't be able to do this, as the tds creates a direct socket connection, which a browser won't be able to do... unless this is building for something like electron?

Personally, I use browserify to bundle server-side code for simpler deployment.

@jholster , I use browserify, but i get 'Error: Cannot find module './tedious' when i try to connect to a database. Any pointers to fix this?

Add tedious to your bundle...  This module dynamically uses the specified (default to tedious) driver upon connecting...
However, as I said, I don't think it will work, as there's no such thing as a direct real socket connection in a web browser.

Thanks, @tracker1, have done that, even updated  @jholster  patch as well. Still, couldn't get it to work. Is there any other way I can use this package?

You can't do a direct socket connection to SQL from the browser... you could create an application with something like electron, or you'll need a service/proxy setup on a server separately from SQL Server.

I met the problem also, cause i'm using electron trying to connect mssql in my app.

Implemented in v4 alpha. Drivers are no longer loaded dynamically so the library is now compatible with Webpack. To use msnodesqlv8 driver, use const sql = require('mssql/msnodesqlv8') syntax.

Looking forward to trying this.

I'm having trouble with figuring this out to work with webpack still.  You mention to use const sql = require('mssql/msnodesqlv8') instead of const sql = require('mssql') but I still get an error during bundling.  Are we required to use this driver instead of Tedious?
> webpack && electron .

Hash: 1693f2f917bb683de147
Version: webpack 2.6.1
Time: 2227ms
      Asset     Size  Chunks                    Chunk Names
renderer.js  1.01 MB       0  [emitted]  [big]  main
   [0] ./~/process/browser.js 5.42 kB {0} [built]
   [4] ./~/object-assign/index.js 2.11 kB {0} [built]
  [10] ./~/react-dom/lib/ReactUpdates.js 9.53 kB {0} [built]
  [16] ./~/react/lib/ReactElement.js 11.2 kB {0} [built]
  [18] ./~/react-dom/lib/ReactReconciler.js 6.21 kB {0} [built]
  [19] ./~/react/lib/React.js 5.08 kB {0} [built]
  [52] ./~/react/react.js 56 bytes {0} [built]
  [94] ./~/react-dom/index.js 59 bytes {0} [built]
  [95] ./renderer.js 2.54 kB {0} [built]
 [137] ./~/mssql/msnodesqlv8.js 49 bytes {0} [built]
 [156] ./~/react-dom/lib/ReactDOM.js 5.17 kB {0} [built]
 [163] ./~/react-dom/lib/ReactDOMInvalidARIAHook.js 3.14 kB {0} [built]
 [164] ./~/react-dom/lib/ReactDOMNullInputValuePropHook.js 1.37 kB {0} [built]
 [170] ./~/react-dom/lib/ReactDOMUnknownPropertyHook.js 4.32 kB {0} [built]
 [173] ./~/react-dom/lib/ReactDefaultInjection.js 3.5 kB {0} [built]
    + 215 hidden modules

ERROR in ./~/mssql/lib/msnodesqlv8.js
Module not found: Error: Can't resolve 'msnodesqlv8' in 'C:\Users\Chris\projects\electron\sysinfo\node_modules\mssql\lib'
 @ ./~/mssql/lib/msnodesqlv8.js 3:18-40
 @ ./~/mssql/msnodesqlv8.js
 @ ./components/App.js
 @ ./renderer.js
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR<EMAIL_ADDRESS>start: `webpack && electron .`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      C:\Users\Chris\AppData\Roaming\npm-cache\_logs\2017-06-16T15_35_32_947Z-debug.log

Same problem for me. Cannot connect to sql from browser. Using latest version of mssql and mssqlnodev8! see here: https://github.com/tediousjs/node-mssql/issues/1025

You can't do a direct socket connection to SQL from the browser... you could create an application with something like electron, or you'll need a service/proxy setup on a server separately from SQL Server.

Why is this not mentioned anywhere? How would anyone come across this information? I'm finding it very difficult to understand how to learn Node development because of these type of issues

@aysiscore, this module (node-mssql) is built for node.js, not for the browser.  You can create a web/api server to do your SQL calls... if you need this on the client, you may want to consider electron[1] or carlo[2]

https://github.com/electron/electron
https://github.com/GoogleChromeLabs/carlo

@aysiscore Normally, you build a web page/application that talks to a server... the server can be written in node, talk to sql server and return the response to your application/page.  The browser doesn't talk directly to the backend.
I only mention electron and carlo, as there are times you do want something directly on a desktop/client that can do these things.  There are other issues with doing these things and have their own issues that I won't get into.
If you don't understand how web API calls (fetch api) and api servers work, I would look into that.  Slightly more advanced is going to be options like web-socket systems as well as binary API interfaces.

@tracker1 thanks for your very clear explanation of this.
My opinion is that we can't put warnings on packages saying what they can't be used for. It's clear this is a node package and not a browser package.
I'm going to lock this issue as resolved beucase @tracker1 has been kind enough to give his time to explain this eloquently, calmly and clearly.