Reducer Composition with Effects in JavaScript

Inspired by this little tweetstorm.
Problem: Side Effects and Composition
We discussed effects in Redux for ages here but this is the first time I think I see a sensible solution. Alas, it depends on a language feature that I personally won’t even hope to see in ES. But would that be a treat.
Here’s the deal. Redux Thunk and Redux Saga are relatively easy to use and have different pros and cons but neither of them composes well. What I mean by that is they both need to be “at the top” in order to work.
This is the big problem with middleware. When you compose independent apps into a single app, neither of them is “at the top”. This is where the today’s Redux paradigm breaks down—we don’t have a good way of composing side effects of independent “subapplications”.
However, this problem has been solved before! If we had a fractal solution like Elm Architecture for side effects, this would not be an issue. Redux Loop implements Elm Architecture and composes well but my opinion its API is a bit too awkward to do in JavaScript to become first-class in Redux. Mostly because it forces you to jump through the hoops to compose reducers instead of just calling functions. If a solution doesn’t work with vanilla combineReducers(), it won’t get into Redux core.
Solution: Algebraic Effects in JavaScript
I think that what @sebmarkbage suggested in this Algebraic Effects proposal is exactly what would solve this problem for us.
If reducers could “throw” effects to the handlers up the stack (in this case, Redux store) and then continue from they left off, we would be able to implement Elm Architecture a la Redux Loop without the awkwardness.
We’d be able to use vanilla combineReducers(), or really, any kind of today’s reducer composition, without worrying about “losing” effects in the middle because a parent didn’t explicitly pass them up. We also would not need to turn every reducer into a generator or something like that. We can keep the simplicity of just calling functions but get the benefits of declaratively yielding the effects for the store (or other reducers! i.e. batch()) to interpret.
I don’t see any solution that is as elegant and simple as this.
It’s 2am where I live so I’m not going to put up any code samples today but you can read through the proposal, combine it with Redux Loop, and get something that I think is truly great.
Of course I don’t hold my breath for that proposal to actually get into ES but.. let’s say we could really use this feature and Redux is one of the most popular JavaScript libraries this year so maybe it’s not such a crazy feature as one might think at first :smile: .

cc people who contributed to relevant past discussions: @lukewestby @acdlite @yelouafi @threepointone @slorber @ccorcos

I'd love this with a twist. Having the previous state to automatically go into an array through object.Assign/other invoke methods  and maybe have them to be pure reducers/actions if you want them to based upon needs. Say you have a() -> 5 and b -> 7 and want c = a() + b(); ...  If you want to use a later, you can reference the function as Sebastian proposed and later use it in another call by doing it from the array of yields. Sorry for explaining this very. Badly and with no code highlighting. 4 am and writing this at my iPad, will clarify tomorrow if needed

Not sure what you mean, code would definitely help :smile: . I’ll try to put up some examples too when I find some time.

Agreed on the awkwardness of composing Effects in middle reducers. The Elm compiler plays a huge role in making it doable with elm-effects. We've already begun to experiment with algebraic effects at Raise by co-opting generators to yield effects or spawn() other generators. I think it's about as close as we've been able to get. I really like @sebmarkbage's point about not needing three different execution style notations for async vs generator vs plain, and we've approximated that by just saying everything is a generator because anything might want to yield an effect at some point. We've been successful so far using it for request handlers in Node, but I'm not really sure how it would look when combined with redux-loop. I think this might be more of an evolution from redux-loop than a combination with it, what do y'all think?

Interesting! So would there be a new concept of effect handlers?
Related reading: http://blog.paralleluniverse.co/2015/08/07/scoped-continuations/
Redux-loop seems to basically be the monadic idea. A reducer in that sense is the argument to flatMap. The delimited continuation model seems similar to that author's scoped continuations.
One thing that seems weird is the use of continuations. Is it too much power to have values injected back into the continuation? Maybe it's just grating against my Redux guide training, but suddenly it seems like reducers are going to get a lot more complicated, with fetching happening in-line. But I suppose that complexity has to live somewhere.
Another thought: with monadic style, you can test each phase independently, but you kind of lose that ability if you have one long thread of control.
I haven't ever tried to compose separate Redux-based components, so my thoughts are all from the perspective of a monolith. More thoughts on how this would make Redux apps more composable would be helpful, to me at least.

Hasn't anyone considered using something such as co to simulate an "effect like" operation? It looks like it could solve the current void

I use co, but I'm not sure what you mean. It's basically just an option for getting async/await with generators. From the use I'm familiar with, it's not so much an effect executor as a way to work with promises.  In other words, you want to be able to yield some value that an effect handler will use to trigger a fetch, and inject the returned value back. But with co, you'd execute the fetch yourself and yield the resulting promise to resolve it. That's not decoupled in the same way.

^ Totally. Promises shouldn't show up anywhere in the reducer since they necessarily represent the execution of an async function. Doesn't always mean a side-effect occurred, but it can and usually does and that's what we're here to avoid.

While co is designed to handle promises there should be a way to do the
handling of synchronous, pure effect spawning within reducers using a
similar approach
On Mar 17, 2016 11:45 PM, "Luke Westby"<EMAIL_ADDRESS>wrote:

^ Totally. Promises shouldn't show up anywhere in the reducer since they
necessarily represent the execution of an async function. Doesn't always
mean a side-effect occurred, but it can and usually does and that's what
we're here to avoid.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/reactjs/redux/issues/1528#issuecomment-198200184

We can experiment with the syntax like Elm provides. Reducer could return not only the next state but also an effect. This effect could be represented as IO monad for example.
function reducer (state, action) {
  if (actionRequiresEffect(action)) {
    return [nextState, IO.of(function () {
      // here is the effect code
    })];
  }
}

The throwing effects idea looks promising, but I have some concerns (even if such a mechanism eventually found its way into ES):

Maybe the solution should not use the return channel in Sebastian's proposal (injecting values back to the reducer, even asynchronously), since IMHO that would make reducers impure or at least more difficult to reason about. The store should immediately call the continuation with no arguments whenever it catches an effect, and then process the effect asynchronously.

If effect handling would be built into core Redux, it would probably make it much more complex, having to handle execution of many types of effects (sync, async, parallel, serial, cancellable, etc.), just as they are now processed by redux-saga. On the other hand, if this would be left to a store enhancer we might end up with deferred continuations and more complex reducers.

I implemented something quite similar to this idea. This was something that I had to come up with to solve the redux-elm-challenge.
You can look at my solution here.
https://github.com/slorber/scalable-frontend-with-elm-or-redux/tree/master/localprovider-redux-saga-ganaraj
import React, {createElement, Component} from 'react';
import { createStore, applyMiddleware } from 'redux';

const localState = (component, localReducer, mws=[]) => {

  class LocalProvider extends Component {
    constructor(props, context) {
        super(props, context);
        this.parentStore = context.store;
        this.listener = this.listener.bind(this);
        this.localStore = createStore(localReducer,
            applyMiddleware.apply(null, [this.listener, ...mws]));
        this.overrideGetState();
    }

    overrideGetState(){
        const localGetState = this.localStore.getState;
        this.localStore.getState = () => ({
                ...this.parentStore.getState(),
                local: localGetState()
            });
    }

    getChildContext() {
        return { store: this.localStore };
    }

    listener() {
        return (next) => (action) => {
            let returnValue = next(action);
            this.parentStore.dispatch(action);
            return returnValue;
        };
    };

    render() {
        return createElement(component, this.props);
    }
  }
  LocalProvider.contextTypes = {store: React.PropTypes.object};
  LocalProvider.childContextTypes = {store: React.PropTypes.object};
  return LocalProvider;
};

export default localState;

The core of the idea is that the Provider given by React-redux is not something that has to be a single entity up the component chain. You could technically have multiple child stores down the chain but you could handle the actions they emit anywhere up the chain. This is an idea quite similar to Angular 2's child injectors.
Currently this has the disadvantage that its not possible to serialize the child/ local stores. But this is something that is easy to handle if we had some support from react itself, if it exposed the _rootNodeId OR if we had a library that provided the path to the current node from the root node. I believe this is a relatively easier problem to solve eventually. But it would be quite interesting to know what others think about a solution like this.

We also realized that the lack of fractability in Redux is a bit limiting for us to build really scalable and complex application.
About a half year ago we were experimenting with changing the shape of reduction into Pair<AppState,List<Effects> which is basically the same like redux-loop is doing and came to exactly the same conclusion like @gaearon. It's simply too difficult to mix effectful and effectless reducers and composition becomes very clumsy.
Frankly, sometimes it's quite clumsy even in Elm because the programmer is still responsible for unwrapping the Pair manually and passing everything appropriately to sub-updaters.
Therefore we've built https://github.com/salsita/redux-side-effects which is at least "somehow" trying to simulate the Continuations with Effect handlers approach. The idea is very simple, if reducer was generator then you could simply yield side effects and return mutated app state. So the only condition is to turn all your reducers into function* and use yield* whenever you want to compose them.
function* subReducer(appState = 0, { type } ) {
  switch (type) {
    case 'INCREMENT':
       yield () => console.log('Incremented');

       return appState + 1;
    case 'DECREMENT':
       yield () => console.log('Decremented');

       return appState - 1;
    default:
       return appState;
  }
}

function* reducer(appState, action) {
  // Composition is very easy
  return yield* subReducer(appState, action);
}

Reducers remain pure because Effects execution is deferred and their testing is a breeze.
And because I am also author of redux-elm I tried to combine those two approaches together and the result? I've ported the Elm architecture examples into Redux and was really surprised how nicely it works, there's no even need for unwrapping the reduction manually
The only drawback so far is that yield* is not automatically propagated in callbacks, therefore if you want to map over some collection and yield side effects inside the mapper function then you can't. The workaround is to map over the collection and yield list of effects. Or use generator "friendly" version of map.
We've been using this approach in production for almost a half year now. Seems like the perfect fit for us in terms of highly scalable architecture for non-trivial applications even in larger teams.

@gaearon the problem you'd like to solve seems quite similar to the one here: scalable-frontend-with-elm-or-redux
The problem is not clearly solved yet weither it's Elm or Redux..., and even Elm fractability does not help that much in my opinion.
Fractal architectures often follow your DOM tree structure. So fractal architecture seems perfect to handle view state. But for which reason exactly the "effects" should also follow that structure? Why should a branch of your tree yield effects in the first place?
I really start to think it's not the responsability of a branch of your view tree to decide to fetch something. That branch should only declare what has happened inside it and the fetch should happen from the outside.
Clearly I don't like at all the idea of view reducers yielding effects. View reducers should remain pure functions that have a single responsability to compute view state.

@gaearon If I understand what you're trying to solve is the boilerplate involved when adopting an Elm approach to side Effects. I'm not yet familiar with delimited continuations but from @sebmarkbage proposal what you're proposing could be something like this
function A(state, action) {
  ... do some stuff
  perform someEffectA
  ... continue my stuff
  return newState;
}

function A(state, action) {
  ... do some stuff
  perform someEffectB
  ... continue my stuff
  return newState;
}

rootReducer = combineReducers({ A, B })

and then somewhere the store do something like
do try {
  let nextState = rootReducer(prevState, action)
  prevState = nextState
  runAllEffects()
}
catch effect -> [someEffectA, continuation] {
  scheduleEffect(someEffectA)
  continuation() // resume reducer A immediately
  /*
    after reducer A finishes, continue to reducer B (?)
    What happens if reducer B throws here
    Or perhaps the do try/catch effect should be implemented
    inside combineReducers
  */
}

catch effect -> [someEffectB, continuation] {
  scheduleEffect(someEffectB)
  continuation() // resume reducer B immediately
}

Maybe I missed something with the above example. But I think there are some issues which still has to be answered

After catching some effect, the store has to schedule the effect for the execution later, because the reducers should be resumed immediately (reducer has still to be synchronous). Then there will be still the issue of managing control flow (long running transactions) using the state machine approach

How would it work if a parent reducer should (say a reducer for a router component since we aim for fractability here) catch effects from reducers of nested components. Does it have to replicate the Store catch Effect/invoke the continuation mechanism ? And if yes, wouldn't the flow of thing be a little complicated to reason about here (where the continuation frame begins and where it ends when we're in the middle of a reducer chain)

And there are also the issues related to Component decoupling raised by @slorber in his Elm challenge,

From a more conceptual POV, AFAIK continuations (call/cc) are about providing a pretty low level way of decomposing/manipulating the flow of a program. I haven't looked in detail into delimited continuations but I guess they provide more flexibility by making possible to capture the continuation from 2 sides (having the continuation return a value). This gives far more power but also would make the flow pretty hard to reason about. My point is that for what you're aiming to achieve, it seems to me that you're using a too much heavy weapon :)
I agree that redux lacks fractability, which Elm achieves by having a different way of decomposing/recomposing things (Model/Action/Update) But IMO the Elm way inevitably introduces the boilerplate of passing things down/up. And with the Effect approach it creates even more boilerplate because Elm reducers (Update) have to unwrap/rewrap all intermediate values (State, ...Effects) that bubbles up the Component tree, without mentioning the wrapping of the dispatch functions when the action tunnels down. (So I'm not sure how Elm language provides an advantage here over redux-loop besides type checking, because the boilerplate seems inherent to the approach itself).
IMO fracttability could be achieved by taking the Store itself as a Unit of Composition. Dont know exactly how this would be concretely but I think this would make Redux apps composable without being opinionated on a specific approach for Side Effects and Control Flow. This way the Store has to worry only about actions and nothing elses

@gaearon the problem you'd like to solve seems quite similar to the one here: scalable-frontend-with-elm-or-redux

Your point is certainly correct, that's definitely a drawback of fractable architecture and there still are some techniques to solve that, like for example Action pattern matching, which as you may protest breaks encapsulation. On the other hand, from my experience this is a quite rare use case because mostly you only need direct parent <-> child communication which can be accomplished in very elegant way using The Elm Architecture.

I really start to think it's not the responsability of a branch of your view tree to decide to fetch something. That branch should only declare what has happened inside it and the fetch should happen from the outside.

Why to put such a constraint to only solve fetch? It's about side effect in general, while fetch may potentially not be tied to the Component, there certainly are different side effects which are Component specific (DOM side effects)

Clearly I don't like at all the idea of view reducers yielding effects. View reducers should remain pure functions that have a single responsability to compute view state.

I disagree, imagine you have a list of items which you may somehow sort using Drag n Drop. Your reducer is responsible for deriving the new application state, therefore the reducer is authoritative entity to define the logic. Persisting sort order is just a side effect of the fact!
In other words, saying that Event log is a single source of truth is utopia because you still need derived state to perform side effects anyway.

@yelouafi

Then there will be still the issue of managing control flow (long running transactions) using the state machine approach

Yes, the Elm Architecture is missing that piece and that's exactly where I believe Saga is very useful and needed. On the other hand using redux-saga or redux-saga-rxjs for side effects is just a side effect of the pattern itself, it solves long running transactions and Request <-> Response is long running transaction. We could say as well that Sagas should be pure and we could combine those two approaches, which IMO seems most reasonable to me.
People misunderstood concept and usefulness of Sagas
const dndTransaction = input$ => input$
  .filter(action => action.type === Actions.BeginDrag)
  .flatMap(action => {
    return Observable.merge(
      Observable.of({ type: Actions.Toggled, expanded: false }),
      input$
        .filter(action => action.type === Actions.EndDrag)
        .take(1)
        .map(() => ({ type: Actions.Toggled, expanded: true }))
    );
  });

This is still a very useful Saga for solving long running transaction, yet it's totally effectless.

As I said on twitter, you don't need continuations here. Continuations are really powerful, and are extremely useful when you need to control the execution flow. But reducers are restricted and we know exactly how they act: they are fully synchronous, always. Sebastian's proposal is great but it's really just a more powerful generator: instead of being a shallow yield, it's a full deep yield.
It's neat because you can control the execution and do stuff like this. Here we don't even call the continuation! But we don't need this with reducers.
try {
  otherFunction();
} catch effect -> [{ x, y }, continuation] {
  if(x < 5) {
    continuation(x + y);
  }
  console.log('aborted');
}

While we could still force reducers to be synchronous (like we can with generators), there is a still a debugging cost. This is probably the main reason TC39 has been against deep yields so far: they are harder to reason about, and debugging can get painful as things jump around (they also like the sync/async interface dichotomy). But we're not going to see this native any time soon, so your only hope is compilation, but compiling this sort of stuff involves implementing true first-class continuations completely and the generated code is really complex (and slow-ish).
I think it goes against the philosophy of reducers as just simple synchronous functions. You realize that with his proposal you will have to always call reducers with a try/catch effect, right? They are no longer simple functions that can be called. This is because perform only works inside the context of a try/catch effect, just like yield only works inside of function*. So in your tests calling a reducer would always be:
try {
  let state = reducer(state, action);
} catch effect -> [effect, continuation] {
  continuation();
}

Even if the reducer has no effects.
Let me show you what I was talking about on twitter. I'm not saying this is a good idea (it may be) but if want to "throw" effects inside reducers and "catch" them outside, all while retaining the existing interface, here's what you do.
You can do this with dynamic variables. JS doesn't have them, but in simple cases they can be emulated.
First you create 2 new public methods that reducer modules can import:
// Public methods that are statically imported

let currentEffectHandler = null;
function runEffect(effect) {
  if(currentEffectHandler) {
    currentEffectHandler(effect);
  }
}

function catchEffects(fn, handler) {
  let lastHandler = currentEffectHandler;
  currentEffectHandler = handler;
  let val = fn();
  currentEffectHandler = lastHandler;
  return val;
}

We're basically making runEffect a dynamic variable. catchEffects may override this variable and run functions in different contexts. We're leveraging the new JS stack to create a stack of effect handlers.
Now let's create a sample store. This does what @gaearon initially described: just pushes effects onto an array. The top-level uses catchEffects itself to gather up effects across the entire call.
let store = {
  dispatch: function(action) {
    let effects = [];
    let val = catchEffects(() => reducer1({}, action), effect => {
      effects.push(effect);
    });
    console.log('dispatched returned', val);
    console.log('dispatched effected', effects);
  }
}

Now some reducers:
// Sample reducers. These would do:
// `import { runEffect, catchEffect } from redux`

function reducer2(state, action) {
  if(action.x > 3) {
    runEffect('bar1');
    runEffect('bar2');
    runEffect('bar3');
  }
  return { y: action.x * 2 };
}

function reducer1(state, action) {
  runEffect('foo');
  return { x: action.x,
           sub1: reducer2(state, action) };
}

Both reducers initiate some effects. Here's the output of store.dispatch({ type: 'FOO', x: 1 });:
dispatched returned { x: 1, sub1: { y: 2 }, sub2: { y: 2 } }
dispatched effected [ 'foo' ]

And here's the output of store.dispatch({ type: 'FOO', x: 5 });:
dispatched returned { x: 5, sub1: { y: 10 } }
dispatched effected [ 'foo', 'bar1', 'bar2', 'bar3' ]

Reducers themselves can use catchEffects to suppress effects:
function reducer1(state, action) {
  runEffect('foo');
  return { x: action.x,
           sub1: reducer2(state, action),
           sub2: catchEffects(() => reducer2(state, action),
                              effect => console.log('caught', effect))};
}

Now the output of store.dispatch({ type: 'FOO', x: 5 });:
caught bar1
caught bar2
caught bar3
dispatched returned { x: 5, sub1: { y: 10 } }
dispatched effected [ 'foo' ]

another way of doing could be something like the Context concept of React. If we suppose that reducers are always non-method functions we can leverage the apply function to call them within a provided context. Throwing an effect would be equivalent to invoking a this.perform(...) where this is the current context.
function child(state, action) {
  //... do some stuff
  this.perform(someEffect)
 // ... continue
 return newState
}

function parent(state, action) {
  //... do some stuff
  this.perform(someEffect)
  return {
    myState: ...
    childState: child.apply(this, state.childState,action)
  }
}

Similarly, catching effects from parent reducers could be done by having parents override the context passed to children
function parent(state, action) {
   const context = this
  //... do some stuff
  const childContext = context.override()
  const childState = child.apply(childContext, state.childState, action)
  if(someCondition) {
    context.perform(childContext.effects)
  }
  return {
    myState: ...
    childState
  }
}

We can also test the reducers simply by passing them a mock context then inspecting the mock context after they return

@yelouafi Kind of exactly what I am doing in LocalProvider - see a few comments above.

Yes, React's context is similar to dynamic variables. That also changes the signature of reducers though, you can't just call them as normal functions.

It would helpful for me at least for API ideas to include both usage and testing code examples. My guess from experience and intuition is that any test for a generator-based solution would look a lot like a test for a saga. What would tests look like using either the dynamic variable approach or the context approach?

@lukewestby
// Test that a reducer returns the right state (literally no difference)
let state2 = reducer(state1, action);
assertEquals(state2.x, 5);

// If you want the effects, use `catchEffects` to get them
let effects = [];
catchEffects(() => reducer(state1, action), effect => effects.push(effect));
assertEquals(effects.length, 3);

Honestly this came out better than expected, I don't know how you could get much simpler and keep the existing interface 100%.

@yelouafi Also, your solution doesn't work across arbitrary frames. If a reducer calls out to any helper methods the this context is lost. I'd like to see this go away from JS, and I don't think we should use it.

@lukewestby yes:
function* reducer(appState = 0,  { type }) {
  switch (type) {
  case 'INCREMENT':
    yield effect(Logger.log, 'Incremented');
    return appState + 1;

  case 'DECREMENT':
    yield effect(Logger.log, 'Decremented');
    return appState - 1;

  default:
    return appState;
  }
}

it('should increment value and log', () => {
  const iterable = reducer(42, { type: 'INCREMENT' });

  assert.deepEqual(iterable.next().value, effect(Logger.log, 'Incremented'));
  assert.equal(iterable.next().value, 43);
});

it('should decrement value and log', () => {
  const iterable = reducer(42, { type: 'DECREMENT' });

  assert.deepEqual(iterable.next().value, effect(Logger.log, 'Decremented'));
  assert.equal(iterable.next().value, 41);
});

@jlongster Great outline of a solution. I've been considering a similar architecture for allowing React components to "yield up" other data besides VDOM.
It seems like in all these cases, we're basically looking for a writer monad -- (well, if you discount the proposed ability to inject data back into your scope). The basic concept is that you have a regular function that returns some core data, but you also want it to return some auxiliary data too. As you compose many of these functions, you provide the way in which pieces of core data should be combined to form a new unitary piece of core data, and the auxiliary data is simply concatenated as a collection. Some top-level machinery decides what to do with your single piece of core data and collection of auxiliary data.
It's funny, because these variations are the same things I considered when writing react-redux-controller. There are basically 3 solutions pushing/pulling auxiliary data from a scope: monads (wrappers for your data), yield, and this. There are also closures, as beautifully illustrated here, but they are restricted to lexical scope. Provided that's a limitation you can live with, it's also a valid way to go. I think the downside there is that now you have to hack the module system if you want to test.

@jlongster

Yes, React's context is similar to dynamic variables. That changes the signature of reducers though, you can't just call them as normal functions.

You're right, you'll have to always use the method form because a parent can't make assumptions about a child reducer (if it needs this or not)

Also, your solution doesn't work across arbitrary frames. If a reducer calls out to any helper methods the this context is lost.

Again it's true.
But anyway I'm not really fond of melting down code that computes the view state with the code that drives the operations. I think it's good to keep the 2 totally separated.
If I wanted a pure/memory-less/state-machine approach for control flow (e.g. to have time travel and hot reload for my control flow) i'd consider having a separate reducer tree for effects (And by reducer tree, I don't mean a tree of states, i.e. all effect reducers should get the top state of the locally scoped component).

I really like @jlongster's solution so far. I'm going to see if I can implement it as a branch of redux-loop and report back with findings!

Seems to work really well after a first pass https://github.com/raisemarketplace/redux-loop/blob/deep_effect_collection/test/index.test.js#L35-L47

@lukewestby: Very cool! Excited to see if it works out. I agreed with @yelouafi that I generally like keeping side effects outside of reducers, however this could prove useful in certain awkward situations. I've been meaning to study redux-sagas so I wonder how this whole thing can fit together.
I'll be keeping an eye on your experiements in redux-loop!
Note that my code was very much a prototype. It might need a few tweaks, for example a finally around the fn call in catchEffects:
try {
  let val = fn();
} finally {
  currentEffectHandler = lastHandler
}

Additionally, if you want to support re-"throwing" effects in a handler, you'll need to keep a stack of handlers instead of using the JS stack. That way you can run currentEffectHandler in the context of the last handler, instead of itself.

This is not a complete solution, but has anyone considered the fact that you can put reducers, middleware, and React components inside the Redux state?  It may not be possible to standardize, because some people use POJOs and some use Immutables, but it definitely allows you to compose multiple apps together if they follow the same standard.  That's what I did to implement a feature-oriented plugin system in my app; here's what the state looks like (though it's actually a big Immutable tree):
{
  plugins: {
    plugin1: {
      reducer: (state, action) => { ... }
      middleware: store => next => action => { ... },
      components: {
         ReactComponent1: ...
         ReactComponent2: ...
      }
   },
   plugin2: {
      reducer: (state, action) => { ... }
      middleware: store => next => action => { ... },
      components: {
         ReactComponent1: ...
         ReactComponent2: ...
      }
   }
}

I have a pluginReducer that composes the plugins' reducers using reselect and applies them to the input state, and a pluginMiddleware that composes the plugins' middleware, also using reselect.
I also made my pluginMiddleware support a LOAD_PLUGIN action that calls an async load method on the plugin with the requested key.  The load method is where I put my Webpack code splitting.  I've been extremely happy with this so far.

@gaearon just so i understand, could you elaborate on why a solution that requires a change to combineReducers wouldn't be allowed? i'm discussing this issue with some other folks and i want to be sure that my explanation is accurate.

@jlongster

I've been meaning to study redux-sagas so I wonder how this whole thing can fit together.

Would be great to read your thoughts on it. I know you've been experimenting with csp so I'm really interested in your opinion (and critics also) on the redux-saga model

@yelouafi my initial impression is it looks awesome :) I'm very attracted to that style of API! I don't know how you represent effects, but the put/take API looks awesome. (sorry if this thread is getting derailed)

(@gaearon sorry too for this deviation)
@jlongster happy you like it :). Effects are plain objects created using functions like take, put, call... the middleware 'executes' the effects & the tests just check the yielded effects are correct

While I'm very excited for this discussion, I'm sorry that I have to admit that I think this is a very terrible idea.
Before telling you why, let me embrace the concept just a little bit.
This reminds me a lot of Meteor's Tracker. I made a presentation and a simplified implementation for people to learn and understand. You could definitely use this same technique to encapsulate computations and access some larger context as you propose.
If you've ever use Meteor, and particularly, Blaze, you'll see that this leads to an amazingly magical programming experience. But the problem with tracker, specifically is in a larger applications, is it becomes very challenging to keep track of whats going on, in what order, and what the performance implications are for any line of code -- something that Redux does a great job at disentangling.

My main objection to this approach is that your functions are no longer pure and side-effect free. A better approach would be to use something like a Task to encapsulate an action. Now you can use the full algebraic powers of map, concat, chain, sequnce, and liftA2 to compose side-effects in the same way as you combine actions.
For example, in order to prevent action collisions in redux, you need to pass the dispatch function down to the children, and wrap the actions in a context.
items.map((item, i) => {
  const
   = (action) => dispatch({type: 'child_action', idx: i, action})
  return <Item data={item} dispatch={childDispatch}/>
})

All that we're really doing here is mapping over a function. We can do this because a function is essentially a funtor if you think of it has holding an eventual value.
# R.map is from Ramda.js
const toChildAction = (action) => ({type: 'child_action', idx: i, action})
const childDispatch = R.map(toChildAction, dispatch)

You can do the same thing with Tasks as well! Suppose you have a task thats going to fetch some data and return an action.
const getFriendsTask = new Task((rej, res) => {
  getFriends()
    .then((friends) => res({type: 'set_friends', friends}))
    .catch((error) => res({type: 'get_friends_error', error}))
})

The parent component now needs to deal with this task, just as with actions, and wrap it up in some context so the action makes it back to the proper reducer. Well, you can map over it just the same!
const childGetFriendsTask = R.map(toChildAction, getFriendsTask)

Now, this may be a little too much for Redux -- I understand that Redux tries to be a simplification of all this functional programming power. But I think under the hood, you'll be a lot better off keeping everything pure, as it currently is, and leveraging monads and functors to do all the heavy lifting.

I don't think this proposal is exclusive of using something Task-like to represent the effect. What about the current discussion prevents "throwing" or enqueuing a Task to be run later by the imperative shell which captures it?

It seems, by the way, that the concepts we're discussing right now are not completely dissimilar from Ember's runloop https://github.com/eoinkelly/ember-runloop-handbook

@lukewestby when you throw an error, that is a side-effect. Rather than throw/catch its best to capture the behavior of a program using a proper data structure. For example, here's what you get when you're using throw/try/catch:
function doSomething() {
  // ...
  if (someProblem) {
    throw new Error('there was some issue')
  } else {
    return result
  }
}

try {
  const result = doSomething()
  const formattedResult = formatResult(result)
  // ...
} catch(e) {
  if (error.message == 'some kind of error') {
    // display a certain message
  } else {
    // if we dont know this error, then dont eat it
    throw e
  }
}

What I (and funtional programming in general) would argue is that you're just not using the right data structure to encapsulate the program. If doSomething could possibly fail, then you need encode that in the data structure that's returned. The Either monad is a great example for this.
function doSomething() {
  // ...
  if (someProblem) {
    return Either.Left.of('there was some issue')
  } else {
    return Either.Right.of(result)
  }
}

const result = doSomething()
// map only applies to Either.Right, but Either.Left just returns itself
const formattedResult = result.map(formatResult)

if (formattedResult.isLeft) {
  // handle the error cases
} else {
  // handle the formattedResult.value
}

Notice in the second example doSomething is pure. There are no side-effects like throw which magically have effects outside of the function. Instead, you encode the fact that this function can have an error by using a proper data structure.
When I was referring to using a Task earlier, I meant it in this sort of way. Using a Task and returning a Task from a function allows the function to remain pure. Thus we can test it easier, and it makes the entire thing easier to reason about. But as soon as we start instroducing side-effects into our functions we lose all the guarantees and benefit that purity gives us...

I'm completely onboard with a solution which can leave everything to the return value as long as it isn't too awkward the way that redux-loop can be, but I have to dispute that Tasks are easier to test because they are opaque. How would I call a reducer that returns a Task and assert that the resulting Task will perform the effect I expect?

Maybe just reducers the ability to dispatch?  That way they can act like normal, and apply future action?  You could theoretically intercept that to control timing, or to route to discreet applications.
https://gist.github.com/robcolburn/3f537a182931f0b297ac

@lukewestby as with all side-effects, you'll have to mock out the side-effect. For example, you could mock out window.fetch to return some value and ensure that when you fork the Task, it returns the expected action.

@ccorcos

Rather than throw/catch its best to capture the behavior of a program using a proper data structure.

That's debatable. If you check out a link I shared earlier, the author makes some great points of the downsides of explicitly using monads, functors, and the like. I'm not entirely sold, but I think it brings up some good points.
I'm not sure that throwing effects necessarily makes a reducer impure, if the standard is that they shouldn't re-inject data into a reducer. But if they're simply resumed, that seems pure to me.

In all the talk about alternatives, I think one option is being left out, which is to represent effects statically as state, making effect enactors regular store subscriber/dispatchers.
So there might be a fetches key in the store, with an an array of representations of what should be fetched. Reducers for actions that should result in fetches merge their requests into this array. A fetcher subscribes to the store, monitors for new fetches, and dispatches success and error actions.
This was the discussion from #1182. It's quite simple, with no middleware, no continuations, no generators, and no impurities. @yelouafi made the point, however, that this requires bundling up action info into these effect descriptions, whereas redux-saga simply listens for the actions directly and makes direct use of their payloads.
But again, I'm still not clear on how the desire for fractal/composable architecture plays into the question of sagas vs. monads vs. continuations vs. static descriptions vs. whatever else for effects.

Dan,
perhaps you would consider taking a deeper look at the SAM pattern? Let's forget about State-Action-Model for a second and just consider the Praxos protocol where state alignment is achieved with three roles: Proposers (Actions in SAM), Acceptors (Model in SAM) and Learners (State function in SAM), what you realize is that Redux, with its choice of making the actions and model anemic and concentrating all the business logic (to propose, accept and learn) in the reducer has created that situation. How could you ever think about composing reducers then?
The Trilogy Redux, Thunks and Sagas should give you a hint that indeed there is somewhat of a problem in the factoring of the reducer and it is what needs to be revisited.
I believe this suggestion is aligned with @acjay "It's quite simple, with no middleware, no continuations, no generators, and no impurities."

I agree with @acjay and @ccorcos on this (and consider throwing errors being impure).
What we look for here is the ability to have deeply nested reducers functions to return their value like usual, AND performing effects. We want reducers to remain pure and testable like before (including the effect part).
The writer monad was done exactly for that. Instead of returning a normal value, the reducer would then return a data structure like { result: result, effects; [{...},{...}] }, and the parent reducer will propagate automatically that structure to the top level. At the top level (middleware) you receive effect descriptors, and you are free to execute them the way you want to (a bit like what redux-saga does)
@ccorcos I prefer the writer monad compared to tasks because for me it looks more testable (but both solutions are quite similar anyway)?
So yes, with this solution, you have to make all intermediate reducers aware of this writer monad infracture (including combineReducers), but is it such a big deal? Deeply nested yields look very dangerous to me and can become a huge mess. I like things being explicit, even if it involves more boilerplate. This kind of feature does not really go in the direction the community loved Redux in the first place, IMHO.
Imagine you have Reducer1 > Reducer2 > Reducer3 and reducer3 yield deep effects. Looking at Reducer1 and Reducer2, you have absolutly no idea of this effect yielding, because nothing in the code makes these effects explicit. This is bad for me :(
Yes monads are hard, yes redux-loop requires boilerplate (and does not look monadic), but at least things are explicit, and by looking at a reducer you know that you can yield effect. I don't think using a fancy syntax in order to avoid modifying combineReducers will help in the first place. This kind of feature may sound convenient right now but will likely be disliked the same way as this is now disliked IMHO.
Side note: I think the Redux-loop project aims to do too much thing. For me Redux-loop and Redux-saga are complimentary and it should not be the responsability of Redux-loop to handle ajax requests: I think its role should only be to translate reusable component actions to app-specific actions, and act as a fractal way to translate local interactions to meaningful business events,

It is also possible to modify composeReducers such that it is still backward compatible and doesn't require every reducer to be updated to return effects.
In the branch I created here https://github.com/reactjs/redux/compare/master...kurtharriger:lift-effects reducers could return a new state withEffects or just return the new state.  There are probably many other ways to do this as well.

@kurtharriger that's true but then the problem would become to find a convention for the data structure to return... it's easy to implement your own combineReducers yourself and no need to rely on Redux one in the first place, so it's not a big deal if Redux does not provide it directly...

@jlongster @sebmarkbage @gaearon I guess you don't like the this based proposal of @yelouafi. However for me the "deep yield" or "dynamic variables" solutions have exactly the same drawbacks as this: when using deep yield, the component that yields assume that it should be used in a given context. By expecting that you implicitly couple the reducer to how it is supposed to be called. For me it really looks like a new fancy this alternative.

The writer monad was done exactly for that. Instead of returning a normal value, the reducer would then return a data structure like { result: result, effects; [{...},{...}] }, and the parent reducer will propagate automatically that structure to the top level. At the top level (middleware) you receive effect descriptors, and you are free to execute them the way you want to (a bit like what redux-saga does)
@ccorcos I prefer the writer monad compared to tasks because for me it looks more testable (but both solutions are quite similar anyway)?

@slorber, you'd love om.next, it does exactly as you describe here. pseudo js version -
class App extends Component {
  static query = () => ql`[count]`
  onClick = () => this.props.transact({ type: 'tick' })
  render() {
    return <div onClick={this.onClick}>
      clicked { this.props.count } times
    </div>
  }
}

function read(env, key /*, params */) {
  return {
    value: env.get()[key]
  }
}

function mutate(env, action){
  if(action.type === 'tick'){
    return {
      value: {
        keys: ['count']
      },
      effect: () => env.store.swap(({ count }) =>
        ({ count: count + 1 }))
    }
  }
}

application({ read, mutate }).add(App, window.app)

(and a lot of other things).
further, it's a fully sync model(!), and all the async stuff is pushed into a send function, where you're only allowed to merge into the store. It looks kinda like this
function send({ search }, cb) {
  for(let expr of search) {
    let { key, params } = expr
    if(key === 'results') {
      searchWiki(params.query,
        (err, res) => res && cb(res))
        // you can call cb as many times as you'd like
    }
  }
}

These are interesting constraints to have, and I'm having fun exploring them.

To address the OP @gaearon , this architecture is interesting in the original context -

data fetching concerns are mostly invisible, and components receive data in a shape they ask for, despite maybe having a different shape in the store/source of truth. This relieves pressure on writing reducers/mapStateToProps. data fetching is fairly simple/automated in this model.
mutations also share the same attribute, modifying the source of truth in a specific manner, again relieving pressure on 'reducers'(ie - the mutate fn).
further, I'm not certain what the composition store here is exactly, but I'd guess you get to compose read / mutate functions, by mapping their keys to sub keys in a combined model. This way, you can create a 'tree of apps' as it were, catching/yielding data at every level/key you need. This doesn't solve 'deep yields', but it does make it simpler to compose apps.

@slorber

Side note: I think the Redux-loop project aims to do too much thing. For me Redux-loop and Redux-saga are complimentary and it should not be the responsability of Redux-loop to handle ajax requests: I think its role should only be to translate reusable component actions to app-specific actions by returning effects. Then sagas can handle these app-specific actions and do the required job. I don't see the point of deciding what effect to execute directly in the state branch, and it's even worse if you need access to state from another branch while yielding the effect (as your app grows, all your effects will likely be more and more moved close to the root reducer, and state shape refactoring would become more complex...)

I wouldn't necessarily agree. It does make sense to use saga when LLT is needed and by LLT I mean something which requires intermediate state to manage the transaction (onboarding...). However, I don't think that Sagas should be effect executors. As you said, it would make sense to declaratively yield Effects in reducers (using either redux-loop or redux-side-effects, it's just different syntax) but why would these effect executors live in Sagas? Saga is pattern for managing LLT, it has nothing to do with side effects.
Because as I have already explained, above:

Imagine you have a list of items which you may somehow sort using Drag n Drop. Your reducer is responsible for deriving the new application state, therefore the reducer is authoritative entity to define the logic. Persisting sort order is just a side effect of the fact!

I found doing these kind of things nearly impossible using redux-saga or redux-saga-rxjs, sometimes Saga simply needs to be parametrised by application state and in that case, I don't really see any benefit of using Sagas which ideally should not be aware of the application state.

@acjay about the link you posted here: http://blog.paralleluniverse.co/2015/08/07/scoped-continuations/
I can also understand the points of the author, but still think it can create a mess. Also the author propose a type-safe implementation by using something very similar to checked exceptions: void foo() suspends A, B so you have to declare in which context your function can be executed.
If you have ever done any Java you can notice that checked exceptions, while looking like a good idea in the first place, were a bad idea that created a lot of beginners just wrapping things in try/catch everywhere C# decided to not ship checked exceptions for these reasons btw.
And I think comparing this to exceptions (weither checked or unchecked) should just warn us. Have to ever find it practical to handle complex control flow in Java or Javascript? I don't remember anything that permits the control flow to jump on the stack having proved to be a good idea.

@threepointone I've taken a look at Om.Next but as I'm not really fluent in Clojure yet and the doc not really good enough I think I'll take a look when it becomes released. But hope to see your experiments in plain JS about that soon :)

@tomkis1 sorry I don't really understand your DnD example. Why would the reducer be responsible to execute the persist effect? For me your reducer should just describe that the order has changed (maybe by emitting an business action as an effect), and the something else should handle the persistance of that new order. I think your reducer should not even have to know that the order of your items is persisted, and even less about the way it is persisted (ajax/localstorage...)

For me your reducer should just describe that the order has changed

That's what I am talking about, if State is derived in reducer and API call clearly depends on the derived State, why can't we treat the persistence of the derived state as side effect of the derivation? Or maybe you can enlighten me what's the benefit of delegating that responsibility to saga? Because I can clearly see the Saga here as effect executor which IMO is not main purpose of Saga.

I'm going to play devil's advocate here and use redux to poke holes in the broader FRP approach to constructing interfaces. I have been on a Rich Hickey kick lately and am going to hammer his point of simple versus easy. I really would like people to share their thoughts and (hopefully) provide some meaningful counterpoints.
...
In the beginning everything was easy. We had state machines driven by top-down, event-based actions that afford easily-traceable control flow. So long as we stayed inside the browser and only listened to user events, this would have been a utopia. But how do we actually communicate any async behavior via ajax/websockets, animations, etc.?
I started with redux-thunk as an easy, no-nonsense way to preform async behavior at the edges of a sync application architecture. After some debate, we decided that thunks are pretty unpredictable, and lack larger control-flow semantics, making it nearly impossible to coordinate multiple components. There was no in-house, direct solution to "component A did something, sibling component B now needs to go do something too" that didn't involve tightly coupling (de-simplifying) A and B.
I found neat libraries like redux-effect and redux-saga (thanks @yelouafi). Having some familiarity with sagas from a DDD perspective, the expressiveness and control flow power of redux-saga was a no-brainer. Now we could coordinate long-running transactions and keep unrelated components decoupled.
But then reusability issues cropped up as the application grew. How do we have dynamic lists of components? How do we reuse components in different places across the app? I started looking for inspiration further afield in FRP conceptualizations like Cyclejs and various flavors of The Elm Architecture. Each one of these approaches "solved" reusability, but also introduced some complexity (I mean this in the "braided-together" sense of the word, not "difficult to understand initially").
It seems like no one has landed on a solution that allows you to build a completely independent component packaged up with its own set of actions, a reducer, and perhaps some side effects, that can be used as-is from the outside world without placing an implementation burden on the component. @slorber is also still on the hunt for a solution to this problem.
...
If I take a step back and ponder the whole predicament: what am I trying to accomplish? I want to build simple (not braided together) interfaces that deliver business value. Simplicity at the minimum affords testability, extensibility and modifiability. The more testable, extensible, and modifiable an application the easier it is to satisfy business demands. An FRP approach helps in what ways?
1. It is relatively easy to keep state in sync, but is it simple?
Using reducers and "data as a first class citizen" is drastically easier and simpler than a lot of other techniques such as object encapsulation, getters/setters, ORMs, etc., but is it as simple as it could be? It's very easy to fall into a "event and handler soup" pattern with FRP where parent components have intricate knowledge of child components (coupled), or child components have knowledge of events outside of their domain (unscalable), or parent components are translating one event into another event that their child can understand (tedious, error prone).
2. Replayability or "global undo" is essentially free
... until it isn't. The moment we introduce side-effects, generators, sagas, loops, or thingamabobs "perfect" time travel becomes either unachievable or places an implementation burden on every single component. How many apps truly need a global undo versus a simple local undo?
A global atom is a great idea because it can be harnessed as a front-end database, and we have been using relational databases for a very long time to build wonderfully simple things. However, I don't think that the redux flavor of reading/writing to this database is a simple as it can be.
3. What other advantages, if any?
No, seriously, I'm trying to figure out what else we are getting by trading away the aforementioned.

@aft-luke in terms of replayability, I've been playing around with this elmish concept, very similar to redux, but all side-effects are handled declaratively in the the render function. One of the best things about this approach is everything is replayable even with side-effects! And since side-effects are declarative, the app is recoverable even when you the app while the side-effects are in flight (without any hacks)! I hope you like it -- I could use some help bouncing ideas around.

or parent components are translating one event into another event that their child can understand (tedious, error prone)

I think this is how Elm succeeds with the Elm architecture insofar as it does. Translation of parent actions into child actions is still tedious, but not error prone. I don't know how to satisfactorily eliminate those errors in JavaScript

So I sat down yesterday and borrowed a lot of ideas from all around to create a new effects library which is based on the Elm Architecture, that solves some of the issues discussed above.
I think the API came out quite nicely, although the implementation is quite hacky at the moment and is tightly coupled to Redux's implementation.
You can find it here (redux-fx)
Please tell me what you think about it.
You can use it basically like this:

First you enhance the createStore with the provided enhancer:

const createStoreWithMiddleware = compose(
  applyMiddleware(someMiddleware),
  enhanceStoreWithEffects(),
  devTools()
)(createStore);

Then you export your reducer wrapped in the higher order function "withFx" and you can now write your reducers Elm-style like so (using "fx" function to create an effect descriptor):

const incrementWithDelay = seconds => dispatch => setTimeout(() => dispatch({type: "INCREMENT"}), seconds * 1000);

const reducer = withFx((state, action) => {
  switch (action.type) {
    case "INCREMENT":
      return [{count: state.count + 1}, fx(incrementWithDelay, 1), fx(incrementWithDelay, 2)];
    case "DECREMENT":
      return {count: state.count - 1};
    default:
      return state;
  }
});

What happens is basically "withFx" is putting makeup on the reducer so you can write it Elm-style but under the hood, strips it back to a normal reducer.
The effect descriptors then "bubble up" and are executed after the new state has been reduced.
The neat thing is that you can also provider your own "effects runner" and support other types of effects or other ways to execute them or not even run them at all.

@doodledood would you be interested in integrating some of the ideas you've come up with into redux-loop?

@lukewestby I thought about doing excatly that but that would actually be a complete rewrite of most of the library so I just opened a new one because it was easier

@doodledood that doesn't seem like a big deal to me, version numbers are free :smile:

@lukewestby sure go ahead :) this is why its called open source

@doodledood @lukewestby Funny thing, I'm currently experimenting with functional effects on the project I'm working on (haven't integrated Redux yet, but experimental mini-redux instances are handwritten in a few components that urge for a state management). I decided to go from scratch to check what I can come up with starting off my own requirements, and the result came naturally as I was writing these things. The code I'm going to show is very fresh, it's only been written today, not run or tested, so there could be errors.
There are several differences to redux-fx. Feedback is very welcome, thanks in advance.
As the effects' functions can't be serialized, I didn't try to make the effect descriptor serializable (the { type: "FX" } thing) -- I made it a constructed instance that can be type-checked with the instanceof operator.
		var NO_EFFECTS = [];
		function WithEffects(state, effects) {
			this.state = state;
			this.effects = effects;
		}
		function withEffects(state, effects) {
			return (effects ? new WithEffects(state, effects) : state);
		}
		function withEffectsGetState(withEffectsObj) {
			return (withEffectsObj instanceof WithEffects ? withEffectsObj.state : withEffectsObj);
		}
		function withEffectsGetEffects(withEffectsObj) {
			return (withEffectsObj instanceof WithEffects ? withEffectsObj.effects : NO_EFFECTS);
		}

I use it like follows (sorry, no Babel yet, lodashExtend should be object spread ... in my dreams):
		function reducer(state, action) {
			switch (action.type) {
				case ACTION_SOMETHING_HAPPENED:
					return withEffects(
						lodashExtend({}, state, {
							_state: STATE_SOMETHING_GOING_ON,
						}),
						[
							somethingHappenedEffect(state, {
								delay: action.delay,
							}),
						]
					);
				default:
					return withEffects(state);
			}
		}

		function somethingHappenedEffect(state, options) {
			return function (dispatch) {
				setTimeout(function () {
					dispatch({
						type: ACTION_SOMETHING_HAPPENED,
						delay: 1000,
					});
				}, options.delay);
			};
		}

And here's my mini-redux implementation:
		(function createStore(initialState, reducer) {
			var _state = initialState;

			function getState() {
				return _state;
			}

			function dispatch(action) {
				var withEffectsObj = reducer(_state, action);
				var nextState = withEffectsGetState(withEffectsObj);
				var nextEffects = withEffectsGetEffects(withEffectsObj);
				if (nextState !== _state) {
					_state = nextState;
				}
				if (nextEffects && nextEffects.length) {
					var effect;
					while ((effect = nextEffects.shift())) {
						effect(dispatch);
					}
				}
			}

			return {
				getState: getState,
				dispatch: dispatch,
			};
		}({
			_state: STATE_INITIAL,
		}, reducer);

Thanks again for any feedback.

@doodledood What do you think about: the effect captures the current state at the moment it was created (my approach), or the effect uses getState to get the state at the moment it is executed (your approach)?

@sompylasar Well, the getState I supplied the effect with is just a bonus enabling stuff like long running tasks and timers.
The idea was to capture state using args when constructing the effect descriptor and then using that to run the effect

@doodledood Yes, makes sense. I tried passing the whole store (i.e the pair of { dispatch, setState }), but then I discovered that this requires me to write var prevState = store.getState() at the effect factory or at the effect start. I don't have long-running effects in this code yet, so probably for a more generic (library) case getState should be passed to the effect instead of just the current state snapshot, or along with it.

@lukewestby @sompylasar Ok, I wanted to take this a little further - so I reimplemented redux-fx from scratch to add support for advanced use cases.
What I gained by doing this was:

getting closer to elm visually
nested reducers/action tagging support
simpler api

The funny thing is, without noticing, what I wrote was really similar to redux-loop except some implementation details and a few features, so maybe we should integrate those changes into redux-loop.
Most, if not all of the exercises on elm-architecture-tutorial can be implemented using this idea quite nicely. I have implemented something similar here .
@slorber @gaearon Can you guys take a look sometime and tell me what you think?

My first blog entry is live<EMAIL_ADDRESS>My input on managing AppState/Side Effects with Redux and Redux-Saga

Interesting, so your side-effect basically shortcut to be global.
One thing I always had in mind that now wouldnt work is if you could encode graphql queries and as an "effect" and compose them to the top level. Just an idea

@ms88privat I've started using redux-saga recently and have also seen this pattern. For example, I have a treeview that issues actions that the user does (expandClicked, nodeSelected, nodeDeleteRequested etc.). Then I have a saga that listens to those actions and issues reducer actions. I think this is applying the pattern you describe on a Widget level. Eg. if you have Widget A, it can issue actions for what the user does, then you have Saga A that watches actions and does side effects and issues actions that alter state. This pattern came naturally for the treeview but I have not formalised it in other parts of the app as I am not sure of it's drawbacks. It feels like it turns user events into commands. The obvious advantage being decoupling so that the Widgets are "dumb". On the other hand it might just be unnecessary indirection if the widgets are not re-usable outside of one specific app?

@jonaskello Thank you for your feedback. It may be unnecessary, but for me the readability is most important. Without the mapping of User-Actions to Reducer-Actions, you can only describe one side of the interaction. E.g. the Action expresses what happened, or will happen, but not both.

@ms88privat I like the pattern but I also remember reading this discussion and became a bit worried it is too imperative. Anyway, I think we are de-railing this thread. Maybe you should open a separate discussion about the user-action/reducer-action pattern on the redux-saga repo?

@ccorcos Right, but also I don't think that a graphql query description should live inside a reducer as effects. I think data needs should live inside the components themselves and only the execution part of the query should be an effect, and that can be implemented using this method

@ms88privat The way you mutate state is not something you make up, it follows a well defined pattern which is best described by the Paxos protocol: Propose, Accept and Learn. Again, this is not something that you are free to design. There could be a class of problems where you could find shortcuts, but this is not the case of Front-End architectures.
The fundamental flaw in redux is to believe that the reducer can accept anything that's presented to it. When you persist the application state (as derived from the reducer) there is a great chance that the persistence layer will respond sometimes in such a way that the proposed values (from the actions) cannot be accepted. Trying to frontload or backload your business logic in such a way that all proposed values must be accepted at all times is simply the absolute worse factoring you could think about because the logic to deal with exceptions will show up everywhere except where it belongs, in the acceptor code.
When you combine with the fact that Sagas break the principle #1 in redux (a single state tree), the question is what are you trying to achieve? Could you at least look at the theory behind state mutation and align with it? then, if there are any optimizations, I am certain really smart people will find them.

@jdubray i've read some of your stuff and it's hard to follow. I've asked you on twitter to submit a solution for scalable-frontend-with-elm-or-redux but did not get any answer nor pull request. It's hard to judge on your ideas if you don't show any code that we can run and that actually solves concrete problems.
What I can say from reading you is that I don't understand at all what can be the relationship between a quorum protocol and a frontend technology where all the code runs on the browser.

Paxos is a family of protocols for solving consensus in a network of unreliable processors

Until you introduce communication with a remote service, I don't see any network of unreliable processors in Redux apps. I don't see any mention of such communication in your post (except maybe the word "persist" which may represent that?)
If no network communication, what I understand is that there is only a single acceptor. And as it does not make sense to present the user a button if he's not allowed to click it, then generally the acceptor will just accept any action (because otherwise we wouldn't put a onclick listener on that button, or even better not show the button at all)
If network communication, it's likely that it's with a single backend (at least for API-based apps) and that the backend takes the decision to accept the state mutation. So in this case I don't see the point of  using a quorum (at least between the frontend and backend, but a cluster of backends can use quorums if needed)

Hello everyone,
I've read the entire topic and would like to propose my solution to you. It's really, really simple but I think it could be a nice addition to the stack, couldn't it?
I'm not sure if it's the correct name, but for now I called it the taskMiddleware. It's implementation is, again, nothing complex:
(reduce) => ({ dispatch, getState }) => (next) => (action) => (
  (action.meta.task) ? reduce(dispatch, action, getState()) : next(action)
)

It essentially introduce a second reducer where all our business logic will reside, it should be a pure function that cannot modify the state directly but will dispatch action accordingly.
As you can see from it's definition, it listen for any action with .meta.task. I think this is the correct way to follow for middlewares, even better if using Symbol.
Consider the Counter example with async and "if odd" action, it will be implemented like this:
import * as A from './actions'

// this is our normal, dumb, state reducer
export function stateReducer(state, action) {
  if(state === undefined)
    state = 0

  switch(action.type) {
    case A.INCREMENT:
      return state + 1

    case A.DECREMENT:
      return state - 1

    default:
      return state
  }
}

// in here will reside all of our business logic
export function taskReducer(dispatch, action, state) {
  switch(action.type) {
    case A.INCIFODD:
      if(state % 2 === 1)
        dispatch( A.increment() )
      break

    case A.INCASYNC:
      setTimeout(() => dispatch( A.increment() ), action.payload)
      break
  }
}

As you can see, it's quite easy to reason about and it's fully composable, in fact you can see this in action in a small repo I've created just today Deku-Redux Experiments.
I've realized a generic List component (with some hacks on Deku) but what it's important here is this part:
// https://github.com/Iazel/deku-redux-experiments/blob/master/list-counters/componenets/list/reducers.js#L33
const createTaskReducer = (reduce) => (dispatch, action, state) => {
  if(action.type !== A.ITEM_ACTION)
    return

  const { index, item_action } = action.payload
  reduce( listDispatch(index)(dispatch), item_action, state.get(index) )
}

Then I can use it like this:
const taskReducer = createTaskReducer(controller.taskReducer)

Done. Inside the taskReducer you can do anything you want, and even build some other abstraction upon it.
In this particular case, createTaskReducer and the List itself has no knowledge about Counter and viceversa.
Side note: listDispatch wrap dispatch to tap in the action flow of the child item.
The store creation is nothing more of the standard one:
const store = createStore(stateReducer, initialState, applyMiddleware(
  taskMiddleware(taskReducer)
))

Because taskReducer is a pure function, we pass anything it needs and because its action are the standard Flux one, it should be quite straightforward to test.
Tell me what you think about it, especially what I'm doing wrong! Thanks :)

@Iazel taskReducer is not pure if it calls dispatch or setTimeout
Not sure but it seems your proposal is quite close these SO answers:
http://stackoverflow.com/a/33829400/82609
http://stackoverflow.com/a/33501899/82609

@slorber Well, you're right, I misused the term xD What I meant is that this reducer is still easily composable and will produce a result depending solely on it's input... More or less...
I was thinking just yesterday if, for complex applications, is it ok to introduce another layer for the side effects, given a total of 3 reducers: state, logic/coordinator (really a pure function this time) and side effects. This kind of technique seems quite flexible to me.
That's said, what kind of use case could you think about where other solutions (e.g. redux-saga, redux-elm, etc.) are more suited for the job? I actually started a new project, my first one with redux, and because it has the potential to be quite complex, I would really like to get it right :)

Hi everyone,
I've spent many time to find solution for side effect composition. Idea to combine 'Algebraic Effects' and Redux-loop is very impressive. Another point: i love redux-saga, because it allows to write simple imperative-like code for business logic.
Few days ago, I write library that solves all my problems (IMHO).
The main idea: reducer should returs a pair { state, effects } like redux loop, but effects should be a generator like redux-saga. Effects generator returns plain objects exactly like actions.
Here sample code:
function reducer(state, action)  {
    if (action == 'Action1') {
        return effected(state, function* {
            yield put({ type: 'AnotherAction' });
            yield take(x => x.type === 'AnotherAction2');
        })
    }
    return state;
}

In nature, effects is a stream of actions that bubbles up through composed reducers and can be transformed. For example:
function reducer(state, action)  {
    if (action == 'Nested.Action1') {

        var [newState, effects] = splitEffected(nestedReducer(state, { ...action, type: action.type.replace(/^Nested\./, '') }))

        return effected[newState, effects::map(effect => {
            if (effect.type === 'PUT') {
                return { ...effect, action: { ...effect.action, type: 'Nested.' + effect.action.type } };
            }
            return effect;
        })
    }
    return state;
}

Another example: bubbling side effects and can be handled at any level of reducer composition:
function nestedReducer(state, action) {
    //...
    if (action == 'Action1') {
        return effected(state, function* () {
            var randomNumber = yield { type: 'GenerateRandomNumber', min: 10, max: 100 };
            yield put({ type: 'UpdateNestedValue', value: randomNumber });
        })
    }
    //...
}

function reducer(state, action)  {
    if (action == 'Action1') {

        var [newState, effects] = splitEffected(nestedReducer(state, action)

        return effected[newState, effects::map(effect => {
            if (effect.type === 'GenerateRandomNumber') {
                return call(function* () {
                    return getRandomInt(effect.min, effect.max)
                });
            }
            return effect;
        })
    }
    return state;
}

I have spent much time to solve problem of easy to reuse 'Confirmation dialog'. This approach solves it easily:
function nestedReducer(state, action) {
    //...
    // I want to delete item with confirmation
    if (action == 'ConfirmedDelete') {
        return effected(state, function* () {
            var confirmed = yield { type: 'Confirm', text: 'Delete item?' };
            if (confirmed)
                yield put({ type: 'Delete', id: action.itemId });
        })
    }
    //...
}

function reducer(state, action)  {
    //...
    var [newState, effects] = splitEffected(nestedReducer(state, action)
    return effected[newState, effects::map(effect => {
        if (effect.type === 'Confirm') {
            return call(function* () {
                yield put({ type: 'ConfirmDialog.Show', text: effect.text });
                var confirmOrDiscard = yield take(x => x.type === 'ConfirmDialog.Discard' || x.type === 'ConfirmDialog.Confirm');
                if (confirmOrDiscard.type === 'ConfirmDialog.Confirm')
                    return true;
                return false;
            });
        }
        return effect;
    })
    //...
}

I create repository with enhancer for createStore that allow to write reducers like examples above.
https://github.com/tihonove/reelm
Plese consider this approach and say 'dead' or 'can live'.
I'am a newbie in front-end development and spent many time to write readable code, so this approach is 'can live’, I ready to provide more examples and recipes.

Why do we want the reducers to provide side effects? I take effect here to mean an imperative description of something to be done, it kind of implies that the reducers need to decide exactly how some desired goal is to be achieved.
In some sense it seems like we're trying to recreate the problem React so elegantly solved for the DOM update problem. And I think the same solution is probably desirable in this case.
What if, instead of focusing on the effects, we put the focus on the demands that make the effects wanted? Just as there's some sentiment that derived data should be handled by selectors and not reducers, effects can be considered derived from demands. By keeping actual effects open to interpretation as long as demands are met we gain some freedom in how to satisfy those demands. An instance of that might be different interpretations in browser, in server and in test environments.
Someone mentioned above that queries should be put in the sate tree, which seems to be in that direction. Consider a relational database, it separates the demand (SQL) from the effect (executing the query) by a pipeline of interpreters, validators and optimizers.
In redux the example might be data demand. Clicking the button changes the application state into one demanding data to be loaded, or navigation state to change, or DOM to change. The DOM demand is not stored in the redux store, bu instead derived through things like react components computing a vdom, and then the effects of updating the DOM is derived from the VDOM using the diff engine. Similarily the data demand might actually be derived from the navigation demand (think angular-ui-router and it's resolve), and the effects what to fetch, how to fetch, and such, derived from that using some data diff engine.
So I guess I'm saying that state, and transitions between states, should be kept high-level and domain specific, and any effects should be derived from chains of meaning ( http://www.vpri.org/pdf/tr2009016_steps09.pdf )
Also speaking of "demands" there might be some insights available from https://awelonblue.wordpress.com/ a lot of thinking about a concept called Reactive Demand Programming ( https://github.com/dmbarbour/Sirea ) : "In RDP, the only effect is observing of the active set of demands by a resource"

@JohnNilsson I believe you're echoing a point I made earlier in the thread: https://github.com/reactjs/redux/issues/1528#issuecomment-198710920

Indeed, I think I am. Perhaps a slight variation is where to keep the fetches state. In on sense I think fetch is too low level for state, somthing identifying a graphql query is probably more what we're after. Also I'm seeing the fetches data structure being differ against some other state, not in the store, things like promises representing outstanding requests f.ex.

Yeah, in my mind, the representation is arbitrary. If there's a store listener that understands fetch demands phrased in GraphQL, so much the better!
And yep, there would be external state in the form of whatever is doing the fetching. I do think the idea would be limited if the store's state didn't also reflect things in progress. That would make it difficult to differentiate between absent data and data actively being loaded in the UI. And it would be difficult to account for data that is present, but stale. In some cases, like incremental search, I agree that the store's representation could be simplified to omit the state of fetching.

it separates the demand (SQL) from the effect (executing the query) by a pipeline of interpreters, validators and optimizers.

This is not the correct way to look at it, the sole purpose of SQL is to propose the new state without the knowledge of the rules that would make it acceptable. SQL proposes values to the acceptor which will validate (constraints), enrich (resolve indices) and optimize (plan) the mutation of the state.
Thinking in terms of demand/effect is simply the wrong way to look at it. I am not quite sure why you guys absolutely refuse to look at the theory behind mutating state. It's like trying to build an airplane and ignoring gravity. Do you really think that all this is arbitrary and you can design your way out of this problem? That's pretty naive if you ask me.

@jdubray, I agree with @slorber that if you do indeed have a better model for managing state and mutations, your descriptions haven't quite sold it. I'm sure I'm not alone in adopting Redux not just because it is a great piece of software, but also because the guide very thoroughly motivates the concepts behind the library. It's pretty clear to me that no one here is trying to ignore great ideas.

This is not "my" description. I am simply encouraging people to look at the theory behind mutation. After all it a has only got a Turing award for it, so it's probably not worth considering in this group.
Here is the book written by Dr. Lamport on TLA+
http://research.microsoft.com/en-us/um/people/lamport/tla/book-02-08-08.pdf
SAM was founded on the principles of TLA+. See for instance page 34 (please click on the embedded links:

HCSpec expresses both how the model mutates (next-state relation) and the next-action predicate. This is coming from the [] operator which is not available in programming languages so you need to keep them separate since they are are inherently separate.

That's an action (p21) TLA+ also defines a state function

How could someone, trying to figure out the best way to mutate state, ignore so openly the theory behind it? Especially when that theory is perfectly aligned (and never breaks) Redux's principle #1 (a single state tree) or when it defines actions as pure functions?
Do you guys really think you can beat TLA+? and find a better theory describing how you mutate state? Really?

@jdubray : yes, yes.  We get that there's a theory, written by a guy with a PhD, and that there's three-letter-acronyms that are magically supposed to solve everything that's wrong with Redux.  You've made that quite clear.
To genuinely answer your first question: you seem to be the only one who has taken meaningful note of Dr. Lamport's book and theory, or at least are the only one who has tried to publicize it.  Therefore, it is very easy to understand that, y'know, just maybe that work wasn't intentionally ignored, it's that no one involved in initially developing Redux had heard of it.  Most of Redux's development process happened here, on Github, and the discussion happened in the issues.  You can go back and see how most of the various concepts evolved into their current form.
As for everything else: it is indeed entirely possible that TLA+, SAM, and NAPs are truly superior, and the answer to all the current discussion regarding Redux, side effects, and state management.  But frankly, your approach to trying to evangelize these ideas is having exactly the opposite effect.
You published an article describing some of your problems with MVC: sure.  You published an article describing your alternative: good.  You published a repo containing your library and some of your ideas: fantastic.  Sharing information is good, sharing code is even better.
  Indeed, it is very hard to reason about mutation and asynchrony, Dan being yet another proof.
Who is Dan Abramov to come around and say, F#&$( that, I'll come up with my own, in a couple of days, publish a couple of cool videos on EggHead, speak to some hipster conferences, tell some stories about how "programming used to be", ... and throw away a lifetime worth of work?

jdubray (Feb 19 16:32)
What redux does is not "stupid", but it is very different from SAM.  I am not quite sure why Dan has not joined the conversation, the article has had 200k views, retweeted hundreds of times. Is he afraid of something? If it was such a slam dunk to show that SAM is in effect Redux, we'd be done.

You repeatedly appeal to authority by citing Dr. Lamport's and his papers as if he were Moses coming down the mountain with the 10 Commandments, point to your "25 years of experience" and numerous certifications, and insist that you therefore have the only viable solution:

jdubray (Feb 10 18:27)
There is someone how ever who spent his entire life thinking about this problem, Dr. Lamport. He came up with TLA+ and these semantics cannot be tweaked. SAM is based on TLA+ semantics.   ... Heck, he even got a Turing award for it.

jdubray (Feb 23 08:19)
I also have 25 MDSD/MDE experience, with deep roots in that community, not to mention 10+ years working deep inside IT, so from all I know things SDO or GraphQL works well in well formed domains, not surprised it works well for Facebook. Now we have seen so many advances in Computer Science that seemed impossible before...
Heck I am even co-author of a patent that specifies a method for automating construction of the flow of data driven applications in an entity model

jdubray (Feb 22 23:30)
Well being a BPEL/BPM expert, having built my a couple of commercial stateful process engines and worked with people like Mark Little and Eric Newcomer on WS-TX and WS-CAF...

jdubray (Feb 22 23:36)
All I can see is that you have been able to build a community around some important concepts that have been critically missing in Computer Science, now, how about carefully landing these semantics so that people end-up doing the right thing rather than continue doing hacks?
Sagas are a hack as far as I am concerned.
Not worthy of all the work you have done so far.

@jdubray (Apr 5, #1528 )
The way you mutate state is not something you make up, it follows a well defined pattern which is best described by the Paxos protocol: Propose, Accept and Learn. Again, this is not something that you are free to design. There could be a class of problems where you could find shortcuts, but this seems not to be the case of Front-End architectures.

Even if you did manage to explain, in clear, simple, understandable terms, exactly what the differences are between Redux and SAM, and convince me that SAM is the superior solution solely on a technical basis... I would refuse to use it, because I would not want to associate myself with something that comes from an attitude of tearing others down.
The React/Redux community is friendly.  This community is positive.  This community writes tutorials, shares information, answers questions, helps others.  This community is coming up with nifty new ideas.  This community is busy writing real applications, with real code, in real-world situations.  And, it would appear that there's a very large number of developers who happen to find these tools useful.
So no, Redux isn't perfect.  There's still some pain points and some concepts that people are trying to work out.  And if you have some genuinely useful contributions to make (like a pull request, or a specific way that Redux could actually be improved), please do submit those so we can discuss them.  But coming in here, insulting everybody, insisting we're doing it the wrong way, and that you are the only one with the right answer, is not a way to win friends and influence people.

I am not the only one talking about TLA+, if facts are what you are looking for, you may check this article: http://research.microsoft.com/en-us/um/people/lamport/tla/formal-methods-amazon.pdf
I know of other people who use it to.
It is true that even Microsoft ignored TLA+ for the last 10 years (how could Ballmer pay any interest to that kind of work?), but if there are any Microsoft employees in this thread, perhaps they can share some recent steps Nadella, himself, took towards using his work profusely at Microsoft. Actually I would not be surprised if at some point Mark (Z) has soon a Satia moment.
Heck, TLA+ is even on ThoughtWorks radar.
I don't "win" or "influence" people. This is not a beauty pageant, if you think it is, you are in the wrong forum. I don't belong to a herd or looking to build one. After 36 years writing code, my experience with the herds in our industry (and the Redux one does not seem to be any different) is that as soon as you bring up some facts such as "Redux seems to be breaking his principle #1 pretty easily", or "you know, that's not really how SQL works" everyone turns stops talking to you. That's exactly what Yassine did as soon as I made that point. He stopped talking within a nanosecond and Dan followed. If you want to talk about "attitude" perhaps you could also comment on that kind of "attitude".
I'll stand by my words, Redux is a piece of garbage, this very thread reflects precisely why, otherwise it would not be necessary to open it, it would already be covered by Redux.

Last person I know who mentioned Redux being inferior wrote a pretty good blog post and has a comprehensive implementation of his ideas to go with it.
http://staltz.com/why-react-redux-is-an-inferior-paradigm.html
I don't agree with him, but it only takes a few minutes to look at Cycle.js and see the pros and cons and understand the whole point.
Elm itself is another good example of someone with an idea and doing a pretty good job at explaining it (if I remember well, Elm is the result of an academia thesis, too).
It would be great to have something like that. PhDs write thesis and papers every day. It's not super useful without a comprehensive proof of concept.

Is just like to chime in and say that the community around redux seems very welcoming and positive. Perhaps there hasn't been much discussion around this because it has, in the past, quickly devolved into negativity and name-calling. I just want to encourage everyone to push hard against the tendency of the Internet, which seems to always be insulting or attacking an imagined, faceless, enemy who is in fact a person dealing with their very own problems. Be encouraging, hold your temper, have a lot of patience, and great conversation will come.

TL;DR: +1 for a redux-effects-saga + EffectsStore that listens to all / some actions, handles effects, and can dispatch actions to Redux store.
It seems like some of the solutions here would like to see redux (or a super-redux future lib) take what redux currently does and expand it to encompass effects.
I'd be much more in favor of an approach, like many have ( e.g. #1528 ) mentioned, of viewing current vanilla redux as perhaps just one composable piece of a larger state/effect management system.
Redux is pretty good at what it does. There are definitely room for improvements and optimizations, but it's amazing to see all the awesome libraries (authored by many of the commentators here) that can be used with it precisely because it only does one thing, does it well, and does it in a composable way.
Redux doesn't manage effects. I really like the idea of keeping it that way, and instead implementing a design pattern in which there are more than one type of action / message, and at least one more message handler / store (EffectsStore) that potentially stands before redux. Because reducers are pure, they would not want to emit messages back to our EffectsStore, but EffectsStore could definitely handle some actions / messages and dispatch to redux.
Right now we are limited by not allowing components to know about effects / messages, only about updated state.
For example, the use case initially mentioned by Sebmarkage for Effects had to do with layout. It would be cool if I could emit a message when I try to resize one component. EffectsStore / EffectsSaga would receive this message, dispatch the message to other components telling them to measure their DOM attributes, do some computations, and only then dispatch to Redux store. The Redux store does not, strictly speaking, need to know about any of this until it's time to update UI-relevant state.
Redux-saga solves a lot of issues, but it currently only handles actions that will be handled by redux store, which makes it unnecessarily wasteful if you just want to pass a message from a component, where that message isn't going to immediately change application state, but is instead meant to trigger other messages / effects that will (perhaps) eventually change application state. We don't even want to run those through our reducers.

For similar reasons, I'm not necessarily a huge fan of storing what we might call "demand state" in Redux
{ loadingUsers: true }` seems like a fine piece of data, because the UI depends on that to display a spinny wheel.
I would argue that UI should be completely agnostic to { computedQuery: { MY_WHOLE_APP_QUERY } }  or anything similar. To the extent that UI is a pure function of redux state, redux state probably shouldn't include irrelevant data.
Application state, on the other hand, could be something like: { redux: {}, effects-in-progress: {} }.
But, again, why update redux with that? Why go through the hassle of updating redux, pinging every reducer, and, if we update redux state with demand that don't affect our rendering, also ask every single connect-ed component to run its "should I update" checks. Even if they are implemented well, and again many folks have helped a ton with awesome libraries, we are still running thousands of pointless computations in the best case, and triggering unneeded re-renders in many cases.

@jtadmor
why would you think that there could be "'effects" without a single change to the application state (assuming the application state is in a single state tree)?
why would you think that when you change the application state, there is a (general) way to know which component will be affected? (if we update redux state with demand that don't affect our rendering,)
why would you think that application is always "visible" to the user? ("redux state probably shouldn't include irrelevant data.")
On the other hand when you separate "proposers" from "acceptors" things like "where that message isn't going to immediately change application state" become trivial to implement, lots and lots of stuff can happen to decide what to propose to the application state. When users changes their mind, I can even cancel an action that has not yet presented its data to the model. I can even trigger multiple actions and let the first one to present to the model win.
When you separate "acceptors" from "learners" then the learners can take a lot of smart decision as to who needs to render or now.
But, what do I know?

jdubray,
I'll give a try at answering what (I think) you're asking:
**1.why would you think that there could be "'effects" without a single change to the application state (assuming the application state is in a single state tree)? **
I can imagine a design pattern in which effects change application state. This would be the { redux: {}, effects-in-progress: {} } I mentioned above. I could also conceive of a design pattern in which we don't record effects at all. The former has some advantages, but also takes work to implement. Up the developer, IMO.
2. why would you think that when you change the application state, there is a (general) way to know which component will be affected? (if we update redux state with demand that don't affect our rendering,)
There are many heuristics that help me know when certain actions will affect certain components. Developers write the app. So they can know whether the current application query state affects the visual output of the app. When I know that a certain part of state is NEVER going to affect my components directly, then I shouldn't be notifying my components when that state changes.
Also, we could easily imagine a pattern in which components register their dependencies, with the Store, in the form shouldUpdate = ({ action, currentState }) => true / false, and components would only be notified when that is true, instead of the current model where each component maintains its own shouldUpdate = ({ myProps, prevState, nextState }) => true / false type logic. But that's another conversation entirely.
Frankly, I think everything else you asked or pointed out are things that are irrelevant to what I said or things that redux already does. I'd echo what others have said: When you stop using jargon that you haven't clearly defined (without asking me to read an entire book) and take the time to charitably read other's comments, you might find your conversations more rewarding.

I'm not necessarily a huge fan of storing what we might call "demand state" in Redux

@jtadmor I would say that these transient states are very relevant. (1) you can display loaders as you already said, but (2) it allows you to recover from from transient async side-effects in a time-travelling debugger -- something I discovered building elmish.

Let me explain what I mean by proposer and acceptor with a simple example. When you say
counter += 1 ;
There is another way to look at it:
const v = counter + 1 ; ... counter = v
It would of course extremely painful if all our code was written that way, but this is how "state" is mutated, one proposes value which are accepted. It's very healthy from a factoring perspective because you can have action specific logic (that "initCountDown" action only proposes values > 3) different from model specific logic (model accepts values >=0). Not to mention what I said before that you can manage proposals in a very smart way (cancellations, races, voting,...).
I'll refrain from commenting further on the reducer structure as I would start to be too negative, but if the Redux community could learn just one thing it would be that there are many advantages to separating proposers from acceptors. The role of an action is to propose values (in the SQL sense, as in I think the value of x is 3)  to the acceptor. Acceptors have full knowledge of the application state to decide whether they want to accept this value or not, while Proposers shouldn't, Actions should never request model.getState().  What Redux calls an "action" is more an "intent". When you trap the proposers in the purity of the Reducer you create unnatural constraints that you have to somehow compensate for somewhere else.
Personality, I find these factoring debates fascinating because you can clearly see how tiny, nearly imperceptible, decisions have such an impact on the structure of your code.

@ccorcos

I would say that these transient states are very relevant.

hum...you would not consider introducing a State function in Redux?

Y'know, part of the problem here is both sides are using the same terms with slightly different meanings...

Yes, I agree that's a very big issue. There are only a handful of nouns and too many variants to make it possible to label them properly. That's also why I tend to like the Paxos labels: proposers, acceptors, learners because these are roles (hence code factoring) rather than things like actions/state,...

@jtadmor
I think the problem is that Redux sells its store as being the "source of truth", while the source of truth is the event log. The state computed by Redux root reducer is just one possible projection of that log, but you can produce as many as you want, so the "state", is not really the source of truth, but it is probably simpler for beginners to think so. For simplicity, Redux somehow combines both receiving events, and projecting them. In backend systems doing EventSourcing/CQRS these 2 parts are often separated but it requires more infrastructure (like Kafka receiving events, and Spark/Storm/Samza/whatever projecting them. Often there is not a single projection logic).
What I want to say is that you should consider indeed every dispatch as a state change, even if the reducers/connected components are not interested by this change (and thus the store's state remains the same), it's still a state change.
But I understand your concern where you want to avoid unnecessary computations when possible. In case you want to dispatch an action that will only target a saga, you can use redux-ignore to avoid hitting reducers uselessly. In case your store state needs to change but it does not affect UI and you don't want connected components to run for nothing, then it's not really a Redux problem but more related to React-redux. I'm proposing here and there more flexibility in React-redux so that we can build custom connect functions that will be more optimized.
These solutions are just workarounds related to the fact that Redux is both sides, while in more complex cases it's worth considering to split (requires more work). For example, instead of dispatching to Redux directly, you could dispatch to your own event log that would produce different event streams to be consumed by multiple redux stores, one of them being in charge to manage the view state (so the stream can be filtered to only contain UI-relevant events).

@slorber
You are correct that some of what I'm proposing bleeds into react-redux optimizations. And I agree that a kafka-esque (hehe) architecture is probably ideal anyway.
Thanks, redux-ignore looks helpful and I'll read over those proposals (like I said, I think there's a lot of room for optimization, but it's also awesome that redux does one thing really well and allows for an ecosystem of helpful tools that play nice).
But part of what I'm saying is that instead of talking about turning redux into effectively a  (action) => { state, effects }, we should let it do what it do, and build other abstractions / libraries that allow for effects, rather than ruining a very nice pure one.

(action) => { state, effects } is exactly what a controller do in MVC, you should stay as far as you can from that kind of factoring.

@slorber The problem with Front-End architectures is that the View tends to be too coupled to the Model. There is a natural progression from user event (click) to intent (submit form) to action (proposed changes to the application state). There are smart/better ways to implement acceptors and you are describing a very important architecture for achieving just that, but you cannot ignore the fact that before you reach that stage, you need to go through an event/intent/action (aka propose) stage. The Acceptor event log is unaware of the View event/intent/action stage. It is the output of the action that makes it to the "event log".
On the other side (M->V) you need as much decoupling, which in SAM is achieved by the S() function.

That presentation is a bit extreme (and not that interesting to watch) in decomposing every step towards mutation (especially if you are into RFP) but it shows that these things are not arbitrary (in spite of every short cut we might have come accustomed to use). If you are serious about answering the #1528 question I believe that's what you have to do.

@jdubray

@slorber The problem with Front-End architectures is that the View tends to be too coupled to the Model. There is a natural progression from user event (click) to intent (submit form) to action (proposed changes to the application state). There are smart/better ways to implement acceptors and you are describing a very important architecture for achieving just that, but you cannot ignore the fact that before you reach that stage, you need to go through an event/intent/action (aka propose) stage. The Acceptor event log is unaware of the View event/intent/action stage. It is the output of the action that makes it to the "event log".

Still don't understand if what you talk about is an app that has a remote backend/storage system or if it's only a pure local app so it's hard to follow you.

There is a natural progression from user event (click) to intent (submit form) to action (proposed changes to the application state).

No there is not. An an UI, when the user wants to click on a button, the following happens:

Intent; the user has in his brain the wish to click on a button
Action: the user move his mouse and use his finger to do the click
Event: the button has been clicked

The UI is not aware of the intent and the action, but is only aware of the event. The "button clicked" event is something that has happened, it is not something questionnable at this point, and it does not need to pass through any "acceptor" layer. The button has been clicked, that's all. That event is added to the log, which in the same time modify the state.
There is no coupling at all to the model. You have a log of events, and then project it has a useful object to fill your views. The recording of events and the projection into an usable form do not even need to know each others. Events emitted should describe what has happened, and not be a mean to "ask" for UI changes. But indeed they do trigger UI changes, in a decoupled way.
@jdubray it should be clear that you can't apply directly rules of distributed systems in an UI. In a distributed system like a simple client-server, the client emits its intent based on stale or eventually consistent data. By the time the intent of the client is shipped to the server that data may have change so it makes sense to have a validation layer. For an UI, if the user should not be allowed to click on a button anymore we would simply make that button disappear immediately and synchronously. There is no way that a "button clicked" event gets fired if the click may be refused, because we would simply not display the button in this case. Note it's worth mentionning that sometimes we display an error message to the user after his click, but it's still a valid click event, that indeed triggered a state change to display that error feedback.

On the other side (M->V) you need as much decoupling, which in SAM is achieved by the S() function. In other words  is not enough.

@jdubray we don't use <V props={state}/> with Redux but we all use <V props={map(reduce(log))}/>. Isn't it enough decoupling for you?

@slorber SAM does not prescribe any approach to the persistence of the model, so I am not talking about persistence at all (though when you mention CQRS and event logs you seem to be talking persistence). Nothing prevents you to use that approach to manage the application state of course.
SAM's factoring is architecture agnostic and I have shown repeatedly that it is readily isomorphic. That means that the whole code can be built in the browser and partially or entirely moved to the server (actions only, actions + model, model only, action + model + state,...).
I have to go catch a plane right now. I'll try to prepare a sensible answer to your reply, though I am not quite sure why you would deny that when the user "submits" some data to change the application state, there is something that must accept that data. In Redux the store is responsible for accepting whatever the Reducer passes it to.

though when you mention CQRS and event logs you seem to be talking persistence

I'm not talking neither about persistence nor remote communication. I'm talking about a simple browser-only application that does not persist state (like TodoMVC). An event-log can be transient and in-memory as well (this is what Redux DevTools do)

I am not quite sure why you would deny that when the user "submits" some data to change the application state, there is something that must accept that data. In Redux the store is responsible for accepting whatever the Reducer passes it to.

When the user submits a form, then he... has submitted that form. It is a non-questionnable fact that can be added to the event log (ie dispatched). The Redux store is a projection of facts of the past. Its role is not to accept or reject facts, it is to project facts into a data structure that is usable for the view (ie JSON). The store never question a fact or reject it, but it can simply ignore a fact it it thinks that fact is not important to construct the UI state. There has never been any accept/reject system in Redux.
However, many people do dispatch intents/commands instead of facts/events, and I think this is wrong. This can be solved by redux-saga

The reason for the demand state, instead of having the event log as the source of truth, is at least two fold. First of all redux doesn't persist the event log so thats a big difference. But I think more importantly is that that even if it did we still would not have a single source of truth.
Even with only the DOM-sync problem we have two separate stores (redux and the dom) with separate histories. And the truth is that we want the two to converge some how, we might want to reflect a change in an input-field in a redux store, so dispatch an action (an effect), or we want to reflect the validity of this new value in the dom so we dispatch a dom update operation (an effect in the other direction).
So in any point in time we have a set of states we somehow want to converge. From the perspective of each process (if we view the states as a process) there is no direct control over the other states, only our current demand (state), and a means to communicate that demand.
I guess, from a strict point of view the actual demand is external from the state, state just is. In the React case demand is declared in the components as a relation between states.
In any case, one reason to keep state declarative in the stores, instead of implicit in sagas, is to not be so complected with time. (Also a saga is its own state, witch increases the set of states to coordinate, isn't that kind of counter to the redux idea of a single state-object?) It seems to me that it should be easier to reach a desired state from demands than from stored effects.
Imagine a situation where some action has just been sent to a server and we do a browser upgrade, or maybe we just resumed from a sleep state. Being back in business now entails figuring out if we can still expect an answer to that request, if we need to execute it again or not and so on.
The state {isLoading:true} is not really enough to determine if any new effects should be triggered. The original FORM_SUBMIT_REQUESTED event might help us create a state to determine that though. So an event source approach would probably be fine just replaying the events. We don't have the event source in redux though, so the next option is to have store keep the demand {formData:{commit:true}} in the store and diff that to our outstainding requests and dom presentations.

@slober while it is true in a theory that the final state of an application could be recreated by replaying the events. There is one very important cavet --- the state change in response to those events MUST be deterministic.
For example, I think it is reasonably well known that a database can be recreated by replaying the transaction log.  But this change log is not simply a copy of the exact statements submitted by the user.  For example, if the user submits a statement update table set field=value where created_date = now() - 7.  If you replay this query at a later time you will get a differnet database state since the value of now() is impure, to make this work the transaction log also includes the value of now and re-execution does not actually use the current timestamp but the recorded value and thus the transaction is once again a pure and deterministic event.
The problem with using user events as a source of truth in redux is that action creators are not required to be pure, thus it is not possible to reconstruct the application state from user events
alone.  For example, if the desired response to clicking a button is to increment a counter by a random amount.  Knowing that the user clicked the button 3 times is insufficient to deterimine the resulting application state.
I think this is what @jdubray is trying to suggest when stating proposers and acceptors are different.
However, while you say event log and give user events as examples.  I think what you are actually proposing is to record a log of the "accpeted" actions created by action creators rather than "proposed" actions such as "click" events.  Thus if clicking a button proposes changing the state by a random increment the actual random value would be included action. Thus the action log is an event log is that has been augmented to include necessary external state such as the randomly generated value in the same way that the database transaction log includes what now() was at the time the statement was recorded.  Thus replaying this action against the store is now once again deterministic and is perhaps now a sufficient source of truth to recreate the application state.
I think action creators are the "acceptors" @jdubray feels are missing are missing from redux.
But... action creators are not ideal place however for accepting actions or initiating side-effects because the transition from a proposed action to an accepted action (and its resulting side-effects) often requires knowledge of the current application state and forces one to move business logic into the action creators which have limited or no visibility of the current application state.
If I complete a form to transfer a million dollars from my checking account to an offshore account and propose this action by clicking submit, the action creator may not want to blindly accept the action without checking that I have sufficient funds.  This may require additional state to unnecessarily be passed through the views into action and possibly aysnc queries to external sources.
Symantically action creators are expected to return an action to dispatch and cannot be null.  Javascript does not allow one to block and await verification from an external service, so the implication is that the action creator must accept the proposed action and we seem to be in trouble.   But thanks to redux-thunk we don't actually have to return an action we could return function instead of an action so it is possible for the action creator to access the full state and to dispatch later or not at all and problem solved right.
So lets say I do have sufficient funds. Where do I put the side-effecting http request that actually initiate the transfer on the server? In redux, the side-effects go in the action creator.  Since the action creator has been aquired the necessary information to accept the transaction it can issue an http request to initiate the actual the transaction.   What if I click submit twice?  Each proposed action must be accepted independantly.  Since javascript is single threaded if an action is returned the the state will be updated before the next action and the next action will see the updated state and we are good right?
Well maybe, if the action creator returns a function and initiates an http query instead of immedialty returning the action then the thread will return to the javascript event handler allowing a second submit to occur before the first action has completed.  Thus it may be possible for the second click to see the same state as the first click since the processing of the first action was deferred by redux-thunk allowing actions to be accepted and state updates to happen non-atomicly.
For this and other reasons.  IMO, it is essentail that side-effects occur after the application state has been updated, not before.
I gave a few additional arguments for this on a related thread 1182.

@kurtharriger I think my proposal (https://github.com/reactjs/redux/issues/1528#issuecomment-207506943) could solve all the issue you stated.

The problem with using user events as a source of truth in redux is that action creators are not required to be pure, thus it is not possible to reconstruct the application state from user events
alone. For example, if the desired response to clicking a button is to increment a counter by a random amount. Knowing that the user clicked the button 3 times is insufficient to deterimine the resulting application state.

Keep in mind that the taskReducer doesn't trigger store.subscribe and cannot directly modify the state, but should dispatch another action. Also the stateReducer must always be pure and focused in changing the state, with possibly zero logic. Your example would be something like:
import * as A from './actions'

function taskReducer(dispatch, action, state) {
  switch(action.type)
  {
    case A.RANDOM_INC: {
      const rand = Math.floor(Math.random() * 10) + 1 // 1 .. 10
      return dispatch( A.inc(rand) )
    }
  }
}

function stateReducer(state = 0, action) {
  switch(action.type) {
    case A.INC:
      return state + action.payload

    default:
      return state
  }
}

When we click the "random button" the log will contain two actions: RANDOM_INC and INC; then if we want to replay the log, we can simply ignore all the actions that doesn't trigger a state change (if you follow my proposal, we can just filter for action.meta === null) and we will end with the same correct state.

If I complete a form to transfer a million dollars from my checking account to an offshore account and propose this action by clicking submit, the action creator may not want to blindly accept the action without checking that I have sufficient funds. This may require additional state to unnecessarily be passed through the views into action and possibly aysnc queries to external sources.
[...]
So lets say I do have sufficient funds. Where do I put the side-effecting http request that actually initiate the transfer on the server? In redux, the side-effects go in the action creator. Since the action creator has been aquired the necessary information to accept the transaction it can issue an http request to initiate the actual the transaction. What if I click submit twice? Each proposed action must be accepted independantly. Since javascript is single threaded if an action is returned the the state will be updated before the next action and the next action will see the updated state and we are good right?
[...] it may be possible for the second click to see the same state as the first click since the processing of the first action was deferred by redux-thunk allowing actions to be accepted and state updates to happen non-atomicly.

The taskReducer will manage all of this:
import * as A from './actions'
import * as m from './model'
import api    from './api'

function taskReducer(dispatch, action, state) {
  switch(action.type)
  {
    case A.TRANSFER: {
      if( m.isTransfering(state) )
        return

      dispatch( A.transfering(true) )

      const { iban, funds } = action.payload
      return api.transferFunds(iban, funds)
        .then((bal) => dispatch( A.setBalance(bal) ))
        .catch((err) => dispatch( A.mapErrorToAction(err) ))
        .finally(() => dispatch( A.transfering(false) ))
    }
  }
}

function stateReducer(state, action) {
  switch(action.type) {
    case A.SET_BALANCE:
      return m.setBalance(state, action.payload)

    case A.TRANSFERING:
      return m.setTransfering(state, action.payload)

    default:
      return state
  }
}

I think it's ok to embed the transfering flag in the state, you usually want to display some loader to notify the user and this also remove the "multi submit issue".
The api.transferFunds will probably issue and ajax request and the server will do all the checks needed (we can also separate the checking part from the actual transfer, the flow is the same and this is just an example), but then an action will be dispatched to mutate the state accordingly.
Thinking on it, the taskReducer could also be seen as an intentReducer. What do you think?

@kurtharriger thank you that's really the kind of discussions I had originally envisaged when I tried to contact Dan in December. I won't reopen the debate on "purity" it seems to be closed, the world impure, we should learn to live with it: "[functional] purity is useless"

it is not possible to reconstruct the application state from user events
So, yes, we agree, this is a purity constraint that severely limits your architecture.

The problem [is] using user events as a source of truth in redux
yes, this is the major issue I have with Redux, I also agree that action creators (aka thunks) would solve that problem, but I am advocating for using action creators by default because there is never/hardly a case where you can "accept" readily the values "proposed" by the user event. There is nearly always a validation step (and often an enrichment one). Furthermore the model constraint might be slightly different from the action constraint. If we go back to your money transfer example, the action might want to validate that your are entitled to transfer $1M, if not, it will propose to the model to accept an exception. If you are entitled, it is then the model that would validate the integrity check that you have enough funds to do so.

All I am advocating is that there is a clear delineation of responsibility between "proposing" and "accepting" values. That delineation is very healthy and unfortunately the "purity" of the Reducer seems to couple both of them, while asking the store to "accept" whatever the reducer came up with.
If you take a more mundane example, such as "Change of Address", the user event/intent will contain the data keyed in by the user, but the action will first get a Postal Address (say from an API call), and it is that postal address that will be proposed to the model (or a list of potential match if a single one could not be found).
There is a huge value in pushing as far down the time you enter the accepting phase. It would be a poor factoring to start mutating the application state while you still need to get the Postal Address. Wouldn't you agree?
The Reducer should be "accepting" values on behalf of the store, nothing less, nothing more. All the code that computes (or gathers) data that will be accepted should be factored out from the reducer. In SAM I prefer a "dataset" style interaction between the actions and the model, but that's not mandatory. However, by coupling directly the user events/actions to the reducer, well you do just that you create a massive coupling between the view and the model. That coupling is not just unnecessary, it is highly undesirable.

Javascript does not allow one to block and await verification from an external service, so the implication is that the action creator must accept the proposed action and we seem to be in trouble.

When you consider that (Store + Reducer) is now an acceptor, then it becomes ok to call APIs in the proposer (Action Creator).

Where do I put the side-effecting http request that actually initiate the transfer on the server?

Unfortunately because the Reducer has to be pure, you cannot put them in the Reducer (or the Store since the Store is completely dumb). In SAM the Model is not pure, it is the application state, to it is "ok" to put this kind of request in the model since they have a direct impact on the ability for the model to accept the proposed values (transfer $1M).
That has been really my point all along (actually ever since I have tried to use React), Redux makes it a lot harder to weave API calls in its programming model and there are no particular reason to do so, except if you want to be "cool" and use a reducer as the foundation of your programming model. Erik Meijer (who works for Facebook) seems to explain that's not really a good idea. In particular the fact that you can access the entire state in an action creator or a sage is fundamentally wrong. The scope of a proposer should be extremely limited, it is the acceptor which should have access to the entire application state. In SAM the model exposes (generally) a single method: present where all the actions present their value.

it is essentail that side-effects occur after the application state has been updated, not before.

So for all the reasons explained above, I would challenge that. This is only true because of some artificial purity constraint.
If we agree on this, I am happy to continue the discussion on the State function S().

@JohnNilsson

Even with only the DOM-sync problem we have two separate stores (redux and the dom) with separate histories. And the truth is that we want the two to converge some how,

So we agree then that this is a "state alignment" problem that in essence a Front-End architecture is de facto a distributed system, and therefore you need a state alignment protocol such as propose/accept/learn to allow the elements of the system to converge?
From the Paxos protocol:
• Only a value that has been proposed may be chosen,
• Only a single value is chosen, and
• A process never learns that a value has been chosen unless it actually
has been.

@Iazel This code would be a lot simpler if it was all implemented in the acceptor, you should not need to dispatch another action to set this property value A.transfering(false). The acceptor boundary should be as wide as it is necessary to accept the proposed values. It is then, when the values have been accepted (or not) that the learners get triggered.
Logically, this kind of code opens the door to some race conditions/dead locks when another action has been processed that creates a condition to reject the proposed value. I don't want to use the word "transaction boundary" but that's pretty close when it comes to accepting proposed values.

@iazal This is an interesting approach. It is similar in a way to what was proposed in https://github.com/reactjs/redux/issues/1182 which I also like, but as a midleware rather than a store subscriber.
The advantage of registering it as a middleware instead of a subscriber is that it also has access to action.  Subscribers only have access to the final state and not the action which made the reactions subscriber proposed in 1182 more complicated then it need be for simpler cases like this.
The biggest issue I see with this approach is that the taskReducer is impure and thus violating the constraint which enables rewinding and replaying actions against a prior state as this may cause undesirable side-effects.  Perhaps its possible to somehow disable the taskReducer when replaying actions or to filter the actions that are replayed, but a reducer with side-effects will be difficult to sell.
Your taskReducer however may be more similar to sagas middleware rather than a reducers.  Thus you might have better luck promoting the idea if you don't call it a reducer.  As far as I know sagas cannot be replayed either, but perhaps this problem has already been solved or is an easier problem to solve.
I'll have to play with it a bit more, but I like the idea in theory.

@slorber I don't believe we are in a disagreement at all, but the way you state it seems to imply that we are rifts apart:

When the user submits a form, then he... has submitted that form. It is a non-questionnable fact that can be added to the event log (ie dispatched). The Redux store is a projection of facts of the past.

It really depends what you mean by "projection", if "projection" means propose+accept, then we are just saying the same thing. All I am arguing is for separating proposing from accepting values.

All I am advocating is that there is a clear delineation of responsibility between "proposing" and "accepting" values. That delineation is very healthy and unfortunately the "purity" of the Reducer seems to couple both of them, while asking the store to "accept" whatever the reducer came up with.

However, by coupling directly the user events/actions to the reducer, well you do just that you create a massive coupling between the view and the model. That coupling is not just unnecessary, it is highly undesirable

That delineation is exactly the purpose of redux-saga. Again look here how clicking on a username gets decoupled from loading an user profile.