Deep merge

import {merge} from "ramda";

console.log(
  merge(
    {age: 30, tags: ["a"] },
    {tags: ["b"] }
  )
); // {age: 30, tags: ["b"]} -- ok

console.log(
  merge(
    {model: {age: 30, tags: ["a"] }},
    {model: {tags: ["b"] }}
  )
); // {model: { tags: ["b"] }} -- ??? where is age ???

So now merge behaves like Object.assign and does not merge nested objects.
console.log(
  Object.assign(
    {},
    {model: {age: 30, tags: ["a"] }},
    {model: {tags: ["b"] }}
  )
); // {model: { tags: ["b"] }} -- this is still lame but expectable from Object.assign

The note from docs

This function will not mutate passed-in objects

is not clear. Do we talk about being immutable or do we talk about mimicking Object.assign behavior?
If it's expected behavior I'd recommend to reconsider it.
To my experience, the business cases where merges of nested objects are required are
much more common. And, even more important, you may emulate "shallow merge" behavior with "deep merge" but not vice versa.
// emulate SHALLOW merge for specific field
result = assoc("model", data2.model, merge(data1, data2));

just because you want mergeDeep does not mean "merge is broken".
For the short-term, I recommend we explicitly document that merge is shallow, not recursive.
For the longer term, we should decide if we want to tackle this problem. I think it is thornier than it may appear on the surface. Have to handle cases for constructed objects, undefined properties, cycle detection, clobbering with different types.
Who wants to take a whack at it? (not it!)

Not it!

There are several requests that have come up multiple times.  Some, like foldObj are shot down for solid API reasons.  Some, like a deep clone, were delayed because of implementation difficulties. This one is a little trickier.  @paldepind has mostly convinced me that there are no technical issues with mergeDeep that don't also plague our current shallow merge.  And yet the uses and limitations of our current merge seem clear in my mind, but I can't figure them out for a deep merge.
What happens with cyclic references?  Copying the references at the root doesn't seem to be an issue, but how about if we want to walk the tree?
What happens with constructed object properties and their prototypal members?  Do we duplicate all prototypal values in the merged object?  Ignore them?
It's not that I don't think we can solve these issues.  It's that I think these are difficult and yet uninteresting questions.  So it makes it hard for me to want to work on them.
So @ivan-kleshnin, are you interested in working on an implementation?

Is the reason for not doing this that it is complicated? Is it not possible to take an existing version, shamelessly copy it and the ramdaify it if needed?

I actually intended to submit a PR for with a deepMerge to put some action behind my words. But I haven't got around to do it.
This is how I would implement deepMerge:
1/ Deep merge recurses into arrays and non-atomic objects. Non-atomic objects are objects that have keys in the eyes of Object.keys. It will for instance recurse into an object literal but not into a date object.
2/ Values from objects are copied into a new plain object.
3/ Value from arrays are copied into a new array. Sparse arrays are not handled.
I think the above rules are very simple both to understand and to implement. They don't promise too much. The implementation will do what users expects but not attempt to do more than that (deal with inherited properties for instance).
What do you think?

@paldepind I look forward to seeing what you come up with. I think you will need cycle detection. And the name should probably be mergeDeep as we have cloneDeep

I think you will need cycle detection.

You don't think a stack error is good enough?

And the name should probably be mergeDeep as we have cloneDeep

That I have no opinion on. But where is cloneDeep?

But where is cloneDeep?

my mistake -- we dropped shallow clone and renamed cloneDeep to clone. Even I can't keep up with all the API changes

Your approach sounds good to me, @paldepind!

Yeah "broken" is an overstatement, so I renamed thread to be neutral.

Will we concat tags as Lodash does by default?

lodash doesn't concat arrays when merging.

Yes, I refreshed that old issue in memory. It was not about concatenation. It was by unpredictable merge behavior:
var merge = require("lodash.merge");

var oldErrors = {
  model: {
    tags: ["foo"],
  }
};

var newErrors = {
  model: {
    tags: [],
  }
};

console.log(merge({}, oldErrors, newErrors));
// {model: {tags: ['foo']}} vs {model: {tags: []}}

Well, it's quite funny that I raise the same issue with the same name for different lib.

 I want to revert back "broken" title.
dict({"tags": ["foo"]}, **{"tags": []})
# {'tags': []}

user=> (merge {:tags ["foo"]} {:tags []})
{:tags []}

I think you're confusing merge and assign. Merge as the name suggests will attempt to merge values while assign/extend/(immutable merge) will assign values (overwriting previous values).

I think that's you confuses merge and something bizarre where second argument overrides first but sometimes first overrides second.

Naw, I got this ;)

Python:
dict({"tags": ["foo"]}, **{"tags": []})
# {"tags": []}

Clojure:
(merge {:tags ["foo"]} {:tags []})
; {:tags []}

Ramda:
R.merge({tags: ['foo']}, {tags: []})
// {tags: []}

This looks consistent to me. This is the behaviour I expect from a shallow merge function.

console.log(Ramda.merge({}, modelA, modelB));

Ramda's merge only takes two arguments, modelB is never considered here. There is also mergeAll.

A number of libraries, going back at least to jQuery's extend used this function to mutate the first parameter, which lead to the pattern of passing an empty object as the first parameter.  I'm sure that's what R.merge({}, modelA, modelB) is all about.  Ramda never mutates user data, so this is not an issue.  As @kedashoe says, just use R.merge(modelA, modelB), and we should get similar behavior to most of the other libraries and languages.  The array concatenation behavior is odd.

@davidchambers sorry it was late and I made a typo.
Correct one is
R.merge({tags: ['foo']}, {tags: []})
// {tags: ['tags']}

Ramda's merge only takes two arguments, modelB is never considered here.

Yes, it was late and lost attention. Sorry about misinformation.

Still, what do you guys think about this one:

my mistake -- we dropped shallow clone and renamed cloneDeep to clone. Even I can't keep up with all the API changes

That's it. How about making the same with equality and merging :smiling_imp: ?

===
My rant about it.
Equality is equality. Objects are either equal or not. All this deep vs shallow comparison is very imlementation specific. IMO, functional programming is mostly about data, not even functions.
So we win when we concentrate on data, not it's representation.
Simple example:
user=> (= '(1 2 3) [1 2 3])
true

What's going on here? In Clojure there is two datatypes: lists and vectors. They have fundamentally different memory representation (linked lists vs trees, that's about memory persistance if you care).
But at comparison Clojure says they are equal. Because they contain the same data.
Smart reader will interrupt me and ask about strings. Aren't those the same data but with different represantation he questions:
(= "123" ["1" "2" "3"])
false

Good catch. Strings are arrays of chars. Clojure sucks here because it was made upon Java and inherited bad parts from it. Performance was improved but along with that we've got a ton of uncomfortable consequences: you can't use the same functions for arrays and strings to begin with.
Best thought-out languages like Haskell or Racket which weren't bound by compatibility issues chose to treat strings as an arrays of chars.
So, back to original question. Is there something useful in providing both eq and deepEq?
Did you guys think about gt, lt and other useful primitives for type-safe data-oriented comparison.
Will you also provide deepGt, deepLt to be consistent ;) ? I bet you not.

I'm afraid I may need to rant about you rant.  :wink:

Equality is equality. Objects are either equal or not. All this deep vs shallow comparison is purely imlementation specific. It's brittle.

We live in a world of implementations.  People use Ramda only inside Javascript environments... obviously.  When you design an API, you have to decide what restrictions you're going to live with.  Ramda has from the beginning been a library that has tried to work with the grain of the language.  We certainly pick the parts of the language that interest us, but we're not trying to be Haskell in Javascript clothing.  This means that we have to consider questions that deluge the Javascript ecosystem, including ones about references and whether two distinct handles refer to the same item.

IMO, functional programming is mostly about data, not even functions.
So we win when we concentrate on data, not it's representation.
Simple example:
user=> (= '(1 2 3) [1 2 3])
true

You make it sound as though any container that holds the values 1, 2, and 3 in that order should compare equal.  Is that really your position?  For it seems absurd to me.  It's quite easy to imagine a binaryTree function that works like this:
var x = binaryTree(binaryTree(1, 2), binaryTree(3));
var y = binaryTree(binaryTree(1), binaryTree(2, 3));
x.display(); //=> (1, 2, 3)
y.display(); //=> (1, 2, 3)
x.add(4); y.add(4);
x.display(); //=> (1, 2, 3, 4)
y.display(); //=> (1, 4, 2, 3)
foldl1(average, x); //=> 25/8
foldl1(average, y); //=> 21/8

Meanwile in JS people can't even decide whether to believe about two empty arrays being equal:

What makes two arrays equal for you? Same elements? Same order of elements? Encoding as JSON only works as long as the element of the array can be serialized to JSON. If the array can contain objects, how deep would you go? When are two objects "equal"? –  Felix Kling Oct 20 '11 at 14:31 source...

This is the most ridiculous and absurd programming statement I ever heard. I always cite it.
That guy was so blinded by JS specific implementation so he totally forgot about reality.

No, that guy (one of the most respected members of the StackOverflow Javascript community, BTW) was answering a question titled "Comparing two arrays in Javascript" (emphasis added) posed by someone who was unsurprised that a simple == doesn't work.  His answer was on the mark, given the context of the JS language.  If you answered that question with a lecture about value equality and the interesting trade-offs made by Clojure that were unnecessary in the purer Haskell and Racket, yours would be the most ridiculous and absurd answer imaginable.  ("How can I clean the fuel injectors for my 2004 Ford Taurus with a 3.8L V6?"  "Well, in an ideal world, no one would be driving internal combustion engine vehicles, so there would be no need for any fuel delivery systems whatsoever."  "Sure, dude.")
So  what, to your mind, should the result of this snippet be?
var a = [];
var b = [];
a.push(b);
b.push(a);
eq(a, b); //=> ??

So, back to original question. Is there something useful in providing both eq and deepEq?

Well, they answer very different questions when it comes to non-primitive values.  If
eqDeep(a, b); //=> true

Then for any fn,
eqDeep(fn(a), fn(b)); //=> true
var a2 = clone(a);
eqDeep(a, a2); //=> true

But if
eq(a, b); //=> true

then
eqDeep(a, b); //=> true
var a2 = clone(a);
var b2 = mutate(b);
eq(fn(a), fn(b2)); //=> true
eqDeep(fn(a), fn(a2)); //=> false

I guess whether that seems useful is in the eyes of the beholder.  But for anyone who is working with any mutable data, reference equality may well turn out to be important.

Equality is equality. Objects are either equal or not. All this deep vs shallow comparison is purely imlementation specific. It's brittle.

I think you're very right if we only talk about values (we values of course can't change, 7 is always 7). And that's all you have in a purely functional programming language. But in JavaScript we also have pointers to values. And then you suddenly have a new type of equality. Pointers can be considered equal if they point to the same value or they can be considered equal if they point to the same location in memory. I.e. equality is not as simple as you make it when the language you're using supports pointers to mutable values.

I think you confuses merge and something bizarre where second argument overrides first but sometimes first overrides second.

No, that is not what is going on. The array elements from the second overrides the first. Thus when there are no elements in the array in the object in the second parameter nothing will be overridden in the resulting object. I think the lodash behavior makes sense.

No, that is not what is going on. The array elements from the second overrides the first. Thus when there are no elements in the array in the object in the second parameter nothing will be overridden in the resulting object. I think the lodash behavior makes sense.

That means Lodash treats Arrays like Objects and offsets like keys. This is understable only from the point of JS implementation. But is weird from all other point of views. Lodash just totally lost Array (the only ordered collection!) concept by this decision. This behavior measn no sense from business tasks. Can you provide any use-case when this behavior would be desirable? No, you never need to merge arrays. Because offsets are not keys. You always either replace or concat.

I think you're very right if we only talk about values (we values of course can't change, 7 is always 7). And that's all you have in a purely functional programming language. But in JavaScript we also have pointers to values. And then you suddenly have a new type of equality. Pointers can be considered equal if they point to the same value or they can be considered equal if they point to the same location in memory. I.e. equality is not as simple as you make it when the language you're using supports pointers to mutable values.

Well, of course there're cross-data pointers. But a doubt that a library which provides and promotes functional approach to JS should be concerned with that. Do you handle cases where developer put something strange (like an object of his own class) to the data? No you believe that's his own problem to send "broken" data to the Ramda. Same think with links. Just ignore them. With memory leak if it's inevitable. It's just not your problem.

@CrossEye

You make it sound as though any container that holds the values 1, 2, and 3 in that order should compare equal. Is that really your position? For it seems absurd to me.

Yes, it's my position and it's how Clojure behaves.

No, that guy (one of the most respected members of the StackOverflow Javascript community, BTW) was answering a question titled "Comparing two arrays in Javascript" (emphasis added) posed by someone who was unsurprised that a simple == doesn't work. His answer was on the mark, given the context of the JS language. If you answered that question with a lecture about value equality and the interesting trade-offs made by Clojure that were unnecessary in the purer Haskell and Racket, yours would be the most ridiculous and absurd answer imaginable. ("How can I clean the fuel injectors for my 2004 Ford Taurus with a 3.8L V6?" "Well, in an ideal world, no one would be driving internal combustion engine vehicles, so there would be no need for any fuel delivery systems whatsoever." "Sure, dude.")

Well, I don't agree at all.  He should tell something like:

Good question, pal. JS treats arrays much like objects (Array has an Object in prototype chain)
So your question is the same as why {} != {}.

And then explain it as an OOP heavy legacy.

So what, to your mind, should the result of this snippet be?

var a = []; var b = [];
a.push(b); b.push(a);
eq(a, b); //=> ??

In the best case: exception with Ramda detected cross-data reference link. Cleanup your data, dude
In the worst case: equal, because [[]] == [[]] and let the inevitable consequences of dirty data will bubble in the other place.

That means Lodash treats Arrays like Objects and offsets like keys in this exact case. This is understandable only from the view of JS implementation.

... which, by itself, does not make this an invalid idea.
I agree with you here.  At least from Ramda's point of view, where we mostly want to think of Arrays as lists, this behavior is fairly odd:
var a = {tags: ['foo', 'bar', 'baz']};
var b = {tags: ['qux', 'norf']};
delete b.tags[0];
_.merge({}, a, b); //=> {tags: [undefined, 'norf', 'baz']}

(Note especially the undefined first element.  Even though '0' in b.tags; //=> false, its non-existent element still overrides the one from a.tags.)
I would find simply replacing the first array with the second and not descending into it to be a bit more sane.  And I guess this is a response to @paldepind, too, then.

@ivan-kleshnin

You make it sound as though any container that holds the values 1, 2, and 3 in that order should compare equal. Is that really your position? For it seems absurd to me.

Not "any container". Any container that meant to represent plain sequential data for user eyes.

I guess I really don't know what that means.  I can imagine two different implementations of a sequence, each with an add method.  One pushes data to the beginning of the list, the other to the end.  By your suggestion, they should be reported as equal if they both currently contain the sequence 1, 2, 3.  But then they lose the notion of substitutability, which is really what equality should be all about.  For -- with some appropriately dispatching add function -- add(4, a); //=> (4, 1, 2, 3) and add(4, b); //=> (1, 2, 3, 4).  I don't understand this idea of super-type equality.

@ivan-kleshnin:
[regarding a comment on StackOverflow]:

I hate when people make you think there is something wrong with you, where's it's definitely something wrong with their tool. He should tell something like:

Good question, pal. JS treats arrays just like objects (because Array has an Object in prototype chain, you know) So your question should be really why {} != {}.

And then explain it as an OOP heavy legacy. His answer was totally opposite.
Like that's a normal thing and fairly logical that two empty arrays are different.

I've read that comment, and the entire exchange on that question over and over, and try as I might, I can't see Felix King's answer as condescending.
The original question about testing two arrays for equality in a more efficient manner than through a JSON conversion.  It sounds as though you would turn it into a debate on the design of the Javascript language choices.  While such a debate could be interesting and fun, it certainly would be out of context as an answer to such a question, and is far off-topic for StackOverflow.
And as to your snide implication that it's crazy to imagine a situation where two empty arrays are considered different, while I have absolutely no data to back it up, I'd be willing to bet that in the history of programming languages, there have been more where such is possible than where it's impossible.

Maybe I've got Immutability and side-effect free functions are at the heart of its design philosophy declaration in the docs too literally

No, you can take that very literally. But you maybe missed the part that said "A practical functional library for Javascript programmers.". I find it a bit odd that you initially told me that just doing compose in JavaScript was freaky. And now you want to be even more extreme and pretend that JavaScript doesn't have pointers and compare everything by value only. Don't get me wrong. Equality by value is important. But pointer equality is a part of JavaScript. And supporting equality across such
@CrossEye

I guess I really don't know what that means. I can imagine two different implementations of a sequence, each with an add method. One pushes data to the beginning of the list, the other to the end.

I think he is talking about different physical implementations of the same abstract data structures. For instance an ordered list is an abstract data structure. A physical implementation could be an array, a linked list or the kind of trees that Clojure uses. Providing value equality across such data structures is a solid feature IMO.

I would find simply replacing the first array with the second and not descending into it to be a bit more sane. And I guess this is a response to @paldepind, too, then.

I've been convinced that not considering the properties in the first array is sane. But we should surely descend into the second array and deep copy any arrays or objects it may contain. Right?

@ivan-kleshnin

I guess whether that seems useful is in the eyes of the beholder. But for anyone who is working with any mutable data, reference equality may well turn out to be important.

If you want Ramda to cover mutable data cases, then of course a lot of what I've said is inapplicable.
Maybe I've got Immutability and side-effect free functions are at the heart of its design philosophy declaration in the docs too literally :smile:

Perhaps you do.  Ramda is designed to help users work in a particular style.  But it does not attempt to be proscriptive.  While Ramda will never mutate your input data, it doesn't care if you do so, and it is meant to be flexible enough that nothing will break if you do.  Ramda's own functions are all free of side-effects, but it certainly doesn't except that all you functions will be.
Ramda wants to make a functional style easier in Javascript.  But it's not trying to replace Javascript with a more functionally pure language.  There are some great languages that do so, things like PureScript, [ClojureScript][cs], and Elm.  But that's not Ramda's place.
So yes, we do want to cover the mutable case.  People use Javascript to work with all sorts of mutable data, including DOM nodes.  We certainly are not trying to write a library in which they have to create immutable wrappers for any such structures.  Nor are we trying to insist that users work solely with FP techniques.  We just want to be able to offer them useful  FP functions for when they choose to do so.

@paldepind:

I would find simply replacing the first array with the second and not descending into it to be a bit more sane. And I guess this is a response to @paldepind, too, then.

I've been convinced that not considering the properties in the first array is sane. But we should surely descend into the second array and deep copy any arrays or objects it may contain. Right?

Probably.  I guess we need to decide if any reference equality is acceptable, or if we simply need to clone in such cases.

@paldepind:
One other thing.  I like your general approach, but I'm not sure how you mean to deal with this:

Sparse arrays are not handled.

Does this go away if you stop worrying about the first array?  Or is it something that you still want to deal with for the second one?  Obviously you can detect sparse arrays: loop through the indices up to the array length and test idx in arr.  But it seems a bit of overkill in general.

@ivan-kleshnin

That means Lodash treats Arrays like Objects and offsets like keys in this exact case. This is understandable only from the view of JS implementation. But is weird from all other point of views. Lodash just totally lost Array (the only ordered collection!) concept by this decision.

lodash iterate arrays as a list not as an object, meaning it ignores non-index properties in the merge.
@CrossEye

(Note especially the undefined first element. Even though '0' in b.tags; //=> false, its non-existent element still overrides the one from a.tags.)

I would find simply replacing the first array with the second and not descending into it to be a bit more sane.
lodash treats sparse arrays as dense in it's methods just as Ramda does in its forEach.

lodash treats sparse arrays as dense in its methods just as Ramda does in its forEach and others.

Yes, understood.  lodash clearly has the most sophisticated merge around.  I haven't actually dug into the implementation, but I have played with it a bit, and I'm very impressed, especially with how it handles complex cycle-detection.  In the example I cited above, a = []; b = []; a.push(b); b.push(a), lodash handles it with aplomb, announcing a == b; //=> true, which seems like the best possible result.
But odd edge-cases like the merging of sparse arrays into others makes me suspect that for Ramda we might want a little less sophistication.

@CrossEye

I guess I really don't know what that means. I can imagine two different implementations of a sequence, each with an add method. One pushes data to the beginning of the list, the other to the end. By your suggestion, they should be reported as equal if they both currently contain the sequence 1, 2, 3. But then they lose the notion of substitutability, which is really what equality should be all about. For -- with some appropriately dispatching add function -- add(4, a); //=> (4, 1, 2, 3) and add(4, b); //=> (1, 2, 3, 4). I don't understand this idea of super-type equality.

That's a very keen observation. As I already mentioned in Clojure Vectors ("binary" trees) and Lists (linked lists) are treated as equals e.g.
(= `(1 2 3) [1 2 3]) ; true

There is a function conj which adds first element to List and last to Vector.
And there are a bunch of operations in Clojure which change input type (mostly they accept any sequence and always return List). So basically in Clojure situation where
x == y
fn(x) != fn(y)

is very much possible. I'm not a Clojure expert but I asked Clojurers how they deal with it and do they see more benefits or drawbacks with it. Basically, they say there are just a tiny set of functions where you need to be sure about exact datatype you deal with. There you do explicit type convertion.
Most of the times, though, it's just mapping / filtering and letting Clojure decide about datatype is rather cool. They mostly start with a Vector, pass data through all the layers letting it to be autoconverted to List and convert to Vector only where that's really required.
While I agree that sounds a bit scary, they ensure me it's quite unobtrusive and becomes second nature very fast.
@paldepind

I find it a bit odd that you initially told me that just doing compose in JavaScript was freaky.

I haven't said that about compose. It was more about avoiding this in the first place if you care.

I think he is talking about different physical implementations of the same abstract data structures. For instance an ordered list is an abstract data structure. A physical implementation could be an array, a linked list or the kind of trees that Clojure uses. Providing value equality across such data structures is a solid feature IMO.

Yes. Not only an equality. There should be common operators.
Why even newer JS Map has no filter / map / reduce method combo?
Isn't it just bad?

Equality by value is important. But pointer equality is a part of JavaScript.

What I can't get is really why not just make eq(deep) and shallowEq
instead of eq(shallow) and eqDeep?

@CrossEye

Ramda wants to make a functional style easier in Javascript. But it's not trying to replace Javascript with a more functionally pure language. There are some great languages that do so, things like PureScript, [ClojureScript][cs], and Elm. But that's not Ramda's place.

Fair deal. I mostly want to define this limit you will not cross in our discussion. To make it clear for me.
Not to push it in any direction.

@ivan-kleshnin

Ok, let me be more precise. What I can't get is not that you have two comparison functions.
It's more about why not make eq (deep) and shallowEq instead of eq (shallow) and eqDeep?

It's an interesting suggestion.  I can think of one major reason not to do this:  Right now, add is a functional equivalent of the operator '+'; gt, of '>', and of '&&', and similarly too for subtract, multiply, divide, or, not, lt, modulo, and, so too, for now is eq a functional equivalent of '==='.  I find it likely that more users would expect this similar behavior from eq than would expect a value-equality function.

I haven't said that about compose. It was more about avoiding this in the first place if you care.

Ok. But you expressed it right after I showed a simple example with compose.

Yes. Not only an equality. There should be common operators.

Of course. Many programming languages have this.

Ok, let me be more precise. What I can't get is not that you have two comparison functions.
It's more about why not make eq (deep) and shallowEq
instead of eq (shallow) and eqDeep?

Because there are two different types of equality in JavaScript. Pointer equality and value equality. And I agree with CrossEye that the current behavior is what people expect since it maps to the === operator.
@CrossEye

Probably. I guess we need to decide if any reference equality is acceptable, or if we simply need to clone in such cases.

I've changed my mind. If we're simply overriding arrays in the first object with arrays in the second object then just copying them over seems reasonable. In that case handling sparse arrays is trivial.

In that case handling sparse arrays is trivial.

If you handle them in one place, you should probably handle them in others.

If you handle them in one place, you should probably handle them in others.

A function that doesn't touch arrays but merely assigns them somewhere else obviously handles sparse arrays. R.always handles sparse arrays as well. But that's very different from functions that actually does something with the arrays.

But that's very different from functions that actually does something with the arrays.

I think, merging is doing something with the arrays. Something like handling sparse arrays is easy to document and handle in a blanket way. By saying there is essentially no sparse arrays (treating them as dense) it can avoid the baggage with them and punt on them in forEach, slice, filter, contains, etc.

I think, merging is doing something with the arrays.

Did you read the foregoing discussion? This is not the same as _.merge. The mentioned merge behavior is exactly not doing touching the arrays. Unless you consider assigning an array to a property to be doing something with an array.

Ok. But you expressed it right after I showed a simple example with compose.

Yes. Because of cumulative effect and a chat limitations.

It's an interesting suggestion. I can think of one major reason not to do this: Right now, add is a functional equivalent of the operator '+'; gt, of '>', and of '&&', and similarly too for subtract, multiply, divide, or, not, lt, modulo, and, so too, for now is eq a functional equivalent of '==='. I find it likely that more users would expect this similar behavior from eq than would expect a value-equality function.

Ok, this is a strong counter-argument. I believe this correspondance should be mentioned in docs then.

Because there are two different types of equality in JavaScript. Pointer equality and value equality. And I agree with CrossEye that the current behavior is what people expect since it maps to the === operator.

And (at least) four are possible. Imagine two clocks. One counting clockwise and other – counterclockwise. There are 4 possible and valid questions about the clocks which may be called "equality questions".

Do they show the same time right now?
Do they share the same behavior?
Do they have the same manufacturer and model?
Are they actually one physical clock?

First question is of highest abstraction level. You just need to look at the numbers. Second question is on lower level and so on.
What abstraction(s) to rely on for programming language? From practice we know that higher-level languages give us more productivity:
1. Do they show the same time right now? :: ==
2. Do they share the same behavior? :: objects are immutable, so questions 2 and 3 are the same
3. Do they have the same manufacturer and model? :: type
4. Are they actually one physical clock? :: Fallback to Java ops

JS fails miserably to model this situation:
1. Do they show the same time right now? :: nothing like that
2. Do they share the same behavior? :: == vs ===
3. Do they have the same manufacturer and model? :: typeof vs instanceof
4. Are they actually one physical clock? :: nothing like that

@ivan-kleshnin
That analogy seems very arbitrary. There's a thousand ways two clocks can be "equal" (do they have the same color?). In a language that's not purely functional pointer equality is a thing. In a purely functional language only value equality makes sense. Your example only obscures that.

@paldepind

Did you read the foregoing discussion?

I've seen discussion over the behavior of assigning values in regards to sparse arrays. If you respect the holes in sparse arrays for merges then you should probably do the same for contains. For example R.contains(undefined, Array(1)) currently returns true. If R.contains says there's an undefined value in the array why wouldn't it be merged? Then there's also the baggage in older environments like IE < 9 which will treat undefined values in array literals as holes (ex:  0 in [undefined] is false) which isn't the case in others.

That analogy seems very arbitrary. There's a thousand ways two clocks can be "equal" (do they have the same color?).

I don't think so. They are not just questions but types of questions. You may change exact sentences staying withing the same structure.
"color" is either an implicit characteristic if values are truly immutable (same as question 3) or explicit if values are mutable (same as question 2).

In a purely functional language only value equality makes sense.

I demonstrated that at least four valid questions can be applied to any language. It's just JS fails to follow this.
Python
1. Do they show the same time right now? (nothing like that)
2. Do they share the same behavior? (==)
3. Do they have the same manufacturer and model? (isinstance, type)
4. Are they actually one physical clock? (id)

I demonstrated that at least four types of valid questions can be applied to any language.

Just because you can ask the questions doesn't mean they make any sense to ask in any given language.
1/ That is value equality. That probably applies to all languages yes. 2/ Values don't have behavior and == certainly don't test for behavior. Duck typing only applies to objects. So it does not make sense in any language. 3/ Not all languages have a concept of type. 4/ In a purely functional languages you don't have pointer equality and thus your 4. type doesn't apply.

@jdalton:

Did you read the foregoing discussion?

I've seen discussion over the behavior of assigning values of sparse arrays. What I'm saying is that if Ramda respects the holes in sparse arrays for merges then it should probably do the same for others methods...

What @paldepind probably meant was, "Did you notice that we were discussing not merging arrays, sparse or not, in this process?"  The suggestion was to simply overwrite them with the new one.
Ramda's (totally unenforceable) point of view is mostly that we work with lists.  Since there is no actual list type in JS, we lean on dense arrays.  While we don't actually test to see that an array you pass us is dense, all bets are off if it's not.  You've pried off the cover and voided the warranty.  In this case, a property of an object to be merged, there's less of an issue, especially now that we seem to be leaning toward not iterating its properties.

Just because you can ask the questions doesn't mean they make any sense to ask in any given language. 1/ That is value equality. That probably applies to all languages yes. 2/ Values don't have behavior and == certainly don't test for behavior. Duck typing only applies to objects. So it does not make sense in any language. 3/ Not all languages have a concept of type. 4/ In a purely functional languages you don't have pointer equality and thus your 4. type doesn't apply.

Ok, I surrender. You're trying to look at the reality inside-out, through programming language experience. I proposed to do the opposite – see on language ouside-in e.g. from reality and real-world abstractions to our PL primitives. Probably it will take too much words.

What @paldepind probably meant was, "Did you notice that we were discussing not merging arrays, sparse or not, in this process?" The suggestion was to simply overwrite them with the new one.

Ah, I missed that in the walls of text.

Ramda's (totally unenforceable) point of view is mostly that we work with lists. Since there is no actual list type in JS, we lean on dense arrays. While we don't actually test to see that an array you pass us is dense, all bets are off if it's not.

I've found that sparse array issues pop up at a higher rate than object iteration order issues. I donno, maybe that will help weight it in terms of something Ramda is concerned with. If you were to tackle them you can look towards ES6 which is treating them as dense too, e.g. 0 in Array.from(Array(1)) is true.

In this case, a property of an object to be merged, there's less of an issue, especially now that we seem to be leaning toward not iterating its properties.

If assignment is the goal of the method maybe naming it something closer to the lang, like assign, is a better route :bike: :house:.

If assignment is the goal of the method maybe naming it something closer to the lang, like assignDeep, is a better route :bike: :house:.

Assign is somewhat misplaced in functional library I think. It merges properties from two objects into one. I think the name makes sense.

@jdalton:

I've found that sparse array issues pop up at a higher rate than object iteration order issues.

I'm guessing you're speaking mostly in the context of lodash here, correct?  If so, then I think Ramda has an advantage in that it's not trying to be as general-purpose as lodash is.  If a use-case strays too far from our expected ones, we feel more comfortable simply saying, "Then don't do that."  We've simply not found any reason to work with sparse arrays, which do not map well to lists.  In the context of something like merge, it's a little different, since we're not working with the object as a list but only as a collection of properties.  (It doesn't fit into the standard list-processing model of, "Process the first, then process the rest.")  In this case, we could decide to do something different.  But I haven't seen anything convincing.  lodash's version is extremely impressive overall, but the handling of sparse arrays here still feels off.  And there's nothing else close.  That's why I'm more than happy to punt on it.  If @ivan-kleshnin's reports above are accurate, many other languages also choose this route.

If you were to tackle them you can look towards ES6 which is treating them as dense too, e.g. 0 in Array.from(Array(1)) is true.

I think that's a bit of an exaggeration.  It's simply that Array.from is based on iterators.  There is nothing that removes sparse arrays or automatically assigns the indices to them.  Iterators will act as though there are undefined values there, just as an ES3/5 for loop that simply addresses arr[idx] for an unassigned indices will act as though there  are undefined values there.
In general, I feel no reason to consider tackling them.  I don't know if others feel any differently.

I think that's a bit of an exaggeration.

Generally sparse arrays are dead. Anywho it's your call.

If assignment is the goal of the method maybe naming it something closer to the lang, like assignDeep, is a better route :bike: :house:.

Assign is somewhat misplaced in functional library I think. It merges properties from two objects into one. I think the name makes sense.

And I really wish we'd just stuck with mixin back when...  Anyone for extend?

For the sake of completeness I will mentioned another Lodashdesign flaw that tripped me up today.
It's related to what was discussed, so I just leave it there for history.
Lodash does not see the difference between missing key and value == undefined.
import Ramda from "ramda";
import Lodash from "lodash";

console.log(
  Ramda.merge({}, {name: undefined}) // {name: undefined} -- expected
  Ramda.merge({name: "foo"}, {name: undefined}) // {name: undefined} -- expected
);

console.log(
  Lodash.merge({name: "foo"}, {name: undefined}) // {} -- wtf?
  Lodash.merge({name: "foo"}, {name: undefined}) // {name: "foo"} -- wtf?
);

Again, we can speculate about design decisions and opinionated, does not mean broken but for me it's getting clear that Lodash does not follow established conventions about Arrays and Objects which mean my personal farewell to this lib.

Lodash does not see the difference between missing key and value == undefined and does not treat undefined the same as other values.

So file a bug.

Lodash does not follow established conventions about Arrays and Objects which means my personal farewell to this lib.

So long.

You could always customize the merge behavior to taste with a customizer callback

Not in this case :disappointed:  If you return undefined from callback default behavior occurs:
Lodash.merge({name: "xxx"}, {name: undefined}, function(a, b) {
  return b;
}); // {name: "xxx"}

This aspect is done right in deep-merge by @Raynos
import DeepMerge from "deep-merge";

let merge = DeepMerge(function mergeStrategy(a, b, key) {
  return b;
});

merge({name: "xxx"}, {name: undefined}, function(a, b) {
  return b;
}); // {name: undefined}

Not in this case :disappointed: If you return undefined from callback default behavior occurs

Ah, yap :P
Patched 882d84f1. So
_.merge([1], [undefined]);
// => [1]

Besides aligning with defaults the merge behavior of undefined aligns with jQuery
$.extend(true, [1], [undefined])
=> [1]

$.extend(true, { 'a': 1 }, { 'a': undefined })
// => {a: 1}

I'm satisfied with lodash's merge behavior. It goes above and beyond in terms of environment support and trying to accommodate developers varying opinions.

@ivan-kleshnin:
While this is a fine place to describe differences between Ramda and lodash (or any other library), it's probably an inappropriate place to take on a sustained critique of any library except for Ramda.
I actually have quite mixed feelings about the behavior in this specific case, and I think it's because Javascript itself is fairly schizophrenic about the meaning of undefined.  While it is a value (otherwise the language have to throw an error on accessing it), it is specifically the value that declares that "no one has defined a value here"; it's more severe than null, which that says "hey, there's nothing here yet".  So I'm not at all certain that there should be any real difference in handling of {foo: 1} and {foo: 1, bar: undefined}.
Several people in a FantasyLand discussion may have convinced me of the need to deal with any value, however obscure, as a value in its own right, but something still nags at me in this case.

Fixed by #1088

So I'm not at all certain that there should be any real difference in handling of {foo: 1} and {foo: 1, bar: undefined}.

But there if a very real difference:
'bar' in {foo: 1}
> false
'bar' in {foo: 1, bar: undefined}
> true

Nice example, @paldepind! Two objects must have the same keys in order to be considered equal.

So I'm not at all certain that there should be any real difference in handling of {foo: 1} and {foo: 1, bar: undefined}.

But there if a very real difference:

I recognize that they're distinguishable.  The point was whether we decide to distinguish them.  As I said, some discussion (that I'm too lazy to track down) on FantasyLand demonstrated that there are times when we cannot avoid choosing to use undefined and/or null as values no different from any other.  But especially for undefined, I think there are also good arguments against it at least in some cases.

Objects in JavaScript serve two roles: they are sometimes used as faux dictionaries and other times are used as faux records.
When an object is used as a record, my mental model is as follows:

the object's property names are the object's most significant characteristic;
less significant is the type of each property; and
least significant is the value of each property.

I'm not arguing that this is the right way to think about record-like objects. :)

The point was whether we decide to distinguish them.

How do you think I discover those issues in Lodash? By inspecting their code with microscope or by facing real troubles with by business-logic layer :smile:? They cause a lot of practical troubles.
For example, my example was resetting of filters. If you treat undefined values as missing keys you just can't reset the filters by setting key value to undefined via merging. This means you need to create another logic branch especially for resets.

I'm not arguing that this is the right way to think about record-like objects. :)

I would like to point out that in a way this is also how the JavaScript VM thinks of objects. If a group of objects has the same keys defined in the same order they will share the same hidden class. Basically, if you use objects as record the optimizer will turn it into something a bit like a C struct. Again, here it is the key and the order of the keys that is most significant.

There seem to be three defensible possible outputs for this:
merge({foo: 1, bar: 2}, {bar: undefined});

{foo: 1, bar: 2}, the current lodash result
{foo: 1, bar: undefined}, the current Ramda result
{foo: 1}, not seen in the wild but certainly arguable

None seems to me to have an overwhelming advantage.  While I do respect David's understanding of objects as records, Ramda has no particular reason to choose that interpretation of the objects passed to it over a dictionary one.
I'm not really suggesting any changes here.  I like what Simon has done with set and don't see any real reason to revisit these things.  But let's not pretend that we've made the single obvious choice in these matters.

I don't get it. If an object has key that is undefined then that undefined has deliberately been put there. The programmer has actively taken the decision to set it to undefined. If you don't want a key you delete it.
How can merge({foo: 1, bar: 2}, {bar: undefined}) turn into {foo: 1}? I have a named key that I've deliberately set to undefined and the library just throws it away? By that logic it seems like this would be expectable behavior as well:
R.keys({foo: 1, bar: undefined}); //=> ['foo']

And then you're clearly disagreeing with JavaScript.

And then you're clearly disagreeing with JavaScript.

You say that like it's a bad thing! I think what @CrossEye is driving at is, what does this mean: {foo: undefined} and apart from the for .. in test, how is that different from {}? It's not at all clear that the meaning is different.

You say that like it's a bad thing! I think what @CrossEye is driving at is, what does this mean: {foo: undefined} and apart from the for .. in test, how is that different from {}? It's not at all clear that the meaning is different.

It means the future of JS is even darker than I was afraid.
By treating undefined values in a special way you do very similar thing Brendan Eich did with null addition: create entirely new edge cases, create unnecessary complexity out of thin air. I'm pretty shocked that someone does not see the difference between missing keys and guard values. Or never met cases where it ruined data-flow.

:ghost:

@ivan-kleshnin for the record, I am not advocating this position. I am simply suggesting -- since @CrossEye enumerated three candidate solutions and @paldepind objected to one of them --  simply suggesting that one could start arguing from there. See the difference?

See the difference?

Yes. I dont want to start a new rant.

closed by #1088

The point is that a user may not have put the value there intentionally.  This is what the language does, and what therefore Ramda does too in some functions, to deal with cases of "Well, I don't have the thing you're looking for, but I need to put a value here."
It is that sense of the language that makes undefined trickier than it might seem.
    var overrides = {foo: 1, baar: 2, baz: 3. corge: 4};
    var important = ['foo', 'bar', 'baz'];

    var current = {foo: 10, bar: 20, qux: 30};
    var final = R.merge(current, R.pickAll(important, overrides));

Here final in Ramda is {foo: 1, bar: undefined, baz: 3, qux: 30}.  In lodash, it would presumably be {foo: 1, bar: 20, baz: 3, qux: 30}.  And here it's arguable that such would be the preferred choice.  I wanted the values in current, overridden by those specific values of overrides listed by name in important.  The whole undefined thing becomes in this scenario is simply an unfortunate accident.  The delete-on-undefined possiblity is certainly less likely, but is clearly supportable, as it is related to the design of the language.
Of course if there was no typo in overrides this would be {foo: 1, bar: 2, baz: 3, qux: 30}.
Still, this is all academic.  We have to make a choice, and I think we've made ours for Ramda.  I'm not unhappy with it.  I just don't like pretending that it was a clear, unambiguous choice.

@CrossEye I actually think that the current result from Ramda is the desired one.

The point is that a user may not have put the value there intentionally.

Using R.pickAll instead of R.pick is in my opinion intentionally saying "I want keys with undefined values to appear". Getting undefined keys is the whole point of R.pickAll! With the other merge behavior this:
var final = R.merge(current, R.pick(important, overrides));

Would be exactly the same as this (pick replaced with pickAll):
var final = R.merge(current, R.pickAll(important, overrides));

Which seems bad because merge does not respect the users choice of using pickAll and then there's suddenly no way to get the original behavior.
My mental model of merge is something like "create new object, copy keys from first object to new object and then copy keys from second object to new object". Nothing in that mental model considers the actual values at the keys. If you write a merge that does consider the values then your adding a special case and thus undeniably creating a more complex merge. This would IMO also be a harder to understand merge.

The point is that a user may not have put the value there intentionally

If you could show an example of that I might actually be convinced. But I'd still say that undefined keys should be respected because the user has intentionally written {foo: undefined} instead of {}, or written obj.foo = undefined instead of delete obj.foo or used pickAll instead of pick.
I understand that this theoretical and not a discussion about how Ramda should actually implement things. I think this is an interesting discussion but I would also be understanding if you do not wish to discuss this further.

The point is that a user may not have put the value there intentionally.

Using R.pickAll instead of R.pick is in my opinion intentionally saying "I want keys with undefined values to appear". Getting undefined keys is the whole point of R.pickAll!

  I coded something like this:
    var choose = function(names, obj) {
      return R.fromPairs(R.zip(names, R.map(R.prop(R.__, obj), names)));
    };

and started writing up my previous message with this in place when I realized that it was equivalent to R.pickAll.  I didn't think about the fact that the actual function was intended to specifically deal with undefined.  But the point is that one can get undefined from the language in all sorts of ways.
I agree that this is an interesting discussion, even if it doesn't affect how we proceed with Ramda for now.  I think the trouble is that how JavaScript works, and what it constructs mean here simply don't play well with standard practice from the FP world.  A library trying to bridge these two worlds has to occasionally deal with such issues.

Ok. Then your example makes more sense. But I could still argue that the choose function is explicitly written in a way that makes these undefined keys appear and thus they must have meaning. And if those undefined keys where not intended then the problem is in the definition of choose.
Here is an attempt at a counter example. Consider data like this (from code I wrote earlier today):
var model = {selected: undefined, items: [/* some items */]};

Here model.selected is an index into model.items when an element is selected and undefined when an element is not selected. Clearly the value undefined here has as much meaning as if model.selected had been a number. So if a library in some cases treated the undefined as a not being an actual value and threw it away in favor of something else then the correctness of my data would not be preserved.

Here model.selected is an index into model.items when an element is selected and undefined when an element is not selected.

To me, that smacks of abuse of the language.  null would be appropriate here.  For without some obnoxious gymnastics (if ('selected' in model && ...) ) there is no way to distinguish the case where the user intentionally put an undefined value on the model and one where the model was accidentally left incomplete, or where it had a typo:
var model = {selceted: 4, items: [/* some items */]}; // TYPO
model.selected; //=> `undefined`

So I guess that's part of the problem.  I feel that using undefined for anything other than "hey, no one ever told me about such a property" is abusing the semantics of the language.  But I also feel that from the FP side, ever treating anything that can be assigned as a value as uniquely privileged is extremely problematic.  So I end up in a quandry.

Ok. That's probably a big part of the reason why we have different opinions of this. I never use null and I always use undefined in cases like described above. I don't know is that's abusing the semantics of the language. I've also programmed in JS that way and I've never heard that is should be "bad".

I never use null and I always use undefined in cases like described above.

Me too. null should be removed from the language (by making it strict equal to undefined and gradual deprecation). If I'm not wrong the only place where null is used is DOM. Native JS never returns it. And DOM will be replaced with VDOM as we all hope. So the opportunity is real.

I don't know is that's abusing the semantics of the language. I've also programmed in JS that way and I've never heard that is should be "bad".

I believe that putting some semathics in null vs undefined weird difference is real abuse.

I guess my main reason for not using undefined as a value this way is that it does not allow me to distinguish between accidentally skipping a property (or having a typo) and intentionally using the undefined value for it, except with the very awkward prop in object, which I rarely see people use.
So it would not exactly be abuse in my mind if the user does choose to use in this way, but it also seems odd to me when the language has another perfectly good value in null which doesn't share this problem.

I believe that putting some semathics in null vs undefined weird difference is real abuse.

ISTM that the language already does so.  This would simply be a matter of using the language the way it was designed.  I'm not claiming that this was one of the best-designed parts of JS, but it is what we have; I've seen no great reason to work against the grain on it.

So it would not exactly be abuse in my mind if the user does choose to use in this way, but it also seems odd to me when the language has another perfectly good value in null which doesn't share this problem.

It seems to me like the design of Ramda itself disagrees with you.
R.head([]); // => `undefined`
R.find(R.lt(20), [1,2,3]) // => `undefined`
R.nth(2, ['only me']); // => `undefined`

If you assign the returned value of any of these functions to an object you have the problem you're talking about.

It seems to me like the design of Ramda itself disagrees with you.
R.head([]); // => `undefined`
R.find(R.lt(20), [1,2,3]) // => `undefined`
R.nth(2, ['only me']); // => `undefined`

If you assign the returned value of any of these functions to an object you have the problem you're talking about.

Interesting.  I find that at least the first and third examples absolutely support what I'm talking about.  The second one is more questionable, as Ramda has more of a choice in deciding what to report for the result of a find when the value is not there, but undefined still seems quite reasonable.  The point is "if you assign" this to an object.  Ramda does not provide that level of error-checking for you.  A library like Sancutary is a great companion for things like this.
The problem is when you make the request, "get me such-and-such", a language will return the value to you, unless such-and-such doesn't exist.  Then the language can do one of several things.  It can generate an exception.  (PHP does this, I believe.)  Or it can return a token to say "hey that wasn't here."  In LISP that token also serves as the empty list, which is incredibly convenient.  In some languages, there is a single (e.g. null) token, which then is often is mostly useless as a value in its own right, but which can be assigned to a reference.  Javascript has two, with somewhat different meanings.  null means that the reference you were looking for (say obj.foo) has been defined but not assigned to.  undefined means that the reference you were looking for has not been defined.  (obj has no foo property.)
Obviously one can choose to ignore this distinction and treat null and undefined as essentially identical.  Or one can go the Sanctuary route and add error-checking so that no null/undefined references propagate.  Ramda has mostly chosen to stick closer to the core of what Javascript does.  I don't feel that this is set in stone, but it's been long-standing in the library and it would take some serious persuasion to change it, I believe.

Javascript has two, with somewhat different meanings. null means that the reference you were looking for (say obj.foo) has been defined but not assigned to. undefined means that the reference you were looking for has not been defined. (obj has no foo property.)

I better unbsubscribe from this thread to protect my sanity.

At this point I better unbsubscribe.

Up to you, of course.  I know you're not happy with Ramda living with some of the quirks of the language.  But that's how it's always worked.  It is a Javascript library, and not an attempt to build Clojure, Haskell, Erlang, or some other more functional language inside JS.

null means that the reference you were looking for (say obj.foo) has been defined but not assigned to. undefined means that the reference you were looking for has not been defined. (obj has no foo property.)

I have not heard of that distinction before. But what if I put a number in obj.foo? Then there is no reference since it's a primitive. If you treat null like a null pointer then you can only use it in places where a pointer, i.e. an object, is expected.
My point with my last post was that pretty much everything returns undefined as a value meaning "nothing to see here" (and then there are the odd cases of -1 and null). Thus, if I wanted to use null in objects for meaning "nothing to see" here I'd have to do a lot of converting myself.
Considering my example again:
var model = {selected: undefined, items: [/* some items */]};

If I where doing what you suggest and used null instead of undefined to represent "nothing is selected". Then I, for instance, couldn't do this
model.selected = R.head(model.items); // select first item if any

I'd instead have to do this
model.selected = R.head(model.items) === undefined ? null : R.head(model.items);

That's really annoying and I'd had to write even more if a wanted to avoid two invocations of R.head.

FWIW (I did not come close to reading this whole thread) I routinely use and see used null and avoid undefined knowing that JavaScript as a language semantically reserves by convention undefined to represent nothingness whereas APIs by convention null to represent nothingness.

I have not heard of that distinction before

@paldepind I've recently found some background to supply @CrossEye opinion.
JSON.stringify({x: null}); // -> '{"x":null}'
JSON.stringify({x: undefined}); // ->  '{}'

This is very dangerous and wonky behavior of language, but that's how it is...
My opinion about merge didn't change, though.

@ivan-kleshnin

This is very dangerous and wonky behavior of language, but that's how it is...

I think it has more to do with the fact that JSON has null but not undefined.

@paldepind they could convert undefined to null like they do with NaN.

I've recently found some background to supply @CrossEye opinion.
JSON.stringify({x: null});      // '{"x":null}'
JSON.stringify({x: undefined}); // '{}'

This is very dangerous and wonky behavior of language, but that's how it is...

While this does offer some minor support to the position I've taken, the main reason for this is that JSON was explicitly designed to be extremely easy to port to almost any languages.  Most languages have some notion of null (regardless of whether they're now seen as the billion dollar mistake) and Javascript's null was the clear map for that notion.  There's simply no portable idea for undefined.

The idea behind undefined can be expressed as "it's possible to use undefined variables, their value is... undefined". Most languages don't allow that.
Because such value exist and does not exist at the same time.
> Object.keys({}).length ==Object.keys({x: undefined}).length
false
> JSON.stringify({}) == JSON.stringify({x: undefined})
true

This is a mess which can cause (and causes) a lot of bugs.

Most languages don't allow that.

For example:
❯ ghci
GHCi, version 7.10.2: http://www.haskell.org/ghc/  :? for help
Prelude> [1,2,3,undefined]
[1,2,3,*** Exception: Prelude.undefined

The idea behind undefined can be expressed as "it's possible to use undefined variables, their value is... undefined"

Maybe it amounts to the same thing, but I would express it very differently: "Rather than throw when you try to access a value that hasn't been set, the language returns the signal value undefined."  There are two somewhat palatable alternatives to this.  First, the language could offer an operator like PHP's isset and then require anyone accessing a possibly undefined value to check this or accept the risk of a thrown exception.  Second, the language could do something akin to what Sanctuary does, creating a wrapper around potential missing value, returning the appropriate wrapped or signal (Nothing) value for further processing.  Neither of these alternatives thrills me.  The only other widespread possibility is the single null value that causes such a problem in Java and C#.  Maybe it's better, but I'm certainly not convinced.

Most languages don't allow that.

For example:

[ ... Haskell code ...]

There is, however, a big difference between how Haskell and JS use their undefined value.  The one in JS is the standard means in the language of simply reporting, "Hey, that thing you're looking for?  It ain't there!"  Haskell generally uses Nothing for that.

The one in JS is the standard means in the language of simply reporting, "Hey, that thing you're looking for? It ain't there!" Haskell generally uses Nothing for that.

I think I disagree with your definition.
undefined is a language-level concept of non-existence, e.g.:
> typeof a
'undefined'

> ({}).a
undefined

> ((a) => console.log(a))()
undefined

> ((a = 1) => console.log(a))()
1

Whereas null is a user-level concept of non-existence:
> ((a = 1) => console.log(a))(null) // Default **does not apply**
null

null in JavaScript is Nothing in Haskell, and so on.


ModuleNotFoundError: No module named 'chatterbot_corpus'

I install the ChatterBot and write a simple program follow the tutorial, but when I run it, it show the error below:
Traceback (most recent call last):
  File "/home/simon/Documents/Files/Coding/Coding Files/Python/Daily/Day 65/Chatter Bot.py", line 1, in <module>
    from chatterbot import ChatBot
  File "/home/simon/.local/lib/python3.8/site-packages/chatterbot/__init__.py", line 4, in <module>
    from .chatterbot import ChatBot
  File "/home/simon/.local/lib/python3.8/site-packages/chatterbot/chatterbot.py", line 2, in <module>
    from chatterbot.storage import StorageAdapter
  File "/home/simon/.local/lib/python3.8/site-packages/chatterbot/storage/__init__.py", line 1, in <module>
    from chatterbot.storage.storage_adapter import StorageAdapter
  File "/home/simon/.local/lib/python3.8/site-packages/chatterbot/storage/storage_adapter.py", line 3, in <module>
    from chatterbot.tagging import PosHypernymTagger
  File "/home/simon/.local/lib/python3.8/site-packages/chatterbot/tagging.py", line 4, in <module>
    from chatterbot.tokenizers import get_sentence_tokenizer
  File "/home/simon/.local/lib/python3.8/site-packages/chatterbot/tokenizers.py", line 4, in <module>
    from chatterbot.corpus import load_corpus, list_corpus_files
  File "/home/simon/.local/lib/python3.8/site-packages/chatterbot/corpus.py", line 5, in <module>
    from chatterbot_corpus.corpus import DATA_DIRECTORY
ModuleNotFoundError: No module named 'chatterbot_corpus'

I'm no idea why it shows this error because ChatterBot is installed successfully.

What version have you installed?
It seems it has been dealt with in the latest stable version (1.0.8)
https://github.com/gunthercox/ChatterBot/blob/1.0.8/chatterbot/corpus.py#L8

What is the solution?


Paginator docs

Add docstrings to paginator's paginate method.  It also renames the paginator class returned from client to a class that falls in line with what is in the documentation e.g. S3.Paginator.list_objects.
In [1]: import botocore.session

In [2]: session = botocore.session.get_session()

In [3]: client = session.create_client('s3')

In [4]: paginator = client.get_paginator('list_objects')

In [5]: paginator.__class__.__name__
Out[5]: 'S3.Paginator.list_objects'

In [6]: help(paginator.paginate)

I realize that I have a lot of PRs out right now and many depend on the other. This is the order in which how I would like to see them reviewed/merged:

https://github.com/boto/botocore/pull/642
https://github.com/boto/botocore/pull/643
This one
https://github.com/boto/boto3/pull/239

Or you can just review this one, which accounts for 1 through 3.
cc @jamesls @mtdowling @rayluo

I primarily have the same question as the waiter doc PR.  I'm not sure if we want to create a new class on every single call to get_paginator.  I know do that for session.create_client, but I would envision a paginator/waiter within a client to be called much more than a create_client call.

Doing some performance testing with the paginators as well. Here are some preliminary results in terms of speed. I used this script:
import time
import botocore.session

session = botocore.session.get_session()
s3 = session.create_client('s3')

total = 0
iterations = 5
for i in range(iterations):
    start_time = time.time()
    s3.get_paginator('list_objects')
    end_time = time.time()
    diff = end_time - start_time
    print('%s round instantiation: %s seconds' % (i, diff))
    total += diff

average = total/iterations
print('Average %f seconds' % average)

For the develop branch, I got these results:
0 round instantiation: 0.00303792953491 seconds
1 round instantiation: 3.79085540771e-05 seconds
2 round instantiation: 2.40802764893e-05 seconds
3 round instantiation: 3.19480895996e-05 seconds
4 round instantiation: 4.10079956055e-05 seconds
Average 0.000635 seconds

For my PR I got:
0 round instantiation: 0.0030620098114 seconds
1 round instantiation: 8.98838043213e-05 seconds
2 round instantiation: 6.29425048828e-05 seconds
3 round instantiation: 6.69956207275e-05 seconds
4 round instantiation: 6.00814819336e-05 seconds
Average 0.000668 seconds

So in terms of speed it looks like it was only affected by a little.

So I ran some memory leak tests similiarly to what I did for the waiter docstring pull request. Here is my results for s3 ListObjects paginator.
I ran each test five times and took the average difference between the end memory usage and the beginning memory usage (i.e. end - start):
develop test_create_single_paginator_memory_constant: 4.46 MB
develop test_create_memory_paginators_in_loop: 1.63 MB
PR test_create_single_paginator_memory_constant: 4.04 MB
PR test_create_memory_paginators_in_loop: 1.93 MB
Surprisingly the memory leakage decreased for one of the tests using my PR. Similarly to the Waiter docstring PR, I do not think we need to cache the Paginator class. Thoughts?

:shipit:


Added pattern support + merged with file deletion branch

Added pattern support. Now you can remove messages using some specific regex pattern.

Hi @5ap , many thanks for the PR. Having regex support is just awesome 👍
I've merged #14 manually, could you rebase your commits (without those from #14) onto the current master? So I can merge them and make the commit history more clear. Thank you a lot! 😄

I'm sorry, but how can I do this? :)


Add image reshaping for statically reshaped OpenVINO model

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Cow: Watch muabanlaptopcuhcm.com

Cow requests the watch of the watch_keyword muabanlaptopcuhcm\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by Jeff Schaller in Charcoal HQ


fix: upgrade npm to 3.x to fix node-sass postinstall

This should solve https://github.com/kingsreach/kingsreach-online/issues/14

Should also solve https://github.com/sass/node-sass/issues/1621

Yep. This fixed it.


Ignores data-gif attribute

URL in data-gif attribute seems to be ignored and the plugin simply attempts to load a file with .gif extension, replacing the .png extension in src attribute.
Looks like getGifSrc function is to blame.

I can only confirm. The data attribute has no function.

Hi , I just updated the plugin to fix this and other bugs, also include support for videos.
Please update. And thanks for your feedback.


Support for caret NISS notation

Ahoy! In order to prepare for some cubing ecosystem changes, I'm trying to promote compatibility with caret NISS notation: https://www.speedsolving.com/threads/caret-niss-notation-b-l-u-f2.88529/
Some FMC folks use parentheses, but unfortunately this is incompatible with various cubing software. Caret NISS notation solves this by using a ^ (caret symbol) before any "inverse scramble" moves in parentheses:
^(B L' U F2) // Nice start on inverse scramble
D' B' U B2 R2 // Nice continuation
^(R B R' U R' U2 R B) //F2L
^(U2 R' U2 R' D' L F2 L' D R2) // LL
// From https://fmcsolves.cubing.net/fmc_tutorial_ENG.pdf

I'd like to offer a change to Insertion Finder to accept caret NISS notation as input, which would allow such solutions to be e.g. pasted to and from Twizzle, and shared on speedsolving.com with an animated player. It wouldn't break any existing functionality — it would just allow a caret in front of (.
This seems to be fairly straightforward in the JS code, but I can only find the minified code. If compatibility with caret NISS notation is amenable, Is there a frontend codebase I could contribute to?

@xuanyan0x7c7 Any chance I could ask for your reasoning?
Without an alternative to regular parentheses, FMC notation will become steadily incompatible with other notation.
Caret NISS notation has been supported by Twizzle Editor for a while, and is supported by https://fmcworkshop.com/ as well.

@xuanyan0x7c7 Any chance I could ask for your reasoning?
Without an alternative to regular parentheses, FMC notation will become steadily incompatible with other notation. I'd like to ensure that algs/solutions can be freely pasted between tools where possible, since that lowers the cognitive overhead for cubers that are in the middle of exploring solutions.
Caret NISS notation has been supported by Twizzle Editor for a while, and is supported by https://fmcworkshop.com/ as well.

This is the core repo of insertionfinder, it only accept face moves as scrambles and skeletons, users should format algorithms before passing to the program.

@xuanyan0x7c7 Any chance I could ask for your reasoning?
Without an alternative to regular parentheses, FMC notation will become steadily incompatible with other notation. I'd like to ensure that algs/solutions can be freely pasted between tools where possible, since that lowers the cognitive overhead for cubers that are in the middle of exploring solutions.
Caret NISS notation has been supported by Twizzle Editor for a while, and is supported by https://fmcworkshop.com/ as well.

@lgarron If you are looking for the web tool insertion finder, it has been supported. Here's an example: https://333.fm/if/2b19f271932909005042bf4d3df6dc2c

@lgarron If you are looking for the web tool insertion finder, it has been supported. Here's an example: https://333.fm/if/2b19f271932909005042bf4d3df6dc2c

oooooh, I'm so glad to see it! 🤩


Fix for #3959

This is my fix for #3959.
There are two bugs racing each other here, which is why it sometimes crashes and sometimes the app just silently exists
Bug #1
When the window is recreated a Destroyed event arrives (due to the Drop of the old window). The code that receives this event does not look to see if its the main viewport or a secondary one and unconditionally closes the app. The code path for other platforms is slightly different and does check.
I have moved the code that handles the destroy to be in the same place and have the same behavior as the other platforms.
Bug #2
At recreate time the window and viewpoer entries of the glow integration are set to None. But the surface is still bound to the old window, this causes the next context switch to fail. So I sim ply added a gl_surface = None too,
This is my first egui PR so I hope I have not broken anything. If nothing else I understand a little better how egui works.

As part of my testing to make sure I didn't break anything I found that the same destroy behavior (bug #1) exists on Mac too. But my fix does not fix it. Investigating

fix works on mac - it showed the same behavior as bug1, immediate exit when recreating secondary viewport. Did no exhibit bug 2
I cannot get egui to build on linux so cannot test


Add payroll section to main menu

The human capital section often grows too big since payroll modules are added under it

Added.


checklist encoding errors

We need to update the checklists to avoid encoding errors produced by bad usage of certain special chars
An example of this can be seen in checklist ERC000022, on the "temperature" fields with the units; They used different chars for the "degree" character (º), which leads to errors when displaying validation errors.
A suggestion on how to improve this came from a meeting, to add checks on checklist generation

Not sure if still relevant and if in scope for 1st prod version.

The "temperature" fields in the mentioned checklist are:

mean seasonal temperature
mean annual temperature
temperature

On reflection,  I prefer the simplicity of MIxS’ YAML wordy approach although it looks less cool, is less error prone: “degree Celsius”. My preference would be to move all temperature to “degree Celsius” as the unit, then we continue to converge on the GSC MIxS standard where no good reason not to, which is our general strategy. (This continues to increase the interoperable nature of our metadata).  Would do you all think?

Enrique pointed out that "degree Celsius" is also the Unit Ontology's preferred term . It might make sense to do this switch.
This would be two parts:

Changing existing fields that use the special character to 'degree Celsius'
Check that doesn't break data

Post production task as emulates current behaviour


fix deprecated ssl directive

fixes mjstealey/wordpress-nginx-docker#12

+1
resolves deprecated ssl on; configuration setting


Some texts with the same font is undetected while others are detected

Hi,
I'm running into a very unusual problem. I have a bunch of images with the same font. I taught it a new font which improved the detection, however they are detected fine with the base eng.traineddata file as well. The problem is that the first image is detected just fine by tesseract but the second image is completely off. The  unusual part is if I crop part of the second image then it gets detected fine. This holds even if i crop the last 2 lines instead of the first 2. How do I deal with this issue?

EDIT: Let me know if you need more images to test with

updated version of charlesw link https://raw.githubusercontent.com/svn2github/Tesseract/master/doc/tesseract.1.html


Remove spirv optimization hack for  AMD drivers on macOS

macOS should use the Metal backend.

I'm closing this because this might cause some desktop apps to stop functioning. Some apps are relying on shared GL contexts.


unsupported type MONEY in macros with Postgres

Running a query on a Postgres DB table with a field of MONEY type gives the following error with the  query_as! macro. This happens despite the fact that the related field in the Struct field is set as "PgMoney" documented in https://docs.rs/sqlx/0.5.2/sqlx/postgres/types/struct.PgMoney.html.
unsupported type MONEY of column #5 ("paymentamount")
Am I missing something here?

Using the money type in PostgreSQL is a horrible idea.

+1 for moneytype!

First, I still contend this is deprecated and provides no real advantage. There is no reason to ever use this over numeric. But that aside, if the upstream devs acknowledge that that the "parsing from the text format is pretty sloppy" what's the plan to recreate the sloppyness here to know if something can be converted to money?

The PgMoney type in SQLx does not support text parsing (because it's vague and highly locale dependent). However, the type support does exist.
If you want to use the money type in macros today, all that would need to be done is SELECT money as "money: PgMoney", .... This is the type override syntax to tell the macros to use a specific type for a column that the macros wouldn't otherwise choose. For native support, that linked PR would need to be merged.
I also am firmly against the idea of the money type. It's about as bad as using the timestamp type (over timestamptz). Either store it as a numeric or store it as an integer of the lowest denomination. However, SQLx tries to not have an opinion for things like this.

Closed by #1218


StackTraceParser: Support StackTraces with 'http' but without the parentheses

Support for stacktraces without parentheses for the http filename

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: LazyTarget sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for your contribution!


[TASK] Send mails using a server

Estimated story points: 5
Description

They should be automatically sent by the supervisor

Acceptance criteria

time: 10h


[Gally]: master <- dev

Automatically created by Git-Ally

:tada: This PR is included in version 1.2.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


New Rochelle - reach out to local government

Stuart identified this person: https://www.newrochelleny.com/100/District-6
Stuart will email him.


refactor dmno server - reuse existing parent server if applicable

stop vite from watching files unnecessarily, if we are not in watch/dev mode

also reduces the number of files being watched when we are watching
note we are still always watching files from integrations (ex: astro) even if just running astro build, because when the dmno server is booted and dmno vite server initialized, we don't yet know if we are in build/dev mode

use uWebSockets.js instead of node-ipc (we were using a safe fork of it)
also use uWS as the web server for the dmno dev web ui
use mTLS for all communication over uWS
stop spawning a dmno dev as the server for vite/other plugins
dmno servers should detect an existing dmno server in a parent process and use that rather than spawning their own config loader
this means all communication with the dmno server must deal with serialized objects instead of accessing the actual objects in case we are using a parent server
collapse the "config server" and "config server client" into a single "dmno server" that is aware of a parent
dmno server launches its own "config loader" rather than getting one passed in
generally did an audit of all potential performance bottlenecks and tackled anything major and easy wins

Additionally worked through issues for a large monorepo user

vite was setting process.env.NODE_ENV, which treated it as an override
nextjs integration now has cjs build, so usable with cjs/ts config files
fixed issues with vite-node loading its own copy of dmno, which was breaking instanceof and ALS. Now we inject the current loaded instance into vite's module cache
introduced the concept of dmno errors being warnings which will not block validation
re-introduced useAt so we can ignore errors for items that are not being used in the current phase (ie something required but injected at boot time will only warn during build"
fixed an issue where dmno run was not passing through exit codes
fixed some timing issues when running many simultaneous dmno commands a big monorepo via turbo
upgraded turbo

few small things to fix

general cleanup, removing of console.log and timing messages
fix cache clear cli command

closing this - decided to split up the PR and clean it up a bit using gitbutler. See  #163 and #164.


Color picker (0.3.0) not working in Gatsby (3.0.1)

Color picker does not work with SSR React pages.
Color picker version: 0.3.0
Gatsby version: 3.0.1
React version: 17.0.1
Build error message:
 ERROR #95312

"window" is not available during server side rendering.

See our docs page for more info on this error: https://gatsby.dev/debug-html

  1 | export var onDrag =
  2 | // for IE, Edge, Firefox, Chrome
> 3 | 'PointerEvent' in window ?
    | ^
  4 |     function (element, onDragStart, onDragMove) {
  5 |         element.addEventListener('pointerdown', function (event) {
  6 |             if (event.button === 0 && onDragStart(event) !== false) {

Probably it should check for typeof window somewhere in code like this:
if (typeof window !== "undefined") {
  doSomething();
}

@primozpadar
Thank you for your report. It reproduced.
I fixed to not use window without checking if it exists, and published v0.3.1.
Could you please check if the problem has been fixed?

I think it works properly now! 😄
I tried to build project with gatsby build and also gatsby develop and it works as it should.

Thanks!
Closing.


@listens for multiple events

Expected behavior
I'd like @listens to support multiple events. There are generic event handlers which accept particular types of events, and it would be helpful to be able to enumerate them directly as part of @listens (rather than defining an event called "generic" and setting its type as a union of the individual events).
Also, while @fires seems to be permitted multiple times, it would seem to be beneficial to allow a union so that only a single @fires is needed for multiple firings.
Current behavior
Currently, the docs indicate only an "eventName" can be supplied as part of the tag.
Your environment

Software
Version

JSDoc
3.5.5

Node.js
9.2.1

npm
6.1.0

Operating system
Mac OSX

it will be great to allow using multiple listeners.
for example in our angularJS project we collect all watchers/listeners in one function. And currently is not possible to describe all this actions...


[2.1] Wait on ZooKeeper instead of Exhibitor during bootstrap

High-level description
Back-port of #7463
Wait for ZooKeeper to enter a suitable mode for the cluster, rather than waiting for Exhibitor.
Corresponding DC/OS tickets (required)

D2IQ-70393 Wait for ZooKeeper instead of Exhibitor in bootstrap
D2IQ-70419 Back-port bootstrap ZK wait

Related tickets (optional)

D2IQ-ID JIRA title / short description.

@mesosphere-mergebot request-review @jkoelker


PoseNet pretrained weights

Hi @noahzn, thank you for releasing the code for this paper! Would it be possible to share the pretrained weights of the pose networks? Currently, only the pretrained weights of the depth networks are provided. Thank you!

Hi, here are some related issues and you can download the weights.
https://github.com/noahzn/Lite-Mono/issues/111
https://github.com/noahzn/Lite-Mono/issues/92
https://github.com/noahzn/Lite-Mono/issues/65

Thank you!


Formalized UI / UX Audit

The UI needs a lot of love... It is almost like a developer ahem built it without any feedback from the end user aside from the initial input. User testing should be done so that product devs/owners can see if their product is fitting users' needs.

**** vital for any kind of wide adoption as UI is not intuitive and somewhat clunky to maneuver through

#106

tagging related issue which has some mock ups @michael-collins has worked on. We've got several other documents internally that I am not at liberty to share as to the direction the UI will be heading.
It's also had quite a bit of user input and feedback as well as the mental model explained in UX concepts -- https://haxtheweb.org/ux-concepts but your assessment is right that we need a more formalized UX audit then early adopters and Mike's moon shot ui.

The latest  discussion about UI should be on #247 , #106 is an older post. Feel free to jump in if you have specific ideas!

closing as round 1 took place at NARA and this is now on-going, splitting off into it's own issues https://github.com/elmsln/issues/issues?q=is%3Aissue+is%3Aopen+label%3A"UX+Study+1"


Unable to uninstall/update/install new version due to missing dependency

Microsoft PowerToys version
0.53.1 or any version
Running as admin

[ ] Yes

Area(s) with issue?
Installer
Steps to reproduce
Getting an error when I try to update, uninstall, or try to install a new version:

"The feature you are trying to use is on a network resource that is unavailable.
Click OK to try again, or enter an alternate path to a folder containing the installation package 'PowerToysSetup-0.51.1x64.msi' in the box below."
Problem is, this file is nowhere to be found, and all releases contain .exe files, but no .msi files.
Log:
[2244:1698][2022-06-29T10:37:47]i001: Burn v<IP_ADDRESS>26, Windows v10.0 (Build 19044: Service Pack 0), path: C:\Users\User\AppData\Local\Temp{AD856216-3EC6-43DB-94E6-56BEF8D88F3E}.cr\PowerToysSetup-0.59.1-x64.exe
[2244:1698][2022-06-29T10:37:47]i000: Initializing string variable 'InstallFolder' to value '[ProgramFiles64Folder]PowerToys'
[2244:1698][2022-06-29T10:37:47]i000: Initializing string variable 'MsiLogFolder' to value '[LocalAppDataFolder]\Microsoft\PowerToys'
[2244:1698][2022-06-29T10:37:47]i000: Initializing version variable 'DetectedPowerToysVersion' to value '<IP_ADDRESS>'
[2244:1698][2022-06-29T10:37:47]i000: Initializing version variable 'TargetPowerToysVersion' to value '0.59.1'
[2244:1698][2022-06-29T10:37:47]i000: Initializing version variable 'DetectedWindowsBuildNumber' to value '0'
[2244:1698][2022-06-29T10:37:47]i009: Command Line: '-burn.clean.room=C:\Users\User\Downloads\PowerToysSetup-0.59.1-x64.exe -burn.filehandle.attached=564 -burn.filehandle.self=560'
[2244:1698][2022-06-29T10:37:47]i000: Setting string variable 'WixBundleOriginalSource' to value 'C:\Users\User\Downloads\PowerToysSetup-0.59.1-x64.exe'
[2244:1698][2022-06-29T10:37:47]i000: Setting string variable 'WixBundleOriginalSourceFolder' to value 'C:\Users\User\Downloads'
[2244:1698][2022-06-29T10:37:47]i000: Setting string variable 'WixBundleLog' to value 'C:\Users\User\AppData\Local\Temp\powertoys-bootstrapper-msi-0.59.1_20220629103747.log'
[2244:1698][2022-06-29T10:37:47]i000: Setting string variable 'WixBundleName' to value 'PowerToys (Preview) x64'
[2244:1698][2022-06-29T10:37:47]i000: Setting string variable 'WixBundleManufacturer' to value 'Microsoft Corporation'
[2244:23E0][2022-06-29T10:37:47]i000: Setting numeric variable 'WixStdBALanguageId' to value 1033
[2244:23E0][2022-06-29T10:37:47]i000: Setting version variable 'WixBundleFileVersion' to value '<IP_ADDRESS>'
[2244:1698][2022-06-29T10:37:47]i100: Detect begin, 4 packages
[2244:1698][2022-06-29T10:37:47]i000: Setting version variable 'DetectedPowerToysVersion' to value '<IP_ADDRESS>'
[2244:1698][2022-06-29T10:37:47]i000: Setting string variable 'DetectedWindowsBuildNumber' to value '19044'
[2244:1698][2022-06-29T10:37:47]i000: Setting numeric variable 'HasDotnet605' to value 1
[2244:1698][2022-06-29T10:37:47]i000: Registry key not found. Key = 'Software\Microsoft\EdgeUpdate\Clients{F3017226-FE2A-4295-8BDF-00C3A9A7E4C5}'
[2244:1698][2022-06-29T10:37:47]i000: Setting numeric variable 'HasWebView2PerUser' to value 0
[2244:1698][2022-06-29T10:37:47]i000: Setting numeric variable 'HasWebView2PerMachine' to value 1
[2244:1698][2022-06-29T10:37:47]i000: Setting string variable 'DetectedVCRedistVersion' to value '32'
[2244:1698][2022-06-29T10:37:47]i052: Condition 'HasDotnet605' evaluates to true.
[2244:1698][2022-06-29T10:37:47]i052: Condition 'HasWebView2PerMachine OR HasWebView2PerUser' evaluates to true.
[2244:1698][2022-06-29T10:37:47]i052: Condition 'DetectedVCRedistVersion >= 32' evaluates to true.
[2244:1698][2022-06-29T10:37:47]i103: Detected related package: {DAA80B28-95F8-4E2C-BED4-3446474AEDA4}, scope: PerMachine, version: <IP_ADDRESS>, language: 0 operation: MajorUpgrade
[2244:1698][2022-06-29T10:37:47]i103: Detected related package: {DAA80B28-95F8-4E2C-BED4-3446474AEDA4}, scope: PerMachine, version: <IP_ADDRESS>, language: 0 operation: MajorUpgrade
[2244:1698][2022-06-29T10:37:47]i101: Detected package: DotnetRuntime6, state: Present, cached: None
[2244:1698][2022-06-29T10:37:47]i101: Detected package: WebView2, state: Present, cached: None
[2244:1698][2022-06-29T10:37:47]i101: Detected package: VCRedist, state: Present, cached: None
[2244:1698][2022-06-29T10:37:47]i101: Detected package: PowerToysSetup_0.59.1_x64.msi, state: Absent, cached: None
[2244:1698][2022-06-29T10:37:47]i052: Condition 'TargetPowerToysVersion >= DetectedPowerToysVersion OR WixBundleInstalled' evaluates to true.
[2244:1698][2022-06-29T10:37:47]i052: Condition 'DetectedWindowsBuildNumber >= 18362 OR WixBundleInstalled' evaluates to true.
[2244:1698][2022-06-29T10:37:47]i199: Detect complete, result: 0x0
[2244:23E0][2022-06-29T10:37:49]i000: Setting numeric variable 'EulaAcceptCheckbox' to value 1
[2244:1698][2022-06-29T10:37:49]i200: Plan begin, 4 packages, action: Install
[2244:1698][2022-06-29T10:37:49]w321: Skipping dependency registration on package with no dependency providers: DotnetRuntime6
[2244:1698][2022-06-29T10:37:49]w321: Skipping dependency registration on package with no dependency providers: WebView2
[2244:1698][2022-06-29T10:37:49]w321: Skipping dependency registration on package with no dependency providers: VCRedist
[2244:1698][2022-06-29T10:37:49]i000: Setting string variable 'WixBundleRollbackLog_PowerToysSetup_0.59.1_x64.msi' to value 'C:\Users\User\AppData\Local\Temp\powertoys-bootstrapper-msi-0.59.1_20220629103747_000_PowerToysSetup_0.59.1_x64.msi_rollback.log'
[2244:1698][2022-06-29T10:37:49]i000: Setting string variable 'WixBundleLog_PowerToysSetup_0.59.1_x64.msi' to value 'C:\Users\User\AppData\Local\Temp\powertoys-bootstrapper-msi-0.59.1_20220629103747_000_PowerToysSetup_0.59.1_x64.msi.log'
[2244:1698][2022-06-29T10:37:49]i201: Planned package: DotnetRuntime6, state: Present, default requested: Present, ba requested: Present, execute: None, rollback: None, cache: No, uncache: No, dependency: None
[2244:1698][2022-06-29T10:37:49]i201: Planned package: WebView2, state: Present, default requested: Present, ba requested: Present, execute: None, rollback: None, cache: No, uncache: No, dependency: None
[2244:1698][2022-06-29T10:37:49]i201: Planned package: VCRedist, state: Present, default requested: Present, ba requested: Present, execute: None, rollback: None, cache: No, uncache: No, dependency: None
[2244:1698][2022-06-29T10:37:49]i201: Planned package: PowerToysSetup_0.59.1_x64.msi, state: Absent, default requested: Present, ba requested: Present, execute: Install, rollback: Uninstall, cache: Yes, uncache: No, dependency: Register
[2244:1698][2022-06-29T10:37:49]i299: Plan complete, result: 0x0
[2244:1698][2022-06-29T10:37:49]i300: Apply begin
[2244:1698][2022-06-29T10:37:49]i010: Launching elevated engine process.
[2244:1698][2022-06-29T10:37:51]i011: Launched elevated engine process.
[2244:1698][2022-06-29T10:37:51]i012: Connected to elevated engine.
[3E88:4B58][2022-06-29T10:37:51]i358: Pausing automatic updates.
[3E88:4B58][2022-06-29T10:37:51]i359: Paused automatic updates.
[3E88:4B58][2022-06-29T10:37:51]i360: Creating a system restore point.
[3E88:4B58][2022-06-29T10:38:00]i361: Created a system restore point.
[3E88:4B58][2022-06-29T10:38:00]i370: Session begin, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall{799616eb-c851-42bb-806a-a8a136f32dda}, options: 0x7, disable resume: No
[3E88:4B58][2022-06-29T10:38:00]i000: Caching bundle from: 'C:\Users\User\AppData\Local\Temp{780CA0AA-3A9B-4E57-8B85-F3020F265E56}.be\PowerToysSetup-0.59.1-x64.exe' to: 'C:\ProgramData\Package Cache{799616eb-c851-42bb-806a-a8a136f32dda}\PowerToysSetup-0.59.1-x64.exe'
[3E88:4B58][2022-06-29T10:38:00]i320: Registering bundle dependency provider: {799616eb-c851-42bb-806a-a8a136f32dda}, version: <IP_ADDRESS>
[3E88:4B58][2022-06-29T10:38:00]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall{799616eb-c851-42bb-806a-a8a136f32dda}, resume: Active, restart initiated: No, disable resume: No
[3E88:179C][2022-06-29T10:38:01]i305: Verified acquired payload: PowerToysSetup_0.59.1_x64.msi at path: C:\ProgramData\Package Cache.unverified\PowerToysSetup_0.59.1_x64.msi, moving to: C:\ProgramData\Package Cache{8881A0B0-BBCB-4171-8783-DBAE6F7158C8}v0.59.1\PowerToysSetup-0.59.1-x64.msi.
[3E88:4B58][2022-06-29T10:38:01]i323: Registering package dependency provider: {8881A0B0-BBCB-4171-8783-DBAE6F7158C8}, version: 0.59.1, package: PowerToysSetup_0.59.1_x64.msi
[3E88:4B58][2022-06-29T10:38:01]i301: Applying execute package: PowerToysSetup_0.59.1_x64.msi, action: Install, path: C:\ProgramData\Package Cache{8881A0B0-BBCB-4171-8783-DBAE6F7158C8}v0.59.1\PowerToysSetup-0.59.1-x64.msi, arguments: ' ARPSYSTEMCOMPONENT="1" BOOTSTRAPPERINSTALLFOLDER="C:\Program Files\PowerToys"'
[3E88:4B58][2022-06-29T10:39:38]e000: Error 0x80070643: Failed to install MSI package.
[3E88:4B58][2022-06-29T10:39:38]e000: Error 0x80070643: Failed to execute MSI package.
[2244:1698][2022-06-29T10:39:38]e000: Error 0x80070643: Failed to configure per-machine MSI package.
[2244:1698][2022-06-29T10:39:38]i319: Applied execute package: PowerToysSetup_0.59.1_x64.msi, result: 0x80070643, restart: None
[2244:1698][2022-06-29T10:39:38]e000: Error 0x80070643: Failed to execute MSI package.
[3E88:4B58][2022-06-29T10:39:38]i318: Skipped rollback of package: PowerToysSetup_0.59.1_x64.msi, action: Uninstall, already: Absent
[2244:1698][2022-06-29T10:39:38]i319: Applied rollback package: PowerToysSetup_0.59.1_x64.msi, result: 0x0, restart: None
[3E88:4B58][2022-06-29T10:39:38]i329: Removed package dependency provider: {8881A0B0-BBCB-4171-8783-DBAE6F7158C8}, package: PowerToysSetup_0.59.1_x64.msi
[3E88:4B58][2022-06-29T10:39:38]i351: Removing cached package: PowerToysSetup_0.59.1_x64.msi, from path: C:\ProgramData\Package Cache{8881A0B0-BBCB-4171-8783-DBAE6F7158C8}v0.59.1
[3E88:4B58][2022-06-29T10:39:38]i372: Session end, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall{799616eb-c851-42bb-806a-a8a136f32dda}, resume: None, restart: None, disable resume: No
[3E88:4B58][2022-06-29T10:39:38]i330: Removed bundle dependency provider: {799616eb-c851-42bb-806a-a8a136f32dda}
[3E88:4B58][2022-06-29T10:39:38]i352: Removing cached bundle: {799616eb-c851-42bb-806a-a8a136f32dda}, from path: C:\ProgramData\Package Cache{799616eb-c851-42bb-806a-a8a136f32dda}
[3E88:4B58][2022-06-29T10:39:38]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall{799616eb-c851-42bb-806a-a8a136f32dda}, resume: None, restart initiated: No, disable resume: No
[2244:1698][2022-06-29T10:39:38]i399: Apply complete, result: 0x80070643, restart: None, ba requested restart:  No
✔ Expected Behavior
No response
❌ Actual Behavior
No response
Other Software
No response

Try this:
If you have an issue where the MSI is not accessible, you can download the installer, that corresponds with the installed version, via the PowerToys release page and then run the following command. You'll want to change the EXECUTABLE_INSTALLER_NAME to what the file name actually is.
In PowerShell, run .\EXECUTABLE_INSTALLER_NAME.exe --extract_msi and this will extract the MSI to your desktop
https://docs.microsoft.com/nl-nl/windows/powertoys/install#fixes-for-uninstalling-051-and-earlier-builds-issues

This is happening again, when trying to upgrade from version 0.59.1 to 0.60.0.

This is the first time I'm seeing this issue.

Try this:
In PowerShell, run .\EXECUTABLE_INSTALLER_NAME.exe --extract_msi and this will extract the MSI to your desktop

This no longer appears to be working - adding that command line option to the installer just brings up the installer as normal.

Did you use the .51 installer?

Hello, I also can't update the 0.66 version.I get the same messages as mentionned above.
I have tried the .\PowerToysSetup-0.66.0-x64.exe --extract_msi but it justs run the setup without extracting the .msi
I have also tried to run .\PowerToysSetup-0.66.0-x64.exe /uninstall and I get e message that it has been successuflly unsinstalled but powertoys is still here.
I really don't know how to get rid this version off...

I have the same problem in version 0.70,  I read one guy who was stuck with this like me because he did a rollback from windows 11. This might be the cause for this happening. Aso for later version than 0.51 it doesn't work the extract_msi for the installer. Maybe the only option available is to download from another computer and in the installing process trying to get the msi somehow, but seems impossible for me rn

Try this:
If you have an issue where the MSI is not accessible, you can download the installer, that corresponds with the installed version, via the PowerToys release page and then run the following command. You'll want to change the EXECUTABLE_INSTALLER_NAME to what the file name actually is.
In PowerShell, run .\EXECUTABLE_INSTALLER_NAME.exe --extract_msi and this will extract the MSI to your desktop
https://docs.microsoft.com/windows/powertoys/install#fixes-for-uninstalling-051-and-earlier-builds-issues

Sorry for my late reply, but this is what solved it for me!
Thanks for the help.