Implement __get__ to enable bound methods?

I initially followed the lead of partial and didn't implement __get__.
From back when I kept design decision notes in the README:

Not providing descriptor support like functools.partialmethod for now, until a need for it becomes apparent which a "normal function" variant (see last point) does not satisfy well enough.

Opening this issue to reconsider this more thoroughly.

You can just do this:
>>> from compose import compose
>>> from functools import partialmethod
>>> composemethod = compose(partialmethod, compose)
>>> class C:
...  def f(self, *args, **kwargs):
...   return args, kwargs
...  g = composemethod(str, f)
...
>>> c = C()
>>> c.f()
((), {})
>>> c.g()
'((), {})'

The downside of composing compose with partialmethod is that it's only obvious and memorable after already understanding Python descriptors.
Prior to that, it's easy for it to read as an opaque magic invocation, and to have it contribute to misinterpretations of what partialmethod even is or does.

Of course the other thing you can already do is mentioned in the README and the above quote:
>>> def fcompose(*functions):
...     composed = compose(*functions)
...     return lambda *args, **kwargs: composed(*args, **kwargs)
...
>>> class C:
...     def f(self, *args, **kwargs):
...         return args, kwargs
...     g = fcompose(str, f)
...
>>> c = C()
>>> c.f()
((), {})
>>> c.g()
'((), {})'

The downside of wrapping compose in a function through fcompose is that it's also not very obvious unless you already understand what's going on that a plain function or lambda can solve your problem.

Competing design values:

Good when composed callables are consistent with regular functions.
Good when composed callables are consistent with callable objects, especially ones already in functools.

Tentative verdict: start to implement __get__, but remain open to abandoning it if that reveals problems.
Because:

If we don't implement the descriptor __get__, then users' ability to use function composition as an ergonomics improvement for writing methods is complected with knowing about the descriptor protocol.

If we do implement __get__, then the complexity is limited to an uncommon edge case and is the same complexity you already have to deal with when using normal functions: if you assign a callable as an attribute on a class, and then access it through an attribute on an instance, well, a normal function would cause you to get a bound method in that case!

Simplicity for a library is relative to the complexity wrinkles already in the language. Much of my resistance to implementing automatic bound method behavior is that it adds absolute complexity to the class. But! - conceptually, it's the opposite: not having that behavior adds complexity by creating a difference from what's normal in the language.

Yes composition is good, so much so that in some ways, compose(partialmethod, compose) is clearly the better and the righter way to solve this than having this library implement a custom __get__.
But notably, the results of partialmethod aren't directly callable, they're only callable as methods. This means that if I provide (or direct users to self-provide) that composemethod made from composing compose and partialmethod, they will have two distinct composer functions, neither of which works in the other case.

I could provide a generic bindable_as_method or whatever, outside this library, which can take any callable, and wrap it with an object that implements __get__ and does the bound method logic, but still works directly as a callable function. And can be pickled in all cases. Then you could compose(bindable, compose) and it's a decent solution!
Maybe as I work on implementing it in compose, I'll just realize it works better as a separate library, and that's what it becomes.
The rest of this numbered point goes increasingly "off" topic (related, but increasingly distantly):
Of course it's not a perfectly transparent proxy in that case. But maybe that's fine. Also, should it even be? If you do attribute lookup on compose you don't expect that to fall through to any composed functions (because which one would you pick, and how, and when is each choice of that useful?), but if you set attributes on a bindable method wrapper, maybe you want it to transparently pass through to the underlying thing... I've been thinking a lot recently about transparent proxy objects as part of revisiting #1 as well - if that's worth doing at all, it's worth doing right, but I don't want my little tightly focused libraries to either re-implement or depend on an external library which provides a thoroughly correct implementation of that, like wrapt. Luckily I have at least some ideas for how users can combine these classes (compose, acompose, wrapt.ObjectProxy, the bindable idea from above, and so on), so I'll keep playing around with combining them and see how ergonomic that is - if it's a trivial recipe that users can use nicely, then it's probably fine to leave these things out of the library. If it turns into an annoying or care-requiring mess to compose it in externally, or somehow bad for transitive users of this library to leave it out (like if you get a compose object from some other library, it might kinda suck if it's not bindable, and you don't want to either know when libraries might give you non-bindable objects, nor preemptively wrap everything you get with bindable just-in-case...; similarly, it might suck if you want to give users bindable wrappers and you don't realize it think to do something to make them transparent proxies, but I'm having trouble thinking of a real-world use-case, let alone in good code, where that's would happen) then I might need to figure out how to combine some or all of them in ways I'm happy with.
Relatedly, I've been thinking of maybe doing an optional dependency, progressive enhancement thing - if wrapt is installed, use it, otherwise don't. (But then that has problems with Pythons like Transcrypt, which can't handle conditional/try-and-move-on-if-not-available dependencies during transpilation.)

On the one hand, functools does establish a pattern that we can follow, and trust/expect that it's familiar to developers of Python: partial vs partialmethod and singledispatch vs singledispatchmethod. On the other hand, that's because partial and singledispatch are doing something that conflicts with trying to naively bind them - compose does not have that problem.
With partial you have to ask: should self be bound before all partially applied positional arguments, or after? Well it actually depends on what you're trying to do. So I presume partial initially just erred on the side of not working either way, and then partialmethod provided one way of doing it, which is specifically useful when you want to take something that is already a method (already written to take self as the first argument) - but notably, you can legitimately have a use-case for factoring code into a function which operates on a class instance as an nth parameter, partially apply the first n-1 arguments, and bind that as a method. So arguably partial could or even should implement __get__ for that use case, while partialmethod covers the other use case. Anyway, singledispatch has a similar problem because it works on the first argument, but the first argument is always self in a method (or cls for class methods, but same concept). So there is good reason for a separate {partial,singledispatch}method in both cases.
But a composed function doesn't have any challenges like that: if the first argument of the innermost composed function is an instance of a class, there's no problem making that an instance method; or if the first argument is a class, as a class method.

There is still the appeal of portability conservatism - I want code that behaves the same on as many Pythons as possible, and the more tricky I get in the implementation, the more likely we are to hit weird incompatibilities. This isn't just about supporting old versions of Python 2 and 3 for people who might still have good reason to be stuck on them, I also have no idea how __get__ will interact with unusual Python implementations like Transcrypt, MicroPython, or Skulpt (Skulpt is much less important because it's a less complete and accurate implementation, and isn't meant for real production work, but it's an active project, and a nice plus if students learning Python through Skulpt-based teaching websites are able to just copy-paste my compose and go). But by now I'm convinced enough that if it works, it's worth doing it, so it's time to try it and test it, and if it doesn't work then I can reevaluate further. And if I can think of a good backpiling/transpiling story for the old and broken Pythons then I am happy to do what's best on the reference Python and compatible Pythons.

The idea of operator overloads for composition in #1 definitely warrants a __get__, and this is relevant because I've mostly decided that I do want to implement the foundation in this library to make composition-with-operators the best it can be, so I'll be looking to implement __get__ for that anyway.
And you absolutely don't want to play the alternating wrapper dance of making an object operator-composable only to lose bindability, then making it bindable only to lose operator-composability. (This also ties back to point 6 in this list, relevant to whether or not a separate independent bindable-as-method decorator/wrapper is viable, or if they have to be combined.)

You'd think this would be easy:
def __get__(self, obj, objtype=None):
    if obj is None:
        return self
    return types.MethodType(self, obj)

But of course portability is what ruins this:

Python 3 dropped the optional third argument of MethodType. Okay, fine, so we can always initialize it with two arguments right?

Skulpt requires a third argument. Of course Skulpt is like my lowest-priority-to-support Python variant (along with Transcrypt) (higher only than Pythons which aren't even maintained or remotely close to correct - I just like testing my stuff to see how it works on those Pythons to get a sense of some of the worst portability edge cases).

MicroPython's MethodType can't actually be used to construct new instances (it also doesn't have the types module built in, but you can get it from PyPI as micropython-types, or the same way the types module does by using type on a normal bound method). Actually it seems like MicroPython just doesn't support descriptors at all.

So I suspect the most portable version is actually to use some basic function-oriented Python stuff: closures:
def __get__(self, obj, objtype=None):
    if obj is None:
        return self
     def bound_compose(*args, **kwargs):
         return self(obj, *args, **kwargs)
     return bound_compose

But then that has a regression vs bound methods in terms of readability.
Also both methods so far have a problem with pickling. MethodType instances pickle fine if the function bound as a method is a regular function with a __name__ or whatever, but they do not pickle fine when the underlying "function" being bound is actually a callable class instance without a __name__.
Could do a partial:
def __get__(self, obj, objtype=None):
    if obj is None:
        return self
    return functools.partial(self, obj)

This might really be the best option. It's available since Python 2.5, is pickleable so long as all of its arguments are pickleable (before Python 3 you need to set the version of the pickle protocol higher, but otherwise it's fine), the basic functionality is easily polyfillable with pure Python (and this can be seen in action in the micropython-functools package, although Python's own standard library provides a pure-Python implementation of partial too). Notably the only place where I would expect types.MethodType to work out-of-the-box but functools.partial not to, is Skulpt, but as mentioned above, Skulpt couldn't be handled by the same code anyway.
However, I don't actually know how well functools.partial is handled as a bound method by various code out there. There might be pieces of code out there which would get rather confused.
A solution that is slightly more portable in some ways, but less so in others, is to implement my own little class:
class BoundComposedMethod(object):
    __slots__ = ('_method', '_self')
    def __init__(self, composed_method, bound_self):
        self._method = composed_method
        self._self = bound_self
    def __call__(self, /, *args, **kwargs):
        return self._method(self._self, *args, **kwargs)

# and then
def __get__(self, obj, objtype=None):
    if obj is None:
        return self
    return BoundComposedMethod(self, obj)

But I have a vague memory of at least one weird incomplete Python being implemented so that callable objects with __call__ actually didn't work - __call__ was just ignored. If I had to guess I'd say it was Transcrypt, since that one generally doesn't support operator overloading in the first place.
In fact that might've been one of the reasons why I initially came up with (note the explicit .__call__):
def fcompose(*functions):
    composed = compose(*functions)
    return lambda *args, **kwargs: composed.__call__(*args, **kwargs)

That recipe ultimately got simplified down for the README, because I figured the vast majority of people would never need to know or care about adding that explicit .__call__, and so I removed it. But in principle that works - it perfectly wraps compose in a function-only behavior that works even on Pythons which don't respect the __call__ magic method.
Of course compose has other struggles on those Pythons anyway, For example, Transcrypt tries to transpile modules referred to by import even if they are explicitly inside try-except ImportError blocks, so you couldn't use any copy of compose on there without either polyfilling reprlib.recursive_repr (not necessarily well, could be a no-op) or deleting that import line and the use of that decorator.
So the partial way is the most portable (as is often the case with functions over magic methods - something has to actually call magic methods so if the language doesn't do it you have to find some location to inject the magic method use, but functions are already being called as part of their usage and they can be polyfilled or replaced more easily with whatever implementation works in the language implementation or for that code specifically).
But even if we eliminate the unhealthy portability obsession I have (could just not care about how to write portable code to the almost-Pythons, or handle that with separate files like I do for older Pythons), functools.partial has a win over types.MethodType for pickling in this case on current proper Python too.

Oh, also wrapping/composing compose instances with partialmethod creates bound methods that can pickle but can't actually unpickle.

One irregularity with using partial instead of the native bound method type is that it doesn't implement __get__, so it can't recursively be subjected to __get__.
I can't think of a situation where you'd want to do this though, because the typical behavior of a bound method's __get__ is to just remain a bound to the self that it was already bound to. And anything that is calling __get__ on bound methods should already be capable of handling things that don't have __get__, right?
So I don't see this as a major bad reason.
However at the same time, if partial is polyfilled as a simple Python function returning a closure, then __get__ will work on partial instances, but it will work wrongly.
So thus we find an unlikely but important edge case where using partial could works badly and inconsistently.
But does this actually matter, or is this a corner so tiny and out-of-the-way that it can be justifiably left unhandled.

Actually I logically followed my argument against a custom bound method class some more, and realized that any Python impl. where __call__ doesn't work already has to wrap compose in a function anyway just to use it, so on those implementations, what __get__ returns doesn't really matter.
So fleshing that out a bit more, with the recursive __get__ behavior matching native bound methods, and more portable pickling support (out-of-the-box on Python 2, which is even better than the partial approach gives):
class BoundComposedMethod(object):
    __slots__ = ('_method', '_self')
    def __init__(self, composed_method, bound_self):
        self._method = composed_method
        self._self = bound_self
    def __call__(self, /, *args, **kwargs):
        return self._method(self._self, *args, **kwargs)
    def __get__(self, obj, objtype=None):
        return self
    def __reduce__(self):
        return (BoundComposedMethod, (self._method, self._self))

# and then in the `compose` classes:
def __get__(self, obj, objtype=None):
    if obj is None:
        return self
    return BoundComposedMethod(self, obj)

Taking a step back, what do I want, without all the portability concerns?
Putting portability aside, I want:
class C:
   method = compose(foo, bar)

c = C()
c.method()  # equivalent to foo(bar(c))
c.method  # pickleable

Basically I want compose instances to work as bound methods, which can be pickled and unpickled to the same bound method (provided that the class instance can be pickled and unpickled).
Why do I care about pickling? That's a good question. I feel like ideally, things which in principle can be pickled should be pickleable, because if it's ever useful to pickle something, I don't want my code to break that. I also think more generally edge cases where behavior changes without good obvious reason are bad - equivalently, I think regularity of behavior across the whole possibility space is good. It makes it simpler to reason about. So I want callables made with compose to be as function-like as possible, and normal functions turn into bound methods that can be pickled and unpickled. If a library implementer is writing a class, no one should have to worry if using compose to implement one of the methods would break something.
I'd love it if types.MethodType worked to achieve that, but it doesn't achieve the pickling-unpickling goal.
I'd also love it if functools.partialmethod worked to achieve that, because then compose.__get__ can be implemented by calling partialmethod(self).__get__. But that also fails the pickle-unpickle goal.
So then I kinda want to just go with a __get__ which uses partial to bind the argument.
But, even putting aside portability, I am cautious about implementing it with partial because then compose_instance.__get__(...).__get__(...) would work inconsistently with function.__get__(...).__get__(...).
The closest relevant example I can think of is iterable.__iter__().__iter__(), which is something that every iterator should be consistent with, by implementing def __iter__(self): return self. Because then you can shove iterators into anything that iterates on an iterable, and it will Just Work. In other words, iterators are already iterators, so trying to get an iterator out of them should be a idempotent success. And that way you can compose code together, like a function that reads part of an iterator and then passes the rest to another. (In fact the recent sacompose addition in v1.3.0 of this library takes advantage of that exact thing.)
But the shape of usage is relevantly different for __get__ versus for __iter__:

The normal code path calling __iter__ unconditionally assumes it is implemented on the object, while the normal code path for __get__ conditionally calls it only if it is implemented.
__iter__ is is regularly called in normal code (through for x in y loops, or the iter builtin function) while __get__ is only called in specialized code which is implementing or wrapping attribute access.
Calling __iter__ on an iterator has meaningful useful semantics: "I want an iterator over whatever iterable stuff you just gave me", and iterators are naturally already what's wanted in that case so they return themselves. But what does calling __get__ on the result of a __get__ mean? For methods, __get__ means something like "bind yourself to this thing if you're not already bound". But it's not even necessarily obvious that this is what we want, without a separate deeper think (though it seems reasonable - if a bound method could become un-bound that would probably be more surprising and unexpected and require more defensive coding all over the place).
It's not obvious that bound methods in Python implement __get__ as because it's good for bound methods to be descriptors. Instead, it seems to just be an implementation side-effect of bound and unbound methods being implemented with the same underlying type, going back all the way to early Python - maybe it is or was simpler or easier or cleaner in C to do it that way. But iterators implementing idempotent (self-returning no-op) __iter__ makes sense as a proactive design decision, per all of the above.

So if an iterator fails to also be an iterable, it becomes less compatible (without any obvious good reason) with a bunch of mainstream code written by normal developers who have good reason to not be intimately familiar with the innards of Python.
But if something returned from a descriptor is not also a descriptor, it would only be incompatible with code which

is hand-written by programmers deliberately getting involved in the descriptor protocol,
rarely needs to be written, and
is making assumptions that it shouldn't be making.

Rant to be continued...

I like the idea of implementing my own bound method class, like the one above.
Then if I ever find problems with that, I would reassess using functools.partial instead.
Now the question that remains is: does this belong inside compose?
I am strongly tempted to make a separate package for just this BoundMethod class anyway - so the only remaining question is, do I bother to replicate it here, or just tell users to install and use that package once I make it?

Notably, the bound method implementation (generic, maximally portable) has grown to all this:
from types import MethodType as _MethodType

class _BoundMethod(object):
    __class__ = _MethodType

    __slots__ = ('_function', '_instance', '__weakref__')

    def __init__(self, function, instance):
        self._function = function
        self._instance = instance

    def __call__(self, /, *args, **kwargs):
        return self._function(self._instance, *args, **kwargs)

    def __get__(self, obj, objtype=None):
        return self

    def __repr__(self):
        return repr(self._function) + '.__get__(' + repr(self._instance) + ')'

    def __reduce__(self):
        return (_BoundMethod, (self._function, self._instance))

That's a very large amount of boilerplate gunk to include in this library, and then maintain in two separate packages.
I don't really like that idea.
On the other hand, I am not thoroughly repulsed by it.

Verdict - adding it to compose.
May do a separate library with the same functionality for others to reuse later, but for now I want to try having it out-of-the-box and with zero dependencies in compose.

Conciser recap of all the rationale:

MethodType is almost perfect, but cannot be pickled if the bound function can't be looked up on the instance class with getattr(type(instance), function.__name__). So using MethodType for cases like compose adds a regression from normal bound methods.
If MethodType(compose(...), obj) could be pickled and unpickled, I would've probably just used MethodType in __get__, at least in the canonical normal.py source.

functools.partial is almost perfect, but compose(...).__get__(...).__get__(...) would silently behave wrongly on any system that polyfills partial with a function returning a closure. The corner case where this would happen is if class A implements method m with compose and then other code assigns instance_of_A.m as attribute m on class B, expecting it to work just like a bound method normally does in that case when called as instance_of_B.m(...).
If this was breakage that users could only get into by knowingly opting in to it, or which failed fast quickly enough to probably get exposed in tests before it's in production, I would've probably used partial, once MethodType was ruled out.

Pickling has greater portability constraints - we can't just provide different code for different Python versions and implementations if pickling serializes them differently.
(Imagine a microcontroller using MicroPython or web app using a JavaScript/WebAssembly Python talking to a server running regular Python.)

Pythons which don't properly implement the descriptor protocol still require you to use the fcompose recipe rather than compose for methods, but we should give users on fuller Pythons an out-of-the-box experience that works like they would expect.

It's still possible to do staticmethod(compose(...)) to achieve what compose previously did on a class, which is more consistent with how Python functions work.

Wait a minute! I can do this, can't I?
def __get__(self, obj, objtype=None):
    return type(self)(*reversed(self._wrappers), self.__wrapped__.__get__(obj, objtype))

That's the clearest version, but here's a more efficient version which might be more obvious for some people:
from copy import copy as _copy

def __get__(self, obj, objtype=None):
     bound_self = _copy(self)
     bound_self.__wrapped__ = self.__wrapped__.__get__(self, obj, objtype)
     return bound_self

Key point is that instead of creating a bound wrapper around the compose instance, we can create a modified compose instance whose "first"/"innermost" function is bound.
This alternative is

simpler (we're not flipping back and forth between the different levels of abstraction where functions are stored in execution order but need to be given to the constructor in traditional composition-writing order),
more efficient (because _wrappers can be shallowly copied).

Of course to make this more like a normal bound method, we would not create a new compose instance if the binding of the innermost function is a no-op, since then the bound composition is the same as the unbound composition:
from copy import copy as _copy

def __get__(self, obj, objtype=None):
    wrapped = self.__wrapped__
    bound_wrapped = wrapped.__get__(obj, objtype)
    if bound_wrapped is wrapped:
        return self
     bound_self = _copy(self)
     bound_self.__wrapped__ = bound_wrapped
     return bound_self

But we actually don't want copy.copy, because we don't want to bypass our __init__ checks for the binding of the innermost function (because the result of __get__ to bind it might not be a callable), so the final version is:
def __get__(self, obj, objtype=None):
    wrapped = self.__wrapped__
    bound_wrapped = wrapped.__get__(obj, objtype)
    if bound_wrapped is wrapped:
        return self
    bound_self = type(self)(bound_wrapped)
    bound_self._wrappers = self._wrappers
    return bound_self

Luckily even without using copy.copy we can still shallow copy _wrappers.
Over the previous implementation ideas, this is:

more correct (we are properly transparent to the descriptor protocol),
more elegant (less concepts, classes, and interactions for the same functionality),
more efficient (smaller code size, calling a bound composed method no longer goes through an extra wrapper, the binding itself falls through to the underlying callable which can be implemented in native optimized code).

The only "downside" (arguably upside) is that it won't automagically create bound methods if the innermost callable does not support binding. But I mean, that's actually more regular and consistent, isn't it? compose(foo, partial(bar, qux)) should have the binding behavior of partial(bar, qux), because that way it's truly just composing callables without subtly changing anything else.
So I'm going to give that some portability testing and if it works well enough in enough places, that's great.

Okay first item, a normal instance.method works without throwing exceptions whether or not type(type(instance).method) has a .get`, it just behaves differently.
But type(instance).method.__get__ will throw an AttributeError if type(type(instance).method) doesn't have a .__get__.
In other words:
class C:
    m = partial(...)

C().m  # == C.m == partial(...)
# but
composed = compose(C().m)
composed.__wrapped__.__get__()  # throws AttributeError

So I think it has to be dragged out like this:
    def __get__(self, obj, objtype=None):
        """Get the composed function as a bound method."""
        wrapped = self.__wrapped__
        try:
            bind = type(wrapped).__get__
        except AttributeError:
            return self
        bound_wrapped = bind(wrapped, obj, objtype)
        if bound_wrapped is wrapped:
            return self
        bound_self = type(self)(bound_wrapped)
        bound_self._wrappers = self._wrappers
        return bound_self

Yes, good, that last one works on every Python where __get__ works (specifically I tested CPython, Brython, Skulpt, and MicroPython - I trust that PyPy, Jython, IronPython, and Pyodide have a fully properly working descriptor protocol like CPython, and I know that Transcrypt doesn't like MicroPython).

An interesting TIL: print doesn't implement the descriptor protocol and so can't automatically become a bound method.


Fix not storing mesh name

Very small fix. But there you go!

Thanks! I'm so sorry, I just saw your PR now...


Mistral

Hey @fakerybakery I think you raised this in reference to the OpenFunctions release? We have a mistral trained model, but the performance is not that great 👀
It might just be that we need to tune the hyperparameters better, but in any case if you want to try it out I just hosted it . Replace the model-id in the colab with gorilla-openfunctions-mistral-v0

Is the code available for how you fine tuned Mistral?  May want to try it out and tweak it.
Ps @ShishirPatil I have compute available for you if you need some.  Hosted TPUs at Google GCP.


missingDeps.some not a function

https://github.com/facebook/create-react-app/blob/69cc07f2e3400d01c5db134659c762eb850ae3c8/packages/react-dev-utils/WatchMissingNodeModulesPlugin.js#L25
I am getting
TypeError: missingDeps.some is not a function

It's a SortableSet from here https://github.com/webpack/webpack/blob/master/lib/util/SortableSet.js and it doesn't have a .some method, so I am not sure how to fix this.

Please upgrade your Node version.

https://www.ecma-international.org/ecma-262/8.0/index.html#sec-properties-of-the-set-prototype-object
Set doesn't have some, SortableSet doesn't have some. I don't understand, did some node version have Set with a .some method, ever?

Thanks @Timer, you really "helped" me. I wish for you to be helped by everyone the same way you handled this issue.

Actually, your template says that if I don't know if it's a bug, I don't have to use it.
Second, I think you didn't even read what I wrote first, just closed it being so sure that I am using some 6 year old node version... but it all doesn't matter, because now I took the time which I don't think I should've had, and found this https://github.com/facebook/create-react-app/issues/4366
Yesterday I didn't search between the closed issues because I didn't assume that you would close unsolved issues. Now I know better, having experienced otherwise how things go around here.
I have installed https://www.npmjs.com/package/react-dev-utils @ 5.0.1 which is the latest.
May I ask when the fix will be published on npm too?

Unfortunately, we do not have enough information to help you.
If you're installing packages like react-dev-utils it sounds like you're ejected, otherwise I'm not sure what you're doing.
I'm sorry if I gave you a terse first response, but most of these issues that come through like this are related to an old Node version.
If you can replicate this with a non-ejected setup, please provide all of the information the issue template states and reproduction steps. I'll close this until more information is provided, as this issue doesn't seem wide-spread.

Publishing that version is a few months out.

@vireshshah we are going to have to wait "a few months". :) Until then I suggest to you to use a local copy of the repo where you can apply the patches you need. (or use an earlier version of webpack).

Also having the same issue. Not sure how to come around it yet. @ashnur which webpack version would solve the problem, do you know?

The current version of react-dev-utils only supports webpack 3.
You cannot use webpack 4 unless if you re-write a significant portion of tooling yourself, or wait for the release of react-scripts@^2 (which is a couple months out).

@ashnur This appears to be fixed in the Webpack 4 upgrade PR, but this particular change did not make it into the alpha release mentioned here. Not sure if there is a newer alpha to try.

Finally we got this working by using 5.0.0-webpack4 of react-dev-utils-for-webpack4 package.

npm install  react-dev-utils-for-webpack4  ?
The problem still exists and has not been resolved。

@echenley Thanks man! Next round of beers on my 🍻

@echenley Thanks!


Add option to force rebuild

Ninja lacks an option to force re-building of a particular target, like the make -B option.
I would be willing to implement this if someone could guide me through what needs to be done.

what about definig a "always" helper dependency that is used as implicitly dependency then?
build always: phony

rule doxygen
  command = doxygen $in

build doxy: doxygen Doxyfile | always

This is derived from what I'm found work nicely for building opaque sub projects:
rule subProjectBuild
    command = somewhere/whatever/magic.sh
    restart = 1
build subProject.out: subProjectBuild | always

Would you consider this -B feature now? I think it would be useful.

@edbaunton what has changed for it to be considered now? Actually I would like to have it too, as at least once I needed to force a rebuild, but I don't remember what was the case tbh.
Maybe it would be better if someone could come with a patch? Then we could see if the code added does not add too much complexity to Ninja.

One use case for -B is when trying to fix warnings for a target.  The build succeeds, but you still want to rerun all the commands to see the output.

One more use case is to allow IDE like Eclipse CDT to scan file dependencies with -B -n -v even without having to do a full build

I'm also in favour of ninja -B. I'd often like to rerun the entire build without having to run clean first in order to check whether there are any compiler warnings left.

@jhasse that’s fundamentally flawed. Output from previous incremental builds cannot be assumed to be representative of the same output starting from scratch.

It is not assumed: It's an option to explicitly show them.
In case of compiler warnings the output will be the same though, unless you're missing some header dependencies.

Closing this without fixing it seemed pretty unpopular. I also miss this feature. Please reconsider?

@ericniebler ninja -t clean <target> && ninja <target> will rebuild . Ninja tries to stay as small as possible, so we're very wary of new command line flags, especially if they can be achieved in other ways like in this case.

@ericniebler ninja -t clean <target> && ninja <target> will rebuild <target>. Ninja tries to stay as small as possible, so we're very wary of new command line flags, especially if they can be achieved in other ways like in this case.

This is not equivalent. Say I have a build target and test target. test depends on build, naturally. I would like to test multiple times, but currently after the first time ninja says no work to do. If I do ninja -t clean test and ninja test, the result is ninja rebuilding the project, which can take a lot longer.

I assume by "rebuilding the project" you mean that ninja rebuilds the build target in addition to the test target? This is equivalent to make -B AFAIK.

I second @jhasse on this, I understand the minimalistic approach, but I also miss this feature.
Any chance this could be reconsidered?

Actually I was making a point against reconsidering, since make -B <target> is equivalent to ninja -t clean <target> && ninja <target>. Am I missing something here?

I have different use case.
Microsoft likes to update their compilers every two-three weeks. That makes me crazy, because all precompiled headers should be regenerated after the compiler update.
Even ninja tells me, that I need to rebuild precompiled headers.

......\baselibs\src\core\sql\statementhints.cpp: fatal error C1853: 'C:/viadat/projects/develop_cmake/out/build/x64-Release/baselibs/src/core/pch/CMakeFiles/vs_core_pch.dir/./cmake_pch.cxx.pch' precompiled header file is from a different version of the compiler, or the precompiled header is C++ and you are using it from C (or vice versa)

The problem is that in the project we have several precompiled headers for different libraries.
And cleaning with ninja -t clean all doesn't remove any of header.

So, force rebuild is must have at least in case of precompiled headers regeneration.
Maybe I need to use another tricky command for the cleaning them, but only manual deletion of each file actually helps.
Do you know any?

And cleaning with ninja -t clean all doesn't remove any of header.

This is the bug.

CMake generates a build.ninja with a target that rebuilds it when any of the cmake files change.
Running clean is obviously not an option. One hack is to touch one of the dependencies and run ninja build.ninja.
Or you could do mv build.ninja build.ninja.old; ninja -f build.ninja.old build.ninja && rm build.ninja.old || mv build.ninja.old build.ninja but it's even more hacky.
A simple ninja -B build.ninja would be very useful.

Do you want to re-run cmake? cmake .

The cmake command under the RERUN_CMAKE rule I have here is more complicated than a simple cmake .. Maybe it's the same thing? I just wouldn't want to do some research to find out. touch ../proj/CMakeLists.txt; ninja build.ninja is easier. ninja -B build.ninja would be even easier-er. Just my two kuruş.

It's the same thing. Also try: ninja rebuild_cache.

I'd like to add some extra arguments in favour of re-opening, instead of resorting to manually touching or deleting the intermediate files:

It saves 10+ seconds of time figuring out what to type, and this can grow to a couple minutes for complicated build systems that are not your own project. Even for your own project, sometimes you forget exactly how you named everything. Over all users over all projects, this is a lot of time.
For complicated build projects, there can be intermediate temporary build products arranged in a complex dependency tree. You need to figure out what all of these are, delete them, then run ninja again. This defeats the point of a dependency-tracking build system that is supposed to figure things out for you.

In general, arguments along the lines of "but you can do it with " are bad arguments. By the same argument, the whole of ninja is redundant since I can "do it with ". The point of all computing is to save people time.

[...] instead of resorting to manually touching or deleting the intermediate files [...]

You don't have to, use ninja -t clean <target> && ninja <target>.

This seems to clean all intermediate files that are an input of <target> except for source files. If other targets also depend on those files, then a later run of ninja will also rebuild those targets. Instead I'd like to clean only inputs of <target> that nothing else depends on, to keep the effect of the rebuild localised. Example:
y.c -> y.o -> y
           \
            v
x.c -> x.o -> x

I'd like "rebuild x" to only clean x.o and x, and not y.o - nor y later. For example, if I updated the tool that generates x.o in a way that I know doesn't affect anything else. It would also be useful for the "test but don't rebuild" scenario that @mahrud mentioned above, or for rebuilding docs if you updated doxygen but not the compiler.
Yes this is not equivalent to make -B.
You could put the functionality in another subcommand like clean-unique  / clean-unique-deps-of and force people to do the shell-command dance, but I also wonder how many people actually want to clean specific targets but not rebuild them afterwards?

Slightly more complex example:
z.c -> z.o -> z
           \
            v
y.c -> y.o -> y
           \
            v
x.c -> x.o -> x

"rebuild x y" would look at the ancestors of x or y and exclude ancestors of other (top-level) targets i.e. z, the result being to clean y.c, y.o, x.c, x.o. In general terms you'd only clean ancestors(requested top-level targets) - ancestors(unrequested top-level targets).

Can you open a new issue for that functionality?

Add option to force rebuild #317 Closed
mgaunard opened this issue on May 26, 2012 · 38 comments

Goddamn, 12+ years and we still not there 😡

nico commented on Apr 5, 2018
People tell me I should be more decisive about closing out things, so let's close this.

👆 That dude swallowed hole 30+ downvotes for blatantly closing this topic.
If anybody uses CMake see my workaround for rebuilding the target

@Challanger524 As it was explained the functionality is already there: ninja -t clean <target> && ninja <target>
Providing a shortcut for that (i.e. -B) is a bad idea IMHO, because most of the time it hides a different problem, e.g. missing dependencies and shouldn't be encouraged.
For missing functionality see #2318.
Having an option which will reprint the output of every edge (e.g. to fix warnings) is also something that I would be willing to implement. Feel free to create an issue for that.


Question: WinUI interfering with MIDI APIs

I am using the midiIn  APIs to get a callback when I press notes on my connected (piano) keyboard. This works fine with a console application, but I never get any callbacks in my WinUI application. The midi objects are created in the Application.OnLaunched method, so they should have main thread affinity.
Does WinUI does not run a standard Win32 PeekMessage/DispatchMessage loop on the main thread? Is there a way to make it do so?
It would be possible to create a separate thread for my midi objects and run my own message loop there, but this is a good amount of extra complexity that I would rather avoid, and I'm not sure that would fix the issue.
Thanks for any suggestions

I might have spoken too fast, I added an artificial message loop after my call to midiInStart and I'm still not receiving callbacks. What could be different in a WinUI app preventing the callbacks? :/
        while (GetMessage(out var msg, default, 0, 0))
        {
            TranslateMessage(in msg);
            DispatchMessage(in msg);
        }

It appears to be a bug with the .NET debugger. I filed an issue below but I'll leave the WinUI team close this question in case they have insights.
https://github.com/dotnet/runtime/issues/64668

@MikeHillberg thoughts?


Added a new disable-install-crds flag to Pilot Discovery.  (#8813)

Added a new disable-install-crds flag to Pilot Discovery.  This flag when passed disables creation of CRDs when they cannot be detected.  In the case of a failure during CRDs creation, the call to deregister CRDs has been removed.

Updating disable-install-crds ucommand usage documentation

Add new line to disable-install-crds usage message line

Run fmt.sh against main.go

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: rshriram
If they are not already assigned, you can assign the PR to them by writing /assign @rshriram in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @kdillane. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/assign @rshriram

/ok-to-test

@kdillane: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
8b6ca36f10eb55ebf98a6114888898d27b6903e3
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

Looks like the tests are failing for a couple different reasons:
racetest: https://github.com/istio/istio/issues/7517
codecov: https://github.com/istio/istio/issues/8907 - this appears to be fixed by a config change for jobs
e2e-pilot-auth: I don't have a root cause here.  Given the default behavior is preserved from master, and this commit passed all e2e-pilot-auth tests previously, it doesn't seem as though it should be related.

I'm going to attempt to merge from the main Istio upstream into my fork and re-run the tests.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: rshriram
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [rshriram]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kdillane: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
3cf81928faaf8273ad8833b276b691a6d29778a4
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Miso tries to set read-only properties instead of using setAttribute

Currently miso checks if an attribute exists as a property and if so it tries to set that property, if not it falls back to setAttribute. However some properties, e.g., the form attribute on HTMLButtonElement are read only and cannot be set via the JS property. At least form can however be set via setAttribute. While I’ve only seen this problem with form I doubt that this is the only property affected by this. It would thus be nice to find a general solution for this instead of adding a bunch of exceptional cases.

Yea, this does seem problematic. I've been told that form and list are the only two exceptions.
So, if you can make a PR on https://github.com/haskell-miso/miso/blob/master/jsbits/diff.js#L83 that looks like:
} else if (c in node && c !== "list" && c !== "form") {

same for line 102

55e5927a21fc4ff3504e0f861c35c342990a3989

Thanks for the fix!

np 🍻


Bulk Move Freezes Page

When I hit *v everything freezes up. Can't type anything, can use mouse, can't use shortcuts. Nothing. Using the 'v' shortcut works just fine for single moves though.

Hey, thanks for reporting.  Are you still seeing this issue?  The "v" shortcut seems to work fine for me, perhaps it was a Todoist bug that got fixed.

Closing from now response.


[Feature] Social Hover Animated Button

Description
A button in circular shape divided into four sections for social media links
Screenshots

Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSOC'24 contributor.
[ ] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

@divyansh-2005
I am assigning u
As admin is busy for some days
Do your task and provide your PR asap.
Then admin merge it with the proper labels .


Include dot product space for Lucene hnsw

Hi Team,
Thanks for the great work.
Do you have any plan or roadmap to include the dot product space for hnsw method using Lucene?
https://opensearch.org/docs/latest/search-plugins/knn/knn-index/#supported-lucene-methods
Thanks and regards,
Li

Duplicate of #865. Closing


Typo

In GitHub repository description, "compatability" should be "compatibility".

Oops... thanks @remram44


storagepb: minor updates from genproto

I ran genproto with the most recent protocol buffer
and it adds one line of missing comment.

LGTM

Thanks!


iavl/tests: BenchmarkImmutableAvlTreeCLevelDB crashes

On branch unstable, hash 25b700b87a45619cb6bd85330ab4a81b7b7fbb0d, when I went into iavl
and then ran
$ go test -v -run=^$ -bench=.

the code crashes with this stack trace
goos: darwin
goarch: amd64
pkg: github.com/tendermint/merkleeyes/iavl
BenchmarkImmutableAvlTreeCLevelDB-4   	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x43d3645]

goroutine 13 [running]:
github.com/tendermint/tmlibs/db.NewDB(0x45200dd, 0x4, 0x45211cd, 0x8, 0x451fac7, 0x2, 0x479a2e4, 0x204059250)
	/Users/emmanuelodeke/go/src/github.com/tendermint/tmlibs/db/db.go:55 +0x95
github.com/tendermint/merkleeyes/iavl.BenchmarkImmutableAvlTreeCLevelDB(0xc420001e00)
	/Users/emmanuelodeke/go/src/github.com/tendermint/merkleeyes/iavl/iavl_test.go:590 +0x83
testing.(*B).runN(0xc420001e00, 0x1)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/benchmark.go:141 +0xb2
testing.(*B).run1.func1(0xc420001e00)
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/benchmark.go:214 +0x5a
created by testing.(*B).run1
	/Users/emmanuelodeke/go/src/go.googlesource.com/go/src/testing/benchmark.go:207 +0x80
exit status 2
FAIL	github.com/tendermint/merkleeyes/iavl	0.059s

Seems like this initialization
db := db.NewDB("test", db.CLevelDBBackendStr, "./")

is trippy, or there is supposed to be a backing DB available?

Okay the problem is that some files in there have a build tag
+build gcc

which was the red herring, gcc as a compiler is needed for cleveldb. Seems like we need to make some documentation and clarify what's up.
/cc @ebuchman @adrianbrink

Thanks @odeke-em . I don't have much experience with c-go. Can you update the docs/makefile to fix this ?

So I've mailed https://github.com/tendermint/merkleeyes/pull/48 which adds the appropriate build tag to only run that test when the compiler they have is gcc.


move test api retries into api stage

previously, if the api encountered a delay or an error, it wouldn't attempt a retry, as the retries/delay code was added to the debug stage. This PR moves the logic into the correct stage, and introduces a slight jitter into the delay to avoid more potential race conditions.

Thinking about this more, I'm not sure if retries are the best thing to do here. If the first request processes but times out after 30s, the second request that is tried could return an error still, but errors are still 200 responses?
Which means until: apiresponse.status == 200 will just waltz on through even on genuine failures, since we aren't checking the apiresponse for an error condition?
It might be better to set the timeout parameter in the URI request from 30 to longer, and then checking the response for errors (but this is also not ideal as some things error in a successful request)https://docs.ansible.com/ansible/latest/collections/ansible/builtin/uri_module.html#parameter-timeout

no longer required


A walkthrough the UX components of the MRTK SDK - new guide

Overview
Users need a clear and concise guide for how to utilise the UX / UI components provided by the MRTK SDK
Requirements
Users need to be able to have good guidance for how to utilize

Interactables
Lines
Collections / Utilities

This might possibly need to be broken up in to multiple guides
Acceptance Criteria

[ ] As a user, I need to be able to add UX interactivity in my scene
[ ] As a user, I need to be able to coordinate the relationships between different objects in my project
[ ] As a user, I need to be able to organise UX components in my scene relative to the player

https://microsoft.github.io/MixedRealityToolkit-Unity/Documentation/README_Interactable.html


Contributors

Hi!
As anyone can see, this repository is a little bit abandoned.
It'd be cool if @ziadoz could give access to other people, so they can manually review, approve or close all those PR's.
To start, I offer myself :smile:

https://github.com/ziadoz/awesome-php/issues/870


pthread_mutex_lock assertion failure on WritesDone in a stream

I get the following backtrace when attempting to call WritesDone on a synchronous stream:
Program received signal SIGABRT, Aborted.
0x00007ffff56e7267 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
55      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff56e7267 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#1  0x00007ffff56e8eca in __GI_abort () at abort.c:89
#2  0x00007ffff6d8da2f in gpr_mu_lock (mu=0x78c830) at src/core/support/sync_posix.c:48
#3  0x00007ffff6529ede in grpc_cq_begin_op (cc=0x78c7f0) at src/core/surface/completion_queue.c:119
#4  0x00007ffff6527d41 in grpc_call_start_batch (call=0x77bd40, ops=0x7fffffffdce0, nops=1, tag=0x7fffffffdfc0, reserved=0x0) at src/core/surface/call.c:1733
#5  0x00007ffff6256b47 in grpc::Channel::PerformOpsOnCall (this=0x77a330, ops=0x7fffffffdfc0, call=0x786ec0) at src/cpp/client/channel.cc:92
#6  0x00007ffff625e1a3 in grpc::Call::PerformOps (this=0x786ec0, ops=0x7fffffffdfc0) at src/cpp/common/call.cc:89
#7  0x00000000004f53f3 in grpc::ClientReaderWriter<OutgoingEventPB, IncomingEventPB>::WritesDone() ()

This particular line is attempting to lock a GRPC_POLLSET_MU in what I presume is the completion queue for the synchronous stream.

I should add, this is a GRPC built from source at tag release_0_11_1

Hey thanks for the bug report. We'll take a look on Monday (it looks like a
memory deallocation that shouldn't be happening). Any chance you have a
simple reproduction you could share?
On Sun, Oct 4, 2015, 11:18 AM alexgolec<EMAIL_ADDRESS>wrote:

I should add, this is a GRPC built from source at tag release_0_11_1
—
Reply to this email directly or view it on GitHub
https://github.com/grpc/grpc/issues/3617#issuecomment-145370766.

Unfortunately I can't provide a reproduction, but I can provide a little more information:
OS: Xubuntu Linux 64 bit
GRPC Version: release_0_11_1
Protobuf Version: proto3 alpha 3
I've tried compiling the hello world client and server and found no issues.

Another data point: When GRPC is compiled with the dbg build config, I see the following error:
../nptl/pthread_mutex_lock.c:355: __pthread_mutex_lock_full: Assertion `(-(e)) != 3 || !robust' failed.

I've reproduced this error on OS X and got the following stack trace (LLDB format):
  * frame #0: 0x00007fff8dd3e0ae libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff893c0500 libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fff968a937b libsystem_c.dylib`abort + 129
    frame #3: 0x00000001013ca3bb libgpr.dylib`gpr_mu_lock(mu=0x0000000105801028) + 75 at sync_posix.c:48
    frame #4: 0x000000010140d78c libgrpc.dylib`lock(call=0x0000000105801000) + 28 at call.c:596
    frame #5: 0x000000010140e065 libgrpc.dylib`grpc_call_start_ioreq_and_call_back(call=0x0000000105801000, reqs=0x00007fff5fbfd610, nreqs=1, on_complete=(libgrpc.dylib`finish_batch at call.c:1555), user_data=0x00007fff5fbfda80) + 37 at call.c:1288
    frame #6: 0x000000010140f322 libgrpc.dylib`grpc_call_start_batch(call=0x0000000105801000, ops=0x00007fff5fbfd760, nops=1, tag=0x00007fff5fbfda80, reserved=0x0000000000000000) + 3042 at call.c:1735
    frame #7: 0x000000010167503b libgrpc++.dylib`grpc::Channel::PerformOpsOnCall(this=0x0000000101b25ac0, ops=0x00007fff5fbfda80, call=0x0000000103000550) + 155 at channel.cc:92
    frame #8: 0x00000001016750e7 libgrpc++.dylib`non-virtual thunk to grpc::Channel::PerformOpsOnCall(this=0x0000000101b25ac0, ops=0x00007fff5fbfda80, call=0x0000000103000550) + 55 at channel.cc:86
    frame #9: 0x0000000101685f87 libgrpc++.dylib`grpc::Call::PerformOps(this=0x0000000103000550, ops=0x00007fff5fbfda80) + 87 at call.cc:89
    frame #10: 0x000000010005d28e IntegrationTests_test`grpc::ClientReaderWriter<OutgoingEventPB, IncomingEventPB>::WritesDone() + 62
    frame #11: 0x000000010000c634 IntegrationTests_test`EventStreamHandler<int>::Close() + 68

Closing after not having heard about this issue for a while.


add functions to convert tf.events files to pd.Dataframe

[ ] add logdir2df and logdir2csv from tweetynet notebooks to summary_writer module
[ ] rename that module to tensorboard: clarity, a couple less letters to type, not just about summary writer itself
[ ] add unit tests for the functions in the module

looking at what I have in tweetynet notebooks now.
think I adapted originally from this or one of its antecedents.
should use "records" approach instead of making a dataframe from every row
https://github.com/theRealSuperMario/supermariopy/blob/master/scripts/tflogs2pandas.py
also not sure whether it's better to use long or wide form, i.e. have a "metric" column?
and I don't know what's going on with "not-scalar" stuff


Support dragging and dropping locations onto the sidebar

I keep finding myself wanting to drag-and-drop folders onto the locations sidebar (instead of opening prefernces). This is is a pretty natural paradigm for macOS, so we should support it.

            .onMove { source, destination in
                applicationModel.directories
            }
            .onInsert(of: [.folder, .url]) { index, items in

            }


⚠ Jenkins Plugins Artifactory has degraded performance

In cd14bb1, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1232 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 4b91804 after 12 minutes.


Endpoint not found: GET /deviation/{deviationid}

I cannot reach the "get deviation" endpoint from the DeviantArt Developers console nor directly using cURL.
I am getting the deviation ID from the deviation's url:

It gives me a 404 when I use the web console, but it feels like a "this api is missing" 404:

It gives me a 404 when I use cURL, and it's definitely an api missing 404:
I feel like that url has either broken or the documentation is incorrect.

That is not the deviantid. It should look like AAAAAAAA-E303-BEF1-DE89-698B122667E3

Oh, okay, that would explain it. I didn't realize I should be looking for a UUID. Thanks!
Do you have any suggestions on the best way to go from a given deviation on the website to discovering the deviation ID? I would prefer to avoid having to pull a user's gallery and iterate through each item--potentially crossing several pages--in an attempt to find the json object whose url matches the url of the deviation.

I found it. There is a meta tag on the deviation page. Thanks for pointing out my error.


Missing simulation.launch

When following the installation instructions on https://github.com/davetcoleman/ros_control_boilerplate and cloning the gazebo_ros_demos.git, it doesn't contain the rrbot_simulation.launch in it's control folder that is necessary for running the simulation demo. But it can be found manually when digging through the base of the git https://github.com/davetcoleman/ros_control_boilerplate/tree/indigo-devel/rrbot_control/launch

Thanks for reporting, this should be fixed in https://github.com/davetcoleman/ros_control_boilerplate/pull/14 once it is re-released into ROS disto and the build farm syncs


Text Sharing

Hi, Thanks for this plugin development.
I want to share text for android and iOS games.
So how to do that? I tried this but this not seam easy to me.
Please give some help in this.

It is giving me this Error "File Format not Supported".

Hi, I've taken over this repository, due to the age of the issue I'll be closing this issue
I've rewritten most of the plugin to be easier to use, and there is a high possibility that your issue has been fixed
Please download and see if your issue is still present. If it is please create a new issue and I'll get to it as soon as I can


API get_result() method does not inform when job fails

Feature details
When querying the API for the result of a job, the call get_result(job_id) returns None for any case that the job did not run.
If the job ran but it failed, it only prints a warning, but it returns None nevertheless. It would be useful for users waiting for their results if this is changed.
That is to say, the function behaves differently for results that need to be waited vs results that have failed and waiting for them is pointless.
Implementation
I suggest returning either False or raising an Exception
How important would you say this feature is?
1: Not important. Would be nice to have.
Additional information
No response

We suggest to use get_job() and from there get the result, instead get_result() directly. Nevertheless, in the future get_result() will ne deprecated


[Accessibility][Reflow] Error analysis tree map & Data analysis & Feature importance Reflow

Description

Note: 2D information is not required for reflow, such as error analysis tree
Error analysis tree map

Data analysis

Feature Importance

Checklist

[ ] I have added screenshots above for all UI changes.
[ ] I have added e2e tests for all UI changes.
[ ] Documentation was updated if it was needed.

Codecov Report

Merging #1678 (b2683ea) into main (cb46dc3) will increase coverage by 3.08%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1678      +/-   ##
==========================================
+ Coverage   88.88%   91.97%   +3.08%
==========================================
  Files         105       86      -19
  Lines        5553     4238    -1315
==========================================
- Hits         4936     3898    -1038
+ Misses        617      340     -277

Flag
Coverage Δ

unittests
91.97% <ø> (+3.08%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...iwidgets/raiwidgets/explanation_dashboard_input.py

raiwidgets/raiwidgets/dashboard.py

raiwidgets/raiwidgets/error_handling.py

...iwidgets/raiwidgets/model_performance_dashboard.py

raiwidgets/raiwidgets/interfaces.py

...dgets/raiwidgets/error_analysis_dashboard_input.py

raiwidgets/raiwidgets/error_analysis_constants.py

raiwidgets/raiwidgets/constants.py

raiwidgets/raiwidgets/explanation_constants.py

raiwidgets/raiwidgets/cohort.py

... and 9 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Fix typos in Plugins subdirectory

Found via codespell
Signed-off-by: luz paz<EMAIL_ADDRESS>

Rebased. Anyone care to review ?

bumping for exposure

rebased

Thanks! Merged with a few corrections of wrongly corrected typos.


Roadmap?

Is there a roadmap of things that need to be completed before the generator is deemed "ready for production?"

lol bit late now


T017: Website rendering is incorrect

The rendering of T017 (and only T017) is off on our website
https://projects.volkamerlab.org/teachopencadd/talktorials/T017_advanced_nglview_usage.html#Practical
Maybe this is related to saved widget states (though we do save them in other notebooks, too).
Let's wait a few days and check in again if the problem is still there.

@dominiquesydow

The rendering of T017 (and only T017) is off on our website
May be: https://github.com/nglviewer/nglview/issues/889

and FYI: Using view.gui_style = 'ngl' might mess up your website's font: https://github.com/nglviewer/nglview/issues/988
By the ways, great nglview tutorials. Thanks.

Hi, I think these are indeed related nglviewer/nglview#988. @dominiquesydow it'd be great if you manage to solve this!


Add upper bound on OrderedCollections

Where do we even use OrderedCollections? git blame suggests I added it when I re-wrote the MOI wrapper, so it was probably before we moved the implementation into CleverDict.
I wish Julia was like Go and threw an error if Project.toml includes an unincluded package.


[C4AEM] [Pictogram Card Enhancements]: Option to add a 1:1 icon app image to the pictogram card

Engineering info:

Status: These design modifications have been moved to Carbon for AEM (https://github.ibm.com/webstandards/carbon-for-aem)
Innovation Team FE JIRA Ticket: https://jsw.ibm.com/browse/HC-2483
DPO consulting engineer: N/A

Contribution type
Components
Contribution type (other)
No response
Intent
The interfaces team is working through how an integrations catalog can be displayed in a template form in AEM. We've identified a need to include a condensed card that displays an app icon and title in order to display the types of integrations products need to display on the site. We've identified that the pictogram card would be a good fit for this and that a few modifications would be needed in order for the card to display an app icon.
Solution
We've started to work with the Carbon and C4IBM.com teams to identify the best approach for including the app icon in the in the pictogram cards. There's been discussion in Figma, and @jeanservaas and @oliviaflory have given their feedback during office hours. While exploring these during office hours, some ways on how we could approve the accessibility of this card and better align it with Carbon was also discussed. Below are the proposed modifications to this card:

Include an option to have a 1:1 ratio app icon at 48px x 48px to display within the pictogram card in place of the pictogram
The pictogram / app icon will be displayed at the bottom the the card
Try to keep the card at a 2:1 ratio at desktop breakpoints
For pictogram cards that are clickable, we will need an arrow icon placed in the lower right of the card to indicate that it's clickable to users
Reduce the card headline size to body-long-02 to include more character count

Here's a link to the Figma file of work in progress: https://www.figma.com/file/WXXBuSLQYXMrFwTkONjbwN/Product-Skills-and-Integrations?node-id=542%3A28744
Below are some visual examples of work in progress

IBM Organization or Company you are part of
Interfaces Team: Our design teams main goal is to design, support and to create processes for templates in AEM for ibm.com
Code of Conduct

[X] I agree to follow this project's Code of Conduct
[X] I checked the current issues for duplicate issues

Per the discussion with @oliviaflory @RichKummer, we have decided to move these modifications to the Carbon for AEM library.
cc: @jwitkowski79

@proeung Hi! I have design specs for the updated pictogram card. https://www.figma.com/file/EYcGXWm9O71Ma97MT1yflL/Components?node-id=32%3A2437. Let me know if you need anything else!

@jwitkowski79 Thanks! Here's the JIRA ticket that I just created to track this work on the Carbon for AEM side. https://jsw.ibm.com/browse/HC-2483

Engineering info:

Status: These design modifications have been moved to Carbon for AEM (https://github.ibm.com/webstandards/carbon-for-aem)
Innovation Team FE JIRA Ticket: https://jsw.ibm.com/browse/HC-2483
DPO consulting engineer: N/A


add vietnamese translation

this is a good mod, so i want to support this small things

Thank you very much! Though just a concern in that line above


Generic OAuth2.0 Clients

What: An OAuth2.0 client that can be used with any OAuth 2.0 service (theoretically). It allows the user to define all of the URLs to make the relevant requests to. In addition, the user can define a custom AttributesDefinition (that extends the new GenericAttributesDefinition) to extract whatever profile details they expect.
Why: My organization uses pac4j for Google SSO in our apps, and we like how simple it makes the process. We also released our own internal OAuth provider and have migrated our Play applications to use that instead of each app having their own login screen to maintain. I created this because we would like to use pac4j with our internal OAuth provider, but needed more flexibility than a hardcoded Api. The purpose for this client is for use with internal OAuth2.0 services.
I also was looking into making BaseOAuth20Client and BaseOAuth20StateClient non-abstract with this functionality, but am limited on time. Perhaps that could be a future change.
Also, I'm not sure about tests for this. Perhaps I could create a test that configures this to authenticate with Google and run the same Google tests?

I like the idea. Maybe we'll need to do more, like being able to setup how the access token is sent to query the profile url (as a parameter or header), but this is a good first step for now.
See my comments. Plus you must fix the build, there are some Findbugs issues.
I agree with the test proposal:  create a RunXXX class in the run package which uses the Generic OAuth client and authenticates with Google.

Merging as is. I'll rename the DefaultGenericAttributesDefinition and move it in the core later on.


Snackbar with no action

Is there a way to show this with no action, so it's basically just an info notification (like the android counterpart?)

@stonesam92 I've implemented this on my branch - would you care for a PR? https://github.com/EddyVerbruggen/SSSnackbar


MINOR: [Java] Bump org.mockito:mockito-junit-jupiter from 2.25.1 to 5.12.0 in /java

Rationale for this change
Bump version now that Java 8 is deprecated. Follow up PR from https://github.com/apache/arrow/pull/39408.
What changes are included in this PR?

Bump mockito to 5.12.0

Are these changes tested?
CI
Are there any user-facing changes?
No

@github-actions crossbow submit -g java

Same as in the other minor Java bump PRs, this seems to be a C++ link error: https://github.com/ursacomputing/crossbow/actions/runs/9981283644/job/27584571994#step:6:4099 and it is not related to this change.

I agree, the failure isn't related. I'll go ahead and merge these since they look good otherwise!


failing with "dial tcp: lookup minio.docker on <IP_ADDRESS>:53: no such host"

Always fails at this point. I am new to it, please help.
Downloading source tar ball master....
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100   134    0   134    0     0    104      0 --:--:--  0:00:01 --:--:--   104
100 10.4M  100 10.4M    0     0   357k      0  0:00:29  0:00:29 --:--:--  294k
Unpacking tarball.
downloading the CLI tool ...
downloading cli for linux
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100   626    0   626    0     0    478      0 --:--:--  0:00:01 --:--:--   478
100 3391k  100 3391k    0     0   315k      0  0:00:10  0:00:10 --:--:--  487k
pulling the docker images short list...
Using default tag: latest
latest: Pulling from openwhisk/controller
Digest: sha256:aa2498cb3c7b7ae08271019c88f7832b4daf28b1da72fe96d8f6cd59e594eba5
Status: Image is up to date for openwhisk/controller:latest
Using default tag: latest
latest: Pulling from openwhisk/invoker
Digest: sha256:b5e1c96450df2d3bb4645bf322bcf84041b0d65d93746818a9314e5e6b939118
Status: Image is up to date for openwhisk/invoker:latest
Using default tag: latest
latest: Pulling from openwhisk/nodejs6action
Digest: sha256:c4afd017e382ba52e789fb63a8801998a758745e22746afcdbcb7441d85acf04
Status: Image is up to date for openwhisk/nodejs6action:latest
Using default tag: latest
latest: Pulling from openwhisk/dockerskeleton
Digest: sha256:833a76ab0e61b6ff32cff8baee0cab18791ce49f0e66d03e3303eb4abbe95951
Status: Image is up to date for openwhisk/dockerskeleton:latest
host ip address: <IP_ADDRESS>
checking required ports ...
... OK
... preparing api-gateway configuration
pinging minio...
.Creating network "openwhisk_default" with the default driver
Pulling minio (minio/minio:RELEASE.2018-07-13T00-09-07Z)...
Creating openwhisk_minio_1_1c9d6f246d33 ... done
...  Did you solve this already ?
checking required ports ...
 ... OK
  ... preparing api-gateway configuration
pinging minio...
Creating openwhisk_minio_1_1e6e41817cb1 ... done
 ... OK
2018/11/21 07:39:13 ERROR : Attempt 1/3 failed with 1 errors and: RequestError: send request failed
caused by: Head http://minio.docker:9000/api-gateway: dial tcp: lookup minio.docker on <IP_ADDRESS>:53: no such host
2018/11/21 07:39:59 ERROR : Attempt 2/3 failed with 1 errors and: RequestError: send request failed
caused by: Head http://minio.docker:9000/api-gateway: dial tcp: lookup minio.docker on <IP_ADDRESS>:53: no such host
2018/11/21 07:40:39 ERROR : Attempt 3/3 failed with 1 errors and: RequestError: send request failed
caused by: Head http://minio.docker:9000/api-gateway: dial tcp: lookup minio.docker on <IP_ADDRESS>:53: no such host
2018/11/21 07:40:39 Failed to mkdir: RequestError: send request failed
caused by: Head http://minio.docker:9000/api-gateway: dial tcp: lookup minio.docker on <IP_ADDRESS>:53: no such host
Makefile:204: recipe for target 'setup' failed
make: *** [setup] Error 1

@karun19 @bipinm I just submitted a fix for this. Could you both pull in the new update and see if this issue still persists?

@Juice10 I just tried devtools off master and had no issues deploying correctly. Will close speculatively as fixed.


(GH-200) Make SecurityProtocol changes immediately

Changes to the SecurityProtocolType should be made before creating any
request objects, otherwise those settings aren't applied to the request
object that already exists.
Relates to #200

@gep13 To make things easier to validate in the future, extra tests containing the URLs from #200 might be a good idea. But they would be harder to maintain as URLs could change and therefore the tests could fail.
@ferventcoder your thoughts?

@mkevenaar said...
extra tests containing the URLs from #200 might be a good idea

I had the same thought.  I have added some tests to include the URL's that came up in issue #200


Marked thread-spawning constructors as deprecated

Thread-spawning constructors hide errors and may lead to complex program state when a thread is declared.
The explicit Thread::start member function should be used to spawn a thread.
More info:
https://github.com/ARMmbed/mbed-os/pull/79
https://github.com/mbedmicro/mbed/issues/2024
cc @pan-, @niklas-arm, @0xc0170

Looks good.
Is there any link to our deprecation policy ?

Is there any link to our deprecation policy ?

Fair point, not yet. I would suggest stating explicitly in the message for how long it stays deprecated (removed by x.x.x release) - to track the lifetime of deprecated attributes.

LGTM


Add a skore.create_or_load

Is your feature request related to a problem? Please describe.
When iterating on my modeling, I might often rerun the same notebook, sometimes dropping/removing entire intermediary or debug runs in skore when I don't need them (by deleting the project on my disk). So, I often switch between skore.create() and skore.load().
Describe the solution you'd like
Instead, I'd rather have a single skore.create_or_load() function that I could reuse all the time.
Describe alternatives you've considered, if relevant
No response
Additional context
No response

The output of the skore.create function seems to be outdated. It returns the project.

Ok, I haven't noticed the overwrite option. It's good to have a fresh start at every run during debugging phases, but for iterating on metrics I have to catch the exception as you showed.
I still think having a single entry point like create_or_load() would make sense, so that users like me don't need to bother with the exception catching.

Or project = skore.create([...]) just loads if the project already exists?

Or project = skore.create([...]) just loads if the project already exists?

I disagree. It's important for user to explicitly know if he's working on an existing project, otherwise it could unknowingly overwrite important information.
The question is if we need create_or_load() or wheter letting the user try/catch is sufficient.

I like @augustebaum's suggestion.
And I agree with both of you:

try/catch is not the target persona here. (though it would be acceptable for ML Engineers: it depends on the situation)
We should avoid adding a new function.

And what about merging create and load in a open function ?
It's signature could be:
def open(project_path: str | Path="project.skore", create:bool=True):
    """Open a project given a project name or path.
    Parameters
        ----------
        project_path: str | Path
            The path where te project exists on disk.
        create: bool (defautl True)
            If set to false the function will raise an FileNotFoundError.
   """

I like @rouk1 proposal and more in favour of create name since it reflect to the create function in the API. An alternative is autocreate=True/False but I'm not sure it is better.

And what about merging create and load in a open function ?

@rouk1 There should be an overwrite option as well but otherwise I think this is ready for dev

The create and overwrite options should coexist.

def open(
    project_path: str | Path = "project.skore",
    *,
    auto_create: bool = True,
    overwrite: bool = False,
):
    """Open a project given a project name or path.

    Parameters
    ----------
    project_path: str | Path
        The path where te project exists on disk.
    auto_create: bool (default True)
        If ``True`` the function will create the projetc if it does not exists.
        If ``False`` the function will raise a FileNotFoundError if path dfoes not exist or is not a valid project.
    overwrite: bool (default True)
        If ``True``, overwrite an existing project with the same name.
        If ``False``, raise an error if a project with the same name already exists.

    Returns
    -------
    Project
        The corresponding Project instance.

    Raises
    ------
    FileNotFoundError
        Raised if:
        - path is not found and auto_create is set to ``False``
    ProjectCreationError
        Raised if:
        - project name is invalid
        - project already exists and overwrite is set to ``False``
    """

WDYT ?

I'm implementing this and having second thoughts about the name open, since it conflicts with the standard library function of the same name. Any thoughts? @rouk1 @MarieS-WiMLDS

🤔
Pillow style ?
from skore import Project

p = Project.open()

Let it go and encourage user to write ?
import skore as sk

p = sk.open()

Why did we drop an option like open_or_create?  access_project, or open_project?
Just dropping ideas.

What should happen if the user calls open("project.skore", create=True, overwrite=False) (the default), but project.skore already exists? Should we try to create a project (and thus fail), or should we load the existing project, which contradicts the create setting?
I'm starting to circle back to
# Equivalent to current `overwrite=True`
project = skore.create("my_project", existing="overwrite")

# New behaviour (could be the default IMO)
project = skore.create("my_project", existing="load")

Why did we drop an option like open_or_create? access_project, or open_project? Just dropping ideas.

I have a slight preference for one-word commands when possible, but these are all valid.

what about project = skore.load("my_project", default=<"raise"|"create"|"overwrite">)

What should happen if the user calls open("project.skore", create=True, overwrite=False) (the default), but project.skore already exists? Should we try to create a project (and thus fail), or should we load the existing project, which contradicts the create setting?

To me you should just open the existing project. For a user open should return a project no matter if it exists already or not. Also we should not delete user data when calling open. Deleting must be a conscious action.

Also we should not delete user data when calling open.

Are you saying the overwrite option should be removed?

Also we should not delete user data when calling open.

Are you saying the overwrite option should be removed?

Maybe yes, at least it's default value should be false. (imo)

I really like the overwrite option for testing! Let's keep the option but indeed, set the default value to false.

From this issue https://github.com/probabl-ai/skore/issues/1000, I wonder why don't we do skore.Project(name: "blabla", overwrite = True)? It would be an init function. instead of having a separate one.


Allow disabling authentication

This will allow running a solr server withouth authentication

Please resolve merge conflicts

Yes I noticed it. I will. Have to do some other stuff first ;-)

Done


How to test FCM::sendTo with Feature test

I have a project integrating Laravel-FCM to send push
I've created a helper class to work with FCM::sendTo and i want to test this class
I use snippets for testing from your README.md file
$numberSucess = 2;
$mockResponse = new \LaravelFCM\Mocks\MockDownstreamResponse(numberSucess);

$mockResponse->addTokenToDelete('token_to_delete');
$mockResponse->addTokenToModify('token_to_modify', 'token_modified');
$mockResponse->setMissingToken(true);

$sender = Mockery::mock(\LaravelFCM\Sender\FCMSender::class);
$sender->shouldReceive('sendTo')->once()->andReturn($mockResponse);

$this->app->singleton('fcm.sender', function($app) use($sender) {
	return $sender;
});

Running this in Test case return following error:
Mockery\Exception\InvalidCountException : Method sendTo(<Any Arguments>) from Mockery_2_LaravelFCM_Sender_FCMSender should be called
 exactly 1 times but called 0 times.

how can you solve this problem

As the error stated LaravelFCM_Sender_FCMSender should be called  exactly 1 times but called 0 times
You should call $sender->send('args here') in singleton callback

@saber13812002
I'm still waiting for a proper answer from repository owner


Owner channels

I'm having issues with the credentials file, I'm trying to use the docker method of setting this up, using 2 owner IDs and I've set it up multiple times so far but every time I do so it tells me that there are no owner channels set up in the credentials.json file. I've deleted the image and re-pulled it, I've even deleted the VM and remade that but none of it seems to change the fact that it is not reading owner channels for me and my friend and saying I might need to check the IDs.

Have you invited your bot to one of your servers?

Invite the bot to your server.

Bot won't work if it isn't already in a server.

You need to invite the bot before initializing. ;)


Tasks.json example is out of date

Environment data
dotnet --info output:
SDK do .NET Core (refletindo qualquer global.json):
Version:   2.2.300
Commit:    73efd5bd87
Ambiente de tempo de execução:
OS Name:     Windows
OS Version:  10.0.18362
OS Platform: Windows
RID:         win10-x64
Base Path:   C:\Program Files\dotnet\sdk\2.2.300\
Host (useful for support):
Version: 2.2.5
Commit:  0a3c9209c0
.NET Core SDKs installed:
2.1.700 [C:\Program Files\dotnet\sdk]
2.2.300 [C:\Program Files\dotnet\sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.2.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
VS Code version: 1.35.1
C# Extension version: 1.19.1
Steps to reproduce
Start a new project using cli dotnet new webapi -o src/WebAPI -n Name.Of.YourProject. Open it
with Visual Studio Code.
Ignore the popup asking to create assets to build.
Try to use this documentation:
Check debbuger.md
When article talks about 'Creating configuration files manually'
It is using an out of date json file.
Check json exemple
After create a file in vscode, actual version, a message says:

Task version 0.1.0 is deprecated. Please use 2.0.0

Expected  behavior
Use the correct exemple file to configure vscode.
Actual behavior
The example is not adequated to version 2.0.0 or above.

Thanks for taking the time to report this! I updated the example.


[feat] Quest: NFT mint quest

Create a quest for minting an NFT. Have the quest take the nft's token_id as call data, and use a dispatcher on the canvas_nft contract to check that the NFT's metadata contains the user as minter on the corresponding NFT metadata.
Also create corresponding tests.
References

NFT Metadata
NFT store to dispatch
Example Quest

Thanks for volunteering for this issue @mubarak23
Please comment here so I can assign you, if you're still interested.

Hey @b-j-roberts  is it up for grab or not?

Yes, kindly assign it to me
thank you

Hey @b-j-roberts is it up for grab or not?

@TAdev0 Mubarak had already expressed interest in this task, I pinged you on another similar issue if you'd like it instead.

sure, i didnt see any comment before your first one, i thought you commented the wrong issue. Sorry.

my nftQuest is going to implement IQuest trait  right @b-j-roberts


1D pooling layers

Need to implement 1D pooling layers for average and maximum techniques.

1D conv layer is also not included in the current release either. This is being tested in the message passing development. This will be ported over.

input2d_layer_type also needs to be added to handle conv1d_layer_type input shape


Enable multi input and add support for gitlab input

Make gitfortress support multiple input and allow gitlab to be an input

Codecov Report
Attention: Patch coverage is 21.69811% with 83 lines in your changes missing coverage. Please review.

Project coverage is 57.29%. Comparing base (bbc806f) to head (7e5c618).

Files with missing lines
Patch %
Lines

internal/interfaces/gitlab/gitlab.go
0.00%
43 Missing :warning:

cmd/app/main.go
0.00%
32 Missing :warning:

config/config.go
68.18%
5 Missing and 2 partials :warning:

internal/interfaces/system_git/git.go
85.71%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #29      +/-   ##
==========================================
- Coverage   66.49%   57.29%   -9.20%
==========================================
  Files          13       14       +1
  Lines         585      733     +148
==========================================
+ Hits          389      420      +31
- Misses        151      266     +115
- Partials       45       47       +2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Read/update model states in native (sigma) coordinates

In order to use neuralgcm for data assimilation, we need access to the model variables on sigma levels (plus surface pressure).  The api.PressureLevelModel provides access to decoded data on pressure levels.   I'd like to add the ability to extract sigma-level data and surface pressure from a prediction, then restart a model prediction using updated sigma-level/surface pressure fields.   How would you recommend doing this with the public API?
@nniraj123 @frolovsa

Are you interested in particular version of NeuralGCM?
The internal state of the model (outputs of PressureLevelModel.encode, inputs/outputs of encode and inputs/first output of unroll) are actually already on sigma coordinates, e.g.,
>>> jax.tree_util.tree_map(np.shape, final_state)
ModelState(state=StateWithTime(vorticity=(32, 256, 129), divergence=(32, 256, 129), temperature_variation=(32, 256, 129), log_surface_pressure=(1, 256, 129), sim_time=(), tracers={'specific_cloud_ice_water_content': (32, 256, 129), 'specific_cloud_liquid_water_content': (32, 256, 129), 'specific_humidity': (32, 256, 129)}), memory=StateWithTime(vorticity=(32, 256, 129), divergence=(32, 256, 129), temperature_variation=(32, 256, 129), log_surface_pressure=(1, 256, 129), sim_time=(), tracers={'specific_cloud_ice_water_content': (32, 256, 129), 'specific_cloud_liquid_water_content': (32, 256, 129), 'specific_humidity': (32, 256, 129)}), diagnostics={}, randomness=RandomnessState(core=(256, 128), nodal_value=(256, 128), modal_value=(256, 129)))

These variables are stored as spherical harmonic coefficient, so they need to be transformed back into real space for visualization, e.g., to visualize temperature near the surface in the demo notebook:
temp_variation = neural_gcm_model.model_coords.horizontal.to_nodal(final_state.state.temperature_variation)
xarray.DataArray(temp_variation[-1, :, :], dims=['x', 'y']).plot.imshow(size=4, aspect=2, x='x', y='y')

There a few tricks for conversion (e.g., to handle units and reference offsets for temperature). We'll work on documenting these as part of https://github.com/google-research/neuralgcm/issues/11.

@shoyer we're most interested in the stochastic version, since we'll be running ensembles for data assimilation.  What we need is to:

run the model ensemble for 9 hours, saving the trajectory state every 3 hours (or perhaps 1 hour).
extract the model state on sigma levels from the trajectory, use it compute the "observation equivalents" for observations valid between 3 and 9 hours.
use the model states (in grid space and also in observation space) to compute an update to the model state at 6 hours using the Ensemble Kalman Filter algorithm.
use these updated model states to re-initialize the model, and then rinse and repeat.
You example looks like exactly what we need to do steps 1-3.  For step 4, we need a way to re-encode the updated sigma-level states.

@shoyer we've got an initial version of a cycling EnKF for neuralgcm running and assimilating surface pressure observations.  The results suggest that the orography that we are using for the forward observation operator (from aux_features['xarray_dataset']['geopotential_at_surface'] is perhaps not the actual orography the dycore uses (and is therfore not consistent with the surface pressure in the model state).  Does the dycore use a filtered version of that orography, and if so how can I access it?

@jswhit Indeed, for the dycore we use a learned filtered version of orography that is a bit smoother. You can extract it from the trained models:
import dataclasses
import functools
import pickle

import gcsfs
from dinosaur import spherical_harmonic
import haiku as hk
from neuralgcm import api
from neuralgcm import orographies
import numpy as np
import matplotlib.pyplot as plt
import xarray

gcs = gcsfs.GCSFileSystem(token='anon')

@hk.transform
def get_orography():
  base_orography = functools.partial(
      orographies.FilteredCustomOrography,
      orography_data_path=None,
      renaming_dict=dict(longitude='lon', latitude='lat'),
  )
  orography_coords = dataclasses.replace(
        neural_gcm_model.model_coords,
        horizontal=spherical_harmonic.Grid.with_wavenumbers(longitude_wavenumbers=126)
  )
  return orographies.LearnedOrography(
      orography_coords,
      neural_gcm_model._structure.specs.dt,
      neural_gcm_model._structure.specs.physics_specs,
      neural_gcm_model._structure.specs.aux_features,
      base_orography_module=base_orography,
      correction_scale=1e-5,
  )()

model_name = 'neural_gcm_stochastic_1_4_deg_v0.pkl'

with gcs.open(f'gs://gresearch/neuralgcm/03_04_2024/{model_name}', 'rb') as f:
  ckpt = pickle.load(f)

neural_gcm_model = api.PressureLevelModel.from_checkpoint(ckpt)

dycore_coords = spherical_harmonic.Grid.with_wavenumbers(longitude_wavenumbers=126)

learned_correction = neural_gcm_model.params[
    'stochastic_modular_step_model/~/stochastic_physics_parameterization_step/'
    '~/custom_coords_corrector/~/dycore_with_physics_corrector/~/learned_orography'
]['orography']

learned_orography_modal = get_orography.apply(
    {'learned_orography': {'orography': learned_correction}}, rng=None
)
learned_orography = neural_gcm_model.from_nondim_units(
    dycore_coords.to_nodal(learned_orography_modal), units='meters'
)

default_orography_modal = get_orography.apply(
    {'learned_orography': {'orography': np.zeros_like(learned_correction)}}, rng=None
)
default_orography = neural_gcm_model.from_nondim_units(
    dycore_coords.to_nodal(default_orography_modal), units='meters'
)

# learned orograpy
xarray.DataArray(learned_orography, dims=['x', 'y']).plot.imshow(x='x', y='y', aspect=2, size=4, robust=True, vmin=0)

# default orography
xarray.DataArray(default_orography, dims=['x', 'y']).plot.imshow(x='x', y='y', aspect=2, size=4, robust=True, vmin=0)

Thanks @shoyer, using the learned orography really moved the needle for the data assimilation.

For future reference, details of how to work with our model on sigma coordinates are now described in the NeuralGCM documentation: https://neuralgcm.readthedocs.io/en/latest/trained_models.html

@shoyer for the deterministic 0.7 degree model it looks like the learned orography is in stochastic_modular_step_model/~/dimensional_learned_weatherbench_to_primitive_with_memory_encoder/~/learned_weatherbench_to_primitive_encoder_1/~/learned_orography and correction_scale should be 2.e-6.  Is this correct?

@jswhit good catch! Yes, we did switch to using slightly different values for that resolution.
Probably worth noting that in the model there are up to 3 different orography values (1 in encoder, 1 for the simulation and 1 in decoder).
I believe that for the model that @shoyer showed has fixed orography (modal representation of conservatively regridded ERA5 orography) used by the encoder and decoder and learned orography in the advance step (extracted by @shoyer's snipped)
In the 0.7 degree model all three components were learned. In a few experiments we found that in this setting decoder tends to keep more sharp features in the orography compared to the encoder. The variables that you've tracked down are looking at the encoder orography. Please let me know if the other one under "dycore_with_physics_corrector " doesn't show up.
We will also prioritize easier access to these in the new API that we are starting to work on.

Thanks @kochkov92, I was able to find 'stochastic_modular_step_model/~/stochastic_physics_parameterization_step/~/custom_coords_corrector/~/dycore_with_physics_corrector/~/learned_orography'.  However, when I use it with the get_orography function @shoyer provided
@hk.transform
def get_orography():
  base_orography = functools.partial(
      orographies.FilteredCustomOrography,
      orography_data_path=None,
      renaming_dict=dict(longitude='lon', latitude='lat'),
  )
  orography_coords = dataclasses.replace(
        neural_gcm_model.model_coords,
        horizontal=spherical_harmonic.Grid.with_wavenumbers(longitude_wavenumbers=256)
  )
  return orographies.LearnedOrography(
      orography_coords,
      neural_gcm_model._structure.specs.dt,
      neural_gcm_model._structure.specs.physics_specs,
      neural_gcm_model._structure.specs.aux_features,
      base_orography_module=base_orography,
      correction_scale=2.e-6,

orogpath='stochastic_modular_step_model/~/stochastic_physics_parameterization_step/~/custom_coords_corrector/~/dycore_with_physics_corrector/~/learned_orography'
learned_correction = neural_gcm_model.params[orogpath]['orography']
learned_orography_modal = get_orography.apply(
    {'learned_orography': {'orography': learned_correction}}, rng=None

ValueError: 'learned_orography/orography' with retrieved shape (65023,) does not match shape=(66047,) dtype=<class 'jax.numpy.float32'>

please disregard my previous question - it works if I use longitude_wavenumbers=254

For anyone who stumbles upon this in the future, take a look at the "deep dive" section of the documentation:
https://neuralgcm.readthedocs.io/en/latest/deepdive_into_models.html


Updated examples spring 2021

This PR updates the examples to include all proposed changes for spring 2021.
The examples were expanded to include additional road events modeled after the scenarios here.  Below is a summary of the examples that were created:

Scenario 1  - Simple Workzone

First work zone shown with basic lane information
Second work zone shown with detailed lane information
Third work zone showing a work zone broken into multiple road events

Scenario 2 - Lane Shift (simple case)

Single work zone with lane shift

Scenario 3 - Shoulder Closure

Single work zone but two road events (one in direction of shoulder closure and second in opposite direction)

Scenario 4 - Road Closure with Detour on Arterial

Single work zone with a detour

Scenario 5 - Work Zone with Bridge Work Overhead

This is a simple work zone but added a recurring component.  Four additional events were created with a pending status

Local Access Example

Single work zone with local access created.  Also included bike lane in example for reference.

Throughout the examples, various optional fields were used as well as a variety of enumerations including the verified location/time.  I also tried to remove all depreciated fields. I expect that addition examples can be added in the future based on feedback.
I included reference to the new examples but didn't refer to the scenarios in the discussions.  I think it would be valuable to add the link if this is acceptable.

@sknick-iastate wow! Lots of new examples—impressive content.

@sknick-iastate I checked a few of these example files again the schema, using https://www.jsonschemavalidator.net/s/ykYbPPxM, and they did not pass, mostly due to the formatting of road_names. Can you validate each example against the current schema (wzdx_vNext_feed.json in the spring-2021-release branch) and make the necessary changes so that each pass?

Thanks @j-d-b !  I went through and resolved the issues.


Check sessions in sessions page

Overview (Required)
Currently, the user can check/uncheck sessions from SessionDetail page.
When the user long tap the session item in Sessions page, it's better to check/uncheck the session.
It's just my idea.

I'm gonna take this one, if no one work on this.

Maybe no one is working on. Please go ahead!


404 Site antd-admin.zuiidea.com is not served on this interface

404 Site antd-admin.zuiidea.com is not served on this interface

It's work now.


Changes to allow applications to set OIDC scopes

So looks like this is targeted for the current broker release?

Can we add a description to PR covering what's being added, how/why?

Closing this as this is no longer required for Teams