[RFC][mxnet 2.0][item 10.1] MXNet Imperative Op Invocation Overhead

MXNet imperative operator invocation overhead is as large as 30-60us, which is significant compared to the official NumPy operators with ~600ns overhead. This has negatively impacted the performance of applying MXNet to the models where many operators' kernel runtime duration is short, especially in the area of classic machine learning. We plan to address the problem in two steps:

Short term: Use pybind11 to replace Python op API and ctypes/c api. Preliminary experiments show that the pure Python-C++ turnaround time by using Pybind is between 400-600ns, while the current Python op API using ctypes/c api costs more than 10us. We believe with the correct implementation, we can reduce the op invocation overhead to 2us including the time on FFI and engine.

Long term: Adopt Python's C extension interface. NumPy did this by developing its own C API. This provides considerably less overhead compared to other solutions. However, it would cost much more engineering efforts by integrating this with our existing operator workflow in C++.

@hzfan @hgt312

Hi @reminisce , I have several questions.

Does the time of asynchronous execution cover that of invocation overhead?
MXNet Imperative mode creates a computational graph, but NumPy computes operations directly. How long will they take?

To my knowledge, if using pybind11 or Python’s C extension, it need to re-build interfaces to support other version of Python.

I have a good idea, I think we can build our own custom ctypes system or some other middleware to solve it.

@wkcn that can be done as part of setup.py locally on the users machine, or would you see any problem?

How much engineering efforts does it take to do (1)?

I think we can first replace the code-gen here https://github.com/apache/incubator-mxnet/blob/521c477ad32864d887481abf6c53acae3b717cf6/python/mxnet/ndarray/register.py#L115-L269 with pybind11.

I have another candidate that would highly recommend: adopt TVM's FFI convention.
The historical problem of MXNet FFI was the blowing amount of the C API bindings as we add new features. This creates a huge amount of maintenance burden.
The real problem was not really about which FFI system to adopt(cython and pybind are fine in that end, except for the cost of compilation), but more of the cost to maintain the FFI. MXNet used to have a fast cython binding, but that was abandoned because we keep add new APIs we cannot keep up both ctypes and cython.
When developing TVM we learnt from the lesson and restrict the API to a limited set of runtime APIs that does not change, and have a stable cython, ctypes binding for them. The runtime support a type-erased function(PackedFunc), which can be efficiently called from any of the frontend language, and all the APIs are exposed through the PackedFunc. On the python side an additional wrapping is created for better documentation and call into the PackedFunc. See more in https://docs.tvm.ai/dev/runtime.html The system works great for over a few years now.
Of course I understand there has been legacy issues in MXNet that is why I did not bring this proposal up. But given this is a proposal for 2.0, I would encourage everyone to give a serious thought about this possibility.

@tqchen thanks for sharing this! Is there any reference benchmark result that stress tests the ffi overhead?

I don't have any benchmarks at hand, but would be great if someone can help creating one with cython enabled.

@reminisce do you think we should also consider/benchmark tvm ffi? It's used in MNM, and probably having that for mxnet 2.0 will make migration to MNM easier.

OK here is a script to quickly check
import timeit

setup = """
import tvm
x = tvm.nd.array([0])
y = tvm.nd.array([1])
nop = tvm._api_internal._nop
"""
timer = timeit.Timer(setup=setup,
                     stmt='nop(x, y)')
timer.timeit(1)
num_repeat = 1000
print(timer.timeit(num_repeat) / num_repeat)

Results on my laptop
$ TVM_FFI=cython python3 test.py
9.749500000000299e-08
$ TVM_FFI=ctypes python3 test.py
7.226789000000011e-06

@tqchen can you also copy the results you posted in #14883 to here? This shows convincing benchmark results that tvm ffi is comparable with numpy.

I think interested folks can follow the last post here https://github.com/apache/incubator-mxnet/issues/14883#issuecomment-567726765 the general takeaway seems to be that the overhead is quite close to numpy's one

@tqchen Thanks for sharing the benchmark results. We did consider using TVM FFI as a candidate and I strongly agree with your suggestion on making a limited set of runtime API for sustainable maintainability. The Python op API overhead in general is more obvious when passing Python native data structures to the backend, such as lists, tuples, etc. I modified your script by passing a Python tuple as an argument and the overhead is around 19us with Cython enabled, while pybind is normally less than 400ns, and numpy comes with even much lower overhead. Did I make any mistakes in coding up the test script or it's something that can be addressed in TVM FFI? Thanks.
import timeit

setup = """
import tvm
#x = tvm.nd.array([0])
#y = tvm.nd.array([1])
nop = tvm._api_internal._nop
"""
timer = timeit.Timer(setup=setup,
                     stmt='nop((1, 2, 3, 4))')
timer.timeit(1)
num_repeat = 1000
print(timer.timeit(num_repeat) / num_repeat)

First of all, it would really be great if the issue of native data structure could been bought up at the first place in the RFC :)
Every design has a tradeoff, and most of the TVM FFI's design started as an artifact of of Amdahl's law.  In particular, it is interesting to ask how much native python data structure we need. e.g. most of the fast path(e.g. add/sub) and common operators the problem does not involve tuple as an argument.  As a layman example, I don't care if my save to json function takes 100us to finish(because that is not going to be bottleneck of my program).
When tuple and strings are involved, the typical operators (such as conv2d) are usually large. Moreover, when such operators are constructed in a two phase manner, or passed through an partially specialized annotator that translates the program, the cost of the constant tuple are only constructed once rather than in a loop (just like the following program).
class Module:
      def __init__(self):
          self.conv2d = conv2d(kernel=(2,2))

      def forward(self, x):
          return self.conv2d(x)

Of course, if we really say that "hey, we want the int tuple case to be fast", just like the case of copy in #14883 .  We can introduce native tuple objects into the TVM FFI to improve the performance of the passing tuple as arguments.
The general design philosophy is we only need make the things that needs to be fast fast. The FFI can handle objects that it recognizes efficiently, while have a reasonable slow fallback for cases that are not necessarily the bottleneck.
As I said in the beginning of the post, the spectrum of "fast argument" should really be discussed in the RFC at the first place. It would be great if we can have a more constructive discussion about these use cases, then add technical reasoning, before reaching a verdict.

I think we need to support tuple if we are going to adopt the PackedFunction approach. It is used in some functions like split, hsplit, reshape, transpose.

@tqchen Thanks for explaining things inside out. Please know that I'm not against TVM FFI design. In fact, it's great to know that you think passing Python native data structures can be accelerated by engineering through TVM FFI. This is vital for keeping the future MXNet runtime API in a limited set for scalable and sustainable maintainability.
Putting the design decision aside, I want to share that there has been extremely strong motivation and need of squeezing out the latency of passing Python native data structures in op interface. Since MXNet is embracing NumPy compatibility, we want to get the op invocation performance on par with NumPy to be appealing to classic machine learning community. We have compared the performance between MXNet and NumPy using a bunch of classic machine learning models and found that optimizing passing Python native data structures is critical for MXNet to be on par with NumPy. Even for deep learning itself, this is also important in some applications. In https://github.com/apache/incubator-mxnet/pull/16716, reset_arrays was corroborated to get rid of passing tuple objects too many times in mx.nd.zeros so that the training performance can be increased by 5%.
With that being said, please allow me to summarize what we have reached here. I think we are aligned on exploring TVM FFI to have a clear engineering view of accelerating passing Python native data structures as arguments. We can start from tuples, and extend the findings to lists and strings later.
Thank everyone for a great discussion and sorry for the late responses since I have been on vacation this week. I didn't expect this task item of PoC to become a full-fledged RFC that has involved this many interested folks. I will be sure to make the post more descriptive and self-explanatory next time. Have a nice holiday! :)

To followup on this thread about brining native support to tvm ffi, I will first discuss the ways to address tuple, and then discuss some of the pros and cons.
First of all, we all know that at a time point we are going to translate the python data structure we know into C++. The main question is where that translation can happen. In the pybind case, the translation happens in the c++ side by directly passing pyobject to the c++. In the case of cython, the translation happens in the C API level. In the case of TVM FFI, the translation can happen at a python wrapping.
The following code gives two examples of such translation(myempty0, and myempty1), the support for myempty1 requires one additional commit in this branch
In the first approach(myempty0), we directly unpack the tuple as positional arguments, and encode the data structure as a flattened argument. In the second approach(myempty1), it first creates a IntTuple object that the C++ side can recognize and then pass it to the C++ side. Note that at the moment all the operations are done through python, if there is a concern in terms of wrapping, we can certainly bring some of them into cython.
We could introduce a third approach(myempty2): which directly passes a PyObject* to the c++ side, then the developer can additional packages(e.g. pybind or related) to process the tuple. This third approach would achieve the same perf as pybind. However, there are some trade-offs, see discussion section.
import timeit
import tvm

nop = tvm._api_internal._nop

setup = """
import tvm
x = tvm.nd.array([0])
y = tvm.nd.array([1])
nop = tvm._api_internal._nop
"""
timer = timeit.Timer(setup=setup,
                     stmt='nop((1, 2,1))')
timer.timeit(1)
num_repeat = 1000
print("tvm.nowrap:", timer.timeit(num_repeat) / num_repeat)

setup = """
import numpy as np
"""

timer = timeit.Timer(setup=setup,
                                   stmt='np.empty((1,2,1))')
timer.timeit(1)
print("numpy.emmpty:", timer.timeit(num_repeat) / num_repeat)

def myempty0(shape):
    return nop(*shape)

def myempty1(shape):
    return nop(tvm.container.IntTuple(*shape))

setup = """
import numpy as np
import tvm
from __main__ import myempty0, myempty1
"""

timer = timeit.Timer(setup=setup,
                                  stmt='myempty0((1,2,1))')
timer.timeit(1)
print("tvm.myempty0:", timer.timeit(num_repeat) / num_repeat)

timer = timeit.Timer(setup=setup,
                                   stmt='myempty1((1,2,1))')
timer.timeit(1)
print("tvm.myempty1:", timer.timeit(num_repeat) / num_repeat)

Here are results on my computer:
$ TVM_FFI=ctypes python test.py
tvm.nowrap: 5.209704500000001e-05
numpy.emmpty: 2.8674499999997715e-07
tvm.myempty0: 7.312735000000015e-06
tvm.myempty1: 1.3553711000000024e-05
$ TVM_FFI=cython python test.py
tvm.nowrap: 1.3689438999999998e-05
numpy.emmpty: 2.86522999999983e-07
tvm.myempty0: 1.7771199999999653e-07
tvm.myempty1: 9.764679999999804e-07

As we can see, the myempty0 was quite fast. myempty1 was a bit slower due to the creation of the object(but still within the order of magnitude that we can use), in the case where we want to keep a gradient tape,  myempty1 may not be a bad approach as the data structure is created.
Discussion
As explained in the beginning, the real question was where should the wrapping happen.
In the case of TVM, usually the wrapping happens at the native language(python) level, because we know there is a need of the python side wrapper for better code, type checking and docs. The translation forces the python arguments into arguments that can be recognized by the runtime.
In the case of pybind, the translation happens at the C++ level, by calling into the python C API(the myempty2 approach is similar to this one).
The advantage of exposing PyObject and related operations to the c++ level is certainly the deferred marshaling of data structures. On the other hand, such approach directly ties the FFI with python. It  means other language frontends can no longer take benefit of the new FFI. On a similar direction, if we want to package some of the functions into a minimum runtime that is independent of python, we can no longer do that. This is why while in theory we could bring PyObject(or a related Proxy) to TVM runtime, we have not done so far.
Of course this is an interesting tradeoff, and everyone is welcomed to discuss their thoughts.

Some additional thoughts along the line:

It would be great to discuss what are the collection of "fast path" data structures
Given that this looks like a proposal, perhaps it should be tagged with an RFC so that it can be  mirrored to dev@

@reminisce Do you have data on what is the time split between the FFI itself and engine scheduling inside MXNet backend?

For the "fast path" structures that we need to support, I'm considering the following:

py_slice, Ellipsis for basic indexing.

Also, there are two scenarios when list will be involved:

For concatenate, the arguments may be wrapped in a python list.
In the case of split and hsplit, the return value can be a list.

@sxjscience here are some quick thoughts (of course passing pyobject kind of "solves" the problem, so I am discussing the wrapping that can be done through the tvm ffi).

concatenate seems can be achieved through the python side wrapping concat-> concat_internal(*args)
return list value can be addressed by introducing a Tuple object to tvm runtime, we recently did that.

The py_slice is the most tricky case, my guess is that it could be accelerated through a cython layer that translate the slice into flattened representation, of course it is not too ideal and pybind maybe better for this case if we only want to handle it through c++.

@ptrendx I benchmarked mx.nd.zeros((1,), out=out) in async engine. The total time including FFI and engine scheduling is 40us, and FFI itself takes 21us.

Ok, so it is about 50:50 split. Is there also work underway to profile what is the reason of the time spent in the engine?

@ptrendx Yes, there is an effort of profiling engine code flow using VTune. We hope the exercise can pinpoint the hotspots that contribute to the most part of latency. Further time split for pure C++ part between setup code (shape/type inference, memory allocation, dependency setup) and op scheduling is also around 50% vs. 50%.
For the "fast path" data structures, I'm summarizing the items as follows (including the ones suggested by @sxjscience):

tuple and list since they can be interchangeable in NumPy semantics to represent shapes and axes.
str because einsum has this parameter and the op can be intensively used in transformer models.
py_slice, Ellipsis, None for basic indexing. We can do one step further by moving the whole indexing dispatch logic to backend.
np scalars.
mx.context.Context. One call of mx.cpu() can be as large as 600ns using ctypes. One thought is do it in the pybind way by creating a Python binding for the backend Context class.
np.dtype. Similar to Context.

The following fast-path can be addressed in the TVM FFI:

tuple``, list``` via translation in python/cython side (see benchmark above)
str is already fast (seem benchmark above)
Context can be quite fast if the object is a tvm object, around same mangitude as passing NDArray
np.dtype can get around by by str conversion, or introduce a type structure, TVM FFI support DLDataType natively
None: natively supported by FFI

The following items needs to be discussed

py_slice, Ellipsis Can be supported by adding pyobject support, however that introduces dispatching into the FFI layer(making the function not accessible to other language frontends). Would be interesting to discuss alternatives

After some thoughts along the direction, I find a better and fun answer to the above question: support tuple/ellipsis/slice in tvm ffi effectively.
I quickly hacked up a POC in https://github.com/tqchen/tvm/tree/pyffi that supports the following benchmark script(disclaimer: it is only a POC so not intended for use or fully optimized, but it demonstrates all the technical flows necessary to make a fully functioning FFI).
import timeit
import tvm
nop = tvm._api_internal._nop

setup = """
import tvm
nop = tvm._api_internal._nop
"""
timer = timeit.Timer(setup=setup,
                                  stmt='nop((None,..., slice(0, 100, 2)))')
timer.timeit(1)
num_repeat = 1000
print("tvm.tuple_slice_ellipsis_combo:", timer.timeit(num_repeat) / num_repeat)

setup = """
import numpy as np
"""

timer = timeit.Timer(setup=setup,
                                  stmt='np.empty((1,2,1))')
timer.timeit(1)
print("numpy.emmpty:", timer.timeit(num_repeat) / num_repeat)

setup = """
import tvm
nop = tvm._api_internal._nop
"""
timer = timeit.Timer(setup=setup,
                                  stmt='nop("mystr")')
timer.timeit(1)
num_repeat = 1000
print("tvm.str_arg:", timer.timeit(num_repeat) / num_repeat)

On my laptop(macbook 13inch), the results are as follows
$ TVM_FFI=cython python benchmark_ffi.py
tvm.tuple_slice_ellipsis_combo: 4.615739999999924e-07
numpy.emmpty: 2.7016599999998834e-07
tvm.str_arg: 2.3390799999997714e-07

What is Implemented in the POC
In the POC, we introduced specific objects for Ellipsis, Slice and Tuple(already supported in ADT). During a PackedFunc call, a python tuple/ellipsis/slice was  converted into the object that is supported by the backend. We implemented a cython version(the previous recursive conversion was in python) to back it up.
The reason that we are able to create Object in the cython side is because all TVM object has been recently converted to be POD-C compatible, so the object can be created in the cython side without crossing DLL boundary and passed to the c++ backend.
We can see from the benchmark that the cost of such deep-copy was at a reasonable level. We also only used the default memory allocator, so there could be space for further improvements.
Discussions
Please also see tradeoff discussions in the last post. As we can see, the main difference here is where to do the conversion, and whether do we do lazy/deep copy:

In the case of pybind: conversion is happened in the c++ side, data structures are lazily created.
In the case of the POC: conversion is happened in cython, data structures are deeply translated into another in-memory format.

The laziness certainly avoids a copy in cases where we do not necessarily need to book-keep the created argument. On the other hand, supporting a common data structure in the c++ side means the binding can potentially be reused by other language frontends.

Thank @tqchen for sharing the PoC code within such a short timeframe. :) The numbers look promising even with Python native objects deeply copied. Pybind performs deep copy by default unless the receiving object in C++ end is marked as opaque so that the Python object passed by reference. That is often used for propagating large object changes from C++ to Python. In our op invocation use cases, there has been no such urgency of introducing this level of complexity so far since the Python objects are small and parameter passing is a one-way trip. The 300ns overhead should give us a good start to squeeze the total overhead into the 2us range. If there is really a need of passing PyObjects in the future, we can always add that with a compile flag option. I think it's worth following this branch to integrate TVM FFI with MXNet op invocation flow to get more comprehensive benchmark results.

Following this branch I made a simple POC on MXNet side (code here). It turns out that passing a python tuple and receiving it as TShape takes around 200ns.

@hzfan thanks for implementing a poc:) However, these is a subtle but important difference which worth discusses here :) I will use cython-ffi to refer to the above approach, and tvm-ffi to refer to tvm's approach

In cython-ffi, both data structure and functions are exposed, this means a in order to grow the set of functions, we need to expand the set of C API. In another words, we need to grow the set of FFI API as we add more functions
In the tvm-ffi, the set of C API is fixed, and only data structure constructions are exposed to the cython side, given that a set of supported data structures are also fixed. In this way, we do not have to grow the set of FFI API as we add functions.
Another subtle point is that we are passing data structure across dll boundaries. In the case of cython-ffi, it could be a c++ container(Tuple). TVM's object structure is designed to be C ABI compatible, which allows us to construct in one dll and pass to another, however it is not necessarily true for all c++ classes. There is a potential danger when passing c++ container across DLL boundaries(when two dll has different allocator, calling push_back in another dll could cause error).

The difference again boils down to the design point of what is a clear cut of FFI conventions. Ideally, it would be: a stable set of C API and container structures that does not change over time.

What's the point of having an API if you type erase it? Then you might as
well have a single function API with a type erased callback name to select
the function to call. In the end you move the burden away from the API to
the callers and inside the API to the dispatchers. For going this route of
uber-clever template tricks to generate code, I think it's better to just
put in place proper code generation for maintainability. Could you provide
a bit more details about tradeoffs? Everything has tradeoffs, I don't
believe any solution which is sold as a panacea, there's no silver bullet.
On Thu, Dec 19, 2019 at 10:21 AM Tianqi Chen<EMAIL_ADDRESS>wrote:

I have another candidate that would highly recommend: adopt TVM's FFI
convention.
The historical problem of MXNet FFI was the blowing amount of the C API
bindings as we add new features. This creates a huge amount of maintenance
burden.
The real problem was not really about which FFI system to adopt(cython and
pybind are fine in that end, except for the cost of compilation), but more
of the cost to maintain the FFI. MXNet used to have a fast cython binding,
but that was abandoned because we keep add new APIs we cannot keep up both
ctypes and cython.
When developing TVM we learnt from the lesson and restrict the API to a
limited set of runtime APIs that does not change, and have a stable cython,
ctypes binding for them. The runtime support a type-erased
function(PackedFunc), which can be efficiently called from any of the
frontend language, and all the APIs are exposed through the PackedFunc. On
the python side an additional wrapping is created for better documentation
and call into the PackedFunc. See more in
https://docs.tvm.ai/dev/runtime.html The system works great for over a
few years now.
Of course I understand there has been legacy issues in MXNet that is why I
did not bring this proposal up. But given this is a proposal for 2.0, I
would encourage everyone to give a serious thought about this possibility.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-mxnet/issues/17097?email_source=notifications&email_token=AAHCV2JHKIW6ZV2HVDYDR6DQZO3RLA5CNFSM4J3WHPZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHKPJXA#issuecomment-567604444,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHCV2NICGEG3VMB2JTNHW3QZO3RLANCNFSM4J3WHPZQ
.

Pybind is nice, I used Boost python many years ago, which I think is based
on. The problem with this is the hourglass C bindings, you have to go from
Python to C++ / Pybind, down to C and to the engine, this seems like a lot
of boilerplate.
On Mon, Dec 16, 2019 at 10:02 PM reminisce<EMAIL_ADDRESS>wrote:

MXNet imperative operator invocation overhead is as large as 30-60us,
which is significant compared to the official NumPy operators with ~600ns
overhead. This has negatively impacted the performance of applying MXNet to
the models where many operators' kernel runtime duration is short,
especially in the area of classic machine learning. We plan to address the
problem in two steps:

Short term: Use pybind11 to replace Python op API and ctypes/c api.
Preliminary experiments show that the pure Python-C++ turnaround time by
using Pybind is between 400-600ns, while the current Python op API using
ctypes/c api costs more than 10us. We believe with the correct
implementation, we can reduce the op invocation overhead to 2us including
the time on FFI and engine.
2.
Long term: Adopt Python's C extension interface. NumPy did this by
developing its own C API. This provides considerably less overhead compared
to other solutions. However, it would cost much more engineering efforts by
integrating this with our existing operator workflow in C++.
@hzfan https://github.com/hzfan @hgt312 https://github.com/hgt312
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-mxnet/issues/17097?email_source=notifications&email_token=AAHCV2KR5Z4L2T62UJCEVPTQZBTOZA5CNFSM4J3WHPZ2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IA6JXSA,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHCV2J7KYUKKTFZTKN6TB3QZBTOZANCNFSM4J3WHPZQ
.

@larroy indeed every solution has trade-offs, and these tradeoffs are discussed in the above posts when we compare solutions, and they are backed by benchmarks :) it would be great if you can also suggest potential tradeoffs here.
When you expose an API from typed language(c++) to a dynamic language(python), you have to type erase it, given that the python functions don't have the type, and you have to pass the information along.
The only difference is where you do the type checking(that the python type corresponds to the right c++ type), and translation(translating to the c++ type).
For example, in the case of pybind, the erasure is done implicitly when you call the python function, then checking and translation happens when you call into the c++ function.
In the case of creating a C API for each feature and wrap things in the python side, the type checking is done in the python side, and translation as well.
In the case of tvm ffi, the type translation is done in the python/cython side,  while the type checking is done in the c++.
To dive deeper into the tradeoffs for PackedFunc calling convention. The convention erases the type by having the type code stored into the arguments. This brings additional cost of passing arguments into heap, as opposed to registers. So they might not be designed for inline functions that needs to happen at the order of 1e-9s, however, for API functions that needs to run around 1e-7 or even 1e-8 level, this convention is pretty good.
In terms of the calling cost, it really depends on whether the caller and callee are strongly typed.

If caller is strongly typed, then assigning type code is O(1)
If caller is a dynamic type(like python) then we need to have a dispatcher to dispatch and select the right type code
If callee is strongly typed, then the cost of checking is O(1) by just check the code to be the correct one
If the callee is dynamic type, then a dispatching need to happen, which have another level of hashtable lookup O(1)

As we can see, the only place where dispatching is necessary is the dynamic type handling case. Even in these cases, if there is a strong need of specialization, we can directly force the type by running checking on the caller, and pass in the right type code (the engineering burden is the same as wrapping the C API). However, the benchmark suggests that the dynamic dispatching cost is reasonable, and satisfies the API speed.
Coming back to the tradeoff, the main tradeoff here is the engineering burden to keep an hourglass design(with fixed set of API) vs efficiency. While my post did not suggest that TVM's ffi is a silver bullet, it does works pretty well for our use cases. hope it helps

Thanks for the explanation. I'm not so concerned about complexity of
dispatching. If I understood you correctly the main benefit that you
explain for the TVM project was not having to change the C API, but still
you need to do type checking in both ends, or at least on the receiving end
of the API, correct? I think we have discussed similar things in the past
and we might have different views on strongly typed vs dynamic typed. A
priori I prefer to see an API which can be evolved and changed, I find it
more explicit and clearer that what I think you do with PackedFun which I
have looked at briefly but not used extensively.  If one is going to call
into the C API using pybind, does it make sense to layer a C++ API on top
of the C API for this?
Also these microbenchmarks are nice, but we also need to consider the
overhead in typical workloads and see if it's still significant.
CFFI is also another alternative.
I couldn't access your pointers like:
https://github.com/tqchen/tvm/tree/pyffi
On Thu, Dec 26, 2019 at 2:00 PM Tianqi Chen<EMAIL_ADDRESS>wrote:

@larroy indeed every solution has trade-offs, and these tradeoffs are
discussed in the above posts when we compare solutions, and they are backed
by benchmarks :) it would be great if you can also suggest potential
tradeoffs here.
When you expose an API from typed language(c++) to a dynamic
language(python), you have to type erase it, given that the python
functions don't have the type, and you have to pass the information along.
The only difference is where you do the type checking(that the python type
corresponds to the right c++ type), and translation(translating to the c++
type).
For example, in the case of pybind, the erasure is done implicitly when
you call the python function, then checking and translation happens when
you call into the c++ function.
In the case of creating a C API for each feature and wrap things in the
python side, the type checking is done in the python side, and translation
as well.
In the case of tvm ffi, the type translation is done in the python/cython
side,  while the type checking is done in the c++.
To dive deeper into the tradeoffs for PackedFunc calling convention. The
convention erases the type by having the type code stored into the
arguments. This brings additional cost of passing arguments into heap, as
opposed to registers. So they might not be designed for inline functions
that needs to happen at the order of 1e-9s, however, for API functions that
needs to run around 1e-7 or even 1e-8 level, this convention is pretty good.
In terms of the calling cost, it really depends on whether the caller and
callee are strongly typed.

If caller is strongly typed, then assigning type code is O(1)
If caller is a dynamic type(like python) then we need to have a
dispatcher to dispatch and select the right type code
If callee is strongly typed, then the cost of checking is O(1) by just
check the code to be the correct one
If the callee is dynamic type, then a dispatching need to happen, which
have another level of hashtable lookup O(1)

As we can see, the only place where dispatching is necessary is the
dynamic type handling case. Even in these cases, if there is a strong need
of specialization, we can directly force the type by running checking on
the caller, and pass in the right type code (the engineering burden is the
same as wrapping the C API). However, the benchmark suggests that the
dynamic dispatching cost is reasonable, and satisfies the API speed.
Coming back to the tradeoff, the main tradeoff here is the engineering
burden to keep an hourglass design(with fixed set of API) vs efficiency.
While my post did not suggest that TVM's ffi is a silver bullet, it does
works pretty well for our use cases. hope it helps
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-mxnet/issues/17097#issuecomment-569139957

LOL, the last one was my comment, not @szha :-D

Test
On Fri, Dec 27, 2019 at 11:54 AM Pedro Larroy<EMAIL_ADDRESS>wrote:

Thanks for the explanation. I'm not so concerned about complexity of
dispatching. If I understood you correctly the main benefit that you
explain for the TVM project was not having to change the C API, but still
you need to do type checking in both ends, or at least on the receiving end
of the API, correct? I think we have discussed similar things in the past
and we might have different views on strongly typed vs dynamic typed. A
priori I prefer to see an API which can be evolved and changed, I find it
more explicit and clearer that what I think you do with PackedFun which I
have looked at briefly but not used extensively.  If one is going to call
into the C API using pybind, does it make sense to layer a C++ API on top
of the C API for this?
Also these microbenchmarks are nice, but we also need to consider the
overhead in typical workloads and see if it's still significant.
CFFI is also another alternative.
I couldn't access your pointers like:
https://github.com/tqchen/tvm/tree/pyffi
On Thu, Dec 26, 2019 at 2:00 PM Tianqi Chen<EMAIL_ADDRESS>wrote:

@larroy indeed every solution has trade-offs, and these tradeoffs are
discussed in the above posts when we compare solutions, and they are backed
by benchmarks :) it would be great if you can also suggest potential
tradeoffs here.
When you expose an API from typed language(c++) to a dynamic
language(python), you have to type erase it, given that the python
functions don't have the type, and you have to pass the information along.
The only difference is where you do the type checking(that the python
type corresponds to the right c++ type), and translation(translating to the
c++ type).
For example, in the case of pybind, the erasure is done implicitly when
you call the python function, then checking and translation happens when
you call into the c++ function.
In the case of creating a C API for each feature and wrap things in the
python side, the type checking is done in the python side, and translation
as well.
In the case of tvm ffi, the type translation is done in the python/cython
side,  while the type checking is done in the c++.
To dive deeper into the tradeoffs for PackedFunc calling convention. The
convention erases the type by having the type code stored into the
arguments. This brings additional cost of passing arguments into heap, as
opposed to registers. So they might not be designed for inline functions
that needs to happen at the order of 1e-9s, however, for API functions that
needs to run around 1e-7 or even 1e-8 level, this convention is pretty good.
In terms of the calling cost, it really depends on whether the caller and
callee are strongly typed.

If caller is strongly typed, then assigning type code is O(1)
If caller is a dynamic type(like python) then we need to have a
dispatcher to dispatch and select the right type code
If callee is strongly typed, then the cost of checking is O(1) by just
check the code to be the correct one
If the callee is dynamic type, then a dispatching need to happen, which
have another level of hashtable lookup O(1)

As we can see, the only place where dispatching is necessary is the
dynamic type handling case. Even in these cases, if there is a strong need
of specialization, we can directly force the type by running checking on
the caller, and pass in the right type code (the engineering burden is the
same as wrapping the C API). However, the benchmark suggests that the
dynamic dispatching cost is reasonable, and satisfies the API speed.
Coming back to the tradeoff, the main tradeoff here is the engineering
burden to keep an hourglass design(with fixed set of API) vs efficiency.
While my post did not suggest that TVM's ffi is a silver bullet, it does
works pretty well for our use cases. hope it helps
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-mxnet/issues/17097#issuecomment-569139957

re the need for explicit type checking code in TVM FFI.
Actually there is no explicit code for type checking as they are generated automatically via template expansion(on the receiving end), also we also have a "strong typed" signature that wraps the packed function interface, which gives you compile time type checking https://github.com/apache/incubator-tvm/blob/master/include/tvm/runtime/packed_func.h#L191
For dynamic language side(python) the exposed function is still type erased(as python is a dynamic language).
Note that the view dynamic vs static typed language does not really apply to this case, because the main goal(exposing to python) means type-erasure(as python is dynamically typed). The main goal would be how to reduce the number of abstraction layers.

Also these microbenchmarks are nice, but we also need to consider the
overhead in typical workloads and see if it's still significant.

If we apply reasoning, most API cost is going to be FFI cost + exec cost, and I think the conclusion so far is we want FFI cost to be around 1e-7s to 1e-6s, which is the limit of any cost .

@tqchen Thanks for sharing this. I don’t know if I understand correctly. For now arguments except primitives pass through FFI via Object (like ADTObj). It is then converted to TShape in backend and TShape is not involved in FFI directly.
As you said, Object allows me to conveniently put various kinds of things into a container (ADTObj), without losing their types. For example, now a tuple of tuples like ((2, 2), (2, 2)) is allowed.
Also sorry for the late reply. I have been on a vacation this week :)


try-catch problem

There are some bugs in the logic of try-catch, need fix!
Below is the explanation of the problem:
try {
}
catch e1 {
}
catch e2 {
}
In the try block, if there is no throw, only error, then it will only return error, the catch block will not be evaluated.

rewrite the try-catch-finally evaluatation logic, so it should work for now.


keyserver receive failed: No name

Got this error when installing Temperature Machine
Executing: /tmp/apt-key-gpghome.Hd8dHFNGHw/gpg.1.sh --keyserver pool.sks-keyservers.net --recv-keys 00258F48226612AE
gpg: keyserver receive failed: No name
Any suggestions??

Just use Stretch 4.14, works every time, T.W. will probably not update this ;)

I found a workaround that is not all that satisfactory. If you make a clone
of your working model - In the rpi menu - accessories - cd card copier -
one can make a copy of the card that is running the pi. It works fine but
if you change the host name of the new device - it will read the
temperature of the sensor and display it through the web interface but will
not display the log graph. As I know very little about the logging and web
interface - I could not change it. If you just keep using the original host
name - you can put 2 devices on the same network. I just use the unique IP
address to select the device on the network but have to remember where each
is.
Bill Rayment
On Sat, 16 Oct 2021 at 11:03, AlistairFielden @.***>
wrote:

I'm having the same issue.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tobyweston/temperature-machine/issues/109#issuecomment-944955519,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWWWP7JKLETTZSC5H5OZODUHGV6FANCNFSM5COXW53Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

It seems that the SKS keyserver has been deprecated, all you need to do is use an alternative key server. This worked for me:
   sudo apt-key adv --keyserver keyserver.ubuntu.com  --recv-keys 00258F48226612AE

Thanks, I'll check it out.
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sat, 6 Nov 2021 at 11:22, Patrick @.***> wrote:

It seems that the SKS keyserver has been deprecated, all you need to do is
use an alternative key server. This worked for me:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
00258F48226612AE
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tobyweston/temperature-machine/issues/109#issuecomment-962482805,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWWWPZHYPKZQE4Z2R67LL3UKVP5BANCNFSM5COXW53Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Well it worked using Stretch Version and
Patrick's
It seems that the SKS keyserver has been deprecated, all you need to do is
use an alternative key server. This worked for me:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
00258F48226612AE
Got 3 sensors reading and graphing. It is shows only temperatures but no
graphs with Buster
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sat, 6 Nov 2021 at 15:18, Bill Rayment @.***> wrote:

Thanks, I'll check it out.
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sat, 6 Nov 2021 at 11:22, Patrick @.***> wrote:

It seems that the SKS keyserver has been deprecated, all you need to do
is use an alternative key server. This worked for me:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
00258F48226612AE
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tobyweston/temperature-machine/issues/109#issuecomment-962482805,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWWWPZHYPKZQE4Z2R67LL3UKVP5BANCNFSM5COXW53Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Well I was wrong - it did present a graph with the temp range but did not
log. Sad it did work well in the past.
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sun, 7 Nov 2021 at 02:50, Bill Rayment @.***> wrote:

Well it worked using Stretch Version and
Patrick's
It seems that the SKS keyserver has been deprecated, all you need to do is
use an alternative key server. This worked for me:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
00258F48226612AE
Got 3 sensors reading and graphing. It is shows only temperatures but no
graphs with Buster
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sat, 6 Nov 2021 at 15:18, Bill Rayment @.***>
wrote:

Thanks, I'll check it out.
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sat, 6 Nov 2021 at 11:22, Patrick @.***> wrote:

It seems that the SKS keyserver has been deprecated, all you need to do
is use an alternative key server. This worked for me:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
00258F48226612AE
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tobyweston/temperature-machine/issues/109#issuecomment-962482805,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWWWPZHYPKZQE4Z2R67LL3UKVP5BANCNFSM5COXW53Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Yeah man just load home assistant on that raspberry pi you will be much
better off.   Hass.io
On Mon, Nov 8, 2021, 1:07 PM Billir @.***> wrote:

Well I was wrong - it did present a graph with the temp range but did not
log. Sad it did work well in the past.
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sun, 7 Nov 2021 at 02:50, Bill Rayment @.***> wrote:

Well it worked using Stretch Version and
Patrick's
It seems that the SKS keyserver has been deprecated, all you need to do
is
use an alternative key server. This worked for me:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
00258F48226612AE
Got 3 sensors reading and graphing. It is shows only temperatures but no
graphs with Buster
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sat, 6 Nov 2021 at 15:18, Bill Rayment @.***>
wrote:

Thanks, I'll check it out.
Bill Rayment
Project Bee
1<PHONE_NUMBER>
On Sat, 6 Nov 2021 at 11:22, Patrick @.***> wrote:

It seems that the SKS keyserver has been deprecated, all you need to do
is use an alternative key server. This worked for me:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
00258F48226612AE
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<
https://github.com/tobyweston/temperature-machine/issues/109#issuecomment-962482805
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAWWWPZHYPKZQE4Z2R67LL3UKVP5BANCNFSM5COXW53Q

.
Triage notifications on the go with GitHub Mobile for iOS
<
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android
<
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tobyweston/temperature-machine/issues/109#issuecomment-963484079,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALFB6DBVI75TU6H52IJWVKLULANY5ANCNFSM5COXW53Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Got this error when installing Temperature Machine
Executing: /tmp/apt-key-gpghome.Hd8dHFNGHw/gpg.1.sh --keyserver pool.sks-keyservers.net --recv-keys 00258F48226612AE
gpg: keyserver receive failed: No name
Any suggestions??

Got this error when installing Temperature Machine
Executing: /tmp/apt-key-gpghome.Hd8dHFNGHw/gpg.1.sh --keyserver pool.sks-keyservers.net --recv-keys 00258F48226612AE
gpg: keyserver receive failed: No name
Any suggestions??

Yes, SKS keyserver has been deprecated. See my answer above.

It i now April 2022 and this key provided earlier still works
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 00258F48226612AE

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 00258F48226612AE    is working now in decembre 2022, thanks


Create mockups

Describe the task
Using wireframes, create mockups including visual design and style choices for each screen of the application
Acceptance Criteria
Mockups created and agreed upon by team

Discord discussion:
https://discord.com/channels/1093609969867296780/1115695807694372904/1116114162632892467
Not closing issue yet because more mockups may follow with more features depending on time/capacity.

Updated second to not have linked/underlined text and using 313132 instead of FFFFFF 87%


Double pore

This implements a double pore, which simply adds another clone of the few-layer graphene to construct a second pore. For systems like below, this is slightly improved behavior, as I would not want to have the chunk duplicated twice.

This could possibly be extended to a system of n pores of different sizes, but I don't currently have a use for that type of structure.

Looks like Travis is failing due to a known bug https://github.com/mosdef-hub/mbuild/issues/424

Stalled and closing - feel free to re-open/takeover if this feature is of interest


TypeError: 'ABCMeta' Object is not subscriptable when configuring mmseg model for PTQ and QAT

Hi,
I am working on configuring a custom config file for the stdc1 model in MMsegmentation using the Cityscapes dataset, with the aim of performing both PTQ and QAT. For reference, I have looked at the following MMRazor config files:

ptq_openvino_resnet18_8xb32_in1k_calib32xb32.py
qat_openvino_resnet18_10e_8xb32_in1k.py

However, I am encountering the following error during both PTQ and QAT:

Since the provided config is limited for mmcls, could you please guide me on how to properly modify the config file for the stdc1 model and Cityscapes dataset?
Below is the config files I wrote:
For ptq_openvino_stdc1_4xb12-80k_cityscapes-512x1024.py
_base_ = [
    'mmseg::stdc/stdc1_4xb12-80k_cityscapes-512x1024.py' ,
    # '../../deploy_cfgs/mmseg/segmentation_openvino_dynamic-1024x2048.py'
]

val_dataloader = dict(batch_size=32)

test_cfg = dict(
    type='mmrazor.PTQLoop',
    calibrate_dataloader=val_dataloader,
    calibrate_steps=32,
)

global_qconfig = dict(
    w_observer=dict(type='mmrazor.PerChannelMinMaxObserver'),
    a_observer=dict(type='mmrazor.MovingAverageMinMaxObserver'),
    w_fake_quant=dict(type='mmrazor.FakeQuantize'),
    a_fake_quant=dict(type='mmrazor.FakeQuantize'),
    w_qscheme=dict(
        qdtype='qint8', bit=8, is_symmetry=True, is_symmetric_range=True),
    a_qscheme=dict(
        qdtype='quint8', bit=8, is_symmetry=True, averaging_constant=0.1),
)

float_checkpoint = 'mmsegmentation/work_dirs/stdc1_4xb12-80k_cityscapes-512x1024/best_mIoU_epoch_0.pth'  # path to my local model

model = dict(
    _delete_=True,
    type='mmrazor.MMArchitectureQuant',
    data_preprocessor = dict(
        type='mmseg.SegDataPreProcessor',
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        bgr_to_rgb=True,
        pad_val=0,
        seg_pad_val=255),
    architecture=_base_.model,
    float_checkpoint=float_checkpoint,
    quantizer=dict(
        type='mmrazor.OpenVINOQuantizer',
        global_qconfig=global_qconfig,
        tracer=dict(
            type='mmrazor.CustomTracer',
        skipped_methods=[
                'mmseg.models.decode_heads.decode_head.BaseDecodeHead.predict_by_feat',
                'mmseg.models.decode_heads.decode_head.BaseDecodeHead.loss_by_feat'
            )
            ))

model_wrapper_cfg = dict(type='mmrazor.MMArchitectureQuantDDP', )

For qat_openvino_stdc1_4xb12-80k_cityscapes-512x1024.py:
_base_ = ['mmseg::stdc/stdc1_4xb12-80k_cityscapes-512x1024.py' ]

float_checkpoint = 'mmsegmentation/work_dirs/stdc1_4xb12-80k_cityscapes-512x1024/best_mIoU_epoch_0.pth'  # path to my local model

global_qconfig = dict(
    w_observer=dict(type='mmrazor.PerChannelMinMaxObserver'),
    a_observer=dict(type='mmrazor.MovingAverageMinMaxObserver'),
    w_fake_quant=dict(type='mmrazor.FakeQuantize'),
    a_fake_quant=dict(type='mmrazor.FakeQuantize'),
    w_qscheme=dict(
        qdtype='qint8', bit=8, is_symmetry=True, is_symmetric_range=True),
    a_qscheme=dict(
        qdtype='quint8', bit=8, is_symmetry=True, averaging_constant=0.1),
)

model = dict(
    _delete_=True,
    _scope_='mmrazor',
    type='MMArchitectureQuant',
    data_preprocessor = dict(
        type='mmseg.SegDataPreProcessor',
        mean=[123.675, 116.28, 103.53],
        std=[58.395, 57.12, 57.375],
        bgr_to_rgb=True,
        pad_val=0,
        seg_pad_val=255),
    architecture=_base_.model,
    float_checkpoint=float_checkpoint,
    quantizer=dict(
        type='mmrazor.OpenVINOQuantizer',
        global_qconfig=global_qconfig,
        tracer=dict(
            type='mmrazor.CustomTracer',
        skipped_methods=[
                'mmseg.models.decode_heads.decode_head.BaseDecodeHead.predict_by_feat',
                'mmseg.models.decode_heads.decode_head.BaseDecodeHead.loss_by_feat']
            )))

optim_wrapper = dict(
    optimizer=dict(type='SGD', lr=0.0001, momentum=0.9, weight_decay=0.0001))

# learning policy
param_scheduler = dict(
    _delete_=True, type='ConstantLR', factor=1.0, by_epoch=True)

model_wrapper_cfg = dict(
    type='mmrazor.MMArchitectureQuantDDP',
    broadcast_buffers=False,
    find_unused_parameters=False)

# train, val, test setting
train_cfg = dict(
    _delete_=True,
    type='mmrazor.QATEpochBasedLoop',
    max_epochs=10,
    val_interval=1)
val_cfg = dict(_delete_=True, type='mmrazor.QATValLoop')

# Make sure the buffer such as min_val/max_val in saved checkpoint is the same
# among different rank.
default_hooks = dict(sync=dict(type='SyncBuffersHook')) ```

Hi, I remember having this issue at first when using MMRazor, but I can't remember the exact solution. I think it was coming from the Python version. I was using Python 3.8, now I'm using Python 3.10 and I don't have this error anymore. I hope this can help

Hi, I remember having this issue at first when using MMRazor, but I can't remember the exact solution. I think it was coming from the Python version. I was using Python 3.8, now I'm using Python 3.10 and I don't have this error anymore. I hope this can help

Upgrading Python to version 3.10, while keeping the rest of the package versions unchanged, resulted in the following error:
ImportError: torch>=1.13 is not installed properly
Here is my current setup:
mmcv: 2.1.0
mmengine: 0.10.4
mmrazor: 1.0.0
torch: 2.4.1+cu118
Could you share the specific versions of Python and other packages in your working setup? Thanks!

Hi, MMRazor does not work with PyTorch 2.x unfortunately, or you have to modified the code deeply. You can see in several script that some code parts are not run if you don't have the right Torch version (I don't know why they did this...)
Try downgrading your PyTorch version to 1.13.1. You can find below my requirement file for the experiments I made, there may be useless dependencies in it.
# Base requirements
ftfy
future>=0.18.2
joblib>=1.4.2
numpy==1.26.4
onnx>=1.13.0
onnx-graphsurgeon
onnxscript
regex
setuptools==60.5.0
tensorboard>=2.16.2
torch_tb_profiler  # for model profiling visualization
torch==1.13.1
torchtext==0.14.1
torchvision==0.14.1

# MMLab requirements
openmim
mmengine
mmcv>=2.0.0
mmpretrain>=1.0.0
# Already installed from source
mmcls>=1.0.0rc6
mmdet>=3.0.0
mmsegmentation>1.0.0
mmrazor>=1.0.0

@Veccoy Thank you ! Indeed, downgrading PyTorch version to 1.13.1 works.


Support reusing Layer instances

In my application, I have a Map component that tracks viewport state and sets some common properties, and other components simply provide the deck.gl layers to display on the map:
class Map extends React.Component {
  ...
  render() {
    return (
      <MapGL {...this.state.vp} {...this.state.size} onViewportChange={vp => this.setState({vp})} ...>
        <DeckGL {...this.state.vp} {...this.state.size} layers={this.props.layers}/>
      </MapGL>
    );
  }
}

class Page extends React.Component {
  ...
  render() {
    const layer = new ScatterplotLayer({ id: 'xyz', ... });
    return <Map layers={[layer]}/>;
  }
}

However, this arrangement hits the "deck.gl sanity check - Matching layer is same" assertion every time the Map component is re-rendered without the parent component being re-rendered, as is the case when you pan or zoom the map. This is not exactly expected behavior, although the docs do hint at it.
Instead of requiring new layer instances, I believe a better approach would be to either clone the layer when this condition is detected, or change the underlying code to not rely on old/new layers being separate instances.

@austinhyde
Correct. deck.gl currently requires freshly minted layers each render. But as you say, it would not be terribly hard to support "reusing" layers.
And we could optimize deck.gl pretty heavily when reusing layers, essentially bypassing change detection, and sublayer generation. (Especially if we call Object.freeze on the layer's props in the Layer constructor - React does something similar, I believe).
Would be very happy to see this as a PR or mark it as a prioritized ticket. It is mostly a question of the "lifecycle management" code being the most complex part of deck.gl, so this change should be carefully reviewed, and ideally be backed up with documentation clarifications as well as additional test cases to help make sure we don't break things.

@gnavvy - do you want to try your hand at this?

Looks like this won't be making it into v4.0.
We've added new benchmarks (npm run bench) that shows that a macbook pro can construct about 300.000 layer instances per second - so this should not be a significant performance bottleneck in practice.
Deprioritizing but keeping open.

Closing, since listed in #83


recent_view: Fix transparent avatar background color being changed.

We don't set a background color for transparent avatars to avoid changing what the avatar is supposed to be.
discussion: https://chat.zulip.org/#narrow/channel/9-issues/topic/transparent.20background.20profile.20picture.20reappears

before
after

Merged, thanks @amanagr!


Let coin selection use (some?) unconfirmed coins

Currently, our automated coin selection only considers confirmed coins. Considering unconfirmed coins as well introduces new challenges and was left for future work (see https://github.com/wizardsardine/liana/pull/560#issuecomment-1784834125).
It would be nice to consider them though. Should we have a bias toward spending confirmed coins first? It's maybe part of a larger bias toward preferring to spend older coins (see https://github.com/wizardsardine/liana/issues/827)? Should we only consider unconfirmed coins we created ourselves (change) and not unconfirmed payments?

What about running 3 rounds in coin selection:

first round only confirmed coins, if  match break
second round confirmed coins + our unconfirmed coins, if  match break
third round confirmed coins + all unconfirmed coins, if match at least notice user we will try spend unconfirmed coins

We keep stumbling upon new situations where https://github.com/wizardsardine/liana/issues/811 (i mean in the general sens of being able to pass warnings from the spend creation code to the GUI) would be useful.

I think it's only ever reasonable to spend our own unconfirmed change by default. We could allow to spend unconfirmed external deposits but only if the user explicitly ask for it, and i think we can leave this toggle to future work (they can already set them manually).
Note that if we ever implement this toggle, a later "safe" spend could spend one of our unconfirmed change which itself was created by and "unsafe" spend which spends an unconfirmed deposit. We should either check this or be explicit about it (maybe the toggle should be a global configuration option in this case).

With regard to RBF, we currently consider as candidates those (possibly unconfirmed) input coins from the transaction to be replaced together with confirmed coins.
Policy 2 in https://github.com/bitcoin/bitcoin/blob/master/doc/policy/mempool-replacements.md#current-replace-by-fee-policy restricts which unconfirmed coins can be used in the replacement:

If the transaction to be replaced is in the mempool, then this should be the only directly conflicting transaction with the replacement and so we would not be able to include any other unconfirmed coins, and so no changes are needed.
In case the transaction to be replaced is not in the mempool and there is some other directly conflicting transaction, it may be that this other transaction is spending unconfirmed change that is not an input in the transaction to be replaced. According to the policy, I think we could include this unconfirmed change as a candidate in the replacement.

For now, I'll make the change for a regular new spend and consider RBF separately.

This is now done.


set docker env variable to force it build images for linux/amd64 platform

Since all our images are running in linux/amd64 platform, we need to force docker to build linux/amd64 compatible images. Without this patch, the M1 mac will fail to deploy the fleetshard operator due to the error:
State:         Waiting
      Reason:      CrashLoopBackOff
    Last State:    Terminated
      Reason:      Error
      Message:     exec /bin/opm: exec format error

Note: It won't impact any non-arm64 chip users because they will build with linux/amd64 originally.
ref: https://docs.docker.com/engine/reference/commandline/cli/#environment-variables

LGTM. Do you know if podman considers this env also?

Had a quick search, but didn't find any. Podman only provides the --platform for each command. That works for most cases, but for this opm command, we can't set any docker/podman options into it, which will also fail:
https://github.com/bf2fc6cc711aee1a0c2a/kas-installer/blob/f04743304ea79c87b7b6913da06a50dfb71962a9/operators/generate-kas-fleetshard-olm-bundle.sh#L352
ref: https://docs.podman.io/en/latest/markdown/podman-build.1.html#platform-os-arch-variant

Had a quick search, but didn't find any.

Same, the best I could find was a Reddit comment that seems to imply it would work (given the context of the post): https://www.reddit.com/r/docker/comments/r0r86r/comment/hlwub5r/?utm_source=share&utm_medium=web2x&context=3
I think we can go ahead and merge this. @k-wall are you using podman on an M1?

Had a quick search, but didn't find any.

Same, the best I could find was a Reddit comment that seems to imply it would work (given the context of the post): https://www.reddit.com/r/docker/comments/r0r86r/comment/hlwub5r/?utm_source=share&utm_medium=web2x&context=3
I think we can go ahead and merge this. @k-wall are you using podman on an M1?

I don't think currently podman supports running in M1 mac:
https://github.com/containers/podman/issues/14303
I ran into the same error in the issue, and then decided to use docker instead.


How to send dialog.delegate directive

Hi,
I have an intent songIntent which plays the requested song. I have a slot songname. if the songname is empty i need to send a dialog.delegate directive. How to send this directive?

Just use .directive() and send the necessary json. There are no helper methods for this yet.

Thanks a lot. It worked.


Trips Example Won't Compile -- Unexpected token

I cloned the git repo and tried to compile the trips example. After running npm install, I got the following output:
npm WARN optional Skipping failed optional dependency /chokidar/fsevents:
npm WARN notsup Not compatible with your operating system or architecture<EMAIL_ADDRESS>npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>No repository field.

Running npm start I get the following error:

ERROR in ./app.js
Module build failed (from ./node_modules/buble-loader/index.js):
BubleLoaderError:
88 :     const {viewState, controller = true, baseMap = true} = this.props;
89 :
90 :     return (
91 :       <DeckGL
92 :         layers={this._renderLayers()}
                   ^
Unexpected token (92:14)
    at Object.module.exports (/home/connor/Documents/deck.gl/examples/website/trips/node_modules/buble-loader/index.js:24:69)
 @ multi (webpack)-dev-server/client?http://localhost:8080 (webpack)/hot/dev-server.js ./app.js app[2]

I'm on npm version 3.5.2
[Update]
I was able to update npm to 6.4.1 and I installed acorn manually, then ran npm install again. I'm still getting this error.

use yarn instead of npm install

That fixed my issue -- thanks

@jianhuang01 This should be fixed after #2888 ?


How to save metadata without re-encoding?

ffpmeg has this -acodec copy how to do it with FFMpeg\FFMpeg?
$ffmpeg = \FFMpeg\FFMpeg::create();
$file = $ffmpeg->open('foo.mp3');
$file->filters()->addMetadata([ 'title' => 'foo' ]);
$file->save(new \FFMpeg\Format\Audio\Mp3, "foo.new.mp3");

this re-encodes the file and is obviously lossy.
I cannot find anything on this matter. thanks!

Hi @owzim
I have created a new format named Copy.
use FFMpeg\Format\Audio\DefaultAudio;

class Copy extends DefaultAudio
{
    public function __construct()
    {
        $this->audioCodec = 'copy';
    }

    public function getAvailableAudioCodecs(): array
    {
        return ['copy'];
    }
}

Then you can save your mp3 with the updated meta like this:
$ffmpeg = \FFMpeg\FFMpeg::create();
$file = $ffmpeg->open('foo.mp3');
$file->filters()->addMetadata([ 'title' => 'foo' ]);
$file->save(new Copy(), "foo.new.mp3");


support for file/blobs

Blob's are a bit tricky, they can only be read lazily. But i happen to know a trick of how to read them synchronous. it's a bit tricky and might not always work for each and every env/context
const blob = new Blob(['123'])
const xhr = new XMLHttpRequest()
const url = URL.createObjectURL(blob)

// required if you need to read binary data (you can't use responseType while using sync xhr)
xhr.overrideMimeType('text/plain; charset=x-user-defined')
xhr.open('GET', url, false)
xhr.send()

const uint8 = Uint8Array.from(xhr.response, c => c.charCodeAt(0))
const text = new TextDecoder().decode(uint8)
const arrayBuffer = uint8.buffer
const json = JSON.parse(text)

if you are using workers then it might be possible to spin up a new worker and use Atomics wizard to pause/resume

just some idea that i got...
if you need to create a file list: then there is a roundabout way of creating a list using something like:
function FileListItems (files) {
  var b = new ClipboardEvent("").clipboardData || new DataTransfer()
  for (var i = 0, len = files.length; i<len; i++) b.items.add(files[i])
  return b.files
}

var fileList = FileListItems([
  new File(['content'], 'sample1.txt'),
  new File(['abc'], 'sample2.txt')
])

it's a bit tricky and might not always work for each and every env/context

I prefer not having these at all than having these "sometimes broken" + this module is not for browsers only, it works pretty well in Node.js too and that's desired when it comes to its JSON helpers for client/server exchange.
I don't think I want to work on this.

That's alright if it dosen't.
All doe it's also possible turn something async into something sync in NodeJS using atomics
found this post just the other day: https://giuseppegurgone.com/synchronizing-async-functions
Trying to do something similar in Browser is more tricker cuz you would need shared arrayBuffer and enabling those requires you to respond with extra response headers in order to use it...


Support .deb auto detect

Is there a way to auto detect Ubuntu / deb and download the appropriate file, like we currently have for Mac and Windows?

According to Electronjs "Currently, only macOS and Windows are supported. There is no built-in support for auto-updater on Linux, so it is recommended to use the distribution's package manager to update your app."


walredo: regression test that will ensure we use posix_spawn

Decided gainst testing this, too much effort for too little gain.
For posterity, testing this would likely require LD_PRELOAD or worse.


using _all_channels to update efficiently

When not changing between low and high, the channel#_data was not being updated.  Also, the clear and nir data is available when _low_channels_configured, so no need to change to high.

Do you need to "get" _all_channels?  The CI is angry since it thinks it's a "pointless statement".  You may need to assign the result to underscore (e.g. _ = self._all_channels) or disable the pylint warning for it in that case.

I pushed a commit to catch the return from all_channels into _ as mentioned.
The specifics of this change are a bit over my head. Hoping we can have someone with more knowledge weigh in, or at least with hardware to try it out.

Thanks for the change.  The only hardware I have tried is with a Raspberry Pi Zero and a Raspberry Pi 3B connected to the AS7341 via SDA/SCL pins, using Python 3.9 or 3.10.  I haven't tried any microcontrollers using MicroPython to do this.

I've experienced a similar issue in CircuitPython running on a Feather RP2040 and attempting to read only the clear channel. It does not update unless i am also reading other channels.


improve edgeview tcp timeout

add keep-alive on client side so it does not timeout by dispatcher
move the server timeout from tcp channel into global
some minor timer value changes.

add the reason for the process being killed by restarting.


[DO NOT MERGE] nxphlite-v3:Add HolyBro GPS on I2C 2 using ist8310

@igalloway - plase test this with your HW. If it works we will merge it.

Tested on hardware revB, ist8310 driver is started correctly

@Nielesonimusso - thank you! Were calibrations and rotations correct?

Haven't payed attention to exact values during tests today, but it was able to calibrate, and at the moments I looked at the heading in QGC it seemed to correspond to the actual heading of the drone. We have performed some position control flights using this driver, which were not a success (main issues are mentioned in the "additional feedback" of the logs, I put a more detailed flight explanation in the rddrone_fmuk66 channel on Slack), but you might be able to see the compass performance from their logs (https://logs.px4.io/plot_app?log=1fc9ab04-ad86-4fe8-941c-d30480fa2a1d and
https://logs.px4.io/plot_app?log=951f6392-12f2-43f3-941c-649e0f26cf26)
Between these test runs, we had an issue that at some point the IST8310 did not appear on the I2C bus at all (manually starting the driver indicated that there was no device on the bus), but this was magically solved by a couple of reboots, and I haven't been able to reproduce the error through numerous reboots, either by reset switch, reboot command or battery removal.

@dagar - let merge this, if it needs tweaking we can do that later.


[do not merge] test with OVS 3.0.0

Need to have an iqtlabs/openvswitch v3.0.0 image built, to be able to use this.

@anarkiwi image should be pushed now.


Add xla_client.Device type annotation to jit & pmap.

Looks like there are some merge conflicts internally... can you rebase this branch on master and we'll try again?

OK, have synced and rebased. Can you trigger Copybara again?


add more template functions according to fermyon/bartholomew#26 fermy…

Added more helpers as per fermyon/bartholomew#26, fermyon/bartholomew#27, fermyon/bartholomew#28 & fermyon/bartholomew#31
Signed-off-by: karthik Ganeshram<EMAIL_ADDRESS>

@rajatjindal text wrapping seems to require a bit more logic and testing than what has been presented so far. What would your thoughts be on the deffering adding wrap to a later PR (potentially just pulling in a library)?

sounds good to me. we can add that in a later PR

This is ready for a rereview

thanks a lot for your PR.

Would you like to PR the changes into Bartholomew or should I go ahead and make the PR?

if you could pls do it, that would be great.

Will do it today!


fix-c-style-bracket

Fixing c style bracket position

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Luís Carlos seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

I don't know if it's the case here, but some coding standards want curly braces on their own line.

I don't know if it's the case here, but some coding standards want curly braces on their own line.

Yap. Also, that is also one of the  c style recomendations from the repository author
https://github.com/MaJerle/c-code-style

So why is then only one file modified?


add feature 'shallowClone'

same as clone described in #1 excepts shallowClone only clones the first level

Resolved here - https://github.com/iShafayet/evolvenode-stdlib/commit/a95e56c2f264e804603006ea86d9442eafa6c10a


api/types/container,client: gofmt

- What I did
I've fixed three issues discovered by gofmt.
- Description for the changelog
api/types/container,client: gofmt

ping @dnephin as I think api/types/container is generated from swagger.

Nope, these files are not generated (yet). All generated files have a comment at the top of the file about it.

LGTM
Did these slip in from PRs opened during the period where validation was not being properly enforced by CI?

Hum probably 😣
LGTM 😇🐳


⚠ Punto de Fuga has degraded performance

In 965f920, Punto de Fuga (http://puntodefugapodcast.com) experienced degraded performance:

HTTP code: 200
Response time: 12628 ms

Resolved: Punto de Fuga performance has improved in 9ff75ef.


Tracking-changes

[x] In order to determine the changes between different pages, we need to track the difference between text files and corrector output files, and corrector output file and verifier output files, at character and word level.

The calculations are done in page level and metrics like Accuracy, Error rate, Change percentage and so on are calculated and saved in a new csv file every time the project is saved.


🐛 Bug Report: Concurrent Modification Error in RealtimeMixin Causing Crashes

👟 Reproduction steps
Create a Flutter project and integrate the Appwrite SDK.
Set up real-time subscriptions using RealtimeMixin.subscribeTo.
The error occurs intermittently, particularly when subscriptions are being closed or created while the _createSocket method is running.
E/flutter (22154): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Concurrent modification during iteration: _Map len:0.
E/flutter (22154): #0      _CompactIterator.moveNext (dart:collection-patch/compact_hash.dart:714:7)
E/flutter (22154): #1      RealtimeMixin._createSocket.<anonymous closure> (package:appwrite/src/realtime_mixin.dart:91:49)
E/flutter (22154): #2      _RootZone.runGuarded (dart:async/zone.dart:1582:10)
E/flutter (22154): #3      _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:418:13)
E/flutter (22154): #4      _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:428:7)
E/flutter (22154): #5      _BufferingStreamSubscription._close (dart:async/stream_impl.dart:317:7)
E/flutter (22154): #6      _ForwardingStream._handleDone (dart:async/stream_pipe.dart:99:10)

👍 Expected behavior
I'm encountering a "Concurrent modification during iteration: _Map len:0" error when using the Appwrite Flutter SDK for real-time subscriptions. This error seems to be originating within the RealtimeMixin._createSocket method (line 91).
The error appears to be related to concurrent access to the _subscriptions map within the RealtimeMixin.
This issue causes the app to crash unexpectedly, disrupting the real-time functionality.
👎 Actual Behavior
The app crashes: The "Unhandled Exception: Concurrent modification during iteration" is a critical error. In most cases, this would lead to an immediate crash of the Flutter application.
Disrupted real-time updates: The error originates from the RealtimeMixin, which is responsible for managing real-time data updates. So, even before the potential crash, the functionality related to receiving and handling real-time events would likely be disrupted.
Difficulty pinpointing the issue: The stack trace points to the Appwrite SDK's internal code. This makes it challenging for developers to directly diagnose and fix the issue, as it might require modifications to the SDK itself.
🎲 Appwrite version
Version 1.0.x
💻 Operating system
MacOS
🧱 Your Environment

Self-hosted Appwrite: I'm using Appwrite self-hosted on an Azure virtual machine Azure Spot (Standard D2s v3 with 2 vCPUs and 8 GB memory, running Ubuntu 22.04).
Appwrite Version: The Appwrite server version I'm running is appwrite/appwrite:1.5.5.
Cloudflare Access: I access my Appwrite instance through Cloudflare by pointing my custom domain to it.
Flutter App Configuration: My Flutter app is configured to use Appwrite in self-signed certificate mode.

👀 Have you spent some time to check if this issue has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

@monzim, thanks for raising 🙏🏼 . It looks like this is a duplicate of https://github.com/appwrite/sdk-for-flutter/issues/200 so I'm going to close it as so.

@monzim, thanks for raising 🙏🏼 . It looks like this is a duplicate of #200 so I'm going to close it as so.

After making some changes to the _createSocket and the _cleanup  methods in the /realtime_mixin.dart file, the error has been resolved. This worked for me.
Can I make a pull request to incorporate these changes? I guess this repo is readonly. If so, how can I proceed?


Add inital quick-git-setup slide

refs #25
I have added a slide that instructs users on how to install git before we tell them to clone a repo for the first time since we have no prior documentation on this. I tell them to get git from git-scm.com because it includes a shell for windows users however we can tell them to use the GitHub tools instead which include similar tools. My thoughts are that I want to stress Git is not the same as GitHub however the GH tool might be better and easier to setup. Thoughts? @ramereth @edunham @iankronquist @lucywyman [others]

@iankronquist I tested this last night with a windows user. The shell is limited, we couldn't find a way to copy-paste into the shell however we were able to navigate directories, create files, clone, vagrant up, vagrant ssh, vagrant destroy, etc.

@iankronquist the git shell command is not the same as the git shell that gets used in windows. A number of git commands rely on some basic Unixy utilities and functions, and so the windows version has to replicate those.

lgtm -- will merge


build(capacitor): :arrow_up: upgrade for capacitor v5

Description

Adds support for Capacitor v5

Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

[ ] Test A
[ ] Test B

Test Configuration:

Device/OS:
Browser:
Version:

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

Screenshots (if appropriate):
Add any screenshots or screen recordings that help visualize the changes made in this pull request.

@mrozema sorry to get back to you so late. We haven't yet updated into v5, but will do so in next week and upgrade this plugin as well.


zplug: Add initial support for zplug package manager

Hi,
I've added support for the zplug package manager for zsh.
I'd be grateful for any feedback.
s1341

Thanks for the contribution! I've added a few comments.

Thanks for the contribution! I've added a few comments.

I'll get on those. I'll start with a PR for zplug in nixpkgs. That might take a few days to get through their process.

zplug PR in nixpkgs: https://github.com/NixOS/nixpkgs/pull/83856

@rycee I've resolved all the issues you mentioned. Do you want to take another look?

Thanks, I added a test case and rebased to master in 1b210e7143547ce0f41e8082b8d27e9c7d220351.


[Feature]: X11 Forwarding Support?

Source problem
X11 forwarding is useful if you want to work with graphical applications
Feature you like
X11 Forwarding by default
Describe alternatives you've considered
No response
Additional context
No response

Hi @WassimBenzarti X11 forwarding enables us to run GUI applications over SSHwithout the need to install any VNC or RDP. If I want to render something and want to visualize it X11 forwarding is one way to do it and efficient compared to full RDP.
This video demonstrates how to use it and some use cases in the beginning.


Restore Danger on Circle 2

This was omitted during the migration to Circle 2 because Danger support landed after the Circle 2 PR was originally opened.
Test Plan
We'll see how Circle 2 handles this config, hopefully the bot will comment here.

Let's wait for master to go green and test. I also need to make sure this doesn't break on release branch.

Circle isn't running here, so I'll have to create a new PR.


Fix the session store. ( do not use memory store on production )

Hey there,
Currently the session are using a memory store even on production. We need to change it to a database store.
Kind regards.
Massiles.

I will take charge of this one.

this was fixed on 54f031b4c393bcfe4daeec9aa011d89dc714834a


Green line route switcher

Summary of changes
Asana Ticket: ⏱Schedules | Green line special case for route selection
Added Green line route switching capability

Assigned to: @NAME

.m-schedule-direction__route-pattern should be Helvetica.
Hooray! We get to use my new docs maybe:

Green Line Direction picker documentation
https://zeroheight.com/2fedee66c/p/33616f/b/59a3ae/t/869217
Referring to the examples contained in the link above, looks like we're using route-long-name instead of the direction-destinations field. For example, we should say "[Green line B] Boston College"  and "[Green Line B] Park St" rather than "Green Line B" for both the eastbound and westbound label.
Since it's semantically important, the branch icons need aria descriptions so they say "Green Line B".

From an accessibility standpoint, we've heard some complaints when focus doesn't return to the direction picker after it's been interacted with and it transforms the page. But since we're navigating to a new page entirely, I'm wondering if changing the focus is still a good idea?

@ryan-mahoney took another look, don't have anything further to add at the moment.

@phildarnowsky is this still in review? can we merge this work?

@ingridpierre I don't believe @ryan-mahoney has made the changes I called for, correct me if I'm wrong Ryan.

yes, there is still some work remaining

The Green Line icons should not be aria-hidden. We should have them read out "Green Line", "Green Line B", etc since they are pretty important in this context.

Seeing an issue with branch direction names. It's coming through as Eastbound: North Station. Should say Eastbound: Park Street.

See direction_destination here, subway lines should use that field instead of headsign, shape names, or route pattern name:
https://api-v3.mbta.com/routes/Green-B

@ingridpierre please take another look. I fixed the Green B issue and added screen-reader only text to address the icons being invisible to screen-readers

@ryan-mahoney LGTM


Cilium agent logs complain about Unable to fetch kubernetes labels

Affected versions:

Cilium 1.7.x
Potentially later versions, not yet observed.

At this stage, only observed in CI (https://github.com/cilium/cilium/issues/10442).
Symptoms
Always observed

Every 15s or so, the cilium-agent log prints a warning about Unable to fetch kubernetes labels:
level=warning msg="Unable to fetch kubernetes labels" containerID=015ef4fdbf datapathPolicyRevision=2 desiredPolicyRevision=1 endpointID=1596 error="pod.core \"app3-c6c587577-fq6kr\" not found" identity=5 ipv4=<IP_ADDRESS> ipv6="f00d::a0a:0:0:aeec" k8sPodName=default/app3-c6c587577-fq6kr subsys=resolve-labels-default/app3-c6c587577-fq6kr

cilium status reports failing controllers with resolve-labels-xxx:
Failed controllers:
 controller resolve-labels-default/app1-68cb4f68c5-cftnr failure 'pod.core "app1-68cb4f68c5-cftnr" not found'

There is no corresponding pod for these endpoints.

Endpoints show up in the cilium endpoint list with the identity reserved:init, and they never get a proper identity:
cmd: kubectl exec -n kube-system cilium-56pmk -- cilium endpoint list
Exitcode: 0
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])
IPv6                 IPv4          STATUS
	            ENFORCEMENT        ENFORCEMENT
	 898        Enabled            Enabled           5          reserved:init                                          f00d::a0a:0:0:af1    <IP_ADDRESS>   ready
	 1479       Enabled            Enabled           5          reserved:init                                          f00d::a0a:0:0:969e   <IP_ADDRESS>   ready

May also be observed

Cannot deploy new Kubernetes pods. Pod deploy logs report IPAM exhaustion.

Impact
~There is no known impact other than the failure logs potentially consuming disk space.~ The related application pods were already deleted so there is no traffic impact.
UPDATE 2020-02-14: One impact if pods continue to be deployed to this node while the node is in this state, is that each "phantom" endpoint consumes an IP address. If the total number of "phantom" endpoints reaches the size of the podCIDR allocated to the node, then IPAM will begin to fail and subsequent pod deployments onto the node may fail until user action is taken to mitigate.
Migitation
Restarting the cilium-agent should cause Cilium to reevaluate the existence of this endpoint and clean up.
If you have more details on this issue, for example because you observe it in a real Cilium deployment, please post details below and react to this issue with :+1: so we know how widely it afffects the community.

Seeing this currently happening to one Cilium 1.7.4 agent in k8s 1.16:
Controller Status:      157/159 healthy
  Name                                                                  Last success   Last error   Count   Message
  resolve-labels-deployment-4459/webserver-deployment-c7997dcc8-tgq7m   never          2m26s ago    231     pod.core "webserver-deployment-c7997dcc8-tgq7m" not found
  sync-to-k8s-ciliumendpoint (3912)                                     never          2m8s ago     231     namespaces "deployment-4459" not found

the pod and namespace in question were created and removed by Sonobuoy. It doesn't appear to be causing actual issues. Running cilium endpoint disconnect 3912 resolved the complaints.

Observed again with Cilium v1.7.11.

Observed again with Cilium v1.7.11.

@diversario have you seen this issue again recently? Are you still using Cilium v1.7.x or have you moved to a newer release at this point?

@diversario have you seen this issue again recently? Are you still using Cilium v1.7.x or have you moved to a newer release at this point?

Observed in v1.9.1. Will update to v1.9.4 and hope for the best.

@diversario have you seen this issue again recently? Are you still using Cilium v1.7.x or have you moved to a newer release at this point?

I've not looked into this recently so I can't tell one way or another. I'm currently at 1.8.x.

I just ran into this too with Cilium 1.9.6 and Kubernetes 1.19.10 too.
The 'phantom' pods were all previously-deleted jobs.

This is a really hacky thing, but maybe someone can use it too.
 kubectl -n kube-system get pods -l k8s-app=cilium -oname | xargs -I{} kubectl -n kube-system exec {} -- /bin/bash -c "cilium endpoint list | grep serviceaccount=failed-job-reaper -B2 | grep uid | cut -d' ' -f1 | xargs -n1 -P20 cilium endpoint disconnect"

failed-job-reaper was the service account, play with it and let it do your thing.

I've reopened the issue since it seems like community users are still observing this.
As for next steps, I think we need a short set of reliable steps to reproduce the issue. It's a bit tricky to understand how Cilium manages to get into this state in the first place at the moment. Is it due to use of k8s jobs or batches? Or are there other common steps that trigger this behaviour? It may also help if someone can upload a sysdump from a cluster where the problem is actively occurring.

We will get a sysdump next time this triggers

@ulfox @mnaser @uhthomas I'm going to close this GH issue. Feel free to open a new GH issue with a sysdump in case you see it again. Thank you!

We're observing this behaviour on one of our AKS clusters. Unfortunately, before doing a sysdump, we've killed the agent. We'd provide a sysdump once this happens again. Can you re-open this issue, @joestringer ?

/cc @mblaschke

@joestringer fyi: generated
sysdump while facing described issue

I'm seeing this in k8s 1.26.3 with cilium 1.13.2
level=warning msg="Unable to fetch kubernetes labels" containerID= datapathPolicyRevision=0 desiredPolicyRevision=0 endpointID=527 error="pod.core "api-687d68f54d-4fp9h" not found" identity=5 ipv4= ipv6= k8sPodName=/ subsys=resolve-labels

Doing a cilium ip list
Error: Cannot get ipcache entries. err: Cilium API client timeout exceeded

Just want to say we're occasionally running into this as well.  At some point, a Cilium pod stops being able to communicate with the apiServer.
We're seeing this error - Unable to fetch kubernetes labels
cilium ip list returns a time out, and new pods cannot be created on the node with this error:
3m38s       Warning   FailedCreatePodSandBox   pod/blah                  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "75ae743023fa39316767c49aefb58833863f09f28793a4c019e8bb52c536b5da" network for pod "blah": networkPlugin cni failed to set up pod "blah" network: Unable to create endpoint: Cilium API client timeout exceeded

cilium status looks green, IPAM shows 17/254 IPs allocated.

We were experiencing the same problem in EKS 1.24 and 1.25, we created a cronjob to restart the cilium agent every 3h and this helped us to reduce 95% of the incidents.
However since we upgraded cilium to 1.13.4 the issue disappeared completely, posting this after two weeks of testing and monitoring in prod, as we can't reproduce the issue anymore.

Hi we had a similar issue with Cilium 1.13.2. One of the worker node isn't able to contact the API server with cilium ip list and every new pods scheduled on that node can't contact any other services. With cilium monitor -t drop on the agent of the broken node all the packets are dropped with Policy denied.

Observe this in v1.14.6.


Adds building static libraries

Adds building static libraries in both Linux and Windows.
Note: This is the #730 PR but with the Windows ^M characters stripped.

I've tested this in Linux and Windows and it works for me in both environments.
The only real hesitation I have has to do with any possible licence incompatibilities although it would appear BSD-3 and Apache are compatible

Going to close this issue and create a new PR if I can implement the functionality without licensing issues


Component resizing

Hi,
if I have only one component on GridStackLayout, I can't vertical resize this component. You can check this issue on the demo site: http://app.siika.fi/GridStackDemo/
Remove all components, except the Form, then try to resize it...
Have you got any idea?
Thanks.

Yes, I can only change size horizontally. Looks like single item gets limited to height of 1 slot. I will check as soon as I find some time.

min-height: 100% applied to .v-panel-content > .grid-stack breaks resizing vertically


Lua: unexpected interaction between set_viewport_size and set_viewport_full

When there's a site that comfortably fits into the viewport horizontally, its preferredContentsSize().width() matches that of the viewport, so set_viewport_full usually only changes the height of the viewport.
However, if one does
splash:set_viewport_size(1920, 1080)
splash:wait(0.1)
splash:set_viewport_size(640, 480)
w,h = splash:set_viewport_full()

w and h will somewhat unexpectedly be 1920 and 1080 respectively. Most likely this happens because page layouting is asynchronous and hence the code that should relayout the page and change mainFrame()->contentsSize is not invoked between the last-but-one and the last lines.

Note: if you do
splash:set_viewport_size(640, 480)
png = splash:png()

PNG is still laid out properly, because rendering forces a synchronous layout just before the operation is performed. The problem with splash:png{render_all=1} is that fetching contentsSize happens before the relayout and thus new size is not taken into account.
So, one could fix this synchronously by doing
splash:set_viewport_size(640, 480)
splash:png()
w, h = splash:set_viewport_full() -- w=640, h=480 here

Turns out, there's a way that doesn't involve generating an image and discarding it (see #171).


CSV Input parser and line breaks in quoted columns fails

Env:
Embulk 0.8.15, OSX
With minimum configuration
in:
  type: file
  path_prefix: test.csv
  parser:
    type: csv
    delimiter: ";"
    charset: utf-8
    newline: CRLF
    columns:
    - {name: a, type: string}
    - {name: b, type: string}
    - {name: c, type: string}
    - {name: d, type: string}
out:
  type: stdout

and test.csv
"1","2","
3","4"

Embulk gives
2016-12-29 23:15:49.334 +0200 [WARN] (0016:task-0000): Skipped line 1 (Unexpected extra character ',' after a value quoted by '"'): "1","2","
2016-12-29 23:15:49.337 +0200 [WARN] (0016:task-0000): Skipped line 2 (Too few columns): 3","4"
2016-12-29 23:15:49.339 +0200 [INFO] (0001:transaction): {done:  1 / 1, running: 0}
2016-12-29 23:15:49.345 +0200 [INFO] (main): Committed.
2016-12-29 23:15:49.346 +0200 [INFO] (main): Next config diff: {"in":{"last_path":"test.csv"},"out":{}}

I'm kinda expecting ruby like behaviour;
CSV.open('test.csv', headers: %w(a b c d)).first
=> #<CSV::Row "a":"1" "b":"2" "c":"\n3" "d":"4">

Embulk should handle this file. Please specify and try "," as delimiter option instead of ";" like:
    delimiter: ","

Aah, yes. My bad, I was somehow mixing the error messages, thought that this was a parsing issue when reality it was related to java.lang.NumberFormatException (I was using long instead of string).
Closing.


Cache and Pre cache videos

Use case
Adding the feature of caching video while playing
It will be nice to add preCaching function without having a controller for every cached video as in some cases we need to precache few seconds from a lot of videos ( that can be streamed later).

Proposal
In android native there's  CacheUtil inside exoplayer like an example. It will be great if we have the some feature on flutter too

Hi @saddem123
I see there are opens issue addressing the case you described #28094 and #92597 (Web)
Closing as a duplication, so please follow up on them for further updates.


Please clarify the statement in the doc

[Enter feedback here]
The doc states "This data is stored for 30 days", but if I build a LUIS app, I want the data to live for as long as my app exist.  What does this 30 days limit mean?  Thank you.
Sidney

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 98177a83-fc2d-6818-a190-b5e3623ba008
Version Independent ID: 901d4552-20bf-cde3-c18c-bfe157777c1c
Content: Data storage - LUIS - Azure Cognitive Services
Content Source: articles/cognitive-services/LUIS/luis-concept-data-storage.md
Service: cognitive-services
Sub-service: language-understanding
GitHub Login: @aahill
Microsoft Alias: aahi

Thanks for the feedback! We are currently investigating and will update you shortly.

Thanks @yfphoon
Data used to train the model such as entities, intents, and utterances will be saved in LUIS for the lifetime of the application. If an owner or contributor deletes the app, this data will be deleted with it. If an application hasn't been used in 90 days, it will be deleted.
Application authors can choose to enable logging on the utterances that are sent to a published application. If enabled, utterances will be saved for 30 days, and can be viewed by the application author. If logging isn't enabled when the application is published, this data is not stored.
I'll update the documentation with this information soon!
#please-close


Switch to an another JSON schema validator

Ditching verify, and using jsonschema instead would probably bring several improvements.
The original idea of using verify was to avoid looking up the positions invalid sections after validation, however it is very slow, not extensible, and I don't have time to maintain it.
Using an actually maintained validator would be an improvement already, especially if it supports resolving references. On top of that jsonschema can be up to 10000x faster (yes, indeed), and even with the additional lookups validation should be much faster.

For anyone wondering why there is no progress on this, this is blocked on https://github.com/Stranger6667/jsonschema-rs/issues/246. Taplo currently works in WASM environments which is a feature that we'll likely keep.