Reducing read_metadata output size in pyarrow/parquet

This is possible fix for the large-dask-graph problem raised in #5357.  We avoid passing around a pq.ParquetDataset "piece", and instead pass around the path and partition_keys members of each piece.  This dramitically reduces the amount of metadata stored in the task graph.
Since this solution does require the metadata to be parsed by each task in read_partition, I am including the results of a simple benchmark (performed on my local machine):
import dask
import dask.dataframe as dd
import cloudpickle

ddf = dask.datasets.timeseries(...)
ddf.to_parquet(file, engine="pyarrow")

read_df = dd.read_parquet(file, engine="pyarrow")
fun = read_df.__dask_graph__().values()[0]
print("Graph Size:", len(cloudpickle.dumps(fun)))
%timeit read_df.compute()

OUTPUT...
OLD ~365 partitions:
Graph Size: 206328
5.71 s ± 22.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

NEW ~365 partitions:
Graph Size: 2022
5.74 s ± 101 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

OLD ~36k partitions:
Graph Size: 2144624
16.5 s ± 648 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

NEW ~36k partitions:
Graph Size: 2022
16 s ± 1.24 s per loop (mean ± std. dev. of 7 runs, 1 loop each)

[x] Tests added / passed
[x] Passes black dask / flake8 dask

Since this solution does require the metadata to be parsed by each task in read_partition, I am including the results of a simple benchmark (performed on my local machine):

@rjzamora if you're willing to humor me, can you try one of these benchmarks also with the distributed scheduler?  To do this you would add the following line of code sometime before you call .compute()
from dask.distributed import Client
client = Client()

I would expect this to accentuate the slowdown of moving large graphs around.

@birdsarah , this might help out some of your workloads (if you're still interested in engaging here).  I suspect that this might have been slowing you down on many-partition workloads.

if you're willing to humor me, can you try one of these benchmarks also with the distributed scheduler? To do this you would add the following line of code sometime before you call .compute()

Absolutely - Sorry, should have done this the first time around :)

dask.distributed version of the benchmark above...
OLD ~365 partitions:
Graph Size: 206328
9.63 s ± 139 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

NEW ~365 partitions:
Graph Size: 2022
8.54 s ± 191 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

For dask.distributed, the ~36k-partition example takes ~13.6s for the new code, but does not work for the "old" code.  I get the following output:

Click to Expand

```
/Users/rzamora/.local/lib/python3.7/site-packages/distributed/worker.py:2794: UserWarning: Large object of size 2.14 MB detected in task graph:
  (future = client.submit(func, big_data)    # bad

big_future = client.scatter(big_data)     # good
future = client.submit(func, big_future)  # good

% (format_bytes(len(b)), s))
distributed.nanny - WARNING - Worker exceeded 95% memory budget. Restarting
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
tornado.application - ERROR - Exception in callback <bound method Nanny.memory_monitor of <Nanny: tcp://<IP_ADDRESS>:56387, threads: 3>>
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: 'Process' object has no attribute '_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 342, in wrapper
ret = self._cache[fun]
AttributeError: _cache
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 359, in catch_zombie
yield
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
ProcessLookupError: [Errno 3] No such process
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/tornado/ioloop.py", line 907, in _run
return self.callback()
File "/Users/rzamora/.local/lib/python3.7/site-packages/distributed/nanny.py", line 266, in memory_monitor
memory = proc.memory_info().rss
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/init.py", line 1170, in memory_info
return self._proc.memory_info()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 469, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 339, in wrapper
return fun(self, *args, **kwargs)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_common.py", line 345, in wrapper
return fun(self)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 400, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/contextlib.py", line 130, in exit
self.gen.throw(type, value, traceback)
File "/anaconda3/envs/dask-parquet-dev/lib/python3.7/site-packages/psutil/_psosx.py", line 372, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=6545)
distributed.nanny - WARNING - Worker process 6545 was killed by signal 15
</details>

Woot!  Nice result.

Thoughts on testing?  Maybe we could test that the size of the serialized graph was small like the following:
df = ... # some dataframe with lots of columns
df.to_parquet(tmppath)
df = dd.read_parquet(tmppath)
assert len(pickle.dumps(df.__dask_graph__())) < 10000  # or some other suitable number

Update:  I revised the fastparquet engine to avoid passing around row-group and ParquetFile objects in the task graph (when possible).  This was a bit trickier to do in fastparquet - Not sure if I implemented the best solution here.

@rjzamora I'm testing out this PR. The failing test makes me nervous. Can you contextualize?

Well, turns out I can only offer quick feedback as I can't actually use this branch due to s3fs problems but I think this solves my large partitions with metadata problem. Without this branch on 2.4.0 I'm seeing "large item in task graph" and un-ending wait time for job to start. With it, work starts in a timely manner.
Small piece of feedback, reading is still slower though. In this particular case I was reading 33 parquet files with a '00 to '000 of partitions. Repartitioning to smaller number and writing out.
To do the read of 33 separate days and concat step on dask 2.1.0 takes ~40s on dask 2.4.0 takes ~1min 30 (sped up from ~1min 40 to ~1min 25 moving from dask 2.4.0 to this branch but that might be a coincidence not this branch). Not the end of the world and with potential other performance savings it's liveable with, just an observation.

@birdsarah - Are you testing this PR with the fastparquet or pyarrow engine?  Since the original issue was for pyarrow, and the fastparquet changes are causing CI failures, I was planning to revert the fastparquet changes and revisit them later (if needed).
If the fastparquet changes are useful to you, I will do my best to address the CI problem (and prehaps the performance problem) here.

I tested on fastparquet. And that's what I'm using for all my data. Unfortunately pyarrow did not support the reading in of complex datatypes at the start of my pipeline so I'm fastparquet all the way.
Do not rush on my account. The fsspec/s3fs problems mean that anything after dask 2.1.0 is completely unusable for me. Last night I was hoping that https://github.com/dask/s3fs/pull/237 would have solved the s3fs problems but it hasn't.

I take it back @rjzamora: do rush on my account!! I will shortly be done writing large volumes of data to s3, which means I should be able to use latest dask for analysis and, therefore upgrade. Let me know what / when you'd like me to test.

@birdsarah - I believe the CI problem has been addressed.  Unfortunatley,  I am not sure how to test the performance regression from 2.1.0 to this branch.  I am personally finding a degredation in performance when I move to 2.1.0 --  Can you provide a simple tests case?
Note that I just tried:
ddf = dask.datasets.timeseries(
    start = "2000-01-01",
    end = "2000-12-31",
    freq = "1S",
    partition_freq="1D",
    seed = 42,
    id_lam=30,
)

Reading this dataset changed from ~1:06 to ~1:50 when I swapped out this branch for 2.1.0

Can you provide a simple tests case?

No. Sorry. I can only report what I see on my cluster when switching between environments. There are so many moving pieces between switching from s3fs=0.2.0 and dask=2.1.0 (env a) and s3fs and dask latest (env b). To provide a rigorous isolated test case is something I simply don't have time for especially given the limited value. I appreciate you trying to find one. The fact that we are seeing performance differences in the opposite direction doesn't entirely surprise me and these performance differences don't appear to be deal breakers (as compared to the fact that dask 2.4.0 can't read large datasets with _metadata right now).

I believe the CI problem has been addressed.

Woot :D

@martindurant - Any chance you can take a look at this?

@martindurant last chance :)
Merging this afternoon if there are no further comments

Thanks for reviewing @martindurant!  Hopefully both of your suggestions have been addressed now

I didn't look for long, but I think things are good.

Merging this in.  Thanks @rjzamora for fixing this and @birdsarah @martindurant for review.


⚠ Bofrost.es has degraded performance

In 6b4a7a1, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1817 ms

Resolved: Bofrost.es performance has improved in 728a636 after 26 minutes.


Got exception when running on windows 10

Expected Behavior
Following the instructions from this page https://www.doyler.net/security-not-included/gimme-aws-creds I am able to install gimme-aws-creds but when running it got some issues described below
Current Behavior

Steps to Reproduce (for bugs)

Install python from the official page (3.12)
Verify paths with where python and where pip
Install gimme-aws-creds pip3 install --upgrade gimme-aws-creds
Execute gimme-aws-creds

Your Environment

App Version used: Python 3.12
Environment name and version:
Operating System and version: Windows 10

I removed the method_whitelist parameter and it resolved the issue. Per the urllib3 documentation it seems allowed_methods is the correct parameter but POST is not an acceptable value. Could be something they missed on their end, but either way I don't think it will work here.
https://urllib3.readthedocs.io/en/latest/reference/urllib3.util.html#urllib3.util.Retry
I can create a PR but I've only tested on Windows with python3.11.

It looks like they've fixed it in code here but there hasn't been a release for this project since the fix was made.
Until they release a newer version of the python module, one way to update to get the new code would be to call pip3 install --upgrade git+git://github.com/Nike-Inc/gimme-aws-creds.git as outlined in the readme of this project.

fixed in 2.8.2


Restructure expiry date and bought date to use singleton design pattern

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@2c675be). Click here to learn what that means.
The diff coverage is 76.47%.

:exclamation: Current head e7e409a differs from pull request most recent head 38e6183. Consider uploading reports for the commit 38e6183 to get more accurate results

@@            Coverage Diff            @@
##             master     #290   +/-   ##
=========================================
  Coverage          ?   77.72%
  Complexity        ?      679
=========================================
  Files             ?      117
  Lines             ?     2070
  Branches          ?      223
=========================================
  Hits              ?     1609
  Misses            ?      373
  Partials          ?       88

Impacted Files
Coverage Δ

.../java/seedu/foodrem/model/util/SampleDataUtil.java
0.00% <0.00%> (ø)

src/main/java/seedu/foodrem/ui/ItemCard.java
0.00% <0.00%> (ø)

.../java/seedu/foodrem/model/item/ItemBoughtDate.java
87.50% <88.88%> (ø)

.../java/seedu/foodrem/model/item/ItemExpiryDate.java
87.50% <88.88%> (ø)

...in/java/seedu/foodrem/logic/parser/ParserUtil.java
96.87% <100.00%> (ø)

src/main/java/seedu/foodrem/model/item/Item.java
81.66% <100.00%> (ø)

...l/item/itemvalidators/ItemBoughtDateValidator.java
92.30% <100.00%> (ø)

...l/item/itemvalidators/ItemExpiryDateValidator.java
92.30% <100.00%> (ø)

...in/java/seedu/foodrem/storage/JsonAdaptedItem.java
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


A page about settings in pharo

I fixed the links and removed the credits for the table of contents


Do not add a cmd:awk dependency as nothing will ever provide cmd:awk

Do not add a cmd:awk dependency as nothing will ever provide cmd:awk
awk is only provided by busybox or gawk.  Both of those do so with a
symlink.  That means that we have to ignore shbang that have awk or
we'll get a 'cmd:awk' dep that will never be provided.
Sad face.
Somehow we need to fix the symlink provider thing, but for
now, just this.
This was noticed in wolfi-dev/os/check.yaml . Melange said:
Added shbang dep cmd:awk for usr/bin/checkmk

Could teach apk to understand and respect symlinks for executables in the $PATH?  I sure wish "apk add cmd:awk" worked as one would have expected...

IIRC @xnox said that was not desirable behavior.

Why?  Because of the ambiguity of cmd:awk either being busybox or gawk?  Or something else?

Could we teach apk to understand and respect symlinks for executables in the $PATH?  I sure wish "apk add cmd:awk" worked as one would have expected...

We can, but they all must be generated without version, and then all of them will be coinstallable.
Or none should provide it.
It is ambiguous which thing should provide awk, and we may not prevent co-installing gnu-awk & busybox.
Such resolution is possible in APK & APKO, but we currently do not generate such resolutions.
I.e. symlinks need to be weak provides.


#497 bugfix: match specific installed version with *

Fixes #497

Fixes a bug where get-version would still print install instructions even if the tool is installed and no specific version is configured
Improves tests for get-version and set-version

Pull Request Test Coverage Report for Build<PHONE_NUMBER>3
Details

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.03%) to 62.826%

Files with Coverage Reduction
New Missed Lines
%

com/devonfw/tools/ide/version/VersionIdentifier.java
1
85.71%

Totals

Change from base Build<PHONE_NUMBER>1:
0.03%

Covered Lines:
5527

Relevant Lines:
8459

💛  - Coveralls


Android strings.xml indented with tabs instead of spaces

In generated strings.xml files for Android, tabs are used instead of spaces for indentation causing Android Studio to complain. Can a config flag be added that'll help developers adhere to the coding standard they follow?

ok I am wondering if tabs are even needed, as when i look at other adapter, only spaces are used.
@raragod Do you know if eclipse or other editors require tabs?

I'm not sure about the other editors but this is the default formatting setting in Android Studio which follows the Code Style Guide defined for Android by Google: https://source.android.com/devices/architecture/hidl/code-style


Ripley stats for categorical variables

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Breaking change (change that would cause existing functionality to not work as before)
[x] New feature (non-breaking change which adds functionality)

Description

Implement various ripley's alphabet statistics for point patterns analysis.
Will also move stuff around a bit on the _patterns module
How has this been tested?

...
Closes

...

Ok, ready to review, best to be tested with the lymphnode dataset that has some spatial patterns for clusters

Details
import scanpy as sc
import squidpy as sq

adata = sc.datasets.visium_sge("V1_Human_Lymph_Node")
sc.pp.normalize_total(adata)
sc.pp.log1p(adata)
sc.pp.highly_variable_genes(adata)
sc.pp.pca(adata)
sc.pp.neighbors(adata)
sc.tl.leiden(adata)
sc.tl.umap(adata)
sc.pl.umap(adata, color="leiden")
sc.pl.spatial(adata, color="leiden")

mode = "L"
sq.gr.ripley(adata, cluster_key="leiden", mode=mode)
sq.pl.ripley(adata, cluster_key="leiden", mode=mode)

What's now available is ripley F,G and L. L is like K (which we used to have before) but sqrt(K/pi) normalized (preferrable in practice). F and G give the sme results for this example, but if the convex hull of the tissue has "holes" it would proabbly be different (essentially G would account for that, F not cause it alwasy compute the score wrt a spatial point process.
It is not optimized but I abstracted the core functions enough to make it flexible. For sure there is room for improvement but can be done later, first let's see if people use.

doics seem to be failing for napari (weird)

doics seem to be failing for napari (weird)

Strange, neither nbformat, nor nbsphinx had a new release recently afaik, will look into this and review this.

ah damn missed literal sorry for that. Now looks good! probably temporary bug

must say was a very gentle review 😅 😅 thanks ! Addressed pretty much everything like you suggested, ready to review again.

Thanks a lot @giovp , really nice PR, you can go ahead and merged (I've only fixed some typo + link to neighbors).


Migrate to eslint

Since TSLint is deprecated in 2019, typescript-eslint should be used as a successor. And I could help with migration. I did it in several of my projects, so it would be not so hard.
Pretty sure that config based on @typescript-eslint/recommended plugin will fit perfectly
What do you think?

Yeah, we should migrate the repo code to use it, and also change some of the plugins generate eslint ignore comments instead of tslint.
Thanks @standy

Done in v1.13.1 🎉  thanks @standy


Raspberry Pi to ESP8266

So far I've seen issues related to using the PI itself to generate the data, but what if I wanted to connect my ESP8266s to my Pi. Would this be feasible? Also, would it be too under powered?
The ideal setup I have is
(1) Set up Pi so my phone can connect to it through bluetooth
(2) Play music on my phone
(3) Music sends signal to my ESP8266s
Update: I got this working for one ESP8266, will try pairing multiple and reporting back the results of it.
Update2: Tried 2 and the lag was insane. The pi couldn't keep up with two lights. I might have to rethink how I could use my phone to stream music and interact with the esp8266s that have the LEDs.

Did u have any issues running main.py in RPI? I'm trying to running on Linux but a have a bunch of error, I would like to know if u have any error running this in raspian

What kind of errors are you getting? I did have to modify the main.py a bit to function correctly and can share my fixes. It's not entirely perfect but it works.

Sure, thanks man, I just came for work I'll try it in a sec
On Sun, Feb 3, 2019, 2:30 PM David Bour<EMAIL_ADDRESS>wrote:

when i run it, this show up
Could not find settings.ini No user colours found No user gradients found
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear ALSA
lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side ALSA
lib pcm_route.c:867:(find_matching_chmap) Found no matching channel map
ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching channel
map ALSA lib pcm_route.c:867:(find_matching_chmap) Found no matching
channel map ALSA lib pcm_route.c:867:(find_matching_chmap) Found no
matching channel map connect(2) call to /dev/shm/jack-1000/default/jack_0
failed (err=No existe el archivo o el directorio) attempt to connect to
server failed Traceback (most recent call last): File "main.py", line 2046,
in  microphone = Microphone(microphone_update) File "main.py", line
847, in init self.setDevice(self.default_device_id) File "main.py",
line 858, in setDevice self.device_id = self.devices[device_id]["index"]
IndexError: list index out of range
it would be a lot of help if you share your fixes man
Try this and let me know how it works
#71 https://github.com/not-matt/Systematic-LEDs/pull/71
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/not-matt/Systematic-LEDs/issues/70#issuecomment-460085495,
or mute the thread
https://github.com/notifications/unsubscribe-auth/As6sG68sxiInqyCF363Sur9ftkZRbDIfks5vJ0bLgaJpZM4afdX-
.

hi, it works good, thanks man, but it seems like its a little unestable when im trying to change the mic set up, same with the color control, and first i was playing with the effects options and it crashed, all of this things happend to your raspberry? also, im curious of how you can connect your phone to the RPI and play music? in fact, im working on this project and trying to do the same as you. cheers
pd. if someone is facing the same issue i modify main.py with those lines of code #71 and #53

Glad it works. There's some still GUI functions that crash that I haven't gotten around to fixing. Is this the issues you're referring to? Is it something along the lines of a KeyError?
As for the Raspberry Pi, I used bluetooth to connect my phone to my Pi, then ran the systematic-led script. Here's a tutorial of how you can activate bluetooth (https://didier.io/post/raspberrypi-stream-sound-over-bluetooth) Note, it will be very laggy and not sure if this approach is worth it.

Yeah that was with the effects option, but sometimes happen and sometimes
it goes well,  a little wierd, do you think it only happens with
Linux based systems??
What if a aux cable is used trough a USB audio device?
On Sun, Feb 3, 2019, 11:47 PM David Bour<EMAIL_ADDRESS>wrote:

hi, it works good, thanks man, but it seems like its a little unestable
when im trying to change the mic set up, same with the color control, and
first i was playing with the effects options and it crashed, all of this
things happend to your raspberry? also, im curious of how you can connect
your phone to the RPI and play music? in fact, im working on this project
and trying to do the same as you. cheers
pd. if someone is facing the same issue i modify main.py with those lines
of code #71 https://github.com/not-matt/Systematic-LEDs/pull/71 and #53
https://github.com/not-matt/Systematic-LEDs/issues/53
Glad it works. There's some still GUI functions that crash that I haven't
gotten around to fixing. Is this the issues you're referring to? Is it
something along the lines of a KeyError?
As for the Raspberry Pi, I used bluetooth to connect my phone to my Pi,
then ran the systematic-led script. Here's a tutorial of how you can
activate bluetooth (
https://didier.io/post/raspberrypi-stream-sound-over-bluetooth) Note, it
will be very laggy and not sure if this approach is worth it.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/not-matt/Systematic-LEDs/issues/70#issuecomment-460137823,
or mute the thread
https://github.com/notifications/unsubscribe-auth/As6sGyrHvciG4gVZLcjZfP9ZzruosDkoks5vJ8mAgaJpZM4afdX-
.

The aux might help with the lag a bit as its one less layer, but I think the main issue is just the Pi isn't powerful/fast enough to output the data to many ESP8266s at once.

Maybe with some pi alternatives like these, may have different results
https://www.zdnet.com/pictures/top-12-raspberry-pi-alternatives-february-2019/3/


🅰 feat: dynamic font size

Introduced font-sizing in settings with all translations, Added a CSS var for font-size which changes according to the localstored font-size or default i.e 16px. #3207
Pull Request Template
⚠ Before Submitting a PR, Please Review:

Please ensure that you have thoroughly read and understood the Contributing Docs before submitting your Pull Request.

⚠ Documentation Updates Notice:

Kindly note that documentation updates are managed in this repository: librechat.ai

Summary
Please provide a brief summary of your changes and the related issue. Include any motivation and context that is relevant to your changes. If there are any dependencies necessary for your changes, please list them here.
Change Type
Please delete any irrelevant options.

[x] New feature (non-breaking change which adds functionality)

Testing
Please describe your test process and include instructions so that we can reproduce your test. If there are any important variables for your testing configuration, list them here.
Test Configuration:
Checklist
Please delete any irrelevant options.

[x] My code adheres to this project's style guidelines
[x] I have performed a self-review of my own code
[x] I have commented in any complex areas of my code
[x] My changes do not introduce new warnings
[ ] I have written tests demonstrating that my changes are effective or that my feature works
[x] Local unit tests pass with my changes
[ ] Any changes dependent on mine have been merged and published in downstream modules.
[ ] A pull request for updating the documentation has been submitted.

also i am fairly new to open source, please point out the points of improvement. Here to learn : )

Hi @krakenftw thanks for your contribution.
At the moment, I can't accept this PR, there are too many linting changes in the stylesheet for me to review. In general, you should prevent any linters that don't result in git differences. If you would like to contribute a CSS linter setup, that would be in a separate PR.
Also, I think it makes sense to make the font size configurable for messages exclusively.
Your addition to the settings tab is still useful, and I can take over from here to revise this if you'd like.

Hey @danny-avila
The linting changes were mess for me honestly, Prettier was doing the stuff and changed the whole thing.
So basically that font-size would only affect the user/model messages?. I am ready to do this.
Also an idea of which method i should opt to do this would be helpful. Since i am fairly new to open source.

Sorry for the long wait, applying my changes here: https://github.com/danny-avila/LibreChat/compare/feat/dynamic-font-size?expand=1


Level 3

Not sure why my timings for serve and done event are wrong.

I think you can check whether your server's end time (to know when to serve the next customer) is updated correctly.

I think you need to check that there are no customers currently being served before servign the next one

you can check whether the next available time of your server is being updated correctly at each event. the state of the shop has to be updated as you update your servers

2 is being served while 1 is still being served so I think you need to make sure 1 is done being served before Serve event for 2 is invoked

Server 1 is serving customer 2 before it is done serving customer 1. Maybe you would want to check the flow from a wait to a serve event, the timing that you input when you create a serve event

faced this problem initially too, like what the others said u need to check the flow from wait to serve, what I did personally was to create 2 different serve methods to differentiate the flow where one is serving a new customer and one is serving the customer in the queue


fix submit form after failing

Closed #1641.

Current coverage is 100% (diff: 100%)

Merging #1654 into master will not change coverage

@@           master   #1654   diff @@
=====================================
  Files          52      52
  Lines        1097    1098     +1
  Methods         0       0
  Messages        0       0
  Branches        0       0
=====================================
+ Hits         1097    1098     +1
  Misses          0       0
  Partials        0       0

Powered by Codecov. Last update 247e596...cbb8b62

Nice work. 👍

Published in v6.0.2.


Extend linear approximation of aponeuroses in apoLocation function

Description
apoLocation function uses radon transform function from skimage.transform module. This radon function works on a circle within the image -> the result of the inverse transform does not appear in the entire image but only in the part of the image that was processed for the radon transform.
Hence, the line approximating the aponeuroses does not reach the right and left borders of the image. A way to extend this line to the border must be found.

The equation of the line has been identified. Let x be the line index and y the column index, the equation has the shape:
x = ay + b
Then, for each column y, x is calculated.


Dashes shown in the examples are not the same chars that the CLI recognizes

If you copy and paste many of these examples, they will fail with an 'unrecognized arguments' error. You would need to re-type the dash/hyphen chars with the standard ones on your keyboard.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 643e1686-5729-90e8-83df-85f7f9d1b577
Version Independent ID: 6258c40a-5043-7cd9-a151-f5b700b69a8c
Content: How to tag an Azure Linux virtual machine
Content Source: articles/virtual-machines/linux/tag.md
Service: virtual-machines-linux
GitHub Login: @mmccrory
Microsoft Alias: memccror

This may be addressed by issue https://github.com/MicrosoftDocs/azure-docs/pull/6581

@kjhosein
Thanks for the feedback. We are actively investigating and will get back to you soon.

@kjhosein thanks for pointing this out. I have noticed in a few examples on other docs that when you copy and paste commands sometimes the space between commands is off.
I will assign this to the content author to double check that the dashes are correctly formatted.
@mmccrory  can you please take a look?

Thanks for the feedback, @kjhosein. I've created a PR that updates this. When the PR merges, this issue will automatically close. The doc will then publish live later on this afternoon.
CC @mmccrory for awareness


한국이름 시체를 수거 하지않아요

한국이름 캐릭터입니다
죽으면 시체를 수거하지않아요
어떻게 해야하나요
이전버전은 수거가 됩니다

How about using google translate?

Korean name character
We don't collect the body when we die.
What should I do
The previous version will be collected
(Google Translated line for line.)

The other question is that is the Korean character only in Korean or are you running the client with Korean language selected as well. Bot has to be ran in English language only. Rerun as English if you're not and run autosetting configuration tool.

It is embarrassing.. korean..

english pls

창피하다 한국 욕먹이지 말고 영어좀 익히고 글싸라


Clarify 3.6

https://github.com/egouldo/EcoConsPreReg/blob/b5d4ed469c4ceb09c2f57db28f570960785baea0/analysis/preregistration_template.Rmd#L219-L222

[x] to move "For data-driven and model-selection approaches that determine model structure and parameters, describe any initial model specifications and parameterisations, including for any tune-in parameters. (Should this go here or in model calibration?)" to model calibration section.

Evaluate, as per HF's comments whether this item is needed at all. I think it's actually just a part of the modelling workflow but is unnecessary in the template itself.
I think I'm finding the task of figuring out where the process of preregistration ends, and the doing begins, to be cognitively difficult...
I guess I was getting at some form of documentation of the fully-specified model. But "translate the conceptual model into the quantitative model" seems to be more of a reporting item than an item that documents any decision-making process by the author / analyst.

I've updated this on Hannah's PR branch, but wording will need to change.

f753664e0919c0cb04295fbb02aa0bf0a9a4827f


Net dmg extraction

Adds dmg support for signature extraction and hash generation.

Could you please rebase on master without any changes in .github/workflows
Please completely remove all private non-static fields from DmgFile like _stream.
Make _signatureData public and remove HasSignature() and SignatureData() properties.

Please add tests for ComputeHashInfo

PList can be binary format not only XML.
Can we read PList blob only by request? I think PList partsing is not a part of this pull-request.
RsrcForkOffset and DataForkOffset are not using in your code
All internal classes should be in Impl directory
Why BLKXEntry.AttributesAttributes has string type?


Image component should respect objectFit even if height isn't provided

The height prop can be used with objectFit to "crop" images. This is inconvenient for responsive designs since setting height in CSS is better. Currently, if height is passed in through CSS, the objectFit is ignored. We should explore changing the logic so that height is not required for objectFit.

This was fixed in #696.


realtime bounding box

i use tensorflow project for android
I trained my custom datsets by yolo and then i made pb file.
This detected bounding box where i used picture not realtime
But when i put yolo file  in the android project, this cannot detect.
So i screenshot the scene of android project and i trained this screenshot in pc by darkflow.
It detected well. I don't know why. I think option value is correct. plz help me T.T

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

I'm not clear what you're asking. Is this question about tracking or YOLO?
The realtime tracking is provided by libtensorflow_demo.so. You can double-check that it's packaged in the APK with unzip -v tensorflow_demo.apk. If it's not there, you'll just see stationary boxes for detections. Bazel builds of the demo will include it, but if you use the "none" buildType in build.gradle then you won't get it.

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

Closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

is it possible to include the libtensorflow_demo.so manually and avoid building with bazel ? i downloaded the file from here created a jniLibs in my project but still the ObjectTracker can't find the library. Any solutions ?


Error with new aws/aws-sdk-php

Laravel Version: 9.40.1
PHP Version: 8.1.5
Database Driver & Version: Mysql 8.0.28

Description:
We are currently using a filesystem that is hosted at DigitalOcean (space), we are using the AWS adapter requiring a few packages, one of the packages: aws/aws-sdk-php
By upgrading from 3.242.1 to 3.243.0 we suddenly got issues in our system (a nasty Xdebug has detected a possible infinite loop, and aborted your script with a stack depth of '256' frames)

We tested by pinning in our composer.json
"aws/aws-sdk-php": "3.242.1", -> OK
"aws/aws-sdk-php": "3.243.0", -> NOT OK
After investigation, we identified this bug reported on that repo:
https://github.com/aws/aws-sdk-php/issues/2563
We tried to apply the patch proposed on src/Illuminate/Filesystem/FilesystemManager.php (before $client = new S3Client($s3Config)):
$s3Config['endpoint_provider'] = PartitionEndpointProvider::defaultProvider();
and that fixes the issue...
Seems like it's linked to a file  that starts with / (example : /private/image.gif )

Seems like they fixed it in 3.244.1 so please upgrade to that version.

We updated to 3.245.0 but the issue was still present.

Please open an issue with the AWS PHP SDK as this is a breaking change with them.

Possibly related to https://github.com/aws/aws-sdk-php/issues/2567

Seems like Laravel injects token option to some sdk clients as a root option, instead of it being inside the credentials key.

I'm not sure it's related because by patching src/Illuminate/Filesystem/FilesystemManager.php and adding
$s3Config['endpoint_provider'] = PartitionEndpointProvider::defaultProvider(); it's solving the issue. But there is something wrong with the combination laravel and aws-sdk-php 3.245.0 (after 3.242.1 at least)

@labomatik can you try the PR by @georgeboot to see if that resolves your issue? https://github.com/laravel/framework/pull/44979

That's not working with that PR

Please open an issue on the AWS PHP SDK then as this does not seem like a bug within Laravel.

Yes i'll do ;-)

Hi @driesvints, @georgeboot,  @labomatik,
Apologies for all of the issues.
@labomatik,
Glad you were able to patch this for now, but I'd like to get this addressed if this is an issue with our implementation. Could you open an issue against our repository with more information about the operations/config you're using for this library?
Thanks!


pattern suggestion for commands

fixes https://github.com/rust-lang/cargo/issues/4352

@BurntSushi can you review it for me? :)


 in insert mode does not delete previous character

Describe the bug
<C-h> in insert mode does not trigger <BS> /  does not delete previous character
To Reproduce

Enter insert mode
write some text
press <C-h>
nothing happens...

Desktop

OS: macOS 13.5.6
Terminal: kitty

Neovim Version
:version
NVIM v0.9.5
Build type: Release
LuaJIT 2.1.1703358377
system vimrc file: "$VIM/sysinit.vim"
fall-back for $VIM: "/usr/local/Cellar/neovim/0.9.5/share/nvim"

Just reproduced this on my Mac.
Also validated that the problem lies with kickstart because starting nvim --clean and Ctrl-h does indeed act as backspace as one might expect.
Interestingly, hitting Backspace performs the expected behavior of deleting the last character typed.

If you type: :map
You'll see it's already bound:
s  <C-H>       * <Lua 94: ~/.local/share/nvim/lazy/nvim-cmp/lua/cmp/utils/keymap.lua:127>
                 cmp.utils.keymap.set_map
n  <C-H>       * <C-W><C-H>
                 Move focus to the left window

Not a bug.
Please see the remap in url below.

You'll see it's already bound:

@feoh it is bound in here:
https://github.com/nvim-lua/kickstart.nvim/blob/773e482d4b40cec4095e4b60fbd753cb69b3f51b/init.lua#L714-L718
You can safely delete those 5 lines to make C-h work normally.
That's what I did


chore:jenkinsfile

Codecov Report

Merging #154 (94a93bc) into main (0e009ac) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #154   +/-   ##
=======================================
  Coverage   27.09%   27.09%
=======================================
  Files          49       49
  Lines        4145     4145
=======================================
  Hits         1123     1123
  Misses       2922     2922
  Partials      100      100

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e009ac...94a93bc. Read the comment docs.


Smart Charging: Add a button to jump to the Logs filtered with Smart Charging in Charging Plan

Merge and Check the feature.
Thanks.

Closed due to double issue in one item.


Add onClose() callback

Add an onClose() callback. This is very useful for folks who want to dually stream and accumulate the response (for saving to a DB).

The intention right now is that you can chain off of the returned Promise to accomplish things like this. Would that work for you?

Thanks @jaredpalmer ! The API has been overhauled, so this PR is no longer relevant.
I believe you can now accomplish this with stream.controller.on('abort', myCallback) if needed, though the following should work for most use-cases:
let text;
for await (const part of await client.completions.create({...params, stream: true})) {
  text += part.completion
  mystream.push(part)
}
await db.save(text)


Change Voucher Client server configuration to avoid conflict with environment variables

In some alpine images, when calling voucher_client from a container. The hostname would be overridden by an environment variable present in the OS causing voucher_client to make a call to the wrong endpoint.
This pr aims to rename that configuration variable so we won't run into that issue

I will merge this in once I am oncall

This introduces a breaking change for whoever is using voucher_client


Extension not working in Code 2023.1.1/Kit 105.0.2

Last night, Code 2023.1.1 was released. This uses Kit 105.0.2 and our extension fails to properly load there in Linux due to the removal of cudart libraries. Once this is resolved, as with #420, we still run into issues as the tilesets fail to load with Fabric enabled.
There are also other issues, including the Cesium Window not appearing when Code launches and the reminder to enable fabric not showing up.

We can close this now that https://github.com/CesiumGS/cesium-omniverse/pull/420 and https://github.com/CesiumGS/cesium-omniverse/pull/423 are merged.
The other issue was that Code 2023.1.1 no longer has a default light in the scene, but if you add a light the tileset is there.


wise_enum_detail.h: add inline to strcmp compare

missing inline for compare strcmp
it creates multiple defs symbols

I merged another pull request that was doing nearly the same thing. Note that in your PR, you also add inline on a function template which isn't necessary (and is unusual in C++ IME). Thanks for pointing it out!

thanks @quicknir
Yeah, the 3rd one can be removed. thanks!
Kobi


Remove duplicate headline

Hallo @wulmer,
unfortunately our commit checks failed.
We have to stick to the rules, which means that this PR needs (see also contribution.md):

a conventional commit message and a dco sign-off footer in the commit message.
your name in the header of the changed file(s)
your name in the contributors list in notice.md

Considering this is just a small fix we could also do this for you if you like and do not want to take the burden to do all this changes.
However thanks for you contribution we highly appreciate that!!


Builds take too long -- can we use bazel cache?

A clean build takes a very long time, largely due to the need to build V8 and other dependencies. #8 introduces GitHub Actions for CI and those builds are now at 2 hours and counting. On my local 32-core EPYC I can build in 10 minutes.
Incremental builds are comparatively very fast (as long as you didn't modify V8).
We need to do something about this. Probably, the answer is "something something bazel cache something", but I don't know enough about bazel cache at the moment to say exactly how this should work.

Bazel's remote caching looks like it implements quite nicely with GCS. I would have quite liked to have a try at using R2 as the caching backend, but for ease of use GCS might be the best choice.
There's also the cache GitHub action - I assume it'll be the 'fastest' choice since you'll be fetching GitHub from a GitHub runner but I haven't checked if the build output will fit within their limits.

GitHub will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited to 10 GB.
If you exceed the limit, GitHub will save the new cache but will begin evicting caches until the total size is less than the repository limit.

It looks like one of our builds is 7GB or so, which means that between the Linux and MacOS builds we'd exceed the 10GB GitHub cache limit. The docs don't seem to make clear if there's any way to, like, pay for more space. Remote caching may be the way to go.
@penalosa offered to dig into this.

Still working on this, but initial results with a worker proxying to R2 as a bazel remote cache are looking promising. The Linux build is down to ~1:30, and can hopefully be optimised further.


Not connecting

Describe the bug
Same as  #307 but I'm on Android 12 so idk if this may cause the problem

@CoolGuyMK if you want to enable wireless debugging on your mobile, read this


Unicode query String containing special character is truncated and query with non special character is sent for quering solr Indexing backend

I am running janusgraph with cassandra as storage backend and solr as Indexing backind .When I index unicode string with UTF-8 character sequenc it is successfully store and index but when searching the query string is truncated before it is sent to solr server from janusgraph server
my query looks like this
g.V().has('contentNepali',textContains('आइतबार'))
but query string becomes "आइतब" when the request goes from janusgraph to solr
I have seen the query string in janusgraph log

Hi @pra527vin
This is the Microsoft BotBuilder repository, for BotBuilder issues and feature requests.  Your issue appears to be related to janusgraph, cassandra and solr.  Have you tried their forums?


ROX-25292: Increase the timeout

Description

Verify process visualization on default: #depName fails due to some missing processes but those processes are later found in Verify process paths, UIDs, and GIDs on #depName and Verify process arguments on #depName. This suggest that the failures are due to timing issues. Increasing the waiting time for the processes to appear should fix the issue.
User-facing documentation

[x] CHANGELOG is updated OR update is not needed
[x] documentation PR is created and is linked above OR is not needed

Testing

[ ] inspected CI results

Automated testing

~[ ] added unit tests~
~[ ] added e2e tests~
~[ ] added regression tests~
~[ ] added compatibility tests~
[x] modified existing tests

How I validated my change

This is particularly flaky in aro. Running CI multiple times to verify the solution.

/test aro-qa-e2e-tests

/test aro-qa-e2e-tests

/test aro-qa-e2e-tests

/test aro-qa-e2e-tests

/test aro-qa-e2e-tests

/test aro-qa-e2e-tests

/test aro-qa-e2e-tests

The loop where the MAX_SLEEP_TIME is used could be replaced by a withRetry, but that's completely optional

You are right. I think I'll do that in a follow-up PR though. I'll also change it in the rest of the test in this class.


Setting to disable "$this can only be used in an object context."

From @Gauss23 on November 20, 2017 15:23
Hello,
we just wondered about a lot of "syntax"-errors in our code. The message is:

$this can only be used in an object context.

Before your update to version 2.0.0 everything was fine. The files IntelliSense is now marking as wrong are config files, which are required/included in object context, so they are perfectly working. Of course IntelliSense cannot know that. Is there a chance to make this check optional?
Best regards,
Christopher
Copied from original issue: felixfbecker/vscode-php-intellisense#201

@Gauss23 is there a way to identify these config files? E.g. do they have a top-level return?

Unfortunately no, we are using this in config files and templates. Our renderer is an object, the templates it is parsing are PHP files and are using $this for multiple reasons. PHP is not complaining and everything is working. We don´t want to make too much trouble about it,.
A simple config switch would be sufficient for us. For the majority of the people, the error will be correct.

Want to do a PR?

I just looked into the sources. I can see how to add a config var in extension.ts and how to push it as a cmd option to bin/php-language-server.php.
How do you want to push the config variable to the TreeAnalyzer? I saw it´s created by PhpDocument.php.
Using a global would not be the preferred method I think. Is there already some kind of infrastructure to pass something from the outside deeply into the the objects?
Maybe there will be more config settings in the future.
Best regards

No, it would have to implement didChangeConfiguration: https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#workspace_didChangeConfiguration

I thought I can add a line in: vscode-php-intellisense/src/extension.ts
Like this line:
    const memoryLimit = conf.get<string>('memoryLimit') || '-1';
I would add:
    const checkThisInClasses = conf.get<string>('checkThisInClasses') || '1';
And then add a command option in the same file.
Can´t see how to implement didChangeConfiguration, sorry.

That wouldn't help. The configuration can change.

At the moment you´re not handling config changes, at least I can´t find an implementation of this didChangeConfiguration in intelliSense.
I´ll try to find an example for this implementation.

Because at the moment there is no config

Memory limit and php executable are config variables in VS Code. Am I wrong? What if those settings are changed? Where is the trigger to restart the language-server?
Sorry, those are my first steps with vs code extensions.

Memory limit and PHP executable are settings for the VS Code extension that modify how the language server is spawned, it's not a setting that is used in the language server at runtime or can change. Customisation of the static analysis should happen through the protocol.

I see, I think I´ll leave that one for a more experienced vs code extension developer.
Thank you for your patience.

Hi! I'm getting the same error message now in 2.0. In my case (current project: Magento) This happens in different files. I think this message should be ignored if $this is used in the document's root scope.  I also remember in CakePHP views, $this was used a lot.
I'm trying to find when this error can be useful, and I can only think of, if $this is called within a function (not class method)

I have the same problem

Same problem

I was expecting to config rules like ESlint

@luispuig You may want to implement the protocol ;)

@felixfbecker I want to take a shot. I added a method to Server\Workspace called didChangeConfiguration. How can I manage that this is being called automatically? It looks like the AdvancedJsonRPC dispatcher is not recognizing it.

@jens1o if you're going to look into this, I just wanted to offer the suggestion to be able to configure a whitelist or blacklist for where (not) to check for the error. Then people could just add the path to their views.

First of all, I want to implement the ConfigurationWatcher, before I can resolve this issue(and others) ;)

@jens1o you don't need to do anything to make it called automatically as long as the client calls it. But you may have to declare a ServerCapability in initialize.

@MaartenStaa a lot of people are hitting this issue, I think it's reasonable to ignore the error if it is in the global context and not in a function. Would you be up for implementing that?

@felixfbecker I don't find the server capability in the protocol... I'm sorry.
Could you give me a pointer? https://github.com/jens1o/php-language-server/tree/jens1o-workspace-settings

Hi,
I have the same problem. I use kahlan a test framework https://github.com/kahlan/kahlan that uses a describe-it syntax to write tests and in these tests (which are functions) $this can be called a lot.
So throwing an error in the context of a function would not work for kahlan users.
I'm not sure what the solution should be.

Is reverting #528 off the table? It seems apparent that the rules that it checks against are too simplistic and result in many false positives. In my case, it complains that the use of $this in the Slim Framework's route group closures is invalid.
I don't like the idea of a configuration option that suppresses this error. It seems like the PHP parser could conceivably validate that $this is available at compile-time, it just doesn't at the moment. It should not be up to the developer to suppress errors for code that is safe and valid.

I'd sooner have $this error turned off completely then put up with it giving false positives like what it's doing now. Red swirleys under every single $this variable in a script full of closures is very, very distracting.


Difference between default terminal creation and picking from list

While troubleshooting https://github.com/microsoft/vscode/issues/144272, with terminal.integrated.inheritEnv set to false, creating terminals through + button (or open terminal panel when there is no active terminal instance) will always fail but picking zsh from the dropdown (^ chevron) picks up env successfully.
Picking shell from the dropdown is probably not respecting terminal.integrated.inheritEnv.

I can't reproduce the issue - maybe because my .zshrc is different from yours. Could you pls share yours?

@meganrogge here is mine
export ZSH="/Users/penlv/.oh-my-zsh"
ZSH_THEME="robbyrussell"
plugins=(git)
source $ZSH/oh-my-zsh.sh

# User configuration

# export MANPATH="/usr/local/man:$MANPATH"

# You may need to manually set your language environment
# export LANG=en_US.UTF-8

# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
#   export EDITOR='vim'
# else
#   export EDITOR='mvim'
# fi

# Compilation flags
# export ARCHFLAGS="-arch x86_64"

# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
# alias zshconfig="mate ~/.zshrc"
# alias ohmyzsh="mate ~/.oh-my-zsh"

export PATH="$(yarn global bin):$PATH"
. ~/.z_z/z.sh

export NVM_DIR=~/.nvm
source $(brew --prefix nvm)/nvm.sh
# if command -v pyenv 1>/dev/null 2>&1; then
#   eval "$(pyenv init -)"
# fi

closing as i can't repro. @rebornix please reopen this if you see it again

I also couldn't repro a difference here, I checked when launching vscode both from the dock and a terminal by diffing the shell launch config on the spawn call. Both always seemed to be identical

Could not repro either. Marking as verified pending any logs of reports of reproductions.


Upgrade to Django 1.9

Need to upgrade through 1.5, 1.6, 1.7, 1.8 and 1.9.  This closes #4

As part of this, I also want to ensure Travis-CI compatibility.


fix: compiler error when secp256k1 is disabled

Resolves an error that was happening when you disabled default features for revm:
failed to resolve: use of undeclared crate or module `secp256k1`
  --> revm_precompiles-1.1.1\src\secp256k1.rs:80:21
   |
80 |     let out = match secp256k1::ecrecover(&sig, &msg) {
   |                     ^^^^^^^^^ use of undeclared crate or module `secp256k1`

I am fine with build failure, as you would not have erecover precompile if you don't enable k256 or default secp256k1.
Probably need to add comments somewhere, Cargo.toml and readme.


Add certificate caching

Request to add certificate caching so that it does not need to request the certificate every time it validates. See line 82 here for an example of this in a python implementation.

Great thought @LucasMoskun.  Will most likely use lru-cache, as there is no need to reinvent the caching wheel.

Awesome!

Yeah, it will be.  Did some noodling in my head as I drifted off last night.  Unfortunately, this will have to change the constructor, so that the cache stays in scope.
Example:
const Validator = require('sns-payload-validator');
const validator = new Validator({ useCache: true, maxCerts: 1000 });

const validPayload = await validator.validate(payloadFromRequest);

That said, I still think this is a worthwhile feature.  Just wanted to give you the heads up that it may take through next weekend to get it implemented and all of the tests configured correctly.  Stay tuned!!

Right on. Let me know if there's anything I can do to help.

@LucasMoskun - Feel free to check out #43 and leave comments.  I will address any concerns you may bring up.  If we are all good, I will test in my own AWS Account tomorrow night and then ship it. :shipit:
I live in the Central timezone in the US, to keep track of what tomorrow night means.

Caching has been released to NPM in v2.0.0.  Thank you for your interest in this project.  Feel free to leave any feedback or open a new issue.

Caching has been released to NPM in v2.0.0. Thank you for your interest in this project. Feel free to leave any feedback or open a new issue.

Looks great, I'll let you know if anything pops up in production. Thanks for smashing this through!

@LucasMoskun - Great to hear.  Not surprised by the TypeScript thing.  I type my modules for the VsCode intellisense, I'm not a big TypeScript person.
Unfortunately, due to NPM rules, I will have to release a v2.0.1 to fix the README, but probably worth doing.  Created issue #45.  Thanks for the catch


New weapon doesn't properly replace previous weapon

Describe the bug
Although the new weapon can be switched to & from and both weapons used normally, the old weapon's mesh is displaying underneath that of the new weapon.
To Reproduce
Steps to reproduce the behavior:

Run game
Go to Rifle pickup
Equip Rifle
See error

Expected behaviour
The previous mesh should be hidden/removed upon picking up & equipping a new weapon.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Windows 7 Ultimate
Version: Unity 2021.1.24f

Additional context
Add any other context about the problem here.

Fixed - part of the old weapon's mesh was incorrectly placed within the prefab's heirarchy.


No Output to Screen for JTC Commands in the Demos

Is your feature request related to a problem? Please describe.
In the demos ros2 launch ros2_control_demo_bringup test_joint_trajectory_controller.launch.py publisher_config:=rrbot_namespace_joint_trajectory_publisher.yaml  doesn't output the commands to the screen like the other controllers do.
Describe the solution you'd like
I was planning on adding the output to the test nodes here
Describe alternatives you've considered
Could assume the user knows how to echo the topic, or explain this in the tutorials. But I think printing the command to the console matches the other examples.
Additional context
Reference conversation can be found here in ros-controls/ros2_control_demos#170

I am planning to work on this.

Close because the node is moved to another repo and should be solved in #389


Update members.csv

added member

Your request has been denied as some of this site's content is deemed too unmarked-NSFW for the Cuddler Webring.

Thank you for taking the time to both look it over and inform me why.


Remove Spaces from Tags

Social media doesn't like it - Twitter especially will not accept multi-word tags. Standardize on dashes or underscores. Ideally, build something small to validate this at build time.

Done for existing works in 5432ba474183afc4c1bc87c2c4923d08dec72ea4
Should be careful not to readd any


Fix references to syncing passwords to syncing password hashes

There are multiple places in the content where the reference to syncing a password versus the password hash exist (the first is in the very first sentence). Those all need to be fixed to not cause confusion around this feature.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f4c06608-fe23-4097-f78a-123667ac763f
Version Independent ID: e334ab72-d55e-87dd-3cdb-cd9d88d6a844
Content: Implement password hash synchronization with Azure AD Connect sync
Content Source: articles/active-directory/connect/active-directory-aadconnectsync-implement-password-hash-synchronization.md
Service: active-directory
GitHub Login: @billmath
Microsoft Alias: billmath

@Roblarson Thank you for the feed.  I am currently working on re-working the Azure AD Connect content and this is one of the items that is part of that.  You should see these issues addressed over the next couple of months.  Thanks!

#please-close

@Roblarson We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Fix npm build on MS window native

Summary | Résumé
This fixes the installation and build scripts when executed directly on the MS Windows operating system.
Test instructions | Instructions pour tester la modification

Clone the repository on a Windows workstation where only NodeJS 18/20 is installed.
run npm install:

Without the fix: The installation build stops during the test execution of `generate-value-accessors.spec.ts``.
With the fix: The installation will proceed and successfully finish as expected.

run npm run build:

Without the fix: The build fails for react-ssr, stating that local file systems imported starting with c:/ must be prefixed with the file:// URI scheme.
With the fix: The build will proceed and successfully finish as expected.

With this proposed fix, I was able to successfully use Node 20 for my setup on my MS Windows computer and also on my Armbian computer. I only tested the web package and the Storybook locally, and both worked as expected. I did not check if the Angular, ReactJS, and VueJS builds were working as expected.

TY @duboisp ! If you could provide an update with the requested changes, I think we should be good to go on merging this.

@daine FYI - I completed the requested changes. Thank you


OpenApi descriptions should contain the x-ms-kiota-info extension

To enable better self serve experiences, the descriptions should contain that extension
The idea being to push Graph consumers to use Graph Core (which will transitively bring kiota dependencies) instead of Kiota dependencies directly. The main benefits for them: easier client initialization, fewer direct dependencies to manage, Task (LFU, paging, batching)...
We could do that in the weekly pipeline process via a PowerShell script.
The main properties we want to override from kiota default:

dependencies for each language so it points to graph core instead
client class name (MicrosoftGraphClient)

Note: we might be missing a placeholder in that extension to enable/disable the backing store


StateScoped should support OnRemove observers

What problem does this solve or what need does it fill?
A StateScoped(SomeState) entity might have a component that triggers OnRemove observers. However, any side effects of those observers potentially race with cleanup systems that run in OnExit(SomeState), since StateScoped cleanup happens in StateTransitionSteps::ExitSchedules.
Additionally, one potentially common side effect of OnRemove observers is emitting events that are handled in SomeState. It's very likely that most such events should be isolated to that state, and hence should be cleaned up in scope. Adding ergonomics here would be a win.
What solution would you like?

Move StateScoped cleanup to a new system set that runs before StateTransitionSteps::ExitSchedules.
Add .add_state_scoped_event<E>(s: impl States) extension to App. This should register the event for queue cleanup in a system set that runs after StateScoped entity cleanup and before StateTransitionSteps::ExitSchedules. Or on the other side (or both), it should run immediately before StateTransitionSteps::EnterSchedules when entering the target state.

What alternative(s) have you considered?
None
Additional context

You can't easily spawn entities within an observer. This turns out to be advantageous for OnRemove handlers that run when StateScoped cleans stuff up because it means you can't easily spawn new StateScoped entities that will leak outside the state (Even though you may want to do so as part of OnRemove handling. For example I have a handler in my last jam game that runs when a CollectableDrop component is removed from any entity in order to spawn a new collectable where the dying entity was. I use an event to 'extract' the spawn command out of the observer.).
There is some nuance to state-scoped events. If you have an event that is sent OnRemove for a StateScoped(S) entity, then the event will be naturally cleaned up without leaking back into the next S if the next S is not set for the tick immediately after the tick with the state that S exited into (since events only live for 2 ticks).

Can you provide a more concrete example? What is the OnRemove observer doing and what is it racing with? Not being able to despawn entities in OnExit(..) seems problematic.

I will say "don't do it that way" for this case because despawning != dying, and you ought to clear any lingering "spawn drops" events on exit as well. This sounds to me like a logic issue similar to system ordering bugs that are difficult to track down but still the user's responsibility to fix.

It's certainly possible (and maybe even generally better) to perform cleanup in OnEnter when re-entering a state as part of state initialization, and only use OnExit to collect information that should be passed out of the state.

All cleanup should go in OnExit, so I wouldn't suggest otherwise. The primary purpose of the separate schedules is to guarantee that tearing down the previous state occurs before setting up the next state. As for whether e.g. setting a resource to its default value counts as "setup" or "teardown", that may not always be immediately obvious I guess.

My take on this is that trying to control all possible points where your entity could be despawned is not feasible. If you're setting an OnRemove observer, you need to be prepared for the consequences that it could be triggered at any point in the frame.

We didn't design states with observers in mind, they were still in development back then.
Something we can consider is moving the entire state architecture to observers, but this is quite a big change that I don't think is worth it until we unify all 3 state traits.

Another thing, we can add an OnAdd hook to StateScoped that will check the state and remove the entity if it's incorrect.

Another thing, we can add an OnAdd hook to StateScoped that will check the state and remove the entity if it's incorrect. This compliments the existing on-exit behavior by doing on-spawn validation.

This is off-topic for the issue, but I don't believe that's the correct complementary behavior. The current behavior is "despawn on exit", so its complement would be "spawn on enter", which would be genuinely useful but is blocked on the better scenes work.


Update sat.cpp

added some header files

Thanks for the PR, but I will not accept it, I'm afraid. They are not needed, as iostream is the only thing from the std lib that is directly used in this example.


install microk8s issue

Summary

Installation issue. Microk8s is not ready after installation.
sudo snap install microk8s --classic
microk8s (1.29/stable) v1.29.4 from Canonical✓ installed
log is attached:
inspection-report-20240707_181410.tar.gz

Hi @yage11,
I see the following line repeated a lot in the Kubernetes logs:
Jul 07 17:40:58 ya-1064-5511 microk8s.daemon-kubelite[102624]: E0707 17:40:58.549013  102624 kubelet.go:1542] "Failed to start ContainerManager" err="system validation failed - wrong number of fields (expected 6, got 7)"

Can you share some more details about the system, OS version, host, arch that you are running MicroK8s on?

I have same issue when install ubuntu with proxy setting. if I not set proxy while installing, I can install and set microk8s up and run normally. I don't know where is ubuntu keep proxy setting while install and how to fix it.

ps. It can install and run eventhough I use ubuntu 24.04 with microk8s 1.30


Unable to create migrations

What you are doing?

I want to create a migration, but I am encountering an error.
sequelize migration:generate --env postgres --name my-migration

What do you expect to happen?
Have a migration created.
What is actually happening?
I am getting this error:

(utils) [my-pc]/home/.../my-project$ sequelize migration:generate --env postgres --name my-migration
Sequelize CLI [Node: 8.9.4, CLI: 3.2.0-c, ORM: 4.31.2]
WARNING: This version of Sequelize CLI is not fully compatible with Sequelize v4. https://github.com/sequelize/cli#sequelize-support
migrations folder at "/home/me/projects/my-project/migrations-ts" already exists.
/home/me/projects/my-project/node_modules/sequelize-cli-typescript/node_modules/yargs/yargs.js:1079
else throw err
^
Error: ENOENT: no such file or directory, open '/home/me/projects/my-project/node_modules/sequelize-cli-typescript/lib/assets/migrations/skeleton.js'
at Object.fs.openSync (fs.js:646:18)
at Object.fs.readFileSync (fs.js:551:33)
at Object.read (/home/me/projects/my-project/node_modules/sequelize-cli-typescript/lib/helpers/asset-helper.js:19:30)
at Object.render (/home/me/projects/my-project/node_modules/sequelize-cli-typescript/lib/helpers/template-helper.js:28:42)
at Object.exports.handler (/home/me/projects/my-project/node_modules/sequelize-cli-typescript/lib/commands/migration_generate.js:30:115)
at Object.self.runCommand (/home/me/projects/my-project/node_modules/sequelize-cli-typescript/node_modules/yargs/lib/command.js:233:22)
at Object.Yargs.self._parseArgs (/home/me/projects/my-project/node_modules/sequelize-cli-typescript/node_modules/yargs/yargs.js:990:30)
at Object.get [as argv] (/home/me/projects/my-project/node_modules/sequelize-cli-typescript/node_modules/yargs/yargs.js:927:19)
at Object. (/home/me/projects/my-project/node_modules/sequelize-cli-typescript/lib/sequelize:84:16)
at Module._compile (module.js:643:30)
at Object.Module._extensions..js (module.js:654:10)
at Module.load (module.js:556:32)
at tryModuleLoad (module.js:499:12)
at Function.Module._load (module.js:491:3)
at Function.Module.runMain (module.js:684:10)
at startup (bootstrap_node.js:187:16)
at bootstrap_node.js:608:3

I see that this repo contains a file sequelize-cli-typescript/lib/assets/migrations/skeleton.ts. Perhaps that file should be renamed with a .js extension?
Dialect: postgres
Database version:  PostgreSQL 10.1 on x86_64-pc-linux-musl, compiled by gcc (Alpine 6.3.0) 6.3.0, 64-bit
Sequelize CLI version: 3.2.0-c
Sequelize version: 4.17.0

https://github.com/douglas-treadwell/sequelize-cli-typescript/pull/2 fixes the issue. Please merge this.

FYI, there is more fresh fork that is still being maintained:
https://github.com/wildcookie007/sequelize-cli-typescript
Fix for this issue is there, plus numerous of other fixes  🎉
Can be included in package.json like this:
"dependencies": {
  "sequelize-cli-typescript": "git+ssh://git@github.com:wildcookie007/sequelize-cli-typescript.git#master",
}

After, run npm i to install it.


Fix a bug where background color was not rendered properly on custom fgColor

On dark backgrounds, fgColor was not working correctly between the icon and the background of the icon itself (see example).

Before the fix, the icon looks like this:

After the fix, the icon looks like that:

Bug only happens in Chrome/Edge browsers. On Safari/Firefox it is working properly.

I opened another one because I sent an extra commit in the last PR by accident

Thank you for updating the PR, I was about to close the last one.

You’re welcome! I will work from now on to support mouse hover on icons since it is a highly requested feature from what I could see.

Released in https://github.com/jaketrent/react-social-icons/releases/tag/v5.10.0, thank you for contributing.


GitHub Actions: Add Python 3.12 release candidate to the testing

https://github.com/actions/checkout/releases
https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#allow-pre-releases

See #683.

Closing in favor of #686


Add ability to search tags

Please add the ability to filter the tag list! When a decent amount of tags are in the bot it's super unwieldy to page through the list.
Related: showing only six tags per page is slightly annoying, but I do think that having the author info right there is worthwhile

 Add searching into tags


taglib_1_9, tomahawk:Remove unmaintained packages

Tomahawk has been marked unmaintained for a few years now: https://github.com/tomahawk-player/tomahawk/commit/c8389592488c07079f40e7edb9195cfabbc58740#diff-04c6e90faac2675aa89e2176d2eec7d8 and been broken on nixpkgs since 2018.
taglib_1_9 has a few vulnerabilities and was only used by tomahawk, therefore it's now also removed.
Neither package has any maintainers.

:fire:


reference setup.sh

In the oracle-ksql-elasticsearch example, the  file that is inside docker-compose/scripts should/setup.sh be in the docker-compose/setup.sh directory because the references inside are made to that directory in addition to the context of docker-compose:
docker-compose exec connect-debezium bash -c '**/scripts/**create-ora-source-debezium-xstream.sh'
can I do a pull request?

Hi, a PR would be great, thank you!
On Fri, 23 Oct 2020 at 08:50, Andres Carmona<EMAIL_ADDRESS>wrote:

In the oracle-ksql-elasticsearch example, the setup file that is inside
docker-compose / scripts should be in the docker-compose / setup.sh
directory because the references inside are made to that directory in
addition to the context of docker-compose:
docker-compose exec connect-debezium bash -c
'**/scripts/**create-ora-source-debezium-xstream.sh'
can I do a pull request?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/confluentinc/demo-scene/issues/161, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA4AMHRT2AGN732VA7S2JW3SMEYVLANCNFSM4S4GUPXQ
.


We need a proper, shareable URL for a given occurrence

Important remark from @timadriaens (people will share by email)

Yes, I agree.

Update: due to a higher risk of GBIFIDs changing (for example if the if the institution code gets updated), we're actually leaning to another solution: generation our own stable occurrence identifier by using a hash function that takes the datasetKey and occurrenceNumber as an input.
This is a different issue in itself, but one of the consequences is that it's that identifier that would be part if the URL for the occurence page. The GbifId field is important and should still be shown to the user, but not considered as a stable identifier.

We now have this thanks to the stable_id field and associated mechanism.


how to resolve shasum check failed while installing @angular/material

Bug, feature request, or proposal:
Bug
What is the expected behavior?
What is the current behavior?
What are the steps to reproduce?
Providing a Plunker (or similar) is the best way to get the team to see your issue.
Plunker template: https://goo.gl/DlHd6U
What is the use-case or motivation for changing an existing behavior?
Which versions of Angular, Material, OS, browsers are affected?
Windows 10
Is there anything else we should know?

Closing this issue as it does not follow the issue template. Please file a new issue that follows the template, which gives the team the information needed to investigate.


Add type annotations and type checking to AgentOS code

Python 3 supports annotating code with type information (see PEP-3107 added in Python 3.0 and PEP-484 added in Python 3.5).
I propose we start using annotations in our code and running a third party type checking library (in the same workflow that we run lint).
Options if we want to do type checking that uses the annotations:

Google's Pytype - used by DeepMind's Acme RL Framework
Microsoft's Pyright
Facebook's Pyre
Mypy - the OG

This article summarizes each of these 4 options, but doesn't weigh-in on which to use.
Also note that Pycharm has built-in typechecking.

Update: We have been using Mypy some.


Rx.range(0, NaN) results in infinite Observable instead of throwing an error

Bug Report
Current Behavior
Calling Rx.range with anything other than a number as second the parameter results in an Observable that emits values indefinitely.
Reproduction
Rx.range(0, NaN).subscribe(console.log);

Expected behavior
Doing so should result in an error. An infinite Observable makes sense if the second parameter is null or undefined but defined, nonnumerical values should be invalid inputs and throw errors.
Environment

RxJS version: 6.3.3

We do not validate input values for runtime, instead use type definition as api contract should be supplied into. But even in that case, this doesn't guarded since params are not strict to allow some unguarded values may have chances to improve it.

But NaN is valid type of number, not sure how to subtract those types.. 🤔


feat: Add service catalog client and necessary calls

This is first commit that starts building out functionality to import service catalog Cfn constructs for provisioned products.
This does not add any reachable code paths to tooling, we simply add our client/facade methods and test suite.
This adds the necessary client and wrapped API calls to generate all the needed information to codegen our imports.
We intend to support at least 2 paths at release that enable a user to either generate all available products,
or if they supply input parameters to generate a singular provisionable product - hence the optional interface that
will pull info from command line args or if optional we will determine at runtime.
Co-authored-by: Dillon Ponzo<EMAIL_ADDRESS>

@dponzo