Quadmesh + datashader + dask raises numba error

ALL software version info
I am using the latest released versions of all the relevant libraries.
Hvplot 0.5.2
Datshader 0.10.0
Numba 0.48.0
Description of expected behavior and the observed behavior
Complete, minimal, self-contained example code that reproduces the issue
I am trying to plot an xarray dataset backed by dask using hvplot. Here is how I load the data. (Should be reproducible with intake installed.)
from intake import open_catalog
cat = open_catalog("https://raw.githubusercontent.com/pangeo-data/pangeo-datastore/master/intake-catalogs/ocean.yaml")
ds  = cat["sea_surface_height"].to_dask()

The following code works fine:
ds.sla.hvplot.image('longitude', 'latitude', rasterize=True, dynamic=True)

However, if I try to do quadmesh instead
ds.sla.hvplot.quadmesh('longitude', 'latitude', rasterize=True, dynamic=True)

I get a big long error.
Stack traceback and/or browser JavaScript console output
WARNING:param.dynamic_operation: Callable raised "TypingError('Failed in nopython mode pipeline (step: nopython frontend)\nnon-precise type pyobject\n[1] During: typing of argument at <extend_cpu> (4)\n\nFile "<extend_cpu>", line 4:\n<source missing, REPL/exec in use?>\n\nThis error may have been caused by the following argument(s):\n- argument 4: cannot determine Numba type of <class \'dask.array.core.Array\'>\n')".
Invoked as dynamic_operation(numpy.datetime64('1993-01-01T00:00:00.000000000'), height=300, scale=1.0, width=700, x_range=None, y_range=None)
WARNING:param.dynamic_operation: Callable raised "TypingError('Failed in nopython mode pipeline (step: nopython frontend)\nnon-precise type pyobject\n[1] During: typing of argument at <extend_cpu> (4)\n\nFile "<extend_cpu>", line 4:\n<source missing, REPL/exec in use?>\n\nThis error may have been caused by the following argument(s):\n- argument 4: cannot determine Numba type of <class \'dask.array.core.Array\'>\n')".
Invoked as dynamic_operation(numpy.datetime64('1993-01-01T00:00:00.000000000'), height=300, scale=1.0, width=700, x_range=None, y_range=None)
---------------------------------------------------------------------------
TypingError                               Traceback (most recent call last)
/srv/conda/envs/pangeo/lib/python3.7/site-packages/IPython/core/formatters.py in __call__(self, obj, include, exclude)
    968
    969             if method is not None:
--> 970                 return method(include=include, exclude=exclude)
    971             return None
    972         else:

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/dimension.py in _repr_mimebundle_(self, include, exclude)
   1302         combined and returned.
   1303         """
-> 1304         return Store.render(self)
   1305
   1306

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/options.py in render(cls, obj)
   1393         data, metadata = {}, {}
   1394         for hook in hooks:
-> 1395             ret = hook(obj)
   1396             if ret is None:
   1397                 continue

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/ipython/display_hooks.py in pprint_display(obj)
    280     if not ip.display_formatter.formatters['text/plain'].pprint:
    281         return None
--> 282     return display(obj, raw_output=True)
    283
    284

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/ipython/display_hooks.py in display(obj, raw_output, **kwargs)
    256     elif isinstance(obj, (HoloMap, DynamicMap)):
    257         with option_state(obj):
--> 258             output = map_display(obj)
    259     elif isinstance(obj, Plot):
    260         output = render(obj)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/ipython/display_hooks.py in wrapped(element)
    144         try:
    145             max_frames = OutputSettings.options['max_frames']
--> 146             mimebundle = fn(element, max_frames=max_frames)
    147             if mimebundle is None:
    148                 return {}, {}

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/ipython/display_hooks.py in map_display(vmap, max_frames)
    204         return None
    205
--> 206     return render(vmap)
    207
    208

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/ipython/display_hooks.py in render(obj, **kwargs)
     66         renderer = renderer.instance(fig='png')
     67
---> 68     return renderer.components(obj, **kwargs)
     69
     70

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/plotting/renderer.py in components(self, obj, fmt, comm, **kwargs)
    363             doc = Document()
    364             with config.set(embed=embed):
--> 365                 model = plot.layout._render_model(doc, comm)
    366             return render_model(model, comm) if embed else render_mimebundle(model, doc, comm)
    367         else:

/srv/conda/envs/pangeo/lib/python3.7/site-packages/panel/viewable.py in _render_model(self, doc, comm)
    363         if comm is None:
    364             comm = state._comm_manager.get_server_comm()
--> 365         model = self.get_root(doc, comm)
    366
    367         if config.embed:

/srv/conda/envs/pangeo/lib/python3.7/site-packages/panel/viewable.py in get_root(self, doc, comm)
    534         """
    535         doc = doc or _curdoc()
--> 536         root = self._get_model(doc, comm=comm)
    537         self._preprocess(root)
    538         ref = root.ref['id']

/srv/conda/envs/pangeo/lib/python3.7/site-packages/panel/layout.py in _get_model(self, doc, root, parent, comm)
    134         if root is None:
    135             root = model
--> 136         objects = self._get_objects(model, [], doc, root, comm)
    137         props = dict(self._init_properties(), objects=objects)
    138         model.update(**self._process_param_change(props))

/srv/conda/envs/pangeo/lib/python3.7/site-packages/panel/layout.py in _get_objects(self, model, old_objects, doc, root, comm)
    124             else:
    125                 try:
--> 126                     child = pane._get_model(doc, root, model, comm)
    127                 except RerenderError:
    128                     return self._get_objects(model, current_objects[:i], doc, root, comm)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/panel/pane/holoviews.py in _get_model(self, doc, root, parent, comm)
    225             plot = self.object
    226         else:
--> 227             plot = self._render(doc, comm, root)
    228
    229         plot.pane = self

/srv/conda/envs/pangeo/lib/python3.7/site-packages/panel/pane/holoviews.py in _render(self, doc, comm, root)
    284             kwargs = {}
    285
--> 286         return renderer.get_plot(self.object, **kwargs)
    287
    288     def _cleanup(self, root):

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/plotting/bokeh/renderer.py in get_plot(self_or_cls, obj, doc, renderer, **kwargs)
     71         combining the bokeh model with another plot.
     72         """
---> 73         plot = super(BokehRenderer, self_or_cls).get_plot(obj, doc, renderer, **kwargs)
     74         if plot.document is None:
     75             plot.document = Document() if self_or_cls.notebook_context else curdoc()

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/plotting/renderer.py in get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)
    200
    201         # Initialize DynamicMaps with first data item
--> 202         initialize_dynamic(obj)
    203
    204         if not renderer:

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/plotting/util.py in initialize_dynamic(obj)
    249             continue
    250         if not len(dmap):
--> 251             dmap[dmap._initial_key()]
    252
    253

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/spaces.py in __getitem__(self, key)
   1278         # Not a cross product and nothing cached so compute element.
   1279         if cache is not None: return cache
-> 1280         val = self._execute_callback(*tuple_key)
   1281         if data_slice:
   1282             val = self._dataslice(val, data_slice)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/spaces.py in _execute_callback(self, *args)
   1052
   1053         with dynamicmap_memoization(self.callback, self.streams):
-> 1054             retval = self.callback(*args, **kwargs)
   1055         return self._style(retval)
   1056

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/spaces.py in __call__(self, *args, **kwargs)
    691
    692         try:
--> 693             ret = self.callable(*args, **kwargs)
    694         except KeyError:
    695             # KeyError is caught separately because it is used to signal

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/util/__init__.py in dynamic_operation(*key, **kwargs)
    983
    984         def dynamic_operation(*key, **kwargs):
--> 985             key, obj = resolve(key, kwargs)
    986             return apply(obj, *key, **kwargs)
    987

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/util/__init__.py in resolve(key, kwargs)
    976             elif isinstance(map_obj, DynamicMap) and map_obj._posarg_keys and not key:
    977                 key = tuple(kwargs[k] for k in map_obj._posarg_keys)
--> 978             return key, map_obj[key]
    979
    980         def apply(element, *key, **kwargs):

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/spaces.py in __getitem__(self, key)
   1278         # Not a cross product and nothing cached so compute element.
   1279         if cache is not None: return cache
-> 1280         val = self._execute_callback(*tuple_key)
   1281         if data_slice:
   1282             val = self._dataslice(val, data_slice)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/spaces.py in _execute_callback(self, *args)
   1052
   1053         with dynamicmap_memoization(self.callback, self.streams):
-> 1054             retval = self.callback(*args, **kwargs)
   1055         return self._style(retval)
   1056

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/spaces.py in __call__(self, *args, **kwargs)
    691
    692         try:
--> 693             ret = self.callable(*args, **kwargs)
    694         except KeyError:
    695             # KeyError is caught separately because it is used to signal

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/util/__init__.py in dynamic_operation(*key, **kwargs)
    984         def dynamic_operation(*key, **kwargs):
    985             key, obj = resolve(key, kwargs)
--> 986             return apply(obj, *key, **kwargs)
    987
    988         operation = self.p.operation

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/util/__init__.py in apply(element, *key, **kwargs)
    980         def apply(element, *key, **kwargs):
    981             kwargs = dict(util.resolve_dependent_kwargs(self.p.kwargs), **kwargs)
--> 982             return self._process(element, key, kwargs)
    983
    984         def dynamic_operation(*key, **kwargs):

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/util/__init__.py in _process(self, element, key, kwargs)
    962         elif isinstance(self.p.operation, Operation):
    963             kwargs = {k: v for k, v in kwargs.items() if k in self.p.operation.param}
--> 964             return self.p.operation.process_element(element, key, **kwargs)
    965         else:
    966             return self.p.operation(element, **kwargs)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/operation.py in process_element(self, element, key, **params)
    170             self.p = param.ParamOverrides(self, params,
    171                                           allow_extra_keywords=self._allow_extra_keywords)
--> 172         return self._apply(element, key)
    173
    174

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/operation.py in _apply(self, element, key)
    130         element_pipeline = getattr(element, '_pipeline', None)
    131
--> 132         ret = self._process(element, key)
    133         for hook in self._postprocess_hooks:
    134             ret = hook(self, ret, **kwargs)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
   1413                                        if k in transform.param})
   1414             op._precomputed = self._precomputed
-> 1415             element = element.map(op, predicate)
   1416             self._precomputed = op._precomputed
   1417

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/data/__init__.py in pipelined_fn(*args, **kwargs)
    212
    213             try:
--> 214                 result = method_fn(*args, **kwargs)
    215
    216                 op = method_op.instance(

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/data/__init__.py in map(self, *args, **kwargs)
   1167
   1168     def map(self, *args, **kwargs):
-> 1169         return super(Dataset, self).map(*args, **kwargs)
   1170     map.__doc__ = LabelledData.map.__doc__
   1171

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/dimension.py in map(self, map_fn, specs, clone)
    703             return deep_mapped
    704         else:
--> 705             return map_fn(self) if applies else self
    706
    707

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/operation.py in __call__(self, element, **kwargs)
    190             elif ((self._per_element and isinstance(element, Element)) or
    191                   (not self._per_element and isinstance(element, ViewableElement))):
--> 192                 return self._apply(element)
    193         elif 'streams' not in kwargs:
    194             kwargs['streams'] = self.p.streams

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/core/operation.py in _apply(self, element, key)
    130         element_pipeline = getattr(element, '_pipeline', None)
    131
--> 132         ret = self._process(element, key)
    133         for hook in self._postprocess_hooks:
    134             ret = hook(self, ret, **kwargs)

/srv/conda/envs/pangeo/lib/python3.7/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
   1081
   1082         vdim = getattr(agg_fn, 'column', element.vdims[0].name)
-> 1083         agg = cvs.quadmesh(data[vdim], x.name, y.name, agg_fn)
   1084         xdim, ydim = list(agg.dims)[:2][::-1]
   1085         if xtype == "datetime":

/srv/conda/envs/pangeo/lib/python3.7/site-packages/datashader/core.py in quadmesh(self, source, x, y, agg)
    770                 dims=list(xarr.dims)))
    771
--> 772         return bypixel(source, self, glyph, agg)
    773
    774     # TODO re 'untested', below: Consider replacing with e.g. a 3x3

/srv/conda/envs/pangeo/lib/python3.7/site-packages/datashader/core.py in bypixel(source, canvas, glyph, agg)
   1159     with np.warnings.catch_warnings():
   1160         np.warnings.filterwarnings('ignore', r'All-NaN (slice|axis) encountered')
-> 1161         return bypixel.pipeline(source, schema, canvas, glyph, agg)
   1162
   1163

/srv/conda/envs/pangeo/lib/python3.7/site-packages/datashader/utils.py in __call__(self, head, *rest, **kwargs)
     91         typ = type(head)
     92         if typ in lk:
---> 93             return lk[typ](head, *rest, **kwargs)
     94         for cls in getmro(typ)[1:]:
     95             if cls in lk:

/srv/conda/envs/pangeo/lib/python3.7/site-packages/datashader/data_libraries/xarray.py in xarray_pipeline(df, schema, canvas, glyph, summary)
     17 def xarray_pipeline(df, schema, canvas, glyph, summary):
     18     cuda = cupy and isinstance(df[glyph.name].data, cupy.ndarray)
---> 19     return glyph_dispatch(glyph, df, schema, canvas, summary, cuda)
     20
     21

/srv/conda/envs/pangeo/lib/python3.7/site-packages/datashader/utils.py in __call__(self, head, *rest, **kwargs)
     94         for cls in getmro(typ)[1:]:
     95             if cls in lk:
---> 96                 return lk[cls](head, *rest, **kwargs)
     97         raise TypeError("No dispatch for {0} type".format(typ))
     98

/srv/conda/envs/pangeo/lib/python3.7/site-packages/datashader/data_libraries/pandas.py in default(glyph, source, schema, canvas, summary, cuda)
     43
     44     bases = create((height, width))
---> 45     extend(bases, source, x_st + y_st, x_range + y_range)
     46
     47     return finalize(bases,

/srv/conda/envs/pangeo/lib/python3.7/site-packages/datashader/glyphs/quadmesh.py in extend(aggs, xr_ds, vt, bounds)
    186                 do_extend = extend_cpu
    187
--> 188             do_extend(xs, ys, *aggs_and_cols)
    189
    190         return extend

/srv/conda/envs/pangeo/lib/python3.7/site-packages/numba/dispatcher.py in _compile_for_args(self, *args, **kws)
    399                 e.patch_message(msg)
    400
--> 401             error_rewrite(e, 'typing')
    402         except errors.UnsupportedError as e:
    403             # Something unsupported is present in the user code, add help info

/srv/conda/envs/pangeo/lib/python3.7/site-packages/numba/dispatcher.py in error_rewrite(e, issue_type)
    342                 raise e
    343             else:
--> 344                 reraise(type(e), e, None)
    345
    346         argtypes = []

/srv/conda/envs/pangeo/lib/python3.7/site-packages/numba/six.py in reraise(tp, value, tb)
    666             value = tp()
    667         if value.__traceback__ is not tb:
--> 668             raise value.with_traceback(tb)
    669         raise value
    670

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type pyobject
[1] During: typing of argument at <extend_cpu> (4)

File "<extend_cpu>", line 4:
<source missing, REPL/exec in use?>

This error may have been caused by the following argument(s):
- argument 4: cannot determine Numba type of <class 'dask.array.core.Array'>

I recognize that the best place for this issue might be in another repo, but I thought I would try here first.

Here another MWE without intake/google cloud storage.
import xarray as xr
import hvplot.xarray
ds = xr.tutorial.open_dataset('rasm', chunks={'x': 25, 'y': 25, 'time': -1})
ds = ds.assign_coords(x=ds.x, y=ds.y)

import holoviews as hv
hv.extension('bokeh')

I think the error above boils down to:
# fails:
ds.Tair.isel(time=0).hvplot.quadmesh('x', 'y', rasterize=True)

# works
ds.Tair.isel(time=0).hvplot.image('x', 'y', rasterize=True)

As a workaround, you could shade a Points element (also allows irregular spacing). I know it's not the same, but maybe it helps you in the short term?
# works:
import datashader as dsh
from holoviews.operation.datashader import datashade
ds.hvplot.points('x', 'y', c='Tair', rasterize=True, x_sampling=1, y_sampling=1, aggregator=dsh.mean('Tair'))

# equivalent:
l = hv.DynamicMap(
    lambda t: hv.Points(ds.sel(time=t), ['x', 'y'], 'Tair'),
    kdims=hv.Dimension('time', values=ds.time.values),
)
datashade(l, x_sampling=1, y_sampling=1, aggregator=dsh.mean('Tair'))

I believe this should be fixed in the next release but @jonmmease or @jbednar can correct me.

So I was wrong, it seems we don't yet have dask quadmesh rendering support in datashader, for now I'll have to insert a compute sadly.

Thanks for the update @philippjfr. My understanding was that quadmesh just underwent a big refactor to improve performance. Datashader quadmesh + dask is very important to the Pangeo project. @jbednar - I'm happy to see our NSF grant support such work.

Oh it turns out it was done by @jonmmease here, it was just never merged! We'll get that merged this weekend and hopefully get a release out by Monday.

I'm happy to see our NSF grant support such work.

Indeed, your NSF grant did support this work (datashader PR #855). :-)  We just need to track down the last few issues and release it. Sorry for the confusion!

Didn't think to look in the PRs, since I assumed such a great change would have been merged asap but I did have a vague recollection of Jon working on it so I was surprised. Anyway, the to-do list for a datashader release is pretty short and I'll do my best to tick off items tonight and tomorrow.

@rabernat I'm trying to test this but am running into this error:
ValueError: Bad Request: https://www.googleapis.com/download/storage/v1/b/pangeo-cmems-duacs/o/.zmetadata?alt=media
b'User project specified in the request is invalid.'

The data are requester pays. So you'll need a google cloud account to pay the egress fees. @martindurant can probably advise. (xref  https://github.com/pangeo-data/pangeo/issues/774)

Okay, I've tested with this example instead, which should be sufficient:
from intake import open_catalog
ds = xr.tutorial.load_dataset('rasm').Tair

ds.xc.values = da.from_array(ds.xc.values)
ds.yc.values = da.from_array(ds.yc.values)
ds.values = da.from_array(ds.values)

ds.hvplot.quadmesh('xc', 'yc', rasterize=True, dynamic=True)

Should be fixed now datashader 0.11 is released.

I am trying to use quadmesh + datashader + dask which is requiring datashader to be at version 0.11 or higher because of the above problem.
But, I cannot have datashader be v0.11 because it requires that numba cannot be 0.49.* or 0.50.*
And I am using xgcm.grid.transform which requires that numba be version 0.49 or above.
Any suggestions for how to proceed from either the datashader or xgcm (@rabernat @jbusecke) side? Thanks.

Numba 0.51 should be available now. Could you try that?

@philippjfr Yes! Thank you a newer version is available and it has fixed the conundrum. I had checked the numba release notes to see the newest version and they apparently aren't quite updated so I thought I was stuck at 0.50.*. Thanks again!


Added Microsoft OpenJDK 11 Preview version <IP_ADDRESS>.

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

On a more general comment, I believe all input data should be "quoted". Because... YAML...

The schema validator hasn't been requiring quotes, @denelon do you have an opinion? I can fix it real quick if I need to.

I mean... I'm fine with strings not being quoted. But then.. it's YAML! :D
LGTM!

If it ain't broke, don't fix it is my general opinion. I've gone back and forth between all strings quoted. If the client works with the manifest as is, I'm fine. We may change guidance in the future, but I don't want to block on them :)

@jedieaston FYI: @brunoborges is our resident MSFT Java expert :)

@jedieaston
https://twitter.com/richturn_ms/status/1379557808136318978


Add postgres instance to Azure for testing

Create postgres db in azure and add data for testing.

I can take this one...is there a specific Azure subscription that we need to be using for this?

@jgarverick - I will send you the Azure details for this via email

Details received and PG instance has been created.  All applicable secrets have been added to the Key Vault in the resource group.


Animation doesn't work for point changes from/to zero

Version
4.9.0
Reproduction link
https://echarts.apache.org/examples/en/editor.html?c=area-basic
Steps to reproduce
Updating series from/to zero doesn't have animation.
What is expected?
Animation
What is actually happening?
No animation

Change from/to 0

Change from/to 1

Seems it is not supposed to happen. Will fix it


Not authorized error when fetching app usage events with global auditor role

Issue
We receive a not authorized error when fetching the app usage events of a cloud foundry foundation using an user account with the global auditor role assigned.
Basically this is the same issue as described and fixed with https://github.com/cloudfoundry/cloud_controller_ng/issues/1403 but on a different endpoint.
Context
We did create a user with the global auditor role. We chose the role as described in the docs
The user has the correct scope when checking with uaac client.
Steps to Reproduce

Create user with role global auditor
login via cf cli
run cf curl /v2/app_usage_events

Expected result
We expect to get a list of app usage events as described in the docs.
Current result
We receive a not authorized error.
Possible Fix
We found that the request cycle ends up with an authorization check in https://github.com/cloudfoundry/cloud_controller_ng/blob/master/app/access/app_usage_event_access.rb#L68-L70
From our point of view the implementation is missing the global auditor part. As such the method should look like
def index?(*_)
  admin_user? || admin_read_only_user? || global_auditor?
end

Unfortunately the link to pivotal tracker is not working for me (i get a 404). Is there any update on this?

Hey @mlin1001 & @mhergerdt ,
The v3 implementation of app usage events should correctly allow global auditors to retrieve app usage events. Does this meet your needs?
Thanks,
Greg

I'm going to close this issue, since it is fixed on the v3 implementation of usage events. Feel free to re-open if you continue to have issues.

Hey @mlin1001 & @mhergerdt ,
The v3 implementation of app usage events should correctly allow global auditors to retrieve app usage events. Does this meet your needs?
Thanks,
Greg

Hi @Gerg,
thanks for pointing that out - I will give it a try.


Output segmentation should be overlaid with opacity=100 in display_syntax_viewer()

Description
At the end of the process, the display_syntax_viewer() currently sets the opacity to 70%, which makes the segmentation appear a bit dark (esp. on dark images). Since the best way to assess segmentation accuracy is to switch on/off the overlaid seg, we should simply make it with 100% opacity. They will appear clearer at the first place.

When I assess the quality of a spinal cord segmentation, I usually set the opacity to 40%. It is easier than switching on/off.

@jcohenadad : I totally agree that 70% is too high.
I used to set my threshold at 20%. And.. I think I would prefer 20% compared to 0% (of transparency, ie 100% opacity). Indeed, 20% allows to clearly see the borders of the target structure (at least for the SC, not sure if that s also the case for GM) without having to switch on/off the overlaid seg.
Also, let's say I run sct_deepseg_sc, the terminal kindly asks me to visualize the results of the segmentation, fslview window pops but... no segmentation appears at first glance --> I am wondering if this could be disturbing.

@charleygros i meant 100% opacity, as defined by fsleyes --> the segmentation appears very clear (it is 100% opaque).
@benjamindeleener i see your point. Have you tried the "CMD+F" on the new fsleyes? it changed my life (you can even call it while z-scrolling). That's where i felt that the transparency was not required anymore.

The problem with opacity is that you won't see if there is a false negative or a false positive far away from your region of interest.

all, i think there is some misunderstanding as to what we call opacity (otherwise @perone's comment doesn't make sense, because a false positive far away from the ROI is only seen with high opacity, not low opacity). What i call 100% opacity is when the object is completely opaque. Below example so we are all on the same page:

so, from this example i understand @benjamindeleener's comment about the need to lower the opacity, because the 70% (current default) is too opaque. My approach was to set 100% by default (because it is clearer) and have the user switch on/off the overlay.
now the debate can continue :-)

Sorry my comment was misleading I agree with the opacity=100%, it will make it easier to see if there are false positives or false negatives in the predictions.


Adding COMPATIBLE_SPEC support for BUP payloads

This is already present in the JetPack 5/R35.1.0-based branches, but was more involved for R32.7.x because:

More platforms are involved
tegra-boot-tools also needed updating

When using the stock NVIDIA bootloader update tools, the setup-nv-boot-control service will add a COMPATIBLE_SPEC line to the configuration file so that BUP payload entries either matching the actual TNSPEC or the compatible spec will be considered valid for an update.  The new version of the tegra-bootloader-update tool implements similar logic for its compatibility checks.
With the above changes, the TEGRA_BUP_SPECS settings for all machines needed revising.  In most cases, fewer specs are required, making BUP packages smaller and taking less time to build/sign.
For reviewing, I'd appreciate it if someone could cross check the setup-nv-boot-control.sh changes here (and the translation of the logic in that script to the C code here in tegra-boot-tools) to derive the compatible spec string against NVIDIA's implementation in their nv-l4t-bootloader-config.sh script.  To get a copy of the NVIDIA script, extract it from the nvidia-l4t-init package, which you can download here, with:
   $ dpkg-deb --fsys-tarfile nvidia-l4t-init_32.7.2-20220417024839_arm64.deb | tar -x -f- --strip-components=4 ./opt/nvidia/l4t-bootloader-config/nv-l4t-bootloader-config.sh

The relevant function in that script is called generate_compatible_spec_info.

Thanks @madisongh.  I needed to use  nvidia-l4t-init_32.7.2-20220420143418_arm64.deb with the dpkg-deb line above, so the full dpkg-deb command was:
dpkg-deb --fsys-tarfile nvidia-l4t-init_32.7.2-20220420143418_arm64.deb |  tar -x -f- --strip-components=4 ./opt/nvidia/l4t-bootloader-config/nv-l4t-bootloader-config.sh

I see the logic in update_jetson_xxx functions within  nv-l4t-bootloader-config.sh which you changed from bash string checks to sh and grep.  The logic looks correct to me.
I'll try testing this branch with tegrademo-mender next.

Thanks @dwalkes.  Looks like I pasted in the link to the t210 version of that deb package, rather than the t186/t194 one.  The script is the same in either one, though. I'll fix the link.

I'll try testing this branch with tegrademo-mender next.

Verified with jetson-agx-xavier-devkit.
LGTM, thanks @madisongh !


initialChainSelection takes too long

Using the db-validator tool, I ran two experiments on part of the chain post hard-fork. In both cases the Imm db tip was
At (Block {blockPointSlot = SlotNo {unSlotNo = 4493500}, blockPointHash = 1236d7a62ed84e0c5bedcc8a47545f3f24026f7a144a51c12f60aa75490ee6b5})
and the ledger snapshot was very close.
In the first the vol db is empty, while in the second there are 3 files (size in bytes):
6803992  blocks-4492.dat
3963152  blocks-4493.dat
726766   blocks-4494.dat
The output with time in the first case was:
DB tip: At (Block {blockPointSlot = SlotNo {unSlotNo = 4493500}, blockPointHash = 1236d7a62ed84e0c5bedcc8a47545f3f24026f7a144a51c12f60aa75490ee6b5})

96.14user 2.76system 1:39.34elapsed 99%CPU (0avgtext+0avgdata 920356maxresident)k
9380336inputs+161576outputs (0major+461170minor)pagefaults 0swaps

while in the second
DB tip: At (Block {blockPointSlot = SlotNo {unSlotNo = 4536700}, blockPointHash = 1edc563f6d0d0b11174730e16050b8d82a6816813a70a3677d6cd2eec276a99f})

321.71user 3.83system 5:27.20elapsed 99%CPU (0avgtext+0avgdata 1420308maxresident)k
9608608inputs+96outputs (0major+588266minor)pagefaults 0swaps

Note 5:27.20 vs  1:39.34 with the majority of additional time spent on initialChainSelection.

Running the same from cardano-node I see the following logs:
[kostas-u:cardano.node.ChainDB:Info:5] [2020-08-03 12:40:33.92 UTC] Opened lgr db
[kostas-u:cardano.node.ChainDB:Info:5] [2020-08-03 12:42:38.73 UTC] Opened db with immutable tip at (Point 4493500, "1236d7a62ed84e0c5bedcc8a47545f3f24026f7a144a51c12f60aa75490ee6b5") and tip (Point 4536700, "1edc563f6d0d0b11174730e16050b8d82a6816813a70a3677d6cd2eec276a99f")

Note ~2 mins gap after the Opened lgr db log. (the time seems longer on the db-validator but maybe this is because of different ghc/rts flags).
Just before the hard fork the time is much smaller:
[kostas-u:cardano.node.ChainDB:Info:5] [2020-08-03 13:38:32.00 UTC] Opened lgr db
[kostas-u:cardano.node.ChainDB:Info:5] [2020-08-03 13:38:32.97 UTC] Opened db with immutable tip at (Point 4488438, "fe723a2e159f55fbc7b41e1892659ccf6775ee9ac42185749edf8b8023ce6cbd") and tip (Point 4490598, "5f6be24a2f9d0fccb240ac717748eae61f4ba72d862af1d0b6dcb742f7507771")

So we have
1s     for 2160 slots before the fork
125s for 43200 after
Big differences are of course expected but maybe there is room for optimizations.

This is probably the result of applyBlock being slow. It would be interesting to see what are the improvements after optimizations there.

Indeed, Edsko and I noticed this too.
While our profiling results indicated that a lot of time was spent in fn_3 (indiv), which doesn't make much sense given its definition, we also believe it is due to ledger validation.
After adding some INLINEs to fn_3 and other functions, we got:
COST CENTRE                      MODULE                                               SRC                                                                   %time %alloc

tickThenApply                    Ouroboros.Consensus.Ledger.Abstract                  src/Ouroboros/Consensus/Ledger/Abstract.hs:(88,1)-(90,60)              70.6   68.7
compute                          Control.Iterate.SetAlgebraInternal                   src/Control/Iterate/SetAlgebraInternal.hs:(976,1)-(1076,33)            14.2   17.3
liftSTS                          Control.State.Transition.Extended                    src/Control/State/Transition/Extended.hs:259:1-30                       6.0    4.0
provideBoth                      Ouroboros.Consensus.HardFork.Combinator.Util.InPairs src/Ouroboros/Consensus/HardFork/Combinator/Util/InPairs.hs:102:7-17    1.8    0.5
parseCanonicalJSON               Text.JSON.Canonical.Parse                            Text/JSON/Canonical/Parse.hs:(118,1)-(119,37)                           0.3    1.0
fromCompactTxId                  Cardano.Chain.UTxO.Compact                           src/Cardano/Chain/UTxO/Compact.hs:(175,1)-(176,62)                      0.2    2.0
fromCompactRedeemVerificationKey Cardano.Crypto.Signing.Redeem.Compact                src/Cardano/Crypto/Signing/Redeem/Compact.hs:(77,1)-(82,46)             0.1    1.4

There's not much consensus can do about it, it's a concern for the ledger, and they're already more than aware of this. Closing.


Cannot install lwc/rest/jest

Trying to install via cli setup:

sfdx force:lightning:lwc:test:setup --loglevel debug
ERROR running force:lightning:lwc:test:setup:  npm command not found. Verify npm is properly installed and try again.

Running npm shows:
<EMAIL_ADDRESS>C:\Program Files\nodejs\node_modules\npm

Could you please provide more info about the environment you're running the command in? From your post it looks like you're on Windows. Is that being run in the standard Windows Command Prompt?

Running on Windows 10 Enterprise. VisualStudioCode also installed and CLI working without problems. I'm behind a restrictive corporate firewall, but that hat nothing to do with the problem at hand.
In setup.ts the following seems to be failing:
 const npmVersionRet = spawnSync('npm', ['-v']);
    if (npmVersionRet.error || npmVersionRet.status !== 0) {
      throw new SfdxError(messages.getMessage('errorNpmNotFound'));
    }

Note that

npm -v
6.9.0

It looks like npm -v fails when ran via spawnSync on your machine. This is strange since you also mention that running npm -v command directly in the console is successful.
Could you please patch the code locally, to log the value of npmVersionRet, so would help us investigate. I am wondering what is reported in stdout, stderr, error and status.

Unfortunately I'm not coding, so I can't patch it locally. I tried a simple script:
var runSync = require('child_process').spawnSync; const npmVersionRet = runSync('npm', ['-v']);
which returns:
{ error:    { Error: spawnSync npm ENOENT        at Object.spawnSync (internal/child_process.js:1002:20)        at spawnSync (child_process.js:614:24)        at Object.<anonymous> (C:\Users\***\sf-org\testsync.js:2:23)        at Module._compile (internal/modules/cjs/loader.js:778:30)        at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)        at Module.load (internal/modules/cjs/loader.js:653:32)        at tryModuleLoad (internal/modules/cjs/loader.js:593:12)        at Function.Module._load (internal/modules/cjs/loader.js:585:3)        at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)        at startup (internal/bootstrap/node.js:283:19)      errno: 'ENOENT',      code: 'ENOENT',      syscall: 'spawnSync npm',      path: 'npm',      spawnargs: [ '-v' ] },   status: null,   signal: null,   output: null,   pid: 0,   stdout: null,   stderr: null } { Error: spawnSync npm ENOENT     at Object.spawnSync (internal/child_process.js:1002:20)     at spawnSync (child_process.js:614:24)     at Object.<anonymous> (C:\Users\****\***\sf-org\testsync.js:2:23)     at Module._compile (internal/modules/cjs/loader.js:778:30)     at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)     at Module.load (internal/modules/cjs/loader.js:653:32)     at tryModuleLoad (internal/modules/cjs/loader.js:593:12)     at Function.Module._load (internal/modules/cjs/loader.js:585:3)     at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)     at startup (internal/bootstrap/node.js:283:19)   errno: 'ENOENT',   code: 'ENOENT',   syscall: 'spawnSync npm',   path: 'npm',   spawnargs: [ '-v' ] } null
Not sure if this helps. Also, have a look at this problem. Of course, it's not the same code as in the test, but it might be a hint.

@architectonicus you're not using Git Bash instead of the Windows Command Prompt are you?
I wonder if we need to pass in { shell: true } or use execSync instead of spawnSync like discussed in this issue: https://github.com/nodejs/help/issues/728

@trevor-bliss , nope, I use the powershell most of the time. If not, I use the Terminal inside VisualStudioCode. The "classic" windows console behaves for me just like power shell or the VSC terminal.
I tried execSync and also shell: true. In these cases my script doesn't return.
This produces an output, though:
 var runSync = require('child_process').spawnSync; const npmVersionRet = runSync('npm', ['--version'],{shell: true, stdio: [0, 1, 2] }); console.log(hello node); console.log(npmVersionRet) console.log(npmVersionRet.error) console.log(npmVersionRet.status);
Prints:
6.9.0 hello node { status: 0,   signal: null,   output: [ null, null, null ],   pid: 16296,   stdout: null,   stderr: null } undefined 0
Picked up from here.

Interesting. Changing the check from spawnSync('npm', ['-v']) to  spawnSync('npm', ['-v'], { shell: true, stdio: [0, 1, 2] }) sounds reasonable to me.
Any concerns @pmdartus?

This issue is from 2015, and the child process builtin module changed a lot since then, so I wouldn't recommend following if we are not 100% certain of what we are doing.
The stdio is unrelated to the issue until proven otherwise. The default value pipes the results out. The [0, 1, 2] is equivalent to inherit which would print out the command output to the parent process output which is undesirable in the case of the CLI.
Concerning the shell flag, this flag is false by default on the spawnSync which is probably the root of the problem. Going through the documentation of the child process module, this module also exposes wrapper methods for common use cases. Based on the documentation, I would recommend swapping spawnSync('npm', ['-v']) with execSync('npm', ['-v']) that spawn a new shell automatically.

“I would recommend swapping spawnSync('npm', ['-v']) with execSync('npm', ['-v']) that spawn a new shell automatically.”
Please note that the execSync as executed in my test script never returns.

I found the issue, the execSync method doesn't accept the argument a second parameter. It should be execSync('npm -v']) and not execSync('npm', ['-v']). I tried on a Windows VM and it works.
@architectonicus Could you confirm you also observe the same thing?

@pmdartus I observe the same. execSync('npm -v') works for me.


Added bugfix and smoke test

My laptop is messed up, so I borrowed a friend's. I couldn't figure out how to remove the commit with his name.

forgot that the first pull request wasn't merged yet. What's the best way to keep commits on separate pull requests? separate branches?

:fire_engine: :cloud:


Unsafe Handling of DateFormatter

Problem
The ISO8601DateFormatter in swift-corelibs-foundation is not thread-safe.
However, URLEncodedFormDecoder and URLEncodedFormEncoder use this class as a singleton. This can cause a runtime crash, such as a segmentation fault.
Reference to the code:
https://github.com/hummingbird-project/hummingbird/blob/d7fb66e1a8d6bc815d335e794910b1d43c8b3e07/Sources/HummingbirdFoundation/Codable/URLEncodedForm/URLEncodedForm.swift#L51-L55
Steps to Reproduce
The following code, when run multiple times in a Linux Swift environment, sometimes causes a crash:
import Foundation

let iso8601Formatter: ISO8601DateFormatter = {
  let formatter = ISO8601DateFormatter()
  formatter.formatOptions = .withInternetDateTime
  return formatter
}()

let now = Date()
for i in 0..<1000 {
  DispatchQueue.global().async {
    let date = now.addingTimeInterval(Double(i) * 60 * 60 * 24)
    print(iso8601Formatter.string(from: date))
  }
}

sleep(1)

Tested Environment

Swift 5.9 (swift-5.9-RELEASE)
Target: x86_64-unknown-linux-gnu

I couldn't get it to crash, but I've still got rid of the global instance.
Fixed in #274
I thought this had been fixed in corelibs foundation, but looking through issues it doesn't look like it has. I doubt it will now and we'll have to wait for the new Foundation

Fix in v1.9.1


Indicator for showing whether player state is in sync

It would be a nice addition to have some kind of indicator for the dm (or everyone) that shows whether the state off all connected people is in sync.
This could be easily implemented by just sending an message to the server once the latest map/fog got loaded.

This would be very useful, packet loss is very rare but the player can go out of sync very rarely so having a thumbs up in the corner that says "Everything is good" would really help.

This would be very useful, packet loss is very rare but the player can go out of sync very rarely so having a thumbs up in the corner that says "Everything is good" would really help.


Add support for > path-0.7.1

The last update from path-0.7.0 to path-0.7.1 broke our tests and it looks like there are changes not mentioned on the changelog about exposing different things. Later it was re-released as 0.8 but stackage starting from LTS-15.14 is using the marked package from the package. Effectively, breaking our test suite.

Apparently, the path package was returned to the version 0.7.0 on LTS 16. I just pushed another commit to verify that it's working


Separate Asset loading into its own module

The way asset loading is currently integrated into the Render module is convenient from the outside, but a mess from the inside. The render module shouldn't need to know about N file formats, only how to create textures, vertex buffers, index buffers from a Stream object, and a 'cannonical' data description (e.g. what pixel format, width, height, offsets into Stream object where mip surfaces start etc...).
Stuff like the RawMeshLoader, MeshBuilder, ShapeBuilder don't belong into the Render module, but in a separate Asset module:

the job of the Asset module is to provide asynchronously loaded or generated texture, vertex- and index data in a 'canonical' format to the Render module
all IO code is removed from Render module
plugin loaders are removed from the Render modules, only the hardwired loader code for the cannonical format remains
no extra data copying
don't link code for file formats that are not needed (supported file formats must be declared when Asset module is setup)
Asset module: allocates resource Id, creates and fires IO request (via IOQueue...?), when IO done, find loader which accepts the format, let loader convert to canonical format, then either:

directly talk to Render module to create Resource..
and/or: return Stream object and data description struct???

Open questions:

consider moving loader into IO thread?
procedural generation? also in separate thread?
first version should probably not work in IO thread, textures and mesh data doesn't require any expensive decoding
remaining problem: OpenGL resources are still created on main thread

Asset creation/loading has been refactored and moved into its own module.


xorg: Remove xcb-proto dependency

I'm happy with these changes once CI is 🍏

CircleCI
==> Installing xcb-proto from linuxbrew/xorg
git config --replace-all homebrew.private false
==> Downloading https://xcb.freedesktop.org/dist/xcb-proto-1.12.tar.bz2
Already downloaded: /home/linuxbrew/.cache/Homebrew/xcb-proto-1.12.tar.bz2
==> Verifying xcb-proto-1.12.tar.bz2 checksum
tar xjf /home/linuxbrew/.cache/Homebrew/xcb-proto-1.12.tar.bz2
==> ./configure --prefix=/home/linuxbrew/.linuxbrew/Cellar/xcb-proto/1.12_2 --sysconfdir=/home/linuxbrew/.linuxbrew/etc --localstatedir=/home/linuxbrew/.linuxbrew/var --disable-silent-rules
.
Last 150 lines from /home/linuxbrew/.cache/Homebrew/Logs/xcb-proto/01.configure:
2018-02-05 21:41:46 +0000

./configure
--prefix=/home/linuxbrew/.linuxbrew/Cellar/xcb-proto/1.12_2
--sysconfdir=/home/linuxbrew/.linuxbrew/etc
--localstatedir=/home/linuxbrew/.linuxbrew/var
--disable-silent-rules

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for xmllint... no
configure: WARNING: xmllint not found; unable to validate against schema.
checking for a Python interpreter with version >= 2.5... none
configure: error: no suitable Python interpreter found

https://circleci.com/gh/Linuxbrew/homebrew-xorg/963
https://circleci.com/api/v1.1/project/github/Linuxbrew/homebrew-xorg/963/output/111/0?file=true

TravisCI
/usr/lib/ruby/1.9.1/rubygems/version.rb:191:in `strip!': can't modify frozen String (RuntimeError)
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:12:in `<main>': Homebrew must be run under Ruby 2.3! You're running 1.9.3. (RuntimeError)

https://travis-ci.org/Linuxbrew/homebrew-xorg/builds/337755276#L1550
This bug is fixed upstream in Homebrew/brew. It's easily worked around here though.

@sjackman @maxim-belkin are you OK with the new version of this?

Looks good to me. Thanks, Michka.
Do either you or Maxim have the time/interest to the the TravisCI failure? It should be a two-line fix.
See https://github.com/Linuxbrew/homebrew-xorg/pull/348#issuecomment-363249607

It would also maybe cool if we could move the formulae to a Formula folder, and bring the travis/circle files in sync with core. Not sure I have much time at the moment though.

Agreed, moving formulae to Formula/ would be great. Maxim, you interested?


without with...as..., I get an

When use with...as.. statement, the ArchiveWriter works well.
with ArchiveWriter("foo.ark", "foo.scp") as writer:
for i in range(10):
mat = np.random.rand(100, 20)
writer.write("mat-{:d}".format(i), mat)
but, when I don't use with...as... statement, I get an AttributeError

If I missed something?

No, it's my implementation's problem. All the subclass of the Writer are forced using with with ... as statement because the files are initialized in __enter__ function.

It's an issue and I will fixed it later. Before that you can modify them based on you habits.


Error activating the extension in tests of another extension

Here's the output when running Jupyter tests via a CLI
Error: Trying to add a disposable to a DisposableStore that has already been disposed of. The added object will be leaked!
    at H3.add (file:///Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:24:1439)
    at I8.B (file:///Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:24:2007)
    at I8.getScopedEnvironmentVariableCollection (file:///Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:114:49475)
    at bO.getScoped (file:///Applications/Visual%20Studio%20Code%20-%20Insiders.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:116:441)
    at R.getEnvironmentVariableCollection (/Users/donjayamanne/.vscode-insiders/extensions/ms-python.python-2024.17.2024102201-darwin-arm64/out/client/extension.js:2:907142)
    at R._applyCollectionImpl (/Users/donjayamanne/.vscode-insiders/extensions/ms-python.python-2024.17.2024102201-darwin-arm64/out/client/extension.js:2:902069)
    at R._applyCollection (/Users/donjayamanne/.vscode-insiders/extensions/ms-python.python-2024.17.2024102201-darwin-arm64/out/client/extension.js:2:901766)
    at R.activate (/Users/donjayamanne/.vscode-insiders/extensions/ms-python.python-2024.17.2024102201-darwin-arm64/out/client/extension.js:2:901505)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 3)
    at v.activateWorkspace (/Users/donjayamanne/.vscode-insiders/extensions/ms-python.python-2024.17.2024102201-darwin-arm64/out/client/extension.js:2:2405)

@DonJayamanne this looks like a VS Code core issue: https://github.com/microsoft/vscode/blob/9ae3d699dc78dd273eb7bd815499615c73a8e35b/src/vs/workbench/api/common/extHostTerminalService.ts#L990


Challenge 41 doesn't submit

CatPhotoApp
Click here for cat photos.

Things cats love:

cat nip
laser pointers
lasagna

Top 3 things cats hate:

flea treatment
thunder
other cats

 Indoor
 Outdoor
 Loving
 Lazy
 Energetic

Submit

Hi @Rickhad ,
Thanks for reaching out. Have you tried the common troubleshooting steps?
We request you to please consult the Help Chat Room for any assistance to Coding.
and
Please go through Help I have found a bug
Closing, if you are confirmed on the chat that what you are facing is a bug feel free to comment, we can re-open this.


Could not check Schema compatibility: unknown protocol

Description
Unable to start KSQL server.
Error: Could not check Schema compatibility: unknown protocol: cp-shcema-registry-cp-schema-registry
Statement: CREATE TABLE PAGEVIEWS_REGIONS WITH ....
How to reproduce
Install KSQL server using an existing kafka cluster and SchemaRegistry:
helm install my-ksql-server -n kafka --set cp-zookeeper.url="my-kafka-zookeeper:2181",cp-schema-registry.url="cp-shcema-registry-cp-schema-registry:8081" cp-ksql-server
Ksql Server didnt start due error:
Caused by: io.confluent.ksql.util.KsqlException: Could not check Schema compatibility: unknown protocol: cp-shcema-registry-cp-schema-registry
at io.confluent.ksql.util.AvroUtil.isValidAvroSchemaForTopic(AvroUtil.java:56)
Using KSQL version 5.3.1
SchemaRegistry version: 5.3.1
confluentinc/cp-kafka version: 5.0.1
Apache Kafka version: 2.0.1
Error Log
[2019-11-19 10:16:14,753] ERROR Failed to start KSQL Server with query file: /etc/ksql/queries/queries.sql (io.confluent.ksql.rest.server.StandaloneExecutor:121)
io.confluent.ksql.util.KsqlStatementException: Could not check Schema compatibility: unknown protocol: cp-shcema-registry-cp-schema-registry
Statement: CREATE TABLE PAGEVIEWS_REGIONS WITH (REPLICAS = 1, PARTITIONS = 4, VALUE_FORMAT = 'avro', KAFKA_TOPIC = 'PAGEVIEWS_REGIONS') AS SELECT
PAGEVIEWS_ENRICHED.GENDER "GENDER"
, PAGEVIEWS_ENRICHED.REGIONID "REGIONID"
, COUNT() "NUMUSERS"
FROM PAGEVIEWS_ENRICHED PAGEVIEWS_ENRICHED
WINDOW TUMBLING ( SIZE 30 SECONDS )
GROUP BY PAGEVIEWS_ENRICHED.GENDER, PAGEVIEWS_ENRICHED.REGIONID
HAVING (COUNT() > 1);
at io.confluent.ksql.engine.EngineExecutor.execute(EngineExecutor.java:114)
at io.confluent.ksql.engine.SandboxedExecutionContext.execute(SandboxedExecutionContext.java:97)
at io.confluent.ksql.engine.SandboxedExecutionContext.execute(SandboxedExecutionContext.java:86)
at io.confluent.ksql.rest.server.StandaloneExecutor$StatementExecutor.handlePersistentQuery(StandaloneExecutor.java:335)
Checklist
Please provide the following information:

[x] A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
[x] Confluent.Kafka nuget version.
[x] Apache Kafka version.
[ ] Client configuration.
[ ] Operating system.
[x] Provide logs (with "debug" : "..." as necessary in configuration).
[ ] Provide broker log excerpts.
[ ] Critical issue.

this issue is unrelated to the .net client.
.. but it looks like you're not specifying a url when you need to since it's complaining that 'cp-shcema-registry-cp-schema-registry' is not a valid protocol (i.e. http or https). a valid schema registry url would be something like "http://localhost:8081" (but the domain or ip depends on your setup).
Also, schema is spelled incorrectly in cp-shcema-registry-cp-schema-registry.


refactor(connlib): implement NAT46 using etherparse

Using etherparse, we can implement the NAT46 implementation in the gateway more elegantly because it allows us to parse the IP and protocol headers into a static and much richer representation. The conversion to the IPv6 equivalent is then just a question of transforming one data structure into another and writing it to the correct place in the buffer.
This will help with #6371 because it means we can more easily NAT the ICMP errors back that we are receiving on the host system, thus telling the client on the other end that a certain IP is not reachable.

Still a minor error in the tests apparently but ready for conceptual review @conectado !

Ready for review @conectado !

Replaced by https://github.com/firezone/firezone/pull/6524.


[WIP] Add connection pool

What is the purpose of the change
Brief changelog
Verifying this change

Checklist

[x] Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
[ ] Add some description to dubbo-website project if you are requesting to add a feature.
[ ] GitHub Actions works fine on your own branch.
[ ] If this contribution is large, please follow the Software Donation Guide.

Codecov Report

Merging #9950 (69b4d5d) into 3.0 (7c72487) will decrease coverage by 0.09%.
The diff coverage is 39.79%.

@@             Coverage Diff              @@
##                3.0    #9950      +/-   ##
============================================
- Coverage     65.70%   65.61%   -0.10%
+ Complexity      319      294      -25
============================================
  Files          1213     1217       +4
  Lines         52949    53127     +178
  Branches       8021     8045      +24
============================================
+ Hits          34792    34858      +66
- Misses        14367    14458      +91
- Partials       3790     3811      +21

Impacted Files
Coverage Δ

.../org/apache/dubbo/remoting/api/ConnectionPool.java
0.00% <0.00%> (ø)

...moting/api/MultiplexProtocolConnectionManager.java
100.00% <ø> (ø)

...ache/dubbo/remoting/api/DefaultConnectionPool.java
33.72% <33.72%> (ø)

.../remoting/api/SingleProtocolConnectionManager.java
84.61% <100.00%> (ø)

...nt/metadata/ServiceInstanceHostPortCustomizer.java
65.78% <0.00%> (-21.06%)
:arrow_down:

...he/dubbo/rpc/listener/ListenerExporterWrapper.java
58.82% <0.00%> (-14.71%)
:arrow_down:

...ian2/dubbo/AbstractHessian2FactoryInitializer.java
50.00% <0.00%> (-11.12%)
:arrow_down:

...g/apache/dubbo/remoting/api/ConnectionHandler.java
45.45% <0.00%> (-9.10%)
:arrow_down:

...e/dubbo/remoting/transport/netty/NettyChannel.java
54.54% <0.00%> (-5.69%)
:arrow_down:

...o/rpc/protocol/tri/transport/GracefulShutdown.java
81.48% <0.00%> (-4.24%)
:arrow_down:

... and 35 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c72487...69b4d5d. Read the comment docs.


better alignment for double-width Unicode characters

Usually a character occupies one column on the terminal screen. But this is not true for, e.g. CJK scripts. '中文' is a string of two characters but is as wide as 'abcd'.

I merged the pull request, thanks for your help!
Quick tip for next time: use the develop branch instead of the master one. It was not difficult to merge it, but it initially created a conflict.

OK, I'll check for a develop branch next time I send a pull request.


Plutus Playground DecodingError when Evaluating any script

Summary
In our hosted Plutus Playground, when evaluating any compiled script the following error then a DecodingError is displayed and it is not possible to proceed to view transactions. Building and hosting locally evaluates correctly, logs and transactions can be seen.
Steps to reproduce the behavior

Go to https://playground.plutus.iohkdev.io/
Pick any demo file (e.g. Hello world or Starter)
Click "Compile"
On successful compilation, click "Simulate"
On Simulator page, click "Evaluate"

Actual Result
Error is displayed at bottom of screen and it is not possible to view transactions.
DecodingError: An error occurred while decoding a JSON value: Under 'EvaluationResult': At object key 'emulatorLog': Under 'Array': At array index 0: Under 'EmulatorTimeEvent': At object key '_eteEvent': Under 'EmulatorEvent'': At object key 'contents': Under 'ChainEvent': At object key 'contents': At array index 1: Under 'Tx': At object key 'txData': Expected value of type 'Object'.
Please try again or contact support for assistance.

Expected Result
Evaluation succeeds, no error, logs and transactions are displayed.
Describe the approach you would take to fix this
No response
System info
OS: Linux Mint 20.2
Current plutus-apps commit is 835ce24b7c89fa0d49e985029defc15b6732785e (works locally, not sure what we're currently hosted but should be head of main)

We are looking into this issue, and a fix should be implementing in the coming time.

Hey there,
I Have same issues can we have resolution here ?
I don't know what happening but my code is not working anymore on playground https://playground.plutus.iohkdev.io/
Can you please do let me know here and process what exactly what happening with the playground..
And please let me know alternative way to run Plutus program run and deploy for the same so that I continue on that part till issues fixed..
thanks :)

@developerdarsh You can run a local instance of the Plutus Playground. See readme here

@james-iohk I don't believe this will help, in this instance, as the bug is present on staging, which is the latest changes from the repo.

Although this issue is occurring at both the production and staging environments, I can confirm that the latest commit 7f53f18 works locally.  What I did to make it happen, in bash terminal:

At top of this repo, nix-build -A plutus-playground.server
Enter nix-shell then

cd plutus-playground-client
plutus-playground-server

Wait for server to start, it'll log Interpreter ready
3. In second terminal at top of this repo, enter nix-shell then:
cd plutus-playground-client
npm run start

Wait for client to start, it'll log webpack compiled with ...
4. In web browser, navigate to url https://localhost:8009/ (need to accept that ssl certificate is invalid)
5. Compile and evaluate plutus code without DecodingError

Thanks @james-iohk ! :)

I followed the steps to build and run the playground locally, but when I try to compile in the playground, I see this error now:

When I try to follow the steps to install it locally with this commit, I get the following error:
sudo nix-build -A plutus-playground.server
zsh: segmentation fault  sudo nix-build -A plutus-playground.server

Hi James,
I got the following error when running the "npm run start" command:
plutus-playground-server: user error (Error evaluating simulation: InterpreterError (CompilationErrors [CompilationError {filename = "\n/private/tmp/web-ghc-work-30f080fdeb208c5b/Main.hs", row = 2, column = 1, text = [" error:","    Could not find module \8216Playground.Contract\8217","    Use -v (or :set -v in ghci) to see a list of the files searched for.","  |","2 | import Playground.Contract","  | ^^^^^^^^^^^^^^^^^^^^^^^^^^"]},CompilationError {filename = "/private/tmp/web-ghc-work-30f080fdeb208c5b/Main.hs", row = 3, column = 1, text = [" error:","    Could not find module \8216Plutus.Contract\8217","    Use -v (or :set -v in ghci) to see a list of the files searched for.","  |","3 | import Plutus.Contract","  | ^^^^^^^^^^^^^^^^^^^^^^"]},CompilationError {filename = "/private/tmp/web-ghc-work-30f080fdeb208c5b/Main.hs", row = 4, column = 1, text = [" error:","    Could not find module \8216PlutusTx.Prelude\8217","    Use -v (or :set -v in ghci) to see a list of the files searched for.","  |","4 | import PlutusTx.Prelude","  | ^^^^^^^^^^^^^^^^^^^^^^^"]}]))
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>start: plutus-playground-generate-purs && npm install && npm run install:spago && npm run build:webpack:dev
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!      /Users/justin/.npm/_logs/2022-01-13T03_54_22_317Z-debug.log

@rwhiten1 @derneuere @specpro30. The steps I listed are proven working on Ubuntu. If you are using a Mac, particularly with silicon M1 hardware, then additional steps are required. The most proven community-produced guide I have seen is: https://github.com/renzwo/cardano-plutus-apps-install-m1

If you're using Linux and my steps didn't work for you then it might be best to start with a clean checkout of plutus-apps and try again. Another community member has put this guide together, it has received positive feedback: https://docs.google.com/document/d/15QR25eWgg587FTPnuf6Si_lpux-uFQsY_2Gj_4vf4lM

The issue is still existing. Can someone in IOG fix it? As it is the only place for me to practise plutus.

This is now fixed on all our hosted environments, including https://playground.plutus.iohkdev.io


pagination failure (Invalid pagination token given) on a function

Description: We have a user view which presents records from a DynamoDB table to the end user. The control causes records to be fetched 10 at a time and a 'next token' is retrieved for pagination to advance the listing when more than 10 records are available. The feature had been working as expected but recently has started throwing an error when the 'next' button is used.

Expected result: clicking the next button should advance the listing to the next 10 records (if more than 10 are available).
Actual result: An error is thrown as follows:
----  ERROR OBJECT ----
[
  {
    "path": [
      "listSecurSamples"
    ],
    "data": null,
    "errorType": "DynamoDB:UserIllegalArgumentException",
    "errorInfo": null,
    "locations": [
      {
        "line": 2,
        "column": 3,
        "sourceName": null
      }
    ],
    "message": "com.amazonaws.deepdish.common.pagination.InvalidPaginationTokenException: Invalid pagination token given."
  }
]

Screenshot of the function and obtained error

A similar unresolved issue is reported here and can be followed:
https://github.com/aws-amplify/amplify-cli/issues/3411

@AshwiniRaysecur are you using the Amplify Console for hosting this app?

@swaminator Yes, we are using the Amplify Console to host the app

@AshwiniRaysecur  Hi, we cut a ticket in our customer oncall list, now is in pending of investigation.

Thanks we'll stand by and continue monitoring this ticket...

Please note this issue is the same as reported here: aws-amplify/amplify-cli#3411
And it has been fixed in aws amplify version 4.13.4 ...

This isn't an issue with the Amplify Console and is instead an issue with the service. Closing since reported fixed.


Feature/tx proto validation

Description of the Change
Adds proto tx validator, which checks iroha::protocol::Transaction on whether all commands are set and permissions in commands are set with valid values
Benefits
No need to do static cast from interface::Transaction to proto::Transaction in transaction validator
Possible Drawbacks
None
Usage Examples or Tests
proto_tx_validator test has been included

CI is red :(


Changelog for v0.15.4 (#5869)

Issue #, if available:
Description of changes:
Testing (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm
/approve


[BUG] array_min does not fall back to CPU when hasNan = true

Describe the bug
A clear and concise description of what the bug is.
Steps/Code to reproduce bug
scala> spark.conf.set("spark.rapids.sql.hasNans","true")

scala> val df = Seq((Seq(1.0, 2.0, 3.0)), (Seq(1.1, 2.1, Double.NaN))).toDF("a")
df: org.apache.spark.sql.DataFrame = [a: array<double>]

scala> df.select(array_min($"a")).collect
22/08/16 16:15:32 WARN GpuOverrides:
! <LocalTableScanExec> cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.LocalTableScanExec
  @Expression <AttributeReference> array_min(a)#6 could run on GPU

res3: Array[org.apache.spark.sql.Row] = Array([1.0], [1.1])

array_min should runs on CPU but not GPU
Expected behavior
A clear and concise description of what you expected to happen.
Environment details (please complete the following information)

Environment location: [Standalone, YARN, Kubernetes, Cloud(specify cloud provider)]
Spark configuration settings related to the issue

Additional context
Related to #6330 and #6317

After removing the hasNan config fromGpuArrayMin(#6317), we can close this issue without fixing it.


Setup step is not awaited

🐛 Bug
The setup step doesn't wait for all the defined processes to finish. This means that the application is declared working, even though the model is still not loaded.
To Reproduce
Please take a look at the code sample below.
Code sample
import litserve as ls
from gliner import GLiNER

# (STEP 1) - DEFINE THE API
class SimpleLitAPI(ls.LitAPI):
    def setup(self, device):
        # setup is called once at startup. Build a compound AI system (1+ models), connect DBs, load data, etc...
        self.model = GLiNER.from_pretrained("urchade/gliner_mediumv2.1")

Logs:
sp-gliner-api-v1  | INFO:     Started server process [31]
sp-gliner-api-v1  | INFO:     Waiting for application startup.
sp-gliner-api-v1  | INFO:     Application startup complete.
sp-gliner-api-v1  | uvloop is not installed. Falling back to the default asyncio event loop.
Fetching 5 files: 100%|██████████| 5/5 [00:00<00:00, 17832.93it/s]
sp-gliner-api-v1  | /usr/local/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
....

Expected behavior
Application startup complete should be triggered after the model is loaded.
Environment
Local / Docker
Additional context

Hi @andreieuganox,
I think the application startup complete (logged from uvicorn) might be separate from the setup phase. If we wait for the model to load completely, it logs the setup completion for the specific worker afterward.

https://lightning.ai/docs/litserve/features/full-control#health-check

I just have a few thoughts on the backend implementation, which is quite interesting.
It seems there are 2 primary sets of processes in the application:

One to host the FastAPI servers (uvicorn server, actually)
One or more destined to handle the requests/workload, depending on the number of processors.

Although the FastAPI might be ready to receive requests, the workers are still not yet ready to consume those requests and generate responses.
I'm not sure those processes can always be easily synchronized to ensure the workers are ready before the Uvicorn server.

yes, as mentioned by @bhimrazy, you can check if the server has started or not using the /health endpoint.


Updating 'Codeforces tag' link

The previews link is broken, updated to one that works

Thanks for the PR :)!


ROS 2 Support?

Hi,
I have an A1 with the expressed goal of making it work with the new and improved ROS 2 navigation2 system. Do you have drivers that work with ROS 2 foxy or any of the ROS2 distributions?

Hello @SteveMacenski , it's nice to see you here (I am familiar with your work on spatio temporal voxel filter!). I have been working on this exact task recently. Please let me know if your team is working on this, or looking for someone to collaborate. Thank you.

I've essentially given up on it and am waiting for someone else to do it. I love the Unitree hardware, the robot itself is great, but the software support is terrible and a ton of false advertising

Hello, until now we does not work on ROS2, and does not plan to support ROS2 recently.

It is unfortunate Unitree is so non-responsive to requests like this. Github is festering with questions that remain unanswered.

This seems to address the question at hand? https://github.com/unitreerobotics/unitree_ros2_to_real/blob/main/ros2_unitree_legged_msgs/CMakeLists.txt

Any updates? Is ROS2 officially supported?


[Explicit Module Builds] Introduce an InterModuleDependencyOracle abstraction for tracking and sharing dependency scanning results

This PR refactors the handling of inter-module dependencies to centralize their handling in a new, reference-type, oracle.
InterModuleDependencyOracle will now be responsible for responding to queries about dependencies of a given module.
The intent behind it is to abstract the actual mechanism used for tracking/computing/caching inter-module dependencies. A single oracle reference is intended to be passed around to multiple Driver invocations, corresponding to different targets, by build-system clients (e.g. SwiftPM), replacing today's use of ModuleInfoMap-based API, to communicate already-computed dependency-scan results.
As implemented, it is a simple store of ModuleInfo value objects, which a Driver invocation populates by invoking a -scan-dependencies job; not unlike passing around an accumulating ModuleInfoMap.
In the future, the oracle's internal implementation may be replaced with a direct dependency-scanner library access, without having to break the clients' API.
There will be a followup PR to cleanup some of the shims required to ensure that this does not break the API currently used by SwiftPM.
Corresponding PR to adapt this API in SwiftPM: https://github.com/apple/swift-package-manager/pull/3042
Resolves rdar://71209183

@swift-ci please test

Thank you for taking a look, @cltnschlosser, and for your suggestions.

Any negative here when trying to do explicit module builds solely from the command line? Didn't seem like it, but I don't know enough about how this works to tell.

This does not make a difference to command line builds today; but, it does open up a possibility for the future to no longer have the driver launch -scan-dependencies jobs, instead talking to the dependency scanner directly via a library that the oracle will wrap.

@swift-ci please test

@swift-ci please test

@swift-ci please test

@DougGregor @nkcsgexi ping for a review.
This PR includes a bunch of shims/adaptors to ensure that the API currently in-use by SwiftPM continues working, but it would be nice to get this in to help with the transition to:
https://github.com/apple/swift/pull/34786

@swift-ci please test

@swift-ci please test


Race condition in server start up logic

Currently all services uses the same dataStore map
When service starts up, it will modify the maxQPS field of the dataStore and use the modified value to create a persistence factory. This introduces a race condition on the maxQPS field.
To fix it, we need to make a deep copy of persistence config for each service so that services won't affect each other.

The issue was addressed by the pr https://github.com/uber/cadence/pull/3062


Add image reconstruction from 2D patches

Checklist
GitHub

[x] I've given this PR a concise, self-descriptive, and meaningful title
[x] I've linked relevant issues in the PR body
[x] I've applied the relevant labels to this PR
[x] I've assigned a reviewer

PR contents

[x] I've consulted ivadomed's internal developer documentation to ensure my contribution is in line with any relevant design decisions
[x] I've added relevant tests for my contribution
[x] I've updated the relevant documentation for my changes, including argparse descriptions, docstrings, and ReadTheDocs tutorial pages

Description

This PR is part of the inference refactoring project to include microscopy data (#306).
It implements an image_reconstruction function in inference.py to reconstruct images from 2d patches.
The function is triggered both in testing.py/run_inference and inference.py/segment_volume at inference time with command test and segment from the config file respectively.
As discussed in #306, the feature is working only for MRI data at the moment due to dependencies with NifTI/Nibabel in the inference process.
The current reconstruction is done by averaging the predictions in the overlap between patches (as it is done in volume_reconstruction for 3D subvolumes).
The PR includes a functional test.

Linked issues
Fixes #781

Pull Request Test Coverage Report for Build 825875201

24 of 47 (51.06%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 69.969%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

ivadomed/testing.py
13
17
76.47%

ivadomed/inference.py
9
28
32.14%

Totals

Change from base Build 825590869:
-0.2%

Covered Lines:
4012

Relevant Lines:
5734

💛  - Coveralls

This PR is ready for review.
To test the PR:

You can run a 1 epoch training on 1 subject with the config_new_loader.json file
Then use the same config file with the command "segment".
You should have something like this on sub-cardiff03_T2w.

Note 1:
The "patch pattern" is visible because it's only a 1 epoch and 1 subject training. After many tests on my part, this remains the best way to see if the reconstruction is working on MRI data.
Note 2:
The 2d reconstruction of the image from the patches is done in the same way than sub-volumes in 3D i.e. by averaging the prediction on the overlaps.  I first wanted to do a proper long training and assess the quality of the reconstruction (no borders visible, good predictions in the overlap, etc) on MRI data, and optimize the reconstruction algorithm based on that. But I changed my mind and decided to wait until the inference process is completed for microscopy to do that. The quality assessment will be 100x easier to do on axon/myelin segmentations than on spinal cord segmentation.

and optimize the reconstruction algorithm based on that.

Just curious: what did you have in mind here?

I've done different tests: Mouhahah that's so good! 🥇 💯 👍

and optimize the reconstruction algorithm based on that.

Just curious: what did you have in mind here?

On microscopy data in ADS, we sometimes see the "patch pattern" or loss of continuity at the borders of patches in the segmentations. In ADS, the overlap is entirely cropped to remove predictions at the borders of patches (and the stride is adjusted to cover all the images). In the current implementation of ivadomed, the overlap is averaged between patches (but include the pixels at the borders). So, the borders we see on my "1 epoch, 1 subject" training may remain on microscopy segmentations even after proper training.
One possibility we discussed a while ago was to do "the best of both world", which would be to crop part of the overlap (pixels at the borders of patches) and average the rest of the overlap.


Fixing https://github.com/cliftonc/seguir/issues/12

removed timestamp from client and updated docs accordingly

Thanks - merged in separately with fixes to the post API.


Initial commit to prioritize capturing expressions

In particular, now if a captured value has a length attribute,
the value of that attribute is shown in the status field.
Also, the length of a watched expression is now bounded by
config.maxDataSize instead of config.maxProperties.
Unit tests still need to be addressed, and more fine grained
control of the total length of the data captured needs to be
added.

@GoogleCloudPlatform/node-team I'm still working on this, but I opened this pull request to get early feedback so that if my changes will have a negative impact that I am not aware of, I can fix it sooner rather than later.

PTAL

LGTM


Constrain against Forcebalance 1.9.4

This is based off of release 0.1.3 17519ec0c38be90541ef6b713372a3c84bc55a3e

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

$ curl -sL https://github.com/openforcefield/openff-bespokefit/archive/0.1.3.tar.gz | openssl sha256
SHA256(stdin)= 8b9ac214a056611c126557bb37290019d7775a856d916ab039239e2a2bc4ebf2


TraceX - Introduce title for alerts and investigations

Description
It is somehow unclear why alerts and investigations do not have a title, but only descriptions
Impact

Usability feature

Additional information

[x] Notifications requires clear and short identification in the kind of title or short summary
[x] I'm willing to contribute to this feature

Hello @wiegerterik
Since the feature is a 24.05 feature and the development phase for 24.08 is coming to an end, we need a status on the feature. Can you please update the status?

[ ] Currently you are assigned (Responsible) → Is this correct? If not, please assign the correct contact person
[ ] Please check whether the status (backlog, work in progress ...) is set correctly
[ ] Please comment on the current status of the feature
[ ] Are all SubTasks (issues from other repositories that deal with the feature) linked? → The easiest way is to mention the feature here in the issue (via the ID) so we can see which teams/repositories are involved.
[ ] Is there a spillover planned?

If you need any clarification, please get in touch, thank you very much.
Stephan

Already implemented.


Improvements to Keywords feature

Along with including a hyperlink to keywords on the resource detail pages (e.g. http://aashe-hub-dev.herokuapp.com/browse/center/7/The-Institute-for-Sustainable-Cities), (already included this in another ticket). We'd like to request the following features related to keywords to improve functionality.

Auto-complete feature - as you begin typing a keyword, a list of already existing keywords shows up, so you can select one of the pre-existing words rather than creating a word that is slightly different (e.g. "AASHE2016" rather than "AASHE 2016")
Index of keywords - in the keyword help-text, we'd like to link to a keyword index that displays all keywords and updates automatically - ideally, clicking on a single word would take you to the list of resources tagged with that keyword.
Add space after comma between keywords (e.g. http://aashe-hub-dev.herokuapp.com/browse/video/57/Introduction-Dorm-vs-Dorm-Sustainability-Competitions)

Hey, @bartTC. What do you think about switching to something like django-tagulous. We could use the auto-complete feature and a bunch of the others...

Note: I'll have to update elasticsearch to include this relation.

Ok, this is fixed in the admin. Take a peek and let me know if it needs any tweaks...

Reopening because:

Tags don't seem to be clickable links that take you to other resources with the same tag. Example: http://aashe-hub-dev.herokuapp.com/browse/outreachmaterial/105/Shut-the-Sash-Sticker
Tags don't seem to auto complete in submission forms or on admin side.

Also, the previous system allowed for tags with multiple words (e.g., "bottled water" or "fume hood") but this one doesn't seem to. It closes the tag as soon as you hit space. This isn't necessarily a deal breaker but we'll have to change the help text because it currently uses "bottled water" as the example tag.
Finally,

Would creating a filter for the keyword/tags be a possible solution for the first issue Julian mentioned?

Sorry, that was a late-night commit. I should have waited until morning. I changed the name of the field to "tags" at first and forgot to change it back to "keywords," so that caused some problems.
So here's how it works:

Keywords will now concatenate themselves when you add a comma(",") during submission
Once tags have been added, they will then auto-complete for further resources
Tags are now linked in the detail view

NOTE: Until #176 is resolved, these keyword links might not work immediately. We re-index the search every hour, but are working on a way to incrementally update it for each newly approved resource.

@monika-aashe I didn't create another filter, just used them as search terms.

Thanks Ben! It's seems to be working well now! However, the key word tool on the admin side still concatenates when you hit space so it doesn't seem to be possible to add multi word tags once a resource has been submitted. Sorry to open this ticket again!

Ok, I'll see what I can do on the admin side of things.

Modifying the admin is proving to be pretty tricky and could be a bit of a rabbit hole. Can you give me a sense of the priority of this feature? I may want to build a workaround or just push this back a phase or two...

We can add to the project team call tomorrow.
On Tue, Jan 19, 2016 at 4:57 PM, Benjamin W Stookey <
<EMAIL_ADDRESS>wrote:

Modifying the admin is proving to be pretty tricky and could be a bit of a
rabbit hole. Can you give me a sense of the priority of this feature? I may
want to build a workaround or just push this back a phase or two...
—
Reply to this email directly or view it on GitHub
https://github.com/AASHE/hub/issues/88#issuecomment-173000996.

--
Monika Urbanski
Data & Content Manager
Association for the Advancement of Sustainability in Higher Education
www.aashe.org |<EMAIL_ADDRESS>|<PHONE_NUMBER> ext. 26
--
Join us in Baltimore, Maryland from Oct. 9 - 12 for the AASHE 2016
Conference & Expo, themed Beyond the Campus. Call for proposals and
reviewers now open https://ww4.aievolution.com/ash1601!
Connect with us:   Website http://www.aashe.org/ | Facebook
https://www.facebook.com/aasheorg | Twitter
https://twitter.com/AASHENews | LinkedIn
http://www.linkedin.com/company/aashe

Email Confidentiality Notice
http://www.aashe.org/email_confidentiality_notice

Decision: rename to "tags" and add an auto-complete filter for tags;

Here's a perfect example of why just searching for the tag text is suboptimal: clicking on the "labs" tag (http://aashe-hub-stage.herokuapp.com/browse/?search=labs) ends up also including a bunch of resources with the "living labs" tag, which refers to something that is mostly unrelated to laboratories.

But, if I was looking for living labs stuff, wouldn't I click on the "labs"
tag?
On Fri, Feb 19, 2016 at 12:58 PM, juliandsmith<EMAIL_ADDRESS>wrote:

Here's a perfect example of why just searching for the tag text is
suboptimal: clicking on the "labs" tag (
http://aashe-hub-stage.herokuapp.com/browse/?search=labs) ends up also
including a bunch of resources with the "living labs" tag, which refers to
something that is mostly unrelated to laboratories.
—
Reply to this email directly or view it on GitHub
https://github.com/AASHE/hub/issues/88#issuecomment-186332689.

--
Robert Erb
Web Application Developer
Association for the Advancement of Sustainability in Higher Education
www.aashe.org |<EMAIL_ADDRESS>|<PHONE_NUMBER> ext 35
--
Join us in Baltimore, Maryland from Oct. 9 - 12 for the AASHE 2016
Conference & Expo, themed Beyond the Campus. Call for proposals & reviewers
extended until Feb. 22! Submit your proposal today
https://ww4.aievolution.com/ash1601!
Connect with us:   Website http://www.aashe.org/ | Facebook
https://www.facebook.com/aasheorg | Twitter
https://twitter.com/AASHENews | LinkedIn
http://www.linkedin.com/company/aashe

Email Confidentiality Notice
http://www.aashe.org/email_confidentiality_notice

Not really. You'd click on the "living labs" tag. This is especially true because you are most likely to see the tag when you are already on a resource with that tag. If you were looking at a living labs resource, you wouldn't even see a "labs" tag.

A related issue is that multi-word tags seem to split into separate tags for each word when accessing a resource via admin. For example, I just submitted a resource with a tag of "civic engagement". When I accessed the same resource in the admin side, the tag had been split into separate tags for "civic" and "engagement". It is possible to delete these two and reapply the original tag manually but obviously it would be better not have to remember to double check the tags.
Note that this issue doesn't seem to happen consistently. Multiword tags for some resources appear to stay whole.

Is this not deployed yet? Clicking on tags still sends you to a search page instead of a filtered set of results.
Also, there is a new problem in admin: before, I could select a two-word tag that was added during migration and it would show up correctly in the front-end. But now, selecting a previously used two-word tag breaks it into two separate tags.

Ok, I have performed the following and tested them:

[x] added a filter for tags to the search view
[x] updated the links in a specific resource to point to the page filtering on that tag
[x] fixed the issue with multi-word tags being split up

This was a tough one! Closing now.

Thanks Ben, I love how this works and can't wait to start using it when someone asks, "where can I find..."!
I added a 1d ticket for updating "Search results" to "Filter results", but that's pretty minor.
Nice work!


Error trying to reach service: 'dial tcp <IP_ADDRESS>:8080: connect: no route to host'

Issue
Installation of openebs via Kubera failed giving the following error - Error trying to reach service: 'dial tcp <IP_ADDRESS>:8080: connect: no route to host'. Ref -

weave pod on node5 was giving the below error -
ipsec init SA local failed: send InitSARemote: write tcp4 <IP_ADDRESS>:54826-><IP_ADDRESS>:6783: write: connection reset by peer

Expected Issue

https://github.com/weaveworks/weave/issues/3619

The following solution resolved the issue - https://github.com/weaveworks/weave/issues/3310#issuecomment-416885365


SyntaxError in /clan/turkiyem-lly00ju0

Error in ClashLeaders
SyntaxError in /clan/turkiyem-lly00ju0
Unexpected token < in JSON at position 0
View on Bugsnag
Stacktrace
webpack:///js/components/SearchBox.vue:56 - a.<anonymous>

View full stacktrace
Created automatically via Bugsnag

An error linked to this issue has been marked as fixed in Bugsnag
SyntaxError in /clan/turkiyem-lly00ju0

An error linked to this issue has been reopened in Bugsnag
SyntaxError in /

An error linked to this issue has been reopened in Bugsnag
SyntaxError in /


Clarify that disabling kubernetes built in secret store can only be d…

…one with an annotation, not by setting a helm argument
Clarify that disabling kubernetes built in secret store can only be done with an annotation, not by setting a helm argument
Thank you for helping make the Dapr documentation better!
Please follow this checklist before submitting:

[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)
[x] Read the contribution guide
[ ] Commands include options for Linux, MacOS, and Windows within codetabs
[ ] New file and folder names are globally unique
[ ] Page references use shortcodes instead of markdown or URL links
[ ] Images use HTML style and have alternative text
[ ] Places where multiple code/command options are given have codetabs

In addition, please fill out the following to help reviewers understand this pull request:
Description
Clarify that disabling kubernetes built in secret store can only be done with an annotation, not by setting a helm argument
Issue reference

@holopin-bot @james-bartlett Thank you!


Fix cmd in the kubernetes.adoc

Missing the symbol  "-"

@vjuranek could you review and apply this one? Note it should not be merged as is, as there's a merge commit part of this PR. Easiest is to use "Squash and merge" above, which will create a single (non-merge) commit. Thanks!

Hi @minchowang , thanks for your contribution! Could you please setup your email to one associated the your account and update the commit to avoid "octocat" commit? Thanks you!

contains octocat commits

Ok，i resolve it now

Thanks @minchowang ! Merged.


net.minecraft.world.level.entity package mappings

Most of this PR is copied straight from Mojang's official mappings, which are forbidden in Yarn (see CONTRIBUTING.md) since their license is incompatible.


Back port optimization to broadcast_axis to MXNet1.7.x

Description
Back port optimization to broadcast_axis for CPU#17882 and GPU#18168 to MXNet1.7.x
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[x] Changes are complete (i.e. I finished coding on this PR)
[x] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[x] To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

@leezu @sandeep-krishnamurthy can you help merge ? These are cherry-picked from master(already merged there)

Looks good since it's a cherry-pick.
For functionality check, broadcast_axis specific tests in unix-cpu should verify that on CI.
For performance check, do you mind pasting comparisons for w/ and w/o this change for 1.x branch just to be sure these perf achievements are valid for 1.x branch as well?

Looks good since it's a cherry-pick.
For functionality check, broadcast_axis specific tests in unix-cpu should verify that on CI.
For performance check, do you mind pasting comparisons for w/ and w/o this change for 1.x branch just to be sure these perf achievements are valid for 1.x branch as well?

@ChaiBapchya
the results are identical to the one I posted on my original PR. These results haven't changed over time. Do you still want additional runs ?

That PR was against master [2.0]. This PR is for 1.x branch [which forked off from master awhile ago]. Adding the numbers would ensure we do our due diligence before merging this PR.

 Shouldn't it be v1.x?

Why is it master? Shouldn't it be v1.x?

master of 1.x


Update Forest Focus Theme

Hello, I improved the JS support for the Forest Focus theme and added my name to the credits in README. Thanks for the awesome support, I'd love to help maintain this project if you're looking for assistance!

thank you


added a tooltip to patterns icon

Signed-off-by: Aditya Chaterjee<EMAIL_ADDRESS>Description
Added a tooltip to the config icon in patterns
https://user-images.githubusercontent.com/53532328/132197969-cf8dd301-eba7-414a-8b70-6fa9562781a5.mov
This PR fixes #
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

Codecov Report

Merging #4094 (7ed0f42) into master (b7c8d8e) will decrease coverage by 7.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4094      +/-   ##
==========================================
- Coverage   26.31%   19.28%   -7.04%
==========================================
  Files          64       59       -5
  Lines        5004     5005       +1
==========================================
- Hits         1317      965     -352
- Misses       3282     3745     +463
+ Partials      405      295     -110

Flag
Coverage Δ

gointegrationtests
?

integrationtests
?

unittests
19.28% <ø> (+0.37%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

mesheryctl/internal/cli/root/system/status.go
0.00% <0.00%> (-62.83%)
:arrow_down:

mesheryctl/internal/cli/root/system/start.go
1.01% <0.00%> (-53.72%)
:arrow_down:

mesheryctl/internal/cli/root/system/stop.go
0.90% <0.00%> (-39.10%)
:arrow_down:

mesheryctl/internal/cli/root/system/restart.go
1.78% <0.00%> (-32.15%)
:arrow_down:

mesheryctl/internal/cli/root/system/update.go
1.13% <0.00%> (-29.46%)
:arrow_down:

mesheryctl/internal/cli/root/system/check.go
1.39% <0.00%> (-22.38%)
:arrow_down:

mesheryctl/internal/cli/root/system/logs.go
0.00% <0.00%> (-21.60%)
:arrow_down:

mesheryctl/pkg/utils/platform.go
3.91% <0.00%> (ø)

mesheryctl/internal/cli/root/system/system.go
70.37% <0.00%> (ø)

...esheryctl/internal/cli/root/system/context/list.go

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6972059...7ed0f42. Read the comment docs.


Bot do 2 tasks to the infinity

Hi, when I do @gamer, it gives me the task: " exploring" to infinity or " getting unstuck from block"
With that, altoclef does not work (but baritone works well, I tested it), yet I tested the nightly and long term release version.  Altoclef only randomly walks to infinity.
Crashlogs and Screenshots (if applicable)
(you may drag + drop text logs and images here, thanks GitHub)
No crashlog, 2 screenshots

Other mods (if applicable)
Nothing
Discord username (if you want updates)
Wan_s²#2353
Developer Notes (leave this blank)

what version did you use?

It explores for "Infinity" until it can find a block. On 1.18.2, the block finder is broken, so won't work. Because it believes it's making no progress, it tries to get unstuck from the block by moving randomly.

The current version of Altoclef is made for 1.18.1.

The current version of Altoclef is made for 1.18.1.

okay so I'm waiting for version 1.18.2 :)
I'll come back if it still doesn't work in this next version, for now I'm closing the issue
Thanks


how to enabling nnpack

I'm trying to run inference on Android with nnpack enabled. I did the following, but didn't notice any performance differences:
(1) make sure USE_NNPACK is ON in CMakeLists.txt
(2) take the .pb files from zoo model, and convert them to use NNPACK, according to this page. I saw that the init_net.pb wasn't changed, but predict_net.pb was changed.
The inference time of bvlc_googlenet on my device was the same, before and after conversion.
Am I missing something?

update:
I just realized that I used a binary without NNPACK compiled previously.
Now I used the right binary, but observed that the network with NNPACK is slower (the inference time is 3x compared to network without NNPACK).
Any advice?

Hmm, it might be because NNPACK was not built with optimized flags - because it is C files, you might want to change the CMAKE_C_FLAGS in addition to CMAKE_CXX_FLAGS as you mentioned in the other issue - want to give that a shot?

Also, you might want to check if the default build script targets your platform: the default is -march=armv7-a -mfloat-abi=softfp -mfpu=neon and if you have aarch64 you may want to change it.

hi @kindloaf, @Yangqing, could you tell me how to properly convert a pretrained model to nnpack enabled, I built an ios demo and manually set op.set_engine("NNPACK") for those Conv Layers, however, when I traced the function calls, I saw that the conv_op were still done by Eigen.

Did I miss something? By the way, I used the intact predictor class in C++ context, which seemed to never switch to other engines, neither CUDNN nor NNPACK.

@power0341
Did you do the following?
(1) compiled the binary with nnpack enabled (make sure the switch is on in CMakeLists.txt)
(2) use the python script in this page to convert the pre-trained model to use "NNPACK" and "BLOCK" engine.

@kindloaf
I did switch on the nnpack option in CMakeLists.txt. This led to two additional libraries "libCAFFE2_NNPACK.a" and "libCAFFE2_PTHREADPOOL.a" been created.
For the second step, which is the one confused me most, I have it done in C++ source code, specifically, I loaded the NetDef predict_net and set the engines of convolution layers to be "NNPACK", then I created the net as a Predictor object. When printing the predictor->def().DebugString(), we see that
  arg {
    name: "engine"
    s: "NNPACK"
  }

I believe this is equivalent to the python version.
Any suggestion?

@power0341
From the debug string, it seems right to me. Not sure why it's still using Eigen...

@power0341 Hi, how did you set the engine to "NNPACK" in the c++ source code? I am trying to do this too. Could you share your code?

@kindloaf I also observed that the network with NNPACK is slower. Did you have any progress on that?

Hi @raininglixinyu I don't have any update on this.

@kindloaf @Yangqing @power0341  Hi, how did you set the engine to "NNPACK" in the c++ source code? I am trying to do this too. Could you share your code?

#726 solve the proplem.

I missed option "--recursive" when I clone the git.
You should command "git clone --recursive https://github.com/caffe2/caffe2.git"
to clone 3rd party gits in "https://github.com/caffe2/caffe2/tree/master/third_party".
Hope it works.

I can run inference code with NNPACK as engine, but how can I set the threadpool size of NNPACK which is default the total number of CPU's core? But I don't need the all CPU's cores use for test. What should I do for it?