Hierarchical coordinates in DataTree

Proof of concept implementation of https://github.com/pydata/xarray/issues/9056
Still probably needs more tests, suggestions welcome!
It's also possible that I've implemented this in a rather inelegant fashion, I am still wrapping my head around the tree-node abstraction.
CC @TomNicholas, @keewis, @owenlittlejohns, @flamingbear, @eni-awowale

This is cool to see! And it shows that simply calling align on the data in parent nodes can be neat way to perform the check for inheritability.
However this PR doesn't actually implement the suggestion in #9056, because it places a new strong restriction on the datatree data model. As well as restricting the range of valid DataTree objects, it would actually mean that there are netCDF files that DataTree objects cannot represent:
# using `main`
import xarray as xr
from xarray.core.datatree import DataTree

# create the unalignable tree from your tests
dt = DataTree.from_dict(
    {
        "/": xr.Dataset({"a": (("x",), [1, 2])}),
        "/b": xr.Dataset({"c": (("x",), [3])}),
    }
)

# write a valid netCDF file...
dt.to_netcdf('test.nc')

# switch branch to `datatree-hierarchy`
from xarray.backends.api import open_datatree

dt2 = open_datatree('test.nc')  # will raise an error from align

ValueError: inconsistent alignment between node and parent datasets:

The alternative suggestion I made in #9056 was instead to provide an API which performs inheritance like this, but doesn't apply that restriction to all trees at construction time.
I was working on my own PR in #9056 which does a similar alignment check but only when the user calls a specific .inherit method. The downside of my approach is that the alignment would be checked every time the user calls .inherit, which is really inefficient:
dt.inherit['x']  # calls `align` once
dt.inherit['x']  # calls `align` again!

I think what we actually want is somewhere between the two: we want to calculate all the inheritable coordinates at tree construction time like you've done, but not actually enforce that by raising errors from align. Instead we cache the list of which variables are inheritable (again like you've done), but keep them hidden until we merge them into the node once the user calls .inherit (like I've tried to do).

However this PR doesn't actually implement the suggestion in https://github.com/pydata/xarray/issues/9056, because it places a new strong restriction on the datatree data model.

I withdraw this complaint 😁  - see #9077 for discussion as to whether or not to impose this contraint.

@shoyer this suggestion might make no sense, but IIUC currently in this PR inherited variables & indexes are literally copied onto child nodes, relying on xarray's shallow copying features to not duplicate any underlying data. What I wonder is whether instead we could simply store the list of names of the inherited variables on the child node as ._inherited_var_names: set[str], and anytime we want access to them we simply look back up the tree until we find it. With a strictly-aligned model we can be sure that they definitely do exist further up the tree.
We could hide this detail by having properties like .variables and ._local_variables or similar, so that code which accesses these variables doesn't have to be changed. But then the model is a bit simpler as we're only storing string keys to represent the inherited variables (it's a little more like how coord_names is internally just a set[Hashable]).
The main tricky part I see with this approach is making sure the correct variable is altered when you try to write to an inherited variable, but I think we have to handle that carefully with either approach.

@shoyer this suggestion might make no sense, but IIUC currently in this PR inherited variables & indexes are literally copied onto child nodes, relying on xarray's shallow copying features to not duplicate any underlying data. What I wonder is whether instead we could simply store the list of names of the inherited variables on the child node as ._inherited_var_names: set[str], and anytime we want access to them we simply look back up the tree until we find it. With a strictly-aligned model we can be sure that they definitely do exist further up the tree.

Indeed, this is how I implemented it currently. Can you think of use-cases for distinguishing node vs inherited data? I guess this could be a nice option for .to_dataset() and for writing data to Zarr?

Oh right! I think we might need to be able to distinguish internally when doing map_over_subtree

Oh right! I think we might need to be able to distinguish internally when doing map_over_subtree

I agree, it seems pretty clear that we need support for pulling out "node local" versions of datasets, without inheritance.
For this PR, I'll implement a keyword option on DataTree.to_dataset to allow for only pulling out these local variables. This might be spelled local=True or inherited=False, and will ensure that we have the right information in the DataTree data model.
I'll also add a test to verify that to_zarr() works properly, without duplicating coordinates.

I implemented to_dataset(local=True). Would love feedback!
One messy thing is that it appears that assignment via .coords is broken on DataTree even at main. No error is raised when you try to set coordinates, but they don't actually get changed, e.g.,
In [1]: from xarray.core.datatree import DataTree
   ...: import xarray as xr
   ...:
   ...: tree = DataTree.from_dict(
   ...:     {
   ...:         "/": xr.Dataset({"e": (("x",), [1, 2])}, coords={"x": [2, 3]}),
   ...:         "/b": xr.Dataset({"f": (("y",), [3])}),
   ...:         "/b/c": xr.Dataset(),
   ...:         "/b/d": xr.Dataset({"g": 4}),
   ...:     }
   ...: )

In [2]: tree
Out[2]:
<xarray.DataTree>
Group: /
│   Dimensions:  (x: 2)
│   Coordinates:
│     * x        (x) int64 16B 2 3
│   Data variables:
│       e        (x) int64 16B 1 2
└── Group: /b
    │   Dimensions:  (y: 1)
    │   Dimensions without coordinates: y
    │   Data variables:
    │       f        (y) int64 8B 3
    ├── Group: /b/c
    └── Group: /b/d
            Dimensions:  ()
            Data variables:
                g        int64 8B 4

In [3]: tree.coords['y'] = 5

In [4]: tree
Out[4]:
<xarray.DataTree>
Group: /
│   Dimensions:  (x: 2)
│   Coordinates:
│     * x        (x) int64 16B 2 3
│   Data variables:
│       *empty*
└── Group: /b
    │   Dimensions:  (y: 1)
    │   Dimensions without coordinates: y
    │   Data variables:
    │       f        (y) int64 8B 3
    ├── Group: /b/c
    └── Group: /b/d
            Dimensions:  ()
            Data variables:
                g        int64 8B 4

So I'm not quite sure how to test modifying coordinates yet. Maybe we'll leave that for a follow-up PR.

One messy thing is that it appears that assignment via .coords is broken on DataTree even at main.

Ah yes - I forgot there's a TODO deep somewhere for that 😅 I left it because it for later because it seemed like it might require changing the DatasetCoordinates class that ds.coords returns. (Surely that class could just be replaced by the new xr.Coordinates class now??)

What I wonder is whether instead we could simply store the list of names of the inherited variables on the child node as ._inherited_var_names: set[str], and anytime we want access to them we simply look back up the tree until we find it. With a strictly-aligned model we can be sure that they definitely do exist further up the tree.

Indeed, this is how I implemented it currently.

Hmm, looking again I'm not sure this is true. You have the attributes defining the DataTree node as
class DataTree:
    _name: str | None
    _parent: DataTree | None
    _children: dict[str, DataTree]
    _cache: dict[str, Any]  # used by _CachedAccessor
    _local_variables: dict[Hashable, Variable]
    _local_coord_names: set[Hashable]
    _local_dims: dict[Hashable, int]
    _local_indexes: dict[Hashable, Index]
    _variables: dict[Hashable, Variable]
    _coord_names: set[Hashable]
    _dims: dict[Hashable, int]
    _indexes: dict[Hashable, Index]
    _attrs: dict[Hashable, Any] | None
    _encoding: dict[Hashable, Any] | None
    _close: Callable[[], None] | None

i.e. every local variable (and index) are explicitly stored on every node, and every inherited variable (and index) are also stored explicitly on every node. What I was suggesting was something more like
class DataTree
    _name: str | None
    _parent: DataTree | None
    _children: dict[str, DataTree]
    _cache: dict[str, Any]  # used by _CachedAccessor
    _local_variables: dict[Hashable, Variable]
    _local_coord_names: set[Hashable]
    _local_dims: dict[Hashable, int]
    _local_indexes: dict[Hashable, Index]
    _inherited_coord_names: set[Hashable]  # note this is now the only new attribute needed compared to current non-inheriting model
    _attrs: dict[Hashable, Any] | None
    _encoding: dict[Hashable, Any] | None
    _close: Callable[[], None] | None

    @property
    def _inherited_coords(self) -> dict[Hashable, Variable]:
        # we know these exist somewhere further up the tree
        return {self._get_inherited_coord_from_above(coord_name) for coord_name in self._inherited_coord_names}

    @property
    def variables(self) -> dict[Hashable, Variable]:
        # we know these are all alignable because we checked it at construction time
        all_variables = {**self._local_variables, **self._inherited_coords}
        return all_variables

where the set of all variables if defined as a property that looks up the tree to fetch the correct inherited variables (so they aren't stored twice).
Basically the point is that the only extra thing I need to know is the names of the variables to inherit, and because I know they definitely exist, I can just generate any combination of local/inherited/all variables dynamically.

@TomNicholas  what do you think of this approach? I'll elaborate on some details later, this was a little harder than I expected.

OK, so here's the tricky case: How should we handle checking alignment when there is an inherited dimension that is not found on a direct parent dataset?
Example:
from xarray.core.datatree import DataTree
from xarray import Dataset

tree = DataTree.from_dict(
    {
        "/": xr.Dataset({"a": (("x",), [1, 2])}),  # x has size 2
        "/b/c": xr.Dataset({"d": (("x",), [3])}),  # x has size1
    }
)

This should clearly be an error in the inherited DataTree data model, because the grandchild node has an inconsistent size for the dimension x.
Checking this gets a little messy: The dataset attached to the intermediate node /b technically should not have an x dimension, because there are no data or coordinate variables on it. However, it turns out we can create Dataset objects with extra unused dimensions (via direct constructors), and everything works in align() as you would expect.
My current plan is to do just that: create technically invalid Dataset objects (actually via DatasetView) for purposes of checking alignment, but don't expose them to users.

My current plan is to do just that: create technically invalid Dataset objects for purposes of checking alignment, but don't expose them to users.

This is complete. The PR is ready for another review.

@TomNicholas what do you think of this approach?

This seems like an improvement!
My original objection to the ChainMap idea was not understanding how you would conditionally build it out of only variables that aligned. Now that everything has to align, that problem goes away, which is great.
So now our list of DataTree attributes looks like
class DataTree:
    _name: str | None
    _parent: DataTree | None
    _children: dict[str, DataTree]
    _cache: dict[str, Any]  # used by _CachedAccessor
    _data_variables: dict[Hashable, Variable]
    _coord_variables: ChainMap[Hashable, Variable]
    _dims: ChainMap[Hashable, int]
    _indexes: ChainMap[Hashable, Index]
    _attrs: dict[Hashable, Any] | None

Pros:

We have fewer internal attributes
It's obvious what the correct set of all coords/dims/indexes for the node is (it's just the ones found under unique keys in the ChainMap)
The ChainMap idea is quite neat - I'm surprised at how few lines of code you needed to make this work!

Cons:

The Variable objects in _coord_variables will still be pointed to from multiple places
Our list of internal attributes on a DataTree node is still not just those on a Dataset plus the inherited coordinates
The ChainMap of coordinates is potentially storing irrelevant information, i.e. anything that gets overridden. Specifically

dt = DataTree.from_dict(
    {
        "/a": xr.Dataset({"x": (("x",), [1, 2])}),  # x coordinate of size 2
        "/a/b": xr.Dataset({"x": (("x",), [3, 4])}),  # x would be inherited, but gets overridden (as this new x coordinate still aligns)
    }
)

Because a ChainMap doesn't overwrite keys, just obscures them, the object dt['/a/b'] now stores a ChainMap which has a reference to the coordinate ('x', [1, 2]). This is despite us knowing at tree construction time that this original x coordinate will never be used or even accessible by the dt['/a/b'] node.
I guess I still don't really understand why we are trying to implement this inheritance by storing all the things that might have been inherited (or might have been overridden) in attributes on the local node. The ChainMap seems like a clever way to avoid iterateing upwards through .parents, but I'm not sure that means that every node should store a ChainMap. I'm curious what you see as the downside of this approach:

Store every coordinate variable only once, keeping only the names of the alignable variables on the child node. Then we iterate up the tree to go fetch the variable only when we need it.

However, it turns out we can create Dataset objects with extra unused dimensions (via direct constructors), and everything works in align() as you would expect.
My current plan is to do just that: create technically invalid Dataset objects for purposes of checking alignment

This also seems potentially fragile? 😕

I'm curious what you see as the downside of this approach:

Store every coordinate variable only once, keeping only the names of the alignable variables on the child node. Then we iterate up the tree to go fetch the variable only when we need it.

I had a go at implementing that approach in #9187, starting from the tests you have here.

OK, so here's the tricky case: How should we handle checking alignment when there is an inherited dimension that is not found on a direct parent dataset?

Ah, now I realise that this is probably why my grandchild tests in #9187 don't yet work...
This does seem tricky. IIUC the basic problem is that _check_alignment currently checks that local variables align with the local variables in the immediate parent node, then recursively calls itself again to check that variables on each child node align with those on the current node. At no point does it do the alignment check between two nodes that are not immediately adjacent, so if alignment information is not propagated, the chain is broken.
Other ideas for solving this:

Check alignment between every node and every other node - would work, but would be combinatorially expensive so clearly a terrible idea.
Check alignment between local variables and all the variables on the parent, including inherited variables. Inheritance will effectively populate those empty nodes, solving the issue. However if there the alignment check raises it then becomes really hard to generate a diff showing where the offending variables were defined. (Also not totally sure that this isn't circular logic...)

How should we handle checking alignment when there is an inherited dimension that is not found on a direct parent dataset?

Thinking about this more I have two questions:

Are we checking alignment against all variables (data and coordinate), but only propagating down coordinate variables? Couldn't that lead to weird situations where a child dataset that aligns with a grandparent's coordinate variables but not with its grandparent's data variables doesn't raise an error? (because the grandparents' data variables aren't inherited down to the empty parent node)
Does it even make sense for _check_alignment to raise a ValueError of the form
"group {path1} is not aligned with its parent: see {repr(group1)} vs {repr(group2)}"? The alignment criterion is that the current node aligns with the parent node including all coordinates inherited onto that parent node. But those inherited coordinates could have come from multiple different ancestor groups. So a single alignment error message could actually have happened due to variables in two different ancestor groups. For example:

tree = DataTree.from_dict(
    {
        "/": xr.Dataset({"x": (("x",), [1, 2])}),
        "/b": xr.Dataset({"y": (("y",), [1])}),
        "/b/c": xr.Dataset({"d": (("x", "y"), [[3, 4]])}),  # d has size {'x': 1, 'y': 2}, so doesn't align with either parent group.
    }
)

Here "d" doesn't align with "x" or "y", but you can't really say that the alignment error is caused by group "/b" misaligning with group "b/c" or by group "/"misaligning with group"/b/c"`, it's caused by both! It would make much more sense to say that group "/b/c" does not align with its parent group including all variables inherited onto the parent.

The Variable objects in _coord_variables will still be pointed to from multiple places

I see this as a pro, rather than a con. Using the exact same Variable objects means it is possible to modify metadata like attrs in-place .

Our list of internal attributes on a DataTree node is still not just those on a Dataset plus the inherited coordinates

This is indeed a bit of a con, but in my mind the right fix is probably to adjust the Dataset data model to using dictionaries of data_variables and coord_variables, rather than the current solution of a dict of variables and a set of coord_names. Using a separate dictionary for coord_variables would also be more aligned with how DataArray is implemented. The internal Dataset data model is a hold-over from the very early days of Xarray, before we had a notion of coordinate variables that are not indexes.

The ChainMap of coordinates is potentially storing irrelevant information, i.e. anything that gets overridden. Specifically

dt = DataTree.from_dict(
    {
        "/a": xr.Dataset({"x": (("x",), [1, 2])}),  # x coordinate of size 2
        "/a/b": xr.Dataset({"x": (("x",), [3, 4])}),  # x would be inherited, but gets overridden (as this new x coordinate still aligns)
    }
)

Because a ChainMap doesn't overwrite keys, just obscures them, the object dt['/a/b'] now stores a ChainMap which has a reference to the coordinate ('x', [1, 2]). This is despite us knowing at tree construction time that this original x coordinate will never be used or even accessible by the dt['/a/b'] node.

This particular example would actually be an alignment error (x is used as an index), but you are correct that this sort of scenario could (rarely) happen. Specifically, it could happen for coordinate variables that are not indexes, e.g.,
dt = DataTree.from_dict(
    {
        "/a": xr.Dataset(coords={"y": (("x",), [1, 2])}),
        "/a/b": xr.Dataset(coords={"y": (("x",), [3, 4])}),  # y would be overridden
    }
)

I don't really see this as a problem, though. It's true that the higher-level coordinates are not currently accessible, but they are still around if the coordinate on the lower-level group is modified.

I guess I still don't really understand why we are trying to implement this inheritance by storing all the things that might have been inherited (or might have been overridden) in attributes on the local node. The ChainMap seems like a clever way to avoid iterateing upwards through .parents, but I'm not sure that means that every node should store a ChainMap.

On the contrary, basically all ChainMap does is systematize iterating upwards through .parents! I highly recommend taking a look under the hood of ChainMap -- it's quite simple.

Store every coordinate variable only once, keeping only the names of the alignable variables on the child node. Then we iterate up the tree to go fetch the variable only when we need it.

To be clear, my intention was quite similar -- the underlying (node-level) maps  at each level of _coord_variables, _dims and _indexes should each exist exactly once. The ChainMap objects are just proxies that link them together.
That said, there's no reason why the ChainMap objects need to be constructed when the DataTree objects are built. Instead, it probably makes more sense to do this dynamically, to avoid a separate redundant representation of the tree structure, e.g.,
@property
def _chained_coord_variables(self):
    maps = [self._coord_variables]
    maps.extend(p._coord_variables for p in parents)
    return ChainMap(*maps)

However, it turns out we can create Dataset objects with extra unused dimensions (via direct constructors), and everything works in align() as you would expect.
My current plan is to do just that: create technically invalid Dataset objects for purposes of checking alignment

This also seems potentially fragile? 😕

It's a little fragile, but these objects never get exposed to users, so the risk is quite contained.

Check alignment between local variables and all the variables on the parent, including inherited variables. Inheritance will effectively populate those empty nodes, solving the issue. However if there the alignment check raises it then becomes really hard to generate a diff showing where the offending variables were defined. (Also not totally sure that this isn't circular logic...)

This is the solution I have currently implemented. I think it's OK -- the inherited coordinate variables are listed on the repr of the parent node -- though indeed it's not as clear as it could be. Ideally we would say exactly at which level all the conflicting coordinates came from.

Are we checking alignment against all variables (data and coordinate), but only propagating down coordinate variables?

We're checking alignment against inherited dimensions and indexes, which exactly what xarray.align() checks -- it doesn't worry about data variables at all, or coordinate variables that are not also stored in indexes.

Does it even make sense for _check_alignment to raise a ValueError of the form
"group {path1} is not aligned with its parent: see {repr(group1)} vs {repr(group2)}"? The alignment criterion is that the current node aligns with the parent node including all coordinates inherited onto that parent node. But those inherited coordinates could have come from multiple different ancestor groups.

Note that the repr for the parent includes inherited variables.

So a single alignment error message could actually have happened due to variables in two different ancestor groups. For example:
tree = DataTree.from_dict(
    {
        "/": xr.Dataset({"x": (("x",), [1, 2])}),
        "/b": xr.Dataset({"y": (("y",), [1])}),
        "/b/c": xr.Dataset({"d": (("x", "y"), [[3, 4]])}),  # d has size {'x': 1, 'y': 2}, so doesn't align with either parent group.
    }
)

Here "d" doesn't align with "x" or "y", but you can't really say that the alignment error is caused by group "/b" misaligning with group "b/c" or by group "/"misaligning with group"/b/c"`, it's caused by both! It would make much more sense to say that group "/b/c" does not align with its parent group including all variables inherited onto the parent.

The current error message (from evaluating your snippet on this branch) is:
ValueError: group '/b/c' is not aligned with its parent:
Group:
    Dimensions:  (x: 1, y: 2)
    Dimensions without coordinates: x, y
    Data variables:
        d        (x, y) int64 16B 3 4
Parent:
    Dimensions:  (x: 2, y: 1)
    Coordinates:
      * x        (x) int64 16B 1 2
      * y        (y) int64 8B 1

I think is pretty descriptive, though perhaps we could more clearly indicate that it includes all parent nodes, even just by switching "parent" to "parents".

That said, there's no reason why the ChainMap objects need to be constructed when the DataTree objects are built. Instead, it probably makes more sense to do this dynamically, to avoid a separate redundant representation of the tree structure, e.g.,
@property
def _chained_coord_variables(self):
    maps = [self._coord_variables]
    maps.extend(p._coord_variables for p in parents)
    return ChainMap(*maps)

This is now implemented -- do you think?

say if they are following the general gist of what's being done here 😁

I have been following along, I may be a bit behind.  I do like the properties for _coord_variables, _dims and _indexes, I think that captures what Tom was going for in a really clean way.
I have at least one dumb question I will ask inline.

👍 let's try to indicate that the group represent coordinates from the entire tree above.

Here's what you currently see, which I think is pretty clear:
ValueError: group '/b' is not aligned with its parents:
Group:
    Dimensions:  (x: 1)
    Dimensions without coordinates: x
    Data variables:
        c        (x) float64 8B 3.0
From parents:
    Dimensions:  (x: 2)
    Dimensions without coordinates: x

[having intermediate datasets that are technically invalid is] a little fragile, but these objects never get exposed to users, so the risk is quite contained.

This still makes me a little worried - I would rather have structures that always obey their own contracts by design, but I agree its a problem that we can completely isolate from the users.

I don't love this either but I think it's an expedient choice for now. DatasetView is also problematic for similar reasons -- maybe even worse because it's exposed to users.

I guess my concern is that
xr.open_dataset(..., group="/a")

results in a dataset that is different from either of
xr.open_datatree(...)["/a"].ds
xr.open_datatree(...)["/a"].to_dataset()

We are going to have a similar issue when we add group to open_datatree, too: open_datatree(..., group="/a") will be different from open_datatree(...)["/a"].
One way to solve this would be to add an inherited argument to open_dataset and open_datatree, but I'm not sure anybody would actually use it. It's probably better to just to document the discrepancy for now.
In practice, setting the group argument to either open_dataset or open_datatree is something I would expect users to will only do for very large files, in performance critical cases. For these power users, I'm OK with a few more surprising edge cases

@keewis that's a good point, though I'm not sure what we can do about it other than document the subtlety clearly.
Note that the proposed
xr.open_as_dict_of_datasets(...)["/a"]
would behave the way you are expecting, i.e. the same as
xr.open_dataset(..., group="/a")

Thanks everyone for your reviews and feedback, especially @TomNicholas ! This definitely improved the final design.
I'm merging this for now so we can start testing and iterating upon it.


Render buildings

At high zooms, Americana should render building. Thoughts?
Here are some ideas with plain gray fill:

Most topo-maps use solid black, but this is a heavy treatment that may be too dark.

I don’t have very many city-scale maps, but many state- and county-scale maps have city insets that show select landmark buildings identifiable by their shapes, like these maps of downtown Cincinnati that depict the former Riverfront Stadium or “Cynergy Field” [sic], which was a bowl stadium:

Some city-scale maps color-code buildings by their use:

Some maps leave out the building use but label house numbers:

The more tourist-oriented, the more detailed the buildings:

I agree that solid black will probably be too heavy looking.  I think a shade of gray just dark enough to contrast with the backround with a very slightly darker border should look good.
I don't think the offset border for psuedo 3D look works terribly well, but if we want that look there's no reason we can't do actual 3D buildings.  Examples of the kind of thing that is possible from MapTiler:

I'm not sure if 3D buildings are appropriate for this style necessarily, but maybe.  Anyway, it is cool that it's an option with webGL.

3D buildings could run up against the road map aesthetic that this style has adhered to so far, but it wouldn’t be so jarring if we bring in other changes at the same zoom level that deliberately depart from that aesthetic.
A really fitting way to convey building heights and shapes would be axonometric outlines. However, only MapLibre’s Node.js bindings support this perspective.
/ref mapbox/mapbox-gl-native#10064 mapbox/mapbox-gl-native#11882

It wouldn’t be necessary to fake a 3D appearance using a border: we can add a fill-extrusion layer with a fixed fill-extrusion-height. Then we can tweak the light source for better contrast.

Exactly.  We don't have to opt for full 3D buildings with real world height.  We can just pretend the buildings are really short for a nice little 3D effect.  Could look something like this from above:


Reading TCF consent from within a module

Regarding this segment: https://docs.prebid.org/dev-docs/integrate-with-the-prebid-analytics-api.html#reading-tcf2-enforcement-actions
The doc states that analytics adapter can listen for TCF events, but when I add an event listener for CONSTANTS.EVENTS.TCF2_ENFORCEMENT (had to look for it in the prebid codebase), it doesn't fire. I assume it should fire with every page load, but the docs are vague. I also tried getting all events before the module is loaded, still the event wasn't there either.
Can you please clarify when is the event fired and add a working example how to get the stated TCF consent information from within an analytics module?
Thank you.

Now I've noticed the event is being fired after auctionEnd. I don't think that's every useful. Is there a reason for it being fired so late? The consent should be finalised at the start of the auction, right?
Prebid MESSAGE: Emitting event for: auctionEnd
Prebid MESSAGE: Emitting event for: tcf2Enforcement

Sorry @EskelCz - this is the documentation repo and this issue seems to be code-related. Adding @Fawke who I believe was involved in the development of the GDPR module.

The code is pretty clear on this. From https://github.com/prebid/Prebid.js/blob/81eae0b8ad3bf1fabfb606166734b425fdd98751/modules/gdprEnforcement.js
  const tcf2FinalResults = {
    storageBlocked: formatArray(storageBlocked),
    biddersBlocked: formatArray(biddersBlocked),
    analyticsBlocked: formatArray(analyticsBlocked)
  };

  events.emit(CONSTANTS.EVENTS.TCF2_ENFORCEMENT, tcf2FinalResults);
}

events.on(CONSTANTS.EVENTS.AUCTION_END, emitTCF2FinalResults);

This is meant to be an after-the-fact accounting of which modules were blocked for various reasons. What is it you're looking for? The title is "reading TCF consent within a module". Other analytics adapters (e.g. https://github.com/prebid/Prebid.js/blob/master/modules/rubiconAnalyticsAdapter.js) read consent data - please search for 'consent' there. Also, there are two new PRs relating to obtaining consent metadata:
https://github.com/prebid/Prebid.js/pull/8043
https://github.com/prebid/Prebid.js/pull/8071

@bretg I just figured out I can read it from the bid requests, therefore it's no issue. Just my intuition was off, kind of mislead by the docs.
Thanks for the investigation, much appreciated.


feat(linter): Implement no_this_before_super with cfg

Implements eslint/no-this-before-super in #479

The best span would be 1 span pointing to the constructor keyword, and the other error span pointing to the this keyword.

Do we have a mechanism for this? My concern is that, say there are 2 uses of this before super(), if we highlight the first usage only, then the user fixes that one and then we show the error again with the second usage, it could be confusing. If we are able to have a diagnostic with multiple spans, we can do that here. Or else should we have multiple diagnostics? I'm happy to start by making sure to use the span for the entire method definition node.

Here's an example for multiple spans:
https://github.com/oxc-project/oxc/blob/0c225a49aa36f5d9c9738cdf2cb6d6c7d1810443/crates/oxc_linter/src/rules/eslint/no_sparse_arrays.rs#L58-L63
Feel free to to pick either one depending on your time availableness.

I'm going to submit with the option highlighting the whole method for now and open an issue to make it better. I'd like to explore a bit more what the right approach would be; I have a few ideas.

I'm going to submit with the option highlighting the whole method for now and open an issue to make it better. I'd like to explore a bit more what the right approach would be; I have a few ideas.

Opened #2290


x/build/cmd/gerritbot: uses too much GitHub quota

gerritbot makes GitHub API requests for each open PR. There are enough open PRs across all the Go repositories that each gerritbot restart consumes most (maybe all?) of our 5000 requests per hour.
In response, today I closed all the PRs that were opened more than a year ago. This has had the effect of annoying a number of people and probably is not a tenable plan long-term. It seems we'll have to figure out a way to avoid looking at unmodified PRs that works across restarts.

Perhaps related: gerritbot could automatically close PRs fixing proposal issues that have been declined. That is, the issue is closed, has the Proposal label, but not the Proposal-Accepted label.
I found one manually right now; I'm sure there are at least a handful of others right now, at least. The authors rarely close the PRs themselves, and auto-closing these shouldn't be too hard in terms of code logic.


Evaluate making the docs render the README instead of defining it's own

See https://stackoverflow.com/questions/46278683/include-my-markdown-readme-into-sphinx/68005314#68005314

More of an internal thought process than an issue...


lsp does not resolve package within dependency code

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Whenever I navigate to a file within a dependency of my project and that file actually import something from a package, the lsp cannot lookup the dependency.
Expected Behavior
I should be able to navigate dependency files without having any lsp error regarding resolving dependencies of the file.
Steps To Reproduce

Open a file within a dependency of your project that imports using 'package:...'
LSP cannot resolve the dependency

Environment
- OS:OSX 13.4.1
- Flutter version: 3.10.6
- Is flutter in $PATH: yes
- neovim version: 0.9.1

Anything else?

I am probably overlooking something obvious but I can't seem to figure out what it could be.
Thanks in advance for your help.

I've got the same issue. I've pretty much tried everything I can think of to fix it:

lsp.settings.analysisExcludedFolders = {}
Set lsp.root_dir to current file's directory

Like the OP, LSP works perfectly for files in my project, but I can't get LSP to work within dependencies.
This is very useful for exploring the implementation of a dependency.

Just a quick word on this. The issue here is not the same as the analysisExcludedFolders issue. Since you are seeing diagnostics then the lsp is not disabled for that file. As to why the LSP can't resolve the path of the dependencies I have no idea. I've never needed to do that and am not currently working with flutter so not quite in the space nor have the time to dig in. Contributions/investigations v. welcome. @sidlatau any change you happen to have hit this or know anything about it maybe (sorry to pick on you 🙏🏾)

Hm, for me LSP in the package works, I can find references, go to definition, etc. - tested on the same dio package:

So I suspect something in the local configuration causes the issue.

I see, thanks for the clarifications guys. @sidlatau If you don't mind could you post your flutter-tools config? And are you also on a Mac? Mine is basically the default except for some customization around the debugger. I also lazily load flutter-tools based on filetype:
  { "akinsho/flutter-tools.nvim", ft = "dart", cmd = { "FlutterDevices" }, ... }

  ...
  require'flutter-tools'.setup {
    debugger = {
      enabled = true,
      exception_breakpoints = "default",
      register_configurations = function(paths)
        local my_launchers =
          my_config.launchers and
          my_config.launchers.dart
        if my_launchers then
          local launchers = vim.deepcopy(my_launchers)
          for _, launcher in ipairs(launchers) do
            if not launcher.dart_sdk then
              launcher.dart_sdk = paths.dart_sdk
            end
            if not launcher.flutter_sdk then
              launcher.flutter_sdk = paths.flutter_sdk
            end
          end
          require'dap'.configurations.dart = launchers
        else
          require'dap'.configurations.dart = {
            {
              type = "dart",
              request = "launch",
              name = "Launch app",
              dartSdkPath = paths.dart_sdk,
              flutterSdkPath = paths.flutter_sdk,
              program = "${workspaceFolder}/lib/main.dart",
              cwd = "${workspaceFolder}",
            },
            {
              type = "dart",
              request = "attach",
              name = "Connect to running app",
              dartSdkPath = paths.dart_sdk,
              flutterSdkPath = paths.flutter_sdk,
              program = "${workspaceFolder}/lib/main.dart",
              cwd = "${workspaceFolder}",
            },
          }
        end
      end,
      run_via_dap = true,
    },
    decorations = {
      statusline = {
        app_version = true,
        device = true,
        project_config = true,
      },
    },
    dev_log = {
      enabled = false,
    },
    dev_tools = {
      auto_open_browser = true,
      autostart = true,
    },
    ui = {
      border = "single",
    },
  }

Hmm, one possible difference could be that I am using FVM for flutter version management - are you using it too?
Also, could you try the plugin without custom lazy loading? It already does lazy loading here: https://github.com/akinsho/flutter-tools.nvim/blob/561d85b16d8ca2938820a9c26b2fe74096d89c81/ftplugin/dart/init.lua#L1, maybe that could be an issue?

Thanks @sidlatau. I've just set up FVM, and removed lazy loading, but still no go to definition in packages. It seems like putting pub-cache in analysisExcludedFolders completely disables LSP for packages, and since go to definition is an LSP feature, then it makes sense that it doesn't work unless I'm missing something.

I'm also getting this issue and have tried many different things. The one thing I have noticed is that when navigating to a dependency file, the LSP is reporting it is operating in single file mode (see first screenshot) which is likely why the LSP is reporting errors and I can't use gD to delve further into the package implementation.
Is there something I'm missing?


Updates to latest version of javascript node plugin

Is this project still being maintained? If not I would consider taking ownership if you like?

 Let me see what's necessary to transfer ownership and how we handle it at plugins.gradle.org

@michaelruocco I investigated a little and here are some points we should talk about:

I can transfer the repository to you or an organization you name, but according the github transfer manual, you must not have a fork to this repo to receive it. I think I should merge your pull request, let you remove your fork and can transfer it to you.
About the plugin registration at plugins.gradle.org, I think the easiest would be if you just change the plugin id and re-publish it.
What do you think? What did I miss?

sounds fine to me, I would be happy to give that a go.

Ok, just remove your fork and I can transfer it to you @michaelruocco

thanks, done.


Solana rebuilding from image 333 to 666

Summary
I have the json and metadata.json files from the previous build.  A total of 333 images from 0 to 332 - I would like to continue generating from 333 to 665 - My files were generated for Solana - Do I need a hash id like on ETH metadata or will it search through the values on the json files to see if there is any match before generating the image? Thank you
Basic example
No response
Motivation
No response

@dropoutgang you would need to add dna: tempMetadata.dna to solana's metadata, on line ~248:
https://github.com/bolshoytoster/hashlips_art_engine/blob/708bf21426b11f3592697bde959caadcf11a722a/src/main.js#L248-L249


multiple output formats, e.g. srt and txt at the same time

How can I make multiple output formats at the same time, like:
whisper ./*.mp4 --language ko --model large-v2 --output_format srt,txt
(that command doesn't work, but just to show you what I think it should do.)

--output_format srt,vtt,json,tsv,ass (support formats) should now work with the new commit.


Placeholder text for TextBox

Is it possible to set from a Theme or otherwise?

what's the use case that we need this placeholder for textbox?

I was just thinking there could be separate shortcuts for
e.g. App Launching vs Web Search (or other types).
And then the TextBox could show which type of Search you triggered (some visual feedback).

@galambalazs why plugin icon can't satisfy this requirement?


Uninitialized constant Delayed::Job (NameError)

Hi I did what your README says but it's not working for me. Running rails 3.0.3 and Ruby 1.9.2.
When I add the following line to my config/environments/development.rb
Delayed::Job.scaler = :local

I get the error:
 /config/environments/development.rb:30:in `<top (required)>': uninitialized constant Delayed::Job (NameError)

Hi, sorry for the long delay, I've just released v1.0.1 of workless, the next step is to add parallelism to the scalers, I'd love some input as to how you'd like to see that working, i.e the rules surrounding it


Add disk io tests

LGTM, some thoughts:
We should add a connection between the data transformation (min_max, gaussian blur, etc) and the DiskDataLoader. Eg. the transformation should be given in the instantiation of the class, and passed down into the disk dataset.
Also, we need docstrings everywhere, copilot is very useful for that.

Done in #11


Double-clicking on icon doesn't close window

Issue Type: Bug
Double-click on the VS Code icon in upper left corner. The window resizes.
The expected behavior is that the window should close.
VS Code version: Code 1.28.1 (3368db6750222d319c851f6d90eb619d886e08f5, 2018-10-11T18:13:53.910Z)
OS version: Windows_NT x64 6.1.7601

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz (8 x 3392)

GPU Status
2d_canvas: unavailable_softwarechecker_imaging: disabled_offflash_3d: unavailable_softwareflash_stage3d: unavailable_softwareflash_stage3d_baseline: unavailable_softwaregpu_compositing: unavailable_softwaremultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: unavailable_softwarevideo_decode: unavailable_softwarevideo_encode: unavailable_softwarewebgl: enabled_readbackwebgl2: unavailable_off

Memory (System)
15.96GB (6.89GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (1)

Extension
Author (truncated)
Version

crane
Hvy
0.3.8

I can repro, looks like this must have regressed recently as it definitely worked in the initial version I tested.

Double click on the system menu active the default action of the menu. I think this changed in the 1.27 or 1.28
The defaut action has been changed and as far as I can see, it is not possible to configure it in the settings.

VS Code version: Code 1.28.1 (3368db6750222d319c851f6d90eb619d886e08f5, 2018-10-11T18:13:53.910Z)
OS version: Windows_NT x64 6.1.7601

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz (4 x 2400)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: enabledvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabled

Memory (System)
7.88GB (0.79GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (8)

Extension
Author (truncated)
Version

overtype
ada
0.2.0

vscode-edifact
DAX
0.4.0

xml
Dot
2.3.2

gitlens
eam
8.5.6

reg
ion
0.0.2

mssql
ms-
1.4.0

PowerShell
ms-
1.9.0

vsc-jsonsnippets
wmo
0.1.4

this is a duplicate of #58199

@sbatten may I honestly ask how much more duplicate issues you need before you realize that this is a bug?
There is nothing "modern" in breaking user expectations, or so I hope.


NULL Pointer Dereference in block_type_get_result_types

Version
commit 06df58f20eeb46351b0ca7ff46d21eb541d2784a (HEAD -> main, origin/main, origin/HEAD)
Compile
cd wasm-micro-runtime/product-mini/platforms/linux/
cmake -DCMAKE_C_COMPILER="gcc" -DCMAKE_C_FLAGS="-g -fsanitize=address -fno-omit-frame-pointer" -DCMAKE_LINKER_FLAGS="-g -fsanitize=address -fno-omit-frame-pointer" ..
make -j

ASAN Log
AddressSanitizer:DEADLYSIGNAL
=================================================================
==866714==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x565157bc383f bp 0x7ffe6cfb60d0 sp 0x7ffe6cfb5f90 T0)
==866714==The signal is caused by a READ memory access.
==866714==Hint: address points to the zero page.
    #0 0x565157bc383e in block_type_get_result_types /root/wasm-micro-runtime/core/iwasm/interpreter/wasm.h:875
    #1 0x565157bc383e in block_type_get_result_types /root/wasm-micro-runtime/core/iwasm/interpreter/wasm.h:863
    #2 0x565157bc383e in wasm_loader_check_br /root/wasm-micro-runtime/core/iwasm/interpreter/wasm_loader.c:7092
    #3 0x565157bc383e in check_branch_block /root/wasm-micro-runtime/core/iwasm/interpreter/wasm_loader.c:7143
    #4 0x565157bd0a73 in wasm_loader_prepare_bytecode /root/wasm-micro-runtime/core/iwasm/interpreter/wasm_loader.c:8127
    #5 0x565157bd0a73 in load_from_sections /root/wasm-micro-runtime/core/iwasm/interpreter/wasm_loader.c:3852
    #6 0x565157bdf654 in load /root/wasm-micro-runtime/core/iwasm/interpreter/wasm_loader.c:4215
    #7 0x565157bdf654 in wasm_loader_load /root/wasm-micro-runtime/core/iwasm/interpreter/wasm_loader.c:4391
    #8 0x565157b5571c in main /root/wasm-micro-runtime/product-mini/platforms/linux/../posix/main.c:885
    #9 0x7f54f8c2a082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082)
    #10 0x565157b56b4d in _start (/root/wasm-micro-runtime/product-mini/platforms/linux/build_asan/iwasm+0x22b4d)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /root/wasm-micro-runtime/core/iwasm/interpreter/wasm.h:875 in block_type_get_result_types
==866714==ABORTING

Reproduce
./iwasm /root/wasm-fuzz-instruction/fuzz_out/crashes/id\:000000\,sig\:06\,src\:028368\,op\:python\,pos\:0

PoC
PoC

@haruki3hhh Thanks for reporting this issue! I submitted PR https://github.com/bytecodealliance/wasm-micro-runtime/pull/3133 to fix it, please try again.

Close this issue since it was resolved.


Predict unseen data

Hi, Thanks for sharing your code.
I am training your model using a different dataset, which works pretty well (F1 of 0.78).
I am now trying to use the trained model to predict new data from a different dataset.
Perhaps I missed it but I don't see any methods for doing predictions, and what format should the data have for doing so?  Could you point me in the right direction? Thank you.

When I use this code , ( F1 R  P) which  I got were the same ;  But  I don't know why this thing happend, do you know?


Support Proxying/Mirroring

The main use case for trow is running inside a cluster. Therefore it will often work alongside other registries, such as a public registry like Docker Hub & Quay or an enterprise wide registry (often artifactory or Nexus). It's essential that trow is able to transparently and easily mirror or proxy images from these clusters.
There is some interesting discussion on this topic here: https://github.com/opencontainers/distribution-spec/issues/12

Although mirroring seems to be functional at this point, there are commonly encountered edge cases where it causes a problem.
For example, pulling a multi-platform image through Trow on one architecture, and then attempting to pull it again on another architecture, causes Trow to serve up the image for the former (now incorrect) architecture, rather than the manifest referencing all the supported architectures and subsequently proxying the download for the correct architecture.
Example:

Dev runs docker pull registry.local/f/docker/nginx, pulling down the linux/amd64 version of the image
2a. Dev deploys app to arm64 K8s cluster
2b. Admission check passes because registry.local/f/docker/nginx is now available on the registry
K8s pulls in the linux/amd64 image, unaware that there is actually a multi-platform manifest under that tag, rather than an image

This sort of highlights another issue - admission check fails for proxied images when they aren't pre-pulled in Trow. This may actually be a feature, allowing a cluster administrator to control which images from Docker Hub may be used on the cluster, by having to pre-pull them into Trow, however, there should at least be an option to make Trow check availability of proxied images in upstream registries before the admission check fails.

Good catch. I'll have to think about the right solution here.

So, I just found out through further I was completely wrong - trow completely ignores what platform was requested and always requests linux/amd64.
Not sure, but it's probably got something to do with how Docker Hub (and probably every other registry) treats image requests that don't ask for a specific platform. I.e. It's literally impossible to get another platform. I even tested with explicitly specifying the linux/arm64 platform, and got this:
$ docker pull registry.local/f/docker/alpine --platform linux/arm64
Using default tag: latest
latest: Pulling from f/docker/alpine
59bf1c3509f3: Already exists
Digest: sha256:e7d88de73db3d3fd9b2d63aa7f447a10fd0220b7cbf39803c803f2af9ba256b3
Status: Downloaded newer image for registry.local/f/docker/alpine:latest
WARNING: image with reference registry.local/f/docker/alpine was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64
registry.local/f/docker/alpine:latest

I mean, idk how Docker's own registry does it - but it doesn't have this issue when it comes to pulling images through it in Docker Hub Proxy mode..


Scritp 15 works WELL but its not really multitennant

Took me a week to fix it, but when last night I finally got it to work, I've noticed that script 15-solution2-xlb-GFE3-traffic-split.sh is not 100% multitennant.
While it's now able to Load Balance with traffic splitting a generic app01 // app02, some artifacts (like FWD_RULE) have a fixed name which doesnt allow me to have it parametric in APP_ID. this means that once you call it with app01, it creates GCP infrastructure to route to app01, and then you do it with app02 and it changes and route to app02.
There's currently NO WAY to route both differently and separately.
Things I need to change in 15-solution2-xlb-GFE3-traffic-split.sh:

URLMAP to add APPXX
VirtualHosts to add APPXX

To keep things easy, I will enforce this best PRactice: every new name shall be:
new_var := $APPXX-$OLDNAME
and to make this even obviouser ;) I will rename the variables from
OLD_VAR_NAME => OLD_VAR_NAME_MTSUFFIX
for easy of grepping.

example in .env.sh/epost:
export FWD_RULE="${URLMAP_NAME}-fwdrule-obsolete" # we wont use it again, because of a bug:
export FWD_RULE_MTSUFFIX="${URLMAP_NAME}-frmt"  # FwdRule Multitenant, see https://github.com/palladius/clouddeploy-platinum-path/issues/14

fixed it today. Now it works well and beautifully! Also thanks to the new Array i've created everything is parametric in just appXX.


Add a finder for rrdtool.

Hi,
I developed a finder for rrdtool. This allowed me to use graphana on my existing installations, where collectd is configured to create .rrd files only.

Thanks for this -- would you be OK with publishing this as a separate distribution, like (graphite-cyanite)[https://github.com/brutasse/graphite-cyanite] and (graphite-influxdb)[https://github.com/vimeo/graphite-influxdb]? This would also allow graphite-web users to benefit from this.
Let me know if you need any help but you can have a look at what graphite-cyanite does to publish it to PyPI. I'm happy to merge a docs patch mentioning the rrd finder's existence, though!

Thanks for the response.
Making a separate distribution works for me. I think I will have some time in November to do it.
Besides, could you think of adding it into the 'extras_require' of setup.py ?

Ah yes sure, let me know when you publish it :) Thanks!


Fix com.fasterxml.jackson.core:jackson-databind vulnerabilities found in pom.xml

Details:
CVE-2019-14379
More information
moderate severity
Vulnerable versions: < <IP_ADDRESS>
Patched version: <IP_ADDRESS>
SubTypeValidator.java in FasterXML jackson-databind before <IP_ADDRESS> mishandles default typing when ehcache is used, leading to remote code execution.
CVE-2019-14439
More information
moderate severity
Vulnerable versions: < <IP_ADDRESS>
Patched version: <IP_ADDRESS>
A Polymorphic Typing issue was discovered in FasterXML jackson-databind 2.x before <IP_ADDRESS>. This occurs when Default Typing is enabled (either globally or for a specific property) for an externally exposed JSON endpoint and the service has the logback jar in the classpath.

Fixed by PR - https://github.com/oracle/coherence-operator/pull/287


Create loopback interface and use /32 length instead 24

          The loopback IP I believe can be /32 length. Is there any reason to use /24 lentgh ? @venkatmahalingam WDYT ?

and
          Yes, loopback interface IP should be /32, but we need to create a loopback interface and assign the IP, should be assigned on the VRF interface.

Originally posted by @mardim91 in https://github.com/opiproject/opi-evpn-bridge/pull/66#discussion_r1271947746

from @JanScheurich also Also the vtep_ip_prefix must be /32


Getting Error

I followed all the instructions given in the README.md
For REST, all of the GET requests are working but the POST requests aren't working.
For "Get Appointment Slots" and "Get Appointment Candidates" POST requests, I am getting this error:
[{"errorCode":"BAD_REQUEST","message":"Value does not match expected type at [line:2, column:2]"}]
Any ideas to resolve this?

Hi !
You need to put values in the body of the post to make it works following your needs.
The list of attributes and behaviour are in the documentation https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/requests_ls_getappointmentcandidates.htm
In the request, go to the body tab and change the values in the JSON Body.
Let me know if it is ok for you
Seb

Thanks.


邮箱验证405无效的连接

Which version of Blessing Skin are you using?
最新版
Which version of PHP are you using?
8.0
What browsers are you seeing the problem on?
Microsoft Edge
Which web server are you using?
Nginx
What happened?
用户收到验证的邮箱点解连接显示40无效的连接
Error Logs
No response
Steps to Reproduce

检查你的账号是否验证成功，可能反垃圾自动给你点了。

并没有，检查了没有验证成功，先是发送验证，然后打开邮箱点击连接，进去就是无效的连接，检查后台显示未验证

我也遇到了这个问题。


GzDecoder doesn't implement BufRead

Hello! The docs for the GzDecoder struct say:

This structure exposes a ReadBuf [sic] interface that will consume compressed data from the underlying reader and emit uncompressed data.

If I've understood correctly, that would suggest that GzDecoder is intended to implement BufRead, which it currently doesn't. (Or maybe it's referencing the fact that GzDecoder wraps an R: BufRead, in which case the phrasing just needs to be tweaked?)

Oh oops I think this is just a mistake, while this may have been intended oh-so-long-ago I think nowadays it's not as easily implementable and not available in the base library. This should probably just be a bug to tweak the docs and update the to remove this clause!

Fixed in https://github.com/rust-lang/flate2-rs/commit/2a6dc3b66fe4313da1a58ebf6b4442998a39b841 . Will be in the next release.


Sign in with Mastodon isn't working

The server returns an HTTP 500 when the client makes a request for the OAuth2 URL, reproducible with curl:
$ curl -siL https://seattle.carinbikelane.com/api/Mastodon/GetOAuthUrl --data-raw '{"serverUrl":"https://metasocial.com"}' --header 'Content-Type: application/json; charset=utf-8'
HTTP/1.1 500 Internal Server Error
Content-Length: 0
Date: Thu, 15 Dec 2022 19:32:02 GMT
Server: Microsoft-IIS/10.0
Set-Cookie: ARRAffinity=…d53;Path=/;HttpOnly;Secure;Domain=seattle.carinbikelane.com
Set-Cookie: ARRAffinitySameSite=…d53;Path=/;HttpOnly;SameSite=None;Secure;Domain=seattle.carinbikelane.com
X-Powered-By: ASP.NET

Trying locally right now but https://metasocial.com seems to be down right now. Once it's back up I'll be able to investigate further.

Hmm, it's up for me right now. Is it still down for you? (In general I haven't noticed blips with it.)


Meilisearch crashes on start: Permission denied (os error 13)

Sail Version: 1.12.11
Laravel Version: 8.76.1
PHP Version: 8.1
OS: Windows 10 with WSL
docker image: getmeili/meilisearch latest d9d9cb7a7c66
docker image: sail-8.1/app latest ae27463add64

Description:
Full error message is
meilisearch_1   | Error: An internal error has occurred. `Permission denied (os error 13)`.

Could this be related to #299 ?
Rebuilding/pulling images doesn't help and I don't know what folder I should add more permissions to
Steps To Reproduce:

curl -s https://laravel.build/example-project | bash
cd example-project
./vendor/bin/sail up

Could also be upstream problem https://github.com/meilisearch/MeiliSearch/issues/1969 I'm unsure if things are supposed to be working ATM

@caugner can you confirm that things are working on your end after your PR?
@Kehet have you tried running sail build --no-cache? (Please note that this will clear any data).

I'm able to reproduce the issues following the steps above.
If I run docker run --rm -it -v /tmp/data.ms:/data.ms -e MEILI_DB_PATH=/data.ms getmeili/meilisearch:v0.24.0, MeiliSearch runs without a charm and writes into the specified directory. I'm surprised that it doesn't work as soon as that volume mount is replaced by a Docker volume, but it matches with what folks are reporting in https://github.com/meilisearch/MeiliSearch/issues/1969. 🤔
Probably it would make sense to pin a specific MeiliSearch version as soon as the db path and the API has stabilised. After all, you need to have the corresponding version of meilisearch-php installed (e.g. meilisearch-php v0.20 for MeiliSearch v0.24 or later VS meilisearch-php v0.19 for MeiliSearch v0.23 or older).

@caugner @Kehet does it work properly in Sail v1.12.10 without the aforementioned PR?

@driesvints Yes and no. It runs without errors, but MeiliSearch's data are lost as soon as the container is removed.

@caugner that's a Meilisearch issue, not a Sail issue. I'm reverting your PR since it now just throws errors. We'll need to wait for a Meilisearch release.

We reverted this.

Sorry I wasn't able to return to this ticket earlier but I can also confirm reverse fixed this issue


Minor german localization issues

Hi there, first of all, thanks for this great tool :)
I found two minor flaws in the german translation, which you might want to fix:
1.) On the left hand side (the menu), the second option in german reads "Überprüfen edition". A better translation would be "Edition überprüfen".
2.) When being on the startpage, on the right hand side the product information is shown. If no OEM-key was found in the firmware, the last entry reads currently in german: "OEM-Schlüssel in Firmwar nicht vorhandene". This could be fixed to "OEM-Schlüssel in Firmware nicht vorhanden"
Thanks again for your effort and best regards!

Hi SteLei
Thanx ;)
I had a issue posted requesting translations but got little response so I just did my own thing LOL
I will definitely make the changes with next release... still hoping we get Win 11 keys before then
Cheers


Should we consider removing the tests files from the final package?

Comment:
Pandas is the largest package in my environments reaching ~73 MB, almost half of that is in the test folder. I don't know how useful the tests are for an installed artifact and I wonder if we should remove them from the final package.
PS: The same happens in the wheel and download times would be greatly improved if the tests were removed there too.

For reference, a pandas issue on this topic: https://github.com/pandas-dev/pandas/issues/30741
My reading is that there is general agreement that this is something we should do. So I think conda-forge could certainly already start doing it

Looks like the size of the tests already decreased a lot in the newer releases. Should we just close this and let upstream keep reducing them? Or does someone wants to tackle the complete removal of the test folder here?


course-v4 and fastbook folders inside fastcore

Hi,
In the fastdotai/fastai-course docker image, the course-v4 and fastbook folders are located inside the fastcore directory instead of /workspace directory.
If the decision is to keep the folders inside the fastcore, then please close this issue.
If not, the issue is because of this line in the dockerfile located in fastai-build
RUN /bin/bash -c "if [[ $BUILD == 'course' ]] ; then echo \"Course Build\" && cd fastai && pip install . && cd ../fastcore && pip install . && git clone https://github.com/fastai/fastbook --depth 1 && git clone https://github.com/fastai/course-v4 --depth 1; fi"
RUN echo '#!/bin/bash\njupyter notebook --ip=<IP_ADDRESS> --port=8888 --allow-root --no-browser' >> run_jupyter.sh
COPY download_testdata.py ./
COPY extract.sh ./
RUN chmod u+x extract.sh
RUN chmod u+x run_jupyter.sh

After the pip install of fastcore library, the script doesn't navigate to workspace directory. I can make the change and raise a PR if you think this is an issue
Thanks
Ravi

Yes please make the change


Service connector - overview page link poirnt to wrong tenant

Dear Team,
In the overview page a service connector (for exemple in a web app), we can click on the resource name.
This link point to the microsoft.onmicrosoft.com tenant and not the customer tenant.
Hence this is not working.

Ideally the link should point to the tenant in use.
best

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fb0435c3-3a25-f8a8-30f2-b8b6270fc7de
Version Independent ID: 25537d47-e122-06af-107b-ecea71c002eb
Content: What is Service Connector?
Content Source: articles/service-connector/overview.md
Service: service-connector
GitHub Login: @shizn
Microsoft Alias: xshi

@zilberd
Thanks for your feedback! We have assigned the issue to author and will provide further updates.

Hi @zilberd thank you for raising this issue. I just tested it using my personal subscription and was able to replicate it. I have informed the product team. This is a great find, thank you!
I will close this issue as this is a docs repo and there's nothing for me to update in the docs. The product team will take over with the information shared.

#sign-off

@zilberd just letting know that this issue has been fixed. Thank you for bringing this to our attention.

#please-close


Not unique table/alias

If you do a search on a model that have 2 different relations that point to the same table, then an error occurs.

edge case.
PR welcome


Date picker calender icon is not coming on chrome on windows 10 machine

I am using
import { AdapterMoment } from '@mui/x-date-pickers/AdapterMoment'
import { LocalizationProvider } from '@mui/x-date-pickers/LocalizationProvider';
import { DatePicker } from '@mui/x-date-pickers/DatePicker';
On a system with windows 10, calendar icon is not appearing. We are using latest version of chrome.
There is no issue on windows 11 machine.
Is it a known issue or I need any configuration update?
Please help.
Search keywords:

Hey @smita-mui, thanks for the report. I'm transferring this issue to the MUI X repo so the team can take a look.


List of markers

Not a duplicate of issue #138. That issue is talking about an events list
When markers are close enough to each other to overlap, you can't click on markers below another. I suggest making a list of markers to select from.

So I don't want to make really significant changes to the UI at this point since I do plan to redo a bunch of marker-related things next year, as I talked about in #138. But, I think a simple way of taking care of this for now would be to add "next marker" and "previous marker" buttons to the edit marker card.
I would also recommend avoiding markers that close together anyways if you can avoid it. But there are situations where you can't so I will work on that.


xCertReq randomly fails with error "RPC server is unavailable"

Hello,
I have reliability issues with xCertReq on Windows Server 2016: randomly (about 1 every 10 times), it fails with error "The RPC server is unavailable. 0x800706ba (WIN32: 1722 RPC_S_SERVER_UNAVAILABLE)".
I use it in a DC hosted in Azure that gets deployed from scratch, below is the relevant portion:
WindowsFeature AddCertAuthority       { Name = "ADCS-Cert-Authority"; Ensure = "Present"; DependsOn = "[xPendingReboot]Reboot1" }
WindowsFeature AddADCSManagementTools { Name = "RSAT-ADCS-Mgmt";      Ensure = "Present"; DependsOn = "[xPendingReboot]Reboot1" }
xADCSCertificationAuthority ADCS
{
	Ensure = "Present"
	Credential = $DomainCredsNetbios
	CAType = "EnterpriseRootCA"
	DependsOn = "[WindowsFeature]AddCertAuthority"
}

xCertReq ADFSSiteCert
{
	CARootName                = "$DomainNetbiosName-$ComputerName-CA"
	CAServerFQDN              = "$ComputerName.$DomainFQDN"
	Subject                   = "$ADFSSiteName.$DomainFQDN"
	KeyLength                 = '2048'
	Exportable                = $true
	ProviderName              = '"Microsoft RSA SChannel Cryptographic Provider"'
	OID                       = '<IP_ADDRESS>.<IP_ADDRESS>.1'
	KeyUsage                  = '0xa0'
	CertificateTemplate       = 'WebServer'
	AutoRenew                 = $true
	#SubjectAltName            = "certauth.$ADFSSiteName.$DomainFQDN"
	Credential                = $DomainCredsNetbios
	DependsOn = '[xADCSCertificationAuthority]ADCS'
}

Here is the full execution of the resource, with the error:
[[xCertReq]ADFSSiteCert]
VERBOSE: [2017-06-20 12:21:08Z] [VERBOSE] [DC]: LCM:  [ Start  Test     ]
[[xCertReq]ADFSSiteCert]
VERBOSE: [2017-06-20 12:21:08Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Test-TargetResource: Testing Certificate with Subject
'CN=ADFS.contoso.local' issued by DC.contoso.local\contoso-DC-CA.
VERBOSE: [2017-06-20 12:21:08Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Test-TargetResource: No valid certificate found with
subject 'CN=ADFS.contoso.local' issued by DC.contoso.local\contoso-DC-CA.
VERBOSE: [2017-06-20 12:21:08Z] [VERBOSE] [DC]: LCM:  [ End    Test     ]
[[xCertReq]ADFSSiteCert]  in 0.0630 seconds.
VERBOSE: [2017-06-20 12:21:08Z] [VERBOSE] [DC]: LCM:  [ Start  Set      ]
[[xCertReq]ADFSSiteCert]
VERBOSE: [2017-06-20 12:21:08Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Set-TargetResource: Starting Certificate request with
Subject 'ADFS.contoso.local' issued by DC.contoso.local\contoso-DC-CA.
VERBOSE: [2017-06-20 12:21:08Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Set-TargetResource: Creating certificate request
'C:\Windows\TEMP\xCertReq-479eb889-ebef-46bc-a56c-3fc0e1449657.req' from
'C:\Windows\TEMP\xCertReq-479eb889-ebef-46bc-a56c-3fc0e1449657.inf'.
VERBOSE: [2017-06-20 12:21:09Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Set-TargetResource: Create certificate request result:
 Active Directory Enrollment Policy
VERBOSE: [2017-06-20 12:21:09Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Set-TargetResource: Submitting certificate request
'C:\Windows\TEMP\xCertReq-479eb889-ebef-46bc-a56c-3fc0e1449657.req' returning
'C:\Windows\TEMP\xCertReq-479eb889-ebef-46bc-a56c-3fc0e1449657.cer' issued by
DC.contoso.local\contoso-DC-CA.
VERBOSE: [2017-06-20 12:21:09Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Set-TargetResource: Submitting certificate request
using separate process.
VERBOSE: [2017-06-20 12:21:10Z] [VERBOSE] [DC]:
[[xCertReq]ADFSSiteCert] Set-TargetResource: Submitting certificate request
result: Certificate Request Processor: The RPC server is unavailable.
0x800706ba (WIN32: 1722 RPC_S_SERVER_UNAVAILABLE)
VERBOSE: [2017-06-20 12:21:10Z] [VERBOSE] [DC]: LCM:  [ End    Set      ]
[[xCertReq]ADFSSiteCert]  in 2.3780 seconds.
VERBOSE: [2017-06-20 12:21:10Z] [ERROR] PowerShell DSC resource MSFT_xCertReq
failed to execute Set-TargetResource functionality with error message:
Certificate file
'C:\Windows\TEMP\xCertReq-479eb889-ebef-46bc-a56c-3fc0e1449657.cer' not found.

Additional information:

When LCM runs config again, xCertReq always completes successfully
DSC config has other xCertReq that run just after this one and they seem to be always successful, even when this one fails

Could it occur because I run xCertReq just after xADCSCertificationAuthority?
Do you have any idea of a possible workaround? I'm thinking about testing a timer between xADCSCertificationAuthority and xCertReq but I don't know if it's relevant.

Hi @Yvand - thanks for submitting this issue - great detail. I do wonder if the request is hitting the ADCS before is has completed installing and the service has started.
Perhaps I could create a new resource to wait for the ADCS to have started up (with a timeout etc). Similar to the xWaitForDisk resources in xStorage.
I'll do some investigation and see if there is a way of detecting if an ADCS is available without connecting to it.

Hi @PlagueHO on my side I implemented a 30 secs timer between xADCSCertificationAuthority and xCertReq and since then I never faced this issue again, so it looks like it's indeed because ADCS is randomly not ready yet when xCertReq runs.

Hi @Yvand - I'll see about creating a new resource that will wait for the ADCS to become available. It shouldn't be too difficult. I'll see if I can put it together this weekend.

Hi @PlagueHO it would be really great, but I wonder if it should be a separate resource, or a logic implemented directly in xCertReq resource? Or Both?

Hi @Yvand - I'm mainly basing that on other patterns we've implemented similar behavior in a separate resource. For example: in xStorage we have the xWaitForDisk and xWaitForVolume.
This reduces complexity of the resource and makes it usable outside xCertReq.
We'd want to use a similar pattern here as well - something like this:
[ClassVersion("<IP_ADDRESS>"), FriendlyName("xWaitForCA")]
class MSFT_xWaitForCA : OMI_BaseResource
{
    [Write, Description("The type of CA in use, Standalone/Enterprise.")] String CAType;
    [Write, Description("The FQDN of the Active Directory Certificate Authority on the local area network. Leave empty to automatically locate.")] String CAServerFQDN;
[Write, Description("The name of the certificate authority, by default this will be in format domain-servername-ca. Leave empty to automatically locate.")] String CARootName;
    [Write, Description("Specifies the number of seconds to wait for the CA to become available.")] Uint32 RetryIntervalSec;
    [Write, Description("The number of times to loop the retry interval while waiting for the CA.")] Uint32 RetryCount;
};

What do you think? Would this address the problem?

Hi @PlagueHO I understand, this makes sense.
Yes the resource looks great, I'm eager to test it :)

Hi @Yvand - I've completed this resource and submitted a PR for it. It may take a little while to get this PR reviewed and merged, so feel free to try out the version in my branch: https://github.com/PlagueHO/xCertificate/tree/Issue-73

Hi @PlagueHO yesterday I tested your PR and it worked fine in my environment, thank you very much for this!

Hi @Yvand - awesome! That is great to know!
FYI: We changed the name of the resource to xWaitForCertificateServices - so once this has been released you'll need to change your configs to use the new name. Sorry about that - we're trying to avoid using abbreviations in names now.


Missing argument 1 for Corcel\Providers\AuthUserProvider

I have installed this package into Laravel, and tried to implement the authentication so that a user can login via the wordpress credentials.
However in the AuthServiceProvider class, the construct() takes a $config variable, but it's not mentioned in the ReadMe under Authentication, and I'm not sure what is supposed to be passed in.
Can you help?

Hi,
If you are using Laravel, you don't need to instantiate the AuthUserProvider class. Just define the Authentication provider and the User provider like it's done in the documentation ; Then, Laravel's built-in authentication system will be using thoses providers to authenticate users in your app.
But if you do have to instantiate AuthUserProvider for some reason, you can just pass null.

Ah I see where I went wrong there.
But after amending the config/app.php and config/auth.php files as advised, when attempting to login, my laravel app is still looking for my laravel users table, instead of the wordpress users.
I thought I might need to create a User model that has a protected $connection pointing to the wordpress data, but still no joy.

Pretty sure your issue was fixed here: https://github.com/jgrossi/corcel/pull/182
You using the latest version?

Yes, I have the latest version.

OK, I figured it out.  I had to change the default connection in Laravel to the wordpress connection.  Then it worked.


Add space between "and" and date

Add a space between "and" and the date to avoid things like "between 10/09/2023and 10/09/2023."

Thank you! Sorry, I missed this notification.


How to distinguish the same text when I click?

Does "ActiveLabel.swift" have any function to know which string I click?
label.text = "@aaa @bbb @aaa"
label.handleMentionTap({ (string) in
//How to do something?
})
How to know which @aaa I click, If I want two of them to do different things?

No this is not possible out of the box. To be honest, as a user if I see mentions with @ or even two hashtags which they are the same I would expect they both end up in the same place.
You can do something with their positions in the text like which one is before or after the other one once the function being triggered. But i guess this only works with only two similar, more than two is complicated.

Regarding this issue
Please update class according to below logic so might it will work for that which exact text has clicked
Take a dictionary to store details for specific text like
If have a user name with more detail (user_id, first name, Las name) then in that dictionary we store all details and when click on that user name then return that same dictionary in call back block so from that dictionary we can identify that exactly which text has clicked

Please email me when update anything on below Email id
<EMAIL_ADDRESS>

The question is different than which element is being tapped on. Of course you’ll get the value and you can map it back to some dictionary or just use it as is to query an API, etc. The question was/is, if all the value are the same how to sperate them to do different actions.
This is a bad practice. As a user you expect to see @maziyarpanahi any where in the text and you expect that goes or does the same thing regardless of its position in the text.
If the question is #aaa #bbb #ccc then yeah just use the value return in the function to map it to dictionary or just quesry the value from an API.


MB-2799 Import fake pricing data to staging/experimental

Description
Follows steps outlined in https://dp3.atlassian.net/browse/MB-2262 to import and upload fake pricing data via secure migration

WIll not be deployed to production

Deployed to Staging & Experimental only

Steps for deploying to experimental:

https://github.com/transcom/mymove/wiki/deploy-to-experimental

https://github.com/transcom/mymove/wiki/migrate-the-database#secure-migrations-for-one-environment

Reviewer Notes

aws-vault login $AWS_PROFILE
open S3

Look for<PHONE_NUMBER>2021_import_pricing_data_june_2020.up.sql in the following folders

transcom-ppp-app-experimental-us-west-2/secure-migrations
transcom-ppp-app-staging-us-west-2/secure-migrations
transcom-ppp-app-prod-us-west-2/secure-migrations (this file should only contain a comment, no sql)

Code Review Verification Steps

[ ] Code follows the guidelines for Logging
[ ] The requirements listed in
Querying the Database Safely
have been satisfied.
Any new migrations/schema changes:

[ ] Follow our guidelines for zero-downtime deploys (see Zero-Downtime Deploys)
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] User facing changes have been reviewed by design.
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

References

Jira story
this article explains more about the approach used.

Screenshots
If this PR makes visible UI changes, an image of the finished UI can help reviewers and casual
observers understand the context of the changes. A before image is optional and
can be included at the submitter's discretion.
Consider using an animated image to show an entire workflow instead of using multiple images. You may want to use GIPHY CAPTURE for this! 📸
Please frame screenshots to show enough useful context but also highlight the affected regions.

Messages

:book:
:link: MB-2799

  Generated by :no_entry_sign: dangerJS against 84dddd9700fcfb2f836c656843bfc79716d11522

It looks like the contract code is set to Pricing. Is that what we want for the test data? I'm thinking we may need to make it more obvious this is test data rather than real pricing. Also, my PR #4250 assumed it was named TRUSS (which I just made up), so I'll need to change that accordingly to whatever the name is here.
Note that if we do decide to change the code, we'll need to restore the experimental DB to make the migration run again there (infra can help us with that -- we've done it several times before).

OK i definitely missed that the code 'Pricing' stuck, i meant for it to be 'june_2020' @jacquelineIO  do you have any guidance on what Contract Code we want attached to this fake data?

I went ahead and had infra roll back the experimental database so we can run this migration against experimental again when we're ready (let's make it the last thing we do after reviewer approval but before merging).
See #4279 for a fix that will need to be merged into this PR after #4279 is approved.
Also, I think we should consider changing the name of the migration since import_pricing_data_june_2020 could imply this is real data.  I'm thinking more like import_fake_pricing_data.  We still need to get confirmation of the contract code we want to use.
With the fix from #4279, I checked all the re_* tables imported from this PR against a psql-loaded migration and they seemed to match. 🎉

I went ahead and had infra roll back the experimental database so we can run this migration against experimental again when we're ready (let's make it the last thing we do after reviewer approval but before merging).
See #4279 for a fix that will need to be merged into this PR after #4279 is approved.
Also, I think we should consider changing the name of the migration since import_pricing_data_june_2020 could imply this is real data. I'm thinking more like import_fake_pricing_data. We still need to get confirmation of the contract code we want to use.
With the fix from #4279, I checked all the re_* tables imported from this PR against a psql-loaded migration and they seemed to match. 🎉

to change the name of the migraiton to import_fake_pricing_data_ i would need to write code in the script that differentiates between an upload of fake data vs real data. i guess it would look like an extra step in the script that asks the user whether this is real or fake data. would it be better to just give the fake data a contract code that indicates that its fake? like (Fake Truss Data)

to change the name of the migraiton to import_fake_pricing_data_ i would need to write code in the script that differentiates between an upload of fake data vs real data. i guess it would look like an extra step in the script that asks the user whether this is real or fake data. would it be better to just give the fake data a contract code that indicates that its fake? like (Fake Truss Data)

I don't think the script has to change.  You should be able to change the migration filename after the pricing-import script creates it -- just remember to also change the migrations_manifest.txt file accordingly.  I'm also fine with import_pricing_data_fake (if you want to keep the same prefix) or whatever you'd like that makes it clear that it's not real data.  That reminds me -- we will also need to clean up S3 to remove the old migration in each environment and add the new one when it's ready.

to change the name of the migraiton to import_fake_pricing_data_ i would need to write code in the script that differentiates between an upload of fake data vs real data. i guess it would look like an extra step in the script that asks the user whether this is real or fake data. would it be better to just give the fake data a contract code that indicates that its fake? like (Fake Truss Data)

I don't think the script has to change. You should be able to change the migration filename after the pricing-import script creates it -- just remember to also change the migrations_manifest.txt file accordingly. I'm also fine with import_pricing_data_fake (if you want to keep the same prefix) or whatever you'd like that makes it clear that it's not real data. That reminds me -- we will also need to clean up S3 to remove the old migration in each environment and add the new one when it's ready.

ok so you're talking about manually changing it right?

ok so you're talking about manually changing it right?

Right.  But if you wanted to change the pricing parser, I'd probably just give the user the option/flexibility to enter a migration name rather than asking them if it's fake data or not.  But I think a manual change is fine for this story.


includeId option

Coverage increased (+0.5%) to 93.407% when pulling b0d7237b3a7a203005fa49624dca36c9a696a371 on optionalId into eb536692721997be4bff4126b19274bb0cd9ad45 on master.

Coverage increased (+0.5%) to 93.407% when pulling b0d7237b3a7a203005fa49624dca36c9a696a371 on optionalId into eb536692721997be4bff4126b19274bb0cd9ad45 on master.


feature: ci with multiple targets

This implements #34 in a way that our CI always tests for multiple node versions. Also removed the old travis file.

Hey  @HappyZombies my bad, changed.
Regarding eslint it runs fine on all three node versions in the ci so I think this should not be a problem, right?


refactor(storagenode): add LogStreamReplicaMetadata to /varlog.snpb.LogIO to replace LogStreamMetadata

What this PR does
Add /varlog.snpb.LogIO/LogStreamReplicaMetadata to replace
/varlog.snpb.LogIO/LogStreamMetadata. The
/varlog.snpb.LogIO/LogStreamMetadata returns
proto/varlogpb/LogStreamDescriptor that have deprecated fields Head and
Tail. Instead, the /varlog.snpb.LogIO/LogStreamReplicaMetadata returns
proto/snpb/LogStreamReplicaMetadataDescriptor that is proper to the intend of
RPC.
Which issue(s) this PR resolves
Anything else

This change is

Codecov Report

Merging #140 (926348d) into main (c4d0c8d) will decrease coverage by 0.24%.
The diff coverage is 41.53%.

@@            Coverage Diff             @@
##             main     #140      +/-   ##
==========================================
- Coverage   62.83%   62.59%   -0.25%
==========================================
  Files         125      125
  Lines       16883    16996     +113
==========================================
+ Hits        10609    10638      +29
- Misses       5727     5804      +77
- Partials      547      554       +7

Impacted Files
Coverage Δ

internal/storagenode/client/log_client.go
45.37% <0.00%> (-3.72%)
:arrow_down:

internal/storagenode/log_server.go
41.05% <0.00%> (-4.83%)
:arrow_down:

pkg/varlog/log.go
0.00% <0.00%> (ø)

pkg/varlog/operations.go
0.00% <0.00%> (ø)

proto/snpb/metadata.go
0.00% <0.00%> (ø)

pkg/varlogtest/log.go
71.46% <70.58%> (-0.15%)
:arrow_down:

pkg/varlogtest/admin.go
42.03% <85.71%> (+0.61%)
:arrow_up:

internal/storagenode/logstream/executor.go
75.94% <0.00%> (-1.90%)
:arrow_down:

internal/storagenode/logstream/committer.go
81.85% <0.00%> (-1.07%)
:arrow_down:

internal/metarepos/report_collector.go
77.86% <0.00%> (-0.81%)
:arrow_down:

... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Readme: Replace Jenkins badge with Travis

Replace badge from jenkins.jquery.org with badge from Travis CI
Follows-up d1564e1a47 and pull #849, which remove TestSwarm from the Jenkins run.
However that means the primary badge isn't very useful anymore and doesn't
include cross-browser results. It also seems that the Jenkins badge plugin doesn't
handle proxy caching properly, causing it to show as "Unstable" on GitHub long
after the build was fixed.
Update both badges to use SVG so that it doesn't look pixelated on high density screens.

LGTM, just want to confirm with @jzaefferer if "readme" is the proper components to use on the commit message.
From the valid names listed at package.json, looks like All would be the best option, but it seems weird for a change on the readme file.

Previous commits to the readme file also used that prefix (see history). I just copied the prefix from there.
I guess those commits are from before the entry in package.json. I've updated it accordingly.

Looks good to me.


rustls & webpki  dependencies are messed up in mesalock

There are inconsistencies between referencing crates with branch=mesalock_sgx and tag=sgx_v1.1.0
i.e. this here causes trouble:
https://github.com/mesalock-linux/rustls/blob/d9a91f5037f5c85f761279d97b838256285a1314/Cargo.toml#L19
rustls tag sgx_v1.1.0 depends on webpki branch=mesaloch_sgx but I'd expect it to depend on tag sgx_v1.1.0
In our Cargo.toml we now have a mess between branch and tag dependencies leading to this (after a cargo update):
error: failed to select a version for the requirement `webpki-roots = "^0.17.0"`
  candidate versions found which didn't match: 0.19.0
  location searched: Git repository https://github.com/mesalock-linux/webpki-roots?branch=mesalock_sgx
required by package `substratee_worker_enclave v4.0.0 (/home/abrenzikofer/substraTEE-worker/enclave)`

I encountered a similar issue:
# Cargo.toml
webpki = { git = "https://github.com/mesalock-linux/webpki", tag = "sgx_1.1.0" }
webpki-roots = { git = "https://github.com/mesalock-linux/webpki-roots", tag = "sgx_1.1.0" }

Causes errors:
error[E0308]: mismatched types
   --> src/ra.rs:187:68
    |
187 |     let mut client = rustls::ClientSession::new(&Arc::new(config), dns_name);
    |                                                                    ^^^^^^^^ expected struct `webpki::name::DNSNameRef`, found a different struct `webpki::name::DNSNameRef`
    |
note: Perhaps two different versions of crate `webpki` are being used?
   --> src/ra.rs:187:68
    |
187 |     let mut client = rustls::ClientSession::new(&Arc::new(config), dns_name);
    |                                                                    ^^^^^^^^

cargo tree confirms that two different webpki version are used:
$ cargo tree -p webpki
error: There are multiple `webpki` packages in your project, and the specification `webpki` is ambiguous.
Please re-run this command with `-p <spec>` where `<spec>` is one of the following:
  https://github.com/mesalock-linux/webpki#0.21.2
  https://github.com/mesalock-linux/webpki#0.21.2

In mesalock-linux/webpki-roots:sgx_1.1.0 we can see:
webpki = { git = "https://github.com/mesalock-linux/webpki", branch = "mesalock_sgx" }

I suggest replacing branch with the tag:
webpki = { git = "https://github.com/mesalock-linux/webpki", tag = "sgx_1.1.0" }

maybe make everything depend on tag is a better choice?

I endorse your suggestion @dingelish. tag everywhere.
It's not perfect because you still don't know the upstream version. The question is if you want to include that version in the tag too (We might have discussed this before): v1.2.3_sgx1.1.0

@brenzi
Plan to merge from branch v1.1.1-testing to master this weekend, followed by tagging every dependencies. It'll be a breaking change.
We wonder if using + is better, which means v1.2.3+sgx1.1.0. please advise. thanks!

The + makes sense IMO

The + makes sense IMO

how do we handle rustcrypto styled repo? such as https://github.com/mesalock-linux/rustcrypto-utils-sgx
they contain multiple crate in one single repo. it's really hard to tag it ... any idea?

tricky one. Basically I think it is good practise to have one repo per version number and vice versa. This way this issue couldn't happen. But as you're not in charge of the upstream repo we're out of luck. You could of course split the fork into single repos, but that makes rebasing a pain.
Maybe going down to the toml?
https://github.com/mesalock-linux/rustcrypto-utils-sgx/blob/2a0f1222fcb0b26405f3d731a091121a9db293f4/blobby/Cargo.toml#L3
version = "0.1.2.sgx1.1.1"

And then the tag for the repo would have to be
sgx1.1.1

but in the depending crates we could refer to "version".

thanks! I just tried with toml version. result shows that we can sgx info in Cargo.toml like
[package]
name = "xxx"
version = "0.1.0+sgx1.1.1"
authors = ["Yu Ding<EMAIL_ADDRESS>edition = "2018"

For dependents, it's the same as before:
[dependencies]
xxx = { version = "0.1.0" }

cargo would warn about unused metadata (warning: version requirement 0.1.0+sgx for dependency xxx includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion) on the following:
xxx = { version = "0.1.0+sgx1.1.1" }

and i have a tool to inspect dependencies: cargo-deplist
https://github.com/dingelish/cargo-deplist
it outputs a topological sort of dependency graph along with source metadata. it seems we can find duplication or errors with the help of it.
future improvements: high-lighting the duplicated/suspicious one according to some rules


Keystrokes are not properly swallowed

Notice how we scroll and we move the sprite down.

Just a note, this is the same bug described in #125

Duplicated.


Quil output with writable classical memory

Is your feature request related to a use case or problem? Please describe.
In QuilOutput, the measurement id convention returns PyQuil memory declarations that are read-only. From the PyQuil docs:
The only memory that is writeable is the region named ro, and only through MEASURE instructions.
All other memory is read-only.

This means that measurements are always empty when you run any program from QuilOutput (unless you manually change the memory declarations and measurements).
Describe the solution you'd like
QuilOutput to return a program with writeable memory - namely, a program with a single memory declaration called "ro" with a number of bits equal to the number of measurements in the circuit, and all measurements writing into "ro".
[optional] Additional context (e.g. screenshots)
Current behavior:
import cirq

q = cirq.LineQubit(0)
circuit = cirq.Circuit(cirq.H(q), cirq.measure(q))
print(circuit.to_quil())

Output:
# Created using Cirq.

DECLARE m0 BIT[1]

H 0
MEASURE 0 m0[0]

Desired output (m0 -> ro):
# Created using Cirq.

DECLARE ro BIT[1]

H 0
MEASURE 0 ro[0]

[optional] Describe alternatives/workarounds you've considered
Asking the PyQuil folks to allow additional writable memory references... 🙃
What is the urgency from your perspective for this issue? Is it blocking important work?

P2 - we should do it in the next couple of quarters

@rmlarose If this is functionality in Cirq-core, do you think we should move it so that it becomes something in cirq-rigetti so that they can look after it ?

Sorry for the late response! I think that certainly makes sense here, but is part of a larger discussion which I don't have the full context on. Thanks Matt for opening the above issue.
Perhaps relevant to the discussion is that Rigetti QCS now installs Cirq and you can, AFAIK, run Cirq circuits directly.

So if you can do Cirq -> QCS directly instead of Cirq -> Quil -> QCS, maybe Cirq -> Quil doesn't need to be user-facing?

Cirq -> QCS

This is via cirq-rigetti I'm assuming. If that's the case:

Maybe Cirq -> Quil doesn't need to be user-facing

I agree. We added QuilOutput because Cirq was not operable with Quil 2 years ago. Now that it is, I don't see a good case for keeping the more "primitive" functionality in Cirq.


Support Rancher V3 Api

Support Rancher V3 Api with following metrics:
rancher_cluster_component_status gauge
rancher_cluster_state gauge
rancher_node_state gauge

Thanks for this, sorry it took me a while to review.


Add $BORK logo to Nova

WOOF Nova team,
1st
Our $WIFD main token logo update is still stuck in the Pull Request #2707:
https://github.com/novasamatech/nova-utils/pull/2707
2nd
New token $BORK
Ticker: $BORK
Name: BORK
Asset Hub ID: 690
HydraDX ID: 1000148
Total Supply: 690
BORK logo in black-white (round):

BORK logo in black-white (square):

Many BORK in advance for your support!
Dogwifdots dev

@Leemo94 can you please support our pups to get their $WIFD and $BORK a nice logo? 🐶🩷

Thank you, added here: https://github.com/novasamatech/nova-utils/pull/2904


xargs is async

kind of annoying it doesn't warn about mismatch between types

not sure if there is a lint , or way to write things that improve things to get caught at compile time, since now it feels like the deceleration of types and the actual functions can go out of sync easily

ah actually they are in sync, the root problem is that it seems in javascript Promise is considered the same as void for types

Thank you!
I notice it might be more convenient to return CommandBuilder[].
How do you think about this?

yeah true, there is no reason to stop the chain call in xargs

I wonder how can that be implemetned
  async xargs(command: (arg0: string) => CommandBuilder) {
    const processes: CommandBuilder[] = [];
    for await (const line of this.stream) {
        processes.push(command(line));
    }
    await Promise.all(processes);
  }

should we await all process expect the last one or something (does that even work) , or I'm missing something straightforward

I see...
We'll just return Promise.all for now.

The original LineStream.xargs was not returning a value, so could you please take a look at this one?

questions:

why PromiseLike instead of Promise (it currently gives a type error)
instead of CommandResult[] I think we should return the last CommandResult, seems more intuitive to me

why PromiseLike instead of Promise (it currently gives a type error)
I made a mistake looking at the return value of CommandBuilder. (And I missed the LSP warning...)
instead of CommandResult[] I think we should return the last CommandResult, seems more intuitive to me
I think this xargs implementation executes the commands in parallel, so it is not clear which result is last.
Given the behavior of xargs, this function should execute one command at a time...! (And add the -P option)

I noticed that xargs should be spawned to generate the process.
Once #5 is taken in, close this PR.


How to get/set source settings for Xcomposite source

I am trying to alter settings for my Xcomposite window capture source and first trying to get them, but I keep acing the issue of type inside \async` block must be known in this contextwith theget_source_settings()` function.

The source_settings field of the SourceSettings response is generic and can be any value that can be deserialized by the serde crate. That means you have to pass in the type that should be deserialized.
For example:
// Define some custom settings struct that is valid for the source.
#[derive(serde::Deserialize)]
struct XCompositeCapture {
    value_1: String,
    value_2: u64,
}

// Load the settings, passing the type to `get_source_settings`
let settings = client.sources().get_source_settings::<XCompositeCapture>().await?;

// Or define the type on the variable directly
let settings: XCompositeCapture = client.sources().get_source_settings().await?;

If you're not sure yet how the data of the source may look like, you can use for example serde_json::Value and inspect the data.
let settings = client.sources().get_source_settings::<serde_json::Value>().await?;
// Print the `serde_json::Value` back out as JSON string
println!("{:?}", serde_json::to_string_pretty(&settings.source_settings)?);

Ah I see, got it working properly now. Thanks!


Using LLM Agents to Complete Simulation Agent Action

Hello,
I would like to implement a scenario where a simulation agent action is complete by accessing an external service. For example, if an agent's attempted action was:
'Alex searches on CupcakeHub for strawberry cupcake recipes.'
I would like to trigger a call from code to the CupcakeHub API* to retrieve said recipe, and then feed it back to the agent as an observation. Eg:
'Alex finds on CupcakeHub that the recipe for the strawberry cupcake recipe involves 100g of flour, 2 eggs...'
Is this possible?
My current thoughts:

The most suitable place would be the update_from_player method of the GameMaster class;
An LLM would be used to infer if the player action matched the use-case to trigger the external service, and if so to produce the request payload. That would be achieved via a prompt similar to:

Given the following attempted action <action-attempt>{action_attempt}</action-attempt>, output an answer in the following format

<evaluation>YES if the player wants to access CupcakeHub, NO otherwise</evaluation>
<explanation>An explanation of how you reached said conclusion</explanation>
<query>Query to submit to cupcake hub</query>

Based on that, a conditional logic which, if the evaluation is YES, calls the CupcakeHub API via its client, obtains the response and feeds it back to the agent. (This could be achieved perhaps using LangChain agents, which allow integrations of LLM with external services https://www.promptingguide.ai/research/llm-agents#llm-agent-applications)

I am wondering if my reasoning is correct or if you think this could be better implemented by 'hooking into' other components of the simulation life-cycle?
*Made up service, but you get the idea

Yep, I think that's a great way to implement that. It's very similar to the way we have done this kind of thing ourselves.


Package request: flutter324

Project description

Flutter SDK version 3.24 is released on Aug 6th.
Metadata

homepage URL: https://flutter.dev/
source URL: https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_3.24.1-stable.tar.xz
license: bsd3
platforms: windows, linux, macOS

Add a :+1: reaction to issues you find important.

Already been PR'd #336650