3D Tiles - Batch Table Hierarchy

For #3241
Spec: https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/66
This adds initial support for hierarchies in the batch table to help reduce the amount of duplicate metadata. More details behind this are in the linked spec issue. The 3D Tiles Hierarchy demo has some example styles

New features:
Cesium API:

Cesium3DTileFeature.isClass(className) - returns whether the feature's class is className. This does not check ancestor classes.
Cesium3DTileFeature.isDerived(className) - returns whether the feature derives from className. This checks the feature's class and ancestor classes.

Styling language:

isClass - same as above - isClass('door') - styles just doors
isDerived - same as above - isDerived('door') - styles doors and doorknobs
Styling a feature based on ancestor properties: "show" : "${building_name} === 'building0'" will style building0 and all descendants of building0

Todo:

[x] Support multiple parents
[ ] Support updates across multiple tiles
[x] Update Cesium3DTileBatchTable.hasProperty, Batched3DModel3DTileContent.hasProperty, etc to take the hierarchy into account. It will need to accept a batchId argument because not all features have the same properties anymore. Or this doesn't change and just continues to check only base-level batch table properties (like area and height in the sample below).
[x] Same idea for getPropertyNames
[ ] For parity with the styling language the Cesium API could include functions like

getFeaturesByProperty(propertyName, value)
getFeaturesByClass(className)
There can be many variants to these. It may be best to leave these out of the Cesium API and let the apps themselves do this checking.

[ ] Add documentation
[x] Better error checking

[x] Recognize cyclical dependencies? Or let validator point that out?

[ ] Add tests
[ ] Naming
[ ] Think about supporting for GLSL backend

[ ] If not supported, write tests that disallow any of the new functions like isClass and isDerived

Questions:

Should hierarchy properties exist in the tileset.json properties section?
Currently a parentId of -1 signifies no parent. However this makes it harder to store the parentIds in binary unless the only allowed componentType are SHORT or INT.
Is it useful to have a ${CLASS} property for styling? This would allow regular expressions against the class name. Or is isClass good enough?
Should a feature be able to set a property of its ancestor's class. Say the feature is a door, should feature.setProperty('building_name', 'new_name') be allowed? Currently this is supported in the code and is probably ok because otherwise we would need some other interface to represent abstract instances - like an analog to Cesium3DTileFeature, and that can get complicated.

Sample hierarchy:
Doorknobs are children of doors
zone0
  building0
    roof0
    wall0
    door0 - doorknob0
    door1 - doorknob1
    door2 - doorknob2
    door3 - doorknob3
  building1
    roof1
    wall1
    door4 - doorknob4
    door5 - doorknob5
    door6 - doorknob6
    door7 - doorknob7
  building2
    roof2
    wall2
    door8 - doorknob8
    door9 - doorknob9
    door10 - doorknob10
    door11 - doorknob11

Sample batch table json:
{
    "height": [0.1, 0.1, 0.1, 0.1, 5, 5, 5, 5, 10, 6, 0.1, 0.1, 0.1, 0.1, 5, 5, 5, 5, 10, 6, 0.1, 0.1, 0.1, 0.1, 5, 5, 5, 5, 10, 6],
    "area": [0.2, 0.2, 0.2, 0.2, 10, 10, 10, 10, 20, 12, 0.2, 0.2, 0.2, 0.2, 10, 10, 10, 10, 20, 12, 0.2, 0.2, 0.2, 0.2, 10, 10, 10, 10, 20, 12],
    "HIERARCHY": {
        "instancesLength": 34,
        "classes": [
            {
                "name": "doorknob",
                "length": 12,
                "instances": {
                    "doorknob_size": [0.3, 0.43, 0.32, 0.2, 0.21, 0.35, 0.3, 0.23, 0.43, 0.32, 0.41, 0.33],
                    "doorknob_name": ["doorknob0", "doorknob1", "doorknob2", "doorknob3", "doorknob4", "doorknob5", "doorknob6", "doorknob7", "doorknob8", "doorknob9", "doorknob10", "doorknob11"]
                }
            },
            {
                "name": "door",
                "length": 12,
                "instances": {
                    "door_mass": [10, 11, 14, 7, 8, 12, 3, 6, 3, 5, 9, 10],
                    "door_width": [1.2, 1.3, 1.21, 1.5, 1.1, 1.15, 1.32, 1.54, 1.8, 2, 2.1, 1.3],
                    "door_name": ["door0", "door1", "door2", "door3", "door4", "door5", "door6", "door7", "door8", "door9", "door10", "door11"]
                }
            },
            {
                "name": "roof",
                "length": 4,
                "instances": {
                    "roof_paint": ["red", "blue", "yellow", "green"],
                    "roof_name": ["roof0", "roof1", "roof2", "roof3"]
                }
            },
            {
                "name": "wall",
                "length": 4,
                "instances": {
                    "wall_paint": ["pink", "orange", "blue", "gray"],
                    "wall_windows": [1, 2, 4, 2],
                    "wall_name": ["wall0", "wall1", "wall2", "wall3"]
                }
            },
            {
                "name": "building",
                "length": 4,
                "instances": {
                    "building_area": [20, 21.98, 39.3],
                    "building_name": ["building0", "building1", "building2"]
                }
            },
            {
                "name": "zone",
                "length": 1,
                "instances": {
                    "zone_buildings": [4],
                    "zone_name": ["zone0"]
                }
            }
        ],
        "classId": [0, 0, 0, 0, 1, 1, 1, 1, 2, 3, 0, 0, 0, 0, 1, 1, 1, 1, 2, 3, 0, 0, 0, 0, 1, 1, 1, 1, 2, 3, 4, 4, 4, 5],
        "parentId": [4, 5, 6, 7, 30, 30, 30, 30, 30, 30, 14, 15, 16, 17, 31, 31, 31, 31, 31, 31, 24, 25, 26, 27, 32, 32, 32, 32, 32, 32, 33, 33, 33, -1]
    }
}

Same example using the batch table binary:
{
    "height": {
        "byteOffset": 0,
        "componentType": "FLOAT",
        "type": "SCALAR"
    },
    "area": {
        "byteOffset": 120,
        "componentType": "FLOAT",
        "type": "SCALAR"
    },
    "HIERARCHY": {
        "instancesLength": 34,
        "classes": [
            {
                "name": "doorknob",
                "length": 12,
                "instances": {
                    "doorknob_size": {
                        "byteOffset": 512,
                        "componentType": "FLOAT",
                        "type": "SCALAR"
                    },
                    "doorknob_name": ["doorknob0", "doorknob1", "doorknob2", "doorknob3", "doorknob4", "doorknob5", "doorknob6", "doorknob7", "doorknob8", "doorknob9", "doorknob10", "doorknob11"]
                }
            },
            {
                "name": "door",
                "length": 12,
                "instances": {
                    "door_mass": {
                        "byteOffset": 560,
                        "componentType": "FLOAT",
                        "type": "SCALAR"
                    },
                    "door_width": {
                        "byteOffset": 608,
                        "componentType": "FLOAT",
                        "type": "SCALAR"
                    },
                    "door_name": ["door0", "door1", "door2", "door3", "door4", "door5", "door6", "door7", "door8", "door9", "door10", "door11"]
                }
            },
            {
                "name": "roof",
                "length": 4,
                "instances": {
                    "roof_paint": ["red", "blue", "yellow", "green"],
                    "roof_name": ["roof0", "roof1", "roof2", "roof3"]
                }
            },
            {
                "name": "wall",
                "length": 4,
                "instances": {
                    "wall_paint": ["pink", "orange", "blue", "gray"],
                    "wall_windows": {
                        "byteOffset": 656,
                        "componentType": "FLOAT",
                        "type": "SCALAR"
                    },
                    "wall_name": ["wall0", "wall1", "wall2", "wall3"]
                }
            },
            {
                "name": "building",
                "length": 4,
                "instances": {
                    "building_area": {
                        "byteOffset": 672,
                        "componentType": "FLOAT",
                        "type": "SCALAR"
                    },
                    "building_name": ["building0", "building1", "building2"]
                }
            },
            {
                "name": "zone",
                "length": 1,
                "instances": {
                    "zone_buildings": {
                        "byteOffset": 684,
                        "componentType": "FLOAT",
                        "type": "SCALAR"
                    },
                    "zone_name": ["zone0"]
                }
            }
        ],
        "classIds": {
            "byteOffset": 240,
            "componentType": "INT"
        },
        "parentIds": {
            "byteOffset": 376,
            "componentType": "INT"
        }
    }
}

Styling a feature based on ancestor properties: "show" : "${building_name} === 'building0'" will style all descendants of building0

Do you mean "style building0 and all descendants of building0?"

CC @pmconne

isClass - same as above - isClass('door') - styles just doors
isDerived - same as above - isDerived('door') - styles doors and doorknobs

Are you happy with these names?  isClass is awkward because, for example, with parent-child A->B, a B still is an A.  Likewise, isDerived is awkward because B is not derived from B.
Did you have alternatives perhaps borrowed from popular OO languages?  isExactClass and isClass come to mind but better names may exist.

Update Cesium3DTileBatchTable.hasProperty, Batched3DModel3DTileContent.hasProperty, etc to take the hierarchy into account. It will need to accept a batchId argument because not all features have the same properties anymore. Or this doesn't change and just continues to check only base-level batch table properties (like area and height in the sample below).

We can brainstorm in person, but ideally we can expose each feature's full set of properties just like C++ inheritance puts everything into the derived class.  If there are real performance or synchronization problems, maybe we would rely on the user being able to get parent classes and read/write those properties.

For parity with the styling language the Cesium API could include functions like

getFeaturesByProperty(propertyName, value)
getFeaturesByClass(className)
There can be many variants to these. It may be best to leave these out of the Cesium API and let the apps themselves do this checking.

Let's roadmap these; I suspect they will be useful, but let's wait until requested and perhaps they will be good beginner issues or external contributions.

Recognize cyclical dependencies? Or let validator point that out?

We probably have to detect this in Cesium (and in the validator); for example, we do this for shaders: https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Renderer/ShaderSource.js#L129

Think about supporting for GLSL backend

Let's discuss in person when you are ready, but I think it can be done by storing class info in the batch table; we need to think through the memory usage.

Should hierarchy properties exist in the tileset.json properties section?

I suspect all properties should be listed there.  We may need to update the wording in the spec to clarify this.

Currently a parentId of -1 signifies no parent. However this makes it harder to store the parentIds in binary unless the only allowed componentType are SHORT or INT.

Maybe use MAX_TYPE, but I suspect that will be error prone for both the client and server.
There could also be a dummy parent (all classes inherit from Class) that is index 0.
Let me know if you want to discuss in person.

Is it useful to have a ${CLASS} property for styling? This would allow regular expressions against the class name. Or is isClass good enough?

${CLASS} is likely useful for the gmail labels use case where there could be a ton of labels.
If it is not hard to implement, I suggest we do it.

Should a feature be able to set a property of its ancestor's class. Say the feature is a door, should feature.setProperty('building_name', 'new_name') be allowed? Currently this is supported in the code and is probably ok because otherwise we would need some other interface to represent abstract instances - like an analog to Cesium3DTileFeature, and that can get complicated.

This is what I meant above:

We can brainstorm in person, but ideally we can expose each feature's full set of properties just like C++ inheritance puts everything into the derived class. If there are real performance or synchronization problems, maybe we would rely on the user being able to get parent classes and read/write those properties.

We need to think through basically if each derived class gets a copy of its parent's properties or not.  It sounds like no in the current implementation.  That could be fine, but let's discuss in person to consider all the use cases.

Looking good so far.  Love the Sandcastle example.

Did you have alternatives perhaps borrowed from popular OO languages? isExactClass and isClass come to mind but better names may exist.

Ruby has instance_of to represent isExactClass and kind_of/is_a to represent isClass.
Java has obj.getClass().equals(Foo.class) for isExactClass and instanceof/isInstance for isClass.
C# is somewhat similar to Java
At this point I'm fine with using isExactClass and isClass

Think about supporting for GLSL backend

Thinking some more, I don't believe this is needed in the GLSL backend. For points at least, it would seem kind of overkill to store class ids and parent ids for every point. If someone needs to represent hierarchies within point clouds they can create a batched point cloud (setting BATCH_ID in the feature table), which uses Cesium3DTileBatchTable under the hood and not the GLSL backend.

Maybe use MAX_TYPE, but I suspect that will be error prone for both the client and server.
There could also be a dummy parent (all classes inherit from Class) that is index 0.

As discussed offline, we may try something else. Instead of using -1, if the parent id equals itself then it does not have a parent. This allows unsigned types to be used and is possibly simpler than a dummy at index 0.

${CLASS} is likely useful for the gmail labels use case where there could be a ton of labels.

Instead this may become a built-in function, like getClassName.

We need to think through basically if each derived class gets a copy of its parent's properties or not. It sounds like no in the current implementation. That could be fine, but let's discuss in person to consider all the use cases.

Yes, the current implementation does not create copies.

We can brainstorm in person, but ideally we can expose each feature's full set of properties just like C++ inheritance puts everything into the derived class. If there are real performance or synchronization problems, maybe we would rely on the user being able to get parent classes and read/write those properties.

Supporting .parent and .children along with a Cesium3DTileInstance type class seems doable. This would act similar to the existing system, but in a more organized way, but also more complexity. I want to wait a bit on this though in case I have other ideas.

All sounds good, except:

Thinking some more, I don't believe this is needed in the GLSL backend. For points at least, it would seem kind of overkill to store class ids and parent ids for every point. If someone needs to represent hierarchies within point clouds they can create a batched point cloud (setting BATCH_ID in the feature table), which uses Cesium3DTileBatchTable under the hood and not the GLSL backend.

So point cloud users would use BATCH_ID to emulate a hierarchy batch table?  Instead, could the hierarchy batch table replace the BATCH_ID.
If not, update https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/140 with this limitation.

So point cloud users would use BATCH_ID to emulate a hierarchy batch table? Instead, could the hierarchy batch table replace the BATCH_ID.

More like, the BATCH_ID attribute and batch table hierarchy work together just like it would for batched models.

More like, the BATCH_ID attribute and batch table hierarchy work together just like it would for batched models.

If I understand correctly, your saying that the Cesium implementation falls back to the JavaScript backend in this case so it just works?

Yes

Updated based on the initial round of comments. It now supports multiple parents. In the process I generalized the hierarchy traversal and added back hasProperty and getPropertyNames both of which now require a batchId parameter.

building0 has two parents - zone0 and classifier_new
buiding1 has two parents - zone0 and classifier_old
building3 has three parents - zone0, classifier_new, and classifier_old
{
    "color" : {
        "conditions" : [
            ["isClass('classifier_new') && isClass('classifier_old')", "color('purple')"],
            ["isClass('classifier_new')", "color('red')"],
            ["isClass('classifier_old')", "color('blue')"],
            ["true", "color()"]
        ]
    }
}

Sample JSON:
{
    "HIERARCHY": {
        "instancesLength": 36,
        "classes": [
            {
                "name": "doorknob",
                "length": 12,
                "instances": {
                    "doorknob_size": [0.3, 0.43, 0.32, 0.2, 0.21, 0.35, 0.3, 0.23, 0.43, 0.32, 0.41, 0.33],
                    "doorknob_name": ["doorknob0", "doorknob1", "doorknob2", "doorknob3", "doorknob4", "doorknob5", "doorknob6", "doorknob7", "doorknob8", "doorknob9", "doorknob10", "doorknob11"]
                }
            },
            {
                "name": "door",
                "length": 12,
                "instances": {
                    "door_mass": [10, 11, 14, 7, 8, 12, 3, 6, 3, 5, 9, 10],
                    "door_width": [1.2, 1.3, 1.21, 1.5, 1.1, 1.15, 1.32, 1.54, 1.8, 2, 2.1, 1.3],
                    "door_name": ["door0", "door1", "door2", "door3", "door4", "door5", "door6", "door7", "door8", "door9", "door10", "door11"]
                }
            },
            {
                "name": "roof",
                "length": 4,
                "instances": {
                    "roof_paint": ["red", "blue", "yellow", "green"],
                    "roof_name": ["roof0", "roof1", "roof2", "roof3"]
                }
            },
            {
                "name": "wall",
                "length": 4,
                "instances": {
                    "wall_paint": ["pink", "orange", "blue", "gray"],
                    "wall_windows": [1, 2, 4, 2],
                    "wall_name": ["wall0", "wall1", "wall2", "wall3"]
                }
            },
            {
                "name": "building",
                "length": 4,
                "instances": {
                    "building_area": [20, 21.98, 39.3],
                    "building_name": ["building0", "building1", "building2"]
                }
            },
            {
                "name": "zone",
                "length": 1,
                "instances": {
                    "zone_buildings": [4],
                    "zone_name": ["zone0"]
                }
            },
            {
                "name": "classifier_new",
                "length": 1,
                "instances": {
                    "year": [2000],
                    "color": ["red"],
                    "name": ["project"],
                    "architect": ["architect"]
                }
            },
            {
                "name": "classifier_old",
                "length": 1,
                "instances": {
                    "description": ["built in 1980"],
                    "inspection": [2009]
                }
            }
        ],
        "classIds": [0, 0, 0, 0, 1, 1, 1, 1, 2, 3, 0, 0, 0, 0, 1, 1, 1, 1, 2, 3, 0, 0, 0, 0, 1, 1, 1, 1, 2, 3, 4, 4, 4, 5, 6, 7],
        "parentIds": [4, 5, 6, 7, 30, 30, 30, 30, 30, 30, 14, 15, 16, 17, 31, 31, 31, 31, 31, 31, 24, 25, 26, 27, 32, 32, 32, 32, 32, 32, 33, 34, 33, 35, 33, 34, 35],
        "parentCounts": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 3, 0, 0, 0]
    },
    "height": [0.1, 0.1, 0.1, 0.1, 5, 5, 5, 5, 6, 10, 0.1, 0.1, 0.1, 0.1, 5, 5, 5, 5, 6, 10, 0.1, 0.1, 0.1, 0.1, 5, 5, 5, 5, 6, 10],
    "area": [0.2, 0.2, 0.2, 0.2, 10, 10, 10, 10, 12, 20, 0.2, 0.2, 0.2, 0.2, 10, 10, 10, 10, 12, 20, 0.2, 0.2, 0.2, 0.2, 10, 10, 10, 10, 12, 20]
}

There's a JSHint warning: https://travis-ci.org/AnalyticalGraphicsInc/cesium/builds/177147085

Is the tasklist at the top up-to-date?

Looking good.
Do you think all the corner cases for the semantic hierarchy are sufficiently covered?  Do we need one with multiple diamonds?

This is ready with the exception of Support updates across multiple tiles which is very involved and possibly not feasible to support.

If the marker is reset then the array also needs to be reset, which I'm trying to avoid. Rather than resetting I just increment it so on the second call tiles that are visited are marked 2, on the third call they are marked as 3, etc.

Is the array likely to grow very large over time?

I can't think of a great way that doesn't involve closures since name has to get the the function in same way. Possibly using scratch options objects and pass that to traverseHierarchy as a third parameter would work, but is not as clean.

OK, just leave it.  We can reevaluate if needed.

Will aim to do another review tomorrow.

This is ready with the exception of Support updates across multiple tiles which is very involved and possibly not feasible to support.

OK for now since this is only a Cesium API issue.
What is the state of this?  I would be OK with all classes being readonly in the Cesium API so the only thing that can change is the leaf nodes, which are the actual features.

Looking good, just those comments!

Is the array likely to grow very large over time?

Not too large. At most the size of the array will equal the tile with the most instances.

What is the state of this? I would be OK with all classes being readonly in the Cesium API so the only thing that can change is the leaf nodes, which are the actual features.

This might be a good idea... I'm okay with the keeping the code as it is now since it gives a little bit more flexibility within the tile. But if others are confused that setting hierarchy properties don't apply to different tiles then it may be worth making it read only.

Updated.

What is the state of this? I would be OK with all classes being readonly in the Cesium API so the only thing that can change is the leaf nodes, which are the actual features.

This might be a good idea... I'm okay with the keeping the code as it is now since it gives a little bit more flexibility within the tile. But if others are confused that setting hierarchy properties don't apply to different tiles then it may be worth making it read only.

OK, let's go with readonly.  Freeze it too.  Will this remove the clone, https://github.com/AnalyticalGraphicsInc/cesium/pull/4625#discussion_r89315401

Is the array likely to grow very large over time?

Not too large. At most the size of the array will equal the tile with the most instances.

That's actually kind of a lot - could be, for example, 50,000.  Assuming there is no reasonable way around this, please add a comment about the potential bounds.

OK, let's go with readonly. Freeze it too. Will this remove the clone, #4625 (comment)

The whole hierarchy can't be frozen since you are still allowed to set properties on an instance's own class. Clone still needs to exist just like it does with the normal setProperty. I'm now just throwing a developer error if it detects that you are setting an inherited property.

This is ready.

Very nice, looking forward to the spec updates


Implement SSZ

Implement SSZ in accordance with the spec listed here. Additional resources can be found here here

@dangerousfood What is the status of this?  Do you have an implementation in Java, or are you going to use Antoine's version in Cava?


Cannot read property time of null

This package looks perfect for what I need, but I hit issues trying to use the example w/ react-native.
Redbox:

Also noticed the example code is missing a closing ``` tick in the onRender method

Hey mate,
I don't think the problem is ReactNative (thats what i used this for too).
Had a quick look at the source, and the documentation could be wrong.
Try changing the prop 'endTime' to 'endsAt'.
lmk how you go.

endsAt works but that conflicts with the readme?
Also, the docs show this.state.completed which isn't defined on state?
Confused...

It appears I wrote the docs incorrectly. I have update it to reflect the endsAt value in the code.
this state.completed is set inside the tick function, when the timer runs out.


how to use it

how am I supposed to use it there are no kexts files, I am totally new to this and want to use it with OpenCore can anybody help me with that also will it work with Intel Corporation Centrino Advanced-N 6205 [Taylor Peak]

nevermind I found the release section sorry my bad.


Add support for suffix="collection" routes

Falcon allows defining one resource class that provides multiple routes:
  suffix (str): Optional responder name suffix for this
                route. If a suffix is provided, Falcon will map GET
                requests to ``on_get_{suffix}()``, POST requests to
                ``on_post_{suffix}()``, etc. In this way, multiple
                closely-related routes can be mapped to the same
                resource. For example, a single resource class can
                use suffixed responders to distinguish requests for
                a single item vs. a collection of those same items.
                Another class might use a suffixed responder to handle
                a shortlink route in addition to the regular route for
                the resource.

Is there a way to have these routes appear in the generated spec?

I am currently not using Falcon at work -- would love help in getting this feature implemented!
Please let me know if you need a hand in getting started.

there is probably a way to get the suffixes directly from a prior app.add_route("/random", random_pet_resource, suffix="root") call

I couldn't find a way to get that from falcon's routes, so I went another way - subclass falcon.API to override add_route and keep a list of added routes (path, resource and suffix). Then I just call spec.path for each route added to the API:
for uri_template, resource, suffix, include_in_spec in api.routes:
        if include_in_spec:
            spec.path(uri_template, resource=resource, suffix=suffix)

Because one resource can handle multiple paths now, I keep track of which resources have already been seen so it doesn't add duplicates to the spec. Cleaned up a bit, this is what I ended up with:

Show code

class FalconPlugin(apispec.BasePlugin):
    """
    APISpec plugin for Falcon resources
    docs: https://apispec.readthedocs.io/en/latest/

    API doc GUI rendered with: https://github.com/swagger-api/swagger-ui
    """

    param_regex = re.compile(r"\{(\w+)\}")

    default_error_responses = {
        "400": {"description": "Bad request"},
        "401": {"description": "Unauthorized"},
        "404": {"description": "Not found"},
        "429": {"description": "Too many requests (rate limit exceeded)"},
        "5xx": {"description": "Unexpected error"},
    }

    def __init__(self, app):
        super().__init__()
        self.app = app  # currently unused
        self.known_resources = set()

    def init_spec(self, spec):
        super().init_spec(spec)
        self.spec = spec

    def path_helper(
        self, path, operations, parameters, resource, suffix=None, **kwargs
    ):
        """Path helper that allows passing a Falcon resource instance.

        This uses some assumptions about our specific implementation to provide smart
        defaults for API documentation.
        """

        suffix = "" if suffix is None else "_" + suffix
        operations.update(
            yaml_utils.load_operations_from_docstring(resource.__doc__) or {}
        )

        # create tags from resource class docstrings
        # swagger-ui uses tags for grouping
        resource_doc = yaml_utils.load_yaml_from_docstring(resource.__doc__)
        tag_name = resource_doc.get("name")

        # one resource class can be used for multiple paths, so keep track of which
        # which we've seen to prevent adding duplicates to the spec
        if resource not in self.known_resources:
            self.known_resources.add(resource)
            if tag_name:
                description = resource_doc.get("description")
                self.spec.tag({"name": tag_name, "description": description})

        # parameters for the path apply to each operation, unless overridden
        path_parameters = resource_doc.get("parameters") or []

        # automatically add placeholder path parameters if not manually documented so
        # you can at least enter a value to call the endpoint through swagger-ui
        for name in self.param_regex.findall(path):
            if not any(param["name"] == name for param in path_parameters):
                param_doc = {
                    "name": name,
                    "in": "path",
                    "required": True,
                }

                if name.endswith("_id"):
                    # pretty safe guess, if it's wrong just define the parameter yourself
                    param_doc["schema"] = {"type": "integer"}

                path_parameters.append(param_doc)

        parameters.extend(path_parameters)

        for method in falcon.constants.HTTP_METHODS:
            http_verb = method.lower()
            method_name = "on_" + http_verb + suffix
            method = getattr(resource, method_name, None)
            if method:
                method_doc = yaml_utils.load_yaml_from_docstring(method.__doc__) or {}

                assert isinstance(
                    method_doc, dict
                ), f"incorrect docstring format for {method}"

                # unless explicitly defined, group the method under the resource
                if tag_name and "tags" not in method_doc:
                    method_doc["tags"] = [tag_name]
                operations[http_verb] = method_doc

                # If you don't specify responses, swagger-ui won't show any "try it out"
                # response, not even just a "200 everything is great, here's some JSON".

                # We could either always include the default responses, or leave them
                # out entirely if a resource defines its own. For now, always add the
                # defaults unless they have been overwritten by something more specific
                # on the resource itself.

                method_responses = method_doc.setdefault("responses", {})

                def add_defaults(defaults):
                    for k, v in defaults.items():
                        # dict keys in YAML that look like ints are parsed into actual ints!
                        try:
                            int_k = int(k)
                        except ValueError:
                            int_k = k

                        if k not in method_responses and int_k not in method_responses:
                            method_doc["responses"][k] = v

                add_defaults(self.default_error_responses)
                if http_verb == "post":
                    add_defaults({"201": {"description": "Created"}})
                else:
                    add_defaults({"200": {"description": "OK"}})

        return path

Feel free to steal what you feel is useful :)

I second @chretienmn on this, we want to use the plugin but need the feature. If you don't mind I can implement it this weekend.

@hharutyunyan would love help getting this feature completed. I can review and cut a release over the next week.

@alysivji Added the feature, pls have a look

Saw the PR -- will review it this week. Thanks for your help!

Just cut a release and uploaded it to PyPI.
Thanks @hharutyunyan for making the fix!

Just cut a release and uploaded it to PyPI.
Thanks @hharutyunyan for adding support for suffixes!

Thank you too @alysivji

@hharutyunyan the issue now is that only one path per resource is mapped. All paths should be mapped for a given resource since that's why suffixes were added to Falcon in the first place.
I'm thinking calls to spec.path for multiple paths should look like:
spec.path('/accounts', resource=accounts)
spec.path('/accounts/{account_id}', resource=accounts)

And the plugin should be more like:
class FalconPlugin(BasePlugin):
    """APISpec plugin for Falcon"""

    def __init__(self, app):
        super(FalconPlugin, self).__init__()
        self._app = app

    @staticmethod
    def _generate_resource_uri_mapping(app):
        routes_to_check = copy.copy(app._router._roots)

        mapping = {}
        for route in routes_to_check:
            uri = route.uri_template
            resource = route.resource
            if resource not in mapping:
                mapping[resource] = {}
            if uri not in mapping[resource]:
                mapping[resource][uri] = {}

            if route.method_map:
                for method_name, method_handler in route.method_map.items():
                    if method_handler.__dict__.get("__module__") == "falcon.responders":
                        continue
                    mapping[resource][uri][method_name.lower()] = method_handler

            routes_to_check.extend(route.children)
        return mapping

    def path_helper(self, path, operations, resource, base_path=None, **kwargs):
        """Path helper that allows passing a Falcon resource instance."""
        resource_uri_mapping = self._generate_resource_uri_mapping(self._app)
        if resource not in resource_uri_mapping:
            raise APISpecError("Could not find endpoint for resource {0}".format(resource))
        if path not in resource_uri_mapping[resource]:
            raise APISpecError("Could not find path {0} for resource {1}".format(path, resource))

        operations.update(yaml_utils.load_operations_from_docstring(resource.__doc__) or {})

        if base_path is not None:
            # make sure base_path accept either with or without leading slash
            # swagger 2 usually come with leading slash but not in openapi 3.x.x
            base_path = '/' + base_path.strip('/')
            path = re.sub(base_path, "", path, 1)

        methods = resource_uri_mapping[resource][path]

        for method_name, method_handler in methods.items():
            docstring_yaml = yaml_utils.load_yaml_from_docstring(method_handler.__doc__)
            operations[method_name] = docstring_yaml or dict()
        return path

@alysivji should I create a new issue for this?

An issue would be great!
Also looks like there is a solution mostly implemented above. Thank you for that! If you have some time and can write up an example, we can create a test case and get this merged.


set environment to use etcd3 api

What this PR does / why we need it:
Set ETCDCTL_API=3 to ensure etcdctl uses the right api endpoints
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #3444
Special notes for your reviewer:
Please see question on 3444, I am curious about other /opt/azure environment files, and if we need to maintain backwards compatibility.
If applicable:

[ ] documentation
[ ] unit tests
[ ] tested backward compatibility (ie. deploy with previous version, upgrade with this branch)

Release note:

@ryanhartje I think all the other pre-reqs are still // TODOs. I infer from @CecileRobertMichon's comments that this env var needs to be set as we install etcd so that it bootstraps itself to use the v3 configs

accidentally pulled in other commits into this branch, going to reopen for maximum cleanliness.


GmLive doesn't work

When I compile with GmLive open GmLive never realizes the game compiling and never attempts to connect to anything.

I've never used GMLive unfortunately so am not aware of how it starts its server or what it expects from Igor. I do own the asset so could look into it, but that will likely be a good while away as I've got very limited time of late to work on Constructor.


Change behavior of complete_slice to fit numpy 1.12.0

Change behavior of complete_slice to fit numpy 1.12.0

Check flake8

LGTM


[zeromq] Fix mingw build issues (24711)

What does your PR fix?

https://github.com/microsoft/vcpkg/issues/24711#issue-1235979108

Fix: zeromq build failure on mingw

Added -DZMQ_HAVE_IPC=0 to PLATFORM_OPTIONS,

Fix: zeromq warning on builds

Set dynamic LINKAGE_OPTIONS flag list:

WITH_PERF_TOOL option only used if -DBUILD_SHARED=ON:

https://github.com/zeromq/libzmq/blob/master/CMakeLists.txt#L1558

option values are set explicitly to avoid issues with future libzmq changes

Which triplets are supported/not supported? Have you updated the CI baseline?

No changes were made to ci.baseline.txt

Tested with
# MSYS2: mingw64 env, toolchain v11.3.0
pacman -S --needed base-devel mingw-w64-x86_64-toolchain
./vcpkg.exe install zeromq:x86-mingw-static zeromq:x64-mingw-static zeromq:x64-mingw-dynamic zeromq:x86-windows zeromq:x64-windows zeromq:x64-windows-static

# MSYS2: mingw32 env, toolchain v11.3.0
pacman -S --needed base-devel mingw-w64-i686-toolchain
./vcpkg.exe install zeromq:x86-mingw-dynamic
# mingw32 required to avoid incorrect architecture with -DBUILD_SHARED=ON: "Expected x86, but was: x64"

Does your PR follow the maintainer guide?

yes

If you have added/updated a port: Have you run ./vcpkg x-add-version --all and committed the result?

yes

Problem

The USE_PERF_TOOLS flag is only used, and should only be set if -DBUILD_SHARED=ON:

https://github.com/zeromq/libzmq/blob/master/CMakeLists.txt#L1558
https://github.com/zeromq/libzmq/blob/master/CMakeLists.txt#L1569
https://github.com/zeromq/libzmq/blob/master/CMakeLists.txt#L1621

message(FATAL_ERROR "Shared library disabled - perf-tools unavailable.")
Wrongly indented but clearly state the intent

The USE_PERF_TOOLS flag was always declared, triggering warnings on some builds, as I mentioned in https://github.com/microsoft/vcpkg/issues/24711#issue-1235979108
So I would disagree with the statement that the "The only valid part is..." (https://github.com/microsoft/vcpkg/pull/24782#discussion_r876631378), and all CI checks passed.

Solution

Dynamically declare the USE_PERF_TOOLS flag if -DBUILD_SHARED=ON

ON being the default option value: https://github.com/zeromq/libzmq/blob/master/CMakeLists.txt#L1304

But as I said, to keep the port future proof these default option values shouldn't be used, as they are subject to changes.

Current implementation

With the intent of saving space, seemingly a concern in other files and PRs:

The USE_PERF_TOOLS flag is only declared with dynamic builds, instead of directly only when -DBUILD_SHARED=ON
Single lines are used to declare LINKAGE_OPTIONS flags
Comments aren't used, as it's almost never done in other ports, if deliberately to save space I will mention:

Better to lose a couple of lines in a file, than a couple of hours going through commits and PRs to understand choices

I'm not sure what is meant by "more clearly", I shared the relevant, libzmq CMakeLists lines, went into more details in the issue (https://github.com/microsoft/vcpkg/issues/24711#issue-1235979108) and this message, but I agree with long lines being indigest, redeclarations too, even dangerous.
For these reasons I would propose the following changes.

Proposed changes

1: Both dynamically set and declared flags based on VCPKG_LIBRARY_LINKAGE
string(COMPARE EQUAL "${VCPKG_LIBRARY_LINKAGE}" "static" BUILD_STATIC)
string(COMPARE EQUAL "${VCPKG_LIBRARY_LINKAGE}" "dynamic" BUILD_SHARED)

set(LINKAGE_OPTIONS
    -DWITH_LIBSODIUM_STATIC=${BUILD_STATIC}
    -DBUILD_STATIC=${BUILD_STATIC}
    -DBUILD_SHARED=${BUILD_SHARED}
)
if(BUILD_SHARED)
    list(APPEND LINKAGE_OPTIONS -DWITH_PERF_TOOL=OFF)
endif()

2: With a comment on USE_PERF_TOOLS
string(COMPARE EQUAL "${VCPKG_LIBRARY_LINKAGE}" "static" BUILD_STATIC)
string(COMPARE EQUAL "${VCPKG_LIBRARY_LINKAGE}" "dynamic" BUILD_SHARED)

set(LINKAGE_OPTIONS
    -DWITH_LIBSODIUM_STATIC=${BUILD_STATIC}
    -DBUILD_STATIC=${BUILD_STATIC}
    -DBUILD_SHARED=${BUILD_SHARED}
)
if(BUILD_SHARED)
    # The USE_PERF_TOOLS option flag should only be set if -DBUILD_SHARED=ON:
    # - <https://github.com/zeromq/libzmq/blob/master/CMakeLists.txt#L1621>
    list(APPEND LINKAGE_OPTIONS -DWITH_PERF_TOOL=OFF)
endif()

LINKAGE_OPTIONS could also be renamed.

Waiting for responses on the prefered ways to handle this.

Thank you for your quick review.
Additional notes on the issues I fixed:

The breaking changes were introduced the Apr 30, 2021 or before, more than a year ago, with a libzmq update
The warning issue was probably there for years.
My PR was functionally valid from the start, it was trivial but challenged, declared invalid, for not following arbitrary undocumented styling rules, as a show of good faith I detailed my design choices as best I could, with an interest in learning more which wasn't met.

@k-128 Did you intentionally close the PR? It is not merged yet.

The breaking changes were introduced the Apr 30, 2021 or before, more than a year ago, with a libzmq update

I take this as an indication that the vcpkg port doesn't  have users on mingw. Mingw support is mostly community-based here.

The warning issue was probably there for years.

The maintainer function which issues the warning learned this in the past 12 months, complemented by the MAYBE_UNUSED_VARIABLES option. These new warnings are usually adressed with other updates to the port,

My PR was functionally valid from the start, it was trivial but challenged, declared invalid, for not following arbitrary undocumented styling rules, as a show of good faith I detailed my design choices as best I could, with an interest in learning more which wasn't met.

I'm sure your PR is appreciated even if it the feedback didn't come as positively as you might have expected.
In fact I also thought hat initial PR seemed to change more than necessary for the things it tried to fix. This port has patterns which are also found in other ports, and retaining the patterns helps with maintenance.
What I can add is that I would prefer to see the warning resolved with MAYBE_UNUSED_VARIABLES instead of putting WITH_PERF_TOOL into LINKAGE_OPTIONS.

Closed it thinking I was closing the issue my bad...

I take this as an indication that the vcpkg port doesn't have users on mingw.

My thoughts too, or they fix their issues privately without contributing.

What I can add is that I would prefer to see the warning resolved with MAYBE_UNUSED_VARIABLES instead of putting WITH_PERF_TOOL into LINKAGE_OPTIONS.

I mentioned this solution in: https://github.com/microsoft/vcpkg/pull/24782#issuecomment-1131605514

"A less explicit solution could be to use vcpkg_cmake_configure MAYBE_UNUSED_VARIABLES to declare USE_PERF_TOOLS"

But a less obvious potential future issue could be the mistake I mentioned in libzmq CMakeLists.txt:

https://github.com/zeromq/libzmq/blob/master/CMakeLists.txt#L1621

  if(WITH_PERF_TOOL)
    ...
  elseif(WITH_PERF_TOOL)
    message(FATAL_ERROR "Shared library disabled - perf-tools unavailable.")
  endif()

The indentation is wrong, what was meant was:

if(BUILD_SHARED)
  ...
elseif(WITH_PERF_TOOL)
  message(FATAL_ERROR "Shared library disabled - perf-tools unavailable.")
endif()

If this gets fixed in libzmq, the declaration of WITH_PERF_TOOL=ON with BUILD_SHARED=OFF will trigger a FATAL_ERROR:

CMake Error, stop processing and generation, on builds (https://cmake.org/cmake/help/latest/command/message.html)

I'm sure your PR is appreciated even if it the feedback didn't come as positively as you might have expected.
In fact I also thought that the initial PR seemed to change more than necessary for the things it tried to fix. This port has patterns which are also found in other ports, and retaining the patterns helps with maintenance.

I was mainly looking for clarity, to get as many questions out of the way if I ever wanted to do another PR. I did actually try to inspire myself from multiple other ports, hence the mentioned note, raising the possibility of standardizing and documenting styling practices, I didn't like the look of my initial PR either, but it was functional.
Thank you for taking the time to go through all that.

So is there any conclusion here?

Ping for response.

@JackBoosY What is your opinion with regard to MAYBE_UNUSED_VARIABLES?

@JackBoosY What is your opinion with regard to MAYBE_UNUSED_VARIABLES?

We must verify the correctness of MAYBE_UNUSED_VARIABLES every time the port is changed, remove the expired items and add the items reported in the output.

We must verify the correctness of MAYBE_UNUSED_VARIABLES every time the port is changed, remove the expired items and add the items reported in the output.

What does this mean with regard to this PR?


Added the String Contains Ignorecase criteria support

Code changes for the Contains Ignore case criteria support

Can I be so bold to ask if you could make that last commit into a new pull request?

Seems there is an elasticsearch2 integration in the code base now: https://github.com/querydsl/querydsl-contrib/tree/master/querydsl-elasticsearch2
So we just need the two first commits now?


Custom measurement docs should indicate that only numeric values are supported

Azure documentation issue guidance
Thanks for opening an issue in the Azure technical documentation repository.
We use GitHub issues as the primary channel for customer and community feedback about the Azure documentation.
Creating an issue
We prefer that you create documentation feedback issues using the Feedback link on the published article - the feedback control on the doc page creates an issue that contains all the article details so you can focus on the feedback part.
You can also create a feedback issue here in the repo. If you do this, please make sure your issue lists:

[ ] The relevant Azure service or technology.
[ ] A link to the published documentation article that you have feedback about.
[ ] Clear, specific feedback that the author can act on.

Pull requests and article contributions
If you know the change that is needed in an article, we encourage you to submit the changes directly using a pull request. If the change is large, or if you want to contribute an entire article, follow these guidelines:

[ ] Don't surprise us with a big pull request or a pull request with a new article! Submit an issue that describes the details of the proposed large change or new article.
[ ] Include the service or technology area.

We'll route the issue to the appropriate content team for review and discussion.
Tech support and product feedback
If you would like to contact Microsoft about other things, such as product feedback or tech support, please review these guidelines:

If you need technical support using Azure, the paid and free support options are described here: https://azure.microsoft.com/support/options/.

Each article in the Azure technical documentation contains a product feedback button - it's best to submit product feedback directly from a relevant article. Otherwise, you can submit product feedback for most Azure products in the following product feedback forum: https://feedback.azure.com/d365community/forum/79b1327d-d925-ec11-b6e6-000d3a4f06a4.

I wasn't able to find a feedback link on the article.
In these docs: https://github.com/MicrosoftDocs/azure-docs/blob/main/includes/application-insights-data-model-measurements.md
...it does not indicate that only numeric values are supported for custom measurements. If you try to include a string value, for example, the telemetry request succeeds without error, but the log simply doesn't show up.  It took a day of trial and error for me to figure out why my logs were not showing up.

@joshuanathanson
Thanks for your feedback! We will investigate and update as appropriate.

Removing my assignment as I no longer work on Application Insights, and have not for the past three years.

@AaronMaxwell
Please review it.


chore: fix typo in german language

Just a typo fix in the german language file.

Thank you so much @jstrobel !

Thanks, will do a new PR for the contributor thing.


update path for hmake

The path given in README (cd $GOPATH/src/github.com/truechain/truechain-consensus-core) is incorrect it should be cd $GOPATH/src/github.com/truechain/truechain-consensus-core/pbft
@arcolife

This is correct. However what's your opinion wrt restructuring the project so that the project root contains the files HyperMake, LICENSE and README.md instead of them residing under pbft/ ? If moving them up makes more sense, this change wouldn't be required.
Ideally the project should look like this?
.
├── docs
│   ├── DEV.md
│   └── TODO.md
├── HyperMake
├── LICENSE
├── pbft
│   ├── cmds
│   │   └── main.go
│   ├── pbft-core
│   │   ├── client.go
│   │   ├── config.go
│   │   ├── genkeys.go
│   │   ├── node.go
│   │   ├── server.go
│   │   ├── transportation.go
│   │   └── utils.go
│   └── vendor
│       └── manifest
├── README.md
├── support
│   ├── config.yaml
│   ├── scripts
│   │   ├── build.sh
│   │   ├── check.sh
│   │   └── test.sh
│   └── toolchain
│       └── Dockerfile
└── tests
├── core-consensus
│   └── keygen_tests.go
└── README.md

The current aim for this repo is to have a modularized consensus, which would very soon be integrated into go-ethereum's fork of truechain.
@procrypt yeah the path change is required after this change was committed. It was a precursory to inject go-ethereum codebase into this repo. But after a brief internal discussion, we decided to pull go-ethereum repo out and keep this repo modularized instead, so we can focus on pure consensus research specific changes here.
@samikshan eventually the bft committee in hybrid consensus would need PBFT code in a separate package and the hypermake, as per your tree, is going to be modified to build the entire modularized consensus itself, rather than just PBFT. So we'd still need the project root structure like the one your specified, but the pbft would reside in a package.

Here's what I have in my head:
.
├── docs/
│   ├── DEV.md
│   └── TODO.md
├── HyperMake
├── LICENSE
├── trueconsensus/ (code for hybrid consensus)
│   ├── cmds
│   │   └── main.go [calls both fast and snailchain]
│   ├── fastchain/ *
│   ├── slowchain/ **
│   ├── config.yaml ***
│   ├── bft_committee.cfg ****
│   └── vendor/
│       └── manifest
├── README.md
├── support/
│   ├── scripts/
│   └── toolchain/
└── tests
    ├── core-consensus

* fastchain/ (contains current PBFT codebase as currently under src/pbft/)
 ** slowchain/ is the snailchain in our hybrid consensus, a POW variant called fruitchain, which for this development purpose, is going to be a dummy block producer.
*** config.yaml contains node environment settings, tunables, etc.. just like https://github.com/truechain/truechain-consensus-core/issues/11
**** bft_committee.cfg contains log directory path, network config paths and so on..

FYI
This derives design decisions from https://github.com/truechain/py-trueconsensus (parallel effort, faster prototyping) ..which is what we're working on at the moment and which will communicate with main network (based on go-ethereum).
The slowchain is responsible for electing BFT committee, so essentially for single node testing purpose, we'd have multiple instances of slowchain and fastchain, all interacting through gRPC interfaces. The number of fastchain instances would be controlled by a tunable.

@samikshan I like the way you wants the file structure. I was also thinking the same that WHY a separate PBFT folder in the repo, but I think @arcolife comments make sense.

For the readme,   the initial goal to use hmake and script is for building from anywhere.
That is to say,  after you git clone the code and enter into the dir.
you can build anythere with hmake build as long as you are in the cloned dir.

covering this in https://github.com/truechain/truechain-consensus-core/pull/57#issuecomment-406927564 (will wrap any commits in there so as to not loose author. Example https://github.com/grafana/grafana/pull/6043/commits/2fdda7973cbc594d89dc93e93a1146c0ff88e2a5)


i18n(zh-cn): Update client-side-scripts.mdx

Description (required)
Update client-side-scripts.mdx
Related issues & labels (optional)

#9240

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

File
Note

zh-cn/guides/client-side-scripts.mdx
Localization changed, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

Thank you peeps!


ngrok.io redirected you too many times Error

When I try to locally host and debug using ngrok I get an error "*****.ngrok.io redirected you too many times". The tab worked fine when it was hosted publicly, so I don't think there's an issue with the code itself? Also the ngrok link works fine when accessed regularly on a browser. I just get this error when it's loaded as a tab on Teams (This is the configuration page).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8ba912c7-3f70-3406-ad5c-06b6a0fc3875
Version Independent ID: d3717482-cd3c-32ec-c7e0-28612bf13c37
Content: Run and debug your app - Teams
Content Source: msteams-platform/concepts/build-and-test/debug.md
Product: msteams
GitHub Login: @laujan
Microsoft Alias: lajanuar

Could you please share the screen shot of the error you are seeing on ngrok and tab screen?

Yeah this is what the error looks like:

This is only shown in the web version of Teams. In the desktop client nothing is shown:

works normally with a broswer:

The page works fine when hosted on the cloud instead of tunneling.

Could you please share network trace (either using the Chromium dev tools or Fiddler)?

It turns out there was a global redirect in our system when accessing internal resources that I wasn't aware of. We added an exception for the Teams' pages and solved this issue. Thanks for your help.


[extractor/twitter] Base extractor api overhaul

IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

The PR introduces the following changes:

Provide GraphQL API for future usage (this + #4864).

GraphQL errors are handled gracefully, extracting the error from the returned data.

Allow usage of GraphQL for TwitterIE which transforms the GraphQL result into a v1.1 result.

In this transformation, Tombstones are filtered and returned as an ExtractorError with explicit message.

Allow the use of cookies for requests (Fixes #1605), using GraphQL.

This also enables extraction for NSFW tweets with multiple media.

Fallback to the old token if the new token gives a 404 (Fixes #5233).
Stabilize multiple media support for cards (e.g. in this Tweet).
Add an extractor arg (force_graphql) to force the usage of GraphQL on unauthenticated api requests.

All tests working with the v1.1 API work with GraphQL as well. Exceptions are:

17: NSFW fallback is not implemented for GraphQL, so it will issue an error if login cookies are not passed.
18 and 19: While GraphQL returns mostly the same data there are instances it does not.
For these the returned url entities do not match, resulting in the description lacking some https://t.co urls.

Massive thanks to @bashonly for helping with this PR.
Template

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests
[x] Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

Can you explain what are the consequences of forcing graphql?

The transformation we need to do in _graphql_to_legacy shows that the formats are not absolutely compatible, which might have implications we do not know about yet.

In that case, imo it is better to always default to graphql. This makes it so that the code path is same with and without login. We can give an extractor-arg for the other path, and if the new API doesnt cause any issues, we can completely remove the old path in future

default to graphql

I see a couple of problems with doing this:

we would always need to keep the (old) API 1.1 code as a fallback since NSFW videos are inaccessible via GraphQL without login cookies. Ditching API 1.1 entirely would be regressive, and only switching the default behavior wouldn't simplify the code.
scripting twitter's API 1.1 has been real-world tested by many users over multiple years and has been shown not to cause IP bans, strict rate-limiting, etc. Twitter encourages use of the API, whereas they specifically warn against scripting their GraphQL in their documentation. Making GraphQL requests the default (logged-out) behavior could have unforeseen negative repercussions

I would also add to this that most of the code path is and would stay the same, since we just have a layer of abstraction for the GraphQL version on top. Inside we still use the _call_api method and we also parse the data using the legacy format extraction.
It would be a different story if we were to completely use a new, GraphQL only format. For that to work we would have to understand what the parameters and variables each do though, which we currently do not.

Since this will create merge conflicts with https://github.com/yt-dlp/yt-dlp/pull/4864, could you pull that into this PR as well? I'll add @nixxo and bashonly as co-authors

hi @pukkandan
i merged this PR with my PR about twitter spaces and pushed it to my fork as a new branch.
How do I append to this PR the 2 commits that I made?
https://github.com/nixxo/yt-dlp/commits/twitter-api-overhaul


Advertise Kubernetes Service external IPs

Expected Behavior

Calico already can advertise ClusterIPs. It would be awesome if we could also advertise external IPs. Raising this so we can gather some requirements and use-cases, since this is something we hear about a lot.
Current Behavior

We only advertise ClusterIPs.
Possible Solution

Some things to think about:

The code should be straightforward and mimic what we do in confd for cluster IPs
This allows anyone who can create a service to advertise an arbitrary IP address into the network. We probably want to put this behind a feature flag at least, or do some admission control.

Use case:
Announcing ExternalIPs will allow Services via Kube-proxy and IPVS to be a native L3/L4 load balancer with virtual IPs while maintaining the logical split between CluserIPs being for clients within the cluster and ExternalIPs being for external clients.
Today with only ClusterIP announcement, the CALICO_ADVERTISE_CLUSTER_IPS subnet needs to be routable to external clients, to be used in this fashion.
There is a concern on restricting users from arbitrarily announcing IPs. One idea, ExternalIP address announcement should be restricted to be within one or more ipPools. IPPools will likely need a new attribute, eg. external: true. This setting would also need to have the added effect of the pool not being considered for pod IPs. Like other ipPools, assignment to one or more namespaces should be supported, thus restricting the use and announcement of that IP block to the configured namespace.

I would be very interested in working to make this a reality. As far as I can tell, the high-level requirements are:

Automatically advertise all ExternalIP's attached to a service.
Whitelist these advertisements, to prevent users from advertising arbitrary addresses.
Implement this whitelisting in a way that is easily and simplify configurable, and works well with Calico's existing data model.

@caseydavenport @neiljerram @bensallen Any other thoughts on the requirements?

@bensallen Regarding the IPPool external: true idea, I wonder if it might be cleaner to simply whitelist all advertisements that are within any IP Pool? Then to define an "external" IP Pool the user could just create a disabled IP Pool.

@tompntn your high-level requirements look good.
There also needs to be a way to associate the specific BGP peers and AS numbers to use for announcing the external IPs. It seems possible that different routers may be used for external announcement versus pod networks, or there may also be different peers depending on the external IP pool.
Per your second point, I'd think allowing an advertisement of an IP within any pool that is either global or scoped within the namespace will work.
Perhaps instead of external: true, if a IP pool is associated with a BGP Peer configuration then announce it, otherwise complain in some fashion and do nothing.

There also needs to be a way to associate the specific BGP peers and AS numbers to use for announcing the external IPs.

That sounds like over-complicating things to me.  It can be added later if someone comes along with a concrete need.  For those who want to do such things, they can always do them at the router side by configuring their import policy.

That sounds like over-complicating things to me. It can be added later if someone comes along with a concrete need. For those who want to do such things, they can always do them at the router side by configuring their import policy.

Agreed, I think for now we should focus on the core feature of advertising whitelisted IP's.

After some discussion on possible approaches with @caseydavenport, we could do this by adding to BGPConfiguration. It might look like:
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  logSeverityScreen: Info
  nodeToNodeMeshEnabled: true
  asNumber: 63400
  externalServiceIPs:
  - <IP_ADDRESS>/32
  - <IP_ADDRESS>/24

Some advantages over using IPPools or environment variables (on the calico/node daemonset):

Allows updating external IPs advertisement without a rolling restart of the calico/node daemonset
Allows immediate feedback and input validation
Moves us towards a more narrowly-defined Calico configuration model (over env variables).

IPPools are probably not the best place since ippools are for endpoint IPs (so we're either overloading the concept of ippools or widening it into a generic pool of IPs).
As part of this feature, we'll probably want to migrate ClusterIP advertisement over to BGPConfiguration as well.

@lmm Extending BGPConfiguration seems like a pretty elegant approach.
Is it possible to extend BGPConfiguration to limit advertisements to specific namespaces? Perhaps via the naming scheme currently used for node specific configuration, eg. namespace.<name>. Although it might be more extensible to select namespaces in the same fashion as IPPools via annotation or perhaps an attribute similar to nodeSelector.
In addition to support ClusterIP and ExternalIP advertisement, LoadBalancerIP advertisement would be useful for integration of a service like MetalLB.

@bensallen if an operator wanted to limit ingress to a service to a specific namespace, they could use network policy.

In addition to support ClusterIP and ExternalIP advertisement, LoadBalancerIP advertisement would be useful for integration of a service like MetalLB.

I agree! 😄It would certainly be nice to improve our integration with MetalLB for our bare-metal users.

@bensallen sorry for the delay. What I had in mind was that you could employ network policy to limit access to the set of pods backing a service to a specific namespace: https://docs.projectcalico.org/master/security/namespace-policy

@lmm, no worries. Ideally I'd like to go the next step and allow users in a multi-tenant environment provision their own services, but I can see how network policies could be used as an interim solution, where services are only allowed to be provisioned via admins.

@bensallen I see. Tigera Secure has a bit more flexibility in this regard with the ability to slot policies into tiers: https://docs.tigera.io/v2.5/reference/resources/tier
Admins can set Tiers with baseline policies and give users access only to certain tiers/policies:
https://docs.tigera.io/v2.5/reference/cnx/rbac-tiered-policies#policy-and-tier-rbac


License of the project

Hi,
The library looks very good and I would like to use it in a project of mine but what's currently holding me back is that there's no license defined for it so I don't know what I am allowed to use it for. Could you clarify by choosing and adding a license?
Thank you very much!

Hello, I have released v1.4.2 which contains the MIT license file.

Thank you very much for the quick reply and the great license choice! Looking forward to using your library!


Set up some standard actions for all workflows

Originally posted by @andrewgordstewart in https://github.com/statechannels/monorepo/pull/1333

@tomclose  @andrewgordstewart any specific examples of standard actions we could write now?

https://github.com/statechannels/monorepo/pull/1333/files#diff-af9d021eea475841e669f823043ab0ffL92-L97
I think those are what instigated this issue


startcli - Couldn't get a file descriptor referring to the console

i used this to create the chroot
sudo sh -e installer/main.sh -r arch -n arch.cli-extra -t cli-extra
when i try "sudo startcli", it gives me this error
Entering /mnt/stateful_partition/crouton/chroots/arch.cli-extra...
Couldn't get a file descriptor referring to the console
Unmounting /mnt/stateful_partition/crouton/chroots/arch.cli-extra...
sudo enter-chroot #works fine

I have this problem too. Just installed crouton -t cli-extra on an Asus CN60.

BTW, should this issue perhaps be filed on the parent repository (dnschneid/crouton)?


Install

A short todo list, beyond the obvious remaining unitTest ports and testing:
we probably need an install
requirements.txt
setup.py
And clone the readme and add some install instructions.
and maybe eventually upload to pypi?

Agreed

Brian, can you give Bruce some suggestions on how to package his code for distribution?

Hi all, Just diving into this code.
Regarding distribution, how do you imagine the usage of the tool?
I think based on the pattern seen in your example.py a pip style installation may be the most convenient for the user. Then they can lace the prompt construction content into their larger script most easily.

If that's the case, I can assist (hopefully) on some packaging steps

Sounds great. This is an area where I have zero experience.
If you want to take a shot that would be great. I'll be offline on this project for a couple of days while I work on a couple of other things. I'll take a look when I return and if you haven't had a chance to take a look I'll look at the link
thanks!


Docstrings - update to conform to any of the three most common conventions (google, numpy, sphinx)

It did not seem that they were picky about the style but rather that we should be consistent.
Any suggestions? otherwise I just pick one like Numpy-style...

I would prefer using Google Style, since it is shorter and good readable:
Google Style
"""
This is an example of Google style.

Args:
    param1: This is the first param.
    param2: This is a second param.

Returns:
    This is a description of what is returned.

Raises:
    KeyError: Raises an exception.
"""

Numpy Style
"""
My numpydoc description of a kind
of very exhautive numpydoc format docstring.

Parameters
----------
first : array_like
    the 1st param name `first`
second :
    the 2nd param
third : {'value', 'other'}, optional
    the 3rd param, by default 'value'

Returns
-------
string
    a value in a string

Raises
------
KeyError
    when a key error
OtherError
    when an other error
"""


Add more Payloads

Added several payloads
VueJS v2 and v3, Angular JS, svg onload and script.

Please squash commits before merge.

This is cool. Only thing is; your changes breaks all styling on the page. I will fix this.


[MCS-DE-GO] Unknown Intent not triggering in Prompt Actions (Ask a question)

Whenever we set any Prompt Action with "allow interruptions" the triggers of the dialogs which have benn confirguren work except the "Unknown Intent" trigger.
Please enable that also the "Unknown Intent" triggers an interruption.
@joaollq @goergenj @sgellock @benbrown @bindsi

@cwhitten Any timeline update on this?

Transferring this to botbuilder-dotnet

Transferring this to botbuilder-dotnet


These are suggestion due to my obessive compulsion to have things readable...

Hey @pasichDev,
I've just had a quick look at your changes and I think you did great, with even picking up a bit of the MVC with controllers :smiley:
I would wish that you would use CamelCase in your code's name spaces and/or classes, but again, not necessary for the code to work.
I have this compulsion to make my code as readable as possible, so I tend to adhere to a lot of the PHP conventions(not rules, I know) that I've picked up with 15+ years on PHP, but this does not mean that everybody should be as "anal" as I am :smiley:
I have this thing about an unspoken rule that involves lead programmers being killed by buses. If they are killed, and the code they leave behind is only readable/understandable to that dev, then the poor dev that comes to continue that code will have a very bad time. And I'm a firm believer of : Do unto others, like you would want others do unto you.
All in all, I love that you were able to understand my template and you even did a mini router and controller!! This means you really have a good understanding of how it should be :smiley:
Congrats!!
Cheers,
Gus

Hi @gcarreno , i understand you, everything is fine! I think we can close the question!


Implement apoc.meta.types function: returns a map of keys to types

Is your feature request related to a problem? Please describe.
While this function is not supported in Apache-AGE, but they are implemented in Neo4j. I found it useful, particularly for answering this particular case scenerio.
https://stackoverflow.com/questions/75892961/how-do-i-find-the-data-type-of-a-cypher-query-result-in-apache-age.
Descriptions of this function is exactly similar to the one described here: https://neo4j.com/labs/apoc/4.3/overview/apoc.meta/apoc.meta.types/
Describe the solution you'd like
N/A
Describe alternatives you've considered
N/A
Additional context
N/A

@jrgemignani Is this feature already in implementation phase? If no, do you consider working on this would be worthwhile?

@Zainab-Saad I am not aware of it being scheduled or worked on by anyone. If Neo4j has it, it is worth considering.

Since this function is not yet implemented in AGE, so I am working on a PR to implement this function

Since this function is not yet implemented in AGE, so I am working on a PR to implement this function

@Zainab-Saad It is worth noting that if this is part of a larger package of functions, by adding this function, it invites others to want the rest of the package. That would be my only caveat.

This package has many useful functions, do you think we should consider implementing the larger package of functions that would have this type check function as well as other functions? Because we need to make AGE as close to Neo4j as possible.

@Zainab-Saad Yeah, for now we could look into the functions that make sense and will add value.


Data update 2022

Initial upload of 2021 data update is working but Energy Star and EUI scores are not properly displaying (at least)
Check formatting issues

[ ] Remove NA and replace with blanks (or whatever was previously used for no data)

When all checks complete

[ ] Join with existing 2020 data seattle_buildings_master_copy_for_update2020

┆Issue is synchronized with this Asana task

When replacing NA with blank, I'm noticing some unexpected blanks in the following fields:

city, state (fixed)
zip (~50 blanks, not fixed)
councildistrict (13 blanks, not fixed)

There is one new field EPAPropertyType. I'm leaving this in, but it is not used in the application, and it will be blank for all years except 2021

This has been combined with previous years and updated on CARTO
https://cityenergy-seattle.carto.com/dataset/seattle_buildings_2021_update
Initial tests on localhost look good:


Add Developers' Assetes Folder

Add a _dev/ folder to collect developers and maintainers' assets and documents, which should be excluded from the final package shipped via Package Control.

[ ] Create _dev/ folder.
[ ] Exclude it from final package?

I'm not sure whether it's possible to exclude some files and folders from the final packed package on Package Control. I have a vague recollection that there used to be some JSON settings to handle this, but I might be wrong or this might no longer be the case.
Devs/maintainers' assets don't really need to be included in the auto-installed package, it's a waste of space; they should only be available when cloning the repository via Git.

Ignoring Package Files
The Creating Package Files page of the Package Control documentation mentions various Package Profiles settings that allow to exclude certain files and folders from the final .sublime-package archive (packed package):

dirs_to_ignore — A list of folders to ignore when creating a .sublime-package file.
files_to_ignore — A list of file glob patterns to ignore when creating a .sublime-package file by the Create Package command.

The instructions are for custom repository hosting instead of using GitHub or BitBucket, so I don't think they apply to normal packages delivered by Package Control — there doesn't seem to be an equivalent settings file for a package repository intended to be delivered via the official channel.
Package Profiles
Profiles refer to the Package Control package that's bundled with ST, i.e. settings that the users can tweak via the Preferences » Package Settings » Package Control » Settings – Default/User menus.
The file allows a package_profiles setting:

package_profiles — The key of the object will be the profile name and a list of them will be presenting when running the "Create Package File" command. The profile "Default" will use the top-level version on the following settings:

E.g., this is the default profile:
{
    "Binaries Only": {
        // Exclude all .py files, but consequently include all .pyc files
        "files_to_ignore": [
            "*.py", ".hgignore", ".gitignore", ".bzrignore",
            "*.sublime-project", "*.sublime-workspace", "*.tmTheme.cache"
        ],

        // Include __init__.py so Sublime Text will load the package
        "files_to_include": [
            "__init__.py"
        ]
    }
}


Create mapping_set_reference endpoint

Extend the LinkML MappingSetReference, and add everything needed.
mapping_set_title needs to find first in the mapping_set, if not add in the mapping_set_reference

Instead of creating this endpoint, we prefer creating a sub API for the UI needs.


[X] Read the common issues list.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Module federation issue and not a framework-specific issue.
[X] The provided reproduction is a minimal reproducible example of the bug.

For your information, I'm running this on macOS, but this shouldn't make any difference because I saw person with the same problem on Windows

set manifest: false on the plugin for now as workaround

@VityaSchel example seems to work for me. Supply a repo that reproduces the issue

@VityaSchel example seems to work for me. Supply a repo that reproduces the issue

it does not matter which project you take, the problem is in path, you can run empty project with spaces in path to it

i did and it worked. Supply a repo broken from the start

i did and it worked.
[x]  Read the common issues list.
[x]  Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[x]  Make sure this is a Module federation issue and not a framework-specific issue.
[x]  The provided reproduction is a minimal reproducible example of the bug.

@VityaSchel  Thankyou for sharing this it really work.

i did and it worked. Supply a repo broken from the start

no way
it's actually broken and my issue helps people
mind finally fixing this?

[x]  Read the common issues list.
[x]  Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[x]  Make sure this is a Module federation issue and not a framework-specific issue.
[x]  The provided reproduction is a minimal reproducible example of the bug.

Thank you for sharing. This solved the issue I was facing as well.

i am unable to replicate the issue

i did and it worked. Supply a repo broken from the start

example repo with nextjs v14 does not work in my case with bun.sh if you can't reproduce and won't fix this, you can close this, it still will be a starting point for developers who have the same issue

You are using bun? not node?

i am unable to replicate the issue

i did and it worked. Supply a repo broken from the start

example repo with nextjs v14 does not work in my case with bun.sh if you can't reproduce and won't fix this, you can close this, it still will be a starting point for developers who have the same issue

You are using bun? not node?

yes, in fact I have never used node since switching to bun, so as everyone who did the same thing


[Free Trial] Make the "Add payment card" button redirect to settings/subscription/add-payment-card so the RHP opens

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: v9.0.8-1
Reproducible in staging?: Y
**Reproducible in production?:**Y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @trjExpensify
Slack conversation: https://expensify.slack.com/archives/C036QM0SLJK/p1721181508995759?thread_ts=1721057227.335969&cid=C036QM0SLJK
Action Performed:

Create a workspace
Submit an expense to the workspace to start a free trial
Go to supportal and set the free trial date to a date in the past (to faux it being ended)
Go to the "Free trial has ended" message in the Expensify/Conci DM
Click the "Add payment card" button

Expected Result:
Redirected to settings/subscription/add-payment-card, so the RHP to add a payment card opens.
Actual Result:
Redirected to settings/subscription.
Workaround:
N/A
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/5b45ca95-a9d8-4e4e-828d-ec3b60dd0faf

View all open jobs on GitHub

Proposal
Please re-state the problem that we are trying to solve in this issue.
Make the "Add payment card" button redirect to settings/subscription/add-payment-card so the RHP opens
What is the root cause of that problem?
We need to update the navigation action here:
https://github.com/Expensify/App/blob/6851dc11848d38b033c973c2ba4f34fee76f8fca/src/pages/home/report/ReportActionItem.tsx#L408-L411
What changes do you think we should make in order to solve the problem?
We will update this line to:
Navigation.navigate(ROUTES.SETTINGS_SUBSCRIPTION_ADD_PAYMENT_CARD)

Sorry I picked this up and already finished it internally since it was a quick fix for me.

No payment needed here, closing.


reorder author photos for the blog post

Signed-off-by: Alice Williams<EMAIL_ADDRESS>Description
reorder photos for the blog post
Check List

[ x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

thanks @krisfreedain !


Make Trace object-safe, allowing dynamic types and slices to be garbage collected

The simple part is making a dynamically dispatched version of 'Trace' and 'GcVisitor'. I just added where Self: Sized bounds to all the generic methods and added an object-safe fallback.
This main problem is that 'Trace' and 'GcSafe' have associated
constants 'Trace::NEEDS_TRACE' and 'GcSafe::NEEDS_DROP'.
I've separated these associated consts into their own GcTypeInfno
trait.
I've messed around with this all day.
So far it looks like we might end up requiring significant amounts of
nightly features D:
Fixes #15
Required for #16
Goals

Include 'NEEDS_TRACE' optimization to avoid un-needed tracing

Possibly just use NullTrace maker as a simpler alternative

Avoid invoking auto-derived dummy drops

These are generated by the derive code, forbidding the user from implementing a custom drop that resurrects objects
In that case std::mem::needs_drop may return a false-positive true

Avoid adding new nightly features

If we do need to add any they must have an accepted RFC
Ideally, we don't want to add very complex features (IE: not specialization)

Preserve static type information and avoid dynamic dispatch wherever possible

Can we include some sort of StaticTypeLayout struct in the future
This depends on static type information being perserve

Approaches
I see three main approaches that keep the current type hierarchy, and there are pitfalls to each one.
Right now I'm leaning towards separate DynTrace types or having separate trait requirements for sized and unsized types.
Require where *const Self: GcTypeInfo for all Trace types
Directly extending Trace from GcTypeInfo doesn't work. The rust compiler requires supertypes to be object-safe.
When wrapped with a pointer, the type can then implement non-object-safe constants. We simply add functions needs_trace and needs_gc_drop which query <*const T as GcTypeInfo>::NEEDS_TRACE (or NEEDS_DROP).
The big advantage is this approach works on stable!
The disadvantage is that the bound isn't implied for generic parameters.
If you have impl<T: Trace> Trace for Box<T> then the compiler complains that *const T doesn't implement GcTypeInfo.
You must explicitly add a where-bound where *const T: GcTypeInfo to each and every generic parameter :roll_eyes:
The implied generic parameters would be fixed by RFC 2089. However, this would probably require nightly, maybe even for API clients.
Specialization
We could use specialization internally to implement needs_trace and needs_drop functions.
trait HiddenTypeInfo {
    const NEEDS_TRACE: bool;
    const NEEDS_DROP: bool;
}
default impl<T: ?Sized> HiddenTypeInfo for T {
    const NEEDS_TRACE: bool = true; // conservatively assume trace
    const NEEDS_DROP: bool = true; // conservative assume drop
}
impl<T: NullTrace> for T {
    const NEEDS_TRACE: bool = false;
}
impl<T: DummyDrop> for T {
    const NEEDS_DROP: bool = false;
}

Problems

Unfortunately, this doesn't work with min_specialization

The full specialization feature is unsound, so we absolutely want to avoid it
It breaks requirement 3

I think this has something to do with always applicable impls
This could be fixed (in theory) if we used the compiler-internal attribute #[rustc_unsafe_specialization_marker]

Maybe we should just ignore it since we know that our specific use-case is fine?

How would we require GcTypeInfo to be implemented for all Trace + Sized types?

Associated Type
We could require an associated type type Info: GcTypeInfo. By default, the associated type would be Self for any Sized types. However, for unsized types we would have Info = DefaultTypeInfo that gives conservative defaults.
Associated type defaults were accepted in RFC 2532.
Problems

The feature is unstable (though not as bad as specialization).
This is only theoretical. I haven't tried it yet
Does this work if we don't have a Self: GcTypeInfo bound on the trait?
Kind of an ugly approach

Separate erased traits: DynTrace and DynGcVisitor
This is what erased serde did. This is the first approach I tried. It was complex because I had to essentially make the GC accept both Trace and DynTrace. They often needed to be treated separately and there was unessicarry code duplication :(
Maybe we should make the Trace trait object safe by default, but keep GcSafe sized?
The main type that requires GcSafe is Gc<'gc, T> (for safety). That requirement could certainly be relaxed to only the constructors.......
Could we have separate traits GcSafe and GcStaticType? Any sized type must implement GcStaticType and give us the flags (NEEDS_TRACE/NEEDS_DROP) that we want.

This needs to be reimplemented with the new pointer metadata API.
I'm considering just making everything thin pointers so that it is easier for the collector to trace, and more consistent with the .
The most important ?Sized types to trace are slices. I'd really like it if the collector was able to allocate arrays/slices and not just individual objects.

Superceeded by #29


Modin DataFrame to_parquet Path Param Differences Between Dask and Ray

Thank you for the fantastic work on Modin! In some recent exploration of Modin capabilities, I noticed that to_parquet appears to seek slightly different path parameter depending on whether Dask or Ray are used as the engine. With Dask, path seems to be a file. With Ray, path seems to be a directory (by default, exporting potentially multiple files based on the dataset). Is the difference between Ray and Dask with Modin expected in this instance? If so, is there any documentation I may reference to learn more? Appreciate any guidance on this.

@d33bs thank you for your report. Unfortunately, while Modin on Ray supports distributed to_parquet, Modin on Dask does not yet. Modin parallelizes to_parquet by writing chunks of rows to separate files. You see multiple files with Ray because each file was written in parallel, while with Dask you should get warning UserWarning: to_parquet defaulting to pandas implementation. and end up with a single file.
For now, let's treat this issue as a feature request for to_parquet on Dask.

Hi @mvashishtha - thank you again! I'd be more than happy to assist with #4628 , appreciate the opportunity to contribute.


Logo Idea

Hey there @basepi
We met briefly at the Adobe Open Source Summit and I said I was interested in creating you a logo for this project. I think I remember you asking for something flattish design? In any case, I've come up with an idea attached here. Let me know if this is something that you're looking for.
Large:

Small:

This is pretty awesome! I wish I were more artistic.
The logo itself looks great but is a touch busy, especially when it's small. It also doesn't really evoke a connection with hubblestack outside of the space telescope theme -- I wonder if we could modify the dimensions to evoke more of an H shape.
For example, this is one that we've been looking at:

It kind of is reminiscent of a satellite, but is still very recognizable as an H as well. It's modified from one of the monograms available on logojoy. I'm still on the prowl for something that makes me go "that's the one!" but I think even simpler is the direction we'd like to go.

I think overall timkim's design is pretty to look at and fits great on stickers. If we want to fit it into a favicon format it cold lose details like the solar panel grids, but that's for really small formats.

Ooh, yeah, now I really want a sticker! We may have to try to find a way to incorporate both, maybe for different uses.

@timkim any chance you have an SVG version I could toy with?

Yep - attached is a zip of the svg and ai files.
hubble.zip
I'll see what I can come up with the H shape in mind.

Ok, here's another iteration of the logo. I have kept in mind the H shape and the busyness of the previous attempt. We can talk about colours next if you like the shape of this design.

Ooh, I like this one quite a bit. Once we get some color in it it could be really awesome! @cedwards thoughts?

Okie dokie, here's a colour scheme that I'm trying. Still playing around with it, but I'm liking it.

I'm not sure I'm digging the yellow, but I'm also not sure what I would replace it with.... turns out I'm not a designer.

We can go red or just go with a circle.

I like the yellow or the white. Not loving the red though.

Yeah, red didn't do it either. White is probably my favorite. @cedwards any thoughts?

Comparison - the first one looks the the best, aesthetically speaking.
But the latest one has the idea of eye being in the middle. My only suggestion is to try and see how it looks with a dark background and somehow change/make the Hubble colors work with that. I think that would make the eye stand out more.

I think we've mostly settled on the logo that's currently on our org, but these are good options for the future. Thanks @timkim!


Import: Change merge logic

In order to support some cases of the import behavior, additional checks were needed to be added to the merger logic.
This PR includes those changes and tests for the import behavior.
It's recommended to review #6250 (the PrefixIterator is sufficient)
Closes #6162

Like I said in #6250, while it's great to reuse the merge logic for the import, I don't think the import is part of the merge.
This PR makes this a bigger issue.
I feel responsible for taking us down this path but I can only suggest two things:

Try to solve this by adding the following logic to the previous PR: the destination iterator should split any range that (may) include a prefix. For example, if the prefix is b and the original range is [a,c], that dest iterator will return [a,b] and [b,c]. This may solve the relevant issue and still will allow you to leave the merge code as is.
Go back to a dedicated import solution. :(


SkyLightProbe produces NaN texels under certain conditions

Related to https://github.com/takram-design-engineering/three-geospatial/issues/7

Confirmed that some vertices of Google Photographic 3D Tiles have normals as zero vectors. I'd rather say this is a bug in TileCreasedNormalsPlugin, or normals aren’t natively supported by Google Photographic 3D Tiles. Keeping this issue open for the record.

This is odd... I wrote the toCreasedNormals function so I'll take a look later. The only reason I can imagine this happening is if we have a degenerate triangle with all vertices at a single point - but in that case we shouldn't see the triangle rasterized to the screen.
From https://github.com/takram-design-engineering/three-geospatial/commit/2b93b01be25cd8ad9f4079a0d48f7e019ed56f8b - do you know how are NaNs making it into that shader? Is it downsampling normals? Or are NaN values making it into the color buffer due to a 0,0,0 normals?

That shader takes the color buffer from the render pass, and NaN values are in the color buffer. I suspect the source is here, which invokes this.

but in that case we shouldn't see the triangle rasterized to the screen

That’s true, now that you mention it…

"Fixing up" these degenerate normals seems to fix the issue:
for ( let i = 0, l = normal.count; i < l; i += 3 ) {

  v0.fromBufferAttribute( normal, i + 0 );
  v1.fromBufferAttribute( normal, i + 1 );
  v2.fromBufferAttribute( normal, i + 2 );
  if ( v0.length() < 0.5 || v1.length() < 0.5 || v2.length() < 0.5 ) {

    normal.setXYZ( i + 0, 0, 0, 1 );
    normal.setXYZ( i + 1, 0, 0, 1 );
    normal.setXYZ( i + 2, 0, 0, 1 );

  }

}

I took a look and it looks like 0, 0, 0 normals are only generated on triangles with an area of 0. Though the triangles are degenerate lines - ie two vertices are in an identical location and the third is offset. I'd think this would result in no pixels being rasterized but this doesn't seem to be the case - though there may be something else going on? computeVertexNormals results in the same issue while setting material.flatShading does not.

An image of 0, 0, 0 normals being rasterized, highlighted in red by modifying a normal material 🤷 :

Fixed in https://github.com/takram-design-engineering/three-geospatial/commit/7fd13501de0df68c4591e6a8aa0f5282852b1cbf


Changes needed for current version of angular material

Please change the MaterialModule import to MatIconModule.  The use of MaterialModule has been deprecated in the current version of angular material and causes an error.
ERROR in D:/data/workspace/XXX/src/main/webapp/node_modules/ngc-float-button/ind
ex.ts (3,10): Module '"D:/data/workspace/IML/src/main/webapp/node_modules/@angul
ar/material/material"' has no exported member 'MaterialModule'.
Additionally, change md-icon to mat-icon as the md prefix has been deprecated as well.

I will fix this problem in this weekend, thank you.

We couldn't wait any longer :) Please find my PR #7

Hey @rayearth42 , unfortunately, I was pretty sick because of this I took so long to fix that, sorry and I just published the new version and it's compatible with angular >= 4.4.4 and >= angular material beta 12, enjoy.
just run npm install --save ngc-float-button


Rename Instances

Self explanatory feature request ;D

No it doesn't.
What is it about?

Close due to inactivity

Can we reopen it? For a couple of services, I have multiple instances. It would be very helpful if I could rename them, give them nicknames and/or even change the usericons to differentiate between them. For instance, now I have 3 WhatsApp-icons below eachother and I have to actually click on each to see which is which.

Can we reopen this issue? I have multiple instances of more than one service. I would love to have the possibility to rename the instance (for example: instead of 'WhatsApp' 'WA Personal') or to give them a nickname behind the service name (for example 'Whatsapp (Personal)') and/or to change the icon per instance of a service.
I know Ferdium offers this possibility and it is very useful.


Added None case to repeat

If this is not added the line 8 crashes:
if (len(src) > length):
TypeError: object of type 'NoneType' has no len()

When invoking something like:
voxel_grid(data.pos, data.batch, size = 10)

Thanks :)


ARTEMIS-4752 upgrade ErrorProne to 2.27.0

@gemmellr, I don't meant to step on your toes with your work on the JUnit 5 upgrade, but these test changes were necessary for the upgrade. If necessary we can certainly wait until your done if you prefer.

I only asked not to merge the other one and have me roll related changes into my work since it was quite big but wasnt actually 'fixing' anything that was 'broken', so it would have been annoying to deal with the conflicts given that...this is fixing stuff, so it goes in and I deal with it along with the other rebase stuff needed. (Happily its also smaller).


display status of gps signal

in the beginning there is an overlay now that shows a loading spinner until fiírst gps fix is received


fix journey manager umd

I haven't tested this extensively (across other packagers, for instance) but it appears to at least work for journey manager

used this html file to test the results of this change
In no way a comprehensive test!


[Ray.so] Add new color themes and preview images for partners

Description

Add new color themes and preview images for Vercel & Evil Rabbit partners.

The preview image come from Vercel's Assets in Geist Design System.
While the Evil Rabbit one comes from his X/Twitter profile picture.

Screencast
Here are some screenshots on how they look in the Color dropdown under the Create Image From Code command:

And a couple videos from using both the Generate Image and Create Image From Code commands:
https://github.com/raycast/extensions/assets/64607766/9fbd68d3-3440-4732-96d6-a2c78f9c0bb8
https://github.com/raycast/extensions/assets/64607766/ffa85f27-f595-45ba-99a5-876ea81f2e2d
Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Diff might appear substantial since updating the extension via npx ray migrate, which upgraded some dependencies from @raycast/api v1.25.3 to v1.72.0.
However, the changes are mainly the addition of the new Vercel and Evil Rabbit color themes.

Looks good to me, could you check this @samuelkraft


Page is not in the proper language

Navigating to https://docs.microsoft.com/nl-nl/azure/azure-monitor/app/proactive-application-security-detection-pack/app/proactive-diagnosticss
The content is shown in language Czech in stead of Dutch.
Url says nl-nl
Language setting in the footer of the page says "Nederlands"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 914de10e-eaf5-d7b5-09be-c6d7b32408b2
Version Independent ID: 5d35c099-0f3c-667e-3a43-66f2cf513461
Content: Smart detection in Azure Application Insights - Azure Monitor
Content Source: articles/azure-monitor/app/proactive-diagnostics.md
Service: azure-monitor
Sub-service: application-insights
GitHub Login: @bwren
Microsoft Alias: aaronmax

Thanks for bringing this to our attention. We will investigate it further and update you shortly.

Hello. There have been a number of localization issues across Docs that are not specific to any one page.
These have been addressed by Sunny, our localization PM. (dengyun)
The issue does not appear to repro at this time, but please comment if you continue to experience the problem after a CTRL+F5 refresh of the page(s).

#please-close