GH-33923: [Docs] Tensor canonical extension type specification

Rationale for this change
There have been quite a lot of discussions connected to the tensor support in Arrow Tables/RecorBatches. This PR is a specification proposal to add tensors as a canonical type extensions and is meant to be sent to the Mailing list for discussion and vote.
What changes are included in this PR?
Specification for canonical extension type for fixed sized tensors.
Open question
Should metadata include the "dim_names" key to (optionally) specify dimension names when creating the Arrow FixedShapeTensorArray?

Closes: #33923

Also: a question I hit on the C++ implementation last week:
what happens when one wants to work with two fixed_size_tensor extensions (let's say they have different shapes) at once? I think this will be a common occurrence and because of the namespace collision two extensions can't be registered at once.

When working on pyarrow implementation locally, this wasn't an issue.
From the Python documentation:

Registration needs an extension type instance, but then works for any instance of the same subclass regardless of parametrization of the type.

And you can see this case in the tests I uploaded to gist:
https://gist.github.com/AlenkaF/95fb41f461fb792396bb20dd502b4112#file-02_tensor_extension_tests-py-L77-L96
I guess it is the same in C++?

Only one extension type with a given name can be registered at a time. Multiple can coexist, but register can only keep one extension per name. I suppose users won't really want to register these types (or is that a requirement for using compute?), but will want an easy way to instantiate them.

Only one extension type with a given name can be registered at a time. Multiple can coexist, but register can only keep one extension per name.

It's only the name that is being registered, along with the methods to serialize / deserialize. The actual metadata (i.e. the only thing that differs between two different extension type instances of the same type) isn't part of the type class that is being registered, and so for a single registered (parametrized) type, you can have many instance alive at the same time with a different parametrization (i.e. different metadata).
That's the goal of having parametrized types (and so you don't want to have those parameters (metadata) in the name of the extension type. Otherwise you would have to register every possible parametrization).

Thank you for reviewing Rok!
If Joris agrees this can be sent to the ML when the C++ implementation is ready. If I remember correctly, I would start a new discussion thread for the tensor canonical extension specification adding some explanation, link to the C++ implementation and pyarrow example. Please correct me if I am wrong.
I am working on PyArrow extension example to have it ready for illustration.

Hey @lhoestq, you (and/or other people at HuggingFace) might be interested in reviewing this proposed addition. It will also be discussed on the Arrow dev ML.

@jorisvandenbossche

I noticed that in the HuggingFace Datasets implementation, there is a comment about using variable instead of fixed size list array. That is resolved now, and it's fine to use fixed-size lists?

Yes, the fixed-sized version seems to work now (requires pyarrow>=10.0.0 to pass the tests).

I think none of the existing custom tensor extension type implementations have this (eg neither of Ray and HuggingFace Datasets have this. @mariosasko do you know if that has come up in HuggingFace?)

No, that has yet to be requested on our side (so not super important for us right now).

Hello @changhiskhan (plus anybody from Lance), @clarkzinzow, @scottjlee (plus anybody from Ray), you might also be interested in reviewing this proposed spec for fixed shape tensor type. A discussion e-mail has also been sent to the Arrow dev ML.

@paleolimbot thanks for the R example (it's nice to see how easy it is!)
One small comment on the conversion to/from an R matrix in your example: typically, if you start from an array of a certain dimension, the dimension of the actual shape in the metadata of the TensorType is the original dimension - 1 (since the first dimension becomes the length of the logical TensorArray). In your example, you are converting a single array to a 1-len TensorArray (which is probably related to what you said earlier about having a "chunked array where every chunk had a single element", but just to note that I think this won't be the typical way to convert to/from TensorArray)

From inline discussion at https://github.com/apache/arrow/pull/33925#discussion_r1094350467:

[@thomasw21] Sometimes users will manipulate non_contiguous formats without realising. Typically torch uses a to(memory_format=torch.channels_last) which essentially is a equivalent to .permute(0,2,3,1).contiguous(). This API is often used in computer vision as some operation run faster when the tensors are not actually stored in "row major" fashion. So loading images in that specific format directly would help, which means storing non row major tensors.
...
More on this: https://pytorch.org/blog/accelerating-pytorch-vision-models-with-channels-last-on-cpu/

@thomasw21 the page you link mentions the difference between "physical order" and "logical order". So the actual, physical layout in memory that pytorch uses can either be channels first (NCHW) or channels last (NHWC). But in both cases, this physical order is row-major. When interacting with those tensors as a user, pytorch however always shows the data as if it's NCHW, independent of the physical layout, i.e. what they call the "logical order". And it's when viewing the data as NCHW while the physical layout is NHWC that you get custom strides / non-C-contiguous layout for that view. But when considering the physical layout NHWC, the data is still row-major.
And for the purpose of this specification and being able to transfer FixedShapeTensor data using Arrow, it's the physical order that matters, and optionally the dimension order that matches the physical row-major layout (i.e. NHWC or NCHW for pytorch).

So loading images in that specific format directly would help, which means storing non row major tensors.

So given the above, my understanding is that it is still perfectly possible to store either formats that pytorch uses as row-major data in the proposed FixedShapeTensor array, and be able to load it directly (without copy).
You could then use the proposed "dim_names" to keep track of the actual layout (for example if the physical layout is "channels last" (NHWC), you could specify "dim_names": ["H", "W", "C"], and when reading those data into a pytorch tensor, you know the physical order, but are still free to view as a different logical order).

@ezyang and @fchollet or their colleagues might have some input here as well.
TLDR: We're discussing design of tensor extension arrays for Arrow. Biggest discussion point so far is whether we should store strides or is dimension order enough.

Regarding what dim_names would imply for pyarrow and numpy:
If I have a numpy array, convert it to arrow using pa.array, and then call to_numpy(), ideally I would expect to get the exact same numpy array.
If we use dim_names and the array has a physical order that is different that the logical order (but still row-major), does that mean I would have to give names to the dimensions to do [numpy -> arrow] and to do [arrow -> numpy] to end up with the same array back ?
Arrow could also give arbitrary names and save the original logical order for the dimensions.

Some recommendations from me (not officially representing PyTorch team, but just based on my experience.)

Don't encode dimension names. We tried this with named tensor, but it's too restrictive and the propagation rules are too difficult to right. The correct way to do it is first class dims (https://github.com/facebookresearch/torchdim) but this is a UX problem that shouldn't be in arrow
You can choose to represent strides or dimension ordering. Dimension ordering is good enough to represent memory ordering, but not good enough to represent a broadcasted input. We are internally experimenting with only recording memory ordering for our mobile runtime, since strides are too expressive and they do not support aliasing in their runtime anyway. It is a lossy conversion (distinct strides can map to the same dimension ordering), but it is not too difficult to define a "best" stride for any given dimension ordering.

@ezyang am I right to read this as: dim_names are good enough for exact in-memory representation but for wouldn't suffice for pytorch's internal representation due to propagation rules etc?

They work perfectly fine if you don't do anything with the names, but yeah, in PyTorch, when you start doing operations on the tensor, it's totally nonobvious how to propagate them.

@ezyang So to be clear: the proposal as is (with dim_names) is good for pytorch?
As I understand it you could zero-copy use it but would have to add your own way of handling strides which would not hurt performance/memory?

Actually, what exactly is the dim_names proposal. I assumed that the ordering of the names tell you how to interpret the physical data layout (outside in). If so, if you loaded one of these tensors into PyTorch, we would just ignore the dim names entirely and just give you a tensor in the physical layout. To reinterpret the meaning of the data is application specific; after all, who decides if you call it N or BATCH or something else? Which wouldn't really be in the purview of a framework to decide

If you wanted PyTorch to automatically restride; eg give the same logical view for both row-major and col-major matrix, you need a different rep. A layout permutation is sufficiently general that it would work for us. Strides would work too. A fixed enum of names and their logical positions would work as well (but people typically dislike that.)

Actually, what exactly is the dim_names proposal.

The proposed logic for calculating strides from dim_names is here and tests it should pass would be:
auto ext_type_4 = extension::tensor_array(int64(), {3, 4, 7}, {"x", "y", "z"});
ASSERT_EQ(ext_type_4->strides(), (std::vector<int64_t>{224, 56, 8}));
ASSERT_EQ(ext_type_4->strides({"x", "y", "z"}), (std::vector<int64_t>{224, 56, 8}));
ASSERT_EQ(ext_type_4->strides({"y", "z", "x"}), (std::vector<int64_t>{56, 8, 224}));
ASSERT_EQ(ext_type_4->strides({"z", "x", "y"}), (std::vector<int64_t>{8, 224, 56}));

I assumed that the ordering of the names tell you how to interpret the physical data layout (outside in). If so, if you loaded one of these tensors into PyTorch, we would just ignore the dim names entirely and just give you a tensor in the physical layout. To reinterpret the meaning of the data is application specific; after all, who decides if you call it N or BATCH or something else? Which wouldn't really be in the purview of a framework to decide

Right! Our current thinking is to provide dim_names in the extension and the ->strides(dim_names)) helper to allow applications derive their own strides if they chose another dimension order. That implies the application has some knowledge of dim_names, but I think that's a given?

If you wanted PyTorch to automatically restride; eg give the same logical view for both row-major and col-major matrix, you need a different rep. A layout permutation is sufficiently general that it would work for us. Strides would work too. A fixed enum of names and their logical positions would work as well (but people typically dislike that.)

Storing a layout permutation is an interesting idea too, but that requires knowledge of the "natural order".

Storing a layout permutation is an interesting idea too, but that requires knowledge of the "natural order".

The natural order is whatever the physical layout is, with contiguous dimensions on the right.

Storing a layout permutation is an interesting idea too, but that requires knowledge of the "natural order".

The natural order is whatever the physical layout is, with contiguous dimensions on the right.

Sure, but if application is not images and we just receive a shaped tensor without dim_names we can't know what they are?

I mean, one way or another the out of bound communication has to happen. In the cases of names, the out of band communication is "sender and receiver agree on common meaning of particular string names" and works pretty OK for specific domains (e.g., images) where these conventions agree. With the case of a layout permutation, the communication is "sender and receiver agree on the semantics of the positions of logical dimensions (e.g., after you apply the permutation)." This is more general because you don't have to have agreed on names for a particular dimension. The user code can then do whatever it needs (e.g., pass channels_last=True/False to an operator on the image) to line up the logical interpretation.

That makes sense yeah. Though I think passing dim_names is equally general to agreeing on natural order + passing permutation in terms of passing information. Except that dim_names gives both in one variable (if we assume dimension names are universal), while natural order + permutation requires two.
If I understand correctly it's more about usability/readability tradeoffs. Am I correct to assume that at pytorch entry time tensor would be interpreted and passed on as a pytorch object and wouldn't cause cause friction?

if we assume dimension names are universal

This is a bonkers thing to assume though, like just get out of batch dimension and image and text data and like, the tensors dimensions can mean literally anything. You're not going to be able to standardize for every case.

Stepping back a little, ISTM that there are two major categories of use cases that should be satisfied:

applications where the dimensions have well-known names that can map to physical order
applications where the dimensions don't have well-known names (they are anonymous) but logical order is a non-trivial permutation of physical order (for example a Numpy array)

Is there a third category of use cases?

if we assume dimension names are universal

This is a bonkers thing to assume though, like just get out of batch dimension and image and text data and like, the tensors dimensions can mean literally anything. You're not going to be able to standardize for every case.

Indeed :D. And that's just on the application entry/exit. If you want to serialize a tensor somewhere mid computation graph you might as well use UUID for dimension naming.

Is there a third category of use cases?

Nothing significant comes to mind.

@pitrou given the two cases - first could be solved with dim_names and second with either strides or permutation. Or did you want to make another point?

@rok I'm just saying that these should be satisfied by the final proposal that you (or someone else) will submit on the ML.

I was hoping you had a solution :D.
I'm starting to think optional strides might be most robust proposal here.

strides will be annoying to consume by implementations, while not bringing any additional flexibility

How so? I see mostly as metadata to be passed between application. Only case where I'd use strides is when creating TensorArray with n-dim tensor elements from a single n+1-dim tensor.

How so? I see mostly as metadata to be passed between application.

Ah, fair enough, perhaps I should have said "applications" instead of "implementations". But the point still holds.

Ah, fair enough, perhaps I should have said "applications" instead of "implementations". But the point still holds.

Strides are the most common API (that is not C or F). For applications that don't use strides we could provide helpers for conversion to C or F.

I will circle back to https://github.com/apache/arrow/pull/33925#issuecomment-1422986313 and let you take that into account.

Personally I am not sure what strides would provide that dim_names and/or permutation couldn't do, while being more difficult to work with and not being a full solution (it doesn't allow for names).
So I would go for either just an optional dim_names (with the convention that if you don't have actual names for dimensions but would like to indicate a typical permutation, you can use integer names to do that), or optional dim_names + optional permutation (if we think that it is important to be able to specify both separately, or if it's cleaner to not mix integers in the dim_names).
Both of those should cover the two use cases that Antoine mentioned above at https://github.com/apache/arrow/pull/33925#issuecomment-1422986313
(in the end, it's maybe mostly a choice between strides and permutation, as both hold the same information but in a different form, and we need dim_names anyway if we want to give the option to preserve dimension name information for applications that do natively have those)

I think optional dim_names + optional permutation would be a good design.
One potential issue would be: when a row-major application would pass data to a column-major application we'd want to force passing of permutation so data is not misinterpreted. I propose permutation is mandatory (it can be derived from C/F by our implementation easily).

I have updated the spec according to the discussion in this PR and the ML plus the talk with @rok and @jorisvandenbossche. You can find the change in Description of the serialization section where dim_names and permutations are now included as an optional metadata.
Please have a look at the text and give comments/suggest changes. Once the PR is ready I will send the new version to the ML for a vote.
Thank you everybody for the participation and feedback!

Thank you everyone for this discussion. The proposal looks good to me, and I would like your help checking my understanding.
In RAPIDS libcudf, we would use an nested List type to represent the proposed Tensor type. In the case where permutation is not provided, I believe we could use the elements zero-copy and then create offsets based on the shape parameter.
So this Tensor example with 2 rows of (2,3) tensors:
value_type: int32
shape: (2,3)
elements: [1,2,3,4,5,6,7,8,9,10,11,12]

would correspond to these List column children:
data_type: int32
offsets: [0,2,4]
  offsets: [0,3,6,9,12]
  elements: [1,2,3,4,5,6,7,8,9,10,11,12]

Would you please let me know if my conversion looks correct?

In RAPIDS libcudf, we would use an nested List type to represent the proposed Tensor type. In the case where permutation is not provided, I believe we could use the elements zero-copy and then create offsets based on the shape parameter.

@GregoryKimball it seems that cudf only has a variable size List type (https://docs.rapids.ai/api/cudf/stable/user_guide/cudf.listdtype)? In the proposal here we use a fixed size list type, which doesn't have offsets. Of course you can use a variable size List type as well, but because each tensor element has the same shape (for this extension type, there is discussion to add a second type with variable shape), the offsets will increase regularly and are somewhat superfluous.
But the main buffer with all values can still be used zero-copy (which is most important here).

would correspond to these List column children:

It's up to cudf to decide which data type to use exactly, but I am not fully sure if it is needed to use a nested list type. For your example where you are using LIST<LIST<INT32>>, you could also use LIST<INT32> where each element of that list array is one flat tensor element (the elements array is still the same, you just have less offsets). But in the end it will depend on what you exactly want to do with this data which representation will be most useful.

Great to see libcudf underlying buffer layout is compatible with this proposal!
I do think variable shape case will additionally need to store tensor shapes as they generally probably cannot be derived.
The variable shape tensor array issue by @chrish42 is #24868.

Thank you @jorisvandenbossche for your reply. Yes, in libcudf we only have a variable-length List type. It's expensive for us to introduce a new type, so our first step to support the proposed fixed-length Tensor type would be to convert to variable-length List type. Thank you for confirming that you expect the elements could be used zero-copy.
Also, thank you for your idea to use flat lists (e.g. LIST<INT32>) instead of nested lists to represent tensors. Unless I'm mistaken, libcudf algorithms for joins, aggregations, stream compaction, etc would all correctly operate on flattened tensors.
Thank you @rok for highlighting the variable-shape Tensor type proposal. I'll take a closer look at #24858 🙏

MATLAB’s Deep Learning Toolbox uses n-dimensional arrays which fit quite well with the proposal. It also has a special datatype called “dlarray” which is responsible for automatic differentiation. This datatype allows labelled dimensions, however, the labels are quite compatible with a permutation. We group dimension labels into categories – spatial/continuous, channel, batch, temporal, unspecified, and potential extensions – where the spatial and unspecified labels (and perhaps others in future) are allowed to be used for multiple dimensions. We find that ‘H’ and ‘W’ are very image-centric and do not extend arbitrarily, while ‘U’ might be used for additional dimensions needed for a variety of purposes in different contexts, such as filter groups, point cloud index and so forth. Consequently we need to keep track of permutations within dimensions with the same name.
Therefore our main input would be to request that permutation and dim_names are not mutually exclusive.
We would also like to ensure that the format will support complex data.
Questions:

Could you clarify whether shape, dim_names and permutation are listed in left-to-right or right-to-left order. As in, is the contiguous dimension the right-most (C, python) or left-most (MATLAB, Fortran) dimension?
Are 1-dimensional and 0-dimensional arrays allowed?
“fixed_shape_array” or variations on multi-dimensional or nd array might be a more appropriate name given the mathematical implications of the term ‘tensor’.

Joss Knight
Development Manager
GPU and Deep Learning Team
MathWorks UK

Thanks you for the input and description of MATLAB's Deep Learning Toolbox @extabgrad !

Therefore our main input would be to request that permutation and dim_names are not mutually exclusive.

As per discussion on the mailing list and this proposal I believe permutation and dim_names will not be mutually exclusive.

We would also like to ensure that the format will support complex data.

Just to be clear: by complex you mean diverse not complex as in complex numbers?

Questions:

Could you clarify whether shape, dim_names and permutation are listed in left-to-right or right-to-left order. As in, is the contiguous dimension the right-most (C, python) or left-most (MATLAB, Fortran) dimension?

shape, dim_names and permutation would map to a row-major (C, python) physical layout of data. Data in the underlying buffer would have row-major layout.

Are 1-dimensional and 0-dimensional arrays allowed?

This wasn't discussed, but the current language doesn't forbid them, so I suppose they are allowed. Do you think we should explicitly allow them?

“fixed_shape_array” or variations on multi-dimensional or nd array might be a more appropriate name given the mathematical implications of the term ‘tensor’.

That's an interesting point. Do you know of a source where this is discussed/argued? I'd be in favor of array-like naming too. The consideration here would be that deep learning frameworks are pushing tensor name and seem to have a lot of momentum.

We would also like to ensure that the format will support complex data.

Just to be clear: by complex you mean diverse not complex as in complex numbers?

And if you mean complex numbers: the value_type in this proposal can be any arrow data type. At the moment, there is no "complex" data type defined in the Arrow format spec, though. But there is a proposal to add a canonical extension type for complex data (https://github.com/apache/arrow/pull/10565), and assuming that this would be included in Arrow, then you can specify a tensor array that uses complex numbers as the individual tensor elements.

5. “fixed_shape_array” or variations on multi-dimensional or nd array might be a more appropriate name given the mathematical implications of the term ‘tensor’.

That's an interesting point. Do you know of a source where this is discussed/argued? I'd be in favor of array-like naming too. The consideration here would be that deep learning frameworks are pushing tensor name and seem to have a lot of momentum.

I don't have an opinion on the whole "array vs tensor" debate, but one aspect in favor of "Tensor" is that it avoids the confusion about whether "FixedShapeArray" is actually an array or type class, and the duplication in the name of "FixedShapeArrayArray".

Just to be clear: by complex you mean diverse not complex as in complex numbers?

No, I mean complex numbers, which are increasingly used in AI workflows.

shape, dim_names and permutation would map to a row-major (C, python) physical layout of data. Data in the underlying buffer would have row-major layout.

I find 'row-major' to be ambiguous. You can have a row major layout and still list dimensions left-to-right [row column page batch].

Are 1-dimensional and 0-dimensional arrays allowed?

This wasn't discussed, but the current language doesn't forbid them, so I suppose they are allowed. Do you think we should explicitly allow them?

I don't know, I just know they are important in numpy.

That's an interesting point. Do you know of a source where this is discussed/argued? I'd be in favor of array-like naming too. The consideration here would be that deep learning frameworks are pushing tensor name and seem to have a lot of momentum.

Are you confident it is clear this datatype is only to be used in machine learning contexts? Perhaps it should be called 'machine-learning-tensor' then?

No, I mean complex numbers, which are increasingly used in AI workflows.

Interesting! As Joris states there is an independent effort to enable that.

shape, dim_names and permutation would map to a row-major (C, python) physical layout of data. Data in the underlying buffer would have row-major layout.

I find 'row-major' to be ambiguous. You can have a row major layout and still list dimensions left-to-right [row column page batch].

This proposal currently states: Elements in a fixed shape tensor extension array are stored in row-major/C-contiguous order.. We can amend that to be more general. Could you state what left-to-right means? I assume it's equal to TensorFlow's minor-to-major.

Are you confident it is clear this datatype is only to be used in machine learning contexts? Perhaps it should be called 'machine-learning-tensor' then?

I hope it'll be used in a wider context. FixedShapeNDArray?

This proposal currently states: Elements in a fixed shape tensor extension array are stored in row-major/C-contiguous order.. We can amend that to be more general. Could you state what left-to-right means? I assume it's equal to TensorFlow's minor-to-major.

I can see it's confusing, since we are distinguishing between the order dimensions are indexed and the order they are stored. Left-to-right means new higher dimensions are added on the right, and right-to-left means they are added on the left.
The minor_to_major field definition states that the left-most entry refers to the contiguous dimension in memory. The order of indexing is then defined by the numbers in this field, so [0 1 2] means that X(i,j,k) is indexing position i+(Mj)+(MN*k). In typical parlance where i represents the row index, this is therefore column-major layout. A minor_to_major of [1 0 2] would therefore be a strict row-major layout so that j can represent the column index. But of course by row-major people typically mean [2 1 0], which means actually k is the column index, since the left-most dimension will be the highest (pages).
If MATLAB data were stored in row-major format but still indexed left-to-right then i would become the column index and no permutation would be needed to translate between MATLAB data and other row-major formats. However, since everyone uses the row,column indexing convention and MATLAB indexes left-to-right, the data is therefore stored in column-major. If 'permutation' is an inherently left-to-right spec then it is [1 0 2] for MATLAB (swap the 'first' two dimensions in memory). But if it's a right-to-left spec then it would be [0 2 1] (swap the 'last' two dimensions in memory).
Hence the ambiguity - so I thought it best to check!
I note that permutation is effectively the inverse of minor_to_major. The former is the memory order relative to the indexing order and the latter is the other way around.


anchor links are needed to prevent excessive scrolling on the endpoints page

Code of Conduct

[X] I agree to follow this project's Code of Conduct

What article on docs.github.com is affected?
https://docs.github.com/en/rest/overview/endpoints-available-for-github-apps
What changes are you suggesting?
I was creating an app with your REST API last night and found myself having to scroll through the endpoints page to find what I needed. I think a list of anchor links for each section should be at the top of the page, right below the intro.
I'm about to submit a PR but am opening an issue first since it was suggested.
Additional information
No response

@christinavoudouris
Thanks so much for opening an issue! This is something that is fixed internally 🐢 So this issue will be transferred to an internal repo 💛


Remove default/test parameters from simulation pipelines

Description

Remove default/test parameters from simulation pipelines
Remove "test" parameter from simulation pipelines
Add test for case when no parameters are enterred for sim
Clarify logging for when no parameters are enterred
Add docstrings for pipelines/simple and pipelines/common
Define test parameters within each pipeline test

Closes #222
Hygiene checklist

[x] Changelog entry
[x] Everything public has a Numpy-style docstring
(modules, public functions, classes, and public methods)
[x] Commit history is cleaned-up with minor changes squashed together
and descriptive commit messages following Tim Pope's style

Cute Animal Picture

Note:
The volume-limited arb CI doesn't pass for this PR or the current main.
In both cases, the strict equality test for per-run D (i.e., the D recorded at the beginning of each run) fails, so I suspect this merely reflects the fixes in #239


add: ohmyzsh-plugins extension

Description

Search Oh My Zsh Plugins and display the plugin README
Screencast

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Hi @k8scat 👋
I think you should be able to do almost the same as I did in the OSS Insight extension, there is no reason for making a call everytime you type 😊


DOESNT WORK

hi dude ı found error pls fix it https://prnt.sc/2583hv3

hi dude ı found error pls fix it https://prnt.sc/2583hv3

can you add more information, like which minecraft version are you using ? which java version? when does this error appear ?

Hey, sorry but the website you posted is not providing any information. You can copy and paste your error directly here.


Augment srb so it can leverage new java gem

This pull-request augments #2298
Specifically, it augments the srb script to do a last-ditch effort to try and match the new mac.sorbet or linux.sorbet binaries based on the uname that are bundled in the java platform gem.
I oped to only do the check at the end since gem env platform might incur some cost.
Testing Plan
I build a local java gem sorbet-static-0.0.0-java.gem and installed it.
I then executed srb tc --help and with the changes it works!

@ptarjan since you have been working with me on https://github.com/sorbet/sorbet/pull/2298
Do you mind if you review this one as well ?

(landing since I gave you a day, but we can always fix it or revert)


Core refactoring

First step for #70
Includes the following

refactor/split code into 2 parts: CLI & API (API needs to be exported in another PR)
replace dynamic entry creation with a static entry file
replace the "glob-like" file resolution with glob

Issues

adding / deleting test entry files in watch mode does not trigger a rebuild
growl needs to be enabled & enhanced to show a notification for webpack's build status for a more consistent developer experience

Current coverage is 76.86% (diff: 73.66%)

Merging #78 into master will increase coverage by 0.88%

@@             master        #78   diff @@
==========================================
  Files            17         15     -2
  Lines           383        389     +6
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits            291        299     +8
+ Misses           92         90     -2
  Partials          0          0

Powered by Codecov. Last update 4d551c2...720d21f


Update list of python requirements to support python 3.9 instead of 3.6

This commit fixes the following build error:
  Command failed: 1

   '/Volumes/D/P/S-0-build/python-install/bin/PythonSlicer' '-m' 'pip' 'install' '--require-hashes' '-r' '/.../ShapeVariationAnalyzer-build/python-ShapeVariationAnalyzer-requirements-requirements.txt' '--prefix' '/.../ShapeVariationAnalyzer-build/python-packages-install'

  See also

    /.../ShapeVariationAnalyzer-build/python-ShapeVariationAnalyzer-requirements-prefix/src/python-ShapeVariationAnalyzer-requirements-stamp/python-ShapeVariationAnalyzer-requirements-install-*.log

The updates was performed following these steps using the updated
"python_package_updater.py" script assocated with PR Slicer/Slicer#6300:
# Set common variables
SLICER_SOURCE_DIR=~/Projects/Slicer
SLICER_DIR=~/Projects/Slicer-Release/Slicer-build
PYTHONSLICER_EXECUTABLE=${SLICER_DIR}/../python-install/bin/PythonSlicer

EXTENSION_SOURCE_DIR=/home/jcfr/Projects/ShapeVariationAnalyzer
EXTENSION_BINARY_DIR=/home/jcfr/Projects/ShapeVariationAnalyzer-Release

# Install packages into the prefix
PYTHON_INSTALL_PREFIX=${EXTENSION_BINARY_DIR}/python-packages-install
${PYTHONSLICER_EXECUTABLE} -m pip install --prefix ${PYTHON_INSTALL_PREFIX} scikit_learn

# Update extension external projects
UPDATER_SCRIPT=${SLICER_SOURCE_DIR}/Utilities/Scripts/python_package_updater.py

${PYTHONSLICER_EXECUTABLE} ${UPDATER_SCRIPT} \
  -s ${EXTENSION_SOURCE_DIR}/SuperBuild      \
  --from-installed-packages \
  --path ${PYTHON_INSTALL_PREFIX}/lib/python3.9/site-packages

This addresses the issue initiated here: https://github.com/DCBIA-OrthoLab/ShapeVariationAnalyzer/issues/55

@jamesobutler  Thanks for the review, it turns out that I had improperly specified the hints and this explaining why the joblib wasn't updated as expected.
I revisited the topic and now all requirements are correctly updated based on what was installed after running:
PYTHON_INSTALL_PREFIX=${EXTENSION_BINARY_DIR}/python-packages-install
${PYTHONSLICER_EXECUTABLE} -m pip install --prefix ${PYTHON_INSTALL_PREFIX} scikit_learn

Ok, looks like the python packages are appropriately updated by the utility. I'll let you all determine if scikit-learn can be updated from 0.23.1 to 1.0.2 without any additional changes. I have not looked into potential scikit-learn API changes.


[typing] Chip definition was missing deleteIcon & more

The TypeScript definition for the Chip component was missing the deleteIcon property.

@cauld Thanks


Transfer libretime.org to Gandi.net

Related to #7
This ticket track the transfer of the domain name from @hairmare's domain name provider to Gandi.net.
See https://github.com/libretime/organization/issues/7#issuecomment-1147517154

@hairmare will you have a chance to do this soon?

@hairmare We are now a little less than a month before the renewal date of the libretime.org domain name.
I think we should do the transfer, could you have a look and start the transfer please ?

I'm preparing the nameserver change to use gandi's LiveDns, here is what I could extract from the current records:
www.libretime.org.          3600 IN CNAME libretime.org.
libretime.org.              3600 IN CNAME libretime.github.io.
discourse.libretime.org.    3600 IN CNAME libretime.hosted-by-discourse.com.
chat.libretime.org.         3600 IN CNAME prod.cluster.bitcast.co.za.

Great, the nameservers are now set, we can now configure emails and other stuff.
jo@jofix: ~ $ dig +noall +answer 'libretime.org' any @ns-144-a.gandi.net
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	AAAA	2606:50c0:8000::153
libretime.org.		300	IN	AAAA	2606:50c0:8001::153
libretime.org.		300	IN	AAAA	2606:50c0:8002::153
libretime.org.		300	IN	AAAA	2606:50c0:8003::153
libretime.org.		10800	IN	MX	10 spool.mail.gandi.net.
libretime.org.		10800	IN	MX	50 fb.mail.gandi.net.
libretime.org.		10800	IN	TXT	"v=spf1 include:_mailcust.gandi.net ?all"
libretime.org.		10800	IN	NS	ns-144-a.gandi.net.
libretime.org.		10800	IN	NS	ns-150-b.gandi.net.
libretime.org.		10800	IN	NS	ns-224-c.gandi.net.
libretime.org.		10800	IN	SOA	ns1.gandi.net. hostmaster.gandi.net.<PHONE_NUMBER> 10800 3600<PHONE_NUMBER>0
jo@jofix: ~ $ dig +noall +answer 'chat.libretime.org' any @ns-144-a.gandi.net
chat.libretime.org.	300	IN	CNAME	prod.cluster.bitcast.co.za.
prod.cluster.bitcast.co.za. 1800 IN	A	<IP_ADDRESS>
jo@jofix: ~ $ dig +noall +answer 'discourse.libretime.org' any @ns-144-a.gandi.net
discourse.libretime.org. 300	IN	CNAME	libretime.hosted-by-discourse.com.
jo@jofix: ~ $ dig +noall +answer 'www.libretime.org' any @ns-144-a.gandi.net
www.libretime.org.	300	IN	CNAME	libretime.org.
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	A	<IP_ADDRESS>
libretime.org.		300	IN	AAAA	2606:50c0:8000::153
libretime.org.		300	IN	AAAA	2606:50c0:8001::153
libretime.org.		300	IN	AAAA	2606:50c0:8002::153
libretime.org.		300	IN	AAAA	2606:50c0:8003::153
libretime.org.		10800	IN	MX	10 spool.mail.gandi.net.
libretime.org.		10800	IN	MX	50 fb.mail.gandi.net.
libretime.org.		10800	IN	TXT	"v=spf1 include:_mailcust.gandi.net ?all"
libretime.org.		10800	IN	NS	ns-144-a.gandi.net.
libretime.org.		10800	IN	NS	ns-150-b.gandi.net.
libretime.org.		10800	IN	NS	ns-224-c.gandi.net.
libretime.org.		10800	IN	SOA	ns1.gandi.net. hostmaster.gandi.net.<PHONE_NUMBER> 10800 3600<PHONE_NUMBER>0
jo@jofix: ~ $

Thanks @hairmare


Upgrade and take advantage of latest versions of GDAL and rasterio

Original report by Oyono (Bitbucket: aoyono, GitHub: aoyono).

For the get_data functionality of eodag, we currently use rasterio/gdal, which are evolving independantly. There were new releases of these tools that may enable eodag to do sub-setting without downloading the data locally. This issue aims at exploring this possibility. As a side work for this, the api name of this functionality should be changed

related to #2


Strip down namespace object and clean up APIs

Part of FIR-12
Remove all remaining functionality related to broadcasting namespaces. It was already broken from previous commits, but this is now to confirm:

any previously-broadcast namespaces that are not spelled out in the config will cease to function in the next release
in the future, if we want to add back dynamic namespace capability, it will probably take the form of editing the config file and adding a local API that reloads the config dynamically

Clean up the /namespaces/{ns} and /namespaces/{ns}/status APIs to be a little more organized.

Codecov Report

Merging #898 (f3b65db) into main (2adbebb) will decrease coverage by 0.00%.
The diff coverage is 96.77%.

@@            Coverage Diff             @@
##             main     #898      +/-   ##
==========================================
- Coverage   99.95%   99.95%   -0.01%
==========================================
  Files         298      297       -1
  Lines       19341    19261      -80
==========================================
- Hits        19333    19252      -81
- Misses          6        7       +1
  Partials        2        2

Impacted Files
Coverage Δ

internal/apiserver/route_get_websockets.go
100.00% <ø> (ø)

internal/database/sqlcommon/sqlcommon.go
100.00% <ø> (ø)

internal/definitions/handler.go
100.00% <ø> (ø)

internal/namespace/persistence_events.go
100.00% <ø> (ø)

internal/reference/reference.go
100.00% <ø> (ø)

internal/txcommon/event_enrich.go
100.00% <ø> (ø)

pkg/core/event.go
100.00% <ø> (ø)

internal/orchestrator/orchestrator.go
97.89% <88.57%> (-2.11%)
:arrow_down:

internal/apiserver/route_get_namespace.go
100.00% <100.00%> (ø)

internal/apiserver/route_get_status.go
100.00% <100.00%> (ø)

... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2adbebb...f3b65db. Read the comment docs.

Merging this, made a task on the board to revisit these comments before release.


Mat/Pat - Make the loader load as soon as you hit the approve button for supervisor

Make the loader load as soon as you hit the validate submission button for supervisor. This has been implemented for telework and should be a very easy change
AC

[ ] Loader should load as soon as supervisor hits the validate submission button
[ ] Add a message to loader saying "Submitting...this can take a few minutes"

Discussed with DEV team, assigned this one to myself.

Discussed with DEV team, assigned this one to myself.

Please test, thanks!

I will request the team at check-in to test this, since I am not able to approve my own submission!

The team while Bug bash tried testing this out, but the message did not appear right away; only the dancing balls appeared.
Can you please look into this Jerry?

the message did come on 1st which is great BUT the dance balls replaced in for a pretty long time before the validation message came up.
Could we keep the message up UNTIL the validati0on page appears, as we would not want user to close the page before the process is complete
Private Zenhub Image

Ready to test, thanks.

The loading message appeared as soon as I hit 'validate submission'. Thanks Jerry!

I see the loader "Submitting...this can take a few minutes" throughout, did not see dancing balls. Looks good


How to trigger a response without user inputs shown?

Hey Ovidijus,
I hope this message finds you well.
I'm trying to make the deep-chat backend response to another button out of the deep-chat. So I need the ai response without a user input shown in the message container.
I have tried to use the 'submitUserMessage'. The message still shows in the messages container, which diverges from my intended outcome.
I also tried to add a div directly in the messages div. But I can't get the div in the container by querySelector. The messages div is undefined.
const chatElementRef = this.$refs.chatElementRef;
const shadowRoot = chatElementRef.shadowRoot;
const messages = shadowRoot.querySelector('#messages');
console.log(messages)

Another question:
It seems that if the user inputs two continuous messages, there will be an avatar image bug.  I can input two continuous messages because I set the initialMessages as follows:
this.initialMessages = [
        { role: 'ai', text: 'Hi!' },
        { role: 'user', text: 'Hello' }
      ]

Actually, I think the two questions are similar.

When I input again, the first user image is hidden:

I would greatly appreciate your insight or guidance on addressing these questions.

Hi @ziao-liu.
If by "I need the ai response without a user input shown in the message container" - you meant that you don't want to see the user messages in the chat, this can simply be achieved by using the messageStyles property as follows:
messageStyles = {
  default: {
    user: {
      outerContainer: {
        display: 'none',
      }
    }
  }
}

In regards to the question about not displaying multiple avatars, this was an intended change that came in version 1.4.11 as repeating avatars and names looked unappealing, especially if they were big. Only using an avatar/name for the last message (in a repeated group of messages) is a common practice for modern chats such as Facebook Messenger.
However, if you want the avatars/names to appear for every message, I have made a small change in our deep-chat-dev and deep-chat-react-dev packages version 9.0.149 where you can use the following to make them visible:
avatars = {
  ai: {
    styles: {
      container: {
        visibility: 'visible',
      }
    }
  }
}

One thing to mention about the dev packages - they behave the same way as the original ones except their names are different. The next release for Deep Chat is going to be a big one as we are migrating a lot of its logic to another web component called active chat and with that we are changing quite a few property names, such as changing the request property to connect and initialMessages to history. The old properties will still keep working as usual except you will see TS/console warnings for their deprecation which can be ignored.
Let me know if you have any further questions.
Thanks!

Thank you for your quick response and the solution provided.
Sorry for the incomplete expression.
"I need the ai response without a user input shown in the message container" is one of my requirements
I have two requirements :

Normal chat with both kinds of texts shown in the deep-chat interface.
A button click, external to the deep-chat interface, displays the AI response in the message container without needing user text input in the chat box.

To meet this, setting display: 'none' addresses the second requirement but not the first.
Maybe I need a way to display text or response directly in the chat-view
Thanks!

I think I know what you are looking for.
Deep Chat has a private method property called _addMessage that you can use to manually add a message for any role. It accepts a MessageContent object as an argument. You can use it like this:
chatElementRef._addMessage({text: 'example text', role: 'ai'});

We plan to make this an official method in the next release and the dev package has it called as addMessage.
Let me know if this helps.
Thanks!

Hey Ovidijus,
Thank you!
I installed the deep-chat-dev and used addMessage to solve my problems successfully.
Thank you so much @OvidijusParsiunas!


Upgrade to <IP_ADDRESS> CTRE Phoenix

This should fix issue with remote reverse limit switch configuration on Falcons.

Upgrade complete in a02d6ab


Squash commits on master

Commits aren't squashed on a PR merge.
This makes the repo history hard to read: we're different devs of different skill levels. People that are just starting out usually commit a lot and people with a lot of experience tend to either commit less (when working on small stuff) or get even more tangled with commits than juniors :)
It's really easy to implement a squash policy:
https://docs.github.com/en/free-pro-team@latest/github/administering-a-repository/configuring-commit-squashing-for-pull-requests
Maybe we should also update the contributing guidelines.
I also wanted to add some extra info to the main .md file so maybe i could bundle that up in a documentation update.

It's already configured to allow squash merges, it's just not the default option. I was considering it myself, since some PRs ended up with a lot of commits just based on code review alone, but I was worried that having big commits (for each PR) wouldn't be too nice either. I wonder where the balance lies - between having very large commits and having a polluted repo history.
How does the PR merge squash deal with merge commits in the PR? As if they don't exist, and just add the entire added code as a new commit to the repo?

Well, any commit on the branch merged is included in the commit
description. So if people use suggestive commit messages, that works.
Define very large. IMO if you need to change 50 files, it’s your fault for
doing it all in one commit. That pr should’ve been an epic, broken into
tasks, than subtasks. Then you’d get much nicer issue tracking, history
logging and less complex code reviews
On Sun, 4 Oct 2020 at 00:01, Ioana Alexandru<EMAIL_ADDRESS>wrote:

It's already configured to allow squash merges, it's just not the default
option. I was considering it myself, since some PRs ended up with a lot of
commits just based on code review alone, but I was worried that having big
commits (for each PR) wouldn't be too nice either. I wonder where the
balance lies - between having very large commits and having a polluted repo
history.
How does the PR merge squash deal with merge commits in the PR? As if they
don't exist, and just add the entire added code as a new commit to the repo?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/acs-upb-mobile/acs-upb-mobile/issues/60#issuecomment-703164197,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACMESVMJQE5SIAHWLMTW2WTSI6GKDANCNFSM4SDHTL4A
.

Yeah, I think you're right and @GeorgeMD also brought this up without even seeing your issue. Please go ahead and update the documentation, and I'll do my best to enforce the squash policy from now on (looks like there's unfortunately no way to make it the default option - https://github.com/mapbox/rasterio/issues/732).


[Product Category] Can top/bottom padding be taken from the AEM Heading component

You can select p-0 - p-10 but this doesn't show on the page.

As discussed yesterday this should ideally not be driven be an author and controlled by the page styles only. Closing for now.


withTimeout

Would be useful to have a function to easily get timeout for a fetch function.
import {withTimeout} from 'fetch-extras';

const fetchWithTimeout = withTimeout(fetch, 5000);

const response = await fetchWithTimeout('/api');
const data = await response.json();

POC (untested):
export function withTimeout(fetchFunction, timeout) {
	return async (urlOrRequest, options = {}) => {
		const controller = new AbortController();

		const timeoutId = setTimeout(() => {
			controller.abort();
		}, timeout);

		const signal = options.signal ?? (urlOrRequest instanceof Request && urlOrRequest.signal) ?? controller.signal;

		try {
			return await fetchFunction(urlOrRequest, {...options, signal});
		} finally {
			clearTimeout(timeoutId);
		}
	};
}

Feedback wanted

If this one works out, I imagine we could have more.
const superFetch = withHttpError(withTimeout(fetch, 5000));

const response = await superFetch('/api');
const data = await response.json();

Alternative API:
const betterFetch = createFetch(
    fetchTimeout(5000),
    fetchRetry(3)
);

const response = await betterFetch('/api');
const data = await response.json();

fetchWithTimeout

I think if you need a reusable function, there's already ky. Timeouts on their own are easy:
const response = fetch('/api', {signal: Abort.timeout(5000)})

I think if you need a reusable function, there's already ky.

You could use that argument on the utilties already here too. Could just use Ky. The point is that this package is composable. You take only what you need.

Timeouts on their own are easy:

Sure, but usually you also want to pass in your own signal, and then it's more difficult.

You could use that argument on the utilties already here too

throwIfHttpError is not 4-tokens long, , {signal: AbortSignal.timeout(5000)} is

this package is composable

👍
I thought it was more like a utilities package like uint8array-extras than a build-your-own-ky

Sure, but usually you also want to pass in your own signal, and then it's more difficult.

A bit, yes
- const fetchWithTimeout = withTimeout(fetch, 5000);
- fetchWithTimeout('/api', {signal: mySignal})
+ fetch('/api', {signal: AbortSignal.any([yourOwn, AbortSignal.timeout(5000)]))


Title: Smooth Tick Option and Explore Button Animation

Pull Request
Title
Smooth Tick Option and Explore Button Animation
Description
Smooth Tick Options Loading: Added a container for tick options to ensure they appear smoothly one by one without affecting the position of other elements.
Explore Button Animation: Implemented an eye-catching animation for the explore button to maintain visibility and encourage user interaction.
Related Issues
Fixes #390
Changes Made
Added CSS for smooth tick option animation.
Implemented animation for the explore button to enhance user engagement.
Checklist

[x] I have tested the changes locally
[ ] Documentation has been updated (if necessary)
[x] Changes are backward-compatible

Screenshots (if applicable)
https://github.com/VaibhavArora314/StyleShare/assets/133522303/8fe1e35b-ddfe-40d4-ae26-2d96b776a543
Additional Notes

Footer

Sir , kindly assign gssoc label and level. Thank you !


Full width/half width numbers treated differently (furigana)

7時 has furigana for only 時. (7 is half width character.)
２週間 has furigana for entire word. (2 is full width character.)
１０時 has furigana for only ０時. (Both 1 and 0 are full width characters.)

Fixed in v1.3.2.


Sealing return value of currentState

I was looking over the source code, and wondered if we truly want to make state read-only, could we apply object.seal to the state in order to make it immutable and read-only?
Here's where I was thinking about implementing it.
Legit link

object.freeze could also be of interest

object.freeze is a bit sketchy, since the prototype is muteable, also you can push things into, i.e an array.

This would need to be a major bump if it was implemented, as it would probably break someones code.

Sure, but isn't that what is the main point of redux? preserving state as read-only? Also, object.freeze is not 100% the best fit, since prototype & pushing to things like arrays are allowed.
@onbjerg do you know if the source code checks if your state has prototypes/constructors of some sort?

We recommend using something like deep-freeze in tests to ensure immutability. However we likely won’t enforce something like this in Redux itself because of potential edge cases where mutation is desired (though they are very rare), and performance implications of deep freezing everything.
You can use something like https://github.com/leoasis/redux-immutable-state-invariant to ensure you don’t mutate the state by accident.

Can you tell me an example of this rare use case? @gaearon
Mutating state by accident is hard, if you allow obj.seal to persist and decide for things like writable, enumerable, configurable

Can you tell me an example of this rare use case?

Redux DevTools is one example where we mutate the state to avoid too many allocations when we know it’s safe to do.

Mutating state by accident is hard

Then why do you wish to seal the state? For stronger guarantees, you can always use ImmutableJS or https://github.com/rtfeldman/seamless-immutable.

I meant in context of object.seal, sorry!

Mutating state by accident is hard

Yes. Sealing the state will make, it hard to override the previous state with non-readable actions. If we want it immutable, then prevState should be different from state when passing state as the previous one. (Leaving state blank). I'm terrible at explaining, but if you seal the state, you'll avoid mutability on the same state object and instead you'll have to create a new one. Please, do correct me if I'm wrong.

Holy macaroni my English is broken asf :cry:

What I'm trying to say, is that you should not override your previousState when you have reduced it or when it has been rendered/ is outside your scope or context.

I understand this. But I wrote that this has performance implications (will be slow!) so we don't want to do this by default. There are ways to enforce this as opt in if you'd like.


Fix geo e2e tests

Description of changes

Address some of the failing e2e tests for Geo.
amplify-prompts automatically makes the selection for the user when there is only one choice in the selection set.
This could interfere with the nspawn process chain of commands defined in the e2e test setup.
We are avoiding such cases by having multiple geo resources in the e2e tests.
Issue #, if available

Description of how you validated changes
Ran e2e tests locally
Checklist

[x] PR description included
[x] yarn test passes
[x] Tests are changed or added
[ ] Relevant documentation is changed or added (and PR referenced)
[ ] New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #8310 (e0f3005) into master (5c9e216) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #8310   +/-   ##
=======================================
  Coverage   55.66%   55.66%
=======================================
  Files         691      691
  Lines       38542    38542
  Branches     7811     7811
=======================================
  Hits        21456    21456
  Misses      16296    16296
  Partials      790      790

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c9e216...e0f3005. Read the comment docs.


SEAB-4269: Ensure safe sourcefile paths

Description
This PR changes the webservice so that SourceFile paths can only be composed of a safe set of characters.  For details regarding the motivation, see https://ucsc-cgl.atlassian.net/browse/SEAB-4269.
I surveyed the ~150k SourceFiles present in a recent db dump, and their paths contained letters, numbers, or /, ., -, _, exclusively.  No other characters were used.  All of these characters are relatively inert on the command line, and thus, they form set of acceptable characters implemented here.
This PR checks the paths both via code in the SourceFile setter methods, and as a @Pattern annotation on the fields.    The former throws a descriptive exception upon the attempted assignment of a bad path value, while the latter serves as a backup, throwing an exception at the point that a bad path would be persisted at the end of request.
I have mixed feelings about limiting the paths so restrictively.  Yes, it's English-centric, but from an interoperability and safety point of view, it may be best.
There will be a companion PR which adds a couple of equivalent db constraints during the 1.13 migration.  The motivation for a "two stage" deployment (source code changes, followed by db constraints) is that once this PR gets into production, it will be very unlikely that any "bad" characters will be subsequently introduced into the database.  Then, before we add the db constraints, we can check the columns in question for an existing violation, and if there is none, add the constraints with minimal risk (of an existing violation, which would make the deploy fail). #4909
The "unit test" portion of the CircleCI is failing, with the "Unauthorized" results that others have been seeing.  It appears that my PRs targeted to develop are passing, but those to hotfix/1.12.1 are not.  This is a blocker at the moment.
Issue
https://ucsc-cgl.atlassian.net/browse/SEAB-4269
Please make sure that you've checked the following before submitting your pull request. Thanks!

[x] Check that you pass the basic style checks and unit tests by running mvn clean install
[x] Follow the existing JPA patterns for queries, using named parameters, to avoid SQL injection
[x] If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
[x] Assume that inputs to the API can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
[x] Do not serve user-uploaded binary images through the Dockstore API
[x] Ensure that endpoints that only allow privileged access enforce that with the @RolesAllowed annotation
[x] Do not create cookies, although this may change in the future

Extra thought: since most filenames come from git, what does git allow anyway? Maybe we can have hosted workflows just match that

Git allows almost every character.  I'm not sure about NULL, but see the repo in the Jira for some examples...

The "unit test" portion of the CircleCI is failing, with the "Unauthorized" results that others have been seeing. It appears that my PRs targeted to develop are passing, but those to hotfix/1.12.1 are not. This is a blocker at the moment.

Hmmm, try cherry-picking a812a8e and then 7b6c7a7

I have mixed feelings about limiting the paths so restrictively. Yes, it's English-centric, but from an interoperability and safety point of view, it may be best.

I don't know how I feel about this yet either. Limiting control characters and non-printing characters seems like a no-brainer. But limiting accented characters ( https://stackoverflow.com/questions/18123501/replacing-accented-characters-with-plain-ascii-ones ) or normally valid filenames ( https://en.wikipedia.org/wiki/Filename#Comparison_of_filename_limitations ) seems a little discuss-able.
Can you take a quick look and see what systems in our field use? Maybe try hacking a bit on Terra, SBG, Nextflow Tower, etc.?

7B, (see near bottom of page, matches our regexp:
https://docs.sevenbridges.com/docs/organize-files
Terra: see section 1.4, workflow name, matches our regexp:
https://support.terra.bio/hc/en-us/articles/360031366091-Create-edit-and-share-a-new-workflow

Terra: see section 1.4, workflow name, matches our regexp: https://support.terra.bio/hc/en-us/articles/360031366091-Create-edit-and-share-a-new-workflow

Hmmm, what about imports though?
CWL has this

If the path contains POSIX shell metacharacters (|,&, ;, <, >, (,), $,`, , ", ', , , and ) or characters not allowed for Internationalized Domain Names for Applications then implementations may terminate the process with a permanentFailure.

https://www.commonwl.org/v1.0/Workflow.html#Data_model

Terra: see section 1.4, workflow name, matches our regexp: https://support.terra.bio/hc/en-us/articles/360031366091-Create-edit-and-share-a-new-workflow

Hmmm, what about imports though?
CWL has this

If the path contains POSIX shell metacharacters (|,&, ;, <, >, (,), $,`, , ", ', , , and ) or characters not allowed for Internationalized Domain Names for Applications then implementations may terminate the process with a permanentFailure.

https://www.commonwl.org/v1.0/Workflow.html#Data_model

Apparently, the POSIX portable filename character set is [-a-zA-Z0-9._]
https://www.google.com/search?q=posix+portable+file+name+character+set
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282

Apparently, the POSIX portable filename character set is [-a-zA-Z0-9._] https://www.google.com/search?q=posix+portable+file+name+character+set https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282

How does this compare with the http://www.iana.org/assignments/idna-tables-6.3.0/idna-tables-6.3.0.xhtml that CWL references? Basically, I'm wondering if we'll end up rejecting workflows that are valid CWL

Apparently, the POSIX portable filename character set is [-a-zA-Z0-9._] https://www.google.com/search?q=posix+portable+file+name+character+set https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282

How does this compare with the http://www.iana.org/assignments/idna-tables-6.3.0/idna-tables-6.3.0.xhtml that CWL references? Basically, I'm wondering if we'll end up rejecting workflows that are valid CWL

Yes, as you mentioned, CWL appears to allow a broader range of characters in filenames than the "POSIX portable" spec.  For example, a CWL workflow filename can include a colon, which is not legal on Windows file names.  It also appears to allow ~, which could theoretically cause problems in downstream systems.

FYI @svonworl  this was probably auto-closed due to the hotfix release


Vernemq crashes after upgrade from 1.7 to 1.10.1

Hello, we have tried to upgrade from vernemq to 1.10.1.
After the upgrade vernemq crashes, version 1.7 works fine.

VerneMQ Version: 1.10.1
OS: Debian buster
Erlang/OTP version : standard, installed vernemq from package
VerneMQ configuration (vernemq.conf) standard config authenticating using a lua mysql script
Cluster size/standalone: standalone

Expected behavior

Upgrade to version 1.10.1 without problem

Actual behaviour

1.10.1 vernemq crashes, this is output at error.log
2020-01-20 10:16:45.173 [error] <0.2321.1>@vmq_cluster:is_ready:69 Ranch listener {{0,0,0,0},1883} terminated with reason: bad argument in call to ets:lookup(vmq_status, ready) in vmq_cluster:is_ready/0 line 69
2020-01-20 10:16:45.173 [error] <0.31842.0>@vmq_cluster:is_ready:69 Ranch listener {{0,0,0,0},1883} terminated with reason: bad argument in call to ets:lookup(vmq_status, ready) in vmq_cluster:is_ready/0 line 69
2020-01-20 10:16:45.173 [error] <0.4052.0>@vmq_cluster:is_ready:69 Ranch listener {{0,0,0,0},1883} terminated with reason: bad argument in call to ets:lookup(vmq_status, ready) in vmq_cluster:is_ready/0 line 69
2020-01-20 10:16:45.173 [error] <0.4956.0>@vmq_cluster:is_ready:69 Ranch listener {{0,0,0,0},1883} terminated with reason: bad argument in call to ets:lookup(vmq_status, ready) in vmq_cluster:is_ready/0 line 69
2020-01-20 10:16:45.173 [error] <0.1130.0>@vmq_cluster:is_ready:69 Ranch listener {{0,0,0,0},1883} terminated with reason: bad argument in call to ets:lookup(vmq_status, ready) in vmq_cluster:is_ready/0 line 69
2020-01-20 10:18:07.676 [error] <0.255.0>@vmq_reg_trie:trie_match_#:378 gen_server vmq_systree terminated with reason: bad argument in call to ets:lookup(vmq_trie, {trie_edge,{[],root},<<"#">>}) in vmq_reg_trie:'trie_match_#'/2 line 378
2020-01-20 10:18:07.676 [error] <0.255.0>@vmq_reg_trie:trie_match_#:378 CRASH REPORT Process vmq_systree with 0 neighbours crashed with reason: bad argument in call to ets:lookup(vmq_trie, {trie_edge,{[],root},<<"#">>}) in vmq_reg_trie:'trie_match_#'/2 line 378
2020-01-20 10:18:07.677 [error] <0.252.0>@vmq_reg_trie:trie_match_#:378 Supervisor vmq_metrics_sup had child vmq_systree started with vmq_systree:start_link() at <0.255.0> exit with reason bad argument in call to ets:lookup(vmq_trie, {trie_edge,{[],root},<<"#">>}) in vmq_reg_trie:'trie_match_#'/2 line 378 in context child_terminated
2020-01-20 10:18:09.396 [error] <0.4862.0>@vmq_mqtt_fsm:check_user:563 can't authenticate client {[],<<"history-collector19W6JUKKGT">>} from <IP_ADDRESS>:33588 due to no_matching_hook_found
2020-01-20 10:18:11.038 [error] <0.7632.0>@vmq_mqtt_fsm:check_user:563 can't authenticate client {[],<<"history-collectorP6PES16YP2">>} from <IP_ADDRESS>:33762 due to no_matching_hook_found

Yes, we kept the old config.
After using the vernemq.conf file (adapted to our setup) and removing /var/lib/vernemq/ everything went well.


Spotify for Artist doesn’t count streaming with new LIBRESPOT

Hi, there is an important problem in the new librespot! Volumio software installed on raspberry pi zero w using this new library for his Spotify connect plug-in and the problem is that Spotify for artist doesn’t count the streaming made from this plug in that use your library for connect to Spotify library. The old plug in was perfect because used the old version of librespot and the streaming count was every time regular, but now ZERO! this is important because is like a pirate protocol streaming now, because Spotify doesn’t pay the royalty to the artists in this case because this library doesn’t permit the count of it! So please check what is change from old to new library and correct otherwise is against the law! Let me know updates so I can try if the problem is correct thanks.

See also https://github.com/librespot-org/librespot/discussions/626

and correct otherwise is against the law!

... OK.
As per the linked issue above, reporting has not worked since at least 2019 - I'm pretty sure reporting has never worked. But you appear to be saying it used to work and now doesn't. So can you please clarify how #626 differs from what you are describing and also provide the revision where it used to work.

Indeed seems like a duplicate to me.


chore(main): release 1.0.1

:robot: I have created a release beep boop
1.0.1 (2024-05-03)
Bug Fixes

Added control if counter collection exists (c6b2d2c)
deps: update dependency mongodb to v6.4.0 (18c2690)
Implemented body request field validation for user creation or modification (c8b4e89)
Implemented the possibility to updated the type of a user (904b5b7)
Modified method that update the user data (dfef51b)
Modified route and method that updated user data (e4f21eb)
Modified User Unique Code Generation (5ac0296)
Updated midleware, it can check if users have permissions to access the defined routes. (64d7ed7)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/WMS-Corporation/wms-users-service/releases/tag/v1.0.1 :sunflower:


Give support functions access to the simulation time

We have the module 'functions' which supplies things like step or if_then_else. We import these functions into the model with import functions and access them as functions.step(a, b), etc. Some of these functions, such as step need access to the current time of the simulation. In v0.3, we handled this by making the functions attributes of a class, the class having a pointer to the components class, etc.
In the interests of simplicity, I'd like to move away from the class architecture, and have these be regular functions. This means that in order to have access to the model time, they need to be instantiated within the components namespace.
The simplest way to do this may be to use (instead of import) the execfile command, which bypasses the module administration. This would create all of the functions within the scope of components.
This isn't quite as clean as having the module import, and there is something floating around the parts of my mind that are generally active when walking through bear country at night, but it is simple.
If the function names are vensim keywords, then any vensim model we write won't have a name conflict. But when we expand to have XMILE imports, we can't trust this. It would be nicer still to have the namespace protection, but just to share the time value with the imported namespace.
Another issue is that we want to make sure that these functions don't get caught up in the cache, as they may be called by different parts of the model, and need to provide different return values based upon that parameter. Namespace protection would help with that.

So we can share the time function with the imported module, but adding it to the __builtins__ dictionary:
from pysd import functions

def time():
    return _t

functions.__builtins__.update({'time':time})

_t = 2
print functions.step(10, 4)

0
_t = 5
print functions.step(10, 4)

10
I like this solution.

This might give us trouble, actually, because the time function might be set as a module attribute, and subsequent models might share this module, and have conflicts. This is something to investigate carefully.

Implemented in https://github.com/JamesPHoughton/pysd/commit/79a9efc4c5cbeff828941be33233b6b3c70f2bf9,
Need to construct 2-model test, still.

@mhy05 can you test this with your models?

So within the ipython notebook, this is problematic:
from pysd import functions
def time():
    return _t
functions.__builtins__.update({'time':time})
_t = 2
functions.step(10, 4)
>>> 0

_t = 5
functions.step(10, 4)

>>> 10

from pysd import functions as f2
def time2():
    return _t2
f2.__builtins__.update({'time':time2})
_t2 = 2
f2.step(10, 4)
>>> 0

functions.step(10, 4)
>>>0

If functions was loaded twice, and had separate namespaces for functions and f2, then the last line should still read 10. So they are sharing namespaces. This is possibly because they are in the same file, and python chose not to import them twice, as they are imported into the same namespace. We'll need a more elaborate test.

Testing in context, it seems to be ok:
mdl_file = 'tests/test-models/tests/input_functions/test_inputs.mdl'
model1 = pysd.read_vensim(mdl_file)
model1.run(return_timestamps=np.arange(0,2,.5), return_columns=['test_step'])

test_step

0.0	0
0.5	0
1.0	1
1.5	1

model1.components.time()

1.5

model2 = pysd.read_vensim(mdl_file)
model2.components.time()

0

model2.run(return_timestamps=np.arange(0,2,.5), return_columns=['test_step'])

test_step

0.0	0
0.5	0
1.0	1
1.5	1

So in the second model, the time function that is being called is the one that belongs to the second model, as it should be. I think we're done here...


Fix images in the plugin marketplace

Summary 🎯
The images aren't showing properly:

pls re-upload after merging :)

Thanks for pointing out! :)


Nested eager with HasManyRelation & ManyToManyRelation: no related data

Hello,
I have trouble with using nested .eager() relations, I can't fetch all nested relations.
It seem's that nested HasManyRelation and ManyToManyRelation requests have not sufficient information to allow the result to retrieve all the related data.
All (nested) requests are well executed, but nested data seem's "lost" when result is recomposing.
Models
Here are Masterdeals, MasterdealsBuyers and the Buyers models
const { Model } = require('objection');

class MasterdealsModel extends Model {
  static get tableName() {
    return 'masterdeals';
  }

  static get idColumn() {
    return 'masterdeal_id';
  }

  static get relationMappings() {
    return {
      buyers: {
        relation: this.ManyToManyRelation,
        modelClass: BuyersModel,
        join: {
          from: 'masterdeals.masterdeal_id',
          through: {
            from: 'masterdeals_buyers.masterdeal_id',
            to: 'masterdeals_buyers.buyer_id'
          },
          to: 'buyers.buyer_id'
        },
      }
    }
  }
}

class MasterdealsBuyersModel extends Model {
  static get tableName() {
    return 'masterdeals_buyers';
  }
}

class BuyersModel extends Model {
  static get tableName() {
    return 'buyers';
  }

  static get idColumn() {
    return 'buyer_id';
  }

  static get relationMappings() {
    return {
      ssp: {
        relation: this.BelongsToOneRelation,
        modelClass: SspsModel,
        join: {
          from: 'buyers.ssp_id',
          to: 'ssps.ssp_id'
        }
      },
      dsp: {
        relation: this.BelongsToOneRelation,
        modelClass: DspsModel,
        join: {
          from: 'buyers.dsp_id',
          to: 'dsps.dsp_id'
        }
      },
    }
  }
}

class DspsModel extends Model { ... }
class SspsModel extends Model { ... }

Queries
The problematic case
const Masterdeals = await MasterdealsModel.query()
  .eager({
    buyers: {
      ssp: true,
      dsp: true
    }
  });

// Output: First relation level OK, second relation level KO
[{
  masterdeal_id: 1,
  masterdeal_name: 'masterdal name',
  buyers: [{
    buyer_id: 1,
    buyer_name: 'buyer name',
    dsp_id: 10,
    ssp_id: 11,
    // dsp: {...}, missing
    // ssp: {...}, missing
  }, {
    buyer_id: 2,
    buyer_name: 'buyer name',
    dsp_id: 20,
    ssp_id: 21
    // dsp: {...}, missing
    // ssp: {...}, missing
  }]
}]

A test just to see that Buyers relation works fine
const Buyers = await BuyersModel.query()
  .eager({
    ssp: true,
    dsp: true
  });

// Output: First relation level OK
[{
  buyer_id: 1,
  buyer_name: 'buyer name',
  dsp_id: 10,
  ssp_id: 11,
  dsp: {
    dsp_id: 10,
    dsp_name: 'dsp name'
  },
  ssp: {
    dsp_id: 11,
    dsp_name: 'ssp name'
  },
}, {
  buyer_id: 2,
  buyer_name: 'buyer name',
  dsp_id: 20,
  ssp_id: 21
  dsp: {
    dsp_id: 20,
    dsp_name: 'dsp name'
  },
  ssp: {
    dsp_id: 21,
    dsp_name: 'ssp name'
  },
}]

The functionnal case with .eagerAlgorithm(Model.JoinEagerAlgorithm), but...
const Masterdeals = await MasterdealsModel.query()
  .eager({
    buyers: {
      ssp: true,
      dsp: true
    }
  })
  .eagerAlgorithm(MasterdealsModel.JoinEagerAlgorithm);

// Output: First relation level OK, second relation level OK
[{
  masterdeal_id: 1,
  masterdeal_name: 'masterdal name',
  buyers: [{
    buyer_id: 1,
    buyer_name: 'buyer name',
    dsp_id: 10,
    ssp_id: 11,
    dsp: {
      dsp_id: 10,
      dsp_name: 'dsp name'
    },
    ssp: {
      dsp_id: 11,
      dsp_name: 'ssp name'
    },
  }, {
    buyer_id: 2,
    buyer_name: 'buyer name',
    dsp_id: 20,
    ssp_id: 21
    dsp: {
      dsp_id: 20,
      dsp_name: 'dsp name'
    },
    ssp: {
      dsp_id: 21,
      dsp_name: 'ssp name'
    },
  }]
}]

But in reality I have a lot of relations in a lot of models, so the maximum SQL Join limit is fastly reached with JoinEagerAlgorithm and this is not a solution for me.

Is this something that I'm doing wrong, is there another solution to fetch nested relations however the type ?

My bad !
All my Models are using modifiers (also in eager relation), inside which there is a pick function that hide the relations...
Sorry


✨ Comments

This PR is a draft Pull Request add all your code to the comments branch or add a pr to the comments branch. This pull request will me merged when all the code is done as per #22

This pull request was self-closed by me, as it was conflicting with the current maintainence of the repository. Another pull request and issue will be made soon 👍🏻


Errors while installing on Raspberry Pi

I am trying to install mraa on a Raspberry Pi using the command line
sudo add-apt-repository ppa:mraa/mraa
However, I am getting an error saying that there is no version for Raspbian, even though the Raspberry Pi seems to be supported (acc. to Documentation).
Traceback (most recent call last):   File "/usr/bin/add-apt-repository", line 95, in <module>     sp = SoftwareProperties(options=options)   File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 109, in __init__     self.reload_sourceslist()   File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 599, in reload_sourceslist     self.distro.get_sources(self.sourceslist)       File "/usr/lib/python3/dist-packages/aptsources/distro.py", line 89, in get_sources     (self.id, self.codename)) aptsources.distro.NoDistroTemplateException: Error: could not find a distribution template for Raspbian/stretch
What am I doing wrong? Can someone help me here?

As you can see in the README, the add-apt-repository is used on Ubuntu, but you're running Raspbian and quick googling shows that's not supposed to work there.
You can add that PPA manually though, see e.g. here for a recipe.

Thanks! I was able to install it after installing dirmngr (with sudo apt-get install dirmngr).
However, my c++ files don't seem to be able to use the library. Is there a way I can check if it is installed properly? And can I just include the mraa library with a #include directive in c++?
Thanks for your answers. I appreciate it!

No worries.
If you could elaborate on how exactly

my c++ files don't seem to be able to use the library

I should be able to advise. Generally there are examples packaged, so you could try those first.

As you can see in the README, the add-apt-repository is used on Ubuntu, but you're running Raspbian and quick googling shows that's not supposed to work there.
You can add that PPA manually though, see e.g. here for a recipe.

I still cannot solve it using the methods provided from the link.
When I try to install
sudo apt-add-repository ppa:pj-assis/ppa
The error receives as below...
Traceback (most recent call last):
File "/usr/bin/apt-add-repository", line 95, in
sp = SoftwareProperties(options=options)
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 109, in init
self.reload_sourceslist()
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 599, in reload_sourceslist
self.distro.get_sources(self.sourceslist)
File "/usr/lib/python3/dist-packages/aptsources/distro.py", line 89, in get_sources
(self.id, self.codename))
aptsources.distro.NoDistroTemplateException: Error: could not find a distribution template for Raspbian/stretch

Is there a solution found?

Not exactly the answer   to the question but if you are in a hurry and have an online translator (for my french pages)  you can check the raspberryPi binary images I am trying to mainain for our project on meteorology and climate sciences, I deliver them with  debian and compilation of mraa and upm among other goodies.
http://mediaserv.climatetmeteo.fr/images/RaspBerry/
And if you have the sufficient skills you can follow my blog and compile it by yourself  #digitalTransformation :-)
https://blog.tremplin.ens-lyon.fr/GerardVidal/RaspberryPi-2_3BuildImage.html


Flow temperature adjustment  accordingly surplus energy from PV

Automating heating and climate control with  flow temperature adjustment , thermostat automation ,  Smart Thermostat with PID controller  ,adjust the temperature accordingly surplus energy from PV ,Automating climate control with Solar PV output   Estimated Power Production - Now (in Watt) ,Estimated Power Production - Next Hour (in Watt) .  The resulting optimized flow temperatures mean increased efficiencies, potential savings,
I need some help to write this scrip or to find a similar script  ( I don't have programming experience )
1.connection  to  heat pump is via local tuya TCPIP   ( number.set4temp ; switch.pdc_on_off ;sensor.21senzor_return;sensor.22senzor_flow)
2.connection to PV inverter  Huawei FusionSolar inverter or via local modbus TCPIP , Power Sensor  (sensor.power_sensor_active_power ;sensor.power_sensor_realtime_power
3.connection with a room thermometer  is via local tuya  (sensor.t1_3_real_temp  second room/  sensor.t2_realtemp )
4.connection  external temperature and humidity is done via API(sensor.wupws_temp,sensor.wupws_humidity)
5.Solar production forecast   (sensor.power_production_now ;sensor.energy_current_hour;sensor.energy_next_hour
Some connection are lost-est over some issue with signal , is necessary a control loop and some extra feedback info .

Not relevant to this integration, closing.


How to force using predefined classes?

For example, if I want to have only class car and license plate and no misspellings are allowed like ccar.
I feel like this should be normally forced because it is so sensitive.
But why it's so easily permitted to put incorrect class name when annotating?

I want to use this software with darkflow.

https://github.com/tzutalin/labelImg/blob/master/data/predefined_classes.txt is what you're looking for? When labelling it should ask you to tick one of the predefined classes.

Yes, it asked me but it also provides the option to type it out. I want to give this to labors to label and we can't guarantee that they will click on the predefined class.

I believe users are more likely to click on the list which pops up, than to type in themselves. There will always be a degree of mislabelling. Providing adequate instruction beforehand should be key.

I submitted a PR not sure if it will be accepted
https://github.com/tzutalin/labelImg/pull/263

I agree w/ awilliamson. If these labels are really mission critical, you can write a simple python script that reads through the xml files and detects any label names that aren't "car" or "license plate", then you can go in and fix those.

It doesn't fix the core issue. It's like you are coping with it instead of providing an option to never make mistake in the first place.
There should be an option to force user to not click ok and that should prevent things from breaking better.

Yeah, that's a fair statement.
The UX of this program is nowhere near perfect, and you're more than welcome to fork the repo and improve it as you see fit. FWIW, your current solution sounds pretty reasonable.


Update Culture Management

~Task - Update Culture Managment
~"User Story": #28
Objective
Cultures should be editable, additionally they need to have space for the new fields we introduced (start/end) note these should be bounded by their associated period.

[x] Add new fields
[x] Constrain by associated period
[x] Relate Periods

Addressing in #43


Don't work RPGitems recipe on 1.12.2

I use last build of spigot.
When i use "rpgitem 1 recipe 1", nothing happens.
In the console I see errors

https://pastebin.com/wP24Wtyq
https://pastebin.com/LwQKW2jP
what the problem and how i can fix that?

please use rpgitems 1.12-v3.6.295


Add more tests for sf::Context

Description
I found some ways to improve the rigor of our sf::Context tests. These should help with #2805.

SFMLBuildMaster: Build this please.


Remove max connection age for GRPC

Motivation
Removes max connection age for all GRPC servers except for the public.
Description
To prevent regular disconnects between the node and the post-service we set the MaxConnectionAge to infinity.
Test Plan
n/a
TODO

[x] Explain motivation or link existing issue(s)
[x] Test changes and document test plan
[x] Update documentation as needed
[x] Update changelog as needed

but this changes it for public/private service as well

bors merge


Mic icon is not visible

I want to show Menu and Mic icon both. Able to see Menu icon but mic icon is not visible. How to achieve this ?

I will check it for next version.


TypeError: Terminal is not a constructor

Describe the bug
The error Uncaught (in promise) TypeError: Terminal is not a constructor occurs when the on mount event is triggered. The error happens while trying to instantiate the Terminal class from the library.
To Reproduce
Steps to reproduce the behavior:

Import the Xterm and XtermAddon components from @battlefieldduck/xterm-svelte.
Attempt to mount the Xterm component on a page.
Look at the error in the browser console.

Expected behavior
The Terminal instance should be created successfully, and the terminal should initialize without any errors.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Windows 11
Browser: Chrome

Additional context
/

What is your svelte version

What is your svelte version

tried with 5.0.0 and 5.1.13

Is it possible to show your code that the error occurs? thanks.

sure:
component:

	<Xterm {options} {onLoad} {onData} {onKey} />

it's just inside a melt tab

how do you load this child component?

Dashboard

This is the dashboard page

				<Terminal2 />

What @battlefieldduck/xterm-svelte version are you using?

What @battlefieldduck/xterm-svelte version are you using?

newest 2.0.0

Try uninstall npm uninstall @battlefieldduck/xterm-svelte and install npm install @battlefieldduck/xterm-svelte again.

Try uninstall npm uninstall @battlefieldduck/xterm-svelte and install npm install @battlefieldduck/xterm-svelte again.

done, doesn't help

What if the <Terminal2 /> component is outside the
?

tried with a blank page, doesnt work

Do you have a GitHub project repository that I can take a look at?

Do you have a GitHub project repository that I can take a look at?

Unfortunately it's not public

I just created a new github repo xterm-svelte-example with sveltekit (svelte5) and xterm-svelte installed.
https://github.com/BattlefieldDuck/xterm-svelte-example
You can test it with npm i and npm run dev. You should be able to see the terminal on while npm run dev.
You may try comparing the settings of the projects. I hope it helps.

Hey, thank you the project you provided works. I will now compare it with my code


License?

Hi Petey,
What is the license for this?

Thanks for the question. I've added the MIT license to the project.


Update configuration-file.md

Fixed some minor typos in hopes of making docs more readable 😊

THanks @tgrecojs


Add match_args to support match case destructuring in Python 3.10

Pull Request Check List
This is just a friendly reminder about the most common mistakes.  Please make sure that you tick all boxes.  But please read our contribution guide at least once, it will save you unnecessary review cycles!
If an item doesn't apply to your pull request, check it anyway to make it apparent that there's nothing left to do.  If your pull request is a documentation fix or a trivial typo, feel free to delete the whole thing.

[x] Added tests for changed code.
[ ] New features have been added to our Hypothesis testing strategy.
[x] Changes or additions to public APIs are reflected in our type stubs (files ending in .pyi).

[x] ...and used in the stub test file tests/typing_example.py.

[ ] Updated documentation for changed code.

[ ] New functions/classes have to be added to docs/api.rst by hand.
[x] Changes to the signature of @attr.s() have to be added by hand too.
[ ] Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.  Find the appropriate next version in our __init__.py file.

[ ] Documentation in .rst files is written using semantic newlines.
[ ] Changes (and possible deprecations) have news fragments in changelog.d.

If you have any questions to any of the points above, just submit and ask!  This checklist is here to help you, not to deter you from contributing!

Sorry for the lint errors. Hopefully it's all fine now.

Changes done :

Updated docs. Removed "Test" from docstrings and updated a few. Feel free to makes changes.
Added test_pattern_matching.py on Python 3.10.
Disabled black since black doesn't support formatting pattern matching : https://github.com/psf/black/issues/2242
Disabled flake8 since pyflakes has an issue with keyword argument matching : https://github.com/PyCQA/pyflakes/issues/634

needs to be added to _next_gen.py

I am not sure about the changes to be made here since match_args is not a signature change in define and field

define, mutable, and frozen all wrap attr.s, so the argument needs to be added to them (and their type stubs) too?

Thanks, added in https://github.com/python-attrs/attrs/pull/815/commits/0cd51b485e560d201b7703c37eeb1b1d854cd8b6

It seems pre-commit  uses Python 3.8 and interrogate fails to parse the Python 3.10 syntax. Excluding the file in pyproject.toml for interrogate doesn't seem to ignore it. debug_statement action of pre-commit also uses the syntax and fails.
black seems not to exclude the files as per pre-commit pinned version. Upgrading black gives some more formatting changes on unrelated files as it seems black has changed the formatting.

I think you'll have to block the match tests to black and interrogate in .pre-commit-config.yaml.

As noted in https://stackoverflow.com/questions/61032281/exclude-some-files-on-running-black-using-pre-commit, it seems that pre-commit gets all python files and just passes it to the command and using --exclude for black doesn't help. I have bumped to Python 3.10 for interrogate and debug_statement. For black I have added exclude from pre-commit end. tox -e lint seems to pass.
tox -e lint
lint installed: appdirs==1.4.4,cfgv==3.2.0,distlib==0.3.1,filelock==3.0.12,identify==2.2.4,nodeenv==1.6.0,pre-commit==2.12.1,PyYAML==5.4.1,six==1.16.0,toml==0.10.2,virtualenv==20.4.6
lint run-test-pre: PYTHONHASHSEED='2125636111'
lint run-test: commands[0] | pre-commit run --all-files
black....................................................................Passed
isort....................................................................Passed
flake8...................................................................Passed
interrogate..............................................................Passed
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Debug Statements (Python)................................................Passed
Check Toml...............................................................Passed
_______________________________________________________________________________ summary ________________________________________________________________________________
  lint: commands succeeded
  congratulations :)

codecov gives me 403, I am not sure about the code coverage error.

codecov is probably failing, because python 3.8 fails and doesn't upload.
now we have the problem, that the linter needs python 3.10 and that's not present in the 3.8 container
I guess there's two options:

block match from interrogate
move lint to py310

I guess you could try option 2 (change the [gh-actions] section in tox.ini) but if that fails for other reasons (or you don't want to deal with it), just go with 1.

Thanks, I guess option 2 makes more sense since eventually the python version has to be upgraded to support parsing. I have pushed commit to bump basepython to 3.10 in lint step.

tox and gh-actions are slightly new to me so I got confused. I pushed the lint job from py38 to py310 and also made basepython 3.10 in lint step. I guess this should run link in python 3.10 container with python 3.10 as base. Please correct me if I am wrong.

Looks good, but I think you still add the tuple on Python <3.10? (On my phone for the rest of the day)

Yes, I am adding __match_args__ tuple always. I don't see any backwards compatibility issues with the same. I will be curious how dataclasses backport also deals with the change too. I can add a version check to add tuple only for Python 3.10+ if you want. In that case I guess match_args=True will always be there in signature and it's just that tuple generation becomes python 3.10+?

Yeah I don’t have any comparability problems in mind – it just seems wasteful to set an attribute on all classes even tho they can’t use it.
And yes, just add something to the tune of  “this argument is ignored on Python versions before 3.10” to the doc string and call it a day.

Thanks @hynek for the review and merge :)


stable/elasticsearch: add new property to define custom affinity rules

What this PR does / why we need it:
Adds a property to define custom affinity rules to the pods and nodes
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #13139

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/chart])

I've updated the PR with some changes and fixed the failed builds.

/ok-to-test

I rebased against master to get the latest changes
@desaintmartin
@rendhalver


[PINOT-6614] Log more messages in the error path for table size estimations and handle estimate of -1 better

The code currently doesn't seem to handle the case where estimated size is -1. The PR updates it to throw an exception in this case. TODO: unit test needs to be added.
Add more log messages when we take the error paths to make it easier to understand how the estimates are arrived at.

Codecov Report

Merging #2935 into master will decrease coverage by 0.04%.
The diff coverage is 64.28%.

@@            Coverage Diff             @@
##           master    #2935      +/-   ##
==========================================
- Coverage   69.26%   69.21%   -0.05%
==========================================
  Files         914      914
  Lines       43416    43429      +13
  Branches     5922     5923       +1
==========================================
- Hits        30070    30061       -9
- Misses      11332    11355      +23
+ Partials     2014     2013       -1

Impacted Files
Coverage Δ

...ontroller/api/resources/ServerTableSizeReader.java
78.04% <100%> (ø)
:arrow_up:

...not/controller/validation/StorageQuotaChecker.java
77.77% <42.85%> (-5.21%)
:arrow_down:

...inkedin/pinot/controller/util/TableSizeReader.java
91.52% <83.33%> (-0.44%)
:arrow_down:

...in/pinot/core/query/pruner/ValidSegmentPruner.java
57.14% <0%> (-28.58%)
:arrow_down:

...a/manager/realtime/RealtimeSegmentDataManager.java
75% <0%> (-25%)
:arrow_down:

...e/operator/dociditerators/MVScanDocIdIterator.java
51.51% <0%> (-15.16%)
:arrow_down:

...lix/EmptyBrokerOnlineOfflineStateModelFactory.java
86.66% <0%> (-13.34%)
:arrow_down:

...impl/dictionary/DoubleOnHeapMutableDictionary.java
68.88% <0%> (-13.34%)
:arrow_down:

...mpl/dictionary/DoubleOffHeapMutableDictionary.java
67.27% <0%> (-9.1%)
:arrow_down:

.../impl/dictionary/LongOffHeapMutableDictionary.java
87.27% <0%> (-5.46%)
:arrow_down:

... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 733c445...d30cd29. Read the comment docs.

Unrelated to this PR but have a question - Are we calling each server for every segment upload. That seems to be overkill for use cases where we have approx million segments.

Are we calling each server for every segment upload?
Yes. In the (longish) email summarizing the issue, I have mentioned the possibility of using a cache that's periodically refreshed.

We could perhaps even do with a controller local in-memory cache; if the table size is close to the limit and we are uploading a large chunk, the controller can kick off an ondemand refresh and/or fire off an alert requiring the quota to be reviewed. Anytime we are around 80% of quota usage, its probably good to review and adjust table quota.