Questions and help on node to node modeling

❓ Questions & Help on node to node modeling
Hi, first thanks for this awesome package, it's a great piece of work.
I've been working on the Molecular Properties challenge on Kaggle lately (https://www.kaggle.com/c/champs-scalar-coupling) and I am struggling a bit. I would really appreciate some help (both in terms of existing codes, papers or your own vision on the problem).
Problem setting
In this competition, we have molecule similar to the molecules in QM9 datasets, the goal is to predict the coupling constant between some atom pairs given atom types and positions. There are 8 differents pair types ('1JHC', '2JHH', '1JHN', '2JHN', '2JHC', '3JHH', '3JHC', '3JHN') and it seems that making different models for each type improve performances.
Here are some specificity I see here and didn't managed to overcome yet :

the edges are not given, so there are different ways of creating the edges. I'll be more specific when explaining my approaches so far.
for a same graph you have multiple targets that don't follow a clear pattern (not one output per node, not one output per edge etc...)

Different approaches
"Real" bond graphs
My first approach consists in creating both node_features, edges and edges_attr by using openbabel and following existing bonds.
Since there is no "real" bond for 3JHC coupling type for example, this method does not allow  to create predictions from edges so  my approach is that one prediction can be done from one graph and two nodes indices where I just stack nodes_1 and nodes_2 representation to then pass then to a MLP (with other external features).
 class RealBondNet(torch.nn.Module):
    def __init__(self, num_node_features, num_edge_features, num_mol_features,
                 dropout=0.0, output_dim=1, num_conv = 5, conv_size=64, num_steps=3):
        super(RealBondNet, self).__init__()
        self.num_conv = num_conv
        self.num_steps = num_steps
        self.num_mol_features = num_mol_features
        self.conv_size = conv_size
        self.output_dim = output_dim
        self.dropout = dropout
        self.graph_conv1 = GraphConv(num_node_features, self.conv_size)
        self.graph_conv2 = GraphConv(self.conv_size, self.conv_size)
        self.pool_S2S = Set2Set(self.conv_size, processing_steps=self.num_steps)

        self.edge_lin0 = torch.nn.Linear(num_node_features, conv_size)

        nn = torch.nn.Sequential(Linear(num_edge_features, 128), ReLU(), Linear(128, conv_size * conv_size))
        self.NNconv = NNConv(conv_size, conv_size, nn, aggr='mean', root_weight=False)
        self.gru = torch.nn.GRU(conv_size, conv_size)
        self.edge_set2set = Set2Set(conv_size, processing_steps=self.num_steps)

        self.fc1 = Linear(self.conv_size*8, self.conv_size*4)
        self.bn1 = BatchNorm1d(self.conv_size*4)
        self.fc2 = Linear(self.conv_size*4, int(self.conv_size))
        self.bn2 = BatchNorm1d(int(self.conv_size))

        # mol features
        self.fcmol1 = Linear(self.num_mol_features, self.num_mol_features)
        self.bnmol1 = BatchNorm1d(self.num_mol_features)
        self.fcmol2 = Linear(self.num_mol_features, self.num_mol_features)
        self.bnmol2 = BatchNorm1d(self.num_mol_features)
        self.fcmol3 = Linear(int(self.num_mol_features), int(self.num_mol_features /2))
        self.bnmol3 = BatchNorm1d(int(self.num_mol_features /2))

        self.before_final_fc = Linear(int(self.num_mol_features /2)+self.conv_size, self.conv_size)
        self.bnbf = BatchNorm1d(self.conv_size)
        self.final_fc = Linear(self.conv_size, 1)

    def forward(self, data):
        x, edge_index, edge_attr, batch, atom_index_0, atom_index_1 = data.x, data.edge_index, data.edge_attr, data.batch, data.atom_index_0, data.atom_index_1

        # edges transformations
        out = F.relu(self.edge_lin0(data.x))
        h = out.unsqueeze(0)

        for i in range(3):
            m = F.relu(self.NNconv(out, edge_index, edge_attr))
            out, h = self.gru(m.unsqueeze(0), h)
            out = out.squeeze(0)

        edge_pool = self.edge_set2set(out, batch)

        # mol information
        mol_out = self.fcmol1(data.mol_attr.view(atom_index_0.shape[0], -1))
        mol_out =  F.relu(self.bnmol1(mol_out))
        mol_out = self.fcmol2(mol_out)
        mol_out =  F.relu(self.bnmol2(mol_out))
        mol_out = self.fcmol3(mol_out)
        mol_out =  F.relu(self.bnmol3(mol_out))
        mol_out = F.dropout(mol_out, p=self.dropout, training=self.training)
        # nodes transformations
        x = F.relu(self.graph_conv1(x, edge_index))
        for i in range(self.num_conv):
            x = F.relu(self.graph_conv2(x, edge_index))

        pool = self.pool_S2S(x, batch)

        nodes_0 = x[atom_index_0,:]
        nodes_1 = x[atom_index_1,:]

        nodes_edge_pool_0 = out[atom_index_0,:]
        nodes_edge_pool_1 = out[atom_index_1,:]

        x = torch.cat([edge_pool, pool, nodes_0, nodes_1, nodes_edge_pool_0, nodes_edge_pool_1], dim=1)
        # BEFORE MLP
        x = self.fc1(x)

        x = F.relu(self.bn1(x))
        x = self.fc2(x)
        x = F.relu(self.bn2(x))
        x = F.dropout(x, p=self.dropout, training=self.training)
        x = torch.cat([x, mol_out], dim=1)

        x = self.before_final_fc(x)
        x = F.relu(self.bnbf(x))
        x = self.final_fc(x)
        #x = self.fc3(x)
        return x

This achieves reasonable but non competitive scores around 0.4 mae while 0.1 is achievable.
I feel like cherry picking nodes representation i and j could break the "flow" since it's external variables which determines what part of the graph representation goes into the MLP.
Fully connected Graphs
An other approach I tried is to systematically creates all edges (n(n-1)/2 edges where n is the number of atoms) and try to make all predictions for all atom pairs at one time using a mask to perform gradient descent only on given connections.
However I had issue with creating my torch geometric dataset because the default collate function does not seem to allow matrix as targets.
I get this error when trying to set data(y=a_matrix) :
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-7-01bfcf89c013> in <module>
----> 1 dataset = MolecularCouplingDataset()

<ipython-input-6-080f091cda03> in __init__(self, root, transform, pre_transform)
      1 class MolecularCouplingDataset(InMemoryDataset):
      2     def __init__(self, root="/home/seb/MolecularCouplingPredictions/output", transform=None, pre_transform=None):
----> 3         super(MolecularCouplingDataset, self).__init__(root, transform, pre_transform)
      4         self.data, self.slices = torch.load(self.processed_paths[0])
      5

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/in_memory_dataset.py in __init__(self, root, transform, pre_transform, pre_filter)
     54                  pre_filter=None):
     55         super(InMemoryDataset, self).__init__(root, transform, pre_transform,
---> 56                                               pre_filter)
     57         self.data, self.slices = None, None
     58

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/dataset.py in __init__(self, root, transform, pre_transform, pre_filter)
     81
     82         self._download()
---> 83         self._process()
     84
     85     @property

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/dataset.py in _process(self)
    125
    126         makedirs(self.processed_dir)
--> 127         self.process()
    128
    129         print('Done!')

<ipython-input-6-080f091cda03> in process(self)
     33             data_list.append(data)
     34
---> 35         data, slices = self.collate(data_list)
     36         torch.save((data, slices), self.processed_paths[0])
     37

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/in_memory_dataset.py in collate(self, data_list)
    148             if torch.is_tensor(data_list[0][key]):
    149                 data[key] = torch.cat(
--> 150                     data[key], dim=data.__cat_dim__(key, data_list[0][key]))
    151             else:
    152                 data[key] = torch.tensor(data[key])

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 5 and 4 in dimension 1 at /opt/conda/conda-bld/pytorch_1556653099582/work/aten/src/TH/generic/THTensor.cpp:711

I tried to overcome this by flattening everything but I struggled with reshaping everything back during  the forward step.
Anyway I decided to create create fake nodes to pad every graph to the maximum size and  I tried to apply a MPNN, then from a matrix (n_graphs, n_conv, n_atom) I tried to create a invariant matrix of shape  (n_graphs, n_conv, n_atom, n_atom) to then apply 1x1 conv2D to get predictions for all edges. Here is the network, results where pretty bad but I am not sure I managed to handle the batches properly.
class SimpleNet(torch.nn.Module):
    def __init__(self, num_node_features, num_edge_features,
                 dropout=0.0, output_dim=1, num_conv = 6, conv_size=64, num_steps=8):
        super(SimpleNet, self).__init__()
        self.num_conv = num_conv
        self.num_steps = num_steps
        self.conv_size = conv_size
        self.output_dim = output_dim
        self.dropout = dropout

        self.edge_lin0 = torch.nn.Linear(num_node_features, conv_size)

        nn = torch.nn.Sequential(Linear(num_edge_features, 128), ReLU(), Linear(128, conv_size * conv_size))
        self.NNconv = NNConv(conv_size, conv_size, nn, aggr='mean', root_weight=False)
        self.gru = torch.nn.GRU(conv_size, conv_size)

        self.conv2d =torch.nn.Conv2d(self.conv_size, 1, kernel_size=1)

    def forward(self, data):
        x, edge_index, edge_attr, batch = data.x, data.edge_index, data.edge_attr, data.batch

        out = F.relu(self.edge_lin0(data.x))
        h = out.unsqueeze(0)

        for i in range(self.num_steps):
            m = F.relu(self.NNconv(out, edge_index, edge_attr))
            out, h = self.gru(m.unsqueeze(0), h)
            out = out.squeeze(0)

        reshaped_out = out.view(data.num_graphs, 29, self.conv_size).permute(0, 2, 1)
        order_invariant_m = reshaped_out.reshape(data.num_graphs, self.conv_size, 29, 1) @reshaped_out.reshape(data.num_graphs, self.conv_size, 1, 29)

        res_conv2D = self.conv2d(order_invariant_m)

        return res_conv2D

So a few questions:
1- do you know about such problems that have been tackled before? Do you have any links?
2- how would you approach the problem yourself?
3- which method looks more promising to you?
4- any idea how to overcome the collate error and simply put matrices as targets?
5- does it make any sense to pad every graph to the maximum graph size? how to do so properly?
6- any idea how to deal with training one or different model to take into account the different coupling types?
I hope this message finds you well (I get that it's a bit hairy) but I would really be glad to have your thoughts on this.
Thanks!

Hi and sorry for answering late, but this is an in-depth question and I hasn‘t habe much time recently. To be honest, I haven‘t looked into the Kaggle competition that closely and I am far from expert on this field, so please take my advice with caution. Overall, I find the best place that describes the problem from a GNN perspective is this thread. IMO, I would tackle this problem similar.
1, 2 and 3: While trying to replicate the results of Gilmer et al, I noticed that it is extremely important to perform graph convolution on the fully-connected graphs with distances and bond types as edge features, so I do believe this is the way to go.
4. This is an extremely interesting problem and you may overcome it by saving sparse target matrices in analogy to edge_index and edge_attr, e.g., y_index and y_value. Batching of those values should work fine. Alternatively, dense matrices should work fine too, but they all need to be of equal size. You can save them as [1, N, N] shape to ensure correct batching.
5. This does make sense when you want to model pair-wise interactions. We provide such functionality with to_dense_batch and to_dense_adj in torch_geometric.utils.
6. Do you want to train a single model for all types? I guess you will need to create targets for each coupling type (in sparse scenario) or with an additional dimension (in dense scenario).

Thanks for your answer, it helped me a lot.
I implemented a fully connected version using n(n-1)/2 edges, with corresponding mask and targets for node to node predictions. Putting only half of the edges since my graph is undirected saves space and time. However I am trying to apply a stack of MetaLayers that seem to be working but not as good as I'd expect. I am wondering if this could be due to using only half edges.
In the original paper, the graphs are directed and so information is aggregated following directed eges. My node model looks like this
class NodeModel(torch.nn.Module):
    def __init__(self, num_node_features, num_u_features, node_conv_dim, edge_model_dim):
        super(NodeModel, self).__init__()
        node_mlp1_dim = node_conv_dim
        in_mlp_1 = num_node_features + edge_model_dim
        in_mlp_2 = node_mlp1_dim + num_node_features + num_u_features
        self.node_mlp_1 = Seq(Lin(in_mlp_1, node_mlp1_dim), ReLU(), Lin(node_mlp1_dim, node_mlp1_dim), BatchNorm1d(node_mlp1_dim))
        self.node_mlp_2 = Seq(Lin(in_mlp_2, node_conv_dim), ReLU(), Lin(node_conv_dim, node_conv_dim), BatchNorm1d(node_conv_dim) )

    def forward(self, x, edge_index, edge_attr, u, batch):
        # x: [N, F_x], where N is the number of nodes.
        # edge_index: [2, E] with max entry N - 1.
        # edge_attr: [E, F_e]
        # u: [B, F_u]
        # batch: [N] with max entry B - 1.
        row, col = edge_index
        out = torch.cat([x[col], edge_attr], dim=1)
        out = self.node_mlp_1(out)
        out = scatter_mean(out, row, dim=0, dim_size=x.size(0))
        out = torch.cat([x, out, u[batch]], dim=1)
        return self.node_mlp_2(out)

But this part out = torch.cat([x[col], edge_attr], dim=1)         out = self.node_mlp_1(out)         out = scatter_mean(out, row, dim=0, dim_size=x.size(0)) makes me wonder, am I missing half of the messages compared to a fully connected n(n-1) version? It seems that my message fly only from smaller nodes index to larger nodes index (because i have edge_index looking like this [[1, 2, 2, 3, 3, 3], [0, 0, 1, 0, 1, 2]]). Isn't that a big problem in my model?
If so I see two ways to overcome this :

double the number of edge_index to have all directions

try to change my NodeModel to pass messages from both ways.
Which one would you recommend? How would you implement 2)?

Hope my questions make sense, thanks a lot!

Yes, you only propagate along half of your edges. I recommend doubling the number of edges. This way you may have more GPU utilization. But honestly, you do not need scatter operations for fully connected graphs. Currently, you implement a sparse version for a very dense graph, which is not that useful. A dense version of your approach should be way faster.

Ok thanks, I doubled the edges and it worked better! Twice longer thoug...
Can I easily switch to a dense implementation with TorchGeometric?

You can, via the ToDense transform and the DenseDataLoader, but we only provide s limited number of dense GNN operators, although it should be straightforward to extend them or build your own.


regex enhancement

Improve regex logic to avoid additional validation after matching

The following regex should do the trick: ^(0[1-9]\d{3}|[1-9]\d{0,3})-W(0[1-9]|[1-4]\d|5[0-3])$


4.20Load failure after packaging

The serial port cannot be read after the project is opened

@SnowflakeSmasher26 What did you try already?
Are you trying to use it on a Blueprint only project? If so, you need the binary files. Just the source code won't do it. You can get the binaries in the releases. The latest release has binaries for 4.24.


MSsql sink writes times as local time #392

Hi,
Here is an implementation for the requested enhancement.
Let me know what you think.
Thanks,
Michael

Looks good to me, just some minor issues mentioned in the comments.
We will also need to version this as 2.0, because the API of the MSSqlServer() method won't be binary-compatible with the old one. (We used to consider source-compatibility the bar for configuration changes, but since it's now cheap to increment sink versions separately, we might as well be more stringent).
Cheers!

Thanks, this looks great, let's kick it off!


[Diagnostics] Port more key path related failures to new framework

This PR completely eliminates the need for FailureDiagnosis::visitKeyPathExpr
by handling contextual mismatches as well as e.g. missing/invalid members via
new diagnostic framework.

@swift-ci please smoke test

@swift-ci please smoke test


[AnimationPlayer] Playing an animation backwards with play("", -2, -1) triggers method tracks out of turn

Tested versions
v4.2.1.stable.official [b09f793f5]
System information
Windows 11
Issue description
Triggering an animation backwards plays all method tracks out of turn.
Steps to reproduce

Play the scene in the MRP
You will see that the trigger method gets called out of turn

Minimal reproduction project (MRP)
Animation MRP.zip

So, this is not at all explained in the documentation, but switching the sign of 'custom_blend' option to positive, while playing backwards, seems to fix the issue.
Not sure if this is by design. If so, it should be documented. If not, just FYI.

Fixed in v4.3.dev.custom_build [7199765d9]


Tom - Added template literal examples

Added template literal examples.

Well darn. Looks like the linting failed. 👎

I like the idea of showing multi line. It also shows the ability to avoid escaping quotes. Can we add a simple one line example to not be so complicated in the example?
Thoughts?

@matt-jarrett Sure thing. I will work on that and get back to you.

Hey @tomLendy - I liked your idea of adding multiline so I added it in #4 . I didn't add a escape example because it's not a feature unique to Template Literals. You can get out of headaches with other options like single quotes. I'll close this, but feel free to continue the discussion.
const phrase = 'Joe said, "Hello Lisa."'
Let me know if you have any feedback?
As for Travis CI, it's rad. I can show you around it if you want 😃 It's easy and pretty straight forward.

Thanks, I was pretty busy this weekend so I didn't get much done on Git
until a bit ago.
I finished the angular tutorial. Interesting thing about that is when you
run the "ng serve -o" command through git it messes with your git console.
So going back to use git gui is difficult.
When I added the escape examples my intent was to show how difficult they
can be. Then seeing the same examples in template literals would really
illustrate how awesome they are. One of those examples wasn't able to use
just alternating quotes because it included contractions. For example
"can't" was a part of the quoted text. That really screwed things up and
needed both alternating quotes and escapes.
Catch you tomorrow. Well chat about Tyler CI then. Thanks.
On Feb 24, 2018 12:49 PM, "Matt Jarrett"<EMAIL_ADDRESS>wrote:

Closed #3 https://github.com/matt-jarrett/es6-in-six-hours/pull/3.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/matt-jarrett/es6-in-six-hours/pull/3#event-1490535759,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Afi8LMPmwtjoI2TaeWWqhIs-3CaJZvk5ks5tYFm2gaJpZM4SQUo_
.


feat: Add Casdoor iframe to page

Signed-off-by: ErikQQY<EMAIL_ADDRESS>

@hsluoyz Plz review

@ErikQQY merged for now, plz send new PRs for update.


[NFC] Add a pip-audit workflow

Well, this ended up being a very large changeset. Let me know if you'd like me to revert 9c384d4, since that's where 99% of the changeset is (and it's all autoformatting noise).


Better user feedback when scripts fail

There are several places where external java and R scripts are called. Should these scripts fail, the user will be bombarded with information. The failing of the scripts should be checked for and handled in a more user-friendly way.

Implemented per bbd0c77d8ff23c8bcc1177391f3fe06c85ce3da9.


Replace base messages with rlang and cli

Replace base message with rlang
Move vignette to package space
Adds missing metadata in vignette

Preparing for cran release.

[ ] Place Closes #<insert_issue_number> into the beginning of your Pull z
Request Title (Use Edit button in top-right if you need to update)
[ ] Code is formatted according to the
tidyverse style guide. Run
styler::style_file() to style R and Rmd files
[ ] Updated relevant unit tests or have written new unit tests, which should
consider realistic data scenarios and edge cases, e.g. empty datasets, errors,
boundary cases etc. - See
Unit Test Guide
[ ] If you removed/replaced any function and/or function parameters, did you
fully follow the
deprecation guidance?
[ ] Update to all relevant roxygen headers and examples, including keywords
and families. Refer to the
categorization of functions to tag appropriate keyword/family.
[ ] Run devtools::document() so all .Rd files in the man folder and the
NAMESPACE file in the project root are updated appropriately
[ ] Address any updates needed for vignettes and/or templates
[ ] Update NEWS.md if the changes pertain to a user-facing function (i.e. it
has an @export tag) or documentation aimed at users (rather than developers)
[ ] Build oak site pkgdown::build_site() and check that all affected
examples are displayed correctly and that all new functions occur on the "Reference" page.
[ ] Address or fix all lintr warnings and errors - lintr::lint_package()
[ ] Run R CMD check locally and address all errors and warnings - devtools::check()
[ ] Link the issue in the Development Section on the right hand side.
[ ] Address all merge conflicts and resolve appropriately
[ ] Pat yourself on the back for a job well done! Much love to your accomplishment!

Hi @ramiromagno
The snapshot issue is fixed #73 or #75 we do not yet decide witch we are merge.
The different was coming from admiraldev package.
For export of mutate.cnd_df thanks was pointing that out, I was thinking I exported that by the mistake.
Let's use cli everywhere for the errors.


Removing redundant null logger after hosting fix

I have resolved https://github.com/aspnet/Hosting/issues/532. I think this is the only place where you added a null logger in https://github.com/aspnet/Mvc/pull/3796 right? @pranavkm

Yup, that's the only location
:shipit:


KAT

https://kat.al/ is one of the new domains,
It would suck if you wouldnt include kat.

we can't trust any of those copycats. the original KAT team is gone.
We're adding more search sources in coming releases.


How to get Max turn count value

I'm trying to give an invalid response to the user based on the Max turn count value but I'm unable to get the value. How can I achieve this?
if bot re-prompt user for first time different invalid response
if it is second time another invalid response
for each turn, I want to catch that increasing max turn count

you can use "this.turncount" @KoradaRajesh

Thank you! @luhan2017


[top] lint fixes

Minor lint fixes.
The pinmux fix might be in another PR already, so just let me know and I can take it out.
@eunchan I'm not really sure if the combo loop fix is right.  I think it makes sense, but you'll probably know best.

ah yes good point. I'll update the markdown as well.
On Thu, Apr 8, 2021 at 1:44 AM Philipp Wagner @.***>
wrote:

@.**** commented on this pull request.
On the pinmux change: The attributes need also changing in
hw/ip/pinmux/doc/padctrl.md. And I'm assuming the attributes should match
prim_pad_wrapper_pkg.sv as well? Maybe you can add a comment to keep the
three in sync, and use the exact same descriptions. (On that note, I'd
prefer 0xf over 1111 e.g. for drive strength.)
Since both comments aren't really trivial "lint" fixes (as in: just add a
width cast somewhere or a whitespace), maybe you can rename them to
something like "Align pad attributes between different files" or something
along those lines)
In hw/ip/pinmux/data/pinmux.hjson.tpl
https://github.com/lowRISC/opentitan/pull/5992#discussion_r609454585:

                          Bit   2: Pull enable.

                          Bit   3: Pull select (0: pull down, 1: pull up).

                          Bit   4: Keeper enable.

                          Bit   5: Schmitt trigger enable.

                          Bit   6: Slew rate (0: slow, 1: fast).

                          Bit 7/8: Drive strength (00: weakest, 11: strongest).

                          Bit   9: Reserved.

                 desc: '''Bit   0:  input/output inversion,

                          Bit   1:  Virtual open drain enable.

                          Bit   2:  Pull enable.

                          Bit   3:  Pull select (0: pull down, 1: pull up).

                          Bit   4:  Keeper enable.

                          Bit   5:  Schmitt trigger enable.

                          Bit   6:  Open drain enable

                          Bit 7/8:  Slew rate (0x0: slowest, 0x3: fastest).

                          Bit 9/12: Drive strength (0x0: weakest, 0xf: strongest).

nit: 9-12
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/lowRISC/opentitan/pull/5992#pullrequestreview-631124694,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAH2RSRXND65RNRZATKFPCDTHVUIDANCNFSM42R7RHOQ
.

let me merge this first (since i'll be gone for a few hours), and i'll follow up with the doc adjustment @imphil pointed out.
That way at least others running lint will have these gone.

Thanks, @tjaychen !


How to resume a download?

OS: Linux Mint 20
Sometimes, a large download stops midway due to internet connectivity issues. Is there any way to resume a download from a given time (e.g. download 63 minutes onward) and then join the new content with the file (partially) downloaded previously?
This is probably not a duplicate of https://github.com/snobu/destreamer/issues/267, I want to resume downloading a single video.

There isn't a way to resume a download since we le ffmpeg handle all the download and it does not support resuming a download.
It may be possible to implement in the new experimental branch but I don't have the time to do so.


feat: TIME variables support

Thank you for contributing to Coraza WAF, your effort is greatly appreciated
Before submitting check if what you want to add to coraza list meets quality standards before sending pull request. Thanks!

Make sure that you've checked the boxes below before you submit PR:

[x] My code includes positive and negative tests.
[x] I have an appropriate description with correct grammar.
[x] I have read Contribution guidelines, maintainers note and Quality standard.
[x] My code is properly linted and passes pre-commit tests.

Thanks for your contribution :heart:
This pull request addresses issue #1220 by introducing support for the following time-related variables in Coraza WAF:

TIME
TIME_DAY
TIME_EPOCH
TIME_HOUR
TIME_MIN
TIME_MON
TIME_SEC
TIME_WDAY
TIME_YEAR

These variables enhance Coraza's functionality by enabling detailed logging and time-based rule creation, improving compatibility with ModSecurity. Additionally, this PR includes tests to ensure the correct implementation and behavior of these new variables.

First of all, thanks a lot @geoolekom for taking the time to put up this PR. That said, I am not fully convinced on the need of this. I believe the case you point out is more like a logger concern as we are only interpolating the variable to display the timestamp. Interpolating a variable isn't cheap, nor it is allocating all the variables we allocated and converted by default from int to string.
Notice native environments are OK with clock calls but other environments like WASM don't get it that simple.
I would say if we could come up with a solid use case where the time is part of the rule I am in to land the variables (only those showing value for the rule) but if the concern is timestamp in the logs we should definitively address that in the logger. What do you think?

Thanks so much for your detailed feedback @jcchavezs! I really appreciate you taking the time to review this PR and share your thoughts. Let me clarify the rationale behind these changes and explore possible solutions.

Modsec compatibility. The main idea behind this PR is to keep Coraza as close as possible to ModSecurity. If we skip these variables, it’ll create a bigger compatibility gap, which I’d really like to avoid.
Ruleset compatibility. These time-related variables are really important because some of popular rulesets, like Imunify360, rely on them heavily. Dropping support for these would make Coraza incompatible with a bunch of existing rulesets, which would make it a lot harder for users to migrate their setups.
Performance. It would be very helpful if you could share an example of a benchmarking report or methodology that seems convincing to you, so we’re addressing your concerns in the best way possible.

If the benchmarks show a real hit to performance, I’ve got a few ideas to make things more flexible:

Add a flag: We could introduce a toggle to enable or disable the timestamp variables, so users can choose based on their needs.
Allow setvar directives to define variables outside the TX collection, so users can define what they want.
Custom variable registration: We could create a plugins.RegisterVariable function (like plugins.RegisterOperator) to let users register extra variables.

Let me know what you think of these ideas.

Thanks @geoolekom! I think most of the points raised here make sense. Let me do a couple of experiments as what I have in mind is that this a good opportunity for trying lazy collections (i.e. collections that are filled on demand) given that a transaction will be read in a sync fashion.

Real-life examples of rules where TIME variables are truly important.
Track time for filescan
We use a set of rules that stores the DURATION value before a scan is started and after completion. This is required to control the time taken for the scan process.
SecRule &TX:lua_present "@eq 0" "id:33363,nolog,auditLog,block,chain,severity:2,phase:2,t:none,msg:'IM360 WAF: Attempt to upload malware||Scan duration:%{TX.py_scan_duration}||Sizes:%{FILES_SIZES}||Combined size:%{FILES_COMBINED_SIZE}||User:%{SCRIPT_USERNAME}||Filename:%{FILES}||Scanned:%{FILES_TMPNAMES}||SC:%{SCRIPT_FILENAME}||WPU:%{TX.wp_user}||RSV:6.52||T:APACHE||',tag:'service_i360'"
SecRule FILES_TMPNAMES "@inspectFile /opt/alt/python35/share/imunify360/scripts/modsec_scan.py" "t:none,setvar:TX.py_scan_duration=%{DURATION},setvar:TX.py_scan_duration=-%{TX.py_scan_start}"

Dynamic IP Tracking and RBL Validation
This example tracks client IPs and combines them with timestamps for uniqueness, then identifies risky actions by matching request filenames against a known list. It verifies risky actions through an external RBL lookup to confirm malicious or undesirable behavior and then blocks requests based on positive RBL matches..
SecAction "id:33368,phase:2,pass,nolog,severity:5,setvar:tx.rbl_ip=%{TIME_HOUR}-%{TIME_MIN}.%{tx.remote_addr},initcol:ip=%{tx.remote_addr}"

SecRule REQUEST_FILENAME "@pmFromFile risky-actions.list" "id:33315,phase:2,block,severity:2,nolog,auditlog,t:none,msg:'IM360 WAF: RBL block risky actions||RSV:6.52||T:APACHE||MV:%{MATCHED_VAR}',chain,setvar:tx.rbl_perf=1,tag:'service_i360'"
SecRule TX:RBL_IP "@rbl risky-actions.v2.rbl.imunify.com." "t:none,chain"
SecRule TX:RBL_IP "!@rbl nxdomain.v2.rbl.imunify.com." "t:none"

@jcchavezs Could you please elaborate, what could we do to get this feature merged?

I worked a PoC on lazy collections in https://github.com/geoolekom/coraza/pull/1


chore: add the missing babel-cli

Missing babel-cli dependency is causing yarn install to fail:
$ yarn install
yarn install v1.3.2
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning " ><EMAIL_ADDRESS>has unmet peer dependency "react-native-vector-icons@*".
warning " ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| ^7.0.0-alpha || ^7.0.0-beta || ^7.0.0".
[4/4] 📃  Building fresh packages...
$ yarn build
yarn run v1.3.2
$ babel --no-babelrc --plugins=syntax-jsx,syntax-class-properties,syntax-object-rest-spread,transform-flow-strip-types src --out-dir dist --ignore '**/__tests__/**'
/bin/sh: babel: command not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Motivation
The pull request adds the missing babel-cli dependency. Also some formatting done automatically by prettier.
Test plan
Run yarn install in the project directory.
Version:
"react-navigation-tabs": "^0.1.0-alpha.3"

@satya164 sure, done


DSA 4.1.16 Stacks: Iterative Tower of Hanoi

Description 📜
Created documentation for the iterative solution of the problem-Tower of Hanoi using Stacks, with C++ code, added time and space complexity as well as the pictorial visualization.
Fixes #7143

Type of change 📝

[ ] Au#dio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)
[ ] Vi#deo (Animations, screen-recordings, presentations and regular explanatory films are all possibilties etc)
[x] Documentation (Content Creation in the form of codes or tutorials)
[ ] Other (If you choose other, Please mention changes below)

Domain of Contribution 📊

[ ] Android Dev #(Flutter)
[ ] Android Dev #(Java)
[ ] Android Dev #(Kotlin)
[ ] Backend Dev #(Java)
[ ] Backend Dev #(.NET)
[ ] Backend Dev #(PHP)
[ ] Backend Dev #(Python)
[ ] C/#CPP
[ ] Competitive #Programming
[ ] Cyber #Security
[x] DSA
[ ] Data#base
[ ] Datascience with #Python
[ ] Datascience with #R
[ ] Frontend Dev #HTML/CSS/JS
[ ] Frontend Dev #React/Angular/Vue
[ ] Go#lang
[ ] Interview #Prep
[ ] Java_#Domain
[ ] JavaScript_#Domain
[ ] ME#RN
[ ] Machine #Learning
[ ] Open #Source
[ ] Python_#Domain
[ ] Ru#st
[ ] Statis#tics
[ ] UI/#UX

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

Screenshots / Gif (Optional) 📸

Please review @nimisha-yadav

@harshsri28 @sneha-nitdgp

@nisha855 keep the file as readme itself for Iterative TOH folder


Wip/6.0 REFRESH cascade verification

The RefreshTests were broken because of some logic loophole. Tried to solve the bug.

not ready yet, so close it for time being


When can AccountInfo be null after login resolves?

Library

[ ]<EMAIL_ADDRESS>or<EMAIL_ADDRESS>[x]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>[ ]<EMAIL_ADDRESS>
Description

I'm upgrading from 2.4.1 to 2.9.0 and I'm running into new type errors when accessing AccountInfo on the resolved value of the login APIs, example:
const ssoResult = await msal.ssoSilent(...);
console.log(ssoResult.account.username); // Object is possibly 'null'. ts(2531)

I'd like to know if there is now a potential scenario where the login api's can resolve but not include AccountInfo data? If so, any way to avoid this scenario?
Source

[x] Internal (Microsoft)
[ ] Customer request

@deasems The account property is marked as null because there are scenarios in other libraries where account may not be populated, but it should always be populated when using msal-browser. We will look into improving this to make it less confusing.

@deasems The account property is marked as null because there are scenarios in other libraries where account may not be populated, but it should always be populated when using msal-browser. We will look into improving this to make it less confusing.

Fixing this will clean up a bunch of unnecessary undefined checks in my code too - super helpful with TS!

Fixing this will clean up a bunch of unnecessary undefined checks in my code too - super helpful with TS!

I will try to put out a PR that addresses this.


Propagate Permission as Claims

Description:
To get the permissions of a relevant user as claims in the id_token.
Eg: User's roles as R1 (perm1 and perm3)  and R2(perm2 and perm3)
User's permission perms: ["perm1", "perm2", "perm3"]
Suggested Labels:
Type/Improvement
Affected Product Version:
WSO2IS-5.4.0

Thank you for your contribution!
We are closing this issue since it has not been prioritized for a long time. Chances are that it has already been solved in more recent versions. If not, we will be re-evaluating this when it becomes a priority.


Add block type editor buttons

Header
Quote

That's probably it for now until we can wrap list items correctly.

Dupe of #1042


add extension function binding system

@kevinlawler there are actually a lot of examples because this is how most of the chrome.* apis work


Did something in the dependency chain break?

 Not even sure if my title makes sense.
I deployed my app to heroku, and was getting a bunch of errors suddenly, after changes that "shouldnt" have led to this (I believe)
TypeError: Cannot use 'in' operator to search for '75em' in 20
  29 |   return paths.join('.').split('.').reduce(function (a, b) {
> 30 |     return a && b in a ? a[b] : null;
  31 |   }, obj);
  32 | };
  33 | export var themeGet = function themeGet(paths, fallback) {

In: ../node_modules/styled-system/dist/index.esm.js:30
Strangely I could not replicate the issue on my local dev version. However, after uninstalling styled-components, and rebass, and reinstalling the packages, I DO get this error, and can not resolve it.
I'm (quite possibly incorrectly) assuming that something in the chain broke, since I didn't get the issue until reinstalling the same version of rebass that i have been using all along. (I believe).
Could of course also be related to styled-components, I have been using the beta for that, which updates frequently. (Probably shouldnt be using the beta)
tldr: reinstalling the rebass and styled components packages has led to the above error, and I'm a stumped noob.
edit: I have rolled back all the way to styled-components 4.0.0-beta.2 and I still have the problem, so I can only guess that the issue is related to rebass or styled-system. (Or me doing something very clueless of course)

I installed styled-system 3.1.6 (the one before the recent updates), and things seem to be working locally.
So maybe I was actually right? :P

Thanks for reporting! I suspect the change in jxnblk/styled-system#297 is the culprit – are you using styled-system directly anywhere in your app? Could also be helpful to see the theme object and some examples of how you're using Rebass components

There are a few instances of me using styled-system to use the responsive css values:
import { height } from "styled-system";

export const FlexWithHeight = styled(Flex)`
	${height};
`;

But I am not sure if that's the issue since the error seems to be related to my usage of font sizes. I also get over 250 of these errors in my app, and I'm not using styled-system (directly) that often.
My theme object looks like this:
export const Theme = {
	breakpoints: ["481px", "769px", "1601px", "2001px"],
	colors: {
		lightShade1: "#FAFAFB",
		lightShade2: "#f2f1f2",
		darkShade1: "#011627",
		darkShade2: "#678",

		main: "#FF9F1C",
		accent1: "#4B9EC1",
		accent2: "#7FA232",

		text1: "#262B4A",
		text2: "#676b80",

		lightShadow: "#CCC",
		darkShadow: "#345"
	},
	fonts: {
		primary: "Open Sans, sans-serif",
		secondary: "PT Sans, Sans-Serif",
		title: "PT Sans, Sans-Serif",
		subtitle: "Open Sans, sans-serif",
		heading: "PT Sans, Sans-Serif",
		label: "Open Sans, sans-serif",
		button: "Open Sans, sans-serif",

		quote: "Alegreya, serif"
	},
	fontSizes: [12, 14, 16, 20, 24, 34, 48, 60, 96, 124, 192],
	shadows: {
		text: "0.05em 0.05em 0.02em rgba(0, 0, 0, .2)"
	}
};

And I (quite frankly) still haven't figured out a one-system-fits-all for how to use rebass components, so I'm all over the place with that.

Going to close this out for now until I have time to dig into the in operator issue in styled-system. Ensure you have styled-system >=3.1.11 installed to avoid this issue if you're still seeing it


Apple Wallet integration

I know it seems the OpenVerify app is for businesses only.
I was just wondering if there will be a version of this app to allow end users to easily add their QR code to Apple Wallet.

Can confirm you can add to Apple Health, but yes, with iOS 15.1 https://www.macrumors.com/2021/09/21/ios-15-vaccine-card-wallet-app/ you can presumably add to Wallet

This gap in convenient mobile support (Apple Wallet and photo pass support for Android) is the gap that https://grassroots.vaccine-ontario.ca was meant to address (full disclosure: I'm one of the developers on the vaccine-ontario.ca team and helped build this app, its repo is at https://github.com/billylo1/covidpass)

That is exactly what I was opening the issue for, to have an easy to click Add to Apple Wallet button as part of the official government website for end users to click and have setup within seconds. Glad to see someone has already done it but I think putting something like it on the government website would make it a lot easier for people to access and know about

I know it seems the OpenVerify app is for businesses only. I was just wondering if there will be a version of this app to allow end users to easily add their QR code to Apple Wallet.

Support for Apple Wallet is coming in iOS 15.1 (currently in pubic beta). If you are running 15.1 beta like I am, you just open you camera, point it your new enhanced certificate and with one tap, you can add it to Apple Health and Wallet.

This gap in convenient mobile support (Apple Wallet and photo pass support for Android) is the gap that https://grassroots.vaccine-ontario.ca was meant to address (full disclosure: I'm one of the developers on the vaccine-ontario.ca team and helped build this app, its repo is at https://github.com/billylo1/covidpass)

That is exactly what I was opening the issue for, to have an easy to click Add to Apple Wallet button as part of the official government website for end users to click and have setup within seconds. Glad to see someone has already done it but I think putting something like it on the government website would make it a lot easier for people to access and know about

100% agree with you, and we are very happy to assist in any way possible with that being done!

This is something we're working on with our Ministry of Health IT colleagues.

You can consider adding a button to download a .smart-health-card file. On iOS, this file will open the Apple Health app for importing, so user's would not need a seperate device to display a QR code from.
More Info:
https://spec.smarthealth.cards/#via-file-download

Just to update everyone, I have tried both the official way of scanning the QR code and the grassroots site and they both work great and are recognised by the OpenVerify/Verify Ontario app. I can say however, the grassroots site was a lot easier to use as I did not need an additional device to display the code to scan with my phone


Passing slices to JS allows mutating arbitrary WASM memory

Describe the Bug
&[u8] can be passed to JS, in the form of a Uint8Array slice into WASM memory. This can then be mutated from JS, kept after the lifetime of the slice, or passed back to Rust - effectively creating a safe version of Uint8Array::view.
Steps to Reproduce
use js_sys::Uint8Array;
use wasm_bindgen::prelude::*;
use wasm_bindgen_test::*;

// wasm-bindgen-test uses node, so this had to be commonjs
#[wasm_bindgen(inline_js = "exports.identity = x => x")]
extern "C" {
    // This is effectively a safe version of `Uint8Array::view`, which shouldn't be able to exist.
    #[wasm_bindgen(js_name = identity)]
    fn safe_view(slice: &[u8]) -> Uint8Array;
}

fn mutate_immutable_slice(slice: &[u8]) {
    let view = safe_view(slice);
    view.set_index(1, 5);
}

#[wasm_bindgen_test]
fn oh_no() {
    let array = [1, 2, 3, 4];
    mutate_immutable_slice(&array);
    // Mutation through an immutable reference - UB!
    assert_eq!(array, [1, 5, 3, 4]);
}

Expected Behavior
It shouldn't be possible to mutate through an immutable reference, particularly not after the lifetime of the reference ends. Either the ability to pass slices to JS should be removed, or the contents of the slice should be copied.
To be fair, JS can already modify arbitrary WASM memory, but you'd have to be explicitly accessing the raw memory and working with raw pointers. With this, seemingly innocuous code can cause undefined behaviour.
Actual Behavior
Passing an immutable slice to JS gives JS a permanent, mutable view of that region of WASM memory.
Additional Context
I was looking at #1187, and thinking it would actually be sound to return a value of any lifetime to WASM, since to be able to return it at all that type would have to explicitly implement IntoWasmAbi for any lifetime. But then I saw that the implementation for slices would create a view into WASM memory, and that they could be safely passed to JS by using an elided lifetime on the imported function.

Thanks for the report! This is sort of both a feature and a bug of wasm-bindgen. Wasm-bindgen itself doesn't know whether passing a slice to a JS function is supposed to be fast or not, so we pessimistically assume they all need to be fast and share direct views of wasm memory as a result. This, however, leads to the consequences you're seeing where there's no protection if wasm memory grows or against mutation from JS (since we can't otherwise prevent mutation).
Overall this decision was made to be pragmatic at the time (speed is good! sometimes...) but is not a great long-term solution. In the future this is what interface-types for WebAssembly is for where that has crisp definitions of all the types and semantic meanings about them, so it's clear that what wasm-bindgen does today would not suffice for those APIs and behavior would need to change.
In the interim though this is a "feature" of wasm-bindgen that I don't think can change. It's an unfortunate feature and one that should be fixed, but I think it's best fixed with the next evolutionary stage of wasm-bindgen (something interface-types based)


chore: publishable generator

This PR adds an nx plugin that is used to ensure a package has a proper package.json

Something happens with test-all/lint-all when there's a project.json in the root. It goes into an infinite loop. Not sure how to solve it...

Nice, I think this solution is much cleaner! There's a lot of boilerplate for nx stuff 😅


Last retry does not act as expected

PR: https://github.com/aneoconsulting/ArmoniK.Extensions.Csharp/pull/98


Company logo in menu

If company logo is set, it could be shown instead of "Support Companion" title in the menu, or alternatively in lower left corner of the menu with Quit button in lower right corner.

How it works at the moment is that if you set BrandName to an empty string, i.e., <string/>, it will not show anything but the logo

Yes, it does - in the app. In the new popdown menu it does not show the company logo. The menu could reflect the settings in the app and show the logo in the popdown menu as well.

Question is, should the logo show in the menu or should it be kept as lightweight as possible?

Also leaving the BrandName empty leaves the extra whitespace in the top of the menu. My suggestion is remove whitespace if there's no BrandName set and display the logo at bottom left corner and resized to appropriate maximum size (in case the logo would be huge otherwise). I don't see small('ish) logo as unnecessary clutter. And you can always include a HideLogoInMenu or such boolean in config for those who don't want it there.

Okay, I will add in the next release so that the logo is shown with a max width of 150 and scales to fit. The logo will be displayed where BrandName is and if you do not want the logo in the tray menu it can be hidden and only keep BrandName if need be

example:


Error installing on EC2 - Reported on BOC

https://discord.com/channels/933320455903907840/1247447145137573949/1247447145137573949
Ddhruv
I was able to clone the ONIX in an ec2 instance, I have configured a SSL and used the domain provided by EC2, now once I give the domain for registry setup, some issue is coming up as
................Installing Registry service................
unknown shorthand flag: 'f' in -f
See 'docker --help'.
Now, this was not expected, but it can be due to different versions of docker, etc. Then finally the Onix shows the registry installation is completed but the server is not started.
Error in some docker command
Installs but server is not  started
Ddhruv
OP
— Yesterday at 12:59 PM
Found the problem, in the installation file, its given as "docker compose -f", where as it should look like "docker-compose -f"

Here is Benoit from Startin'blox (France). We are diving into Beckn for some projects.
We are in the phase of setting up the Onix Project on EC2 Test instances and we do have the same error, could a patch be released please ?

@balessan
I also faced the same issue. And this is how I resolved it.
Step 1
Not sure why I noticed that when running docker-compose, it was showing segmentation fault.
Step 2
I tried deleting all docker files using
sudo apt  purge docker docker-compose docker-init

This did not work.
I was still able to execute docker commands.
Step 3
I used the whereis command to find the where all the docker files are situated and deleted using the rm command
whereis docker
whereis docker-compose

Now deleting it
sudo rm /usr/bin/docker
sudo rm /usr/bin/docker-compose

Like wise, deleted all the docker files manually.
At last for my Debian Machine went to the official docker docs and installed it from there .

@balessan Please install the latest docker version on you system and then try.
Make sure you have Docker version 27.1.2, build d01f264 and above.
The docker-compose no longer being actively maintained. Link
@Mishalabdullah Please remove the docker from you Debian machine and reinstall the latest version of docker.
For Debian please follow official steps here

Docker package installation script is updated which will install the latest docker if docker is not installed previously.
PR: https://github.com/beckn/beckn-onix/pull/190

Review Completed
@prasad-takale-eminds  please merge the PR as @vbabuEM has already approved it


Formatting

Hi @bsrikan @rashiatmarklogic , I've made this one last push to EA-3 niceness.
This PR delivers comments and cleanup, and provides one remaining fix for 40486.
Since comments have changed, this will trash all the keys for conversion module text, but for me it's a big improvement in readability.  If you get word to me this weekend or Monday, I'll merge this bug and move most of my remaining bugs to test or push them to ea-4.

So I'm heading out of town until Thursday.  I would love to push this PR to develop on friday so as to give time to adjust keys.  So just let me know when you're ready for it.

Sure, go ahead will try to absorb. Mainly conversion module keys will be changed. Ok noted. You can merge.

Thanks @rashiatmarklogic .  will make a new build today and run all test than merge (just cause I miss yalls so much)

Merged this.  Thanks folks!


棋子颜色搞混

文件路径
examples/game_gomoku/code/board_agent.py

使用matplotlib描述棋子颜色时搞混
            if board[y, x] == NAME_TO_PIECE[NAME_WHITE]:   ⭐ 这里应该是NAME_BLACK
                circle = patches.Circle(
                    (x, y),
                    0.45,
                    edgecolor="black",
                    facecolor="black",
                    zorder=10,
                )
                ax.add_patch(circle)
            elif board[y, x] == NAME_TO_PIECE[NAME_BLACK]:  ⭐ 这里应该是NAME_WHITE
                circle = patches.Circle(
                    (x, y),
                    0.45,
                    edgecolor="black",
                    facecolor="white",
                    zorder=10,
                )
                ax.add_patch(circle)

Thanks for your reporting! We will check and fix it as soon as possible.


Revert "Roll engine 8fff8da38da2..a544b45f26cc (3 commits)"

Reverts flutter/flutter#55492
TBR: @flar @iskakaushik
To fix the broken devicelab tests.

Merge on red to fix the tree.

An engine commit revert is also merged in https://github.com/flutter/engine/pull/17916

See also flutter/flutter#55519


Epic: TC State Substeps + Deck UI

This epic includes all tickets needed for MVP of TC State substep UI and deck UI (including UI components and hook-up work)

Merged into #5575


Additional XXE mitigation

Reference Issue
#872
What does this implement/fix? Explain your changes.
This prevents loading external images and elements when SvgDocument.ResolveExternalResources is false.
Any other comments?
This is just a draft pull request to get the ball rolling.

Should loading external images and text content like this be disabled by default, or should this behavior be behind a separate flag from ResolveExternalResources? The property name seems to indicate that it controls the loading of all kinds of external resources, but there might be users who expect to be able to include external images?
Should SvgImage and SvgElementIdManager access the static SvgDocument.ResolveExternalResources like this or should it somehow be passed down to them, like for SvgDtdResolver?
It's missing unit tests. I'll add some, but I'm just trying to figure out what external URI's I can use that will always be available and that will allow distinguishing behaviors from each other. Is it and should it be possible to run the tests without an internet connection?

It looks like quite a few (190) of the W3C test suite SVG's use external references, specifically for fonts, for example: https://github.com/svg-net/SVG/blob/062dc7c91cadf5607cceca1103f38628c0e065f8/Tests/W3CTestSuite/svg/color-prop-01-b.svg#L40
I have very little information about how SVG.NET is used elsewhere. It might be common and expected to load and render SVG's with external font references. However, in our use-case, we render user-uploaded SVG's, and would like to prevent them from both loading local files and making web requests. That also seems to be the concern of @gmwalker in https://github.com/svg-net/SVG/issues/869#issuecomment-831729398.
It could be that these W3C suite tests should be run with SvgDocument.ResolveExternalResources = true, or it could be that the behavior of loading external images and fonts should be controlled by another flag that's enabled by default.

For now, I enabled SvgDocument.ResolveExternalResources in the rendering tests.

I've added image comparison tests that render an SVG with an external image reference and one with a text reference, and check that the image and text is only shown if ResolveExternalSources is true. I put the tests in XxeVulnerabilityTests, but made the CompareSvgImageWithReferenceImpl from ImageComparisonTest public and static and used it. Let me know if you'd rather have the tests moved to ImageComparisonTest, have the rendering and comparison logic duplicated in XxeVulnerabilityTests, or think this could be tested better with a different kind of test.

I also think that having ResolveExternalSources default to false is the correct way, we just have to make sure that this is made clear in the release notes. As this is a breaking change, we'll probably need a new major release version, if we want to follow semantic versioning

I agree. If loading external images and other references is disabled by default, it's a breaking change and should be a new major version.

@H1Gdev: I'm not sure I quite understand your questions.

I think setting this feature with Boolean value will significantly reduce usability.

Do you mean to suggest having multiple different Boolean properties controlling the different types of external resources that can be loaded:
public bool ResolveExternalXmlEntites { get; set }
public bool ResolveExternalImages { get; set; }
public bool ResolveExternalElements { get; set }

Or something similar with an enum or flags enum?

In most cases, we want to disable external file reference, but we want to display images.

The current implementation in this pull request allows displaying images via data: URI's, but neither via local file system references nor web references:
<!-- Displayed when SvgDocument.ResolveExternalResources == false -->
<image href="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" />

<!-- Not displayed when SvgDocument.ResolveExternalResources == false -->
<image href="../../MySecretImages/Secret.png" />
<image href="https://example.org/External.png" />

Are you saying that, in most cases, we want to allow loading external images from the local file system and the web?

What do you want to disable in this PR ?

Our use-case is to allow users to upload SVG files and to generate previews of them. When rendering these previews, we want to:

Disable loading files from the local file system, since that could allow including confidential information from the local system in the previews.
Disable loading files from the web, since that exposes us to Server Side Request Forgery, where a user has our server make malicious web requests.

Please add property or method to evaluate.

I'm not sure where I should add a property or method to evaluate what?

@mrbean-bremen sorry for the late reply. Yes, I agree with what you said about the versioning, since it's a breaking change.
Regarding code, it's out of my comfort zone a bit. But from what I've seen, it's definitely an improvement, even if it isn't mitigating all possible vulnerabilities.

@tebjan - thanks! I also think that the changes are good, and we may just wait a bit for any other comments and ideas before merging them in.
@H1Gdev - maybe you can show in a code example what you mean? I also didn't quite get your review comments.

@kimsey0

Or something similar with an enum or flags enum?

I think flags enum is easy to use.
However, we haven't been able to identify use cases, so I'm wondering what kind of interfaces should be defined.

design 1[Flags]
enum ExternalType
{
  None = 0x0,
  Local = 0x1,
  Remote = 0x2,
}
public ExternalType ResolveExternalXmlEntites { get; set } = ExternalType.None;
public ExternalType ResolveExternalImages { get; set; } = ExternalType.Local;
public ExternalType ResolveExternalElements { get; set } = ExternalType.Local | ExternalType.Remote;

design 2[Flags]
enum ExternalType
{
  Local_XML = 0x1,
  Remote_XML = 0x2,
  Local_Image = 0x4,
  Remote_Image = 0x8,
  Local_Element = 0x10,
  Remote_Element = 0x20,
}
public ExternalType ResolveExternalResources { get; set } = ExternalType.Local_Image | ExternalType.Remote_Image;

Are you saying that, in most cases, we want to allow loading external images from the local file system and the web?

This depends on intended use of users.
(I mainly use it for SVG image output.)

Our use-case is to allow users to upload SVG files and to generate previews of them. When rendering these previews, we want to:

It seems that default values are different on server side or client side...
It may be kind to define default values in each case.

I'm not sure where I should add a property or method to evaluate what ?

Please add it to corresponding element.
(For example SvgImage)
public ExternalType ResolveExternalResources { get; set }

                if (ResolveExternalResources.HasFlag(ExternalType.Local))
                {
                }
                if (ResolveExternalResources.HasFlag(ExternalType.Remote))
                {
                }

@H1Gdev

I think flags enum is easy to use.

OK. I went with your design 1, since I think it makes both setup and usage of the properties more clear and makes it harder to accidentally overwrite flags for other resource types. Have a look!

It seems that default values are different on server side or client side...
It may be kind to define default values in each case.

I'm not exactly sure how this should be accomplished. We can't reliably determine which environment SVG.NET is being used in, so it would have to be configurable. We could add additional properties or configuration methods like (shorthand):
public static void ConfigureServerSecurity() => ResolveExternalXmlEntites = ResolveExternalImages = ResolveExternalElements = ExternalType.None;

But I'm not sure that would be any more discoverable or understandable than just setting the properties manually.
I've set ResolveExternalXmlEntites to ExternalType.None and ResolveExternalImages and ResolveExternalElements to ExternalType.Local | ExternalType.Remote for now. That keeps the current behavior on master, but the defaults changes from #870 may still be considered breaking and necessitate a new major version.

Please add it to corresponding element.
(For example SvgImage)
I can add this property to SvgImage, but I'm not sure how to get it initialized. SvgImage seems to be instantiated by SvgElementFactory, which is source generated by AvailableElementsGenerator. I can't find any functionality to initialize this kind of property which isn't bound to anything in the SVG file. Can you point me in the right direction?

@kimsey0

Please add it to corresponding element.
(For example SvgImage)

I can add this property to SvgImage, but I'm not sure how to get it initialized. SvgImage seems to be instantiated by SvgElementFactory, which is source generated by AvailableElementsGenerator. I can't find any functionality to initialize this kind of property which isn't bound to anything in the SVG file. Can you point me in the right direction?

At this time, there is no need to initialize.
A place that has access to direct SvgDocument from SvgImage, ensure that you access through the property or method of SvgImage.
It may be better to have it in SvgElement.

But I'm not sure that would be any more discoverable or understandable than just setting the properties manually.

If we want to make it stronger, you can force users to set it.
If not set, an exception will be thrown and it will not work.

Sorry for the long delay - I missed the question, and I didn't follow up here, as I was busy with other stuff.
I just had another look, and I actually think this is sufficient as is. It does the main task to make the library usage safer by default, and allows to configure it easily.
I will wait a couple of days for further comments and merge if nothing substantial comes up. It also probably makes sense to make a new release soon.

It looks like some code hasn't been fixed yet.

It looks like some code hasn't been fixed yet.

You refer to the private method made public, and the reference to the test? Yes, these could be changed, though these are minor things. Anything else?

Maybe that's it.

Hey @mrbean-bremen ,
is it possible to mark the nuget-versions before this fix as vulnerable, so using the older version will know that there might be security issues?


Add support for explicitly indicating that a function's return type is inferred.

Search Terms
explicit infer return type
Suggestion
Add support for a special infer type that can be used to explicitly indicate that the return type of a function should be inferred from its implementation.
The presence of an explicit infer return type for a function (or arrow function) would have identical observable behavior as if the return type wasn't specified: the return type is inferred from the implementation.
The difference is that the function now technically has a return type specified to indicate that the developer explicitly made a decision to allow the return type to be inferred, rather than simply forgetting to think about what the return type should be.
Use Cases
In a project with strict/explicit code style, it is desirable in general to use a linting rule that requires all functions to have a return type specified (The tslint "typedef" rule for "call-signature", for example: https://palantir.github.io/tslint/rules/typedef/).
There are, however, some situations where the return type of a function is a complex type (involving generics, etc.), such that the return type is more naturally inferred from the implementation of the method, as opposed to it being natural to know the intended return type ahead of time.
In such situations, it would be nice to have the option to explicitly indicate the intent for the compiler to infer the return type of the function. This would both clearly communicate this explicit intent to other developers reading the code, and could be used to satisfy code style and linting rules that require functions to have a return type specified.
Why not just disable the linting rule on a case-by-case basis?
Yes, tslint's support of line-level rule disabling could be used to temporarily disable the linting rule and allow you to omit the return type in such situations. But I feel that language-level support for an explicit infer return type would be much more powerfully clear and expressive.  It is especially worthwhile if it is fairly low effort/risk to implement.
Additionally, there is not enough granularity to disable the "typedef" rule ONLY for the "call-signature" context. Disabling "typedef" on that line would also disable checks for parameter types in the signature, which is undesirable.
Examples
A simple example is helper methods for unit tests that use enzyme. Let's say you have a custom React component named MyButtonComponent, and you would like a helper function to find the "close" button within some other outer component:
import { ReactWrapper } from "enzyme";
import { MyButtonComponent } from "./MyButtonComponent";

function findCloseButton(wrapper: ReactWrapper): infer {
    return wrapper.find(MyButtonComponent).filter(".close-button");
}

In this case, the line return wrapper.find(MyButtonComponent).filter(".close-button") returns a strictly typed ReactWrapper<P> type where P is the Props type of the MyButtonComponent component, which allows type-safe access to the current values of the component's props via ReactWrapper's props() method .
Manually writing out the correct return type of the helper function above would be quite tedious and provide very little benefit. Especially if the Props interface for MyButtonComponent is not readily available either because it is not exported, or because MyButtonComponent was created by a complex generic higher order component (HOC). The effort of correctly writing out the return type outweighs its benefits in a situation like this.
Inferring implicit any
In the following examples, there is no implementation to infer the return type from:
declare function foo(a: number): infer;

interface Bar {
    bar(b: string): infer;
}

The infer type is actually quite useless here and is guaranteed to be an implicit any type.  Perhaps it would make sense to ONLY allow the infer type in contexts where there is something to infer from? If so, then the above examples would be compiler errors because there is nothing to infer from.
Another reasonable option may be to allow it to infer an implicit any type in such a way that will fail to compile when using the --noImplicitAny compiler option. This option is a bit less direct, but perfectly acceptable if it is the most "natural" and low-risk/effort option based on the current structure of code.
Checklist
My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript / JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. new expression-level syntax)

What will we do when TSLint adds a rule that bans : infer ? 🤔

A TSLint rule that bans : infer would be irrelevant, because users of TSLint can choose which rules to enable/disable. That would actually provide a good flexibility. Let's imagine a new rule called "no-infer-return-type" rule is added. You really have 3 levels of "strictness" you can enforce on function signatures:

Disable "typedef" rule for "call-signature" if you want to allow for implicitly inferred return types in your project.
Enable "typedef" rule for "call-signature" and "no-infer-return-type" if you want to force explicit non-inferred return types in your project.
Enable "typedef" rule for "call-signature" and DISABLE "no-infer-return-type" if you want to force explicit return types in your project, but still allow : infer.

You could also explicitly indicate your intent to use an inferred return type by using // tslint:disable-next-line typedef, and then no new TS changes would be needed.

@nmain yes, I acknowledged this already.

Yes, tslint's support of line-level rule disabling could be used to temporarily disable the linting rule and allow you to omit the return type in such situations. But I feel that language-level support for an explicit infer return type would be much more powerfully clear and expressive. It is especially worthwhile if it is fairly low effort/risk to implement.

Additionally, there is not enough granularity to disable the "typedef" rule ONLY for the "call-signature" context. Disabling "typedef" on that line would also disable checks for parameter types in the signature, which is undesirable.

An expansion on this idea would be partially inferred return types:
function foo(value: number): Promise<infer> {
    // return value must be assignable to Promise<any>
    // return type is inferred to be Promise<number>
    return Promise.resolve(value);
}

interface Pair<A, B> {
    a: A;
    b: B
}

function bar(value: number): Pair<infer, infer> {
    // return value must be assignable to Pair<any, any>
    // return type is inferred to be Pair<number, boolean>
    return {
        a: value,
        b: value % 2 === 0
    };
}

This would provide a nice blend of compile time validation that you are returning something in the basic format that you intend, but let the compiler infer complicated parts of the type that are not worth the effort of manually constructing/writing.

Some follow-up thoughts on my previous comment...
I think it should be fairly easy to implement partially inferred return types. The basic concept is:

If infer exists anywhere within the return type, then infer the return type of the function.
Validate that the inferred return type is assignable to the specified return type:

Create a temporary "expected" return type by replacing all infers with any in the specified return type.
Fail if the inferred return type is not assignable to the temporary "expected" return type.


Changed how Cozy gets the list of freebies to use.

Instead of grabbing from hints, it now uses the new states_maintained_by(query) method in Implementation in impls.py.
The new method takes a query and returns all of the state variables that is being maintained by the query.

Looks great!


Fix inconsistency in consuming messages in batches

This issue may need to be transferred to kafka binder, so for now it's just a place holder
See StackOverflow for more details - https://stackoverflow.com/questions/70423758/consuming-kafka-messages-with-its-key-in-batches-using-spring-cloud-stream-kafka/70424028#70424028
tagging @sobychacko

Any news about this issue which is more than one year old ?

It is not compliant with the Spring Cloud Stream reference documentation to receive Message<List> instead of List>
It is really not convenient to have to match key and value of each Kafka records through matching indexes in 2 separate lists


getUserByAccount Error Using Hasura Example

I am seeing the following error when I try the example provided here. I've created the table structure defined in the article. I was trying out the Github OAuth integration. Could be wrong, but it seems like the getUserByAccount function is trying to filter on the accounts field in the Users model/table, but that field doesn't exist in the schema. Thoughts?
API handler should not return a value, received object.
[next-auth][error][adapter_error_getUserByAccount]
https://next-auth.js.org/errors#adapter_error_getuserbyaccount field 'accounts' not found in type: 'users_bool_exp': {"response":{"errors":[{"message":"field 'accounts' not found in type: 'users_bool_exp'","extensions":{"path":"$.selectionSet.users.args.where.accounts","code":"validation-failed"}}],"status":200,"headers":{}},"request":{"query":"query GetUsers($where: users_bool_exp!) {\n  users(where: $where) {\n    ...User\n  }\n}\n\nfragment User on users {\n  __typename\n  id\n  name\n  email\n  image\n  emailVerified\n}","variables":{"where":{"accounts":{"provider":{"_eq":"github"},"providerAccountId":{"_eq":"5010504"}}}}}} {
  message: `field 'accounts' not found in type: 'users_bool_exp': {"response":{"errors":[{"message":"field 'accounts' not found in type: 'users_bool_exp'","extensions":{"path":"$.selectionSet.users.args.where.accounts","code":"validation-failed"}}],"status":200,"headers":{}},"request":{"query":"query GetUsers($where: users_bool_exp!) {\\n  users(where: $where) {\\n    ...User\\n  }\\n}\\n\\nfragment User on users {\\n  __typename\\n  id\\n  name\\n  email\\n  image\\n  emailVerified\\n}","variables":{"where":{"accounts":{"provider":{"_eq":"github"},"providerAccountId":{"_eq":"5010504"}}}}}}`,
  stack: `Error: field 'accounts' not found in type: 'users_bool_exp': {"response":{"errors":[{"message":"field 'accounts' not found in type: 'users_bool_exp'","extensions":{"path":"$.selectionSet.users.args.where.accounts","code":"validation-failed"}}],"status":200,"headers":{}},"request":{"query":"query GetUsers($where: users_bool_exp!) {\\n  users(where: $where) {\\n    ...User\\n  }\\n}\\n\\nfragment User on users {\\n  __typename\\n  id\\n  name\\n  email\\n  image\\n  emailVerified\\n}","variables":{"where":{"accounts":{"provider":{"_eq":"github"},"providerAccountId":{"_eq":"5010504"}}}}}}\n` +
    '    at makeRequest (~/next-auth-example/node_modules/graphql-request/build/cjs/index.js:310:15)\n' +
    '    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' +
    '    at async getUserByAccount (~/next-auth-example/node_modules/next-auth-hasura-adapter/dist/index.js:46:25)',
  name: 'Error'
}

Figured it out. For folks in the future that run into this, you must name the relationship between the accounts and users types properly. The relationship name needs to be accounts. Here's what my relationship looks like after fixing:

@lewisdawson thank you so much for posting the answer to your issue.


API Proposal for Classification Feature

API for #948
Three API use cases need to be addressed:

Start a classification, see status of ongoing classification
View classified items
View meta data about a thing which was part of a classification

Here are proposals for all three:
1. Trigger Classification, see Status
POST /v1/classifications
{
  "class": "Company",
  "classifyProperties": ["materialGroupOne", "materialGroupTwo"],
  "basedOnProperties": ["description"],
  "type": "knn", // optional, can default to knn
  "k": 3 // optional, can default to something reasonable
}

returns GET response + Header 'Location: /v1/classifications/<id>'

GET /v1/classifications/<classificationID>
{
  "id": "abcdefgh",
  "class": "Company",
  "classifyProperties": ["materialGroupOne", "materialGroupTwo"],
  "basedOnProperties": ["description"],
  "status": "running",
  "started": <timestamp>,
  "completed": <timestamp> || null,
  "type": "knn"
  "k": 3
}

2. View classified items
No new APIs required, as classifications are just cross-refs. Any existing measures to view crossrefs (resolved in GQL Get or as beacons in REST GET /v1/things/<id> can be used.
3. View metadata about a thing which received a ref from a classification
Regular GET /v1/thing/<id> without meta parameter
Unchanged, same as always:
GET /v1/things/<id>
{
  "class": "Company",
  "id": "<id>",
  "schema": {
    "description": "Foo bar baz bazzar",
    "materialGroupOne": [
      {
         "beacon": "..."
      }
    ],
   "materialGroupTwo": [
      {
         "beacon": "..."
      }
    ]
  }
}

Get with meta parameter GET /v1/thing/<id>?meta=true
GET /v1/things/<id>
{
  "class": "Company",
  "id": "<id>",
  "schema": {
    "description": "Foo bar baz bazzar",
    "materialGroupOne": [
      {
         "beacon": "...",
         "source": "user-input"  <----- indicates this has not been classified, but set by a user
      }
    ],
   "materialGroupTwo": [
      {
         "beacon": "..."
         "source": "classification"  <----- indicates this has not been classified
      }
    ]
  }
  "meta": {
    "classification": {
      "id": "<classificatinId>",
      "completed": <timestamp>,
      "classifiedFields": ["matericalGroupTwo"],
      "scope": ["materialGroupOne", "materialGroupTwo"]  <----- note that the scope was to classify both fields, but since the user had already set materialGroupOne the actual classified Fields (see one row up) is only a single field (materialGroupTwo)
      "basedOn": ["description"],
      "distance": 0.334
    }
  }
}

What kind of distance metric is used?
I would like to have some kind of (average) distance in the meta object:
"meta": {
    "classification": {
      "id": "<classificatinId>",
      "completed": <timestamp>,
      "classifiedFields": ["matericalGroupTwo"],
      "scope": ["materialGroupOne", "materialGroupTwo"]  <----- note that the scope was to classify both fields, but since the user had already set materialGroupOne the actual classified Fields (see one row up) is only a single field (materialGroupTwo)
      "basedOn": ["description"]
      "distance": 0.334
    }
  }

This can serve as some kind of confidence or quality measure.
Typo in example this should mean: "source": "classification"  <----- indicates this has been classified, but set by a user

Great point abut the distance, @fefi42. I agree.
What would be the best way to calculate the distance?

Mean distance between classified item and all k neighbors?

In addition would it be helpful to include how many neighbors were of the same class.
For example if k=7,

if 4 point to A and 3 point to B, we classify as A
if all 7 point to A, we also classify as A

Could we argue that the second classification was done with a higher confidence? If so how would we express this? As part of the distance or as a separate metric?

One additional idea comes to mind.
Being able to set a where filter:
POST /v1/classifications
{
  "class": "Company",
  "classifyProperties": ["materialGroupOne", "materialGroupTwo"],
  "basedOnProperties": ["description"],
  "type": "knn", // optional, can default to knn
  "k": 3 // optional, can default to something reasonable
  "where": {
    // same as where in GQL query
  }
}

It would allow users to: ”e.g., only classify companies with more than 1000 employees”

Also, this feature seems to become more and more important.
basedOnProperties should not be set to none-indexable.

@bobvanluijt
Great point about the where filter, I also really like the fact that we are enriching the REST API with something that was previously exclusive to the GQL. (Once we have the where filter in one place it should be trivial to also copy it to other places like GET /v1/things)

basedOnProperties should not be set to a nonindexable property.

Fully agreed. In fact it's a technical necessity for the field to be indexed, so that we can even do a classification. I will see that I can prioritize #952 right after the first usable version of classifications.

Thanks
An example of a non-indexable use case might be articles. We want the c11y to index the abstract, but not the meta data.

I am currently not sure whats the best way to calculate an average distance, maybe it would be best to just give three values. Min, arithmetic average, Max This would help to quickly spot if n is chosen correctly. E.g. if min is very small but max and mean are very high this would indicate that n is too high.

Wouldn't a where clause also improve performance? Because it reduces the search space.

Wouldn't a where clause also (potentially) improve performance? Because it reduces the search space.

Your point is a good one, it highlights that there are two spaces we can limit. Let's take the examples that I have Product ---MaterialGroup---> Material and I want to classify products as belonging to a material group. I know how two potential filtering options:
A.) "Only classify those Products which have price > 100"
B.) "Only consider target MaterialGroups where organic==true"
@bobvanluijt which one did you mean when suggesting the where filter? I implicitly assumed we mean A., but the point Felix raises would also apply to B.

I am currently not sure whats the best way to calculate an average distance

@fefi42 I'm not sure I understand. We want to solve this for the end user in Weaviate rather than have the end user do it, right?

I am currently not sure whats the best way to calculate an average distance
Yes we show this to the enduser. I would just start with the normal average, nothing fancy like weighted sums or something. Thats what this was referring to. We can improve on it later

Closing as proposal accepted, currently being implemented in #976

Reopening as we realized, we need the distances per prop, not overall.

Resulting from a discussion on slack:
Updated Proposal
Changes since the original proposal:

Each cross ref now has a meta object (visible only when ?meta=true is set)
The meta object contains the mean distance of the winning and losing group
The overall meta object has no more distance field

Notes / Limitations

The overall meta object only shows the meta data of the last classification run. If multiple classifciations were run the meta data from previous requests is lost

Example
Copied from updated source post
Get with meta parameter GET /v1/thing/<id>?meta=true
GET /v1/things/<id>
{
  "class": "Company",
  "id": "<id>",
  "schema": {
    "description": "Foo bar baz bazzar",
    "materialGroupOne": [
      {
         "beacon": "...",
         "meta": null  <----- indicates this has not been classified, but set by a user
      }
    ],
   "materialGroupTwo": [
      {
         "beacon": "...",
         "meta": {
            "classification": {
              "distanceWinning": 0.5,
              "distanceLosing": 0.6,
            },
          },
      }
    ]
  }
  "meta": {
    "classification": {
      "id": "<classificatinId>",
      "completed": <timestamp>,
      "classifiedFields": ["matericalGroupTwo"],
      "scope": ["materialGroupOne", "materialGroupTwo"]  <----- note that the scope was to classify both fields, but since the user had already set materialGroupOne the actual classified Fields (see one row up) is only a single field (materialGroupTwo)
      "basedOn": ["description"],
    }
  }
}

@fefi42 @bobvanluijt can you quickly confirm this proposal, then I can start implementing it. Thanks.

Confirmed, I will need a more detailed understanding of the changes for the docs but this isn't crucial for now.

Closing again as accepted, currently being implemented in #979.


Exhaustion doesn't work properly for movement (at least)

Describe the bug
Not a very big deal, but exhaustion level are not parameterized the same way than it is in the tidy sheet, and doesn't work properly in DFreds (at least some)
To Reproduce
Steps to reproduce the behavior:
For example, exhaustion level 2, yours on the right :

I'm not sure if it's intended or not, but the thing that made me disturb you about that is the "move reduction" doesn't work properly :
data.attributes.movement.walk
for a barbarian who has extra move with level 5, only the base "9 meters / 30 feet" are divided, not the bonus... But directly from the tidysheet, it does (because of the "multiply" and not "customize" I guess ? )
This issue is just in case you don't know that already... Thanks for your module anyway. (can't wait for the localisation)

Okay, thanks for the heads up. I'm guessing this is in dnd5e 1.6?

Yes absolutely... I admit I don't know if it was already the case before 1.6 ...
I previously used your module only for conditions... I added exhaustion ("épuisement") recently...


Keep .vscode from appearing in untracked files

Reasons for making this change:
Whenever I ran git status in my project directory the .vscode subdirectory would erroneously show up in the untracked files list.

@icylace the rules in this gitignore should cover the specific files that VS Code generates that shouldn't be tracked - I'm 👎 to disabling this globally.

Yes, this gitignore does cover specific files but why not also cover the folder name itself as I'm suggesting ?

@icylace because the existing rules should cover this - line 2 will ignore all files, and the lines below that will then track specific files that should be in version control

Thank you for your information, Ron. I am an amateur to say the least but
can follow instructions, in most cases, quite well. Though I am rather tech
savvy to an extent, I feel as though I will be staying away from any
developments and code, etc. None the less, I thank you very much for your
emails and my apologies for not responding before now.
Regards,
Chris McGahee
On Sun, Mar 4, 2018, 3:20 AM Ron Martinez<EMAIL_ADDRESS>wrote:

Reasons for making this change:
Whenever I ran git status in my project directory the .vscode
subdirectory would erroneously show up in the untracked files list.
You can view, comment on, or merge this pull request online at:
https://github.com/github/gitignore/pull/2618
Commit Summary

Keep .vscode from appearing in untracked files

File Changes

M Global/VisualStudioCode.gitignore
https://github.com/github/gitignore/pull/2618/files#diff-0 (1)

Patch Links:

https://github.com/github/gitignore/pull/2618.patch
https://github.com/github/gitignore/pull/2618.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/github/gitignore/pull/2618, or mute the thread
https://github.com/notifications/unsubscribe-auth/AixW_khCpwTPJiYCu7s5CrSc8ye8Pswiks5ta6PSgaJpZM4SbPmf
.

I've come to realize that my technical skills are not to the level of
riding toad, development, etc. I think all of you for your emails but will
be stepping back from this. Have a great day!
Regards,
Chris McGahee
On Mon, Mar 5, 2018, 1:54 PM Brendan Forster<EMAIL_ADDRESS>wrote:

@icylace https://github.com/icylace because the existing rules should
cover this - line 2 will ignore all files, and the lines below that will
then track specific files that should be in version control
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/github/gitignore/pull/2618#issuecomment-370523470,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AixW_jmWcNiXZILnrOmkarZVqLAFk0Y-ks5tbYnWgaJpZM4SbPmf
.

@icylace I believe the current template should cover what you need. Please try it out and open a new PR if there are any changes that need to be included.


Move internal to pkg to allow library re-use

The usage of internal does not allow us to re-use the packages in other go applications. Changing to pkg will allow us to reference the source code in other applications as well.

Hi @maordavidov
We are using the internal on purpose. That way, we are not obligated to maintain backward compatibility, and we can make any changes we want there.
What functionalities are you looking for to re-use from the oauth2c internal package? I would instead expose only some of the internal functionalities in pkg using a new abstract interface so that way we can limit the api we need to keep backward compatible to a bare minimum.


Qustion - How to convert multiple postman collections at once..?

is it possible to convert 2 postman collections at once..?
for eg: is it possible to do something like  postman-to-k6 collection1.json  collection2.json -o k6-script.js like this..?
if not is there any workaround to achieve the same..?

hi @EdisonEy
That is not possible through the postman-to-k6 CLI, but some libraries could do this before you pass it to the CLI.
Like: https://www.npmjs.com/package/postman-combine-collections (disclaimer, I have not tested the library)


Add method: LeaderBoard#retrieveMember

1、Add a method retrieveMember in LeaderBoard.
2、Combine some duplicate codes.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@ljy0827 Thanks for your contribution.
Current code looks good to me, but you also need add retrieveMember to DistributedLeaderBoard.java.


is it able to change the UI binding port from 8080 to another?

env: Centos
version: 6.6.0-es7
This scenario blocked me a lot that, as a Java dev, we always open & monitor 8080 to offer our backend service, but, when I need to deploy skywalking the same node with my service, I have no idea but changing my port of service from 8080 to other one (e.g. 8081), so that I can access the skywalking UI then
And, I searched the config files & other online doc, but no one else can tell the stable way to change this config.
So, my question is, is it able to change the default UI port of skywalking from 8080 to others instead of  modify the code and re-complie it?

you can change port in apache-skywalking-apm-bin/webapp/webapp.yml


Feature/issue 62 gammu gsm transport

supports:

Unicode messages
Multipart message sending (splits up after 160 bytes)
Multipart message receiving

Doesn't support delivery reports because Gammu's python-gammu bindings for that seem to be broken.

Done commenting. Looks good (modulo minor comments).


fix: docker-file for private packages (#50)

@Cahllagerfeld what do you think?

I think we are quite close.
The question is how can I adjust the action as its not a PR linked to my fork anymore? 😅

@Cahllagerfeld you already have write access to this repo, so you can edit this branch/PR directly

You're right I forgot about that...
I made the adjustments

Great thank you @Cahllagerfeld 👍


Unable to apply isolated style to MudDialog using Class API

Bug type
Component
Component name
MudDialog
What happened?
When displaying dialog using dialogbox service with a dialog template defined in a separate razor component, the class defined in the component css file are not correctly applied.
Here is the Counter.razor page which display the dialog when clicking the button.
@inject IDialogService _dialogService
@page "/counter"

<PageTitle>Counter</PageTitle>

<MudText Typo="Typo.h3" GutterBottom="true">Counter</MudText>
<MudText Class="mb-4">Current count: @currentCount</MudText>
<MudButton Color="Color.Primary" Variant="Variant.Filled" @onclick="IncrementCount">Click me</MudButton>

@code {
    private int currentCount = 0;

    private void IncrementCount()
    {
      _dialogService.Show<Box>("title", new DialogParameters(), new DialogOptions()
      {
        CloseButton = true,
        FullWidth = true
      });
    }
}

Here is the dialog component razor file: Box.razor
<MudDialog Class="redhello">
  <DialogContent>

Hello

the world

  </DialogContent>
</MudDialog>

And here is the style defined in the Box.razor.css
.redhello
{
  color: red;
  background-color: lightyellow;
}

Here is the result after the dialogbox is displayed.

And here is the DOM:

title

Hello

the world

As we can see, that the "redhello" is applied to both muddialog, and the inner div, but due to the lack of b-cgqs55320o attribute in the muddialog level, the style is not applied to the dialog level.
I understand that the b-cgqs55320o comes from the CSS Isolation. The problem is I cannot figure it a way to apply the style contained in the component.razor.css when CSS isolation is not disabled.
Expected behavior
The custom style is applied to the entire dialog:

Reproduction link
https://github.com/aDisplayName/bugsamplecode/tree/main/20220411
Reproduction steps
When displaying dialog using dialogbox service with a dialog template defined in a separate razor component, the class defined in the component css file are not correctly applied.
Here is the page which display the dialog when clicking the button.
@inject IDialogService _dialogService
@page "/counter"

<PageTitle>Counter</PageTitle>

<MudText Typo="Typo.h3" GutterBottom="true">Counter</MudText>
<MudText Class="mb-4">Current count: @currentCount</MudText>
<MudButton Color="Color.Primary" Variant="Variant.Filled" @onclick="IncrementCount">Click me</MudButton>

@code {
    private int currentCount = 0;

    private void IncrementCount()
    {
      _dialogService.Show<Box>("title", new DialogParameters(), new DialogOptions()
      {
        CloseButton = true,
        FullWidth = true
      });
    }
}

Here is the dialog component razor file: Box.razor
<MudDialog Class="redhello">
  <DialogContent>

Hello

the world

  </DialogContent>
</MudDialog>

And here is the style defined in the Box.razor.css
.redhello
{
  color: red;
  background-color: lightyellow;
}

Here is the result after the dialogbox is displayed.

And here is the DOM:

title

Hello

the world

As we can see, that the "redhello" is applied to both muddialog, and the inner div, but due to the lack of b-cgqs55320o attribute in the muddialog level, the style is not applied to the dialog level.
I understand that the b-cgqs55320o comes from the CSS Isolation. The problem is I cannot figure it a way to apply the style contained in the component.razor.css when CSS isolation is not disabled.
Relevant log output
No response
Version (bug)
6.0.9
Version (working)
No response
What browsers are you seeing the problem on?
Microsoft Edge
On what operating system are you experiencing the issue?
Windows
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I found a possible solution/reason. As stated in the following blog:
https://www.daveabrock.com/2020/09/10/blazor-css-isolation
It states that you need to wrap a simple  arround your component.:

  <MudDialog Class="redhello">
    <DialogContent>

Hello

the world

    </DialogContent>
  </MudDialog>

This might be your solution.

I'm also having this problem. Using the
 wrapper suggested by @Xorpio didn't change anything. But using ::deep seems to work for me:
::deep .mud-input-label {
    font-size: 120px !important
}

At least, this way I can modify the labels inside the dialog from within the dialog's css file...

I'm also having this problem. Using the
 wrapper suggested by @Xorpio didn't change anything. But using ::deep seems to work for me:
::deep .mud-input-label {
    font-size: 120px !important
}

At least, this way I can modify the labels inside the dialog from within the dialog's css file...

Make a tryblazor repoducing the issue and tag me I'll look at it


reporting unused imports

made basic reporting working.

[x] basic reporting
[x] use correct message on diagnostic
[x] Add support for the new unnecessary tag to have fading effect (https://microsoft.github.io/language-server-protocol/specifications/specification-3-15/#diagnostic)
[x] check VS and vscode whether it supports new unnecessary tag spec
[x] add tests

... I am splitting out code fix part to different PR

[ ] add remove unused import code fix
[ ] etc (might have more work item once I get to remove unused import such as grouping unused imports based on what gets removed together on 1 code fix and etc)
...

this is with latest lsp client<EMAIL_ADDRESS>+ latest vscode.
without it installed, you will get "..." rather than fading out.

...
to get new lsp client that supports fading, do this.
first, take changes in https://github.com/heejaechang/vscode-python/tree/newlspclient
and then do npm install and npm ci

@Anapo14 can you take a look to see whether this meets your expectations?
@MikhailArkhipov @jakebailey @bschnurr can you code review? this is reporting part. code fix part is blocked by my "add import" PR since it require code action portion I added there.

@Anapo14 can you take a look to see whether this meets your expectations?

based on the gif, it looks good to me!


JPHUB variable not used in all scripts

./skel/.profile
./scripts/install_system.sh
./scripts/kernel_python

Only .profile left at that time.


Release for cdn 539

release for cdn. https://github.com/Azure/sdk-release-request/issues/539

Closing in favor of #10521


aws.ec2.SpotInstanceRequest ignores launch_template

Hello!

Vote on this issue by adding a 👍 reaction
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details
I have an aws.ec2.LaunchTemplate from which I can launch EC2 instances:
aws.ec2.Instance("on-demand-instance",
                 launch_template={"id": launch_template})

I've been trying and failing to use the same to launch EC2 spot instances. According to the documentation it should be as simple as:
aws.ec2.SpotInstanceRequest("on-demand-instance",
                            launch_template={"id": launch_template})

Unfortunately this does not work and I get various error messages like:
        * Error requesting spot instances: MissingParameter: The request must contain the parameter imageId
        status code: 400, request id: 878ebf1d-3edf-4447-b6fb-3b155116ec37

And
        * Error requesting spot instances: MissingParameter: The request must contain the parameter instanceType
        status code: 400, request id: 08998f08-0524-4a49-84cf-e45b6baf12e0

So it seems like aws.ec2.SpotInstanceRequest is completely ignoring launch_template, unlike aws.ec2.Instance.

@stack72 Did you ever manage to replicate this issue with my code examples?

I can reproduce the issue. Something looks odd about this resource setup. The upstream docs have no mention of launch_template at all: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/spot_instance_request
I suspect something is wrong with our use of IncludeArgumentsFrom: "aws_instance": https://github.com/pulumi/pulumi-aws/blob/master/provider/resources.go#L1760C6-L1760C26
Given the note below (from the docs), we may want to deprecate this resource...

AWS strongly discourages the use of the legacy APIs called by this resource. We recommend using the EC2 Instance resource

Since this issue is pretty old and aws has recommended not using this legacy API for several years I am going to close this issue as won't-fix. We recommend using the recommended EC2 Instance resource instead.