[Swift] Unrecognized enum values result in a runtime crash

What version of gRPC and what language are you using?
Excerpt from Podfile.lock (our name redacted):
  - "!ProtoCompiler-gRPCPlugin (1.14.0)":
    - "!ProtoCompiler (= 3.5.0)"
    - gRPC-ProtoRPC (= 1.14.0)
...
  - gRPC (1.14.0):
    - gRPC-RxLibrary (= 1.14.0)
    - gRPC/Main (= 1.14.0)
  - gRPC-Core (1.14.0):
    - gRPC-Core/Implementation (= 1.14.0)
    - gRPC-Core/Interface (= 1.14.0)
  - gRPC-Core/Implementation (1.14.0):
    - BoringSSL (~> 10.0)
    - gRPC-Core/Interface (= 1.14.0)
    - nanopb (~> 0.3)
  - gRPC-Core/Interface (1.14.0)
  - gRPC-ProtoRPC (1.14.0):
    - gRPC-ProtoRPC/Main (= 1.14.0)
  - gRPC-ProtoRPC/Main (1.14.0):
    - gRPC (= 1.14.0)
    - gRPC-RxLibrary (= 1.14.0)
    - Protobuf (~> 3.0)
  - gRPC-RxLibrary (1.14.0)
  - gRPC/Main (1.14.0):
    - gRPC-Core (= 1.14.0)
    - gRPC-RxLibrary (= 1.14.0)
...
  - XxxGRPC (0.0.145):
    - "!ProtoCompiler-gRPCPlugin"
    - XxxGRPC/Messages (= 0.0.145)
    - XxxGRPC/Services (= 0.0.145)
  - XxxGRPC/Messages (0.0.145):
    - "!ProtoCompiler-gRPCPlugin"
    - Protobuf
  - XxxGRPC/Services (0.0.145):
    - "!ProtoCompiler-gRPCPlugin"
    - gRPC-ProtoRPC
    - XxxGRPC/Messages

The gRPC code is used in our Swift 4.0 codebase in Xcode 10 (from generated Objective C code).
What operating system (Linux, Windows, …) and version?
MacOS
What runtime / compiler are you using (e.g. python version or version of gcc)
$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1<IP_ADDRESS>)
Target: x86_64-apple-darwin17.7.0

What did you do?
Cases of an enum were previously handled in a switch like this (minimal example):
guard let field = XXXFields(rawValue: fieldValue)
   else { continue }

switch field {
case .field1: handleField1()
default: break
}

It was then changed to something like this:
guard let field = XXXFields(rawValue: fieldValue)
   else { continue }

switch field {
case .bool1: handleField1()
case .gpbUnrecognizedEnumeratorValue: break
}

Note: after the change, all known enum cases are handled (i.e. the compiler does not complain about missing cases).
Then, after compiling this into our iOS app, the proto was updated to add a new enum case which is not handled by our app.
What did you expect to see?
We expected that, when receiving an enum value that is not recognized by the app, that the unrecognized enum value would be handled gracefully, by either

failing to instantiate an enum object (i.e. guard let field = ... should fail), or
instantiating the gpbUnrecognizedEnumeratorValue case for the enum object.

What did you see instead?
After the change, receiving a new enum case not handled by our version of the proto specification results in a runtime crash with an error message like Fatal error: unexpected enum case 'XXXFields(rawValue: 2)'
The crash did not occur in the first version (with a default case). However, the second version is preferable for us because it lets us use the compiler to enforce that all known cases are handled.

@thomasvl - I feel this is something related to the protobuf codegen. Could you comment a bit?

@muxi sorry, I don't know the grpc code, can you point me at something a can look at specifically to understand what this is referring too?

@thomasvl - Thanks for the prompt response. The code block referred above was not grpc code, but some client code manipulating with the generated ObjC proto message objects. Particularly, it seems they are experiencing issue with GPBUnrecognizedEnumeratorValue when putting it in a switch statement in Swift.

When you say things were working, what changed?  Is this a result of using the exact same code built changing Xcode versions (and hence Swift tools)?  Even if you are still set to using Swift 4 (and not 4.2), there could be changes on the Swift side of things.  The bridging of enums from ObjC (or even just C) to Swift has always been somewhat fragile since ObjC/C have open enums and Swift doesn't.
I know where was a proposal out for getting language support for this, but it has gone a few rounds already, but I though I remember seeing something that Apple had punted on the full solution pending that proposals, but they were do something atleast with respect to the System sdks.  Maybe part of that work included changing some of the other behaviors anyways.
Can you make a simple test case using just protos and some code without gRPC so we have a simpler test case to look?
Adding @allevato incase he has ideas or remembers more about the changes to enum bridging.

@thomasvl Thanks for checking it out.

What changed between the working and non-working cases was simply the way the switch statement was written. In the working case, not all switch cases were handled explicitly; instead, some were caught by default. In the non-working one, all are handled explicitly. But from a Swift syntax point of view, they are equivalent, because the default case only catches one unhandled case, namely gpbUnrecognizedEnumeratorValue.
Good point about using Swift 4 and not 4.2. We experienced this in a project with a large code base, and we don't have capacity to convert it yet.
I agree that a simple test case would be useful in identifying whether this is a Swift 4 vs 4.2 issue, among others, but I'm not sure how we would orchestrate one without gRPC, since the issue occurs when one application is compiled against one version of a protobuf message format, and then receives a message in an updated and mismatching format. Could you suggest a method for receiving and unpacking messages without the use of gRPC?

I agree that a simple test case would be useful in identifying whether this is a Swift 4 vs 4.2 issue, among others, but I'm not sure how we would orchestrate one without gRPC, since the issue occurs when one application is compiled against one version of a protobuf message format, and then receives a message in an updated and mismatching format. Could you suggest a method for receiving and unpacking messages without the use of gRPC?

Just make two messages that are identical with an enum filed in each (MessageV1, MessageV2), in the second one, add another value to the enum.  Then you can create a message with V2 (using the new values also), and serialize it, and decode it as V1.

Closing for no activity.


intermittent SIGSEGV segfault when plotting during training_step

🐛 Bug
If I create specific, simple matplotlib plots during training_step then I get a segfault after training for several batches.
Specifically, if I call plot_data() from within training_step, then I get a segfault:
def plot_data():
    """Plot random data."""
    fig, axes = plt.subplots(ncols=4)  # See note 1 below.
    N_PERIODS = 16
    x = pd.date_range(START_DATE, periods=N_PERIODS, freq="30 min")  # See note 2 below.
    y = np.ones(N_PERIODS)
    for ax in axes:
        ax.plot(x, y)
    plt.close(fig)

Interestingly, I've found two ways to stop the segfaults. You can do one or the other (or both) of these actions to stop the segfaults:

Change ncols to one of {2, 3, 5, 6, 7, 9, 10}.  segfaults only seem to appear if ncols is one of {4, 8, 16, 32}
Convert x from a pd.DatetimeIndex to a numpy array of matplotlib dates by doing x = matplotlib.dates.date2num(x) after line 1.

If I run gdb --args python script.py I get this:
Epoch 0:   1%|█▏| 7/1024 [00:01<02:52,  5.90it/s, loss=0.281, v_num=34]
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x000055555568bb4b in _PyObject_GC_UNTRACK_impl (filename=0x55555587fef0
"/home/conda/feedstock_root/build_artifacts/python-split_1643749964416/work/Modules/gcmodule.c", lineno=2236,
op=0x7fffd05f5e00) at /home/conda/feedstock_root/build_artifacts/python-
split_1643749964416/work/Include/internal/pycore_object.h:76
76  /home/conda/feedstock_root/build_artifacts/python-split_1643749964416/work/Include/internal/pycore_object.h:
 No such file or directory.

The minimal code example below seems to always crash whilst training on epoch 7.
In my "real" code, the crash was intermittent.
If ncols is large (e.g. 32) then the crash changes randomly between:

SIGSEGV segfault
AttributeError: 'weakref' object has no attribute 'grad_fn'
AttributeError: 'builtin_function_or_method' object has no attribute 'grad_fn'

The two AttributeErrors originate from the same bit of code: line 213 of torch/optim/optimizer.py: if p.grad.grad_fn is not None:.  My guess is that the SIGSEGV is the root problem, and the AttributeErrors are symptoms of the underlying SIGSEGV error. I have (once) seen the code compain, at the same time, about a SIGSEGV and an AttributeError.
To Reproduce
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import pandas as pd
import numpy as np
import torch
from torch import nn
import torch.nn.functional as F
from torch.utils.data import Dataset, DataLoader
import pytorch_lightning as pl

START_DATE = pd.Timestamp("2020-01-01")
N_EXAMPLES_PER_BATCH = 32
N_FEATURES = 1

class MyDataset(Dataset):
    def __init__(self, n_batches_per_epoch: int):
        self.n_batches_per_epoch = n_batches_per_epoch

    def __len__(self):
        return self.n_batches_per_epoch

    def __getitem__(self, idx: int) -> tuple[torch.Tensor, torch.Tensor]:
        x = torch.rand(N_EXAMPLES_PER_BATCH, N_FEATURES, 1)
        y = torch.rand(N_EXAMPLES_PER_BATCH, 1)
        return x, y

class LitNeuralNetwork(pl.LightningModule):
    def __init__(self):
        super().__init__()
        self.flatten = nn.Flatten()
        self.nn = nn.Linear(in_features=N_FEATURES, out_features=1)

    def forward(self, x):
        x = self.flatten(x)
        return self.nn(x)

    def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.mse_loss(y_hat, y)
        plot_data()
        return loss

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=0.001)

def plot_data():
    """Plot random data."""
    # ncols needs to be 4 or higher to trigger a segfault.
    fig, axes = plt.subplots(ncols=4)
    N_PERIODS = 16
    x = pd.date_range(START_DATE, periods=N_PERIODS, freq="30 min")
    # The segfaults go away if I do:
    # x = mdates.date2num(x)
    y = np.ones(N_PERIODS)
    for ax in axes:
        ax.plot(x, y)
    plt.close(fig)

dataloader = DataLoader(
    MyDataset(n_batches_per_epoch=1024),
    batch_size=None,
    num_workers=2,
)

model = LitNeuralNetwork()
trainer = pl.Trainer()
trainer.fit(model=model, train_dataloader=dataloader)

Expected behavior
No segfault :slightly_smiling_face:
Environment
* CUDA:
	- GPU:
	- available:         False
	- version:           None
* Packages:
	- numpy:             1.22.2
	- pyTorch_debug:     False
	- pyTorch_version:   1.10.2
	- pytorch-lightning: 1.5.10
	- tqdm:              4.62.3
* System:
	- OS:                Linux
	- architecture:
		- 64bit
		- ELF
	- processor:         x86_64
	- python:            3.9.10
	- version:           #31-Ubuntu SMP Thu Jan 13 17:41:06 UTC 2022

Here's my environment.yml file:
name: segfault
channels:
  - conda-forge
  - pytorch
dependencies:
  - matplotlib
  - pandas
  - pytorch
  - cpuonly
  - pytorch-lightning

Additional context
This may be related to issue #11067.
I have tried and failed to reproduce this problem in two simpler scripts:

A script which just uses matplotlib and pandas (no pytorch.  no pytorch-lightning)
A script which just uses matplotlib, pandas and pytorch (no pytorch-lightning).

These scripts are in a tiny GitHub repo for this bug report: https://github.com/JackKelly/pytorch-lighting-segfault

@JackKelly Thanks for opening this issue and for the reproducibility examples. I'll have a look later today and report back if I find something :)

Hi again,
I ran your code and for me, neither the pure pytorch, nor the lightning version segfaulted (or had any issue at all).

The two attribute errors you mentioned seem to indicate that somewhere torch.Tensor are swapped out with proxies or builtin functions, for which I don't have any explanation.
I am very sorry, but with your exact environment, the code seems to run fine for me so I am a bit clueless on how to investigate this :/

Thanks loads for looking into this so quickly!
Hmm, that is strange!
Would it be possible to try increasing ncols to 32 (on the line fig, axes = plt.subplots(ncols=4)) to see if that forces a segfault?
I've seen segfaults with this code on my local workstation and on GCP VMs.  (All were running very similar Ubuntu 21.10 environments though)
Let me try again with a fresh VM on GCP...

Hmm, interesting, after installing a fresh VM, now I'm seeing the same as you! Here are the steps I took to create a fresh VM... Let me have a tinker to see if I can figure out what's going on!

Create a new GCP VM instance
Use machine config e2-standard-4 (4 vCPU, 16 GB memory).  No GPU.
Change the boot disk to Ubuntu 21.10
Save and start the VM
SSH into the new VM
sudo apt update && sudo apt -y upgrade
install miniconda
log out and log back in again
git clone https://github.com/JackKelly/pytorch-lighting-segfault.git
cd pytorch-lightning-segfault
conda env create -f environment.yml
conda activate segfault
python pytorch-lightning.py

oops, sorry, I'm an idiot!
I had forgotten to comment out x = mdates.date2num(x). I have now commented out x = mdates.date2num(x) and pushed to the repo.
Please pull from the repo again and try running python pytorch-lightning.py.
It segfaults for me on the fresh VM (described in the comment above) and on my local workstation.

Hi, Thanks for the update. Now, I can indeed reproduce this.
I changed your script to the following (which is functionally equivalent):
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import pandas as pd
import numpy as np
import torch
from torch import nn
import torch.nn.functional as F
from torch.utils.data import Dataset, DataLoader
import pytorch_lightning as pl

START_DATE = pd.Timestamp("2020-01-01")
N_EXAMPLES_PER_BATCH = 32
N_FEATURES = 1

class MyDataset(Dataset):
    def __init__(self, n_batches_per_epoch: int):
        self.n_batches_per_epoch = n_batches_per_epoch

    def __len__(self):
        return self.n_batches_per_epoch

    def __getitem__(self, idx: int) -> tuple[torch.Tensor, torch.Tensor]:
        x = torch.rand(N_EXAMPLES_PER_BATCH, N_FEATURES, 1)
        y = torch.rand(N_EXAMPLES_PER_BATCH, 1)
        return x, y

class LitNeuralNetwork(pl.LightningModule):
    def __init__(self):
        super().__init__()
        self.flatten = nn.Flatten()
        self.nn = nn.Linear(in_features=N_FEATURES, out_features=1)

    def forward(self, x):
        x = self.flatten(x)
        return self.nn(x)

    def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.mse_loss(y_hat, y)
        from copy import deepcopy
        old_params = deepcopy(list(self.parameters()))
        self.plot_data()
        return loss

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=0.001)

    def plot_data(self):
        """Plot random data."""
        # ncols which cause segfaults: 4, 8, 16, 32
        # ncols which don't appear to cause segfaults: 2, 3, 5, 6, 7, 9, 10
        from copy import deepcopy
        old_params = deepcopy(list(self.parameters()))
        fig, axes = plt.subplots(ncols=32)
        N_PERIODS = 16
        x = pd.date_range(START_DATE, periods=N_PERIODS, freq="30 min")
        # The segfaults go away if I do:
        # x = mdates.date2num(x)
        y = np.ones(N_PERIODS)
        for ax in axes:
            ax.plot(x, y)
        plt.close(fig)

dataloader = DataLoader(
    MyDataset(n_batches_per_epoch=1024),
    batch_size=None,
    num_workers=2,
)

model = LitNeuralNetwork()
trainer = pl.Trainer()
trainer.fit(model=model, train_dataloader=dataloader)

and when setting breakpoints with a condition of
any(p.grad is not None and not isinstance(p.grad, torch.Tensor) for p in self.parameters())

I can see, that if it happens, it is after the plt.subplots command. Comparing the grad attribute before and after this command gives:
>> for p in old_params:
>>     print(p.grad)
None
None
>> for p in self.parameters():
>>     print(p.grad)
<built-in method pop of dict object at 0x7f44eca02540>
<weakref at 0x7f44d5510f90; to 'CompositeGenericTransform' at 0x7f44ec9e6cd0>

I wasn't able to really debug this further, because it quickly dispatches to the matplotlib backend which is dynamically imported. You could try to simply switch the backend there, which I couldn't try (since I'm only having a remote machine without X-Server.
I strongly think that this is a bug in matplotlib, since they also dynamically pass over all the locals when instantiating their figure_manager.
Beyond that, I am sorry that I cannot help further.

Cool, thank you so much for looking into this, and for doing some debugging!
Should I report this bug to the matplotlib folks? (If I do that, please may I quote your debugging results?)

Yes, I think that would be best as I don't see any reason why this could be caused by us :)
Sure, feel free to quote them :)

Cool, thank you. I'll close this issue for now. I've started a new issue in the matplotlib issue queue: https://github.com/matplotlib/matplotlib/issues/22478
Thanks again for your help!

https://github.com/matplotlib/matplotlib/issues/22478#issuecomment-1043069215
@justusschock Can you check the types of the objects in self.paramaters()?  I am assuming that somewhere in the pytorch/lightning side there are c-pointers to Python objects being held.  If that is true, I suspect there is a bug in your reference counting and somehow resurrected or null objects are being found.  Quoting my self from the mpl issue:

This is weird enough that my guess is that under the hood pytorch/pytoch-lightning is keeping pointers to python objects without properly reference counting them, the objects are getting cleaned up, and then because Python uses an object pool that object is getting resurrected as a new object. I do not have a clear explanation of how the grad is getting miss-mapped to some other attribute (but my guess is that Paramaters is slotted so there is a index-based jump-table in there someplace). This would certainly explain the segfaults in the case when the object did not get resurrected and now instead of grabbing the wrong attribute it is looking at a null pointer.

I would also add I do not see any mechanisim by which Matplotlib would even be aware of the objects in self.parameters(), let alone mutate them.

Hi, I just checked the issue.
thanks for the pointers @tacaswell .
We are indeed doing some pointer mechanics under the hood, but nothing of that is in any sort related to gradient tracking or anything similar (and it is all in Python). Those are all handled by PyTorch directly. So from my understanding it is definitely possible/likely that PyTorch does have some C-/C++-Style Pointers there.
As I really do not have any idea on how to further debug this, I'd actually suggest to ask the torch guys directly. We are never touching the gradients explicitly except through pre-defined torch functions. I am really sorry, that I have to keep the hot-potato-game up and running, but I think this is your best chance to get the issue resolved. At least I am very clueless about what is going on here.
@tacaswell Yes I agree, that matplotlib wouldn't know about the parameters, but my initial guess was, that there might be a collision from one of the matplotlib backends (which might also include some C++ / C code) that might have a collision with the PyTorch C++ part. I am not that familiar with the matplotlib mechanics under the hood though so this is a real wild guess and I am sorry if it actually does not make sense at all.

I think looking at the id(p) and type(p) of the objects in self.parameters() would be useful information to take to torch.  If you run that debugging code mulitple times do you always get the same result (pop and a weakref to a transfrom)?  Is the dict that pop belongs to the p or has a bound method been stashed on an object?
I also suggest replacing the mpl calls with (a couple of ) calls to gc.collect() to see if that will force the issue.
While I can not rule out reference counting bugs on the mpl side, I would be very surprised if we were grabbing someone else's objects and adding things to them (or transmute it to a dict?).

@tacaswell yes, they are always the same result (pop and weakref-wise). The only difference is, if this will result in an AttributeError or a SIGSEGV as @JackKelly pointed out.

Huh, I'm experiencing this error now locally trying to run the PyCharm debugger for a test we have with a specific set of breakpoints enabled... 😕
The test is:
https://github.com/PyTorchLightning/pytorch-lightning/blob/550d3a640d1bfeb731a16b2996d3160f0f7eb071/tests/trainer/test_trainer.py#L1300-L1318
And my environment:
* CUDA:
        - GPU:
        - available:         False
        - version:           None
* Packages:
        - numpy:             1.21.0
        - pyTorch_debug:     False
        - pyTorch_version:   1.10.1
        - pytorch-lightning: 1.6.0dev
        - tqdm:              4.62.3
* System:
        - OS:                Darwin
        - architecture:
                - 64bit
                -
        - processor:         i386
        - python:            3.9.1
        - version:           Darwin Kernel Version 19.6.0: Tue Oct 12 18:34:05 PDT 2021; root:xnu-6153.141.43~1/RELEASE_X86_64

I don't think this is a problem with Lightning. But I can see we both have the same PyTorch and Python version.

Yes. The report at https://github.com/PyTorchLightning/pytorch-lightning/issues/11067 used Python 3.6 but also PyTorch 1.10 so that seems to be the common element atm

It might also be worth manually turning off cyclic gc.  It will burn a bunch of memory, but if the issue is something holding a pointer to a gc'd object it would "fix" the problem.

@JackKelly I encountering a similar segfault when trying to log an image with tensorboard inside the training step (i.e., I use  self.logger.experiment.add_image inside the training step) .
One thing that solves the segfault is adding a plt.pause(0.1) right after I finish plotting or adding the image:
import matplotlib.pyplot as plt
plt.pause(0.1)

Since I only log the images a few times during the epoch, adding a pause does not impact the performance of my training code.


Please update the start command for example

Expected Behavior

The websocket server will started by running command java -cp build/examples:dist/WebSocket.jar ChatServer. where is command from
Current Behavior

Command result is Error: Could not find or load main class ChatServer
Possible Solution

The correct command should be java -cp build/examples:dist/java_websocket.jar ChatServer in current version build
Steps to Reproduce (for bugs)

nope
Debug log (for bugs)

Context

Your Environment

Version used:1.3.8
Java version:1.8.0
Operating System and version: mac OS 11.13.2
Endpoint Name and version:
Link to your project:

Hello @marsend,
thank you for your bug report!
Do you mind open a PR to fix this issue?
Love to give credit where credit is due!
Best regards,
marci4

Hello @marsend,
I changed everything!
Thank you again!
Best regards,
marci4


[Doris on es] Support compound_and predicate push down to Elasticsearch

Relate Issue: https://github.com/apache/incubator-doris/issues/3248
SQL:
select * from test where (k2 = 6 and k3 = 1) or (k2 = 2 and k3 =3 and k4 = 'beijing');

Output filter:
((#k2:[6 TO 6] #k3:[1 TO 1]) (#(#k2:[2 TO 2] #k3:[3 TO 3]) #k4:beijing))~1

SQL:
select * from test where (k2 = 6 or k3 = 7) or (k2 = 2 and k3 =3 and (k4 = 'beijing' or k4 = 'zhaochun'));

Output filter:
(k2:[6 TO 6] k3:[7 TO 7] (#(#k2:[2 TO 2] #k3:[3 TO 3]) #((k4:beijing k4:zhaochun)~1)))~1

SQL:
select * from test where (k2 = 6 or k3 = 7) or (k2 = 2 and abs(k3) =3 and (k4 = 'beijing' or k4 = 'zhaochun'));

Output filter (abs can not be pushed down to es, so doris on es would not process this scenario ):
match_all

@imay OK. please check again.

https://github.com/apache/incubator-doris/issues/3249


Unify auth

Contributor offers to license certain software (a “Contribution” or multiple
“Contributions”) to sewer, and sewer agrees to accept said Contributions,
under the terms of the MIT License.
Contributor understands and agrees that sewer shall have the irrevocable and perpetual right to make
and distribute copies of any Contribution, as well as to create and distribute collective works and
derivative works of any Contribution, under the MIT License.
What(What have you changed?)
Enough of the refactoring of Auth providers that I want to get it out in the open.  High points:

future is BaseAuth and sewer.providers; dns_providers is legacy; http_providers is toast
very simple http_dummy provider for testing and to illustrate new interface
client uses the batch interface (clear_cert) but still uses setup_auth to do them one at a time.  What can I say, I forgot to get back to that.

Of course there's more to do, but this felt like a good place to checkpoint.  Except I might push that change to make client use setup_cert later...
Why(Why did you change it?)
Because it was there.  :-)

 Here is an overview of what got changed by this pull request:

Issues
======
+ Solved 1
- Added 4

Complexity increasing per file
==============================
- sewer/auth.py  4
- sewer/providers/http_dummy.py  2
- sewer/providers/tests/test_dummy.py  2

Clones removed
==============
+ sewer/auth.py  -1
+ sewer/client.py  -1

See the complete overview on Codacy

 Here is an overview of what got changed by this pull request:

Issues
======
+ Solved 1

Complexity increasing per file
==============================
- sewer/providers/http_dummy.py  2
- sewer/providers/tests/test_dummy.py  2
- sewer/auth.py  4

Clones removed
==============
+ sewer/auth.py  -1
+ sewer/client.py  -1

See the complete overview on Codacy

cc @donspaulding @AlecTroemel what do you think?

This PR is dead - as a PR, certainly much of it has been recycled, not only the logging stuff.
@AlecTroemel, I would greatly value your thoughts on #178.  One part of that which is NOT final (and I may make changes this evening, if I can decide what exactly to change) is the handling of the "errata lists" returned by ProviderBase methods.  And some new tests, and at least one example http-01 provider using the new interface, and doc updates, and...
@komuw I just assume you've got an eye on things.  :-)


Update layer1 to load iframe, and communicate with it

Should create an iframe element, append it to the page, and make sure it is not visible.
Should use the layer0 SDK to interact with the iframe.
Should emit events

@rishabhgrg Closing this


Execution failed for task ':app:checkDebugAarMetadata'.

* What went wrong:
Execution failed for task ':app:checkDebugAarMetadata'.
> Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
   > Could not find com.github.PayBox:kotlin-paybox-sdk:0.9.10.
     Searched in the following locations:
       - https://dl.google.com/dl/android/maven2/com/github/PayBox/kotlin-paybox-sdk/0.9.10/kotlin-paybox-sdk-0.9.10.pom
       - https://repo.maven.apache.org/maven2/com/github/PayBox/kotlin-paybox-sdk/0.9.10/kotlin-paybox-sdk-0.9.10.pom
       - https://maven.aliyun.com/repository/jcenter/com/github/PayBox/kotlin-paybox-sdk/0.9.10/kotlin-paybox-sdk-0.9.10.pom
       - https://plugins.gradle.org/m2/com/github/PayBox/kotlin-paybox-sdk/0.9.10/kotlin-paybox-sdk-0.9.10.pom
       - https://jcenter.bintray.com/com/github/PayBox/kotlin-paybox-sdk/0.9.10/kotlin-paybox-sdk-0.9.10.pom
       - https://storage.googleapis.com/download.flutter.io/com/github/PayBox/kotlin-paybox-sdk/0.9.10/kotlin-paybox-sdk-0.9.10.pom
     Required by:
         project :app
   > Could not resolve com.facebook.android:facebook-android-sdk:12.+.
     Required by:
         project :app > project :facebook_app_events
      > Failed to list versions for com.facebook.android:facebook-android-sdk.
         > Unable to load Maven meta-data from https://jcenter.bintray.com/com/facebook/android/facebook-android-sdk/maven-metadata.xml.
            > Could not HEAD 'https://jcenter.bintray.com/com/facebook/android/facebook-android-sdk/maven-metadata.xml'.
               > Read timed out```

Hello @Isko21. Since this is an issue with jcenter itself (see https://github.com/flutter/flutter/issues/114343#issuecomment-1297094212), and there isn't anything Flutter can do about it, I am closing this issue. You can fix this in Flutter by using mavenCentral instead. As for plugins still using jcenter, please raise an issue on those plugins' repos. Thank you

jcenter seems to be permanently down now. Ref: https://stackoverflow.com/questions/74258160/is-jcenter-down-permanently-31-oct


Project is not building with Bazel 1.0.1

Any plans to make exodus Bazel 1.0+ compatible?
Trying to build the source with Bazel 1.0.1 using bazel build //migrator/wix-bazel-migrator:migrator_cli_deploy.jar
ERROR: Analysis of target '//migrator/wix-bazel-migrator:migrator_cli_deploy.jar' failed; build aborted: Analysis of target '@scalac_rules_commons_io//:scalac_rules_commons_io' failed; build aborted

Thanks @D0rmouse!
rules_scala version needs to be updated.
https://github.com/wix/exodus/blob/master/WORKSPACE#L25
any chance you can try to update it and report back?
seems like you can just take the head commit:
https://github.com/bazelbuild/rules_scala#bazel-compatible-versions

Thanks for the advice :)
HEAD didn't work, but I updated to and now it builds.
rules_scala_version="0f89c210ade8f4320017daf718a61de3c1ac4773" # update this as needed
rules_scala_version_sha256="37eb013ea3e6a940da70df43fe2dd6f423d1ac0849042aa586f9ac157321018d"

Great!
If you could be so kind to open a PR with your change, that would be amazing :)

I would like to and I tried, but I am unable to push my branch to remote:
$ git push -u origin feature/update_rules_scala_version
remote: Permission to wix/exodus.git denied to D0rmouse.
fatal: unable to access 'https://github.com/wix/exodus.git/': The requested URL returned error: 403

Cheers, PR raised.

Thank you! merged.
Can I close this issue?

Done!


dep: Use released versions of libraries

Using the released versions of the apps. To get us up to date.
This also uncovered some problems with the OpenAPI validation. Fixed in https://github.com/giantswarm/apiextensions/pull/364.

Tested on geckon. Vendor PR going with a ping.


[Enhancement] Allow username mentions but not of other types

Checks

[X] I confirm that I have searched for existing issues / pull requests before reporting to avoid duplicate reporting.
[X] I confirm that I noted that if I don't follow the instructions, the issue may be closed directly.

Description
This is the only chat discord mod with a pinging feature.
However you can bypass the system and @everyone or any role you'd like seeing as you can't limit webhooks.
It'd be nice if you could make it so from in-game it's only possible to ping users but not roles and @everyone, etc.

I think seperating which mention can be pinged is a good idea too! So we can choose which type of ping the players in game can be used, while not removing the @everyone and @here ping.

"allowedMentions": true

is changed to
"allowedMentions": [
  "everyone",
  "users",
  "roles"
]

everyone is for @everyone and @here, according to docs of Discord.
Leave the list empty to disallow all mentions.


Exercise stubs

Many exercises don't have stubs.  For example Gigasecond is an empty file, but it should have a stub defined like:
// gigasecond.php
<?php

function from(DateTimeImmutable $date) {
    throw new Exception('Not implemented');
}

This will lead to a better experience using the website code editor and also help to define the sandbox of solutions for students.

Is that something where I can through the exercises and check? Here we used a BadFunctionCallException
<?php

function xyz(int $number): string {
  throw new \BadFunctionCallException("Implement the xyz function");
}

So can we agree on using this exception throughout all the stubs?

In exercises that are just functions, yes, I think that exception is the proper one to use (PHP doesn't have a native NotImplementedException and I would like to keep exercises simple).  For exercises which require a class (I'm not sure which ones off-hand) it should probably be something like:
<?php

class ExampleClass
{
    public function doSomething(): int
    {
        throw new \BadMethodCallException("Implement the doSomething method");
    }
}

Fulfilled by #357


Update compress.ts

Add support for font/woff2 compression. I will test this implementation.
🔗 Linked issue
None
❓ Type of change

[ ] 📖 Documentation (updates to the documentation, readme, or JSdoc annotations)
[ ] 🐞 Bug fix (a non-breaking change that fixes an issue)
[x] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
Sending a MonaSans(137k) / HubotSans(170k) file over to the client is quite expense no matter how you want to put it, we're expecting a 50-60% reduction in that size. Their OTF variants (currently supported by Nitro plugins) are out of the question with their more than double size.
📝 Checklist

[ ] I have linked an issue or discussion.
[ ] I have updated the documentation accordingly.

WOFF and WOFF2 are already compressed using Brotli. Compressing them again is not a good idea IMO

So where should the woff files be located to get into compression pipeline?

@thednp Nowhere, they are compressed by default already. No compression (again) needed as it won't save any bytes

@manniL sorry to bother you guys, but I did some digging and found some interesting things you might wanna have a look at.
I modified the file at<EMAIL_ADDRESS>and added   "font/woff",  "font/woff2", and found the following:

now my font IS included into the build compression pipeline
the br file is exactly the same size as the original (137Kb), the gz file is 0.1Kb more for some strange reason
see screenshot below

May I suggest that you download a Woff2 file and make some tests yourselves, perhaps there's some bug you don't know yet about.
Thanks


:bug: Fix: deps

Description
Concise description of proposed changes
Testing
Explain the quality checks that have been done on the code changes
Additional Information

[ ] I read the contributing docs (if this is your first contribution)

Your ENS/address:

Current dependencies on/for this PR:

main

PR #746   👈

This stack of pull requests is managed by Graphite.


Scripted fields incompatible with Elasticsearch/Kibana v8

We're on version 7.x and the upgrade assistant gives the following output:

You have 1 index patterns (cf-http-*...) that use scripted fields. Scripted fields are deprecated and will be removed in future. Use runtime fields instead.

Issues reported by the upgrade assistant must fixed before upgrading to v8. Seems like multiple fields are scripted. That logic will likely need to be moved from scripted fields to either runtime fields or the ingest pipeline.
Reference:
https://www.elastic.co/de/blog/introducing-elasticsearch-runtime-fields
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/runtime.html

I have managed to "migrate" this to runtime fields within index-template some time ago that I plan to start using soon - I decided to use https://github.com/elastic/terraform-provider-elasticstack for that. Once I find some time I could share those snippets, if you like. I do remember there were some challenges with reformatting everything so that you can push the json properly but still maintain some readability of .tf files.


Check point file

Hi,  thanks again for this great work!
May I ask how do you generate the checkpoint file (.pth)? I'm trying to train the model with my own dataset, the training process went well. But in the testing process, I'm confused about whether to use the pre-trained model you provided or if I need to generate my own checkpoint file?
Thank you!

You need to use your own checkpoint if you train the model by yourself. The model should be saved by default in "./work_dirs/"

Thanks for your reply!
I tried to load the pre-trained model in the training process but after training in the "./work_dirs/" there is just a log and a .py file, so I don't have a new check point file generated:

Is it because I loaded the pre-trained model?

How fast is your training progress? Sometimes when you load a pre-trained model, it would store the number of iterations. If your config is 240k iters and the pre-trained model has been trained for 240k iters, then it would not be trained at all. In that case, you should increase your config to 360k, for example, so the model can be trained for another 120k iters.

I think it's 160k as a default, cause I didn't change the file '/configs/base/schedule/schedule_160k.py' for training, in the training process it did have a training record like this:

Should I increase the iterations in this case?

In this case, you don't need to increase the iterations. How long have you been training? The checkpoint is generated every few iterations. You can only see the py file and log file in the beginning.

Thank you so much, now I understand it is because the training time is not enough since the google colab cut the connection if there is no interaction in 90 minutes.

Hai, Updated Ganav run error. how to rectify this?

It has not problems on my end, can you make sure you have everything updated?
I suggest that you check the config file and make sure that the in_channels for decoder_head is an array. In this case I think the in_channels variable is an integer, which caused the issue. Make sure you use the latest ganav_rellis_6.py for config, and the corresponding pth file on the google drive for the checkpoint.

Thank you, how to run ganav_rellis.py which is transnet based segmentation...

Thanks in advance...I want to run latest ganav_rellis.py which is present in legacy/rellis_group6 folder. for this which one is corresponding pth file and I am getting same error i have posted earlier...

I think those pth are still accessible in the google drive under legacy_models/. The current version might not run those checkpoints. If you are still interested, please go back to previous commit (before July 25, 2022).
Those files are in the legacy folder and are not the latest ones with the best performance. If you are aiming for the best performance and efficient running time, please use the model that are not under legacy folder.


KAFKA-8580: Compute RocksDB metrics

A metric recorder runs in it own thread and regularly records RocksDB metrics from
RocksDB's statistics. For segmented state stores the metrics are aggregated over the
segments.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Call for review @guozhangwang @ableegoldman @mjsax @vvcephei @bbejeck @abbccdda

For JDK 11/Scala 2.12 the following tests failed:
kafka.api.PlaintextConsumerTest.testLowMaxFetchSizeForRequestAndPartition
kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl

The other two builds exceeded the timeout.
Retest this, please

In JDK8/Scala 2.11 the following tests failed:
kafka.network.DynamicConnectionQuotaTest.testDynamicConnectionQuota

Retest this, please

Merged to trunk, thanks @cadonna ! Please make sure we have another PR for docs change as well :)


GI3

how much they changed?
may i know some details?
3.6 global-metadata.dat https://mega.nz/file/VD9zTLbC#gAPyVboajcydOVzRNwoOfLELJ_2pV865ooz_8Yfk60o

Greetings,
Since version 3.2 and newer, global-metadata.dat has been changed to MHY\x0 structure, were static dumping got broken.


L0 Masthead load lag

Detailed description

The L0 masthead loads a few seconds after home page loads

Is this a feature request (new component, new icon), a bug, or a general
Bug

Is this issue related to a specific component? Masthead L0

What did you expect to happen? What happened instead? What would you like to
see changed?
Masthead should load with homepage

What browser are you working in?  Don't know.  Rooney reported to me.

What version of Carbon for IBM.com are you using?

What offering/product do you work on? Any pressing ship or release dates we
should be aware of?

Steps to reproduce the issue
enter ibm.com into url
.
Additional information
Screen Recording 2020-11-06 at 2.03.11 PM.mov.zip

Forgot to mention this.  We are running a multivariate test with Products and Solutions megamenu.  Not sure if this is impacting.
I have not been able to replicate the issue.  This came from Rooney

Browser: Safari

The resolution is that this is only happening in a preview URL and not production. Closing this.
@ljcarot @RobertaJHahn


For meteor latest version is error with this.each is not a function for tabular.

Meteor version 2.9
Jquery 3.0.0


in_udp: add test of message_length_limit

Which issue(s) this PR fixes:
None.
What this PR does / why we need it:
Add test of message_length_limit for in_udp because there was no test for this option.
Docs Changes:
Not needed.
Release Note:
Not needed.

At least on non-Windows, the data over message_length_limit  are discarded.
https://pubs.opengroup.org/onlinepubs/009695399/functions/recvfrom.html

If a message is too long to fit in the supplied buffer, and MSG_PEEK is not set in the flags argument, the excess bytes shall be discarded.

And there is no error EMSGSIZE.

On Windows, it seems that WSAEMSGSIZE error happens.
https://learn.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-recvfrom

On Windows, sending data over message_length_limit causes an unexpected error and the data is completely lost.
$ bundle exec fluentd -c .\fluent\fluent.conf -v
2023-03-29 23:14:07 +0900 [info]: fluent/log.rb:360:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-03-29 23:14:07 +0900 [info]: fluent/log.rb:360:info: parsing config file is succeeded path=".\\fluent\\fluent.conf"
2023-03-29 23:14:07 +0900 [info]: fluent/log.rb:360:info: gem 'fluentd' version '1.16.0'
2023-03-29 23:14:07 +0900 [debug]: fluent/log.rb:339:debug: No fluent logger for internal event
2023-03-29 23:14:07 +0900 [info]: fluent/log.rb:360:info: using configuration file: <ROOT>
  <source>
    @type udp
    bind "<IP_ADDRESS>"
    tag "test"
    message_length_limit 32
    <parse>
      @type "none"
    </parse>
  </source>
  <match test.**>
    @type stdout
  </match>
</ROOT>
2023-03-29 23:14:07 +0900 [info]: fluent/log.rb:360:info: starting fluentd-1.16.0 pid=4576 ruby="3.2.0"
2023-03-29 23:14:07 +0900 [info]: fluent/log.rb:360:info: spawn command to main:  cmdline=["C:/Ruby32-x64/bin/ruby.exe", "-rC:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/bundler-2.4.8/lib/bundler/setup", "-Eutf-8", "C:/Ruby32-x64/lib/ruby/gems/3.2.0/bin/fluentd", "-c", ".\\fluent\\fluent.conf", "-v", "--under-supervisor"]
2023-03-29 23:14:10 +0900 [info]: #0 fluent/log.rb:360:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2023-03-29 23:14:10 +0900 [info]: fluent/log.rb:360:info: adding match pattern="test.**" type="stdout"
2023-03-29 23:14:10 +0900 [info]: fluent/log.rb:360:info: adding source type="udp"
2023-03-29 23:14:10 +0900 [debug]: #0 fluent/log.rb:339:debug: No fluent logger for internal event
2023-03-29 23:14:10 +0900 [info]: #0 fluent/log.rb:360:info: starting fluentd worker pid=15020 ppid=4576 worker=0
2023-03-29 23:14:10 +0900 [info]: #0 fluent/log.rb:360:info: listening udp socket bind="<IP_ADDRESS>" port=5160
2023-03-29 23:14:10 +0900 [info]: #0 fluent/log.rb:360:info: fluentd worker is now running worker=0
2023-03-29 23:15:19.367809800 +0900 test: {"message":"hoge"}
2023-03-29 23:15:39 +0900 [error]: #0 fluent/log.rb:402:error: unexpected error in processing UDP data error_class=Errno::EMSGSIZE error="A message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram into was smaller than the datagram itself. - recvfrom(2)"
  2023-03-29 23:15:39 +0900 [error]: #0 cool.io/loop.rb:88:run_once: C:/Users/reang/Documents/work/fluentd/fluentd/lib/fluent/plugin_helper/server.rb:558:in `recvfrom'
  2023-03-29 23:15:39 +0900 [error]: #0 cool.io/loop.rb:88:run_once: C:/Users/reang/Documents/work/fluentd/fluentd/lib/fluent/plugin_helper/server.rb:558:in `on_readable_with_sock'
  2023-03-29 23:15:39 +0900 [error]: #0 cool.io/loop.rb:88:run_once: C:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/cool.io-1.7.1/lib/cool.io/io.rb:186:in `on_readable'
  2023-03-29 23:15:39 +0900 [error]: #0 cool.io/loop.rb:88:run_once: C:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/cool.io-1.7.1/lib/cool.io/loop.rb:88:in `run_once'
  2023-03-29 23:15:39 +0900 [error]: #0 cool.io/loop.rb:88:run_once: C:/Ruby32-x64/lib/ruby/gems/3.2.0/gems/cool.io-1.7.1/lib/cool.io/loop.rb:88:in `run'
  2023-03-29 23:15:39 +0900 [error]: #0 cool.io/loop.rb:88:run_once: C:/Users/reang/Documents/work/fluentd/fluentd/lib/fluent/plugin_helper/event_loop.rb:93:in `block in start'
  2023-03-29 23:15:39 +0900 [error]: #0 cool.io/loop.rb:88:run_once: C:/Users/reang/Documents/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'

On Ubuntu focal,  the data over message_length_limit` are discarded without any error.
$ bundle exec fluentd -c ../work/fluent.conf -v
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: parsing config file is succeeded path="../work/fluent.conf"
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: gem 'fluentd' version '1.16.0'
2023-03-29 23:26:20 +0900 [debug]: fluent/log.rb:339:debug: No fluent logger for internal event
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: using configuration file: <ROOT>
  <system>
  </system>
  <source>
    @type udp
    tag "test"
    bind "<IP_ADDRESS>"
    message_length_limit 32
    <parse>
      @type "none"
    </parse>
  </source>
  <match test.**>
    @type stdout
  </match>
  <source>
    @type monitor_agent
  </source>
</ROOT>
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: starting fluentd-1.16.0 pid=982411 ruby="3.2.0"
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: spawn command to main:  cmdline=["/home/daipom/.rbenv/versions/3.2.0/bin/ruby", "-r/home/daipom/.rbenv/versions/3.2.0/lib/ruby/site_ruby/3.2.0/bundler/setup", "-Eascii-8bit:ascii-8bit", "/home/daipom/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/bin/fluentd", "-c", "../work/fluent.conf", "-v", "--under-supervisor"]
2023-03-29 23:26:20 +0900 [info]: #0 fluent/log.rb:360:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: adding match pattern="test.**" type="stdout"
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: adding source type="udp"
2023-03-29 23:26:20 +0900 [info]: fluent/log.rb:360:info: adding source type="monitor_agent"
2023-03-29 23:26:20 +0900 [debug]: #0 fluent/log.rb:339:debug: No fluent logger for internal event
2023-03-29 23:26:20 +0900 [info]: #0 fluent/log.rb:360:info: starting fluentd worker pid=982431 ppid=982411 worker=0
2023-03-29 23:26:20 +0900 [debug]: #0 fluent/log.rb:339:debug: listening monitoring http server on http://<IP_ADDRESS>:24220/api/plugins for worker0
2023-03-29 23:26:20 +0900 [debug]: #0 fluent/log.rb:339:debug: Start async HTTP server listening http://<IP_ADDRESS>:24220
2023-03-29 23:26:20 +0900 [debug]: #0 fluent/log.rb:339:debug:   0.0s: Async::IO::Socket
      | Binding to #<Addrinfo: <IP_ADDRESS>:24220 TCP>
2023-03-29 23:26:20 +0900 [info]: #0 fluent/log.rb:360:info: listening udp socket bind="<IP_ADDRESS>" port=5160
2023-03-29 23:26:20 +0900 [info]: #0 fluent/log.rb:360:info: fluentd worker is now running worker=0
2023-03-29 23:26:47.859800904 +0900 test: {"message":"hoge"}
2023-03-29 23:26:52.054402922 +0900 test: {"message":"jfiaojeofpiajodkfjwoeeifojwiejok"}

$ netcat -u <IP_ADDRESS> 5160
hoge
jfiaojeofpiajodkfjwoeeifojwiejokdjfoewioweifj

If we rescue Errno::EMSGSIZE, then we can ignore incoming data over message_length_limit  on Windows.
However, we can't discard only the part of data that exceeds message_length_limit in the same way as non-Windows.
https://github.com/fluent/fluentd/blob/206b46b91560177366371c1f05bbcf09ac006d86/lib/fluent/plugin_helper/server.rb#L556-L568

We need to figure out what is the right specification.

Could you try with an undocumented parameter receive_buffer_size?
https://github.com/fluent/fluentd/blob/206b46b91560177366371c1f05bbcf09ac006d86/lib/fluent/plugin/in_udp.rb#L46
https://learn.microsoft.com/en-us/windows-hardware/drivers/network/so-rcvbuf
I'm not sure whether it works as expected or not but it can tune the buffer size of a socket's receive buffer.

Oh, thanks, I was not aware of that parameter.
Now I have confirmed that it doesn't work as expected.

Only receive_buffer_size 32

Incoming data is emitted as is without any limitation and any error.

receive_buffer_size 4096 and message_length_limit 32

The same error occurs.

I think what we need to do is the following 2 things:

Fix the unexpected error and the data over message_length_limit should be simply ignored on Windows.
Add documentation that the specifications differ depending on whether it is Windows or not.

https://docs.fluentd.org/plugin-helper-overview/api-plugin-helper-server#server_create-title-port-proto-nil-bind-<IP_ADDRESS>-shared-true-socket-nil-backlog-nil-tls_options-nil-m
This behavior on Windows looks like not being considered so far.
This can happen in in_syslog too.
As for in_syslog, as long as we can avoid unexpected errors, it looks good.
in_syslog should cut off or ignore the data over message_length_limit.

rfc5424: https://www.rfc-editor.org/rfc/rfc5424#section-6.1

The server can cut off or ignore the data.

All transport receiver
implementations SHOULD be able to accept messages of up to and
including 2048 octets in length.  Transport receivers MAY receive
messages larger than 2048 octets in length.  If a transport receiver
receives a message with a length larger than it supports, the
transport receiver SHOULD truncate the payload.  Alternatively, it
MAY discard the message.

rfc3164: https://www.rfc-editor.org/rfc/rfc3164#section-4.1

The data must be 1024 bytes or less.

The total length of
the packet MUST be 1024 bytes or less.

For in_udp, we can apply the same specification.
If we work hard to cut data off instead of just ignoring it for Windows, it would not be very happy.
Cutting data off would usually only cause a failure of parsing and the data would be eventually ignored.

Refs:
https://learn.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2

Return code/value
Description

WSAEMSGSIZE10040
Message too long.  A message sent on a datagram socket was larger than the internal  message buffer or some other network limit, or the buffer used to  receive a datagram was smaller than the datagram itself.

Ruby's implementation:

https://github.com/ruby/ruby/blob/e51014f9c05aa65cbf203442d37fef7c12390015/win32/win32.c#L3540-L3589
https://github.com/ruby/ruby/blob/e51014f9c05aa65cbf203442d37fef7c12390015/win32/win32.c#L3730-L3782

Fix the unexpected error and the data over message_length_limit should be simply ignored on Windows.
Add documentation that the specifications differ depending on whether it is Windows or not.

It looks good, please go a head :+1:

If we work hard to cut data off instead of just ignoring it for Windows, it would not be very happy.
Cutting data off would usually only cause a failure of parsing and the data would be eventually ignored.

User might want to know which kind of data is lost, there is a similar feature request: #3739
You don't need to care such thing in this PR though.

If we work hard to cut data off instead of just ignoring it for Windows, it would not be very happy.
Cutting data off would usually only cause a failure of parsing and the data would be eventually ignored.

User might want to know which kind of data is lost, there is a similar feature request: #3739 You don't need to care such thing in this PR though.

Oh, I see!
I had thought that it might be better to provide an INFO log about ignored data.
I will change the target of this PR to fixing max_bytes parameter of  server helper.
I'll think about logging in the case of  EMSGSIZE .

I had thought that it might be better to provide an INFO log about ignored data.

Oh, now I understand. It is impossible because of

WinSock might return truncated data but discarded by Ruby.

I made another PR to fix the helper.

#4131

When that PR is merged, this PR's tests will succeed on Windows too.

Thanks!


Azure CXP DOC EDIT - Update groups-quickstart-expiration.md

Updated with recent Azure AD UI Screenshots for Microsoft 365 Groups.
MicrosoftDocs/azure-docs/issues/88003

Hi @Givary - Images that are referenced in MD files need to be added to the repository as image files. We can't reference an image file that's hosted somewhere else. (You can't add image files to a PR that's created by using the GitHub UI; you'll need to use content authoring tools like Visual Studio Code and GitBash, for example.) Also, alt text for images needs to follow the guidance that's described in the internal contributor guide for Microsoft Docs content.
Could you change the pull request accordingly?
Thank you!

@curtand This article has https://docs.microsoft.com/en-us/azure/active-directory/enterprise-users/groups-quickstart-expiration old screenshots due to which customer is getting confused and requesting to add content to the screenshot options mentioned in this thread https://github.com/MicrosoftDocs/azure-docs/issues/88003
As per Kristine, there is a different process to update it. Hence assigning  MicrosoftDocs/azure-docs/issues/88003  to make the changes.


In .NET Core projects, VScode should check if the referenced project has been updated

VSCode Version: 1.9
OS Version: Xubuntu 16.04

Steps to Reproduce:

Create two .NET core projects (ie. Console app and .NET Standard library) in separate workspaces
Reference the library project from the console app project
Make change to library (ie. make new class)
The Console app project remembers the previously compiled version and reports errors if I try to reference the new class, until I close and open te workspace

Reading your description it seems that the issue originates from an extension and not VS Code itself. Thereby I will close this issue and kindly ask you to report this issue against the extension repository.
Most extensions are on GitHub as well so it should be easy to file it against the extension. If you open an extension on the market place, click on the Support link to find out more.

Thanks again for your feedback!


generator raised StopIteration

I am coming into a problem when trying to download a specific bundle: thumbnail/costume.
It shows up in sekai.best, but I want a local version.

I am on the latest app hash, app version and sssekai version. My abcache is up to date.

It does not show up in AssetStudio properly, so I assume it really didn't download the entire bundle and is more or less unusable.

https://github.com/mos9527/sssekai/commit/f8598cbe9affb5900dd8f4cdfe9fa4349a0eb3f3 should fix it.

BTW you may want to check out abserve for browsing the asset bundle index.

Github Actions seem to have broken too...this should be also fixed rn 😵💫
Now available on PyPI: https://github.com/mos9527/sssekai/releases/tag/0.5.9

It works! Thank you.
Do you have a Kofi? I'd love to give a few dollars.

It works! Thank you.
Do you have a Kofi? I'd love to give a few dollars.

Thx! But that won't be necessary^^
I'll close this issue for now.


Exception: Chained function entry cannot be changed

Hi, I am trying to use delvewheel with cibuildwheel to build windows wheels for https://github.com/kuelumbus/rdkit_platform_wheels/ . Unfortunately, I get an error (https://github.com/kuelumbus/rdkit_platform_wheels/actions/runs/1195363995)
+ delvewheel repair --add-path C:\libs -w C:\Users\RUNNER~1\AppData\Local\Temp\cibuildwheelm5ku8_7m\repaired_wheel C:\Users\RUNNER~1\AppData\Local\Temp\cibuildwheelm5ku8_7m\built_wheel\rdkit_pypi-2021.3.4-cp38-cp38-win_amd64.whl
  Traceback (most recent call last):
    File "c:\cibw\python\python.3.8.10\tools\lib\runpy.py", line 194, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "c:\cibw\python\python.3.8.10\tools\lib\runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "C:\cibw\python\python.3.8.10\tools\Scripts\delvewheel.exe\__main__.py", line 7, in <module>
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\delvewheel\__main__.py", line 70, in main
      wr.repair(args.target, no_mangles, args.no_mangle_all, args.lib_sdir)
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\delvewheel\wheel_repair.py", line 374, in repair
      discovered, ignored = patch_dll.get_all_needed(extension_module_path, self._add_dlls, self._no_dlls, self._wheel_dirs)[:2]
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\delvewheel\patch_dll.py", line 160, in get_all_needed
      with PEContext(lib_path) as pe:
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\delvewheel\patch_dll.py", line 17, in __init__
      self._pe = pefile.PE(name)
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\pefile.py", line 2742, in __init__
      self.__parse__(name, data, fast_load)
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\pefile.py", line 3147, in __parse__
      self.full_load()
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\pefile.py", line 3258, in full_load
      self.parse_data_directories()
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\pefile.py", line 3553, in parse_data_directories
      value = entry[1](dir_entry.VirtualAddress, dir_entry.Size)
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\pefile.py", line 3646, in parse_exceptions_directory
      rf.unwindinfo.set_chained_function_entry(
    File "c:\cibw\python\python.3.8.10\tools\lib\site-packages\pefile.py", line 1945, in set_chained_function_entry
      raise Exception("Chained function entry cannot be changed")
  Exception: Chained function entry cannot be changed
  repairing C:\Users\RUNNER~1\AppData\Local\Temp\cibuildwheelm5ku8_7m\built_wheel\rdkit_pypi-2021.3.4-cp38-cp38-win_amd64.whl
  finding DLL dependencies

Any idea?
Chris

I encounter the same error. Seems the dependent pefile is throwing an error

Looks like pefile was just updated yesterday. Pinning pefile to 2021.5.24 solved it to me. pip install delvewheel pefile==2021.5.24

It looks like this was reported to pefile at https://github.com/erocarrera/pefile/issues/334 and fixed in version 2021.9.3 of pefile.


Encoder Attribution

At the very least, we should have something like the "Volunteer encoders" list on openings.moe/hub.
Perhaps we should allow for attribution on each video, which would allow us to do things like aggregate counts for leaderboards and such.
For registered users there should be team membership.

In relation to #223, we can perhaps designate members of this team as a circle of trust of encoders. I.e, certain roles granted in this team allow the encoder to bypass restrictions for limiting outstanding submissions or the global feature flag to shut down new submissions.


TypeError: unsupported operand type(s) for |: '_GenericAlias' and 'NoneType'

TypeError: unsupported operand type(s) for |: '_GenericAlias' and 'NoneType'

the same error.
[START] ComfyUI AlekPet Nodes
Node -> ArgosTranslateNode [Loading]
Node -> ExtrasNode [Loading]
Node -> PainterNode [Loading]
Node -> PoseNode [Loading]
Node -> TranslateNode [Loading]
Traceback (most recent call last):
File "D:\ai\ComfyUI\nodes.py", line 1814, in load_custom_node
module_spec.loader.exec_module(module)
File "", line 850, in exec_module
File "", line 228, in call_with_frames_removed
File "D:\ai\ComfyUI\custom_nodes\ComfyUI_Custom_Nodes_AlekPet_init.py", line 223, in
installNodes()
File "D:\ai\ComfyUI\custom_nodes\ComfyUI_Custom_Nodes_AlekPet_init_.py", line 219, in installNodes
addComfyUINodesToMapping(nodeElement)
File "D:\ai\ComfyUI\custom_nodes\ComfyUI_Custom_Nodes_AlekPet_init_.py", line 167, in addComfyUINodesToMapping
spec.loader.exec_module(module)
File "", line 850, in exec_module
File "", line 228, in call_with_frames_removed
File "D:\ai\ComfyUI\custom_nodes\ComfyUI_Custom_Nodes_AlekPet\TranslateNode\translate_node.py", line 4, in
from googletrans import Translator, LANGUAGES
File "C:\Users\Staff\AppData\Local\Programs\Python\Python39\lib\site-packages\googletrans_init.py", line 7, in
from googletrans.client import Translator
File "C:\Users\Staff\AppData\Local\Programs\Python\Python39\lib\site-packages\googletrans\client.py", line 22, in
from googletrans.models import Translated, Detected, TranslatedPart
File "C:\Users\Staff\AppData\Local\Programs\Python\Python39\lib\site-packages\googletrans\models.py", line 25, in
class Translated(Base):
File "C:\Users\Staff\AppData\Local\Programs\Python\Python39\lib\site-packages\googletrans\models.py", line 42, in Translated
parts: List[TranslatedPart] | None,
TypeError: unsupported operand type(s) for |: '_GenericAlias' and 'NoneType'
Cannot import D:\ai\ComfyUI\custom_nodes\ComfyUI_Custom_Nodes_AlekPet module for custom nodes: unsupported operand type(s) for |: '_GenericAlias' and 'NoneType'
Import times for custom nodes:
2.0 seconds (IMPORT FAILED): D:\ai\ComfyUI\custom_nodes\ComfyUI_Custom_Nodes_AlekPet
Starting server
To see the GUI go to: http://<IP_ADDRESS>:8188
Thanks so much in advance!

it's may occur with python==3.8, upgreed python version to 3.10 may fix

OMG, as you said, after changing to python 3.10, the problem was solved, thank you very much.

The topic is closed because the issue has been resolved.


Use Scala 3 artifacts for discipline-scalatest

Should unblock Scala 3 publishing for Scala.js.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@66d65cc). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #1720   +/-   ##
=========================================
  Coverage          ?   86.85%
=========================================
  Files             ?       58
  Lines             ?     2382
  Branches          ?      142
=========================================
  Hits              ?     2069
  Misses            ?      313
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66d65cc...200d26d. Read the comment docs.


Provide a sane error if key and keyFile are provided

Today when key and keyFile are provided, we don't throw an error.  This lead to someone trying to pass a private secret (not a key) into the key field, and getting a nonsense error:
https://github.com/stephenplusplus/google-auto-auth/issues/45#issuecomment-370584488
Lets do better validation!

It actually looks like this is a valid thing to do :|


Routes for the destination are missing in the virtual network gateway

Hi,
We set up a connection monitor for ICMP between two Windows Virtual Machine in different regions. The communication between these regions is established by a Fortigate NVA (FW/VPN) relaying over the On Premise Network. In both Virtual Networks routing tables exist having a default route (<IP_ADDRESS>/0) set to the Fortigate inbound IP (<IP_ADDRESS>).
Name           State   Source  NextHopType                   NextHopIpAddress AddressPrefix
----           -----   ------  -----------                   ---------------- -------------
               Active  Default VnetLocal                     {}               {<IP_ADDRESS>/22}
DefaultRoute   Active  User    VirtualAppliance              {<IP_ADDRESS>}     {<IP_ADDRESS>/0}
PTVNetOverride Active  User    VirtualAppliance              {<IP_ADDRESS>}     {<IP_ADDRESS>/22}
PPVNetOverride Active  User    VirtualAppliance              {<IP_ADDRESS>}     {<IP_ADDRESS>/22}

Communications work fine, as well as the monitoring and alerting. We have however two issues:

In the topology overview an issue occurs stating that Routes for the destination are missing in the virtual network gateway.
When using the trouble shoot feature on the Network Watcher the error returned is: {"origin":"Outbound","severity":"Error","type":"NoRouteLearned","context":[]}
When the Fortigate Firewall goes down for maintenance, the monitoring stops working. The only way to restart monitoring is to reboot the Windows VM, and everything goes back on line.

My questions are, what is the origin of the issue of missing routes. And is the failure of monitoring when communication stops for a while related to this issue, or is there something else wrong. What would be the best way to restart/continue monitoring in these cases?

EffectiveRoutes.txt

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 3bf662d1-53fb-f651-ee67-6e174836d739
Version Independent ID: 67782b9a-1dd5-acb4-6d93-62ee9ff60d43
Content: Tutorial: Monitor network communication between two virtual machines using the Azure portal
Content Source: articles/network-watcher/connection-monitor.md
Service: network-watcher
GitHub Login: @damendo
Microsoft Alias: damendo

@aggerritsen , Thank you for your feedback. However, since this is a product issue which is not related directly to the doc, I would recommend you to create a thread on the forums - Microsoft Q&A or Stack Overflow
Once you post your issue on forums, it will have visibility across the community which is a better suited audience for such types of issues.
We are closing this issue for now. If there are further questions regarding this matter, please reply and we will gladly continue the discussion.


When you change color property and close the color popup, the property is set twice upon close

Godot version:

2af5723
Steps to reproduce:

Add a Node2D
Click to change modulate property
Change it to different color
Press Escape

First set comes from clicking color, two latter sets are from closing the popup.
This was briefly fixed by #44895, but #45042 reintroduced it. This is very low-priority issue (it was there perhaps since last inspector rewrite and no one reported it) and doesn't really affect anything, but it's strange and redundant that the value is set twice.

Ta for the information.
On Thursday, 14 January 2021, Tomek<EMAIL_ADDRESS>wrote:

Godot version:
2af5723
https://github.com/godotengine/godot/commit/2af5723aee3da89fb4a94f01cf0ac2ba9aa06c18
Steps to reproduce:

Add a Node2D
Click to change modulate property
Change it to different color
Press Escape

[image: image]
https://user-images.githubusercontent.com/2223172/104601161-2f083d00-567a-11eb-9060-94e9ee3689e0.png
First set comes from clicking color, two latter sets are from closing the
popup.
This was briefly fixed by #44895
https://github.com/godotengine/godot/pull/44895, but #45042
https://github.com/godotengine/godot/pull/45042 reintroduced it. This
is very low-priority issue (it was there perhaps since last inspector
rewrite and no one reported it) and doesn't really affect anything, but
it's strange and redundant that the value is set twice.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/45186, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALMZECWAFSOWF25BDIA2X5DSZ33WRANCNFSM4WCPSVSQ
.

--
nicole.gmail.org

Ta for the information.
On Thursday, 14 January 2021, Tomek<EMAIL_ADDRESS>wrote:

Godot version:
2af5723
https://github.com/godotengine/godot/commit/2af5723aee3da89fb4a94f01cf0ac2ba9aa06c18
Steps to reproduce:

Add a Node2D
Click to change modulate property
Change it to different color
Press Escape

[image: image]
https://user-images.githubusercontent.com/2223172/104601161-2f083d00-567a-11eb-9060-94e9ee3689e0.png
First set comes from clicking color, two latter sets are from closing the
popup.
This was briefly fixed by #44895
https://github.com/godotengine/godot/pull/44895, but #45042
https://github.com/godotengine/godot/pull/45042 reintroduced it. This
is very low-priority issue (it was there perhaps since last inspector
rewrite and no one reported it) and doesn't really affect anything, but
it's strange and redundant that the value is set twice.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/45186, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALMZECWAFSOWF25BDIA2X5DSZ33WRANCNFSM4WCPSVSQ
.

--
nicole.gmail.org

Stop already.
On Thursday, 14 January 2021, Tomek<EMAIL_ADDRESS>wrote:

Godot version:
2af5723
https://github.com/godotengine/godot/commit/2af5723aee3da89fb4a94f01cf0ac2ba9aa06c18
Steps to reproduce:

Add a Node2D
Click to change modulate property
Change it to different color
Press Escape

[image: image]
https://user-images.githubusercontent.com/2223172/104601161-2f083d00-567a-11eb-9060-94e9ee3689e0.png
First set comes from clicking color, two latter sets are from closing the
popup.
This was briefly fixed by #44895
https://github.com/godotengine/godot/pull/44895, but #45042
https://github.com/godotengine/godot/pull/45042 reintroduced it. This
is very low-priority issue (it was there perhaps since last inspector
rewrite and no one reported it) and doesn't really affect anything, but
it's strange and redundant that the value is set twice.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/45186, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALMZECWAFSOWF25BDIA2X5DSZ33WRANCNFSM4WCPSVSQ
.

--
nicole.gmail.org

Stop already.
On Thursday, 14 January 2021, Tomek<EMAIL_ADDRESS>wrote:

Godot version:
2af5723
https://github.com/godotengine/godot/commit/2af5723aee3da89fb4a94f01cf0ac2ba9aa06c18
Steps to reproduce:

Add a Node2D
Click to change modulate property
Change it to different color
Press Escape

[image: image]
https://user-images.githubusercontent.com/2223172/104601161-2f083d00-567a-11eb-9060-94e9ee3689e0.png
First set comes from clicking color, two latter sets are from closing the
popup.
This was briefly fixed by #44895
https://github.com/godotengine/godot/pull/44895, but #45042
https://github.com/godotengine/godot/pull/45042 reintroduced it. This
is very low-priority issue (it was there perhaps since last inspector
rewrite and no one reported it) and doesn't really affect anything, but
it's strange and redundant that the value is set twice.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/45186, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALMZECWAFSOWF25BDIA2X5DSZ33WRANCNFSM4WCPSVSQ
.

--
nicole.gmail.org

Overtorment is cyber raping me, I helped you, why would you do this?
On Thursday, 14 January 2021, Tomek<EMAIL_ADDRESS>wrote:

Godot version:
2af5723
https://github.com/godotengine/godot/commit/2af5723aee3da89fb4a94f01cf0ac2ba9aa06c18
Steps to reproduce:

Add a Node2D
Click to change modulate property
Change it to different color
Press Escape

[image: image]
https://user-images.githubusercontent.com/2223172/104601161-2f083d00-567a-11eb-9060-94e9ee3689e0.png
First set comes from clicking color, two latter sets are from closing the
popup.
This was briefly fixed by #44895
https://github.com/godotengine/godot/pull/44895, but #45042
https://github.com/godotengine/godot/pull/45042 reintroduced it. This
is very low-priority issue (it was there perhaps since last inspector
rewrite and no one reported it) and doesn't really affect anything, but
it's strange and redundant that the value is set twice.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/45186, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALMZECWAFSOWF25BDIA2X5DSZ33WRANCNFSM4WCPSVSQ
.

--
nicole.gmail.org

Overtorment is cyber raping me, I helped you, why would you do this?
On Thursday, 14 January 2021, Tomek<EMAIL_ADDRESS>wrote:

Godot version:
2af5723
https://github.com/godotengine/godot/commit/2af5723aee3da89fb4a94f01cf0ac2ba9aa06c18
Steps to reproduce:

Add a Node2D
Click to change modulate property
Change it to different color
Press Escape

[image: image]
https://user-images.githubusercontent.com/2223172/104601161-2f083d00-567a-11eb-9060-94e9ee3689e0.png
First set comes from clicking color, two latter sets are from closing the
popup.
This was briefly fixed by #44895
https://github.com/godotengine/godot/pull/44895, but #45042
https://github.com/godotengine/godot/pull/45042 reintroduced it. This
is very low-priority issue (it was there perhaps since last inspector
rewrite and no one reported it) and doesn't really affect anything, but
it's strange and redundant that the value is set twice.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/45186, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALMZECWAFSOWF25BDIA2X5DSZ33WRANCNFSM4WCPSVSQ
.

--
nicole.gmail.org


Make use of the new Discord features

[X] Bump embed limit to 4096 characters
[x] Use localized timestamps

This is now ready for review! That said since iOS doesn't have support for timestamps yet, I'm leaving this PR in a draft state for the time being.

Should we consider having the integer 4096 as a constant?

The thing is some modules needs 4096, some 4093, 4090 or even 4000. I think it would have been worth it from the start, now I doubt discord will ever change it again. I don't believe it is worth changing and retesting everything to use 4096, minus some magic numbers in some cases.

Agreed. These numbers will remain where they are.

It seems like timestamps have been rolled on iOS 🥳
It can take a bit to be available everywhere, but by the time we merge this PR we should be fine.

It seems like timestamps have been rolled on iOS 🥳
It can take a bit to be available everywhere, but by the time we merge this PR we should be fine.


Update implementation to modern API version (Android SDK)

Working on update-modern-api branch.
For now, will require user to disable battery optimisation for continuous background running.

Implemented in commit 031d28dee4a402fd905e18e483f7f0524fd98619


[GLUTEN-3559][VL] Rewrite GlutenInsertSuite test cases with default values

What changes were proposed in this pull request?
Additional support was added in Spark-3.4 for default values in parquet file scans.
https://issues.apache.org/jira/browse/SPARK-39265
While scanning the files if the column with default value does not have any value then reader appends the default value to it. So, even if the column with default value was added later, file scan still provides values for all records (existing as well as new ones).
Velox does not support back filling the existing records while scan. So, if the column with default value was added later then it will provide null as column value for existing records.
This is a behavior difference and not an inconsistent behavior. Users can update the existing data by running DML commands.
This PR, rewrites those testcases with default value in Gluten.
(Fixes:  #3559)
How was this patch tested?
Unit tests are passing

@JkSelf @PHILO-HE Please review!

@Surbhi-Vijay An issue for this can be opened: Velox does not support back filling the existing records while scan

@Surbhi-Vijay, could you please rebase the code and resolve the conflicts? Thanks!

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_4737_time.csv
log/native_master_02_23_2024_9ceade94a_time.csv
difference
percentage

q1
32.21
34.69
2.488
107.73%

q2
24.57
24.33
-0.240
99.02%

q3
37.99
38.86
0.875
102.30%

q4
36.28
36.08
-0.206
99.43%

q5
70.18
72.24
2.067
102.95%

q6
6.59
7.31
0.725
111.01%

q7
86.44
84.62
-1.818
97.90%

q8
87.29
85.39
-1.895
97.83%

q9
119.14
124.53
5.390
104.52%

q10
41.89
44.23
2.337
105.58%

q11
20.82
19.94
-0.877
95.78%

q12
28.08
26.67
-1.408
94.98%

q13
45.53
44.33
-1.208
97.35%

q14
16.69
20.85
4.155
124.89%

q15
28.86
29.04
0.172
100.60%

q16
14.07
14.19
0.113
100.81%

q17
103.57
102.20
-1.364
98.68%

q18
150.41
150.35
-0.056
99.96%

q19
12.57
14.06
1.485
111.81%

q20
28.70
26.19
-2.511
91.25%

q21
226.08
226.58
0.500
100.22%

q22
13.69
13.90
0.214
101.56%

total
1231.64
1240.58
8.938
100.73%

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query
log/native_master_02_23_2024_time.csv
log/native_master_02_23_2024_9ceade94a_time.csv
difference
percentage

q1
33.31
34.69
1.379
104.14%

q2
22.67
24.33
1.663
107.34%

q3
39.20
38.86
-0.343
99.13%

q4
37.79
36.08
-1.713
95.47%

q5
70.31
72.24
1.936
102.75%

q6
7.14
7.31
0.172
102.40%

q7
84.33
84.62
0.287
100.34%

q8
86.39
85.39
-0.997
98.85%

q9
124.14
124.53
0.386
100.31%

q10
42.39
44.23
1.834
104.33%

q11
21.03
19.94
-1.094
94.80%

q12
26.45
26.67
0.227
100.86%

q13
45.22
44.33
-0.892
98.03%

q14
16.73
20.85
4.124
124.66%

q15
27.23
29.04
1.811
106.65%

q16
14.68
14.19
-0.498
96.61%

q17
101.39
102.20
0.814
100.80%

q18
148.35
150.35
2.000
101.35%

q19
13.42
14.06
0.634
104.72%

q20
26.61
26.19
-0.423
98.41%

q21
226.57
226.58
0.006
100.00%

q22
13.69
13.90
0.205
101.50%

total
1229.06
1240.58
11.518
100.94%


Revert "utils/find-overlay-dependencies.sh: Don't lose Foundation's CoreGraph…"

Reverts apple/swift#9665

@swift-ci please test OS X Platform

This might be the cause of linker problems with libFoundationTestsShared.dylib

Please don't revert this. Our Foundation overlay really does depend on CoreGraphics.

…and this script is only run manually, which means reverting it won't actually change anything.

Are you arguing that this will not give us a data point in the problem finding libFoundationTestsShared.dylib?

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 04c9e08e374b2d0c7d1ac2dfcc9ed7b85db65371
Test requested by - @milseman

Irrelevant


Make contract for a mint resolver committment

If bots are going to be used to chain transactions we need to allow commitments to be spend by them.
Maybe a register of the MintResolverRequest box contains the box id of the commitment, so the commitment doesn't need to be spent when creating a mint request?
Commitment should also be redeemable by the buyer if they want to withdraw their commitment. Seems like this will now require a contract

Closed by #7


avoid static imports in Java sample code

in case this code is copied to an IDE the imports are missing and you have a bit of a challenge to find the right import for the static imports (VS Code Java extension does not support it at all at the moment).
Alternatively we could also use the new 'focus' feature for code blocks and include the static import statements as 'unfocused' but have them included in the clipboard after clicking the copy button.

Can this be merged? I had the feeling it's not final, that's why I'm asking.

Let's at least update with the base branch. @agoerler merge or talk? ;)

@agoerler merge or talk? ;)

talk :-)

We could do sth similar to what we've done in the getting started in a Nutshell. There we have the small snippets that is needed and in addition a detail box that can be expanded if you want the full code example incl. imports.


Adding support for constitution

I'm thinking about if it would be possible for users to use a constitution to get the rewards for training.
https://www.anthropic.com/constitutional.pdf
Right now I'm experimenting with either using a zero-shot bart multi-class classifier
from transformers import pipeline
classifier = pipeline("zero-shot-classification",
                      model="facebook/bart-large-mnli")

candidate_labels = ["well-being", "non-judgmental", "empathetic", "tailored", "privacy", "crisis", "ethical"]

classifier(sequence_to_classify, candidate_labels, multi_class=True)

Or using a open domain model.

Here is some example code I made in a branch.
https://github.com/lvwerra/trl/commit/d785c2274f365c041f9653a4364da7ff6060aeba
I would love some feedback and I'm sure there is a better way of doing this. So think about the code as inspiration for how this could be achieved.
One thing I thought about that you could probably also do which is simpler is just use the included sentiment analysis pipeline if you wanted positive or negative texts in a style based on a dataset that available on the hub.

Sounds really cool! Have you been able to test it already? If you have a working example then we can add it as an example! This might also be interesting to @lewtun.

Closing this for now - feel free to reopen if there's an update!


Add style workflow blog

Description
Adds a blog about the Vale style workflow
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

@pajuric Could you provide meta/keywords for this blog please?

@pajuric - I think we're waiting on meta information and meta keywords from you on this one?

@pajuric Added meta/description and changed date to 8/8

@krisfreedain @dtaivpp - This blog is ready to publish.


fix: add steeringWheelStep: 2

This is needed for vehicles with multiple heating levels on the
steering wheel.  This doesn't allow for customization of the level
(it's always on high), but it's better than not being set.
Tested to work on a 2024 Kia EV9, but I am unsure if it'll cause
problems on vehicles without multiple levels.

I see this is draft.  Is it waiting on something?

Nope, I'm willing to try it if you are!


KeyError: "No object named 'DeformableDetr' found in 'META_ARCH' registry!"

when I run the "configs/BoxSup-DeformDETR_L_R50_2x.yaml", I meet this error, how to solve it

solve it. There is sys.path.insert(0, 'third_party/Deformable-DETR') in train_net.py, but at first i did not compile the deformablr detr correctly

Hello, I compiled DeformableDetr according to the installation prompts, but I also encountered a problem like yours. Can you share your solution in detail?


Fix Ace templates link

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

Hello @jasonm23,
I checked the link with the HEAD of master and the links worked fine. Why did you removed the relref shortcode?

I was viewing the docs through GitHub, and the other links were just valid markdown, and worked as expected.
My assumption is that the links should just work, everywhere, via any markdown processor.

The relref shortcode is part of the cross-reference feature. Hugo allows you to completely rewrite the url of a content file.
For example: links would break if you link to post/foo.md manually and somebody rewrites the url in the frontmatter. In this case the content isn't accessible under the manually defined url. relref would take things like this into account.

My assumption is that the links should just work, everywhere, via any markdown processor.

The usage in the documentation is in some places inconsistent. In most places relref is used. If we follow the cited assumption we would have to remove relref everywhere. It would be nice if links would work with other processors -- but at the end it's a tradeoff.

TBH the inconsistency is something that should be fixed.
relref appears to be applicable to internal and external links.  Is that correct?

(And ref for absolute refs!)

TBH the inconsistency is something that should be fixed.

I agree

relref appears to be applicable to internal and external links.  Is that correct?

I've tested if external ressources are ignored. This is not the case. Hugo tries to link a content file based on the scheme that you to passed to relref. "https://google.com/foo/bar.html" in multiple variants always produced a error message that's displayed in the log.

(And ref for absolute refs!)

Correct. ref prepends the base url.

I will look at fixing some of these inconsistencies and submit pull requests for separate sections / documents as appropriate.

Any updates here?

Closing this as stall. Feel free to reopen if their are any updates for a pull request.

@digitalcraftsman As an FYI, all good on POC, including aliases: https://hugodocs.info/templates/alternatives/#ace-templates


getting error of  NameError: name 'GL_V3F' is not defined

I am new to this i search online i didn't afind anything, Thanks for your help

You can see the file example/requirements.txt which specified the version of pyglet. 1.5.28 does work for me :)

You can see the file example/requirements.txt which specified the version of pyglet. 1.5.28 does work for me :)

I pip installed -r requirements.txt but instead of an error nothing pops up, just runs than closes.

You can see the file example/requirements.txt which specified the version of pyglet. 1.5.28 does work for me :)

This worked for me also

If pyglet 2.x adaption starts picking up we'll to make rendering support for that.
pyglet 2.x completely removed OpenGL fixed pipeline, something we currently use. That's why 1.5.x must be used for now.


[3.2.0] Adding document section for analytics migration edge case

Purpose
Fix issue : #2101
This PR will add an important note to resolve the issue when the external IDP using in the setup, it has not been receiving the tenant domain of users to the analytics dashboard after migration without selecting the "Use tenant domain at local subject identifier" in Analytics SP configuration.
Ported from https://github.com/wso2/docs-apim/pull/2100

@RakhithaRR , isn't this fix applicable to master? If yes can you pls send a PR to master as well.

@RakhithaRR , isn't this fix applicable to master? If yes can you pls send a PR to master as well.


Add myself as a maintainer

At the moment I have 7 merged PRs(with one backport):
https://github.com/go-gitea/gitea/pulls?q=is%3Apr+is%3Aclosed+author%3Aaxifive

Codecov Report

Merging #4345 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #4345   +/-   ##
======================================
  Coverage    20.1%   20.1%
======================================
  Files         153     153
  Lines       30705   30705
======================================
  Hits         6174    6174
  Misses      23588   23588
  Partials      943     943

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e8a17e4...e259d8a. Read the comment docs.


creating a script to clean monthly data

And adding cleaned monthly NOAA data cropped to Acadia.

Fixes and closes issue #3

Fixes and closes issue #3


[Qualcomm feedback] Chapter 5.1. Clarify TSM access to memory regions

Reference: link

A TVM and/or TSM
needs to access both types of memory:
◦ Confidential memory - used for TVM code, data
◦ Non-confidential memory - used for communication between TVM and the non-TCB host
software and/or non-TCB IO devices.

The examples below only show why TVMs needs access to both. Why do TSMs? Is that for case (1), where TSM is nesting the TVM at the same privilege level?

right. added a clarification

A TVM and/or TSM needs to access both types of memory:

Confidential memory - used for TVM/TSM code and security-sensitive data;
including state such as 1st-stage, G-stage page tables.
Non-confidential memory - used only for shared data, e.g. communication
between the TVM/TSM and the non-TCB host software and/or non-TCB IO devices.

Closing as addressed in PR #70
cc @ozkoyuncu


DND mode not hiding notifications

The MercuryTrade dnd mode does not hide incomming notifications since <IP_ADDRESS> for me.

fixed in <IP_ADDRESS>


⚠ Jenkins Plugins Artifactory has degraded performance

In 5d8da69, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 2541 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 7d8ff6e after 16 minutes.


How to use this package?

The docs don't go over much of the setup required to use this package. It seems to look for a nuxt.config.js file in fixture by default, but I do not understand why. Wouldn't we want to test our real apps and configs? I tried setting rootDir and that didn't seem to change anything.

We ended up needing to do:
  setupTest({
    browser: true,
    configFile: resolve(__dirname, '../pages/fixtures/nuxt.config.js'),
    rootDir: resolve(__dirname, '../../../'),
    testDir: resolve(__dirname, '../../'),
    config: {
      rootDir: resolve(__dirname, '../../../')
    }
  });

I think this package should be set to use the root nuxt app as the default, so this is not required.

I would completely agree with @rwwagner90. Documentation for setup is almost non-existing, and It's embarrassing to have testing utilities where even the simplest case (from documentation) is not working.

Related #95


Develop

Why the Interaction Engine document is not the lastest with the current asset package?

Drag the LeapCameraRig prefab into your scene: LeapMotion/Core/Prefabs. -- not found
2.Examples 1 through 5 can be played -- already 8 examples
..., etc

Hi @caibf,
Our documentation matches the currently available package on developer.leapmotion.com/unity .
Working in develop is a different story. We're working on improving our documentation system to handle our versioning more gracefully, but at the moment our latest develop work does not reflect what we publically support. The changes in develop are undergoing QA and we don't merge it until it's proven ready and stable.


What about BonusPack support?

Hello,
Where are we supposed to report issues or ask for support about BonusPack?
Now that BonusPack seems supposed to be included in OSMDroid core (a good thing I think) where does the support will be done?
I wonder if Mathieu still answers to questions on his repository.

Great question. I've hashed some things out with @MKergall regarding this. I think the short term goals are to

get Mathieu write access on this repo to make life easier for him. @neilboyd can you help with this?
osmbonuspack primarily focuses on kml, geoson, routing support
osmdroid to start deprecating the older overlay sets
most likely some refactoring in osmdroid to better integrating the classes migrated over from bonuspack.

#252 is still open for a reason. Primarily for number 3 above. I'm not exactly sure what right looks like here. We also now have a few classes with duplicate names in different packages. There's also a lot of house keeping we should do regarding unused interfaces in the main library.
To more directly answer your question, most of the folks on osmdroid also monitor osmbounspack for new issues and whatnot. I think what was agreed to was the items in number 2 above would remain at osmbonuspack and thus issues related to that should be posted over there, everything else presumably over here

I just wanted to chime in and give my opinion. I haven't been very active on osmdroid lately, but have seen a lot of the evolution of osmdroid over time. If I had stayed active, I would have pushed to reduce the feature list of osmdroid. My feeling is that osmdroid should just cover core functionality - a map view, one or two overlays, a demo project and that's it. The original intent of osmdroid is to be a drop-in replacement for google maps (even if it is v1 which is obsolete). I loved that osmbonuspack was separate - it kept a clear division between core functions and what you can "do" with osmdroid. If we think that merging the two projects is necessary then my response to that would be that we need to expose additional interfaces for developers so osmbonuspack can implement what they need. Overlays should be "drop-in", new tile providers should be possible by extending base classes - we should provide all the tools necessary so that projects like osmbonuspack can do what they do without needing to tightly integrate with the core.
Just wanted to give another view. Best of luck and let me know if I can help!

The original intent of osmdroid is to be a drop-in replacement for google maps
But OSMDroidBonusPack adds gaps to OSMDroid to replace GoogleMaps!
From my point of view, without BonusPack, OSMDroid is not really useable and personnally, I would have choosed another solution, maybe MapBox or I don't know what else.
But that's just my point of view.

Hi all,
Here are my views:

About the philosophy/strategy:
As discussed with @spyhunter99, the idea was to move some classes that can be considered as "core features" of a "modern map library" from OBP to osmdroid.
From my point of view, these key classes are: Marker, Polyline and Polygon (along with underlying classes like InfoWindow stuff).
@spyhunter99 also decided to move CacheManager. It makes sense (as implementation requires a close link with osmdroid internals).
3 other classes have also been moved: FolderOverlay, and the MapEventsOverlay/MapEventsReceiver couple. This is more questionable, but why not.

About the practical aspects, to answer to the question from @2ndGAB :
I'm currently waiting for the release of osmdroid including those additions - once the house keeping mentionned  by spyhunter will be done.
Once this osmdroid version is released, I will update OBP, to remove all redundant classes.
Until then, people can still raise issues about those classes in OBP project, I will try to provide support.
But since the copy of sources to osmdroid, I don't perform any change on copied classes anymore. Their source code repository is now osmdroid.

Everything else in OBP will just stay in OBP: routes, POI and geocoding services, markers clustering, KML and GeoJSON. And I will continue to support related issues in OBP project (welcoming help from others!).


[sival, keymgr] State-agnostic chip_sw_key_derivation_test

keymgr_key_derivation_test_fpga_cw310_sival_rom_ext test was marked as broken in #21706. This commit fixes this and addresses #21500.
The source of the bug is due to the nature of the ROM_EXT that runs through a CDI attestation flow advancing the key manager state to "OwnerRootKey" in the process. This was not accounted for in the test, which assumed an active "CreatorRootKey" state (this works for sim_dv tests but not with ROM_EXT). The solution to this issue lies in rendering the test state-agnostic, meaning it will execute its verifications independently of the current key manager state. This approach thus mimics the keymgr_sideload_{aes,kmac,otbn} tests which follow a similar strategy.

Thank you for your reviews @vogelpi and @jwnrt. I split the commit into two parts to make the removal of the unused includes explicit.

Thanks, I'll merge once CI has finished


Unable to perform inference with Prompt Optimization Example

Hello,
I am getting the following error when running the prompt optimization code on my own custom dataset:

It seems that the model is not outputting any response from this call at line 60 from llm_ops.py. I checked that input  and system values are valid:
response_text = self.engine(input_variable.value, system_prompt=system_prompt_value)
I don't think this has anything to do with my dataset or the contents of the prompt (correct me if I'm wrong!). Does anyone know how to resolve this error?  Thanks!

Without more context it is unclear -- For some reason your response variable is getting initialized with a None value. I am not sure why this may be the case -- perhaps debug what's happening at Line 62 in llm ops?

I am facing the same problem. @nikhilk7153 is it solved?

I am facing the same problem. @nikhilk7153 is it solved?

Seems like the following chunk solves the issue:
if isinstance(y, np.int64):
                y = int(y)

Hi, I was wondering where exactly you added this code @Mehrab-Tanjim ?

def eval_sample(item, eval_fn, model):
"""
This function allows us to evaluate if an answer to a question in the prompt is a good answer.
"""
x, y = item
if isinstance(y, np.int64):
    y = int(y)


[BUG] helm chart crds are invalid in chart versions >=1.4.5

Description
When Running the kubectl apply command to install custom resource definitions for spark-operator I get an error that metadata.annotations are too long. It worked in 1.4.4 of the chart but not in newer versions.
The error is:
The CustomResourceDefinition "sparkapplications.sparkoperator.k8s.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

[x] ✋ I have searched the open/closed issues and my issue is not listed.

Reproduction Code [Required]
kubectl apply -f https://raw.githubusercontent.com/kubeflow/spark-operator/spark-operator-chart-1.4.4/charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml
# customresourcedefinition.apiextensions.k8s.io/sparkapplications.sparkoperator.k8s.io created
kubectl apply -f https://raw.githubusercontent.com/kubeflow/spark-operator/spark-operator-chart-1.4.6/charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml
# The CustomResourceDefinition "sparkapplications.sparkoperator.k8s.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

Steps to reproduce the behavior:
Execute the above statements in terminal
Expected behavior
The crds should be installable without an issue
Actual behavior
Installing CRDs fails hard
Environment & Versions

Spark Operator App version: Irrelevant
Helm Chart Version: 1.4.6
Kubernetes Version: 1.29
Apache Spark version: Irrelevant

@dada-engineer  For client-side apply, it will fail since the definition of the whole CRD will be patched as an annotation, which exceeds the size limit. You can install the CRDs with kubectl create -f:
kubectl create -f https://raw.githubusercontent.com/kubeflow/spark-operator/spark-operator-chart-1.4.4/charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml
kubectl create -f https://raw.githubusercontent.com/kubeflow/spark-operator/spark-operator-chart-1.4.6/charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml

Or use server-side apply:
kubectl apply --server-side -f https://raw.githubusercontent.com/kubeflow/spark-operator/spark-operator-chart-1.4.4/charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml
kubectl apply --server-side -f https://raw.githubusercontent.com/kubeflow/spark-operator/spark-operator-chart-1.4.6/charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml

Oh thanks a lot. this makes a lot of sense ❤