Items not processed, when busy processing previous items

Bug

Which library version?

System.Reactive 5.0.0

What are the platform(s), environment(s) and related component version(s)?

.Net 6

What is the use case or problem?

I'm using Rx to process events in groups of maximal X items or after no new events were sent for Y ms.
For this purpose I use the code suggested in this stackoverflow answer.
When I changed the code to wait until all events in the queue are processed, I noticed that some events get stuck in the queue.

What is the expected outcome?

I would expect, that all events added to queue would be processed eventually, even when the queue is currently busy processing.

What is the actual outcome?

When the queue is busy processing, events get "stuck" in the queue. Therefore the events which were sent while processing, will not be processed until a new entry is added to the queue after the previous processing finished.

Do you have a code snippet or project that reproduces the problem?

To reproduce the behavior I wrote some sample code which adds numbers to the subject and then waits until the number is processed or the timeout is hit (10sec).
If the number is processed another number is added.
This second number is sent while the processing of the previous number is still busy, this leads to the described behavior.
This behavior can also be reproduced without any delay, it just won't happen regularly, because the timing has to be just right for it to happen.
private DispatcherScheduler _schedulerProviderDispatcher = new DispatcherScheduler(Application.Current.Dispatcher);
private Subject<int> _numberEvents = new Subject<int>();
private readonly ConcurrentDictionary<int, object> _numbersInProgress = new ConcurrentDictionary<int, object>();
private IDisposable _disposable;

public async Task SendNumbersAsync()
{
    _disposable = _numberEvents.Synchronize()
        .BufferUntilInactive(TimeSpan.FromMilliseconds(2000), _schedulerProviderDispatcher, 100)
        .Subscribe(numbers =>
        {
            if (numbers.Count == 0)
                return;

            var values = string.Empty;

            // Handle numbers in queue and remove them from the dictionary
            for (var i = 0; i < numbers.Count; ++i)
            {
                var number = numbers[i];
                if (_numbersInProgress.TryRemove(number, out _) == false)
                    Trace.WriteLine($"Failed to remove number '{number}'");

                values += $"{number}, ";
                Trace.WriteLine($"Handled Number: {number}, Count: {i + 1}/{numbers.Count}");
            }

            Trace.WriteLine($"Handled '{numbers.Count}' numbers. Values: '{values}'");

            // delay the execution by 1000ms to simulate a slow processing of the numbers
            // and create a timeframe where new numbers will be sent and received, but the subject is still busy with the previous processing
            var task = Task.Delay(1000);
            Task.WaitAll(task);

            Trace.WriteLine($"Finished handling '{numbers.Count}' numbers. Values: '{values}'");
        });

    // push numbers to the subject
    var number = 0;
    while (_numbersInProgress.Count == 0)
    {
        ++number;
        Trace.WriteLine($"Sending number: {number}");
        _numberEvents.OnNext(number);
        Trace.WriteLine($"Finished sending number: {number}");

        // add the number to the progress dictionary
        if (_numbersInProgress.TryAdd(number, 0) == false)
            Trace.WriteLine($"Failed to add number '{number}'");

        var waitCount = 0;
        var timedOut = false;
        // wait for the numbers to be processed
        // if we waited 100 times (10sec) stop
        while (_numbersInProgress.Count != 0)
        {
            Trace.WriteLine($"Waiting ({waitCount})");
            await Task.Delay(100);
            ++waitCount;

            if (waitCount > 100)
            {
                timedOut = true;
                break;
            }
        }

        if (timedOut)
            Trace.WriteLine($"Timeout waiting: {number}");
        else
            Trace.WriteLine($"Finished waiting: {number}");
    }

    // if we still have unprocessed numbers
    // send another number to the subject to trigger processing of both numbers
    if (_numbersInProgress.Count != 0)
    {
        // Failed
        Trace.WriteLine($"Failed waiting: {number}");

        Trace.WriteLine($"Sending number: {9999}");
        _numberEvents.OnNext(9999);
        Trace.WriteLine($"Finished sending number: {9999}");
    }
}

And the extension method, used to group the numbers:
public static IObservable<IList<T>> BufferUntilInactive<T>(this IObservable<T> stream, TimeSpan delay, IScheduler scheduler = null, Int32? maxCount = null)
{
     var s = scheduler ?? Scheduler.Default;

     var publish = stream.Publish(p =>
     {
          var closes = p.Throttle(delay, s);

          if (maxCount != null)
          {
              var overflows = p.Where((x, index) => index + 1 >= maxCount);
              closes = closes.Amb(overflows);
          }

          return p.Window(() => closes).SelectMany(window => window.ToList());
     });

     return publish;
}

Output
When executing this method, the trace outputs the following lines:

number 2 is sent before the processing of the previous sent number 1 is finished

the loop runs in the timeout condition after 100 wait cycles and then sends a new number, which triggeres the processing of both numbers in the queue

Thanks for the reply,
I tried your suggested solution and with a little change in the extension method the error was fixed.
I needed to ajust the maxCount, without the change the maxCount would never be reseted and after it was hit once, every new event would be over the maxCount.
public static IObservable<IList<T>> BufferUntilInactive<T>(this IObservable<T> stream, TimeSpan delay, IScheduler scheduler = null, Int32? maxCount = null)
{
     var s = scheduler ?? Scheduler.Default;

     var publish = stream.Publish(p =>
     {
          var closes = p.Throttle(delay, s);

          if (maxCount != null)
          {
              Int32 i = 0;
              var overflows = p.Where(x =>
              {
                  ++i;

                  if (i >= maxCount)
                  {
                      i = 0;
                      return true;
                  }

                  return false;
              });

              closes = closes.Amb(overflows);
          }

          return p.Window(closes).SelectMany(window => window.ToList());
     });

     return publish;
}

Thanks again for the help and the insight where my wrong understanding of the code happened.

Sorry to bother you again,
but after some more testing with the production code, the same error occured again, although the test code showed no errors.
It seems, that even with the suggested change, there is still a timeframe in which events won't be processed.
The error behavior is exactly the same as before, some events get stuck in the queue and because I am waiting for them to be processed, I am deadlocked.
I used a workaround for the past 4 Months which resolved the problem, but it only seems to work with the previous implementation of the p.Window(() => closes).
By adding a Delay with any TimeSpan the problem stopped occuring.
Can you give me some insight into the impact of the Delay() to this behavior and why it prevents the error to occur?
public static IObservable<IList<T>> BufferUntilInactive<T>(this IObservable<T> stream, TimeSpan delay, IScheduler scheduler = null, Int32? maxCount = null)
{
     var s = scheduler ?? Scheduler.Default;

     var publish = stream.Publish(p =>
     {
          var closes = p.Throttle(delay, s);

          if (maxCount != null)
          {
              var overflows = p.Where((x, index) => index + 1 >= maxCount);
              closes = closes.Amb(overflows);
          }

          return p.Window(() => closes).SelectMany(window => window.ToList()).Delay(TimeSpan.Zero, s);
     });

     return publish;
}

Do you have any other idea why I am still experiencing this behavior, even after changing the code?

Do you have any other idea why I am still experiencing this behavior, even after changing the code?

Well, the answer last time wasn't a bug in Rx, it was a bug in the code you showed me. Now you're telling me that although the test code you've shown me no longer fails (you said "the test code showed no errors,") you still see problems in your production code.
So one possibility is that this still isn't a bug in Rx, and that this time it's a bug in some of the code you've not shown me.
Although it's not out of the question that there could be a bug in Rx, there's nothing at all for me to go on here. If you can't show me code that exhibits the problem, it's likely to be impossible for me to work out why it's happening, because the cause of the problem could be in some code that's only running in your production system.
With the information you've given me, about the only helpful thing I can possibly try to do here is to answer this:

Can you give me some insight into the impact of the Delay() to this behavior and why it prevents the error to occur?

Using this operator will cause everything that emerges from the SelectMany to be deferred through the scheduler before coming out of the IObservable returned by BufferUntilInactive. This can change the exact timings with which things occur, and if you've got race condition bugs, doing something that changes the timings even slightly might mask (or reveal) those bugs. Since the problem you describe has something to do with whether events fall into a narrow timespan, it's not wholly surprising that a change that makes no logical difference to the behaviour but which will likely change the scheduling will disturb the behaviour.
You've re-opened this issue, but you've not provided any further information that might point to a problem in Rx. (And the information you already provided only exhibited problems fully explained by a bug in the test code.) If you can supply example code that exhibits a problem, I can take a look at it, but if not, I'll have to reclose the issue because there's nothing for me to base any further investigation on.

FYI still there in Rx 6.0 on .NET 8

This appears not to be a bug in Window, but a feature of combining Do and SelectMany in this way. Rx has in fact done precisely what you've asked. I know you didn't intend to ask it to ignore that X event, but the way you've written your code means that Rx is positively obliged not to deliver that X event where you're expecting it to go. (It's not actually dropping it though, as I'll demonstrate. It's that you've created a situation where the event is delivered before your code is has asked to receive it.)

the value 'X' is injected during this event, and it escapes observation by the newly created window.

That statement is not quite accurate. The newly created window receives the event and duly distributes it to all of its subscribers. The problem is that nothing has yet subscribed to that window when you call synchronized.OnNext('X');. And that's happened because that's what your code has asked for.
We can make this a bit more obvious by adding some logging to the SelectMany callback:
.SelectMany(window =>
{
    Print("SelectMany received new window");
    return window.ToList();
})

That produces this output:
07:28:53.092 [1] > Window #1 emitted
07:28:53.113 [1] > SelectMany received new window
07:28:53.174 [7] > Produced: A
07:28:53.284 [7] > Consumed: [A]
07:28:53.286 [7] > Window #2 emitted
07:28:53.287 [7] > Produced: X
07:28:53.287 [7] > After OnNext('X') injection
07:28:53.287 [7] > SelectMany received new window
07:28:53.684 [4] > Producer Completed
07:28:53.685 [4] > Consumed: []
07:28:53.686 [4] > Consumer Completed
07:28:53.688 [4] > Results: [A], []

This makes it clear that the SelectMany callback doesn't run until after your Do callback has injected the X value. And since it's that callback that subscribes to window, your code never receives the item.
So it's not that Window is losing the item. It's that you haven't yet subscribed to the window at the moment that you produce that item.
And you should expect this. You've inserted that Do before the SelectMany, so you are asking Rx to invoke your Do callback before it passes each window on to SelectMany. It has done exactly what you asked it to do: it has given you access to the new window before SelectMany got to see it. So you should not be surprise that Rx gives you the window before it has any subscribers. You should thank it for doing precisely what you asked. 😉
We can further demonstrate that Window really isn't losing anything by adding yet another Do:
IList<char>[] results = await subject
    .Do((char x) => Print($"Produced: {x}"), () => Print($"Producer Completed"))
    .Window(subject.Throttle(TimeSpan.FromMilliseconds(100)))
    .Do(w => w.ToList().Subscribe(l => Print($"Pre-Do window subscription received [{string.Join(", ", l)}]")))
    .Do(_ =>
    {
        i++;
        Print($"Window #{i} emitted");
        if (i != 2) return;
        // Inject the value X in the 2nd window only.
        synchronized.OnNext('X');
        Print($"After OnNext('X') injection");
    })
    .SelectMany(window =>
    {
        Print("SelectMany received new window");
        return window.ToList();
    })
    .Do((IList<char> w) => Print($"Consumed: [{String.Join(", ", w)}]"),
        () => Print($"Consumer Completed"))
    .ToArray();

The extra Do appears before before your injecting Do. Rx does things in the order you tell it to (which is why your example doesn't behave how you had hoped, by the way), so this gets to run before your original Do callback. That means this added code will have a chance to subscribe to each new window before your injection occurs. Here's the output:
07:34:34.316 [1] > Window #1 emitted
07:34:34.336 [1] > SelectMany received new window
07:34:34.392 [7] > Produced: A
07:34:34.502 [7] > Pre-Do window subscription received [A]
07:34:34.504 [7] > Consumed: [A]
07:34:34.505 [7] > Window #2 emitted
07:34:34.505 [7] > Produced: X
07:34:34.505 [7] > After OnNext('X') injection
07:34:34.505 [7] > SelectMany received new window
07:34:34.906 [8] > Producer Completed
07:34:34.906 [8] > Pre-Do window subscription received [X]
07:34:34.906 [8] > Consumed: []
07:34:34.907 [8] > Consumer Completed
07:34:34.909 [8] > Results: [A], []

Note in particular, this line:
07:34:34.906 [8] > Pre-Do window subscription received [X]

There's your X. This proves conclusively that the item did not in fact escape observation by Rx. The window captured the event exactly as it is supposed to. It reported it to all the window's active subscribers exactly as it is supposed to. (So the 'bug' here does not exist. The item is not lost.)
It is true that Rx did not report this item to the subscription that hadn't actually been set up yet at the time that the event occurred. That is not a bug. Window is not designed to hold onto items until such time as you're ready to subscribe.
In short, this code has gone out of its way to be able to send an item into the window before that SelectMany had the opportunity to subscribe to the window. Rx has done precisely what the code asked it to do. The effect is that your code doesn't see the X in the output of the SelectMany, but that's because you told Rx to do things in a way that ensured that this would be the case.
Do or do not?
This is a good example of why I try never to use Do. It can seem very convenient but in my experience it is almost always trouble.
The nature of the trouble is that it's typically very hard to reason about what will actually happen. For example, by adding a little bit of extra log output to your code, I hope it is now clear that the reason the X appears to go missing is because Do runs before SelectMany gets to see the window from which that X can emerge. (And this happens because that's what you told Rx to do.) However, I did not find this obvious. I had to think about this for quite a while to understand that this is what was happening.
I suspect I would not have noticed the problem in a code review. (Although if I had been reviewing this code I would have strongly recommended finding a way of not using Do simply because I know from experience that Do is usually a bad idea.) And that's the problem with Do. It makes logic errors in the code very hard to spot.
Why is that? I think it's a combination of two factors. One is that Do is essentially an imperative style of code, whereas Rx (and LINQ in general) was designed to be used in a functional style. Do provides a kind of escape hatch, and sometimes you can't avoid it, but it's best to think of it as a last resort.
And the second factor follows from the first: by moving into an imperative idiom, you suddenly need think more about how Rx actually does things. If you avoid Do, you can think about Rx subscriptions at a fairly high level of abstraction. You can think a stream of chars being split into windows that are collected by SelectMany. You generally don't need to ask yourself exactly how and when Rx makes the necessary method calls. You don't need to think about the fact that you've passed a callback method in as the argument to SelectMany that will need to run at some specific moment—you can just think about the abstract operation of Window and SelectMany.
In fact, I'd go as far as to say that the ability to think at this higher level of abstraction is Rx's biggest selling point.
But Do defeats that.
By introducing Do the behaviour of the code is now much more sensitive to how Rx actually does things in practice. Actually, in particular it's the fact that one of the Do callbacks pushes new items into the upstream source—this creates a feedback loop that exists entirely outside of the core LINQ abstractions that Rx is modelled on.
That makes this code relatively hard to understand. By using Do and by pushing items into the upstream source in the Do callback, you've now made the fact that SelectMany has to invoke the callback you supplied a relevant factor in the behaviour. (And it could get a whole lot harder to understand once schedulers get involved. I would say that you really never want to be in a position where you have to understand exactly how particular schedulers do things, but if you use Do, you may well be putting yourself in exactly that position.)
So in conclusion:

There is no evidence of a bug in Rx here—it did what you told it
Your code causes items to go into a window before that window has any subscribers
Avoid Do if at all possible

Thanks @idg10 for the superb explanation of my minimal repro. I agree that there is no bug in Rx here, and the behavior that caught me by surprise was the correct behavior to be expected from Rx.
Have you also studied the code posted by @Schand1 in the opening comment of this issue? Their issue might not be identical with the issue in my repro. I am worried that the custom Rx operator BufferUntilInactive that has been posted by Colonel Panic on StackOverflow might be buggy. It's likely that many people have used this operator in their projects, because it performs a function that is both useful and common.

@HowardvanRooijen yes, the Quiescent is essentially the same operator with the BufferUntilInactive, apart from the maximum count functionality. Colonel Panic has posted a simpler BufferUntilInactive version without maximum count policy in another StackOverflow question. Both of his answers were posted the same day (Mar 20, 2012). He has used the Window Rx operator as the basis of his implementations, contrary to the Quiescent that is based on the Buffer Rx operator. Using the Window is more risky, because if you don't subscribe to these windows immediately you might lose messages (as it happened to me in my repro). With the Buffer there is nothing to subscribe to, so this risk factor does not exist.

@theodorzoulias

Have you also studied the code posted by @Schand1 in the opening comment of this issue?

I explained what was wrong with that in my first reply in this thread.
If you look at https://stackoverflow.com/a/75926551/497397 you can see that someone added an answer to that stack overflow page based on my reply.


Add a relation from SphinxDomain -> HTMLFile

This will help with search indexing,
as it will let us get this data through a relation on the model instead of in naive Python code.
Refs #5821

Gonna merge this 🎉


Add Scheduler.Schedule(Action action)

Resolves: https://github.com/dotnet/corefxlab/issues/2000
/cc @davidfowl @pakrym

Superceaded by https://github.com/dotnet/corefxlab/pull/1999


add "withMetadata" and "overwrite" flag

Hello,
I use your maven-plantuml-plugin to created *.png files from *.puml files directly in the source directory.
The *.png files are commited to the architecture documentation so that they can viewed in the git file browser.
During the process I observed that the *.png files are always modified with binary differences if created on different systems. This is because of the image metadata, which contain system environment information.
Another observed effect is that during re-execution of the goal any output file is overwritten without any change in the source *.puml file.
The provided changes contain solutions for both problems.
It would be nice if they can be integrated into your master.

Thanks for the change!


Add support for more mutation types

We should create a different issue for each of this (or we can use a GitHub project):

Invert logical operators (&&, || and !)
Bitwise operator replacement
Remove calls to functions without return
Replace the return of a function with its corresponding zero value
Replace return values that can be nil with nil
Conditionals removal (ex. if a > b { -> if true {
Remove increments/decrements
Invert assignments (ex. += -> -=)
Remove assignments (ex += -> =)
Invert loop break/continue
Empty conditional branches

We created an entire project for this issue, so this one will be closed.


Fix orientation details export

Hello this PR is to deal with the problem discussed here: #973

[x] Add orientation computation and output it
[x] Add language detection model
[x] Add it to predictor and output result
[x] Add tests

Hi @aminemindee :wave:,
the Actions issue on GH side has destroyed anything .. build actions stucks in a Queue and i cannot trigger or cancel it (same for manual merge) :sweat_smile:
Could you please push again that we can rerun CI
git commit --allow-empty -m "trigger CI"
than push again should work without any changes :)

Thanks :)

@frgfm could you check this please i'm not able to merge this PR :thinking: (manual i get also an error: At least 1 approving review is required by reviewers with write access. :thinking: )


feat(gradle): add support for gradle-language-server

Opening a new PR with, basically, #1710 but with docs. It was rather aggressively closed by the Github action bot

It was rather aggressively closed by the Github action bot

Heh, yeah, it does help reduce my PR review burden usually though. Merged!


bump sbt-comms-packaging to enable metaspace limit

This PR was successfully deployed to production by CircleCI build 25 at 2017-03-10 13:20 GMT.


Alternating canonical SMILES produced

Describe the bug
SMILES canonicalization alternates between two different outputs.
To Reproduce
from rdkit import Chem

def canonicalize(smiles):
    return Chem.MolToSmiles(Chem.MolFromSmiles(smiles))

s1 = "O=C(C[C@]1(c2ccc(-c3ccc(F)cc3)cc2)C2CC3CC1CC(C2)[C@@H]3O)N1CC(O)C1"
s2 = canonicalize(s1)
s3 = canonicalize(s2)

print(s1 == s2, s1 == s3)
>>> False True

Expected behavior
Since s1 might not be canonical smiles, I would expect at least s2 == s3, and that all future outputs are the same.
Configuration (please complete the following information):

RDKit version: 2023.03.1
OS: MacOS Monterey
Python version: 3.9.16
Are you using conda? No
If you are not using conda: how did you install the RDKit? pip via pdm

The legacy stereochemistry perception code (default) has known issues.
If you switch to the new stereochemistry perception code the issue disappears.
Note that all stereochemistry will be stripped from the molecule upon canonicalization, since the adamantane has all bridgehead stereocenters undefined, and hence there is not enough information to unequivocally determine the stereochemistry at the centers that carry wedged bonds.
from rdkit import Chem, ForceField
from rdkit.Chem import rdDistGeom, rdForceFieldHelpers

Chem.SetUseLegacyStereoPerception(False)

def canonicalize(smiles):
    return Chem.MolToSmiles(Chem.MolFromSmiles(smiles))

s1 = "O=C(C[C@]1(c2ccc(-c3ccc(F)cc3)cc2)C2CC3CC1CC(C2)[C@@H]3O)N1CC(O)C1"
s2 = canonicalize(s1)
s3 = canonicalize(s2)
print(s1 == s2, s1 == s3)
False False
s2 == s3
True
s2
'O=C(CC1(c2ccc(-c3ccc(F)cc3)cc2)C2CC3CC1CC(C2)C3O)N1CC(O)C1'

However, if you provide a full stereochemical description of the adamantane, stereochemistry will be retained upon canonicalization:
s4 = 'O=C(C[C@@]1(c2ccc(-c3ccc(F)cc3)cc2)[C@@H]2C[C@@H]3C[C@H]1C[C@H](C2)[C@H]3O)N1CC(O)C1'
canonicalize(s4) == s4
True


.Net Processes: Create a sample showing how to use DI with Steps

          I spent a couple hours trying to figure out how to get access to the primary IServiceProvider DI container from within a step but came up with nothing so far.

So, it would be helpful to show one practical example of interacting with a service external to the SK Process/Steps instead of having placeholder comments. The service doesn't actually need to do anything... SDK consumers just need to see how to get access to the service.
For example, in NewAccountStep instead of:
[KernelFunction(Functions.CreateNewAccount)]
    public async Task CreateNewAccountAsync(KernelProcessStepContext context, bool previousCheckSucceeded, NewCustomerForm customerDetails, List<ChatMessageContent> interactionTranscript, Kernel _kernel)
    {
        // Placeholder for a call to API to create new account for user
        var accountId = new Guid();
        ...
    }

It would be:
[KernelFunction(Functions.CreateNewAccount)]
    public async Task CreateNewAccountAsync(KernelProcessStepContext context, bool previousCheckSucceeded, NewCustomerForm customerDetails, List<ChatMessageContent> interactionTranscript, Kernel _kernel)
    {
        // var accountApiService = how do I resolve this service? As far as I can tell, there is no access to the DI container from within a step.
        var accountId = new Guid();
        ...
    }

Originally posted by @rjygraham in https://github.com/microsoft/semantic-kernel/issues/8990#issuecomment-2380356865

Per @alliscode, DI support for Process Steps is available when constructing the Kernel as illustrated in this sample:
https://github.com/microsoft/semantic-kernel/blob/cdb2d5930bf6e0cffec13ec393fb68040cabb940/dotnet/samples/Concepts/DependencyInjection/Kernel_Building.cs#L62
Given this redirect, I'm happy to create a sample showing DI within SK Process Steps. The SK Process documentation lists 3 business process examples in which SK Process Framework could be a fit. We already have a sample for Account Opening, so I could either update existing sample or start new using the Food Delivery, Support Ticket, or some other sample.
Any preferences?

Hi @rjygraham, this would great if you could contribute a sample for this! I like the Support Ticket scenario for this but any of them would be fine.


Time2Vec: Learning a Vector Representation of Time

Relevant information

Are you willing to contribute it (yes/no): yes
Are you willing to maintain it going forward? (yes/no): yes
Is there a relevant academic paper? (if so, where): Time2Vec: Learning a Vector Representation of Time
Does the relavent academic paper exceed 50 citations? (yes/no): no (google scholar counts 15)
Is there already an implementation in another framework? (if so, where): There are several like this in pytorch  and this in tensorflow  but they are lacking: learning supporting multiple features and/or applying the layer in tensors of shape (batch size, seq len, feat size) is not supported.
Was it part of tf.contrib? (if so, where): no

Which API type would this fall under (layer, metric, optimizer, etc.)
Layer
Who will benefit with this feature?
Primarily time series and other sequential tasks
Any other info.
Sample (working properly, but feature-incomplete) implementation:
import tensorflow.keras as keras
import tensorflow.keras.backend as K

class Time2Vec(keras.layers.Layer):
    def __init__(self, kernel_size):
        super().__init__(trainable=True, name='Time2Vec')
        self.k = kernel_size

    def build(self, input_shape):
        # trend
        self.wb = self.add_weight(name='wb', shape=(
            input_shape[1],), initializer='uniform', trainable=True)
        self.bb = self.add_weight(name='bb', shape=(
            input_shape[1],), initializer='uniform', trainable=True)
        # periodic
        self.wa = self.add_weight(name='wa', shape=(
            1, input_shape[1], self.k), initializer='uniform', trainable=True)

        self.ba = self.add_weight(name='ba', shape=(
            1, input_shape[1], self.k), initializer='uniform', trainable=True)
        super().build(input_shape)

    def call(self, inputs, **kwargs):
        bias = self.wb * inputs + self.bb
        dp = K.dot(inputs, self.wa) + self.ba
        wgts = K.sin(dp)

        ret = K.concatenate([K.expand_dims(bias, -1), wgts], -1)
        ret = K.reshape(ret, (-1, inputs.shape[1]*(self.k+1)))
        return ret

    def compute_output_shape(self, input_shape):
        return (input_shape[0], input_shape[1]*(self.k + 1))

In order to use that as-is, you'd have to do:
x = tf.random.normal([8, 7, 5]) # bs, seq len, feat

time_embedding = keras.layers.TimeDistributed(Time2Vec(kernel_size=3))(x)

y = K.concatenate([x, time_embedding], -1) # optionally concatenate with input

There are some design considerations:

Are we going to use 2 kernel sizes?, one for the trend and one for the periodic component, or just keep one kernel size and optionally also provide a second one that takes the value of the first one if it is None
Is the TimeDistributed part going to be automatically included in the layer by inspecting the input_shape?

Thanks for the relevant information and the sample but there aren't enough citations. We will wait for some more time to check if the number of citation grows over time. If it does, you can reopen the issue and submit a PR.

It's been a few years since this original post. The paper indeed has over 50 citations now. Can we re-open this? I'm also trying to adapt this layer for multiple features.


Build ref link should point to commit not compare if only on commit has changed

example:
get: https://codeberg.org/6543/woodpecker/compare/97554816f4b927328ed523a6506aa38a1b99f039...d086e8268fa7713d72c6f1d03bab52f762c99325
want: https://codeberg.org/6543/woodpecker/commit/d086e8268fa7713d72c6f1d03bab52f762c99325

plus, right now it can also be wrong:
https://codeberg.org/Freeyourgadget/Gadgetbridge/compare/0000000000000000000000000000000000000000...943f33e0cec05324c3495dec1e9cbbcbd1aaa6d6
this is from https://ci.codeberg.org/Freeyourgadget/Gadgetbridge/build/139


Deprecation warning for json_decode() for MenutItem model

Laravel version
9.52.0
PHP version
8.2.3
Voyager version
1.6.0
Database
MySQL 8.0
Description
json_decode(): Passing null to parameter #1 ($json) of type string is deprecated in ... vendor/tcg/voyager/src/Models/MenuItem.php on line 93
Steps to reproduce
Setup voyager with Laravel 9, Valet and PHP 8.2. Have a look at tail -f storage/logs/php-deprecation-warnings.log
Expected behavior
The log file should contain deprecation warnings
Screenshots
No response
Additional context
No response

Same problem here

Dupe of #5603


line-offset makes the tile borders visible

Context
Trying to style a layer using line-offset and the tiles borders become visible as a side effect
Steps to Reproduce

Create a map from a polygon dataset or from scratch with an empty layer, and then draw a polygon by hand
Style the layer with an extra border with Npx offset as

#layer {
  polygon-fill: #374c70;
  polygon-opacity: 0;
  line-width: 2;
  line-color:red;
  line-opacity: 1;
}
#layer::2 {
  polygon-fill: #374c70;
  polygon-opacity: 0;
  line-width: 2;
  line-color: blue;
  line-opacity: 1;
  line-offset:-3;
}

Current Result
red lines: actual polygon border
blue lines: polygon border with offset

If offset is < 0: the tiles that intersect my polygon has visible borders inside the polygon

If offset = 0: Behavior as expected

if offset > 0: the tiles that intersect my polygon has visible borders outside the polygon

Expected result
Just no tiles borders :)
Browser and version
Any
.carto file
test_offset.carto.zip
Additional info
Please add any information of interest here below

Sorry I never posted, but found a temporary workaround haven't tested lately, but you can see here

It can be reproduced easily in local, with and without metatiling. I could not confirm it yet, but I'm pretty sure the bug is within mapnik, most likely around these lines:
https://github.com/mapnik/mapnik/pull/2955/files
(offset converter, line symbolizer)
Aside, as @makella pointed out, there's a workaround to it, that I reproduce:
#layer {

  // Use just one layer w/ 2 styles
  // this line-width below prevents the grid effect
  line-width: 0;

  ::top {
  	polygon-fill: #374c70;
  	polygon-opacity: 0;
  	line-width: 2;
  	line-color:red;
  	line-opacity: 1;
  }
  ::2 {
    polygon-fill: #374c70;
  	polygon-opacity: 0;
  	line-width: 2;
    line-color: blue;
    line-opacity: 1;
    line-offset: -3;
  }
}

I created a PR reverting the suspect change: https://github.com/CartoDB/mapnik/pull/59
but did not have time to test it. Mind that this is just a hail mary pass in an attempt of quickly fixing it...

I'm having a look into into this. It can be solved by disabling line-clip in the second layer:
#layer {
  polygon-fill: #374c70;
  polygon-opacity: 0;
  line-width: 2;
  line-color:red;
  line-opacity: 1;
}
#layer::2 {
  polygon-fill: #374c70;
  polygon-opacity: 0;
  line-width: 2;
  line-color: blue;
  line-opacity: 1;
  line-offset:-3;
  line-clip: false; //<<<<<<<<<<
}

I have it partially fixed (it was an issue in where the clipping was being done):

But I've noticed that the clipping has some issues (you can see some lines disappearing now) so I'm looking into this with a minimal test case to have it shipped together:

Left = no clipping, Right = with clipping

The issue is found in how the offsetted points are calculated. After some changes I see both improvements and deterioration in the visual tests, so this is really tricky to get it right:

I'll see if I can find a better compromise since the issue with the test from my previous message is that after clipping the points are too close (closer than the static threshold) so it decides to skip them. I'll look into making that threshold more dynamic (based on the properties of the CartoCSS) or increase the distance to the clipping grid.

Should be fixed with https://github.com/CartoDB/mapnik/pull/63
Since it requires review, performance regression tests and multiple intermediate releases it might take a while to get it into production. I'll let you know once it's ready.

Moving this out of RT. It'll be deployed with the next Mapnik release

Fixed in https://github.com/CartoDB/cartodb/pull/14124

@ramiroaznar I'm not sure that this is related to #14124.
I think it's related to mapnik (engine), but it can be closed since they are going to deploy Mapnik soon.

Yeah, it's not related to #14124. I'm keeping this open until the fix is released and installed in production.

Sorry! Yes, I see it is not the same problem.

Yeah, it's not related to #14124. I'm keeping this open until the fix is released and installed in production.

Well, it's only taken me 13 months but it's now in production.

woot!!

:100:


Credo agent failing to start

In the most recent test runs, the Credo agent is failing to start, resulting in all tests failing.

I believe that the Credo agent is actually starting, however it is taking longer to start in some instances than the 30 second timer we have set waiting to get a status. This is manifesting now because we no longer run tests if we don't get a status on an agent. In the past it would carry on and we wouldn't notice.  I think we need to extend the checking of status to 60 seconds.
I've seen this a couple times with ACAPy as well.

I may be mistaken and the credo agent is actually not starting. I am getting a Require ESM error when building locally now. A bad library version update maybe.

Any luck resolving this?  When I run it locally, no logs are written, so I can't see anything to debug.  AFAIK, the build works fine.

After working with the main Ts.ED maintainers, they have fixed a v6 peer dependancy and a v7 peer dependancy. This has fixed our immediate problem. However, I am still working with them to move the credo backchannel to Ts.ED v7 permanently so this will be less likely to happen again. So if we can get to a working v7 I will still do a PR for that. I'll only spend a few more hours on that and if we can't sort it will revert to v6 as is.
Closing this issue since credo tests behaved normally in last night's run.

Awesome stuff — well done!!


Enhance mobile suspend MainLoop notifications

Implements https://github.com/godotengine/godot-proposals/issues/8453
iOS:
Adds NOTIFICATION_APPLICATION_RESUMED and NOTIFICATION_APPLICATION_PAUSED on iOS, as well as making the application being switched from also fire NOTIFICATION_APPLICATION_FOCUS_OUT and NOTIFICATION_APPLICATION_FOCUS_IN upon returning.
Android:
Makes the application being switched from fire NOTIFICATION_APPLICATION_FOCUS_OUT and NOTIFICATION_APPLICATION_FOCUS_IN upon returning.
Be mindful of https://github.com/godotengine/godot-proposals/issues/8453#issuecomment-1817240294, there may be complications with DeX mode that need to be ironed out.

It would be better to have the Android and iOS implementations in the same PR/commit, so that we don't risk introducing significant platform support differences. If you could only implement one, it's fine to have a PR for it, but since you did both it makes sense to land that change all at once.

@Calinou thoughts?

I'd follow Akien's judgement here, so it makes it easier to compare the Android and iOS code added in the same PR.

Merged

Please squash your commits into one, see here

Done 🙂

Any update on this? I believe this feature is exactly what i'm looking for in my own game, so would be great to see this merged

Any update on this?

The iOS part still needs to be reviewed and we don't have a dedicated iOS maintainer, so it takes some time.

Came back from that notification and forgot I should have included the rationale for this.

[b]Note:[/b] On iOS, you only have approximately 5 seconds to finish a task started by this signal.

The source for that claim added to the docs is this, there might be a better way to word it but that might make review a bit easier. I think this is relevant information because it would probably be hard for someone deep into a Godot project to think to check Apple's docs for applicationDidEnterBackground().

Thanks!


add a example for using SWC plugin

SWC plugin for client-preset was added here https://github.com/dotansimha/graphql-code-generator/pull/8759 we should have example next app for it.

We have it here https://github.com/dotansimha/graphql-code-generator/blob/master/examples/react/nextjs-swr/next.config.js


Recording the data

Hey, I’m a great fan of your app!
Is there an easy way to record the animation?
Thanks
Marco

Never Mind. I found the "Unity Animation Recorder".


Brightness condition not functioning properly with shader power type

(This has ripple effects onto exposed_to_sun)
Important findings at the bottom too!
Brightness condition is always evaluated as higher than 0.5 (I checked by reversing the comparison and using different shaders) for the shader power type.
Affected example:
Notice how weakness is properly applied but the shader is applied even when it shouldn't:

Data for that example:
{
	"type": "origins:multiple",

	"comparison_power_meant_to_show_that_things_are_triggered_properly_for_this_power_type": {
		"type": "origins:stacking_status_effect",
		"min_stacks": 0,
		"max_stacks": 1,
		"duration_per_stack": 20,
		"effects": [
			{
				"effect": "minecraft:weakness",
				"is_ambient": true,
				"show_particles": false,
				"show_icon": true
			}
		],
		"condition": {
			"type": "origins:brightness",
			"comparison": ">",
			"compare_to": 0.5
		}
	},

	"shader_power_that_has_the_issue": {
		"type": "origins:shader",
		"shader": "minecraft:shaders/post/pencil.json",
		"condition": {
			"type": "origins:brightness",
			"comparison": ">",
			"compare_to": 0.5
		}
	},

	"tell_whether_brightness_condition_is_fullfilled_every_second_for_reference": {
		"type": "origins:action_over_time",
		"entity_action": {
			"type": "origins:if_else",
			"condition": {
				"type": "origins:brightness",
				"comparison": ">",
				"compare_to": 0.5
			},
			"if_action": {
				"type": "origins:execute_command",
				"command": "say true"
			},
			"else_action": {
				"type": "origins:execute_command",
				"command": "say false"
			}
		},
		"interval": 20
	}
}

The shader is also active at both noon and midnight instead of deactivating at midnight in the case of a dark one:

This excludes the hypothesis that brightness depends on the post-processed image for shader power types.
When changing ">" to "<":

Reference test (for comparison):
{
	"type": "origins:multiple",

	"comparison_power_meant_to_show_that_things_are_triggered_properly_for_this_power_type": {
		"type": "origins:stacking_status_effect",
		"min_stacks": 0,
		"max_stacks": 1,
		"duration_per_stack": 20,
		"effects": [
			{
				"effect": "minecraft:weakness",
				"is_ambient": true,
				"show_particles": false,
				"show_icon": true
			}
		],
		"condition": {
			"type": "origins:on_block",
			"block_condition": {
				"type": "origins:block",
				"block": "minecraft:grass_block"
			}
		}
	},

	"shader_power_that_has_the_issue": {
		"type": "origins:shader",
		"shader": "minecraft:shaders/post/pencil.json",
		"condition": {
			"type": "origins:on_block",
			"block_condition": {
				"type": "origins:block",
				"block": "minecraft:grass_block"
			}
		}
	}
}

In this test everything goes as expected and the player gets weakness if and only if the shader effect is applied, meaning the shader deactivates and activates properly. I also tested with exposed_to_sky, yielding the same results.
IMPORTANT FINDINGS
Now, changing from "> 0.5" to "== 1" gives us interesting results:
At noon:
Not exposed:

Exposed:

At midnight:
Not exposed:

Exposed:

It seems the brightness value is somehow offset due to the shader. I would like to do more research but it's 3:30 AM so that will be it for now.
Naturally I'm using Origins version 0.7.1.

Judging by the code, the issue seems to be from the game itself. Maybe it can be patched though.

For reference, the following has the expected behavior:
{
	"type": "origins:multiple",

	"is_exposed": {
		"type": "origins:resource",
		"min": 0,
		"max": 1,
		"start_value": 0,
		"hud_render": {
			"should_render": false
		}
	},

	"update_exposed": {
		"type": "origins:action_over_time",
		"entity_action": {
			"type":"origins:if_else",
			"condition": {
				"type": "origins:exposed_to_sun"
			},
			"if_action": {
				"type": "origins:execute_command",
				"command": "resource set @s alluyslorigins:test_is_exposed 1"
			},
			"else_action": {
				"type": "origins:execute_command",
				"command": "resource set @s alluyslorigins:test_is_exposed 0"
			}
		},
		"interval": 20
	},

	"comparison_power_meant_to_show_that_things_are_triggered_properly_for_this_power_type": {
		"type": "origins:stacking_status_effect",
		"min_stacks": 0,
		"max_stacks": 1,
		"duration_per_stack": 20,
		"effects": [
			{
				"effect": "minecraft:weakness",
				"is_ambient": true,
				"show_particles": false,
				"show_icon": true
			}
		],
		"condition": {
			"type": "origins:resource",
			"resource": "alluyslorigins:test_is_exposed",
			"comparison": "==",
			"compare_to": 1
		}
	},

	"shader_power_that_has_the_issue": {
		"type": "origins:shader",
		"shader": "minecraft:shaders/post/pencil.json",
		"condition": {
			"type": "origins:resource",
			"resource": "alluyslorigins:test_is_exposed",
			"comparison": "==",
			"compare_to": 1
		}
	},

	"tell_whether_brightness_condition_is_fullfilled_every_second_for_reference": {
		"type": "origins:action_over_time",
		"entity_action": {
			"type": "origins:if_else",
			"condition": {
				"type": "origins:resource",
				"resource": "alluyslorigins:test_is_exposed",
				"comparison": "==",
				"compare_to": 1
			},
			"if_action": {
				"type": "origins:execute_command",
				"command": "say true"
			},
			"else_action": {
				"type": "origins:execute_command",
				"command": "say false"
			}
		},
		"interval": 20
	}
}

I should mention I'm using Sodium, actually.


Bbox_crop is set to false in stage_2.yaml but not supported by dataset code

Hi, I noticed you set bbox_crop to false in stage_2.yaml and keep quote it in the training procedure

but the tgt_box is necessary in video_dataset.getitem and could only be calculated when bbox_crop is True

I'm sorry, this is indeed a bug. You can modify it by adding a line of code as mentioned above, so that the default tgt_bbox is set to the entire ref_img.

I'm sorry, this is indeed a bug. You can modify it by adding a line of code as mentioned above, so that the default tgt_bbox is set to the entire ref_img.

Thx, so is "bbox_crop" set to true in your paper?


Not getting the SPEAKER SEGMENTS

I have downloaded the model and tested it on my local machine. However, I am facing a critical issue. The [SPEAKER_TURN] information that I was expecting is missing in all the WAV files that I processed using this component. It was working fine when I tested it last time, but now it's not. This absence of speaker segmentation is causing significant disruptions in my workflow. As I need to deploy this system into a production environment, it is crucial that this issue gets resolved as soon as possible.

The project hasn't been meaningfully updated since June. Probably what changed was in your own setup?

@prafulkl this is not a production ready tool so you should not make requests like this.

Hi @prafulkl, as the others have pointed out (ty!):

If things were working previously, it's most likely something on your end. Both the checkpoints and inference code haven't seen any changes. In general, you should be able to share a reproducible example if someone has to help debug an issue. I would suggest making sure your python env isn't somehow clashing with the original OpenAI whisper package https://github.com/akashmjn/tinydiarize/issues/16
This is not a production-ready project, rather a prototype/proof-of-concept that happens to work fairly well. For reasons in https://github.com/akashmjn/tinydiarize/issues/14 it will be the case for some time. I have added a  note to the README to clarify this. Hope that helps.

For these reasons, I'm going to have to close this issue.


Reactivate Inline GitHub Issues

Heya! Bikeshed's author here, just letting you know that you should be able to turn GitHub Inline Issues back on now. (Turned off in https://github.com/openregister/specification/commit/58cef44736f7c5e5333b624a02fa0e742bec51f3.)  Bikeshed now caches the issues it finds, so unless the issue was changed since you last generated the spec, regenning doesn't count any of them against your limit.
As for the styling issue you're running into, you can negate that easily with a .issue > .marker { text-transform: none !important; } to override the rule coming from the default Bikeshed stylesheet. I think it's a bug in Bikeshed's stylesheet, too, but I try to stay aligned with the W3C stylesheet, so I'll be filing an issue to get the W3C stylesheet changed shortly.

Thanks for letting us know!
I think we're happy with the existing behaviour rather than switching the live issues back on, so I'll mark this issue as closed.


Database rights

As a Application admin
I want to set privilages to users
So I set the userrights: read-only, write and tailored XML acces (e.g. masking the IP adresses).
Background
Due to possible sensible data in CoMPAS it is needed to set userrights and management.
Acceptance criteria:

Where do you set the user privelages?
How do you connect them with a central user rights repository? or other microservices can acces the services.
Is direct database access allowed for a microservices architecture?
Documented on the architecture pages

See architecture website


Pytest discovery fails if one import is incorrect

In vscode the whole test discover fails if one import in the code is incorrect. Pycharm is still able to discover almost all tests, even when there are issues with some of the imports (or syntax errors) in the code.

/extPython

It looks like this is caused by the Python extension. Please file the issue to the Python extension repository. Make sure to check their issue reporting template and provide them relevant information such as the extension version you're using. See also our issue reporting guidelines for more information.
Happy Coding!


Unite structs with similar paths through flatten doesn't work

Greetings!
The problem is that I can't split structure into several structures and unite them through flatten.
P.S. serde_json is just to show the problem. I faced with it using config crate.
#[derive(Debug, serde::Deserialize)]
struct First {
    path: Path0,
}

#[derive(Debug, serde::Deserialize)]
struct Path0 {
    begin: Box<str>,
}

#[derive(Debug, serde::Deserialize)]
struct Second {
    path: Path1,
}

#[derive(Debug, serde::Deserialize)]
struct Path1 {
    end: Box<str>,
}

#[derive(Debug, serde::Deserialize)]
struct Unite {
    #[serde(flatten)]
    first: First,
    #[serde(flatten)]
    second: Second,
}

fn main() {
    let json = serde_json::json!({
        "path": {
            "begin": "too",
            "end": "not_too",
        }
    });

    let data = serde_json::from_value::<Unite>(json);

    println!("{data:?}");
}

The error message is clear: "Err(Error("missing field path", line: 0, column: 0))"
Through flattening, the Unite struct looks like this from serde's point of view:
struct Unite {
    // From flattening First
    path: Path0,

    // From flattening Second
    path: Path1,
}

You have the same field declared twice (I don't even know if serde is supposed to handle this), and you're not even giving a field named "path" to your deserializer. So it obviously fails.

You have the same field declared twice (I don't even know if serde is supposed to handle this), and you're not even giving a field named "path" to your deserializer. So it obviously fails.
The reason was clear, it doesn't clear how to deal with it.

Did you forget to flatten First::path and Second::path? Did you forget a #[serde(transparent)] on any of these structs?
I add fully working example in first message (it uses serde and serde_json).
With #[serde(transparent)]:

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct First {
    path: Path0,
}

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct Path0 {
    begin: Box<str>,
}

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct Second {
    path: Path1,
}

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct Path1 {
    end: Box<str>,
}

#[derive(Debug, serde::Deserialize)]
struct Unite {
    #[serde(flatten)]
    first: First,
    #[serde(flatten)]
    second: Second,
}

gives error:
Err(Error("can only flatten structs and maps", line: 0, column: 0))

I think I understand, but this looks very weird and I'm not even sure there is a solution. Why do you need to split a deserialize struct in 2 like this ? It's much simpler to just do:
#[derive(Debug, serde::Deserialize)]
struct Path {
    #[serde(flatten)]
    first: First,
    #[serde(flatten)]
    second: Second,
}

#[derive(Debug, serde::Deserialize)]
struct Unite {
   path: Path,
}

And depending on your use case, you should take a look at figment, another deserialization library (made for configuration files), partially based on serde, that can resolve multiple "layers" of configuration exactly like what you're trying to do. The Rocket web server uses it to allow fairings (extensions) to read from the same configuration file as rocket.

It's much simpler to just do:

It's not an option for my case, because I have struct with generic.

you should take a look at figment, another deserialization library

Hm, thanks, I'll take a look. I use config crate, seems that I can use figment instead of config. I'll definitely take a look if figment solve my problem.

Why you try to flatten fields that are effectively Box<str> (because of flatten)? Of course this will not work, because string has no internal structure.
Your struct equivalent to this:
#[derive(Debug, serde::Deserialize)]
struct Unite {
    #[serde(flatten)]
    first: Box<str>,
    #[serde(flatten)]
    second: Box<str>,
}

Why you try to flatten fields that are effectively Box<str> (because of flatten)?

Sorry, what?
I'm flattening struct First and struct Second which give that (as @edgarogh already said #issuecomment-1232005440) it comes into this:
struct Unite {
    // From flattening First
    path: Path0,

    // From flattening Second
    path: Path1,
}

I misprinted. #[serde(trasparent]) means that structure with one field will be serialized as only field. Actually, you can see this code after expanding macros:
#[derive(serde::Deserialize)]
#[serde(transparent)]
struct First {
    path: Path0,
}

turns into
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
#[serde(transparent)]
struct First {
    path: Path0,
}
#[doc(hidden)]
#[allow(non_upper_case_globals, unused_attributes, unused_qualifications)]
const _: () =
    {
        #[allow(unused_extern_crates, clippy :: useless_attribute)]
        extern crate serde as _serde;
        #[allow(unused_macros)]
        macro_rules! try {
            ($__expr : expr) =>
            {
                match $__expr
                {
                    _serde :: __private :: Ok(__val) => __val, _serde ::
                    __private :: Err(__err) =>
                    { return _serde :: __private :: Err(__err) ; }
                }
            }
        }
        #[automatically_derived]
        impl<'de> _serde::Deserialize<'de> for First {
            fn deserialize<__D>(__deserializer: __D)
                -> _serde::__private::Result<Self, __D::Error> where
                __D: _serde::Deserializer<'de> {
                _serde::__private::Result::map(_serde::Deserialize::deserialize(__deserializer),
                    |__transparent| First { path: __transparent })
            }
        }
    };

Apply chain of that transformations, in the end your struct serialized the same as
#[derive(Debug, serde::Deserialize)]
struct Unite {
    #[serde(flatten)]
    first: Box<str>,
    #[serde(flatten)]
    second: Box<str>,
}

and in that case flatten is impossible to work because absence of the internal structure of a string.

Okey, for now I use this workaround:
impl<'de> Deserialize<'de> for Unite {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        let value = Value::deserialize(deserializer)?;

        let first = First::deserialize(&value).map_err(D::Error::custom)?;
        let second = Second::deserialize(value).map_err(D::Error::custom)?;

        Ok(Self { first, second })
    }
}

You just need to remove #[serde(flatten)]

You just need to remove #[serde(flatten)]

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct First {
    path: Path0,
}

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct Path0 {
    begin: Box<str>,
}

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct Second {
    path: Path1,
}

#[derive(Debug, serde::Deserialize)]
#[serde(transparent)]
struct Path1 {
    end: Box<str>,
}

#[derive(Debug, serde::Deserialize)]
struct Unite {
    first: First,
    second: Second,
}

Err(Error("missing field `first`", line: 0, column: 0))

The solution in https://github.com/serde-rs/serde/issues/2261#issuecomment-1235272468 looks good to me!


⚠ File access has degraded performance

In 2c533e1, File access (https://fyle.uk/file/fish) experienced degraded performance:

HTTP code: 200
Response time: 6871 ms

Resolved: File access performance has improved in f3b5578 after 11 minutes.


Check env

Dont merge, this is a test

This is no longer needed.  It was for diagnostic purposes.


Save theme-builder design-tokens as zip

Use the cli to generate json design-tokens and save as a zip file

We decided to wait with this as it only negates the need to not use node / npm. You still have to create a git repo and push tokens.


Why is 2.0.0 on the PSGallery but not on Github?

The current release show  1.1.33 as latest
https://github.com/googleprojectzero/sandbox-attacksurface-analysis-tools/releases
The PS Gallery has 2.0 published with an alpha as well..
https://www.powershellgallery.com/packages/NtObjectManager/2.0.0

I will release the code soon, it's in the middle of a significant refactor. However I needed to put up a version of 2.0 for the early release of my book which will use the module.

2.0 change log ..had  breaking changes.. (no surprise hence 2.0.. but a change log would of been nice)
WTF-WFP was downstream of this PSM
https://zeronetworks.com/blog/wtf-is-going-on-with-wfp/
Unable to find type [NtApiDotNet.Net.Firewall.FirewallLayerGuids].
At C:\Program Files\WindowsPowerShell\Modules\wtf-wfp\1.0.4\wtf-wfp.psm1:805 char:34
+ ... $layerslist = @([NtApiDotNet.Net.Firewall.FirewallLayerGuids]::FWPM_L ...
+                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (NtApiDotNet.Net...ewallLayerGuids:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

Unable to find type [NtApiDotNet.Net.Firewall.FirewallConditionFlags].
At C:\Program Files\WindowsPowerShell\Modules\wtf-wfp\1.0.4\wtf-wfp.psm1:823 char:5
+     [NtApiDotNet.Net.Firewall.FirewallConditionFlags]$flags,
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (NtApiDotNet.Net...lConditionFlags:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

Yeah, well sorry about that, I at least should have put in a warning in the release notes. That said supporting anything outside of the exported cmdlets has never been my intention, the ability to use the NtApiDotNet library is just a sometimes useful byproduct and shouldn't be relied upon. As in, any script which worked before in v1 and only used the exported cmdlets should hopefully still work in v2, but obviously if you access internal classes I don't make any guarantees.
You could specify in your module that you rely on a specific version of the NtObjectManager module, not sure how well that works in practice.
Unfortunately the v1 branch is basically dead, anti-virus detections killed it. It required a substantial refactor to try and make the detections go away. Some things might not have been required such as changing the main module's name but I thought best to do it one go (this seems to be what has failed for you, going from NtApiDotNet to NtCoreLib). While I was at it a more substantial refactor of the structure is being undertaken which might change things even more.

Interestingly, there are parts of this that might be extracted for more generalized access. The generalized access may be abused less. Specifically a smaller scope that could be combined to make the [sandbox-attacksurface-analysis-tools].
I say this because the full tool is very powerful, but Im focused on the pure WFP api portion.

Certainly it might make sense to pull out what you specifically need into your own library to provide with the PowerShell code, although not sure how easy that would be. You could also ship a copy of NtApiDotNet.dll with the module. I think the main problem with AV is around code linking to the library not necessarily the library itself so it probably won't be detected by AV.


WIP: test cleanup

Clearly divide and specify test types.

 Can you please review so we can merge this as soon as possible -- I will continue in the new PR. What I would like to leave to the next PR(s):

fix the response generating (needs some tweaks on the requre side)
move as much integration tests as possible to requre
clean the fixture a lot more

(impossible to review)

Sorry... That was a yearly-cleanup before Xmass..;)


Undefined but required properties do not fail the validation

Description
The validation does not check whether a required property is actually part of the definition:
A message like Schema properties specified as 'required' must be defined is not created for the validation
Swagger-codegen version
2.2.2
Swagger declaration file content or url
In below the version is required but not a property:
  Sample:
    required:
      - name
      - version
    properties:
      name:
        type: string

Command line used for generation
Steps to reproduce
Related issues
Suggest a Fix

Hi @mischac is this still an issue? I see a closed PR in other project. Did That fixed your issue?

Required properties that do not have a schema defined are a valid use case.
In that case they must only be present, and any value is accepted.


Support UUIDs

Reason/Context
A string with format UUID is often used for ids. It would be very useful if it would map to the java type UUID.
Example:
  messages:
    song:
      payload:
        type: object
        properties:
          id:
            description: ID
            type: string
            format: uuid

Description
I prepared a small example.
kafka-example.yml:
asyncapi: 2.3.0
info:
  title: Record Label Service
  version: 1.0.0
  description: This service is in charge of processing music
servers:
  production:
    url: 'my-kafka-hostname:9092'
    protocol: kafka-secure
    description: Production Instance 1
    security:
      - myScram: []
channels:
  song.released:
    publish:
      message:
        $ref: '#/components/messages/song'
    subscribe:
      message:
        $ref: '#/components/messages/song'
components:
  securitySchemes:
    myScram:
      type: scramSha512
  messages:
    song:
      payload:
        type: object
        properties:
          id:
            description: ID
            type: string
            format: uuid
          title:
            description: Title of a song
            type: string

I executed the generator like this:
ag kafka-example.yml @asyncapi/java-spring-template -o ./output

The generated code contains the property id but as object instead of UUID.

I just saw that the uuid format is not part of the asynapi specification. However I think just as with openapi (where it is also not specified but used very often), it is still very useful to support it.


Add main branch protections

Main branch should be protected generally.  Also, there are unit tests - make these required for merging.

Done with:
https://github.com/bmccool/pyMcCool/pull/5


Improve shell command support in cluster/env run

#667  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @carolineechen and the rest of your teammates on  Graphite


Memory Leak when load() API is called even with option "unload: true"

C3 version: 0.7.20
D3 version: 5.16.0
Browser: Chrome/Firefox official version
OS: Windows 10, Linux, macOS

Same issue as #1961,  I create 7 line-charts with timeseries x-axis, and each chart has 240 data points.
Only on page load, it uses c3.generate() to create charts, and then each chart uses c3.load() with unload option set as “true” (means unload all data mentioned in the c3 reference document) to update its 240 data points per 5 minutes.
I can observe the memory usage will increase 30MB~100MB per update in whole page by checking chrome task monitor.
It is easy to reproduce by using setTimeout() function or create a button to manually trigger.
By the way, the other case I found if the c3.chart is rendered complete, when blur the page to other window or app to do something about several minutes, and then back to the page with the c3.chart, the page will freeze a period of time. The cpu/memory usage grow up very fast until the the page recoveries.
PS. The page is the visible tab.
Is there any idea for this issue ?

Sorry about the wrong report.
I found the my code flow is wrong, the c3.load() with option “unload” as “true” is worked, the memory will be released in the few time.
But the case of freezing a period of time can be seem.


Add blacklisted warnings

Adds an optional array parameter to the disable_warnings method, so that you can pick and choose which warnings to ignore.
Usage:
class Response < Hashie::Mash
  disable_warnings :zip
end

Response.new('zip' => '90210') # does not log a warning
Response.new('merge' => 'yes') # does log a warning

This preserves existing usage.

Why do we need the only keyword? Is there going to be some other exclusion here in the future?
I feel like disable_warnings: [:zip] and disable_warning :zip is the most straightforward interface.

@dblock I suppose we don't explicitly need the only keyword. I can refactor if you think your proposal is more straightforward. Is it worth adding the disable_warning singular case vs just using the plural disable_warnings with an array of 1 (or checking in the function if a symbol was passed vs an array)

I think the behavior when both variations are used is broken here.
I'd expect these to pass:
    it 'does not write to the logger when warnings are disabled' do
      mash_class = Class.new(Hashie::Mash) do
        disable_warnings :zip
        disable_warnings
      end

      mash_class.new('trust' => { 'two' => 2 })

      expect(logger_output).to be_blank
    end

and this one
    it 'has cumulative behavior' do
      mash_class = Class.new(Hashie::Mash) do
        disable_warnings :two, :zip
      end

      mash_class.new('trust' => { 'two' => 2 })
      mash_class.new('address' => { 'zip' => '90210' })

      expect(logger_output).to be_blank
    end

  it 'has cumulative behavior' do
      mash_class = Class.new(Hashie::Mash) do
        disable_warnings :two, :zip
      end

      mash_class.new('trust' => { 'two' => 2 })
      mash_class.new('address' => { 'zip' => '90210' })

      expect(logger_output).to be_blank
    end

This scenario works for the given keys, the error being thrown is due to #trust being defined from Kernal
I'll give some thoughts to the other scenario you mentioned.

After looking at my test, the case where a child calls disable_warnings was incorrect, I resolved that and your first case should be resolved as well.
Now when disable_warnings is called without params after it has been called with them, we will clear the blacklist and ignore all conflicting method keys.

I am happy with this with the minor fixes in my last comments. @BobbyMcWho you should squash your commits, leaving the final review/comments/merge to @michaelherold.

Thanks for the documentation suggestion @dblock, for the sake of clarity I also added:
Disable warnings will honor the last disable_warnings call. Calling without parameters will override the blacklist, and calling with parameters will create a new blacklist. This includes child classes that inherit from a class that disables warnings.
class Message < Hashie::Mash
  disable_warnings :zip, :zap
  disable_warnings
end
# No errors will be logged
Message.new(merge: 'true', compact: true)

class Message < Hashie::Mash
  disable_warnings
end
class Response < Message
  disable_warnings :zip, :zap
end
# 2 errors will be logged
Response.new(merge: 'true', compact: true, zip: '90210', zap: 'electric')

Thanks for all the input from both of you, it's been a pleasure!

Nice work @BobbyMcWho, thanks for hanging in there with us!


Client Incomptabile with PHP Scoper

Issue Summary
PHP Scoper is a tool that adds a unique prefix to all namespaces in the vendor directory of a project in order to prevent namespace collisions - for example, if I'm developing a wordpress plugin and want to avoid namespace collisions or version collisions with the dependencies of other wordpress plugins, I'll use Scoper.
In lib/mail/Mail.php there is this line: https://github.com/sendgrid/sendgrid-php/blob/7bdf97e961f0eb529a0a027ae7d3b8a204faa905/lib/mail/Mail.php#L188. PHP Scoper is not able to detect dynamic class names like this, and so the prefix is not applied.
If using Scoper, the logic in that block fails and a fatal exception is raised. This effectively looks like:
$email = new \MyScopedNamespace\SendGrid\Mail\Mail();

// the logic from the link above
$emailTypeClass = '\SendGrid\\Mail\\' . $emailType; // '\SendGrid\Mail\Mail'
if (!$email instanceof $emailTypeClass) { // If not '\MyScopedNamespace\SendGrid\Mail\Mail' instance of '\SendGrid\Mail\Mail'
    $email = new $emailTypeClass($email, $name, $substitutions); // because of the prefixed namespace, '\SendGrid\Mail\Mail' does not exist
}

There is a workaround for this issue: https://github.com/humbug/php-scoper#patchers. This workaround is less than ideal as it's highly manual.
Steps to Reproduce

Use PHP Scoper to add a unique namespace to the package
Attempt to send an email using that namespaced package

Code Snippet
# paste code here

Exception/Log
2020-10-04T14:39:50+00:00 CRITICAL Uncaught Error: Object of class \Vendor\SendGrid\Mail\To could not be converted to string in /www/mysite/public/vendor/sendgrid/sendgrid/lib/helper/Assert.php:28
Stack trace:
#0 /www/mysite/public/vendor/sendgrid/sendgrid/lib/helper/Assert.php(28): sprintf('"$%s" must be a...', 'emailAddress', Object(\Vendor\SendGrid\Mail\To))
#1 /www/mysite/public/vendor/sendgrid/sendgrid/lib/helper/Assert.php(48): SendGrid\Helper\Assert::string(Object(\Vendor\SendGrid\Mail\To), 'emailAddress', NULL)
#2 /www/mysite/public/vendor/sendgrid/sendgrid/lib/mail/EmailAddress.php(69): SendGrid\Helper\Assert::email(Object(\Vendor\SendGrid\Mail\To), 'emailAddress')
#3 /www/mysite/public/vendor/sendgrid/sendgrid/lib/mail/EmailAddress.php(47): SendGrid\Mail\EmailAddress->setEmailAddress(Object(\Vendor\SendGrid\Mail\To))
#4 /www/mysite/public/build/sendgrid/sendgrid/lib/mail/Mail.php(148): SendGrid\Mail\EmailAddress->__construct(Objec in /www/mysite/public/vendor/sendgrid/sendgrid/lib/helper/Assert.php on line 28

Technical details:

sendgrid-php version: 7.8.4
php version: 7.4

Note - in my error logs, a class not found fatal is not raised as my code is running in a WordPress environment where other plugins have loaded their own versions of the SendGrid client.

For anyone who comes across this, adding this code block to your scoper.inc.php should correct the issue:
'patchers' => [
    function (string $filePath, string $prefix, string $content): string {
        if ($filePath === '/path/to/my/project/vendor/sendgrid/sendgrid/lib/mail/Mail.php') {
            $replacement = str_replace(
                '\\\\SendGrid\\\\Mail\\\\',
                $prefix . '\\\\SendGrid\\\\Mail\\\\',
                $content
            );

            if ($replacement !== null) {
                return $replacement;
            }
        }
        return $content;
    },
],

This issue has been added to our internal backlog to be prioritized. Pull requests and +1s on the issue summary will help it move up the backlog.


[FLINK-31169][k8s] Fix that pod termination may be mis-treated as fatal error.

@flinkbot run azure


Uris with custom schemes are not working in 2.5.2.

This update broke custom scheme urls. (Ex: "capacitor://localhost", "ionic://localhost"). Ionic app in ios or android renders pages with those urls inside webview.
According to oauth rfc 6749 section 3.1.2 custom schemes are allowed.
Issue / Steps to reproduce the problem
Add some CORS origins with custom schemes. (Ex: "capacitor://localhost", "ionic://localhost")
Relevant parts of the log file
Invalid client configuration for client sampleclient: AllowedCorsOrigins contains invalid origin: ionic://localhost

This update broke custom scheme urls. (Ex: "capacitor://localhost", "ionic://localhost"). Ionic app in ios or android shows page in those urls.

Why are you putting those in AllowsCorsOrigins? They don't need to be in there.

No, ionic creates some kind of server inside mobile and opens it in webview. That URL happens to be "capacitor://localhost".
Docs: https://ionicframework.com/docs/faq/cors

Still an issue in version 3.0.0 with custom schemes.

Mobile Applications that perform OAuth have the ability to register custom URL schemes. When such a scheme is specified in the call back URL of the request, the identity server doesn't redirect to the custom scheme and as a result, the mobile application never receives the access token.

@adozenlines what framework are you using? Is it Ionic?

Hi @brockallen this issue is blocking us from updating to .Net Core 3.0. Is there any alternative and can you check PR #3559. Thank you.

@brockallen I have overridden ValidateAllowedCorsOriginsAsync in DefaultClientConfigurationValidator. Everything is working good.

Tracked and fixed here:  #3880


Rustup

@bors r+

:pushpin: Commit 1c3808a28efcc302dfeafcea2ea02ac2e7b57abe has been approved by RalfJung
It is now in the queue for this repository.

:hourglass: Testing commit 1c3808a28efcc302dfeafcea2ea02ac2e7b57abe with merge fbc1a63c0b7e7df8ca80226ace07b28eb152ab85...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing fbc1a63c0b7e7df8ca80226ace07b28eb152ab85 to master...


GettingStarted.md is out of date

https://github.com/microsoft/ebpf-for-windows/blob/main/docs/GettingStarted.md states that the 19041 WDK should be installed.
The problem is that the eBPF-For-Windows will no longer build correctly with this WDK and instead requires the 22000 WDK.
In addition, the setup script installs both 10941 and 22000 WDK, which is redundant.

The issues appear to run deeper:

https://github.com/microsoft/ebpf-for-windows/commit/0543c810a7a98699a6b6e1e30250facad96dc371 added a dependency on WDK > 19041.
Setup-DevEnv.ps1 attempts to install the 19041 and 22000 WDK, but the installer for the 22000 WDK is no longer working.
The latest SDK / WDK is 22621, but this doesn't work with VS2019.

Given that, I don't think the scripted option is going to work anymore.
I think we should hold off fixing this until https://github.com/microsoft/ebpf-for-windows/issues/725 as it will need to be updated once VS2022 is adopted.

Fixed.


Indicated version on python-requests.org higher than what's actually released

python-requests.org shows 0.10.2, but neither PyPI nor gits tags know about that release.

It's the development docs of the next release.

But that means these docs are not for the version I get when following the installation instructions right next to the version number (well, at least not if I use pip or easy_install). That's not really elegant, is it?

Usually the overlap is only for a few hours. I'm just stuck on a few bugs right now.
If there was anything that changed usage, it'd say (dev), and the default would be the latest release.

Fair enough.

I agree that it's not ideal.


Fix path for the testdata file diff_example/error.go

This PR adds the missing example testdata/test_files/diff_example/error.go that is mentioned in the documentation. This file has been mistakenly committed as testdata/test_files/lint_example/test2.go in PR #64.
"Apply the patch" section in README states:

For example if we applied patch ~/s1028 to our testfile error.go
$ gopatch -d -p ~/s1028.patch ./testdata/test_files/diff_example/

Output would be :
gopatch/testdata/test_files/diff_example/error.go:Replace redundant fmt.Sprintf with fmt.Errorf
--- gopatch/testdata/test_files/diff_example/error.go
+++ gopatch/testdata/test_files/diff_example/error.go
@@ -7,7 +7,7 @@

func foo() error {
        err := errors.New("test")
-       return errors.New(fmt.Sprintf("error: %v", err))
+       return fmt.Errorf("error: %v", err)
}

 func main() {

But I checked the repo and this file is absent.
Additionally, I modified the patch s1028 with the comment. So, the output in README is consistent.

Hey @alexandear thanks for catching this! I think you're right, I think there was a mistake in #64 where this file was checked in at the wrong place. I searched the repo for the lint_example path and found no references to it. Do you mind also removing the file from the incorrect path (testdata/test_files/lint_example/test2.go)?

@JacobOaks done. Also, fixed tests.


About page displaying as a blog post .. how to fix ??

I am still having the problem that the about page shows up on the index page as a blog post. Even when I try content/about/_index.md... does anyone know the solution? I tried reverting to a previous commit when this post was made and the problem is still there. It seems like all files in contents get displayed as blog posts. Does anyone know how to fix this?

All regular content shows up as blog posts, this is the reason that the about file needs to be in _index.md which makes it the list page for about. It should then not appear in the list with regular blog posts. Could you show me a list of files in your project?

I'm closing this issue because there has been no activity for a while. If you still want to respond, feel free to reopen it!


Restore-DatabasesInBcContainer | SQL timeout

Freddy,
i use the new-BCContainer with a backup. most backups are not so big, and all is well.
but there are sometimes the backup is big, and the agent server is not so fast or its also bussy with other builds that de restore is slower, then i hit the 5 min timeout of the function below: Restore-DatabasesInBcContainer

is there a change you can change its default into 600 ? (10 min) or make a parameter on the new-BCContainer ?
function Restore-DatabasesInBcContainer {
Param(
[string] $containerName = $bcContainerHelperConfig.defaultContainerName,
[string] $bakFolder = "",
[string] $bakFile = "",
[string] $databaseName = "",
[string[]] $tenant,
[Parameter(Mandatory=$false)]
[string] $databaseFolder = "c:\databases",
[int] $sqlTimeout = 300

I will add a sqlTimeout on New-BcContainer.

Great!, many thanks

Shipped in 2.0.9


feat: 0.3.10rc3 support and refactor to packaging.SpecifierSet [APE-1390]

What I did

fixes: #93
Fixes: APE-1387
How I did it
How to verify it
Checklist

[ ] Passes all linting checks (pre-commit and CI jobs)
[ ] New test cases have been added and are passing
[ ] Documentation has been updated
[ ] PR title follows Conventional Commit standard (will be automatically included in the changelog)

On vyper 0.3.10rc3 regressions, confirm with @big_tech_sux if these are intended

We already discussed with Non-payable check, that one is fine.
The only other thing is still kind of a mystery to us...


Fallback to BytesIO only when needed regarding ZipFile nested zips

See https://github.com/fox-it/log4j-finder/pull/22 for more bug details

Tested with the following Docker images:

python:3.6.11 -> is fallback
python:3.7.2 is no fallback.

Can be verified using -vv and on the following Jar file: https://get.jenkins.io/war-stable/1.409.3/jenkins.war


cmd/power: Another shutdown command fix

It looks like some arguments for shutdown/halt/poweroff have been moved
to a conditional block that works off of how the command was actually
called. However, this value is derived from argv 0, without any sort of
normalization to make sure it matches the relative commands used to
determine how arguments are handled.
This has particular implications when power management commands are
called via absolute commands, as for example in the case of
open-vm-tools which calls /sbin/shutdown -h now specifically when
shutting down a system.
This corrects the situation by passing argv 0 through filepath.Base
before operating on it.

LGTM


SMJobBless fails on 10.9

As I added in a comment in #214, on 10.9 SMJobBless seems to fail with the following:
10/21/20 4:18:50.452 PM launchdadd[23465]: FAILURE: Code <SecStaticCode 0x7fc198f0d7e0 [0x7fff7b209f00]> failed requirement (identifier "com.ridiculousfish.HexFiend.PrivilegedHelper" and certificate leaf[subject.CN] = "Developer ID Application: Kevin Wojniak") with error "The operation couldn’t be completed. (OSStatus error -67050.)"

10/21/20 4:18:50.452 PM launchdadd[23465]: FAILURE: Job com.ridiculousfish.HexFiend.PrivilegedHelper could not be installed from /Applications/Hex Fiend.app/Contents/Library/LaunchServices/com.ridiculousfish.HexFiend.PrivilegedHelper, reason 3.

This behavior only seemed to start from Hex Fiend 2.9 onwards (it works fine in 2.8). The setup of SMPrivilegedExecutables key-value in the info.plist of the main app and embedded in the executable of the helper tool seems identical between the working and broken version, so I'm really not sure what could be the cause.
Since you've previously mentioned that the privilege-escalation part is hacky and not something you're committed to, feel free to close this as a wontfix if needed – I just wanted to document the issue for the future.

Thanks for the details. It's probably related to #257. I'm going to close this out for that thread.


[Bug] Incorrect assertion failure in polynomial degree function due to leading coefficient equal to 0 after DensePolynomial subtraction operation

Incorrect assertion failure in polynomial degree function due to leading coefficient equal to 0 after DensePolynomial subtraction operation
Author(s): Ronan Thoraval from @Fuzzinglabs
Date: 24/11/2023
Executive Summary
Call the degree() function on a polynomial with a leading coefficient equal to 0 leads to the failure of an assertion which checks that the first coefficient is not 0 (except if the polynomial is zero), and so panics.
We can have such a polynomial by subtracting two different polynomials with the same degree and the same leading coefficient. We can't create directly this polynomial with the from_coefficients_vec() function, because it removes all the leading zeros, which is not necessarily done with the subtraction (and others) operation(s).
Vulnerability Details

Severity: High

Environment

Compiler Version: cargo 1.73.0 (9c4383fb5 2023-08-26)
Distro Version: Ubuntu 23.04

Steps to Reproduce
use rand::Rng;
use snarkvm_algorithms::fft::polynomial::*;
use snarkvm_curves::bls12_377::Fr;
use snarkvm_utilities::rand::TestRng;
use snarkvm_fields::Zero;

fn main(){

    let rng = &mut TestRng::default();

    let p1 = DensePolynomial::<Fr>::rand(rng.gen_range(1..10), rng);

    // p2 = leading_coeff_p1 * x^deg_p1
    let mut coeffs = vec![Fr::zero(); p1.degree()+1];
    coeffs[p1.degree()] = *p1.coeffs().last().unwrap();
    let p2 = DensePolynomial::<Fr>::from_coefficients_vec(coeffs) ;

    let p = &p1 - &p2;

    let _ = p.degree();
}

Root Cause Analysis
We constructed a polynomial $p$ of degree $d$ represented by the vector $(c_0,...,c_{d-1},0)$, with $c_i$ the coefficient of $x^i$.
The assert instruction checks that the leading coefficient of the polynomial is not 0, which is the case in our situation, and so it panics.
The key point is that there is the possibility to construct a polynomial with a leading coefficient equal to 0, which is supposedly not possible.
Detailed Behavior
thread 'main' panicked at /home/ronan/Desktop/Projects/ALEO/snarkVM/algorithms/src/fft/polynomial/dense.rs:92:13:
assertion failed: self.coeffs.last().map_or(false, |coeff| !coeff.is_zero())
stack backtrace:
   0: rust_begin_unwind
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5
   1: core::panicking::panic_fmt
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14
   2: core::panicking::panic
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:117:5
   3: snarkvm_algorithms::fft::polynomial::dense::DensePolynomial<F>::degree
             at /home/ronan/Desktop/Projects/ALEO/snarkVM/algorithms/src/fft/polynomial/dense.rs:92:13
   4: tests::polynomial_null_degree
             at ./src/main.rs:43:13
   5: tests::main
             at ./src/main.rs:461:5
   6: core::ops::function::FnOnce::call_once
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:250:5

Recommendations

Remove the assert and replace it with a line that counts the number of leading zeros. Then at the following line, subtract this count:

From
} else {
            assert!(self.coeffs.last().map_or(false, |coeff| !coeff.is_zero()));
            self.coeffs.len() - 1
        }

To
} else {
            let num_leading_zeros = self.coeffs.iter().rev().take_while(|c| c.is_zero()).count();
            self.coeffs.len() - 1 - num_leading_zeros
        }

In the operation functions, remove the leading zeros at the end (like done in the last else of the sub operation).

I tried reproducing it with 0f324520e22d9af9d855be41893355f41a1efc63 (current testnet3), but I was unable to trigger a panic in both release and debug modes; it's likely that it has recently been fixed.

Yes you are right, the problem has already been fixed !


Feature: Literal casting

As the feature name implies, allows casting of literals to other types.

Added the tests. Two things:

not sure if I should actually implement the TODO with "?" in it
the TODOs with "!" in them need to be fixed in the hierarchy as they require inverse conversion functions

[] figure out how to cast in opposite direction of hierarchy


⚠ Cioos Pacific ERDDAP has degraded performance

In 270116c, Cioos Pacific ERDDAP (https://data.cioospacific.ca/erddap/index.html) experienced degraded performance:

HTTP code: 503
Response time: 9652 ms

Resolved: Cioos Pacific ERDDAP performance has improved in 15f4890.


added morpgOne relation to Grid

add check on Relations\MorphToMany instance in handleRelationColumn method of Grid class.

Thanks!


Update request: zed-editor 0.154.3 → 0.155.2

Package name: zed-editor
Latest released version: 0.155.2

Current version on the unstable channel: 0.154.3
Current version on the stable/release channel: not in stable yet

[x] Checked the nixpkgs pull requests

Notify maintainers
@GaetanLepage @niklaskorz

Note for maintainers: Please tag this issue in your PR.

Add a :+1: reaction to issues you find important.

i'm willing to bump the version

i'm willing to bump the version

Sure, go for it.
It is no more complicated than running nix-update --commit zed-editor.

The package was already updated, please look for merged PRs next time: https://github.com/NixOS/nixpkgs/pull/345995

oops; super sorry about that. This is my first time trying to package something.

To be fair, it's a common confusion that merged PRs take a few days to land in unstable.


proxy

Can we add proxy configuration (with a link to .pac file, for example) in bot communication to avoid some contries blocking policy?

@SkyQuant we do not provide help on GitHub. If you have trouble with library please follow issue template
If you need to chat, join our group chat.


error: connect EHOSTUNREACH

Hi I am new here. What should I do with this error?
I have 3 same version of aircon and only have 1 IP address that output this error.

@xinruijr Perhaps your air conditioner's firmware version is not 2.8.0?


rust client  windows 获取硬盘容量有问题

client版本：v1.7.1

pc上有两个硬盘，只能获取到c盘的硬盘容量

./stat_client --sys-info
sys id: 9fb23ca8eb7c501a39239f0e8443fb62
sys info: {"name":"h1","version":"1.7.1","os_name":"windows","os_arch":"x86_64","os_family":"windows","os_release":"Windows Server 2022 Datacenter","kernel_version":"20348","cpu_num":8,"cpu_brand":"AMD EPYC 7T83 64-Core Processor                ","cpu_vender_id":"AuthenticAMD","host_name":"iZ8f9vzjx87lp0Z"}
=> disks:
NTFS<PHONE_NUMBER>6/107372081152 B      removable:false mounted on:C:
NTFS<PHONE_NUMBER>68/805304262656 B     removable:false mounted on:D:
=> networks:
以太网:         1858/2482 B
=> components:
=> system:
total memory:<PHONE_NUMBER>4 bytes
used memory :<PHONE_NUMBER>0 bytes
avai memory :<PHONE_NUMBER> bytes
free memory :<PHONE_NUMBER> bytes
total swap  :<PHONE_NUMBER> bytes
used swap   : 0 bytes
System name:             Some("Windows")
System kernel version:   Some("20348")
System OS version:       Some("10 (20348)")
System host name:        Some("iZ8f9vzjx87lp0Z")
one minute: 0.00, five minutes: 0.00, fifteen minutes: 0.00
CPU Num: 8
CPU Brand: AMD EPYC 7T83 64-Core Processor
CPU VerderId: AuthenticAMD

=> disks:
                NTFS<PHONE_NUMBER>6/107372081152 B      removable:false mounted on:C:\
                NTFS<PHONE_NUMBER>68/805304262656 B     removable:false mounted on:D:\

so seems that D or more drivers is not listed.


clean: remove unused wires even with (* init *) on

(* init *) is not guaranteed to be attached to a DFF cell's Q port. Thus when that DFF gets blown away (e.g. by abc9 -dff) then it would desirable to also remove unused wires -- even with (* init *) on -- that get left behind.

By convention, the init attribute should be removed when the DFF is removed, see for example opt_rmdff. This allows us to check for unhandled init attributes in the check pass. By just removing any dangling init attribute, we are effectively getting rid of that check in some scenarios.

My tentative argument was that if a wire is completely unused and would otherwise (if it did not have (* init *) be optimised away, does that still count as an unhandled dangling init attribute?
I was hoping to rely on clean to remove such signals, since it seems a little wasteful for abc9 to build a SigMap just for the purpose of handling (* init *).
Another solution I think would work is to introduce an invariant that (* init *) must be attached to the $_DFF*.Q wire; the following benchmark demonstrates when this doesn't happen:
read_verilog -icells <<EOT
module top(input clk, output public);
(* init = 1'b1 *) wire private;
$_DFF_P_ dff (.C(clk), .D(1'b1), .Q(private));
assign public = private;
endmodule
EOT
rename -hide w:private
clean
dump

as it prefers the public name over the private name even though the private one has an (* init *) attribute on.


Clarify "scaffolding PWA project" reusable component

Clarify supported Node.js versions in <ScaffoldingPWAProject />.

thx ❤ , I just copy/paste the sentence from vite docs: https://vitejs.dev/guide/#scaffolding-your-first-vite-project

Oh, I see. On a side note, that link currently says "Vite requires Node.js version 18+ or 20+." but I didn't use that because I think it's confusing ("how can it be 18 or above and 20 or above?"). I understand what it means, but I think "18.x.x or 20+" is clearer.

I'm fine with both


Add link to the GitHub Pages deployment

It would be nice to add the link to the GitHub Pages deployment as part of the repo home page so it's easier to try it out: https://emscripten-forge.github.io/sample-python-repl/

cc @DerThorsten since I don't have the rights to do it

fyi, added you to emscripten-forge

done! and thanks!

Nice, thanks!


[rtl] Deny no-match X access in M-Mode while MML=1

In ePMP spec, it specifies as:
Executing code with Machine mode privileges is only possible from memory
regions with a matching Mmode-only rule or a locked Shared-Region rule
with executable privileges. Executing code from a region without a
matching rule or with a matching S/U-mode-only rule is denied.

This change provides that.
Resolves #1736
Signed-off-by: Canberk Topal<EMAIL_ADDRESS>

After looking at this further it is a rule that makes perfect sense. The idea being that during boot/initialization you setup all of the code regions you want M mode to be able to execute. Once done there's no reason to execute outside of that so this provides you with extra protection without needing some catch all rule to fill in the gaps.
There's also this:

Adding a rule with executable privileges that either is M-mode-only or a locked Shared-Region is not possible and such pmpcfg writes are ignored, leaving pmpcfg unchanged. This restriction can be temporarily lifted e.g. during the boot process, by setting mseccfg.RLB

Which is not implemented either. I'm a bit unclear on what it's getting at with the 'temporarily lifted' bit. It doesn't seem to be explicitly saying setting mseccfg.RLB bypasses this requirement but rather it should be clear that setting mseccfg.RLB bypasses the requirement from some other part of the specification and it's just mentioned as an example here.
I think these things must have been late additions, perhaps I just managed to skip over them during my initial implementation. Still an encouraging sign our DV work has already found our lack of compliance with one of the points.

I think these things must have been late additions, perhaps I just managed to skip over them during my initial implementation. Still an encouraging sign our DV work has already found our lack of compliance with one of the points.

Nope, they were in the original proposal document I used for the implementation. I somehow missed them!

I'm a bit unclear on what it's getting at with the 'temporarily lifted' bit. It doesn't seem to be explicitly saying setting mseccfg.RLB bypasses this requirement but rather it should be clear that setting mseccfg.RLB bypasses the requirement from some other part of the specification and it's just mentioned as an example here.

Answering my own question it comes down the comma. What it's saying is setting mseccfg.RLB bypasses this requirement which you may wish to do temporarily, e.g. during the boot process.


New content: Skip Link

Issue Description:
A skip link is a crucial accessibility feature for websites. It allows users of assistive technologies to skip repetitive content and navigate directly to the main content of a web page. Wagtail CMS currently supports this feature, but there is no documentation to guide users.

Why Documentation is needed:

Developers and content creators may not be aware of this feature.
Some that are aware of the feature may struggle  access it.

Solution:
I will work on a documentation that would  provide a step-by-step instructions on how to find and use the skip link.

Thanks @FanyamTerkuma, this all looks solid.


Can we avoid having to post messages ourselves?

This would be much preferable to the hack we currently use.
There is the ChatMessageRender hook.
One issue is that the text is scanned for chat commands, and our /alias command clashes with that.
So maybe we do the text replacement in ChatMessage, and trigger calculating the response in ChatMessageRender.
How do we know if the text contained an unkenny command?
Maybe we need to rethink if we really want to use the /alias Syntax. We are about to release breaking changes, so now would be the time to do that.

I want to remove the chat command feature for the following reasons:

The input text is not recognised as valid input by FoundryVTT. That means that we have to intervene and replace stuff quite early on.
Because of this, we have to use a hack and post the message in the chat ourselves.
It is not seen from the posted text that this addresses an unkenny actor. The @-Syntax on the other hand is very straightforward.

Hjep, getting more complicated again. To post a message, we need to call the processMessage function on ChatLog. For that, we need to get the ChatLog. And as this function only takes the message as argument, setting the speaker will be a small challenge. Maybe we can work with a hook.

Kinda hacky but works now. In triggerResponse now creates a new ChatLog().processMessage("#TeamEmilia" + JSON.stringify(chatData)), where chatData encodes response + message type + actor name. #TeamEmilia is used as a stop word for triggering logic in our overridden version of ChatMessage._preCreate.


implement  '<=>' for Bool

I sometimes wish to sort by boolean, simplified example:
a = [1,2,3]
a.sort_by{|n| n == 3}
today it yields:  undefined method '<=>' for Bool
Consider this a feature request.  Thanks!

Just wondering what the use case for this would be?
Would the partition method be a way to separate the items based on the boolean expression?

@rdp you can monkey-patch Bool urself:
struct Bool
  def <=>(other)
    self == other ? 0 : -1
  end
end

Nice! Yeah partition would work, in my case I just want to send elements with a certain attribute to the "end" or "beginning" of an array.  @Sija thanks, yeah having it in core would be kind... :)

Boolean doesn't have a natural ordering... Is true less than false? No, it's not. I think this is a terrible idea.
You're using sort_by anyway, you can just use sort_by { |n| n == 3 ? -1 : 1 } or the other way around if you want the other sort direction.

It's not hard to imagine "some arbitrary order" (true == 1, false == 1) by default and just let that (thus avoiding the ternary's) but if it's too contentious hopefully people will be able to find this and just use Sija's work around now.  Thanks all!

OK I had some difficulty getting the snippet above to work, but this seems to do the trick, instead of just clumping "trues and falses together" regardless of which one is which:
struct Bool
  def <=>(other : Bool)
    if self == other
      return 0
    elsif self
      return 1
    else
      return -1
    end
  end
end


[derive] Make output tests insensitive to syntax

Our previous output tests asserted on the exact token stream, which is sensitive to subtle differences like > > vs >> in type signatures. With this change, we instead test for string equality after performing a pretty-printing pass, which has the effect of normalizing most such differences, thus making our tests more robust and less sensitive to these inconsequential syntax details.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 88.98%. Comparing base (ba8bf1c) to head (96095cb).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1711   +/-   ##
=======================================
  Coverage   88.98%   88.98%
=======================================
  Files          16       16
  Lines        5830     5830
=======================================
  Hits         5188     5188
  Misses        642      642

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.