How to add ASR

I've been trying a while to get streaming diarization + recognition working together. I've failed trying many different strategies for streaming wav to pyannote. I started with an example that did this for wav files and wound up here. Without yet knowing/being accustomed to rx, my latest attempt:
import random
import time
from queue import SimpleQueue
from typing import Tuple, Text, Optional, Iterable

import sounddevice as sd
import torch
from einops import rearrange
from pyannote.audio.core.io import Audio, AudioFile
from pyannote.core import SlidingWindowFeature, SlidingWindow
from rx.subject import Subject
from pyannote.audio import Pipeline
from transformers import pipeline

class AudioSource:
    """Represents a source of audio that can start streaming via the `stream` property.

    Parameters
    ----------
    uri: Text
        Unique identifier of the audio source.
    sample_rate: int
        Sample rate of the audio source.
    """
    def __init__(self, uri: Text, sample_rate: int):
        self.uri = uri
        self.sample_rate = sample_rate
        self.stream = Subject()

    @property
    def is_regular(self) -> bool:
        """Whether the stream is regular. Defaults to False.
        A regular stream always yields the same amount of samples per event.
        """
        return False

    @property
    def duration(self) -> Optional[float]:
        """The duration of the stream if known. Defaults to None (unknown duration)"""
        return None

    def read(self):
        """Start reading the source and yielding samples through the stream"""
        raise NotImplementedError

class AudioFileReader:
    """Represents a method for reading an audio file.

    Parameters
    ----------
    sample_rate: int
        Sample rate of the audio file.
    """
    def __init__(self, sample_rate: int):
        self.audio = Audio(sample_rate=sample_rate, mono=True)
        self.resolution = 1 / sample_rate

    @property
    def sample_rate(self) -> int:
        return self.audio.sample_rate

    @property
    def is_regular(self) -> bool:
        """Whether the reading is regular. Defaults to False.
        A regular reading method always yields the same amount of samples."""
        return False

    def get_duration(self, file: AudioFile) -> float:
        return self.audio.get_duration(file)

    def iterate(self, file: AudioFile) -> Iterable[SlidingWindowFeature]:
        """Return an iterable over the file's samples"""
        raise NotImplementedError

class RegularAudioFileReader(AudioFileReader):
    """Reads a file always yielding the same number of samples with a given step.

    Parameters
    ----------
    sample_rate: int
        Sample rate of the audio file.
    window_duration: float
        Duration of each chunk of samples (window) in seconds.
    step_duration: float
        Step duration between chunks in seconds.
    """
    def __init__(
        self,
        sample_rate: int,
        window_duration: float,
        step_duration: float,
    ):
        super().__init__(sample_rate)
        self.window_duration = window_duration
        self.step_duration = step_duration
        self.window_samples = int(round(self.window_duration * self.sample_rate))
        self.step_samples = int(round(self.step_duration * self.sample_rate))

    @property
    def is_regular(self) -> bool:
        return True

    def iterate(self, file: AudioFile) -> Iterable[SlidingWindowFeature]:
        waveform, _ = self.audio(file)
        chunks = rearrange(
            waveform.unfold(1, self.window_samples, self.step_samples),
            "channel chunk frame -> chunk channel frame",
        ).numpy()
        for i, chunk in enumerate(chunks):
            w = SlidingWindow(
                start=i * self.step_duration,
                duration=self.resolution,
                step=self.resolution
            )
            yield SlidingWindowFeature(chunk.T, w)

class IrregularAudioFileReader(AudioFileReader):
    """Reads an audio file yielding a different number of non-overlapping samples in each event.
    This class is useful to simulate how a system would work in unreliable reading conditions.

    Parameters
    ----------
    sample_rate: int
        Sample rate of the audio file.
    refresh_rate_range: (float, float)
        Duration range within which to determine the number of samples to yield (in seconds).
    simulate_delay: bool
        Whether to simulate that the samples are being read in real time before they are yielded.
        Defaults to False (no delay).
    """
    def __init__(
        self,
        sample_rate: int,
        refresh_rate_range: Tuple[float, float],
        simulate_delay: bool = False,
    ):
        super().__init__(sample_rate)
        self.start, self.end = refresh_rate_range
        self.delay = simulate_delay

    def iterate(self, file: AudioFile) -> Iterable[SlidingWindowFeature]:
        waveform, _ = self.audio(file)
        total_samples = waveform.shape[1]
        i = 0
        while i < total_samples:
            rnd_duration = random.uniform(self.start, self.end)
            if self.delay:
                time.sleep(rnd_duration)
            num_samples = int(round(rnd_duration * self.sample_rate))
            last_i = i
            i += num_samples
            yield waveform[:, last_i:i]

class FileAudioSource(AudioSource):
    """Represents an audio source tied to a file.

    Parameters
    ----------
    file: AudioFile
        The file to stream.
    uri: Text
        Unique identifier of the audio source.
    reader: AudioFileReader
        Determines how the file will be read.
    """
    def __init__(
        self,
        file: AudioFile,
        uri: Text,
        reader: AudioFileReader
    ):
        super().__init__(uri, reader.sample_rate)
        self.reader = reader
        self._duration = self.reader.get_duration(file)
        self.file = file

    @property
    def is_regular(self) -> bool:
        """The regularity depends on the reader"""
        return self.reader.is_regular

    @property
    def duration(self) -> Optional[float]:
        """The duration of a file is known"""
        return self._duration

    def read(self):
        """Send each chunk of samples through the stream"""
        for waveform in self.reader.iterate(self.file):
            try:
                self.stream.on_next(waveform)
            except Exception as e:
                self.stream.on_error(e)
        self.stream.on_completed()

class MicrophoneAudioSource(AudioSource):
    """Represents an audio source tied to the default microphone available"""

    def __init__(self, sample_rate: int):
        super().__init__("live_recording", sample_rate)
        self.block_size = 1024
        self.mic_stream = sd.InputStream(
            channels=1,
            samplerate=sample_rate,
            latency=0,
            blocksize=self.block_size,
            callback=self._read_callback
        )
        self.queue = SimpleQueue()

    def _read_callback(self, samples, *args):
        self.queue.put_nowait(samples[:, [0]].T)

    def read(self):
        sttapi = STTAPI()
        self.mic_stream.start()
        while self.mic_stream:
            try:
                mic_next = self.queue.get()
                print(mic_next)
                mic_data = torch.from_numpy(mic_next)
                diarized, full_text = sttapi.segmentation({
                    "waveform": mic_data,
                    "sample_rate": 16000
                })
                print(diarized, full_text)
            except Exception as e:
                print(e) # FIXME: Expected more than 1 spatial element when training, got input size torch.Size([1, 60, 1])
                self.stream.on_error(e)
                break
        self.stream.on_completed()

class STTAPI:
    def __init__(self,):
        self.asr = pipeline(
            "automatic-speech-recognition",
            model="facebook/wav2vec2-large-960h-lv60-self",
            feature_extractor="facebook/wav2vec2-large-960h-lv60-self",
        )
        self.speaker_segmentation = Pipeline.from_pretrained("pyannote/speaker-segmentation")

    def segmentation(self, audio):
        speaker_output = self.speaker_segmentation(audio)
        text_output = self.asr(audio, return_timestamps="word")
        full_text = text_output['text'].lower()
        chunks = text_output['chunks']

        diarized_output = ""
        i = 0
        for turn, _, speaker in speaker_output.itertracks(yield_label=True):
            diarized = ""
            while i < len(chunks) and chunks[i]['timestamp'][1] <= turn.end:
                diarized += chunks[i]['text'].lower() + ' '
                i += 1

            if diarized != "":
                diarized_output += "{}: ''{}'' from {:.3f}-{:.3f}\n".format(speaker, diarized, turn.start, turn.end)

        return diarized_output, full_text

def sound_device_info(device_id=None, in_or_out: str = 'input'):
    return sd.query_devices(device_id, in_or_out)

def main():
    input_device = sound_device_info()
    samplerate = int(input_device['default_samplerate'])
    mic = MicrophoneAudioSource(samplerate)
    mic.read()

if __name__ == '__main__':
    main()

for which I get Expected more than 1 spatial element when training, got input size torch.Size([1, 60, 1]). It maybe out of scope here but asking anywhere else will be useless. If anyone is so inclined, it would be much appreciated.

Hi @bitnom, can you post the full stacktrace of the error? Are you sure that mic_data has the correct shape that the segmentation and/or ASR model is expecting?

@bitnom hi have you solved this, I am trying same thing. Any suggestion will be useful. I am trying to using Azure Recognise along with it


[BUG]: VTT2ASS error when converting subtitle.

Program version
4.7.1
Operating System
Windows
Type
GUI
Service
Hidive
Command used
NIL
Show ID
1045
Episode
1
Console Output
VTT2ASS subtitle conversion displays the error:

Subtitle Downloaded: https://dve-subtitles.imggaming.com/537894/818597/vtt/subtitle-en-US-246613-1712172962745.vtt
[ERROR] VTT2ASS: Unknown style: text-decoration:underline
[ERROR] VTT2ASS: Unknown style: text-decoration:underline

The error message seems to refer to this line in the VTT
::cue(.Q2){font-family:"Swis721 BT" ,Arial; font-size:.84em; color:#FFFFFF; text-decoration:underline;}

Additional Information
No response

Fixed in the latest commit of the v5 branch, it will make it's way into the next version

Thank you for the update.
I will close this once it's the new version is out so it won't reemerge.


Fix ci builder

[x] Remove iOS 9.3 simulators for tests to fix broken CI build
[x] Investigate why the build still passes with an error
[ ] Investigate how to enable iOS 9.3 simulator

I asked on https://github.com/travis-ci/docs-travis-ci-com/issues/1106 about 9.3 being gone. I wonder if there's a better place for that.


Add consistency level to request diagnostics

Add consistency level to request diagnostics

@simplynaveen20  - can we also accommodate this request with the current work of diagnostics that you are working on with @aayush3011  ?

Consistency level is already added to the Diagnostics.


[BLOCKCHAIN]Blockchain library comparison: Web3.js vs. Ethers.js

My Resource

Here is the draft below:
Blockchain is a decentralized and distributed digital ledger technology that allows multiple parties to share and maintain a tamper-evident and transparent record of transactions or information.
Web3.js is a JavaScript library that allows developers to interact with the Ethereum blockchain using HTTP or WebSocket connections. It provides a set of APIs that enable developers to build decentralized applications (dApps) that can read and write to the blockchain.
Ethers.js is a modern JavaScript library that provides a simple and intuitive way to interact with the Ethereum blockchain. It is designed to be easy to use, fast, and secure, and provides a set of APIs for developers to interact with the Ethereum blockchain and build decentralized applications (dApps).
Goal
There are several blockchain libraries available for developers who want to build applications or systems that utilize blockchain technology but i will be talking about web3.js and Ethers.js which is the two popular ETH libaries.
Table of Content

Introduction

Javascript for Blockchain

What is Ethers.js

Features of Ethers.js

Pros and Cons of Ethers.js

Modules of Ethers.js

What is Web3.js

Features of Web3.js

Prons and Cons of Web3.js

Modules of Web3.js

Comparison of Web3.js and Ethers.js
-Developer experience
-Popularity
-Documentation
-Ease of use
-Performance
-Maintenance

Conclusion

@auxesis @jstangroome
Here is the link to the topic:
https://www.dropbox.com/scl/fi/e7ut4qf1nflvrq14kjt9k/Blockchain-library-comparison-Web3.js-vs.-Ethers.js.paper?dl=0&rlkey=rauspdoelfoda81v0w7jbedqf


updates to microsoft build mcirosite, add event to resources list

Description
https://github.com/pulumi/marketing/issues/1030

booth corrected
added event to resources
June Azure workshops added to microsite
Happy Hour registration link added to microsite

@shughes26 - I just realized the meta image is now wrong (due to the change in booth number)
can we use this?


Body parts flying off in v0.0.7a

When using a custom skin in version 0.0.7a, if you attempt to offset a base body part, such as the head or arms, any offset in the x or y planes is constantly applied.  This results in the, albeit hilarious, effect of said part flying off into the sunset.  This persists even if you change the skin.
Steps to reproduce:

Load up a modpack with CustomPlayerModels on Forge v35.1.28
Create a new custom skin on 0.0.7a
Attempt to offset a base body part in the x or z planes
Say goodbye to the part

After a bit of testing, I've noticed a few peculiarities:

Arms are working as intended i.e. not flying away.
Legs only move constantly in the x direction.

Hope that helps.


Feature Request:  Support Shift-F10 to invoke right-click context menu

Apologies if this is a duplicate.  I searched and couldn't find an existing issue.
Unfortunately, macOS doesn't natively support invoking the right-click context menu from the keyboard as is possible with the menu keyboard button or SHIFT-F10 on Windows and Linux.   The solution associated with the accessibility feature to invoke CTRL-5(number pad) is insufficient because it opens the context menu where the mouse pointer is, not where the text cursor is.
However, third party applications such as Eclipse and IntelliJ on macOS do support this feature with custom application code.   Eclipse implemented such functionality in this bugzilla:  https://bugs.eclipse.org/bugs/show_bug.cgi?id=219305
This feature request covers adding this functionality to MacVim via the SHIFT-F10 keyboard combination.  To be clear, the behaviour is to open the context menu at the keyboard arrow key cursor location, as opposed to the mouse cursor location.   For example, this means when moving a keyboard cursor over a misspelled word, the context menu will display with, among other things, suggested spelling corrections for that word, irrespective of where the mouse pointer happens to be.

You can bind a key to show a PopUp menu. This is well documented feature in Vim.
:[niv]map <Shift-10> :popup PopUp


fix: collect_columns quadratic complexity

Which issue does this PR close?
-
Rationale for this change
Fix accidental $O(n^2)$ in collect_columns.
What changes are included in this PR?
There are the following ways to insert a clone into a hash set:

clone before check: Basically set.insert(x.clone()). That's rather expensive if you have a lot of duplicates.
iter & clone: That's what we do right now, but that's in $O(n^2)$.
check & insert: Basically if !set.contains(x) {set.insert(x.clone())} That requires two hash probes though.
entry-based API: Sadly the stdlib doesn't really offer any handle/entry-based APIs yet (see https://github.com/rust-lang/rust/issues/60896 ), but hashbrown does, so we can use the nice set.get_or_insert_owned(x) which will only clone the reference if it doesn't exists yet and only hashes once.

We now use the last approach.
Are these changes tested?
Functional tests still pass, performance change is only visible for large inputs (like 100+ columns).
Are there any user-facing changes?
:warning: Breaking change: collect_columns now returns hashbrown::HashSet instead of std::collections::HashSet!

I ran the benchmarks and don't see any negative (or positive) direct effect
Details

crepererum/fix_collect_columns_o2
Benchmarking logical_select_one_from_700: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.5s, enable flat sampling, or reduce sample count to 60.
logical_select_one_from_700
                        time:   [1.0805 ms 1.0827 ms 1.0849 ms]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

physical_select_one_from_700
                        time:   [3.4578 ms 3.4601 ms 3.4625 ms]
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild

logical_select_all_from_1000
                        time:   [17.965 ms 17.977 ms 17.990 ms]
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

physical_select_all_from_1000
                        time:   [43.469 ms 43.504 ms 43.547 ms]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

Then main
Benchmarking logical_select_one_from_700: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.5s, enable flat sampling, or reduce sample count to 60.
logical_select_one_from_700
                        time:   [1.0891 ms 1.0908 ms 1.0926 ms]
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  3 (3.00%) high mild

physical_select_one_from_700
                        time:   [3.5458 ms 3.5485 ms 3.5515 ms]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

logical_select_all_from_1000
                        time:   [18.000 ms 18.012 ms 18.025 ms]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

physical_select_all_from_1000
                        time:   [43.609 ms 43.639 ms 43.671 ms]
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe


Joining a query with inner join + groupBy results in error

Module: quill-sql
Failing - https://scastie.scala-lang.org/mentegy/ZLficzsLRBqdPkWDE6Sc8Q
Working (with a workaround)  - https://scastie.scala-lang.org/mentegy/p2lpqYJlRsCV68PqxhXAiQ
val q4 = quote {
  query[Teams]
  .join(q3)
  .on(...)
  .groupBy {
    case (teams, inner) => teams.id
  }
  .map {
    case (id, q) => Output(id, q.size)
  }
}

// this fails
val q5 = quote {
  query[Teams]
  .join(q4)
  .on { case (teams, inner) => teams.id == inner.teamId }
}

Expected behavior
Produce a correct sql
Actual behavior
exception during macro expansion:
java.lang.IllegalStateException: A `groupBy` clause must be followed by `map`.
	at io.getquill.util.Messages$.fail(Messages.scala:15)
	at io.getquill.context.sql.SqlQuery$.flatten(SqlQuery.scala:143)
	at io.getquill.context.sql.SqlQuery$.flatten(SqlQuery.scala:83)
	at io.getquill.context.sql.SqlQuery$.apply(SqlQuery.scala:76)

Workaround
use implicit join syntax

val q5 = quote {
  for {
    team <- query[Teams]
    inner <- q4
    if team.id == inner.teamId
  } yield (team, inner)
}

@getquill/maintainers

Perhaps it's related to https://github.com/getquill/quill/issues/63

Still happening in quill-jdbc-zio 4.6.1


Re-add Gnosis Chain support

Description
This PR re-adds the missing gnosis chain configuration that was included in version 0.1.20 which got lost at some point as it doesn't exists anymore.
Additional Information

[x] I read the contributing docs (if this is your first contribution)

Thanks for doing this! Going to hold off for now though. I'm working on a way to add a bunch of different chains to the core that is tree-shake-able and easy to maintain.
In the meantime, you can create and use chains in your own project by conforming to the Chain type. Related discussion https://github.com/tmm/wagmi/discussions/370#discussioncomment-2650012


Python 3

What the program do ?
The program writes Hello World in Python 3
In what programming language it is written?
It is written in Python 3

This pull request do not contains files in specific folders. Please fix it and reopen


Add order arg to stackplot

Please confirm that this PR has done the following:

[x] Tests Added
[x] Documentation Added
~Name of contributors Added to AUTHORS.rst~
[x] Description in RELEASE_NOTES.md Added

Description of PR
This PR adds an order arg to the stackplot option. In preparation of that, it makes a few clean-up operations:

pass the IamDataFrame to the plotting library (rather than the data) to keep core.py shorter
deprecate stack_plot() in favor of stackplot() for consistency with Matplotlib
add a page with plotting docs (properly cross-referenced from the IamDataFrame docs)
add an order kwarg to stackplot with the behavior as explained below

The implementation of the ordering is slightly more complicated than necessary, anticipating that this will also be useful for ordering in other plotting functions. This new structure of the function names and docs can then be applied to the other plotting functions later.
The behaviour of the order kwarg:

if not specified, use the order of the run_control()['color'] dictionary to order the stacked levels;
any elements in the data which are not in the dictionary are added alphabetically after
if not specified and nothing relevant defined in the run-control: keep current (alphabetical) behavior
or use the order given by that kwarg if it is a list, dropping all elements in the data that are not in the list

One suggested change - I think 'color' should be passed explicitly as the order if desired, as to not hide plot logic in a configuration file.

One suggested change - I think 'color' should be passed explicitly as the order if desired, as to not hide plot logic in a configuration file.

Disagree for a number of reasons:

The whole point of the run control is to hide logic in a configuration file.
I imagine someone seeing this in a tutorial or code snippet and wondering what order='color' should mean - red before blue?
There is no reason to think that the current approach (alphabetical) is any less surprising to a user than the new default approach (run control plus alphabeting) - as a user, I might expect the order of the input dataframe (when casting to an IamDataFrame) to be the most logical order... At least, the new approach allows to explicitly set the order if the default does not make sense.

The whole point of the run control is to hide logic in a configuration file.
The point of the config file is not to hide logic, but rather to explicitly define configuration in one place. For example, we could add an order section there which can be looked up here so that a single order can be defined once and used in multiple places.

What is suggested here, if I understand, is rather to, by default, infer (implicitly) an ordering parameter through another, arguably unrelated parameter.
In general, we should be explicit instead of implicit.

I imagine someone seeing this in a tutorial or code snippet and wondering what order='color' should mean - red before blue?
Fair point. See above for a more explicit suggestion.

Well... I agree that adding an adding an order option would be more explicit, but on second thought, I don't think it's worth it:

the docstring would change from # levels are ordered based on the `run_control()['color']` dictionary

to# levels are ordered based on the `run_control()['order']` dictionary

it would just mean taking a dictionary of dictionaries and repeating it as dictionary of lists. In the IPCC SR15 context, just 20 extra lines with no additional info - which could just as well be embedded in the order of the dictionaries...

Well not every user may be using their configuration files in the exact same way. Here for example, if we were to pull as is, all plots would change. I think its worth being explicit in this case.

Implemented a separate order feature in the run control per suggestion by @gidden

While testing the ordered stackplot in my real-world use case, I noticed that the ordering was actually overridden by the quick-fix for dealing with data that crosses the zero line. So I fixed that one, too... Not necessarily the prettiest plots, but at least correct.
For illustration of the problem
import pandas as pd
import pyam

TEST_STACKPLOT_DF = pd.DataFrame([
    ['var1', 1, -1, 0],
    ['var2', 1, 1, -2],
    ['var3', -1, 1, 1],
],
    columns=['variable', 2005, 2010, 2015],
)

df = pyam.IamDataFrame(TEST_STACKPLOT_DF, model='model_a', scenario='scen_a', region='World', unit='foo')
df.stackplot(total=True)

An illustration of what was wrong with the previous implementation:

legend not ordered (alphabetically by default)
var1 (green) in 2005 goes from +1 to -1
var3 (orange) goes to -2 in 2015, instead of +1

New implementation:


Cant send a message with an attachment

No mater what i have tried i just cant get this lib to send messages with attachments, some sort of explanation or sample code could be extremely helpfull

By attachment I assume you mean a message attachment, not a file attachment. If the latter, this is no help....
For each attachment you want to attach to a message, create a new Attachment object, let's call it $attachment. Check the constructor for the fields you can set. When sending your message, use the MessageBuilder:
$mb = $client->getMessageBuilder()
            ->setText($messageText)
            ->setChannel($channel);

For each attachment you need to add:
$mb = $mb->addAttachment($attachment)

In my code I have a "send" function that I pass an array of attachments to, and I walk the array to attach them there.
Finally, create a Message object from the builder like so:
$message = $mb->create();

And send your message off:
$client->postMessage($message);

This needs to be documented in the API docs.... took me hours of googling to find this issue that solved my issue. It isn't a standard class implementation and it's frustrating!

@calonmerc There is an example of creating attachments in the README already here.

That example only shows how to add one attachment, but not multiple.

@calonmerc
$builder = $client->getMessageBuilder()
    ->setText('Hello, all!')
    ->setChannel($someChannelObject)

for ($myThingsToAttach as $title => $text) {
    $builder->addAttachment(new Attachment($title, $text));
}

$message = $builder->create();

addAttachment() is just a builder mutator method that adds an attachment to the builder object.

That's my point.... I figured it out AFTER finding this issue, that exact sample needs to be in the docs because it isn't set up as a standard class....
And that code won't work.... the only way I could get it to work would be:
`$builder = $client->getMessageBuilder()
->setText('Hello, all!')
->setChannel($someChannelObject);
for ($myThingsToAttach as $title => $text) {
$builder = $builder->addAttachment(new Attachment($title, $text));
}
$message = $builder->create();`
Notice the loop part, that's the only way it would work in any implementation of the code.

Hope this thread continues to help people, eventually I needed something very specific so I implemented my own message builder class using guzzle for http requests.


packages lock files

Currently the examples have a package-lock.json that is unused as we used jlpm to follow Jupyterlab recommendation. But there is no yarn.lock neither. The question is: it seems logical to remove the npm lock file but should we replace it with a yarn lock file?

I think it's fine for now to not have any lock files.
So we can indeed remove all package-lock.json without adding a yarn.lock.


Spike: Ingestion filtering lifecycle options

What problem does your feature solve?
How should filters handle rule config changes over time, i.e. lifecycle.
Should filter rule changes retro-actively backfill history per changes, etc.
What would you like to see?
Definition of the product expectations for ingestion filtering lifecycle aspect.
HLD for how the filter backfill would work.
What alternatives are there?

spike write-up on filtering life cycle options captured here

@sreuland should we close this?

I think this can go to done, reviewed the Filtering LIfecycle Implications with @jcx120, he has created product feature doc on filtering expectations which references lifecycle aspects and the prior asset/account spikes in the overall filtering story.
I think the next step will be to review the Dev tickets that were drafted up, they are mentioned in  product feature doc on filtering expectations next to the feature they are related to. @2opremio , @ire-and-curses , does this sound ok?

I took a quick look, I think we can go ahead and add them to the sprint :+1:


DCOS-57466 fix(quota): support pods in quota limit

Added a getRole override on Pod struct to support returning the role
value from a Pod using the Pod.spec instead of looking for the role at
the root of the Pod object like it is on the Service object.

Closes DCOS-57466

Testing
See JIRA for steps to reproduce.

Create Group with Quota & use group roles
Create pod inside group
Ensure Pod is not seen as "no limit" (no badges or banners displayed with pod is running in group with quota)

Trade-offs

Dependencies

Screenshots

:tada: This PR is included in version 2.139.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Request: Blog Page Layout

since you have a blog grid, it will be amazing if you hade a blog page layout

Can I take this?

@williamssam If you are not working, I would like to take on this issue?


Is there a good way to extend 45s timeout for starting jobs?

In our deployment, there needs to install some heavy jobs, which need almost 2 mins to start up. But BOSH only sets 45s as the default timeout for starting jobs -> https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/agent_client.rb#L39
Seems this setting is not parameterized, is there a good way to adjust this setting? Thanks.
@jianqiu

@mattcui 45s timeout is not related to amount of time it takes for the job to start. what failures are you actually seeing as part of the deploy?

@cppforlife We saw the following error when jobs take much longer time to start up:
D, [2016-04-26 05:26:34 #24909] [task:78091-checkpoint] DEBUG -- DirectorJobRunner: (0.001116s) COMMIT
D, [2016-04-26 05:26:35 #24909] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:yz-stage0
D, [2016-04-26 05:26:40 #24909] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:yz-stage0
D, [2016-04-26 05:26:45 #24909] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:yz-stage0
D, [2016-04-26 05:26:50 #24909] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:yz-stage0
E, [2016-04-26 05:26:51 #24909] [instance_update(api_worker_dal06/0)] ERROR -- DirectorJobRunner: Error updating instance: #<Bosh::Director::RpcTimeout: Timed out sending `start
' to 45b2363e-ab3a-43bf-9630-a699e4bb7d5c after 45 seconds>
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/agent_client.rb:227:in `block in handle_method'
/var/vcap/packages/ruby/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/agent_client.rb:222:in `handle_method'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/agent_client.rb:291:in `handle_message_with_retry'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/agent_client.rb:326:in `start_task'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/agent_client.rb:302:in `send_message'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/agent_client.rb:82:in `start'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/instance_updater.rb:126:in `start!'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/instance_updater.rb:74:in `block in update_steps'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/instance_updater.rb:88:in `call'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/instance_updater.rb:88:in `block in update'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/instance_updater.rb:87:in `each'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/instance_updater.rb:87:in `update'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/job_updater.rb:94:in `block (2 levels) in update_instance'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh_common-1.3153.0/lib/common/thread_formatter.rb:49:in `with_thread_name'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/job_updater.rb:92:in `block in update_instance'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/event_log.rb:97:in `call'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/event_log.rb:97:in `advance_and_track'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/job_updater.rb:91:in `update_instance'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh-director-1.3153.0/lib/bosh/director/job_updater.rb:85:in `block (2 levels) in update_instances'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh_common-1.3153.0/lib/common/thread_pool.rb:77:in `call'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh_common-1.3153.0/lib/common/thread_pool.rb:77:in `block (2 levels) in create_thread'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh_common-1.3153.0/lib/common/thread_pool.rb:63:in `loop'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/bosh_common-1.3153.0/lib/common/thread_pool.rb:63:in `block in create_thread'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
/var/vcap/packages/director/gem_home/ruby/2.1.0/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'

From the source code, seems that the director waits a response from bosh agent for job startup (a sync command) for 45s.
      result.synchronize do
        while result.empty?
          timeout = timeout_time - Time.now.to_f
          unless timeout > 0
            @nats_rpc.cancel_request(request_id)
            raise RpcTimeout,
              "Timed out sending '#{method_name}' to #{@client_id} " +
                "after #{@timeout} seconds"
          end
          cond.wait(timeout)
        end
      end

If 45s here is not the timeout waiting for the job startup, where can we set a timeout duration for job startup? Thanks.

@cppforlife this problem got more serious, it could be a blocking issue to migrating our environments now. We hope to get support from BOSH community to help us understand the problem. Firstly, we want to know what's the use of this 45s timeout, in what condition, there will trigger this timeout exception? Thanks.
/cc @maximilien @jianqiu

It might be helpful to see /var/vcap/bosh/log/current to know exactly what steps are taking so long for this. Should just be laying down pre-compiled templates and making directories, which shouldn't take long. Perhaps take a look at those, or provide them in anonymized form with full output or just the debug messages.
fwiw, might be similar area of code as https://github.com/cloudfoundry/bosh/issues/1245

@dpb587-pivotal Yes, we checked agent log, we saw the agent got a message from director to start jobs, but we didn't see the agent sent back a response message (may be because jobs are not started within a given time). We always saw "45s timeout" exception in bosh debug log, do you know what kind of problems would trigger this "45s timeout" exception? Thanks.

@mattcui @maximilien can we arrange a debug session or retrieve agent logs? i'd like to understand exactly what's happening on the agent side. this might be related to start being synchronous operation, which means correct thing to do here would be to switch it to async like many other agent ops.

Yeah, what about tonight 6:30p if no resolution.

@cppforlife , @maximilien, @mattcui
Hi Dmitriy,
We found most of the time (45s) was consumed in "monit reload", and did not post "start" action to monit in 45s. But，we could not determine why "monit reload" was blocked,  one possibility is that there are too many monit process (https://github.com/cloudfoundry/bosh/issues/1245), the other is that lots of security jobs failure in the process of starting.
func (m monitJobSupervisor) Reload() error {
var currentIncarnation int
oldIncarnation, err := m.getIncarnation()
if err != nil {
	return bosherr.WrapError(err, "Getting monit incarnation")
}

// Monit process could be started in the same second as `monit reload` runs
// so it's ideal for MaxCheckTries * DelayBetweenCheckTries to be greater than 1 sec
// because monit incarnation id is just a timestamp with 1 sec resolution.
for reloadI := 0; reloadI < m.reloadOptions.MaxTries; reloadI++ {
	// Exit code or output cannot be trusted
	_, _, _, err := m.runner.RunCommand("monit", "reload")
	if err != nil {
		m.logger.Error(monitJobSupervisorLogTag, "Failed to reload monit %s", err.Error())
	}

	for checkI := 0; checkI < m.reloadOptions.MaxCheckTries; checkI++ {
		currentIncarnation, err = m.getIncarnation()
		if err != nil {
			return bosherr.WrapError(err, "Getting monit incarnation")
		}

		// Incarnation id can decrease or increase because
		// monit uses time(...) and system time can be changed
		if oldIncarnation != currentIncarnation {
			return nil
		}

		m.logger.Debug(
			monitJobSupervisorLogTag,
			"Waiting for monit to reload: before=%d after=%d",
			oldIncarnation, currentIncarnation,
		)

		time.Sleep(m.reloadOptions.DelayBetweenCheckTries)
	}
}

return bosherr.Errorf(
	"Failed to reload monit: before=%d after=%d",
	oldIncarnation, currentIncarnation,
)

}

Hello,
I'm facing the same issue. My BOSH deploys very often hit the 45s timeout. Currently I circumvent the problem  with the following script that patches the director and restarts it.
cd /var/vcap/packages/director/gem_home/ruby/2.*.*/gems/bosh-director-1.*.0 || exit 115
patch -p0 <<'END_OF_PATCH'
--- lib/bosh/director/agent_client.rb 2016-05-11 14:13:17.000000000 +0000
+++ lib/bosh/director/agent_client.rb 2016-05-26 10:13:53.971352833 +0000
@@ -36,7 +36,7 @@
       @service_name = service_name
       @client_id = client_id
       @nats_rpc = Config.nats_rpc
-      @timeout = options[:timeout] || 45
+      @timeout = options[:timeout] || 90
       @logger = Config.logger
       @retry_methods = options[:retry_methods] || {}

END_OF_PATCH
/var/vcap/bosh/bin/monit restart director

However I would be happy to find a better way to cure the issue.
In the situation above, what solution did you finally identify?
@cppforlife : how can I know if job start is sync or async in my case? How can I switch it to async?
Thank you for your help.

Hi all,
To answer the original question about extending the timeout, it is not configurable, and this is intentional since the problem is likely symptomatic of other root cause problems that should be addressed.  There have been improvements made to how the Agent interacts with monit since the issue was opened which might address those root causes.  If this issue is still a challenge, please give the latest stemcells a shot (or a stemcell built with the latest Agent code if you're using a stemcell not maintained by the core BOSH maintainers).  We'll close this issue out due to inactivity, please feel free to re-open if the issue persists with the latest Agent code.
Thanks,
Amit
/cc @jfmyers9 @mfine30 @dpb587


CI Testing to Ensure Backward Compatibility with Previous Terra Versions

What is the expected behavior?
Currently there is no testing to ensure that changes made in master do not break previous versions of terra code, e.g. notebooks written for previous releases.
It may be beneficial to add tests to the Travis CI framework that run notebooks for previous versions of terra using the latest master to check for backward compatibility.  As will other tests, this should probably only be run upon merging.

We are not shipping notebooks with terra anymore. Closing this issue.

Notebooks were just an example, hence the e.g.  The point of this was to have a way to be able to report which versions of Qiskit are still compatible with the IQX api. However if this can be accomplished some other way then I can cool with that.

Should that be something that the IQX api might check with each release?

oh.. I made my comment before seeing https://github.com/Qiskit/qiskit/issues/757


Add responsiveness in profile page

Issue solved: #600

@narayan954 You can check now
https://github.com/narayan954/dummygram/assets/94003231/0834f938-da64-46bd-96a9-659f1268f65c


KubernetesDiscoveryClientAutoConfiguration will not create non-reactive DiscoveryClient Bean if WebClient is on the classpath

Describe the bug
in Spring Cloud 2022.0.3, the non-reactive discovery client bean will not be created or autoconfigured if WebClient is on the classpath. This is due to the use of @ConditionalOnMissingClass("org.springframework.web.reactive.function.client.WebClient") .
I'm not sure what the official position is of Spring on this, but the existence of WebClient on the classpath is not a good indicator on if the app itself is reactive or not. More and more dependent libraries are preferring to use WebClient under the hood even for non-reactive (or libraries that support both), as well as Spring 6.x and Spring Boot 3.x seem to have gone down a path of correcting issues with the co-existence of spring-mvc and spring-webflux components on the classpath.
In my opinion, the decision of which DiscoveryClient bean to instantiate should be based on whether the app itself is reactive or not, rather than what is on the classpath.
Would the following annotations help in this regard?
@ConditionalOnWebApplication(type=ConditionalOnWebApplication.Type.REACTIVE)
@ConditionalOnWebApplication(type=ConditionalOnWebApplication.Type.SERVLET)
Sample
I cannot provide a sample at this time. But a simple non-reactive spring-mvc app that also has webflux on the classpath will not create the non-reactive DiscoveryClient, and will not be able to use the ReactiveDiscoveryClient that does get created in this case.

Yes that’s what I meant 😉


Go-to-definition still does not work inside some macros

#202 improved the behaviour of go-to-definition inside macro invocations, but in some cases it stil does not work.
Here is a minimal example:
#define VALIDATE_TYPE(x) (void)x;

#define ASSERT(expr)       \
  do {                     \
    VALIDATE_TYPE(expr);   \
    if (!expr);            \
  } while (false)

bool waldo();

void foo() {
  ASSERT(wa^ldo());
}

I guess the issue is related to the fact that the macro argument appears in the expansion multiple times.
However, all occurrences in the expansion target the same declaration, so in principle this should still work.

It looks like SelectionTree selects the individual nodes in the expansion as desired, but if there are multiple ones, their commonAncestor() will most likely be a node that doesn't target anything (like a CompoundStmt).

Proposed fix


update us-wa-pacific

added one conform and tidied up another.
thanks for the san bernardino addresses.  used them to help a non-profit with a hyperlocal mailer.

Hm, "mail address" might not be the right field to split in us-wa-pacific, I think it’s the postal address of the owner: e.g.:

I think it’s the postal address of the owner

good catch.  the appropriate field appears to be SITUS, but just a heads up, the number and streetname are delimited with a backslash.

You’re right—it’s not great but will probably work!

Some zeroes in SB, but should work.


Fix /collection endpoint accepting Concept iri

Describe the bug
/collection endpoint doesn't raise 404 error when provided a Concept iri but fails internally.
To Reproduce
Steps to reproduce the behavior:

Provide https://api.g.sentier.dev/latest/collection with a Concept iri

Expected behavior
It should raise a 404 error
Additional context
This was identified during development. I will work on it next. I am just raising awarness.

False alarm.


please support non-universal (possibly AOT) version of --mobile flag

I love the fact CLI will soon support back the --mobile however I really don't want to use angular universal.
It would be awesome if we can have a non universal (and possibly AOT) version of  seed cli PWA support,
thanks as always,
Sean

See https://github.com/angular/angular-cli/issues/2228#issuecomment-253053168


can this work with complex numbers?

I did a fork and added tangent function just to test ...  I'm curious to use complex numbers processing ... maybe just tie in x and y for a single operation ?

I'm... not really sure what you mean, where/how complex numbers would be applicable?
I see your fork to add tan as a scale scheduler, and, uh, not sure the goal of that either? Shape of a tangent doesn't really make sense in this context, unless you know something I don't.

I am going to add frequency controls to the scheduling as well...
https://www.naturalspublishing.com/files/published/7px0r964d727hj.pdf
https://www.ventrella.com/mandelbrotgenetics/#:~:text=The%20Mandelbrot%20equation%20is%20z,input%20for%20the%20next%20iteration.

https://www.jstor.org/stable/3211856?read-now=1#page_scan_tab_contents

If you manage to get something useful and functional working, feel free to PR it.


Notification id in OnNotification doesn't match one in database

Description
Nakama returns wrong id for notification in OnNotification callback
Steps to Reproduce
In Unity I execute this code:

Client receives "Want to chat" request
Client receives "Friend request"

 _client.OnNotification = (INNotification notification) =>
{
    Log.Info(">>>>>Notification id: " + notification.Id);

    var message = NNotificationsListMessage.Default(100);

    _client.Send(message, set =>
    {
        foreach (var result in set.Results)
        {
            Log.Info("+++Notification id: " + result.Id);
        }
    }, error =>
    {
        Log.ErrF("Error listing notifications: Code [{0}] : Message [{1}]", error.Code, error.Message);
    });
};

I get this
response
You can see that after first notification I get OnNotification callback with notification id rklj... but after I list notifications from server the id is sT4J...
Expected Result
Identical ids in OnNotification and after lising notifications from server
Actual Result
Different ids. Though not always
Context
Unity client
Your Environment
"Node","server":"nakama-8bf8","name":"nakama-8bf8","version":"1.4.0+93669f8","runtime":"go1.9.2"
"Data directory","server":"nakama-8bf8","path":"/nakama/data"
"Database connections","server":"nakama-8bf8","dsns":["root@cockroachdb:26257"]
"Database information","server":"nakama-8bf8","version":"CockroachDB CCL v1.1.3 (linux amd64, built 2017/11/27 13:59:10, go1.8.3)"

This is fixed in commit 8d208e6.


Show pagination only if the number of items is greater than the items per page

Maybe we should add a check in the twig template whether the number of pagination items is greater than the items per page. We should only show the pagination if we need it because a pagination with only one page makes no sense.

The pagination template is not part of this repository anymore


Allow caching the result of a series of processors

This a proposal for a higher-order cached processor that can wrap a series of processors and cache their results if they successfully process messages batches. The proposed API looks like this:
pipeline:
  processors:
    cached:
      resource: foocache
      key: '${! json("message.id") }'
      ttl: 5m
      processors:
        - resource: expensive_processor_1
        - resource: expensive_processor_2
        - resource: expensive_processor_3

(transcript from Discord)
@disintegrator:

I'm not sure yet how to serialize a []service.MessageBatch to a cache though. is there built-in API for that today or we need to create it?

@Jeffail:

we'd need something custom, there'll be a performance hit with marshalling/unmarshalling so ideally we'd want to do some sort of benchmarking for different options, once it's decided and released we can't change it easily later on

I suspect we need to provide guidance for users to compare the cost of persisting to a cache vs performance cost of the underlying un-cached operation. In my own use cases, this processor's performance will beat out querying BigQuery on every service.MessageBatch and cut BQ costs.
Could the performance be improved by switching to msgpack or similar? I know with msgpack there's nice interop with JSON so jumping between the two representation should be straightforward.

Not sure if switching to msgpack is a good idea. Maybe it's worth trying alternative json implementations (simd), they will have similar performance with msgpack.
Good candidate is https://github.com/bytedance/sonic

TIL. Thanks @packman80!

Yes, i missed this one.


Progress indication for boot sequence

The current boot sequence for WebDash takes between 20-50 seconds to load, depending on connection speed and existing browser cache. It would be useful to have some progress indication for the boot sequence.
One challenge here is that we do not know in advance how many packages will be loaded as that's purely a function of the imports in the Dash app. A good middle ground solution would be to not display percent progress, but simply show which currently loading/loaded packages, until the app is finally done booting.

Pyodide 0.18.0 has the option to set user defined functions for stdout/stderr logging. We could use that to indicate loading progress.


Integration with Templater Folder Templates? enhancement

The creases-in-templates work perfectly when creating new documents with 'Create new note' > 'Insert Template'. But they don't work when templates are automatically applied by Templater's 'Folder Templates' function, which automatically applies a template to new notes created in a certain folder. These contain the creases, but don't fold them.

Having a similar problem

Any news on this?

In fact, when the setting how should creases in templates behave is set to start folded, anytime you insert a template into a note, it causes ALL headings to unfold (not just ones from the incoming template, but folds within the existing note, and even if creases are not used).
This happens whether you're inserting a template using the core Templates plugin or Templater, and it occurs even if you're inserting a template that is just a single character. So there is a bug somewhere.
It's not that big of a deal because it doesn't happen if you change the setting to start unfolded, but it makes this feature unusable sadly. Also I had to spend about 50 minutes figuring out which plugin was causing this. I didn't initially expect it to be creases so I had to bisect the plugins until I realized it was the culprit.
Thanks for making an awesome plugin!


Does not work with anaconda environments

If you run the snyk extension with a project that uses a python interpreter using an anaconda environment it will fail with following error:
[Info] Running "...\snyk-win.exe test h:\Projekte\Projekt1 --json".
[Error] Open Source Security test failed for "h:\Projekte\Projekt1". Required packages missing: nump

Please run `pip install -r requirements.txt`. If the issue persists try again with --skip-unresolved.

Running the snyk-win.exe command manually within the activated anaconda environment (conda activate [env]) will work.

Hi, since GitHub issues are not officially monitored by Snyk support, we will be disabling "Issues" tab on this repository very soon.
Please raise the current issue directly by contact Snyk Support on the web or emailing us at<EMAIL_ADDRESS>


 The issue occurs
Describe the results you received:
root@terry-VirtualBox:/home/terry/docker-node-hello# docker run -d -p 8080:8080 example/docker-node-hello:latest
b8a67d3434d807cc7a5b6edaf3d0b29b1b6ef4b8404170c55cfcc2d121bdda6c
FATA[0000] Error response from daemon: Cannot start container b8a67d3434d807cc7a5b6edaf3d0b29b1b6ef4b8404170c55cfcc2d121bdda6c: [8] System error: exec: "supervisord": executable file not found in $PATH
Describe the results you expected:
Successfully Run
Additional information you deem important (e.g. issue happens only occasionally):
I changed "CMD ["supervisord",-"n"]" to "CMD ["/usr/bin/supervisord","-n"]" in my dockfile and build image again.
When I ran the image,I got the following error message.
Error: statusCode=404 Cannot start container c2416fcf39379712c87402dce0fdd5ae8173e78bdda59ef1557272cb3a2f11bc: [8] System error: exec: "/usr/bin/supervisord": stat /usr/bin/supervisord: no such file or directory
In /usr/bin directory I could find  "supervisord" execute command.
$PATH environment variable that I have checked contained /usr/bin directory.

In /usr/bin directory I could find "supervisord" execute command.

Is that inside or outside of the container? How did you check?
Also, you're running a very old version of Docker, that we no longer maintain (current version is 1.11.1), so even if there's a bug in that version, there won't be a bug fix, unless the issue is still present in the current release.
@tezhao I don't mean to be rude here, but I explained in one of your previous issues that the GitHub issue tracker is not a support forum. This doesn't look like a bug, but something in your image. Please use either one of:

forums.docker.com
the #docker IRC channel
StackOverflow

I'll close this issue


koordlet: add taskids in statesinformer

Ⅰ. Describe what this PR does
Add taskids for each pod as an attributes. It's part of the https://github.com/koordinator-sh/koordinator/pull/1974. Resctrl runtime hook will leverage this to write new taskids to resctrl group.

Ⅱ. Does this pull request fix one issue?
https://github.com/koordinator-sh/koordinator/issues/1831

Ⅲ. Describe how to verify it
Ⅳ. Special notes for reviews
V. Checklist

[ ] I have written necessary docs and comments
[ ] I have added necessary unit tests and integration tests
[ ] All checks passed in make test

@saintube PTAL.

@saintube PTAL.


Cirros VM hangs very long on dhcp discover

When I boot my cirros VM, it takes some time until it finally gets past the dhcp configuration (up to 30 secons).
Console output:
Starting acpid: OK
mcb [info=/dev/vdb dev=/dev/vdb target=tmp unmount=true callback=mcu_drop_dev_arg]: mount '/dev/vdb' '-o,ro' '/tmp/nocloud.mp.ODGQgF'
mcudda: fn=cp dev=/dev/vdb mp=/tmp/nocloud.mp.ODGQgF : -a /tmp/cirros-ds.INNIox/nocloud/raw
Starting network...
udhcpc (v1.23.2) started
Sending discover...

It hangs at the last line very long until it continues. Let's check if we have a general issue, or if it is just cirros related. At the end it always works, but we don't see something on the console for a very long time.

@vladikr FYI

Thanks, I'll take a look.

For the record, we are using Cirros in OpenStack gates and I am not aware of an issue like that one there. If there is indeed an issue, it is probably something kubevirt related.

@vladikr giving this high priority, since it is currently the number 1 issue on ci for flaky tests.
@davidvossel ^^ FYI

@rmohr strange, I ran it several times and I get a response immediately...
I'll try on a different environment.

@rmohr where were you able to reproduce this? Was this on CI or your local machine?
@vladikr Looking at the code, since the DHCP server starts in a goroutine in parallel to the VM starting there's technically a race condition there where the VM could start before DHCP was listening in virt-launcher... that seems really unlikely though.

@rmohr @davidvossel I think I've managed to reproduce the issue, only on openshift though.
However, I see that both the test VMs got an ip from dhcp:
udhcpc (v1.23.2) started\r\nSending discover...\r\nSending select for <IP_ADDRESS>...\r\n [Sending discover...]} {1 Lease of <IP_ADDRESS> obtained, lease time 86313600\r [Lease of <IP_ADDRESS> obtained, lease time 86313600]}]"

And
Sending discover...\r\nSending select for <IP_ADDRESS>...\r\nL [Sending discover...]} {1 ease of <IP_ADDRESS> obtained, lease time 86313600\r\n

Looks like the outbound vm crashes after the login:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0xf06cc2]

goroutine 199 [running]:
kubevirt.io/kubevirt/tests.NewConsoleExpecter.func1(0x12e0e60, 0xc42027c0c0, 0x0, 0xc42000eea8, 0xc42000eec0, 0xc4206d0d80)
	/root/go/src/kubevirt.io/kubevirt/tests/utils.go:957 +0x32
created by kubevirt.io/kubevirt/tests.NewConsoleExpecter
	/root/go/src/kubevirt.io/kubevirt/tests/utils.go:956 +0x329

Roman, is it similar to what you've been seeing?

Looks like the outbound vm crashes after the login:

That looks like a null nil pointer exception in our golang testing code and not really related to the vm.

Roman, is it similar to what you've been seeing?

No, booting the vm always works and nothing crashes. It just sometimes hangs for a very long time until it reached the boot prompt.

This issue and the comment by David above reminded me of a very old OpenStack Neutron bug where DHCP agent was not quick enough to start dnsmasq to serve VM lease before Nova started it and Cirros (or whatever is in the image) starts networking configuration.
The OpenStack bug is https://bugs.launchpad.net/neutron/+bug/1453350 ; the fix is https://review.openstack.org/#/c/181674/
The general idea of the fix is to not update port status to ACTIVE until all entities (L2 agent, DHCP agent...) confirm completion of their part of the job. Only when all entities report back, we bump port status to ACTIVE which triggers a notification message back to Nova Compute service to unpause the VM and proceed with its boot.
We may need a similar mechanism in kubevirt.

@booxter great that you were able to reproduce it. I think that the solution may be much simpler, than to do something that complex, like pausing the vm (but let's see ...). First we are not using dnsmasq, we just have a go-routine which understands dhcp-requests and can respond. That should be extremely fast. If there is no bug in the dhcp code (causing slow start, not understanding a valid dhcp frame, ...), using a waiting-group to wait with booting qemu until dhcp is ready, should be sufficient.
Looking forward to see what finally caused the issue and what will solve it.

using a waiting-group to wait with booting qemu until dhcp is ready, should be sufficient.

We just need to avoid marking the pod as ready until we're certain dhcp is both listening and accepting connections.
Essentially by delaying the execution of cmd/virt-launcher/virt-launcher.go's markReady() function, we delay the possibility of the VM being started in the pod.

@davidvossel yes, if we move the dhcp-server start out from the preStart hook and start it right away (which makes more sense anyway). I guess that we will not even see the issue then, no matter if we delay the marking or not. Sounds good in general.

oh wow, I actually forgot we started dhcp server during sync command. That makes this race condition much more likely than I originally though.

if we move the dhcp-server start out from the preStart hook
I don't see how will it work.
We discover the pod ip and routes during the preStart hook.
What am I missing?

We discover the pod ip and routes during the preStart hook.
What am I missing?

The whole network setup can be done immediately. No need to wait for the preStart hook, which is invoked much later, when virt-handler tells virt-launcher to start the vm.

Some updates from my environment.
First, I don't believe it's an issue with late DHCP server goroutine startup. In docker log for virt-launcher container I see that the server was started long before the VM is spawned (there is a "Starting SingleClientDHCPServer").
I believe the issue is because the spawned server doesn't receive a DHCP request frame when Cirros issues the first discover message. Only when the second message is logged by Cirros, virt-launcher container, and DHCP goroutine, receive the request. (I confirmed it twice, by enabling debug messages and looking for "The request has message type DISCOVER" message in container log; as well as by running tcpdump inside the container and (not) observing BOOTP messages until 1 minute later from the first "Starting discover..." log message.)
I also tried to boot a Fedora image, and it booted just fine; tcpdump shows that BOOTP messages are received almost immediately when network.target is started inside the VM; the lease is immediately served, and in a second I get to getty.
Note it doesn't necessarily mean the issue is with Cirros. There is some space for a time sensitive race condition that could be triggered for Cirros only because it e.g. boots slightly quicker.

First, I don't believe it's an issue with late DHCP server goroutine startup. In docker log for virt-launcher container I see that the server was started long before the VM is spawned (there is a "Starting SingleClientDHCPServer").

This isn't a guarantee that the DHCP server is accepting requests yet.
I think we should make issuing and receiving a response from the DHCP server a condition that has to be met before the pod is marked as ready. If we do that, then we at least know the DHCP server is capable of receiving requests before the VM starts.
If the problem persists after that, then I'm really not sure what else can be done from our end.

@davidvossel while I can't rule out there is an issue with DHCP goroutine, it won't work regardless if we don't even receive DHCP frames into the pod. There must be an issue in VM connectivity to the pod bridge.
One potential lead that @vladikr found is https://bugzilla.redhat.com/show_bug.cgi?id=1411025#c2 that suggests there is expected wait time for libvirt bridges when no traffic from VM is allowed in. The comment suggests that the delay is 2 seconds by default but I see forward delay in brctl showstp for br1 set to 15.0 (seconds?) that would explain why the frame doesn't reach the pod.

An alternative to customizing interface model type via API would be labeling / tagging images with metadata that would specify hardware requirements. Then kubevirt would consume the metadata to make decisions on which model type to choose. (This is how OpenStack Nova does it: Glance images carry hw_vif_model tags that then Nova Compute consumes.)

excellent work on tracking this down!

An alternative to customizing interface model type via API would be labeling / tagging images with metadata that would specify hardware requirements.

I think we're going to have to add VIF info into the VM's domainSpec. This is going to take some discussion
@rmohr did you have any thoughts on now we should represent network devices in the KubeVirt API?

For the record, I reported a bug for Cirros suggesting they should wait for interface link ready before starting udhcpc: https://bugs.launchpad.net/cirros/+bug/1768955

@rmohr did you have any thoughts on now we should represent network devices in the KubeVirt API?

I am in conflict. On the one hand for a huge amount of "legazy" workloads inside VMs, the pod network is exactly what they will need. For more specific use-cases like VNF, ... it is definitely not good enough. I would love the give the pod network interface a place on the API which underlines its very special role and not just make it an entry in "anonymous" list of interfaces. On the other hand I want to keep the interfaces unified.
For now, for the sake of stabilising the tests, I would even be fine to just add an annotation like alpha.kubevirt.io/network.device: virtio, or something similar, just to buy us some time for a reasonable debate.

Just for reference, we are working on a doc to outline our proposal of how to represent multiple nics in the kubevirt API.
At the same time work with upstream kubernetes on (3) proposal of how K8s can be thought to support multiple networks (and other things).

Asked in libvirt-users@ about link establishment slowdown for e1000 in case someone has insight into the issue: https://www.redhat.com/archives/libvirt-users/2018-May/msg00065.html

A user suggested in libvirt-users@ thread that we could explore other model types for our interface defaults, for example, rtl8139. I tried it, and indeed it is not affected by the delay either. That being said, I am told the rtl driver is slower / not well supported. On the bright side, it should work for both Linux and Windows guests.

I also reported a bug against qemu for the slow e1000 initialization: https://bugs.launchpad.net/qemu/+bug/1770724

the e1000e device is causing problems with the test suite apparently.
A solution that came up in the community meeting is for us to create a value in the VM Spec's feature section to define the pod network driver. This avoids us having to define network interfaces in the api for now, and lets us give users the ability to define what drive to use just for the pod network. I think this is something we can support long term possibly.
Example
metadata:
  creationTimestamp: null
  name: vm-windows
spec:
  domain:
    clock:
      timer:
        hpet:
          present: false
        hyperv: {}
        pit:
          tickPolicy: delay
        rtc:
          tickPolicy: catchup
      utc: {}
    cpu:
      cores: 2
    devices:
      disks:
      - disk:
          bus: sata
        name: pvcdisk
        volumeName: pvcvolume
    features:
      podNetworkDriver: virtio
      acpi: {}
      apic: {}
      hyperv:
        relaxed: {}
        spinlocks:
          spinlocks: 8191
        vapic: {}
    firmware:
      uuid: 5d307ca9-b3ef-428c-8861-06e72d69f223
    machine:
      type: ""
    resources:
      requests:
        memory: 2Gi
  terminationGracePeriodSeconds: 0
  volumes:
  - name: pvcvolume
    persistentVolumeClaim:
      claimName: disk-windows
status: {}

My take on this is that we simply need to increase the tests timeout, to make tests more resilient, instead of introducing a new feature.
We are still going to implement a new API to represent network interfaces in order to support various connections methods to the pod network (bridge or proxy) and connecting various devices using device plugins.
As @fabiand  mentioner earlier, we are currently drafting a document and submit a proposal that would include all of the above as well as allowing the users to set a nic model.
The implentation of a proxy connection to the pod networking is being worked on in #787
From point of view it's better to give a bit more time and increase the timeout for now.

I've made a bunch of tests with cirros and alpine images, and it seems like we have no single good option. To list:

e1000 works for both but results in test timeouts;
(despite what I was saying before) e1000e works for neither of 'simple' images (guest drivers are not included; not sure how I missed that, probably libvirt side changes were not applied when I was testing it before;
rtl* makes console hang up;
virtio is not supported by Windows machines.

Overall, it looks like the list of kernel modules included in cirros / alpine is very limited.
There is no "easy" solution here.
We can either revive https://github.com/kubevirt/kubevirt/pull/990 to use virtio for cirros; or have a spin on the PR like using virtio by default but change the model to e1000 for win* matches.
Or we can complete API design for advanced networking to include ability to choose a driver, and then modify example templates for cirros / windows accordingly: https://github.com/kubevirt/kubevirt/issues/718 (which is probably a piece of a more general https://github.com/kubevirt/kubevirt/pull/787)

@booxter just to clrify - virtio should be supported by Windows, just required installing the drivers (not saying it solves our problem, just to make sure).

A nice fix - based around presets and the correct annotations - depends on https://github.com/kubevirt/kubevirt/issues/1067


extension of  CalendarViewController: JTAppleCalendarViewDataSource, JTAppleCalendarViewDelegate

Hi ,
It is asking me to provide another function, I've been following your video tutorial but you only have two protocols outlined in the video. I cloned your project and implemented everything correctly. Could you resolve this issue?
Here is the code:
 func calendar(_ calendar: JTAppleCalendarView, willDisplay cell: JTAppleCell, forItemAt date:      Date, cellState: CellState, indexPath: IndexPath) {         <#code#>     }
I already have these two defined
` func calendar(_ calendar: JTAppleCalendarView, cellForItemAt date: Date, cellState:
  CellState, indexPath: IndexPath) -> JTAppleCell {

    let cell = calendar.dequeueReusableJTAppleCell(withReuseIdentifier: "CustomCell", for:
    indexPath) as! CustomCell

    cell.dateLabel.text = cellState.text

    return cell

}
func configureCalendar(_ calendar: JTAppleCalendarView) -> ConfigurationParameters {
    formatter.dateFormat = "yyyy MM dd"
    formatter.timeZone = Calendar.current.timeZone
    formatter.locale = Calendar.current.locale

    let startDate = formatter.date(from:"2017 01 01")
    let endDate = formatter.date(from: "2018 01 01")

    let parameters = ConfigurationParameters(startDate: startDate!, endDate: endDate!)
    return parameters
}

`
Why is it asking me to conform to another protocol?
Thanks

Nik

You must be  using the latest master branch code.
Yes, in the update to come, i will be forcing every one to implement the willDisplayCell function.
The reason is explained here -> https://github.com/patchthecode/JTAppleCalendar/issues/553


Sidecar tunnel to Istiod XDS

istioctl will contact Istiod indirectly.  istioctl will port-forward to a workload pod on a special port.
The workload pod will expose a port that, with proper security, "tunnels" to Istiod XDS.  istioctl will use this tunnel to do XDS Event queries against Istiod.
The XDS Event document is https://docs.google.com/document/d/1X7DmkP10x4nEcTGpikxmvLbBTTK59B0WL3liQzVy8Ms/edit#heading=h.xw1gqgyqs5b

Is this issue addressed by https://github.com/istio/istio/pull/24398


Correct the tick labeling in show_saxs_qmap.

Thanks! I will take a careful look.
Sent from my iPhone
On Apr 20, 2017, at 8:49 PM, Zheng Zhang <notifications<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/yugangzhang/chxanalys/pull/1
Commit Summary

Correct the tick labeling in show_saxs_qmap.

File Changes

M chxanalys/SAXS.pyhttps://github.com/yugangzhang/chxanalys/pull/1/files#diff-0 (2)

Patch Links:

https://github.com/yugangzhang/chxanalys/pull/1.patch
https://github.com/yugangzhang/chxanalys/pull/1.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/yugangzhang/chxanalys/pull/1, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKO8GSLYRI9Z5NN-jeLviBNZM5IWV_mzks5rx_zxgaJpZM4NDvXZ.

Confirmed! Thanks @zzhangresearch


Why Common Install .sh isn't working?

Hi,
any idea why the install script of the repository https://raw.githubusercontent.com/jesseduffield/lazydocker/master/scripts/install_update_linux.sh is'nt working on a vmware virtuell machine ubuntu 64 bit (image: ubuntu-22.04.3-desktop-amd64.iso). Ubuntu was installed with the minimum of utilites. The host is Win10.
I got it running with these steps:
wget https://github.com/jesseduffield/lazydocker/releases/download/v0.21.1/lazydocker_0.21.1_Linux_x86_64.tar.gz
tar xvzf lazydocker*.tar.gz

sudo install lazydocker /usr/local/bin/
sudo systemctl status docker
sudo lazydocker

The first way with install_update_linux.sh give me normal command in terminal lazydocker is running, but I get the socket error docker.sock: connet: permission denied. The command sudo lazydocker says lazydocker command not found.
The second way gives me both from above. I get the socket error without sudo and with sudo it is running and the magic starts with lazydocker to manage containers.
Probably it would be a good idea to add docker to a group to avoid the use of sudo all the time. But that's not my question here. Or is it the answer?

Same problem here. Did you manage to fix/workaround this?


Acknowledge users with level 2000+ as admins as well

Pull request checklist:

[x] CHANGELOG.md was updated, if applicable
[x] Documentation in docs/ or install-docs/ was updated, if applicable

I've also seen there's some more faulty logic in that file. Once this is merged I'll fix that too.

I didn't pay much attention, but the changelog entry has been mistakenly added under v1.53 section, while it should be added to Unversioned. Might be something to be fixed in another commit / PR.


Update to 2018 edition

Merge conflicts because I got #23 in first, sorry about that, hopefully won't be too hard to fix. Other than that, looks fine.

@raphlinus Rebased and ready to go!


Question: Do you plan to use material2/md-data-table as base for td-data-table?

When material2/md-data-table becomes stable, do you plan to use as base to td-data-table?

Not truly relevant at the moment in my opinion. The data table component of material is still in design/alpha state. So its going to take some (long) time to see it ready 😄
Btw, most likely the dev team of covalent already took care of this by sticking to material design. And for what I saw both components handle the data as an immutable collection, so conceptually they are most likely very similar

we'll take a look at wrapping md-data-table (as we did w/ our autocomplete chips) when it's released, but it will be some time before they release, and we're going to continue adding features to ours. closing since it's a conversation more than an issue


azure-cli-2.51.0.msi upgrade from previous version 2.50.0 no longer non-interactive.

Describe the bug

Package name change.

BEFORE:
PS> Get-Package -ProviderName msi -Name 'Microsoft Azure CLI'
AFTER:
Get-Package -ProviderName msi -Name 'Microsoft Azure CLI (32-bit)'
Name                           Version          Source                           ProviderName

Microsoft Azure CLI            2.50.0                                            msi
Microsoft Azure CLI (32-bit)   2.51.0                                            msi

Upgrade from previous version 2.50.0 no longer non-interactive.
CMD> az upgrade --all --yes

Related command
az upgrade --all --yes
Errors
No error, just change in update behavior.
Issue script & Debug output
<#
.SYNOPSIS
Installs Azure CLI (Command Line Interface).
.NOTES
How to install the Azure CLI
& Start-Process https://learn.microsoft.com/en-us/cli/azure/install-azure-cli
#>
Import-Module -Name PackageManagement -ErrorAction Stop
$az = "${env:ProgramFiles(x86)}\Microsoft SDKs\Azure\CLI2\wbin\az.cmd"
$msiFile = "${env:USERPROFILE}\Downloads\AzureCLI.msi"
if (-not (Test-Path -Path $az -PathType Leaf))
{
# Write-Warning -Message "Install Azure CLI?" -WarningAction Inquire
Invoke-WebRequest -Uri 'https://aka.ms/installazurecliwindows' -OutFile $msiFile -Verbose
Install-Package -ProviderName msi -Name $msiFile -Verbose
}
Get-Package -ProviderName msi -Name 'Microsoft Azure CLI' -Verbose | Format-Table -AutoSize
$azDevopsInstalled = & $az extension list --query "[? name == 'azure-devops']" | ConvertFrom-Json | Select-Object -First 1
if (-not $azDevopsInstalled)
{
# Add the Azure DevOps extension
& $az extension add --name azure-devops
}
NOTE: Avoid stop on error due to stderr: This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
$ErrorActionPreferenceBackup = $ErrorActionPreference
try
{
$ErrorActionPreference = [System.Management.Automation.ActionPreference]::SilentlyContinue
& $az upgrade --all --yes
}
finally
{
$ErrorActionPreference = $ErrorActionPreferenceBackup
}
& $az version
Expected behavior
Silent upgrade from 2.50.0 to 2.51.0
az upgrade --all --yes

Environment Summary
CMD> az --version
azure-cli                         2.51.0

core                              2.51.0
telemetry                          1.1.0

Extensions:
azure-devops                      0.26.0

Dependencies:
msal                            1.24.0b1
azure-mgmt-resource             23.1.0b2

Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\Administrator\.azure\cliextensions'

Python (Windows) 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:05:00) [MSC v.1929 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context
No response

Thank you for opening this issue, we will look into it.

The product name change from Microsoft Azure CLI to Microsoft Azure CLI (32-bit) is expected according to  https://github.com/Azure/azure-cli/pull/26640#issuecomment-1660054026.
az upgrade is never meant to be non-interactive as no /q or /quiet argument is given when invoking msiexec.exe:
https://github.com/Azure/azure-cli/blob/21266e02457263420b62fba1c529f69e28852414/src/azure-cli/azure/cli/command_modules/util/custom.py#L193
It will always prompt for the MSI installation window:

I doubt in your previous executions, there is no update available. The output should look like:
> az upgrade --all --yes
This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
You already have the latest azure-cli version: 2.51.0
Upgrade finished.You can enable auto-upgrade with 'az config set auto-upgrade.enable=yes'. More details in https://docs.microsoft.com/cli/azure/update-azure-cli#automatic-update

See:

https://learn.microsoft.com/en-us/windows/win32/msi/command-line-options
https://learn.microsoft.com/en-us/windows/win32/msi/standard-installer-command-line-options
https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/msiexec


Add support for Kube-Prometheus-Stack monitoring

As a systems operator I want to be able to monitor my applications.

With release v1.10.0 OTS gained Prometheus metrics export which can be scraped by the Kube-Prometheus-Stack in an automated way without the OPS having to configure that themselves.
Acceptance Criteria

A ServiceMonitor is added to the chart scraping /metrics on the pods
A Grafana Dashboard is built to visualize the metrics exposed by OTS and added as an annotated ConfigMap
All of the above must be enabled through a flag in the .Values (default: enabled = false)
When enabled the customization option metricsAllowedSubnets should automatically be enabled for cluster-local access (usual setup is to have the cluster run on <IP_ADDRESS>/8 so allowing that should be sufficient though this almost certainly also exposes the metrics to the public so there needs to be a blocker in the ingress to block access to /metrics)
A way to add alerting rules should be added but no default rules should be specified.

As a side-effect of enabling the metricsAllowedSubnets in customizations it might be a good idea to provide a map in the Values to set customization options which then are {{ .Values.customizations | toYaml }} rendered into a config-map (after being joined with the chart-set customizations) and mounted into the pod.

thanks for your suggestion. I wasn't aware that the metrics are now exposed.
Will try to incooperate the proposed changes next week

I made a first draft (Did not tried it yet). Also the grafana and alerting is missing.
Regarding the "subnet" allow list I'm a little unsure. I don't see a benefit of this setting. If you allow the service subnet it would also be accessible via the ingress. Then you have to also block it ingress-specific. Sounds a little to complicated.
Why not just checking the HTTP Request Header for x-forwarded-host? If the request gets through the ingress this value is set.
What do you think?

Well, not everything is Kubernetes and never ever trust that header without previously whitelisting the proxies which can send that header (which then would be just another setting configuring IPs / Subnets) aside of that in some scenarios you just don't have that header / the real IP. That setting is to - by default - cover up the metrics handler.
Just ensure that /metrics cannot be accessed through the Ingress and open the setting so a cluster-internal Prometheus can access it.

yeah, that's correct. The problem is that the there is no ingress-indepent configuration option for that
I made a simple, but not really elegent approach: https://github.com/ContinuousSecurityTooling/ots-helm-chart/blob/b098a83000e85e035f7b6430891c400d5fd4d822/chart/templates/ingress.yaml#L45-L52

@Luzifer Would it be okay for you to split the Monitoring + Grafana/Alerting features?
I'm currently not having that much time for the complete solution and would like to make to publish an update with this PR: https://github.com/ContinuousSecurityTooling/ots-helm-chart/pull/11


Added a dummy user fixture.

As the title suggests, I've added a default dummy user to the project as instructed in #154.

Ok,  let's merge.


User Icon Images

Edit - 6/4/2022: Updated issue task where users can update avatar icons to be optional.
Currently, it is difficult to visually tell friends apart from each other because they all have the same default user icon image. Users should be able to choose an icon for themselves and this icon should be displayed in place of the current default image.

[x] Add property to ApplicationUser for image icons
[x] Allow user to select an icon during registration
[x] Update views where user icons can be seen in the:

[x] friend list
[x] new friend search
[x] stats of a user
[x] winner selection when adding a score
[x] account management

[ ] allow user to update their icon in account management (optional)

All criteria complete (aside from avatar updating which is now marked optional).


IOTEX Online IDE proposal by Buidllabs

Thanks for the proposal! Looks interesting. Let's have a phone chat next week to discuss the details?  My availability looks like http://raullen.youcanbook.me/.

Hey @raullenchai ,
Just scheduled a call using the link above. Looking forward to our call :)

Close this one as it doesn't look like we will pursue it


Update weaveexec image base container to alpine 3.16

What type of PR is this?

type::security
What this PR does / why we need it:
Weaveexec container image got pinned to the wrong alpine version
Which issue(s) this PR fixes:

Fixes NONE
Special notes for your reviewer:

Steps to reproduce

Does this PR introduce a user-facing change?

The [Weave add-on](https://kurl.sh/docs/add-ons/weave) now uses alpine:3.16 as the base image for container image weaveexec for versions 2.8.1 and 2.6.5.

Does this PR require documentation?

NONE

I'm going to close this as weave security patches have become independent versions in kURL


Client Ping Support (Keep Alive)

Support for client applications to send keep alive ping messages

@abelino, ping is already mentioned in the protocol, albeit it’s not used yet.
https://github.com/citrusbyte/oncilla/blob/fb56514ac095173d2c30c3906f48b8f9ce59214b/docs/network/wsProtocol.md#ping


[#148] Add try/except block around code adding labels to already existing issues

We have noticed when adding a new label to an already existing issue in Jira that the permissions on the firewatch-tool user needed to be updated in our projects. This PR adds some logging around this issue to help users who may not be familiar with the problem. It also adds some specific documentation to the README to help resolve the problem or tell new users what is required before using the tool.
Closes #148

/verified


Eslint fixes preparing for upgrade

These are changes to prepare us to upgrade ESLint.
No behavior changes.

Coverage increased (+0.06%) to 98.907% when pulling 9b30deb6a4939c60fef789db55736cdddb5f87e5 on jessebeach:eslint-fixes-preparing-for-upgrade into a9d415954efedd5f601769d233c3707705fc5972 on evcohen:master.


Resolve MS post link when blacklisting users

This should resolve #1450
I added timeout=5.000 hoping that MS downness will not throttle Smokey too badly.

Coverage remained the same at 64.336% when pulling a29e9b858e4a46acfba779206fdc08ad98dbce4d on resolve-ms into 524f70881f6c0be755ba548c5191c89772a8d88f on master.

What does this do for us? Looks like it changes the SE post link to an MS post link, which... I don't know if we want? It'd be cool to have both, if we could.

@ArtOfCode- Oops, I didn't realize that. I guess it needs some expansion.

@ArtOfCode- Ah yes. Can you add an option to MS so when a post has multiple records, it generates a redirect to the latest occurrence instead of showing a "disambiguation list"? The option could be a GET parameter.

@ArtOfCode- How does it look now?


cannot use qt6 qmake to build project

When I try to use qt6 qmake to build, it shows:
Could not find qmake spec 'win32-msvc'.
Then adding -spec D:\vcpkg\installed\x64-windows-static\share\Qt6\mkspecs\win32-msvc to qmake, it shows:
Project ERROR: Could not find feature thread.
For qmake -query, it puts out:
QT_SYSROOT:
QT_INSTALL_PREFIX:D:/vcpkg/installed/x64-windows-static
QT_INSTALL_ARCHDATA:D:/vcpkg/installed/x64-windows-static/share/Qt6
QT_INSTALL_DATA:D:/vcpkg/installed/x64-windows-static/share/Qt6
QT_INSTALL_DOCS:D:/vcpkg/installed/x64-windows-static/doc/Qt6
QT_INSTALL_HEADERS:D:/vcpkg/installed/x64-windows-static/include/Qt6
QT_INSTALL_LIBS:D:/vcpkg/installed/x64-windows-static/lib
QT_INSTALL_LIBEXECS:D:/vcpkg/installed/x64-windows-static/tools/Qt6/bin
QT_INSTALL_BINS:D:/vcpkg/installed/x64-windows-static/bin
QT_INSTALL_TESTS:D:/vcpkg/installed/x64-windows-static/tests
QT_INSTALL_PLUGINS:D:/vcpkg/installed/x64-windows-static/Qt6/plugins
QT_INSTALL_QML:D:/vcpkg/installed/x64-windows-static/Qt6/qml
QT_INSTALL_TRANSLATIONS:D:/vcpkg/installed/x64-windows-static/translations/Qt6
QT_INSTALL_CONFIGURATION:
QT_INSTALL_EXAMPLES:D:/vcpkg/installed/x64-windows-static/share/examples/Qt6
QT_INSTALL_DEMOS:D:/vcpkg/installed/x64-windows-static/share/examples/Qt6
QT_HOST_PREFIX:D:/vcpkg/installed/x64-windows-static
QT_HOST_DATA:D:/share/Qt6
QT_HOST_BINS:D:/vcpkg/installed/x64-windows-static/bin
QT_HOST_LIBEXECS:D:/vcpkg/installed/x64-windows-static/tools/Qt6/bin
QT_HOST_LIBS:D:/vcpkg/installed/x64-windows-static/lib
QMAKE_SPEC:win32-msvc
QMAKE_XSPEC:win32-msvc
QMAKE_VERSION:3.1
QT_VERSION:6.2.2

OS: Windows 10 19044.1387
Compiler: Visual Studio Community 2022

But for qt5, qmake works well as usual, and for cmake build project, building poppler-qt6 is fine.
So is there any bug with qt6 qmake?

cc @Neumann-A

a) which feature set of qtbase the is the featuren concurrent which enables threads
b) which host triplet? QT_HOST_DATA looks wrong. Maybe qt.conf in the folder of qmake wasn't configured correctly.
c) by default -static triplets are not the native host triplets.

a) which feature set of qtbase the is the featuren concurrent which enables threads b) which host triplet? QT_HOST_DATA looks wrong. Maybe qt.conf in the folder of qmake wasn't configured correctly. c) by default -static triplets are not the native host triplets.

a) I found QThread in 'x64-windows-static\include\Qt6\QtCore', maybe it's qt6 core function?
b) x64-windows-static? I install both x64-windows and x64-windows-static for qt6. Here's the content of x64-windows-static\tools\Qt6\bin\qt.conf:
[DevicePaths]
Prefix=./../../../
Headers=include/Qt6/
Libraries=lib
Plugins=Qt6/plugins
Qml2Imports=Qt6/qml
Documentation=doc/Qt6/
Binaries=bin
LibraryExecutables=tools/Qt6/bin
ArchData=share/Qt6
Data=share/Qt6
Translations=translations/Qt6/
Examples=share/examples/Qt6/
[Paths]
Prefix=./../../../
Headers=include/Qt6/
Libraries=lib
Plugins=Qt6/plugins
Qml2Imports=Qt6/qml
Documentation=doc/Qt6/
Binaries=bin
LibraryExecutables=tools/Qt6/bin
ArchData=share/Qt6
Data=share/Qt6
Translations=translations/Qt6/
Examples=share/examples/Qt6/
HostPrefix=./../../../
HostData=./../../..//share/Qt6
HostBinaries=bin
HostLibraries=lib
HostLibraryExecutables=tools/Qt6/bin
It does exist some different paths from qt5, there're no qt5 in my vcpkg now, and I built qt6 from fresh today.

Could you manually remove the additional / in HostData=./../../..//share/Qt6 and see if it resolves the issue. If not try to set it to HostData=share/Qt6
The reason this is an issue is probably because you didn't do vcpkg install qtbase --triplet=x64-windows-static --host-triplet=x64-windows-static As such the qt.conf in x64-windows-static is invalid since the valid one is in x64-windows

Ok, it works fine when set it to HostData=share/Qt6.
Yeah, I installed qt6 as vcpkg install qtbase:x64-windows. I didn't consider host-triplet before.

Please keep this issue open until #21944 merge.


README.md Building Project with Visual Studio instructions incorrect

Hi!
In https://github.com/microsoft/WSL-DistroLauncher#building-project-visual-studio the step states:

You can also easily build and deploy the distro launcher from Visual Studio. To sideload your appx on your machine for testing, all you need to do is right-click on the "Solution (DistroLauncher)" in the Solution Explorer and click "Deploy Solution". This should build the project and sideload it automatically for testing.
In order run your solution under the Visual Studio debugger, you will need to copy your install.tar.gz file into your output folder, for example: x64\Debug. NOTE: If you have registered your distribution by this method, you will need to manually unregister it via wslconfig.exe /unregister

Right clicking on "Solution 'DistroLauncher'" (Not "Solution (Distrolauncher)") here you can find "Deploy Solution":

However clicking on this does not actually produce any results.
It seems that the user must select their arch and mode here then hit "Start":

However, if the user followed the Getting Started steps https://github.com/microsoft/WSL-DistroLauncher#getting-started they never changed or set OutputPath, so they see this error:
This results in errors still:

No combination or debug/release x86/ARM works.

Hello @Hoverbear
did you get anywhere with it?
I have not installed VS since the previous decade and i faced a bunch of issues.
Managed to get a clean build with the community edition and despite installing 20GB of msbuild tools and forcing  141/142, when i share the launcher gives the infamous msvc140d error.
Runs fine on my desktop so i cant really debug.
If you have some notes for the community version that has a working launcher please share.

I kind of put it down and haven't gotten back to it. :(

when I share the launcher gives the infamous msvc140d error.

Because by default Visual Studio compiles a project dynamically with MS C runtime. You have use Release Mode or pack your application including msvcrt or vcruntime dlls.

@Biswa96
as i dont have experience with VS and this project sais use the default build.bat
why do i need to ship the launcher with all these dll's? None of the official distro launchers do.
Readme sais go modify 15-20 entries for you project in 3 files and away you go.
If you have a nice sanitised way within the readme/project please do share

This project is a joke.


Vulkan: occlusion queries, housekeeping - attempt 2

Second attempt :eyes: Tested on g++9, compiles correctly now.
Old PR here: https://github.com/bkaradzic/bgfx/pull/2505

All good! Thanks! :+1:


Improve efficiency of caching queries

Caches data in fewer queries and is easier on the API.

.deploy


How to make system pause until image processing is complete

Hello,
I am trying to do an acquisition of a set of images, and using each image I zap the nucleus with a 405 laser.
I would like to write an image processing hook that uses the image and finds the centroid of the nucleus and then zaps the corresponding position, but the acquisition loop and the image processing loop are not synchronized and I am not zapping the correct positions.
Is there any way to do image processing, but make the acquisition halt until it completes?
I have tried writing a post_camera_hook function which does the image processing in order to pause the system, but I need to use core.snap to get an image which will bleach the sample faster.
Thanks!

Yes, you to add acquisition events from within the processor, rather than usign acq.acquire.
Though you do need to do acq.acquire with at least one event to kick off the processor
Example, here:
https://pycro-manager.readthedocs.io/en/latest/img_processors.html#adapting-acquisition-from-image-processors


Roll Flutter Engine from 878fc6fc022e to 3712cc7d0c48 (1 revision)

https://github.com/flutter/engine/compare/878fc6fc022e...3712cc7d0c48
2022-12-05<EMAIL_ADDRESS>Roll Skia from 5dd1b7d6a303 to f75f98074f60 (3 revisions) (flutter/engine#38070)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


How to find the default connection method used in SAML 2.0

Kindly let us know, what are all the connection method available in SAML2.0 and which one used by default. Also, please let us know, where we can find those information.

Sorry but this communication channel is to fix issues related to php-saml. Not to learn the SAML 2.0 protocol.
You can start reading the documentation hosted at: http://saml.xml.org/saml-specifications


Custom oom handler that calls panic instead of abort

Currently, the Heap and LockedHeap uses the default oom handler, which calls intrinsics::abort(). On the x86_64 architecture, this generates a ud2 instruction, generating an invalid opcode exception.
While using this library in a kernel, allocating too much memory will call the default oom handler and generate an INVALID OPCODE exception that points to the first ud2 instruction in alloc::oom (0x1073fc in my case):
0000000000107ef0 <_ZN5alloc9allocator5Alloc3oom17h5474e96b9ba411a5E>:
  107ef0:	55                   	push   %rbp
  107ef1:	48 89 e5             	mov    %rsp,%rbp
  107ef4:	48 83 ec 10          	sub    $0x10,%rsp
  107ef8:	48 89 7d f8          	mov    %rdi,-0x8(%rbp)
 >107efc:	0f 0b                	ud2
  107efe:	0f 0b                	ud2
  107f00:	48 89 75 f0          	mov    %rsi,-0x10(%rbp)
  107f04:	66 2e 0f 1f 84 00 00 	nopw   %cs:0x0(%rax,%rax,1)
  107f0b:	00 00 00
  107f0e:	66 90                	xchg   %ax,%ax

I find the exception message unhelpful.  So instead of generating an exception, we can set a custom oom handler by implementing oom for Heap and LockedHeap.  Custom oom handlers will be important later when we want to gracefully handle errors.  My PR results in the following message when we run out of memory.
PANIC in /path/to/linked-list-allocator/src/lib.rs at line 180:
    Out of memory

Testing
You can test this PR by adding the following to src/lib.rs of blog_os
let mut stack: alloc::Vec<usize> = vec![0; 100_000_000];

Thanks a lot!

Published as 0.4.2


Interaction-less scheduling and date creation

Currently the planning gets prefilled 31 days when viewing the planning. That ought to be augmented to allow future planning to be pre-filled automatically. That way it can be a "set it and forget it"-type solution.

Configure a default schedule for each day
Cron job/scheduling that runs nightly to add more dates, and applies the default schedule to it.

Requested by Frederik https://secure.helpscout.net/conversation/1482487715/27369?folderId=46284