bug(MatTree): padding is not recomputed when moving a node inside/outside an expandable node

Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description
As TreeControl is deprecated, I started migrating to the latest API using childrenAccessor. So far so good, except for one case not working any more.
I'm using matTreeNodePadding and matTreeNodePaddingIndent on <mat-tree-node>. When I move a node inside/outside of an expandable node, padding is not recomputed as-is.
Expanding/collapsing the parent node in which the node was moved, will recompute its padding though.
Reproduction
I'm working on an open-source project, so I don't have a small example to share for now.
Steps to reproduce:

Clone branch https://github.com/poirierlouis/FellowImGui/tree/refactor/mat-tree
npm install
npm run start
Open browser in localhost:4200/editor
Replicate actions like in video below:

You can find code related to <mat-tree> in:

fig-tree: component + template

dataSource is set with a plain array.
When changed, dataSource is set with a new array. That is, only the root of dataSource is changed with a new array reference. It doesn't cascade down through children.
Maybe it is related? Should the entire tree be cloned somehow, to trigger changes detection?
Expected Behavior
When a node is moved inside/outside of an expandable node (level is changed), padding should be recomputed based on the new level of the node.
Actual Behavior
Padding of the node is not updated when moving the node inside/outside of an expandable node.
Environment

Angular: 18.2.10
CDK/Material: 18.2.11
Browser(s): Firefox / Chrome
Operating System (e.g. Windows, macOS, Ubuntu): Windows

Hi @poirierlouis, would you be able to provide a stackblitz example of this issue? It is difficult to debug this only from the code & video you shared.

Hi @wagnermaciel thanks for the quick answer. Yes I'll try and write an example to reproduce this issue.

@wagnermaciel , I could definitely reproduce this issue, see Stackblitz example.

@BobobUnicorn Is this resolved by https://github.com/angular/components/pull/29751?


Address a number of consistency issues in view specs

This is worth reviewing commit by commit as the overall diff looks a lot more dramatic than the actual changes made and there are full explanations of each change.
Mostly these changes continue the work and discussion from: https://github.com/alphagov/content-publisher/pull/1977
Happy to drop off the last commit if that is contentious.

Thanks for the review @benthorner. I ended up putting this up in a rush and meant to add some more context about my experiments with a single render call so I'll jot this up now (as it is a very quiet working Saturday 😄).
So I made really good progress with setting all view variables with stubbing and was feeling quite happy with myself (see: https://github.com/alphagov/content-publisher/commit/16a5f0c2fcd56af7def98ee428570a7cfc3babb3) and then all I had to do was rebase against master to the changes from https://github.com/alphagov/content-publisher/pull/1984 and everything went to hell.
It turns out that the subtle difference between methods and variables is exposed where attachment undefined variable overrides the attachment variable  so needs naming differently. I then ventured into govuk_publishing_components and realised that the very common ||= approach wouldn't work with stubs either so retreated.

Perhaps something we could do to address the con here is to raise an issue with rspec-rails.

That's a good idea. I'm not sure how into that they'd be as they are sometimes inconsistent with what they use as description for view tests in their examples (for example https://relishapp.com/rspec/rspec-rails/v/4-0/docs/view-specs/view-spec#view-specs-can-explicitly-render-templates)

On a tangent once I finished putting this together I was quite pleased how all the conventions in the describe names matched the approach across the rails-rspec docs (yay 🎉 ) and then just after I opened this PR I realised I'd been looking at 2.2 of the docs and they've used more Rails style naming in the newer versions. So I'm going to have a quick experiment with that for greater consistency with rspec-rails conventions


Please create dist folder

Even if installation from bower is not possible, could you please create the dist folder in this github repo so that I can just point to this github and the dist/angular-tooltip.min.js exists... Thanks!

+1

At the moment, the current state of the library is not minifiable. Has be updated to inject dependencies using literals.
+1.

+1


Test entity labels, fixed corner cases, changed interface

add tests for entity_labels_system
fixed filling label_entities map
fixed corner cases when removing entities, Labels component
use Changed query filter
changed EntityLabels::get to return slice or empty slice instead of
None or Some empty or non-empty slice

Changing the interface of EntityLabels::get is beneficial, since else
you would get different results in case there was an entity before that
with this missing label or not. You would either get None or Some(&[])
and need to handle both, which is actually not necessary.

as mentioned here, you shouldn't use Changed without changing first the stage the system runs on
goods to have more tests!

Yup this seems good to me. We should be able to add Changed filters using this approach:

Add Changed filter to query
Move system to POST_UPDATE
Add a copy of the system to POST_STARTUP

Without Changed filters, the current implementation is expensive in a "very hard to justify" sort of way. (which isn't a critique of this pr. this is a critique of past @cart back when he was just building things for himself)

Feel free to follow up with those changes (or I will if nobody else does). No harm in merging these fixes now.

Feel free to follow up with those changes (or I will if nobody else does). No harm in merging these fixes now.

I have a try :+1:

Feel free to follow up with those changes (or I will if nobody else does). No harm in merging these fixes now.

@mockersf followed up on that in #1155 :rocket:


Does Veasy support an array or collection of objects that can be added to the form?

My form has:

Name
Set of Domain objects. Each object consists of:
Domain Name, Domain Type

A valid form must have at least 1 Domain object.

Does Veasy support a collection of objects where there must be at least 1 object added to the form?

For example
this.mainFormSchema = {
   name: {minLength: [2, 'min length 2']},
   domainSet: {
      minSize: 1,
      elementContent: {
          domainName: {isUrl: true},
          domainRule: {enum: ['Full', 'None', 'Half']}
      }
   }
}

Or should the Set be a second Veasy form and if so how do I initialise this 2nd form?
eg

this.mainFormSchema = {
   name: {minLength: [2, 'min length 2']},
}

this.domainFormSchema = {
  domainSet: {
      minSize: 1,
      elementContent: {
          domainName: {isUrl: true},
          domainRule: {enum: ['Full', 'None', 'Half']}
      }
   }
}

this.state = createInitialState(this.mainFormSchema, this.myState);
this.domainFormState = createInitialState(this.domainFormSchema, this.myState);

// I bind this in the constructor
updateDomainFormState(state) {
     console.log("updating domain form state...");

     // the form's state is overrided completely each time! Better way to do this so Veasy knows about changes in the state?
     this.domainFormState = state;
}

<VeasyForm schema={this.mainFormSchema} allState={this.state} update={this.updateState}>
.
.
</VeasyForm>

<VeasyForm schema={this.domainFormSchema} allState={this.domainFormState} update={this.domainFormUpdateState}>
.
.
</VeasyForm>

Using the above code, the 2nd Veasy domainForm does not report error messages until onBlur is called on the 1st Veasy mainForm...

If I seperate the 2nd form into its own component then can it have the same  param values as the 1st form? Or will its state then be considered part of the state of the 1st form even though it's in a separate React component?

Let's be more clear about this.
So, you want to validate against a set of entities, then the result should be one?

I can use the form to add many domains. The form is valid if there's at least 1 domain.  A JSON representation would be:
{
   name: 'bob',
   domains: [
      {name: 'http://a.com', type: 'ALL'},
      {name: 'http://b.com', type: 'WHITE_LIST'}
   ]
}

I have realised that it's easier to have 2 VeasyForms:
i) contains 'name' and recevies any domain JSON from form ii)
ii) contains domain name and type. If these are valid then the user can click 'add' and the data is added to form i).
I'll update this task's title to 'How to have multiple Veasyforms play together' ?

What do you mean by domain here?
If I get it right. As you said, I think you can simply add 2 <VeasyForms>, and after validating that 2nd has valid input, them pass it back to form1 somehow.
But connecting these 2 forms shouldn't be coupled with validation steps.

I have a form with inputs for a web domain's url and type. But I agree with you - use 2 forms and pass the valid value of one form to the other to send to the server.


Refactor landing page

I suggested changes that will:

Update the landing page for toota

This is really good work,can't wait to see it on production


Can't extend the same interface with different type parameters

This might sound like obviously correct behaviour. There's a very strong use case in the Node.js type definitions alone though, and I think type aliases show an avenue for this that could become a manageable workaround that's accessible without too much work.
Motivation
In the Node type definitions, we have a huge amount of duplication for EventEmitter types, which could be removed if we could write something like this:
interface EventEmitter<EventName extends string, CallbackType extends Function> {
        addListener(event: EventName, listener: CallbackType): this;
        on(event: EventName, listener: CallbackType): this;
        once(event: EventName, listener: CallbackType): this;
        prependListener(event: EventName, listener: CallbackType): this;
        prependOnceListener(event: EventName, listener: CallbackType): this;
        removeListener(event: EventName, listener: CallbackType): this;
}

interface Server extends EventEmitter<"connection", (s: Socket) => void>,
                         EventEmitter<"error", (e: Error) => void>,
                         EventEmitter<"listening", () => void>,
                         EventEmitter<"close", () => void> {
  // ...other non-event emitter methods
}

Currently in the Node type definitions we implement EventEmitters like the above by duplicating every one of the EventEmitter methods for every single event that an object can have. That results in a huge amount of code in lots of places. These interfaces don't even bother typing all the methods they could (e.g. removeListener, listenerCount, listeners) presumably because that would make this even worse.
This would all be drastically simplified and made much more maintainable if something like the above was possible.
Type alias approach
In a perfect world, it would be wonderful if this Just Worked, and the above code defined method overrides for each of the combinations where they conflicted, more or less equivalent to the currently manually maintained net.Server definition.
I've seen other explanations of why this won't work with interfaces directly though. Unfortunately the suggested workaround for this problem generally is to manually merge the method declarations, which would defeat the point here.
You can get closer with type aliases:
// You can define this just fine, unlike the interface equivalent, solving half the problem.
type ServerEmitter = EventEmitter<"connection", (s: Socket) => void> &
                     EventEmitter<"error", (e: Error) => void> &
                     EventEmitter<"listening", () => void> &
                     EventEmitter<"close", () => void>;

This isn't a fully equivalent alternative though; critically you can't extend this type with another interface, or extend from the intersection expression itself (TS2312: An interface may only extend a class or another interface / TS2499: An interface can only extend an identifier/qualified-name with optional type arguments).
Options
There's a few things TypeScript could do that would solve this:

Make interface extension work as in the first example (sounds like this isn't practical)
Allow extending type aliases in interfaces
Allow extending type expressions (A & B) in interfaces

I haven't dug into this in depth, but either 2 or 3 sound like they would work around the issues with 1 (as the rules for merging are well defined), and would provide relatively clean solutions to solving these problems generally.

Extending a type alias is not the real issue here. The real issue is ordering signatures. Overload resolution in TS is order dependent. So the issue is how these overloads are merged. for instance:
declare function doWork(a: string) : number;
declare function doWork(a: "command" ) : string;

declare function doWork2(a: "command" ) : string;
declare function doWork2(a: string) : number;

doWork("command"); // returns number

doWork2("command"); // returns string;

In your EventEmitter example, you have something like addListener that needs to get an overload from each one, but which one comes first, should order of interfaces in extends clause matter? what if one of them is more generic than the previous ones, should it hide it? etc.. The error you get today is just an indication that the resulting interface needs to re-specify the order of overloads. obviously respesfiying the overloads defines the whole point of minimizing the duplication.
I do agree that it would be good to enable the scenario listed above. but for this to happen we first need https://github.com/Microsoft/TypeScript/issues/10523 addressed with an algorithm that allows for order-independent literal types resolution. then we need to identify when inheriting the same signature from multiple interfaces would be ok under that algorithm and no show an error when it happens.

Not sure if this issue is related, but it seems like it is.
The Redux issue was resolved, but the API it leaves behind is not ideal (because of the inability to overload the same interface with different type parameters).

I think existing solutions are sufficient here given how often the use case arises


unable to ignore .git/ directory

Hello, I'm using the Copy Files task in TFS. All of the application source is in the same directory that .git/ resides.
I want to copy all files except .git/ and I can't seem to make it work. The easy solution is to put everything in a subfolder and use that as the source folder. But are ignore globs allowed? I've tried the following variations:
**
!.git/

**
!.git\

Patterns only match full files, so you would need:
**/*
!.git/**/*

I'll add this as an example.


No module named 'termios'

Tried this on windows 7 and it says no module named 'termios'.
Getting the same error while importing in seperate py file. Also cant install via pip when further researched it is found : no distribution available for windows

so for the time it is safe to say that this package is only linux supportive?

Hi Omkara000,
Windows does not support the pty (pseudo-terminal manipulation tools) of Pyxtermjs, termios and fcntl.
Note :
It is possible to run Pyxtermjs on Windows, however this requires modifying the libraries and the python code of Pyxtermjs. At the moment there is no version of Pyxtermjs for Windows. However, I'm working on it on my PyxtermJS fork named Termsteel. I can't guarantee that the Pyxtermjs project will be around for a long time. I recommend you to use my fork, you will find a more recent version with a nice web interface.
To solve:
Install PyxtermJS on a Linux machine to fix your problem.
If you have no other choice but to use Windows, you can use WSL (requires Windows 10 Version 1903 or later, with Build 18362 or later with a machine supporting virtualization) or install Docker and install a Linux distribution on it

Right now it is only supported on unix-like machines, since that is what the Python standard library supports.
Windows support is technically possible, as node-pty provides this through a single API. https://github.com/Microsoft/node-pty
It does look like there are some 3rd party Python packages that support Windows, for example https://github.com/andfoy/pywinpty may work.
PRs welcome!


Add comments explaining external dependency checksum caching

This PR aims to answer https://github.com/rubocop-hq/rubocop/pull/7496#discussion_r350462926 by @exterm on https://github.com/rubocop-hq/rubocop/pull/7496 "Allow cops to invalidate results cache" with code comments.
An alternative (more invasive) implementation I considered was to pull the cop mobilization logic out of Runner into a new helper module RuboCop::Cop::Mobilizer. Then the runner could just pass the config and options to the ResultCache, which could then create its own "standby" teams internally. This would more cleanly contain the team-level external dependency checksum caching logic within the ResultCache, but it would require more refactoring.

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[ ] Added tests.
[ ] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title and description in grammatically correct, complete sentences.
[ ] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Thanks!


add TurtleRule

Turtle Rule is one of the well known trend following strategy complete with definitive position sizing, exit, and entry rules. This example strategy would be a great addition as an example of a complete trading rule.

faithful implementation of unit_risk_percent would need leverage support in Jesse because otherwise, it will require too much capital and raise negative balance error
for now, change this parameter to 0.5 for backtesting

Thank you for the PR. It's an advanced strategy.  I might create directories such as "beginner" and "advanced" strategies for it later. It's also a good idea to include a sample backtest result in .txt format and maybe even the generated chart for BTCUSD or ETHUSD for the last year or so.


State-dependent system runs twice on wrong add_state call order

Bevy version
main branch on commit 284889c64b2a4a4f67018e42a14062392edbd0aa.
Operating system & version
MacOS Catalina 10.15.7 (19H2)
What you did
Consider following snippet:
use bevy::prelude::*;

fn main() {
    App::build()
        .add_plugins(DefaultPlugins)
        .add_system_set(
            SystemSet::on_update(GameState::Running)
                .with_system(running_input.system()),
        )
        .add_state(GameState::Running)
        .run();
}

#[derive(Clone, PartialEq, Eq)]
enum GameState {
    Running,
}

fn running_input(time: Res<Time>, ) {
    println!("{:?}", time.last_update())
}

What you expected to happen
System running_input is called once per frame.
What actually happened
System running_input is called twice per frame:
/Users/v.kaverin/.cargo/bin/cargo run --color=always --package rust-sandbox --bin rust-sandbox
    Finished dev [unoptimized + debuginfo] target(s) in 0.21s
     Running `target/debug/rust-sandbox`
Some(Instant { t:<PHONE_NUMBER>114726 })
Some(Instant { t:<PHONE_NUMBER>077967 })
Some(Instant { t:<PHONE_NUMBER>077967 })
Some(Instant { t:<PHONE_NUMBER>604616 })
Some(Instant { t:<PHONE_NUMBER>604616 })
Some(Instant { t:<PHONE_NUMBER>654412 })
Some(Instant { t:<PHONE_NUMBER>654412 })
Some(Instant { t:<PHONE_NUMBER>062196 })
Some(Instant { t:<PHONE_NUMBER>062196 })
Some(Instant { t:<PHONE_NUMBER>839161 })
Some(Instant { t:<PHONE_NUMBER>839161 })

Additional information
The result depend of when add_state was called.
After swapping add_state and add_system_set like
...
        .add_state(GameState::Running)
        .add_system_set(
            SystemSet::on_update(GameState::Running)
                .with_system(running_input.system()),
        )
...

everything works just fine:
/Users/v.kaverin/.cargo/bin/cargo run --color=always --package rust-sandbox --bin rust-sandbox
    Finished dev [unoptimized + debuginfo] target(s) in 0.19s
     Running `target/debug/rust-sandbox`
Some(Instant { t:<PHONE_NUMBER>729694 })
Some(Instant { t:<PHONE_NUMBER>376989 })
Some(Instant { t:<PHONE_NUMBER>594279 })
Some(Instant { t:<PHONE_NUMBER>349265 })
Some(Instant { t:<PHONE_NUMBER>412494 })
Some(Instant { t:<PHONE_NUMBER>973065 })
Some(Instant { t:<PHONE_NUMBER>934881 })
Some(Instant { t:<PHONE_NUMBER>073601 })

Another sub-bug of #1120 :(

This... works as expected.
add_state is adding the make_driver system set: https://github.com/bevyengine/bevy/blob/284889c64b2a4a4f67018e42a14062392edbd0aa/crates/bevy_app/src/app_builder.rs#L180-L183
which must be added first, as stated by its doc: https://github.com/bevyengine/bevy/blob/284889c64b2a4a4f67018e42a14062392edbd0aa/crates/bevy_ecs/src/schedule/state.rs#L189-L192

And also seemingly unavoidable: https://github.com/bevyengine/bevy/pull/1424#discussion_r576323512

make_driver is not used directly and looks like implementation details, so it's quite unobvious that  it's necessary to check its docs while add_state's one tells nothing about it this implicit dependency. It worth to add the doc to add_state method as more high level API as well at least.
But running the system twice is still not expected behavior regardless the docs.

This is a product of SystemSet run criteria evaluation order. I think providing some configuration for SystemSet insertion position is probably a nice quick solution to this. All we really need is for the driver system set to be inserted at the front by default and normal sets to be inserted at the back by default.
This is slightly complicated by the fact that index 0 is reserved for the "default" system set (which always exists). But we can make "front" insertion just happen at index 1 instead.
We could even make this configuration private (for now) so users can't accidentally insert a SystemSet at the front with state run criteria. This would also buy us time to design the "final" system set order api.
I'm making these changes now, but feel free to discuss alternatives / pick this apart.

This isn't quite as simple as I made it out to be because of the way we convert SystemSets (the "configuration") to virtual_system_set, parallel_systems, and exclusive_system lists (the "runtime state"). Upon insertion we store an index in each system container to the virtual_system_set index.
This creates a situation where re-ordering virtual_system_sets invalidates all stored indices (which could happen at any point during app building).
We'll either need to "fix" these indices when re-ordering occurs (or at some point before startup), use non-index ids to correlate, or change the lifecycle to ensure that when we populate the system container lists, the virtual system set indices are already stable.
@Ratysz would any of the above changes mess with your planned ecs work?  Do you have an opinion here?


Read depth at distinct sites is non-independent

There must be some strong correlation in read depth along the sequence, which drops off after n bases, where n is some function of the mean read length. E.g. two sites that are adjacent along the genome will have very similar read depths.

Maybe we could construct a Markov model to describe this, from which we could easily simulate read depths?

I agree, simGL right now assumes that each site is totally independent from the rest of sites. But of course, in a real set up, coverage should be correlated across nearby sites. I think this is an ok assumption for distant sites since it makes things very easy. However, depending on the SNP density, I agree that we should incorporate some sort of correlation across sites.
I think we could simulate N reads of length L (normally distributed) that satisfy a mean coverage X along the genome of length G such that X = G/NL. Then, we can randomly place those reads along the genome defining a starting position per read. Finally, we can check the number of reads overlapping per site. I expect that this simulation is going to be slower to produce than the Poisson random sample that is implemented right now in sim_allelereadcount().
But again, this raises other questions such as error rates might be also correlated somehow and different platforms might have different error profiles... and I guess the gold question here is how accurate do we want to go to simulate GL?

I opened this issue to discuss the idea, rather than to suggest that we really should do it. Maby it's not worth the effort.
What if we fix an upper bound on the read depth, say m, then construct an m * m transition matrix by scanning along some bam file (or just simulating reads from a read length distribution like what you suggest). Then to simulate read depths along a sequence, we do a random walk on read depths following the transition matrix. Conceptually fairly easy, and I think it would be faster than simulating reads (but could be wrong).

Ok! I see. So, I understand that what you are suggesting is that we have a precomputation of an actual read simulation (as I described) and from that, we extract the transition probabilities. Then, from those, we perform random walks per individual so that we get the coverage per site with the "dependence" property. I think it's a great idea. The issue I see here is that if the coverage per haplotype is different, then, I guess you'll have to do a simulation per every different mean_coverage.

You're right, each haplotype will need their own chain of read depths (either simulated from reads, or from a transition matrix). But most of the time I think we'll use this to simulate many replicates (i.e. lots of training instances for a neural network), so we ought to be able to cache lots of stuff.
Here's a proof of concept. Sampling the random walk is actually a lot slower than simulating the reads (at least, with the simple model in read_depths_from_reads()), but I'm sure we could improve things.
import matplotlib.pyplot as plt
import numpy as np

def read_depths_from_reads(
    *, num_reads: int, sequence_length: int, read_length: int, seed: int
) -> np.ndarray:
    rng = np.random.default_rng(seed)
    # 5' positions of reads that overlap the interval [0, sequence_length).
    _5p = rng.integers(low=-read_length, high=sequence_length, size=num_reads)
    # Left and right positions on the interval.
    left = np.maximum(0, _5p)
    right = np.minimum(sequence_length, _5p + read_length + 1)
    depth = np.zeros(sequence_length, dtype=int)
    for a, b in zip(left, right):
        depth[a:b] += 1
    return depth

def depth_transition_matrix(depth: np.ndarray, max_depth: int = None) -> np.ndarray:
    if max_depth is None:
        max_depth = np.max(depth)
    depth = np.minimum(depth, max_depth)
    # Count transitions.
    M = np.zeros((max_depth + 1, max_depth + 1))
    for j, k in zip(depth[:-1], depth[1:]):
        M[j, k] += 1
    # To avoid absorbing states, add an epsilon to each count.
    M += 1e-6
    # Make each row sum to 1.
    T = M / np.expand_dims(M.sum(axis=-1), -1)
    return T

def read_depths_from_transition_matrix(
    T: np.ndarray, sequence_length: int, seed: int
) -> np.ndarray:
    n, m = T.shape
    assert n == m
    rng = np.random.default_rng(seed)
    p0 = np.sum(T, axis=0) / np.sum(T)
    dp = rng.choice(n, p=p0)
    depth = np.zeros(sequence_length, dtype=int)
    depth[0] = dp
    for i in range(1, sequence_length):
        dp = rng.choice(n, p=T[dp])
        depth[i] = dp
    return depth

depths = read_depths_from_reads(
    num_reads=10_000, sequence_length=100_000, read_length=150, seed=1234
)
print(depths.mean(), depths.var(), depths.min(), depths.max())

T = depth_transition_matrix(depths)
#print(T)

limit = 1000  # just plot first 1000 bp
x = np.arange(limit)
plt.step(x, depths[:limit], linestyle="--", label="from reads")
for i in range(3):
    depths = read_depths_from_transition_matrix(T, sequence_length=100_000, seed=i)
    plt.step(x, depths[:limit], label=f"markov chain {i}")

plt.legend()
plt.show()

Another option I was considering is to have this function as an independent function that the user can call, and then, use it's output as input for sim_allelereadcounts().

I think I prefer this, as it more cleanly separates simulating depths from allele counts.
Before you go to far with this though, maybe it would be a good idea to check a couple of things?

What is the autocorrelation of read depths in empirical data? Probably we care about modern data (fixed read lengths) and ancient data (with log-normal-ish read length distribution)? It would be nice to understand under which circumstances it is useful to model non-independence of read depths. I.e. for what average read length and what density of segregating sites do we care?
Does the spatial distribution of depths in empirical data resemble what we've simulated? I suspect that in ancient DNA, data is wildly biased by sequence composition (e.g. much higher depths in GC rich regions). In particular, we probably underestimate the empirical variance in read depth by using the simple read depth simulations above.

Ok! I've decided to check these questions comparing to real data values the two simulation strategies suggested:

Independent: Drawing the depth per position from a Poisson distribution with a certain lambda
Linked: Simulating reads by sampling their starting location from a uniform distribution (start and end position fixed) and calculating the depth per position by checking the number of reads spanning each position

1. Check the statistics in a modern sample
First, I downloaded a cram file from a Brahui individual from the HGDP dataset as an example and the reference genome it was mapped to:
$ wget ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/HGDP/data/Brahui/HGDP00001/alignment/HGDP00001.alt_bwamem_GRCh38DH.20181023.Brahui.cram
$ wget ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa

Here are some details about the sequencing described in the HGDP paper:

Libraries were sequenced on Illumina HiSeq X machines, on single lanes for the PCR libraries and multiplexed across 12 lanes for the PCR-free libraries, producing paired-end reads of length 2×151 base-pairs (bp), a mean insert size of 447 bp and a mean coverage of 35.0X

So, I expect this individual to be around 35X with reads of length 151 bp.
To check that, I take the columns corresponding to the leftmost position of a few reads and their bp quality values
$ samtools view HGDP00001.alt_bwamem_GRCh38DH.20181023.Brahui.cram -T GRCh38_full_analysis_set_plus_decoy_hla.fa chr1:1000000-2000000 | awk '{print $4"\t"$11}' > readquality.txt

This is how the file created looks like
$ head readquality.txt
999857  070<<BBB7B0<B77BB7B<<7<<7<<<BB<<0707B<B7<B700777B77<<B7B77B<7<B<7B<BBB<BBBBBB777<0BB0B<<BB<0<B<7BBB<7B<<B<BBB<77<B7BB7BBB<<B<B<B7<BBBB<B7<B<B777<B7<BB<
999866  <B<<B7BB<BBB7B<77B'BBB70<<BBBBBBBBBB<7BB7<B<B0B<7BB<BB7<BB<<B<7B<B7B<B<0B07BBB7<BB7B00<B7770<B00BBBB<B<B<7<BBB7<B<<7077B0B7<B7770<'B70B0<B00B<0<7B07<00
999867  7<<777<BB<<7BB<B<BBBB<777BBBBBBBB777<077777B7B7BB<77B77B<BBBBBBBB7BB<7B<BB7B<BBB77B<B<BBB<7<<BBBBBBB<7BBBBB0<<7<BBBB<B<BBBBB<<<BBB<777BB7B<B7B7BB<B<7<<
999868  <B<7BB<<BB0<B77B7<<B777BB<B<B7BB<<BBBB7'BBBBBB0B<7BB7<0B7<<B7BBB770<<BB<BB<B0<BB7<0BBB7770<B7<B7BB<B0BB7B77B0BB0<770<B'B70B7777<B770'B0077<B00'B70<000'

With a python script, for the first 10000 reads, I create a pandas dataframe in which there is a line per bp of each read with its position and its bp quality, transformed from the phred ASCII code to a numeric value
qual = '!"#$%&\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~'
phred = {}

for i, q in enumerate(qual):
    phred[q] = i

readnum = []
positio = []
quality = []
end = None
with open("readquality.txt") as file:
    for i, line in enumerate(file):
        start, qual = line.strip().split()
        start = int(start)

        if end != None:
            if end < start:
                for p in range(end+1, start):
                    readnum.append(-1)
                    positio.append(p)
                    quality.append(0)

        for q in qual:
            readnum.append(i)
            positio.append(start)
            quality.append(phred[q])
            start += 1
            end = start
        if i > 10000:
            break

df = pd.DataFrame({"readnum" : readnum,
                   "positio" : positio,
                   "quality" : quality})

Finally, in R we can plot some of the basic statistics

Read length

df %>%
    count(readnum) %>%
    mutate(depthx10 = trunc(n/10)) %>%
    count(depthx10) %>%
    mutate(min_len = (depthx10*10)+1, max_len = (depthx10+1)*10, perc = n*100/sum(n)) %>%
    select(min_len, max_len, n, perc)

>  min_len max_len    n        perc
1        1      10    5  0.05254309
2       11      20    4  0.04203447
3       21      30    8  0.08406894
4       31      40  129  1.35561160
5       41      50   18  0.18915511
6       51      60    7  0.07356032
7       61      70   10  0.10508617
8       71      80    9  0.09457755
9       81      90   10  0.10508617
10      91     100    6  0.06305170
11     101     110    6  0.06305170
12     111     120    3  0.03152585
13     121     130    3  0.03152585
14     131     140    3  0.03152585
15     141     150    2  0.02101723
16<PHONE_NUMBER> 97.65657839

As expected, the majority of reads have lengths between 151 - 160. I checked manually, and they are actually all 151 lengths.

Mean depth

Also, as expected, the mean depth per bp is 37.7; very close to the 35X described in the method section.
df %>%
    count(positio) %>%
    pull(n) %>%
    mean()

> [1] 37.70846

Thus, the region that I selected is a good example to check some statistics.

Depth per base pair and distribution of depths per base pair

The profile of depth along the region selected looks like this
df %>%
    count(positio) %>%
    ggplot() +
    geom_point(aes(x = positio, y = n)) +
    geom_hline(aes(yintercept = mean(n)), color = "red") +
    geom_hline(aes(yintercept = median(n)), color = "blue")

Figure1. Depth (y-axis) per bp on the region selected (x-axis). The mean and the median are shown with horizontal red and blue lines.
The distribution of coverage per position is the following
df %>%
    count(positio) %>%
    ggplot() +
    geom_histogram(aes(x = n), binwidth = 1) +
    geom_point(data = data.frame(x = 0:100, y = dpois(x = 0:100, lambda = 37.70846)*37498), aes(x = x, y = y), color = "red")

Figure2. Histogram of the distribution of depths in the positions of the region selected. In red dots, it is shown the expected distribution of the depth for 37498 positions drawn from a Poisson distribution with a lambda value of 37.70846.
We can see that the fit is quite good, pointing towards Poisson distribution to be a good candidate to describe and simulate depths per base pair.

Depth correlation between contiguous positions

As a measure of the correlation between contiguous positions, I calculate the difference between the depth of a position and the next position. This is referred as the autocorrelation measure from now on.
df %>%
    count(positio) %>%
    arrange(positio) %>%
    mutate(n1 = lead(n)) %>%
    mutate(diff = n-n1) %>%
    filter(diff < 25, diff > -25) %>%
    ggplot() +
    geom_point(aes(x = positio, y = diff))

Figure3. Difference of depths between positions in n and n+1 for every position in the region.
We can see that they are highly correlated with differences that range between +-2 mostly
df %>%
    count(positio) %>%
    arrange(positio) %>%
    mutate(n1 = lead(n)) %>%
    mutate(diff = n-n1) %>%
    filter(diff < 10, diff > -10) %>%
    ggplot() +
    geom_histogram(aes(x = diff), binwidth = 1)

Figure4. Histogram of the difference of depths between positions in n and n+1.
and the mean and variance for this measure of autocorrelation is
df %>%
    count(positio) %>%
    arrange(positio) %>%
    mutate(n1 = lead(n)) %>%
    mutate(diff = n-n1) %>%
    filter(!is.na(diff)) %>%
    summarize(mean = mean(diff), var = var(diff))

>   mean      var
1 -8e-04 1.456838

2. Check the statistics in an independent simulation
If I simulate 37498 positions with a mean coverage of 37.71 from a poison distribution and compute this measure of autocorrelation, we get this picture
data.frame(pos = 1:37498,
           depth = rpois(n = 37498, lambda = 37.70846)) %>%
    mutate(depth1 = lead(depth)) %>%
    mutate(diff = depth-depth1) %>%
    ggplot() +
    geom_point(aes(x = pos, y = diff))

Figure5. Difference of depths between positions n and n+1 for every position in the independent simulated region.
data.frame(pos = 1:37498,
           depth = rpois(n = 37498, lambda = 37.70846)) %>%
    mutate(depth1 = lead(depth)) %>%
    mutate(diff = depth-depth1) %>%
    ggplot() +
    geom_histogram(aes(x = diff), binwidth = 1)

Figure6. Histogram of the difference of depths between positions in n and n+1 of the independent simulation.
and the values for the mean and variance are
data.frame(pos = 1:37498,
           depth = rpois(n = 37498, lambda = 37.70846)) %>%
    mutate(depth1 = lead(depth)) %>%
    mutate(diff = depth-depth1) %>%
    filter(!is.na(diff)) %>%
    #filter(diff < 25, diff > -25) %>%
    summarize(mean = mean(diff), var = var(diff))

>          mean      var
1 -0.0003733632 76.16626

We can already see, as expected, that the autocorrelation does not match between this simulation strategy and the real data. In fact, because the depths are Poisson distributed and independent, the variance (76/2 = 38, since the values calculate take positive and negative values) is the same as the mean (lambda) value. Thus, this simulation strategy is not good to simualte depths for all base pairs. Probably, the length of the reads create the autocorrelation observed in the real data that explains the smaller variance.
3. Check the statistics in an linked simulation
In python, I simualte around 9500 reads of length 151 that map in a region of length 37498 and derive the same statistics to compare with our observed results.
seq_len = 37498
n_reads = 9527-12
read_length = 151
df_sim = np.array([int(x) for x in np.random.uniform(low=0.0, high=seq_len, size=n_reads)])
pos = []
for s in df_sim:
    for i in range(s, s+read_length):
        pos.append(i)

df_sim = pd.DataFrame({"pos" : np.array(pos)})

and back to R to plot the autocorrelation statistic

df_sim %>%
    count(pos) %>%
    arrange(pos) %>%
    mutate(n1 = lead(n)) %>%
    mutate(diff = n-n1) %>%
    ggplot() +
    geom_point(aes(x = pos, y = diff))

Figure7. Difference of depths between positions n and n+1 for every position in the Linked simulated region.
df_sim %>%
    count(pos) %>%
    arrange(pos) %>%
    mutate(n1 = lead(n)) %>%
    mutate(diff = n-n1) %>%
    ggplot() +
    geom_histogram(aes(x = diff))

Figure8. Histogram of the difference of depths between positions in n and n+1 of the Linked simulated region.
And the statistics are
df_sim %>%
    count(pos) %>%
    arrange(pos) %>%
    mutate(n1 = lead(n)) %>%
    mutate(diff = n-n1) %>%
    filter(!is.na(diff)) %>%
    summarize(mean = mean(diff), var = var(diff))

>     mean       var
1    0 0.5069734

Although the variance is smaller than the real data one, the fit is much better compared to the other simulation strategy. The difference in variance between the real data and the linked simulated data could potentially be explained by on other factors breaking the autocorrelation among neighbouring regions (GC content, repetitive content...) or just pure noise. As we can see in Figure1, there are certain regions that suddenly change depth which might be the responsible for the bigger variance in the original data. In any case, simulating the fluctuations of depth by other causes than just randomness as discussed would be much more complex to simulate and I believe it is far from the scope of our package. I believe that we are safe with the linked simulation strategy.
4. Check the statistics in a subsample of the whole observed region
Sometimes, we might not want to simulate the depth for all base pairs of a given region, and only a subset (polymorphic sites). For example, in humans, the proportion of polymorphisms is 1e-4. So, what is the autocorrelation of the depths between consecutive base pairs for only polymorphic sites?
We can check the autocorrelation of depths in different subsets of the original dataset of positions.  If we subsample 1% (1e-2) positions in that region, we get that
df %>%
    count(positio) %>%
    sample_frac(size = 0.01) %>%
    arrange(positio) %>%
    mutate(n1 = lead(n)) %>%
    mutate(diff = n-n1) %>%
    filter(diff < 50, diff > -50) %>%
    summarize(mean = mean(diff), var = var(diff))

>       mean     var
1 -0.0802139 68.1276

We can see that the variance value of our measure for autocorrelation is much more similar to the simple independent simulation strategy. Thus, this indicates that for spaced positions in the genome simulated, it is ok to use the naive Poisson distribution. However, if we want to simulate all positions in a given region, we might want to use the linked strategy. There might be a direct relationship between the level of autocorrelation and the length of reads and the spacing between positions and from that, we could decide which strategy to use. However, it is much easier to just recommend the linked strategy of simulations in case of doubt (at some point, I'll test the running time for both strategies).

Cool! This is very useful! Let's discuss in person this afternoon.

I calculated depths with samtools depth -a for the .cram file mentioned above and made autocorrelation plots. My guess is that long stretches of zero depth are an important factor in the autocorrelation.
import numpy as np

def autocovariance(x: np.ndarray):
    """Return the autocovariance (ACVF) of 1-d array x."""
    # Centre the array.
    x = x - np.mean(x)
    # Do ACV in transform space.
    n = 2 ** int(np.ceil(np.log2(len(x))))
    x_trans = np.fft.fft(x, n=2 * n)
    acv_trans = x_trans * np.conjugate(x_trans)
    return np.fft.ifft(acv_trans)[: len(x)].real

def nonzero_segments(depths: np.ndarray, gapsize=10000, maxsegsize=5*1000*1000):
    """
    Break the input into segments of nonzero values.

    Segment breakpoints occur at runs of zeros longer than `gapsize`.
    For ACVF computational tractibility (max memory needed), segments are also
    broken into pieces of length `maxsegsize`.
    """
    nz = np.nonzero(depths)[0]
    dp_list = []
    start = 0
    i = 0
    for j in nz:
        if j - i > gapsize:
            if start != i:
                seg = depths[start: i]
                segs = np.array_split(seg, 1 + len(seg) // maxsegsize)
                dp_list.extend(segs)
            start = j
        i = j
    return dp_list

if __name__ == "__main__":
    import sys
    import matplotlib.pyplot as plt

    if len(sys.argv) != 3:
        print(f"usage: {sys.argv[0]} depths.tsv acf.pdf")
        exit(1)

    file_in = sys.argv[1]
    plot_out = sys.argv[2]

    # Depths from:
    #   samtools depth -a -r chr22 -o chr22_depths.tsv file.bam
    depths = np.loadtxt(file_in, usecols=2, dtype=int)

    # Don't bother with the ACF past this distance.
    dist_max = 7500

    dp_list = nonzero_segments(depths)
    acf = np.zeros(dist_max)
    for dp in dp_list:
        if len(dp) < dist_max:
            continue
        acf += autocovariance(dp)[:dist_max]
    # Normalise to get autocorrelation.
    acf /= acf[0]

    fig, ax = plt.subplots(figsize=plt.figaspect(9 / 16))
    fig.set_tight_layout(True)
    ax.plot(acf)
    ax.set_ylim([-0.01, 1.01])
    ax.set_ylabel("Autocorrelation")
    ax.set_xlabel("Distance (bp)")
    ax.set_title("Autocorrelation of sequencing depth")

    # Read length and mean insert size.
    ax.vlines([151, 447], *ax.get_ylim(), colors="black", linestyles=":")

    fig.savefig(plot_out)

chr22

chr1

That's great! I can see you marked with vertical lines 151 and 447 bp distance. I guess another interesting distance point would be (151*2)+447 = 749 bp since it's the total length of a pair end read. But I would say that the sharpest decrease in autocorrelation can be seen in the first 151 bp (we can see a change after the first horizontal dotted line, especially in chromosome 22). It's curious that the autocorrelation is quite different for each chromosome: in chromosome 22 the autocorrelation drops much faster than chromosome 1.
In any case, we can try to implement both solutions: independent (Poisson) depths and linked depths simulating reads, the former for spaced polymorphic (and error) positions and the latter to simulate the entire desired region.


Update flickr-uploadr.rb for latest download URL

I don't know how to revise the pull request commit, so I've tacked on another commit to fix the license to :gratis. If it looks good, I hope you can squash the branch and merge it :)

If you can’t squash your them, I can do it for you.
Just noticed a small thing, though. Seeing as url doesn’t point to a specific version, this cask should use version :latest and sha256 :no_check. See an example.

Aah, thanks for the link, didn't realize that git push -f is the trick!
I've pushed new commit with version :latest and sha256 :no_check. Please review.

Looks good. But double quotes are reserved for cases where string interpolation is needed, so in url they should be changed to single quotes (should’ve made that explicit in the previous comment).

Aah, sorry I didn't notice this code convention. Just pushed a revised commit.

Thank you.


Pin depends

This is a rebase of @rizo's PR: https://github.com/ocamllabs/opam-monorepo/pull/159
Fixes https://github.com/ocamllabs/opam-monorepo/issues/153

I just rebased and added a changelog entry.

Thanks for the multiple rebase, I'll revive this PR.
There are a couple things that I'd like to improve, in particular the interface for Opam_solve and code sharing for pull and pulling the pin depends.

I made a few improvements:

Clean up the interface of Opam_solve and let it deal with pin_depends rather than merging them into local_opam_files in Lock. calculate now expects a map for local packages and a map for pin-depends, abstracting everything else away from Lock.
I moved the pin_depends extraction further down the chain so that we only load opam states once
I fixed a small quirk in Opam_solve.get_opam_info so that the pin-depends map is only looked up once instead of twice (find_opt instead of mem + find)
I forbid having local packages in a pin-depends field. Before lock would have allowed it but the pin would effectively be ignored since local packages are removed from the solution in the lockfile. There's no way to properly support this as you either want it local and use whatever you have in your repo or use a pin that will be pulled in your duniverse. Having both will lead to a dune error because you have the same package twice in your workspace.
I factored opam pulling code for pin-depends and the pull command
I added deduplication of the repository's pin-depends (the same pin could appear several times in different opam files)
I added rejection of different pins for the same package
we group pin-depends per URL, meaning if several packages are pinned to the same URLs, the repo is pulled once instead of once per package
I implemented @emillon suggestion to use Bos to generate a temporary directory

I need to test this a bit.
@TheLortex would you mind trying this out for mirage and see if it still works?
@emillon could you give this another round of review?


Update rhel7 builder image go version

Signed-off-by: Sebastian Sch<EMAIL_ADDRESS>

@zshi-redhat PTAL

/lgtm
This is an error introduced when adding -modcache in make clean entry that:
make clean fails with golang-1.10: "flag provided but not defined: -modcache"
The proposed change is to update base image to use golang-1.13.
@ahalim-intel PTAL


The UWP Template no longer contains the Publish Symbos path task

I am adding it by hand

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c024acf3-a58a-74c8-25b1-e5c88c85583a
Version Independent ID: 81403e1d-947a-1f7d-f4b7-898105abd745
Content: Set up automated builds for your UWP app - UWP app developer
Content Source: windows-apps-src/packaging/auto-build-package-uwp-apps.md
Product: windows
GitHub Login: @laurenhughes
Microsoft Alias: lahugh

or the publish to store task

Thanks for your feedback, and I apologize for the long delay in responding. Since the time you submitted this feedback we have completely rewritten the article to align to the latest developer experience and recommendations. If you still have concerns/feedback, please feel free to reopen this issue with more details.


State Problem: Unexpected Status

Trying to be low maintenance here. 😁 👼
Consider this PR:
https://github.com/ExtendedXmlSerializer/NextRelease/pull/8
Notice that it makes use of PR-Labeler
At first, there is no label and NextRelease states there's a problem.
So far so good.
Then PR-Labeler identifies and applies enhancement, and NextRelease goes green.
I then apply a breaking change label as it is a feature that introduces one.
NextRelease does not update in the Pending Release:

While the next version is accurate (1.0.0), the item in the list is denoted as a feature.  While that is technically accurate, it is also a breaking change as well.  The expectation here is that the item would be promoted to a breaking change to reflect the new version.

This appears to be a state problem in the NextRelease editor.
It published as expected:
https://github.com/ExtendedXmlSerializer/NextRelease/releases/tag/1.0.0

@Mike-EEE Thanks for reporting this. We'll get this turned around!


add test as npm script

@thepiwo  With this issue do you mean, to add new npm script test, that will run all .js files from the test folder? If yes how is this different from the aeproject test command?

Yes, that's what I would expect. Its helpful as npm scripts are well integrated in many idea/editors and aeproject is not yet.

This issue is fixed in PR:
https://github.com/aeternity/aepp-aeproject-js/pull/256#pullrequestreview-308445592


Mediapipe Holistic Nondeterministic Behaviour

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
None
OS Platform and Distribution
Windows 10
MediaPipe Tasks SDK version
No response
Task name (e.g. Image classification, Gesture recognition etc.)
Holistic landmarks detection
Programming Language and version (e.g. C++, Python, Java)
Python
Describe the actual behavior
Non-deterministic Holistic landmarks detection with track mode off
Describe the expected behaviour
Holistic landmarks detection should be deterministic and should give same result everytime.
Standalone code/steps you may have used to try to get what you need
I am using mediapipe python version 0.10.7. I am using code below to initialize Holistic model:

import mediapipe as mp
mp_holistic = mp.solutions.holistic
holistic = mp_holistic.Holistic(min_detection_confidence=0.3, static_image_mode = True)

I am reading images in a folder sequentially and predicting keypoints using holistic.process(image). I am trying to get rid of images that doesn’t detect right hand keypoints. I run this for a folder in loop. If I re-run this loop with shuffled data, results are changing. My dataset consists of 24k images. In first run, model failed to detect right hand keypoints in 305 images. In the second run, shuffled one, this number becomes 320. I do not get why this happens. This seems non-deterministic and model is not in track mode. Does mediapipe non deterministic?

Other info / Complete Logs
No response

Hi @tugcekiziltepe,
We have upgraded our MediaPipe Legacy Solutions to include new features like the Holistic Task API. You can still use the old solutions if you prefer, but we recommend checking out the new MediaPipe options. New Task APIs make it easier to build and customize machine learning solutions for your applications.
Could you give the new HolisticLandmarker task APIs a try and let us know if you encounter any issues?
Thank you!!


About Page

As a site owner, I would like a page that links to this code base, credits Pump Out and KyleTT, and includes contact information for me.

Merged in #30


New package: ADIOS2 v0.1.0

Registering package: ADIOS2
Repository: https://github.com/eschnett/ADIOS2.jl
Created by: @eschnett
Version: v0.1.0
Commit: a85c01795445bb1beecec8b97f9f026d3d7f5ddf
Git reference: HEAD

This package ADIOS2 wraps the ADIOS2 library https://github.com/ornladios/ADIOS2. It should have the same name, all upper case characters. The case is similar to the HDF5 library.


Show an AlertDialog in Android & iOS

Isn't there an API for showing a simple AlertDialog in Android & iOS?
I only see code for desktop platform.

You can use the Popup composable for both Platforms but it's a different function with different parameters, so i build myself an expected DialogContainer that then actually implements the Popup composable.

@Nailik Can you share your code for that?

Expected:
@Composable
expect fun DialogContainer(content: @Composable () -> Unit)

Actual Android:
@Composable
actual fun DialogContainer(content: @Composable () -> Unit) {

    Popup(content = content) //via popupProperties you can handle outside click and back press etc

}

Actual iOS:
@Composable
actual fun DialogContainer(content: @Composable () -> Unit) {

    Popup(content = content)

}

Usage in shared (rebuild the material3 spec dialog):
@Composable
private fun Dialog(
    modifier: Modifier = Modifier,
    onDismissRequest: () -> Unit,
    icon: (@Composable () -> Unit)? = null,
    headline: @Composable () -> Unit,
    supportingText: (@Composable () -> Unit)? = null,
    confirmButton: @Composable () -> Unit,
    dismissButton: (@Composable () -> Unit)? = null,
    dismissOnOutside: Boolean = true,
    showDivider: Boolean = false
) {
    DialogContainer {

        //Scrim
        Surface(
            modifier = modifier
                .fillMaxSize()
                .clickable(enabled = dismissOnOutside, onClick = onDismissRequest),
            color = MaterialTheme.colorScheme.scrim.copy(alpha = 0.5f)
        ) {

            //Container
            Surface(
                modifier = Modifier
                    .widthIn(min = 280.dp, max = 560.dp)
                    .padding(48.dp)
                    .wrapContentHeight()
                    .clip(RoundedCornerShape(28.dp))
                    .clickable(enabled = false) { },
                shape = RoundedCornerShape(28.dp),
                color = MaterialTheme.colorScheme.surface,
                tonalElevation = 6.dp
            ) {
                Column(
                    modifier = Modifier.padding(24.dp),
                    horizontalAlignment = Alignment.Start
                ) {

                    Column(
                        modifier = Modifier
                            .weight(weight = 1f, fill = false)
                    ) {
                        if (icon != null) {
                            Box(
                                modifier = Modifier
                                    .padding(bottom = 16.dp)
                                    .fillMaxWidth(),
                                contentAlignment = Alignment.Center,
                            ) {
                                Box(
                                    modifier = Modifier.size(24.dp),
                                ) {
                                    CompositionLocalProvider(
                                        LocalContentColor provides MaterialTheme.colorScheme.secondary,
                                    ) {
                                        //Icon (optional)
                                        icon()
                                    }
                                }
                            }
                        }

                        CompositionLocalProvider(
                            LocalContentColor provides MaterialTheme.colorScheme.onSurface,
                            LocalTextStyle provides MaterialTheme.typography.headlineSmall
                        ) {
                            //Headline
                            headline()
                            Spacer(modifier = Modifier.height(16.dp))
                        }

                        if (supportingText != null) {
                            CompositionLocalProvider(
                                LocalContentColor provides MaterialTheme.colorScheme.onSurfaceVariant,
                                LocalTextStyle provides MaterialTheme.typography.bodyMedium.copy(textAlign = TextAlign.Start)
                            ) {
                                //Supporting text
                                supportingText()
                            }
                        }

                        //Divider (optional)
                        if (showDivider) {
                            Divider()
                        }
                    }

                    CompositionLocalProvider(
                        LocalContentColor provides MaterialTheme.colorScheme.primary,
                        LocalTextStyle provides MaterialTheme.typography.labelLarge.copy(textAlign = TextAlign.Start)
                    ) {
                        //Text button
                        Row(
                            modifier = Modifier.padding(top = 24.dp),
                            horizontalArrangement = Arrangement.spacedBy(8.dp),
                            verticalAlignment = Alignment.CenterVertically
                        ) {
                            Spacer(modifier = Modifier.weight(1f))

                            //Dismiss
                            dismissButton?.invoke()

                            //Confirm
                            confirmButton()
                        }
                    }

                }

            }
        }

    }

}

Making AlertDialog available from common sourceset is in our plan. No exact ETA yet, most likely Q4
Closing as duplicate of #762

@Nailik Thanks for your code! It works

@Nailik thanks for the code, it works great. This should be standard library functionality.

At least AlterDialog itself is available since 1.8 see https://github.com/JetBrains/compose-multiplatform/issues/762#issuecomment-1733035632
Also in my Code there might be issues with the buttons if the device is not big enough for the text ;)

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Crashes in Swift 5 in Debug mode

Summary
In Swift 5, I get some fatalErrors, such as:

Fatal error: UnsafeRawBufferPointer.load out of bounds
Fatal error: load from misaligned raw pointer

These are only present in DEBUG mode, though, so maybe it's fine?
Steps to Reproduce
The errors are reproducible using ZIPFoundation's test suite:

Set SWIFT_VERSION to 'Swift 5'
Set the Build Configuration in the Scheme to 'Debug'

Expected Results
All tests pass
Actual Results
Some fatalErrors related to UnsafePointers are triggered
Regression & Version
Xcode 10.2 and Swift 5

Hi,
Thanks for pointing this out - The current development contains a bug that I introduced when I switched to the Swift 5.0 version of withUnsafeBytes:. The crash will be fixed after #105 got merged.

Oh, that's great to hear!
I'll close this then ✅


[gitea.io] Website is only displayed in Chinese.

Visiting https://gitea.io/ directly or e.g. via Google always forward to https://gitea.io/zh-cn/. I see no language select. It should be english by default. (https://gitea.io/en-us/) Btw. when I enter: https://gitea.io/index.html it works. But https://gitea.io/ is always zh-cn. I would also suggest to avoid language in URL.
Tested in Firefox and Chrome 105 on Windows 10 (english is not set as main language, but also not chinese).

Please fire it in https://gitea.com/gitea/website/issues


[conda-lock] Conda lock file generation is currently broken

...and I am looking into it (see fail ). Bear with me :bear:

the issue is raised upstream at https://github.com/conda/conda-lock/issues/396 - reasons are unknown to me, but what is certain is that the conda lock generation works fine with conda=22.9 and the latest, conda=23.3, there is an inherent issue with conda=23.1 so now we are always updating conda base before running the test :+1:

changes from #3148 did the trick locally, and upon triggering manually the flow, and I assumed things would go fine during the cron run too - I assumed wrong and we still keep seeing the Error 143 when the CI runs scheduled. I opened https://github.com/conda/conda-lock/issues/399 to as what that code means, for starters - will have to poke around some more; for now, the file we have is correct and sane, but it's getting older with every day passing...


Add foundation sec members as security contact to gradle-plugin

Description

I would like to propose our help with regards to security maintenance.
Our team at CloudBees will be able to react on security issue reports, by definition time-sensitive. This way we can be notified on the event of a security report to help on the resolution, collaboration with the security team, synchronization, etc.
We have done this already for a number of plugins:
https://github.com/jenkins-infra/repository-permissions-updater/search?q=foundation_security_members&unscoped_q=foundation_security_members
If anybody sees a problem with this, please let me know.
Thank you.

Repo implied: https://github.com/jenkinsci/gradle-plugin
CC @gboissinot, @wolfs for approval from an existing maintainer.
CC @daniel-beck as the security officer since this PR is changing security contacts.

Submitter checklist for adding or changing permissions

Always

[x] Add link to plugin/component Git repository in description above

For a newly hosted plugin only

[ ] Add link to resolved HOSTING issue in description above

For a new permissions file only

[ ] Make sure the file is created in permissions/ directory
[ ] artifactId (pom.xml) is used for name (permissions YAML file).
[ ] groupId / artifactId (pom.xml) are correctly represented in path (permissions YAML file)
[ ] Check that the file is named plugin-${artifactId}.yml for plugins

When adding new uploaders (this includes newly created permissions files)

[x] Make sure to @mention an existing maintainer to confirm the permissions request, if applicable
[x] Use the Jenkins community (LDAP) account name in the YAML file, not the GitHub account name
[ ] Make sure to @mention the users being added so their GitHub account names are known if they require GitHub merge access (see below).
[x] All newly added users have logged in to Artifactory at least once

Reviewer checklist (not for requesters!)

[ ] Check this if newly added person also needs to be given merge permission to the GitHub repo (please @ the people/person with their GitHub username in this issue as well). If needed, it can be done using an IRC Bot command
[ ] Check that the $pluginId Developers team has Admin permissions while granting the access.
[ ] In the case of plugin adoption, ensure that the Jenkins Jira default assignee is either removed or changed to the new maintainer.
[ ] If security contacts are changed (this includes add/remove), ping the security officer (currently @daniel-beck) in this pull request. If an email contact is changed, wait for approval from the security officer.

There are IRC Bot commands for it

@wolfs are you OK with this addition?
Do you still want to maintain the Gradle Jenkins plugin?
Thanks!

@batmat I am fine with this addition. I am still maintaining the gradle plugin.
How does this work after the change? Will I also get notifications about security problems automatically?

Will I also get notifications about security problems automatically?

We're notifying security contacts and maintainers via Jira. Either can develop security fixes, approve staging, etc. (which bypasses regular permission checks).

Thank you all
On Wed, Nov 18, 2020 at 2:30 PM Tim Jacomb<EMAIL_ADDRESS>wrote:

Merged #1708
https://github.com/jenkins-infra/repository-permissions-updater/pull/1708
into master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jenkins-infra/repository-permissions-updater/pull/1708#event-4010672577,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFIMFIXZAGCJCQCZCRIBV4TSQPEABANCNFSM4SIQTY7A
.


[#Wave-Control: Add NetSuite]: Settings Configuration in NewDot: Import - Add Custom Records/Lists

Details

Covers the implementation of Custom segments/records and Custom lists.
It covers Add workflow and has changes from the previous PR (https://github.com/Expensify/App/pull/44792) - Listing, View , Edit and Remove.

Collect workspace with accounting enabled.
NetSuite beta enabled for the user.  You can return true in the canUseNetSuiteIntegration method.
Setup the NetSuite connected workspace.
It has pending translations.
It also contains the refactor of custom lists from text input to picker in this PR.

Fixed Issues

$ https://github.com/Expensify/App/issues/43437
PROPOSAL:
Tests

[ ] Verify that no errors appear in the JS console

Tests for List, View, Update, Remove

Open the app with the pre-setup workspace.
Go to Accounting from the LHS
Verify that you're able to see Net Suite in the active connections along with the Oracle NetSuite logo.
Click on the Import button.
Verify that the list of options appear as per the linked design doc.
Click on the Custom segments/records without any data.
You should see an the empty screen with the folder icon, and the text as following:

Now add a few records in Custom segments/records using the Add custom segments/records button.
Come back to the NewDot to the Import screen.
Click on the Custom segments/records with data now.
Verify that you can see the list of segments. Each segment should have the title Custom segment/record and the segmentName that you added.
Verify that you can see the description as the subtitle.
Click on any one of the records, you see see the view screen with all the fields: Name, Internal ID, Script and Mapping
Click on any one of the fields, and you should see the edit screen. For mapping you should see the dropdown with Tags and Report option. For the other fields it should be a text field.
The form should throw an error if the field is empty.
Update the value for these fields, and then click on Save.
It should update the field value and take you back to the record view screen.
Now to test delete, click on Remove and you should see a confirmation modal for remove.
Clicking on Remove should redirect back to the list screen and the record removed.
Repeat steps 6-19 for Custom lists in the import screen

Tests for List, View, Update, Remove

Start from step 6 in the previous test steps.
You should see the add workflow mentioned here in the design doc.
Verify for each step the page title, field title, field name, data type, subtext if any, error message.
Verify whether all the data you entered shows correct in the last confirmation step.
Clicking on confirm button should take you back to the listing screen with the newly added record.
Repeat the same steps for Custom list.

Offline tests

QA Steps

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
https://github.com/Expensify/App/assets/3069065/f36b22d3-18a4-44f3-a2be-25dc2a4a2c0f
Steps
https://github.com/Expensify/App/assets/3069065/acb70e66-4c3d-4952-8966-36b9190f31cf
Steps Completion

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari
https://github.com/Expensify/App/assets/3069065/de42c895-363f-4730-9f38-34d13f549048
https://github.com/Expensify/App/assets/3069065/0fc57bbb-5654-4622-a8f8-3c764bb76e22

MacOS: Chrome / Safari
https://github.com/Expensify/App/assets/3069065/511a018c-2684-4d07-8cf4-cc536b6a064b
https://github.com/Expensify/App/assets/3069065/b7d60977-f502-492e-bd08-e9a28dc1b95b

MacOS: Desktop
https://github.com/Expensify/App/assets/3069065/3bc3d806-4784-4f04-a39e-b2aba6dce86d

@akinwale Please ignore this PR.
@shubham1206agra @yuwenmemon This is the PR built on top of https://github.com/Expensify/App/pull/44792. You can either review these separately or just this large one. This might have smaller UI, etc. issues, which I'll work on as follow up if they're not blocking.

@Expensify/design did we decide to not use a background color behind these kinds of empty states? I could have sworn we ditched it, but maybe that was just report fields?

Either way, I kind of think all of these should be consistent?

@shawnborton The figma for these screens has no background color. But I was using an existing component so wasn’t sure if I should remove it. Thee components are related to Workspace pages, categories, etc.
What’s the verdict background color or no background color ?

I think I would lean on no background color, but let's see what the rest of the design team thinks. Also, where else is this component used?

I think I would lean on no background color, but let's see what the rest of the design team thinks. Also, where else is this component used?

For Empty Search View, Empty Workspace Categories View. I can easily remove the background. Let me know once we've decided that.

Agree with no background color! Those empty states being referenced for Expenses and Categories are going away anyways.

Okay cool, let's make the BG transparent then.

@shubham1206agra I've applied your refactor diff. Can you please review.

@yuwenmemon There are a few challenges adding the SelectionList with FormWrapper.

Scroll issues and VirtualizedLists should never be nested inside plain ScrollViews (which @shubham1206agra resolved) but we've enabled shouldUseScrollView={false} in ConnectionLayout.

On reload of the page setting initiallyFocusedKey as the value comes async from Onyx.

Shubham helped fix removing the Button for SelectionList, but onInputChange and onSubmitEditing are called async, and hence even before the Onyx is updated the submit is called. Hence we had to add Next button in the Mapping step (TAG, REPORT_FIELD) selection. We might have to revisit on how onSubmitEditing and shouldSubmitForm work

Onyx pending update works on field keys. But we're essentially doing a bulk update. Hence, we don't have an easy way to set pendingFields for a single record. The API is also queued for the whole array. I tried adding index and other checks but they don't work.

@yuwenmemon @Expensify/design Here's the revised video.
https://github.com/Expensify/App/assets/3069065/5f26aa6f-3e60-447b-9dac-f11533c794b3
https://github.com/Expensify/App/assets/3069065/7f098c49-a6ea-4fda-8b9e-e1fc31852c7a

Looks pretty good to me.

Removed draft logic as discussed here.

🚀 Deployed to staging by https://github.com/yuwenmemon in version: 9.0.6-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
failure ❌

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to staging by https://github.com/yuwenmemon in version: 9.0.6-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Cherry-picked to staging by https://github.com/Julesssss in version: 9.0.6-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
failure ❌

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Cherry-picked to staging by https://github.com/Julesssss in version: 9.0.6-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by https://github.com/thienlnam in version: 9.0.6-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/thienlnam in version: 9.0.7-8 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅


Adding wayland launchers

Added support for 3 additional menus. Also added automatic browser detection with 8 different browsers in a similar way to the previous menu detection. Finally, user can now use the -m and -b flags to use their own menus and browsers in case they are still not in the detection list.

Hi @KlosRadieschen! I appreciate the pull request, however I haven't used Arch, i3, or anything related to this project for several years now, so I can't reasonably review this or merge it in, and will be archiving this repo. I hope you'll still find use with this code in your own copy of the repo. Thanks ❤


Update Readme to include the mention about matching package.json syntax for authors/contributors

I was initially having issues with generating codeowners for our project, when I realized my syntax for contributors in package.json was incorrect. Updating the readme here so that other don't make that mistake.

Pull Request Test Coverage Report for Build 307498891

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 299631222:
0.0%

Covered Lines:
197

Relevant Lines:
197

💛  - Coveralls


Schedule meet-up for Google Fiber

done and announced: https://www.meetup.com/Triangle-Code-for-America/events/243684781/
Please sign-up if you plan to attend.

Did you look at my Google doc (short invitation)?


RE for adding Additional Attack Effects

As the title, this is a suggestion to implement a way to add Additional Attack Effects in NPC strikes through Rule Elements.
Low Priority, as you can imitate such effects through the Note Rule Element, with an example as follows:
{
  "key": "Note",
  "selector": "attack",
  "text"<EMAIL_ADDRESS>  "predicate": [
    "item:slug:driftwood-hook"
  ],
  "title"<EMAIL_ADDRESS>  "visibility": "owner"
}

The only issue with that is it does not show up on the sheet, the addition is only visible once you made the attack.

The additional effects are in fact nothing more than roll notes.

Actually, the PR is framed poorly, but this seems to be a feature request for roll notes that impact attacks to be visible from the character sheet. Which is a perfectly valid request.

feature request for things that impact attacks to be visible from the character/npc sheet
It is indeed.

I'd be happy with any solution, just as long as it allows us to actually have the same sort of experience as with normal attack effects (ex. when you add Grab and then tick it in the additional effects menu of an attack), but with the ability to be predicated upon.


Issue selection design enhancements

More inspo from Things 3. Here's how they do it:

Unselected uses hollow tag icon
Selected fills label icon
Title is white

Now I don't think we need to go 100% this, but would be cool to explore designs that make selections more obvious. Ideas:

White title label w/ 2pt border radius for unselected label
Selection fills the label and updates the text color based on background (include checkmark)

cc @kocheck

That’s a beautiful blue
Sent with GitHawk

I love that idea @j-f1   I think if we keep the labels contrast in mind it would make the feature more valuable :)
Sent with GitHawk


Fix Zally issue creation

Remove preview media type from a request.
Relates to #628, #623 and #607

👍

:+1:

@vadeg doesn't this most importantly fixes #667.

@tkrop yes. I will close it.


没有高亮，不能格式化？

with the extension Beautify installed in your VS Code add a settings like this
    "beautify.language": {
        "html": [
            "wxml"
        ]
    }


Refactor engine switching when Ray is installed

Feature or Bugfix

Feature/Refactor

Detail

Refactor engine switching when Ray is installed
Added unit tests

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: f3c0b52aa2519bf57712bb15135f44c28cfddac9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubStandardCodeBuild8C06-llutOAimTATs
Commit ID: f3c0b52aa2519bf57712bb15135f44c28cfddac9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: bdbd40ab0418033708d458a30b040b06e68ef751
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubStandardCodeBuild8C06-llutOAimTATs
Commit ID: e60af1291d2016cf398889d922e45851891d707c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: e60af1291d2016cf398889d922e45851891d707c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubLoadTests5656BB24-s6u9F3qN9oFy
Commit ID: e60af1291d2016cf398889d922e45851891d707c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubLoadTests5656BB24-s6u9F3qN9oFy
Commit ID: e2c4378c6e2819c77563f4f9f1c401875aeab829
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: e2c4378c6e2819c77563f4f9f1c401875aeab829
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubStandardCodeBuild8C06-llutOAimTATs
Commit ID: e2c4378c6e2819c77563f4f9f1c401875aeab829
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


fix(merge documents routes into index): Merge documents routes into index

Merge documents routes into index

:tada: This PR is included in version 2.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


fix(guide): Fix book recommendation folder structure for mis-named file

I merged a PR in the book recommendation section without noticing the file and folder structure didn't conform to the right format (sorry 😞 ) - this PR fixes it.

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

I can not figure out what changed.  Did you change an existing file's name or add a new file containing the information?

@RandellDawson - the file was named "Swift" and was floating in the book-recommendations folder, so I made a "swift" directory and renamed the file to index.md. It should have broken the build, but some of those tests may be off with the repo merge?


Cisco AnyConnect VPN with DNS

AnyConnect will setup utunN interface as well as DNS resolvers I could see with scutil --dns.
This doesn't seem to be propagated into VMs. Is there a way to make it work, do I miss some configuration or it's just not supported?
I also tried to put some known DNS servers to my ~/.lima/default/lima.yaml:
dns:
- x.x.x.x
- y.y.y.y

And then limactl stop && limactl start - but my VM still doesn't have it in the /etc/resolv.conf neither am I able to use curl to get from a private host meaning it is not being injected to the VM in any other way. What am I missing?

This doesn't seem to be propagated into VMs. Is there a way to make it work, do I miss some configuration or it's just not supported?

You have to do this manually via the dns settings in lima.yaml. The only name servers that should be propagated automatically are the ones from the host en0 interface.

my VM still doesn't have it in the /etc/resolv.conf

On a systemd setup they will not be added to /etc/resolv.conf, but should still be configured for the resolved service. You can check with resolvectl status; it should show additional servers beyond <IP_ADDRESS>, e.g.
$ resolvectl status
Global
       Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 2 (eth0)
    Current Scopes: DNS
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: <IP_ADDRESS>
       DNS Servers: <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>

neither am I able to use curl to get from a private host

I suspect the issue is that the VM cannot route to the additional name servers because they too are on the VPN. I don't have an idea right now how to make that work, unfortunately.

I figured the latter question out - modifying list of DNS servers are not having any effect, I had to modify /etc/netplan/50-cloud-init.yaml and now it picked it up after a restart. My DNS server are indeed private but somehow VM can perfectly access it as well as other private hosts behind VPN. This seems to be a DNS only issue.
However, that doesn't solve it entirely. Being able to provide DNS servers is not enough - AnyConnect as well as other VPN clients are using conditional resolvers. There are different DNS servers for different search domains. For example my scutil --dns looking the following way (stripped out private details from the output):
resolver #1
  search domain[0] : enterprise.com
  ...
  search domain[19] : enterprise19.com
  nameserver[0] : <IP_ADDRESS>
  nameserver[1] : <IP_ADDRESS>
  if_index : 6 (en0)
  flags    : Request A records
  reach    : 0x00000002 (Reachable)

resolver #2
  domain   : enterprise2.org
  nameserver[0] : x.x.x.x
  nameserver[1] : y.y.y.y
  flags    : Supplemental, Request A records
  reach    : 0x00000002 (Reachable)
  order    : 1

resolver #17
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

resolver #23
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300200

resolver #24
  domain   : 8.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300400

resolver #25
  domain   : 9.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300600

resolver #26
  domain   : a.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300800

resolver #27
  domain   : b.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 301000

DNS configuration (for scoped queries)

resolver #1
  nameserver[0] : <IP_ADDRESS>
  nameserver[1] : <IP_ADDRESS>
  if_index : 6 (en0)
  flags    : Scoped, Request A records
  reach    : 0x00000002 (Reachable)

resolver #2
  search domain[0] : enterprise.com
  ...
  search domain[19] : enterprise19.tech
  nameserver[0] : x.x.x.x
  nameserver[1] : y.y.y.y
  if_index : 17 (utun2)
  flags    : Scoped, Request A records
  reach    : 0x00000002 (Reachable)
  order    : 1

These x.x.x.x and y.y.y.y DNS servers might not know about anything outside the VPN so I can't just use them as catch-all DNS. Adding both public and private DNS servers to /etc/netplan/50-cloud-init.yaml doesn't seem to work, probably because they are expected to be HA replicas instead of actually different authorities that could return different results.

Upon further research - split DNS as well as systemd-resolved is not supported by underlying coredns anyway. I've found out that lima.yaml (and /etc/netplan/50-cloud-init.yaml) DNS ends up in the /run/systemd/resolve/resolv.conf so for K3s one could use --resolv-conf /run/systemd/resolve/resolv.conf to make it work with the corporate VPN.
It would be nice to fix Lima to refresh DNS from lima.yaml on restart so the user don't have to know about /etc/netplan/50-cloud-init.yaml but otherwise this ticket can be closed.

It would be nice to fix Lima to refresh DNS from lima.yaml on restart so the user don't have to know about /etc/netplan/50-cloud-init.yaml but otherwise this ticket can be closed.

I was just about to say that Lima does refresh the DNS on each restart, but when I just tested it, the new servers were not configured.
Unfortunately this seems to be an issue with cloud-init, as we do pass in the new nameservers:
jan@lima-default:/Users/jan$ sudo cat /mnt/lima-cidata/network-config
version: 2
ethernets:
  eth0:
    match:
      macaddress: '52:55:55:e9:a1:b4'
    dhcp4: true
    set-name: eth0
    nameservers:
      addresses:
      - <IP_ADDRESS>
      - <IP_ADDRESS>

This seems to be processed only during first-boot.

This seems to be processed only during first-boot.

Yes, this is intentional, as far as cloud-init is concerned, but I found a way to work around it, by changing the instance id on every reboot, so it always looks like a fresh install to cloud-init: #270

Upon further research - split DNS as well as systemd-resolved is not supported by underlying coredns anyway. I've found out that lima.yaml (and /etc/netplan/50-cloud-init.yaml) DNS ends up in the /run/systemd/resolve/resolv.conf so for K3s one could use --resolv-conf /run/systemd/resolve/resolv.conf to make it work with the corporate VPN.

This is the part I still don't understand:
If coredns doesn't understand the split setup where each name server only knows about a subset of domains, how does it help to configure all with systemd. coredns would still forward to a randomly picked server from /run/systemd/resolve/resolv.conf, and you can either get lucky, or you could not.
I think leaving <IP_ADDRESS> in the list of name servers is a mistake because in case you explicitly supply VPN name servers, the qemu nameserver will not be able to resolve the name, so you have a 33% chance of failure.
But removing this address will still not make the split-dns scenario work. I'm actually not sure if it is even possible to make it work. This is turning into a rabbit hole...
I'm re-opening this issue, because I think we need to explore this further.

I'm actually not sure if it is even possible to make it work.

On further thoughts, I think it should be possible: we could specify the search domains together with the nameserver addresses, and systemd-resolved should route queries appropriately. And k8s would use <IP_ADDRESS> from /etc/resolv.conf.
For systems without systemd-resolved we could run dnsmasq and configure it for conditional forwarding.
This is now getting quite a bit more complex, so it may take a while before I can look into it.

I just tried it and it works like a charm!

Thanks for confirming!
I'm happy with the result; I love it when stuff works correctly without requiring any configuration. It is good to be able to override settings, but it is even better if you don't need to.

So what was the solution to this? The only way I'm able to currently get network working in lima behind VPN is to copy /etc/resolv.conf from host to lima instance, but it will be rewritten with every restart..

@Kitanotori Can you provide some more details on what exactly is not working?

Which lima version are you using?
Do you just need the nameserver, or is it the search domains that are missing?
Does it fail to resolve any names when the VPN is connected, or only names provided by the VPN?
Where is it failing: from regular shell commands, or when nerdctl is pulling images, or from within k8s?

Lima now provides a DNS server that resolves names on the host, so generally this should work automatically without any configuration. If this is not the case, then we would like to understand how and why it fails, so we can improve the implementation.
Just today I learned that DNS does not work properly when the names are using the *.local domain (which should really only be used for mDNS).
If all you need to do is to specify the DNS server inside /etc/resolv.conf, then you could pass this in via your lima.yaml file:
dns:
- <IP_ADDRESS>
- <IP_ADDRESS>

There is no way to specify a search domain though.
But even if this fixes your problem, please provide the information I requested above to help us improve Lima!

@jandubois
Host /etc/resolv.conf looks like this:
search intra.aaa.bbb.com
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>

lima instance /etc/resolve.conf:
nameserver <IP_ADDRESS>
options <not sure if these are relevant?>
search .

I tried with exact same resolv.conf as host and one with only the nameservers of the host, and both seemed to work. I guess my issue is that the config from ~/.lima/default/lima.yaml is not reflected to lima instance's resolve.conf..?

Just today I learned that DNS does not work properly when the names are using the *.local domain (which should really only be used for mDNS).

We use such domains, but those don't seem to be issue if using host's nameservers.

Also, the nameservers are not reflected to resolve.conf even when specifying the nameservers during lima instance creation. So, how can I get the configuration persist restarts?

They are configured in systemd-resolved, which is what <IP_ADDRESS> points to.
$ grep ^useHost ~/.lima/default/lima.yaml
useHostResolver: false

$ grep -A 2 ^dns ~/.lima/default/lima.yaml
dns:
- <IP_ADDRESS>
- <IP_ADDRESS>

$ lima resolvectl status
Global
       Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 2 (eth0)
    Current Scopes: DNS
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: <IP_ADDRESS>
       DNS Servers: <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>

Looks like the host resolver (<IP_ADDRESS>) is still included in the nameserver list, but as long as the primary server is available it should make no difference.

@jandubois I'm not familiar with systemd-resolved. Does it mean that currently there's no way to persist correct settings at lima instance?

@Kitanotori As I've shown just above, the DNS servers configured via lima.yaml are used by systemd-resolved as name servers, and the systemd resolver is exposed on <IP_ADDRESS>. That address is stored in /etc/resolv.conf, so applications not linking against glibc (e.g. Go programs compiled without CGO) can still take advantage of the extended resolver features.


Jdo 709 spec

Updated specification changes for final review of JDO-709.

Some comments:

Chapter 6: Would it make sense to relabel "Converted Classes" to "Convertible Classes" or "Convertible Types"? In my mind the converted type (a primitive, not a class) is stored in the datastore. The application/domain type is in my mind the un-converted type.
Chapter 14, page 8: Change "... subsequent execute method" to "subsequent execution" or "subsequently called execute method"?
Chapter 14, page 15: The last two change just appear to add an empty bullet?
Chapter 19.1.4, page 4: The first line of the javadoc comment is missing a linebreak after /**: /** * The
Chapter 19.1.5, page 5: The first line of the javadoc comment is missing a linebreak after /**: /** * The

Correction: never mind about the empty bullet in Chapter 14, page 15, it does show up in OpenOffice.
I was using LibreOffice which just showed nothing.


[Question] tibia-api will focus only in Character or will it be more comprehensive?

(the title says all)

Hello @renatorib.
My intention is to create a complete API based each important information got from Tibia Website.
I don't know if you notice them, but I have already implemented Worlds route. 😄

That's great!

I don't know if you notice them, but I have already implemented Worlds route. 😄
:o where??