1.13.0-rc2 cherry-picks : part 2

#27631

d075b83dd6e5bbd4470400d44889ca71cf9dbd59 allow node update API to receive node name and id prefix

#28272

7c29f5beed19b55ac8f14880752d4825ba23eccf Windows: Perf: Use sequential file access

#28316

d7257640835abe5e7c08235b3fd974e319dbd7b6 Update docs for docker info for most recent changes

#28346

9dabde5b01c50cbac3da20943ba3c4498a7516c4 Fix missing versions in example requests of api docs

#28420

b1a91dcc0f0ba0bbad1d70943b423464b500657f update tini to a87614212b3a51a9cad57ff7989103a841546745

#28440

b9b8d8b364a14b6f827c9db94f651dda372ed253 Don't validate platform-dep tmpfs opts on client

#28467

d63dc45f0b34be6e25592ea6322a462497a89507 Remove redundant 'if' statement

#28468

70fdbff5f077e01897b8d284d1e29de64f8041eb Add zsh completion for 'docker system df'

#28475

947464e2f1a8df9addce4911f4c1264b2c606543 Update stats cli reference documentation

#27749

a524c135dce6ce39ccbca68fdd38f5b575ad82f4 Edits to plugin docs after building authz plugin

#28100

7ffb4ad81ab8e4400b0a588552ffb99dc55d13aa Disable v1 protocol for the default registry

#28457

a8139460ff98b081c06ecf28a821d6f456d9a079 explicitly show plugins as unsupported on !linux

#28501

5b0a52f7dcd04017a1def7326f8adbb8547ebe89 Skip cli initialization for daemon command

#28586

271b1f27563b18a125b89d2d2629246213dbfd96 fix infinite recursion

#28584

228ee16df3d738efaf001309370b26561c49edd6 Fix check config script on 4.8 kernels

#28580

1f039a66ac55c937f5dfa281a04009626d191f83 fix typos

#28560

2d08a764211035ec93aa3a97afb2baff074103da fix a typo

#28557

ce33d4e1c3d2ca3672a4d7e8e2b553de842fef30 Update Tini to fix photon build-rpm compilation errors

#28554

bf95472105e5dad55314cfff599abea5b81b134a refactor help func in CLI

#28552

158bb9bbd588aa03a3567f40b367025ccbd81fb3 3a0554585177a55802377423bb4529d305b4566f Fix use of cap in MultiReadSeeker; possible data corruption bug.

#28551

4130eb24e9e844bcfa9a336a7fef7698ad0e5c9f Reduce spew data dumped

#28533

972a246c671b52786a5e57743babfcf7b5b5ae99 judge manager before unlocking

#28530

3484e02590117d175d9c1ab24c583390b4e94a55 api/server/httputils: ensure consistent status code

#28523

b6857e91c1625695abbf5897e13f688028880c33 Fix crash caused by docker service inspect --pretty

#28521

d8b8b129948ef8586f5b2f576533952419e302f2 api: Remove omitempty tag on Parallelism

#28512

1144f8f1d4b1fd2bbf1f41bf5dad8d929d0dc06e Cleanup after plugin install.

#28505

7e03ce3f13baa61ce5547574163059b712b37025 Windows: Revert 27884

#28477

a0d7b34d9cb0876f8583a6e15d1b1aaab8d2b9ab Update images cli reference with reference filter

#28462

89b123473774248fc3a0356dd3ce5b116cc69b29 Fix deadlock on cancelling healthcheck

#28458

d22ac2f3a0de8e2ff6da8f1787e3ac3ac7438f17 Fix error messages for --cpus from daemon

#28447

d910781c66262af95646d7a0b564d254afc4af8f dc1b6341b98c3d009342a878119f683bfe14b471 Using distribution reference pkg (pin image by digest)

#28415

a3c4ab9b657018cf6fc33e70a44c790563c16043 Windows: Allow USER in builder

#28354

039aca05c2eae8a06dc16ff97110539feb44d13b Change the docker-tag usage text to be clearer

#28347

312cc7eebd326cc500e5742115d1e92e94ff60ee Use map[string]bool for preProcessor to ignore unknown fields

#28532

1d90d7604881e29b71819af9c092d565513aeeec api/server/httputils: service unavailable for disable swarm

#28565

aff99136b44c0a3d9ef381afd38a5cd05328ddc6 Fix cfs period calculation

#28367

19e2c9177a111758e304ad9dbd0835cddbd840ab Fix a comment error

#28290

19bd1cee236b85d2e24fdf49d6181edb7e1f7f17 fix "fatal error: concurrent map read and map write" bugs when we concurrently execute lots of delete volumes and create or list volumes tasks

#28421

948e60691e523022f88e7f8129f02106a0f8826c Return warnings from service create/update when digest pinning fails

#28538

ace786e9d517777473bd431e65d6c464d82e4f65 Fix several issues with go vet and gofmt -s

#28590

8b1aeb22febdb60e29b4d1ee52c6d4139af1cd62 Update containerd & runc

#28603

f1b9df91722a229fd0cefd7a0868933b6b811544 swap position of "host" and "ip"

#28610

9f348d5625996faf7a0855b8eb52cdd4def3958e fix gofmt in master

#28613

79a891efb7365e4f7d129a0142e66d2852781608 skip api/types/container/ (like golint) and fix one pkg

LGTM

Couple of remaining golint errors;
02:58:04 cmd/docker/docker.go:84:3: error strings should not end with punctuation

Fixed by https://github.com/docker/docker/pull/28611
02:58:04 daemon/cluster/cluster.go:73:1: comment on exported var ErrSwarmCertificatesExpired should be of the form "ErrSwarmCertificatesExpired ..."

Fixed by https://github.com/docker/docker/pull/28537
02:58:04 daemon/cluster/cluster.go:1058:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
02:58:04 daemon/info_unix.go:16:1: exported method Daemon.FillPlatformInfo should have comment or be unexpected
02:58:04 daemon/info_windows.go:8:1: exported method Daemon.FillPlatformInfo should have comment or be unexpected
02:58:04 plugin/backend_unsupported.go:14:5: exported var ErrNotSupported should have comment or be unexported
02:58:04 volume/local/local.go:272:37: error strings should not end with punctuation

Fixed by https://github.com/docker/docker/pull/28611

@thaJeztah PR updated. it's now :green_heart:
ping @docker/core-engine-maintainers

LGTM


add andFilterWhere condition

It would be a nice feature to write code like this:
// grid filtering conditions
$select->andFilterWhere('code', 'like', $entity->code);
$select->andFilterWhere('name', 'like', $entity->name);

instead of this:
// grid filtering conditions
if ($entity->code !== null) {
    $select->andWhere('code', 'like', $entity->code);
}
if ($entity->name !== null) {
    $select->andWhere('name', 'like', $entity->name);
}

I tried to implement it, but I do not really understand yet how it works.

You can do this with custom repositories
class UserRepository extends Repository
{

    public function withFilterWhere($column, $operator, $value)
    {
        $r = $this;
        if ($value !== null) {

            $r = clone $this;
            $r->select->where($column, $operator, $value);
        }

        return $r;
    }
}

Sample usage
        $code = 1;
        $name = 'cat';

        /** @var UserRepository $repo */
        $repo = $this->orm->getRepository(User::class);

        $users = $repo
            ->withFilterWhere('code', 'like', $code)
            ->withFilterWhere('name', 'like', $name)
            ->findAll();

https://github.com/cycle/docs/blob/master/basic/repository.md
https://github.com/cycle/docs/blob/master/advanced/chained-repository.md

Hi @albertborsos
I'm against adding this feature for 3 main reasons:

The method is confusing, should it skip condition when the value is null/false/empty string?
We are saving 2 lines of code but introducing a new concept of optional conditions, this will make code maintenance harder down the road.
As @genhoi it's much easier to add custom filter logic to repositories (which they are intended for).

Thanks for your suggestion, I'm open to change my opinion if more arguments will surface, this ticket will remain open for the discussion for the next few months.

Thanks! This solution is good enough for me.


Improve metrics to be more analytics friendly

@oscarotero and team, I have these suggestions for improving metrics:

Increase granularity
Allow "baggage" (in open telemetry terms) to be contributed by pages

Increasing granularity. Right now we have the following in metrics.json where the "name" has event details that need to parsed in order to be consumable for analytics:
{
    "name": "Load: /support/generate-static-build.sh",
    "entryType": "measure",
    "startTime": 1009.5581999999999,
    "duration": 0.1815000000000282,
    "detail": null
  }

Instead, I suggest something like this:
{
    "name": "Load: /home", // human-friendly, not useful for analytics
    "nature": { category: "load" }, // analytics-friendly, nature can be like a URL with depth /load/X, /preprocess, /render/Z, etc. and can be merged with metrics contributed by pages being rendered
    "target": { url: "/page/url", ...anything else we know about target... }, // analytics-friendly
    "entryType": "measure", // "measure" is the only entry I see here -- is there anything else?
    "startTime": 1007.8624, // analytics-friendly,
    "duration": 2187.5929, // suggest removing "duration" and only including analytics-friendly "endTime"
    "detail": null // suggest renaming to "baggage" using OpenTelemetry terminology; and allow this to be contributed/merged from generated pages (through filter.metrics({ nature: {  }, baggage: {... anything ...}})
}

Allow metrics "contribution" by pages. Many of my pages run database calls or other expensive calls and I'd like to add my own metrics for those expensive items.
export default async function ( // for Lume
  page: lumeAide.RenderablePage,
  filters: lumeAide.Filters,
) {
  // do something expensive
  filter.metrics({ nature: {  }, baggage: {... anything ...}}); // I can call my own metrics?
}

These metrics are generated with the standard Performance web interface, that allows to create marks and measure the time between these marks (or one mark to now) with performance.measure. I think it's better to use an already implemented and available insterface instead reivent the wheel creating a new system.
The performance object is already available in Deno as a global object, so you can use it in any point of your javascript/typescript code. site.metrics is just a wrapper of the performance object to be used internally by lume: See for example: https://github.com/lumeland/lume/blob/master/metrics.ts#L33
So, if you want to add additional metrics from your typescript pages, just need to do something like this:
export default async function (data, filters) {
    performance.mark("SQL query"); //Create a timing mark named "SQL query"
    const result = await makeExpensiveQuery();
    performance.measure("SQL query"); //Measure the timing from the mark "SQL query" to now.
}

Now, the measure of "SQL query" will be included in the list of your metrics. Note that lume only export the entries of type "measure" (nor "mark") because marks are used only as the started point of a measure. See this
Maybe we could include additional info in the details property but, due marks cannot have duplicated names, we have to contain all info in the entry name to avoid duplications.

Perfect @oscarotero - I completely agree we should stay with Deno's strategy of being as close to web interface as possible. With this new information you've armed me with about metrics I'll do some studying and implement what I need on top. I'm trying to get the metrics to be useful on the server side through Prometheus and OpenTelemetry taxonomy and I'll let you know if I get stuck anywhere. Thanks!

@oscarotero here's a quick suggestion that might get what we need. In metrics.ts if we change the code to this #getMarkName I think it would work fine:
  /**
   * Generate an unique mark name
   */
  #getMarkName(name: string, subject?: unknown, baggage?: Record<string, unknown>) {
    let markName: string | ({ name: string, subject?: unknown, baggage?: Record<string, unknown>}) = name;
    if(subject) {
        markName = { name, subject }
    }
    if(baggage) {
        markName = { name, subject, baggage }
    }
    return JSON.stringify(name);
  }

This way, if only a name is passed in we get:
{
    "name": "Load",
    "entryType": "measure",
    "startTime": 1009.5581999999999,
    "duration": 0.1815000000000282,
    "detail": null
  }

if other details are passed, we can get a parseable JSON:
{
    "name": "{ name: \"load\", subject: \"/support/generate-static-build.sh\", baggage: "...whatever..." }",
    "entryType": "measure",
    "startTime": 1009.5581999999999,
    "duration": 0.1815000000000282,
    "detail": null
  }

@oscarotero if you'd rather put the extra information in detail that's fine too -- my only goal is to have is parseable without doing any string manipulation. Thanks!

I don't think the name is a good place to set info. This is the job of detail property.

@oscarotero on second thought, I think we should do the following -- this will allow the Metrics to be completely turned into a user-supplied strategy and supplied through _config.ts. We can have a default implementation of the InstumentationSupplier which would be exactly what is available today but would allow those of us who need more sophistication to supply their own InstumentationSupplier implementation without bothering the Lume core.
/**
 * Interface to collect a single event's instrumentation
 */
export interface Instrumentable {
    readonly mark: () => Deno.PerformanceMark;
    readonly measure: () => Deno.PerformanceMeasure;
}

export type InstrumentOptions = Record<string, unknown>;

/**
 * Interface to collect all Lume events' instrumentation
 */
export interface InstumentationSupplier {
  readonly beforeBuild: (options?: InstrumentOptions) => Instrumentable;
  readonly copyAssets: (options?: InstrumentOptions) => Instrumentable;
  readonly copyAsset: (options: { from: string; to: string; }) => Instrumentable;
  readonly loadPages: (options?: InstrumentOptions) => Instrumentable;
  readonly loadPage: (options: { page: Page }) => Instrumentable;
  readonly renderPages: (options?: InstrumentOptions) => Instrumentable;
  readonly renderPage: (options: { page: Page }) => Instrumentable;
  readonly savePages: (options?: InstrumentOptions) => Instrumentable;
  readonly savePage: (options?: InstrumentOptions & { page: Page }) => Instrumentable;
  readonly afterBuild: (options?: InstrumentOptions) => Instrumentable;

  // allow unlimited others to be added by site owner?
  // readonly [x: string]: (options?: InstrumentOptions) => Instrumentable;
}

Once the above is available, then the calls would look like this:
   // this.metrics.copyAssets() returns Instrumentable with mark() already called
    const observe = this.metrics.copyAssets();
    for (const [from, to] of this.source.staticFiles) {
      await this.#copyStatic(from, to);
    }
    observe.measure();

Inside #copyStatic:
  async #copyStatic(from: string, to: string) {
   // this.metrics.copyAsset() returns Instrumentable with mark() already called
    const observe= this.metrics.copyAsset({ from, to }); // observation is strongly typed
    const pathFrom = this.src(from);
    const pathTo = this.dest(to);

    if (await exists(pathFrom)) {
      await ensureDir(dirname(pathTo));
      if (!this.options.quiet) {
        console.log(`🔥 ${normalizePath(to)} ${gray(from)}`);
      }
      return copy(pathFrom, pathTo, { overwrite: true });
    }
    observe.measure(); // the instrumentation supplier controls all aspects
  }

The current Metrics could become a default implementation like so:
/**
 * Class to collect and return performance metrics
 */
export default class Metrics implements InstumentationSupplier {
  site: Site;

  constructor(site: Site) {
    this.site = site;
  }

  beforeBuild(options?: InstrumentOptions): Instrumentable {
    const result: Instrumentable = {
        mark: () => performance.mark(this.#getMarkName("Build (entire site)"));
        measure: () => {
            const markName = this.#getMarkName("Build (entire site)");
            return performance.measure(markName, markName);
        },
        ...options // everything passed in is included as part of the result
    }
    result.mark();
    return result; // it's the responsibility of the caller to later call result.measure().
  }

  /// all other instrumentables ...

  copyAssets(options?: InstrumentOptions): Instrumentable {
    const result: Instrumentable = {
        mark: () => performance.mark(this.#getMarkName("Copy (all files)"));
        measure: () => {
            const markName = this.#getMarkName("Copy (all files)");
            return performance.measure(markName, markName);
        },
        ...options // everything passed in is included as part of the result
    }
    result.mark();
    return result; // it's the responsibility of the caller to later call result.measure().
  }

  copyAsset: (options: { from: string; to: string; }): Instrumentable {
    const result: Instrumentable = {
        mark: () => performance.mark(this.#getMarkName("Copy", options.from);
        measure: () => {
            const markName = this.#getMarkName("Copy", options.from);
            return performance.measure(markName, markName);
        },
        ...options // everything passed in is included as part of the result (caller can provide anything)
    }
    result.mark();
    return result; // it's the responsibility of the caller to later call result.measure().
  }

This is too verbose and not so scalable to my taste. It requires to create a method for every thing you want to measure.
But there are good ideas here, so I've modified a bit the metrics class:

instead of the start and end methods, there are only one method: start that returns the callback for the end:

const end = metrics.start("Metric name");
await expensive_operation();
end();

The second argument of start allows to insert any value that will be saved in detail:

const end = metrics.start("Load page", { page });
await loadPage(page);
end();

You can add extra details in the end() function too:

const end = metrics.start("Load page", { page });
const content = await loadPage(page);
end({ content });

The detail object has always the name key with the original name of the metric. This allows to have many metrics with the same name value in detail (but unique entry names):
for (const page as this.pages) {
  const end = metrics.start("Load page", { page });
  await loadPage(page);
  end();
}

// Now, you can get all "Load page" metrics:
const loadPageEntries = metrics.entries.filter((entry) => entry.detail.name === "Load page");

In addition to that, the json file exported is now prettier.

Sure, if this is the best I can get I'll take it :-).
But, it's not as scalable for complex needs because the Lume core is providing the messages and subjects instead of a type-safe instrumentation supplier doing all messages and categorization. With the design I suggested, tossing in OpenTelemetry as the supplier would be trivial. With your (even much improved) design I am still forced to create an adapter in the middle instead of using a proper type-safe interface. In general I agree with you about reducing verbosity but complex sites like mine that have hundreds of thousands of pages with over a dozen developers need different flexibility in different page generators.
I appreciate you taking the input and improving it though, thanks! I'll use what you've created when the next release comes out and give you more feedback.

@oscarotero in case you missed it please see this at the bottom of the InstrumentationSupplier interface:
  // allow unlimited others to be added by site owner?
  // readonly [x: string]: (options?: InstrumentOptions) => Instrumentable;

Also, you can make the code less verbose easily with the following -- maintains type-safety with easy flexibility:
export default class Metrics implements InstumentationSupplier {
  site: Site;

  constructor(site: Site) {
    this.site = site;
  }

  typicalInstrument(subject: string, options?: InstrumentOptions): Instrumentable {
    const result: Instrumentable = {
        mark: () => performance.mark(this.#getMarkName(subject));
        measure: () => {
            const markName = this.#getMarkName(subject);
            return performance.measure(markName, markName);
        },
        ...options // everything passed in is included as part of the result
    }
    result.mark();
    return result; // it's the responsibility of the caller to later call result.measure().
  }

  beforeBuild: (options?: InstrumentOptions) {
    return this.typicalInstrument("Build (entire site)");
  }

  copyAssets(options?: InstrumentOptions): Instrumentable {
    return this.typicalInstrument("Copy (all files)");
  }

  copyAsset: (options: { from: string; to: string; }): Instrumentable {
    return this.typicalInstrument("Copy (all files)", ...options);
  }

The main thing we want from the core is to have type-safe methods called through a proper interface for all core events. This way, we handle it with compile-time checks and not run-time surprises.
If you agree with proper type-safety in the core you would do this:
    const observe = this.metrics.copyAssets(); // type-safe and trappable by non-core implementation
    for (const [from, to] of this.source.staticFiles) {
      await this.#copyStatic(from, to);
    }
    observe.measure();

If you disagree with proper type-safety in the core you would do this:
    const observe = this.metrics.typicalInstrument("Copy (all files)");  // no limits, but not type-safe, still trappable by non-core implementation
    for (const [from, to] of this.source.staticFiles) {
      await this.#copyStatic(from, to);
    }
    observe.measure();

The nice thing with my suggestion is that it's a proper interface and I can trap any/all calls and do whatever I'd like. Whatever you decide in the end please make it an interface that can be replaced by users' own implementations.

I think the solution is very similar to your proposal. The only difference is that instead having one method for every thing to measure, we have just one method. I know that you're concerned about typings (so you can accept only some options for every measure, for example require a Page for every renderPage measurement), but in practice, is almost the same. In fact, there are other metrics that you have not taken into account, like processors and preprocessors. And if you want to measure database queries, for example, you should have to create new methods.
This is why I believe this way is more scalable. It works like events: there isn't an event class for every event type, there's just one event with the property type. This allows to have only two methods: addEventListener and dispatchEvent, instead of addBeforeBuildEvent, dispatchBeforeBuildEvent, addAfterBuildEvent, dispatchBeforeBuildEvent, and so on.

@oscarotero I know it looks similar to my proposal but it's missing the main purpose: flexibility for analytics through multiple implements. In your version, you're making the decision a priori about what to name a subject (which makes it type-unsafe and defeats typing) because you have concrete classes instead of an interface.
By putting in a minor abstraction (Instrumentable) you get unlimited flexibility with your primary goal of allowing everything complex to be done outside of the simplicity-focused core. Lume's initial TypeScript implementation is fantastic but there are many places there are concrete classes where interfaces should be (like Metrics). This may be the first place where you might take the TypeScript-centric approach to creating small interfaces and default simple implementations in Lume core with non-core implementations can do all the complex work on their own. Similar to your plugins concept.
This is mainly an architecture choice difference -- your style is good, solid, JavaScript concrete classes approach ... my suggestion is to use TypeScript-focused style with interfaces and multiple implementations. If it would be helpful to have a quick voice call to discuss this topic I'm happy to jump on a call. The reason I'm spending time on this discussion is that the architecture of Lume is fantastic but not type-safe yet and it would be great to increase type-safety as you progress to 1.0.
Thanks for hearing me out!

What you're proposing is that, instead of returning a function, returns an Instrument object with a measure function?
I mean:
// This is your proposal (the not type-safe option)
const observe = this.metrics.typicalInstrument("Copy (all files)");
for (const [from, to] of this.source.staticFiles) {
  await this.#copyStatic(from, to);
}
observe.measure();

// And this is my proposal
const end = this.metrics.start("Copy (all files)");
for (const [from, to] of this.source.staticFiles) {
  await this.#copyStatic(from, to);
}
end();

I see both are almost the same, but maybe I'm missing something. Maybe you're planning to expand the observe object with more functionalities?
My idea to make this extensible is that you could create your own class extending Metrics:
class DbMetrics extends Metrics {
    measureDatabase() {
        return this.start("Database");
    }
}

I guess what you're proposing is creating an interface that you can use without extending the base class, right?
class MyMetrics implements InstumentationSupplier {

Anyway, I'll release a new version in next days, but if you want to test it right now, just run lume upgrade --dev and this will install the last commit version.

Affirmative @oscarotero -- returning a function is not as extensible as returning an interface (like you said, I'm planning on expanding the observe object) :-).
Also, you're right that I would like to create an interface without extending the base class. Requiring inheritance (class X extends Y) is generally more brittle than using composition (implementing an interface with property functions). With interface implementations the core never worries about any base classes - only default implementations.
As Lume grows and more developers join the core (like I would love to!) the more base classes you have, the more brittle the system will be because base functionality can break in inherited classes. Adding more interfaces with default implementations in favor of base classes with inheritance is ideal - it would allow us to use Inversion of Control (IoC) in _config.ts for all complexity and leave the core super-simple.


tests(rust): allow using a custom ockam home on bats tests

Fixes https://github.com/build-trust/ockam/pull/4257#issuecomment-1444276091

@adrianbenavides I ran shfmt on all *.sh, *.bash and *.bats files in develop so this will need a rebase. Thank you.

ready now @mrinalwadhwa


typing chars to replace a multi-position selection in (empty) virtual space causes caret jump and split text

Create an empty document and enable virtual space in AvalonEdit.
Enter some text at the left margin

 abc

Use <ctrl> with the mouse to select 3 empty positions in the space to the right of "abc":

...nevermind I mistakenly hit the button on this form in GitHub prematurely ... sorry


Implement Google Typescript Standards (Update to ES2018)

Include Google Typescript Standards
Includes update to ES2018
Have not run linter or prettier yet

I did not approve this!


Fix HTTP caching behavior

In order to leverage HTTP caching, the very same HTTP client instance must be reused across calls. This PR uses a memoized client function to only ever create a client once and then return the same instance over and over (which seems the original intent of the code).
Previous behavior is preserved through the base client* function. Docstrings are provided to properly distinguish between the two flavors.

Thanks for this PR @skuro , looks good to me. Allow me some time to test it before merging, I'll keep you posted.


Feat : Add snackbar for internet connectivity

Fix #96

@Pritom14 The changes look good. Squash these commits into single one.

Can I submit a new PR?
I am having a lot of error while squashing them

Sure.


r-modules: R bioc updater revised

This PR supercedes #122408, because I couldn't get the editorconfig checkers to pass on that branch, and merge conflicts had occured since the PR was submitted.
BioConductor releases are tied to specific R releases.
This pull request allows the package update scripts to automatically
select the correct version of bioconductor to match the current
version of R.
Also update language documentation.
Motivation for this change
I updated the r-module package set (#122409, but also see #121819 which was submitted first and also bumps the package set) and encountered a few issues.
Bioconductor needed to have a version set in generate-r-packages.R, and the version must be compatible with the version of R, but that was not mentioned in the documentation. I have automated the version matching so the correct version of bioconductor will be chosen depending on the version of R used to update the packages. The maintainer should not have to update generate-r-packages.R before bumping anymore, as long as they use the R version in master.
I also had issues with some packages in bioc-experiment failing to get a hash. Re-running the scripts seemed to work most of the time.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

@peti, given your comments in https://github.com/NixOS/nixpkgs/pull/93883#pullrequestreview-570738189, is this PR blocked until we can find another maintainer for R in NixOS?

@jbedo,  @bcdarwin, I have been attempting to retest the changes in this PR, particularly the last code comment made: https://github.com/NixOS/nixpkgs/pull/127790#discussion_r699067271
Should the nix-shell generate-shell.nix be made pure?
My version of NixOS is out of date, I haven't had a chance to update recently.  When generate-r-packages.R attempts to call nix-hash it fails because of a mismatch between GLIBC versions.
nix-hash: /nix/store/sbbifs2ykc05inws26203h0xwcadnf0l-glibc-2.32-46/lib/libc.so.6: version `GLIBC_2.33' not found (required by /nix/store/di1fqpncmjbjgvghqr6y4sqsrdi2653k-gfortran-9.3.0-lib/lib/libstdc++.so.6)

If I add -pure and call nix-shell -pure generate-shell.nix then SSL calls fail due to missing certificates, not surprising since the shell only pulls in R.
The quick fix is for me to update to the latest nixos-unstable, but I am able to put in a bit more time to get the shell working anywhere. Would it make a difference for others?

nix-shell generate-shell.nix also rebuilt firefox, which doesn't make sense to me. What tools do you use to look at the dependency trees?

You can:
nix-instantiate generate-shell.nix | xargs nix-store -q --tree

There's also --graph that produces Graphviz dot files.
Firefox get's pulled in via openjdk, required by the base R package.  I
agree this is excessive and we should take a look at it, but maybe not
in this PR.

Would this be solved by adding nix to the nativeBuildInputs of
generate-shell.nix?

I tested some of the changes that @jbedo flagged, and found that they were not helping. I think I added them at a time when I was trying to update the package set over a flakey internet connection. When a package couldn't be downloaded, R didn't crash, it just returned an SHA of NULL.
I have removed the changes, because they were not an attempt to make the package update script robust to poor internet, they were just hacks to get it to "pass" while still failing.
For the same reasons, I have reverted all the language documentation changes.
I believe this is ready to merge. As a bonus from testing, I have a bumped package set ready to put into a PR.

I won't put together a variant of the package bump script that is robust to flakey internet, I don't think enough people would benefit from it.

The bumped package set had a few conflicts with master, no PR at this stage.

Whitespace fixed

The bumped package set had a few conflicts with master, no PR at this stage.

I've been working on an experimental update bot that runs daily, so you can pull the *-packages.nix files out of my tree https://github.com/jbedo/nixpkgs/tree/R if you want to avoid all the downloading and hashing. I merged your BIOC improvements into that branch to make the auto-update more reliable and I look forward to having your changes in nixpkgs.

I'm excited to see an automated package update for R! How often will in land in nixpkgs? If it won't happen often, I'll probably just point my development environment at your repo.

The downloading and hashing just finished anyway. #137751

I'm excited to see an automated package update for R! How often will in land in nixpkgs? If it won't happen often, I'll probably just point my development environment at your repo.

It's still early days, I have some issues to sort out (CRAN deletes packages and this causes problems) and before any automated PRs happen will need to sort out hydra so all the packages are built. I personally would want at least monthly updates since CRAN moves pretty quickly. If we select some "core packages" then we could generate a PR whenever they change (and successfully build).
Feel free to use my branch, I intend to keep it updating daily.

It's still early days, I have some issues to sort out (CRAN deletes packages and this causes problems) and before any automated PRs happen will need to sort out hydra so all the packages are built. I personally would want at least monthly updates since CRAN moves pretty quickly. If we select some "core packages" then we could generate a PR whenever they change (and successfully build).
Feel free to use my branch, I intend to keep it updating daily.

Yes, CRAN deletes packages, and that messes with default.nix, when a package was manually fixed but is now gone. The only way to automate the fixes would be to inspect the system requirements for a package, but they aren't enforced by CRAN.
Would "conserving" dropped packages be OK? Basically, keep the last version that we hashed even if it is removed from CRAN? I think that would be simple to implement in generate-R-packages.

Would "conserving" dropped packages be OK? Basically, keep the last version that we hashed even if it is removed from CRAN? I think that would be simple to implement in generate-R-packages.

I'm not in favour of keeping packages around if they're not available upstream anymore, and hydra shouldn't test them either. We could keep the entry and mark it somehow as deprecated (or use meta.broken?) so it doesn't build.

I'm not in favour of keeping packages around if they're not available upstream anymore, and hydra shouldn't test them either. We could keep the entry and mark it somehow as deprecated (or use meta.broken?) so it doesn't build.

Fair enough, keeping orphaned packages isn't a great plan. I think tracking removed packages in default.nix would be a lot trickier though, with a lot of parsing of default.nix to decide which packages need to be marked as broken and which packages can just be forgotten about because they never touched default.nix. We might as well just automatically find lines in default.nix and remove them using a very careful regex.

And we'd have to watch out for false positives, where a package was removed from CRAN, then the author fixes the package and it goes back up. With a daily update and automatic removal of lines from default.nix, we now have to go in and re-add all the overrides.
It's not obvious what the right approach is. But its probably time to move the discussion to a new place if you are actively working on it.


Why is int32ToString needed for numeric props? (SolidJs)

Why is int32ToString needed for numeric props? (SolidJs)
Doesn't Solid work well directly with numeric values in props  ?
e.g. Attr.x1
test: https://playground.solidjs.com/anonymous/7523d6dc-9e88-43ab-a493-5421c9482aa4

This is perhaps just an over site.
I am not so familiar with SolidJS or the JSX module of Fable. I think the best thing to do, is to test if using an int works and if yes then send a PR and make a new release.
Like for Fable.SolidJS, if you want I can give you maintainer access to this repo and the packages.

Thanks @MangelMaxime I will test changes it in my own fork, and if it works well create a pull request. You gave me maintainer access to the NuGet already 🤓 (but not this repo). But I can't cover the React JSX part 🤷♂.

But I can't cover the React JSX part 🤷♂.

I don't think it is really used by the community, it is still marked as a beta and no one maintained it since 2022.
So I think it is fine to mark as obsolete or keep it as it is for now. Most of the effort are made using Feliz and if a new package needs to be generated it will probably go under Glutinum project as I am working on experimenting on generating bindings automatically and will perhaps needs to have a custom low level API to make it possible.

Actually, unquoted number literals work because simply attributes don't need quotation marks. It's in the html spec.
(SolidJS will create it's internal html template for html rendering also without the quotation marks.)
https://html.spec.whatwg.org/multipage/syntax.html#attributes-2 :
Unquoted attribute value syntax
The attribute name, followed by zero or more ASCII whitespace, followed by a single U+003D EQUALS SIGN character, followed by zero or more ASCII whitespace, followed by the attribute value, which, in addition to the requirements given above for attribute values, must not contain any literal ASCII whitespace, any U+0022 QUOTATION MARK characters ("), U+0027 APOSTROPHE characters ('), U+003D EQUALS SIGN characters (=), U+003C LESS-THAN SIGN characters (<), U+003E GREATER-THAN SIGN characters (>), or U+0060 GRAVE ACCENT characters (`), and must not be the empty string.


Delete content/support/more-dashboard-apps/Cloudflare Stream/Deliveri…

…ng Videos with Cloudflare.md

@goodboy


Sometime the "guess" position is not the one clicked on the map

Randomly, the position used to compute who best guessed is not the one clicked on the map, hence unvalidating the scoring.
This "wrong" position is the one displayed at the end of the round.

Hello sorry I still don't get what you mean.

This "wrong" position is the one displayed at the end of the round.

Isn't this position the same as the one the player clicked? Or does the marker move randomly at the end of the round?

The marker moves randomly at the end of the round :/

Got it. Thanks for your report. I will investigate it. I would appreciate if you can find out how this thing happens.

I have just run into this problem with my friends as well. We played with 4 players and all of our markers got randomly moved at the end of the round. Any news on this?


Still under development?

The alpha version works fine for us, but I was wondering if this library is still under development since the alpha hasn't been updated in over a year.
Thanks for all the great work on this library!

I think they have bigger concerns right now... https://news.ycombinator.com/item?id=32460342

shameless plug here: i’ve wrote a really similar (and smaller) library if y’all are still interested..
it’s called reduxtron and i’ve just wrote a blog post about the journey and my motivations behind it

I, for one, appreciate this shameless plug!
Is it the same as electron-redux in that each process has its own store locally? Or is it handled in some other way? Are there any other notable differences? And can state be segmented by process in any way? Eg if I have a big store but I only need to populate some portion of it in child windows, is there any way to avoid loading the big store in each process?

hey there @Slapbox 👋
let me try answering some of your questions..

the tl;dr is that it might not suit all your needs as of now 😢

Is it the same as electron-redux in that each process has its own store locally? Or is it handled in some other way?

reduxtron only enforces a single store, on the main process, other processes (renderer process, windows) have the 3 main functions (subscribe, getState, dispatch) on their global scope
that being said, you can definitely write your own logic for each process to "drop"/"select" just a bit of the store (eg.: by consuming the global subscribe but only appending the slice of the state that particular window is concerned). but, for now, the ipc messages underneath will likely contain the whole global store state.
i implemented this way to make it easier to ramp-up new applications that eventually share it’s whole state, but i see your use case as really relevant

Are there any other notable differences?

the first one that comes to mind is that reduxtron doesn’t force or has opinions on what you have it on the browser windows, while on electron-redux you have a redux store on the main process + one for each window.
as some examples of this i’ve implemented:

couple boilerplates that the state change updates a frontend framework directly
a bridge between the redux store on the main process and zustand on the view layer, making type-safe hook selectors available on the frontend without a hassle (preventing unnecessary rerenders)

If I have a big store but I only need to populate some portion of it in child windows, is there any way to avoid loading the big store in each process?

as stated above, you can avoid having a store per se, or cherry-picking only what you need on the consumer (browser window) side. i haven’t solved this “wholesale” where you would be able to limit the messages the main thread sends to each browser
for enabling this control i would need to figure a clean and flexible enough API — and i definitely haven’t thought about a specific one.
one idea would optionally leave all the message passing burden on a redux middleware when you want to have finer control on the main side, but i would see as an ”advanced usage” and not the standard configuration

The newest version is just plain broken for me.

newest version of electron-redux or reduxtron?
btw, @Nantris idk if you still need the subset of the state for each window.
but i do have some ideas to achieve your needs, i have not yet needed it (still fast enough to have all redux data on all windows for me) but it might be as easy as just writing a “catch all” reducer and writing the logic to extract each piece of state for each window you need.
sometimes i’m working on my free time on zit, that has reduxtron as a central piece of it


Unbuckling causes a phantom body to appear

Description
Weird phantom body appears when walking diagonally after getting up from bed
Reproduction
Get buckled to a bed
Strip
Get up
Put clothes on
You will have a phantom body attached to you when you move diagonally
Screenshots

Dupe https://github.com/space-wizards/space-station-14/issues/10746
Dupe https://github.com/space-wizards/space-station-14/issues/10745


dji_onboardsdk_ros: 0.1.1-0 in 'indigo/distribution.yaml' [bloom]

Increasing version of package(s) in repository dji_onboardsdk_ros to 0.1.1-0:

upstream repository: https://github.com/dji-sdk/Onboard-SDK-ROS.git
release repository: https://github.com/dji-sdk/Onboard-SDK-ROS-Release.git
distro file: indigo/distribution.yaml
bloom version: 0.5.21
previous version for package: 0.1.0-0

why does the bloom-release change REP 143 into 141?
the version is 0.5.21

@tfoote , I'm not quite clear about the travis-ci check, does it mean the file distribution.yaml is broken? does it caused by bloom, which changes the line of REP143 ?

Please either change the patch to not change the comment line manually or create a new patch using bloom-release ... -p after updating to the latest release of python-rosdistro.

Thanks '@dirk-thomas , I apt-geted it last week so I believe it should be the latest version. Then I'm very puzzled.
Since all updated has done except the pr on distribution, I will re-pr this change by manual.


Topology Exception in valid polygon

I have a list of geometries in GeoPandas, which consist of Polygon and LineString. I am trying to convert LineString to Polygon using buffer because I need to use it in voronoi_diagram.
>>> c =LineString([(8601635.012337072, 2653275.483739082), (8601667.851586856, 2653238.721249395), (8601712.602022154, 2653208.9497218933), (8601871.454935515, 2653149.6478985697), (8602007.376033774, 2653084.3197241016), (8602027.079583647, 2653067.9274569103), (8602037.54361578, 2653052.4994562212), (8602045.224660644, 2653037.0714705535), (8602060.475430883, 2653025.2594291777), (8602152.75928875, 2652992.5955646853), (8602240.813005969, 2652973.55171904), (8602268.308920195, 2652980.0603725724), (8602412.24502179, 2652995.4883027347), (8602467.125530751, 2652987.171682267), (8602620.301150084, 2653000.5505955922), (8602653.140399868, 2653031.165448764), (8602800.304766696, 2653050.691488374), (8602943.684270838, 2653053.1021122164), (8602988.76866461, 2653051.0530819274), (8603045.875563387, 2653043.5801501744), (8603088.176969886, 2653036.830408397), (8603188.25319211, 2653013.6884584087), (8603241.68654769, 2652994.0419336436), (8603286.436982991, 2652970.2973932764), (8603308.144283695, 2652952.5794091793), (8603342.653325839, 2652941.7316735894), (8603396.643278876, 2652906.777909426), (8603411.894049115, 2652898.4613355114), (8603443.508784499, 2652895.448085172), (8603464.436848769, 2652896.65338524), (8603480.24421646, 2652907.621620064), (8603494.938389245, 2652913.8891862305), (8603652.566788208, 2652950.65092231), (8603739.7299495, 2652963.1860911557), (8603771.122045903, 2652964.6324574314), (8603810.751784626, 2652955.231079007), (8603824.555401485, 2652951.012513579), (8603843.924992882, 2652940.285309404), (8603861.290833447, 2652923.652130774), (8603972.165046277, 2652796.1316746194), (8604065.22814058, 2652689.1020649103), (8604110.31253435, 2652658.6084007607), (8604151.166787473, 2652635.346475068), (8604177.994784754, 2652620.8831187948), (8604204.377504071, 2652608.709804162), (8604235.32432251, 2652598.2238871735), (8604262.597597757, 2652589.063781458), (8604290.872748418, 2652583.7605647785), (8604322.264844822, 2652578.698405059), (8604349.983398028, 2652574.23888474), (8604369.130350444, 2652568.935673023), (8604404.641268007, 2652559.052419558), (8604446.497396547, 2652550.4949733634), (8604473.993310772, 2652544.227550915), (8604499.596793655, 2652533.862203839), (8604531.322848532, 2652531.4516589795), (8604565.943210168, 2652534.4648401113), (8604605.35030991, 2652538.5627673697), (8604628.616083486, 2652547.4817891833), (8604651.881857062, 2652549.2896996215), (8604686.279579716, 2652550.133391231), (8604716.6698007, 2652546.9996797675)])

>>> d=c.buffer(10,1,join_style=3)
>>> d.is_valid
True
>>> from shapely.ops import voronoi_diagram
>>> voronoi_diagram(d)
TopologyException: Input geom 1 is invalid: Self-intersection at or near point 8601635.01233707 2653275.483739086 at 8601635.01233707 2653275.483739086
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.6/site-packages/shapely/ops.py", line 222, in voronoi_diagram
    result = geom_factory(func(geom._geom, envelope, tolerance, int(edges)))
  File "/home/ubuntu/.local/lib/python3.6/site-packages/shapely/geometry/base.py", line 69, in geom_factory
    raise ValueError("No Shapely geometry can be created from null value")
ValueError: No Shapely geometry can be created from null value

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.6/site-packages/shapely/ops.py", line 227, in voronoi_diagram
    raise ValueError(errstr)
ValueError: Could not create Voronoi Diagram with the specified inputs.

buffer(0) doesn't work as well. Help is appreciated. The red dot is the point mentioned in the error

@aydal I've confirmed the TopologyError. And I've found that if we make a MultiPolygon of the polygonized edges of the Voronoi Diagram that MultiPolygon is invalid.

However: polys = list(polygonize(edges)) gives a list of 140 valid polygons. This may be useful to you? I think it may be practically impossible to make a valid MultiPolygon from the results of voronoi_diagram due to imperfect floating point precision.
cc @tomplex

I am trying to convert LineString to Polygon using buffer because I need to use it in voronoi_diagram.

Maybe I'm misunderstanding you, but you don't need to pass a polygon into voronoi_diagram. Regardless of what geometry you pass in, the underlying GEOS algorithm is going to explode it into a point set and create a diagram from that. You can pass your LineString in directly:
from shapely.ops import voronoi_diagram
from shapely.geometry import LineString

ls = LineString([(8601635.012337072, 2653275.483739082), (8601667.851586856, 2653238.721249395), (8601712.602022154, 2653208.9497218933), (8601871.454935515, 2653149.6478985697), (8602007.376033774, 2653084.3197241016), (8602027.079583647, 2653067.9274569103), (860203
7.54361578, 2653052.4994562212), (8602045.224660644, 2653037.0714705535), (8602060.475430883, 2653025.2594291777), (8602152.75928875, 2652992.5955646853), (8602240.813005969, 2652973.55171904), (8602268.308920195, 2652980.0603725724), (8602412.24502179, 2652995.4883027347
), (8602467.125530751, 2652987.171682267), (8602620.301150084, 2653000.5505955922), (8602653.140399868, 2653031.165448764), (8602800.304766696, 2653050.691488374), (8602943.684270838, 2653053.1021122164), (8602988.76866461, 2653051.0530819274), (8603045.875563387, 2653043
.5801501744), (8603088.176969886, 2653036.830408397), (8603188.25319211, 2653013.6884584087), (8603241.68654769, 2652994.0419336436), (8603286.436982991, 2652970.2973932764), (8603308.144283695, 2652952.5794091793), (8603342.653325839, 2652941.7316735894), (8603396.643278
876, 2652906.777909426), (8603411.894049115, 2652898.4613355114), (8603443.508784499, 2652895.448085172), (8603464.436848769, 2652896.65338524), (8603480.24421646, 2652907.621620064), (8603494.938389245, 2652913.8891862305), (8603652.566788208, 2652950.65092231), (8603739
.7299495, 2652963.1860911557), (8603771.122045903, 2652964.6324574314), (8603810.751784626, 2652955.231079007), (8603824.555401485, 2652951.012513579), (8603843.924992882, 2652940.285309404), (8603861.290833447, 2652923.652130774), (8603972.165046277, 2652796.1316746194),
 (8604065.22814058, 2652689.1020649103), (8604110.31253435, 2652658.6084007607), (8604151.166787473, 2652635.346475068), (8604177.994784754, 2652620.8831187948), (8604204.377504071, 2652608.709804162), (8604235.32432251, 2652598.2238871735), (8604262.597597757, 2652589.06
3781458), (8604290.872748418, 2652583.7605647785), (8604322.264844822, 2652578.698405059), (8604349.983398028, 2652574.23888474), (8604369.130350444, 2652568.935673023), (8604404.641268007, 2652559.052419558), (8604446.497396547, 2652550.4949733634), (8604473.993310772, 2
652544.227550915), (8604499.596793655, 2652533.862203839), (8604531.322848532, 2652531.4516589795), (8604565.943210168, 2652534.4648401113), (8604605.35030991, 2652538.5627673697), (8604628.616083486, 2652547.4817891833), (8604651.881857062, 2652549.2896996215), (8604686.
279579716, 2652550.133391231), (8604716.6698007, 2652546.9996797675)])
diagram = voronoi_diagram(ls)

You can also do as @sgillies did and only return edges, polygonize them and create a geometry from that:
from shapely.geometry import GeometryCollection
from shapely import ops

diagram = voronoi_diagram(ls, edges=True)
collection = GeometryCollection(list(ops.polygonize(diagram)))

You might also wish to play around with the tolerance argument for voronoi_diagram. It will do some snapping / rounding of coordinates to try and make the algorithm more robust. the PostGIS docs have a decent description.

@sgillies @tomplex Thanks a lot for your help.
As for the need for a buffer to LineString they represent roads and for my application, I need to give them some width.
Going by @sgillies approach, it may hold for individual geometries but malfunctions on a list. (Gray is the edges=True approach)

I will look into the tolerance and see if that helps.

I think the issues can be closed?
With the latest released versions, I also no longer get a TopologyException for the original example in the top post.


Where can I obtain obs?

Where can I obtain the data for obs? Is the obs, the raw data from the sensor?

can you give more details

Closed due to inactivity. If the problem persists, pls feel free to reopen it or create a new one and refer to it.


Not able to create schedule.

tui-date-picker.min.js:8 Uncaught TypeError: Cannot read property 'on' of undefined
at Object.on (tui-date-picker.min.js:8)
at init._setEvents (tui-date-picker.min.js:7)
at new init (tui-date-picker.min.js:7)
at init._initHeader (tui-date-picker.min.js:7)
at new init (tui-date-picker.min.js:7)
at new init (tui-date-picker.min.js:7)
at init._initializePickers (tui-date-picker.min.js:8)
at new init (tui-date-picker.min.js:8)
at Function.a.createRangePicker (tui-date-picker.min.js:7)
at ScheduleCreationPopup../src/js/view/popup/scheduleCreationPopup.js.ScheduleCreationPopup._createDatepicker (scheduleCreationPopup.js:563)

https://nhn.github.io/tui.calendar/latest/tutorial-example00-basic

Dependency packages have been changed. In progress.

Will you please let me know when it'll start working or please let me know a similar working example with different version?

@more-yogesh If you want it now, use "1.11.0" version until we solve this.

@more-yogesh If you want it now, use "1.11.0" version until we solve this.

Not solved same issue continues with 1.11.0

@more-yogesh Oh, do you use directly this repo with npm? OK, then, it'll take some time. Please take a cup of coffee. @jungeun-cho will solve it shortly.

@more-yogesh
https://nhn.github.io/tui.calendar/latest/tutorial-example00-basic
Try again. It should work find now. 😊

Its Done Thanks :)

@jungeun-cho Can you confirm which file you changed. I have download earlier repo and customize it for my purpose. So if you can let me know the changes you have made in your repository them I will add the same to my repo.
Thank You

@coolvijaymca
Added <script src =" https://uicdn.toast.com/tui.dom/v3.0.0/tui-dom.js "></ script> to the example using the CDN.
We will deploy a lighter version of the TOAST UI Calendar 2.0, removing the dependency between tui-code-snippet and tui-dom.


add localized exercise name to LocalizedText

Coverage decreased (-31.06%) to 42.345% when pulling 6c9956a458515f87764ce4296e99d7db814f5020 on localized-exercise-names into cd133e840f88ed4c633e305e7a56043de3423050 on master.

Coverage decreased (-0.006%) to 73.397% when pulling 666011216f927512291cbe23948244ab3b3097b4 on localized-exercise-names into cd133e840f88ed4c633e305e7a56043de3423050 on master.


Packages for other Linux distros

With https://github.com/mull-project/mull/pull/926 in place, we can now follow the same approach and ship more packages for different Linux distros.
Which distros we should support? Debian, Fedora, ...?

I vote for Debian (on this 2 years old issue).


EMR Serverless plugin in conflict with Airflow 2.2.2 constraints file

Hi all,
I'm trying to use the latest release of the serverless plugin on MWAA with Airflow version 2.2.2: https://github.com/aws-samples/emr-serverless-samples/releases/tag/v1.0.1
The install is in conflict with the airflow v2.2.2 constraints file found here: https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.7.txt
Steps to reproduce
Requirements.txt contents:
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.7.txt"
emr_serverless @ https://github.com/aws-samples/emr-serverless-samples/releases/download/v1.0.1/mwaa_plugin.zip

Run:
pip3 install -r requirements.txt

Output:
The conflict is caused by:
    emr-serverless 1.0.1 depends on boto3>=1.23.9 and ~=1.23
    The user requested (constraint) boto3==1.18.65

Facing the same issue

I'm not quite sure why this started happening - will have to do some tests later this week.
Do you need the --constraint in there?

Yea it does work ok without the constraints flag for my proof of concept - I have some more dependencies I'm going to want to add in the future and would like to be able to include it.
Awesome plugin btw

Unfortunately EMR Serverless requires a newer version of boto3 than what's in that constraints file. I don't know if there's a way to override that...

I noticed it doesn't conflict with Airflow 2.4.2 which is out - MWAA is just a little behind on that

Yup, MWAA is still on 2.2.2. I'm curious, can you help me understand why you're including the constraints line? I know you kind of mentioned it, but I'm still not sure what it's used for / why it's needed?

I was following this best practices guide in the MWAA docs: https://docs.aws.amazon.com/mwaa/latest/userguide/best-practices-dependencies.html
There is an Option 2 there using wheel fwiw.. maybe I'll look into that if 2.2.2 is SOL

Ahhh got it thank you. Yea, the boto3 will be an issue just because of when EMR Serverless support was added to it.

@dacort thank you for response. Curious, what features of boto3>=1.23.9 and ~=1.23 are in use by emr serverless operators and sensors that are not present in boto3==1.18.65 ? We for example are using MWAA 2.2.2 on our project and EMR Serverless 6.7.0, and can not use the library because of this boto issue.

@marknorkin EMR Serverless was made generally available this year, and boto3 1.23.9 is when support for EMR Serverless was added. You can still use the Operator on MWAA 2.2.2, you just need to upgrade boto3 (which will happen automatically if you use the Operator from this repo).
I wasn't aware of the recommendation in our docs to add the constraints line to the requirements.txt - that said, I've tried this operator with the upgraded boto3 with MWAA and haven't seen any issues.

Going to close this for now as EMR Serverless requires a newer version of boto3. If you're willing to forego the constraints, you can still use the operator on MWAA, but I don't think there's a workaround. The Operator is in use in MWAA environments.
For reference, this is the dependency tree of the EMR Serverless operator. You could potentially update the constraints file with the relevant versions...or I do see now that there is a constraints-no-provider file as well. Maybe that'll help if the concern is preventing against upgrade of core libraries for Airflow?
https://raw.githubusercontent.com/apache/airflow/constraints-2.4.2/constraints-no-providers-3.7.txt
emr-serverless==1.0.1
  - boto3 [required: ~=1.23,>=1.23.9, installed: 1.26.10]
    - botocore [required: >=1.29.10,<1.30.0, installed: 1.29.10]
      - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
      - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
        - six [required: >=1.5, installed: 1.16.0]
      - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.12]
    - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
    - s3transfer [required: >=0.6.0,<0.7.0, installed: 0.6.0]
      - botocore [required: >=1.12.36,<2.0a.0, installed: 1.29.10]
        - jmespath [required: >=0.7.1,<2.0.0, installed: 1.0.1]
        - python-dateutil [required: >=2.1,<3.0.0, installed: 2.8.2]
          - six [required: >=1.5, installed: 1.16.0]
        - urllib3 [required: >=1.25.4,<1.27, installed: 1.26.12]

Hello, even without constraints files, we are having this issue on a new MWAA 2.2.2 cluster. Our only peculiarity is that we are hosting your released .zip file in our nexus repository (the file is unmodified):
adding trusted host: 'nexus.REDACTED' (from line 1 of /usr/local/airflow/requirements/requirements.txt)
adding trusted host: 'nexusmaster.REDACTED' (from line 2 of /usr/local/airflow/requirements/requirements.txt)
Looking in indexes: https://nexus.REDACTED/repository/pypi-public/simple/
Collecting emr_serverless@ https://nexusmaster.REDACTED/repository/REDACTED/REDACTED/mwaa_plugin.zip
  Downloading https://nexusmaster.REDACTED/repository/REDACTED/REDACTED/mwaa_plugin.zip (6.7 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting boto3>=1.23.9,~=1.23
  Downloading https://nexus.REDACTED/repository/pypi-public/packages/boto3/1.26.15/boto3-1.26.15-py3-none-any.whl (132 kB)
Collecting s3transfer<0.7.0,>=0.6.0
  Downloading https://nexus.REDACTED/repository/pypi-public/packages/s3transfer/0.6.0/s3transfer-0.6.0-py3-none-any.whl (79 kB)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in ./.local/lib/python3.7/site-packages (from boto3>=1.23.9,~=1.23->emr_serverless@ https://nexusmaster.REDACTED/repository/REDACTED/REDACTED/mwaa_plugin.zip->-r /usr/local/airflow/requirements/requirements.txt (line 5)) (0.10.0)
Collecting botocore<1.30.0,>=1.29.15
  Downloading https://nexus.REDACTED/repository/pypi-public/packages/botocore/1.29.15/botocore-1.29.15-py3-none-any.whl (9.9 MB)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in ./.local/lib/python3.7/site-packages (from botocore<1.30.0,>=1.29.15->boto3>=1.23.9,~=1.23->emr_serverless@ https://nexusmaster.REDACTED/repository/REDACTED/REDACTED/mwaa_plugin.zip->-r /usr/local/airflow/requirements/requirements.txt (line 5)) (2.8.2)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in ./.local/lib/python3.7/site-packages (from botocore<1.30.0,>=1.29.15->boto3>=1.23.9,~=1.23->emr_serverless@ https://nexusmaster.REDACTED/repository/REDACTED/REDACTED/mwaa_plugin.zip->-r /usr/local/airflow/requirements/requirements.txt (line 5)) (1.26.7)
Requirement already satisfied: six>=1.5 in ./.local/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.30.0,>=1.29.15->boto3>=1.23.9,~=1.23->emr_serverless@ https://nexusmaster.REDACTED/repository/REDACTED/REDACTED/mwaa_plugin.zip->-r /usr/local/airflow/requirements/requirements.txt (line 5)) (1.16.0)
Building wheels for collected packages: emr-serverless
  Building wheel for emr-serverless (setup.py): started
  Building wheel for emr-serverless (setup.py): finished with status 'done'
  Created wheel for emr-serverless: filename=emr_serverless-1.0.1-py3-none-any.whl size=7414 sha256=da8ce9ab8a2ff91d9a3b883ddaafbc3c9e892133a4ffb499e420236b70068f0f
  Stored in directory: /tmp/pip-ephem-wheel-cache-lpa7pkzp/wheels/13/92/50/475b17c65c8d67d0c9ecba04a3df4e16188d880c57c8d90d8f
Successfully built emr-serverless
Installing collected packages: botocore, s3transfer, boto3, emr-serverless
  Attempting uninstall: botocore
    Found existing installation: botocore 1.21.65
    Uninstalling botocore-1.21.65:
      Successfully uninstalled botocore-1.21.65
  Attempting uninstall: s3transfer
    Found existing installation: s3transfer 0.5.0
    Uninstalling s3transfer-0.5.0:
      Successfully uninstalled s3transfer-0.5.0
  Attempting uninstall: boto3
    Found existing installation: boto3 1.18.65
    Uninstalling boto3-1.18.65:
      Successfully uninstalled boto3-1.18.65
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
apache-airflow-providers-amazon 2.4.0 requires boto3<1.19.0,>=1.15.0, but you have boto3 1.26.15 which is incompatible.
apache-airflow-providers-amazon 2.4.0 requires watchtower~=1.0.6, but you have watchtower 2.0.1 which is incompatible.
Successfully installed boto3-1.26.15 botocore-1.29.15 emr-serverless-1.0.1 s3transfer-0.6.0

Our requirements file is as follows:
--trusted-host nexus.alm.europe.cloudcenter.corp
--trusted-host nexusmaster.alm.europe.cloudcenter.corp
--index https://nexus.alm.europe.cloudcenter.corp/repository/pypi-public/
--index-url https://nexus.alm.europe.cloudcenter.corp/repository/pypi-public/simple/
emr_serverless @ https://nexusmaster.alm.europe.cloudcenter.corp/repository/scq-3rd-party-raw/scq-3rd-party-raw/mwaa_plugin.zip

As a bit of an aside, we have tried getting around this by setting this:
apache-airflow==2.2.2
apache-airflow-providers-amazon>=v5.1.0

This solves the version issue and install works correctly everywhere except on WebServer (as in https://repost.aws/questions/QUmgPhWhgmTFGMc18d7De40A/airflow-webserver-not-installing-python-requirements). However, if we set this and then try to use the operator in a DAG, the DAG gets processed correctly, but we never get a Task to actually run. We have also tried this with different versions of apache-airflow-providers-amazon (3.1.1, 5.1.0, 6.0.0). In the latter case we removed mwaa_plugin.zip as the library itself should already be providing the operator. We are unsure of the reason why this is not working (it may be our fault), hence why we are not opening a new issue yet.
In any case, we just wanted to let you know that just setting the emr_serverless requirement is not working for us, even without constraints.

@dlecina Interesting, thank you for all the detail. I know the MWAA team has been doing some work on Python requirements lately so I wonder if something changed here.
I will try to reproduce this and reopen this if I run into the same. Between the US holiday this week and re:Invent next week it may take me a bit, but I'll try to take a look ASAP.

Thanks @dacort! Yes, I expect there have been some changes in the background that explain the different behavior.
In case it's helpful to anyone, in the end the following combination seemed to work for us; we were able to reach EMR Serverless with this:
--trusted-host nexus.REDACTED
--trusted-host nexusmaster.REDACTED
--index https://nexus.REDACTED/repository/pypi-public/
--index-url https://nexus.REDACTED/repository/pypi-public/simple/
apache-airflow==2.2.2
apache-airflow-providers-amazon==6.0.0
boto>=1.23.9

Context:
Setting apache-airflow-providers-amazon==6.1.0 would be ideal, as it has the correct boto requirement, but then it demands apache-airflow>=2.3.0, which does not work with MWAA 2.2.2, so instead we set boto explicitely and that seemed to work as it does not conflict with either library. Not setting boto explicitely does not work in this configuration because, despite having the EMR Serverless Operator, the boto requirement is set to an older version which does not have the emr-serverless API and it will fail when running the task.
In short:
apache-airflow-providers-amazon==6.1.0 -> apache-airflow>=2.3.0 ❌ boto3>=1.24.0 ✔
apache-airflow-providers-amazon==6.0.0 -> apache-airflow>=2.2.0 ✔ boto3>=1.15.0 ❌
apache-airflow-providers-amazon==6.0.0 + boto>=1.23.9 -> apache-airflow>=2.2.0 ✔ boto>=1.23.9 ✔

Just to confirm, I was still able to use MWAA 2.2.2 with the release from this repository without a problem.
My requirements file is just this plugin, though.
emr_serverless @ https://github.com/aws-samples/emr-serverless-samples/releases/download/v1.0.1/mwaa_plugin.zip

I'm using the CDK stack from this repository.
I'll also try with the constraints-no-provider file as well and see if that works.


fix: mul with endomorphism

previously not correctly handling points at infinity

Benchmark results
Metrics with a significant change:

note_history_trial_decrypting_time_in_ms (10): 397 (+76%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Values are compared against data from master at commit ce51d206 and shown if the difference exceeds 1%.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
128 txs

l1_rollup_calldata_size_in_bytes
45,700
180,580
720,100

l1_rollup_calldata_gas
222,760
871,240
3,466,000

l1_rollup_execution_gas
315,951
989,688
3,686,378

l2_block_processing_time_in_ms
1,188 (+2%)
4,372
17,288

note_successful_decrypting_time_in_ms
322 (-1%)
1,025 (+3%)
3,751

note_trial_decrypting_time_in_ms
33.4 (-10%)
51.4 (-58%)
145 (-6%)

l2_block_building_time_in_ms
17,724 (-2%)
70,275 (-1%)
282,065 (-1%)

l2_block_rollup_simulation_time_in_ms
13,036 (-2%)
51,622 (-1%)
207,787 (-1%)

l2_block_public_tx_process_time_in_ms
4,662 (-1%)
18,576 (-1%)
74,022 (-1%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
13,767
26,081 (-1%)

note_history_successful_decrypting_time_in_ms
2,401 (+2%)
4,672

note_history_trial_decrypting_time_in_ms
94.5
:warning: 397 (+76%)

node_database_size_in_bytes
17,297,488
32,989,264

pxe_database_size_in_bytes
29,923
59,478

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes

private-kernel-init
308
44,528
25,825

private-kernel-ordering
195
44,145
16,225

base-rollup
1,407 (-1%)
128,654
933

root-rollup
82.6
4,192
729

private-kernel-inner
439
71,331
25,825

public-kernel-private-input
241
32,287
25,825

public-kernel-non-first-iteration
237
32,329
25,825

merge-rollup
7.86 (-1%)
2,712
933

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
0 deployed contracts
1 deployed contracts

tx_size_in_bytes
16,859
40,167

Transaction processing duration by data writes.

Metric
0 new commitments
1 new commitments

tx_pxe_processing_time_ms
551 (-1%)
1,309

Metric
0 public data writes
1 public data writes

tx_sequencer_processing_time_ms
0.497 (-4%)
576 (-1%)


[Bug]: The user should not be allowed to connect 2 different apps to a single git repo

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
The user should not be allowed to connect 2 different apps to a single git repo
Steps To Reproduce

In the release instance
Create a new org
Now add 2 apps app1 and app2
Now Connect APP1 to git repo 1 just connect dont do a first commit
Now Connect App2 to same git repo
The connection will be successful
Now do a first commit from app1 after that Try the commit from App2 the commit Fails

Environment
Release
Version
Cloud

@abhvsn Can we auto commit and push the app on the connect CTA? This would be a better approach to avoid edge cases like this?

We have moved away from this as it was taking somewhere around 12-15sec, but let us explore other ways to tackle if not we can come back to the solution you have mentioned

Closing this as this is already fixed


Feature Request: Drag and Drop for Files

Bug, feature request, or proposal:
Feature
What is the expected behavior?
When a user drops a file from their operating system, onto a webpage(, or specific component), it should trigger an event that has the data for that file, so that, that file can be sent through an http request someplace.
What is the current behavior?
There is the cdk-drop, but it doesn't allow for a drop event to be registered if file is from outside of browser onto browser.

You mean something like

@fxck potentially yes, however, I can see (drop) using the above syntax can be seen as something the core Angular framework should deal with and not Material. I was thinking of something more along the lines of:
<cdk-drop (drop)="onDropped($event)">

</>
</cdk-drop>

Closing as duplicate of https://github.com/angular/material2/issues/1347
As file uploads are not part of the material spec, there are not any plans to support file uploads whether via drag or input.


Collapse without transitions doesn't work on nested triggers.

Having i.e. a trigger like:

Trigger for the first panel

works on all browsers when the h4 is clicked but only works on browsers with transitions supported if the anchor glyph is clicked. I think this is because the event propagates from the anchor to the h4 and causes the panel to hide as soon as it's shown by triggering the click on the h4.
My workaround was to create an anchor in the panel title e.g:

Trigger for the first panel

however this is not ideal since I want the entire panel title to trigger the panel collapse/expand.

@apokryfos Would you be so kind as to make a JS Bin or JS Fiddle that demonstrates the problem?

See http://jsfiddle.net/wq428aLz/
The (+) glyph works on Chrome but in IE 9 it closes other open panels but doesn't open the current one.
Clicking on the panel title (anywhere in the grey area) works for both.

Bootstrap 3 is no longer being officially developed or supported.
All work has moved onto our next major release, v4. As such, this issue or pull request is being closed as a "won't fix." For additional help and support, we recommend utilizing our community resources. Thanks for your understanding, and see you on the other side of v4!
<3,
@mdo and team


Added Discord Alias, clearCache command

While I dont know if this is finished yet, you need to register that command in the main

Lmao we both requested the same thing

But like what happens after clearing the data lol

The data is a cache, so any time that someone does
 PublicDataManager.getData(/* path goes here */)

It looks in the cache first, and if not it requests it

I approve flag wanting to increase his commits from 2 to 3 <3

I told him I'll get him crepes whenever he gets to #3 top commiter


[3.x] Fix RigidBody collision update after changing collision layer/mask

Changing the collision layer of a sleeping body was not triggering area updates correctly (found this bug when testing more cases around #53997).
Bodies need to be active for collision to be checked against already overlapping bodies and areas.
Neighbors need to be activated too in order to handle the case where a static body is modified (it can't be activated directly but paired bodies need to check their collision again).
In 3D, moved the call to wakeup() from the physics server to BodySW::_shapes_changed to make it consistent with 2D and also handle the case where shapes are modified (_shapes_changed is called in both this case and collision layer changes).
3.x backport of #54110
(milestone set to 3.5 to give more time for catching potential regressions)

Thanks!


Add gain sub-element and importance attribute to non object type audioBlockFormats

In 2076-2 gain sub-element and importance attribute are now available in all typeDefinitions of audioBlockFormat.

maybe do #72 at the same time


Tiles 25660, 27438 and 27802 from 20231220 have unexplained EFFTIME pattern.

Images of the tile QA attached. Tiles currently marked as unsure due to thrufrac alerts and efftime pattern. For tile 27438 in particular the efftime pattern does not transition smoothly at petal edges.

I took a look at these today and didn't understand them.  e.g., 211321 is particularly bad.  The ratios of imaging to spectroscopic fluxes looks like this

which looks basically fine though there are a number of high outliers on the petals on one side, which may be what is leading to the large THRUFRAC and deep EFFTIME.  But I don't have a sense for what can lead to a random smattering of high points here.  The sframes look fine, so it's hard to imagine the spectroscopic fluxes are dramatically in error.

Let's just mark all of these as bad and reobserve.  I do not understand what happened here.  But it's 3 bright tiles.
20231220/211320, 211321, 211323, on tileid 25660, 27438, 27802.  @abhi0395 @akremin

@schlafly I have marked these exposures as bad and have re-processed these tiles. nightqa and tsnr have been updated. Please close the ticket if everything looks okay to you.

Fixed, closing!


About the Spinal cord grey matter segmentation challenge dataloader

Hi!
I noticed that you only released the M&Ms dataloader in this repo. I downloaded two dataset and I found that the SCGM dataset is much smaller. So I think it is easier to reproduce since I don't have too much GPU resources. Could you also make the SCGM dataloader available?

Hi, thanks for your interest, we did not manage to release that part of the code.
You can find one data loader for SCGM here: https://github.com/wyf0912/LDDG/blob/master/dataset/spinal_cord_challenge.py
It might not be well aligned with my code. But it is not hard to build the data loader based on that for vMFNet.
Best wishes,
Xiao


NO-TICKET: Add more logging to track block proposals. (Backport to 0.9.5)

This adds INFO-level logs for receiving and validating proposed blocks, and whenever we send a witness unit in a round where we didn't receive a block.
Backport of #1191 to 0.9.5.

bors r+


Adds encoding configuration to recorder. see #386

What do you think about this @slandelle

Thanks, I'll have a look soon. I'm in the middle of a check API refactory (understand in the middle of nowhere).


API request hangs in case of error

When using Upload/download example you provided, in case of obfuscation error the process just silently hangs untli you kill it.
The error I am getting Error (code -313) and have no idea what that means.
I put this logging information:
      .then(function (res) {
         console.log("downloading files");
         return jScrambler.downloadCode(client, res.id);
      })

and I can see "downloading files" in the console, this is a last message and after it there is no anything else.

Can you please contact JScrambler's support? The problem appears to be in the services side and not in the client.

Hi José, sorry for late response. I think it would be really cool to have some timeout option on a client side, so it will not hang forever.


ENH: tags list directive

Was talking to @esibinga about how sometimes you need a list of tags in the documentation example and also want a glossary example and thinking it might be nice to have a directive that's kinda a mix of the sphinx-gallery ..mini-gallery and matplotlibs ..math_symbol_table.
So for example. given tags: [a:1, a:2, a:3, b:1, b:2, b:3]
.. list-tags:: a:*
  :wrap: True

would print:

a:1, a:2, a:3, a:4

#60 could be implemented as part of this, e.g.:
tags =  {a:1: A, a:2:B, a:3:C, b:1:D, b:2:E, b:3:F]

.. list-tags:: b*
  :glossary: True
  :wrap: False

would yield:

tag
meaning

b:1
D

b:2
E

b:3
F

So essentially this would be a way to create tag subgroups, and then list those on demand.


Enable Paint Debug while running a Flutter App in Debug Mode

After setting the flag debugPaintSizeEnabled to true, the app the does not display the layout grids while running the app in debug mode.

Please add the output of flutter doctor -v.
Do you invoke a rebuild after you set debugPaintSizeEnabled?
IntelliJ has a toolbar button to toggle while the app is running

I don't know if there is one in intelliJ
Also if you use flutter run you can toggle it by pressing p in the terminal window (might also work in VSCode)

Are these commands supported when using VS Code F5 Debug? Because when I go to the debug console it does not respond to these commands.

@DanTup ?

@Coder-Dan Those keys are just shortcut keys in the console runner, they won't work if typed into the  Debug Console (which just evaluates what you type as Dart expressions). However, there are equivalent commands in the VS Code command palette (which, you can keybind if you wish):

Note: They'll only appear when you're in a debug session that supports them.
Let me know if you have any problems using them!


TST: Fix lightkurve 2.0.x compat, improve test header

Looks like lightkurve finally changed their API.
Also add additional dependencies to test header.
TODO

[x] How to handle lightkurve/lightkurve#985 ?
[x] How to handle lightkurve/lightkurve#986 ?
[x] How to handle lightkurve/lightkurve#987 ?
[ ] How to handle lightkurve/lightkurve#988 (dev job)?

As I was playing I realize that we still need to update vetters.ModShift to use
time, flux, time_offset_str =
lightkurve_utils.unpack_lk_version(lc, self.lc_name)
This code was written so wecould go back and forth between new and old lightkurve, but apparently it wasn't implemented for ModShift.  Do you want to do that as part of this pull request, or should I open a new one?

go back and forth between new and old lightkurve

I am 👎 on this. Given that corazon already uses lightkurve 2.0, I don't see a real need for such compatibility layer. It is only going to increase maintenance burden. Why do you still need lightkurve 1.0?

This compatibility layer already exists. I wrote it so I could work with corazon and lightkurve at the same time.  It seems to me the easiest path forward is to just make ModShift use the same layer and we can phase it out once we are sure everything is working correctly with lightkurve 2.0.  I have no plans to long term support it, but since we already do, it is more work to take it out.

I don't have time to work on this today or the next few days, so feel free to fix modshift if you want. Thanks!

I think this is ready to go.  Merge when you are ready.  I'd officially approve it, but I can't find the github button.

Thanks! I merged it. I'll open a separate issue for the dev job as it is pending reply from Geert.


Generalize signatures for u to z

Summary
This generalizes the signatures for prim/fwd/rev from u to z
Tests
Refactor so no new tests
Side Effects
Not yet, but here's a weird one, we don't call the rev specialization of unit_vector_constrain and if we do the tests fail. This happens because the header guard in rev is the same as the prim one, which happened about 4 years ago. We could fix it or delete it.
Release notes
Generalize signatures for Eigen functions from U to Z.
Checklist

[ ] Math issue #(issue number)

[x] Copyright holder: Steve Bronder
The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[x] the basic tests are passing

unit tests pass (to run, use: ./runTests.py test/unit)
header checks pass, (make test-headers)
dependencies checks pass, (make test-math-dependencies)
docs build, (make doxygen)
code passes the built in C++ standards checks (make cpplint)

[x] the code is written in idiomatic C++ and changes are documented in the doxygen

[x] the new changes are tested

If there's a reverse-mode specialization for unit_vector_constrain that works with the way unit-vectors currently work, then we should use it.

If there's a reverse-mode specialization for unit_vector_constrain that works with the way unit-vectors currently work, then we should use it.

@bgoodri can you look at the code there to see why that test is failing when the header guard is changed?

The header guards should definitely be distinct. Which test is failing if we do that?

@bgoodri fixed the transform and it looks like the reverse implementation matches the new definition's intent in that it has the normal density term lp -= 0.5 * dot_self(y) in the constraining transform.
By the way, that's better written as lp += -0.5 * dot_self(y) because -0.5 will be interpreted as a double so you get an add and multiply instead of the less efificient subtract and multiply.

The header guards should definitely be distinct. Which test is failing if we do that?

Changing the header and adding unit_vector_constrain to rev/fun.hpp causes
./runTests.py ./test/unit/math/mix/fun/unit_vector_constrain_test

to fail

@serban-nicusor-toptal do you know whatsup with the error I'm seeing here?
 Skipping blame.

[LLVM/Clang] [-ERROR-] hudson.plugins.git.GitException: Command "/usr/bin/git rev-parse HEAD^{commit}" returned status code 128:

stdout:

stderr: fatal: Not a git repository (or any of the parent directories): .git

https://jenkins.mc-stan.org/blue/organizations/jenkins/Stan/detail/downstream_tests/1509/pipeline
idt that would be something on my end. Also I think we should throw if the performance tests fail

I agree with @t4c1 and would like to keep the code as simple as possible.  Specifically,

Only forward where necessary.  It isn't necessary for our autodiff types, is it?

Only use requires where we need them.  If the arguments don't match the concept, it won't compile.  In general, I don't like the idea of repalacing a concrete data type with a template parameter plus requires if the set of matches is the same.

Only use requires where we need them. If the arguments don't match the concept, it won't compile.

I disagree. I prefere requires to a template and a funtion that does not compile with wrong arguments. In such cases error messages usually have little or nothing in common with the fact that a function does not support certain types.

In general, I don't like the idea of repalacing a concrete data type with a template parameter plus requires if the set of matches is the same.

+1 to that part. We only need to do that with Eigen types to make functions accept general expressions.

It isn't necessary for our autodiff types, is it?

No, it is not. It is only needed for types that have move costructors/ move assigment operators. Usually those are the types that dynamically allocate memory, such as std::vector or Eigen::Matrix.

Yeah I went a little forward loco here, it was mostly from trying to get  value_of working right. I'll remove it where we have no need for forwarding

@SteveBronder How is this going? If you are too busy with static matrix stuff I can take over.

@SteveBronder How is this going? If you are too busy with static matrix stuff I can take over.

Hey sorry yes got distracted by static stuff. Let me make one more push on this tonight

@t4c1 can you give this a look when you have a minute?

Oh, the ancient PR is alive! ;)

Lol yes I think at one point I was like, "Let me ping you after I read this over" and then it was lost to the sands of time

All the functions you changed should be tested by expression testing framework.

Reading that over how do I add internal functions that are not directly callable in stan like unit_vector_constrain? I removed the ones from the exceptions list like variance.
Can you please stop pushing perfect forwarding everywhere?
Yes, now that I've dived harder into Stan's memory pattern they are really not useful for var types (for anyone reading since vars are just pointers to vari copying a var is the same as a move). Though they are useful when working with floating point matrices!

Did you just generalize whole prim/err?

Yeah sorry is that alright? I was getting errors up in the stan lib that those functions didn't accept Eigen::Map<> types since some of them are used in reader and I'd like reader to work over maps

Those in general don't actually need to accept Eigen matrices. You can either write tests yourself or extend the framework adding a list of non-exposed signatures you want tested.

Yeah I can do that

@SteveBronder Can we try getting this in for the release?

Yes def I'l clean this up today!


Add live overrides

Allow overide sets, which set multiple overrides at once (and allow extending other ones, etc.

[ ] Allow override on each node
[ ] Allow named override sets in top level
[ ] Allow selecting an override set in UI
[ ] override selection shows in URL (omit if '')
[ ] Animate as overrides change

Animation: https://blog.greggant.com/posts/2018/10/10/svg-path-d-animation-in-2018.html

dd4ad4f7bc64201e4862578db09859fb023f2b99 was also part of this but erroneously didn't mention it in the commit message.


AggressiveSplittingPlugin does not support child compilations which emit to static asset file paths

Do you want to request a feature or report a bug?
bug

Actually I've tried asking about this in the webpack gitter channel several times now and haven't seen a response yet.

What is the current behavior?
webpack/lib/optimize/AggressiveSplittingPlugin does not support child compilations which emit to static asset file paths correctly.
Please see https://github.com/jantimon/html-webpack-plugin/pull/670 for more details. This means you cannot use AggressiveSplittingPlugin with any plugin with creates a child compilation that emits to a static asset file name. This is including (but, not limited to) the following webpack plugins:

extract-text-webpack-plugin
html-webpack-plugin
script-ext-html-webpack-plugin

If the current behavior is a bug, please provide the steps to reproduce.
Use AggressiveSplittingPlugin with any webpack config which uses one of the plugins listed above. For example:

run:

git clone https://github.com/simonsmith/github-user-search.git
cd github-user-search
yarn

add AggressiveSplittingPlugin to it's webpack config.
run:

yarn build

You'll see an error which looks something like this:
ERROR in chunk 1 [initial]
extract-text-webpack-plugin-output-filename
Conflict: Multiple assets emit to the same filename 'extract-text-webpack-plugin-output-filename'

What is the expected behavior?
This plugin should support the static asset file name strategy that many plugins use in child compilations.
If this is a feature request, what is motivation or use case for changing the behavior?
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.
Versions of node or webpack probably doesn't matter for this, as it's been an issue for a long time.

I am willing to contribute and provide a PR for this issue. I just need some direction on what would be the preferred solution, as I believe a poor solution could result in adding a lot of unnecessary advanced-optimize-chunks bundle time..

anyone? @sokra i was told you had some insight on this.

change plugin("compilation", ... to plugin("this-compilation", ... in the Plugin.

@sokra I've tried this. The problem is this breaks the minSize/maxSize functionality. More details here: https://github.com/jantimon/html-webpack-plugin/pull/670#issuecomment-301639384

Well not sure this is going anywhere. I suppose just close?


Profile-guided optimization

Add a pgo mode to the compiler, which compiles with logging for runtime types (or more in the future) which is dumped to file. Then, add an argument to regular compile which takes in such file as input and uses it for optimization.

Done in 97bb4f33be95a23eaebd017cea80a90a1d49abef!


With refresh token expiration offset seconds

Hoist P/R Checklist
Pull request authors: Review and check off the below. Items that do not apply can also be
checked off to indicate they have been considered. If unclear if a step is relevant, please leave
unchecked and note in comments.

[ ] Caught up with develop branch as of last change.
[ ] Added CHANGELOG entry, or determined not required.
[ ] Reviewed for breaking changes, added breaking-change label + CHANGELOG if so.
[ ] Updated doc comments / prop-types, or determined not required.
[ ] Reviewed and tested on Mobile, or determined not required.
[ ] Created Toolbox branch / PR, or determined not required.

If your change is still a WIP, please use the "Create draft pull request" option in the split
button below to indicate it is not ready yet for a final review.

Pull request reviewers: when merging this P/R, please consider using a squash commit to
collapse multiple intermediate commits into a single commit representing the overall feature
change. This helps keep the commit log clean and easy to scan across releases. PRs containing a
single commit should be rebased when possible.

Closed in favor of slight rework of API changes just merged here:
https://github.com/xh/hoist-react/commit/0b67abe197e7f3902da0ca4c57b7589ab954b066


Missing Nuget

Hi @dmariogatto ,
I'm converting my project from Xamarin and noticed that the NuGet Package is missing for this library. Could you make it available again?
Many thanks!

@varyamereon
Here you go. Enjoy!

Very nice, thank you!!


Add autofocus attribute

I'm not sure if this should be autofocus or autoFocus. It seems like, given React's docs, it should be autoFocus, but that won't help me solve https://github.com/developit/preact/pull/653, so...
In short, I'm a bit at a loss here 😄

Yeah, I think that you're right it should be autoFocus...

Thank you for your pull request! I recently found a bug in the package regarding the omission of some React specific attributes. There was an update that fixed all that.
https://github.com/jackyho112/react-html-attributes/blob/master/src/react-html-attributes.json#L8
Thanks again for reporting the bug!

Fixed here: https://github.com/jackyho112/react-html-attributes/commit/b4b9450031bc6a6e7ecbac8215dbe22e323892a0

Fixed here: https://github.com/jackyho112/react-html-attributes/commit/b4b9450031bc6a6e7ecbac8215dbe22e323892a0

Awesome, thanks.


Add "orange" theme

Adding @felixhahnweilheim to review and hopefully LGTM the listing.

Yes, looks good. You could also add my Flex Theme (https://github.com/felixhahnweilheim/humhub-flex-theme) if you like.

I created issue #7 as a place to have the conversation around this lists intention and how it would serve people best.


Editing input field causes cursor to jump to end of string

When I try to edit within the date field, the cursor moves to the end of the string and instead of being able to backspace and insert a new value.
Steps to reproduce:

import { formatWithMask } from 'react-native-mask-input'

const Test = () => {
  const [date, setDate] = useState('')

  const { masked } = formatWithMask({
    text: date,
    mask: [/\d/, /\d/, '/', /\d/, /\d/, '/', /\d/, /\d/, /\d/, /\d/],
  })

 return (
   <>
     <input placeholder="mm/dd/yyyy" value={masked} onChange={(val) => setDate(val)} name="date" />
   </>
  )

}

export default Test

Expected behavior: Be able to click within date, backspace to remove value and add new value
Actual behavior: The second slash is removed and the cursor is moved to the back of the input

👆 ++

++

up

This issue is causing us to reconsider using the library, unfortunately. It degrades the UX and confuses users.

There's no reproducible code

are you guys just using the formatWithMask function on a web project with a input html tag? so the cursor position is not function's fault.
You can try use the hook useMaskedInputProps, see the example usage on README and adapt to use within the input html

@CaioQuirinoMedeiros Here is a repro link: https://codesandbox.io/s/immutable-morning-9ioot7?file=/src/App.js
Editing the year is ok, but when you try to edit the day or the month, you'll see the problem. The year gets half deleted. Not a good UX

No. Sorry guys, I'm not having time for this

HI @oliviercperrier, have you picked an alternative library?
The same problem.  😢

HI @oliviercperrier, have you picked an alternative library?
The same problem.  😢

@arhipy97 Ive built my own 😂

Can you share how you done it??

In case it helps any of you, the root cause of the app is described here (not written by me, but the blog post links to this specific specific issue)
https://giacomocerquone.com/keep-input-cursor-still/


Add shared context between components

It would be a good idea to introduce an object like context that is shared between all components. Here's an example:
eryn.render('file.eryn', { test: 'hello' }, { test2: 'This is shared' });

file.eryn
[| context.test |] // hello

[| shared.test2 |] // This is shared

[|% comp.eryn /|]  // Note that no context is passed.

comp.eryn
[| context |] // { }

[| shared.test2 |] // This is shared

Added.


French for 20 seems incorrect

Address expansion should rewrite words to numbers.  Testing with the command-line tool src/libpostal, it fails with this correct French address:
src/libpostal 'Quatre-vingt-douze Ave des Champs-Élysées'
Result:
12 avenue des champs-elysees
12 avenue des champs elysees
12 avenue des champselysees
42012 avenue des champs-elysees
42012 avenue des champs elysees
42012 avenue des champselysees

However, if you misspell "vingt" by swapping the 'g' and the 'n' it works as expected:
src/libpostal 'Quatre-vignt-douze Ave des Champs-Élysées'
Result:
92 avenue des champs-elysees
92 avenue des champs elysees
92 avenue des champselysees

I'm not a native French speaker so I verified the correct spelling of "vingt":
http://dictionary.reverso.net/french-english/vingt
If you enter vignt into the search box on the reverso.net site, it corrects it to vingt and displays the same URL as the one quoted above.
Note: this example came from the blog posting announcing libpostal.  I shared the blog posting and one of my co-workers spotted the misspelling, so I don't get the credit for spotting this.

:open_mouth: never noticed that (strangely no French speakers pointed it out yet either). Thanks! It has been fixed in libpostal and the example is being updated across the bindings.
Just pull latest from libpostal (this repo), then run make clean followed by make and sudo make install.  For the Python bindings, I would also run pip uninstall postal followed by pip install postal for good measure to make sure it recompiles the Python bindings.


Where is the config file sipmask_r50_caffe_fpn_gn_1x_4gpu.py of sipmask-vis?

Hello, thanks for your good work. I cannot find the config file sipmask_r50_caffe_fpn_gn_1x_4gpu.py  in the vis project. There is just the configs/sipmask/sipmask_r50_caffe_fpn_gn_1x.py, and I have train with it, but got near 28 mAP in the evaluation website. So I want to ask  what should I do more to get higher mAP. Thanks.

@zack624 Thanks for interests. how many gpus are you using for training?

@zack624 Thanks for interests. how many gpus are you using for training?

Just one. I run directly "python tools/train.py". And I also download your checkpoints to infer, but there is a lot of missing keys like that:
`unexpected key in source state_dict: bbox_head.cls_convs.2.conv.weight, bbox_head.cls_convs.2.gn.weight, bbox_head.cls_convs.2.gn.bias, bbox_head.reg_convs.3.conv.weight, bbox_head.reg_convs.3.gn.weight, bbox_head.reg_convs.3.gn.bias, bbox_head.track_convs.2.conv.weight, bbox_head.track_convs.2.gn.weight, bbox_head.track_convs.2.gn.bias, bbox_head.track_convs.3.conv.weight, bbox_head.track_convs.3.gn.weight, bbox_head.track_convs.3.gn.bias, bbox_head.fovea_track.weight, bbox_head.fovea_track.bias
missing keys in source state_dict: bbox_head.sipmask_track.bias, bbox_head.sipmask_track.weight`
A lot of input pictures are not be inferred, and the output file of json is just 200K.

@zack624 you means that the provided inference model and the config file in vis donot match?

@JialeCao001  Thanks for your answer. I guess there might be some differences between the provided model file with the sipmask_r50_caffe_fpn_gn_1x.py config, so I ask for the sipmask_r50_caffe_fpn_gn_1x_4gpu.py mentioned in project readme.

@zack624 Hi. I remove the _gpu from file name. I test on my device. It seems no error.

Hi, I have the same problem about the near 28 mAP in the evaluation website. Have you sloved it, please? If sloved it, can you tell me how it do? Thank you very much.