[feature] tasks with dynamic parameter specifications

this one is a long shot, and I would totally understand if you decide that it's just too much for overseer, bloaty, and so on. Overseer is already great as it is, I would totally understand!
But I thought, why not mentioning it. I wrote a custom task for a script I'm using at work.. Well technically I'm registering a template:
require('overseer').register_template(
  {
    name = "my task",
    params = {
      branch = { type = "string"},
    },
    builder = function(params)
     ...

Now the parameter, branch, is in fact the name of a branch for a git repository i have somewhere else on my disk. So currently I'm typing the branch name by hand, or pasting it. But in an ideal world, for instance, params, instead of being a constant, could optionally be a function.. in there i would make a plenary sync() call to git to get the list of branch names for that repo.. then I would return a single parameter, but instead of being of type string, it would be of type enum, enumerating all the possible branch names (I'm not sure how much overseer enums can scale in terms of number of members though.. presumably it would be OK).
That way, with the recent cmp completion for task parameters, I could have nice completion and validation for the 'branch name' parameter. But obviously the 'params' function would have to be called again every time the user would invoke the task.
So yes, as I said... I think it would be totally valid to reject this as completely out of scope for overseer, but I thought i'd mention it :)
And thank you again for an amazing tool!

I love this use case, and it would be great to support it somehow. It sounds like the ideal would be to insert some sort of async step in between selecting the task and doing the UI prompt + run logic. I'd have to do some thinking to figure out what a good API would be for that.
Something you could try now: instead of registering this as a template, instead use a template provider. Then you can run the git branch command as part of the async logic of the provider, and use the branches to set the schema of the template. The downside is that you incur the latency of the git command before your tasks will show up in the list, but the upside is that it will work today!

Thank you for the answer! Regarding the template provider idea, I think there may be an issue, that that the template provider will presumably be invoked only once when overseer is set up, and not every time the user asks for the task list? Or is it possible I'm wrong and it's invoked every time?
(It would be bad if I'm not offered a branch as option if it was added after the task list was first retrieved)

Yeah actually I realized that template providers of course must be invoked for task discovery each time, otherwise mix rake and other new tasks wouldn't be detected. So of course that would be a solution!

Yep, it's invoked every time. Thus the caching logic that I added a while back. So that comes with the caveat that if the provider is slow, the results may become cached and then the branches won't be refreshed until you clear the cache.

Hello, I have the same use case with a cmake based build system for c++.
I have a task to configure cmake and then I have another task for building the project that uses input parameters (compiler, build type and target).
And instead of entering those by typing a string I would be querying the cmake-file-api to create an enum for the available options.
@emmanueltouzery did you manage to make a temporary solution work for this use case ? If so, could you provide some snippet of you solution ?

yes, I used that to great effect, here is an edited portion of code where I take advantage of this:

-- get the list of branches and offer to pick one
require('overseer').register_template(
{
  name = "branch-using-task",
  cache_key = function(opts)
    return "branch-using-task"
  end,
  generator = function(opts, cb)
    local branches = {}
    local jid = vim.fn.jobstart({
      "git",
      "branch",
      "--sort=-committerdate",
      "--format=%(refname:short)"
    }, {
      cwd = opts.dir,
      stdout_buffered = true,
      on_stdout = vim.schedule_wrap(function(j, output)
        for _, line in ipairs(output) do
          table.insert(branches, line)
        end
      end),
      on_exit = vim.schedule_wrap(function(j, output)
        cb({{
          name = "name of the task",
          params = {
            branch = { type = "enum", choices = branches},
          },
          builder = function(params)
            local args = {"rake taskname"}
            return {
              cmd = {'rake'},
              args = args,
              env = {
                BRANCH = params['branch'],
              },
            }
          end,
        }})
      end)
    })
    if jid == 0 then
      log:error("Passed invalid arguments to 'git'")
      cb({})
    elseif jid == -1 then
      log:error("'git' is not executable")
      cb({})
    end
  end
})

I'd argue that this is bad practice, because there is no unique history job id<-> status <-> output log <-> command anymore.
Consider 3 jobs:
RUNNING: gen3
out: waitForMsg: msgty: StopTransaction
────────────────────────────────────────
RUNNING: gen2
out: last_timeline_append 2023-09-22 10:16:47.742601
────────────────────────────────────────
FAILURE: simu16
out: FATAL: port  12000  already used.
────────────────────────────────────────
FAILURE: gen3
out: socket.timeout: timed out
────────────────────────────────────────
RUNNING: simu16
out: 2023-09-22 12:16:38: TestTransactions - 6 - Wait for 21 seconds to avoid too early deauthorization

I do want each task name have a unique meaning in terms of what cmd arguments have been used (ideally also with storing the file hash after the artifacts have been produced meaning build system integration, but that sounds complex to implement).

@matu3ba I don't know what part of this you're referring to is bad practice. Dynamically generating enums for the parameter of a task? Your complaint seems to be that tasks can have the same name, which doesn't have any bearing on this issue. Also, task names can be completely user defined, and default to the executable + all arguments. Doesn't that already do what you want?

Also, task names can be completely user defined, and default to the executable + all arguments. Doesn't that already do what you want?

Agreed, but more so since the design of overseer is to not do execution book keeping besides what the user (recently) started.

I've added support for defining template params as a function. It is not async though, so all i/o operations must be blocking.


error: Import 'https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts' failed: 500 Internal Server Error

I tried to build Attain, but it doesn't work. Apparently, it is referring to a repository that does not exist.
Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download<EMAIL_ADDRESS>Download https://cdn.pika.dev/-/mime-db@v1.44.0-dKNwMIxpxaF4opR7ympR/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/mime-db@v1.44.0-dKNwMIxpxaF4opR7ympR/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/mime-db@v1.44.0-dKNwMIxpxaF4opR7ympR/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/mime-db@v1.44.0-dKNwMIxpxaF4opR7ympR/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/http/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/http/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/http/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/http/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/http/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/http/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
error: Import 'https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts' failed: 500 Internal Server Error

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS> https://deno.land/x/attain/encoding.ts
 https://deno.land/x/attain/http-errors.ts
 https://deno.land/x/attain/type-is.ts
 https://deno.land/x/attain/mediaType.ts
 https://deno.land/x/attain/media-typer.ts
 https://deno.land/x/attain/version.ts
 https://cdn.pika.dev/vary
 https://cdn.pika.dev/-/vary@v1.1.2-4z5NtC6Lp63pgchyGfIE/dist=es2019,mode=types/index.d.ts
 https://cdn.pika.dev/http/dist=es2019,mode=types/index.d.ts
 https://deno.land/x/attain/path-to-regexp.ts
 https://cdn.pika.dev/mime-types
 https://cdn.pika.dev/-/mime-types@v2.1.27-EcRNul1gxVHTgSAtG6fe/dist=es2019,mode=types/index.d.ts
<EMAIL_ADDRESS><EMAIL_ADDRESS> https://cdn.pika.dev/-/mime-db@v1.44.0-dKNwMIxpxaF4opR7ympR/dist=es2019,mode=types/index.d.ts

in my case, it's working fine
can you give me more detail? like your code example and what env are you using?

index.ts
import { App, Request, Response } from "https://deno.land/x/attain/mod.ts";

const app = new App();

const sampleMiddleware = (req: Request, res: Response) => {
  console.log("before send")
};

app.use("/:id", (req, res) => {
  console.log(req.params);
  res.status(200).send(`id: ${req.params.id}`);
});

app.use(sampleMiddleware, (req, res) => {
  res.status(200).send({status: "Good"});
});

app.listen({ port: 3500 });

console.log("http://localhost:3500");

log
$ deno run --allow-net index.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
Download https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts
error: Import 'https://cdn.pika.dev/-/http@v0.0.1-security/dist=es2019,mode=types/index.d.ts' failed: 500 Internal Server Error

Version:
Deno: 1.0.1
Linux: 5.4.41-1-lts

This is weird, It's working with vscode extension. But, without it, it returns an error the same as you.
Thanks for letting me know about this issue.
You may have to reload this module or instantly you can use "https://raw.githubusercontent.com/aaronwlee/Attain/master/mod.ts" this path.


Delay calculating the starting LSN when doing timeline branching

Previously, upon branching, if no starting LSN is specified, we
determine the start LSN based on the source timeline's last record LSN
in timelines::create_timeline function, which then calls Repository::branch_timeline
to create the timeline.
Inside the LayeredRepository::branch_timeline function, to start branching,
we try to acquire a GC lock to prevent GC from removing data needed
for the new timeline. However, a GC iteration takes time, so the GC lock
can be held for a long period of time. As a result, the previously determined
starting LSN can become invalid because of GC.
This PR fixes the above issue by delaying the LSN calculation part and moving it to be
inside LayeredRepository::branch_timeline function.

Related discussion: https://github.com/neondatabase/neon/pull/1569#issuecomment-1177693827

I tried to investigate this issue more. First of all I find out that I have used slightly deteriorated version of neon - when I checkout the latest sources the problem really reproduced quite frequently. So something is recently changed...
Then I tried to understand why GC takes so long time, although it is expected to be cheap operation (reading only layers metadata, do not read or write layer file content - just delete deteriorated layers).  And the reason is compaction_cs which is obtained not only by compaction but also by GC. And compaction can really take very long time. Is it acceptabe to delay creation of new branches and GC until compaction is complete?

Then I tried to understand why GC takes so long time, although it is expected to be cheap operation (reading only layers metadata, do not read or write layer file content - just delete deteriorated layers). And the reason is compaction_cs which is obtained not only by compaction but also by GC. And compaction can really take very long time. Is it acceptabe to delay creation of new branches and GC until compaction is complete?

Yeah, that's not great... Created a new issue to track that.

Then I tried to understand why GC takes so long time, although it is expected to be cheap operation (reading only layers metadata, do not read or write layer file content - just delete deteriorated layers). And the reason is compaction_cs which is obtained not only by compaction but also by GC. And compaction can really take very long time. Is it acceptabe to delay creation of new branches and GC until compaction is complete?

I also realize the same thing. Initially about to comment in the original discussion but forgot to do so. It's a bad thing that there is a delay during branch creation because of GC as it contradicts with our description in the website:

Neon allows to instantly branch your Postgres database to support a modern development workflow

Waiting 16s is not instant :)


Fulfillment approve mutation

I want to merge this change because it adds a mutation to approve a fulfillment.
⚠ This is a PR to featue branch.

Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[ ] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[ ] The changes are tested
[ ] GraphQL schema and type definitions are up to date
[ ] Changes are mentioned in the changelog

Task linked: SALEOR-3841 Add orderFulfillmentApprove mutation


horizon extension is compressing the files with the wrong user compared to horizon

Do NOT include any confidential information, this issue will be created on a public repo
This section is completed by the issue reporter
On which version(s) do you encounter this issue?
Please provide the output to 'git describe --all' for  all deployments where you have confirmed it occurs.
Liberty and up
What type of issue is this? Please tick the appropriate box.

[x] bug - existing functionality that does not work for you as documented
[ ] feature - new functionality that you would like to see added

What is the expected behavior?
Use the same user as horizon
What is actually happening?
The static files have mixed permissions, owned by root rather than the horizon_system_user_name configuration
What were you attempting to do when you encountered this error (give as much detail as possible)?
How can we reproduce this issue?
Install any Liberty + release and roll out the horizon extension
How can we verify your issue is addressed?
Ownership is the same for all static horizon files
If you are using or know of a workaround please describe the necessary steps?
What is your evaluation on the importance of this issue, and why?
Prio 3

This section is completed during the issue resolution process. If you are reporting an issue, please do not modify the section below.
Which branches are impacted by this issue?
Remove branches that are not impacted.
For each pull request, update the branch line to reference it.

[ ] mitaka-13.1
[ ] liberty-12.2

Upgrade impact. Consider the impact on moving between major, minor and patch releases.

[ ] This change does not impact upgrades
[ ] This change addresses a potential upgrade impact
[ ] The upgrade impact is being tracked by another issue.

The issue tracking the upgrade impact is:
Testing impact

[ ] An issue/pull-request has been create or exists to track the testing gap, and is referenced below
[ ] This change includes additional testing
[ ] This change is tested by the existing setup

Additional testing links:
Release note

[ ] This change includes a release note.
[ ] This change does not require a release note.

Doc Impact
Does your change require documentation?

[ ] Yes, and I submitted an issue to Docs issues
[ ] Yes, and I contributed a documentation patch to docs-rpc
[ ] No documentation required for this change

Additional documentation links (please add links if you have ticked yes):

This was merged. Additionally, we're trying to have the extensions removed, as we've done in Newton+, which makes this irrelevant going forward. I'm closing this issue as the original bug has been resolved.


The current SDK language version does not match your analyzer package language version

Describe the bug
When I execute :
flutter_rust_bridge_codegen generate

instead of
cargo run --manifest-path ../flutter_rust_bridge/frb_codegen/Cargo.toml -- generate

I have many problems.
My FRB generator is : flutter_rust_bridge_codegen 2.1.0.
One of the problems that I get is :

[INFO] Precompiling build script......
[INFO] Precompiling build script... completed, took 5.2s

[WARNING] Attempting to enable experiments `[class-modifiers]`, but the current SDK
language version does not match your `analyzer` package language version:

Analyzer language version: 3.4.0
SDK language version: 3.5.0

In order to use experiments you may need to upgrade or downgrade your
`analyzer` package dependency such that its language version matches that of
your current SDK, see https://github.com/dart-lang/build/issues/2685.

Note that you may or may not have a direct dependency on the `analyzer`
package in your `pubspec.yaml`, so you may have to add that. You can see your
current version by running `pub deps`.

[INFO] Initializing inputs
[INFO] Building new asset graph...
[INFO] Building new asset graph completed, took 2.9s

[INFO] Checking for unexpected pre-existing outputs....
[INFO] Deleting 9 declared outputs which already existed on disk.
 stderr=

You have hit a bug in build_runner
Please file an issue with reproduction steps at https://github.com/dart-lang/build/issues

PathNotFoundException: Cannot delete file, path = '/Volumes/mac-J/larpoux/projmac/tau/tau/example/linux/flutter/ephemeral/.plugin_symlinks/tau/rust/target/frb_dump/generator_text/api_dart/third_party/web_audio_api/context.freezed.dart' (OS Error: No such file or directory, errno = 2)
dart:async                                                          _rootRunUnary
package:build_runner_core/src/asset/file_based.dart 106:9           FileBasedAssetWriter.delete.<fn>
package:pool/pool.dart 127:14                                       Pool.withResource
dart:async                                                          Future.wait.<fn>
package:build_runner_core/src/generate/build_definition.dart 533:7  _Loader._initialBuildCleanup
package:build_runner_core/src/logging/logging.dart 25:18            logTimedAsync
package:build_runner_core/src/generate/build_definition.dart 280:7  _Loader.prepareWorkspace
package:build_runner_core/src/generate/build_impl.dart 114:27       BuildImpl.create
package:build_runner_core/src/generate/build_runner.dart 34:26      BuildRunner.create
package:build_runner/src/generate/build.dart 109:17                 build
package:build_runner/src/entrypoint/build.dart 35:18                BuildCommand._run
package:args/command_runner.dart 212:13                             CommandRunner.runCommand
package:build_runner/src/entrypoint/run.dart 26:18                  run
.dart_tool/build/entrypoint/build.dart 46:16                        main

[2024-08-06T08:03:32.448Z WARN /Volumes/mac-J/larpoux/.cargo/registry/src/index.crates.io-6f17d22bba15001f/flutter_rust_bridge_codegen-2.1.0/src/library/codegen/polisher/mod.rs:88] Fail to execute_build_runner, but continue to run.
Error details: Err(Failed to run build_runner for "/Volumes/mac-J/larpoux/projmac/tau/tau": [INFO] Generating build script...
[INFO] Generating build script completed, took 502ms

[INFO] Precompiling build script......
[INFO] Precompiling build script... completed, took 5.2s

[WARNING] Attempting to enable experiments `[class-modifiers]`, but the current SDK
language version does not match your `analyzer` package language version:

Analyzer language version: 3.4.0
SDK language version: 3.5.0

In order to use experiments you may need to upgrade or downgrade your
`analyzer` package dependency such that its language version matches that of
your current SDK, see https://github.com/dart-lang/build/issues/2685.

Note that you may or may not have a direct dependency on the `analyzer`
package in your `pubspec.yaml`, so you may have to add that. You can see your
current version by running `pub deps`.

[INFO] Initializing inputs
[INFO] Building new asset graph...
[INFO] Building new asset graph completed, took 2.9s

[INFO] Checking for unexpected pre-existing outputs....
[INFO] Deleting 9 declared outputs which already existed on disk.
)
[31.4s] Parse
  └── [30.6s] Cargo expand & syn parse
  └── [0.5s] Parse HIR
  └── [0.3s] Parse MIR
[5.6s] Generate
[25.4s] Polish
  └── [21.2s] Run Dart build_runner
  └── [3.4s] Run Dart formatter
  └── [0.8s] Run Rust formatter                                                                                Done!
    Updating crates.io index
^C     Fetch [=================>               ] 142 complete; 9 pending

Steps to reproduce
run
flutter_rust_bridge_codegen generate

Logs
-

Expected behavior
No response
Generated binding code
No response
OS
No response
Version of flutter_rust_bridge_codegen
No response
Flutter info
No response
Version of clang++
No response
Additional context
No response

I switched to the Stable Flutter Channel but it is not very much better.
I confirm that :
cargo run --manifest-path ../flutter_rust_bridge/frb_codegen/Cargo.toml -- generate

works more or less correctely, but :
flutter_rust_bridge_codegen generate

gives me various errors.
Another error that I get is :
error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:3589:89
     |
3579 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
3589 |                         0 => api_that_guard = Some(api_that.lockable_decode_async_ref().await),
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:3595:83
     |
3579 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
3595 |                     web_audio_api::context::AudioContext::close(&*api_that_guard).await;
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:5470:89
     |
5460 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
5470 |                         0 => api_that_guard = Some(api_that.lockable_decode_async_ref().await),
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:5476:85
     |
5460 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
5476 |                     web_audio_api::context::AudioContext::suspend(&*api_that_guard).await;
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:8207:89
     |
8197 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
8207 |                         0 => api_that_guard = Some(api_that.lockable_decode_async_ref().await),
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:8213:91
     |
8197 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
8213 |                     web_audio_api::context::OfflineAudioContext::resume(&*api_that_guard).await;
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:8326:89
     |
8316 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
8326 |                         0 => api_that_guard = Some(api_that.lockable_decode_async_ref().await),
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:8333:26
     |
8316 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
8333 |                         .await,
     |                          ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:8474:89
     |
8464 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
8474 |                         0 => api_that_guard = Some(api_that.lockable_decode_async_ref().await),
     |                                                                                ^^^^^ only allowed inside `async` functions and blocks

error[E0728]: `await` is only allowed inside `async` functions and blocks
    --> src/frb_generated.rs:8484:22
     |
8464 |             transform_result_sse::<_, ()>((move || {
     |                                            ------- this is not `async`
...
8484 |                     .await;
     |                      ^^^^^ only allowed inside `async` functions and blocks

For more information about this error, try `rustc --explain E0728`.
error: could not compile `tau` (lib test) due to 10 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `tau` (lib) due to 10 previous errors
Error: ~/bin/generate.sh

And another error that I have some time when using flutter_rust_bridge_codegen generate :
[INFO] Generating build script completed, took 234ms

[INFO] Precompiling build script......
[INFO] Precompiling build script... completed, took 3.0s

[INFO] Initializing inputs
[INFO] Building new asset graph...
[INFO] Building new asset graph completed, took 1.8s

[INFO] Checking for unexpected pre-existing outputs....
[INFO] Deleting 9 declared outputs which already existed on disk.
 stderr=

You have hit a bug in build_runner
Please file an issue with reproduction steps at https://github.com/dart-lang/build/issues

PathNotFoundException: Cannot delete file, path = '/Volumes/mac-J/larpoux/projmac/tau/tau/example/linux/flutter/ephemeral/.plugin_symlinks/tau/example/windows/flutter/ephemeral/.plugin_symlinks/tau/lib/public/rust/third_party/web_audio_api/context.freezed.dart' (OS Error: No such file or directory, errno = 2)
dart:async                                                          _rootRunUnary
package:build_runner_core/src/asset/file_based.dart 106:9           FileBasedAssetWriter.delete.<fn>
package:pool/pool.dart 127:14                                       Pool.withResource
dart:async                                                          Future.wait.<fn>
package:build_runner_core/src/generate/build_definition.dart 533:7  _Loader._initialBuildCleanup
package:build_runner_core/src/logging/logging.dart 25:18            logTimedAsync
package:build_runner_core/src/generate/build_definition.dart 280:7  _Loader.prepareWorkspace
package:build_runner_core/src/generate/build_impl.dart 114:27       BuildImpl.create
package:build_runner_core/src/generate/build_runner.dart 34:26      BuildRunner.create
package:build_runner/src/generate/build.dart 109:17                 build
package:build_runner/src/entrypoint/build.dart 35:18                BuildCommand._run
package:args/command_runner.dart 212:13                             CommandRunner.runCommand
package:build_runner/src/entrypoint/run.dart 26:18                  run
.dart_tool/build/entrypoint/build.dart 46:16                        main

[2024-08-06T11:31:59.790Z WARN /Volumes/mac-J/larpoux/.cargo/registry/src/index.crates.io-6f17d22bba15001f/flutter_rust_bridge_codegen-2.1.0/src/library/codegen/polisher/mod.rs:88] Fail to execute_build_runner, but continue to run.
Error details: Err(Failed to run build_runner for "/Volumes/mac-J/larpoux/projmac/tau/tau": [INFO] Generating build script...
[INFO] Generating build script completed, took 234ms

[INFO] Precompiling build script......
[INFO] Precompiling build script... completed, took 3.0s

[INFO] Initializing inputs
[INFO] Building new asset graph...
[INFO] Building new asset graph completed, took 1.8s

[INFO] Checking for unexpected pre-existing outputs....
[INFO] Deleting 9 declared outputs which already existed on disk.
)
[20.7s] Parse
  └── [20.1s] Cargo expand & syn parse

What is weird is that I can have many different errors when using flutter_rust_bridge_codegen generate.
cargo run --manifest-path ../flutter_rust_bridge/frb_codegen/Cargo.toml -- generate works better for me.

My FRB generator is : flutter_rust_bridge_codegen 2.1.0.

Ok, looks like https://github.com/fzyzcjy/flutter_rust_bridge/pull/2190 is merged but not yet released. I will release 2.2.0 probably within a day (after CI of https://github.com/fzyzcjy/flutter_rust_bridge/pull/2228 is green)

No need to hurry. I have many other things to work on ...
Take your time

2.2.0 has been released before and this should be solved :)


[swift2objc] Filtering Support

This Pull Request is to add filtering support to the swiftgen tool.

Add filtering support to swiftgen Config
Implementing filtering support into transform logic
Create and test unit tests for such filtering logic

More information can be found here: #1394

coverage: 88.691% (-0.09%) from 88.784%
when pulling 94926ae1590598db02e16b622053c7456360920a on nikeokoronkwo:swift2objc-filters
into d79f68c534b1a642838ca22bcc3fe376be6d01e6 on dart-lang:main.


Consider setting Typescript compile target  to ES6

Given the non-compliance of Typescript to es5 compilation (particularly class inheritance), it might be better to set the compile target of Typescript to ES6 as more features are already standard in ES6. Babel could then be used to further transpile down to ES5.

I'm wondering what kind of problems You have had with class inheritance? I've used superclass for views and custom elements and haven't had any problems with typescript + ES5 so far. Maybe my usecase has been different.

Object.hasOwnProperty and Object.getPrototypeOf don't work in the es5 version. aurelia uses both

Are you getting TypeScript errors and/or errors in the browser?
I found that the es6-collections typescript definition file doesn't cover everything in the aurelia-polyfill.  Try replacing it with es6-shims definition file:
typings install es6-shim --ambient --save

's not about typings. it's how they (purposefully) implemented class inheritance for es5

I think there was some issues addressed in the aurelia-polyfills module.  I remember running into some issues with IE11 a few weeks ago but they were fixed in the latest version of the polyfill
https://github.com/aurelia/polyfills/blob/master/src/object.js

hm yeah, see, fixing aurelia-polyfills is exactly what i want to avoid with this suggestion here (see https://github.com/aurelia/polyfills/issues/24)

Looks like this webpack2 branch addresses the issue
https://github.com/aurelia/skeleton-navigation/pull/371

I think we want to keep the configuration as is for now.


responsiveness

If anyone has any idea how can i make it mroe responsive can just comment here. Or can guide me how can i do it. Or they can do it themselves.

Hello, I'm kinda new to github and contribution to open source (jobs in general). I could try to help out with these issues.

Go ahead mate!!!
Your help is more than appreciated.

The paragraph and Carl Sagans' sentence are supposed to be a bit to the right of the center? Or should I change that as well?

Go ahead.
Try it. let's see


Broadcast BMS_OK, IMD_OK, and BSPD_OK status over CAN

Summary

Changelist

Testing Done

Resolved Issues

Checklist
Please change [ ] to [x] when you are ready.

[ ] I have read and followed the code conventions detailed in README.md (This will save time for both you and the reviewer!).
[ ] If this pull request is longer then 500 lines, I have provided explicit justification in the summary above explaining why I cannot break this up into multiple pull requests (Small PR's are faster and less painful for everyone involved!).

@garethellis0 Ready for review again! Reviewed the concept of "latch" entirely and updated specs.md to reflect that.


Added X-Forwarded-Tls-Client-Cert to the forward-auth headers

What does this PR do?
I've added the X-Forwarded-Tls-Client-Cert to the forward.go (ForwardAuth Middleware) if trustForwardHeader is set to true.
This allows a Forward-Auth Server to check a client certificate. (This requires mTLS on the entrypoint).
The Client Certificate is currently not passed to the forward auth server.
Motivation
Im working on a forward auth server to authenticate non-CAS Services with our CAS service and allowing the clients to use mTLS (Client Cert Authentication)
More

[x] Added/updated tests

Additional Notes
First time working with Go - im from the Javaland ;D

Is there a plan when this will get merged and released? We need this ASAP.. :/

Hi @SeaLife,
I did some tests, I didn't have any issue with chaining the PassTLSClientCert and the ForwardAuth middlewares.
We can see that the header is passed to the fake authentication server.
We will decline this PR accordingly but feel free to reopen an issue if you think we have misunderstood your situation.

Okay, we tested this on 2.0 ? and had this issue, we started our traefik with the latest image and it just works...
I dont know why. But its working now.


Accepting redirectURL and mobile=true as query params

/auth/github/callback route can now accept redirectURL=url_to_redirect_to and mobile=true
For redirecting to a particular page, redirectURL=https://realdevsquad.com query param can be passed
ex: /auth/github/callback?redirectURL=https://realdevsquad.com
And to let backend know weather the request is being made from mobile, mobile=true query param cab be passed
ex: /auth/github/callback?mobile=true

This use case has been solved by other PR


Update readme

Can you please put something in the readme? Something very small how to get start using this plugin.

Pretty much a dead project!


UP supports translation from FHIR of AOE questions

User Story:
As a receiver, I want to have ask on entry (AOE) questions in the HL7 v2 data, so the data is complete.
Description/Use Case
AOE questions are part of COVID reports. For HL7 v2, they show up as separate observations with specific LOINC codes. We need to convert those AOE questions into the proper HL7 v2 segments.
Risks/Impacts/Considerations

Need to make sure the AOE questions are part of the correct segments in the HL7 v2.
There is a possibility the location of AOE questions will change in the future, but for now we will go with what we found in #8271

Dev Notes:

See sample AOE in #8271
A list of AOE LOINC Codes and samples can be found here.
AOEs belong to a specific group of ORC/OBR observations
You will need a way to properly add the AOE questions into the HL7 v2 at the end of all other observations

Acceptance Criteria

The UP can translate AOE questions from a FHIR bundle to an HL7 v2 message.

Hey team! Please add your planning poker estimate with Zenhub @arnejduranovic @carlosfelix2 @jack-h-wang @JessicaWNava @PatriciaPerozoUSDS @thetaurean @victor-chaparro


Disney Now Unable to extract post url - DirecTV

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--list-formats', '--verbose', 'https://disneynow.com/shows/roman-to-the-rescue/season-01/episode-07-asher/vdka26623601', '--ap-mso', 'DTV', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.22000
[debug] exe versions: ffmpeg git-2020-06-28-4cfcfb3, ffprobe git-2020-06-28-4cfcfb3
[debug] Proxy map: {}
[Go] vdka26623601: Downloading JSON metadata
[Go] VDKA26623601: Downloading Provider Redirect Page
[Go] VDKA26623601: Downloading Provider Login Page
ERROR: Unable to extract post url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 815, in wrapper
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 836, in __extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 534, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\go.py", line 234, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\adobepass.py", line 1513, in _extract_mvpd_auth
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\adobepass.py", line 1365, in post_form
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 1021, in _html_search_regex
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 1012, in _search_regex
youtube_dl.utils.RegexNotFoundError: Unable to extract post url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

Free videos work fine. It's with the "premium" videos where I got issues. Right now I'm not logged in with my TV provider at youtube-dl, and it asks me for my user, and when I enter it...
G:\Downloads\Youtube\youtube-dl>youtube-dl --list-formats https://disneynow.com/shows/roman-to-the-rescue/season-01/episode-07-asher/vdka26623601 --ap-mso DTV --ap-username xxxx@yyyy --ap-password aaaa
[Go] vdka26623601: Downloading JSON metadata
[Go] VDKA26623601: Downloading Provider Redirect Page
[Go] VDKA26623601: Downloading Provider Login Page
ERROR: Unable to extract post url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

I read a very old ticket with the same issue, and apparently the problem was the TV provider didn't have support, but DirecTV was working fine.
ap-username and ap-password have been hidden, but they are correct when I enter them.
Plus: During writing this, I used a laptop where I was logged in, and it works fine, so I guess I just can't log in.
Extra: My account was flagged as soon as I posted this last week (being this my first and only repository), so it is invisible, I guess.

Please provide verbose log (-v/--verbose) to show the error backtrace:
youtube-dl --list-formats --verbose https://disneynow.com/shows/roman-to-the-rescue/season-01/episode-07-asher/vdka26623601 --ap-mso DTV --ap-username xxxx@yyyy --ap-password aaaa

Please provide verbose log (-v/--verbose) to show the error backtrace:
youtube-dl --list-formats --verbose https://disneynow.com/shows/roman-to-the-rescue/season-01/episode-07-asher/vdka26623601 --ap-mso DTV --ap-username xxxx@yyyy --ap-password aaaa

However this looks like the same issue as #29191 and this too, though the issue did not include a verbose log that would have confirmed it.
Also: #26139, #28717.

Sorry, completely misunderstood the indication. Just edited the verbose log.

There is also a suggestion that (eg) Comcast requires you to be connected via its ISP, just to make debugging even more tricky.

This is also happening in NBC sites as well. With videos that do not require login they download fine. For videos that require provider login, one also get the error of unable to extract post url. Here is an example:
[debug] Command-line config: ['--ap-mso', 'Charter_Direct', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '-o', '%(series)s - S%(season_number)sE%(episode_number)s - %(episode)s.%(ext)s', '--cookies', '/userfiles/Downloads/Firefox Downloads/cookies.txt', '-vvv', '-s', 'https://www.nbc.com/the-ark/video/every-single-person-matters/9000326419']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[392389b7d] (pip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-67-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2020.06.20, mutagen-1.45.1, pyxattr-0.7.2, secretstorage-3.3.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[NBC] Extracting URL: https://www.nbc.com/the-ark/video/every-single-person-matters/9000326419
[NBC]<PHONE_NUMBER>: Downloading JSON metadata
[NBC]<PHONE_NUMBER>: Downloading JSON metadata
[NBC]<PHONE_NUMBER>: Downloading Provider Redirect Page
ERROR: [NBC]<PHONE_NUMBER>: Unable to extract post url; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
File "/home/vagrant/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 694, in extract
ie_result = self._real_extract(url)
File "/home/vagrant/.local/lib/python3.10/site-packages/yt_dlp/extractor/nbc.py", line 204, in _real_extract
query['auth'] = self._extract_mvpd_auth(
File "/home/vagrant/.local/lib/python3.10/site-packages/yt_dlp/extractor/adobepass.py", line 1707, in _extract_mvpd_auth
provider_login_page_res = post_form(
File "/home/vagrant/.local/lib/python3.10/site-packages/yt_dlp/extractor/adobepass.py", line 1395, in post_form
post_url = self._html_search_regex(r'<form[^>]+action=(["'])(?P.+?)\1', form_page, 'post url', group='url')
File "/home/vagrant/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1278, in _html_search_regex
res = self._search_regex(pattern, string, name, default, fatal, flags, group)
File "/home/vagrant/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1242, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)


ClusterBuildTemplate variations of each BuildTemplate file

$ kubectl apply -f buildpack/buildpack-cluster.yaml
clusterbuildtemplate.build.knative.dev/buildpack created
$ kubectl apply -f buildpack/buildpack.yaml
buildtemplate.build.knative.dev/buildpack created

[x] Includes ./hack/generate-cluster-templates.sh to regenerate *-cluster.yaml files from the kind: BuildTemplate versions.
[x] includes test/build-buildpack-cluster.yaml

/ok-to-test

Was there a reason to not merge this last year? Want me to rebase?

@drnic I don't think so cc @ImJasonH

I'm not sure how I so thoroughly missed this PR! :astonished:
I'm slightly opposed to having to maintain two copies of each build template going forward, it seems like a recipe for them getting out of sync in confusing and unobvious ways, or otherwise like a speed bump to avoid in every future PR that gets proposed.
Perhaps if there was a CI test that checked that they were identical, and a script that could be run to copy-and-update the ClusterBuildTemplate version, that would help a lot.
And in any case I think if we're going to invest in improving this catalog of build templates, we should consider making the same change instead of / in addition to the Tekton Task catalog at https://github.com/tektoncd/catalog.
Up to you if you want to move this forward, I don't feel strongly that we really need it. Maybe just a blurb somewhere that any namespaced BuildTemplate can also be installed as a ClusterBuildTemplate by changing the kind.
WDYT?

Ok, I'll close the issue for now. Someone/myself can reopen in future.


Failed to disassociate GitLab group error appears when any project is enforced to cla with 400 bad request.

Steps :

Login Admin portal
Open easyCla GitLab repository page
Click on connected GitLab group
Enforced Cla to project
Click on additional settings
Click on "Disassociate GitLab group" button
Click on Disassociate at confirmation prompt

Actual Results : "Failed to disassociate GitLab group. Please disable DCO on all GitLab projects before disassociating GitLab group." error toast appears & 400 bad request appears at console
{status: 400, stack: "", details: null,…}
code: "400"
details: null
message: "EasyCLA - 400 Bad Request - problem deleting Gitlab Group with project SFID: a092h000003hQBPAA2 with path: veergitlabdemo - error: the following repositories are still enabled under the GitLab Group/Organization: veergitlabdemo - veergitlabdemo/veerdemogitlabproject02,veergitlabdemo/veergitlabdemoproject01"
requestId: "0f745870-0a2a-11ec-ae36-a9d16a83182d"
stack: ""
status: 400

https://images.zenhubusercontent.com/194341141/076c94af-4fad-4854-a088-306a3e5ffb97/failed_to_disassociate_gitlab_group_.mp4

Added PCC ticket for this case, So making issue status QA-Done
Refer Ticket id : PCC-1611


python3packages.stytra: init at 0.8.26

WIP - wanted to get feedback on help as this is my first time packing using wheel.
It seems like propogatedBuildInputs is not behaving the same way--if I only have a package here I get failures like:
ERROR: Could not find a version that satisfies the requirement pyqtgraph>=0.10.0 (from stytra==0.8.26) (from versions: none)
ERROR: No matching distribution found for pyqtgraph>=0.10.0 (from stytra==0.8.26)
builder for '/nix/store/brscmbwx2nzr56f8wdh3n8vnjq2ih58p-python3.7-stytra-0.8.26.drv' failed with exit code 1

As a result, I'm including in both buildInputs, and propogatedBuildInputs, which feels wrong. It does build, but when I try open a shell the following happens:
> python
>>> import stytra
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/nix/store/prr2jha833ix3qa30w0cbfgcr1zxh75s-python3-3.7.5-env/lib/python3.7/site-packages/stytra/__init__.py", line 3, in <module>
    from stytra.experiments import VisualExperiment
  File "/nix/store/prr2jha833ix3qa30w0cbfgcr1zxh75s-python3-3.7.5-env/lib/python3.7/site-packages/stytra/experiments/__init__.py", line 5, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'

This seems very odd, as in any other derivation I test, I can import from propogatedBuildInputs in python shell. To debug, I switched back to using the tar.gz file, and found an unexpected error:
these derivations will be built:
  /nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv
building '/nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv'...
collision between `/nix/store/10b0xgqaqacaxiwlvcklp9nbl20y5wcx-python3.7-sip-4.19.18/bin/sip' and `/nix/store/nqa4rbzhi6aj0ry54dwc82r2900zlnmr-python3.7-PyQt5.sip-4.19.18/bin/sip'
builder for '/nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv' failed with exit code 255
error: build of '/nix/store/25sy04zv5h91a02nkbdxy5lkf73ibn33-python3-3.7.5-env.drv' failed

Any thoughts or tips? Thanks!
Motivation for this change
Add module for closed-loop experiments for larval zebrafish.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

I don't see you declaring the pyqtgraph package in the propagated build inputs....
python packages should go into propagatedBuildInputs

It's there (via commonInputs). If pyqtgraph is only in propogatedBuildInputs, I get the error above. It goes away if I add to buildInputs, too, as is currently done in the code

This is blocked until https://github.com/NixOS/nixpkgs/pull/72378 is merged, but reviews in the meantime are welcome!

Thanks @jonringer. This is ready for final review

@jonringer done


Collection does not sort itself when elements change

When I change a property defined by the comparator the collection does not sort itself. Is this intended?

No that is not intended, could you give an example of this?

If I'm reading this right, I think what you want is https://github.com/AmpersandJS/ampersand-filtered-subcollection
With ampersand-collection the behavior is this:
var Collection = require('ampersand-collection');
var C = Collection.extend({comparator: 'name'});
var c = new C([{name: 'z'}, {name: 'c'}, {name: 'b'}])

console.log(c.at(0).name, c.at(1).name, c.at(2).name);
// b c z

c.at(2).name = 'a';
console.log(c.at(0).name, c.at(1).name, c.at(2).name);
// b c a
// im assuming you want it to be `a b c`

Oh right, the collection has no way to know if attributes in an object change.  So you either set the object wholly, or call sort, I think.

What if we listened to the change:comparator event when the comparator was a string. I think it would sense for a collection with a comparator to keep itself sorted. If you'd like I can make a pull request.
Ampersand-filtered-subcollection is nice and I use it but I don't want to make a subcollection out of the entire collection for the sake of sorting. Right now I added a change listener to the collection and it works.


shiny doesn't work along with the latest ipython version

Package Compatibility Issue: ipython and shiny with Poetry
Summary
When using Poetry to manage a Python project that includes both ipython and shiny, a version conflict occurs due to incompatible dependencies on prompt-toolkit. This issue prevents the installation of both packages in their latest versions.
Error Message
Because no versions of questionary match >2.0.0,<2.0.1 || >2.0.1
 and questionary (2.0.0) depends on prompt_toolkit (>=2.0,<=3.0.36), questionary (>=2.0.0,<2.0.1 || >2.0.1) requires prompt_toolkit (>=2.0,<=3.0.36).
And because questionary (2.0.1) depends on prompt_toolkit (>=2.0,<=3.0.36), questionary (>=2.0.0) requires prompt_toolkit (>=2.0,<=3.0.36).
Because no versions of shiny match >1.0.0,<2.0.0
 and shiny (1.0.0) depends on questionary (>=2.0.0), shiny (>=1.0.0,<2.0.0) requires questionary (>=2.0.0).
Thus, shiny (>=1.0.0,<2.0.0) requires prompt_toolkit (>=2.0,<=3.0.36).
And because ipython (8.26.0) depends on prompt-toolkit (>=3.0.41,<3.1.0)
 and no versions of ipython match >8.26.0,<9.0.0, shiny (>=1.0.0,<2.0.0) is incompatible with ipython (>=8.26.0,<9.0.0).
So, because polarstar depends on both shiny (^1.0.0) and ipython (^8.26.0), version solving failed.

Steps to Reproduce

Create a new directory (e.g., test)
Initialize a new Poetry project:poetry init

Add ipython to the project:poetry add ipython

Attempt to add shiny to the project:poetry add shiny

Observe the error message above

Root Cause
The conflict arises from incompatible version requirements for prompt-toolkit:

shiny depends on questionary, which requires prompt-toolkit (>=2.0,<=3.0.36)
ipython (version 8.26.0) requires prompt-toolkit (>=3.0.41,<3.1.0)

These requirements cannot be satisfied simultaneously, causing the dependency resolution to fail.
Workaround
Manually specify a lower version of ipython that is compatible with the required version of prompt-toolkit. For example:
poetry add<EMAIL_ADDRESS>
Then add shiny:
poetry add shiny

Proposed Solutions

Update shiny or its dependencies to support newer versions of prompt-toolkit.
Investigate if ipython can be made compatible with a wider range of prompt-toolkit versions.
Consider creating a compatibility layer or shim to allow these packages to coexist.

Interesting, shiny requires questionary>=2.0.0, but poetry seems to be limiting shiny to questionary<=2.0.0 rather than installing the latest version.
Can you try installing questionary v2.0.1, which requires prompt_toolkit = ">=2.0,<4.0"?

Oh wait, I was looking at questionary's GitHub repo which has updates but is still at 2.0.1. It looks like questionary plans to push a new release soon that will remove the prompt_toolkit restriction.


Security assessment via CNCF sig-security

What would you like to be added:
A plan/milestone for a security assessment with the SIG, so we can schedule time with the SIG.
Why is this needed:

We still want to do this

/close
closing this in favor of #886

@fntlnz: Closing this issue.

In response to this:

/close
closing this in favor of #886

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

The assessment is different from an audit -- I talked to @mfdii who said he would talk to @kris-nova about kicking this off

@ultrasaurus @kris-nova we need to open this again then.
/reopen

@fntlnz: Reopened this issue.

In response to this:

@ultrasaurus @kris-nova we need to open this again then.
/reopen

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Looks like there is some activity on this in the sig-security repo.
Linking https://github.com/cncf/sig-security/issues/273

/milestone 1.0.0


update URL after mv to euroargodev org

Thanks @giordano for merging
One aspect that no longer seems to work with this repo is releasing a new version -- https://github.com/euroargodev/ArgoData.jl/issues/37 does not trigger the bot / the automated PR to here.
I am looking for way to fix or pointers on the subject. Maybe I overlooked something.

Is it enabled in the new repository/organisation?

Is it enabled in the new repository/organisation?

Right! It wasn't enabled for the org. Works now. Thank you again @giordano


avn fails when .nvmrc contains additional lines

When .nvmrc contains more than 1 line, e.g. with comments, then avn fails.
This is a problem because a code base I have to work with at work has extended .nvmrc with some additional lines for specifying the yarn version, which is used by a different tool. These extra lines are ignored by nvm, but cause an error in avn.
The simplest fix would be to only read the first line of the file and ignore everything else that follows.
i.e. In lib/hooks.js, change this line:
.then(function(version) { this.version = version.trim(); })

to this:
.then(function(version) { this.version = version.split('\n')[0].trim(); })

Details

avn 0.2.3
node 6.9.4
nvm 0.33.1
bash 4.4.12

The output of __avn_debug in the directory with a .node-version file is:
avn could not activate node 6.9.4
# test
error: no plugin passed predicate
  avn-nvm: no version matching 6.9.4
# test
  avn-n: no version matching 6.9.4
# test

avn is loaded in my ~/.{bash|zsh}{_profile|rc} file with:
[[ -s "$HOME/.avn/bin/avn.sh" ]] && source "$HOME/.avn/bin/avn.sh" # load avn

nvm specific

[ ] As an nvm user I am confirming that I did not install with Homebrew

Yes.

@lachlanhunt as the maintainer of nvm, I absolutely intend to break your use case in the future. Please do not use nvm's config file for anything that's not part of nvm.
However, this is still an issue with avn; avn should not be relying on the file only ever having one line in it.

I agree it shouldn't be abused for things unrelated to node versions. Unfortunately, that code is maintained by a different team and I'm not entirely sure . I'll follow up with them though to see if they'll agree to change it.
But at the very least, ignoring lines beginning with # and treating them as comments would help.

I think this is something that will be somewhat difficult to fix "correctly".
Here are a few related issues that may further the discussion here:

#43
#53
#17

Here are some options to consider:

Remove support for .nvmrc files since it has caused many issues.
Use an nvm call to read the correct info (or even find config files based on the current directory) and get back a version. Perhaps this sort of call already exists. If not, we'd need to first add something to nvm.

As I haven't been using nvm heavily, I'd very much appreciate any PRs related to this.

nvm_rc_version will set the $NVM_RC_VERSION environment variable if an nvmrc file is found.

@ljharb does that traverse from CWD?

yes

@ljharb thanks. So avn should be able to use that to read the file rather than making any assumptions. @lachlanhunt if you want to take a stab at that a PR would be more than welcome!


Add random affine transformation

Random affine transformations with tests.
For usage, see sanity_check.ipynb

Any other feedbacks on the PR is highly appreciated :)

@fmassa A precision on matching between transformed points. Take look at the example with scale.
Here the input image:

which is then transformed with
result = F.affine(img, angle=0, translate=(0, 0), scale=2.0, shear=0.0)

However, manually-verified points gives :
xs = []
ys = []
true_image = np.zeros((100, 100, 3), dtype=np.uint8)
for y, x in pts:
    xs.append(int((x - cnt[0]) * s + cnt[0] + 0.5))
    ys.append(int((y - cnt[1]) * s + cnt[1] + 0.5))
    true_image[ys[-1], xs[-1], :] = (255, 255, 255)

what is correct as input points are transformed to the output and not as an inverse mapping (as in PIL).
So, I think if we keep these tests, we need to match only manually-verified set in the non-null locations of affine transformed image.
What is your opinion ?

I refactored a little the code and tests such that I compare transformation matrices and images with accepted error of max N(=3) different pixels (cause of aliasing). With such approach, tests take about 3 mins to execute because of rather bruteforce tests:
for a in range(-90, 90, 25):
    for t1 in range(-10, 10, 5):
        for s in [0.75, 0.98, 1.0, 1.1, 1.2]:
              for sh in range(-15, 15, 5):
                   _test_transformation(a=a, t=(t1, t1), s=s, sh=sh)

Thoughts @fmassa ?

@fmassa can we merge this ?

thanks @vfdev-5


Add ability to logout and thus turn a refresh token into a machine auth

For WebBrowser logins, machine auths are only issued when you logout.

Hi!
i work on it on my side too.
if i have some result i will post here.
The big problem is that we need to wait the cookie expiration to renew it... It's take sooooo long to test...

Hi!
Here some news, i'm stuck on the first redirection, i need to send 2 new tokens : steamRefresh_steam & steamMachineAuth.
steamRefresh_steam it's ok we have the infos to create it, but steamMachineAuth, i don't know how to make it.
When i decode them, they are very similar.


Doesn't work

This library simply doesn't work and the documentation for it is so poor, making it unusable. Please fix the ReadMe. Getting - "Please provide ShowCaseView context" for no reason.

Hi,
Can you share your code snippet and error log so that we can help you out to solve your problem.
And sure we will look after your recommendation to update whole documentation of package.

@DevarshRanpara Getting Please provide ShowCaseView context error

@akshatjagga @rajeevjaiswal
By mistake you are passing wrong context to ShowCaseWidget.of(context).startShowCase([_one])  method.
You need to wrap your Scaffold with ShowCaseWidget  and pass Builder context of ShowCaseWidget.
Please refer this file for batter understanding.


Trying to use Cuda crashes the JVM

Using the current version and this minimal example the JVM crashes for me:
package com.example;

import org.nd4j.linalg.factory.Nd4j;

public class App
{
    public static void main( String[] args ) throws InterruptedException {
        Nd4j.ones(1);
    }
}

hs_err_pid8524.log.txt
hs_err_pid8556.log.txt

Still now working for me :cry:
But it looks like the stackframe has changed. See the attached logs.
hs_err_pid2916.log.txt
hs_err_pid7436.log.txt

It also crashes in a similar way here on Linux (Fedora 23) with just a Nd4j.ones(1), so it's probably not related to Windows...
hs_err_pid14814.log.txt

With the most current commits by @raver119 on nd4j (branch jita_cuda_1_1) and here on branch scalar, this crash is fixed.


Fix: check permissions before getting the address from a wallet browser extension

Hmmm -- it should actually do neither of these. It is not the library's job to respond to this, it should be done on the wallet side. This ensures that different wallets can implement the correct behaviour for their situation.

I think that it would be insecure if we'd provide a function in the wallet api that does both of these


Polish

adds generics
removes deprecated package
removes redundant groupId and version  from maven which is managed by parent POM
uses Static Instead of reference
removes unused logger

@rajadilipkolli,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by Microsoft. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot


Update libraries and JSX for current versions 1/23/2015

Required a few JSX changes.
Don't use import for commonjs modules
key changed to eventKey for NavItem
Use different names for the the native input HTML elements compared to
the react-bootstrap ones
Could not update the sass-loader without errors
Moved all Sass related parts of webpack to the webpack.hot.config.js
as Rails version does not use the Webpack sass handlers.
Changed webpack.rails.config.js so that the loader for JSX does not
use react-hot
Added the woff2 pattern for the file loader.
Changed from React.renderComponent to React.render in example.jsx.

@mbreining @kimagure Any comments? I'll try to get the sass-loader updated next.

You can import CommonJS modules easily, just requires that you do import * as YOURNAME from 'module'
I admit it's not the best thing ever, but it'll work enough.

@justin808, the code in update-to-latest is deploying to Heroku fine but is not functional (none of the forms are showing up). The same issue exist on master. I'll have to look into it.

You can merge as far as I'm concerned.

@mbreining I rebased master onto my branch and did a push -f.


v0.25.0 RC Post key by sending only the date part YYYY-MM-DD failed

Describe the bug
Post a new key by sending only the date part for expiresAt field failed.
According to the API Keys specification and the API response, the expiresAt value can be defined only with the date such as: YYYY-MM-DD.

field
type
required
description

expiresAt
string
Required
The expiration date and time as ISO-8601 format. null equals to no expiration time. Sending only the date part e.g 2021-12-01 leads to having an expiresAt value set to 2021-12-01T00:00:00. Default: No Default

To Reproduce
Post on /keys route:
{
    "description": "Indexing Products API key",
    "actions": [
        "documents.add"
    ],
    "indexes": ["products"],
    "expiresAt": "2022-01-01"
}

Response:
{
	"message": "expiresAt field value `\"2022-01-01\"` is invalid. It should be in ISO-8601 format to represents a date or datetime in the future or specified as a null value. e.g. 'YYYY-MM-DD' or 'YYYY-MM-DDTHH:MM:SS'.",
	"code": "invalid_api_key_expires_at",
	"type": "invalid_request",
	"link": "https://docs.meilisearch.com/errors#invalid_api_key_expires_at"
}

Expected behavior
The expiredAt must be able to be defined by a date format or the API message must be modified.
MeiliSearch version: v0.25.0rc1

Hey @alallema,
It is indeed an issue on our side, as we must accept ISO-8601 date format we should accept the YYYY-MM-DD format and it doesn't seem to be the case. However, it seems like the chrono library that we use accepts RFC3339 instead, you can see the difference on this website.
I would bet that we always try to parse the string as a chrono DateTime even if there is no time given in the string therefore the chrono library tells us that it is too short and it is right. We should have tried to parse the string as a NaiveDate when the first fully-qualified string was parsed and failed, this way we can accept the YYYY-MM-DD format and specify that the time is in fact midnight and the offset is +0 (UTC). I have done some tests on the Rust playground and it works.

Fixed by #2026


feat: 5 Education Quotes added

PULL REQUEST TEMPLATE
👨💻 What did you do ?
I have added 5 unique Education Quotes added in the last of the Quotes.js file all according to instructions provided by you in Contributing.md file.
Note to reviewers
📷 Screenshots

Not an UI issue.
✔ Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title and description of the PR is clear and explains the approach.
[x] I have read the contribution guideline properly and following the criteria.

thanks for the contribution .
please do star the project if you like it.
merged.


Adds support for Mermaid diagrams in ADRs

Create a codeblock with mermaid as the language tag and it'll render.
See the new test ADR for an example.

This looks great! One comment on the versioning of the mermaid JS import.

Ok, looks like the changes are rather large, so I'll have to set aside some time to look at it more closely at some point in the next week or so.

No worries at all - any contributions are appreciated. After you've landed it, I'll kick up RC2 with this and the latest changes I've made, then if it all looks good I'll release 1.5.0 soon after that.

I think this is ready to go now. GH is saying This branch cannot be rebased due to conflicts but upstream/master is fully merged into this branch…

Yeah, this is rebase problems from my model refactoring.
I've switched the project to allow merge commits temporarily, which GitHub is happy with (I can see the big green "Merge pull request" button), and will fold it in.
Thank you for the change!

Thank you!


[fix] [broker] fix use error TimeUnit to record publish latency

Motivation
recordAddLatency use nanoTime and TimeUnit.MICROSECONDS by mistake
Modifications
change to TimeUnit.NANOSECONDS
Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[ ] Dependencies (add or upgrade a dependency)
[ ] The public API
[ ] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository: https://github.com/TakaHiR07/pulsar/pull/7

Merging...
Thanks for your contribution @TakaHiR07!


[SU-07] copy button component

Changes Made 🎉

[X] feat(icons): 🍱 update icon component with a copy icon
[X] feat(buttons): ✨ Add Copy Button component and related files

Describe Changes
The pull request adds a new Copy Button component to the side-ui component library. The Copy Button component allows users to easily copy text content to the clipboard, enhancing the user experience. It includes the Copy.tsx file with the component implementation, the Copy.stories.ts file for storybook documentation, and an update to the index.ts file to export the Copy Button component.
Related Issue(s)

#7

Visuals (Optional)

Checklist ✅

[X] My code follows the code style of this project.
[X] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

Termine los cambios @serudda, tengo una dudas con el storybook no pude desactivar el htmlType y el children del storybook.
Tampoco pude darle el satisfied para el storybook me da problema con el target pero no encontre una solucion. Igual le echo un ojo mañana

Le dejo el preview

This POR was implemented in another branch.


npm install -g next?

Following the readme example, it indicates to install next with npm install next --save. I couldnt get the example to work this way.
zsh: command not found: next
Shouldn't this be npm install -g next so that the next command is global?

@purplecones I just added a script to package.json like so:
"scripts": {
    "start": "next"
  },

and then ran npm start

@purplecones usually you use npm scripts. Npm knows how to locate next inside the package. It's recommended to install it locally mostly because you can have different versions in different projects
{
  "name": "my-app",
  "dependencies": {
    "next": "*"
  },
  "scripts": {
    "dev": "next",
    "build": "next build",
    "start": "next start"
  }
}

@davemo @impronunciable oh ok. Any reason why this CLI from zeit should be used this way? now and ns that I use are all global.

@purplecones with now you are interfacing the now API and you'll want to be always at the last version. Is not the case of a front end framework

next init probably better work and sense for global installations

@impronunciable @fabricionaweb thanks


Reference rename

I found 2 places in this PR that needs renaming. Please make a global search - there might be other places as well...

The name doesn't have to be escaped any more I guess - but no harm done...

I would say we should remove the quotes because it can cause issues.  By default, the database case normalizes unquoted input and accordingly the "public wisdom" is to either always quote or never quote.  Except for "refereces" we have never quoted, so I think we should standardize on "not quoting" now that we have the option (this implies that we should not quote the name in the SQL either in case we did that).

The name doesn't have to be escaped any more I guess - but no harm done...

I would say we should remove the quotes because it can cause issues. By default, the database case normalizes unquoted input and accordingly the "public wisdom" is to either always quote or never quote. Except for "refereces" we have never quoted, so I think we should standardize on "not quoting" now that we have the option (this implies that we should not quote the name in the SQL either in case we did that).

@nt-as I have removed the quotations.


Mixing properties and patternProperties fails for properties not matching the patternProperties definition

Schema (incomplete, just the important part):
"link": {
  {
    "type": "object",
    "properties": {
      "controller": {
        "type": "string",
        "pattern": "^[A-Z][a-zA-Z0-9]+$"
      },
      "action": {
        "type": "string",
        "pattern": "^[a-z][a-z0-9-]+$"
      }
    },
    "patternProperties": {
      "^[a-z][a-z0-9-]+$": {"pattern": "^[a-z0-9-]+$"}
    },
    "required": ["controller", "action"],
    "additionalProperties": false
  }
}

Data:
"link": {
  "controller": "Dashboard",
  "action": "overview",
  "param1": 123,
  "param2": "foobar"
}

Validating this data will result in an error:
Violations: [link.controller] Does not match the regex pattern ^[a-z0-9-]+$
Maybe I am the problem here, but I understand the "patternProperties" to only require validation for properties that aren't explicitly defined.
Since "controller" already is set, no validation with the regex pattern in patternProperties should occur.
I already found a fix, but I'd like to wait with a PR until someone can tell me whether this is, indeed, a bug or just my misunderstanding of how properties & patternProperties work.

This isn't a bug; it's how the spec says it should behave 🙂

Maybe I am the problem here, but I understand the "patternProperties" to only require validation for properties that aren't explicitly defined.

Your understanding is incorrect. According to the spec, patternProperties applies to all matching properties, and does not exclude those which also match a properties rule. The two keywords operate entirely independently of each other.
The relevant spec wording is:

6.19 patternProperties
The value of "patternProperties" MUST be an object. Each property name of this object SHOULD be a valid regular expression, according to the ECMA 262 regular expression dialect. Each property value of this object MUST be a valid JSON Schema.
This keyword determines how child instances validate for objects, and does not directly validate the immediate instance itself. Validation of the primitive instance type against this keyword always succeeds.
Validation succeeds if, for each instance name that matches any regular expressions that appear as a property name in this keyword's value, the child instance for that name successfully validates against each schema that corresponds to a matching regular expression.
Omitting this keyword has the same behavior as an empty object.

A contrasting example is additionalProperties, which does contain such an exclusion:

Validation with "additionalProperties" applies only to the child values of instance names that do not match any names in "properties", and do not match any regular expression in "patternProperties".

If you are wanting patternProperties to apply only to properties which do not pass validation against properties, the spec-compliant way to do this is with oneOf. For example:
{
    "type": "object",
    "oneOf": [
        {
            "properties": {...},
            "additionalProperties": false
        {,
        {
            "patternProperties": {...}
        }
    ]
}

Note that additionalProperties is necessary in order to ensure that non-matching property names fail validation on the properties branch of the oneOf constraint.

Thanks for the quick reply and clarification. It seems like PHPStorm's implementation is wrong.
Sadly, I cannot make this work using anyOf like this results - it results in errors as well:
"title": {
  "type": "object",
  "anyOf": [
    {
      "properties": {
        "controller": {
          "type": "string",
          "pattern": "^[A-Z][a-zA-Z0-9]+$"
        },
        "action": {
          "type": "string",
          "pattern": "^[a-z][a-z0-9-]+$"
        }
      },
      "additionalProperties": false
    },
    {
      "patternProperties": {
        "^[a-z][a-z0-9-]+$": {"pattern": "^[a-z0-9-]+$"}
      }
    }
  ]
}

[link] The property param1 is not defined and the definition does not allow additional properties
[link] The property param2 is not defined and the definition does not allow additional properties
[link.controller] Does not match the regex pattern ^[a-z0-9-]+$
[link] Failed to match at least one schema

I was able to make it work using a negative lookahead in the patternProperties:
^((?!^(controller|action)))[a-z][a-z0-9-]+$
Thanks again and sorry for any inconvenience.


Could you add some good tests for this SDK?

Hi Guys,
Right now wepay-rails uses HTTParty for communications with the wepay api. I'd like to change that and use your SDK instead. We are starting a major testing effort on the wepay-rails gem and I was hoping that you guys could write some around your SDK that can call your service in a real way vs. stubbing out some responses. This would help us to ensure that our code is good as well.
Thanks in advance,
Adam (wepay-rails)

It's taken way, way, way too long, but they're here in master. Will go out as part of 0.3.0.


Add Project Auxiliator to the list of projects.

Add the project that I hacked up at the "Hack the North" hackathon.
Project Auxiliator is a really cool personal assistant with many integrations across home automation, smartphone functionality and smartwatch functionality.

Added.


Xero Projects

When submitting a connector, please check the following conditions for your PR.

[X] I attest that the connector works and I verified by deploying and testing all the operations.
[X] I validated the swagger file, apiDefinition.swagger.json, by running paconn validate command.
[X] If this is a certified connector, I confirm that apiProperties.json has a valid brand color and doesn't use an invalid brand color, #007ee5 or #ffffff. If this is an independent publisher connector, I confirm that I am not submitting a logo.

[[certify-connector]]


Prefer k8s Node IP for SNAT IP (IPV{4,6}_NODEPORT) when multiple IP addrs are set to BPF NodePort device

Bug report
When Cilium is configured as a kube-proxy replacement, it fails to masquerade the source IP of pods when the target pod is in the hostNetwork of one of the k8s nodes.
General Information

Cilium version

Client: 1.8.2 aa42034f0 2020-07-23T15:02:39-07:00 go version go1.14.6 linux/amd64
Daemon: 1.8.2 aa42034f0 2020-07-23T15:02:39-07:00 go version go1.14.6 linux/amd64

Kernel version

Linux primary-external-workers-uksouth-machine-node-0 5.4.0-1022-azure #22-Ubuntu SMP Fri Jul 10 06:14:37 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Orchestration system version in use

$ kubectl version

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:12:48Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:04:18Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

How to reproduce the issue

Using kubespray (or manually with kubeadm) build a cluster on Azure with cloud-provider disabled. API-server pods will be running as static pods in host OS namespace on 3 controller nodes.

k describe svc kubernetes
Name:              kubernetes
Namespace:         default
Labels:            component=apiserver
                   provider=kubernetes
Annotations:       <none>
Selector:          <none>
Type:              ClusterIP
IP:                <IP_ADDRESS>
Port:              https  443/TCP
TargetPort:        6443/TCP
Endpoints:         <IP_ADDRESS>:6443,<IP_ADDRESS>:6443,<IP_ADDRESS>:6443
Session Affinity:  None
Events:            <none>

Deploy a test pod on any worker node and attempt to connect to the API server clusterIP. This results in timeouts

bash-5.0# nc -zv -w 5 <IP_ADDRESS> 443
nc: connect to <IP_ADDRESS> port 443 (tcp) timed out: Operation in progress

Additional details
I have a test pod (<IP_ADDRESS>) trying to connect to the API server on <IP_ADDRESS>:443. When doing a tcpdump on the underlying node, I cannot see SNAT'ed packets.
root@primary-external-workers-uksouth-machine-node-0:~# tcpdump -i any -n 'host <IP_ADDRESS> or host <IP_ADDRESS> or <IP_ADDRESS>'
14:01:30.463349 IP <IP_ADDRESS>.35832 > <IP_ADDRESS>.6443: Flags [S], seq<PHONE_NUMBER>, win 64860, options [mss 1410,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
14:01:34.687357 IP <IP_ADDRESS>.35832 > <IP_ADDRESS>.6443: Flags [S], seq<PHONE_NUMBER>, win 64860, options [mss 1410,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0

Since the source IP of the pod is not masqueraded, it gets dropped by Azure's networking stack.
According to @brb's comment it should have been translated on TC egress, however I don't see it happening (see above tcpdump).
As soon as I change enable-bpf-masquerade to "false" and restart the agent on the node, connectivity gets restored.
I've got the environment up and running for a while so I'm happy to collect any additional logs/outputs.
Here's the cilium configmap.
apiVersion: v1
kind: ConfigMap
metadata:
  name: cilium-config
  namespace: kube-system
data:
  # Identity allocation mode selects how identities are shared between cilium
  # nodes by setting how they are stored. The options are "crd" or "kvstore".
  # - "crd" stores identities in kubernetes as CRDs (custom resource definition).
  #   These can be queried with:
  #     kubectl get ciliumid
  # - "kvstore" stores identities in a kvstore, etcd or consul, that is
  #   configured below. Cilium versions before 1.6 supported only the kvstore
  #   backend. Upgrades from these older cilium versions should continue using
  #   the kvstore by commenting out the identity-allocation-mode below, or
  #   setting it to "kvstore".
  identity-allocation-mode: crd

  # If you want to run cilium in debug mode change this value to true
  debug: "false"

  # Enable IPv4 addressing. If enabled, all endpoints are allocated an IPv4
  # address.
  enable-ipv4: "true"

  # Enable IPv6 addressing. If enabled, all endpoints are allocated an IPv6
  # address.
  enable-ipv6: "false"
  enable-bpf-clock-probe: "true"

  # If you want cilium monitor to aggregate tracing for packets, set this level
  # to "low", "medium", or "maximum". The higher the level, the less packets
  # that will be seen in monitor output.
  monitor-aggregation: medium

  # The monitor aggregation interval governs the typical time between monitor
  # notification events for each allowed connection.
  #
  # Only effective when monitor aggregation is set to "medium" or higher.
  monitor-aggregation-interval: 5s

  # The monitor aggregation flags determine which TCP flags which, upon the
  # first observation, cause monitor notifications to be generated.
  #
  # Only effective when monitor aggregation is set to "medium" or higher.
  monitor-aggregation-flags: all
  # bpf-policy-map-max specified the maximum number of entries in endpoint
  # policy map (per endpoint)
  bpf-policy-map-max: "16384"
  # Specifies the ratio (0.0-1.0) of total system memory to use for dynamic
  # sizing of the TCP CT, non-TCP CT, NAT and policy BPF maps.
  bpf-map-dynamic-size-ratio: "0.0025"

  # Pre-allocation of map entries allows per-packet latency to be reduced, at
  # the expense of up-front memory allocation for the entries in the maps. The
  # default value below will minimize memory usage in the default installation;
  # users who are sensitive to latency may consider setting this to "true".
  #
  # This option was introduced in Cilium 1.4. Cilium 1.3 and earlier ignore
  # this option and behave as though it is set to "true".
  #
  # If this value is modified, then during the next Cilium startup the restore
  # of existing endpoints and tracking of ongoing connections may be disrupted.
  # This may lead to policy drops or a change in loadbalancing decisions for a
  # connection for some time. Endpoints may need to be recreated to restore
  # connectivity.
  #
  # If this option is set to "false" during an upgrade from 1.3 or earlier to
  # 1.4 or later, then it may cause one-time disruptions during the upgrade.
  preallocate-bpf-maps: "false"

  # Regular expression matching compatible Istio sidecar istio-proxy
  # container image names
  sidecar-istio-proxy-image: "cilium/istio_proxy"

  # Encapsulation mode for communication between nodes
  # Possible values:
  #   - disabled
  #   - vxlan (default)
  #   - geneve
  tunnel: vxlan

  # Name of the cluster. Only relevant when building a mesh of clusters.
  cluster-name: default

  # wait-bpf-mount makes init container wait until bpf filesystem is mounted
  wait-bpf-mount: "false"

  masquerade: "true"
  # (Michael) workaround for https://github.com/cilium/cilium/issues/12699
  enable-bpf-masquerade: "false"
  enable-xt-socket-fallback: "true"
  install-iptables-rules: "true"
  auto-direct-node-routes: "false"
  kube-proxy-replacement:  "strict"
  enable-health-check-nodeport: "true"
  node-port-bind-protection: "true"
  enable-auto-protect-node-port-range: "true"
  enable-session-affinity: "true"
  enable-endpoint-health-checking: "true"
  enable-well-known-identities: "false"
  enable-remote-node-identity: "true"
  operator-api-serve-addr: "<IP_ADDRESS>:9234"
  # Enable Hubble gRPC service.
  enable-hubble: "true"
  # UNIX domain socket for Hubble server to listen to.
  hubble-socket-path:  "/var/run/cilium/hubble.sock"
  # An additional address for Hubble server to listen to (e.g. ":4244").
  hubble-listen-address: ""
  ipam: "kubernetes"
  disable-cnp-status-updates: "true"

  # (Michael) workaround for https://github.com/cilium/cilium/issues/10627
  blacklist-conflicting-routes: "false"

CC @brb

@networkop Thanks for the issue. Two things - could you run cilium bpf ipcache get <IP_ADDRESS> from the node which runs the client pod, and could you provide a sysdump (https://docs.cilium.io/en/v1.8/troubleshooting/#automatic-log-state-collection)?

sure, here's the bpf ipcache output first
$ cilium bpf ipcache get <IP_ADDRESS>
<IP_ADDRESS> maps to identity 6 0 <IP_ADDRESS>

@brb where can I upload the sysdump?

oh wow, didn't know about GH's drag-and-drop cilium-sysdump-20200827-160052.zip

@networkop Could you try w/o tunneling? You can do that by setting auto-direct-node-routes: true and tunnel: disabled in the ConfigMap, and then restarting all cilium-agent pods?

ok, l'll try. do you want me to collect any logs or just verify that it works?

Thanks. Just to verify that it works.

I've done the test, however I still don't have the e2e connectivity. In addition to changing the above flags I also had to specify the native-routing-cidr to stop the agent from crashing. I set it to the same value as pod cidr (in my case <IP_ADDRESS>/18).
Additionally, in Azure I enabled "ip forwarding" on the NIC of the worker node I was testing, to stop it from dropping unknown packets.
The only new things I've noticed was that traceroute now looks a bit more like what you'd expected in the beginning:
15:14:57.811789 IP <IP_ADDRESS>.50440 > <IP_ADDRESS>.6443: Flags [S], seq<PHONE_NUMBER>, win 65280, options [mss 1360,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
15:14:59.162251 IP <IP_ADDRESS>.54334 > <IP_ADDRESS>.6443: Flags [S], seq<PHONE_NUMBER>, win 64240, options [mss 1460,sackOK,TS val 560436112 ecr 0,nop,wscale 7], length 0
15:14:59.163323 IP <IP_ADDRESS>.6443 > <IP_ADDRESS>.54334: Flags [S.], seq<PHONE_NUMBER>, ack<PHONE_NUMBER>, win 65160, options [mss 1418,sackOK,TS val<PHONE_NUMBER> ecr 560436112,nop,wscale 7], length 0
15:14:59.163377 IP <IP_ADDRESS>.54334 > <IP_ADDRESS>.6443: Flags [.], ack 1, win 502, options [nop,nop,TS val 560436113 ecr<PHONE_NUMBER>], length 0
15:14:59.163417 IP <IP_ADDRESS>.54334 > <IP_ADDRESS>.6443: Flags [P.], seq 1:518, ack 1, win 502, options [nop,nop,TS val 560436113 ecr<PHONE_NUMBER>], length 517

So it looks like there's a SNAT'ed packet, however I can't understand why the translated session keeps sending packets while I don't even see a SYN-ACK in the pod. So maybe it's a red herring.

Hmm. Would it be possible to get an access to your Azure cluster (I'm martynas on Cilium's public slack)?

the same bug , mark it

@Mengkzhaoyun What is your setup and configuration?

I deploy the cilium-dev:v1.9.0-rc0 & kubernetes v1.18.8 with out kube-proxy mod (3master ha with kube-vip)
when i test the deploy on 3 ubuntu 20.04 hyper-v win10 vm , i found my kubernetes'dashboard pod can not access https://kube-apiserver:6443 , the error is (apiserver service endpoint) access time out.
I login to bash on dashboard pod , it can not curl access other host's apiserver , but bash on the pod' vm host , the curl https://kube-apiserver.othervm.local:6443/version run success.
I change enable-bpf-masquerade to false then my kubernetes'dashboard pod work correct.
here is my config
{
	"auto-direct-node-routes": "false",
	"bpf-lb-map-max": "65536",
	"bpf-map-dynamic-size-ratio": "0.0025",
	"bpf-policy-map-max": "16384",
	"cluster-name": "default",
	"cluster-pool-ipv4-cidr": "<IP_ADDRESS>/11",
	"cluster-pool-ipv4-mask-size": "24",
	"debug": "false",
	"disable-cnp-status-updates": "true",
	"disable-envoy-version-check": "true",
	"enable-auto-protect-node-port-range": "true",
	"enable-bpf-clock-probe": "true",
	"enable-bpf-masquerade": "false",
	"enable-endpoint-health-checking": "true",
	"enable-external-ips": "true",
	"enable-health-check-nodeport": "true",
	"enable-host-port": "true",
	"enable-host-reachable-services": "true",
	"enable-hubble": "true",
	"enable-ipv4": "true",
	"enable-ipv6": "false",
	"enable-node-port": "true",
	"enable-remote-node-identity": "true",
	"enable-session-affinity": "true",
	"enable-well-known-identities": "false",
	"enable-xt-socket-fallback": "true",
	"hubble-listen-address": ":4244",
	"hubble-socket-path": "/var/run/cilium/hubble.sock",
	"identity-allocation-mode": "crd",
	"install-iptables-rules": "true",
	"ipam": "cluster-pool",
	"k8s-require-ipv4-pod-cidr": "true",
	"k8s-require-ipv6-pod-cidr": "false",
	"kube-proxy-replacement": "partial",
	"masquerade": "true",
	"monitor-aggregation": "medium",
	"monitor-aggregation-flags": "all",
	"monitor-aggregation-interval": "5s",
	"node-port-bind-protection": "true",
	"node-port-range": "20,65000",
	"operator-api-serve-addr": "<IP_ADDRESS>:9234",
	"preallocate-bpf-maps": "false",
	"sidecar-istio-proxy-image": "cilium/istio_proxy",
	"tunnel": "vxlan",
	"wait-bpf-mount": "false"
}

@Mengkzhaoyun Thanks.

@Mengkzhaoyun Can you please ping me (martynas) on the Cilium's public slack?

@brb I'm pretty sure this is related to issue I was last discussing with you guys as well where I had to revert to iptables.

@networkop @Mengkzhaoyun OK, so what happens in your case is that the wrong IPv4 addr is picked for SNAT, as the selected devices for NodePort have more than one IPv4 addr. Currently, cilium-agent can support only one per bpf_host.o instance, and it does not check which IP addr is used to communicate between k8s nodes (aka k8s Node IP). The fix would be to to prefer the k8s Node IP when selecting that single IP addr.
If any of you wants to work on the fix (a good opportunity to familiarize with Cilium internals), I'm happy to guide you.

I'm happy to pick it up @brb

The fix might be simple: in the function https://github.com/cilium/cilium/blob/master/pkg/node/address.go#L128 you need to pass k8s Node IPv{4,6} addr to the invocations of firstGlobalV{4,6}Addr(...) as a second parameter.

add CILIUM_IPV4_NODE env to cilium-agent can temp skip the bug.
        env:
        - name: K8S_NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        - name: CILIUM_IPV4_NODE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName

@Mengkzhaoyun that makes my deployment crash as the nodeName is a hostname...aware of another property that pushes the IP definitively?
level=fatal msg="Invalid IPv4 node address" ipAddr=<node hostname> subsys=daemon

Here's the better way I suppose:
        - name: CILIUM_IPV4_NODE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP

Didn't help my scenario so I must be hitting a different bug altogether.

CILIUM_IPV4_NODE doesn't have anything to do with fixing the issue.

@brb I'm pretty sure this is related to issue I was last discussing with you guys as well where I had to revert to iptables.

As per discussion offline - it's not related.

Here's a brief summary of the issues I was hitting:

The main issue was caused by the fact that one of the k8s nodes had multiple outgoing interfaces. Specifically, the interface that was used to communicate with the affected nodes was wireguard, which doesn't have a L2 (MAC) address and is currently not supported until #12317 gets resolved.

The secondary issue was that cilium agent is picking the wrong SNAT IP for traffic leaving the primary (non-wg) interface. This interface was a bond interface with 1 public and 1 private IP. Cilium was picking the public IP, ignoring the fact that that k8s node had InternalIP set to the private IP. The following patch fixed this issue for me:

diff --git a/pkg/node/address.go b/pkg/node/address.go
index a5fa3dadd..7c438a9d8 100644
--- a/pkg/node/address.go
+++ b/pkg/node/address.go
@@ -129,10 +129,14 @@ func InitNodePortAddrs(devices []string) error {
        if option.Config.EnableIPv4 {
                ipv4NodePortAddrs = make(map[string]net.IP, len(devices))
                for _, device := range devices {
-                       ip, err := firstGlobalV4Addr(device, nil)
+                       ip, err := firstGlobalV4Addr(device, GetK8sNodeIP())
                        if err != nil {
                                return fmt.Errorf("Failed to determine IPv4 of %s for NodePort", device)
                        }
+                       log.WithFields(logrus.Fields{
+                               "device": device,
+                               "ip":     ip,
+                       }).Info("Pinning node IPs")
                        ipv4NodePortAddrs[device] = ip
                }
        }
@@ -140,7 +144,7 @@ func InitNodePortAddrs(devices []string) error {
        if option.Config.EnableIPv6 {
                ipv6NodePortAddrs = make(map[string]net.IP, len(devices))
                for _, device := range devices {
-                       ip, err := firstGlobalV6Addr(device, nil)
+                       ip, err := firstGlobalV6Addr(device, GetK8sNodeIP())
                        if err != nil {
                                return fmt.Errorf("Failed to determine IPv6 of %s for NodePort", device)
                        }
diff --git a/pkg/node/address_linux.go b/pkg/node/address_linux.go
index a05a3f38d..bfa16fc73 100644
--- a/pkg/node/address_linux.go
+++ b/pkg/node/address_linux.go
@@ -86,7 +86,7 @@ retryScope:
        }

        if len(ipsPublic) != 0 {
-               if hasPreferred && ip.IsPublicAddr(preferredIP) {
+               if hasPreferred {
                        return preferredIP, nil
                }

Finally, the last issue was me (PBKAC). I started replicating the issue in the default namespace which also happened to have an egress network policy. cilium monitor -t drop helped identify that.

Thanks @brb for the support, let me know if you want me to do a PR with the above fix or it's ok to just close this issue for now.

@networkop  , good job , it is work.

@networkop can you push a PR with that fix? that would be great!

PR is ready for review :+1:


DSpace 6.3 searching for bitstream identifier

Describe the bug
Searching for the bitstream identifier returns no results and returns a recommended search which also returns no results.
To Reproduce
Steps to reproduce the behavior:

Find a bitstream identifier (I used "f705a8ce-485f-4c4e-ae8e-7b9fc13cffad" from the media extraction cronjob)
Search for it in GUI
Notice that there are no results.
Notice that there's a "Did you mean:" search for a very similar string (usually one character different, which one depends on the exact identifier used)
Click in the recommendation
Notice that the recommended search doesn't have any hits either.

Expected behavior
(a) Items should be findable by their bitstream and other identifiers
(b) Recommended searches should return items.

This sounds like a "new feature" request to me.  I don't believe bitstream Identifiers are currently indexed in DSpace 6.x (or above).  So, I suspect that's why they do not work in searches.