MiniRake::Task#timestamp should use Time.now only

$ cat t.rake
require 'fileutils'

file "a.txt" do |t|
  p t.name
  FileUtils.touch(t.name)
end

task "b.txt" => "a.txt" do |t|
  p t.name
  FileUtils.touch(t.name)
end

file "c.txt" => "b.txt" do |t|
  p t.name
  FileUtils.touch(t.name)
end

$ rm -f {a,b,c}.txt

$ rake c.txt -f t.rake
"a.txt"
"b.txt"
"c.txt"

$ rake c.txt -f t.rake
"b.txt"
"c.txt"

$ rake c.txt -f t.rake
"b.txt"
"c.txt"

$ ./minirake c.txt -f t.rake
(in /Users/ksss/src/github.com/ksss/mruby)
"b.txt"

$ touch b.txt

$ ./minirake c.txt -f t.rake
(in /Users/ksss/src/github.com/ksss/mruby)
"b.txt"

$ rake c.txt -f t.rake
"b.txt"
"c.txt"

I think, minirake should call "c.txt" file task block

Referenced by https://github.com/ruby/rake/blob/af15762f972a3f8322a9b5904d56fe2ff3dd8352/lib/rake/task.rb#L261

I am not 100% sure, but I trust you.  Report if something happens.

@matz Thank you for trust me.
What are the concerns? backward compatibility?
I'll revert if problems arise.

No specific concern. I have to confess I didn't understand the meaning of the change.

I see.


Ensure that better-strings runs before any (?) other annotation processor

Not sure if that can done without a dedicated build-tool-specific plugin (for Maven/Gradle)... anyway, I'd be pretty nice if there was a way to ensure better-strings is run before other common annotation processors like https://github.com/checkstyle/checkstyle or (the very awesome) https://github.com/typetools/checker-framework. Otherwise they won't have the chance to statically verify the Java code within the interpolations.

I think we can do it only with a build tool (maven/gradle).
As I know in the source code of APT you cannot set the ordering. And the order of loading annotation processors depends on the class loader implementation, it's unpredictable.
But I think this is a good point to explain in the documentation. "How to use with other annotation processors"


$\sqrt{x}$ can't be rendered properly after enabled this extension

Hi, I installed the extension "Markdown All in One" before, which can render latex, this function is OK. Recently I installed this extension "vscode-markdown-markmap", while after that some latex can't work properly in markdown.
The picture shows the problem.
If there's no character behind "$\sqrt{x}$", "$\sqrt{x}$" can't be rendered properly, which is not correct.

If you want to reproduce, just install the extensions "Markdown All in One" and "vscode-markdown-markmap", and copy the following code to a markdown file
$\sqrt{x}$

$\sqrt{x}$a

$\sqrt{x}$
n

${\sqrt{x}}$

$\sqrt{{x}}$

It would be great If you can do some check about this. Thank you!:smile:

@m1stm4o
Thanks for your reporting. I'll get it investigated.

This happens because the markdown-it-attr module in "vscode-markdown-markmap" was conflicted with { } curly brackets to the markdown-it-katex module in "Markdown All in One" extension.
I'll add some options to solve this issue soon.
Thank you again for your reporting.

Ah, I see, looking forward to your update.

@m1stm4o
This issues fixed in vscode-markdown-markmap v1.1.0.
Now, you can use { } curly brackets to add attributes with or without the markdown-it-attr module.

@phoihos Thanks.


Add mock based testing

Let's just shamelessly steal our test infrastructure from oak-api
The data provider backend responses are mocked:
@pytest.fixture
def mock_oak_implementation() -> OakImpl:
    mock = Mock(OakImpl)
    mock.basic_search.return_value = ["a", "b", "c"]
    mock.labels.return_value = [("a", "a label"), ("b", "b label"), ("c", "c label")]
    mock.definition.return_value = "mock definition"
    mock.ancestors.return_value = ["ANCESTOR:01", "ANCESTOR:02", "ANCESTOR:03"]
    mock.descendants.return_value = ["DESCENDANT:01", "DESCENDANT:02", "DESCENDANT:03"]
    return mock

Then the testing ensures that the api functions call the right mocked data provider functions:
def test_get_ancestors(client: TestClient, mock_oak_implementation: OakImpl):
    curie = "TEST:000001"
    response = client.get(f"/classes/{curie}/ancestors")
    assert response.status_code == 200
    body = response.json()
    assert "data" in body
    assert "pagination" in body
    assert body["data"][0] == {"id": "a", "label": "a label", "definition": "mock definition"}
    mock_oak_implementation.ancestors.assert_called_with(curie, None)
    mock_oak_implementation.labels.assert_called_with(
        ["ANCESTOR:01", "ANCESTOR:02", "ANCESTOR:03"]
    )

I added some initial mock based testing, I think we can keep it going with new features without keeping this issue around


Making source files overrideable?

Here's my usecase:
I'm writing CMS that uses less for its styles, and currently I have two layers of those:

bootstrap
cms

There's also empty third layer named "flavor" that contains two empty less files: variables.less and flavor.less.
My idea is that to customise way site looks, users can override variables and styles defined in previous two layers by defining custom ones in third layer.
Currently this isn't working because compilation occurs before files are copied into STATIC_ROOT. I understand that there were good reasons for doing this that way, but I am wondering if current implementation leaves place for changing current behaviour for desired one?
I can think of different approaches to this issue, but sadly I am yet to come with one that is elegant and spares users that wish to customize their sites additional maintenance burden of having to copypaste files around.
Anyway thanks for your time and foremost thanks for amazing project! :+1:

Scratch that, I've updated to 1.4.2 and it seems that this behaviour is there, at least in debug.


OVERLAPBLOCK (DESIGN 15)

Overlapblock (design 15)
Design a overlap block that looks like the button in this image...
Create a file "OVERLAP_DESIGN_15.html" in the modules/overlapblock folder.
Use simple HTML and CSS and JAVASCRIPT to create the overlapblock. You can use the interface development https://kwickerhub.com (This project) or you can write the lines of code yourself.
Step by Step Guide.
Fork this project(Use the 'fork' button in the top right corner) and Clone your Fork.
git clone https://github.com/YOUR_USERNAME/frontend
Open your code Editor and Create a file "OVERLAP_DESIGN_15.html" in the "modules/overlapblock" folder of this project you just cloned. No need to create head, title and body tags, Just Add a div tag with some embedded style(i.e use the style tag) and the script tag where necessary.
If you want to add an image resource, please add it in the folder  ••
"modules/overlapblock/images_and_icons"
We recommend you use an svg for your image/icon.
Push your Code: You need to push your recent changes back to the cloud. Use the command below in the main directory of this Repository
git push origin dev
or use a GUI tool to avoid mistakes or complexity. LOL.
Make you Pull Request...
Good-luck.

Hi, I want to contribute, so I need to make a page that looks exactly the same as the image?

Hello sir please Assign me this issue,i want to work on this issue

@sanjaygupta972004 I think someone has already been assigned, but you can still get on with it

Ok sir

@MaverickDe sir, I have a doubt, this issue is already assigned to the another. can i work on this issue

Sure go on
On Fri, Jun 2, 2023, 2:35 PM Sanjay Gupta @.***> wrote:

@MaverickDe https://github.com/MaverickDe sir, I have a doubt, this
issue is already assigned to the another. can i work on this issue
—
Reply to this email directly, view it on GitHub
https://github.com/KwickerHub/frontend/issues/248#issuecomment-1573746124,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYAUKVKYIDRIQYO2VI5E2F3XJHT2TANCNFSM6AAAAAAYMDJ25Q
.
You are receiving this because you were mentioned.Message ID:
@.***>

Thanks sir

I want to work on this issue. Please assign this issue me.

Is anyone working on it ?


Refactor Customer Product UI

[ ] Move button Add Customer Product
[ ] Move Filter Customer Product button and Field

applied

applied


Runs forever, CPU at 145%, when using nodejs v8.15.0

Yeah, that title says it all. Just

run yarn run audit-ci <whatever> using nodejs v8.15.0,
have a coffee and
come back
Still running and CPU is blocking my system.

Changed to nodejs v10 and 12, works fine there. Happens with audit-ci v1.5.0 and latest v2.3.0 both.
Would be nice to have a verbose mode so that I can see where exactly your audit tool is stuck with.

Btw using a private npm registry here and having scoped packages.

Thanks for the thorough bug report. I agree with having a verbose option. I don't think yarn supports having a private repo: https://github.com/yarnpkg/yarn/issues/7012. If this is the case, you can try using NPM as the auditer for Node 8.

Might be that, dunno know. Still doesn't explain why it works when sticking to yarn while using Node.js v10 instead.

I haven't heard much from this issue in a while. Unless someone mentions that they have seen this issue in a week (and it's not this issue: #103), I will close this.
(Sorry for late pings)


[SPARK-40198][CORE] Enable spark.storage.decommission.(rdd|shuffle)Blocks.enabled by default

What changes were proposed in this pull request?
This PR aims to enable the following two configurations by default. Note that these storage configurations are disabled by default by the parent configuration, spark.storage.decommission.enabled, still. In addition, all decommission features including this storage decommissioning are disabled by spark.decommission.enabled by default.

spark.storage.decommission.rddBlocks.enabled
spark.storage.decommission.shuffleBlocks.enabled

Why are the changes needed?
This will help users use the storage decommissioning feature more easily.
Does this PR introduce any user-facing change?
Yes, this will reduce the number of required configurations for decommission users.
However, these two configurations are disabled by two configurations still.
How was this patch tested?
Pass the CIs.

Hi, @viirya . Could you review this? This PR will help the executor rolling and worker decommission users.

Thank you!

Merged to master for Apache Spark 3.4.0.

We do not support decommissioning in yarn iirc - am I missing something here ?
If not, we should not be turning shuffle decom by default - particularly when it is not in k8s

Hi, @mridulm . spark.storage.decommission.enabled is false still.
So, these two configuration changes are no-op for the existing YARN jobs.
IIUC, all YARN jobs have spark.storage.decommission.enabled=false (by default), but also spark.decommission.enabled=false (by default) before and after this PR.

Thanks for clarifying @dongjoon-hyun  !
I knew I was missing something :-)

Thank you always for reviewing, @mridulm . :)


[TEVA-3838] Fix name method bug

Jira ticket URL
https://dfedigital.atlassian.net/browse/TEVA-3838
Changes in this PR:
This PR fixes the bug causing "local authority" to be added to the end of a local authority name and then persisted in the database.

deoes this not mean there is something needs tweaking in a test too?

deoes this not mean there is something needs tweaking in a test too? as in its testing that the string includes the la name when it should be an exact match - i havent actually looked at the test tho!

@alexbowen do you mean the test for the name method on the model spec? In which case, I think the current test seems to cover it (it tests equality to "Camden local authority" for organisation name Camden).
The change here means we just aren't assigning the new concatenated name to an instance variable (this was causing a problem when creating a vacancy, as the organisation name was being updated in the db with the value of the instance variable).


VSCode Extension reports uncaught errors

Describe the bug
I've installed the devskim plugin in VSCode with my Remote WSL instance, and the Runtime status for the plugin shows the following uncaught errors
Request textDocument/devskim/validateddocuments failed with message: Cannot read properties of undefined (reading 'uri')
Additionally, the plugin does not seem to detect any of the problems in my code that are found in my ADO pipeline running the devskim command.
To Reproduce
Steps to reproduce the behavior:

Install plugin
View "Runtime Status"

Expected behavior
No errors and warnings detected.
Screenshots

Versions(please complete the following information):

OS: Windows 11, build 22000.856 (where VS Code is installed), WSL running ubuntu 20.04
Devskim Version: 0.7.1
VSCode Version: 1.70.2

Thanks for the report. The 0.7 version of the vscode extension is still a prerelease, but the flag may not have been set properly in the build. I will try to get it taken down today so it is not installed in error.
if you can select a specific version try installing the latest 0.6 revision instead.

I've installed 0.6.26 instead (which is the latest version) and I'm still seeing the same errors.

@AdmiralGT that is unexpected and I'm sorry it's not working. I can't think immediately of any changes made recently to the vscode extension that would have broken it, but the build itself has been fragile in the past.
I'm coincidentally currently working on a complete rewrite of the extensions (that will be 0.7.x) and I hope to have something available next week.

Sorry for the delay in addressing this.
We've rolled back to a working version for the latest release and marketplace version is now working for me.


Authentication on pull must require auth in UI

Currently, if the "authentication on pull" feature is enabled, any user (logged in or not) can see the crates, docs and so on in the UI. That renders the authentication feature rather useless, as the code of the crates can be seen in the docs anyway.
Implement: If the "authentication on pull" is enabled, require a login to the UI and do not show any data to anonymous users.

Next to-do: Check in the UI if auth if required and if so, show only the login page, instead of an empty list of crates.


ci: fix release step

More details here: https://github.com/cycjimmy/semantic-release-action/issues/79

:tada: This PR is included in version 6.0.0 :tada:
The release is available on:

v6.0.0
GitHub release

Your semantic-release bot :package::rocket:


June release branch

I ran test_data_cycle and tested annotation through the client tool. Everything works after I applied changes as requested during the PR. Please refer to this commit to see the changes.
Here is the list of newly created issues:

BQ annotation method
Getting rid of pandas in model service
Great bq_scripts refactor
Cellarium ML integration
Flexible filters in extract method


Create the PageVectorIndexedMixin class

It's an explicit way to include Wagtail-specific behaviour (i.e. use of PageVectorIndex instead of ModelVectorIndex) when defining a Page model. This also helps with type annotations for the return value of the get_vector_index() method.

I have this WIP patch to wagtail-vector-index lying around my repo since two weeks ago. It's a bit of a refactoring of the (page) model mixin class, to make typing easier, and make the code a bit more robust, but it introduces a breaking change. Not sure if it's worth pursuing, and I don't have bandwidth for it, so I've made it into a draft PR, in case somebody else finds it useful. If not, feel free to close it.


Migrate dds.dart to null safety

https://github.com/flutter/flutter/pull/88015 was reverted due to https://github.com/dart-lang/sdk/issues/46934.
Re-migrate dds.dart.  Add a unit test to check the host port is 0 when unspecified.
Part of #71511

@knopp @bkonyi I don't think this should regress https://github.com/dart-lang/sdk/issues/46934 again, but let me know if you see anything wonky.

/cc @wanglikun7342

Whoops misread the review status, apologies.

@jmagman, I've tested this briefly, attach now works as expected. Thanks!


Remove superfluous "into" from several backend tests.

Several test modules were still named test_*_into.py. I removed the _into.

Thanks!


Vagrantfile: Add option to set the host port to use

Automated message from Dropbox CLA bot
@timabbott, it looks like you've already signed the Dropbox CLA. Thanks!


Small changes in structure and typos

Just one note - should it be goleveldb and goqueue or should it be goleveldb and goqueue?

@Simplychee Thanks, fixed.


Issues with uv scaling

Hello
When setting uv coordinates using the Model , there seems to be an issue of non-tiling uvs when the uv corrdinate is out of the (0.0,1.0) range. For example, I have set the uv coordinates to 10.0 in the image below:

But should it not tile by the scale amount (e.g. in this case tile by 10 times in the y/x direction)? I have attached the file (it uses the demo project you have provided in examples folder)
uvs.tar.gz
Thanks

Hi!
This is weird. I can only suspect some missing WGPU configuration. I will check it ASAP and let you know.
Cheers

Hi!
This is a configuration of sampler. If you change it in dotrix_core/src/renderer.rs to
        ctx.sampler = Some(device.create_sampler(&wgpu::SamplerDescriptor {
            address_mode_u: wgpu::AddressMode::Repeat,
            address_mode_v: wgpu::AddressMode::Repeat,
            address_mode_w: wgpu::AddressMode::Repeat,
            mag_filter: wgpu::FilterMode::Nearest,
            min_filter: wgpu::FilterMode::Linear,
            mipmap_filter: wgpu::FilterMode::Nearest,
            ..Default::default()
        }));

It will work as expected.

Hi, thanks for answer, I did not know that! I close now

The default behaviour will be  changed in the next release

Ok that sounds good, shall I reopen this issues as a 'Feature Request'?

Sure, you can. Thank you!


Project status

Hi,
What is the project status?
1 - It is abandoned?
2 - It has support for PHP 7.4?

@prsolucoes Hello!
Nopes.
The project is not abandoned.
It's simply working as expected.
For PHP7.4 support, we should take a look.
Should be just upgrade the minimum requirements in composer.json

Closing as the project works properly even with php8


fping: 5.1 -> 5.2

Changelog: https://github.com/schweikert/fping/releases/tag/v5.2
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 307425 run on x86_64-linux 1

4 packages built:

fping
smokeping
zs-apc-spdu-ctl
zs-wait4host

Result of nixpkgs-review pr 307425 run on x86_64-linux 1

4 packages built:

fping
smokeping
zs-apc-spdu-ctl
zs-wait4host


SUNNYVALE → dddNYVALE

Obviously this isn't a date, because I'm using parseFormat to help look for dates and times in CSV data.
moment.parseFormat('SUNNYVALE') parsed to dddNYVALE, I expected: SUNNYVALE (or null)
I think we should exclude any month/day name or abbrev matches that have adjacent letters.

I think your use case is out of scope of parseFormat. If your data comes out of a CSV, maybe do a pre-cleanup before you pass the date string into parseFormat?
parseFormat is meant to understand human-input of date/time strings. SUNNYVALE is not something a human would enter, the problem is somewhere else and I would resolve the problem outside of parseFormat


CLI Implementation with Click

Description
This PR implements a simple CLI using click. Note the following entrypoint:
    entry_points={
        "console_scripts": [
            "axolotl=axolotl.cli.main:main",
        ],
    },

This means, once pip installed, axolotl's CLI can be used as:
axolotl [command] [args]

It support the following commands:

axolotl preprocess
axolotl train
axolotl inference
axolotl shard
axolotl merge_sharded_fsdp_weights
axolotl fetch

The last command is new functionality which allows the user to download (really, sync) the contents of the examples/ and deepspeed_configs/ directories at the top level of the axolotl project. For example:
axolotl fetch examples
axolotl fetch deepspeed_configs

An optional argument --dest [directory] allows the user to specify an output location; otherwise, this defaults to examples/ and deepspeed_configs/ in the local directory.
Motivation and Context
This change was requested in this Notion ticket. This allows axolotl users that have installed the package via pip to use the existing CLI commands, and source users to use them with a slightly simpler interface axolotl [command] ....
How has this been tested?
Ad hoc testing on axolotlai/axolotl-cloud:main-latest image on a Runpod A40 instance on this feature branch. I sparsely tested the preprocess, train, and fetch commands.
# pre install
$ axolotl train examples/openllama-3b/lora.yml
-bash: axolotl: command not found

$ pip install -e .
...

# works
$ axolotl preprocess examples/openllama-3b/lora.yml

# works
$ axolotl train examples/openllama-3b/lora.yml
...

# config file overrides? works with replacing "_" with "-" (click limitation)
$ axolotl train examples/openllama-3b/lora.yml --max-steps 1 --micro-batch-size 1 --val-set-size 0

# etc.

TODO:

[ ] Add pytest coverage for the CLI (?)
[ ] Test via PyPI package installation (?)

Types of changes
This PR adds a new file src/axolotl/cli/main.py which implements the Click CLI commands, and minor changes elsewhere to accommodate them.

Question for the team: does it make sense to write tests against this code path? I noticed there weren't tests against the existing axolotl.cli module, but I can pad these out if desired. In the interest of moving fast, we can also backlog this.

Question for the team: does it make sense to write tests against this code path? I noticed there weren't tests against the existing axolotl.cli module, but I can pad these out if desired. In the interest of moving fast, we can also backlog this.

yeah, let's go ahead and take the time now to add some tests against the cli.

I think the test errors:
    def test_inference_all_options(
        cli_runner, default_config, mock_inference_deps
    ):  # pylint: disable=redefined-outer-name
        """Test inference with all possible options"""
        mock_cfg = MagicMock()
        mock_inference_deps["load_cfg"].return_value = mock_cfg

        cli_runner.invoke(
            cli,
            [
                "inference",
                str(default_config),
                "--load-in-8bit",
                "--base-model",
                "base/model/path",
                "--lora-model-dir",
                "lora/path",
                "--prompter",
                "my_prompter",
            ],
        )

        # Check all options were passed through
>       cli_args = mock_inference_deps["do_inference"].call_args[1]["cli_args"]
E       TypeError: 'NoneType' object is not subscriptable

is because some of the commands run accelerate as a new subprocess, so don't directly call the do_inference function in this scope.

Assuming tests pass here (they're passing in Runpod at least), this PR is ready for re-review. I think I got the testing mostly right; I'm testing mostly shallowly where I assert that args that are expected are in fact passed to the relevant train / do_inference / do_clil etc. methods in similarly named axolotl.cli.preprocess / axolotl.cli.train etc. modules. A small subset of tests (marked as pytest integration tests) are more end-to-end and should be useful for preventing unexpected bugs from making it to main.
As an aside, once we deprecate the old way of running axolotl commands on the command line (e.g., accelerate launch axolotl.cli.train ...), we can probably refactor out a good bit of cruft.

Adding context from Slack:

I think something weird is going on with the logging. I couldn't debug it after a few hours of trying so I've moved the tests to tests/e2e/patched for now. I think maybe the test_validations.py could be moved there instead as I think that's what's causing the issue with the caplog logic, but I'm not sure.

I also simplified some of the CLI tests so they're no longer running simple preprocess / train / shard etc. commands end to end for the sake of runtime. We have other test coverage that actually tests this functionality, so I think there's no need to duplicate.

I need to add a way to pass accelerate launch args as well.

need to update pytest calls in https://github.com/axolotl-ai-cloud/axolotl/blob/main/.github/workflows/tests.yml#L82 and on line 126 as well (and probably the nightly tests too)


Component usage as props not working

Hello,
I'm encountering an issue where components used as props seem not to be recognized during the build process.
Here's a snippet of my configuration:
// vite.config.ts
export default defineConfig({
  plugins: [
    components({
      dirs: 'src/components',
      dts: 'src/components.d.ts',
      external: [
        {
          from: 'svelte-routing',
          names: ['Link', 'Route', 'Router'],
          defaultImport: false,
        },
      ],
    }),
  ],
  // ...
})

And here is how I'm using the components in my Svelte app:
// App.svelte
<Router>
  <Route path="/" component={ListPage} />
</Router>

While Router and Route work as expected, during the build process I receive an error stating that ListPage is not defined.
I believe this might be a limitation when using components as props, which is a feature I find very useful. Would it be possible to extend the functionality to support components used in this manner?
Thank you for your work on this plugin and for considering this enhancement.
Best regards

Hi @kevin82222,
This is an interesting feature, but I have a couple of questions before implementing it.

First, how can we figure out whether a prop is a component or not? For example, if the user creates a local variable with the same name as a component in the auto-imports folder, it will be difficult to tell which import is correct. Here is an example:
<script>
  import ListPage from "X"; // this will be auto-imported by unplugin-svelte-components, because the user passed `ListPage` as a prop in the component

  let ListPage = "Something"; // this is a variable the user created that will be passed to the component
</script>

<Route path="/" component={ListPage} />

This is most likely to cause an error, because the user created a prop with the same name as a component, and unplugin-svelte-components found the component with that name and tried to import it.

Second, while we checking whether a prop is a component, that may slow down development and build times? This is because we would need to check every component that the user uses and check its props.


workflows: fix L4LB test missing PR reporting on issue_comment

Unlike pull_request-triggered runs, issue_comment-triggered runs cannot automatically report back to the PR they were triggered from.
Adding manual reporting as per the other workflows.

L4LB test failed: https://github.com/cilium/cilium/actions/runs/1012162379
But this is the expected unrelated failure that's been happening in all L4LB runs:
+ kubectl -n kube-system wait --for=condition=Ready pod cilium-2kpf2 --timeout=5m
error: timed out waiting for the condition on pods/cilium-2kpf2

The manual PR check did get added properly to the PR:

So I would say the changes are validated.
Removing temp commit.

Btw I see this note in the workflow yaml [1] -
### FOR TESTING PURPOSES
  # This workflow runs in the context of `master`, and ignores changes to
  # workflow files in PRs. For testing changes to this workflow from a PR:
  # - Make sure the PR uses a branch from the base repository (requires write
  #   privileges). It will not work with a branch from a fork (missing secrets).
  # - Uncomment the `pull_request` event below, commit separately with a `DO
  #   NOT MERGE` message, and push to the PR. As long as the commit is present,
  #   any push to the PR will trigger this workflow.
  # - Don't forget to remove the `DO NOT MERGE` commit once satisfied. The run
  #   will disappear from the PR checks: please provide a direct link to the
  #   successful workflow run (can be found from Actions tab) in a comment.
  #
  # pull_request: {}
  ###

[1] https://github.com/cilium/cilium/blob/master/.github/workflows/tests-l4lb.yaml

@nbusseneau @brb Does this GH action need to be run as a cron job if it's run on ever PR?

@nbusseneau @brb Does this GH action need to be run as a cron job if it's run on ever PR?

It's there for testing master every 6 hours so that we can detect when merging multiple concurrent PRs might be breaking things :)


Change Index Criteria to Not Allow Floats. Add Tests.

 The previous logic allowed floats, but it did not seem like float indexes made sense, as PHP cannot have floats as an array index, and the previous behavior was casting the float to an int and using that as the index value.
return $this->source[$this->convertIndex(\intval($offset))];

Consider if this is the behavior you want.
For reference, if you cast a float to an int:
php > echo intval(1.4);
1
php > echo intval(1.5);
1
php > echo intval(1.9);
1
php > echo intval(-1.9);
-1

Added some tests to show that floats and numeric floats as well as special floats no longer work.
For reference in Python:
>>> [1, 2, 3][1.5]
<stdin>:1: SyntaxWarning: list indices must be integers or slices, not float; perhaps you missed a comma?
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers or slices, not float

Also, it was not clear what the distinction between KeyError and IndexError is. The unit tests assert what is currently being thrown.

Thank you @markrogoyski for you PR! I merged it.


Pods created with istio injection are not accessible if no service is created

Looking at the new changes: https://github.com/istio/istio/blob/b8e30e07ec7b36cf1ba1b445b78c81fdf0d417b9/pilot/cmd/pilot-agent/status/ready/probe.go#L42
sidecar proxy will mark itself as not ready if no service is created(no listeners being pushed from the pilot). This somehow makes senses if you think service is a switch for network layer.
This also breaks liveness/readiness check for the main container as it will return 404.
@hzxuzhonghu

@iandyh I am not sure if you have noticed the pr by @incfly ,which rewrite the liveness/readiness probe. Can help resolve the probe failure.  But It does not help to access directly from outside.
@rshriram @costinm Any commments?

https://github.com/istio/istio/pull/10470/

For kubernetes  pod, I think we can get the container port and generate a listener for it.     Just as it has a associated service with ip = podIP and port=conatinerPort.  We should also assume the protocol is TCP

@hzxuzhonghu Yes. I fully agree. Sidecar proxy should not break any kubernetes semantics in this case.
I saw the probe fixes is being merged to 1.1. Do you think fixes to this issue will be merged into 1.1 as well? Or we have to wait next releases?

Since 1.1 will be released very soon, I think we have to wait until 1.2.

or 1.1.1

If I understand the problem correctly,

Injector is correct, passing application port into the sidecar cmd flag as applicationPorts
No service definition however, No xDS.
Prober doesn't report ready.

So this has nothing to do with health check rewrite with mTLS, correct?
Regarding solution,
I think we can get the container port and generate a listener for it. Just as it has a associated service with ip = podIP and port=conatinerPort. We should also assume the protocol is TCP

You mean do this at Pilot level such that we always have a default LDS so this case will be covered?
My thoughts of this problem,

Can Pilot generate a bit to tell Sidecar, "hey sidecar, you should consider yourself ready in terms of xDS config"
Pilot agent only look at this bit, instead of reading applicationPorts and compare that to the number of xDS received? (Maybe I miss something, but I don't understand why we pass containerPort from app to sidecar originally? Shouldn't it be xDS per service not per deployment?)

Thoughts? @hzxuzhonghu @rshriram @costinm

if the crux of the matter is that our readiness is tied to listeners, and these pods don't have listeners..., isn't it better to not have readiness for these pods at all? If they don't expose a service what is readiness doing anyway.
If there is a way to disable highjacking readiness for this deployment, lets us it. Otherwise we can add a new annotation that exempts it from readiness checks.

if the crux of the matter is that our readiness is tied to listeners, and these pods don't have listeners..., isn't it better to not have readiness for these pods at all?

Do you mean checking service existing on a particular deployment during injecting time?
If so, I don't think fixing on injector side will work, given deployment & injector is once and static. Service definition are dynamic by its nature. Only pilot can capture that anatomically.

if the crux of the matter is that our readiness is tied to listeners, and these pods don't have listeners..., isn't it better to not have readiness for these pods at all? If they don't expose a service what is readiness doing anyway.
If there is a way to disable highjacking readiness for this deployment, lets us it. Otherwise we can add a new annotation that exempts it from readiness checks.

Not sure if I understand the reply correctly. Firstly, HTTP liveness will also fail due to sidecar injects the traffic, not just readiness check. Secondly, those pods do listen on certain ports but those ports become inaccessible due to no Envoy side car proxy listeners being pushed from the pilot.

If pods are listening on ports then they are implementing a service. In which case it is ok to attach a service to these pods, right?

Some applications will just use this port as a liveness check, which will become pointless due to current design. Deployment should be decoupled from Service.


NDRS-891: DB integrity check

REF: https://casperlabs.atlassian.net/browse/NDRS-819
CHANGELOG:

Added a check in the intializtion of a reactor to check if the missing_trie_keys is empty. If it is not, the node will panic
Added a pub function to allow the reactor during initialization to get the current highest block from storage if it exists
Added a pub function within the contract runtime to allow for querying of the trie store

I would like to see this PR tested in various contexts:

node starts, gets restarted and then tries synchronizing
node upgrades
node joins an upgraded network (including Global State changes)

Just to be sure that going through various states of the Global State and the block storage does not break this.

@goral09 I tested this yesterday by running a 5 node network with nctl and performing an upgrade from era 1 to era 2 and the node continued to function nominally

bors r+


[master] Upgrade to latest dependencies

Produced via:
./hack/update-deps.sh --upgrade
./hack/update-codegen.sh

/cc tcnghia n3wscott
/assign tcnghia n3wscott

/lgtm


Add a max interval between waterings.

Coverage decreased (-0.03%) to 80.032% when pulling e082f971d080bd0ab46c782c2bb23c1ed0297113 on pump_interval into 87a60842d34bfe450a9b8a7c86515df4bd9b41b4 on master.

Coverage increased (+0.06%) to 80.128% when pulling 3e769db515714d0e73e396d3a98797457b82f156 on pump_interval into 87a60842d34bfe450a9b8a7c86515df4bd9b41b4 on master.

Coverage increased (+0.06%) to 80.128% when pulling b39b6be53974a087fddf42424f26ade326783cd9 on pump_interval into 87a60842d34bfe450a9b8a7c86515df4bd9b41b4 on master.

Coverage increased (+0.06%) to 80.128% when pulling e53118b47924aec6dadc187c01896757078c432d on pump_interval into 87a60842d34bfe450a9b8a7c86515df4bd9b41b4 on master.


Pydantic validator not working on imported hook

If the hook is local (ie in hooks/thing.py) then validators aren't working.  Validators do work normally as in the random_string hook.
tackle.exceptions.UnknownHookTypeException: Error parsing input_file='/home/code/robcxyz/tackle-box/docs/scratch.yaml' at key_path='do'
The hook type="do_stuff" is not available in the providers. Run the application with `--verbose` to see available hook types.

do->: do_stuff a_map

from pydantic import validator
from tackle import BaseHook

class DoStuffHook(BaseHook):
    hook_type: str = "do_stuff"
    stuff: str = None
    args: list = ['stuff']

    @validator('things')
    def validate(cls, value):
        # Check if the input is valid - throw error otherwise
        if value == 'not-things':
            raise Exception
        return value

    def exec(self):
        return self.stuff

Derr....
    @validator('stuff')

This errored out so it never was imported.  This is the right behaviour I think.  All good then.


Performance regression

Seems like FCS is slower on Travis in comparison with AppVeyor
and latest versions become slower.
Repro steps
I use FCS to test SwaggerProvider and ensure that TP is compilable on ~390x2 available Swagger schemas.
I see following test suite execution time

FCS
AppVeyor
Travis (Linux)

11.0.6
15 min 56 sec
40 min 37 sec

11.0.9
24 min 37 sec
> 50 min

11.0.10
29 min 30 sec
> 50 min

12.0.1
26 min 28 sec
> 50 min

Does it expected regression that is start from 11.0.9?
Do you understand why it is slower on Mono?

@sergey-tihon Well it could be a number of things

There may indeed have been a performance regression
Neither AppVeyor nor Travis are reliable performance environment, and their VM configurations may have changed, or you may randomly get allocated a different VM under different load during the day. No one knows.
AppVeyor and Travis are likely use different VMs and core allocations so are not comparable
Mono in general is a bit slower

But still, it's concerning and your numbers are worth investigating.  Can you re-run them a few more times?

Looking at this diff between 11.0.6 and 11.0.9 it's hard to see anything at all suspicious w.r.t. performance.
Was your version of Mono identical in each case?

If you think that this is not FCS issue - feel free to close this issue anytime.
On Travis i use latest Mono, so it is (always the same)

Mono JIT compiler version 4.8.0 (Stable <IP_ADDRESS>0/8f6d0f6 Wed Mar 15 16:18:35 UTC 2017)

I've limited number of tests for FSC to 200 in order to fit TravisCI execution time limitation. Here are the results for FCS 12.0.1:

Execution
AppVeyor
AppVeyor - Paket
Travis (Linux)
Travis  - Paket

Attempt 1
16 min 10 sec
45 seconds
33 min 10 sec
27 seconds

Attempt 2
18 min 24 sec
42 seconds
34 min 18 sec
28 seconds

Attempt 3
16 min 57 sec
50 seconds
32 min 42 sec
21 seconds

Attempt 4
16 min 28 sec
44 seconds
31 min 27 sec
22 seconds

Attempt 5
18 min 57 sec
44 seconds
34 min 10 sec
22 seconds

We can conclude that

Travis (Mono/Linux) is ~2 times slower in average than AppVeyor (may be hardware difference)
Both AppVeyor and Travis are relatively reliable (show similar results between runs)
Paket restore is consistent before build (so there is no network delays before the build start)
Network on Travis is about 2 times faster than on AppVeyor (paket restore is faster)

The same setup (200 FCS tests) with different FCS versions

FCS
FSharp.Core
AppVeyor
Travis (Linux)

11.0.6
4.1.2
16 min 43 sec
33 min 47 sec

11.0.9
4.1.2
16 min 7 sec
33 min 35 sec

11.0.10
4.1.2
16 min 21 sec
34 min 31 sec

12.0.1
4.1.2
16 min 10 sec
33 min 10 sec

12.0.1
<IP_ADDRESS>
16 min 38 sec
34 min 16 sec

Seems that regression is not related with version of FCS.
My bad. Sorry 😐

That's good to know! Thanks for the doing the analysis again!


Refresh token and access token are interchangeable

Version: 0.35.1
If you pass either refresh token or access token to casdoorsdk.ParseJwtToken(token) then token will pass verification and there's no good way to distinct one from another using returned casdoorsdk.Claims. The downside of this is that security risks are higher.
You can use jwt parser directly without casdoor sdk to ensure token type. Refresh tokens have a claim TokenType that equal to refresh-token

@pi-kei Casdoor's refresh token and access token are very similar, this is by design. You can check refreshClaims["TokenType"] = "refresh-token" by yourself to see if it is a refresh token

@hsluoyz To make it clear: Does it make sense to add the claim TokenType to casdoorsdk.Claims?

@pi-kei isn't it already there? See:

@pi-kei didn't you see that the value is already set in the claims before generating the JWT?

@hsluoyz Yes, I see that the value is already set in the claims before generating the JWT. I even write about that when I created this issue.
I am talking about github.com/casdoor/casdoor-go-sdk/casdoorsdk package and it's usecases. How can I use this package to get TokenType claim from JWT? Using this package I can get every other claim from JWT except TokenType.

@pi-kei if you use: https://jwt.io/ to parse the token, is TokenType there already?

@hsluoyz Yes, it is there. But the question is how to get it in the code using this sdk package. And the answer is I can't until it will be modified. Should it be modified?

@pi-kei yes, can you make a PR?

@hsluoyz Sure.


Add backend validation for add team member form

Issue #:
https://github.com/bcgov/met-public/issues/1151
Description of changes:

Implement flag to get users without roles
Add validation  for groups in backend
set react hook form error to display the backend validation

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the met-public license (Apache 2.0).

Codecov Report

Merging #1179 (9e85d0a) into main (4902c9d) will decrease coverage by 0.21%.
The diff coverage is 58.10%.

:exclamation: Current head 9e85d0a differs from pull request most recent head c7e70fb. Consider uploading reports for the commit c7e70fb to get more accurate results

@@            Coverage Diff             @@
##             main    #1179      +/-   ##
==========================================
- Coverage   73.77%   73.57%   -0.21%
==========================================
  Files         266      266
  Lines        7481     7518      +37
  Branches      508      514       +6
==========================================
+ Hits         5519     5531      +12
- Misses       1885     1908      +23
- Partials       77       79       +2

Flag
Coverage Δ

metapi
79.39% <63.82%> (-0.31%)
:arrow_down:

metweb
67.76% <48.14%> (-0.11%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...s/userManagement/listing/UserManagementContext.tsx
80.64% <ø> (+2.52%)
:arrow_up:

met-web/src/services/userService/api/index.tsx
45.45% <ø> (ø)

met-api/src/met_api/services/membership_service.py
40.47% <30.00%> (-6.75%)
:arrow_down:

met-api/src/met_api/resources/user.py
82.45% <33.33%> (-2.73%)
:arrow_down:

...components/userManagement/listing/AddUserModal.tsx
44.14% <41.66%> (+0.45%)
:arrow_up:

...ent/form/EngagementFormTabs/AddTeamMemberModal.tsx
51.21% <53.33%> (-1.64%)
:arrow_down:

met-api/src/met_api/services/user_service.py
75.00% <69.23%> (-6.12%)
:arrow_down:

met-api/src/met_api/models/membership.py
84.61% <100.00%> (+0.61%)
:arrow_up:

met-api/src/met_api/models/user.py
92.06% <100.00%> (+1.29%)
:arrow_up:

met-api/src/met_api/schemas/memberships.py
100.00% <100.00%> (ø)

... and 3 more


potential bug loading a pruned model with no masks

load_pruned_model in the cofi_utils file seems to take a model as a first argument, however load_model(..) calls load_pruned_model by passing a string. In this case the program crashes as the string doesn't have a "config" property for example

I also noticed that actually load_pruned_model sometimes is called with model_path, and model_class arguments and sometimes is called with model and weights arguments. Maybe this was changed but some things were not updated?

Hi,
Sorry for the late reply!!
For the first issue, I think load_model is always used for loading a model along with the masks zs. If zs is not provided, as you said, the program crashes. I updated the function to remove the second path, which is never triggered in the codebase. Sorry for the confusion!
For the second issue, yes! I think only in test.py file, load_pruned_model is called with model_path and model_class, which is a deprecated use.
Let me know if you have more questions and I am happy to help!


Green, flickering bar over camera preview

Steps to Reproduce

Make an app exactly like this sample: https://pub.dartlang.org/packages/camera
Deploy the app to a Huawei Y6 2018 with Android 8.0.0
See this:

Logs
Launching lib\main.dart on ATU L21 in debug mode...
Built build\app\outputs\apk\debug\app-debug.apk.
I/CameraManagerGlobal(30810): Connecting to camera service
I/CameraManagerGlobal(30810): single back camera, do not need hide aux camera, device number: 0
I/CameraManagerGlobal(30810): single back camera, do not need hide aux camera, device number: 1
I/CameraManagerGlobal(30810): single back camera, do not need hide aux camera, device number: 2
I/CameraManagerGlobal(30810): single back camera, do not need hide aux camera, device number: 2
E/BufferQueueProducer(30810): [] Can not get hwsched service
E/BufferQueueProducer(30810): [] Can not get hwsched service
I/PermissionManager(30810): camera remind result:true
I/CameraManagerGlobal(30810): single back camera, do not need hide aux camera, device number: 2
I/CameraManagerGlobal(30810): single back camera, do not need hide aux camera, device number: 2
I/zygote64(30810): Do partial code cache collection, code=27KB, data=27KB
I/zygote64(30810): After code cache collection, code=20KB, data=24KB
I/zygote64(30810): Increasing code cache capacity to 128KB

   info - Duplicate import - lib\main.dart:3:8 - duplicate_import
   info - Unused import: 'package:fluff/main.dart' - test\widget_test.dart:11:8 - unused_import
  error - The constructor returns type 'dynamic' that isn't of expected type 'Widget' - test\widget_test.dart:16:29 -
         strong_mode_invalid_cast_new_expr
  error - Undefined class 'MyApp' - test\widget_test.dart:16:33 - undefined_class

[√] Flutter (Channel beta, v0.10.2, on Microsoft Windows [Version 10.0.17134.345], locale da-DK)
    • Flutter version 0.10.2 at c:\workspace\flutter
    • Framework revision d8cbb80206 (3 weeks ago), 2018-10-26 01:30:21 -0400
    • Engine revision 6c2ade9fa2
    • Dart version 2.1.0-dev.8.0.flutter-bf26f760b1

[√] Android toolchain - develop for Android devices (Android SDK 28.0.3)
    • Android SDK at C:\Users\asgerhallas\AppData\Local\Android\sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
    • All Android licenses accepted.

[√] Android Studio (version 3.2)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin version 30.0.1
    • Dart plugin version 181.5656
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)

[√] VS Code, 64-bit edition (version 1.28.2)
    • VS Code at C:\Program Files\Microsoft VS Code
    • Flutter extension version 2.20.0

[√] Connected device (1 available)
    • ATU L21 • GSM9X18A07G03187 • android-arm64 • Android 8.0.0 (API 26)

• No issues found!

Deploy the app to a Huawei Y6 2018 with Android 8.0.0

does this mean on other devices (and emulator) this doesn't reproduce?

@zoechi I am meeting the same problem with  xiaomi 4 Android 6.0. By the way, the same problem with the video_player plugin. (Flutter Channel dev, v0.11.3)

same issue in video player , xiaomi 4 Android 6.0

@zoechi I couldn't get access to another device, sorry.

@asgerhallas that's ok, it just wasn't clear to me if you meant to point out if you see this only on the mentioned device or if this is the only device you tried it with. Thanks for the update.

@songfei do you mean this issue is not present in Flutter 0.7.0?
I have experienced a similar issue on an LG-G2 (D802) Device (https://www.lg.com/uk/mobile-phones/lg-D802) running Android KitKat 4.4.2 when using the video_player. I'll try to get my hands on a screenshot.

@recastrodiaz   , YES,  I confirmed it,
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel unknown, v0.7.3, on Mac OS X 10.14 18A391, locale zh-Hans-CN)
[✓] Android toolchain - develop for Android devices (Android SDK 27.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
[✓] Android Studio (version 3.2)
[✓] VS Code (version 1.29.0)
[✓] Connected devices (2 available)
video_player and Camera is OK， same code in Flutter 0.11.3 is NOT OK.

Thanks @songfei
From this comment in https://github.com/flutter/flutter/issues/24402#issuecomment-439237830, it would seem the regression started in v0.10.2  Would you mind checking if v0.10.1 (https://github.com/flutter/flutter/releases/tag/v0.10.1) works as expected? This might help the flutter team confirm where the regression started. Thanks!

@asgerhallas this should be fixed in the latest release (v0.11.10, available on the dev and beta channels).  Can you please run flutter upgrade and then verify to see if it's fixed for you?

Flutter 0.11.11-pre.2 • channel master • https://github.com/flutter/flutter.git
Framework • revision a2a1311a1a (5 hours ago) • 2018-11-26 19:43:10 -0800
Engine • revision 84e3b137a1
Tools • Dart 2.1.0 (build 2.1.0-dev.9.4 f9ebf21297)
XiaoMi 4 , Android 6.0
green block is still exist.

facing the same issue. 0.11.10 didn't solve it

Ok, second take... can you please try v0.11.11 (on the latest dev and beta channels - available via flutter upgrade)?  Thanks!

Ok, v0.11.13 is live on the dev and beta channels, with the latest attempt at fixing this.

Ok, v0.11.13 is live on the dev and beta channels, with the latest attempt at fixing this.
It is OK on my device. The green block not occur any more.

Great!  I'm going to close this issue.  Feel free to reopen it if you see this happen again

Hãy giúp tôi sửa cái này
E/BufferQueueProducer: [] [hwsched]Can not get hwsched service
E/BufferQueueProducer: [] [hwsched] Sched_config_cust.xml is not exit or file is invalid!
[] [hwsched] LoadFile error, path:(null), error code:3

Hãy giúp tôi sửa cái này
E/BufferQueueProducer: [] [hwsched]Can not get hwsched service
E/BufferQueueProducer: [] [hwsched] Sched_config_cust.xml is not exit or file is invalid!
[] [hwsched] LoadFile error, path:(null), error code:3

same issue

i have problem when i try to build final apk or using --release parameter


These options need to be set for CDAP HA

This is not an HA configuration, just the options necessary to set for HA.

LGTM


Enhancement: Adopt area for mass_players

The mass_players could automatically adopt the area assigned to the source players for the mass versions

It would be nice if it could copy the (active) icon of the source as well.
I'm not using the default icons for all of my sources and I'm seeing different icons on the mass versions.
OTOH ... some kind of visial indicator that this is the mass version when you're chosing not to hide them would be nice to have.

Not applicable for MA2 approach


clustering function for features

When writing tutorials, I find myself defining the same clustering function in several notebooks.
def cluster_features(features: pd.DataFrame, like=None):
    """Calculate leiden clustering of features.

    Specify filter of features using `like`.
    """
    # filter features
    if like is not None:
        features = features.filter(like=like)
    # create temporary adata to calculate the clustering
    adata = ad.AnnData(features)
    # adata.var_names_make_unique()
    # important - feature values are not scaled, so need to scale them before PCA
    sc.pp.scale(adata)
    # calculate leiden clustering
    sc.pp.pca(adata, n_comps=min(10, features.shape[1] - 1))
    sc.pp.neighbors(adata)
    sc.tl.leiden(adata)

    return adata.obs["leiden"]

This essentially does scaling+PCA+neighbors+leiden on a set of features.
I was wondering if we should include this in squidpy as a convenience function (maybe made a bit more general)? Or should we rather leave these sort of functions outside of squidpy? Is there a solution that I can avoid defining the same function in several notebooks? @giovp

it would be cool to have a multiplex partition based on layers/obsm
see this https://github.com/theislab/scanpy/issues/1818


Install popmon at the beginning of notebooks

This provides a more seamless experience when opening notebooks in e.g. Google Colab without needing to explicitly add !pip install popmon code cell in the notebook (since popmon is not installed in Colab by default like pandas or numpy).

Explicitly stating the import is a nice addition.

!pip install popmon will not generally work. The correct way of installing packages is import sys !{sys.executable} -m pip install popmon. See: https://jakevdp.github.io/blog/2017/12/05/installing-python-packages-from-jupyter/
Both this PR and #46 pin the isort version. It should be mentioned that we do this to ensure the CI of popmon stays working until we're updated the code to work with isort 5.0.0 (see https://github.com/timothycrosley/isort/releases/tag/5.0.0).


[rom-e2e] rom_e2e_watchdog_reconfig

Testpoint name: rom_e2e_watchdog_reconfig
Contact person: @alphan
Description: Verify that ROM reconfigures the watchdog timer correctly.

Should be disabled in TEST and RMA.
In DEV, PROD, and PROD_END, the threshold should be
OWNER_SW_CFG_ROM_WATCHDOG_BITE_THRESHOLD_CYCLES if greater than or equal to
kWatchdogMinThreshold, disabled otherwise.

Note: for this, we can either write a simple self-checking test or use the ujson library (the former is simpler but the latter is more scalable). See chip_specific_startup.

Here are some notes on my understanding so far (end of Monday). @alphan I'd appreciate if you glance over this and let me know if I'm on the right track.
I should create OTP images with varying values of OWNER_SW_CFG_ROM_WATCHDOG_BITE_THRESHOLD_CYCLES. This particular value is a plain 32-bit unsigned integer, with no hardening or fixed set of permissible values.
I should create a test image in sw/device/silicon_creater/rom/e2e which checks the values of watchdog registers against expected values.
It's still unclear to me how the test image should determine the expected values of the watchdog registers. Is this where ujson comes in -- should those be fed in via UART as JSON?
Build system wise, is the plan to compile the test image once and run it multiple times with different OTP images spliced in? Or maybe the test image is compiled multiple times, once per OTP image?

Here are some notes on my understanding so far (end of Monday). @alphan I'd appreciate if you glance over this and let me know if I'm on the right track.
I should create OTP images with varying values of OWNER_SW_CFG_ROM_WATCHDOG_BITE_THRESHOLD_CYCLES. This particular value is a plain 32-bit unsigned integer, with no hardening or fixed set of permissible values.

Correct, I think 2 different OWNER_SW_CFG_ROM_WATCHDOG_BITE_THRESHOLD_CYCLES values would be good:

OWNER_SW_CFG_ROM_WATCHDOG_BITE_THRESHOLD_CYCLES = 0, i.e. kWatchdogMinThreshold - 1. Watchdog should be disabled in all life cycle states, including DEV, PROD, PROD_END.
OWNER_SW_CFG_ROM_WATCHDOG_BITE_THRESHOLD_CYCLES = 0xC350, i.e. 50k cycles, 0.25 s assuming a clock freq of 200 kHz. Watchdog should be disabled in TEST_UNLOCKEDN and RMA and enabled in DEV, PROD, PROD_END.

We should test each case in all 5 life cycle states, i.e. TEST_UNLOCKED0, DEV, PROD, PROD_END, and RMA, so 10 images in total.

I should create a test image in sw/device/silicon_creater/rom/e2e which checks the values of watchdog registers against expected values.
It's still unclear to me how the test image should determine the expected values of the watchdog registers. Is this where ujson comes in -- should those be fed in via UART as JSON?
Build system wise, is the plan to compile the test image once and run it multiple times with different OTP images spliced in? Or maybe the test image is compiled multiple times, once per OTP image?

For the on-target program, you can either

Write a simple program (please see the examples in sw/device/silicon_creator/rom/e2e) that specifically checks for the register values -- these values can be hardcoded to match the values in the BUILD file,
Use the ujson lib, but this would also require an off-target rust component, or
(My preference, simplest) Use empty_test.c with a custom message like here. You can set this custom message to print the registers you are interested in and set the exit_success and exit_failure arguments accordingly. Please see top_earlgrey.h for the base address of the watchdog timer and aon_timer_regs.h (this one is autogenerated during build) for the register offsets. You can use the abs_mmio library to read these registers.


Fix the non-deterministic generation order

Related to #225.
Before this commit, the integration tests sometimes fails, like so:
field has incomplete type 'struct swift_bridge$tuple$I32ResultTestOpaqueRustTypeString'

This PR fixes the issue.

Test session results, code coverage, and logs:
	Command CompileSwift failed with a nonzero exit code
	/Users/runner/Library/Developer/Xcode/DerivedData/SwiftRustIntegrationTestRunner-eunjzlpbulmkoedlbxmntxkdbksm/Logs/Test/Run-SwiftRustIntegrationTestRunner-2023.06.17_06-29-48-+0000.xcresult
	Cannot find 'swift_reflect_already_declared_struct' in scope
	Testing cancelled because the build failed.

I rebased #226 into this branch, and then I fixed the immediately above issue.

@chinedufn
I guess that I addressed your review. If we need to support Result<tuple, tuple>, we couldn't yet merge this PR into master.


Revert "IRGen: generate static arrays in read-only data sections."

This reverts commit aca0d8358b52ba5b4f20463a26bc4b579c3f4312.
I need to fix a problem before this can actually land.

@swift-ci smoke test and merge


Use Vitepress for serving all documentation

This PR adds vitepress to the docs folder to consolidate all documentation of the project into one central folder. Additionally, a GitHub Action is created to deploy the site to GitHub Pages.

LGTM, I will merge this to see if the domain already works.


janet: 1.5.1 -> 1.6.0

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/janet/versions.
meta.description for janet is: '"Janet programming language"'.
meta.homepage for janet is: '"https://janet-lang.org/"
Release on GitHub
Compare changes on GitHub

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.6.0 with grep in /nix/store/wqn54qm1c6ixsfhazm25152mav91zify-janet-1.6.0

directory tree listing: https://gist.github.com/72437e87544caa4f75b9dc4c3ee12e10

du listing: https://gist.github.com/1ce10e3424b645cbaf896b6586570065

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)
1 package rebuild(s)
1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
janet

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/wqn54qm1c6ixsfhazm25152mav91zify-janet-1.6.0 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A janet https://github.com/r-ryantm/nixpkgs/archive/7e9851475ac5186cc70bc28e6342508586bf2008.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/wqn54qm1c6ixsfhazm25152mav91zify-janet-1.6.0
ls -la /nix/store/wqn54qm1c6ixsfhazm25152mav91zify-janet-1.6.0/bin

cc @andrewchambers for testing.

lgtm.


[SPARK-2016][WebUI]RDD partition table pagination for the RDD Page

Add pagination for the RDD page to avoid unresponsive UI when the number of the RDD partitions is large.

Merged build triggered.

Merged build started.

Test build #38531 has started for   PR 7692 at commit 612c18c.

Test build #38531 timed out     for PR 7692 at commit 612c18c     after a configured wait of 175m.

Merged build finished. Test FAILed.

retest this please

Merged build triggered.

Merged build started.

Merged build triggered.

Merged build started.

Test build #126 has started for   PR 7692 at commit 612c18c.

Merged build finished. Test FAILed.

Test build #126 has finished for   PR 7692 at commit 612c18c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

retest this please

Merged build triggered.

Merged build started.

Merged build triggered.

Merged build started.

Test build #131 has started for   PR 7692 at commit 612c18c.

Test build #38671 has started for   PR 7692 at commit 612c18c.

Test build #131 has finished for   PR 7692 at commit 612c18c.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test build #38671 has finished for   PR 7692 at commit 612c18c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

@zsxwing could you review this?

@carsonwang would you mind posting a screenshot of what this looks like before and after?

Screenshots attached.

Bump.  @zsxwing (or maybe @sarutak if you have cycles), let's see about pulling this in for 1.5?

O.K, I'll take a look soon.

Merged build triggered.

Merged build started.

Thanks @sarutak . The style issues are fixed.

Test build #39636 has started for   PR 7692 at commit 03c7168.

Works well in my machine.

Test build #39636 has finished for   PR 7692 at commit 03c7168.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

LGTM

O.K. Merging into master and branch-1.5. Thanks @carsonwang !


[Site support request] stream.me

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.09.18. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.09.18

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[x] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
youtube-dl -v https://www.stream.me/archive/narcissa/narcissas-stream/kl46vDd62A
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.stream.me/archive/narcissa/narcissas-stream/kl46vDd62A']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2018.09.18
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.16299
[debug] exe versions: ffmpeg N-87130-g2b9fd15, ffprobe N-87130-g2b9fd15
[debug] Proxy map: {}
[generic] kl46vDd62A: Requesting header
WARNING: Falling back on generic information extractor.
[generic] kl46vDd62A: Downloading webpage
[generic] kl46vDd62A: Extracting information
ERROR: Unsupported URL: https://www.stream.me/archive/narcissa/narcissas-stream/kl46vDd62A
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpipjeparb\build\youtube_dl\YoutubeDL.py", line
92, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpipjeparb\build\youtube_dl\extractor\common.py"
 line 507, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpipjeparb\build\youtube_dl\extractor\generic.py
, line 3298, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.stream.me/archive/narcissa/narcissas-stream/kl46vDd
2A

<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.stream.me/archive/narcissa/narcissas-stream/kl46vDd62A
Single video: https://www.stream.me/archive/narcissa/narcissas-stream/kl46vDd62A?t=03:36:33
Playlist: None

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
Stream.me is a gaming video streaming site which currently is not listed in the supported URLs list. There is a large amount of content on there so I think it would be good to add support for this site.

This one is easy. From the page, look in the <script> tags for the line defining __context. In that JSON there are several m3u8 URIs under the key href and there is a JSON URI as well. The JSON URI is what the desktop will follow first to get m3u8 URIs. URL: https://video.creekcdn.com/mediasvc/v1/playlist/aiw/v/0/res/3/kl46vDd62A.json

There are two formats for streaming and one format for the MP4.
The single video link would be supported but it would just do the same thing as the one without a timestamp. For splitting the video from that point you should use something like ffmpeg.


Fix warning messages about config.json when the base model_id is local.

It should be possible for the user to specify a local directory as the base model in the library.
However, currently the library only checks for remote presence of config.json, and fails to check the actual config.json when using a local repo.
This PR adds check for a local model_id and fixes the behavior.

Thanks for the PR, what you describe sounds reasonable. Do you have a small example where this change would apply? Ideally, we can use that to create a unit test.

Sure, a simple example is to create a LoRA adapter for a local base model and saving it.
for example, create a PeftModel for a local snapshot of mistralai/Mistral-7B-v0.1, and saving the adapter issues a warning saying
warnings.warn(
    f"Could not find a config file in {model_id} - will assume that the vocabulary was not modified."
)

And the configuration is not checked therefore.

from transformers import AutoModelForCausalLM
from peft import LoraConfig, PeftModel
from peft import prepare_model_for_kbit_training, get_peft_model

local_dir = 'path/to/model'

base_model = AutoModelForCausalLM.from_pretrained(local_dir)
peft_config = LoraConfig(
    lora_alpha=16,
    lora_dropout=0.1,
    r=64,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj","gate_proj"]
)
peft_model = get_peft_model(base_model, peft_config)
peft_model.save_pretrained("test")

Is this sufficient? @BenjaminBossan

Nice, thanks for providing the example. I used it to create a test:
class TestLocalModel:
    def test_local_model_saving_no_warning(self, recwarn, tmp_path):
        model_id = "facebook/opt-125m"
        model = AutoModelForCausalLM.from_pretrained(model_id)
        local_dir = tmp_path / model_id
        model.save_pretrained(local_dir)
        del model

        base_model = AutoModelForCausalLM.from_pretrained(local_dir)
        peft_config = LoraConfig(
            lora_alpha=16,
            lora_dropout=0.1,
            r=64,
            bias="none",
            task_type="CAUSAL_LM",
            target_modules=["q_proj", "k_proj", "v_proj", "o_proj","gate_proj"]
        )
        peft_model = get_peft_model(base_model, peft_config)
        peft_model.save_pretrained(local_dir)

        for warning in recwarn.list:
            assert "Could not find a config file" not in warning.message.args[0]

We could for instance put it into tests/test_hub_features.py, WDYT? Running it locally on main, it currently fails, but on your branch, it should pass.

Certainly, I followed the syntax in testing_common.py and created a test unit for the issue.
Maybe some further checks? @BenjaminBossan

Certainly, I followed the syntax in testing_common.py and created a test unit for the issue.

The way you added the test, it's not executed. You would have to add corresponding methods that call this method in test_decoder_models.py, test_encoder_decoder_models.py, etc. But this is overkill, we don't really need to check this with all kind of different model architectures. Instead, as I suggested earlier, just add this test to tests/test_hub_features.py and it should be good. Let's also add a comment that explains why we need this test.

The test case is fixed as advised and comments are added to explain the issue.
Please review the changes, thanks.

@elementary-particle Thanks for the update. Could you please run make style?

The merge conflict is resolved.


EID-1721 Pass countrySignedResponseContainer to MSA for user account creation

When the MSA doesn't find a user we enter the user account creation flow. In the case of unsigned assertions, we need to send the data from the countrySignedResponseContainer object as an assertion in the attribute query, rather than the original encrypted assertion.
Work has already been done to ensure that this object is available in the correct states, as well as the work to include the unsigned response assertion rather than the encrypted assertion (in HubEidasAttributeQueryRequestToSamlAttributeQueryTransformer).

If it helps with the 2nd review, the new tests from https://github.com/alphagov/verify-acceptance-tests/pull/73 pass when this PR is applied.


Added callback support

Improved readability of much of the code.
Added support for a callback that is triggered after package installation is complete.
Updated readme to reflect these changes.

:+1:

Finally I've put some time on this, so now it's released in v1.1.0


position error when inside form

Can you provide a jsfiddle example or so?
https://jsfiddle.net/


Help Wanted

I'd like to humbly acknowledge that I have not actively maintained this project for some time. I don't expect my time commitments to change any time soon, so I'd like to ask for some interested parties to maintain and eventually take ownership.
If you have the time and patience to maintain awsgi, please use this issue to volunteer and discuss. Thanks.

Like most people, I'm also kind of busy, but I would like to help to maintain this project ;)

@slank Recently AWS launched Lambda function URLs which uses Amazon API Gateway payload format version 2.0
Fortunately, @c3ko has already developed the new format and a PR (#58) is waiting for merge. I'm my opinion, the new code does not break the format version 1.0. It would be nice if you merge that PR or give someone rights in the repo to do it (me for example :))


Many Win32-style enums would be better promoted to module-scope constants

Compare the usability of WM_QUIT with D3D12_RESOURCE_BARRIER_TYPE_TRANSITION:
let message = WM_QUIT;
let barrier_type = D3D12_RESOURCE_BARRIER_TYPE::D3D12_RESOURCE_BARRIER_TYPE_TRANSITION;

The barrier type one is much more awkward to use.  IMO it would be much more convenient to be able to write:
let barrier_type = D3D12_RESOURCE_BARRIER_TYPE_TRANSITION;

As all the naming conventions were picked with the assumption that no further qualification would be required.
Alternatively, the projection could try and figure out better names:
let barrier_type = D3D12_RESOURCE_BARRIER_TYPE::TRANSITION;

Yes, I think that's a good idea. Enums from WinRT APIs tend to already provide scoped names, but Win32 enums are based on C-style enums, as you know. I could make them module-scoped but still preserve their type.

Blocked on https://github.com/microsoft/win32metadata/issues/442 and https://github.com/microsoft/win32metadata/issues/443

If D3D12_RESOURCE_BARRIER_TYPE exists as a type used in structs already, I'd prefer simplified variants:
let barrier_type = D3D12_RESOURCE_BARRIER_TYPE::TRANSITION;

Module-scope constants could simply reference those variants I guess?

I was going to propose something like this originally, but there's a couple of drawbacks that it'd be interesting to get your take on.  Firstly, it diverges from what these things are called in other tools or languages.  For example, being able to search for D3D12_RESOURCE_BARRIER_TYPE_TRANSITION and get hits in C++ and Rust is pretty valuable.  Tools such as PIX or the debug layer display the Win32 ABI names for these types, so as it diverges we'll be asking developers to make the translation in their head.
The second issue, which is more of an implementation detail, is that there isn't always a mechanical way to reliably generate the simplified names.  This can be addressed with enough determination :)

Yes, searchability and portability is very valuable. Having already tried it, there are definitely a lot of edge cases that don't work well when you try to shorten the names. The metadata also doesn't deal with these constants consistently. Some are enums, some are constants, so in the end I'm also looking for more consistency in how this is presented in Rust. Scoping these also means that all of them must be generated as bindings. This is very problematic if you refer to an API that say uses WIN32_ERROR and suddenly your generated windows.rs file has an extra few thousand lines of code.

Yes, renaming stuff is hard and I ultimately went with preserving (#646) the original names everywhere (the win32 metadata project has taken the same originalist approach). The challenge is to make the bindings as rich/useful as possible while letting the developer limit/filter as much as possible to keep build time/complexity under control. That's why I'd like fine-grained filtering and not force huge chunks of code to be generated. The flip side is having pre-generated bindings that include everything (#432) but we're a ways from getting that to work.

@kennykerr That's only casing, not stripping/deduplicating enum variant names, right? I agree on keeping the names similar, using original casing wherever possible. Tools like rust-analyzer do a good job helping one find symbols, though.
Unless I misunderstood you can still have fine-grained control over generated code if enum variants have the redundant part of the name stripped off? Again, I'm okay with module-level D3D12_RESOURCE_BARRIER_TYPE_TRANSITION or associated enum variant D3D12_RESOURCE_BARRIER_TYPE::TRANSITION (no need to strip off D3D12 even if implied by the module, I guess?), but D3D12_RESOURCE_BARRIER_TYPE::D3D12_RESOURCE_BARRIER_TYPE_TRANSITION is too verbose for me.

One downside of this is that when generating         Windows::Win32::System::Diagnostics::Debug::WIN32_ERROR, none of the associated constants are generated anymore. The only way to get all of them is to generate Windows::Win32::System::Diagnostics::Debug::*, which is a bit more than necessary.

@roblabla yes that's a tradeoff I'm still wrestling with.
@mikebattista are there plans to reduce the size of the Windows::Win32::System::Diagnostics::Debug namespace or move the common WIN32_ERROR type to a less cluttered namespace?

I think when generating a scoped enum, all its module-scope constants should be generated as well.

The trouble is that it is often the case that the particular error code isn't needed where the value is simply checked for success/failure and passed on to some Result object for propagation. Such large enums can really clutter up the the generated bindings. Generating all of the WIN32_ERROR constants amounts to over 3,000 extra lines of code.

@kennykerr we could move WIN32_ERROR and GetLastError to the Foundation namespace. Would that work?

Hmm. While this is true for WIN32_ERROR, there are many other enums that are rather small and currently live in very cluttered namespaces too (VER_FLAGS above). Those would have minimal impact in being always generated when the type is present.
Another avenue that could fix it would be giving the ability to have a prefix on wildcard, e.g. build!(        Windows::Win32::System::WindowsProgramming::VER_*). It's a bit of a bandaid though.

In the latest metadata, VER_FLAGS is currently in a much more scoped Windows.Win32.System.SystemInformation namespace.
The SystemServices and WindowsProgramming namespaces won't exist anymore once we're done refactoring.

Another option is to treat the scoped enums as follows: if the build macro only includes specific constants like ERROR_PATH_NOT_FOUND then only the listed constants are defined, but if the build macro includes the enum type itself like WIN32_ERROR then all of its enclosing constants are included. How does that sound? I've been trying to keep as close to Rust syntax and semantics as possible but this sounds like it could be useful.

Thanks @mikebattista, I'm not sure whether moving it would help. I am curious what the difference is between Windows::Win32::Foundation and Windows::Win32::System but that's a different topic. 😊

There's nothing in Windows.Win32.System so I'm not sure what you're referring to there.

I mean how do you decide whether something is in Foundation vs System::* or somewhere else? With WinRT, Windows.Foundation is for things that are core to WinRT (e.g. collections and async support) without which other APIs could not be defined.

There is a need to put common shared types like HANDLE, HWND, string types, etc. in a shared namespace. Foundation is currently the plan for that to align with Windows.Foundation. The expectation would be that Foundation would always be included like Windows.Foundation for WinRT or System for .NET. Given that WIN32_ERROR and GetLastError are used with many APIs, it's a possible candidate for that Foundation namespace. The goal is to improve discovery of the basic things you need and minimize needing to add seemingly obscure namespace references just to use APIs in some other completely unrelated namespace.
System.* is for system-level APIs that are mostly self-contained and not as commonly required as Foundation APIs would be: System.Memory, System.Power, System.Threading, etc. If you're using System.Memory you're not necessarily also using System.Power.  If you think APIs like CreateFileMapping and VirtualAlloc should be in Foundation, that would be a separate discussion.

Ah yes,  I remember us chatting about a common namespace that others could depend on to keep circular references to a minimum.

So yes, I'd err on the side of keeping it to a minimum and GetLastError doesn't seem like it needs to be there to reduce dependencies.

Thanks. It's a work in progress so happy to iterate based on feedback on where APIs belong.
For sure, though, I want to get rid of SystemServices and WindowsProgramming.


ambr fails to match/replace some match with -

I tried to use ambr to replace *  with -  in some files and I got
$ ambr '* ' '- ' .
error: Found argument '- ' which wasn't expected, or isn't valid in this context

USAGE:
    ambr [FLAGS] [OPTIONS] <KEYWORD> <REPLACEMENT> [PATHS]...

For more information try --help

After I dug a little bit, I found that structop fails to parse positional arguments that start with -  (dash + space), hence ambr returns that error if something starting with -  is passed to <KEYWORD> or <REPLACEMENT>.

Revolved on by https://github.com/TeXitoi/structopt/issues/534


Fix Javadoc errors in module client/rest

Description
Fix Javadoc comments to fix Javadoc errors as described in issue https://github.com/opensearch-project/OpenSearch/issues/221.
Issues Resolved
Fixes Javadoc errors in the client/rest module.
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[X] New functionality has javadoc added

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@adnapibar Thanks for your review, I've made the suggested updates.

Noticed it was merged into the javadoc branch. Was that intentional? This needs to go into main, doesn't it? What are the labels we need for this @adnapibar?

Yes this was intentional but I agree this needs to go in main as well. The problem is, missingJavadoc check is only available on javadoc branch. It is not added to main because of issue #449 . Once that is fixed, we can merge javadoc branch to main (while keeping missingJavadoc checks excluded from gradlew check) which will also merge all incoming javadocs. I think we should prioritize the fix as it gets harder to maintain a feature branch. However, until that happens, we have below options

Merge and keep new incoming javadocs in javadoc branch.Fix #449 asap, and then rebase and merge javadoc branch commits into main.
Merge new incoming javadocs in javadoc branch and port it to main branch as well. (But then it becomes a  backporting nightmare as there are other branches as well)
Merge incoming javadocs to main branch (and backport to 1.x as other PRs) and let javadoc branch pull from main. However the problem with merging to main is there is no check available to validate the new javadocs. The checks are only available in javadoc branch. So if we go with this, contributor has to first verify against javadoc branch, then push to main and 1.x branch. (javadoc branch can be maintained by maintainers)
@dblock i'm fine with both 1 & 3, but 3 seems to have slightly more friction. WDYT?

This is the second time I've seen this precommit failure:
* What went wrong:
A problem was found with the configuration of task ':client:benchmark:validateNebulaPom' (type 'PomValidationTask').
> File '/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Precommit/search/client/benchmark/build/publications/nebula/pom-default.xml' specified for property 'pomFile' does not exist.

We shouldn't get into a habit of merging w/ failing precommit. Is this a CI config issue? Red herring?

This is the second time I've seen this precommit failure:
* What went wrong:
A problem was found with the configuration of task ':client:benchmark:validateNebulaPom' (type 'PomValidationTask').
> File '/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Precommit/search/client/benchmark/build/publications/nebula/pom-default.xml' specified for property 'pomFile' does not exist.

We shouldn't get into a habit of merging w/ failing precommit. Is this a CI config issue? Red herring?

@nknize the precommit fails because of #449 . That's why we are keeping these changes in separate feature (javadoc) branch, until we fix that. Good news though, I have figured a fix for #449 , and once that's merged, the precommit should start passing and then we can merge javadoc branch to main


[SPARK-19223][SQL][PySpark] Fix InputFileBlockHolder for datasources which are based on HadoopRDD or NewHadoopRDD

What changes were proposed in this pull request?
For some datasources which are based on HadoopRDD or NewHadoopRDD, such as spark-xml, InputFileBlockHolder doesn't work with Python UDF.
The method to reproduce it is, running the following codes with bin/pyspark --packages com.databricks:spark-xml_2.11:0.4.1:
from pyspark.sql.functions import udf,input_file_name
from pyspark.sql.types import StringType
from pyspark.sql import SparkSession

def filename(path):
    return path

session = SparkSession.builder.appName('APP').getOrCreate()

session.udf.register('sameText',filename)
sameText = udf(filename, StringType())

df = session.read.format('xml').load('a.xml', rowTag='root').select('*',input_file_name().alias('file'))
df.select('file').show() # works
df.select(sameText(df['file'])).show()   # returns empty content

The issue is because in HadoopRDD and NewHadoopRDD we set the file block's info in InputFileBlockHolder before the returned iterator begins consuming. InputFileBlockHolder will record this info into thread local variable. When running Python UDF in batch, we set up another thread to consume the iterator from child plan's output rdd, so we can't read the info back in another thread.
To fix this, we have to set the info in InputFileBlockHolder after the iterator begins consuming. So the info can be read in correct thread.
How was this patch tested?
Manual test with above example codes for spark-xml package on pyspark: bin/pyspark --packages com.databricks:spark-xml_2.11:0.4.1.
Please review http://spark.apache.org/contributing.html before opening a pull request.

Test build #71366 has started for PR 16585 at commit 5fd215f.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71366/
Test FAILed.

Merged build finished. Test FAILed.

retest this please.

Test build #71368 has started for PR 16585 at commit 5fd215f.

Test build #71368 has finished for PR 16585 at commit 5fd215f.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71368/
Test FAILed.

retest this please.

Test build #71370 has started for PR 16585 at commit 5fd215f.

Test build #71370 has finished for PR 16585 at commit 5fd215f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71370/
Test PASSed.

Merged build finished. Test PASSed.

ping @rxin @cloud-fan

should the proper fix be the python thread transfers the proper information over?

@rxin Thanks for looking at this. I think the simplest way to transfer the info is using InheritableThreadLocal to replace ThreadLocal in InputFileBlockHolder. As I tested, it works. What do you think? It is ok for you?

SGTM

@cloud-fan SGTM is for current approach or InheritableThreadLocal?

InheritableThreadLocal

Test build #71474 has started for PR 16585 at commit e2d872c.

Test build #71475 has started for PR 16585 at commit 1563e03.

BTW please add a test case for this. Thanks.

Test build #71474 has finished for PR 16585 at commit e2d872c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #71475 has finished for PR 16585 at commit 1563e03.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71474/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71475/
Test PASSed.

Merged build finished. Test PASSed.

Test build #71509 has started for PR 16585 at commit b1bfc50.

Test build #71510 has started for PR 16585 at commit 157f70f.

Test build #71511 has started for PR 16585 at commit d7c05d2.

Test build #71511 has finished for PR 16585 at commit d7c05d2.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class InputFileBlockHolderSuite extends SparkFunSuite

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71511/
Test FAILed.

Test build #71509 has finished for PR 16585 at commit b1bfc50.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class InputFileBlockHolderSuite extends SparkFunSuite with LocalSparkContext

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71509/
Test PASSed.

Test build #71510 has finished for PR 16585 at commit 157f70f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class InputFileBlockHolderSuite extends SparkFunSuite with LocalSparkContext

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71510/
Test PASSed.

Test build #71540 has started for PR 16585 at commit 8380617.

Test build #71540 has finished for PR 16585 at commit 8380617.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class InputFileBlockHolderSuite extends SparkFunSuite

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71540/
Test FAILed.

Test build #71559 has started for PR 16585 at commit 3444499.

Test build #71559 has finished for PR 16585 at commit 3444499.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71559/
Test FAILed.

Merged build finished. Test FAILed.

Test build #71561 has started for PR 16585 at commit 2ce65cb.

Test build #71561 has finished for PR 16585 at commit 2ce65cb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71561/
Test PASSed.

Test build #71575 has started for PR 16585 at commit 2b61d47.

LGTM, pending jenkins

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71575/
Test FAILed.

retest this please.

Test build #71581 has started for PR 16585 at commit 2b61d47.

Test build #71581 has finished for PR 16585 at commit 2b61d47.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71581/
Test FAILed.

retest this please.

Test build #71596 has started for PR 16585 at commit 2b61d47.

Test build #71596 has finished for PR 16585 at commit 2b61d47.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71596/
Test PASSed.

thanks, merging to master!

@rxin @cloud-fan Thanks!


Modern Authentication

Hi! I love this script and have been using it for months without any problems. But unfortunately it doesn't work since a couple of days. I think this is due to the type of authentication which is used. Since Oct 1st, MS set the "modern" authentication as requirement for lots of things.
At the moment, the script fails on the following line:
$CalendarFolder = [Microsoft.Exchange.WebServices.Data.CalendarFolder]::Bind($service, [Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Calendar);
-> Error 401.
I just ran a connectivity test via https://testconnectivity.microsoft.com/tests/O365EwsAccess/input . When I run the test with my username and password, a specified target mailbox and "Calendar" as target folder I get:

The same error when using the standard authentication
No error when using OAuth
-> So I guess the script has to be changed in order to incorporate the modern authentication.

Can anybody please tell me what exactly has to be changed?
Thanks in advance and kind regards

Thanks for the quick reply. Sorry, I am referring to the "Import-CalendarCSV.ps1".

I'll add the OAuth parts to that one when I get a chance.  Looks like a long time since I updated it!

Sounds awesome, thanks a lot! I assume you are pretty busy and I can totally understand if this might take some time. As this is a highly relevant topic for our company (booking options etc rely on this), could you maybe tell me in which of your other scripts those parameters are already incorporated? Maybe I can change the relevant lines by myself by comparing the script to the updated scripts. Thanks so much!

awesome, thanks! I'll see what I can do. No matter what, I'm looking forward to your official update :)

Update: I was not able to get the script to work =/ Looking forward to the official update!

I'd also be interested.
Best
Keith

+1. Thank you for your work on this script David! It's much appreciated.

Please check out v2.0.0 of the script (just published).  I tested it with these parameters (which uses the default app registration):
.\Import-CalendarCSV.ps1 -CSVFileName "c:\temp\importtest.csv" -GenerateSampleCSV -Mailbox<EMAIL_ADDRESS>-Office365 -OAuth -OAuthTenantId "tenant id"
The above uses the default app registration which is multitenant, and would trigger log-on for the specified mailbox and require consent (which may need a tenant administrator).  I don't recommend using my app registration, though.  Better would be to register an application in your own tenant and pass the relevant info in the OAuth parameters.
I'll add a Wiki page at some point.  I haven't done a huge amount of testing, so if any issues are encountered please raise a new report here and I'll take a look.

Hi David, Thanks a lot!
I'm already testing the new script.

First I received errors because the script was looking for the dll file in strange locations, but I managed to load the dll file properly after a few tries.

But now the script fails when binding to the calendar ($CalendarFolder = [Microsoft.Exchange.WebServices.Data.CalendarFolder]::Bind($service, [Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Calendar)).

It fails with error 401 when $OAuth=False . Impersonation is set to $True and I pass the Credentials of an administrator user (who is definitely allowed to access the user calendars and create appointments) to the script.

It fails with error 403 when $OAuth=True . I registered a new app in Azure AD and pass all the necessary OAuth information to the script.

Maybe you can give me a hint, where these problems come from? Much appreciated!
Kind regards
Tim


#example() fails for code that results in content with relative width

Here's a minimal example with three modules given as inline code:
#import "@preview/tidy:0.2.0"

#tidy.show-module(
  tidy.parse-module(````
/// A function
///
/// #example(mode: "markup", ```
/// #rect(width: 2em)
/// ```)
#let func-a() = {}
````.text),
  style: tidy.styles.minimal
)

// #tidy.show-module(
//   tidy.parse-module(````
// /// A function
// ///
// /// #example(mode: "markup", ```
// /// #rect(width: 100%)
// /// ```)
// #let func-b() = {}
// ````.text),
//   style: tidy.styles.minimal
// )

#tidy.show-module(
  tidy.parse-module(````
/// A function
///
/// #example(mode: "markup", scale-preview: 100%, ```
/// #rect(width: 100%)
/// ```)
#let func-c() = {}
````.text),
  style: tidy.styles.minimal
)

The first module contains a func-a, and its documentation has an example. The width is 2em (plus some border) - no problem.
However, the second module (commented out) has a rectangle with width: 100%. This leads to a division by zero, it seems:
error: cannot divide by zero
   ┌─ @preview/tidy:0.2.0/src/show-example.typ:59:16
   │
59 │     calc.min(1, available-preview-width / preview-size.width) * 100%
   │                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The third module circumvents this: the division only happens if scale-preview == auto, so I set scale-preview: 100%. However, this seems to give the example content a "viewport" width of zero: width: preview-size.width * (scale-preview / 100%).
So in total: examples can't have a width that is a ratio, as measuring that leads to either dividing by or multiplying with zero.

Hi @SillyFreak ,
Yes this is kind of the intended behaviour. The example should not have a size that is dependent on the parent container because how would scaling then work? Do you have a use-case where you would want to have an example with a width that is a ratio in this scenario?

I see, mmh.
Well even for a fixed scale factor, a width of 100% seems conceptionally a bit problematic.
No, I think we can make it work.

Fixed via beca07c


Install community.general

@flaper87 you needed to do that for it to include the roles?
on which ansible version?

@flaper87 you needed to do that for it to include the roles?
on which ansible version?

ansible 2.9 on EL8. Basically, on a fresh node, it'll fail to load the roles unless this package is installed.
It should be a noop if it's already present.