Fix data loss in echo.c

Please note that while we are transitioning from travis-ci to AWS CodeBuild, some tests are run on each platform. Non-AWS contributors will temporarily be unable to see CodeBuild results. We apologize for the inconvenience.
Resolved issues:
related to #1758
Description of changes:
In the echo function used by s2nd/s2nd only bufsize data would be read on each poll event.  This created a condition where a POLLHUP would be detected, and the function would return, before all the data was actually read.
This change processes all the data from stdin before returning on POLLHUP.
Call-outs:
It may be worth writing a test an examining the s2n socket code as well. I haven't seen any error with this code, so I can't determine if something is broken and needs to be fixed.
Testing:
Ran this change against a couple thousand integration tests. The tests used to fail frequently because 2048 bytes of expected data was never sent. Now the tests pass.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #1906 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1906   +/-   ##
=======================================
  Coverage   81.51%   81.51%
=======================================
  Files         229      229
  Lines       17111    17111
=======================================
  Hits        13948    13948
  Misses       3163     3163


Flsion/feat scrollbar

Types of changes

[x] New feature
[ ] Bug fix
[ ] Enhancement
[ ] Component style change
[ ] Typescript definition change
[ ] Documentation change
[ ] Coding style change
[ ] Refactoring
[ ] Test cases
[ ] Continuous integration
[ ] Breaking change
[ ] Others

Background and context

Solution

How is the change tested?

Changelog

Component
Changelog(CN)
Changelog(EN)
Related issues

overflow-list
新增 OverflowList 组件
Added OverflowList component

scrollbar
增加虚拟滚动条组件
Add virtual scrollbar component

scrollbar
table、select、list、cascader、dropdown 组件替换虚拟滚动条
Table, select, list components replace virtual scroll bars

Checklist:

[ ] Test suite passes (npm run test)
[ ] Provide changelog for relevant changes (e.g. bug fixes and new features) if applicable.
[ ] Changes are submitted to the appropriate branch (e.g. features should be submitted to feature branch and others
should be submitted to main branch)

Other information

table、select、list的虚拟滚动，有计划替换成虚拟滚动条吗？


Rounding to 0 decimals?

Hi,
I have my set up with round: 0, however I have some entities showing with a single decimal.
Some of my lights will report sometimes a residual 0.4W or 0.1W, I was hoping to dismiss them by using "round: 0", however it does not seem to be working? Ie, them keep showing on my chart despite the rounding being set to 0.
Thanks,

This is intentional, because of #18 . If they were rounded to 0 decimals, it would say 0W, and if it is 0, then it shouldn't be displayed and you would wonder where that 0.4 went. Or if it is displayed as 0W, you would wonder why it is displayed but other zero values are not. So for values that would round to zero we round to the first significant digit. Ex 0.0027 would show as 0.003
I have to add a note in the docs about this.

Thanks for the clarification, I understand now, seems logical.

I would recommend to do something like 0.0027 -> <0.01 (round to 2 dec). This limits the number of character. For instance 0.0000000027 will takes a huge among of space


YAML Frontmatter

In Atom it's marking errors in the YAML frontmatter of jekyll markdown files

This is a false flag and should not be marked imho.

Also, it does not recognize comments in the frontmatter:

Hi @mcnesium!
remark itself supports YAML through remark-frontmatter (it used to be in the core, but was externalised recently).
This issue is unrelated to remark-lint. I’m not sure how you use remark-lint is Atom. Maybe linter-markdown? If you can use plugins in Atom, you should add remark-frontmatter there. Otherwise, raise an issue on the thing using remark-lint in Atom.


Apply to individual div or other DOM object

Would be great to be able to target a specific object with this besides the background!

+1 to this.
In fact, I initially thought that this was the case before using the plugin

+1

https://github.com/MasonWang025/brusher
Download the new brusher.js file in my repo to play around with it.


Update pystrace to work with recent versions of strace

Luis, I had forked this to work on it a bit myself but I don't have much time to maintain this and it has been a while since I worked on it. I will try to take a look in the next couple of days. Perhaps maintaining it is something you could do? Apologies for not really paying attention to this one.


Add method to get total object counts per category in a folder (that pass filter threshold)

What is this for?  Who will be using it?

Counting instances of object categories, e.g. there are 10 people total in this dataset, or 100 juvenile seals
Anyone with counting problems would use it

Blocked on #306

@subdavis do we need more info on this issue? Now that #306 is in process, we can at least triage this issue.

I'm going to let #1080 eat this issue.  Closing as duplicate.


Problems when compiling to Rust

Hi @delneg! Thanks a lot integrating Fable with Tauri, this looks really interesting! In this tweet you mention that you also tried to compile to Rust (the code is in fable4-rust branch in this repo) but faced some problems.
Would it be possible to quickly summarize the issues? I'm sure that @ncave and @alexswan10k will be happy to hear your feedback and maybe they know how to fix/work around :)

Hello,
Well first of all it's Fable version in .config/dotnet-tools.json.
I had to switch between Fable 4.0.0-snake-island-alpha-021 and Fable 3.7.18 because Feliz requires Fable 3 or it won't compile. I don't know if there's any way to use specific dotnet tool for specific projects though.
Secondly, I've had much trouble with getting a global function (maybe it's called 'crate-level function', I don't remember)
I need it for build.rs which is a special file used to build the project:
fn main() {
    tauri_build::build()
}

The other issue with that file is that if it's the last file in fsproj, other file (src/main.rs) gets 'mod'-ed into it, which is a no-go because it's a build file:
#[path = "./src/main.rs"]
mod module_e43b409d;

Also, I didn't understand how to get
#![cfg_attr(all(not(debug_assertions), target_os = "windows"),
           windows_subsystem = "windows")]

to appear at the very top.
Note that it's not #[cfg_attr( but a #![cfg_attr(
Another very important issue is I don't know how to get &str, because it's what #[tauri::command] expects - it doesn't work with any type of Rc (and definitely not Lrc<str> that fsharp string compiles into.
I've read Fable.Core.Rust source code, read through tests in Fable repo, even searched in issues & pull requests - but didn't find any examples of that.
[<ByRef>] or inref<string> that I've found didn't work, they just produced &Lrc<str>.
Also, I didn't understand how get get my functions compile into regular functions without a module, they always seem to be showed into some pub mod Main or pub mod Build, and that doesn't work for binaries because I want to have my own pub fn main() { , not the one that Fable generates from [<Entrypoint>].
Anyway, I'm not an expert in neither F# nor Rust and I didn't write too much code using Fable Rust so far, so maybe those issues have some pretty easy solutions.
I hope that I've outlined them correctly and it's possible to get if not exactly the same but close to the existing Rust code to be generated by Fable.

@delneg

Currently you can't do #![cfg_attr at file level, but you can do #[cfg_attr at file module level, which is the same:

[<Fable.Core.Rust.OuterAttr("cfg_attr", [| "something"; "feature = \"something\""; "etc" |])>]
module MyModule

We are still considering changing the string type representation to lower the impedance with interop code. In the mean time, Rust interop is still a work in progress, but you can emit the code you need, for example:

open Fable.Core
open Fable.Core.Rust

[<Erase; Emit("String")>]
type String = interface end

[<Erase; Emit("&str")>]
type StrRef = interface end

[<Emit("$0.as_ref()")>]
let asRef x = nativeOnly

[<OuterAttr("tauri::command")>]
let my_custom_command (invoke_message: String): StrRef = asRef invoke_message

This is just an example how to emit types and funcs, not actual code, so don't read too much into it.
Also, we don't have any serialization support yet, so perhaps for now the whole command layer needs to be in native Rust and transform the command payload before calling into your F# code.

Yes, there are no global functions in F#, everything is in a module.

I want to have my own pub fn main() { , not the one that Fable generates from [].

All that the generated main does is to call the F# main, passing the arguments, that's it.
So you can just put your F# code in the F# main, like you normally do.
Or, you can always have your own Rust main.rs/lib.rs/build.rs top level file that does whatever you need to do and then calls into the rest of the code.

I think @ncave  covered my thoughts really, but I just wanted to quickly summarize a couple of things
How to get various commonly used strings from an Lrc<str> in Rust:
                    let s:&str = string("3.14").as_ref();
                    let s2:String = s.to_string();

You can also do this using emit as ncave pointed out. We will probably be able to standardize this better down the road.
As to your question about integrating with existing libraries, the way I have been experimenting with personally is to actually keep all of the F# generated code in a separate crate, and assemble the F# code with Rust libraries in a consuming root crate. By doing this you do not need to work around any F# generation limitations when trying to interop, although this is definitely a scenario that would be nice to have working well.
Another point of interest - you can pull in rust files into the compilation process in F# using
importAll "./ExtInteropTests.rs"

We have examples of this in the Fable Rust tests.
At some point I hope to be able to post an example but not quite there yet, sorry.

Hello and thank you for your answers.
I've played around and pushed a somewhat practical version here https://github.com/delneg/Fable-tauri-vite-feliz-starter/commit/c7b77c2406e702136b8731dd9c28f673bf134c35
However, right now I guess it doesn't make much sense to force yourself to write tauri code in F#, because it's very hard to define commands without serde.
I've left build.rs & main.rs "as is" and added a separate file "commands.fs" to define Tauri commands.
Although, it's still not usable for development because of Feliz.HookAttribute incompatibility with Fable 4 currently
./fable_modules/Feliz.1.68.0/React.fs(1,1): error EXCEPTION: Plugin Feliz.HookAttribute expects v3.0 but currently running Fable v4.0.0-snake-island-alpha-021
Anyway, even if it's not practical it's a fun experiment

FWIW, the inner attributes were fixed and will work as expected in the next released version.

@delneg I've published alpha-022 which includes latest Rust fixes if you want to give it a new try 👍 This version should be also compatible with Feliz so you can use it to replace Fable 3 (please let us know if you find any regression). You just need to install the latest Feliz plugins with:
dotnet add package Feliz.CompilerPlugins --version 2.0.0-prerelease-001

@delneg I've published alpha-023 which includes latest Rust fixes if you want to give it a new try 👍 This version should be also compatible with Feliz so you can use it to replace Fable 3 (please let us know if you find any regression). You just need to install the latest Feliz plugins with:
dotnet add package Feliz.CompilerPlugins --version 2.0.0-prerelease-001

@alfonsogarciacaro Yes, after updating it looks like it works, tauri dev works fine for me locally


Catch not work

Hello and thank you for using dotnet/reactive. Please select a category and detail your issue by answering the questions there:
Bug
Despite our best efforts, bugs can slip into releases or corner cases forgotten about. We will try our best to remedy the situation
and/or provide workarounds. Note that certain (odd) behaviors are by design and as such are not considered bugs.

Which library version?
5.0

What are the platform(s), environment(s) and related component version(s)?
Window11 22H2 22621.1265，Visual Studio 2022(64 bit); .Net Framework 4.8

Exception not catch.
            Observable.Defer(() => Observable.Return("aac"))
            .ObserveOn(SynchronizationContext.Current)
            .Select(_ =>
            {
                throw new Exception("test execption");
                return _;
            })
            .Catch<string, Exception>(ex => Observable.Throw<string>(ex))
            .Subscribe(str =>
            {
                //Log("Response: " + response.ToString());
                log.Info("test " + str);
            }, err =>
            {
                log.Warn("test ", err);
            }, () =>
            {
                log.Warn("test completed");
            });

SynchronizationContext.Current is null by default, and is generally only set if you're using a framework that supplies a SynchronizationContext (e.g., WPF or Windows Forms), if if you've set one yourself.
So I'm guessing this code probably runs in the context of some UI framework? Could you tell me which framework you are using?

SynchronizationContext.Current is null by default, and is generally only set if you're using a framework that supplies a SynchronizationContext (e.g., WPF or Windows Forms), if if you've set one yourself.
So I'm guessing this code probably runs in the context of some UI framework? Could you tell me which framework you are using?

Yes, I'm using on winform

I need to changed the code this way to be catched
            Observable.Defer(() => Observable.Return("aac"))
            .ObserveOn(SynchronizationContext.Current)
            .Select(_ =>
            {
                try {
                    throw new Exception("test execption");
                } catch(Exception ex) {
                    return Observable.Throw(ex);
                }
                return _;
            })
            .Catch<string, Exception>(ex => Observable.Throw<string>(ex))
            .Subscribe(str =>
            {
                //Log("Response: " + response.ToString());
                log.Info("test " + str);
            }, err =>
            {
                log.Warn("test ", err);
            }, () =>
            {
                log.Warn("test completed");
            });

Thank you for your valuable time, This is indeed the case.


[Feature] Support more sites

Thanks for the script, it's working great!
I was wondering if you could write something similar for Bumble :)

Sorry, I'm not on Bumble and at the same time don't know how their API works. Also, I am really busy at least until the end of the year, so I don't want to create a new Bumble profile and study Bumble's API just for a challenge.

How about supporting more functionalities?

⚠UPDATE⚠

The project started as a simple wrapper around a javascript function posted on Reddit. I gained way more traction than I anticipated and I couldn't keep up as well and fast as I should have.
What didn't help, was Tinder patching the exploited bug and later changing both API and frontend's identifiers. With me not on Tinder anymore, I am not able to test possible changes and/or find new exploits.
I am accepting any PRs I get and will do so until the project is archived. If you, the reader, are willing to take over the project, please, let me know.
If something isn't working, sometimes there's newer version available on the develop branch. To try it, click this link.
Once there's a newer stable version, your userscript extension should update you automatically to it.


i want to get user share library list,but i can't find any way to do  it

[ ] Question
i want to get user share library list,but i can't find any way to do it, can you help me?

Thank you.

Could you please clarify your scenario?
Are you asking about "shared with me? https://docs.microsoft.com/en-us/graph/api/drive-sharedwithme?view=graph-rest-1.0

i am sorry,I didn't reply in time。but i not asking 'share with me', i want get 'shared library' in one drive bueiness, i want to get 'share library' list

As part of a repository clean up effort we are closing older issues. If this issue remains, please: open a new issue, reference this issue, and provide any additional details that may help in resolution. Thank you for your understanding as we work to improve our responsiveness.


Adapt Swift type metadata lookup to τ_depth_index convention.

This is the LLDB work to adapt the new compiler convention to use
depth and index instead of name to lookup type metadata artificial
variables.
rdar://problem/38306926

test with https://github.com/apple/swift/pull/16937
@swift-ci test

test with https://github.com/apple/swift/pull/16937
@swift-ci test

test with https://github.com/apple/swift/pull/16937
@swift-ci test

test with https://github.com/apple/swift/pull/16937
@swift-ci test

test with https://github.com/apple/swift/pull/16937
@swift-ci test

@dcci: Can you take another look?

@dcci: Can you take another look?

This looks good to me but we should really add a test for type reconstruction in the compiler.

I did a new type reconstruction test over in the last revision of apple/swift#16937.


Proof Login/logout

Right now I have just a simple button to login/out for PROOF

It's not great. I'm used to a login being in the top right of a navbar, and I think other people will be too. I haven't landed on anything that seems like a good fit yet. Some notes:

An SO solution to add the login/logout button in top right
package shinyauthr - looks nice, but I think it won't work as I think it only works by checking user input username/password against a local data.frame, which won't work as we need to check against the proof API, but may be worth checking if we can just use the UI components
https://www.polished.tech/ but it only does auth against their own cloud solution, so no go
https://github.com/treysp/shiny_password
heard from someone that mentioned https://github.com/r-lib/gargle/pull/157 & https://github.com/r-lib/httr2/issues/422 but even if those did help they aren't released yet
https://datastorm-open.github.io/shinymanager/ is an option. apparently you can use a custom function to pass to the function that handles auth, so could try this. downside is that the framework "takes over the UI", so we may be limited in the UI if we use this
a solution from Jason Bryer on mastodon https://gist.github.com/jbryer/d563b3147c7e5ea8c8d7910601676e39 - looks too complicated
others ...

Asked in the Posit forum for ideas

To surface some Slack discussion about this: scott summarize any salient points here

Currently going the route of forking shinyauthr and using a modified version of it for our login/logout flow.
It's at https://github.com/getwilds/shinyauthr - changes are not pushed there yet but should be soon
Current issues not resolved:

[ ] xxxx
[ ] xxx

@sckott -- Looks like you did some work on this in the commit from March 1st listed above and the login button is currently in the top right like you wanted. Would you consider this complete or is there still more work you'd like to do on it?

I think it can be closed, we have login that works.
I'd definitely improve on this in v2 but we can open issues for that later


error when updating hyprland-git

Hyprland Version
v0.29.1-88-gb0a82c04
Bug or Regression?
Bug
Description
I get file conflict when updating hyprland got 0.30
How to reproduce
yay -Syu
Crash reports, logs, images, videos
error: failed to commit transaction (conflicting files)
hyprland-git: /usr/share/xdg-desktop-portal/hyprland-portals.conf exists in filesystem (owned by xdg-desktop-portal-hyprland)
Errors occurred, no packages were upgraded.
 -> error installing: [/home/stovejeebs/.cache/yay/hyprland-git/hyprland-git-0.30.0.r29.7b002d60-1-x86_64.pkg.tar.zst] - exit status 1

Just delete that file manually and try again


updated ordino text

the layout of products is not ideal for such a long text...

looks good, thanks!


python310Packages.angr: 9.2.35 -> 9.2.36

Description of changes
Update angr stack to 9.2.36

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 213922 run on x86_64-linux 1

16 packages built:

python310Packages.ailment
python310Packages.angr
python310Packages.angrcli
python310Packages.angrop
python310Packages.archinfo
python310Packages.claripy
python310Packages.cle
python310Packages.pyvex
python311Packages.ailment
python311Packages.angr
python311Packages.angrcli
python311Packages.angrop
python311Packages.archinfo
python311Packages.claripy
python311Packages.cle
python311Packages.pyvex


Feature: support passing dialect to parser when using evaluate function

Which issue does this PR address?
https://github.com/nikku/feelin/issues/86
Closes #86

Good job! I cleaned up your PR and ensured this is also supported for unary tests via https://github.com/nikku/feelin/pull/92.

Released as v3.2.0


Consider adding full-stack-todo-rust-course to SeaORM showcase!

Hey @BrooksPatton, thanks for adopting SeaORM!
It's our pleasure to see more inspirational projects were built on top of SeaORM :)
Let us know if you have any feature recommendation or feedback.
Some learning resources for you: Documentation, Tutorial, Cookbook, Q&A
Join our Discord server to chat with others in the SeaQL community!
Feel free to submit a PR to showcase your project, https://github.com/SeaQL/sea-orm/issues/403.

I didn't realize this was a thing! Thanks @billy1624

I added a pr for the course! https://github.com/SeaQL/sea-orm/pull/1454


[BUG][SPARK] ABFSS and WASBS report all files being used as Input Sources

When using OpenLineage's Spark Integration and using ABFSS or WASBS to connect to an Azure Data Lake Gen 2 / Blob Storage account, Open Lineage's Input data source will report back all file paths being used by the spark job rather than just the folder path in some scenarios.
The table below shows 12 scenarios testing ABFSS and WASBS vs Databricks Mount Points where we varied the file path input string to be either a single file, a folder path, or a folder path that uses a wildcard.  In addition we tested the read with inferSchema on or off which generates a second open lineage emit event.

input source
single file
folder path
wildcard path
infer schema

wasbs
Returns the file path
Returns the folder path
Returns all file paths used
Returns all file paths

abfss
Returns the file path
Returns the folder path
Returns all file paths used
Returns all file paths

dbfs mount point
Returns the folder path
Returns the folder path
Returns the folder path
Returns the folder path

We tested these scenarios using a scala spark command such as
val df = spark
.read
.format("csv")
.load(FILE_OR_FOLDER_PATH)

FILE_OR_FOLDER_PATH would take on a pattern such as:

Single File<EMAIL_ADDRESS>Folder Path<EMAIL_ADDRESS>Wildcard Path<EMAIL_ADDRESS>
This is an example of what we mean by returning the file path and NOT the folder path.  It was generated by using<EMAIL_ADDRESS>{
    "inputs": [
        {
            "namespace"<EMAIL_ADDRESS>            "name": "/newretail/retail-1.csv",
            "facets": "..."
        },
        {
            "namespace"<EMAIL_ADDRESS>            "name": "/newretail/retail-2.csv",
            "facets": "..."
        }
    ]
}

Desired / Expected Behavior:

input source
single file
folder path
wildcard path
infer schema

wasbs
:white_check_mark:Returns the file path
:white_check_mark:Returns the folder path
:collision:Returns the folder path
:collision:Returns the folder path

abfss
:white_check_mark:Returns the file path
:white_check_mark:Returns the folder path
:collision:Returns the folder path
:collision:Returns the folder path

dbfs mount point
:collision:Returns the file path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path

If a single file is mentioned, it would be ideal if Open Lineage always reported out that single file
If wildcard path is mentioned, Open Lineage should only report back the appropriate parent folder
InferSchema jobs should report only the single parent folder in all cases

I did some more digging on this one and it looks like there might be an issue due to integration\spark\src\main\common\java\io\openlineage\spark\agent\util\PlanUtils.java and the getDirectoryPath method doesn't know how to instantiate / use the ABFSS file path. It requires instantiating the filesystem with ouath2 workflow versus WASBS which follows an older convention and uses the fs.azure.account.key that PlanUtils seems to be looking for (WASBS doc).
When comparing the logical plans between abfss and mount points, it's the exact same plan and the only difference is this warning it seems.
22/01/19 15:31:40 INFO OpenLineageRunEventBuilder: Visiting query plan Optional[== Parsed Logical Plan ==
InsertIntoHadoopFsRelationCommand<EMAIL_ADDRESS>false, CSV<EMAIL_ADDRESS>Overwrite, [id, postalCode, streetAddress, city, stateAbbreviation]
+- Repartition 1, true
   +- Project [id#71, postalCode#72, streetAddress#73, city#78, stateAbbreviation#79]
      +- Join Inner, (id#71 = id#77)
         :- Relation[id#71,postalCode#72,streetAddress#73] csv
         +- Relation[id#77,city#78,stateAbbreviation#79] csv

== Analyzed Logical Plan ==

InsertIntoHadoopFsRelationCommand<EMAIL_ADDRESS>false, CSV<EMAIL_ADDRESS>Overwrite, [id, postalCode, streetAddress, city, stateAbbreviation]
+- Repartition 1, true
   +- Project [id#71, postalCode#72, streetAddress#73, city#78, stateAbbreviation#79]
      +- Join Inner, (id#71 = id#77)
         :- Relation[id#71,postalCode#72,streetAddress#73] csv
         +- Relation[id#77,city#78,stateAbbreviation#79] csv

== Optimized Logical Plan ==
InsertIntoHadoopFsRelationCommand<EMAIL_ADDRESS>false, CSV<EMAIL_ADDRESS>Overwrite, [id, postalCode, streetAddress, city, stateAbbreviation]
+- Repartition 1, true
   +- Project [id#71, postalCode#72, streetAddress#73, city#78, stateAbbreviation#79]
      +- Join Inner, (id#71 = id#77)
         :- Filter isnotnull(id#71)
         :  +- Relation[id#71,postalCode#72,streetAddress#73] csv
         +- Filter isnotnull(id#77)
            +- Relation[id#77,city#78,stateAbbreviation#79] csv

== Physical Plan ==
Execute InsertIntoHadoopFsRelationCommand<EMAIL_ADDRESS>false, CSV<EMAIL_ADDRESS>Overwrite, [id, postalCode, streetAddress, city, stateAbbreviation]
+- AdaptiveSparkPlan isFinalPlan=true
   +- == Final Plan ==
      ShuffleQueryStage 1, Statistics(sizeInBytes=456.0 B, rowCount=5, isRuntime=true)
      +- Exchange RoundRobinPartitioning(1), REPARTITION_WITH_NUM, [id=#241]
         +- *(2) Project [id#71, postalCode#72, streetAddress#73, city#78, stateAbbreviation#79]
            +- *(2) BroadcastHashJoin [id#71], [id#77], Inner, BuildRight, false
               :- *(2) Filter isnotnull(id#71)
               :  +- FileScan csv [id#71,postalCode#72,streetAddress#73] Batched: false, DataFilters: [isnotnull(id#71)], Format: CSV, Location<EMAIL_ADDRESS>PartitionFilters: [], PushedFilters: [IsNotNull(id)], ReadSchema: struct<id:int,postalCode:string,streetAddress:string>
               +- BroadcastQueryStage 0, Statistics(sizeInBytes=1024.0 KiB, rowCount=5, isRuntime=true)
                  +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [id=#161]
                     +- *(1) Filter isnotnull(id#77)
                        +- FileScan csv [id#77,city#78,stateAbbreviation#79] Batched: false, DataFilters: [isnotnull(id#77)], Format: CSV, Location<EMAIL_ADDRESS>PartitionFilters: [], PushedFilters: [IsNotNull(id)], ReadSchema: struct<id:int,city:string,stateAbbreviation:string>
   +- == Initial Plan ==
      Exchange RoundRobinPartitioning(1), REPARTITION_WITH_NUM, [id=#78]
      +- Project [id#71, postalCode#72, streetAddress#73, city#78, stateAbbreviation#79]
         +- BroadcastHashJoin [id#71], [id#77], Inner, BuildRight, false
            :- Filter isnotnull(id#71)
            :  +- FileScan csv [id#71,postalCode#72,streetAddress#73] Batched: false, DataFilters: [isnotnull(id#71)], Format: CSV, Location<EMAIL_ADDRESS>PartitionFilters: [], PushedFilters: [IsNotNull(id)], ReadSchema: struct<id:int,postalCode:string,streetAddress:string>
            +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, false] as bigint)),false), [id=#75]
               +- Filter isnotnull(id#77)
                  +- FileScan csv [id#77,city#78,stateAbbreviation#79] Batched: false, DataFilters: [isnotnull(id#77)], Format: CSV, Location<EMAIL_ADDRESS>PartitionFilters: [], PushedFilters: [IsNotNull(id)], ReadSchema: struct<id:int,city:string,stateAbbreviation:string>
] with input dataset builders [io.openlineage.spark.agent.lifecycle.plan.LogicalRelationVisitor<org.apache.spark.sql.execution.datasources.LogicalRelation,D>, io.openlineage.spark.agent.lifecycle.plan.LogicalRDDVisitor<org.apache.spark.sql.execution.LogicalRDD,D>, io.openlineage.spark.agent.lifecycle.plan.BigQueryNodeVisitor<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,D>, io.openlineage.spark.agent.lifecycle.plan.KafkaRelationVisitor<org.apache.spark.sql.execution.datasources.LogicalRelation,D>, io.openlineage.spark.agent.lifecycle.plan.CommandPlanVisitor<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,io.openlineage.client.OpenLineage$InputDataset>]
22/01/19 15:31:40 WARN PlanUtils: Unable to get file system for path
Failure to initialize configurationInvalid configuration value detected for fs.azure.account.key
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.SimpleKeyProvider.getStorageAccountKey(SimpleKeyProvider.java:51)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getStorageAccountKey(AbfsConfiguration.java:552)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.initializeClient(AzureBlobFileSystemStore.java:1679)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:210)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:130)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
	at io.openlineage.spark.agent.util.PlanUtils.getDirectoryPath(PlanUtils.java:159)
	at io.openlineage.spark.agent.lifecycle.plan.LogicalRelationVisitor.lambda$handleHadoopFsRelation$0(LogicalRelationVisitor.java:93)
        <SNIP>
	at io.openlineage.spark.agent.lifecycle.plan.LogicalRelationVisitor.handleHadoopFsRelation(LogicalRelationVisitor.java:101)
	at io.openlineage.spark.agent.lifecycle.plan.LogicalRelationVisitor.apply(LogicalRelationVisitor.java:71)
	at io.openlineage.spark.agent.lifecycle.plan.LogicalRelationVisitor.apply(LogicalRelationVisitor.java:48)
	at io.openlineage.spark.agent.util.PlanUtils$1.apply(PlanUtils.java:78)
        <SNIP>
	at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$visitLogicalPlan$13(OpenLineageRunEventBuilder.java:399)
	at io.openlineage.spark.agent.util.ScalaConversionUtils$3.apply(ScalaConversionUtils.java:126)
       <SNIP>
	at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.lambda$buildInputDatasets$5(OpenLineageRunEventBuilder.java:350)
	at java.util.Optional.map(Optional.java:215)
	at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildInputDatasets(OpenLineageRunEventBuilder.java:348)
	at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.populateRun(OpenLineageRunEventBuilder.java:307)
	at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:293)
	at io.openlineage.spark.agent.lifecycle.OpenLineageRunEventBuilder.buildRun(OpenLineageRunEventBuilder.java:248)
	at io.openlineage.spark.agent.lifecycle.SparkSQLExecutionContext.end(SparkSQLExecutionContext.java:80)
	at io.openlineage.spark.agent.OpenLineageSparkListener.sparkSQLExecEnd(OpenLineageSparkListener.java:148)
        <SNIP>
Caused by: Invalid configuration value detected for fs.azure.account.key
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.diagnostics.ConfigurationBasicValidator.validate(ConfigurationBasicValidator.java:49)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.diagnostics.Base64StringConfigurationBasicValidator.validate(Base64StringConfigurationBasicValidator.java:40)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.SimpleKeyProvider.validateStorageAccountKey(SimpleKeyProvider.java:70)
	at shaded.databricks.azurebfs.org.apache.hadoop.fs.azurebfs.services.SimpleKeyProvider.getStorageAccountKey(SimpleKeyProvider.java:49)
	... 101 more

@wjohnson is this comment for the second issue? https://github.com/OpenLineage/OpenLineage/issues/458
From what you posted before here, there is same behavior (file/directory/wildcard) between WASBS and ABFSS.

I see - we have this if condition to return the file's parent if we can detect it is a file, but that exception being thrown is preventing us from determining the type of the path - https://github.com/OpenLineage/OpenLineage/blob/main/integration/spark/src/main/common/java/io/openlineage/spark/agent/util/PlanUtils.java#L159 .
It looks like the LogicalRelationVisitor grabs the Hadoop Configuration object directly from the SparkContext. https://github.com/OpenLineage/OpenLineage/blob/main/integration/spark/src/main/common/java/io/openlineage/spark/agent/lifecycle/plan/LogicalRelationVisitor.java#L91 . Based on the links you share, I would think the configuration should be in the configuration. I can try to replicate this (though I don't have an Azure Databricks account) and see if the configuration keys propagate

@wjohnson, can you try replacing the handleHadoopFsRelation method with the following code and try again?
This changes the hadoopConfig construction using the same code that the InMemoryFileIndex code uses, which means it should match the way those files are being read for processing. If it works for the execution, it should work for the metadata collection 🙏🏽
  private List<D> handleHadoopFsRelation(LogicalRelation x) {
    HadoopFsRelation relation = (HadoopFsRelation) x.relation();
    return context.getSparkSession()
        .map(session -> {
          Configuration hadoopConfig = session.sessionState().newHadoopConfWithOptions(relation.options());
          return JavaConversions.asJavaCollection(relation.location().rootPaths()).stream()
              .map(p -> PlanUtils.getDirectoryPath(p, hadoopConfig))
              .distinct()
              .map(
                  p -> {
                    // TODO- refactor this to return a single partitioned dataset based on static
                    // static partitions in the relation
                    return datasetFactory.getDataset(p.toUri(), relation.schema());
                  })
              .collect(Collectors.toList());
        }).orElse(Collections.emptyList());
  }

@collado-mike BAM! That's it!  I no longer receive the PlanUtil exception and the behavior is now consistent across ABFSS, WASBS, and mount points in Azure Databricks.

input source
single file
folder path
wildcard path
infer schema

wasbs
Returns the folder path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path

abfss
Returns the folder path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path

dbfs mount point
Returns the folder path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path
:white_check_mark:Returns the folder path

Closing this issue since it was resolved in #491


[Feature Request] Allow Sheet uploads greater than 10 MB

I have a process that uploads a sheet with about 25 MB worth of data when formatted into a Google Sheets API compatible object (the actual data itself is about 3 MB when exported as CSV).
When trying to use Export-GSSheet on this data, I receive the following:
Export-GSSheet : Exception calling "Execute" with "0" argument(s): "Google.Apis.Requests.RequestError
Request payload size exceeds the limit: 10485760 bytes. [400]
Errors [
        Message[Request payload size exceeds the limit: 10485760 bytes.] Location[ - ] Reason[badRequest] Domain[global]
]
"
At line:94 char:35
+ ... t = $billingTable | Export-GSSheet -NewSheetTitle $SheetName -Verbose
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Export-GSSheet

Seems like it wouldn't be hard to add a recursive call, just re-using the existing code with the -Append switch. I'll test it out when I get a chance next week.


iOS: crash on hot reload

Steps to Reproduce
When running the latest master (see below) hot reload doesn't work anymore from within VisualStudio Code.  The application crashes after the first change. Using the command line still works, I think.
I haven't checked with Android.
Logs
Here's a same crash from the Debug panel:
Launching lib/main.dart on iPhone X in debug mode...
Xcode build done.                                           13,5s
../../third_party/dart/runtime/vm/object.h: 8954: error: Handle check failed: saw Instance of 'SplashPage' expected Closure
Dumping native stack trace for thread 303
[0x0000000107bf6454] dart::Profiler::DumpStackTrace(void*)
[0x0000000107bf6454] dart::Profiler::DumpStackTrace(void*)
[0x0000000107e6e462] dart::Assert::Fail(char const*, ...)
[0x0000000107c480e0] dart::DRT_InvokeClosureNoSuchMethod(dart::NativeArguments)
[0x0000000124f00902] Unknown symbol
[0x0000000124f02dc4] Unknown symbol
[0x0000000128f78efd] Unknown symbol
[0x000000013367dc54] Unknown symbol
[0x00000001289e6363] Unknown symbol
[0x00000001289e5873] Unknown symbol
[0x00000001289e4bad] Unknown symbol
[0x00000001289dae43] Unknown symbol
[0x00000001299b38cf] Unknown symbol
[0x00000001299b35d1] Unknown symbol
[0x00000001299b34fe] Unknown symbol
[0x0000000125a1fbf9] Unknown symbol
[0x0000000129998e93] Unknown symbol
[0x00000001299b2bd4] Unknown symbol
[0x00000001299b27d8] Unknown symbol
[0x0000000129990679] Unknown symbol
[0x0000000125a1fbf9] Unknown symbol
[0x000000012998f8bb] Unknown symbol
[0x000000012998dd4d] Unknown symbol
[0x000000012998d93a] Unknown symbol
[0x0000000125a1fbf9] Unknown symbol
[0x0000000125a1fb10] Unknown symbol
[0x0000000124f00d63] Unknown symbol
[0x0000000107affcbf] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
Lost connection to device.
[0x0000000107b03aa7] dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&)
[0x0000000107b2d0d6] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
[0x0000000107b5b4fe] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
[0x0000000107b5b600] dart::MessageHandler::HandleNextMessage()
[0x0000000107de07b5] Dart_HandleMessage
[0x00000001079cab0a] tonic::DartMessageHandler::OnHandleMessage(tonic::DartState*)
[0x00000001079cadf0] std::__1::__function::__func<tonic::DartMessageHandler::OnMessage(tonic::DartState*)::$_0, std::__1::allocator<tonic::DartMessageHandler::OnMessage(tonic::DartState*)::$_0>, void ()>::operator()()
[0x00000001076c1056] fml::MessageLoopImpl::RunExpiredTasks()
[0x00000001076c43d4] fml::MessageLoopDarwin::OnTimerFire(__CFRunLoopTimer*, fml::MessageLoopDarwin*)
[0x0000000109a8bf34] __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__
[0x0000000109a8bb32] __CFRunLoopDoTimer
[0x0000000109a8b39a] __CFRunLoopDoTimers
[0x0000000109a85a1c] __CFRunLoopRun
[0x0000000109a84e11] CFRunLoopRunSpecific
[0x000000010e72c1dd] GSEventRunModal
[0x000000011393681d] UIApplicationMain
[0x0000000106cd6f64] main
[0x000000010d999575] start
-- End of DumpStackTrace
Exited (sigterm)

Flutter doctor:
[✓] Flutter (Channel master, v1.1.5-pre.8, on Mac OS X 10.14.1 18B75, locale de-DE)
    • Flutter version 1.1.5-pre.8 at /Users/sma/Work/flutter
    • Framework revision 6f0f0c4b60 (vor 5 Stunden), 2018-12-21 02:03:28 -0500
    • Engine revision 6a9041853c
    • Dart version 2.2.0 (build 2.2.0-dev.1.1 f9ebf21297)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/sma/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.1, Build version 10B61
    • ios-deploy 1.9.4
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 31.3.1
    • Dart plugin version 181.5656
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)

[✓] VS Code (version 1.30.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 2.21.1

[✓] Connected device (3 available)
    • iPhone 5s           • 9BE708BE-3CA3-48A9-8697-6E706D91E2F5 • ios • iOS 12.1 (simulator)
    • iPhone X            • 183CCFF2-9461-4F0D-BD88-DFABE027C0C2 • ios • iOS 12.1 (simulator)
    • iPad Pro (9.7-inch) • 971975FF-8290-4575-B2F6-10CFFA30168D • ios • iOS 12.1 (simulator)

• No issues found!

If you switch back to dev or stable it works again as well?

It's master only and probably started to break yesterday. The dev channel seems to work just fine.
I tested this version:
Flutter 1.1.4 • channel dev • https://github.com/flutter/flutter.git
Framework • revision 7a88fbc5fd (vor 18 Stunden) • 2018-12-20 11:38:07 -0800
Engine • revision dbdb5e6f8c
Tools • Dart 2.2.0 (build 2.2.0-dev.1.1 f9ebf21297)

I am not familiar with the vs code plugin, but does it reproduce when using flutter run on the terminal?

Unable to repro, removing TODAY, but leaving open for further investigation

@clocksmith, it worked with flutter run and manually pressing r on the terminal.
It did not work using VSC on two different machines with two different projects, though.
Using the latest master (see below), it seems to work again, so I'm closing this issue.
Flutter 1.1.5-pre.17 • channel master • https://github.com/flutter/flutter.git
Framework • revision 2e2f19d3e9 (vor 6 Stunden) • 2018-12-21 19:07:01 -0800
Engine • revision 215ca15600
Tools • Dart 2.2.0 (build 2.2.0-dev.1.1 f9ebf21297)

In case anyone is also experiencing a crash on hot reload.
On doing hot reload through Android Studio, my app crashes with
version=2.4.0 (Wed Jun 19 11:53:45 2019 +0200) on "ios_dbc"
thread=24323, isolate=main(0x103826c00)
  pc 0x00000001014fe4c0 fp 0x000000016f5402f0 dart::Profiler::DumpStackTrace(void*)
-- End of DumpStackTrace
Lost connection to device.

However, using flutter run, I can hot reload by pressing r.
This problem was resolved by adding this line to the .xcconfig file my project was using:
TRACK_WIDGET_CREATION=true

I'm also experiencing the same problem with vscode, ios version 9.0 and using vscode debug for hot reload:
flutter --version Flutter 1.7.8+hotfix.4 • channel stable • https://github.com/flutter/flutter.git Framework • revision 20e59316b8 (7 weeks ago) • 2019-07-18 20:04:33 -0700 Engine • revision fee001c93f Tools • Dart 2.4.0
And here is the report from Google crashlytics:

Crashed: io.flutter.1.ui 0  Flutter                        0x102669190 dart::Function::QualifiedName(dart::Object::NameVisibility) const + 208 1  Flutter                        0x10266918c dart::Function::QualifiedName(dart::Object::NameVisibility) const + 204 2  Flutter                        0x10270f900 dart::DRT_NoSuchMethodFromPrologue(dart::NativeArguments) + 836 3  Flutter                        0x102731008 dart::InvokeRuntime(dart::Thread*, dart::Simulator*, void (*)(dart::NativeArguments), dart::NativeArguments const&) + 96 4  Flutter                        0x102730e8c dart::Simulator::Call(dart::Code const&, dart::Array const&, dart::Array const&, dart::Thread*) + 23828 5  Flutter                        0x1025d9d94 dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long) + 888 6  Flutter                        0x1025dd104 dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&) + 800 7  Flutter                        0x102607968 dart::IsolateMessageHandler::HandleMessage(std::__1::unique_ptr<dart::Message, std::__1::default_delete<dart::Message> >) + 1404 8  Flutter                        0x1026333dc dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool) + 232 9  Flutter                        0x10263354c dart::MessageHandler::HandleNextMessage() + 72 10 Flutter                        0x1028e6a90 Dart_HandleMessage + 236 11 Flutter                        0x1024c3500 tonic::DartMessageHandler::OnHandleMessage(tonic::DartState*) + 224 12 Flutter                        0x1024c38fc std::__1::__function::__func<tonic::DartMessageHandler::OnMessage(tonic::DartState*)::$_0, std::__1::allocator<tonic::DartMessageHandler::OnMessage(tonic::DartState*)::$_0>, void ()>::operator()() + 52 13 Flutter                        0x1021c77c4 fml::MessageLoopImpl::FlushTasks(fml::FlushType) + 116 14 Flutter                        0x1021cc1bc fml::MessageLoopDarwin::OnTimerFire(__CFRunLoopTimer*, fml::MessageLoopDarwin*) + 32 15 CoreFoundation                 0x1fd248d60 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 28 16 CoreFoundation                 0x1fd248a90 __CFRunLoopDoTimer + 864 17 CoreFoundation                 0x1fd2482c4 __CFRunLoopDoTimers + 248 18 CoreFoundation                 0x1fd243214 __CFRunLoopRun + 1844 19 CoreFoundation                 0x1fd2427c0 CFRunLoopRunSpecific + 436 20 Flutter                        0x1021cc2e0 fml::MessageLoopDarwin::Run() + 88 21 Flutter                        0x1021c7700 fml::MessageLoopImpl::DoRun() + 40 22 Flutter                        0x1021cb7f8 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0> >(void*) + 200 23 libsystem_pthread.dylib        0x1fced52c0 _pthread_body + 128 24 libsystem_pthread.dylib        0x1fced5220 _pthread_start + 44 25 libsystem_pthread.dylib        0x1fced8cdc thread_start + 4

TRACK_WIDGET_CREATION=true is already in my .xcconfig, so it does nothing for me.


Bounding boxes drawn on training data

As I was debugging, I decided to look at some of the input images in training.  I noticed that bounding boxes and labels were drawn on them, and then I traced it to this bit of code in the __getitem__ method in BatchGenerator:
            # assign input image to x_batch
            if self.norm != None:
                x_batch[instance_count] = self.norm(img)
            else:
                # plot image and bounding boxes for sanity check
                for obj in all_objs:
                    if obj['xmax'] > obj['xmin'] and obj['ymax'] > obj['ymin']:
                        cv2.rectangle(img[:,:,::-1], (obj['xmin'],obj['ymin']), (obj['xmax'],obj['ymax']), (255,0,0), 3)
                        cv2.putText(img[:,:,::-1], obj['name'],
                                    (obj['xmin']+2, obj['ymin']+12),
                                    0, 1.2e-3 * img.shape[0],
                                    (0,255,0), 2)

                x_batch[instance_count] = img

So it looks to me that unless norm function is set, bounding boxes are drawn on the training data, and the norm function is not set in either train.py or the Step-By-Step notebook.  Is this the desired behavior?

@alanjschoen The norm is the normalization function, which is different for different backends. I set it in frontend.py, when creating the train generator and validation generator (lines 403 and 406).

@experiencor I looked at frontend.py and I see what youre talking about, so this should not affect train.py.  But this problem might still affect the Step-By-Step notebook because it creates the BatchGenerator objects directly instead letting the train function in frontend.py create them, and it does not set the norm function.

@alanjschoen You are right. My bad. I did not change the code in the notebook accordingly.

I changed the notebook to work with the new preprocessing.py.

Why is the normalization function different for different backends? In particular, why mobilenet's normalize is the following?
`def normalize(self, image):
image = image / 255.
image = image - 0.5
image = image * 2.
    return image`

Because it fallows the original normalizations for each model.
In the mobileNet you can see the original implementation here

images = tf.cast(images, tf.float32) / 128.  - 1

and the keras implementation here. The formulas are quite different, but gives very close results.
And by default when the model doesnt specific the normalization, it is used 1/255 that in theory converges more quickly than using values from 0..255


Add nil to BeEql

For matching nil, I think we should recomend using be_nil over be(nil).

Well this cop specifically helps improve the semantics of instances where users were already writing things like eql(false). There should certainly be a cop that enforces either be(nil) or be_nil (I think someone is working on it?) but it shouldn't be this cop


chore: inconsistencies with figma

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
This PR solves the inconsistencies listed on issues #109
Fixes # (issue)
Screenshots/Screen Recording of Implementation
check landing page on preview branch

Oops sorry, forget to leave a comment when approving the PR. I got this space below the footer, could you check it @andrehadianto? Other than that, everything's good. Thanks, Andre! 🙌


package: Update testbotsdk to include TX2 support

Change-type: patch
Signed-off-by: Alexandru Costache<EMAIL_ADDRESS>Connects to:

https://github.com/balena-io-hardware/testbotsdk/pull/45
https://github.com/balena-os/balena-jetson/issues/332

A testbot and TX2 DUT are set up in the rig app. Once this is merged, deployed and updated in leviathan it should start running tests on PRs


2.0.0

Added async support for most of the library.
Fixed a few performance issues with steppers.
Added docs.

Coverage remained the same at 100.0% when pulling 4437c0115f80d6349937ef4a64976a737e4a813f on 2.0.0 into d91f00813e568752a975c0676160577bc2b00e35 on master.

Coverage remained the same at 100.0% when pulling 4437c0115f80d6349937ef4a64976a737e4a813f on 2.0.0 into d91f00813e568752a975c0676160577bc2b00e35 on master.

Coverage remained the same at 100.0% when pulling 4437c0115f80d6349937ef4a64976a737e4a813f on 2.0.0 into d91f00813e568752a975c0676160577bc2b00e35 on master.

Coverage remained the same at 100.0% when pulling a666c4034414ba6e337f0ea135b88aede31fe828 on 2.0.0 into d91f00813e568752a975c0676160577bc2b00e35 on master.

Coverage remained the same at 100.0% when pulling e97c606e6ff5139e92ea6e33a720413fe6e511d5 on 2.0.0 into d91f00813e568752a975c0676160577bc2b00e35 on master.

Coverage remained the same at 100.0% when pulling e97c606e6ff5139e92ea6e33a720413fe6e511d5 on 2.0.0 into d91f00813e568752a975c0676160577bc2b00e35 on master.


[HOLD for payment 2023-05-29] Updating the Details Page

https://docs.google.com/document/d/18uQqP0eTe1kWPrQiQwjv_A6u3cBO3VtiwA55XndjCnY/edit#bookmark=id.dvyh4dxac0ja
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01bdb5f277d0358b5a
Upwork Job ID:<PHONE_NUMBER>898828800
Last Price Increase: 2023-05-11

@grgia , @stitesExpensify Is this simply a [$1000] payout for the PR Review to Fedi? (and qualifies for bonus if no regression?)

@fedirjh I've sent you the Upwork Contract for this payment.

@abekkala yep! Thanks for taking care of it :)

@fedirjh Paid and contract ended - Thank you! 🎉
Closing


[FEATURE] add a login frontpage

add a login frontpage possibly with 2 step auth.

Duplicated of https://github.com/roflcoopter/viseron/issues/435


fix(slides): Updated lockSwipes methods to match the new swiper.js API

Short description of what this resolves:
Updated lockSwipes methods to match the new swiper.js API since 4.0.0 ( http://idangero.us/swiper/api/ )
Ionic Version: 4.0
Fixes: #15462

Can someone review or consider to make the changes for lockSwipes @manucorporat

The PR looks perfects! thanks a lot!


Change parameter type of ValueStream.skip from int to long

Fixes #3563
More than 2G positions can legitimately be skipped at once.

ping @dain

looks good

Merged #3579


Shares Per Second at Zero in API when miners are hashing

I've got a MiningCore installation with LTC running. All seems to be working fine, but for some reason, the "Pool" Shares per second is at Zero, even though you can see the two connected miners are hashing and each have their own Share Per Second information being displayed. You can view the API live here: http://<IP_ADDRESS>:4000/api/pools/ltc1
Or look at the attached JPEG file
Why is the pool counting Zero as the Shares Per Second?
Is there a possibility that the Shares Per Second needs to collectively be above "1.0" for the software to count it?

poolStats.sharesPerSecond is of type int so rounds down to nearest whole number
        public int SharesPerSecond { get; set; }
https://github.com/coinfoundry/miningcore/blob/master/src/MiningCore/Mining/PoolStats.cs#L30

This is simply an oversight on my part. On a production pool you usually have a couple dozens to hundreds of shares per second. Anyway it needs to be fixed.


"unable to add mic as InputDevice"

I keep getting this error

I've been having the same issue.
It looks like a race condition because when we initialize a CameraEngine, it performs all session initialization in background (see setupSession() in CameraEngine.swift). As soon that the initializing part is over (on our end), we ask it to change the device (going from .Back to .Front) - causing a new call to addInputDevice that happens at the same time as the background initializer one.

@delannoyk I failed to follow up but never solved the problem.
Where you able to?

You can solve it by dispatching the call to the sessionQueue (requires modifying the source code)
public func changeCurrentDevice(position: AVCaptureDevicePosition) {
    self.cameraDevice.changeCurrentDevice(position)
    // Note: to avoid multi-threading crash
    dispatch_async(self.sessionQueue) { () -> Void in
        self.configureInputDevice()
    }
}


Corrected typo in autosaver.

Corrected typo in autosaver in a method openTempFilePrompt from "unusial" to "unusual".

Any objections?


Will unfinished span caused by some-deadlock be recognized?

we are using zipkin to inspect the database-sql-execute in lots of services. some-time, the sql-execution will dead-locked unpredictable. this dead-lock will leave some span un-finished.  It seems these un-finished span will not be report to database.
So if we using zipkin, how to easily recognize such sql-dead-lock problems  in lots of services?

No more feedback received since reporting, or perhaps it was solved through gitter chat.


Enable full state runs with scaling

Covasim can run pretty well with populations up to ~100k. This is statistically enough to model even multi-milion pops, assuming we can generate sample similar to overall population.
We can include functions that will automatically scale results and iterations to show numbers in real state population. So, create 100k pop, calculate fraction of actual pop (say, for 5M state, it's going to be 0.02) and multiply all numbers by it.
@Cliff one open question I'd have in this approach would be how to handle initial infections? Should we also scale them and round to ceiling? So for 10 initially infectious in 5M state, during sims we'll run 1? That will skew results quite drastically if I'm correct. Ideas?

We won't want to start from a single infection -- too much stochasticity anyway. This question is a good one, related to this:
https://github.com/amath-idm/covasim/issues/23

Idea @cliffckerr had and I'll prototype:
When we reach certain threshold of infections (say, 50%, but configurable) we scale up frame of reference by 2 (or later, configurable).
That means:

De-infect random 50% of infected
Resurrect 50% of dead
Re-susceptible(?) 50% of recovered
Multiply every summary stat by 2

@inc0 sounds good. the main change that needs to happen is to expand the definition of scale to have it be an array of length n_days rather than a scalar (or better, keep scale a scalar but introduce a scalevec parameter for this). but yeah basically just loop over 50% of all non-susceptibles and restore them to the susceptible state (no method for this currently, but should be added)

Doesn't scaling affect elimination dynamics and the time to the epidemic peak? You can't just re-scale the y-axis on your plots. And now if you want to start with 500 cases in the real population of 5,000,000 but it is implemented as 1 case in an actual population of 10,000 people, then you will get tons of stochasticity when there shouldn't be with 500 initial cases.

@dlchao it's a good point, we'll need to be able to scale down as well as up! but the idea is to get around exactly that -- we start with 500 cases in a population of 50k, and then when we get to 5000 cases we rescale to 500k, etc. -- so we never have fewer than 500 simulated cases but also never have more than 5000

Another thing is that we're rescaling by factor of 2 by default and scaling multiple times (every few days once epidemy picks up).
example: 10k internal population and 2M target population, start with 10 infections:

keep going until you reach 5000 or less susceptibles
cut make half of non-susceptibles susceptible again, multiply all numbers by 2
reach 10000 (5000 * 2), multiply by 2 again
repeat

as i mentioned in my comment on the PR we definitely don't want to let prevalence reach 50% -- maybe not even 5% -- before rescaling

I've set it up to 5% and it's configurable, we can experiment with even lower thresholds if need be

How do you make transmission trees when you have dynamic rescaling of the population?

you wouldn't be able to use transmission trees with this -- assuming you probably wouldn't be doing a full transmission tree for >1m infections

Sorry, I'm still novice in epi simulation and learning nomenclature, what do you mean by transmission trees? We don't touch contact network underneath, so one school will represent 2 schools after rescaling.

Right, this is mainly to produce numbers per day for big populations, probably good to document when not to use dynamic resampling


[Bug]:  Errors when call postgrest.update with Moshi codegen

General info

[X] I checked the troubleshooting page for similar problems

What is your question?
I make my own MoshiSerializer with below
defaultSerializer = MoshiSerializer(
                Moshi.Builder()
                    .add(ColorAdapter())
                    .add(LocalDateTimeAdapter())
                    .build()
            )

And I am not using the .addLast(KotlinJsonAdapterFactory()) because I want to use kotlin codegen as descriped in Moshi - Github, I add annotations to all my data class that used in supabase.
@JsonClass(generateAdapter = true)
data class Post(
    ...
)

but When I use the update, it shows me the error.
java.lang.IllegalArgumentException: No JsonAdapter for class kotlinx.serialization.json.JsonObject, you should probably use Map instead of JsonObject (Moshi only supports the collection interfaces by default) or else register a custom JsonAdapter.
     at com.squareup.moshi.ClassJsonAdapter$1.throwIfIsCollectionClass(ClassJsonAdapter.java:125)
     at com.squareup.moshi.ClassJsonAdapter$1.create(ClassJsonAdapter.java:69)
     at com.squareup.moshi.Moshi.adapter(Moshi.java:146)
     at com.squareup.moshi.Moshi.adapter(Moshi.java:106)
     at com.squareup.moshi.Moshi.adapter(Moshi.java:75)
     at com.squareup.moshi._MoshiKotlinExtensionsKt.adapter(-MoshiKotlinExtensions.kt:40)
     at io.github.jan.supabase.serializer.MoshiSerializer.encode(MoshiSerializer.kt:17)

How can i write the custom JsonAdapter or any other solutions?

yeah , it is working well with version 1.3.2-dev-1. Waiting for this version to release.

yeah , it is working well with version 1.3.2-dev-1. Waiting for this version to release.

Great! The release should be out pretty soon!


How can I animate the change of the widgets generated by the status widgets builders?

I would like to make a change more smooth, or animated, for example between firstProgressIndicatorBuilder to itemBuilder, I do not know if it is currently possible.
for example I think an AnimatedSwitcher would do the job.
https://github.com/EdsonBueno/infinite_scroll_pagination/blob/master/lib/src/ui/paged_sliver_builder.dart#L164-L208

Hi @basketball-ico.
That's not possible currently, but I liked your idea.
Maybe we can let developers choose with a parameter whether they want it animated or not.
I'll edit the issue title in order to make it a feature request.

Would also love to see this.

This is now available in the 3.1.0 release.
Thank you!


getTimePosition returns wrong value after calling goToPosition more than once

Bug Description
After calling goToPosition more than once, getTimePosition returns wrong value
How To Reproduce
await mpv.start()
await mpv.load('test.flv')
await mpv.goToPosition(10)
console.log(await mpv.getTimePosition())
await mpv.goToPosition(20)
// await new Promise((resolve) => setTimeout(resolve, 1000))
console.log(await mpv.getTimePosition())

The output is
10
10

Though the expected output is
10
20

If you call getTimeposition after a while, the output is roughly correct
10
20.433 (roughly)

Software Versions

Node-Mpv: Node-MPV-2, commit b4e39d1 (latest when writing)
MPV: mpv-x86_64-20200308-git-28ea1ed from SourceForge
OS: Windows 10, x64, 10.0.18362

Hey there, thanks for opening this issue and taking the time to fill out the template.
I just looked at the code and I'm pretty sure I know where the error is coming from, I will try to provide a fix asap.
Cheers

After a little thinking I decided to push this commit, it fixes your issue, however it still has another issue.
If the seek() or goToPosition()  method will go beyond the tracks duration, the promise will be resolved before the next track is started. That means that getTimePosition()will throw an error, because the property will not be available in that short moment, where no track is playing.
I thought a lot about it and unfortunately, it's very complicated to try to check for every edge case. I could check myself if, what is entered as seconds will skip the track, and skip it myself using   this.next() but then no seek event will be fired. I could try to check for all possible combinations of events happening, but that makes it really complex and prone to a lot of errors.
I also thought about having the two methods return a bool to mark, if the track was skipped or not by the entered value.
I still have to think about it, by any means, feel free to reopen this issue, if your problem still remains :) it helps a lot.

Thanks for your timely response.
It now works fine in the example. I will check edge cases in my code so the issue you mentioned shouldn't be a problem for me.
Yet another issue arises. When a user is dragging the progress bar, goToPosition() will be called frequently. It seems it takes a while for goToPosition() to resolve because it needs to wait for data to arrive.
    // Just use a loop to represent frequent calls caused by user's dragging action
    for (let i = 1; i <= 10; i++) {
    	await mpv.goToPosition(i)
    }

    console.log(await mpv.getTimePosition())

So I call them parallelly like this
    let seekingPromises = []

    for (let i = 1; i <= 10; i++) {
    	seekingPromises.push(mpv.goToPosition(i))
    }

    await Promise.all(seekingPromises)
    console.log(await mpv.getTimePosition())

My problem is, what mpv/node-mpv's behavior is when it receives another goToPosition()call while previous one is pending. Will they finally resolve in proper order?

My problem is, what mpv/node-mpv's behavior is when it receives another goToPosition()call while previous one is pending. Will they finally resolve in proper order?

That's a good question, to be honest I don't know. Node-MPV is just a wrapper around MPV's json ipc socket API. Since goToPosition is an absolute call, meaning that every subsequent call of goToPosition(100) will point to the 100 second mark, it should be fine.
Have you tried it? What does happen in your case?
I don't think there's a guarantee, that they will resolve in proper order to be honest, since they are executed in parallel.

It just works fine at least from the user’s point of view. Though the resolving order is indeed chaotic.
I think I should ignore the problem temporarily since it seems harmless.

I use goToPosition() on my appa s well and never noted this issue, actually.
We also have some kind of progressbar in place, and you can click that progress bar to go back or forward in the video.

@AxelTerizaki
You're really active in this project :)
I'm writing a video player capable of synchronizing multiple users based on electron and mpv. Your code actually helps me a lot.

Is that so? Which part of it did you use? :)
I'm just trying to help because I fiddled with it a lot and it's a central part of Karaoke Mugen so it's normal I help out Jan even if a little.

@AxelTerizaki
Sorry for the late reply.
I'm just a JS beginner so I learn many basic things from your code.
For example, how to use async function and the way work with node-mpv.


E0905 08:27:54.318826       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:27:59.324674       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0905 08:28:00.324427       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
E0905 08:28:04.325323       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:28:09.328599       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:28:14.334846       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:28:19.335425       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:28:24.339249       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:28:29.344837       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:28:30.325554       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: Error: 'dial tcp <IP_ADDRESS>:443: i/o timeout'
Trying to reach: 'https://<IP_ADDRESS>:443/openapi/v2', Header: map[]
I0905 08:28:30.325738       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0905 08:28:34.353742       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
I0905 08:28:35.382404       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
E0905 08:28:39.357140       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:28:44.362900       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:28:49.363586       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:28:54.367842       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:28:59.373741       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:29:04.377102       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:29:05.383398       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: Error: 'dial tcp <IP_ADDRESS>:443: i/o timeout'
Trying to reach: 'https://<IP_ADDRESS>:443/openapi/v2', Header: map[]
I0905 08:29:05.383543       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0905 08:29:09.379964       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:29:14.385691       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0905 08:29:15.386815       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
E0905 08:29:19.386415       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:29:24.390118       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:29:29.395293       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:29:34.398331       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:29:39.401547       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:29:44.408221       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:29:45.387682       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: Error: 'dial tcp <IP_ADDRESS>:443: i/o timeout'
Trying to reach: 'https://<IP_ADDRESS>:443/openapi/v2', Header: map[]
I0905 08:29:45.388070       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0905 08:29:49.409295       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0905 08:29:50.445377       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
E0905 08:29:54.412934       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:29:59.417707       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:30:04.420670       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:30:09.423931       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:30:14.432440       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:30:19.432788       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:30:20.446450       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: Error: 'dial tcp <IP_ADDRESS>:443: i/o timeout'
Trying to reach: 'https://<IP_ADDRESS>:443/openapi/v2', Header: map[]
I0905 08:30:20.446468       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0905 08:30:24.437765       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
I0905 08:30:27.302380       1 client.go:352] parsed scheme: ""
I0905 08:30:27.302526       1 client.go:352] scheme "" not registered, fallback to default scheme
I0905 08:30:27.302586       1 asm_amd64.s:1337] ccResolverWrapper: sending new addresses to cc: [{<IP_ADDRESS>:2379 0  <nil>}]
I0905 08:30:27.302652       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>}]
I0905 08:30:27.311738       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>}]
W0905 08:30:27.312071       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
W0905 08:30:27.312424       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
I0905 08:30:27.315022       1 client.go:352] parsed scheme: ""
I0905 08:30:27.315104       1 client.go:352] scheme "" not registered, fallback to default scheme
I0905 08:30:27.315169       1 asm_amd64.s:1337] ccResolverWrapper: sending new addresses to cc: [{<IP_ADDRESS>:2379 0  <nil>}]
I0905 08:30:27.315780       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>}]
I0905 08:30:27.323314       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>}]
W0905 08:30:27.323642       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
W0905 08:30:27.323936       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
E0905 08:30:29.443442       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0905 08:30:30.202752       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
E0905 08:30:34.448603       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
I0905 08:30:38.677338       1 client.go:352] parsed scheme: ""
I0905 08:30:38.677465       1 client.go:352] scheme "" not registered, fallback to default scheme
I0905 08:30:38.677534       1 asm_amd64.s:1337] ccResolverWrapper: sending new addresses to cc: [{<IP_ADDRESS>:2379 0  <nil>}]
I0905 08:30:38.677611       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>}]
I0905 08:30:38.687983       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>}]
W0905 08:30:38.688404       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
W0905 08:30:38.693026       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
I0905 08:30:38.709996       1 client.go:352] parsed scheme: ""
I0905 08:30:38.710211       1 client.go:352] scheme "" not registered, fallback to default scheme
I0905 08:30:38.710283       1 asm_amd64.s:1337] ccResolverWrapper: sending new addresses to cc: [{<IP_ADDRESS>:2379 0  <nil>}]
I0905 08:30:38.710365       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>}]
I0905 08:30:38.720942       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>}]
W0905 08:30:38.721313       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
W0905 08:30:38.721618       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
I0905 08:30:38.733197       1 client.go:352] parsed scheme: ""
I0905 08:30:38.733342       1 client.go:352] scheme "" not registered, fallback to default scheme
I0905 08:30:38.733429       1 asm_amd64.s:1337] ccResolverWrapper: sending new addresses to cc: [{<IP_ADDRESS>:2379 0  <nil>}]
I0905 08:30:38.733531       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>} {<IP_ADDRESS>:2379 <nil>}]
I0905 08:30:38.807918       1 asm_amd64.s:1337] balancerWrapper: got update addr from Notify: [{<IP_ADDRESS>:2379 <nil>}]
W0905 08:30:38.809671       1 asm_amd64.s:1337] Failed to dial <IP_ADDRESS>:2379: grpc: the connection is closing; please retry.
E0905 08:30:39.451425       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:30:44.459178       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:30:49.459956       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:30:54.501775       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:30:59.502101       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:31:00.204182       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: Error: 'dial tcp <IP_ADDRESS>:443: i/o timeout'
Trying to reach: 'https://<IP_ADDRESS>:443/openapi/v2', Header: map[]
I0905 08:31:00.204294       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
E0905 08:31:04.508701       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I0905 08:31:05.509838       1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
E0905 08:31:09.510148       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:31:14.518403       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:31:19.518687       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:31:24.525698       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Get https://<IP_ADDRESS>:443: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
E0905 08:31:24.543563       1 available_controller.go:353] v1beta1.metrics.k8s.io failed with: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.metrics.k8s.io": the object has been modified; please apply your changes to the latest version and try again
E0905 08:31:35.510550       1 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: Error: 'dial tcp <IP_ADDRESS>:443: i/o timeout'
Trying to reach: 'https://<IP_ADDRESS>:443/openapi/v2', Header: map[]
I0905 08:31:35.510707       1 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
 I also had to pass IP address as --node-name parameter to the agent join command:
--node-name <IP_ADDRESS> --address <IP_ADDRESS> --internal-address <IP_ADDRESS>

Strange though that otherwise one node worked and the other one node not. Because both k8s-node-2 and k8s-node-3 DNS names are not resolvable.


How to Control the Frequency Interval of Until Activity

I want Until Activity to Run every 30 seconds or whatever the amount i decide for it could be 1 minute 5 minute and so on. How can i configure that Interval. Currently i have to use Wait and Until activity together

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 73ff4df3-ae31-335a-bf60-b97dbe98fa58
Version Independent ID: 1c769558-db53-2e3d-f3b7-cdb23c2689d8
Content: Until activity in Azure Data Factory
Content Source: articles/data-factory/control-flow-until-activity.md
Service: data-factory
GitHub Login: @sharonlo101
Microsoft Alias: shlo

@nss12 Thank you for the valuable feedback,we are investigating the issue.

@nss12 Unfortunately the until activity doesn't provide scheduling on its own. You have to use it in conjunction with Wait activity to achieve scheduling.

@nss12 I will now proceed to close this issue. Please feel free to comment and tag me for any more queries.


Question about 'TORCH_HOME'

Good job! Is the parameter 'TORCH_HOME' required?

'TORCH_HOME' could be a local path to save weights of pre-trained models provided by pytorch. Actually, the default path is "/username/.cache/torch/hub/checkpoints".


⚠ Bofrost.lu has degraded performance

In fc6e230, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 2240 ms

Resolved: Bofrost.lu performance has improved in b5dce55 after 9 minutes.


Polyglot seems broken on ProseMirror's journal pages?

I no longer see the Polyglot font choice in the journals.  We used to be able to give handouts in "languages" but that appears to only be an option in chat at the moment.

I have tried disabling all of the modules except for polyglot and libwrapper and it shows the same font choices.

The new journal editor, ProseMirror, currently isn't module-friendly as the old one. For now, the solution is to change the Journal Editor to TinyMCE.

The new journal editor, ProseMirror, currently isn't module-friendly as the old one. For now, the solution is to change the Journal Editor to TinyMCE.

Would that interfere with Monk's Enhanced Journals do you know?

TinyMCE doesnt work with V11.  any other options?

TinyMCE doesnt work with V11. any other options?

Use the TinyMCE setting inside Monk's Enchanced Journals, not the TinyMCE module.

I am not sure about @grepheavy's issue. It works fine on a world with only Polyglot running.
Anyway, I've found out a way to get it running on ProseMirror today (based on a PF2e change posted on the FVTT discord).
It has its issues, but it is not a "100% not supported" thing anymore.


Paa Error

I run paa_r101_fpn_2x_coco in torch:1.6, mmcv:1.2.0 and mmdet:2.6.0, the code report error:
I try in  torch:1.6, mmcv:1.1.6 and mmdet:2.4.0, the code also report error.
/home/zgz/mmdetection-2.6.0/mmdetection-2.6.0/mmdet/models/dense_heads/paa_head.py:485: UserWarning: This overload of nonzero is deprecated:
	nonzero()
Consider using one of the following signatures instead:
	nonzero(*, bool as_tuple) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:766.)
  pos_inds.append(pos_mask.nonzero().view(-1))
[W TensorIterator.cpp:924] Warning: Mixed memory format inputs detected while calling the operator. The operator will output channels_last tensor even if some of the inputs are not in channels_last format. (function operator())
[W TensorIterator.cpp:918] Warning: Mixed memory format inputs detected while calling the operator. The operator will output contiguous tensor even if some of the inputs are in channels_last format. (function operator())
2020-11-23 19:22:17,484 - mmdet - INFO - Epoch [1][50/58633]	lr: 9.890e-04, eta: 10 days, 15:56:31, time: 0.655, data_time: 0.054, memory: 5105, loss_cls: 1.5923, loss_bbox: 0.7098, loss_iou: 0.3477, loss: 2.6498
2020-11-23 19:22:49,508 - mmdet - INFO - Epoch [1][100/58633]	lr: 1.988e-03, eta: 10 days, 13:07:58, time: 0.640, data_time: 0.007, memory: 5105, loss_cls: 1.2279, loss_bbox: 0.6568, loss_iou: 0.3483, loss: 2.2330
2020-11-23 19:23:19,759 - mmdet - INFO - Epoch [1][150/58633]	lr: 2.987e-03, eta: 10 days, 7:34:17, time: 0.605, data_time: 0.006, memory: 5105, loss_cls: 1.5554, loss_bbox: 0.7115, loss_iou: 0.3505, loss: 2.6174
Traceback (most recent call last):
  ... ...
  File "/home/zgz/mmdetection-2.6.0/mmdetection-2.6.0/mmdet/models/dense_heads/paa_head.py", line 155, in loss
    anchor_list,
  File "/home/zgz/mmdetection-2.6.0/mmdetection-2.6.0/mmdet/core/utils/misc.py", line 26, in multi_apply
    return tuple(map(list, zip(*map_results)))
  File "/home/zgz/mmdetection-2.6.0/mmdetection-2.6.0/mmdet/models/dense_heads/paa_head.py", line 343, in paa_reassign
    gmm.fit(pos_loss_gmm)
  File "/usr/local/python/lib/python3.7/site-packages/sklearn/mixture/_base.py", line 193, in fit
    self.fit_predict(X, y)
  File "/usr/local/python/lib/python3.7/site-packages/sklearn/mixture/_base.py", line 246, in fit_predict
    self._m_step(X, log_resp)
  File "/usr/local/python/lib/python3.7/site-packages/sklearn/mixture/_gaussian_mixture.py", line 679, in _m_step
    self.covariances_, self.covariance_type)
  File "/usr/local/python/lib/python3.7/site-packages/sklearn/mixture/_gaussian_mixture.py", line 333, in _compute_precision_cholesky
    raise ValueError(estimate_precision_error_message)
ValueError: Fitting the mixture model failed because some components have ill-defined empirical covariance (for instance caused by singleton or collapsed samples). Try to decrease the number of components, or increase reg_covar.

I don't know which cause the warn Warning: Mixed memory format inputs detected while calling the operator. The operator will output channels_last tensor even if some of the inputs are not in channels_last format. (function operator()).

Hi @as754770178
What is your batch size? When the batch size is too small, the model may diverge.

my batchsize is 2. I only have one GPU, so what cause this error? I want to avoid it by myself.

Hi @as754770178
When the batch size is too small, the model may diverge.
You may use the provided checkpoint as pretrain.

I not find the resnext152 pth.


Update search.md

Overwrite with new ones_zh

Hi @arthuo , Thank you for your contribution again. Please check the comments I've made. There is a small conflict since last merge, could you please also check it out?

Hi @arthuo , Thank you for your contribution again. Please check the comments I've made. There is a small conflict since last merge, could you please also check it out?

Hi, thank you very much for your careful review and feedback. I have made corresponding changes to the submitted documents based on the feedback.

That's very nice of you to do that. Merging the PR.


Feature Request: Component References

I would like the documentation to indicate which components are used by other components.
If this cannot be generated automatically, a manual link (like JsDoc @see directive) would work.
If this feature is already available, I apologize. Could you give me a pointer or example? Thanks.

Any position about this request?

The referencing relationship will be a tree structure, do you have any good ideas for showing this section?


Chat update and delete chat thread properties

Added the below operations:

Update Chat Thread Properties
Delete Chat Thread

If this is your first time submitting to GitHub and you need some help, please sign up for this session.

[ ] I attest that the connector doesn't exist on the Power Platform today. I've verified by checking the pull requests in GitHub and by searching for the connector on the platform or in the documentation.
[x] I attest that the connector works and I verified by deploying and testing all the operations.
[x] I attest that I have added detailed descriptions for all operations and parameters in the swagger file.
[x] I attest that I have added response schemas to my actions, unless the response schema is dynamic.
[ ] I validated the swagger file, apiDefinition.swagger.json, by running paconn validate command.
[x] If this is a certified connector, I confirm that apiProperties.json has a valid brand color and doesn't use an invalid brand color, #007ee5 or #ffffff. If this is an independent publisher connector, I confirm that I am not submitting a connector icon.

If you are an Independent Publisher, you must also attest to the following to ensure a smooth publishing process:

[ ] I have named this PR after the pattern of "Connector Name (Independent Publisher)" ex: HubSpot Marketing (Independent Publisher)
[ ] Within this PR markdown file, I have pasted screenshots that show: 3 unique operations (actions/triggers) working within a Flow. This can be in one flow or part of multiple flows. For each one of those flows, I have pasted in screenshots of the Flow succeeding.
[ ] Within this PR markdown file, I have pasted in a screenshot from the Test operations section within the Custom Connector UI.
[ ] If the connector uses OAuth, I have provided detailed steps on how to create an app in the readme.md.

@vmanoharas Looks like validations are passed

Hello @vmanoharas , I have created a new PR. Here's a link to that: https://github.com/microsoft/PowerPlatformConnectors/pull/3311

Hello @anmolbohra97,
As you created a new PR and the same is approved, so I am closing this PR.


How to get  row from data-item and vice versa?

Hi
Thanks for your great job!
I wonder how can i get grid row from data-item or date-item from a row?
In a loop over my data or grid rows i want to add a css class to the related <tr> based on value of a property of data-item. i mean something like conditional formatting.
I think it would be nice to have an event fired on every row added to the grid.

To set class to the row use the rowClass callback http://js-grid.com/docs/#rowclass
Get item by row:
var item = $row.data("JSGridItem");

Get row by item:
var $row = $jsGrid.jsGrid("_rowByItem", item);

Could you share your use cases, to know whether we need public API for this.

rowClass did the job for me!
Thanks.

can u please elaborate ,how  can i use rowClass for row coloring

Just define css styles like following
.my-row-custom-cls td {
    background: red;
}

Hi,,,thaxs fr de rply,,I gt de thing for row previously,but can u help me hw to clr a particular cell?

Please, open a separate issue clearly stating your question.


How to derive the negative edge from a clock

I have a myhdl background and maybe that's why this seems like a very easy task to me.
in myhdl you can define a elaboratele thing like this (http://docs.myhdl.org/en/latest/manual/reference.html#myhdl.always_seq)
@always_seq(clk.negedge, reset=reset)
def logic():
    self.output = self.input
return logic

I can't seem to find an easy way to do something similar in nmigen, without creating a new clockdomain, registering it in my module (as well as defining a clock source, ...).
I would like to write something like this:
m.d.sync.neg += self.output.eq(self.input)

What is the idiomatic way to do this?
(Ps, if you open the discussion tab, you can easily move an (/this) issue to discussions, so they don't litter the issue tab ;) )

Currently this is done by creating a new clockdomain, driving it by the inverse of the clocksignal and adding it to your module.

Yep, creating a new clock domain is the idiomatic way (arguably, it is a new domain--at least if you include the polarity in the control set, which I do). You probably want to drive it with the exact same clock signal though, not the inverse.

Whoops yes, I completely missed the clk_edge=... argument of ClockDomain, sorry!

RobertBaruch has exactly this topic covered in his tutorial:
https://github.com/RobertBaruch/nmigen-tutorial/blob/master/3_modules.md#tip-clock-domains-with-the-same-clock-but-different-edges
I should go a read more of him, thanks everybody.