[Go Live][Sept 29]

Push to live

Learn Build status updates of commit 0a1efc1:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

msteams-platform/tabs/how-to/create-personal-tab.md
:white_check_mark:Succeeded
View

msteams-platform/TOC.yml
:white_check_mark:Succeeded
View

msteams-platform/toolkit/teams-toolkit-fundamentals.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


[Bug]

Describe the bug
I was unable to run the source code in visual studio.
Please How to run Source code in visual studio. How to setup envoirment.
Desktop (please complete the following information):

OS: [e.g. Windows 10 1809 17763.593]
Version [e.g. v<IP_ADDRESS>]

https://github.com/JasonStein/Notepads/blob/master/CONTRIBUTING.md


Generated Id

Cannot compare roomNoList.get(i) and id (both are integers). I get "java.lang.String cannot be cast to java.lang.Integer" error.

Solved.


Adapt nginx port to 81

To avoid struggle with native installed nginx on the server.
@chrismayer @JakobMiksch @weskamm

Does the port need to be exposed at all? Only the workers are using that nginx via its name, so the port does not to need to be opened i think.

@weskamm good point. Indeed for the moment only the pygeoapi and the GeoServer use the COG directory. But we thought it might be useful for users to have access to the raw COGs as well.
But the 'mocked webspace' indeed does not need to be available public at all

Thanks, all remarks resolved.

after discussing, all was well


[Word] Add citation management sample

Q
A

Bug fix?
no

New feature?
no

New sample?
yes

Related issues?
none

What's in this Pull Request?
Adds a sample that showcases using a Word add-in for citation management in a Word document.

Everything seems to be working fine in the latest iteration.


npm run dev does not return to cli in laravel-mix

What version of @tailwindcss/jit are you using?
v0.1.5
What version of Node.js are you using?
v14.11.0
What browser are you using?
N/A
What operating system are you using?
macOS
Reproduction repository
https://www.youtube.com/watch?v=3O_3X7InOw8
When you add the @tailwindcss/jit postcss processor in webpack.mix.js example:
const mix = require('laravel-mix');

mix.js('resources/js/app.js', 'public/js')
    .vue()
    .postCss('resources/css/app.css', 'public/css', [
        require('postcss-import'),
        require('postcss-nested'),
        require('@tailwindcss/jit'),
        require('autoprefixer'),
    ])
    .webpackConfig(require('./webpack.config'));

if (mix.inProduction()) {
    mix.version();
}

When you run npm run dev the process "hangs" and not exited.
This happens like npm run watch or npm run hot but then it reloads the code.
So when you run npm run dev you will have to CTRL+c to exit the process so you can type again. This should not happen with npm run dev :)

Hey! This is expected albeit unintuitive, but it's covered in the README:

Make sure you set NODE_ENV=development if you are running a watcher, or Tailwind won't watch your template files for changes. Set NODE_ENV=production for one-off builds.
If you want to control whether Tailwind watches files or not more explicitly, set TAILWIND_MODE=watch or TAILWIND_MODE=build to override the default NODE_ENV-based behavior.
For example if you want to do one-off builds with NODE_ENV=development, explicitly set TAILWIND_MODE=build so that Tailwind knows you are just doing a one-off build and doesn't hang.

Add the TAILWIND_MODE=build environment variable if you want to do one-off development builds instead of running a dev server 👍🏻
Duplicate of #113.

Ahh! I totally missed that part in reading the documentation!!!
Thank you for the extreme fast response!
I will read the docs better next time!
Really loving jit tough.

Just in case it helps anyone else using Mix who doesn't necessarily want/need separate dev/prod build processes you can amend the watch script in your package.json's scripts section as follows to do a full production build locally on watch:
"watch": "NODE_ENV=production TAILWIND_MODE=watch mix watch"
The speed of JIT makes this a totally feasible way of working in my opinion... at least for simple projects that build very fast and don't require a separate npm run prod.  The only downside is that you obviously end up with minified files locally, but for me that's a small price to pay to have a nice simple singular build process. (I also like the fact that I'm testing the minified files locally to avoid any nasty minifier surprises when pushing to production).


Support OpenID Connect auth

We switched to CoreOS/Dex recently which uses OpenID for auth.
Using ktail was generating an error:

No Auth Provider found for name "oidc"

This PR should fix that issue (we just need to import it to register the oidc plugin).
It might actually be better to import all plugins:
See: https://github.com/kubernetes/client-go/blob/master/plugin/pkg/client/auth/plugins.go

@atombender could we get this merged and released to support OIDC?

I've merged it until we can add the suggested change.


[SPARK-16508][SparkR] Fix CRAN undocumented/duplicated arguments warnings.

What changes were proposed in this pull request?
This PR tries to fix all the remaining "undocumented/duplicated arguments" warnings given by CRAN-check.
One left is doc for R stats::glm exported in SparkR. To mute that warning, we have to also provide document for all arguments of that non-SparkR function.
Some previous conversation is in #14558.
How was this patch tested?
R unit test and check-cran.sh script (with no-test).

Test build #64015 has started for PR 14705 at commit 8d60676.

Test build #64015 has finished for PR 14705 at commit 8d60676.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64015/
Test PASSed.

Thanks @junyangq  -- I just ran the check on branch-2.0 with this PR and in addition to glm there were two warnings for ... in first and unpersist that we can fix in this PR.
For glm its kind of unfortunate but I think we need to just go ahead and add those args and say that those are not used by SparkR.  Is there any other workaround you have in mind ? We can do that in a separate PR if required.
Undocumented arguments in documentation object 'first'  ‘...’
Undocumented arguments in documentation object 'glm'
  ‘weights’ ‘subset’ ‘na.action’ ‘start’ ‘etastart’ ‘mustart’ ‘offset’
  ‘control’ ‘model’ ‘method’ ‘x’ ‘y’ ‘contrasts’ ‘...’
Undocumented arguments in documentation object 'unpersist'  ‘...’

I don't have a good answer in mind now. I'm not sure how much we gain by making glm also applicable to SparkDataFrame as well. Would spark.glm be enough, like other ML methods?

Test build #64019 has started for PR 14705 at commit b0e8459.

Yeah I think we will be more careful about adding new algorithms to override existing R methods -- but given that glm is already exposed I'd think we can make an exception for just this one. @mengxr Any thoughts on this ?

Test build #64019 has finished for PR 14705 at commit b0e8459.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64019/
Test PASSed.

@shivaram BTW do you mind if I ask if we have R style guidelines somewhere please? I remember I made a PR for R referencing only the other codes. It seems it is also missing in https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide.

@shivaram I found perhaps a neat way to document R'glm if we don't want to remove it is to use @inheritParams stats::glm. That will bring in all the parameters from stats::glm not listed in SparkR's glm. That also means we need slight modification of the data description: something like "a SparkDataFrame or R's glm data for training."

@HyukjinKwon - we don't have a coding style guide for R. We have some style check with lint-r.
In addition, the document style you are looking at is a bit different from coding style - this document style I'm planning to write one after this is merged. Perhaps a coding style could be good too, for things like eg. "what to do with method without parameter".

@inheritParams would be the way to go.

looking good - looks like we are very close.

@felixcheung Thanks for kind explanation. BTW, it'd be great too if it just has a sentence, for example, "For R code, Apache Spark follows lint-r" in the wiki just like Python has "For Python code, Apache Spark follows PEP 8 with one exception: lines can be up to 100 characters in length, not 79." for just correctness and references for new contributors if it makes any sense maybe :).

surely - we should have said lint-r as the baseline. There's definitely more we could add though. It would be great if we have bandwidth to write more linters at some point.

Test build #64047 has started for PR 14705 at commit 870279a.

Test build #64047 has finished for PR 14705 at commit 870279a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64047/
Test PASSed.

Merged build finished. Test PASSed.

Hmm, I'm beginning to think we could do this:
generics.R
setGeneric("spark.naiveBayes", function(data, formula, ...) { standardGeneric("spark.naiveBayes") })

mllib.R
#' @param ... additional argument(s) passed to the method. Currently only \code{smoothing}.
 setMethod("spark.naiveBayes", signature(data = "SparkDataFrame", formula = "formula"),
           function(data, formula, smoothing = 1.0) {
           # note, no `...`

That we wouldn't need ... in the function definition?
I'd argue this because:
> a <- function(correct, ...) { print("foo") }
> a(correct = "a", wrong = "b")
[1] "foo"
> a(correcy = "b")
[1] "foo"

# versus
> a <- function(correct) { print("foo") }
> a(correcy = "b")
Error in a(correcy = "b") : unused argument (correcy = "b")
> a(wrong = "b")
Error in a(wrong = "b") : unused argument (wrong = "b")

With ... in the function definition, typo or misuse would not be detected easily.
Would that be better off? what do you think?

Yeah, I totally agree that in terms of usage this is safer. Then the doc for ... would be an issue. If we keep to the principle that doc be close to the function, then ... would be in generic part. That  seems not what we want to do. Putting it above the function definition is also a little weird since there is not ... in the arguments of the function itself. No sure which is the best way though...

I think it's ok to put @param ... on top of the function in the order we want it. Or @param na.rm for sd, var etc. Yes it is a bit odd to have param in the documentation block that is not in the immediate signature, but to me it seems better than making the API contract weaker just so we could document it (or get some warnings to go away).
We could consider this as you need to include both function definition and generic definition for documentation and the place to do that is on top of the function definition (which is just for maintainability and convenient).
Other packages have opted to maintain separate Rd files which could have other issues.
I don't want to drag this PR on though. We could address the warnings now and revisit later - I am concerned with adding params to function signature that weaken it.

That makes sense. Perhaps this could be done in another PR?

Agree with @felixcheung . Let do the setGeneric(, ...) and do not use ... in function definition if it is not required. Note that having every param documented is not a strict requirement of CRAN submission. It would be great if we can merge this PR fist and fix minor issues after since it touches many files in SparkR.

Test build #64107 has started for PR 14705 at commit be767f0.

Test build #64107 has finished for PR 14705 at commit be767f0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64107/
Test PASSed.

Yes agreed. I will do one more pass tonight and merge if there isn't any blocking issue.
Thanks!

Thanks @felixcheung!

I think a subset of this should go to 2.0.1 as well (as requirement to fix warning for CRAN in 2.0.x), but it's a non-trivial port: mllib isoreg are new in 2.1.0 only.
What's the best way to proceed?

Yeah so we can do a couple of things. One is we try to cherry-pick this PR to branch-2.0 and then fix all the merge conflicts that are thrown. I think that should handle cases where the method doesn't exist in 2.0 etc.
The other option is to create a new PR that is targeted at branch-2.0 (i.e. the cherry-pick / merge can be done as a part of development) and then we can review, merge it.
Let me know if you or @junyangq want to try the second option -- If not I can try the first one and see how many conflicts there are.

Thanks.
Reviewing each change, I think we need this PR (14705) and PR #14734 in 2.0.1 - so maybe only a few lines of conflicts.

I cherry-picked this PR into branch-2.0 in https://github.com/apache/spark/commit/0297896119e11f23da4b14f62f50ec72b5fac57f -- The merge was a little awkward and but I think I got it to work correctly. The CRAN checks look fine to me locally, but it would be good if somebody else can also verify this.


Create hactoberfest

Amazing experience to pull request

Thank you, for opening this pull request.
This is a standard message notifying you that we've reviewed your pull request and have decided not to merge it. We would welcome future pull requests from you.
Thank you, and happy coding.

Hope to contribute further


Can this only be tested locally? Can I test the server-side interface on the server?

Can this only be tested locally? Can I test the server-side interface on the server?

@zhangtiedong0912 There are no requirements that the client and server be run on the same host
If you can elaborate on the test you're trying to perform, we may be able to offer some clearer guidance.


Transcript of Fortran programs

Hey, I had a transcript of some of the Fortran programs in the book here. You might find them helpful to verify your Python program versions against. Do you accept pull requests? I might be able to contribute.

Hey @cibinjoseph that's great! I've transcribed most of them as well, here: https://github.com/alwinw/KatzPlotkinPy/tree/master/tests/f77
I think I'm only missing programs 15 and 16 and would be happy to accept a pull request for them, thanks!

Program 15 has been added in pull request #16


Matlab: basic pointcloud demo

pointcloud_example - basic depth pointcloud representation with Matlab. no texture mapping.

#2576

Add to readme?

I added the documentation is in the latest commit, the rest appear updated due to rebase


Audio file and binks selector as part of the AICE and eventually having spaces for new AIs.

I think there should be a feature as part of the AICE where we can select any audio files and binks in the game to be used for any new AIs we may create to replace existing ones. The speech files and binks do add a lot of character to AIs I think and it would be good if we could use the ones we wanted from the game for our new AIs easily with the AICE.
Also eventually there should be spaces for completely new AIs rather than having to replace existing AIs with new tactics but I do think that would be hard to do.

Maybe you an also change the AI pictures in my GM1 Converter and add to the new AIV the GM1 File with new pictures of the lords.


Changed CMakeLists to work with ROOT 5 and 6

Travis is using repositories to install root-system and not setting enviroment variables. Therefore a little workaround was needed - proper cmake file is downloaded.
However user does not need to download that file - after running script bin/thisroot.[c]sh every ROOT connected enviroment variable is set and cmake will find apropriate files on it's own

Every time I was running framework it was with ROOT6 - even with previous releases (for example at the time of writing HitFinder module). And Unpacker worked then - however I have not made any tests related to his pull request.
As for the Travis tests - that would require either downloading approximately 100MB precompiled binary from ROOT web page and instaling it or making our own .deb package and then downloading it. Both ways seems doable.

I think downloading from the official site would be ok.
Can you try to use this version with ROOT6 and check that everything is working according to this instruction:
http://sphinx.if.uj.edu.pl/~alek/framework_integration_tests/framework_integration_tests.txt

run all tests and check that there are no errors.

Unfortunately there seems to be error with unpacker:
"Error exec():The hld file doesnt exist: àŸi"
Log in the attachment
JPet_a8363fa5-8375-492a-b354-7bcc7367098d.log

This version is working on my PC with
ROOT 6.08/06
g++ 5.4
gcc
It also builds .pcm dictionaries in proper directory
Tested with Alek's integration tests - I need access to jenkins to fix that checks

All tests are defined in jenkins/root_6/docker-entrypoint.sh file, so to change tests, you do not need direct access to Jenkins.
Line: "rc=$?; if [[ $rc != 0 ]]; then exit $rc; fi" is just for returning error code if some statement fails, and should be added after every command(later I will wrap it into function).
All environment are defined in Dockerfile.
For now it seems that jenkins can't find root and trying to use fallback, but it is not here:
CMake Error at CMakeLists.txt:20 (find_package):
By not providing "FindROOT.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "ROOT", but
CMake did not find one.
Could not find a package configuration file provided by "ROOT" with any of
the following names:
ROOTConfig.cmake
root-config.cmake

Add the installation prefix of "ROOT" to CMAKE_PREFIX_PATH or set
"ROOT_DIR" to a directory containing one of the above files.  If "ROOT"
provides a separate development package or SDK, be sure it has been
installed.
Do you have some special environment variables set?

Only thing I can see right now is that test failed - don't know why or where.
As for the environment variables - during compilation of ROOT user is requested to execute script "thisroot.sh" (and add it to bash.rc). This script sets "CMAKE_PREFIX_PATH" variable to ROOT directory and therefore to proper file FindROOT.cmake. That file differs between different ROOT versions so it should be provided by ROOT version installed on users PC.

Ok, sorry, I forget I didn't set up access for your jenkins account. It is done, can you check, do you can view jobs/console output in jenkins?
I'm executing "source /root-system/bin/thisroot.sh"  before "cmake ..".

Yes, I can see console log now.
From what I see now root is build via ./configure rather than CMake - maybe that is causing problem (I'm compiling ROOT in the way it is build on Jenkins to see if file exists, update when it is finished)

After compiling and installing ROOT with the same procedure as in Jenkins (and in the same location) CMAKE_PREFIX_PATH is set to "/root-system". And I'm getting the same error.
However, ROOT version installed by CMake, in location: "/home/user/root_install_dir" has CMAKE_PREFIX_PATH set to that location. And compilation runs without problems.
That suggests that CMake is not able to access/find file in location "/root-system". I have no idea why.


Radio group updates

Description

Adds dark theme for radio groups
Adds new "grid" layout, with new option of "columns" to render (updated docs too)

Related Issue(s)
Required as a pre-cursor to: https://github.com/flowforge/flowforge/issues/2362
Checklist

[x] I have read the contribution guidelines
[x] Documentation has been updated

local testing all good.


Update action to use Node.js 16 (or later) (Github deprecation warning)

Since yesterday, Github flags using this action with the following deprecation warning below:
Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: philips-software/inner-source-checkout-action
As our repository uses internal repositories as submodules, we depend on this action to (continue to) work.

Using the latest release (v1.2.1) I still receive this deprication warning. I noticed that the main branch is newer and changed from node12 to node16. However, no new release has been made with these updates.
@JeroenKnoops  @Brend-Smits
Could you make a release which includes the latest updates, at least with the update to node16?

You're right. I will check and update some documentation and will make a release.


Extension not working

I have downloaded the extension, restarted my vs, and waited for it to finish downloading it. vs shows it as enabled and working but it looks just like it is unmodified. any help would be appreciated.

Vs 19 or 22?


Add cached state

This PR is a big rewrite of the core functionality. Now the tags and entries are stored locally to a file in a OS cache directory. This allows for much better output and faster execution.
Closes: #6
Closes: #29

I'm unsure how to add a pull request to a pull request, but just a suggestion on adding color to the help message.
File: opt.rs
// --snip--

use clap::{Clap, AppSettings};
use wutag_core::Error;

pub const APP_NAME: &str = "wutag";
pub const APP_VERSION: &str = "0.4.0";
pub const APP_AUTHOR: &str = "Wojciech Kępka<EMAIL_ADDRESS>pub const APP_ABOUT: &str = "Tool to tag and manage tags of files.";

#[derive(Clap)]
#[clap(
    version = APP_VERSION,
    author = APP_AUTHOR,
    about = APP_ABOUT,
    global_setting = AppSettings::ColoredHelp,
    global_setting = AppSettings::ColorAlways,
)]

pub struct Opts {
    #[clap(short, long)]

// --snip--

Thanks for checking out the PR! This is very much helpful :) I added the color to the help message - it's a worthy addition and fits well with the style of wutag.
I also updated how wutag rm and wutag clear works, it should now delete the tags from cache even when it fails to untag them on the file system. If you have any other issues please let me know. I will leave this PR open for now while I'll still polish it and cleanup the code.

The issue still isn't resolved and I get the same behavior when using rm or clear. I checked to make sure I was building the correct commit by checking the git log output.
I can run the following command many times in a row and nothing happens except for this output. I'm unsure if it is both deleting the xattrs and the path in the registry. Here's the output:
wutag rm '*.sh'

/Users/lucasburns/test/clap/count-tests.sh:

/Users/lucasburns/test/aaa.sh:

saving to /Users/lucasburns/Library/Caches/wutag.registry

It is different with this command (I think this is how it's supposed to be used). If the command is ran again, I get an error saying that the tag doesn't exists, but when listing the files and the tags (these are the only files with the tags) they still show up.
wutag rm '*.sh' shell2

/Users/lucasburns/test/clap/count-tests.sh:
        X shell2
/Users/lucasburns/test/aaa.sh:
        X shell2
saving to /Users/lucasburns/Library/Caches/wutag.registry

wutag rm '*.d' jjj

/Users/lucasburns/projects/rust/repos_example/wutag/target/release/libwutag_core.d:
        ERROR tag `jjj` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/target
/release/libwutag_core.d

/Users/lucasburns/projects/rust/repos_example/wutag/target/release/wutag.d:
        ERROR tag `jjj` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/target
/release/wutag.d

/Users/lucasburns/projects/rust/repos_example/wutag/target/release/deps/anyhow-f4d68ea32d12b485.d:
        X jjj
/Users/lucasburns/projects/rust/repos_example/wutag/target/release/deps/walkdir-fdf07dfba53e1aa8.d
:
        X jjj

Using the following does work:
wutag edit <tag> --color 0x1f1f1f

Another issue I am having (though I am unsure if this is default behavior), is that whenever I list files or list tags, the entire system is traversed every time. And whenever I do:
wutag -d $PWD list files
# or
wutag -d . list files

the entire file system is also traversed.
If there is anything that you'd like me to try out to see if I can resolve the problem, I can.

cargo run
This post should be much more complete and show every command.
set -- Works
$ cargo run -- set 'runner.rs' rust                                                         ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag set runner.rs rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
        + rust
saving to /Users/lucasburns/Library/Caches/wutag.registry

edit -- Works

However, no error is thrown if there is an invalid tag given.

cargo run -- edit rust --color 0xFFFFFF                                                    ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag edit rust --color 0xFFFFFF`
rust ==> rust
saving to /Users/lucasburns/Library/Caches/wutag.registry

cp -- Displaying that it works, but doesn't copy
$ cargo run -- cp runner.rs config.rs
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag cp runner.rs config.rs`
/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
saving to /Users/lucasburns/Library/Caches/wutag.registry

However, listing files afterward does not display that file as being tagged:
$ cargo run -- list files
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/aaa.sh
/Users/lucasburns/test/clap/count-tests.sh

clear -- Backtrace
$ cargo run -- clear '*.rs'
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag clear '*.rs'`thread 'main' panicked at 'removal index (is<PHONE_NUMBER>219654000) should be < len (is 1)', library/alloc/src/vec/mod.rs:1355:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

rm -- Displays that it works, but doesn't
If the tag is set, when removing the tag the first time, no backtrace is given; however, if the command is ran again, a backtrace is given as shown below. This should not display the rust tag at all because it should've been deleted from the xattrs the first time.
Also, I'm not sure if the other errors are supposed to be shown or if it is just supposed to show the files that it is supposed to remove the tags from.
cargo run -- rm '*.rs' rust                                                               ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag rm '*.rs' rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/registry.rs:90:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
        X rust

After running the above command twice and getting that error message, when listing the files again, the path is still displayed:
$ cargo run -- list files                                                                   ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/aaa.sh

search -- This displays what the problem is
If a tag is set in the directory and I search for it, it is shown:
$ cargo run -- search 'rust'                                                                ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag search rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs: rust

If I remove the tag:
$ cargo run -- rm '*.rs' rust                                                               ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag rm '*.rs' rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/util.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/util.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/registry.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/registry.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
        X rust
/Users/lucasburns/projects/rust/repos_example/wutag/src/main.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/main.rs

saving to /Users/lucasburns/Library/Caches/wutag.registry

And search again, it is gone:
$ cargo run -- search 'rust'                                                                ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag search rust`

However, if I list the files, it is still there:
$ cargo run -- list files                                                                   ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/test/aaa.sh

The issue still isn't resolved and I get the same behavior when using rm or clear. I checked to make sure I was building the correct commit by checking the git log output.

I missed a few bugs, should be better now.

I can run the following command many times in a row and nothing happens except for this output. I'm unsure if it is both deleting the xattrs and the path in the registry. Here's the output:
wutag rm '*.sh'

/Users/lucasburns/test/clap/count-tests.sh:

/Users/lucasburns/test/aaa.sh:

saving to /Users/lucasburns/Library/Caches/wutag.registry

This should not display the paths anymore unless the tags are actually deleted.

Another issue I am having (though I am unsure if this is default behavior), is that whenever I list files or list tags, the entire system is traversed every time. And whenever I do:
wutag -d $PWD list files
# or
wutag -d . list files

the entire file system is also traversed.
If there is anything that you'd like me to try out to see if I can resolve the problem, I can. Also, all the tests are passed.

This shouldn't be the case, current implementation doesn't touch the file system except for the cache file from which the list of entries or tags is read. The -d flag only applies to commands that take a pattern as an input (it should be noted in the help message).
I saw you also got a panic when doing a clear, that should also be resolved now.

cargo run
This post should be much more complete and show every command.
set -- Works
$ cargo run -- set 'runner.rs' rust                                                         ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag set runner.rs rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
        + rust
saving to /Users/lucasburns/Library/Caches/wutag.registry

edit -- Works
* However, no error is thrown if there is an invalid tag given.

cargo run -- edit rust --color 0xFFFFFF                                                    ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag edit rust --color 0xFFFFFF`
rust ==> rust
saving to /Users/lucasburns/Library/Caches/wutag.registry

cp -- Displaying that it works, but doesn't copy
$ cargo run -- cp runner.rs config.rs
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag cp runner.rs config.rs`
/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
saving to /Users/lucasburns/Library/Caches/wutag.registry

However, listing files afterward does not display that file as being tagged:
$ cargo run -- list files
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/aaa.sh
/Users/lucasburns/test/clap/count-tests.sh

clear -- Backtrace
$ cargo run -- clear '*.rs'
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag clear '*.rs'`thread 'main' panicked at 'removal index (is<PHONE_NUMBER>219654000) should be < len (is 1)', library/alloc/src/vec/mod.rs:1355:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

rm -- Displays that it works, but doesn't
If the tag is set, when removing the tag the first time, no backtrace is given; however, if the command is ran again, a backtrace is given as shown below. This should not display the rust tag at all because it should've been deleted from the xattrs the first time.
Also, I'm not sure if the other errors are supposed to be shown or if it is just supposed to show the files that it is supposed to remove the tags from.
cargo run -- rm '*.rs' rust                                                               ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag rm '*.rs' rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/registry.rs:90:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
        X rust

After running the above command twice and getting that error message, when listing the files again, the path is still displayed:
$ cargo run -- list files                                                                   ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.22s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/aaa.sh

search -- This displays what the problem is
If a tag is set in the directory and I search for it, it is shown:
$ cargo run -- search 'rust'                                                                ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag search rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs: rust

If I remove the tag:
$ cargo run -- rm '*.rs' rust                                                               ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag rm '*.rs' rust`
/Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/opt.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/util.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/util.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/registry.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/registry.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/config.rs

/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs:
        X rust
/Users/lucasburns/projects/rust/repos_example/wutag/src/main.rs:
        ERROR tag `rust` doesn't exist - /Users/lucasburns/projects/rust/repos_example/wutag/src/main.rs

saving to /Users/lucasburns/Library/Caches/wutag.registry

And search again, it is gone:
$ cargo run -- search 'rust'                                                                ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag search rust`

However, if I list the files, it is still there:
$ cargo run -- list files                                                                   ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.06s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag list files`
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/test/aaa.sh

-d
Doesn't affect which files are listed.
$ cargo run -- -d . list files                                                              ─╯
    Finished dev [unoptimized + debuginfo] target(s) in 0.28s
     Running `/Users/lucasburns/projects/rust/repos_example/wutag/target/debug/wutag -d . list files`
/Users/lucasburns/test/aaa.sh
/Users/lucasburns/test/clap/count-tests.sh
/Users/lucasburns/projects/rust/repos_example/wutag/src/runner.rs

-m
I tested max-depth earlier when setting tags and it worked.

Please try the latest commit 3c5dea7 , it should fix most of the issues.

I also noticed that wutag clear doesn't work at all, it was because it was only executed when the --verbose flag was passed. Should work in all cases now. Tested locally.

A sidenote, testing out some of the version of the code I ended up with files without tags that are in cache. To clear it up I had to manually remove $XDG_HOME_DIR/.cache/wutag.registry. I think I'll add a subcommand that lets the user clear the cache easily.

I removed the registry manually, and when running both clear and rm this time, they work. However, clear does not display anything showing that it cleared the tags, rm does though. The only issue that is still there is that when listing the files and tags both, the file that was just either cleared or rm'd is still being shown. Would this have anything to do with the way the registry is being stored across systems?

I removed the registry manually, and when running both clear and rm this time, they work. However, clear does not display anything showing that it cleared the tags, rm does though. The only issue that is still there is that when listing the files and tags both, the file that was just either cleared or rm'd is still being shown. Would this have anything to do with the way the registry is being stored across systems?

I don't think the way it's stored matters in this case. Clear doesn't display anything by default, only with --verbose or -v. My initial thought was that when clearing usually a lot of files are cleared so there would be a lot of output but I think it's better to keep it the same across all subcommands so I removed the --verbose flag.

Would it also be possible to have an option to maybe implement this function list_entries_with_tags from the command line to show all files and tags beside each other without having to specifically search for the tags to get that display?
Perhaps, wutag list files-tags, or files-n-tags

I added a flag so that you can now do wutag list files --with-tags or wutag list files -t and it will display:
❯ cargo r -- list files --with-tags
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/wutag list files --with-tags`
/home/wojtek/dev/rust/wutag/target/CACHEDIR.TAG: trash ...
/home/wojtek/dev/rust/wutag/src/opt.rs: test123
/home/wojtek/dev/rust/wutag/src/config.rs: test123
/home/wojtek/dev/rust/wutag/target/debug: trash ...
/home/wojtek/dev/rust/wutag/target/.rustc_info.json: trash ...
/home/wojtek/dev/rust/wutag/src/util.rs: test123
/home/wojtek/dev/rust/wutag/src/main.rs: test123
/home/wojtek/dev/rust/wutag/src/runner.rs: test123
/home/wojtek/dev/rust/wutag/src/registry.rs: test123

The only issue that is still there is that when listing the files and tags both, the file that was just either cleared or rm'd is still being shown. Would this have anything to do with the way the registry is being stored across systems?

Would you mind removing the registry again and noting exactly what commands you issue to get into this state? I cannot really reproduce it.

I think I will merge this PR for now as it contains way too many changes. If you find any new bugs please report them in separate issues :) Thanks!

Awesome, it is working now! The tags and files both are no longer being displayed after removing the tags or clearing the tags. I like how it shows that it is cleared as well. I have been using jdberry's tag for macOS and wrote a fairly long script for it that also included wutag, but I think that I may start using wutag only


Adding optional config to HAProxy for logging, buffer size, and stats socket.

This pull request was discussed in two group threads:
https://groups.google.com/a/cloudfoundry.org/forum/?utm_medium=email&utm_source=footer#!msg/vcap-dev/70MYeQbDnQc/mmSRWfKM0a0J
https://groups.google.com/a/cloudfoundry.org/forum/?utm_medium=email&utm_source=footer#!msg/vcap-dev/iWO7QDqt0AM/RqQslk_QYcYJ
This will also close two current trackers:
https://www.pivotaltracker.com/n/projects/966314/stories/88373146
https://www.pivotaltracker.com/n/projects/966314/stories/87802174
Changes:

Add optional config to log HAProxy to /var/vcap/sys/log via rsyslog configuration (default is /var/log/syslog)

This requires some permissions changes to /var/vcap/sys and /var/vcap/sys/log so rsyslog can traverse the folders

Add optional config to enable a stats socket for HAProxy
Add optional config to enable the "dontlognull" setting

Upstream load balancers that use TCP checks to HAProxy result in larger numbers of "null" requests that get added to the logs
Enabling this option will disable logging of the null requests, but that will also eliminate logging of port scanning requests or anything else that hits HAProxy without actually transmitting data

Add configuration for the buffer size, with the default value being the HAProxy default

Increasing the buffer size is required to handle requests with large query strings or cookies

The new configuration is optional and no manifest changes are required, and if the new settings aren't added to the manifest then the new config will be identical to the old config.
In addition, the "http-server-close" option was removed. This was required in older HAProxy versions in order to make the X-Forwarded-Proto header work correctly, but it resulted in disabling server-side keep-alive.  The new HAProxy version does not require this entry and removing it should result in a (potentially significant) performance improvement as HAProxy can now keep a pool of connections to the routers open instead of initiating new connections for every request.
Aaron Huber
Intel Corporation

Note that based on the forum discussion the desire is to remove the permissions changes and add syslog to the vcap group:
https://www.pivotaltracker.com/n/projects/956238/stories/89303062
This will require a stemcell update, so there are two options:

Pull in this change as-is for now with the permisisons changes, and then I'll submit a new one once the syslog changes are made to remove the 2 chmod commands.
Wait until the stemcell change is made and then I'll add a commit to remove the chmod commands.

Aaron Huber
Intel Corporation

Thanks @amhuber! We're just going to go with option 2 and stick this pull in our backlog until the bosh changes are made. We'll pull it in as soon as that happens!
@zaksoup + @crhino CF Runtime Team

OK, I'll pull out the two chmod commands and add a commit, then this will be ready to pull in once the BOSH changes are ready.

Sounds good. We'll ping you when it gets pulled in.

@amhuber Looks like bosh made changes for this in stemcell 2915.  I'll move this into the backlog, probably a few weeks out.

Hi there!
We use Pivotal Tracker to provide visibility into what our team is working on. A story for this issue has been automatically created.
The current status is as follows:

[ ] #99824330 Adding optional config to HAProxy for logging, buffer size, and stats socket.

This comment, as well as the labels on the issue, will be automatically updated as the status in Tracker changes.

@amhuber Sorry for the long delay.  This story has been prioritized now in the new Routing team's backlog:
https://www.pivotaltracker.com/story/show/89303250

Hello @amhuber  couple of requests:
Please update the property name ha_proxy.buffer_size to include the units of measure. Looking at http://www.haproxy.org/download/1.4/doc/configuration.txt (tune.bufsize), I believe this is bytes. So our property should beha_proxy.buffer_size_bytes
Also, how does one connect to the stats socket? On a particular port? Would you ever want to change this port?
Thank you

You can connect to the stats socket using either socat or nc (examples at http://www.mgoff.in/2010/07/14/haproxy-gathering-stats-using-socat/ and https://tech.fawk.eu/111/), so there is no port involved, just the socket file.
Have you already done any work on resolving the conflicts since this is is based off such an old config?  If not then it's probably easier for me to rebase this against a current release and re-test it submit a new pull request, assuming you would be able to pull it in more quickly so it doesn't get out of sync again?  That way I can also confirm that the change to the stemcell for the syslog permissions is working correctly.
Let me know and if that works for you I'll close this and submit a new request next week.
Aaron

Hello @amhuber, I just realized how long ago your submitted this. Sorry about that! We'll take care of the property name change.

Works for me, thanks!

@shalako @amhuber Have whatever desired property name changes been done?  If so, can this PR be closed?

The commit made it to release-candidate: https://github.com/cloudfoundry/cf-release/commit/ead532949b697c4a837e499075fcc8a3805f432e
Closing


Enabling telemetry causing multiple issues due to socket leak

What kind of request is this (question/bug/enhancement/feature request):
Bug
Steps to reproduce (least amount of steps as possible):
Upgrade from 2.3.x to 2.4.5
Keep telemetry on (default)
Result:
After a few days, Helm installation fails with below
Failed to install app <---->. Error: UPGRADE FAILED: Get https://localhost:443/k8s/clusters/c-djpgp/api/v1/namespaces/ugi-sb-kcapp001-otw-dsp/configmaps?labelSelector=NAME%!D(MISSING)jupyterhub%!C(MISSING)OWNER%!D(MISSING)TILLER: dial tcp <IP_ADDRESS>:443: connect: cannot assign requested address

Failed to install app <----> . Error: validation failed: [unable to recognize "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1", unable to recognize "": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1", unable to recognize "": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1", unable to recognize "": no matches for kind "Deployment" in version "apps/v1"]

Error message from Rancher Apps:
# monitoring-operator
Failed to install app monitoring-operator. Error: UPGRADE FAILED: unknown service hapi.services.tiller.ReleaseService

Failed to install app monitoring-operator. Error: validation failed: [unable to recognize "": no matches for kind "ConfigMap" in version "v1", unable to recognize "": no matches for kind "ServiceAccount" in version "v1", unable to recognize "": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1", unable to recognize "": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1", unable to recognize "": no matches for kind "Service" in version "v1", unable to recognize "": no matches for kind "Deployment" in version "apps/v1", unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"]

# project-monitoring
Failed to install app project-monitoring. Error: validation failed: [unable to recognize "": no matches for kind "Secret" in version "v1", unable to recognize "": no matches for kind "ConfigMap" in version "v1", unable to recognize "": no matches for kind "ServiceAccount" in version "v1", unable to recognize "": no matches for kind "Service" in version "v1", unable to recognize "": no matches for kind "Deployment" in version "apps/v1", unable to recognize "": no matches for kind "Prometheus" in version "monitoring.coreos.com/v1", unable to recognize "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"]

Other details that may be helpful:
Environment information

Rancher version : 2.4.3
Installation option (single install/HA): HA

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): Custom
Machine type (cloud/VM/metal) and specifications (CPU/memory): VM
Kubernetes version :  v1.17.5
Docker version

Client:
 Debug Mode: false

Server:
 Containers: 40
  Running: 29
  Paused: 0
  Stopped: 11
 Images: 37
 Server Version: 19.03.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.14.138-rancher
 Operating System: RancherOS v1.5.5
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.42GiB
 Name: pn4-it-kwran001
 ID: 2TE2:XG25:LS7F:UR6W:JNZS:PKO6:4E7E:KRCN:VQWK:57J6:5LHO:OQSB
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
   ...
  <IP_ADDRESS>/8
 Registry Mirrors:
  ...
 Live Restore Enabled: false
 Product License: Community Engine

gz#10241
gz#11086
gz#11356
gz#12428
gz#12355
gz#12936
gz#13318
gz#14722

Similar issue: https://github.com/rancher/rancher/issues/28360

etcd snapshots also fails with the below error when the container run out of free sockets.
Error starting driver: failed retrieving port for driver: listen tcp <IP_ADDRESS>:0: bind: address already in use

Submitted PR rancher/telemetry#47 to address the issue. More info, https://github.com/rancher/rancher/issues/28360#issuecomment-771134043

Rancher version master-head (02/03/2021) 54b3a755b
I was able to check this with the steps provided https://github.com/rancher/rancher/issues/28360#issuecomment-771134043
I see the number of connections is kept steadily low even after multiple request to the report endpoint.
I also validated the /v1-telemetry endpoint was properly updated after adding clusters.
Still waiting for a build of 2.5-head to validate that branch.

Rancher version v2.5-head-head (02/03/2021) c6cdf815a
I was able to check this with the steps provided https://github.com/rancher/rancher/issues/28360#issuecomment-771134043
I see the number of connections is kept steadily low even after multiple request to the report endpoint.
I also validated the /v1-telemetry endpoint was properly updated after adding clusters.


throw errors for mandatory columns that don't match the regex

Current behavior

Mandatory column

Yes
No

Format does not match
Error
Warning

Format does not match but value = NA
Warning
Warning

Alternative behavior

Mandatory column

Yes
No

Format does not match
Error
-

Format does not match but value = NA
Warning
-

Which one do you prefer @alecristia ?

Opted for "Alternative behavior 2".


[WIP] etcdmain: correctly check return values from SdNotify()

Since https://github.com/coreos/go-systemd/commit/0d7fccc155921b2b4023b0acfd60981d4a27b3da, SdNotify() in go-systemd started to return 2 values, sent and err. So startEtcdOrProxyV2() in etcd needs to check the 2 return values correctly. As the 2 values are independent of each other, error checking logic needs to be slightly updated too. SdNotifyNoSocket, which was previously provided by go-systemd, does not exist any more. In that case (false, nil) will be returned instead.
This was a direct reason of a travis-ci failure in fleet like this: https://travis-ci.org/coreos/fleet/builds/146093283. Although fleet has nothing to do with this API change, travis always fetches the most recent master branch for each git repo.

@dongsupark vendor updates need to be done through godep so the vendored packages match an upstream revision. Try running scripts/updatedep.sh with the recent go-systemd in the gopath?

@heyitsanthony Thanks. I tried to update both Godeps.json and vendor tree together, but several tests started to fail. I'm not sure why. I'll look into that tomorrow.

@donguspark it looks like this is upgrading some extra deps which may be causing the failures. I think it'll be OK to keep the go-systemd updates to Godeps.json and vendor/ but keep the rest the same.

@heyitsanthony Thanks. That did the trick! :-)
Removing go-semver and clockwork from vendor, both unit tests and functional tests work without failures. Interesting.
Please have a look.

@dongsupark please squash the commits cmd/Godeps: update Godeps.json and cmd/vendor: update vendor trees according to go-systemd update into vendor: update go-systemd and it'll lgtm. Thanks!
/cc @xiang90

@heyitsanthony Done. Thanks!

lgtm

Thanks. Now travis error from fleet is gone. :-)
BTW as for the test failure in TestServeMembersCreate, culprit seems to be a recent change in github.com/jonboulle/clockwork. I'm not sure about how to fix that, and anyway that doesn't look like a critical issue, as long as etcd doesn't update the repo in the vendor tree.


Flag incomplete DNS config errors with 'configuration problem' error code

How to categorize this PR?

/area ops-productivity
/kind enhancement
/priority normal
What this PR does / why we need it:
With this PR we are flagging incomplete DNS errors with the ERR_CONFIGURATION_PROBLEM error code.
Special notes for your reviewer:
/cc @tim-ebert
Release note:

Shoots with 'incomplete DNS configuration' errors are now flagged with the `ERR_CONFIGURATION_PROBLEM` error code.

/ok-to-test


Missing video codec requirements on Readme

[x] I have read the FAQ.
[x] I have searched in existing issues.

Environment

OS: Fedora 33
scrcpy version: 1.18
installation method: copr
device model: Realme XT (RMX1921)
Android version: Android 10

Describe the bug
Hello I noticed that in the readme/faq that there wasn't any instructions to install additional video codecs. If the codecs are missing on a user's computer, they won't be able to install the package. You can see which codecs are missing in the terminal outputs section.
I found that installing ffmpeg seemed to install any missing video codecs. The following guide was helpful in installing ffmpeg
https://computingforgeeks.com/how-to-install-ffmpeg-on-fedora/
The solution is to have the package install with the video codecs, or have a section listing the codec requirements and how to install them.
terminal output
sudo dnf install scrcpy
Copr repo for scrcpy owned by zeno               10 kB/s | 2.4 kB     00:00
Error:
 Problem: conflicting requests
  - nothing provides libavcodec.so.58()(64bit) needed by scrcpy-1.18-2.fc33.x86_64
  - nothing provides libavcodec.so.58(LIBAVCODEC_58)(64bit) needed by scrcpy-1.18-2.fc33.x86_64
  - nothing provides libavdevice.so.58()(64bit) needed by scrcpy-1.18-2.fc33.x86_64
  - nothing provides libavdevice.so.58(LIBAVDEVICE_58)(64bit) needed by scrcpy-1.18-2.fc33.x86_64
  - nothing provides libavformat.so.58()(64bit) needed by scrcpy-1.18-2.fc33.x86_64
  - nothing provides libavformat.so.58(LIBAVFORMAT_58)(64bit) needed by scrcpy-1.18-2.fc33.x86_64
  - nothing provides libavutil.so.56()(64bit) needed by scrcpy-1.18-2.fc33.x86_64
  - nothing provides libavutil.so.56(LIBAVUTIL_56)(64bit) needed by scrcpy-1.18-2.fc33.x86_64
(try to add '--skip-broken' to skip uninstallable packages)
$ sudo dnf install scrcpy --skip-broken

Please do not post screenshots of your terminal, just post the content as text instead.

Error:
Problem: conflicting requests
 - nothing provides libavcodec.so.58()(64bit) needed by scrcpy-1.18-2.fc33.x86_64

-1.18-2.fc33.x86_64
That's not missing codec, but just missing dependencies (ffmpeg) managed by your package manager.

Thanks! I'll update the title


Fix reported performance degradation on aarch64 [1.1.1]

This restores the implementation prior to
commit 2621751 ("aes/asm/aesv8-armx.pl: avoid 32-bit lane assignment in CTR mode")
for 64bit targets only, since it is reportedly 2-17% slower,
and the silicon errata only affects 32bit targets.
Only for 32bit targets the new algorithm is used.
Fixes #18445

Will this apply cleanly to master branch?

[minor] typo in title: degradiation should be degradation

(note: looks good from our side, too. sorry, I have been away on vacation)

This pull request is ready to merge

Merged to 1.1.1. Thank you.


NH-36351: Removed use of swmetricstransform

This processor is no longer needed since filterprocessor can filter by datapoint values
value_int does not work, all values in our pipeline are value_double
filtering is done in separate step of the pipeline.
Verified manually.
I change both (image and helm) in the same time, because Helm does not require new updated image.

Could you add a note to the unreleased section of the chart's changelog that we are replacing all usages of swmetricstransformprocessor for filterprocessor?


Possible Circular Dependency [Issuer and Certificate Don't Deploy]

I've been trying to deploy this chart with Argo CD omitting the cert-manager deployment as I already have a deployment running.
ArgoCD can't deploy the DNSClass as  the mutating webhook mdnsclass.kb.io fails.
The mutating webhook mdnsclass.kb.io won't work because the pods that handle it's requests won't boot.
The pods won't boot because they are looking for a Certificate, via a secret, that hasn't been created yet and can't be due to the following annotations.
I assume the same issue problem occurs with a Helm deployment.
serving-cert.yaml
  annotations:
    "helm.sh/hook": post-install,post-upgrade
    "helm.sh/hook-weight": "2"

selfsigned-issuer.yaml
  annotations:
    "helm.sh/hook": post-install,post-upgrade
    "helm.sh/hook-weight": "1"

Hi @mlladb , thank you for bringing this issue to my attention! I think we should avoid adding the Helm Hooks If cert-manager installation is not enabled for certificates.

I was able to reproduce your case and the new PR fixes the issue on my test setup.

The fix has been merged. If the issue persists, please feel free to reopen this ticket.

Hi @eminaktas. Thanks for the quick turnaround. The fix works in that I now have Issuer and Certificate resources being deployed. However I now have another circular dependency as ArgoCD can't deploy the DNSClass. 😄
I believe this is because the application pods need to be running in order for the mutating webhook and validating web hook to run. Both of which intercept requests to create or update dnsclasses. Perhaps these can be deployed last after all other resources are deployed?


Create 2021-11-30_gnuradio.yaml

New job Listing from GNU Radio

Thanks, @saminatrachier! The job is now on the board and also at permalink https://jobs.numfocus.org/job/2021-11-30_gnuradio


Attempt to flash or erase LPC55s28 triggers an assert

I hit the assert below while attempting to flash or simply erase an LPC55s28 (LPCXpresso55S28 dev board). pyOCD GDB server runs without a problem.
Attempted pyocd versions: 0.24.0, 0.25.0, and 0.26.1.  (the log below is from 0.25.0)
pyocd erase -t lpc55s28 --chip -Osmart_flash=0 -vvv
0001919:DEBUG:session:Project directory: C:\Users\u106788
0002132:DEBUG:pack_target:Loading target 'lpc55s28' from CMSIS-Pack
0002151:INFO:board:Target type is lpc55s28
0002156:DEBUG:dap_access_cmsis_dap:CMSIS-DAP probe FTA0CQAR firmware version: 1.10
0002161:DEBUG:sequencer:Running task load_svd
0002186:DEBUG:sequencer:Running task create_flash
0002187:DEBUG:sequencer:Running task pre_connect
0002281:DEBUG:sequencer:Running task dp_init
0002375:DEBUG:dap:Default wire protocol selected; using SWD
0002380:DEBUG:dap:Sending deprecated SWJ sequence to select SWD
0002411:INFO:dap:DP IDR = 0x6ba02477 (v2 rev6)
0002411:DEBUG:sequencer:Running task power_up
0002415:DEBUG:sequencer:Running task find_aps
0002422:DEBUG:sequencer:Running task create_aps
0002424:DEBUG:sequencer:Running task create_ap.0
0002430:INFO:ap:AP#0 IDR = 0x84770001 (AHB-AP var0 rev8)
0002432:DEBUG:ap:AP#0 default HPROT=3 HNONSEC=0
0002434:DEBUG:ap:AP#0 implemented HPROT=f HNONSEC=1
0002439:DEBUG:sequencer:Running task create_ap.2
0002443:INFO:ap:AP#2 IDR = 0x002a0000 (proprietary)
0002443:DEBUG:sequencer:Running task find_components
0002446:DEBUG:sequencer:Running task init_ap.0
0002453:INFO:rom_table:AP#0 ROM table #0 @ 0xe00fe000 (designer=015 part=000)
0002457:INFO:rom_table:[0]<e00ff000:ROM class=1 designer=43b part=4c9>
0002457:INFO:rom_table:  AP#0 ROM table #1 @ 0xe00ff000 (designer=43b part=4c9)
0002462:INFO:rom_table:  [0]<e000e000:SCS-M33 class=9 designer=43b part=d21 devtype=00 archid=2a04 devid=0:0:0>
0002465:INFO:rom_table:  [1]<e0001000:DWT class=9 designer=43b part=d21 devtype=00 archid=1a02 devid=0:0:0>
0002468:INFO:rom_table:  [2]<e0002000:BPU class=9 designer=43b part=d21 devtype=00 archid=1a03 devid=0:0:0>
0002470:INFO:rom_table:  [3]<e0000000:ITM class=9 designer=43b part=d21 devtype=43 archid=1a01 devid=0:0:0>
0002475:INFO:rom_table:[1]<e0040000:TPIU-M33 class=9 designer=43b part=d21 devtype=11 archid=0000 devid=0:0:ca0>
0002475:DEBUG:sequencer:Running task create_cores
0002477:DEBUG:coresight_target:Creating SCS-M33 component
0002481:INFO:cortex_m_v8m:CPU core #0 is Cortex-M33 r0p3
0002488:INFO:cortex_m:FPU present: FPv5-SP
0002495:DEBUG:sequencer:Running task set_default_reset_type
0002495:DEBUG:sequencer:Running task create_components
0002495:DEBUG:coresight_target:Creating DWT component
0002499:INFO:dwt:4 hardware watchpoints
0002499:DEBUG:coresight_target:Creating BPU component
0002503:INFO:fpb:8 hardware breakpoints, 1 literal comparators
0002503:DEBUG:fpb:fpb has been disabled
0002504:DEBUG:coresight_target:Creating ITM component
0002510:DEBUG:coresight_target:Creating TPIU-M33 component
0002512:DEBUG:sequencer:Running task check_for_cores
0002513:DEBUG:sequencer:Running task halt_on_connect
0002514:DEBUG:cortex_m:halting core 0
0002518:DEBUG:sequencer:Running task post_connect
0002518:DEBUG:sequencer:Running task notify
0002518:INFO:eraser:Erasing chip...
0002523:DEBUG:cortex_m:halting core 0
0002526:DEBUG:cortex_m:set reset catch, core 0
0002527:DEBUG:cortex_m:halting core 0
0002532:DEBUG:cortex_m:reset, core 0, type=SW_SYSRESETREQ
0004560:DEBUG:session:uninit session <pyocd.core.session.Session object at 0x000001F25DEE53C8>
0004560:DEBUG:board:uninit board <pyocd.board.board.Board object at 0x000001F25D2B4748>
0004564:DEBUG:cortex_m:cannot resume: target not halted
0004569:DEBUG:pywinusb_backend:closing interface
0004569:CRITICAL:__main__:uncaught exception:
Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\__main__.py", line 360, in run
    self._COMMANDS[self._args.cmd](self)
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\__main__.py", line 555, in do_erase
    eraser.erase(addresses)
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\flash\eraser.py", line 76, in erase
    self._chip_erase()
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\flash\eraser.py", line 97, in _chip_erase
    region.flash.init(region.flash.Operation.ERASE)
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\flash\flash.py", line 235, in init
    self.target.reset_and_halt(Target.ResetType.SW)
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\core\coresight_target.py", line 398, in reset_and_halt
    return self.selected_core.reset_and_halt(reset_type)
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\coresight\cortex_m.py", line 966, in reset_and_halt
    xpsr = self.read_core_register('xpsr')
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\coresight\cortex_m.py", line 1033, in read_core_register
    regValue = self.read_core_register_raw(regIndex)
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\coresight\cortex_m.py", line 1047, in read_core_register_raw
    vals = self.read_core_registers_raw([reg])
  File "c:\programdata\anaconda3\lib\site-packages\pyocd\coresight\cortex_m.py", line 1107, in read_core_registers_raw
    assert dhcsr_val & CortexM.S_REGRDY
AssertionError

That assertion indicates that the CPU is still running. The LPC55xx series has a boot ROM that requires some custom logic to handle. Since the LPC552x devices aren't supported yet by pyOCD, this logic isn't being used.
As a temporary workaround, you might try telling pyocd that the target is an LPC55S69: -tlpc55s69. This will enable the boot ROM handling code, which is common for the entire LPC55xx series.
For a real solution, there are two possible paths:

Support debug sequences in CMSIS Packs. This will allow pyocd to execute any custom debug logic included in the device descriptions in CMSIS Packs.
Add a builtin target for LPC552x devices.

I'm already working on debug sequences, so this will happen at some point. It may not be necessary to have a builtin target if debug sequences work well enough.

Thanks @flit for the quick reply! I attempted using lpc55s69 as target and ended up hitting a different problem. Is there anything I can do on the command line to get around this one?

pyocd erase -tlpc55s69 --chip -Osmart_flash=0 -vvv
0001289:DEBUG:session:Project directory: C:\Users\u106788
0001291:INFO:board:Target type is lpc55s69
0001295:DEBUG:dap_access_cmsis_dap:CMSIS-DAP probe FTA0CQAR firmware version: 1.10
0001299:DEBUG:sequencer:Running task load_svd
0001300:DEBUG:sequencer:Running task create_flash
0001300:DEBUG:sequencer:Running task pre_connect
0001302:DEBUG:sequencer:Running task dp_init
0001324:DEBUG:dap:Default wire protocol selected; using SWD
0001326:DEBUG:dap:Sending deprecated SWJ sequence to select SWD
0001345:INFO:dap:DP IDR = 0x6ba02477 (v2 rev6)
0001347:DEBUG:sequencer:Running task power_up
0001352:DEBUG:sequencer:Running task find_aps
0001367:DEBUG:sequencer:Running task create_aps
0001368:DEBUG:sequencer:Running task create_ap.0
0001373:INFO:ap:AP#0 IDR = 0x84770001 (AHB-AP var0 rev8)
0001377:DEBUG:ap:AP#0 default HPROT=3 HNONSEC=1
0001381:DEBUG:ap:AP#0 implemented HPROT=f HNONSEC=1
0001414:DEBUG:sequencer:Running task create_ap.2
0001418:INFO:ap:AP#2 IDR = 0x002a0000 (proprietary)
0001420:DEBUG:sequencer:Running task find_components
0001422:DEBUG:session:uninit session <pyocd.core.session.Session object at 0x0000027582B843C8>
0001424:DEBUG:pywinusb_backend:closing interface
0001428:CRITICAL:main:uncaught exception: 'init_ap.1'
Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\site-packages\pyocd_main_.py", line 360, in run
self.COMMANDSself._args.cmd
File "c:\programdata\anaconda3\lib\site-packages\pyocd_main.py", line 550, in do_erase
with session:
File "c:\programdata\anaconda3\lib\site-packages\pyocd\core\session.py", line 299, in enter
self.open()
File "c:\programdata\anaconda3\lib\site-packages\pyocd\core\session.py", line 415, in open
self._board.init()
File "c:\programdata\anaconda3\lib\site-packages\pyocd\board\board.py", line 83, in init
self.target.init()
File "c:\programdata\anaconda3\lib\site-packages\pyocd\core\coresight_target.py", line 163, in init
seq.invoke()
File "c:\programdata\anaconda3\lib\site-packages\pyocd\utility\sequencer.py", line 208, in invoke
resultSequence = call()
File "c:\programdata\anaconda3\lib\site-packages\pyocd\utility\sequencer.py", line 131, in
self._calls[name] = lambda : wrapper(orig())
File "c:\programdata\anaconda3\lib\site-packages\pyocd\target\builtin\target_LPC55S69JBD100.py", line 174, in _modify_ap1
('set_ap1_nonsec',        self._set_ap1_nonsec),
File "c:\programdata\anaconda3\lib\site-packages\pyocd\utility\sequencer.py", line 160, in insert_before
raise KeyError(beforeTaskName)
KeyError: 'init_ap.1'

Ah, that makes sense. Sorry I forgot about that! This is caused by the LPC552x being single-core instead of dual-core. Unfortunately there's no workaround from the command line. Let me see if I can put together a fix for you, but I might not get to it until the coming weekend.

Great, thank you for doing this!

I applied the changes in #884 pull request to target_LPC55S69JBD100.py of version 0.25.0 and below is what I got. It goes further than before but then hangs. Just putting it out there in case it helps, as this may not be valid since I don't have all the other changes currently in the head.
>pyocd erase -tlpc55s69 --chip -Osmart_flash=0 -vvv
0001517:DEBUG:session:Project directory: C:\WINDOWS\system32
0001519:INFO:board:Target type is lpc55s69
0001523:DEBUG:dap_access_cmsis_dap:CMSIS-DAP probe FTA0CQAR firmware version: 1.10
0001527:DEBUG:sequencer:Running task load_svd
0001528:DEBUG:sequencer:Running task create_flash
0001528:DEBUG:sequencer:Running task pre_connect
0001528:DEBUG:sequencer:Running task dp_init
0001551:DEBUG:dap:Default wire protocol selected; using SWD
0001553:DEBUG:dap:Sending deprecated SWJ sequence to select SWD
0001569:INFO:dap:DP IDR = 0x6ba02477 (v2 rev6)
0001572:DEBUG:sequencer:Running task power_up
0001576:DEBUG:sequencer:Running task find_aps
0001592:DEBUG:sequencer:Running task create_aps
0001593:DEBUG:sequencer:Running task create_ap.0
0001597:INFO:ap:AP#0 IDR = 0x84770001 (AHB-AP var0 rev8)
0001600:DEBUG:ap:AP#0 default HPROT=3 HNONSEC=1
0001604:DEBUG:ap:AP#0 implemented HPROT=f HNONSEC=1
0001613:DEBUG:sequencer:Running task create_ap.2
0001617:INFO:ap:AP#2 IDR = 0x002a0000 (proprietary)
0001646:DEBUG:sequencer:Running task find_components
0001647:DEBUG:sequencer:Running task init_ap.0
0001660:INFO:rom_table:AP#0 ROM table #0 @ 0xe00fe000 (designer=015 part=000)
0001670:INFO:rom_table:[0]<e00ff000:ROM class=1 designer=43b part=4c9>
0001671:INFO:rom_table:  AP#0 ROM table #1 @ 0xe00ff000 (designer=43b part=4c9)
0001706:INFO:rom_table:  [0]<e000e000:SCS-M33 class=9 designer=43b part=d21 devtype=00 archid=2a04 devid=0:0:0>
0001752:INFO:rom_table:  [1]<e0001000:DWT class=9 designer=43b part=d21 devtype=00 archid=1a02 devid=0:0:0>
0001797:INFO:rom_table:  [2]<e0002000:BPU class=9 designer=43b part=d21 devtype=00 archid=1a03 devid=0:0:0>
0001811:INFO:rom_table:  [3]<e0000000:ITM class=9 designer=43b part=d21 devtype=43 archid=1a01 devid=0:0:0>
0001813:INFO:rom_table:[1]<e0040000:TPIU-M33 class=9 designer=43b part=d21 devtype=11 archid=0000 devid=0:0:ca0>
0001813:DEBUG:sequencer:Running task create_cores
0001817:INFO:cortex_m_v8m:CPU core #0 is Cortex-M33 r0p3
0001823:INFO:cortex_m:FPU present: FPv5-SP
0001829:DEBUG:sequencer:Running task enable_traceclk
0001833:DEBUG:sequencer:Running task create_components
0001833:DEBUG:coresight_target:Creating DWT component
0001837:INFO:dwt:4 hardware watchpoints
0001838:DEBUG:coresight_target:Creating BPU component
0001842:INFO:fpb:8 hardware breakpoints, 1 literal comparators
0001842:DEBUG:fpb:fpb has been disabled
0001843:DEBUG:coresight_target:Creating ITM component
0001846:DEBUG:coresight_target:Creating TPIU-M33 component
0001848:DEBUG:sequencer:Running task check_for_cores
0001848:DEBUG:sequencer:Running task halt_on_connect
0001849:DEBUG:cortex_m:halting core 0
0001851:DEBUG:sequencer:Running task post_connect
0001851:DEBUG:sequencer:Running task notify
0001851:INFO:eraser:Erasing chip...
0001852:DEBUG:cortex_m:halting core 0
0001864:DEBUG:cortex_m:reset, core 0, type=SW_SYSRESETREQ
0001907:DEBUG:cortex_m:resuming core 0
0001908:DEBUG:manager:added=[] removed=[]
0001909:DEBUG:manager:bps after flush={}
0001919:DEBUG:cortex_m:resuming core 0
0001920:DEBUG:manager:added=[] removed=[]
0001920:DEBUG:manager:bps after flush={}

Thanks for testing! It looks like it's hanging while waiting for the chip erase to complete.
This might not be specific to the LPC55S28. There have been reports of some lingering issues with flash programming in certain cases, and I think I saw this same hang a couple days ago. Unfortunately it's been very intermittent and difficult to reproduce. (I just tried a bunch of chip erases on my LPC55S69 without issue.)
I'm not sure if this will change anything, but you might try other flash operations like sector erase and programming something.

I updated to head and tried again. No luck with sector erase or flashing attempt. It all fails at the same place, and every time. I guess there is some difference b/n LPC5528 and S69 with regards to erase.  Is there another issue that is open for the erase hang?
The halt failure is resolved so closing this particular issue. Thanks for the great support.

Fyi, I went ahead and added LPC55S28 support in #885. I also updated the flash algo used for LPC55S69 to the latest from NXP.
It turns out that the flash algos for '28 and '69 are almost exactly the same (just for the code). There's only 1 byte that differs between the two. Hopefully the new version of the algo will fix the hang, but if not then it must be that intermittent bug.


update: touchDragTool add event listener

update: touchDragTool add event listener 'cornerstoneimagerendered' to execute strategy.

Please check if the PR fulfills these requirements

[ ] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

Bug fix

What is the current behavior? (You can also link to an open issue here)

update touchDragTool's dragCallback

What is the new behavior (if this is a feature change)?

none

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

none

Other information:

Coverage decreased (-0.003%) to 4.38% when pulling 31c665ac4f4993748803f2f3c1656847fed3ddf3 on adbu:patch-1 into 16a12026d62d76032edb28a6c50fbb8d1c4e2a63 on cornerstonejs:master.

Hi there!
Could you provide some more details about the issue you are trying to solve? Is the touch version of this tool not displaying anything on the image?

Hi !
I want to fix the bug of 'dragProbeTouch '. Because there is no event listener 'cornerstoneimagerendered ', the strategy was not be executed. As the notes of  the previous version say , the strategy will be executed once cornerstoneimagerendered is triggered.
// The strategy can't be execute at this moment because the image is rendered asynchronously // (requestAnimationFrame). Then the eventData that contains all information needed is being // Cached and the strategy will be executed once cornerstoneimagerendered is triggered. function dragCallback (e) {}

Yep this is definitely a real issue. I think we need to fix it in the touchDragTool. I'll leave this open for now.

I pushed a commit which should close this. The change will be in the next release. Thanks for the report!


Add the ToyMix results to the Baseline page

https://graphium-docs.datamol.io/stable/baseline.html

Done


[Go Live][Apr 30]

Go live

Docs Build status updates of commit ad24141:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/assets/images/authentication/sample-app-demo-bot-configuration.png
:white_check_mark:Succeeded
View

msteams-platform/bots/how-to/authentication/add-authentication.md
:white_check_mark:Succeeded
View

msteams-platform/resources/samesite-cookie-update.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Exception when calling flask.abort() with mod_wsgi and connexion 1.1.6+

Description
After upgrading to connexion 1.1.6 and 1.1.7 from connexion 1.1.5, I get an exception (error 500) when calling flask.abort(...) on servers using mod_wsgi + Apache + Python 3.4. I cannot reproduce the problem in my local development machine (I don't have Apache installed, tried with uwsgi and flask development server and it works fine on both of them).
Expected behaviour
Return a nice error message, example:
{
    "detail": "Please authenticate first (send authorization header with a valid token or apikey)",
    "status": 401,
    "title": "Unauthorized",
    "type": "about:blank"
}

Actual behaviour
[Wed Apr 05 00:56:11.722938 2017] [:error] [pid 8604] (Dummy-1) connexion.apis.flask_api - DEBUG - Getting data and status code
[Wed Apr 05 00:56:11.723226 2017] [:error] [pid 8604] (Dummy-1) connexion.apis.flask_api - DEBUG - Got data and status code (401)
[Wed Apr 05 00:56:11.723547 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951] mod_wsgi (pid=8604): Exception occurred processing WSGI script '/opt/python/current/app/application.py'.
[Wed Apr 05 00:56:11.723581 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951] Traceback (most recent call last):
[Wed Apr 05 00:56:11.723631 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951]   File "/opt/python/run/venv/lib/python3.4/site-packages/flask/app.py", line 1997, in __call__
[Wed Apr 05 00:56:11.723636 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951]     return self.wsgi_app(environ, start_response)
[Wed Apr 05 00:56:11.723657 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951]   File "/opt/python/run/venv/lib/python3.4/site-packages/flask/app.py", line 1989, in wsgi_app
[Wed Apr 05 00:56:11.723661 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951]     return response(environ, start_response)
[Wed Apr 05 00:56:11.723695 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951]   File "/opt/python/run/venv/lib/python3.4/site-packages/werkzeug/wrappers.py", line 1277, in __call__
[Wed Apr 05 00:56:11.723701 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951]     start_response(status, headers)
[Wed Apr 05 00:56:11.723719 2017] [:error] [pid 8604] [remote <IP_ADDRESS>:11951] ValueError: status message was not supplied

Steps to reproduce

Use mod_wsgi
Call flask.abort()

Additional info:
As soon as I have more information, I update this post.
Output of the commands:

python --version

Python 3.4.3

pip show connexion | grep "^Version\:"

Version: 1.1.6

Shot in the dark here, but I've seen similar issues in other libraries before which stemmed from propagate exceptions being turned on in the development server, but off during production. Are you running with app.debug=True on your development setup? If you turn that to false, do you get the same errors you do in your production server?

Ok, I think I've found the problem. Connexion 1.1.6+ omits the HTTP Reason-Phrase.
Before (1.1.5):

*   Trying <IP_ADDRESS>...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8080 (#0)
> GET /v1/playlists HTTP/1.1
> Host: <IP_ADDRESS>:8080
> User-Agent: curl/7.50.1
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 401 UNAUTHORIZED
< Content-Type: application/problem+json
< Content-Length: 170
< Access-Control-Allow-Origin: *
< Server: Werkzeug/0.12.1 Python/3.5.2+
< Date: Wed, 05 Apr 2017 02:08:35 GMT
<
{
  "detail": "Please authenticate first (send authorization header with a valid token or apikey)",
  "status": 401,
  "title": "Unauthorized",
  "type": "about:blank"
}
* Closing connection 0

After (1.1.7):
*   Trying <IP_ADDRESS>...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8080 (#0)
> GET /v1/playlists HTTP/1.1
> Host: <IP_ADDRESS>:8080
> User-Agent: curl/7.50.1
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 401
< Content-Type: application/problem+json
< Content-Length: 170
< Access-Control-Allow-Origin: *
< Server: Werkzeug/0.12.1 Python/3.5.2+
< Date: Wed, 05 Apr 2017 02:09:06 GMT
<
{
  "detail": "Please authenticate first (send authorization header with a valid token or apikey)",
  "status": 401,
  "title": "Unauthorized",
  "type": "about:blank"
}
* Closing connection 0

@hjalves thanks, do you mind doing a PR?

@hjalves What version of Werkzeug are you using? Changing the line your mentioned break many tests.

@rafaelcaricio I'm using Werkzeug==0.12.1.

@rafaelcaricio Really? Did you change flask_response.status or flask_response.status_code?
Changing flask_response.status causes many errors, as status is expected to be a string.

@hjalves I didn't change the status -> status_code. Thank you for the PR! 😉


Updates shields.io badges

Updated shields.io badges to include community extension badge, and updated Stable badge to include alt text.

@celanthe as there are rwo PRs which apparently have similar changes, please merge the one you want


 For this reason, this PR introduces a docker-compose configuration for setting everything up in one command.

One related bug found when working on #124 - the time in the docker container is UTC which makes it a mess when working on local CEST system (state planning is acting "weird"). This is, however, related to #55 so I don't think there is really any point in addressing it in this PR. In the rare case that it would influence the functionality, the developer can always set the time to UTC locally for the sake of development

One related bug found when working on #124 - the time in the docker container is UTC which makes it a mess when working on local CEST system (state planning is acting "weird"). This is, however, related to #55 so I don't think there is really any point in addressing it in this PR. In the rare case that it would influence the functionality, the developer can always set the time to UTC locally for the sake of development

Considering the fact that this will definitely run in a CE(S)T environment, wouldn't it be better to force the container to use this timezone (at least until #55 is addressed)?


Connexion App is not accepting the celery instance even after setting the context manually

Description
While trying to add the celery instance with Connexion Flask Context, its not getting added due to which we are unable to register the tasks.
Expected behaviour
App context should accept Celery Instance as well.
Actual behaviour
Not accepting the celery instance to the context of Connexion app.
Kindly advise whether connexion framework has the capability to accept celery instance and also if you can provide an example, it would be great.
Output of the commands:

python --version
pip show connexion | grep "^Version\:"

It's hard to help with just the information you provided.
Are you using the underlying flask app?

Yes we are using connexion.app which tries to set the context with celery

can you provide a minimal working example please?

i am using the below example
import connexion
from celery import Celery

app = connexion.FlaskApp(__name__,
                         specification_dir='./')
app.add_api('api.yaml')

def make_celery(app):
    celery = Celery(
        app.import_name,
        backend=app.config['CELERY_RESULT_BACKEND'],
        broker=app.config['CELERY_BROKER_URL']
    )
    celery.conf.update(app.config)

    class ContextTask(celery.Task):
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return self.run(*args, **kwargs)

    celery.Task = ContextTask
    return celery

app.config.update(
    CELERY_BROKER_URL='redis://localhost:6379',
    CELERY_RESULT_BACKEND='redis://localhost:6379'
)

celery = make_celery(app)

@app.route('/')
def my_app():
    pass

if __name__ == '__main__':
    app.run(host='<IP_ADDRESS>',
            debug=False)

Now in another module i am importing celery from anomaly_server import celery and im having the same issue with error circular import

Please use the underlying Flask app, which you can access via app.app.


[SPARK-24371] [SQL] Added isInCollection in DataFrame API for Scala and Java.

What changes were proposed in this pull request?
Implemented isInCollection  in DataFrame API for both Scala and Java, so users can do
val profileDF = Seq(
  Some(1), Some(2), Some(3), Some(4),
  Some(5), Some(6), Some(7), None
).toDF("profileID")

val validUsers: Seq[Any] = Seq(6, 7.toShort, 8L, "3")

val result = profileDF.withColumn("isValid", $"profileID". isInCollection(validUsers))

result.show(10)
"""
+---------+-------+
|profileID|isValid|
+---------+-------+
|        1|  false|
|        2|  false|
|        3|   true|
|        4|  false|
|        5|  false|
|        6|   true|
|        7|   true|
|     null|   null|
+---------+-------+
 """.stripMargin

How was this patch tested?
Several unit tests are added.

Test build #91068 has started for PR 21416 at commit ec91220.

Test build #91068 has finished for PR 21416 at commit ec91220.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91068/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3524/
Test PASSed.

Test build #91069 has started for PR 21416 at commit 6459015.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3525/
Test PASSed.

Test build #91069 has finished for PR 21416 at commit 6459015.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91069/
Test FAILed.

Merged build finished. Test FAILed.

+@cloud-fan

Test build #91089 has started for PR 21416 at commit de3296c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3540/
Test PASSed.

Test build #91089 has finished for PR 21416 at commit de3296c.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91089/
Test FAILed.

test this again

retest this please

Test build #91091 has started for PR 21416 at commit de3296c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3542/
Test PASSed.

Test build #91141 has started for PR 21416 at commit 20650d6.

Test build #91142 has started for PR 21416 at commit da10307.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3575/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3576/
Test PASSed.

retest this please

Test build #91143 has started for PR 21416 at commit da10307.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3577/
Test PASSed.

Test build #91144 has started for PR 21416 at commit 730b19b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3578/
Test PASSed.

Test build #91145 has started for PR 21416 at commit 6ff2806.

Test build #91144 has finished for PR 21416 at commit 730b19b.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #91143 has finished for PR 21416 at commit da10307.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #91145 has finished for PR 21416 at commit 6ff2806.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #91141 has finished for PR 21416 at commit 20650d6.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #91142 has finished for PR 21416 at commit da10307.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91145/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91144/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91143/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91142/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91141/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3580/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3581/
Test PASSed.

Test build #91147 has started for PR 21416 at commit 6ff2806.

Test build #91147 has finished for PR 21416 at commit 6ff2806.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91147/
Test PASSed.

Test build #91178 has started for PR 21416 at commit 286a468.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3604/
Test PASSed.

I just found that PySpark's isin is capable of similar feature. One issue is Scala/Java API now doesn't match with PySpark API.

Test build #91178 has finished for PR 21416 at commit 286a468.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91178/
Test PASSed.

@cloud-fan Let me know if the new API looks good to you. Thanks.

@viirya good point. One thing I'm not sure is, does isin(collection: Iterable) conflict with isin(list: Any*)? if they don't conflict, they we can follow pyspark.

Yes. The design of PySpark API seems to be a bit different than Scala/Java API at beginning. If we are going to make them consistent, either we break Scala queries like col.isin(Array[Byte](1,2,3)) or PySpark queries like col.isin([1, 2, 3]).

@cloud-fan unfortunately,  scala vararg can not be overloaded, and scala will return the following error.
Error:(410, 32) ambiguous reference to overloaded definition,
both method isin in class Column of type (values: Iterable[_])org.apache.spark.sql.Column
and  method isin in class Column of type (list: Any*)org.apache.spark.sql.Column
match argument types (Seq[Int])
    checkAnswer(df.filter($"a".isin(Seq(1, 2))),

I'm leaning toward to using isInCollection now, and implement the corresponding python APIs.

Test build #91212 has started for PR 21416 at commit 1332406.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3629/
Test PASSed.

LGTM

LGTM

Test build #91212 has finished for PR 21416 at commit 1332406.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91212/
Test FAILed.

retest this please

retest this please

Test build #91220 has started for PR 21416 at commit 1332406.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3637/
Test PASSed.

Test build #91220 has finished for PR 21416 at commit 1332406.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91220/
Test PASSed.

+1, LGTM.

Test build #91242 has started for PR 21416 at commit fed2846.

@rxin I simplified the test cases as you suggested. Thanks.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3656/
Test PASSed.

Merged build finished. Test PASSed.

LGTM (I didn't look that carefully though)

Test build #91242 has finished for PR 21416 at commit fed2846.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91242/
Test FAILed.

retest this please

retest this please

Test build #91254 has started for PR 21416 at commit fed2846.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3667/
Test PASSed.

Test build #91254 has finished for PR 21416 at commit fed2846.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged into master. Thank you everyone for reviewing.
Followup PR will be created for

Adding tests in Java.
Adding docs about automagical type casting.


[WIP] Upgrade to Rust 2018 and parser optimizations

The NATS codec has been broken due to premature optimizations that actually didn't work. I'm thinking of another way to do so.
The problem being that the old way of doing things was just reading the buffer by borrowing and copying data all around, creating unreasonable slowness on some cases.
Right now it needs to be rewritten IMO since we actually need to make sure we have a correct command to parse before splitting off the buffer and consuming that split in the parser.