Saving tournament report [Closes #249]

Added parameters in compare parsers to enable saving tournament report in csv or md format.
compare_parser.add_argument(
        "--report-format",
        help="define the output format for the command line report (default: markdown).",
        choices=["markdown", "csv"],
        default="markdown")
compare_parser.add_argument(
        "--report-file",
        help="write the command line report also to the provided file",
        default="")

Thanks for the PR @keyur9. I had a quick look. It's a good start but you also need to use these parameters, i.e. ComparisonReporter (in reporter.py) needs to write the report file (see https://github.com/elastic/rally/blob/master/esrally/reporter.py#L242 how it is done in the SummaryReporter that is used for normal reports).

Thank you for the feedback @danielmitterdorfer. As suggested, have updated the ComparisonReporter (wrote the report file similar to how its done in the SummaryReporter) in reporter.py.

Thanks for the follow-up. Could you please refactor it in a way that both reporters call just one implementation? I'd just implement write_single_report as a function and have then both reporter implementations call it.

Thank you for the quick response. Sure, I'll work on refactoring and complete it before end of the week.

Hello @danielmitterdorfer, as suggested I've implemented write_single_report as a function and call the same function from both reporter. I'd appreciate your feedback / suggestions if there is any way I could refactor the code.

Hello @danielmitterdorfer , Thank you for all the feedback. I've fixed all those issue and both implementations are working as expected. However, I faced one issue while generating Tournament report file. The output of difference between baseline and contender is shown in color in console and the diff method returns the difference in color. But when writing the color output in file it is getting converted to something like this \[31;1m+1.49723\[0m.  So, I've changed the code that will not print the colored output in console as it will give appropriate value in report file. Would love to hear your thoughts on it.
Attached output Console colored: ON and Console colored: OFF

@I'd really love to keep the colors in the command line output. How about you generate the report twice, once for console consumption (with the ANSI escape sequence) and once for file output (without the ANSI escape sequences). Generating the report is quite fast and I wouldn't bother doing it twice.
Something like this:
class ComparisonReporter:
    def report(self, r1, r2):
        # ...
        metric_table_plain = self.metrics_table(baseline_stats, contender_stats, plain=True)
        metric_table_rich = self.metrics_table(baseline_stats, contender_stats, plain=False)
        # writes metric_table_rich to console, writes metric_table_plain to file
        self.write_report(metric_table_plain, metric_table_rich)

    def metrics_table(self, baseline_stats, contender_stats, plain):
        self.plain=plain
        metrics_table = []
        # ...
        return metrics_table

    def diff(self, baseline, contender, treat_increase_as_improvement, formatter=lambda x: x):
        diff = formatter(contender - baseline)
        if self.plain:
            color_greater = lambda x: x
            color_smaller = lambda x: x
            color_neutral = lambda x: x
        elif treat_increase_as_improvement:
            color_greater = console.format.green
            color_smaller = console.format.red
            color_neutral = console.format.neutral
        else:
            color_greater = console.format.red
            color_smaller = console.format.green
            color_neutral = console.format.neutral

        if diff > 0:
            return color_greater("+%.5f" % diff)
        elif diff < 0:
            return color_smaller("%.5f" % diff)
        else:
            # tabulate needs this to align all values correctly
            return color_neutral("%.5f" % diff)

You just need to add a new parameter to the function that actually writes the report data and write  metric_table_plain to the output file and metric_table_rich to console. For the summary reporter, you can just provide the same metric table twice (as there are no ANSI escape sequences). Apart from that your change looks good. What do you think?

Hello @danielmitterdorfer, Thanks for the feedback. Generating the report twice didn't took long and have made the changes. Now we would be able to see the console output in color and save the tournament report in md and csv file.

Thanks for following-up @keyur9! I'll try to review it later today.

I did a few tests and unfortunately, a few of them failed. Can you please have a look @keyur9?
Command:
esrally --distribution-version=5.3.0 --test-mode --report-format=csv --laps=2 --report-file=/Users/daniel/Desktop/report.csv

Trace:
Traceback (most recent call last):
  File "/Users/daniel/Projects/rally/esrally/racecontrol.py", line 303, in run
    pipeline(cfg)
  File "/Users/daniel/Projects/rally/esrally/racecontrol.py", line 42, in __call__
    self.target(cfg)
  File "/Users/daniel/Projects/rally/esrally/racecontrol.py", line 244, in from_distribution
    return race(Benchmark(cfg, distribution=True))
  File "/Users/daniel/Projects/rally/esrally/racecontrol.py", line 216, in race
    benchmark.teardown(cancelled)
  File "/Users/daniel/Projects/rally/esrally/racecontrol.py", line 160, in teardown
    reporter.summarize(self.race_store, self.metrics_store, self.cfg, self.track)
  File "/Users/daniel/Projects/rally/esrally/reporter.py", line 15, in summarize
    SummaryReporter(race_store, metrics_store, cfg, lap).report(track)
  File "/Users/daniel/Projects/rally/esrally/reporter.py", line 267, in report
    self.write_report(metrics_table, meta_info_table)
  File "/Users/daniel/Projects/rally/esrally/reporter.py", line 276, in write_report
    write_single_report("%s.meta" % report_file, headers=["Name", "Value"], data=meta_info_table, show_also_in_console=False)
TypeError: write_single_report() missing 2 required positional arguments: 'report_format' and 'cwd'

Command (obviously you need to choose two different timestamp depending on the output of esrally list races):
esrally compare --baseline=20170331T081942Z --contender=20170331T082445Z

Expected result: The report is printed once on the console using colors. No file is written.
Actual result: The report is printed twice: Once without colors, once with colors

Command:
esrally compare --baseline=20170331T081942Z --contender=20170331T082445Z --report-format=csv --report-file=/Users/daniel/Desktop/comparison.csv

Expected result: The report is printed once on the console using colors. The file /Users/daniel/Desktop/comparison.csv is written and contains the same data but without any ANSI escape sequences.
Actual result: The report is printed twice: Once without colors, once with colors. Also the file contents are written twice, once with ANSI escape sequences and once without.

Hello @danielmitterdorfer. Thank you for the feedback. I have fixed the issues and now report is printed once in console and written once to file without ANSI escape sequence.
Now the output for the command -
esrally compare --baseline=20170331T081942Z --contender=20170331T082445Z

Expected result: The report is printed once on the console using colors. No file is written.
Actual result: The report is printed once on the console using colors. No file is written.

esrally compare --baseline=20170331T081942Z --contender=20170331T082445Z --report-format=csv --report-file=/Users/daniel/Desktop/comparison.csv

Expected result: The report is printed once on the console using colors. The file /Users/daniel/Desktop/comparison.csv is written and contains the same data but without any ANSI escape sequences.
Actual result: The report is printed once on the console using colors. The file /Users/daniel/Desktop/comparison.csv is written and contains the same data but without any ANSI escape sequences.

Please let me know if you're fine with changes.

@keyur9 We're almost there. The only test case that failed for me was:
esrally --distribution-version=5.3.0 --test-mode --report-format=csv --laps=2 --report-file=/Users/daniel/Desktop/report.csv

Rally writes not only the actual report but also a second file containing the git commit hash of the Elasticsearch version that it has benchmarked. When Rally attempts to write it with your modified version, we get (trace is shortened):
  File "/Users/daniel/Projects/rally/esrally/reporter.py", line 276, in write_report
    write_single_report("%s.meta" % report_file, headers=["Name", "Value"], data=meta_info_table, data_rich = meta_info_table, show_also_in_console=False)
TypeError: write_single_report() got an unexpected keyword argument 'data'

The keyword argument is now data_plain instead of data and you also forgot to add the new arguments report_format and cwd. If you change line 276 instead to
write_single_report("%s.meta" % report_file, report_format, cwd, headers=["Name", "Value"], data_plain=meta_info_table, data_rich=meta_info_table, show_also_in_console=False)

everything works fine.
I think this is the last change to get this PR in.
So you need

Hello @danielmitterdorfer, Thank you for the feedback. I added both the missing parameters and changed the keyword to data_plain.
Please let me know if you're fine with those changes.


Use Rails 5 version suffix.

Added support for suffixes in Rails 5 migrations.

Tested locally. New seed migrations have proper version now.


Updated: Some dependencies

I have updated some nodejs dependencies to the latest available version. React itself takes some time as I have to mess around quite a bit with the code itself.

P.S. Could one enable "issues" and dependabot warnings so we can fix potential vulnerabilities faster?

I think one of the updates breaks something, the UI only shows loading... for me, and nothing happens. Does it work for you?

Looks like the major upgrade of mobx is what broke things, after reverting those three packages the web UI is working again.
I've also tested them in the following versions (only MINOR updates, not MAJOR), and still works:
    "mobx": "^5.15.7",
    "mobx-react": "^6.3.1",
    "mobx-utils": "^5.6.1",

I don't have enough knowledge about mobx or even React to be able to debug the exact cause. I'd say we should go with the above versions until somebody figures out how to make it work with the latest release.

I got it working using this npx mobx-undecorate tool.
Will push a commit, which also migrates the lockFileVersion from 1 to 2, which should speed up builds and general npm commands, and is backwards compatible to npm 6 and earlier.


Support newer React Native 0.40 and greater

Add conditional import to fix build.
See "breaking changes" in https://github.com/facebook/react-native/releases/tag/v0.40.0

After e1577df, Native code on iOS must refer to headers out of the react namespace. Previously the following would work:
#import "RCTUtils.h"
But now all headers have been moved:
#import <React/RCTUtils.h>

+1

Thanks @RobertBiehl


Autopricer doesn't work for all the items.

Describe the bug

To Reproduce
Steps to reproduce the behavior:

Add an item(For example !add sku=666;6)
Close the bot and change the time of that item to 0 and the price to 0
Start the bot again and that item wont be affected by the autopricer.

Expected behavior

Version: 3.5.1

I mean, it's not really a "bug" since you are changing it's time which normally isn't changed by an ordinary user.

I mean, it's not really a "bug" since you are changing it's time which normally isn't changed by an ordinary user.

Well It doesnt change the price too when you change it manual. Also it wont work even if the prices change on prices.tf

I mean, it's not really a "bug" since you are changing it's time which normally isn't changed by an ordinary user.

It is a bug because the bot can't update the prices using the pricestf pricelist


Save datas in user_ iOS using Alamofire

How could I store all data forms in a particular user id? I have a registration page, so each user will have a different user_id. Then I have forms to fill their basic information, education etc.
For example, suppose user(person) 'A' has registered in the registration page for that "ABCD" random user_id is created and for user(another person) 'B' also registered so "EFGH" user_id is generated.
So their data has to be stored in a particular user id according to their registered user id( A' datas have to store in "ABCD" and B's further information have to store in "EFGH". How could I do this in Alamofire?
My post request for Alamofire is as follows,
@IBAction func eduSave(_ sender: UIButton) {
let parameters: Parameters=[
        "institute_name":univ.text!,
        "degree_name":grad.text!,
        "start_year":cityP.text!,
        "end_year":enYear.text!,
        "degree_city":proj.text!,
        "degree_country":eduCountry.text!
    ]

]
[   //Sending http post request
Alamofire.request(URL_USER_EDUCATION, method: .post, parameters: parameters).responseJSON
{
response in
//printing response
print(response)
            //getting the json value from the server
            if let result = response.result.value {

                //converting it as NSDictionary
                let jsonData = result as! NSDictionary
                  let retrievedString: String? = KeychainWrapper.standard.string(forKey: "user_id")

}
}
let experience = self.storyboard?.instantiateViewController(withIdentifier: "experience") as! ExperienceViewController
self.present(experience, animated: true)
}

Sorry, we use our GitHub project for bug reports and feature requests. In the future, you should open questions like this on Stack Overflow and tag alamofire.
Cheers. 🍻

From our Contribution Guidelines
Asking Questions
We don't use GitHub as a support forum. For any usage questions that are not specific to the project itself, please ask on Stack Overflow instead. By doing so, you'll be more likely to quickly solve your problem, and you'll allow anyone else with the same question to find the answer. This also allows maintainers to focus on improving the project for others.


Reduce iterations in tests

Now that the time for ZQ2 CI / check is down to ~16m, is the motivation for this change weaker?

Now that the time for ZQ2 CI / check is down to ~16m, is the motivation for this change weaker?

Only slightly - the CI still bombs out I think if there are more than 2 PRs running in parallel?

I think we should resolve the CI problems, not reduce the number of tests to work around them.

Yes, that's fair. Closing


semanticTokens/full/delta not working

I am using
Version: 1.71.0 (user setup)
Commit: 784b0177c56c607789f9638da7b6bf3230d47a8c
Date: 2022-09-01T07:36:10.600Z
Electron: 19.0.12
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19042
Sandboxed: No
and following TS modules for client support:
vscode-languageclient: "8.0.2"
What I try to achieve:
I have a very large file which takes several seconds to generate the semantic tokens for the semanticTokens/full Response. I would like to split it into several ranges to keep the system responsive. To achieve this I append resultIdfield to the Result message. Only the desired semanticTokens/full/delta request does not appear.
[Trace - 6:12:15 PM] Received response 'textDocument/semanticTokens/full - (10)' in 1244ms.
Result: {
    "resultId": "someString",
    "data": [
        8,
        7,
        12,
        0,

The server correctly advertises its semanticTokens capabilities:
Received response 'initialize - (0)' in 6ms.
Result: {
    "capabilities": {
        "textDocumentSync": {
            "openClose": true,
            "change": 1,
            "save": {}
        },
        "completionProvider": {
            "triggerCharacters": [
                "."
            ]
        },
        "hoverProvider": true,
        "signatureHelpProvider": {},
        "definitionProvider": true,
        "typeDefinitionProvider": false,
        "implementationProvider": false,
        "referencesProvider": true,
        "documentSymbolProvider": true,
        "codeActionProvider": false,
        "codeLensProvider": {},
        "documentLinkProvider": {},
        "documentOnTypeFormattingProvider": {
            "firstTriggerCharacter": ""
        },
        "foldingRangeProvider": false,
        "executeCommandProvider": {
            "commands": null
        },
        "semanticTokensProvider": {
            "documentSelector": [
                {
                    "language": "ttcn3",
                    "scheme": "file",
                    "pattern": "**/*.ttcn3"
                }
            ],
            "legend": {
                "tokenTypes": [
                    "namespace",
                    "type",
                    "class",
                    "enum",
                    "interface",
                    "struct",
                    "typeParameter",
                    "parameter",
                    "variable",
                    "property",
                    "enumMember",
                    "event",
                    "function",
                    "method",
                    "macro",
                    "keyword",
                    "modifier",
                    "comment",
                    "string",
                    "number",
                    "regexp",
                    "operator"
                ],
                "tokenModifiers": [
                    "undefined",
                    "declaration",
                    "definition",
                    "readonly",
                    "static",
                    "deprecated",
                    "abstract",
                    "async",
                    "modification",
                    "documentation",
                    "defaultLibrary"
                ]
            },
            "range": true,
            "full": {
                "delta": true
            }
        },
        "workspace": {
            "workspaceFolders": {
                "supported": true,
                "changeNotifications": "workspace/didChangeWorkspaceFolders"
            }
        }
    },

What already works:

semanticTokens/full
semanticTokens/range

Is there something I miss in understanding how the interaction between server and client should be or does the lsp client just not react correctly?

The delta request is sent only after a full request has successfully be return and the user types in the document. The server is then ask to send a delta which will move a full result FR1 into a FR2.
What what you describe I think you don't want to have a full request at all and only provide range results. Hover this might cause flickering in the UI and will lead to a uncolored mini map.
I will close the issue. Please ping if things are still unclear.

@dbaeumer Thanks for clarification. I still have some more general questions:

does the typescript lsp client library I mentioned above block the whole lsp client until it receives the semanticTokens/full Response or is it capable to issue other requests in the meantime
is it possible to cancel the semanticTokens/full Request without displaying this event in the UI. Further on it should not lead to further semanticTokens/full Request being issued by the client.

does the typescript lsp client library I mentioned above block the whole lsp client until it receives the semanticTokens/full Response or is it capable to issue other requests in the meantime

No. LSP allows parallel requests. It is up to the server to decide what to do with them

is it possible to cancel the semanticTokens/full Request without displaying this event in the UI. Further on it should not lead to further semanticTokens/full Request being issued by the client.

It is possible for a server to cancel a request. However the client will re-issue the request after a timeout. If you want a server to not respond to these requests anymore use dynamic registration and un-registration.


support for native list types

we store as native arrays in postgres but json strings in sqlite

here also https://github.com/lolopinto/ent/pull/375


Event system to also trigger event on interface classes [feature or improvement]

I am not sure weather it could be useful for others or not. but i got a situation where i wanted to bind event on Interface instead of main class.
See this.
Right now i've to write this.
Event::on(ActiveRecord::class,CommonConstants::EVENT_INIT,function($e){
    if($e->sender instanceof TranslatableModel){

    }
});

I hope i could use  directly
Event::on(TranslatableModel::class,CommonConstants::EVENT_INIT,function($e){

});

TranslatableModel is interface being implemented by various models.
Let me know your thoughts on this, if you think it would be usefull then i wolud like to contribute on Yii2 to make it happen.
little question: i know Event class is almost static, we can't extend anything. if i want little tweaks there then how can i do that without doing things in vendor folder. (if you think any possible way let me know,  afaik i can't do anything for extending it unless i copy whole file and preload it.)

just found that master branch already did this..
Thanks guys..


improve bolthold.Where / query.And

Hi,
some consideratons regarding the query api
The current scheme using bolthold.Where / Query.And over a pointer is not in my opinion the best strategy.
In this silly example, the user has to deal with repetitive code
var q *bolthold.Query
if title!="" {
if q == nil { q = bolthold.Where("Title").Eq(title) }
else{ q = q.And("Title").Eq(title) }
}
if otherthing="" {
if q == nil { q = bolthold.Where("Otherthing").Eq(otherthing) }
else{ q = q.And("Otherthing").Eq(otherthing) }
}
if q == nil { q = new(*bolthold.Query) }
q.Limit(10)
store.Find(some{}, q)

something is broken in this code, could be factorized, a much better out of the box version could look like
var q bolthold.Query
if title!="" {
q.And("Title").Eq(title)   // or q.Where.
}
if otherthing="" {
q.And("Otherthing").Eq(otherthing)  // or q.Where.
}
q.Limit(10)
store.Find(some{}, q)

the and / where mechanic is simplified. in the example they should behave exactly the same way. no matter what who was called first etc.
because q is declared as value type there is no need to new or anything else.
as q is value type, Find method change to a value type also. so writing a criterion less query is painful
var q bolthold.Query
store.Find(some{}, q) // vs nil previously

a solution is to adjust Find method in order to take a variadic query parameter, so the user can write
store.Find(some{})

am i missing someting ?

  Just instantiate it so you don't have to check for nil.  A nil query and a query with no criteria behave the same.
q := &bolthold.Query{}
if title!="" {
q.And("Title").Eq(title)   // or q.Where.
}
if otherthing="" {
q.And("Otherthing").Eq(otherthing)  // or q.Where.
}
q.Limit(10)
store.Find(some{}, q)

i met some issues doing that.


makeotf supported char

I see that you’re using a dash as a separator between the "script" component and the rest of the glyph names. As of November 2015, MakeOTF allows the use of several additional characters in glyph names (+ * : ~ ^ !) but not -. Would you consider perhaps changing the separator to one of these allowed characters instead of the dash (I'd say :), e.g. U+0C8C could become kn:lvocal instead of kn-lvocal?

I have a very simple fix in my fork of the afdko to allow hyphen in glyph names and still support ranges in feature code. Those are the only reason Read Robert didn’t add them to the list of supported chars.

@schriftgestalt would you be willing to contribute this back upstream? :)

I did, but the pull request was never applied.

Oh right yes, I remember now. Found it, https://github.com/adobe-type-tools/afdko/pull/71#issuecomment-148570163

I'm curious what @LettError thinks :)

Thanks for finding this reference. So, makeotf does not support - and it only works in Georg's fork?

Yes. But I don't think that should stop us using hyphens. They are the most natural way to connect/divide different parts.

I understand that you like them. I do also like the slightly different semantics of : suggested by Adam.

The colon looks a bit technical to me. After all we are making the names for people and not for machines. to read.

Reading the discussions about the support and legacy in FDK, I would also opt for : as Adam proposed, its super Read added those extra characters and I understand not to support a hyphen in glyph names.
This package is mainly build to replace AGL and AGD, as Miguel discouraged us to use AGD  (agl was first embedded in fontTools on dec 16 1999, by Just) and we share this knowledge across the world, across different editors, across different workflows, different formats...
Some scripts like cyrillic already pointed out they prefer just to add cyr to the end, so no : and this is also possible.

Some scripts like cyrillic already pointed out they prefer just to add cyr to the end, so no : and this is also possible.

But there should be a solution that works for all scripts. So adding a delimiter for some and not for others is confusing. And is less readable and you need to maintain a list of script suffixes to be able to process it by scripts. For ligatures I add the script tag only once (lam_alef-ar, not lam-ar_alef-ar). So it is much easier with a distinct delimiter.

If we care about legacy and meeting users expectations, then the huge growth in type designers working with non-latin scripts who predominantly use Glyphs is - I think - an important legacy and a big set of expectations...
But I suggest not being too concerned about the past, thinking more about the next 20 years rather than the last 20 years, and setting out a formatter which is ideal and letter other projects adopt it (optionally or by default) as they wish.
With that in mind, I think the script clearly ought to be prefixed, not affixed as Glyph does, so that when sorting lists of glyph names, the glyphs that belong to the same script are sorted together.
On the face of it I would prefer - to be used, and I am mystified why @readroberts and @miguelsousa didn't accept @schriftgestalt 's patch because he handled the issue they said blocked such an improvement.
If they are sure that they won't accept that patch, then it does not seem so important to me and seems like an okay compromise to use :. I don't think it or the other choices are as easy to scan-read as - (since they are narrower or have less white space) or as easy to type (they all require a shift key on English keyboards)

I am in favor of supporting the future, but am reluctant to invalidate the past when there are alternatives. I hadn't seen the comment by schriftgestalt that his patch did support legacy use of hyphen for ranges, but now that I look at it, I may not fully understand it. Looking at the regular expression for glyphnames in the patch, I don't see how it could differentiate the glyph name 'a-z' from the range 'a-z', and I can't come up with a regular expression that would do so.  For new feature files, you can require spaces around the  hyphen, but we have a collection of feature files going back to 2001. The issue also is larger than just the feature file. We have a large library of command lines tools and shell and Python scripts that call them and use hyphens for glyph name ranges.
The one thing I have been able to think of is to add a check when a feature file range is seen: assume that the string is a glyph name if the full string matches a known glyph and one side of the range produces a name which is not in the font. If someone is up for doing this, I'd be happy to accept it.I might get around to it myself eventually.  This check does not solve the  problems with command line tools, and there are still cases the logic would not correctly solve (when it is a glyph name and when treated as range, the names on both sides are also glyph names), but it would cover almost all real use cases in feature files. The Adobe Type group and others who use command line tools are still free to not use hyphens in glyph names, and since these won't show up in final font names, the command line issue wouldn't be much of a problem for anyone other than the developers who do use hyphens in glyph names. I'm all for letting people make their own decisions about risks vs value for such UI issues.

Looking at the regular expression for glyph names in the patch, I don't see how it could differentiate the glyph name 'a-z' from the range 'a-z'

It doesn’t it is not handled on that level.

when it is a glyph name and when treated as range, the names on both sides are also glyph names

The one thing I have been able to think of is to add a check when a feature file range is seen: assume that the string is a glyph name if the full string matches a known glyph and one side of the range produces a name which is not in the font. If someone is up for doing this, I'd be happy to accept it

It will first assume it is a glyph name and if it doesn’t find it, it break the name down into its pieces and tries again. see here
I know that it isn’t the most elegant solution but it works. All names I defined in my glyph lists can’t be mistaken for a range. So if some uses ranges there are no possible glyph pairs that can be mistaken for a glyph name. And all legacy code that has ranges doesn’t have hyphen names to begin with. The only downside is a tiny performance hit for the missed check for a not existing glyph name. But that should be in the milliseconds.
And if someone for whatever reason comes up with a glyph name that is made from two other names in the same font and he likes to use ranges, he simply needs to add spaces before and/or after the hyphen.
This is exactly whats happening.

@readroberts What problems do you see for using : ?

My error for not looking through your  Sep 28, 2015 commit carefully enough to see the feat.c changes: I looked only at the featgram.g changes. The logic you have implemented looks good to me, and will certainly avoid any problems with pre-existing feature files. I'll fold this into makeotf the next time I work on it, which is in the next week or two.
I would still caution font developers to avoid using a hyphen because it will in some cases cause problems with the FDK command line tools. However, I also recognize that this is not an issue for many font developers.
for ":", I don't see any issue. It is not a Unix shell wild card, and very few tools care about the fact that it was once a directory path separator under Mac OSX 9. It is currently supported in  glyph names by makeotf.

Awesome Read, it was a concern for me that glyphs had an incompatible fork
of fdk, so great news this can go upstream after all.
Does anyone prefer : to -?
On Saturday, 12 March 2016, Read Roberts<EMAIL_ADDRESS>wrote:

My error for not looking through your Sep 28, 2015 commit carefully enough
to see the feat.c changes: I looked only at the featgram.g changes. The
logic you have implemented looks good to me, and will certainly avoid any
problems with pre-existing feature files. I'll fold this into makeotf the
next time I work on it, which is in the next week or two.
I would still caution font developers to avoid using a hyphen because it
will in some cases cause problems with the FDK command line tools. However,
I also recognize that this is not an issue for many font developers.
for ":", I don't see any issue. It is not a Unix shell wild card, and very
few tools care about the fact that it was once a directory path separator
under Mac OSX 9. It is currently supported in glyph names by makeotf.
—
Reply to this email directly or view it on GitHub
https://github.com/LettError/glyphNameFormatter/issues/11#issuecomment-195816881
.

--
Cheers
Dave

I’d vote for -.
I must say that if @readroberts does after all accept the contribution of @schriftgestalt to also allow - in glyph names, then I’d say we should all settle on -. I do agree with Georg that - is the most obvious choice, and anything else such as : is a bit more technical. - can be input with one hand while : mostly needs Shift. My only reservation towards using - was that Read had reservations about accepting it in the mainstream FDK so - was difficult to use for other tools.
But given now that Read is willing to put - into the mainstream FDK, and that Georg has published GlyphsInfo which already includes a curated list of human-readable glyph names, I think the most sensible course of action would be to:

Start using - instead of : in glyphNameFormatter
Work towards aligning GlyphsInfo and glyphNameFormatter

This reminds me of the history of ISO 10646 and Unicode. In 1989, they started as two separate, incompatible “universal character sets”. Fortunately, soon enough the ISO working group and the Unicode consortium realized that having two “universal” character sets was not a good idea, so they combined the efforts and started producing a synchronized standard. Today, ISO 10646 is mostly “the repertoire” while Unicode is “the repertoire plus lots of other metadata”.
I could see how glyphNameFormatter could become ISO 10646 (being largely a vehicle of generating the names plus a repository of those), while GlyphsInfo/GlyphData.xml could be its own “Unicode” in the sense that it can define many more additional properties and metadata.
But the name set in both projects should be the same, really. :)
What do you think, @typemytype , @LettError and @schriftgestalt ? :)
Best,
Adam

GlyphNameFormatter is a generator: a suffix, prefix, a - or : or even ✌ is optional and all possible...
As Read still pointed out some reservations for - I would still not advice to use the hyphen. But yeah this is a generator, use whatever you like :) see https://github.com/LettError/glyphNameFormatter/tree/master/Lib/glyphNameFormatter/names for different outputs.
GlyphNameFormatter is not a fixed list, editors can use it to recommend readable glyph names from unicode data. But users are still free to name glyph names whatever they want. Editors could use this to build glyph names on the fly with some users prefs.
GlyphNameFormatter is also open and very clear how glyph names are constructed. Exceptions are documented. It is not a manually constructed list.

But the name set in both projects should be the same, really. :)

why? I guess there are afii uni fans out there or simple camelCaseFans. The names are already different from the AGL and AGD lists.

That is a good point. Would it be possible to have presets? So that if someone needs to make some names, he could pick the style he needs to match the environment he needs the names in?

On 24 March 2016 at 07:08, Frederik Berlaen<EMAIL_ADDRESS>wrote:

But the name set in both projects should be the same, really. :)
why?

Because some users like to move between tools

The lovers of afii or camelCase won’t find much fun with this generator, I think.
Glyph names are akin to the vocabulary of a natural language. Most people are comfortable with a certain standardized vocabulary (when discussing things on forums or using menus in apps).

Sure, there is a place for that — but most of those who design type don’t want to invent a new language at the same time. They want to learn an existing one — perhaps inventing one or two new words on occasion.
In my experience, glyph naming has been a major obstacle in exchanging linguistic metadata (e.g. sharing glyph repertoires for dedicated fields of use), in effective functional QAing and in writing of scripts or extensions that deal with the linguistic metadata of fonts.
I think the customization aspect of the formatter is great, but in real life, we probably also want a sensible default — 98% of people will stick to it.
I don’t particularly enjoy the fact that Unicode texts exist in NFC and NFD forms (and mixes of those), and in UTF-16, UTF-8 and UTF-32. The constant dilemma between cubic and quadratic curves in font also doesn’t help much.
Sure, options are nice for the edge cases, but for the 98% of projects, a common set of glyph names that most font tool makers agree to and present to the users as defaults would reduce the transactional costs of collaborative type design and of development of auxiliary tools.

I see a standardized glyph naming scheme an extension of the UFO philosophy. In the original presentation of UFO, Tal spoke how UFO would allow different the same font project to be moved between different tools, and how in a 50 years of now, some could take the sources of a UFO project and recreate a usable form from it. For this, UFO strives to have a common vocabulary to express what’s a point, a hint or a guideline. But we lack a common vocabulary to express what’s a “small-cap б”.
The Adobe-curated glyph lists have been a good help, but their origins lie in the 1980s and they have been constructed over a long time, so they now suffer from fatigue and inconsistencies, like FontLab Studio 5 or RoboFab.
This project gives us a chance to make a clean start, and since most makers of type design apps are here talking, it seems that we all see that the interest and need is there.
I appreciate that there is a dynamic generator, though I think that — since any given version of Unicode is finite — the finite static list that will be the output of the generator will be the most “popular” product. Any tool will be able  to integrate such a static list, though code written in Python and returning names dynamically will, no doubt, be useful to some as well.

@schriftgestalt This line shows how to make control the generation of lists. I don't mean this answer to be glib, it is just to illustrate that it does not need much.
The lists contain a link to version on github that generated the names, so you can always find the right state.

@twardoch I don't have an illusion that this initiative can be normative for old names. Regardless of what conclusion might be reached in this forum, there is too much history and there are too many legacy projects in too many tools at too many foundries. Existing lists are incomplete, there are many gaps in the AGD and perhaps in @schriftgestalt's as well, just because the glyphs were obscure and someone somewhere decided to not include it in a font. Not including something can be valid decision when publishing a font, a tool on the other hand should not have such a bias. If it has a unicode name it should have a glyph name. The tool should not decide what is relevant or not.
The challenge we all face that our lists are not complete, and each unicode revision will bring new objects that also need naming. Only a parametric approach can guarantee coverage. And this is where collaboration will be beneficial to all don't you think?

I absolutely agree. From what Georg told me, his method of building the names in GlyphData.xml was also "parametric with tweaks". I definitely prefer a parametric method with tweaks than a hand-made list, sure :)
I think the best outcome were if GlyphData.xml and this formatter were turned into "the best of both worlds" (three actually, given AGL/AGD is still part of the legacy). This might mean that we could look at the GlyphData.xml names (now opensourced) and see where our parametric system is better or worse from GlyphData.xml.
I tried to make the point earlier that if FDK adopts - and Glyphs has successfully used the -suffix then we can make it default.
In the end, perhaps Glyphs could change some of the GlyphData.xml names "our way", we could change some "Glyphs' way", and in the end we'd unify and use one system (i.e. the formatter or the formatter-made lists) everywhere.
I'm saying that many ideas that Georg has done in the Glyphs names are good, and he's already painstakingly combed through the names himself, so if exceptions are needed, his exceptions should be first to look at.

Georg has definitely done a lot of work - the problem is that the list contains hardcoded suffixen which makes it difficult to compare to or even find the exceptions. Maybe @schriftgestalt can contribute a list with base names and separate script extensions?

In the end, perhaps Glyphs could change some of the GlyphData.xml names "our way", we could change some "Glyphs' way", and in the end we'd unify and use one system (i.e. the formatter or the formatter-made lists) everywhere.

That sounds good

the problem is that the list contains hardcoded suffixen which makes it difficult to compare to or even find the exceptions.

One reason I insist of having a unique delimiter is that it is very easy to parse. If you need to list of plain names, ignore everything after the hyphen or period.

It's the choice of delimiter that is the problem.

Didn’t you pick the hyphen as a potential delimiter yourself?


NonNullList funkiness when using KeyValuePair<TKey,TValue> as its generic type

I'm not able to properly store KeyValuePair<TKey,TValue> elements inside a NonNullList<T>. Accessing an element's Key or Value property produces very funky behaviour.
Min repro code:
using ProductiveRage.Immutable;
using System;
using System.Collections.Generic;

namespace NonNullListFunkyStructMinRepro
{
    public class App
    {
        public static void Main()
        {
            var item = new KeyValuePair<string, string>("abc", "def");
            Console.WriteLine(item);

            var list = NonNullList<KeyValuePair<string, string>>.Empty;
            list = list.Add(item);
            Console.WriteLine(list[0]);
        }
    }
}

The first Console.WriteLine call emits as expected:
[abc, def]

The second Console.WriteLine should do the same, but it instead prints this:
[function (key, value) {
                if (key === undefined) {
                    key = Bridge.getDefaultValue(TKey);
                }

                if (value === undefined) {
                    value = Bridge.getDefaultValue(TValue);
                }

                this.$initialize();
                this.key = key;
                this.value = value;
            }, ]

And the output from using the browser's console with the underlying JS directly:
> item
Object { $initialized: core.emptyFn(), key: "abc", value: "def" }
> list.getItem(0)
Object { $initialized: core.emptyFn(), key: .ctor(key, value), value: null, $$fullname: "System.Collections.Generic.KeyValue…", $kind: "struct", $$name: "System.Collections.Generic.KeyValue…" }

Things to note:

KeyValuePair<TKey, TValue> is a struct
This behaviour doesn't happen when I have my own custom generic struct (MyStruct<TKey, TValue>) that has getter-only Key and Value properties
Other plains structs like DateTime work fine
Similar behaviour happens when using KeyValuePair<int, int>, though list[0].Value then seems to emit 0 instead of nothing, making it seem like it just emits the default value for the expected type.

Given that NonNullList<T> works well with custom generic structs, it looks like there's some magic going on with Bridge's KeyValuePair<TKey,TValue> implementation.

Strange that it works for other structs for you, I get the same problem if I use custom structs! eg.
using System;
using Bridge.Html5;
using ProductiveRage.Immutable;

namespace Tester
{
	public static class Class1
	{
		[Ready]
		static void Main()
		{
			Console.WriteLine("HI!");

			var item = new MyKeyValuePair<string, string>("abc", "def");
			Console.WriteLine(item.Key);

			var list = NonNullList.Of(item);
			Console.WriteLine(list[0].Key);

			var retrievedItem = list[0];
			Console.WriteLine(retrievedItem.Key);
		}
	}

	public struct MyKeyValuePair<TKey, TValue>
	{
		public MyKeyValuePair(TKey key, TValue value)
		{
			Key = key;
			Value = value;
		}
		public TKey Key { get; }
		public TValue Value { get; }
	}
}

Confirmed that this is a Bridge bug (see forums.bridge.net/forum/bridge-net-pro/bugs/3947-generic-type-issue-with-struct-encountered). The good news is that it's already fixed in their main branch. The bad news is that I have no idea when this will be released (I asked in that bug report but no answer is forthcoming yet),

The version of Bridge that addresses this should be getting released to a beta test period early next week. So hopefully it will be possible to close this issue soon!

Bridge 16 fixes this but it's been in beta forever, don't think there's much point leaving this issue open any longer!


Update go 1.16.7

Description
Updates go to 1.16.7 using these (steps)[https://github.com/transcom/mymove/wiki/upgrade-go-version#how-to-upgrade-go-version].
Set up
See instructions in https://github.com/transcom/mymove/wiki/upgrade-go-version
Or:
Update .tool-versions to point to new version of golang
If you've done some PATH sorcery to point to a specific Go version (as detailed here), you'll have to update that as well
(if you do not yet have asdf installed, see How to get up and running)
Run asdf install to install the new version
Run asdf global golang to update global version as well
go version and asdf which go to check it worked
Add this change to the PR created in the next step

go version
go version go1.16.7 darwin/amd64

asdf which go
/Users/ryan/.asdf/installs/golang/1.16.7/go/bin/go

Code Review Verification Steps

[ ] If the change is risky, it has been tested in experimental before merging.
[ ] Code follows the guidelines for Logging
[ ] The requirements listed in Querying the Database Safely have been satisfied.
Any new migrations/schema changes:

[ ] Follow our guidelines for zero-downtime deploys (see Zero-Downtime Deploys)
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] User facing changes have been reviewed by design.
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

Warnings

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against ea5df3642279beac398e54c1ec889fb06383421c

Change good. I am using nix, probably should get confirmation from someone with asdf setup too.

oops, accidently closed it. working fine with asdf.


[test] GPU dgemm and pointer chasing tests

New dgemm and pointer_chase regression tests. Like all the other microbenchmark tests, these run with both cuda and hip.
Pointer chase checks:

Check the latency of calling the clock function.
Average latency on a single device. This test uses several stride values.
Average P2P latency. The stride is fixed to 1 here.
L1 latency, L1 miss rate, and miss latency on a single device.
L1 latency, L1 miss rate and miss latency when executing the pointer chase through P2P

Hello @jjotero, Thank you for submitting the Pull Request!

In the file cscs-checks/microbenchmarks/gpu/pointer_chase/pointer_chase.py:

Line 114:80: E501 line too long (81 > 79 characters)

Do see the ReFrame Coding Style Guide

Hello @jjotero, Thank you for submitting the Pull Request!

In the file cscs-checks/microbenchmarks/gpu/pointer_chase/pointer_chase.py:

Line 114:80: E501 line too long (81 > 79 characters)

Do see the ReFrame Coding Style Guide

Codecov Report

Merging #1687 (7779679) into master (1160d2d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1687   +/-   ##
=======================================
  Coverage   87.21%   87.21%
=======================================
  Files          45       45
  Lines        7546     7546
=======================================
  Hits         6581     6581
  Misses        965      965

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1160d2d...7779679. Read the comment docs.

Codecov Report

Merging #1687 (7779679) into master (1160d2d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1687   +/-   ##
=======================================
  Coverage   87.21%   87.21%
=======================================
  Files          45       45
  Lines        7546     7546
=======================================
  Hits         6581     6581
  Misses        965      965

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1160d2d...7779679. Read the comment docs.

@jenkins-cscs retry all

@jenkins-cscs retry all

The clockLatency test is ver very far from meeting the performance in Daint. However, it passes in Dom without any problems.

The clockLatency test is ver very far from meeting the performance in Daint. However, it passes in Dom without any problems.

Great work, seeing a working hip/cuda abstraction really makes me want to try it out for my projects too!
Also impressive level of detail with the timers.
I'm sorry I just saw the review request now, will add my review shortly.

Great work, seeing a working hip/cuda abstraction really makes me want to try it out for my projects too!
Also impressive level of detail with the timers.
I'm sorry I just saw the review request now, will add my review shortly.

Hmm, not sure there is a need for more node jumps. All these reframe tests do not use the random init, and despite having only 64 nodes, the timings are pretty consistent.
There are two types of tests in here. The first type measures the overall traversal time, and the second one measures each jump independently. For the second type, the reframe tests work out the L1 hits and misses, and compute the average timing only for these two categories. As it can be seen in the figure below (data from a P100), the timings for an L1 hit are pretty consistent at 158 cycles. If we were to capture the latency of L2 (which sits somewhere below 400 cycles), then I'd agree that we might need more nodes. But in that case, extracting the data that only belongs to an L2 hit is a challenge on its own.
Though, as you say, if we make it circular we could easily tune the list to fit in any memory level. Let me experiment a bit.

The linked lists are now circular and the number of node jumps is fixed to 256. The number of nodes is now an input parameter from the user, which means that we can do something like a list with 2 nodes (resident in L1) and jump around 256 times to collect all the data we need.
At the moment, there is some inconsistent behaviour with the clock latency on the P100s. This needs further testing in the other cards present in Ault.

The timings don't seem converged. As I mentioned previously, setting the number of hops to a fixed number is not a viable approach and also 256 jumps are not enough by a long shot to get converged results.
I took the code and made a small modification: I added num_jumps as a command line parameter, changed the traversal to a while loop as described above and increased num_jumps for each list size until the time per jump converged.
Here are the results on V100:
V100.pdf
and P100:
P100.pdf
List linkage was random, with no additional node-padding.
On the V100, the L1 latency now is 28 cycles which agrees with other reports that I found online.
The reason why 256 jumps are not nearly enough is in fact not due to timing overhead. Like CPUs, GPUs nowadays have multiple (opaque hardware-controlled) levels of cache and GPUs also use virtual memory, which means that page table lookup also happens. Despite a warm-up pass through the list, where exactly an address resides and whether or not there's a page-fault is beyond your control for individual accesses. You can only hope to iron out these effects by averaging over multiple repeat passes. The faster the device and the faster the cache, the more pronounced these effects become.
In this context, I would say that trying to time individual jumps is an interesting experiment, but in practice actually just yields random linear combinations of the different cache/DRAM values. Again, not because the timer is not accurate enough, but due to the mentioned background effects that you cannot control.

The tests are in now.


What is Microsoft recommendation on automation of Advanced eDiscovery cases in Microsoft 365?

What is Microsoft recommendation on automation of Advanced eDiscovery cases in Microsoft 365?

Create advanced eDiscovery Cases
Manage advanced eDiscovery Cases
Export advanced eDiscovery search reports
Dashboard for total no. of cases, total no. of holds, total no. of SPO sites on hold, total no. of teams on hold etc.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 46b65228-2f7b-17bc-bba4-1735618208da
Version Independent ID: ef891290-e3f0-d63b-5586-267733d45b01
Content: Create and manage Advanced eDiscovery cases in Microsoft 365 - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/create-and-manage-advanced-ediscoveryv2-case.md
Service: o365-seccomp
GitHub Login: @markjjo
Microsoft Alias: markjjo

Any update on this would be great. Thanks.

@jaganpt I believe you can use Microsoft Graph APIs to automate some if not all of these tasks. Please see: https://docs.microsoft.com/en-us/graph/api/resources/ediscovery-ediscoveryapioverview?view=graph-rest-beta
For the last one about holds, we have published a PowerShell script that generates a holds report; see https://docs.microsoft.com/en-us/microsoft-365/compliance/create-a-report-on-holds-in-ediscovery-cases?view=o365-worldwide
Finally, there are plans to release a new holds report functionality in Advanced eDiscovery. This report will provide info about holds in both Core and Advanced eDiscovery. See this roadmap item for a brief description: https://www.microsoft.com/en-us/microsoft-365/roadmap?filters=&searchterms=88540
Hope this helps.


Help with getting the Video stream working over RTSP

Trying to play around with my Mebo using your repo. Is there any way I can get the Video stream working? Trying to tinker around with the code in the stream folder, but no luck.
m.media.start_streams() gives me the Exception: supply stream password
Where do I get the stream password?
Any help is appreciated. Thanks.

Figured out how to get it working. Thanks. Closing issue.


Create Mammasse_Amine

ok thx i will fix that . i fixed the Octocat img

for the Octocat, you have to upload the PNG file itself


how can i change monthView to Weekview with animation

how can i change monthView to Weekview with animation
when i pull up the calendar view

https://github.com/prolificinteractive/material-calendarview/issues/278#issuecomment-218274439
You can set android:animateLayoutChanges="true" and LayoutTransition#enableTransitionType(CHANGING) on your parent layout.

Closing this issue, feel free to comment #278 if you need more info.


Update designer wording check

I updated japanese wording.
Please check.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


LUIS bot creations fails with "ajaxExtended Call Failed" error

The Microsoft Bot Framework team prefers that "How To" questions be submitted on Stack Overflow. The official Bot Framework GitHub repository  is the preferred platform for submitting bug fixes and feature requests.
Bot Info

SDK Platform:
SDK Version:
Active Channels:
Deployment Environment:

Issue Description
LUIS bot framework fails with error "ajaxExtended Call "
Code Example

Reproduction Steps

Expected Behavior

Actual Results

This is a duplicate of https://github.com/Microsoft/BotBuilder/issues/4893 and https://github.com/Microsoft/BotBuilder/issues/4892


Allure-results folder is getting bigger and bigger

Hi, I have a question about the results folder. What is the best practice to deal with it? Because it will generate a lot of files everytime I run the test. Are there any methods to clean it? Thanks,

It is up to you. This question is not related to reporter itself. Please, ask in official allure gitter chanel https://gitter.im/allure-framework/allure-core

Thanks @BorisOsipov


[SIS-041] Add and configure redux

WIP
Replace Library.js to manually test that everything works lol.
import React from 'react';
import { SafeAreaView, Text, Button } from 'react-native';
import { isTokenValid } from '../services/login/AuthService';
import { listDriveFiles } from '../services/routes/Drive';

const Library = ({ navigation }) => {
  return (
    <SafeAreaView>
      <Text>Library of user's lesson plans here!</Text>
      <Button
        onPress={() => listDriveFiles()}
        title="List drive files"
        color="#841584"
      />
      <Button
        onPress={() => console.log(isTokenValid())}
        title="Is token valid?"
        color="blue"
      />
    </SafeAreaView>
  );
};

export default Library;

good news: thanks to emily's ginormous brain the onBlur issue is fixed on iOS
bad news: it only works if you tap certain areas, not the entire screen (like if it's too far on the edge it doesn't work because the TouchableWithoutFeedback component doesn't wrap around a big enough area)
is this a problem tho? thots? @yuemily24 @Lemiex


Rancher 2.5 (Single Install) not starting after nf_conntrack_max value adjustment

What kind of request is this (question/bug/enhancement/feature request): bug
Steps to reproduce (least amount of steps as possible):
In this scenario, the Docker Host where the Rancher container is started is a LXC container.
root@ubuntu4:~# sysctl -a|grep conntrack_max
net.netfilter.nf_conntrack_max = 524288

root@ubuntu4:~# docker run -d --name rancher --restart=unless-stopped -p 80:80 -p 443:443 --privileged rancher/rancher:v2.5.8

Installation and setup works fine.
Now the nf_conntrack_max value is set to another value on the LXC host:
root@host ~ # sysctl -w net.netfilter.nf_conntrack_max=525000
net.netfilter.nf_conntrack_max = 525000

Inside the LXC container, the nested Rancher container is restarted:
root@ubuntu4:~# docker restart rancher

root@ubuntu4:~# docker logs --follow rancher
[...]
I0624 07:38:23.051838      54 node.go:136] Successfully retrieved node IP: <IP_ADDRESS>
I0624 07:38:23.051867      54 server_others.go:143] kube-proxy node IP is an IPv4 address (<IP_ADDRESS>), assume IPv4 operation
I0624 07:38:23.052887      54 server_others.go:186] Using iptables Proxier.
I0624 07:38:23.053288      54 server.go:650] Version: v1.19.8+k3s1
I0624 07:38:23.053960      54 conntrack.go:103] Set sysctl 'net/netfilter/nf_conntrack_max' to 524288
F0624 07:38:23.053999      54 server.go:495] open /proc/sys/net/netfilter/nf_conntrack_max: permission denied
2021/06/24 07:38:23 [FATAL] k3s exited with: exit status 1

Result:
Start of Rancher container fails. At least if the current value is same or higher than the remembered value.
Expected Result:
Start of Rancher container succeeds.
Other details that may be helpful:
The problem seems to be in the func realConntracker in k3s/conntrack.go:
https://github.com/k3s-io/k3s/blob/master/vendor/k8s.io/kubernetes/cmd/kube-proxy/app/conntrack.go#L98
This function checks the current sysctl value and compares it to a (somewhere internally retained?) internal value:
func (realConntracker) setIntSysCtl(name string, value int) error {
	entry := "net/netfilter/" + name

	sys := sysctl.New()
	if val, _ := sys.GetSysctl(entry); val != value {
		klog.Infof("Set sysctl '%v' to %v", entry, value)
		if err := sys.SetSysctl(entry, value); err != nil {
			return err
		}
	}
	return nil
}

If the current system value doesn't match the internal value, sys.SetSysctl is launched to set the previously known value. Yet this fails in a LXC container as the value is defined at Host level:
root@ubuntu4:~# sysctl -w net/netfilter/nf_conntrack_max=524288
root@ubuntu4:~# sysctl -a|grep conntrack_max
net.netfilter.nf_conntrack_max = 525000

Probably related: https://github.com/kubernetes-sigs/kind/pull/2241
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): 2.5.8

Installation option (single install/HA): Single Install

Docker version (use docker version):

root@ubuntu4:~# docker version
Client:
 Version:           20.10.2
 API version:       1.41
 Go version:        go1.13.8
 Git commit:        20.10.2-0ubuntu1~20.04.2
 Built:             Tue Mar 30 21:24:57 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.2
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.8
  Git commit:       20.10.2-0ubuntu1~20.04.2
  Built:            Mon Mar 29 19:10:09 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4-0ubuntu1~20.04.2
  GitCommit:
 runc:
  Version:          1.0.0~rc95-0ubuntu1~20.04.1
  GitCommit:
 docker-init:
  Version:          0.19.0
  GitCommit:

This issue appears to be dependent on upstream: https://github.com/k3s-io/k3s/pull/3505#issuecomment-867795852

This also affects/breaks deploying a Rancher managed Kubernetes cluster to nodes. The Kubernetes deployment is stuck on kube-proxy deployment. kube-proxy tries to set a specific value as nf_conntrack_max but fails:
I0723 07:30:08.943595   12314 node.go:172] Successfully retrieved node IP: <IP_ADDRESS>
I0723 07:30:08.943627   12314 server_others.go:142] kube-proxy node IP is an IPv4 address (<IP_ADDRESS>), assume IPv4 operation
W0723 07:30:08.944553   12314 server_others.go:584] Unknown proxy mode "", assuming iptables proxy
I0723 07:30:08.944629   12314 server_others.go:182] DetectLocalMode: 'ClusterCIDR'
I0723 07:30:08.944652   12314 server_others.go:185] Using iptables Proxier.
I0723 07:30:08.944760   12314 proxier.go:287] iptables(IPv4) masquerade mark: 0x00004000
I0723 07:30:08.944857   12314 proxier.go:334] iptables(IPv4) sync params: minSyncPeriod=1s, syncPeriod=30s, burstSyncs=2
I0723 07:30:08.944977   12314 proxier.go:346] iptables(IPv4) supports --random-fully
I0723 07:30:08.945162   12314 server.go:650] Version: v1.20.8
I0723 07:30:08.945589   12314 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072
F0723 07:30:08.945634   12314 server.go:495] open /proc/sys/net/netfilter/nf_conntrack_max: permission denied

kube-proxy is then restarted in a loop, trying to fix it.
On Rancher server side, the following error is logged:
2021/07/23 07:40:55 [ERROR] Failed to upgrade worker components on NotReady hosts, error: [Failed to verify healthcheck: Failed to check http://localhost:10256/healthz for service [kube-proxy] on host [<IP_ADDRESS>]: Get "http://localhost:10256/healthz": dial tcp <IP_ADDRESS>:10256: connect: connection refused, log: F0723 07:40:32.920751   19619 server.go:495] open /proc/sys/net/netfilter/nf_conntrack_max: permission denied]

This breaks Rancher Kubernetes deployments in LXC and LXD environments.
My Kubernetes PR is still open unfortunately.

On master-head - commit id: 94bca993f
docker install rancher works and rancher server comes up.

@sowmyav27 Did you try this inside a container? Now change the value of nf_conntrack_max on the host. Then restart Rancher. What happens? Please don't just close issues without proper validation, thanks.
Also, this does not only apply to the Single Rancher Install (as mentioned in https://github.com/rancher/rancher/issues/33360#issuecomment-885462965) but also to deployments of new Kubernetes clusters from within Rancher (Rancher deployed Kubernetes clusters).

@Napsty It has been fixed in k3s version: v1.21.3+k3s1 Related PR - https://github.com/k3s-io/k3s/pull/3341 and this version is embedded in the rancher version that was tested on .

@sowmyav27 Alright, I was missing that information then, thanks for sharing the PR.
That covers the k3s part but not the k8s from Rancher launched clusters. How will this be solved?

Hi @Napsty - the issue was originally filed for single node install, as mentioned in the title, so that's why we validated for it in particular.
However, I do see your recent comment (https://github.com/rancher/rancher/issues/33360#issuecomment-885462965), so we should validate that as well.

not the k8s from Rancher launched clusters
How are you launching these clusters? Is there a reproducible workflow? I presume this is with RKE.

Yes, the title of the issue is a bit misleading. I only got aware that the same issue also affects k8s installations later.
To reproduce you can basically just create a new Kubernetes cluster from within the Rancher interface (create new cluster, custom). Just make sure the cluster nodes are actually containers. I was able to reproduce with LXC and LXD containers. I am not 100% certain but I believe this method uses rke launched from Rancher itself.
As mentioned in the previous comments, a workaround is to change kube-proxy conntrack to ignore higher values. With this workaround the Kubernetes deployment should not get stuck at kube-proxy deployment anymore as long as the hosts value is high enough (which can be prepared in advance). As conntrack comes from Kubernetes upstream, it probably needs to be fixed there but my PR there seems to attract nobody 😅
Thanks for looking into this.

Just make sure the cluster nodes are actually containers

Rancher 2.5 creates downstream clusters with RKE when you select "custom". When you are given the Docker run command, it sets up a node for an RKE cluster. I believe you may be encountering an unsupported use case
From the RKE docs: "It works on bare-metal and virtualized servers."

https://rancher.com/docs/rancher/v2.5/en/cluster-provisioning/node-requirements/
https://rancher.com/docs/rke/latest/en/

Does this fail when you create cluster nodes that aren't containers?

Does this fail when you create cluster nodes that aren't containers?

No, it works fine in VMs and bare-metal.
Thanks for the edit part ;-). Yes, it would mean a large improvement. Rancher managed clusters could be rolled out much quicker (in LXC/LXD environments). Thx for fixing it in k3s with a workaround. I guess to work in k8s (rke deployed Kubernetes) this needs to be fixed in k8s upstream (see my PR).
We can leave this ticket closed or if you prefer I change the issue subject.

Actually I have to revise my last comment.
I just wanted to install a clean new install of Rancher 2.5.9 Single Node Install inside a KVM. Installation of Rancher fails.
Environment: Virtual Machine (KVM), Ubuntu 20.04 (guest OS), 4vCPU, 8GB Memory
2021/08/03 10:47:49 [INFO] Rancher version v2.5.9 (3c5418944) is starting
2021/08/03 10:47:49 [INFO] Rancher arguments {ACMEDomains:[] AddLocal:true Embedded:false BindHost: HTTPListenPort:80 HTTPSListenPort:443 K8sMode:auto Debug:false Trace:false NoCACerts:false AuditLogPath:/var/log/auditlog/rancher-api-audit.log AuditLogMaxage:10 AuditLogMaxsize:100 AuditLogMaxbackup:10 AuditLevel:0 Agent:false Features: ClusterRegistry:}
2021/08/03 10:47:49 [INFO] Listening on /tmp/log.sock
2021/08/03 10:47:49 [INFO] Running etcd --data-dir=management-state/etcd --heartbeat-interval=500 --election-timeout=5000
2021-08-03 10:47:49.106840 W | pkg/flags: unrecognized environment variable ETCD_URL_arm64=https://github.com/etcd-io/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-arm64.tar.gz
2021-08-03 10:47:49.106904 W | pkg/flags: unrecognized environment variable ETCD_URL_amd64=https://github.com/etcd-io/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-amd64.tar.gz
2021-08-03 10:47:49.106914 W | pkg/flags: unrecognized environment variable ETCD_UNSUPPORTED_ARCH=amd64
2021-08-03 10:47:49.106927 W | pkg/flags: unrecognized environment variable ETCD_URL=ETCD_URL_amd64
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-08-03 10:47:49.106982 I | etcdmain: etcd Version: 3.4.3
2021-08-03 10:47:49.106995 I | etcdmain: Git SHA: 3cf2f69b5
2021-08-03 10:47:49.107001 I | etcdmain: Go Version: go1.12.12
2021-08-03 10:47:49.107012 I | etcdmain: Go OS/Arch: linux/amd64
2021-08-03 10:47:49.107024 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2021-08-03 10:47:49.107141 N | etcdmain: the server is already initialized as member before, starting as etcd member...
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2021-08-03 10:47:49.107849 I | embed: name = default
2021-08-03 10:47:49.107880 I | embed: data dir = management-state/etcd
2021-08-03 10:47:49.107888 I | embed: member dir = management-state/etcd/member
2021-08-03 10:47:49.107895 I | embed: heartbeat = 500ms
2021-08-03 10:47:49.107902 I | embed: election = 5000ms
2021-08-03 10:47:49.107915 I | embed: snapshot count = 100000
2021-08-03 10:47:49.107977 I | embed: advertise client URLs = http://localhost:2379
2021-08-03 10:47:49.107995 I | embed: initial advertise peer URLs = http://localhost:2380
2021-08-03 10:47:49.108009 I | embed: initial cluster =
2021-08-03 10:47:49.114885 I | etcdserver: restarting member 8e9e05c52164694d in cluster cdf818194e3a8c32 at commit index 577
raft2021/08/03 10:47:49 INFO: 8e9e05c52164694d switched to configuration voters=()
raft2021/08/03 10:47:49 INFO: 8e9e05c52164694d became follower at term 3
raft2021/08/03 10:47:49 INFO: newRaft 8e9e05c52164694d [peers: [], term: 3, commit: 577, applied: 0, lastindex: 577, lastterm: 3]
2021-08-03 10:47:49.122161 W | auth: simple token is not cryptographically signed
2021-08-03 10:47:49.124712 I | etcdserver: starting server... [version: 3.4.3, cluster version: to_be_decided]
raft2021/08/03 10:47:49 INFO: 8e9e05c52164694d switched to configuration voters=(10276657743932975437)
2021-08-03 10:47:49.126010 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2021-08-03 10:47:49.126184 N | etcdserver/membership: set the initial cluster version to 3.4
2021-08-03 10:47:49.126290 I | etcdserver/api: enabled capabilities for version 3.4
2021-08-03 10:47:49.126844 I | embed: listening for peers on <IP_ADDRESS>:2380
raft2021/08/03 10:47:55 INFO: 8e9e05c52164694d is starting a new election at term 3
raft2021/08/03 10:47:55 INFO: 8e9e05c52164694d became candidate at term 4
raft2021/08/03 10:47:55 INFO: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 4
raft2021/08/03 10:47:55 INFO: 8e9e05c52164694d became leader at term 4
raft2021/08/03 10:47:55 INFO: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 4
2021-08-03 10:47:55.116855 I | etcdserver: published {Name:default ClientURLs:[http://localhost:2379]} to cluster cdf818194e3a8c32
2021-08-03 10:47:55.117072 I | embed: ready to serve client requests
2021-08-03 10:47:55.119245 N | embed: serving insecure client requests on <IP_ADDRESS>:2379, this is strongly discouraged!
2021/08/03 10:47:55 [INFO] Waiting for server to become available: Get "https://<IP_ADDRESS>:6443/version?timeout=15m0s": dial tcp <IP_ADDRESS>:6443: connect: connection refused
time="2021-08-03T10:47:55.404406940Z" level=info msg="Starting k3s v1.19.8+k3s1 (95fc76b2)"
time="2021-08-03T10:47:55.404989488Z" level=info msg="Cluster bootstrap already complete"
time="2021-08-03T10:47:55.435299095Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=unknown --authorization-mode=Node,RBAC --bind-address=<IP_ADDRESS> --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --enable-admission-plugins=NodeRestriction --etcd-servers=http://localhost:2379 --insecure-port=0 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=k3s --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range=<IP_ADDRESS>/16 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
I0803 10:47:55.438166      34 server.go:652] external host was not specified, using <IP_ADDRESS>
I0803 10:47:55.439186      34 server.go:177] Version: v1.19.8+k3s1
I0803 10:47:55.460009      34 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0803 10:47:55.460074      34 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
I0803 10:47:55.463217      34 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0803 10:47:55.463254      34 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
I0803 10:47:55.516041      34 master.go:271] Using reconciler: lease
W0803 10:47:55.926248      34 genericapiserver.go:412] Skipping API batch/v2alpha1 because it has no resources.
W0803 10:47:55.945512      34 genericapiserver.go:412] Skipping API discovery.k8s.io/v1alpha1 because it has no resources.
W0803 10:47:55.981580      34 genericapiserver.go:412] Skipping API node.k8s.io/v1alpha1 because it has no resources.
W0803 10:47:56.045832      34 genericapiserver.go:412] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
W0803 10:47:56.049188      34 genericapiserver.go:412] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
W0803 10:47:56.068143      34 genericapiserver.go:412] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
W0803 10:47:56.091699      34 genericapiserver.go:412] Skipping API apps/v1beta2 because it has no resources.
W0803 10:47:56.091734      34 genericapiserver.go:412] Skipping API apps/v1beta1 because it has no resources.
I0803 10:47:56.102792      34 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0803 10:47:56.102831      34 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
time="2021-08-03T10:47:56.117486702Z" level=info msg="Running kube-scheduler --address=<IP_ADDRESS> --bind-address=<IP_ADDRESS> --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --port=10251 --profiling=false --secure-port=0"
time="2021-08-03T10:47:56.117526487Z" level=info msg="Waiting for API server to become available"
I0803 10:47:56.117996      34 registry.go:173] Registering SelectorSpread plugin
I0803 10:47:56.118024      34 registry.go:173] Registering SelectorSpread plugin
time="2021-08-03T10:47:56.118586884Z" level=info msg="Running kube-controller-manager --address=<IP_ADDRESS> --allocate-node-cidrs=true --bind-address=<IP_ADDRESS> --cluster-cidr=<IP_ADDRESS>/16 --cluster-signing-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --port=10252 --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=0 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.key --use-service-account-credentials=true"
time="2021-08-03T10:47:56.121603426Z" level=info msg="Node token is available at /var/lib/rancher/k3s/server/token"
time="2021-08-03T10:47:56.121676173Z" level=info msg="To join node to cluster: k3s agent -s https://<IP_ADDRESS>:6443 -t ${NODE_TOKEN}"
time="2021-08-03T10:47:56.125076618Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
time="2021-08-03T10:47:56.125163251Z" level=info msg="Run: k3s kubectl"
time="2021-08-03T10:47:56.125534450Z" level=info msg="Module overlay was already loaded"
time="2021-08-03T10:47:56.125590165Z" level=info msg="Module nf_conntrack was already loaded"
time="2021-08-03T10:47:56.125623247Z" level=info msg="Module br_netfilter was already loaded"
time="2021-08-03T10:47:56.125662752Z" level=info msg="Module iptable_nat was already loaded"
time="2021-08-03T10:47:56.150691402Z" level=info msg="Cluster-Http-Server 2021/08/03 10:47:56 http: TLS handshake error from <IP_ADDRESS>:55380: remote error: tls: bad certificate"
time="2021-08-03T10:47:56.159468634Z" level=info msg="Cluster-Http-Server 2021/08/03 10:47:56 http: TLS handshake error from <IP_ADDRESS>:55386: remote error: tls: bad certificate"
time="2021-08-03T10:47:56.182242979Z" level=info msg="certificate CN=local-node signed by CN=k3s-server-ca@1627987647: notBefore=2021-08-03 10:47:27 +0000 UTC notAfter=2022-08-03 10:47:56 +0000 UTC"
time="2021-08-03T10:47:56.189805213Z" level=info msg="certificate CN=system:node:local-node,O=system:nodes signed by CN=k3s-client-ca@1627987647: notBefore=2021-08-03 10:47:27 +0000 UTC notAfter=2022-08-03 10:47:56 +0000 UTC"
time="2021-08-03T10:47:56.202788256Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
time="2021-08-03T10:47:56.203314847Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
W0803 10:47:57.183700      11 warnings.go:80] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
time="2021-08-03T10:47:57.205871940Z" level=info msg="Containerd is now running"
time="2021-08-03T10:47:57.218505485Z" level=info msg="Connecting to proxy" url="wss://<IP_ADDRESS>:6443/v1-k3s/connect"
time="2021-08-03T10:47:57.230260755Z" level=info msg="Handling backend connection request [local-node]"
time="2021-08-03T10:47:57.230981283Z" level=warning msg="Disabling CPU quotas due to missing cpu.cfs_period_us"
time="2021-08-03T10:47:57.231143909Z" level=info msg="Running kubelet --address=<IP_ADDRESS> --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=<IP_ADDRESS> --cluster-domain=cluster.local --cni-bin-dir=/var/lib/rancher/k3s/data/ab1b521f47b3cbc273e0c667521084cff1ea3540d55b425ac7ed30c27daf92f8/bin --cni-conf-dir=/var/lib/rancher/k3s/agent/etc/cni/net.d --container-runtime-endpoint=/run/k3s/containerd/containerd.sock --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --cpu-cfs-quota=false --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=<IP_ADDRESS> --hostname-override=local-node --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --kubelet-cgroups=/k3s --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/etc/resolv.conf --runtime-cgroups=/k3s --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
time="2021-08-03T10:47:57.232737862Z" level=info msg="Running kube-proxy --cluster-cidr=<IP_ADDRESS>/16 --healthz-bind-address=<IP_ADDRESS> --hostname-override=local-node --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables"
Flag --cloud-provider has been deprecated, will be removed in 1.23, in favor of removing cloud provider code from Kubelet.
Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
I0803 10:47:57.238111      34 server.go:407] Version: v1.19.8+k3s1
W0803 10:47:57.239400      34 server.go:226] WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP.
W0803 10:47:57.240983      11 warnings.go:80] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0803 10:47:57.244216      34 proxier.go:639] Failed to read file /lib/modules/5.4.0-80-generic/modules.builtin with error open /lib/modules/5.4.0-80-generic/modules.builtin: no such file or directory. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
W0803 10:47:57.245847      34 proxier.go:649] Failed to load kernel module ip_vs with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
W0803 10:47:57.247811      34 proxier.go:649] Failed to load kernel module ip_vs_rr with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
W0803 10:47:57.250566      34 proxier.go:649] Failed to load kernel module ip_vs_wrr with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
W0803 10:47:57.253242      34 proxier.go:649] Failed to load kernel module ip_vs_sh with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
W0803 10:47:57.255003      34 proxier.go:649] Failed to load kernel module nf_conntrack with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
time="2021-08-03T10:47:57.275604610Z" level=info msg="Waiting for node local-node CIDR not assigned yet"
E0803 10:47:57.290613      34 node.go:125] Failed to retrieve node info: nodes "local-node" is forbidden: User "system:kube-proxy" cannot get resource "nodes" in API group "" at the cluster scope
W0803 10:47:57.300698      11 warnings.go:80] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
I0803 10:47:57.311211      34 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt
I0803 10:47:58.368243      34 dynamic_cafile_content.go:167] Starting request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt
I0803 10:47:58.368488      34 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt
I0803 10:47:58.368657      34 dynamic_serving_content.go:130] Starting serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key
I0803 10:47:58.369515      34 secure_serving.go:197] Serving securely on <IP_ADDRESS>:6444
I0803 10:47:58.369583      34 apiservice_controller.go:97] Starting APIServiceRegistrationController
I0803 10:47:58.369593      34 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
I0803 10:47:58.369612      34 tlsconfig.go:240] Starting DynamicServingCertificateController
I0803 10:47:58.370321      34 available_controller.go:475] Starting AvailableConditionController
I0803 10:47:58.370332      34 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
I0803 10:47:58.370367      34 autoregister_controller.go:141] Starting autoregister controller
I0803 10:47:58.370371      34 cache.go:32] Waiting for caches to sync for autoregister controller
I0803 10:47:58.371163      34 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
I0803 10:47:58.371174      34 shared_informer.go:240] Waiting for caches to sync for cluster_authentication_trust_controller
I0803 10:47:58.371256      34 controller.go:83] Starting OpenAPI AggregationController
I0803 10:47:58.371648      34 customresource_discovery_controller.go:209] Starting DiscoveryController
I0803 10:47:58.371959      34 crdregistration_controller.go:111] Starting crd-autoregister controller
I0803 10:47:58.371968      34 shared_informer.go:240] Waiting for caches to sync for crd-autoregister
I0803 10:47:58.371995      34 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt
I0803 10:47:58.372108      34 dynamic_cafile_content.go:167] Starting request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt
I0803 10:47:58.372987      34 dynamic_serving_content.go:130] Starting aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key
I0803 10:47:58.374442      34 controller.go:86] Starting OpenAPI controller
I0803 10:47:58.374841      34 naming_controller.go:291] Starting NamingConditionController
I0803 10:47:58.375297      34 establishing_controller.go:76] Starting EstablishingController
I0803 10:47:58.375748      34 nonstructuralschema_controller.go:186] Starting NonStructuralSchemaConditionController
I0803 10:47:58.376140      34 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
I0803 10:47:58.376542      34 crd_finalizer.go:266] Starting CRDFinalizer
time="2021-08-03T10:47:58.419755275Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --allow-untagged-cloud=true --bind-address=<IP_ADDRESS> --cloud-provider=k3s --cluster-cidr=<IP_ADDRESS>/16 --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --node-status-update-frequency=1m --profiling=false --secure-port=0"
Flag --allow-untagged-cloud has been deprecated, This flag is deprecated and will be removed in a future release. A cluster-id will be required on cloud instances.
I0803 10:47:58.423592      34 controllermanager.go:127] Version: v1.19.8+k3s1
W0803 10:47:58.423633      34 controllermanager.go:139] detected a cluster without a ClusterID.  A ClusterID will be required in the future.  Please tag your cluster to avoid any future issues
I0803 10:47:58.423731      34 leaderelection.go:243] attempting to acquire leader lease  kube-system/cloud-controller-manager...
E0803 10:47:58.426840      34 leaderelection.go:325] error retrieving resource lock kube-system/cloud-controller-manager: endpoints "cloud-controller-manager" is forbidden: User "cloud-controller-manager" cannot get resource "endpoints" in API group "" in the namespace "kube-system": RBAC: [clusterrole.rbac.authorization.k8s.io "cloud-controller-manager" not found, clusterrole.rbac.authorization.k8s.io "system:public-info-viewer" not found, clusterrole.rbac.authorization.k8s.io "system:basic-user" not found, clusterrole.rbac.authorization.k8s.io "system:discovery" not found]
E0803 10:47:58.439901      34 controller.go:156] Unable to remove old endpoints from kubernetes service: no master IPs were listed in storage, refusing to erase all endpoints for the kubernetes service
I0803 10:47:58.449587      34 node.go:136] Successfully retrieved node IP: <IP_ADDRESS>
I0803 10:47:58.449615      34 server_others.go:143] kube-proxy node IP is an IPv4 address (<IP_ADDRESS>), assume IPv4 operation
I0803 10:47:58.450447      34 server_others.go:186] Using iptables Proxier.
I0803 10:47:58.450606      34 server.go:650] Version: v1.19.8+k3s1
I0803 10:47:58.450909      34 conntrack.go:103] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072
F0803 10:47:58.450942      34 server.go:495] open /proc/sys/net/netfilter/nf_conntrack_max: permission denied
2021/08/03 10:47:58 [FATAL] k3s exited with: exit status 1

Inside the KVM itself I am able to change the nf_conntrack_max value:
root@pgk8s:~# sysctl net/netfilter/nf_conntrack_max
net.netfilter.nf_conntrack_max = 262144

root@pgk8s:~# sysctl -w net/netfilter/nf_conntrack_max=131072
net.netfilter.nf_conntrack_max = 131072

root@pgk8s:~# sysctl net/netfilter/nf_conntrack_max
net.netfilter.nf_conntrack_max = 131072

The fact that a container (rancher) wants to change nf_conntrack_max interferes with the Linux Kernel's non net-init policy - hence permission denied. So the bug report actually also affects installations other than inside LXC/LXD environments.
@sowmyav27 You mentioned this:

It has been fixed in k3s version: v1.21.3+k3s1 Related PR - k3s-io/k3s#3341 and this version is embedded in the rancher version that was tested on .

Which Rancher version was this tested on?

@Napsty , the testing was done with master-head version which is basically the latest in-development version and it doesn't have a formal version number like 2.5.9 (see https://hub.docker.com/r/rancher/rancher/tags?page=1&ordering=last_updated). This fix is slated to go into the next release (2.6.0) and also backported into next 2.5.x release (2.5.10) per https://github.com/rancher/rancher/issues/33762.
Hope this helps.

@snasovich Thanks, yes this helps. So will keep my hopes up for 2.5.10 :-)

Did some additional testing and I was unable to install Rancher 2.5 Single Install in a Ubuntu 18.04 and 20.04 KVM (not LXC), due to the nf_conntrack_max permission denied error. Tested with both 2.5.9 and 2.5.8 - same error. According to the support matrix (https://rancher.com/support-maintenance-terms/all-supported-versions/rancher-v2.5.9/) this should work.
Installation of Rancher 2.4.16 works.
For the sake of other people trying to install 2.5.x and failing, please re-open this ticket as the bug is as of the current available releases still existing. Close it when the fix was released and is available. It's just confusing right now. Thanks in advance.

Close it when the fix was released and is available. It's just confusing right now

Hi @Napsty. Are you suggesting that we keep the issue open to track the k8s upstream fix?

Hi @nickgerace
No, only keep the issue open until the Rancher 2.5.x release with the fix (for Single Install) is out .
As the issue title mentions and what the issue initially was filed for, was concerning the 2.5.x Single Install (on Docker).
The k8s upstream fix is a wider issue, as mentioned by @briandowns in https://github.com/kubernetes/kubernetes/pull/103174#issuecomment-890100905. Not sure if it's helpful to keep this Rancher issue open to track the k8s upstream. Doesn't seem to be moving really there (unfortunately).
I'm open for other suggestions of course.

@Napsty , thank you for the additional information. Regarding leaving the issue open, our current process is to close the issues as soon as they are validated by QA on master-head image (for latest upcoming release) or related 2.x - head image (for backports to 2.x releases). Milestone value can then be used to determine where the fix is/will be available.
Note the issue to get this into 2.5.10 is https://github.com/rancher/rancher/issues/33762

@snasovich that's fine, but can you please update the milestone of this issue to 2.5.10? right now it's v2.6.

@Napsty , https://github.com/rancher/rancher/issues/33762 is backport of this issue that targets milestone 2.5.10.

Why is this closed? I just installed rancher:latest and it's broken.

@lindhe , at this time rancher:latest tag is the same as v2.5.9 release and it doesn't have this issue. This will be fixed in the upcoming v2.6.0 release. It is also planned to be backported to next v2.5.x release (v2.5.10) as per separate backport issue https://github.com/rancher/rancher/issues/33762.
Please also note our current process is to close issues once they are validated by QA in internal/RC builds as per this comment - https://github.com/rancher/rancher/issues/33360#issuecomment-892777075.
Hope this helps with clearing things out.


Create ACI with new VNET and new subnet may fail with invalid subnet range

Describe the bug
In a brand new resource group, run:
az container create -g demo1 -n australia-vnet --image yangl/netutils --vnet acivnet --subnet aci --vnet-address-prefix <IP_ADDRESS>/24 --ip-address Private -l australiaeast
the CLI command assumed the subnet range to <IP_ADDRESS>/24, and it won’t be accepted by NRP since the vnet has <IP_ADDRESS>/24 ip range. It seems the command should require --subnet-address-prefix if --vnet-address-prefix is specified.
To Reproduce
In a brand new resource group, run:
az container create -g demo1 -n australia-vnet --image yangl/netutils --vnet acivnet --subnet aci --vnet-address-prefix <IP_ADDRESS>/24 --ip-address Private -l australiaeast
Expected behavior
Fail with validation, since we cannot guess subnet range.
Environment summary
Cloud Shell
Additional context
Add any other context about the problem here.

@samkreter


Fix network metric name constants

https://jira.mesosphere.com/browse/DCOS-14186

Ha, great catch. Let me fix those.


Use a consistent obo-namespace tag in obo format exports

Originally submitted by @ukemi on https://github.com/OBOFoundry/OBOFoundry.github.io/issues/101

WARNING: This issue has been automatically closed because it has not been updated in more than 3 years. Please re-open it if you still need this to be addressed – we are now getting some resources to deal with such issues.


Shift-Up/Down scrolls Cygwin window.

Shift-Up/Down scrolls terminal window instead of selecting text.
Environment: Cygwin terminal + ssh + screen.
Workaround with Ctrl/Alt+Shift+Up/Down inserts character instead of selecting text.
According to error message on Ctrl/Alt+Shift+Up/Down, Enter:
SyntaxError: found "\u001b[1;6A", expected TopPkgSeq | Import | Prelude ~ BlockDef | Expr | End at index 0

Ctrl+Shift+Up - "\u001b[1;6A"
Ctrl+Shift+Down - "\u001b[1;6B"
Alt+Shift+Up - "\u001b[1;4A"
Alt+Shift+Down - "\u001b[1;4B"

This and other cygwin related bugs should probably be reported under https://github.com/lihaoyi/Ammonite/issues/119 just to keep all the windows related stuff in one place


It keeps pulling data again and again from input source

Hi,
I am using kafka output plugin with elasticsearch and input plugin.
Goal is to index every document pushed in the ES to kafka.
My config file.
input {
        elasticsearch {
                hosts => ["localhost:9200"]
                index => "liferay-20116"
        }
}
output {
      kafka {
        topic_id => "elastic-topic-index"
        codec => json
      }
}

Now once I restart logstash and put a consumer to the topic, it reads data from ES. That is working.
But after reading the content from topic, it waits for a while and then re-read everything again! Which is weird to me.
Please help, as this is bit urgent.

the elasticsearch input does not perform a tail on index, every time it executes it will perform a query and dump results to the outputs. Until elasticsearch implements the changes API this will be how the plugin works.

closing this since it is not related to the kafka output.


pytest-xdist slave fail on Interrupted system call

Hi,
I get IOError Interrupted system call in pytest-xdist slave.  Don't know why, can't see signal anywhere but fail
creating slavegateway on <__main__.Popen2IO instance at 0x20fa2d8>
RECEIVERTHREAD: starting to run
received <Message.CHANNEL_EXEC channelid=1 len=285>
received <Message.CHANNEL_DATA channelid=1 len=123>
execution starts[1]: "\nimport os\npath, nice, env = channel.receive()\
execution finished
sent <Message.CHANNEL_CLOSE channelid=1 >
1 sent channel close message
received <Message.CHANNEL_EXEC channelid=3 len=210>
1 channel.__del__
execution starts[3]: '\nimport sys, os\nchannel.send(dict(\n    executa
sent <Message.CHANNEL_DATA channelid=3 len=295>
execution finished
sent <Message.CHANNEL_CLOSE channelid=3 >
3 sent channel close message
received <Message.CHANNEL_EXEC channelid=5 len=5308>
3 channel.__del__
execution starts[5]: '"""\n    This module is executed in remote subpro
received <Message.CHANNEL_DATA channelid=5 len=1492>
RECEIVERTHREAD Traceback (most recent call last):
 File "<string>", line 622, in _thread_receiver
 File "<string>", line 802, in load
 File "<string>", line 86, in read
IOError: [Errno 4] Interrupted system call

RECEIVERTHREAD entering finalization
RECEIVERTHREAD leaving finalization
sent <Message.CHANNEL_DATA channelid=5 len=451>
sent <Message.CHANNEL_DATA channelid=5 ('collectionstart', {})>
sent <Message.CHANNEL_DATA channelid=5 len=115>
sent <Message.CHANNEL_DATA channelid=5 len=163>
sent <Message.CHANNEL_DATA channelid=5 len=244>
sent <Message.CHANNEL_DATA channelid=5 len=53>

gateway_base.Unserializer.load(self) can be made work by retry if get IOError with errno == 4
while True:
    opcode = self.stream.read(1)  # Retry this if IOError raised here
    if not opcode:
        raise EOFError

Bitbucket: https://bitbucket.org/hpk42/execnet/issue/6
Originally reported by: memedough
Originally created at: 2012-04-14T03:00:30.080

Original comment by: Ronny Pfannschmidt