Questions regarding season search & 1337x

@elgatito
Hi,
When I'm trying for example to make a search for "Black Mirror S04S02" with 1337x
It gives me only results from:
https://www.1337x.to/search/black+mirror+s04e02/1/
But when I ran a manual search in the site for season I get additional results (The torrents doesn't work, but the Magnet link does work)
https://www.1337x.to/search/black+mirror+s04/1/
So my questions is:

Why I don't get also the season results in additional to the episodes results?
Does it because the torrents links are broken? (Because the Magnet link is working)
Does it should search also season results in addtion or only at specific conditions?
Does burst return also Magnet results in additional to Torrents results?

Kodi debug log:
https://paste.ubuntu.com/p/7PWbFQ98JZ
raw: https://paste.ubuntu.com/p/7PWbFQ98JZ/plain/

Burst 0.0.70
Elementum 0.1.86
Kodi 19.3

hi.
just as into:
1st search is for specific episode so it gives you specific results.
2nd search is for season so it gives you a lot of results but vast majority of then are irrelevant (for different episodes).
With that in mind:

Why I don't get also the season results in additional to the episodes results?

because search for season gives you a lot of results but vast majority of then are irrelevant (for different episodes).

Does it because the torrents links are broken? (Because the Magnet link is working)

No. See answer above. But just as side note: torrents (or magnets, same in this context) can be filtered depending on your settings (so seeds, wrong screen resolution, type of rip, size, etc) - see your settings to learn more.

Does it should search also season results in addtion or only at specific conditions?

Depends on configuration of provider, but usually only when:

you search for season (you can click on season and search a torrent for it)
if nothing was found for episode and provider has "fallback" query (that query usually uses season search).

Does burst return also Magnet results in additional to Torrents results?

Depends on configuration of provider - it is either torrent file or magnet (for 1337x it is magnet iirc).

You can see technical search details about 1337x here - https://github.com/elgatito/script.elementum.burst/blob/f66d4dd1cd4da968f8e2233e0189b6920be9bacf/burst/providers/providers.json#L69
e.g. fallback search for episode (when nothing is found) - is to search for season - https://github.com/elgatito/script.elementum.burst/blob/f66d4dd1cd4da968f8e2233e0189b6920be9bacf/burst/providers/providers.json#L94

Thanks. @antonsoroko
But I had a lot of cases that searching for a specific episode still returns a season reasults which contains the specific episode in additoin to others episode results for the same provider. So I don't think like you said in (2) that if nothing was found for episode or standing on a season and choose streaming from the context menu.
By the way even when I tried to make a season search I still don't get this specific result that I found manually, why is that?
This is the missing result I found by myself:
https://www.1337xxx.to/torrent/2696524/Black-Mirror-S04-COMPLETE-INTERNAL-1080p-WEB-x264-STRiFE/

But I had a lot of cases that searching for a specific episode still returns a season reasults which contains the specific episode in additoin to others episode results for the same provider.

as i said -  depends how provider is configured. some might have 2 queries (for episode and season at the same time), but most have 1 query for episode and then fallback for season. See providers.json for details.

So I don't think like you said in (2) that if nothing was found for episode or standing on a season and choose streaming from the context menu.

I do not understand what you wanted to tell. Anyway - what i wrote is based on burst code and providers.json code.

By the way even when I tried to make a season search I still don't get this specific result that I found manually, why is that?
As i said - maybe it was filtered out by your settings. See log files for details.
Also iirc elementum will actually try to resolve number of seeds - so if site says 10 but in reality it is zero - it will be filtered out. You can enable "zero seed" torrents in settings and check if this is the case (also it should be reflected in logs iirc).

I will open a PR.

please don't.
right now behaviors is correct like i described above:

because search for season gives you a lot of results but vast majority of then are irrelevant (for different episodes).

you can make local override if you want such behavior - see https://github.com/elgatito/script.elementum.burst#adding--editing-providers

Why not to update all the providers with their "tv_keywords2" to support season search in addition to the specific episode?

let me answer again :-)

because search for season gives you a lot of results but vast majority of then are irrelevant (for different episodes).

so navigating a lot of irrelevant result will make you mad after some time.

Why some of the providers their "tv_keywords" is empty?

maybe they are: anime only? movie only? (or maybe person who added them was lazy :-) )

log: https://paste.ubuntu.com/p/4qHkDhRSVK

open link from log
2022-01-24 18:53:30.111 T:23164 WARNING <general>: [script.elementum.burst] [1337x] >                1337x search URL: https://www.1337x.to/search/black+mirror+s04/1/
also see here
https://github.com/elgatito/script.elementum.burst/blob/f66d4dd1cd4da968f8e2233e0189b6920be9bacf/burst/providers/providers.json#L55
it uses only first page.
your torrent is on second page.

So I summarize this issue with another idea, a little fix.
Lets take an example for this case: Black Mirror S04E02

Confiduration: 720p and 1080p is acceptable results.
1337x has only 4 results for this episode, non of them is 1080p.
Because of that the fallback (season query) isn't called.
In addition if I run a season search (By standing on the season and selecting "Choose stream.." from the context menu)
I don't get the mentioned result since it is on page 2 and the provider is set to get results only from page 1.

My advice is to change from the current provider's configuration is:
        "base_url": "https://www.1337x.to/search/QUERYEXTRA/1/",
        "season_keywords": "{title} s{season:2}",
        "season_keywords2": "{title} season {season}",
        "season_keywords_fallback": "{title} {season:2}x01",
        "tv_keywords": "{title} s{season:2}e{episode:2}",
        "tv_keywords2": null,
        "tv_keywords_fallback": "{title} s{season:2}",
        "tv_keywords_fallback2": "{title} season {season}",
        "tv_keywords_fallback3": "{title} {season:2}x01"

To this one:
        "base_url": "https://www.1337x.to/search/QUERYEXTRA/",
        "season_keywords": "{title} s{season:2}/1",
        "season_keywords2": "{title} s{season:2}/2",
        "season_keywords_fallback": "{title} season {season:2}",
        "tv_keywords": "{title} s{season:2}e{episode:2}/1",
        "tv_keywords2": "{title} s{season:2}e{episode:2}/2",
        "tv_keywords_fallback": "{title} s{season:2}/1",
        "tv_keywords_fallback2": "{title} season {season}/1",
        "tv_keywords_fallback3": "{title} {season:2}x01/1"

The mainly change is to move the 'page' query to the keywords,
Regarding my case that now the season results will be returned with more results from both pages.
And also the episode will have now option to look for page 2 if it has.
Also made some fixes with the queries.

heh, interesting hack :-)
just a reminder - you would also need to change queries for anime and movies since you changed base_url and now they will not work w/o extra /N/.
btw there are up to 5 queries per type (e.g. from tv_keywords up to tv_keywords4).
also it might be more flexible to use _extra keyword, for example
https://github.com/elgatito/script.elementum.burst/blob/master/burst/providers/providers.json#L1692-L1694
https://github.com/elgatito/script.elementum.burst/blob/master/burst/providers/providers.json#L2956
so you will make "base_url": "https://www.1337x.to/search/QUERY/EXTRA/", (note / between QUERY and EXTRA) and then set appropriate _extra to 1/2/3/4/5.

also note: all _keywords* executed at the same time so you will create more load to tracker (although it also depend of how many elementum's users uses 1337x), and also not every search will have multiple pages to it will generates some errors in log. as usual in life - evaluate tradeoffs to find "acceptable" solution.

@antonsoroko

just a reminder - you would also need to change queries for anime and movies since you changed base_url and now they will not work w/o extra /N/.

You're right, I will fix it.

btw there are up to 5 queries per type (e.g. from tv_keywords up to tv_keywords4).

I think 2 pages is enough :)
And regarding you note. This small error won't effect so much (again 2 page is enough. so it's a one error)

so you will make "base_url": "https://www.1337x.to/search/QUERY/EXTRA/", (note / between QUERY and EXTRA) and then set appropriate _extra to 1/2/3/4/5.

What is the difference between 'QUERYEXTRA' to 'QUERY/EXTRA'?
I thought it's a one argement. Does it actually two argments? What is the QUERY and what is the EXTRA?
But I think I will keep it as I already did.

Does it actually two argments?

yes.

What is the QUERY and what is the EXTRA?

this is a bit confusing :-) for example, season_keywords is QUERY and season_extra is EXTRA.
then in base_url (or in season_query - which might be confusing because of its name) burst will replace replace QUERY and EXTRA with appropriate _query and _extra for specific type of search (movie, episode, etc).
So you can use QUERY and EXTRA not only in base_url but also in _query, like in my examples.
I used EXTRA to implement search in multiple categories in rutor (e.g. local and foreign shows have different categories), same for torrent.by.
So it is also possible to implement "pages" hack with the same logic.
Since _extra is rarely used you can see a lot of QUERYEXTRA - so basically since EXTRA is empty it becomes just  QUERY.

you moved
        "season_keywords2": "{title} season {season}",
to
        "season_keywords_fallback": "{title} season {season:2}/1",
i would test first how many results this query gives - maybe we will lose some good results b/c of this change.

also JFYI - all _keywords* are run at the same time, but _keywords_fallback* are run 1 by 1 until found something (so most likely not all of them will be run).

Can someone post updated 1337x provider configuration?
Thanks :)

@kovacicjanez
This is my configuration for now, you can use it:
Copy and replace.
(If I will change it in the future I will update here)
    "1337x": {
        "anime_extra": "",
        "anime_keywords": "{title} s{season:2}e{episode:2}/1",
        "anime_keywords_fallback": "{title} s{season:2}/1",
        "anime_keywords_fallback2": "{title} season {season}/1",
        "anime_keywords_fallback3": "{title} {season:2}x01/1",
        "anime_query": "",
        "base_url": "https://www.1337x.to/search/QUERYEXTRA/",
        "color": "FFF14E13",
        "enabled": true,
        "general_extra": "",
        "general_keywords": "{title}",
        "general_query": "",
        "language": null,
        "languages": "en",
        "login_failed": "",
        "login_object": "",
        "login_path": null,
        "movie_extra": "",
        "movie_keywords": "{title} {year}/1",
        "movie_query": "",
        "name": "1337x",
        "parser": {
            "infohash": "",
            "name": "item('a', order=2)",
            "peers": "item(tag='td', order=3)",
            "row": "find_once(tag='tbody').find_all('tr')",
            "seeds": "item(tag='td', order=2)",
            "size": "item(tag='td', order=5)",
            "torrent": "'https://www.1337x.to%s' % item(tag='a', attribute='href', order=2)"
        },
        "predefined": true,
        "private": false,
        "season_extra": "",
        "season_extra2": "",
        "season_keywords": "{title} s{season:2}/1",
        "season_keywords2": "{title} s{season:2}/2",
        "season_keywords_fallback": "{title} season {season:2}/1",
        "season_query": "",
        "separator": "+",
        "show_query": "",
        "subpage": true,
        "tv_extra": "",
        "tv_extra2": "",
        "tv_keywords": "{title} s{season:2}e{episode:2}/1",
        "tv_keywords2": "{title} s{season:2}e{episode:2}/2",
        "tv_keywords_fallback": "{title} s{season:2}/1",
        "tv_keywords_fallback2": "{title} season {season}/1",
        "tv_keywords_fallback3": "{title} {season:2}x01/1"
    },

Been a long time, and I'm still never see any correct search results from 1337x, it always looks like spam.
Sample 1: And Just Like that s02e06
https://i.imgur.com/0SWGoFa.png
Sample 2: Star Trek s02e06
https://i.imgur.com/caOtdgu.png
@burekas7 Did you manage to overcome this specific "spam" issue?

@elgatito Sooo simple, bingo! I change the interface to English and... Boom! All 1337x results are now correct.
So embarrassing, didn't think of this almost 2 years. 🙈

@SubEdict  for reference https://github.com/elgatito/script.elementum.burst/commit/8799e2eaebdb34b3ae03dd8fc0893e22f0767587

Although I never had this issue because I used this option:
https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/resources/language/messages.pot#L55

Yep, yep, great! all good now. I changed back to my default interface language and 1337x finally works as it should.
Thanks again @elgatito, @antonsoroko for kipping everything up and running.
BTW, why would you use "Use language preference...." for? Mine is turned off (default) and all links on the web are English anyway.

Oh, I meant that I do not use that option. But since it is disabled by default - it just means that I do not remember how things were, and I did have issues with 1337 when i searched something non-english.
About what it does:
https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/burst/filtering.py#L463-L465
here is some bad-ass logic about what to use as title for search - language defined in tracker (providers.json) or language of kodi or original language of video.

The thing that confuse me was why in the first place if provider not in self.language_exceptions ...I mean all my providers are English, and movies/shows are also English. So why search algorithm care about GUI language preference? I always thought GUI translation is just visual, and search commands look for the original ID_English_Title-Number (like IMDB, tvdb, Trakt, etc.).
I am a simple man I don't like complications
Those 3 lines of code make me think that for the best result maybe it's better to use English after all. 🤔

@SubEdict

I mean all my providers are English, and movies/shows are also English. So why search algorithm care about GUI language preference?

Hah, you just extrapolated your preferences to everybody else :-D
I use my language as GUI and I watch sometimes TV shows and movies in my language and in other languages (with subtitles). That's why burst also uses original title, which is not english for those cases. And with english title search will fail in many cases, because releases often use original language.
language_exceptions meaning is
https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/resources/language/messages.pot#L95
so you can search all trackers in your language (maybe to find releases with dubbing audio) except those trackers that do not understand/support other languages.
I guess this is just a "legacy" idea from original code. You can use "git blame" feature in github UI to see history of any line of code, like:
https://github.com/elgatito/script.elementum.burst/commit/93bbc867c45efa26ee0477b2d0ce9522395a6e2b
https://github.com/elgatito/script.elementum.burst/commit/04ceee429d68f5dcb41964c301abacdf38bfb7c4
this is 6.5 years old code, probably back then it was used to fix some issues with local (per-country) trackers.
Anyway, we also can specify language(s) in tracker definition (json) - if this information is correct then you will proper results for queries in your language and will not get garbage results.
But not all of them have language(s) defined, which also can be a reason why this option exists.
Also, language is used to prepare correct search query, like:
https://github.com/elgatito/script.elementum.burst/blob/8799e2eaebdb34b3ae03dd8fc0893e22f0767587/burst/filtering.py#L478
So, now you know that searching is more complicated than it sounds :-)


fix: return a single chat message instead of an empty list when template has no {% chat %} tags

When no {% chat %} tag is present in the template, Prompt.chat_messages() returns an empty list.
With this PR instead, we return a single user message containing the rendered template for convenience.

coverage: 91.416% (+0.06%) from 91.361%
when pulling 3f36713d8b856f8ce360a821b3d0214fbffb9058 on massi/chat
into 7811818ca1333430da81983e796f6f4ae95ef998 on main.


Add MCU and MCU Interfacing

This pull request adds

Microcontroller-side code (mcu-code/mcu-code.ino)
Microcontroller interfacing library (mcu-lib/mcu.py)
a simple CLI for testing the microcontroller and its library (mcu-lib/mcu_cli.py)

You also appear to have included a binary file  mcu-lib/__pycache__/packets.cpython-39.pyc. Is this intentional?

I will work on the changes in the morning.

I don't expect any more significant changes to this pull request, so to speed up development, I'll merge it into main so @arora767287 can start work on comms.

Wait on line 168, I'm probably not interpreting it the right way but if what is not packet?

Wait on line 168, I'm probably not interpreting it the right way but if what is not packet?

End the function early.


kubectl apply artifacts/operator-amd64.yaml

Expected Behaviour

It should deploy the operator
Current Behaviour

The deployment never gets ready, CrashLoopBack on pod
Possible Solution

Modify ingress-operator/artifacts/operator-amd64.yaml
command:
- ./ingress-operator
- -logtostderr
- -v=2
remove --v=2
After that it worked for me.
Steps to Reproduce (for bugs)

Context

Your Environment

Docker version docker version (e.g. Docker 17.0.05 ):

What version and distriubtion of Kubernetes are you using? kubectl version

1.6

Operating System and version (e.g. Linux, Windows, MacOS):

Link to your project or a code example to reproduce issue:

What network driver are you using and what CIDR? i.e. Weave net / Flannel

Fixed by: 0f12b7b


Test support in Visual Studio 2012

Someone should verify that this extension works properly in Visual Studio 2012, especially in regards to the use of the Just My Code image moniker.

Does the command show up in the right place?
Does the command have the proper icon?
Does the command function correctly?

It looks like Visual Studio 2012 doesn't support these icons at all.


Remove unused header

Header unused and was causing problems for clang compilers as it was an extension specific to GNU compilers.

This should go into stable, let's me rebase.


TEN-55 added view for associated accounts

Closing as this page is no longer going to be in the flow


CardField handle validation errors

Is your feature request related to a problem? Please describe.
I'm using CardField and createToken method in my application. Everything is working perfectly fine.
But I want to do customizations with CardField, and I'm unable to do that.
CardField is handling all the validations which are really great, but I also want to display errors to users, that which specific fields have issues.
I was able to handle the error message for ExpiryDate, as shown in the below screenshots.

How I handled this
CardField has a method on onCardChange, in which I have access to expiryMonth and expiryYear. I can validate it, and show errors to the user. But I can't do it for the card number. Because when the card number is empty or is not valid, onCardChange gives null for last4 digits of the card. Same for CVC.
Describe the solution you'd like
CardField has a method on onCardChange, which gives cardDetails. Addition of some bool attributes like validCard, validExpiryDate, and validCVC, so I can show & hide errors at my end. Just like we have complete in cardDetails, which is false when any of the fields are invalid, and true when all the fields are valid.
Describe alternatives you've considered
N/A
Additional context
N/A

Please use expo install @stripe/stripe-react-native to install the correct version for your Expo SDK version: https://github.com/stripe/stripe-react-native#expo


Revised weather code metadata changes

Reference issue #294
Testing:

[X] Ran tests and they passed OK
[X] Added new tests for the new feature(s)

Let's see if this updates.


Add post announcing 4.161

Looks good! Don't see anything I would add in particular aside from checking that the new API pages are sufficiently descriptive.


Failing test

This commit should pass tests as far as I can tell, but RangeTests fails.
/Users/dave/src/hylo/Tests/LibraryTests/TestCases/RangeTests.hylo:1: error: -[LibraryTests.LibraryTests test_compileAndRun_RangeTests] : success was expected, but processing failed with thrown error: NonzeroExit(
  terminationStatus: 6,
  standardOutput: "/Users/dave/src/hylo/Tests/LibraryTests/TestCases/RangeTests.hylo:13: precondition failure\n",
  standardError: "",
  commandLine: ["/var/folders/tt/443nx5hn3djbkn_vcx1nl_900000gq/T/94F55AEA-5A93-40B1-A461-A919524C01CE"])

The problem is that the wrong method of Range gets called. It disappears if we comment out the version that accepts a bound and not a range, or if we add an argument label to either function.
The bug is probably about name mangling or name resolution. Since I've seen another bug related to the latter today, I suspect something fishy might be going on with overloading.

The problem is due to an interesting set of circumstances.
The current implementation of the type checker ignores the constraints defined in the where clause when:

It is type checking a call whose parameter should satisfy this constraints: (e.g., f(1) where the argument of f has a constraint)
It is comparing two solutions for specificity to disambiguate an overloaded call.

In this particular case, we have the following declaration of Range:
/// A half-open interval from a lower bound up to, but not including, an uppor bound.
public type Range<Bound: SemiRegular> {

  ...

  /// Creates a half-open interval [`lower_bound`, `upper_bound`).
  ///
  /// - Requires: `lower_bound <= upper_bound`.
  public init(from lower_bound: sink Bound, up_to upper_bound: sink Bound) {
    // precondition(lower_bound <= upper_bound)
    &self.lower_bound = lower_bound
    &self.upper_bound = upper_bound
  }

  /// Returns `true` iff `x` is contained in `self`.
  public fun contains<B: Comparable where Bound == B>(_ x: B) -> Bool {
    let e = Pointer<Bound>(type_punning: pointer[to: x])
    return (e.unsafe[] >= lower_bound) && (e.unsafe[] < upper_bound)
  }

  /// Returns `true` iff `other` is contained in `self`.
  public fun contains<B: Comparable where Bound == B>(_ other: Range<B>) -> Bool {
    let e = Pointer<Self>(type_punning: pointer[to: other])
    return (e.unsafe[].lower_bound >= self.lower_bound) && (e.unsafe[].upper_bound <= self.upper_bound)
  }

}

When the checker examines r0.contains(5 ..< 15), it concludes that it can be dispatched to the first overload because, absent of the constraints in the where clause, it is "more specific" than the second one.
I recommend we add a label to the second declaration (e.g., contains(range:)) until constraints in where clauses of functions are properly handled at call sites.

And if anyone is wondering by type inference chooses the first method over the second, I think it's because of yet another bug https://github.com/hylo-lang/hylo/issues/1149#issue-2000937094


[ModSecurity 3.x] Json output log, missing information?

Describe the bug
When reviewing the output from a blocked request (intentionally) I am missing several pieces of information which are in the older modsecurity version (2.x).
The parts missing:

timings (duration, phase1 to phase5, garbage collection timing, storage read / write etc).
the action taken: I expect somewhere 'Access denied with code ..' or 'Warning.'

After further reviewing the error log reveals that information but I would like if this information could be integrated in the json output.
Logs and dumps
Output of:

DebugLogs (level 9)
Not relevant

AuditLogs

{
    "transaction":
    {
        "client_ip": "<ip>",
        "time_stamp": "Mon Sep  2 14:42:59 2019",
        "server_id": "c9e2a02f58b8fc1cf7865733a9bfdb70b9ebe800",
        "client_port": 61596,
        "host_ip": "<ip>",
        "host_port": 443,
        "unique_id": "156742817931.641393",
        "request":
        {
            "method": "POST",
            "http_version": 2,
            "uri": "/?bla=cat /etc/passwd",
            "body": "param1=value1&param2=SELECT * FROM users;",
            "headers":
            {
                "host": "www.example.com",
                "accept": "*/*",
                "user-agent": "masscan",
                "content-type": "application/x-www-form-urlencoded",
                "content-length": "41"
            }
        },
        "response":
        {
            "body": "\r\n<title>403 Forbidden</title>\r\n\r\n
403 Forbidden
\r\n
nginx\r\n\r\n\r\n",
            "http_code": 403,
            "headers":
            {
                "Server": "nginx",
                "Date": "Mon, 02 Sep 2019 12:42:59 GMT",
                "Content-Length": "162",
                "Content-Type": "text/html",
                "Connection": "close"
            }
        },
        "producer":
        {
            "modsecurity": "ModSecurity v3.0.3 (Linux)",
            "connector": "ModSecurity-nginx v1.0.0",
            "secrules_engine": "Enabled",
            "components":
            [
                "OWASP_CRS/3.1.1\""
            ]
        },
        "messages":
        [
            {
                "message": "Found User-Agent associated with security scanner",
                "details":
                {
                    "match": "Matched \"Operator `PmFromFile' with parameter `scanners-user-agents.data' against variable `REQUEST_HEADERS:user-agent' (Value: `masscan' )",
                    "reference": "o0,7v84,7t:lowercase",
                    "ruleId": "913100",
                    "file": "/usr/share/modsecurity-crs/rules/REQUEST-913-SCANNER-DETECTION.conf",
                    "lineNumber": "33",
                    "data": "Matched Data: masscan found within REQUEST_HEADERS:user-agent: masscan",
                    "severity": "2",
                    "ver": "OWASP_CRS/3.1.1",
                    "rev": "",
                    "tags":
                    [
                        "application-multi",
                        "language-multi",
                        "platform-multi",
                        "attack-reputation-scanner",
                        "OWASP_CRS/AUTOMATION/SECURITY_SCANNER",
                        "WASCTC/WASC-21",
                        "OWASP_TOP_10/A7",
                        "PCI/6.5.10"
                    ],
                    "maturity": "0",
                    "accuracy": "0"
                }
            },
            {
                "message": "Invalid HTTP Request Line",
                "details":
                {
                    "match": "Matched \"Operator `Rx' with parameter `^(?i:(?:[a-z]{3,10}\\s+(?:\\w{3,7}?://[\\w\\-\\./]*(?::\\d+)?)?/[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?|connect (?:\\d{1,3}\\.){3}\\d{1,3}\\.?(?::\\d+)?|options \\*)\\s+[\\w\\./]+|get /[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?)$' against variable `REQUEST_LINE' (Value: `POST /?bla=cat /etc/passwd HTTP/2.0' )",
                    "reference": "v0,35",
                    "ruleId": "920100",
                    "file": "/usr/share/modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf",
                    "lineNumber": "47",
                    "data": "POST /?bla=cat /etc/passwd HTTP/2.0",
                    "severity": "4",
                    "ver": "OWASP_CRS/3.1.1",
                    "rev": "",
                    "tags":
                    [
                        "application-multi",
                        "language-multi",
                        "platform-multi",
                        "attack-protocol",
                        "OWASP_CRS/PROTOCOL_VIOLATION/INVALID_REQ",
                        "CAPEC-272"
                    ],
                    "maturity": "0",
                    "accuracy": "0"
                }
            },
            {
                "message": "OS File Access Attempt",
                "details":
                {
                    "match": "Matched \"Operator `PmFromFile' with parameter `lfi-os-files.data' against variable `ARGS:bla' (Value: `cat /etc/passwd' )",
                    "reference": "o5,10v11,15t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWin,t:lowercase",
                    "ruleId": "930120",
                    "file": "/usr/share/modsecurity-crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf",
                    "lineNumber": "78",
                    "data": "Matched Data: etc/passwd found within ARGS:bla: cat /etc/passwd",
                    "severity": "2",
                    "ver": "OWASP_CRS/3.1.1",
                    "rev": "",
                    "tags":
                    [
                        "application-multi",
                        "language-multi",
                        "platform-multi",
                        "attack-lfi",
                        "OWASP_CRS/WEB_ATTACK/FILE_INJECTION",
                        "WASCTC/WASC-33",
                        "OWASP_TOP_10/A4",
                        "PCI/6.5.4"
                    ],
                    "maturity": "0",
                    "accuracy": "0"
                }
            },
            {
                "message": "Remote Command Execution: Unix Shell Code Found",
                "details":
                {
                    "match": "Matched \"Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:bla' (Value: `cat /etc/passwd' )",
                    "reference": "o4,10v11,15t:urlDecodeUni,t:cmdLine,t:normalizePath,t:lowercase",
                    "ruleId": "932160",
                    "file": "/usr/share/modsecurity-crs/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf",
                    "lineNumber": "481",
                    "data": "Matched Data: etc/passwd found within ARGS:bla: cat/etc/passwd",
                    "severity": "2",
                    "ver": "OWASP_CRS/3.1.1",
                    "rev": "",
                    "tags":
                    [
                        "application-multi",
                        "language-shell",
                        "platform-unix",
                        "attack-rce",
                        "OWASP_CRS/WEB_ATTACK/COMMAND_INJECTION",
                        "WASCTC/WASC-31",
                        "OWASP_TOP_10/A1",
                        "PCI/6.5.2"
                    ],
                    "maturity": "0",
                    "accuracy": "0"
                }
            },
            {
                "message": "Inbound Anomaly Score Exceeded (Total Score: 17)",
                "details":
                {
                    "match": "Matched \"Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `17' )",
                    "reference": "",
                    "ruleId": "949110",
                    "file": "/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf",
                    "lineNumber": "80",
                    "data": "",
                    "severity": "2",
                    "ver": "",
                    "rev": "",
                    "tags":
                    [
                        "application-multi",
                        "language-multi",
                        "platform-multi",
                        "attack-generic"
                    ],
                    "maturity": "0",
                    "accuracy": "0"
                }
            },
            {
                "message": "Inbound Anomaly Score Exceeded (Total Inbound Score: 17 - SQLI=0,XSS=0,RFI=0,LFI=5,RCE=5,PHPI=0,HTTP=0,SESS=0): Remote Command Execution: Unix Shell Code Found; individual paranoia level scores: 17, 0, 0, 0",
                "details":
                {
                    "match": "Matched \"Operator `Ge' with parameter `5' against variable `TX:INBOUND_ANOMALY_SCORE' (Value: `17' )",
                    "reference": "",
                    "ruleId": "980130",
                    "file": "/usr/share/modsecurity-crs/rules/RESPONSE-980-CORRELATION.conf",
                    "lineNumber": "76",
                    "data": "",
                    "severity": "0",
                    "ver": "",
                    "rev": "",
                    "tags":
                    [
                        "event-correlation"
                    ],
                    "maturity": "0",
                    "accuracy": "0"
                }
            }
        ]
    }
}

Error logs
2019/09/02 14:42:59 [error] 18578#18578: *3714036 [client <IP_ADDRESS>] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator Ge' with parameter 5' against variable TX:ANOMALY_SCORE' (Value: 17' ) [file "/usr/share/modsecurity-crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "80"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 17)"] [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname ""] [uri "/"] [unique_id "156742817931.641393"] [ref ""], client: , server: www.example.com, request: "POST /?bla=cat /etc/passwd HTTP/2.0", host: "www.example.com"

If there is a crash, the core dump file.
Not relevant

Notice: Be carefully to not leak any confidential information.
To Reproduce
Steps to reproduce the behavior:
A curl command line that mimics the original request and reproduces the problem. Or a ModSecurity v3 test case.
curl -D - -s -H "User-Agent: masscan" -X POST -H "Content-Type: application/x-www-form-urlencoded" -d "param1=value1&param2=SELECT * FROM users;" 'https://www.example.com/?bla=cat /etc/passwd'
Expected behavior
I expected more information in the audit log, action taken, timings etc.
Server (please complete the following information):

ModSecurity version (and connector): ModSecurity version 3.03 with nginx-connector (latest master)
WebServer: [e.g. nginx-1.14.2]
OS (and distro): [e.g. Linux, Debian]

Rule Set (please complete the following information):

Running the public rule set (from modsecurity-crs.org).
What is the version number? Version 3.1.1

Hi @matthijs,
The timing information was removed on propose in favor of linux instrumentations. What kind of information you want to see on those logs?

I am going to close this issue until we hear from @matthijs again.

dear,
 How to include duplicated rule(id is equals).
 I find second rule cannot be match!
 I doesn’t want to change ‘id’, is existing some hash or other key attributes?
------------------ Original ------------------
From: Felipe Zimmerle<EMAIL_ADDRESS>Date: Tue,Sep 17,2019 10:07 PM
To: SpiderLabs/ModSecurity<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: Re: [SpiderLabs/ModSecurity] [ModSecurity 3.x] Json output log, missing information? (#2161)
Closed #2161.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Hi @nibaozhu,
Not sure if I can help you. My advice is to create an specific issue for your question, try to give a few details on your use case.


Remove entries from quick open history when root folder is removed

Testing #29510

Open VS Code with multiple roots A and B
Use Quick open to open files from B
Remove the root folder B
Quick open still shows files from B in recent
Behaviour does not change even after Reload

I would argue this works as designed, we never remove entries from the recent history in quick open unless a file gets deleted and we can detect that.
The same is true for files which are opened which do not belong to the workspace, we keep them around.
The quick open history is a history of recently opened files, independent from the origin. I think it should stay agnostic to multi-root. The only impact multi-root should have is to honour files.exclude setting per folder if any.

Not sure if I like this behaviour of treating explicit and implicit actions same.
I think opening a file that does not belong to workspace and opening a file that belongs to workspace are different. Because, in first case user explicitly open the file using a different user action (Open File / Drag n Drop.. ). In later case, its more implicit. When user removes the root folder from the workspace, the implicit behaviour should not be treated explicit all of a sudden. I think, removing a root folder is like removing all details belonging to that folder.

Yeah, let's keep it open and see what others have to say. This may or may not be expected behaviour.

Yeah, Makes sense to me as I am asking to remove the root folder.
Retaining the editors might leads to expectation to retaining the settings.

I find the current behavior strange, especially in the context of multi folder workspaces. If I remove a folder from my workspace, I expect those files to disappear from quickopen immediately.
I would also be happy if open files from that repo automatically closed. There has been at least one instance where I was scratching my head trying to figure out what I wasn't getting code intelligence on a file and I eventually figured out it was because I had closed the workspace folder.

Similar behavior when deleting a single file:

the file still shows up in quick open as "recently opened"
selecting the entry results in a "Unable to open '': File not found" popup, followed by it being removed.


Adjust font sizes by language

I find the fonts a bit small on my 13inch MacBook, especially when I am tired.
Chinese and Japanese glyphs have many more strokes than Roman characters so would look better larger.
It would be worth having the default font variable by language so that we can tweak it.

We should also respect/detect the current font settings from the operating system. We have a TODO in the code to cover this.


Fix scheduleUpdate call if this._popper does not exist

Refs #32
Defensive PR to avoid error if this._popper not defined. Should inspect to understand why not defined in the first place, but really needed to fix it for my project.
Merge if it helps =)
Best

Thank you! 🙏


Incorrect extraction.

For the attached PDF file, both stream and lattice generates incorrect results
160.pdf

Which cells are extracted incorrectly? It's recommended that you exclude headers from your selection, since those often confuse Tabula. (And vertical text is not extracted.)

Which cells are extracted incorrectly? It's recommended that you exclude headers from your selection, since those often confuse Tabula. (And vertical text is not extracted.)


docker build error

When I enter "docker build -t caffe:cpu https://raw.githubusercontent.com/BVLC/caffe/master/docker/standalone/cpu/Dockerfile"
return 404 error

URL for the docker file appears to now be:
https://raw.githubusercontent.com/BVLC/caffe/master/docker/cpu/Dockerfile


Use in multiple domains...

I have a scenario were the backend is on localhost:3001 and the frontend is on localhost:3000
I'm tring to login from localhost:3000 but I was not luky to get the session. How can I use nuxt-auth-utils when the backend is on a different domain?

If your backend provides you a JTW, you cam simply use useCookie from Nuxt to store the session and useState() to keep your user.

I actually created a cors middleware
import type { H3CorsOptions } from 'h3'

export default defineEventHandler(async (event) => {
  const _headerOrigin = getRequestHeader(event, 'origin')

  let origin: H3CorsOptions['origin']
  if (_headerOrigin) {
    origin = [_headerOrigin]
  }

  handleCors(event, {
    origin,
    credentials: true
  })
})

but I'm not sure how secure is this.
This is the scenario:

Nuxt frontend
Nuxt backend

the frontend and the backend are open source, the goal is to use the frontend to connect to other backend (for example blusky)

One solution is to to create an API route in your frontend that call your backend so you can avoid using CORS.
I don't see many issues for CORS as long as you whitelist the hosts.

I don't want to use an api route or a proxy because this is also a trpc router that an user could use. I think one of the best way is to create a cors middleware and allow only trusted origins, something like this
import type { H3CorsOptions } from 'h3'

const ALLOWED_ORIGINS = [
  'https://domain0...',
  'https://domain1...',
]

export default defineEventHandler(async (event) => {
  const _headerOrigin = getRequestHeader(event, 'origin')

  let origin: H3CorsOptions['origin']
  if (_headerOrigin && ALLOWED_ORIGINS.includes(_headerOrigin)) {
    origin = [_headerOrigin]
  } else {
    origin = ['https://defaultdomain']
  }

  handleCors(event, {
    origin,
    credentials: true,
    methods: ['GET', 'POST', 'PUT', 'DELETE'],
    allowedHeaders: ['Content-Type', 'Authorization'],
    maxAge: 3600
  })
})


How to train in different devices?

Hello,
If I have 4 gpu in 1 node, and I want to separately train a model in one gpu.(such as training1 -> gpu0, training2 -> gpu1...)
So how do I specify a gpu for training?(such as gpu0, gpu1).

The code is based on the lightning hydra template, which itself builds on PyTorch lightning. Please have a look at the corresponding documentations:

lightning hydra template
lightning choosing device

Thank you!


Translate cache for fast page loading

It is possible use $translateCache as Angular $templateCache. It allow to create plugin like https://github.com/ericclemmons/grunt-angular-templates which can include translations into corresponding module (not only in config phase). It useful for project with lazy loading, when we can use $translatePartialLoader for loading .json for development and cache it .json by $translateCache into one compiled file for build for production.
In case with message format $templateCache can store compiled message format functions like https://github.com/blueimp/angular-localize
f.e. $translateCache.put('locals/en.json', {STR_ID: "String translation", ...})

I guess this can be closed as it was added with b685601?

Yep, that's actually correct ;) Thank you.


Can't render notification

version: 0.10.2
❯ RUST_BACKTRACE=full wired
A wired socket exists; taking ownership.  Existing wired processes will not receive CLI calls.
Notification bus name acquired.
Config reloaded.
[src/manager.rs:97] &notification = Notification: {
        id: 2,
        app_name: Wired,
        summary: Wired,
        body: Config was reloaded.,
        actions: {},
        app_image: false,
        hint_image: false,
        urgency: Low,
        percentage: None,
        time: 2022-06-21 10:41:28.024392670 +08:00,
        timeout: 5000
}
[src/manager.rs:97] &notification = Notification: {
        id: 3,
        app_name: notify-send,
        summary: Summary,
        body: A different body,
        actions: {},
        app_image: false,
        hint_image: false,
        urgency: Normal,
        percentage: None,
        time: 2022-06-21 10:41:42.253420686 +08:00,
        timeout: 4000
}

Notification can be displayed in log, but no notification window popup

Post the config so we can see what it is happening.

I use this config https://github.com/Toqozz/wired-notify/issues/63#issuecomment-1100855910

[src/manager.rs:97] &notification = Notification: {
        id: 4,
        app_name: xxx,
        summary: Approval,
        body: yyy,
        actions: {"default": "Activate"},
        app_image: true,
        hint_image: false,
        urgency: Normal,
        percentage: None,
        time: 2022-06-21 11:11:33.608194615 +08:00,
        timeout: 4000
}

I find the notification with actions like the above can be show, although the popup window is in the central of  the screen.

Well, it's not that hard to fix. I'll see later if I can update a few things on the Wiki later on, but the render part changed quite a bit:
See #62.
Modify the following lines for:
render_criteria: [AppImage],    -> render_criteria: [ And([AppImage]) ],
render_criteria: [Progress],    -> render_criteria: [ And([Progress]) ],
render_criteria: [HintImage],   -> render_criteria: [ And([HintImage]) ],

It worked after I delete render_criteria. I'll try to see the detail of all config fields. Thanks!

I wouldn't recommend just taking it out, since all of them are not supposed to be rendered at once. Please try to modify the lines I posted on the reply before yours later on.

I wouldn't recommend just taking it out, since all of them are not supposed to be rendered at once. Please try to modify the lines I posted on the reply before yours later on.

Thanks, I'll try to adjust the config. Now the display is very strange, but at least, it's worked.


[MM-41817] : Change global header z-index causing issue with profile menu if RHS if is open

Summary
Change global header z-index causing issue with profile menu if RHS if is open.
Ticket Link
Fixes https://mattermost.atlassian.net/browse/MM-41817
Related Pull Requests
NONE
Screenshots

before
after

Release Note
NONE

@koox00 @ashishbhate FYI, this would be nice to fix before March 2nd cloud release.

/cherry-pick cloud

Cherry pick is scheduled.


ASAN error

I've started to update this repo to LVGL v8 but got Address sanitizer error:
==9072==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55d071d02366 at pc 0x7f3ba84b5733 bp 0x7ffd438a5a00 sp 0x7ffd438a51a8
READ of size 512 at 0x55d071d02366 thread T0
    #0 0x7f3ba84b5732  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x79732)
    #1 0x55d071ab240d in input_func ../lv_lib_split_jpg/lv_sjpg.c:289
    #2 0x55d071ab6faa in huffext ../lv_lib_split_jpg/tjpgd.c:315
    #3 0x55d071ab9838 in mcu_load ../lv_lib_split_jpg/tjpgd.c:515
    #4 0x55d071abe696 in jd_decomp ../lv_lib_split_jpg/tjpgd.c:952
    #5 0x55d071ab4f62 in decoder_read_line ../lv_lib_split_jpg/lv_sjpg.c:725
    #6 0x55d071a38549 in lv_img_decoder_read_line ../lvgl/src/draw/lv_img_decoder.c:164
    #7 0x55d0719f800f in lv_img_draw_core ../lvgl/src/draw/lv_draw_img.c:306
    #8 0x55d0719f6f74 in lv_draw_img ../lvgl/src/draw/lv_draw_img.c:84
    #9 0x55d07190b8ba in draw_img ../lvgl/src/widgets/lv_img.c:628
    #10 0x55d07190974a in lv_img_event ../lvgl/src/widgets/lv_img.c:485
    #11 0x55d071a402b9 in lv_obj_event_base ../lvgl/src/core/lv_event.c:96
    #12 0x55d071a438ab in event_send_core ../lvgl/src/core/lv_event.c:368
    #13 0x55d071a3ff2e in lv_event_send ../lvgl/src/core/lv_event.c:73
    #14 0x55d071a91553 in lv_refr_obj ../lvgl/src/core/lv_refr.c:681
    #15 0x55d071a91782 in lv_refr_obj ../lvgl/src/core/lv_refr.c:717
    #16 0x55d071a91161 in lv_refr_obj_and_children ../lvgl/src/core/lv_refr.c:618
    #17 0x55d071a90c82 in lv_refr_area_part ../lvgl/src/core/lv_refr.c:551
    #18 0x55d071a8fe0a in lv_refr_area ../lvgl/src/core/lv_refr.c:466
    #19 0x55d071a8edc8 in lv_refr_areas ../lvgl/src/core/lv_refr.c:389
    #20 0x55d071a8d750 in _lv_disp_refr_timer ../lvgl/src/core/lv_refr.c:216
    #21 0x55d071968fd4 in lv_timer_exec ../lvgl/src/misc/lv_timer.c:306
    #22 0x55d0719682af in lv_timer_handler ../lvgl/src/misc/lv_timer.c:97
    #23 0x55d071af87c0 in main ../main.c:141
    #24 0x7f3ba6d6cbf6 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21bf6)
    #25 0x55d0718cc349 in _start (/home/kisvegabor/projects/littlevgl/eclipse-workspace/lv_sim_eclipse_sdl_v8.0/Debug/lv_sim_eclipse_sdl_v8.0+0x21b349)

0x55d071d02366 is located 0 bytes to the right of global variable 'wallpaper_jpg_map' defined in '../lv_lib_split_jpg/example/images/wallpaper_jpg.c:12:69' (0x55d071ced800) of size 84838
SUMMARY: AddressSanitizer: global-buffer-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x79732)
Shadow bytes around the buggy address:
  0x0aba8e398410: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e398420: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e398430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e398440: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e398450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0aba8e398460: 00 00 00 00 00 00 00 00 00 00 00 00[06]f9 f9 f9
  0x0aba8e398470: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e398480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e398490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e3984a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0aba8e3984b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==9072==ABORTING

To reproduce:
  lv_split_jpeg_init();
  LV_IMG_DECLARE( wallpaper_jpg );

  lv_obj_t * img1 = lv_img_create(lv_scr_act());
  lv_img_set_src( img1,  &wallpaper_jpg);

@vinodstanur could you take a look at it?

Okay, I will check.

@kisvegabor I fuzzed TJpgDec a while back and found that it doesn't do a great job of validating input. I fixed several errors in my fork of the project which is at https://github.com/cmumford/TJpgDec. You should be able to copy tjpgdec.c from https://github.com/cmumford/TJpgDec/commit/1fc035d2e34c50589ef33e51c192ba15679b50e6 to see if your crash goes away.

@cmumford
The issue still persists :slightly_frowning_face:
I haven't fully understood the issue yet, but I think the problem is that the library always tries to read 512 bytes of data but when the last chunk is read the jpeg data can be read out of bounds (if its size if is not Nx512).

Thx @kisvegabor Is this image available so that I can try to reproduce?

Sure, I've used this image from the example folder. Copy pasting the code snippet from my first comment should work out of the box.

@kisvegabor I think that wallpaper_jpg is OK - at least for TJpgDec. I hacked my fuzzer to only fuzz that image data, and it produced no errors on Ubuntu. My branch is at https://github.com/cmumford/TJpgDec/tree/wallpaper-test. I believe there is a bug in lv_lib_split_jpg's input_func in lv_sjpg.c on line 291. TJpgDec calls infunc with the number of bytes that it wants to read (or skip), but infunc needs to verify the request size and return the number of bytes read/skipped (very similar to fread). My fuzzer does this here.

Thank you @cmumford. I've checked it, and I agree the issue could be there.
I've tried to add the read size validation but it's rather complex because of the custom JPEG handling of SJPG.
@vinodstanur do you have some time to check it?

This should be addressed and not closed. I may give it a try if I have some time.

I looked into this a bit, and I think the only real solution is to add a src_len parameter to io_source_t so that the buffer size can be known. Unfortunately this has far reaching consequences as all API's which pass a src parameter will need to have a new src_len parameter added. So, not a simple change.
I'm happy to do the work, but because so many API's will be changed I wanted to contact the project owners to see if this is a desired change. IMHO if we want LVGL to be able to display images from untrusted sources then this change is a must. @kisvegabor what do you think?

I agree that this improvement is required. Thank you for offering your help!
Fortunately, the only public API is lv_split_jpeg_init so we are free to add any parameters to the internal functions.

@kisvegabor I just uploaded a candidate fix at https://github.com/cmumford/lv_lib_split_jpg/tree/img-bug.  So far I've only compiled it - haven't run the example program, or anything else.  Are you in a position to give this a try?

Made a few changes to initial upload. I did build and run the demo_jpg_sjpg example, but don't have a handy project for ASAN testing.


Adding application launch tracking for new tab and webtop launches

Will allow for the HUD to report application launches to metrics server in 2 categories, launches to webtop, and launches to a new tab.
This depends upon the launch-type-link branch from ozp-react-commons ozone-development/ozp-react-commons#29

:+1:

:+1:


DBSQLOperation Refactoring (3 of 3)

Continuation of databricks/databricks-sql-nodejs#163 and databricks/databricks-sql-nodejs#185
Modify FetchResultsHelper and result handlers so they have unified interface and allow chaining. The idea is to form a sort of pipeline that will fetch operation results and then process it on each step. Currently it's just fetching and parsing, but later we'll add an additional step to ensure that chunk size matches maxRows, or steps that may convert data to different representation (or return raw data)

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Files
Coverage Δ

lib/DBSQLOperation/index.ts
95.20% <100.00%> (-0.07%)
:arrow_down:

lib/result/CloudFetchResultHandler.ts
100.00% <100.00%> (ø)

lib/result/JsonResultHandler.ts
100.00% <100.00%> (ø)

lib/result/RowSetProvider.ts
88.88% <100.00%> (ø)

lib/result/ArrowResultHandler.ts
86.56% <80.00%> (ø)

:loudspeaker: Thoughts on this report? Let us know!.


Can`t run the software at the beginning

Hey, I am using the CUT-RUNTools2.0 to do some tests and I encountered some problems at the beginning. I think I correctly all the required softwares and correctly set them in the bulk-config.json file. And I got an empty line following your guide which means the validate.py script runs without errors. However, when I try to do the bulk analysis, there is an error. Here is my code and error:
codes:
./run_bulkModule.sh bulk-config.json SRR891269
error:
./run_bulkModule.sh: line 89: /home/ryan/controlsoftware/CUT-RUNTools-2.0-master/install2/src/bulk/bulk-pipeline.sh: No such file or directory
I checked the installation directory where there was no such file or directory.
Do you have any idea about these issue? I would be really appreciate that if you could give me some advice. Thanks very much!
Ryan,

Hi Ryan,
it seems that a install2 is the issue in /home/ryan/controlsoftware/CUT-RUNTools-2.0-master/install2/src/bulk/bulk-pipeline.sh. can you check the directory you run the tool with pwd?


Removed deprecated sensor param

This entirely removes the sensor parameter, which has been deprecated by Google. See discussion in PR #32.

Thank you so much sir :beers:

No worries - thank YOU for maintaining this excellent repo :)

Any chance 3.2.0 could be released soon? It'd be good to remove Google's deprecation warning from master :)

@scriptist right now :wink:

:+1:
Just realised I didn't remove the 'sensor' bit from the readme. Oops.

@scriptist ouch.... :laughing:
I released also 3.2.1

:+1: :+1: Even better :D


feat: create @waku/react

Problem
@waku/react has it's release process and is detached from main codebase
Solution
move to js-waku
Notes

Resolves https://github.com/waku-org/waku-react/issues/43

Closing for now


Catching NaNs during runtime

In C/C++ on system with glibc it can be done like:
#include <cmath>
#include <fenv.h>

int main() {
    double x, y;
    feenableexcept(FE_ALL_EXCEPT);
    x = -1;
    y = log(x);
    return 0;
}

It raises SIGFPE on all floating point exceptions which usually causes core dump. With Intel compilers in order to be effective it is necessary to enbable floating point exceptions with option -fp-model. With g++ it becomes ineffective with some compiler flags like -Ofast.


Fix typos & grammar in doc/UI/Input

Thanks so much for this! FYI, StereoKit's documentation is auto-generated from the commentdocs during version releases, so you'd want to make your modifications over here. Changes should also be applied to the /develop branch! I generally treat /master as a snapshot of the latest full release.
It looks like I may have at least updated the spelling on that one since v0.3.5, but it is indeed still a pretty rough sentence!

Was unaware that's how you had the workflow setup, thanks for letting me know! :) Since you already fixed it in develop, I'll close this PR.


Refactor: Optimise payment form event handlers and logic

An option for optimising this would be to just apply the click handler to the containing element and use event delegation to check which target has been clicked. This is slightly better for performance and also means we don't have to iterate over the elements.
Would also be good to optimise logic of the payment banner and popover by deprecating jQuery in these handlers.

No longer relevant


Enable manual processing of Queue

You can successfully set  autoProcessQueue=false and the component behaves accordingly. However their is no convenient way to call processQueue().

I posted a solution in #26 that also solves this problem.


News app using React

This is a projectunder SSoC'23.
This project will showcase latest news through the app.
Please assign this project to me.

This issue assign to me, I made it by using newsAPI

React projects are not allowed


Fix: better description for steps 12&15 in registration form

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

Found these two while reviewing a PR.
The text in step 12 was against guidelines as it used a "we".
While the text in step 15 was unclear.

I worry that "smushed" might cause a problem since it is not a common word. Perhaps something like:
"As label elements are inline by default, they are all displayed on the same line, causing their text to run together."
Although I guess "run together" could be an idiom that is not understood by some ESL users as well.
"As label elements are inline by default, they are all displayed side by side on the same line, making their text hard to read."
Just throwing out a few possible alternatives.

@bbsmooth if you add a suggestion you can appear as co-author

Let's go with this:

"As label elements are inline by default, they are all displayed side by side on the same line, making their text hard to read."


When giving Levels directly (with /xp L) player.get(Keys.TOTAL_EXPERIENCE) isn't updated

When players recieve levels directly via the /xp <level>L [player] command the value of player.get(Keys.TOTAL_EXPERIENCE) remains unchanged. However the value is correct when reciving XP orbs or using the /xp <experience> [player] command.
I am currently running...

SpongeVanilla version: 1.11.2-6.0.0-BETA-253, 1.11.2-6.0.0-BETA-252 and 1.11.2-6.0.0-BETA-243
Java version: 1.8.0_121
Operating System:

[x] Linux (Debian)
[ ] macOS
[x] Windows

What is the expected result?
That the value of total experience is increased when giving levels directly.
What is the current result?
The value remains unchanged.
See also: SpongePowered/SpongeVanilla#312

Also I noticed that when enchanting or repairing the value doesn't decrease.

Does receiving experience by an orb, after gaining it with the command, give you the correct total?

@ryantheleach no it does not.

When I tried to reproduce @BrainStone's bug report on SpongeVanilla I noticed that the total experience and the experience levels are tracked separately (and also saved separately) in the internal Minecraft server. I'm not entirely sure in which cases it converts between these values (I haven't checked yet).

Any updates so far?

I know what the issue is, I'm just unsure as to whether something should be done about it, and if so, whether it should be optional, because it's fundamentally a Vanilla issue - if indeed, it is an issue (and I'll explain my thinking in a moment).
The problem is, as stated, that the Minecraft server tracks total experience and levels (and the current fraction of experience between two levels) separately. When changing a player's total experience via the /xp command, Minecraft also ensures that the experience levels are updated. However, this does not happen the other way around, if the levels are updated, the xp is not. This doesn't just happen using the /xp command, you also observe this behaviour when you repair an item using an anvil and enchant an item using the enchantment table.
The issue is that the method on the EntityPlayer that adds to the total experience also does the calculations to work out the level change, but directly calling the level change does not change the experience, unless the level is set to zero. I see no other synchronisation points. It could be that this is intentional, because the XP can the become some sort of score, whilst the levels are expendable - and a lower level will still allow you to benrfit from a lower level cap. However, given that a zero level returns you to zero experience, this seems unlikely.
The fix I have would be to ensure that, when levels are changed outside of the method that adds experience points, is to check to see if there is a mismatch between the current experience and the current level, and, if so, recalculate the total experience (using the current percentage to the next level as a guide). To be clear, I have this ready to go - but I don't know whether this patch should be in an optional mixin.

@dualspiral I don't see a point of it being optional. When would there be a need to control them separately. It just causes confusion for you as a plugin developer but also other plugins manipulating experience.
Having it as an optional mixin makes plugin developers not know what functionality to expect and is harder to develop for.
I think this would be a great addition, just not optionally :)


Data is not exporting.

Hi @JanKoppe. I am using this tool.
I ran a file with ffmpeg and stored the data into a .txt file (/tmp/ffmpeg)
After completion of ffmpeg progress.
I am exporting to an IP address. But it shows the output like this.
Why it isn't showing any data that is in my txt file.

How does it works..? Please help me to make work on this.
Thanks.

Hey! Thanks for trying this out :)
Can you show me the full command that you use to run the ffmpeg-exporter?
Do note please, that the -i parameter expects a path to a directory that contains the progressfile, not the path to a single file.
It sounds like you are writing the progress to a file called /tmp/ffmpeg, and then using the default location of /tmp/ffmpeg for the lookup with this exporter. This does not work, as /tmp/ffmpeg in this case is a file, not a directory. You'd need to write the ffmpeg progress to a file called e.g. /tmp/ffmpeg/foobar.
Then the exporter will pick up this foobar file, and expose the metrics with a identifier="foobar" label, so that you can distinguish between multiple processes.

Hi thanks for the reply.
This is the command that i use to write process into a file.
ffmpeg -i input.mp4 -vf scale=w=100:h=50 output.mp4 -progress /tmp/ffmpeg/myfancytranscodejob.txt
my txt file.
myfancytranscodejob.txt
This is the command that i use to run the ffmpeg exporter.
./ffmpeg-exporter.py -i /tmp/ffmpeg -p 1234
How can i run that.?

Hm, that's weird. One thing to note: The exporter will ignore files with progress=end, which will happen once an ffmpeg process has finished. Your log indicates that the ffmpeg process was running for just ~7 seconds. Could it be that you requested the metrics after the ffmpeg process has finished already?
The primary usecase for this exporter was to have an eye on livestream transcodes. These are very long running ffmpeg processes (sometimes several days). For very short batch jobs (as in your case), this exporter might not be the best approach. Can you elaborate a bit on your use case?

Hi @JanKoppe .
I tried this when my ffmpeg process is on live and it is sending the data to a file located in /tmp/ffmpeg.
But it shows this error while exporting.
command :
./ffmpeg-exporter.py -i /tmp/ffmpeg -p 1234
Error :
[INFO]: watching /tmp/ffmpeg
Traceback (most recent call last):
File "./ffmpeg-exporter.py", line 102, in
REGISTRY.register(FfmpegCollector(args.watch_path))
File "/.local/lib/python3.6/site-packages/prometheus_client/registry.py", line 26, in register
names = self._get_names(collector)
File "/.local/lib/python3.6/site-packages/prometheus_client/registry.py", line 66, in _get_names
for metric in desc_func():
File "./ffmpeg-exporter.py", line 77, in collect
g_speed.add_metric([identifier], float(value.removesuffix('x')))
AttributeError: 'str' object has no attribute 'removesuffix'

Hi. This has found in my searching.
https://stackoverflow.com/questions/66683630/removesuffix-returns-error-str-object-has-no-attribute-removesuffix
It says remove suffix is 3.9+ python version method in str. my python version is 3.6. So, I installed python3.9 and ran it again.
It works.
Command :
python3.9 ffmpeg-exporter.py -i /tmp/ffmpeg -p 1234

Hm, nice catch. Will rework it to not rely on that method, as Python 3.7 and 3.8 are not EOL yet.
Thanks!

Fixed with https://github.com/JanKoppe/ffmpeg-exporter/releases/tag/v0.1.1

Thanks @JanKoppe :smile:


[YouTube Import] MPWG Trademark & Compatibility Call with the EF- EMO

Thank you, @jclingan for helping with import!
Please add the agenda link  as the description of the video
Compatibility Program Discussion
[Paul- EF EMO] presents the process by which the MP community works with the EF trademark team to get things accomplished.
Paul presented for the full hour the content in the spreadsheet  MicroProfile Trademark & Compatibility Program Components
The EMO will use the MPWG forum to communicate with the MPWG on tasks they depend on and require feedback from the MPWG
MPWG will continue to use Git Issues under the WG repo for Compatibility matters
MPWG will continue to use the mpwg forum to communicate with the EMO on the progress of the tasks required for a MPcompatibility project.
The Compatibility spreadsheet Q&A used the full hour.

Uploaded it to youtube last night.


Compile without GPU

Hi, thank you for the great work.
Is that possible to compile "OpenPCDet" without GPU？
Our GPU will automatically cut down the script if the utilization rate of the GPU is lower than 5% for 5min. But the compile process takes longer than 5min. Therefore, I would like to know if it is possible to compile the "setup.py" without GPU.
Thank you in advance!

I think you have to read through the old issues, somebody did it with CPU only I guess, but I'm not sure.

I think you have to read through the old issues. Somebody did it with CPU only, I guess, but I'm not sure.

Hi, thank you for your reply. However, I checked the old issues, and I think someone trained it by CPU. But I am wondering if it is doable to compile it by CPU. For me, the problem would be during compiling, the GPU usage rate is too low, which is not allowed.

Then it seems there is no other way, you must bypass the low GPU usage rate or you can't install OpenPCDet properly.

Okay. Thank you for your suggestion.

@XiangyPeng you can reopen the issue later, but for now it seems you have to compile it with GPU.


Update support library version etc

This sample has been deprecated/archived (check README for more information on newer samples related to this technology).
As recommended by GitHub, we are closing all issues and pull requests.


Add convert.yazi

Thanks for creating your package and adding in awesome-yazi.
Please add your package in the following format to allow users to access your plugin as well as to increase ease to download them.

convert.yazi - Convert between image formats using `magick`

# This contains downloading instruction, prefer `ya pack`, if not, git clone instruction will do.
ya pack -a atareao/convert

[X] I have read the CONTRIBUTING.md file.
[X] I have added my package in ascending order in the sub-section of chosen category.
[X] I have added in the format mentioned above.

Thanks!
If I can suggest feature requests in the plugin, maybe add other formats of images if possible,
Maybe also add on conversion between videos formats or like documents too. Since "convert.yazi" is a very general name.

Oki. I will do it

I'll keep the PR open for now. Once you have implemented them, pls change the descriptions and I'll merge it later on.
Is that okay with you?

I'm not sure if it would be better three different plugins,

image-convert
document-convert
movie-convert

Or just one, because each one uses different tools. magick , pandoc or ffmpeg.
What do you think?

See it's all my personal opinions. why have 3 diff plugins although the plugin size will become big, but then its all in 1.
you can always set multiple args to support each style.
I am personally looking to make an optimizer plugin which supports all different types of files, and optimizes it in space. multiple libraries are there for each type but then! its all in 1.
I would say if you are comfortable in making 3 plugins 1 to one, give it a shot!


Offroad UI enhancements

[ ] Allow editing tethering password while enabled
[ ] Show username for current SSH keys
[ ] Refresh developer page periodically
[ ] Cache home screen state (drive stats, prime status, etc.)

[ ] show cached version instead of "No Internet connection" when offline

[ ] Allow scrolling for release notes on update screen
[ ] Don't shift drive stats down when alert/update button is shown
[ ] Click outside of confirmation dialog to dismiss
[ ] Close confirmation dialogs after settings timeout
[ ] Ability to forget a network (either only the current one, or show a list of all know networks with a delete button next to each one)
[ ] Make driver view fast
[ ] Description text for toggles
[ ] Adjust volume based on speed (old UI implementation: https://github.com/commaai/openpilot/blob/master/selfdrive/ui/android/ui.cc#L169)

[ ] WPA2-EAP support
[ ] Remove saved wifi networks
[ ] UI change to denote the difference between connecting over wifi and ethernet

[ ]  View mount placement degrees like on C2

[ ] Reset SSH keys to default

[ ] comma prime upgrade widget overlaps the home statistics view.

[ ]  comma prime upgrade widget overlaps the home statistics view.
I think that's only true on PC because it uses the narrower screen(https://github.com/commaai/openpilot/blob/fb5873bb6d00b4f625e9ee0cd1767fe8805b2476/selfdrive/ui/qt/qt_window.hpp#L16)


Update mirrorlist to 4.3 Beta 3

Issue description: Redot website needs https://godotengine.org/mirrorlist/4.3.beta-3.json referencing https://github.com/Redot-Engine/redot-engine/releases/download/redot-4.3-beta.3/Redot_v4.3-beta.3_export_templates.tpz, should coincide with a news article for 4.3 Beta 3.

I think we better automate this instead of always making json file. Seems pretty feel unnecessarily creating these files. Or fetching github release from engine side.


Detect website locale based on HTML "lang" attribute and language request HTTP headers

@sfionov can we have page language in the onResponse callback, hence allow platforms to use it for automatic filters activation logic?

Yes, we can add it.


Upgrade __main__.py CLI to be more interactive

I'd like to update the CLI to be more user-friendly and interactive. I think originally the CLI was good for testing and debugging when there weren't too many LIFX products, but given there are now quite a few I thought a few areas could use improvements:

Code is a bit hard to read when we have branching options to support different lifx products/features. E.g. options 9 and 10 were either hev or multizone effects
When setting values, the user needs to know what to input, and in what order to put arguments. E.g. for set_color they need to know hue is first, and accepts 0 through 360; saturation is next and accepts 0 through 100 etc.
When unpacking user inputs in the code, we need to remember that lov[1] is hue, lov[2] is saturation etc.
We also need to validate and error if the numbers aren't numbers or are out of range

Particularly given the changes in:

https://github.com/frawau/aiolifx/pull/75
https://github.com/frawau/aiolifx/pull/67
we are now supporting some really long input arguments. One example is for setting button configs: <haptic_duration_ms> <backlight_on_color_hue> (0-360) <backlight_on_color_saturation> (0-100) <backlight_on_color_brightness> (0-100) <backlight_on_color_kelvin> (2500-9000) <backlight_off_color_hue> (0-360) <backlight_off_color_saturation> (0-100) <backlight_off_color_brightness> (0-100) <backlight_off_color_kelvin> (2500-9000). This doesn't even mention that if you set a hue the kelvin is ignored.

So I had a look around at a few libraries for creating an CLI which can guide the user, while still being quick to repeat the same command over and over (e.g. if you're testing like us).
Originally I looked into both argtypes and click and settled on click for the initial entry point and its built in --help command.
We need a choice method, which lets users pick from a number of options. Both argtypes and click support a choice type syntax, but I wasn't a fan as you'd need to create your own numbering, and then parse the number to get the equivalent value.
Instead I settled on InquirerPy (PyInquirer is a dead project with breaking bugs). This has a number of highly interactive CLI methods which I think dramatically reduces the complexity for a user of the CLI. Example video:
https://user-images.githubusercontent.com/1314053/232798796-f4afc830-532e-49a3-bf39-022de7f1b57d.mov
This gives us a few options to present choices in different ways, depending on whether its a long list, a shorter list, yes or no or a number input. We can ask a few questions in a row instead of needing the user to remember a long series of numbers for a set command. It restricts the choices of the user to ones we support, meaning we don't need to add our own validation to every input. I'm also putting each value into its own variable so the code should be easier to read.
I also removed the initial need to hit Enter to start, replacing with a 1 second timeout.
Given all of these changes, I've updated the minor version to 0.9.0 as it might be a breaking change to someone who uses the CLI.

Let me know what you think. I had to add a few aio.sleep() to make the console output messages come in the right order given the callbacks (you can see the issue in the video above when I select uptime). I guess we need to change the methods in aiolifx.py to be awaitable, but I'm not very familiar with async in Python.

Not sure about this... The CLI was meant as a quick and dirty way to check  device and control them, not as a full fledged app.  For a real CLI app, I would go with textual
What we could do is have a second CLI tool with the fancy stuff....

I think Textual is trying to build a fully fledged app. I'm just trying to solve the UX issues described in the description around entering lots of numbers; and also trying to grab those numbers in code. I think introducing this as a second CLI wouldn't really offer much benefit because

This CLI does the same as the existing one, so why would a user prefer to use the original one?
It becomes two code paths to maintain. One of the benefits of this is easier to read code, but if a contributor still needs to update the original one then it doesn't really matter

WDYT?

Alright... I tried it and it is not bad..... I think I will merge it but it needs a couple of minor changes..
First it initially did not work with
python3 -m aiolifx
You need to add
if name == "main":
cli()
At the end of main
Second, I do not like that "quit" is the default action, can we change that? The default should be the first light IMO
Third can we have a help message at the start ? Something like
Use the arrow keys to select what you want to do. You can also type the bulb/action you want to select
Cheers

Good ideas; have addressed each comment.

Updated to include changes from my other PR


Chapter 3  - CI Error communicating with database: Connection refused (os error 111)

Hi, I've been trying to follow the book and fix the broken CI as it said in Chapter <IP_ADDRESS> Updating Our CI Pipeline
It said to grab the updated version of the GitHub Actions setup here which I did. But my CI fails every time on the same step with error: error communicating with database: Connection refused (os error 111)
https://github.com/bnzone/zero2prod/actions/runs/8442471032/job/23123940815#step:5:207
My workflow - https://github.com/bnzone/zero2prod/blob/main/.github/workflows/general.yml
My init_db script - https://github.com/bnzone/zero2prod/blob/main/scripts/init_db.sh
Am I missing some step?
Any help is appreciated 🙏🏻

You're at a similar place to me in the book. The workflow file used for that run isn't the same as the one you've updated.
It looks like you've copy/pasted the workflow file without changing the on key to reflect your repo so new workflows aren't running for you. In other words, try changing it to this:
...
on: [push, pull_request]
...

@techotron
Thank you so much for taking a look!  🙂
I updated the on key and the workflow POSTGRES_DB env variable to be "newsletter" and that worked!


[Question]: navigation composition

Huge fan of your work and this great OSS project! One discussion I often get into with others is ...should we include menu/nav "clickable" inside specific page objects (and use composition to reuse a set of functions that do this) or ... should you just use more than 1 page object in your tests? One for the navigation /menu work and one for the content you are testing specifically ?
Thanks for the feedback!

@toranb we are glad you like it :smile:. That's actually a good question, I guess it's a matter taste, in my opinion, I like my page objects to reflect everything that the user sees when they get to a page (including repeated content as a menu or nav).
Anyway there are same cases that having more than one PageObject is more clear to me, for example testing that a transition is done when the user does a particular action on a page.
  const createUserPage = PageObject.create({
    visit: visitable('/users/new'),
    fillInName: PageObject.fillable('#name')
  });

  const usersPage = PageObject.create({
    users: PageObject.collection({
      scope: 'users',
      itemScope: 'tr',
      item: {
        name: PageObject.text('td')
      }
    })
  });

test("user is redirected to user list page to see created user", function(assert) {
  createUserPage.visit().fillInName("toranb").clickOn('Save');

  andThen(function() {
    assert.equal(usersPage.users(0).name, "toranb");
  });
});

Hope this answers your question :smile:

@juanazam excellent (sorry I didn't utilize the slack channel first) !

Thanks for the kind words @toranb.
It's really an interesting topic of discussion. Personally, I prefer to implement the nav menu as a page object component and include it on every page that needs it.
We're also discussing how to implement page object extensibility (#109) to DRY this type of definitions so you can implement a base page object that has all the common parts of the application (main menu, etc).
I would like to see how people is implementing this in general, so if you have examples you can share, It would be great to see them.

@san650 I looked over #109 and realized it isn't (yet) in master so for now I'll just follow that and watch what lands in master officially :)
I'm really new to adopting this myself but like mixin model (or any extend option you end up with) as I can include modal/ or navigation as needed. Thanks again!


added upgrade for master nodes

this adds and upgrade.sh which will update master nodes one by one

@DStorck is this PR covered by #320?

@DStorck do we still need this PR or is it covered by #320?

@coffeepac I think  #320 is the real one


Added filename-changes.txt

I have added a list of file name changes in filename-changes.txt, e.g.,
#CfA_Ia/sn1997ef/
old-file.dat new-file.dat
Let me know if you would like me to restructure that.

@guillochon Interesting activity happening in this PR.
https://travis-ci.org/astrocatalogs/sne/builds/120626217

Yep that's Travis in action, as none of your changes affect the build you
can ignore it (it's more for if you edit something that affects the import
scripts).
On Mon, Apr 4, 2016 at 10:24 AM, J. Parrent<EMAIL_ADDRESS>wrote:

@guillochon https://github.com/guillochon Interesting activity
happening in this PR.
https://travis-ci.org/astrocatalogs/sne/builds/120626217
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/astrocatalogs/sne/pull/12#issuecomment-205319611

--
James Guillochon
Einstein Fellow at the Harvard-Smithsonian CfA
<EMAIL_ADDRESS>

That's cool. Reading more.


Make hardcoded timeouts configurable

When attempting to reproduce some rare errors in firecracker-containerd, I've at times increased load quite a bit which has caused me to hit various hardcoded timeouts in the go sdk. When I increase the timeouts in my fork the operations succeed, so the timeouts are just set too low for the tests I'm running. The current values seem fine as defaults, but they should be configurable.
Ones I've encountered:

https://github.com/firecracker-microvm/firecracker-go-sdk/blob/9e2ff62839b228665976be79191e33cffce5f760/machine.go#L495
https://github.com/firecracker-microvm/firecracker-go-sdk/blob/9e2ff62839b228665976be79191e33cffce5f760/firecracker.go#L29
https://github.com/firecracker-microvm/firecracker-go-sdk/blob/9e2ff62839b228665976be79191e33cffce5f760/firecracker.go#L173

@sipsma Totally make sense to me, but how do you plan to configure it?

It's probably reasonable for this sort of thing to be settable via environment variables.

I think I much prefer these to live in the config, rather than as environment variables. We can support both, but I think having them inside the config at the bare minimum is preferable. And honestly, this can probably just use the request timeout.

What config?

@nmeyerhans - The config structure, https://github.com/firecracker-microvm/firecracker-go-sdk/blob/master/machine.go#L52

Right. That's exactly why I think we should use environment variables. That structure comes from the client application, meaning that if we require these settings to come from there, then they can only be adjusted by a client application that knows about them. Since this is specifically useful for debugging, and affects the internal behavior of the SDK, there's no need for the client to know anything about them. Environment variables let us manipulate the SDK settings directly, independent of whatever client is in use. This makes SDK debugging much more practical.
Unless there are use-cases for adjusting these settings in production deployments, we should not. require clients to handle this.

Hm, how do you feel about supporting both then? I mean that definitely adds a level of complexity of figuring out what value to use when both are set, but I can definitely see users wanting to adjust the timeouts in the config.

I'd start with environment variables, since that functionality will address the use cases we know about now and is simple to implement.
If at a later time it makes sense for these values to be settable by the calling application, we can do that. I would not want to add support for that now, though, given a lack of any documented need for it.


790 fix - Resample's dimensions error

Solving one of problems from Issue #790. The bug was merging features and targets data. I don't know why, but after the updates release, target array became a 1-dim. Solution is to reshape it into 2-dim.

Hello @aPovidlo! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file temp.py:

Line 34:1: W391 blank line at end of file

@nicl-nno Временный файл temp в конце удалю


b2d autoformat test and fix

Signed-off-by: Sven Dowideit<EMAIL_ADDRESS>

mmm, TestUpgradeFromImage again - and it passes when run on its own.
++ eval 'hd_amd64 /go/src/github.com/rancher/os/state/hd.img'
1691s
3335
+++ hd_amd64 /go/src/github.com/rancher/os/state/hd.img
1691s
3336
+++ echo '-drive if=virtio,file=/go/src/github.com/rancher/os/state/hd.img'
1691s
3337
+ exec qemu-system-x86_64 -nographic -serial stdio -display none -rtc base=utc,clock=host -enable-kvm -cpu host -m 2048 -net nic,vlan=0,model=virtio -net user,vlan=0,hostfwd=tcp::2222-:22,hostname=rancher-dev -drive if=virtio,file=/go/src/github.com/rancher/os/state/hd.img -smp 1
1691s
3338
--- QemuSuite.TestUpgradeFromImage: qemu stopped
1762s
3339
----------------------------------------------------------------------
1762s
3340
FAIL: installer_test.go:84: QemuSuite.TestUpgradeFromImage
1762s
3342
installer_test.go:126:
1762s
3343
    s.RunQemuWith(c, runArgs...)
1762s
3344
common_test.go:55:
1762s
3345
    c.Assert(err, IsNil)
1762s
3346
... value *errors.errorString = &errors.errorString{s:"Failed to connect to SSH: exit status 255"} ("Failed to connect to SSH: exit status 255")
1762s
3348
--- QemuSuite.TestWriteFiles: starting qemu ../scripts/run, [--qemu --no-rebuild --no-rm-usr --fresh --cloud-config ./tests/assets/test_24/cloud-config.yml]

looks like this fixes the auto-format issue:
sven@i7:~/src/rancher/os$ docker-machine create --driver=virtualbox --virtualbox-boot2docker-url=./dist/artifacts/rancheros.iso  test
Running pre-create checks...
(test) You are using version 4.3.36_Debianr105129 of VirtualBox. If you encounter issues, you might want to upgrade to version 5 at https://www.virtualbox.org
(test) Image cache directory does not exist, creating it at /home/sven/.docker/machine/cache...
Creating machine...
(test) Downloading /home/sven/.docker/machine/cache/boot2docker.iso from ./dist/artifacts/rancheros.iso...
(test) Creating VirtualBox VM...
(test) Creating SSH key...
(test) Starting the VM...
(test) Check network to re-create if needed...
(test) Found a new host-only adapter: "vboxnet0"
(test) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Error creating machine: Error detecting OS: Error getting SSH command: Something went wrong running an SSH command!
command : cat /etc/os-release
err     : exit status 1
output  : cat: can't open '/etc/os-release': No such file or directory

sven@i7:~/src/rancher/os$ docker-machine ssh test
[docker@rancher ~]$
[docker@rancher ~]$
[docker@rancher ~]$ ls -la
total 12
drwxr-xr-x    3 docker   docker        4096 Jan 20 11:55 .
drwxr-xr-x    4 root     root          4096 Jan 20 11:55 ..
drwx------    2 docker   docker        4096 Jan 20 11:55 .ssh

#1532 should fix the os-release warning


Rename subject/topic in API types, and fix orig/new names in UpdateMessageEvent

The name "subject" is long obsolete for Zulip topics.  It persists
in a few places in the API, but we should keep it out of our code.
We had it here on the Message type really just because this is a
part of the code that I wrote in the very first hours of the
prototype of this app, before I'd read up enough on the
json_serialization package to find JsonKey.name.

In the server API on UpdateMessageEvent, "stream_id" is the old value and contrasts
with "new_stream_id", but "subject" is the new value and contrasts
with "orig_subject".
This makes them confusing to read, and is the way it is only for
historical reasons: I believe the story is that before Zulip
Server 3.0 added support for moving topics between streams, three
of those properties were already present and then "new_stream_id"
was squeezed in at the best name still available.
Fixing that in the API will be a pain because it'll need a migration
path for compatibility.  But we can fix it here within our code.

The renames in UpdateMessageEvent are from a draft I'd started a few weeks ago. Then while I was polishing that up to send (in particular, writing tests for it), I decided to go back and take care of the Message.subject rename too, since that's been an occasional small annoyance.
/cc @PIG208 because when you go to handle UpdateMessageEvent for #750 / #171, you'll want to have rebased atop these renames.
I have some other draft commits that touch the MessageStoreImpl.handleUpdateMessageEvent implementation itself. I'll take a look through those next and see if any of them are relevant to #171, or if they become relevant only when you get to #150. If there are changes relevant to #171, I'll get those in shape too and send a PR.

Thanks for the review! Rebased, and replied above; merging.


[BSI][v1.7][20201118] Vulnerability to execute arbitrary SQL statements on the key package database

Rating: High
Description:
The version 1.7.0 of the iOS CWA app implements a feature to revoke key
packages that are already stored in the device's local database. This is
implemented via the app's configuration, which is fetched each time the
app is launched. This config file can contain a list of E-Tags. These
tags describe which key packages need to be removed. Two different SQL
queries are used to remove the revoked packages. Both of these queries
are assembled using string concatenation [1, 2].
In case [1], the injection occurs in the list variable that is
created
from the list of E-Tags out of the configuration file. In the second
case [2], the result of this first query is used, also by string
concatenation, to construct another query.
A malicious server or an attacker controlled server can use this
vulnerability to execute arbitrary SQL statements on the key package
database of iOS CWA apps. In addition to that, due to the usage of
sqlite, the attacker can potentially write (and potentially even read)
files in the app's sandboxed filesystem.
While an attacker controlled, or fake backend is unlikely, it would
certainly increase the attack surface an attacker had available if they
compromised the cwa-server (by potentially gaining control over the
connected iOS apps). Additionally, it could be interpreted as an
intentionally placed backdoor.
1:
https://github.com/corona-warn-app/cwa-app-ios/blob/94b28ccd8cff0ca2457e407ab5aebc0899f9d6cc/src/xcode/ENA/ENA/Source/Services/DownloadedPackagesStore/V2/DownloadedPackagesSQLLiteStoreV2.swift#L308
2:
https://github.com/corona-warn-app/cwa-app-ios/blob/94b28ccd8cff0ca2457e407ab5aebc0899f9d6cc/src/xcode/ENA/ENA/Source/Services/DownloadedPackagesStore/V2/DownloadedPackagesSQLLiteStoreV2.swift#L429

Internal Tracking ID: EXPOSUREAPP-3851

Copy cats 😸

Hey @BSI-TF-CWA ,
I have created a Jira ticket EXPOSUREAPP-3851 for this issue so that the developers are notified.
Thanks again for your input!
Best regards,
SG
Corona-Warn-App Open Source Team

Just for the record: this is a vulnerability in a piece of code that has not been released yet.

This Issue should be closed by #1539 but the automatic closing somehow didn't work @svengabr


ReshardAllDocsTest errors for Elixir tests in CouchDB test suite

-CouchDB v2.3(master branch)
-Erlang OTP v21
-Elixir v1.8.2
Following error messages are found intermittently while running CouchDB test suite:-
ReshardAllDocsTest
  * test all_docs after splitting all shards on node1 (16905.6ms)

  1) test all_docs after splitting all shards on node1 (ReshardAllDocsTest)
     test/elixir/test/reshard_all_docs_test.exs:21
     ** (RuntimeError) timed out after 10162 ms
     code: wait_job_completed(jobid)
     stacktrace:
       (couchdbtest) test/elixir/lib/couch/db_test.ex:301: Couch.DBTest.retry_until/4
       test/elixir/test/reshard_all_docs_test.exs:32: (test)

  * test all_docs after splitting the same range on all nodes (15679.8ms)

  2) test all_docs after splitting the same range on all nodes (ReshardAllDocsTest)
     test/elixir/test/reshard_all_docs_test.exs:39
     ** (RuntimeError) timed out after 10164 ms
     code: |> Enum.each(fn id -> wait_job_completed(id) end)
     stacktrace:
       (couchdbtest) test/elixir/lib/couch/db_test.ex:301: Couch.DBTest.retry_until/4
       (elixir) lib/enum.ex:769: Enum."-each/2-lists^foreach/1-0-"/2
       (elixir) lib/enum.ex:769: Enum.each/2
       test/elixir/test/reshard_all_docs_test.exs:51: (test)

Entire test suite for CouchDB v2.3(current master) executes successfully/passes with Erlang v21 and Elixir v1.8 on PowerPC64LE. Closing this issue. Thanks for all your help and support in getting this through.


Assignment of float to int changes data type instead of truncation

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

Ubuntu 19.10

Modin version (modin.__version__):

0.7.3+15.gc09e61f.dirty

Python version:

Python 3.7.5

Code we can use to reproduce:

import modin.pandas as pd

col = pd.Series([1, 2, 3])
print(type(col[0]))
print(type(col[1]))
col[0] = 1.1
print(type(col[0]))
print(type(col[1]))

Describe the problem

In this reproducer assignment of float to an integer Series changes data type of the whole Series object. Pandas instead truncates floats to integers and retains int64 data type.
Source code / logs

This is interesting because pandas is not consistent between Series.__setitem__ and Series.iloc.__setitem__:
import pandas

col = pandas.Series([1, 2, 3])
print(type(col[0]))
print(type(col[1]))
col[0] = 1.1
print(type(col[0]))
print(type(col[1]))

col = pd.Series([1, 2, 3])
print(type(col[0]))
print(type(col[1]))
col.iloc[0] = 1.1
print(type(col[0]))
print(type(col[1]))

@gshimansky Is this a problem in a workflow you're testing? Does the workflow assume the data type will not change?

No this code doesn't appear in any benchmark. I noticed this bug when I was writing reproducer tests and filed it as a side work because I saw difference with pandas behavior.

pandas 1.4.4 seems to also change datatype to float when you do assignment with either Series.__setitem__ or Series.iloc.__setitem__. Modin does the same as of version af6f8277cf9caeb7dec8b390ef3652ec0fa04774. Both Modin and pandas seem to behave the same way for the code in https://github.com/modin-project/modin/issues/1431#issuecomment-663650672.


[UNDERTOW-1886] [master] Request dispatcher is returned when the path points to outside the servlet context

Issue: https://issues.redhat.com/browse/UNDERTOW-1886
The CanonicalPathUtils.canonicalize now receives an optional boolean to return null when out of context. It's only used in the request dispatcher related methods. A system property -Dio.undertow.REQUEST_DISPATCHER_NULL_DISALLOWED=true restores the old behavior just in case. Tests addded.
PR for master.
PR for 2.1.x: https://github.com/undertow-io/undertow/pull/1118
PR for 2.0.x: https://github.com/undertow-io/undertow/pull/1119

@fl4via I added the property just in case (it can be removed in that case) but the CanonicalPathUtils.canonicalize is used in a lot of places that maybe now don't expect a null returned. That's why I created a second variant with the nullAllowed parameter that is used for the request dispatcher. If you are sure that we can return null anywhere please go ahead with your modification.

There is an intermitent failure in DefaultServletCachingListenerTestCase in JDK11 proxy-ajp scenario.
I created this Jira to track it: https://issues.redhat.com/browse/UNDERTOW-1895


A Stream must be seekable when opening package on .NET Core

While refactoring tests, I found that on .NET Core, the following fails, while it succeeds on .NET Framework:
using (var stream = TestFileStreams.Presentation)
using (var package = Package.Open(stream))
{
    var openSettings = new OpenSettings
    {
        MarkupCompatibilityProcessSettings = new MarkupCompatibilityProcessSettings(MarkupCompatibilityProcessMode.ProcessAllParts, FileFormatVersions.Office2013)
    };

    using (var doc = PresentationDocument.Open(package, openSettings))
    {
        var v = new OpenXmlValidator(FileFormatVersions.Office2013);
        var errs = v.Validate(doc);

        Assert.Equal(94, errs.Count());
    }
}

The input stream needs to be made writeable (ie copied to a MemoryStream):
using (var stream = TestFileStreams.Presentation.AsMemoryStream())
using (var package = Package.Open(stream))
{
    var openSettings = new OpenSettings
    {
        MarkupCompatibilityProcessSettings = new MarkupCompatibilityProcessSettings(MarkupCompatibilityProcessMode.ProcessAllParts, FileFormatVersions.Office2013)
    };

    using (var doc = PresentationDocument.Open(package, openSettings))
    {
        var v = new OpenXmlValidator(FileFormatVersions.Office2013);
        var errs = v.Validate(doc);

        Assert.Equal(94, errs.Count());
    }
}

This fails because ZipArchive requires a stream that has read, write, and seek capabilities in .NET Core:
Failed   DocumentFormat.OpenXml.Tests.PptxTests01.P005_PptxCreation_Package_Settings
Error Message:
 System.ArgumentException : Update mode requires a stream with read, write, and seek capabilities.
Stack Trace:
   at System.IO.Compression.ZipArchive.Init(Stream stream, ZipArchiveMode mode, Boolean leaveOpen)
   at System.IO.Packaging.ZipPackage..ctor(Stream s, FileMode packageFileMode, FileAccess packageFileAccess)
   at System.IO.Packaging.Package.Open(Stream stream, FileMode packageMode, FileAccess packageAccess)
   at System.IO.Packaging.Package.Open(Stream stream)
   at DocumentFormat.OpenXml.Tests.PptxTests01.P005_PptxCreation_Package_Settings() in C:\Users\tasou\Projects\Software\Open-XML-SDK\DocumentFormat.OpenXml.Tests\TestPptx01.cs:line 60

This has been fixed as of v4.4.0 of System.IO.Packaging (see https://github.com/dotnet/corefx/commit/4589ecea6de8524d89ed194da52db71c47974722#diff-158b13054282356f81163626ee7d36ad). We should update to the latest System.IO.Packaging to get this fix