[GRIDFIELD] BUG: pagination in GridField does not work if DataObject default_sort field values are not unique

SUMMARY:
Pagination does not work well when the data is provided by MySQL 5.6 / 5.7 (php 7.1) where the data-set is sorted by a field (or fields) that are not enforced to be unique on a database level.  The reason for this is that MySQL may return datasets in a random order (beyond what it has been asked to sort by).
You can possibly replicate this bug by yourself by using the tools listed below:

https://github.com/sunnysideup/silverstripe-sort-test/: see the bug in action
https://github.com/sunnysideup/silverstripe-dataintegritytests/blob/master/code/CheckForMysqlPaginationIssuesBuildTask.php: analyse your own project

This means, for example, that:

batch processing may not work correctly (you will process some records twice and other ones not at all)
grid field pagination is broken (some items listed twice or more, on separate pages, others not at all)
pagination on the front-end is broken (some items listed twice or more, on separate pages, others not at all)

There are still a lot of questions about this bug, but I have been able to consistently demonstrate this error on several machines with clean installs of 3.6.1
Details
the issue: when the default_sort value for a dataobject contains non-unique values for one or more rows then the GridField ends up NEVER showing some records while showing other records more than once:
How to replicate:

install SS 3.6.1 using standard installer

add the following dataobject (MyDataObject):

<?php

class MyDataObject extends DataObject
{

    private static $db = array(
        'Title' => 'Varchar(20)',
        'SameSame' => 'Varchar(20)'
    );

    private static $default_sort = array(
        'SameSame' => 'ASC'
    );

    function requireDefaultRecords()
    {
        DB::query('DELETE FROM MyDataObject');
        for($i = 1; $i < 47; $i++) {
            $filter = array('Title' => "item #".$i);
            if(! MyDataObject::get()->filter($filter)->first()) {
                $obj = MyDataObject::create($filter);
                $obj->SameSame = 'same';
                $obj->write();
            }
        }
    }

}

then add modeladmin (MyModelAdmin.php):

<?php

class MyModelAdminTest extends ModelAdmin {

    private static $managed_models = array('MyDataObject');

    private static $url_segment = 'test';

    private static $menu_title = 'Test';
}

You get the following results:

For my client, I was sorting a GridField by EndDate and StartDate. Some of the items has the same EndDate and StartDate and thus were missing.  Making it impossible for them to be found by the client.

Have you tried adding ID as the last item in the list?
This does look like a genuine bug, but perhaps adding ID would be a workaround for it

@flamerohr  - you of course that is a work around, but this is a serious bug - because anytime you do not sort by a field that mysql enforces to be unique, i.e. in basically almost all cases (e.g. sorting by Title, by Created, by LastEdited, by Date, and so on....  you will potentially end up with missing records in the GridField.

unfortunately, this is a potential problem in any given relational database, many queries do not explicitly provide a sort (or not a sort with at least a unique column) and has the "implied" sorting by ID, which is what happened here.
I've marked as critical because of data disappearing, but I encourage always adding explicit ID sort to the end of any sort calls.

I can't replicate this on 3.6.1. Can you test it on a plain install?

I run this on a plain 3.6.1

I wonder if the auto increment on your ID is isn't working properly. Can you try deleting the entire DB and starting over? I'm curious if you're getting IDs that are non sequential with your Item numbers due to all the deletions. Try adding ID as a summary_field to help debug.

I run the following code:
    public function MyDataObjectRaw()
    {
        $this->html .= '
-------------------------------------';
        $rows = DB::query('SELECT Title, ID FROM MyDataObject ORDER BY SameSame ASC;');
        foreach($rows as $row) {
            $this->html .= '
'.$row['Title'].' = '.$row['ID'];
        }
        $this->html .= '
-------------------------------------';
        $rows = DB::query('SELECT Title, ID FROM MyDataObject ORDER BY SameSame ASC LIMIT 0, 10;');
        foreach($rows as $row) {
            $this->html .= '
'.$row['Title'].' = '.$row['ID'];
        }
        $this->html .= '
-------------------------------------';
        $rows = DB::query('SELECT Title, ID FROM MyDataObject ORDER BY SameSame ASC LIMIT 10, 10;');
        foreach($rows as $row) {
            $this->html .= '
'.$row['Title'].' = '.$row['ID'];
        }
        $this->html .= '
-------------------------------------';
        $rows = DB::query('SELECT Title, ID FROM MyDataObject ORDER BY SameSame ASC LIMIT 20, 10;');
        foreach($rows as $row) {
            $this->html .= '
'.$row['Title'].' = '.$row['ID'];
        }
    }

and the output that I get is:

item #36 = 369
item #25 = 358
item #26 = 359
item #27 = 360
item #28 = 361
item #29 = 362
item #30 = 363
item #31 = 364
item #32 = 365
item #33 = 366
item #34 = 367
item #35 = 368
item #24 = 357
item #37 = 370
item #38 = 371
item #39 = 372
item #40 = 373
item #41 = 374
item #42 = 375
item #43 = 376
item #44 = 377
item #45 = 378
item #46 = 379
item #13 = 346
item #2 = 335
item #3 = 336
item #4 = 337
item #5 = 338
item #6 = 339
item #7 = 340
item #8 = 341
item #9 = 342
item #10 = 343
item #11 = 344
item #12 = 345
item #1 = 334
item #14 = 347
item #15 = 348
item #16 = 349
item #17 = 350
item #18 = 351
item #19 = 352
item #20 = 353
item #21 = 354
item #22 = 355
item #23 = 356
item #1 = 334
item #2 = 335
item #3 = 336
item #4 = 337
item #5 = 338
item #6 = 339
item #7 = 340
item #8 = 341
item #9 = 342
item #10 = 343
item #12 = 345
item #1 = 334
item #10 = 343
item #9 = 342
item #8 = 341
item #7 = 340
item #6 = 339
item #5 = 338
item #4 = 337
item #3 = 336
item #12 = 345
item #11 = 344
item #10 = 343
item #9 = 342
item #8 = 341
item #7 = 340
item #6 = 339
item #5 = 338
item #4 = 337
item #3 = 336

I'm not convinced this is a SilverStripe issue. The way that MySQL resolves an ambiguous sort is highly unpredictable. See (https://dba.stackexchange.com/questions/6051/what-is-the-default-order-of-records-for-a-select-statement-in-mysql).

MySQL sorts the rows any way it wants to, without any guarantee of consistency.

That I can't replicate it should be an indication that we're dealing with some kind of unknown. It could be as simple as differing storage engines, MySQL versions, or both. In short, don't use ambiguous sorts.

Hi Aaron,

In short, don't use ambiguous sorts.

That is true in theory, but I dont think that is true practice, most of the time you are sorting by an ambiguous sort: e.g. we tend to sort by Sort / SortNumber (which are usually not database enforced uique), Title, Name, Calendar Start Date, Created, LastEdited, etc... etc....  Anytime they are not unique, you will end up with this problem.
So, I reckon we should fix this, for the following reasons:
(1)
I am sure that most default_sort entries, such as SiteTree.php do not sort by a database enforced unique key. If any of them are displayed in a paginated list (e.g. a gridfield, search results, filtered list, etc... etc... ) there is a real chance that some will show up twice on different pages and that other items will never show.
(2) There is an easy fix - add an ORDER BY BaseTable.ID to the end of all SQL Sort statements
(3) The danger with this bug is that no one will notice until it has real life consequences.  There is no error, there lists seem to operate fine, etc...
(4) A perfect developer should remember to add the Sort By ID to the end of the sort statement - but if (a) you always need to add this (b) few developers actually do this in practice - then this is another reason for adding it.

I could not replicate this behaviour on my lappy:
+-------------------------+-------------------------+
| Variable_name           | Value                   |
+-------------------------+-------------------------+
| innodb_version          | 5.7.19                  |
| protocol_version        | 10                      |
| slave_type_conversions  |                         |
| tls_version             | TLSv1,TLSv1.1           |
| version                 | 5.7.19-0ubuntu<IP_ADDRESS> |
| version_comment         | (Ubuntu)                |
| version_compile_machine | x86_64                  |
| version_compile_os      | Linux                   |
+-------------------------+-------------------------+
I will check mysql version at work.

Here is another test I ran:
Page:

    function requireDefaultRecords()
    {
        for($i = 1; $i < 400; $i++) {
            DB::alteration_message('Creating Pages: '.$i);
            $filter = ['Title' => 'Page No. '.$i];
            $page = Page::get()->filter($filter)->first();
            if(! $page) {
                $page = Page::create($filter);
            }
            if($i > 10) {
                $parentPage = Page::get()->filter(array('ID:LessThan' => 10))->sort('RAND()')->first();
                if($parentPage) {
                    $page->ParentID = $parentPage->ID;
                }
            } else {
                $page->ParentID = 0;
            }
            $page->writeToStage('Stage');
            $page->publish('Stage', 'Live');
        }
    }

Page_Controller:

    public function UnpaginatedPages()
    {
        return Page::get();
    }

    public function PaginatedPages()
    {
        $list = Page::get();

        $list = new PaginatedList($list, $this->getRequest());
        $list->setPageLength(395);
        return $list;
    }

Page.ss
    <% loop PaginatedPages %>

$Title

    <% end_loop %>

    <% if $PaginatedPages.MoreThanOnePage %>
        <% loop $PaginatedPages.Pages %>
            <% if $CurrentBool %>
                $PageNum
            <% else %>
                <% if $Link %>
                    $PageNum
                <% else %>
                    ...
                <% end_if %>
            <% end_if %>
            <% end_loop %>
    <% end_if %>

(2) There is an easy fix - add an ORDER BY BaseTable.ID to the end of all SQL Sort statements

I really think this is dangerous scope creep for the ORM, and it sets a really bad precedent. The ORM is an abstraction layer between you and SQL. It's primary purpose is to make querying more succinct, more secure, and more declarative. It is not the ORM's job to fix your bugs. It should be very possible to write bad queries with the ORM, and sorting by something you know not to have a singular value is one of those areas.

(a) you always need to add this no matter what

Personally, I've never had to do this. Keep in mind, you're testing a very extreme case, where all the values of a column are equal, and in any real-world scenario, it would be apparent to the developer that he was sorting on a column with uniform values. If, by contrast, I sort by Title ASC, and 2/100 records have the same title, I'm not sure I've ever noticed any consequences for the logic MySQL uses to resolve that conflict. If I did, I'd add a second column.

The best practice, in these scenarios, is to encourage developers to implement composite default_sort values for their models, if they are aware that their sort column is not unique. I concur with @unclecheese on this.
E.g.
private static $default_sort = '"BaseTable"."Sort" ASC, "BaseTable"."ID" ASC';

I agree, documentation to encourage this would be the best outcome for this...
I don't believe enforcing an ID sort at the end of all queries which uses a sort is a good idea.
The SiteTree.default_sort also not having ID doesn't mean it's best practice :) in fact if anyone created a pull request to add ID in for SiteTree, I think it'd be a welcomed change.

The best practice, in these scenarios, is to encourage developers to implement composite default_sort values for their model

@tractorcow
What about SiteTree.Sort and  Blog:
https://github.com/silverstripe/silverstripe-blog/blob/master/src/Model/BlogPost.php#L134-L139
Both models are bound to have non-unique sort values.

We could add ID to those models exclusively, but that's still preferable to baking it into datalist for all classes globally.

Check out my scenario:
On an e-commerce website we offer discount vouchers. We had set the default_sort to EndDate DESC, StartDate DESC as that made sense at the time. In the modelAdmin that works great.
Then the client adds a whole bunch of discount vouchers with the same start and end dates.
When he is trying to delete them, there are around 8 / 46 that  are not showing up in the ModelAdmin (and another 8 that show up twice) and calls me. I immediately add the ID value and that sorted it.  However, as you can see, BlogPost and Page have the same issue and this is obvious not ideal.
Another example
Member List for Club, Ms Smith can not find herself in list, calls admin, admin finds her on page three of list, sends her a screenshot and a link to page three. Ms Smith can not find herself on that page (there are a few Smith people, list is sorted by last name).

SiteTree.Sort isn’t supposed to be unique, is it? The sort is relative to the parent page

@kinglozzer no, SiteTree.Sort is not supposed to be unique, but what I found is that when you sort by a non-unique field (e.g. SiteTree.Sort) then pagination (including GridField pagination) can do pretty crazy things as Mysql is rather unpredictable in its sort order when there are non-unique values.

Here is my code as a repository:
https://github.com/sunnysideup/silverstripe-sort-test
I did some further testing.
It does not happen with pages, it only happens with DataObjects.

I think this is really getting into a philosophical discussion on what is the role of framework/ORM. That seems to be where we're divided. My thoughts:
If you assign a singular sort clause to a DataObject for a column that is not entirely unique, and it produces unexpected results,  that's on you as the developer who wrote that code. To expect the framework to jump in and save you is undue, for the same reason the template parser doesn't close your
 tags.
If we were to implicitly put ID ASC in the sort of every DataList, we would be punishing every developer who went about it the right way, and unique columns, or composite clauses. There's a performance impact for having multiple sort columns, and it's nothing I would want to impose on every query without a full opt-in from the user. If I'm sorting on something almost guaranteed to be unique, like Created, I don't need or want ID in the mix as well, and if I have to write special code to tell the framework not to do that, that creates a really suboptimal developer experience.

A Framework is there to make its users happy.

add sort by ID to GridField paginated lists?
add sort by ID to Paginated lists (optional)?

Also - Created is often not unique (e.g. import of data - import through API - high volume sales).

Both GridField and PaginatedList are ORM agnostic. They're designed to work only with instances of SS_List, so injecting concerns over ID into either of them is inappropriate.
The fact that many users would be punished by this safety feature is really a non-starter for me. I don't think it aligns well with the core values of the product.

I tried sorting by Created - same issue (most objects are created within one second).

That's not the point, though. In a real-world scenario where you were using data that was all generated programatically in a matter of milliseconds, you would intuit the Created column to be unreliable for sort, and use a different column, or a composite sort clause. What you can demonstrate in test code should not influence the decisions we make that affect the developer experience.
Developers are responsible for understanding the idiosyncrasies of their data, and to install guardrails in the framework to protect them from writing bad code sets a terribly dangerous precedent. If it is acceptable for the ORM to audit the efficacy of your sort clause (at the expense of performance), it logically follows that the framework is permitted to make all kinds of other overreach in the name of developer hand-holding, like closing
 tags, preventing redirect loops, removing unnecessary queries, etc.

:-)
Thank you for all your attention to detail in this bug report.
These are my assumptions right now:

the problem exists
it may negatively affect our users in a significant way
most Silverstripe code out there does not follow best practice in relation to this issue (i.e. most code does not add a Sort By Unique Field to pagination / gridfields).

Reply from SS core team is: fix your own code, it is not good to punish best practice by adding a hack for sloppy code (always sorting by ID).
I think we need to test the assumptions a bit more first.  I have set up a small repo other developers can see the bug in action: https://github.com/sunnysideup/silverstripe-sort-test/.
Thank you again for looking into all of this.

Have added an actionable issue to the backlog.

Closing as

I just realised then when you sort by one the headers in a gridfield, you end up reproducing the same bug.
I am now sorting my dataobjects by ID:

But when I click on SameSame, you end up having some items that never show up and other items that show up twice on two pages.
I still have a nagging feeling that the consequences of this bug are underestimated.

page 2:

I get that this is an issue when you have duplicate field values. For me, the best practice when it was known that you could have duplicate values and it was important to have consistent and predictable ordering (particularly in the case of pagination), I was always certain to include a secondary sort field to reduce ambiguity at the database level.  That said, I just wanted to point out:

... but sort by ID ASC/DESC should always be added as the last sort criteria, no matter what.

I would have to disagree with this, since it I think it potentially adds an extra level of magic ✨ which is the last thing you want in an already complicated ORM. What happens when the ORM incorrectly parses/interprets the user's intent in their $default_sort property? How do we know they want older or newer records first (when there are dupes)?  What happens if you need to use FIELD(...) or RAND(...) functions? I think it adds more issues than it solves, since:

Adhere to best practice; when sorting order must be consistent, ensure you resolve ambiguity on your own.
KISS principle: Let the database do the heavy lifting and reduce complexity as much as possible (increases transparency).

@patricknelson, I agree with your theory, but in practice this is not always so easy or practicable.  My thinking right now is that we should add a Sort by ID ONLY when you limit a set, but I have not really found a solution yet.
Have you considered that, for example, by default the GridField allows the user to sort by all Summary Fields that are database fields? Thus, when you sort by a non-unique value in the GridField this bug will raise its ugly head, even though your default_sort may have included the ID field.

That's GridField, however, which allows you to select sort columns. I was replying to:

... but sort by ID ASC/DESC should always be added as the last sort criteria, no matter what.

... and then clarified that this shouldn't happen at the ORM (or framework) level when sorting in general. When you're in a GridField, I think it might be fine to enable that feature (possibly by default) and then maybe allow disabling of it via some extra setting on your GridField instance, if it gets in the way.

Hi Patrick,
Thank you for our reply. I think the first thing is really understand the issue, when does it happen, where does it happen. I also jumped to the "solution" but I think the key is to understand the issue first. It does not seem to happen with all DataObjects, so I want to know, for example why that is.
It seems to me when you add DataObjects programatically (i.e. many at the same time) then it is more likely to happen.  Also, I could not replicate it on Pages, why is that?

We've had a bit of a chat about this internally, and there's some consensus that a halfway measure to mitigate this is appropriate.
First, the condition of MySQL returning a non deterministic result set when ambiguous sort columns are used is absolutely a feature, not a bug. From the MySQL docs:

If multiple rows have identical values in the ORDER BY columns, the server is free to return those rows in any order, and may do so differently depending on the overall execution plan. In other words, the sort order of those rows is nondeterministic with respect to the nonordered columns. One factor that affects the execution plan is LIMIT, so an ORDER BY query with and without LIMIT may return rows in different orders.

While MySQL is providing the "expected result" (albeit non deterministic) in this case, it's fair to say DataList is not. As a implementor of the Sortable interface, DataList is expected to sort deterministically. Further, it is the job of the ORM to obscure the "how" from the user, and create the list in a consistent, uniform way, whether you're using MySQL, Postgres, etc, or just plain ArrayData.
All that said, to force ID into the sort is still too much magic, but ensuring default_sort is added seems like a reasonable halfway point. The problem is, with the direction being included as part of the default sort, it makes sort direction toggling (I.e. GridField column headers) unreliable. Therefore, a more sensible setting might be something more like default_sort_column, with null being allowed to disable forced deterministic sorts.
Thoughts?

Good thoughts ;-) I agree wit hall of that.
Here are a couple of thoughts I had:

limit without reliable order of records is useless (i.e. limit 10, 10 may return exactly the same as limit 0, 10).
database enforced unique fields are not a good option in silverstripe: https://github.com/silverstripe/silverstripe-framework/issues/6819.

Putting these two together ... whenever you use limit in an ORM call, you will have to sort by ID as the last sort fragment (e.g. turn ORDER BY "StartDate" DESC into ORDER BY "StartDate" DESC, "ID" ASC).
So, what I would recommend is: whenever we limit a recordset in an ORM call, we add sort by ID as the last sort phrase (with the ability to turn it off) by default.  This is how I would do it:
When putting together the final SQL for a database query - IF:

limit is included
there is no sort by a database enforced unique field
if the datbase is MySQL (and others???)
then add sort by ID
unless the user turns it off

Research questions:

why does this random sorting no occur on Pages - AFAIK?
does it also occur on other Databases (apart from MySQL)?
what is the time difference between SELECT * FROM Table AND SELECT * FROM Table ORDER BY ID - for a table with a few records and a table with a ton of records...
is 4.0 also affected?
are there faster ways to ensure a reliable sort order other than adding ORDER BY ID
how we do we ensure that there is no ambiguous ID added (i.e. the baseclass / table is always added).

So the answer to most of your questions/concerns is that it's not a bug or oddity with MySQL or Postgres or pages or DataObjects. This is something that's encoded in the SQL standard. The server is free to return the rows in any order it wants when identical values are encountered. That means, potentially, all database platforms are affected, and the way that this issue may manifest itself in various contexts, be it with the application of certain clauses, implementation of / version of storage engines, or just plain phases of the moon, is irrelevant. The point is that the SQL standard in this case tells us the result is nondeterministic, and we have an API that promises to be deterministic, so we need to reconcile that conflict somehow.
I think if we were to add a default_sort_column setting, it would have to be applied to all queries -- not just those that use LIMIT. If you had page of results that was all-inclusive, with ambiguous sort, that URL now becomes non-cacheable, as the results could in theory be returned differently on subsequent page loads. The goal should be to make DataList deterministic, not to negotiate the random behaviour of your database platform.

are we able to infer default_sort_column from getting the PrimaryKey type fields in DataObjectSchema::databaseFields()?
Probably a bit too much magic for my taste, but it's an idea to bounce around.

Since I think this is just in reference to DataObjects (where the default_sort_column setting will exist), the ID column should also  always exist. Is it possible to change change a `DataObject''s primary key? I don't think so, but if it is, then maybe that'd be a concern.

How does it work in PHPMyAdmin?
I think we need to do more research. I found that only some tables return with duplicates between limited segments of the table, while other tables do not (e.g. SiteTree).

A challenge I've considered with a default_sort_column is that there needs to be a way to tell a datalist to reverse it in the case where a non-default sort column (E.g. Title) is reversed, otherwise groups of rows with the duplicate Title column would not reverse. That makes the solution not able to be applied so transparently.

IMPORTANT
A:
SELECT * FROM `SiteTree` LIMIT 20, 10

B:
SELECT * FROM `SiteTree` LIMIT 20, 10 ORDER BY ID

C:
SELECT * FROM `SiteTree` LIMIT 20, 10 ORDER BY `sort`

NOT sorting and sorting by ID are equally fast, but sorting by Sort is significantly slower ... see: https://github.com/silverstripe/silverstripe-framework/issues/7272
This evidence shows that sort by ID vs no sort key has no performance impact and sorting by ID is faster than sorting by the default_sort.

Yeah it's not just in sorting in ASC/DESC but also if/when you're using functions to sort (e.g. FIELD(...) or IF(...) and maybe some others, mentioned his above). I don't know what the solution to this would be since this is known behavior, generally speaking. You could always suggest people include a secondary sort column in the meantime (separate issue, update doc's). Depending on how users apply sort, it may be possible to infer the direction if you apply ID as the second column. Often time it's just a second parameter passed to ->sort('Sort', 'DESC'). It's when you get into custom sort queries that his coildnt be parsed reliably (you could try) but I think at that level the dev should have more control with less magic.
Ask: Yeah @sunnysideup, typically sorting by indexed fields is a bit faster than unindexed.

OK, there's about twenty screenpages of discussion here ;) If you care strongly about resolution, can somebody please summarise the state of the discussion with options going forward?

Summary:

When an ambiguous sort column is used, the database is free to return the rows in any order it wants, making the operation non-deterministic.
This makes pagination unpredictable (e.g. repeated entries across pages)
As an implementor of Sortable, DataList should sort deterministically.
Therefore, we should patch it
Forcing ID ASC into the query is too much magic. Something like default_sort_column would be more appropriate
The bug has existed at least since SS2, and no one raised it until now, so impact/low is fitting.

Forcing ID ASC into the query is too much magic. Something like default_sort_column would be more appropriate

I think we need to disambiguate the use (or the nomenclature) of default_sort_column since we already have default_sort which can define both column and direction. As such, since we're addressing a specific problem (deterministic sorting) I think we should take a specific approach that makes it a bit more transparent, both in nomenclature and in implementation.
So, I'd suggest we do the following:

Utilize ID as our secondary sorting column failover. These are all DataObjects and, as such, all will contain this column.
Still add a new config option, but instead of default_sort_column (which is somewhat ambiguous in meaning), we use the name such as: sort_deterministic, sort_secondary_id or  sort_failover (or similar).

Note: I'd stay away from the default_sort_ prefix since we're not defaulting anything per se (as you'll see below). Instead, we're enabling/disabling a deterministic sorting algorithm at the database level, which (yes) does mean we have to inject extra fluff into the ORDER BY clause and (yes) this setting is enabled by default. However, this is a boolean (noted below)).

Per above (and discussion): This should be an on/off setting and (per above) this should default to true.
Direction: We can automatically adjust the direction of secondary ID sort failover by defaulting to ASC in our algorithm and then parsing the existing order setting in our query prior to final execution for a direction and, if not blank/ASC, we can then set it to DESC.
Disabling/overriding:  This can be done one of either one or two ways (uncertain on second):

Override the sort_deterministic setting and set it to false.
Parse final order clause for any secondary sort fields and, if they exist, remove.

NOTE: I'm unsure if this is necessary, as you can have as many additional sort columns as you like (as far as I'm aware) and a developer with the confidence of adding a secondary sort column may already have a very unique sort, so adding ID may be redundant for several reasons but not necessarily negative or have an impact on performance.
At minimum, if we retain ID even if a secondary column is manually provided, we may still want to do a quick de-dupe check to ensure this ID column isn't already specified (to prevent accidentally overriding a contrary direction, e.g. DESC instead of ASC).

Beyond that, I think we'd be set. Thoughts?

I agree with @patricknelson that having another config setting called default_sort_something is ambiguous and conflicts with the existing default_sort, but the idea is the same - to have a failover column to enable deterministic sorting.
Having ID as a default for whatever this setting is called is a good idea, as all DataObjects have this (maybe rare exceptions).
If the idea can be kept simple such as a single config private static $determined_sort = ['ID' => 'ASC']; and that can be extended/disabled as desired by developers through php or yml that would be good - yes you can set a config to false or null in yml in ss4.

I think some research may be useful:

what are the consequences of adding a sort by ID ASC to all sql statements (after any other sort statements - if any), so that you always have a predictable outcome.  This would be my solution and apart from performance reductions, I can't see any other reason why this is not a good solution.

I noticed that dataobjects do sort randomly but pages do not and I wonder why that is (maybe to do with joins?)

Also - while no one has noticed this issues, this is not a benign one.  At least two of my clients noticed it (and were unable to edit data because of it) in gridfields (where I had set the sort order to a field that was something like a date field).

Adding an index to SiteTree.Sort should be a no-brainer AFAIK.

Actually, what if we just had a default_sort_secondary field? This could not only be consistent with the existing default_sort field, but can be automatically defaulted to ID ASC. Let me explain some scenarios and why I think this might work.
Functionality:

Default value for default_sort_secondary is  ID ASC.
Behind the scenes, the secondary column and direction are parsed separately before being handed to the database. This way, we can handle scenarios when the user begins to customize sorting. The sort direction is just an initial value that we start with and then negate when the user negates the initial direction. That way if the user ever selects another primary sort column or ever chooses to negate the direction, we just negate our initial direction. No magic/mystery to this, just keep it logical. So, we're just shifting perspective. Logically, if you reverse the current sort, the secondary sort must also be reversed as well for consistency.
If default_sort is defined with a secondary sort value, that secondary sort will be used. e.g. Date ASC, Title ASC will imply a default_sort_secondary value of Title ASC.
Conflict resolution: If current sort column (e.g. Title) is equal to the above resolved secondary sort, secondary sort fails over to the ID column (since we know ID will always be unique).

This satisfies:

Nomenclature consistency (default_sort_secondary as an extension to default_sort).
Sane initial defaults which are less magical (e.g. if the user sets up a secondary sort column in default_sort, this is parsed/retained in default_sort_secondary and functionality is extended).
Logical, consistent and deterministic results for pagination.

What do you guys think about that? Am I missing any scenarios or edge cases, possibly?

Just skim read through this delightful thread. While the suggested feature would provide some benefit for some very specific use cases, it sounds like it's somewhat low value for most common scenarios. Those people who would benefit from this have a relatively easy alternative (just manually include a ID sort in their queries).
My instinct would be to close this issue has a "won't fix" since there's no clear agreed upon action to take.

Reread this. That would be fine with me as it's still possible (as the developer) to manually define the secondary sort column via ::$default_sort. However, how do you (as SilverStripe) conclude this and resolve @unclecheese's point that DataList is intended to be deterministic, per https://github.com/silverstripe/silverstripe-framework/issues/7249#issuecomment-321949846?
That is ultimately what I think this issue morphed into, which makes it more broad and less about a specific use case like you'd find just in GridField pagination (or whatever). It's probably still low priority since while it may occur frequently, it's easy to fix (i.e. via best practices @tractorcow mentioned), but would still represent an important low level change, considering the breadth of it.
So to summarize: Is DataList deterministic, despite the underlying database layer or not?

FWIW I experienced this bug on SS4 today with 'out of the box' models - in the Security->Users section. A site had many members with no first or last name, so I got repeat/missing members on subsequent pages. Looks like it would remain an issue on SS5.
Putting aside the bigger debate for now, quick win would be to fix up all core models to ensure consistent sorting (e.g. add ID and maybe Email to Member default_sort), and maybe update the docs to flag this issue as suggested previously?

Think the scope is bigger than the title suggests as it's not just GridField (it's all DataLists) and not just about default_sort. Even if you fix your default_sort to include ID, that won't come for the ride if you choose a different sort by using the sortable headers that are baked in to Silverstripe CMS.
Another pain point is that it doesn't appear that you can sort on multiple fields with the GraphQL module. sort expects on object instead of an array so passing multiple fields already wouldn't let you specify which field should come first, but in any case if you do it seems that only the field name that's alphabetically last is used for the sort. So overall this doesn't appear to be an issue that developers can easily workaround by tweaking a few sort clauses in their own code.

BUMP BUMP BUMP


error removing old index fe518b69: blob not removed, server response: 403 Forbidden (403)

Hi, I have the following problem.
At the time of making a check of the snapshots I find the following messages and suggests running restic rebuild-index
pack 6ebe5339 contained in several indexes: {16b3e9b6 611c4a4a 633867da 6956e51f 84b15cee 912fdbb2 b9869b9d bff7b50f c5642da1 d871110b f29e3b72}

run `restic rebuild-index' to correct this
check all packs
check snapshots, trees and blobs
no errors were found

Then when executing the rebuild-index this type of messages appear.
Remove(<index/0013fb40cb>) returned error, retrying after 558.730644ms: blob not removed, server response: 403 Forbidden (403)

I do not know if this topic has already been discussed here, but I need help with this.
PD: I apologize for my english if something was not understood.

Which backend are you using? What are the permissions of the file (and its parent directories)?

I use the backend  in a docker container.
Version rest-server 0.9.4
and the permission off data direcctory
-rw-------    1 root     root         155 Nov 14 02:00 config
drwx------  258 root     root        8.0K Nov 14 02:00 data
drwx------    2 root     root       28.0K Mar 22 15:51 index
drwx------    2 root     root          78 Nov 14 02:00 keys
drwx------    2 root     root           6 Mar 22 16:53 locks
drwx------    2 root     root       24.0K Mar 22 12:40 snapshots

The restic client version is restic 0.7.3

I tried also with version 0.8.3

Hey, you did not fill out the issue template. Would you mind doing that? It is rather rude to not fill it out and let us query you for every detail.
From your description (without having any details) it sounds like you've configured the REST server to use a password, and you're not providing this in the rebuild-index run. But I'm guessing here, please fill out the issue template.

I'll close this issue for now, please fill out the issue template, then we can reopen this issue and help you. Thanks!

Output of restic version
restic 0.7.2
compiled with go1.9 on linux/amd64

How did you run restic exactly?
export RESTIC_SERVER=<IP_ADDRESS>
export RESTIC_PORT=9002
export RESTIC_USER=mkmi
export RESTIC_PASSWORD=[pass]
export RESTIC_REPOSITORY=rest:http://$RESTIC_USER:$RESTIC_PASSWORD@$RESTIC_SERVER:$RESTIC_PORT/

# restic snapshots

# restic forget 3fab1cec

What backend/server/service did you use to store the repository?
The backend is running over a docker container
rest-server 0.9.4 compiled with go1.8.3 on linux/amd64

Expected behavior
# restic forget 3fab1cec

Delete the snapshot with ID 3fab1cec
Actual behavior
# restic forget 3fab1cec
blob not removed, server response: 403 Forbidden (403)
github.com/restic/restic/internal/backend/rest.(*restBackend).Remove
        src/github.com/restic/restic/internal/backend/rest/rest.go:283
main.runForget
        src/github.com/restic/restic/cmd/restic/cmd_forget.go:124
main.glob..func8
        src/github.com/restic/restic/cmd/restic/cmd_forget.go:24
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).execute
        src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:650
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).ExecuteC
        src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:729
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).Execute
        src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:688
main.main
        src/github.com/restic/restic/cmd/restic/main.go:69
runtime.main
        /usr/local/go/src/runtime/proc.go:185
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:2337

Steps to reproduce the behavior
export RESTIC_SERVER=<IP_ADDRESS>
export RESTIC_PORT=9002
export RESTIC_USER=mkmi
export RESTIC_PASSWORD=[pass]
export RESTIC_REPOSITORY=rest:http://$RESTIC_USER:$RESTIC_PASSWORD@$RESTIC_SERVER:$RESTIC_PORT/

# restic snapshots
ID        Date                 Host        Tags           Directory
----------------------------------------------------------------------
3fab1cec  2018-03-05 00:01:38  mkmi-vps1   mkmi-vps1      C:\Backups
09e028ce  2018-03-06 00:01:33  mkmi-vps1   mkmi-vps1      C:\Backups
1510eb58  2018-03-07 00:01:37  mkmi-vps1   mkmi-vps1      C:\Backups
dce1bf42  2018-03-08 00:01:32  mkmi-vps1   mkmi-vps1      C:\Backups
2248928e  2018-03-09 00:01:34  mkmi-vps1   mkmi-vps1      C:\Backups
1107cdb0  2018-03-10 00:01:32  mkmi-vps1   mkmi-vps1      C:\Backups
dc1a95c8  2018-03-11 00:01:26  mkmi-vps1   mkmi-vps1      C:\Backups
fc9cf84b  2018-03-12 00:01:32  mkmi-vps1   mkmi-vps1      C:\Backups
5eac0941  2018-03-13 00:04:46  mkmi-vps1   mkmi-vps1      C:\Backups
f9eb5096  2018-03-14 00:01:25  mkmi-vps1   mkmi-vps1      C:\Backups

# restic forget 3fab1cec
blob not removed, server response: 403 Forbidden (403)
github.com/restic/restic/internal/backend/rest.(*restBackend).Remove
        src/github.com/restic/restic/internal/backend/rest/rest.go:283
main.runForget
        src/github.com/restic/restic/cmd/restic/cmd_forget.go:124
main.glob..func8
        src/github.com/restic/restic/cmd/restic/cmd_forget.go:24
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).execute
        src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:650
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).ExecuteC
        src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:729
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).Execute
        src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:688
main.main
        src/github.com/restic/restic/cmd/restic/main.go:69
runtime.main
        /usr/local/go/src/runtime/proc.go:185
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:2337

Do you have any idea what may have caused this?
This because when executing restic check and i don't have errors
# restic check
Create exclusive lock for repository
Load indexes
Check all packs
Check snapshots, trees and blobs

Do you have an idea how to solve the issue?
This happens with several repositories that I have configured
I waiting for your awnsers. Thanks

Hey, thanks for filling out the issue template. How do you run the rest-server? Can you paste the command-line?
Can you please check:

Is the rest-server running with --append-only?
Does the system user the rest server runs as have permissions to delete files in snapshots/?

Is the rest-server running with --append-only?
I used a docker container with this options.
# docker run -ti -d \
	--name rest-mkmi \
	--restart=always \
	-p 9002:80 \
	-v /mnt/rest-mkmi/:/data \
	-e OPTIONS="--append-only" \
	restic/rest-server

# docker exec -it rest-mkmi create_user mkmi
# docker restart rest-mkmi

Does the system user the rest server runs as have permissions to delete files in snapshots/?
I saw this through a docker portainer console
# ls -hl
total 24
drwxr-xr-x    7 root     root         105 Jan 11 12:53 OLD
-r--------    1 root     root         155 Jan 11 13:00 config
drwx------  258 root     root        8.0K Jan 11 13:00 data
drwx------    2 root     root        4.0K Apr  2 04:18 index
drwx------    2 root     root          78 Jan 11 13:00 keys
drwx------    2 root     root           6 Apr  2 14:40 locks
drwx------    2 root     root        4.0K Apr  2 04:18 snapshot

About the password
Yes, it is intentional. Because it facilitates the deployment and because we handle many passwords.  a more password can be complicated.
Thank again for you reply.

Ok, this is solved: You've run rest-server in append-only mode (with --append-only). It instructs the server to only allow adding data, no data can be deleted. The idea behind this mode is that when attackers gain access to a machine, they cannot destroy old backup data, but restic is still able to add new data.
For being able to run forget and prune, you need to either access the repo directly, or via a rest-server started without --append-only.
I'm closing this issue as it is solved. Please feel free to add further comments. Thanks!

Thank.  I modified this option. I didn't know completely how the option worked.
Solved!!


base-64 response did not display

Originally reported on Google Code with ID 193
What steps will reproduce the problem?
1. Our REST resource returns a JSON array with 2 variables. A status string and a base64
encoded PDF.

Expect the JSON array. The same function works in the REST Console app. Instead the
response header populates, but the output gives me a green PacMan icon that does not
disappear.

On what operating system, browser and browser version?
Win 7 x64, Chrome 29.0.1547.0 dev-m

Please provide any additional information below.
This bug has existed since I started using the program about a year ago.

Reported by johnschram on 2013-07-22 21:10:40

I believe it is fixed already since I can't reproduce the problem.
There was some major changes in this module and then the problem was fixed.


Remove legacy plot_psd* funcs/methods in our docs and codebase

closes #11554
@larsoner WDYT about the changes to Report in 9b481b71eda44871ac267a203a554993c1735b3e? The kwargs used to be passed to plot_psd, now are passed to spectrum.plot().  Needs a deprecation cycle?

I've compared all the spectrum plots in the edited examples/tutorials against the corresponding plots in our dev docs, and they all look unchanged.  This is ready for review.


⚠ Bofrost.hr has degraded performance

In 7e36cdd, Bofrost.hr (https://www.bofrost.hr) experienced degraded performance:

HTTP code: 200
Response time: 3050 ms

Resolved: Bofrost.hr performance has improved in 0285207 after 2 minutes.


error importing package

I have tried both import methods
import CanvasScrollClip from 'canvas-scroll-clip';

const CanvasScrollClip = require('canvas-scroll-clip').default

They result in the same error
/home/kaleb/code/ttf-cms/node_modules/canvas-scroll-clip/dist/esbuild/main.esm.js:393
export {
^^^^^^

SyntaxError: Unexpected token 'export'
    at internalCompileFunction (node:internal/vm:73:18)
    at wrapSafe (node:internal/modules/cjs/loader:1149:20)
    at Module._compile (node:internal/modules/cjs/loader:1190:27)
    at Module._extensions..js (node:internal/modules/cjs/loader:1280:10)
    at Object.newLoader [as .js] (/home/kaleb/code/ttf-cms/node_modules/pirates/lib/index.js:141:7)
    at Module.load (node:internal/modules/cjs/loader:1089:32)
    at Module._load (node:internal/modules/cjs/loader:930:12)
    at Module.require (node:internal/modules/cjs/loader:1113:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (/home/kaleb/code/ttf-cms/components/Media/index.tsx:15:62)

Node.js v18.14.0

This is the project I am using:
https://github.com/KalebKloppe/nextjs-custom-server

You are trying to use it without considering it a client-side component. Use 'use client'; on top of the files that are using it and then it will work.

You are trying to use it without considering it a client-side component. Use 'use client'; on top of the files that are using it and then it will work.

I was using the legacy pages router, not the App router. No "use client"

@KalebKloppe did you managed to solve this?


Update Status display

The status currently shows as green no matter the status.
Colour-coding the status label will help to clearly display the application's progress.
Let's format the status label to change colour depending on the status.

Codecov Report
Base: 72.49% // Head: 72.22% // Decreases project coverage by -0.26% :warning:

Coverage data is based on head (f01d01a) compared to base (017f128).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##             master      #70      +/-   ##
============================================
- Coverage     72.49%   72.22%   -0.27%
  Complexity      431      431
============================================
  Files            74       74
  Lines          1345     1350       +5
  Branches        147      147
============================================
  Hits            975      975
- Misses          326      331       +5
  Partials         44       44

Impacted Files
Coverage Δ

src/main/java/seedu/intrack/ui/InternshipCard.java
0.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add and enforce editorconfig

Add the editorconfig from the dotnet/docs repo so this matches our stated style.
Then, fix or suppress any warnings generated.
@adegeo I suppressed the one warning in repoman because I think its use is required by the GH API. If there's a way to fix it rather than suppress it, let me know.

Correct, they use SHA1 encryption on the signature, so AFIAK no way around that.


Resolve #218, Resolves kubernetes-client/python#1335

Fixed the bug from https://github.com/kubernetes-client/python/issues/1335

added exception to JWT validation
added a rule to fix the url unsafe characters
changed base64.b64decode to base64.urlsafe_b64decode for correct oidc token encoding

I already have a CLA


Record battery information on watchOS

Goal
Records the battery level and charging status when the app starts.
Since watchOS doesn't have a notification API for battery like iOS does, only the initial status is recorded.
Testing
Manually tested on all platforms.

Can we go back to something closer to the original version of this PR?
The latest version (with BSGHardware) adds over 1KB to the binary size whereas the original added 0.

The naming convention for functions in Bugsnag is UpperCamelCase.
snake_case should be restricted to code that lives under /Bugsnag/KSCrash.


Origins requires 0.4.1 version or higher crash

Could not find required mod: origins-classes requiers {origins @ [>=0.4.1]}

update origins?

@noobplayerz pls close this its just a depency warning.


Replaced dex frontend logo with correct URL.

Used github raw image to correctley load raspbernetes logo.
Signed-off-by: anthr76<EMAIL_ADDRESS>Description
Currently in Dex the non raw image was being loaded. If you are seeing the logo currently it's likely it was cached by cloudflare. Current usage without raw prevents the logo from being loaded onto the dex front end.
Checklist

[X] All commits are signed off. See the section Developer’s Certificate of Origin
[ ] Provide a title or release-note blurb suitable for the release notes.
[ ] All pre-commit hook validation passed successfully.

[ ] All workflow validation and compliance checks are passing.

Issue Ref (Optional)
Which issue(s) this PR fixes (optional, using fixes #(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged): Fixes #
Notes
Add special notes for your reviewer here.

Greatly appreciated! Thanks for raising this change @anthr76 🎉


PooledByteBufAllocatorTest#testThreadCacheToArenaMappings random failure

Just happened on my machine: OSX, Hotspot 1.8.0_112
<<< FAILURE! - in io.netty.buffer.PooledByteBufAllocatorTest
testThreadCacheToArenaMappings(io.netty.buffer.PooledByteBufAllocatorTest)  Time elapsed: 0.215 sec  <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
	at io.netty.buffer.PooledByteBufAllocatorTest.testThreadCacheToArenaMappings(PooledByteBufAllocatorTest.java:238)

interesting... never saw this. Can you reproduce it in a regular fashion @slandelle ?

I haven't investigated nor tried to reproduce yet. It happened while working on an unrelated PR on codec-http (so built downstream this failing test). I wanted to report before I might forget.

will check

Ok I think what is wrong... But no worries its just a race in the test. Looking into it now.

cool :)

@slandelle fixed by https://github.com/netty/netty/pull/6077

@normanmaurer Yeah, I saw that :) Great job!


[BUG] Can't build - ModuleNotFoundError: No module named 'parse_config'

I cloned the repo and opened NeuralAmpModeler-macOS.xcodeproj in Xcode 14.3.1 (MacOS 14.0, M2). When I try to build, I get the following error:
Traceback (most recent call last):
  File "/_Plugins/NeuralAmpModelerPlugin/NeuralAmpModeler/projects/../scripts/update_version-mac.py", line 19, in <module>
    from parse_config import parse_config, parse_xcconfig
ModuleNotFoundError: No module named 'parse_config'
Command PhaseScriptExecution failed with a nonzero exit code

I'm sure I'm missing something. I tried creating a new conda environment and pip installing https://github.com/sdatkinson/neural-amp-modeler, which seemed to install dozens of packages, but when I run update_version-mac.py from that env I get the same error.

You don't need the trainer to build this--all of the dependencies are self-contained.
I think the issue is that you didn't properly clone the submodules used by this repo.
In short, clone with git clone --recurse-submodules [path].
Look up git submodules if you're unfamiliar with them.
Let me know if that works.

Haven't heard back; closing since I assume this did it


Reloading pmtiles header on file change

Before working on a patch I wanted to have your opinion.
In the context of the Caddy pmtiles plugins: it's using the modification time of the file to compute the etag (at least in the file implementation but same for s3 https://github.com/protomaps/go-pmtiles/blob/main/pmtiles/bucket.go#L121 ),  but it seems nothing is reloading the header, then the tile requests are pointing to wrong offsets returning bogus data.
Since we can detect the file changed we could reload and set the proper offsets?
Note while following the code I'm not sure the cache is working correctly if you host multiple pmtiles since the key is just formed on the zxy without the path?
Thanks!

nothing is reloading the header

The JavaScript client should detect the new etag. https://github.com/protomaps/PMTiles/blob/main/js/index.ts#L1019
What client are you using where this isn't working?

Note while following the code I'm not sure the cache is working correctly if you host multiple pmtiles since the key is just formed on the zxy without the path?

The key is part of the cache: https://github.com/protomaps/go-pmtiles/blob/main/pmtiles/server.go#L19

Sorry the lack of context, I'm using the xyz tiles served by caddy and the pmtiles plugin, in that context the js client does not have access to the pmtiles headers.
Only the server can detect the file modification time changed (it does) but it is not reloading the correct seek position, so it returns back garbage.

OK, that sounds like a bug. Just to be sure, what OS are you using?

OS is Linux, it has been observed and reproduced while using the Azure blobstorage from the gcloud library, but from the code I think it would happen on any sources.

So are you observing this only on serving archives stored on Azure Blob, or also on the filesystem?
@msbarry may have mentioned in the PR that Azure etags aren't supported yet: https://github.com/protomaps/go-pmtiles/pull/130

thanks! This is a serious lead, I tested it again with the local file adapter and you are right it worked, the etag changed and the correct file is returned.

Basically this line is responsible for turning the abstract Bucket into a provider-specific one. Only the logic for S3 is implemented correctly right now: https://github.com/protomaps/go-pmtiles/blob/main/pmtiles/bucket.go#L216
So we would need to check if the low-lying azblob library supports If-Match, and if so populate it there.

I've tested this PR on the local azure blob emulator, will do more testing against the real storage and let you know.

Cut a release in https://github.com/protomaps/go-pmtiles/releases/tag/v1.21.0


How to set bounds in mvp_grasp.yaml?

Hi,
I'm confused about bounds in mvp_grasp.yaml. I try to change them but either I get out of bounds error or some dimension miss match error.
I use  this_pose = self.active_group.get_current_pose() to find my desire pose and save it as
self.active_group.remember_joint_values('grip_ready', this_pose)
If I keep your setting for bounds the robot moves but very far from my  grip_ready location.
Have you hard cord the searching position somewhere in the code?
In panda_base_grasping_controller.py why do you have two initial position?
self.pc.goto_named_pose('grip_ready', velocity=0.25)
start_pose = list(self.pregrasp_pose)
start_pose[0] += np.random.randn() * 0.05
start_pose[1] += np.random.randn() * 0.05
self.pc.goto_pose(start_pose, velocity=0.25)
The way understood the bounds is set around grip_ready position. I can make this two working but out of bounds error seems to come from update_grid.
Any suggestion?

The histogram bounds in mvp_grasp.yaml are defined in world coordinates in the robot base frame.  In my case, I believe the workspace was to the right of the robot (negative y direction), so if you are working in front of the robot (positive x direction) then you will need to adjust them accordingly.
It is also expecting the area to be a square, if that helps.


updated for newest prerelease

Docs Build status updates of commit 86000b0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-edge/webview2/releasenotes.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit d9d57d6:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-edge/webview2/releasenotes.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit f8fe212:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-edge/webview2/releasenotes.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Event will self trigger when element create dynamically

I bind touch:tap on my list item and when item rendered, the tap event self triggers
<template>

    <dir-ser
      class="navigate"
      :path.sync="syncdirpath"
    ></dir-ser>

{{ item.name }}

</template>

 private itemclick(event: any, item: serverItem) {
    switch (item.type) {
      case FileType.file:
        this.cmd(item);
        break;
      case FileType.directory:
        this.cd(item.name);
        break;
      case FileType.link:
        break;
    }
  }

What was your solution to this?

@andorfermichael  See this https://github.com/jerrybendy/vue-touch-events#how-to-add-extra-parameters

Hi Jerry, Nice Library! Thanks for providing it to the community :) I have a usecase where I am using virtual scrolling, as well as intersection observers as sometimes I am dealing with 1000s of elements. These get dynamically created as you scroll around. This leads to a lot of misfires which while I can probably filter them out in a method, I assume isn't great for performance, is there any way to make this only trigger on a touch event?

@RichardCr  I'm not understand you very clearly. Do you mean the disableClick in configuration?
If not, you can open a new issue and give me some more details.


Fix exception raised by subgoal

Coverage remained the same at 80.978% when pulling 0593f6c0474bcc73ed70e43e742412e158e28715 on fix/subgoals into ecd8c267faf361a9140f63edec136c9528370106 on master.

Coverage remained the same at 80.978% when pulling bf7edc4897026ec83f6da16cbc6231ec3ee4ad63 on fix/subgoals into 3ce50ecff210d3c158843e6c3109258004f07122 on master.

Coverage remained the same at 80.978% when pulling 81b04e7c71ed52a553e0048bbb91f64a05ba63f0 on fix/subgoals into 8f82768235438a21e476fde7ca48fe12452ed55e on master.

Coverage remained the same at 80.978% when pulling 6f936e0abaa09a096e6942a4f473a21cf9ceb279 on fix/subgoals into f983f47e60c937e4e0e747a6e921748479e6d688 on master.


Implement .NET Core 3.0 GC APIs

Is there anything left for this after #7517?

Yes, there's https://github.com/dotnet/corert/blob/b62a807cf8a0fd59c87caa6bdc6c5465f2c4c98d/src/System.Private.CoreLib/src/System/GC.cs#L635-L639 left to be implemented, which i've skipped because it's non trivial (requires SuspendEE and RestartEE which seems only defined on the GCToEEInterface).
Not directly related to this issue but to the GC class is https://github.com/dotnet/corert/blob/b62a807cf8a0fd59c87caa6bdc6c5465f2c4c98d/src/System.Private.CoreLib/src/System/GC.cs#L661-L664
which calls into a fair amount of vm code so i've skipped that as well.

Ah, I see. Didn't realize this issue was reused for those. I renamed it to reflect that.

Also i don't know the exact difference between InternalCall, RuntimeExport and DllImport and their equivalent runtime declarations. InternalCall seems to be a cooperative mode call and is defined with the COOP_PINVOKE helper macro. RuntimeExport appears to use cdecl but also cooperative mode and DllImport might use cdecl and preemptive mode?

Yes, you got it right.

Is there a guideline when to use which? In particular between InternalCall and RuntimeExport, which only seem to differ in fastcall vs. cdecl. Are there things i'm not allowed to do in an internal call, like switch to preemptive mode?

RuntimeExport is method implemented in C# that is called from C/C++ runtime code. InternalCall is method implemented n C/C++ that is called from C#.

Are there things i'm not allowed to do in an internal call, like switch to preemptive mode?

Correct, switch to preeemptive mode is not allowed in internal call in CoreRT.


fix: hover on cards

Description
Fix hovering effect on cards.
Fixes #129
Type of change
Please delete options that are not relevant.

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Responsive Testing?
(Only applicable for Frontend changes)
Checked for: (width in px)

[X] Laptop L - 1440px
[X] Laptop - 1024px
[X] Tablet - 768px
[X] Mobile M - 375px

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my code
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in modules

Hey @christianfds, sorry for replying late.
Can you please re-check this once. The "go to contests" button stopped working

Not sure why the button isn't working just by the card.css changes

lol, that weird, I will take a look at it. The ::before tag isn't behaving as I expected

That should fix it @pranshugupta54

Looks great now!
Sorry for not checking this PR on time


Can't resize address bar

Hello,
Very nice theme and thanks for sharing it with the community.
One thing I am trying to achieve is to resize the address bar.
Do you have any idea how to achieve it in this setup? At the moment it's really narrow (as it appears in your images) and I'd like it to take up more space in the middle of the window

6th line of the userChrome.css defines the starting point of the navigation bar contents.
  --uc-window-control-width: 898px; /* Space reserved for window controls */
Decreasing it will cause extension icons to shift right. The address bar will expand automatically.

Pad the address bar to shift it. It can be centered this way.

Here is a mirrored look:

Or big chungus:

Incredible. Thanks a ton. I tried increasing this space but didn't think that I need to decrease it. It worked just fine.


Add nanoflow property

Codecov Report

Merging #5 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #5   +/-   ##
=======================================
  Coverage   95.45%   95.45%
=======================================
  Files           2        2
  Lines          44       44
  Branches       15       15
=======================================
  Hits           42       42
  Misses          2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea5b456...0e0c84c. Read the comment docs.


feat: Implement optimized TreeMap iterator functions

Closes #426
This PR just makes the implementation more efficient, to avoid unnecessary storage reads
This should be functionally the same externally, but since it will reduce storage reads it's a breaking change in a smart contract context

Probably makes sense to add some smoke tests here. There was a funny bug in std when a combination of nth and rev flat out gave wrong results in some cases, and nobody noticed )

Probably makes sense to add some smoke tests here.

not related to the PR per-se, but it'd be cool to look into setting up property-based/fuzz testing for collections. std collections give us a good, free oracle we can compare against. The two relevant crates in the rust ecosystem are (from most important to least importat)

https://lib.rs/crates/arbitrary
https://lib.rs/crates/quickcheck
https://lib.rs/crates/proptest

Which reminds me that I should dig into https://github.com/graydon/photesthesis at some point...

Probably makes sense to add some smoke tests here.

not related to the PR per-se, but it'd be cool to look into setting up property-based/fuzz testing for collections. std collections give us a good, free oracle we can compare against. The two relevant crates in the rust ecosystem are (from most important to least importat)

lib.rs/crates/arbitrary
lib.rs/crates/quickcheck
lib.rs/crates/proptest

Which reminds me that I should dig into graydon/photesthesis at some point...

Yeah, I added some basic tests, but yeah fuzzing probably is out of the scope of this. I'd love to tackle this when I have time.
I assume when you mention arbitrary you mean to be used with cargo-fuzz? quickcheck is already used in the sdk for tree map tests, so probably least cost to add those to be run as tests, but definitely would be nice to extensively fuzz all data structures and interactions.

Not necessary -- I believe that the arbitrary crate is setup in such a way that is useful even without coverage-guided fuzzers, so it should be possible to use it the same way quickcheck is used, via simple #[test]. I don't think the ecosystem is quite there yet, so it might be that we'll have to do some ground-work here ourselves. arbitrary does seem to be a very well-designed crate that captures the essense of randomized testing, so I'd like to find a way to marry convenient tests with it.

And the benefit of doing this over using quickcheck? More control?

And the benefit of doing this over using quickcheck? More control?

The arbitrary crate is a good ecosystem-wide interface. I expect that, over time, a lot of other crates would implement arbitrary::Arbitrary. So, if we make our stuff Adbitrary, then we'll be more inter-operable.
That's I am betting on arbitrary: it's a 1.0 crate which defines just the narrow interface for randomization, without being opinionated about how actual property testing is commenced.

The arbitrary crate is a good ecosystem-wide interface. I expect that, over time, a lot of other crates would implement arbitrary::Arbitrary. So, if we make our stuff Adbitrary, then we'll be more inter-operable.
That's I am betting on arbitrary: it's a 1.0 crate which defines just the narrow interface for randomization, without being opinionated about how actual property testing is commenced.

You okay with this being done in a separate PR? I can create an issue later.
Rather not increase the scope of this PR if possible

You okay with this being done in a separate PR? I can create an issue later.

Sure, I am always for splitting up PR, and for non-blocking PRs!


fix wrap intrinsic height calculation

Description
Possible fix to wrap intrinsic problem
Related Issues
https://github.com/flutter/flutter/issues/61204
https://github.com/flutter/flutter/issues/63302
Tests
I added the following tests:
TBD
Checklist
Before you create this PR, confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[ ] I signed the CLA.
[ ] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] All existing and new tests are passing.
[ ] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[ ] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Did any tests fail when you ran them? Please read Handling breaking changes.

[ ] No, no existing tests failed, so this is not a breaking change.
[ ] Yes, this is a breaking change. If not, delete the remainder of this section.

[ ] I wrote a design doc: https://flutter.dev/go/template Replace this with a link to your design doc's short link
[ ] I got input from the developer relations team, specifically from: Replace with the names of who gave advice
[ ] I wrote a migration guide: https://flutter.dev/go/breaking-changes-template Replace this with a link to a pull request that adds the migration guide to https://flutter.dev/docs/release/breaking-changes

ready for another look @goderbauer @Piinks


Remove dependency on rustc bootstrap

Description of changes:
Replace the last call to bootstrap by direct call to cargo. I also did a few cosmetic changes to our scripts to use target packages instead of changing the current directory.
Resolved issues:
Resolves #641
Call-outs:
The initial version of this PR includes two commits, but only the second commit is relevant.
The first commit is an update to the toolchain which is covered in this PR (#764) but it is required for the second one. Once I merge the original PR, this should go away.
Testing:

How is this change tested? N/A

Is this a refactor change? Maybe

Checklist

[ ] Each commit message has a non-empty body, explaining why the change was made
[ ] Methods or procedures are documented
[ ] Regression or unit tests are included, or existing tests cover the modified code
[ ] My PR is restricted to a single feature or bugfix

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

This PR seems to include the same changes as PR #764

This PR seems to include the same changes (as a superset) as PR #764

You are totally correct! Please see the call-outs section. It is included because it is a requirement. But #764 should be merged first.

This PR seems to include the same changes (as a superset) as PR #764

You are totally correct! Please see the call-outs section. It is included because it is a requirement. But #764 should be merged first.

Yup, I understand it now. Thanks!


Update ms.topic values

Docs Build status updates of commit b47a7a3:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/console-functions.md
:white_check_mark:Succeeded
View

docs/console-reference.md
:white_check_mark:Succeeded
View

docs/console-structures.md
:white_check_mark:Succeeded
View

docs/using-the-console.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

@miniksa - updating ms.topic values per IM thread, please merge.


Implement grid concatenation and standardize datatype casting

This PR has two main aims:

Standardize and simplify how casting between different datatypes works
Implement concatenation for all datatypes

Before this PR was applied past both casting and concatenation were limited to columnar data formats, which meant that certain operations could not be applied to gridded data, e.g. a HoloMap collapse. Having a dedicated concat implementation for both columnar and gridded data also allows much more efficient concatenation than what is currently in use by methods like .table and .dframe and will generalize them so that we can eventually replace the column specific .table implementation with a general one that returns a dataset of arbitrary type.
Implementing concatenation along HoloMap dimensions also means that Dataset.groupby operations are now reversible and fixes HoloMap.collapse.

[x] Fixes https://github.com/ioam/holoviews/issues/1417
[x] Adds unit tests

Made some additional comments to clarify certain implementation details.

Ready for review.

Other than a few minor comments this looks good and I'm happy to merge.

Tests are green. Merging.


linter privatelink.database.windows.net

Bicep version
Bicep CLI version 0.11.1 (030248df55)
Describe the bug
I am gettign this warning:

Environment URLs should not be hardcoded. Use the environment() function to ensure compatibility across clouds. Found this disallowed host: "database.windows.net"bicep core https://aka.ms/bicep/linter/no-hardcoded-env-urls

However it's not clear to me how to remedy - the linked document says I should use environment() instead - but upon inspecting that I can't see how I get the database url from environment():

Which property above do I use to obtain this value?
To Reproduce
As per above
Additional context

I fell into this too, it would be really nice if the linter could tell us exactly what we should be replacing the host with.
@dazinator In this case you need environment().suffixes.sqlServerHostname which should work for you
There's a little bit of info here about what properties are returned: https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-functions-deployment#return-value-1

Glad we found the right property. The larger issue of auto-fix is covered in #2999, so I'm going to close this.


Implement selection sort using python

Fixes issue #1 partially.
Let me know for any changes. Also please do add hacktoberfest-accepted label to it.

@AdityaDKale Make the program take the whole input at once like in the problem statement, instead of taking input one by one.

@Chitresh-code Thanks for your comment. Please check if following code suits your needs..

@AdityaDKale Yes it works, thanks for your contribution.
Happy Hacktoberfest!


TSServer: Refactor trigger reason

Follow up on #34930
Problem
VS Code lets user create keybindings for specific refactorings like extract constant. At present, TS Server does not know if refactorings are requested explicitly by the user using a keyboard shortcut or if they are requested automatically when the user simply makes a selection in the current document.
This distinction is important for a few reasons brought up in #34930:

There are cases where we do not want the code action lightbulb to show up in VS Code but refactorings still should be availible if the user requests them
The TypeScript server could be less strict about the selection for manually requested refactorings. For example, if I request to extract constant with no selection, TS Server could assume that I want to extract the nearest expression.

Protocol Proposal
In the getApplicableRefactors request, add a new triggerReason field:
interface GetApplicableRefactorsRequestArgs extends FileLocationOrRangeRequestArgs {
    triggerReason?: RefactorTriggerReason;
}

The trigger reason would capture why the refactoring was requested:
type RefactorTriggerReason = RefactorInvokedReason;

interface RefactorInvokedReason {
    kind: 'invoked';
}

At the moment, I think we only need a single trigger reason: invoked. I have based this design around the existing server API for parameter hints.
How TS Server could use triggerReason
As a first step, when refactorings are explicitly invoked by the user, if the current selection is not extractable, we should automatically expand the selection for extract constant and extract function to nearest extractable expression.
Possible extension: hierarchal refactorings
VS Code uses a hierarchal system to assign the class of code actions / refactorings. For example, extract constant has a CodeActionKind of refactor.extract.constant. Users can setup keybindings for refactor.extract (show all extract refactorings) or for refactor.extract.constant (show only the extract constant refactorings).
We should consider putting some form of hierarchical refactorings into the TS Server api as well. A few reasons this may be helpful:

If TS Server knows the type of refactorings being requested, it can only compute that subset of refactorings
In the future, we can specialize behavior for individual refactoring requests even more. For example, if a user explicitly requests extract constant refactorings for TS Server, we could show more than one expression around the current selection that can be extracted.

To support this in the server api, we would need to add a new another field on RefactorInvokedReason:
interface RefactorInvokedReason {
    kind: 'invoked';
    refactorType?: RequestedRefactoringType;
}

VS Code uses dotted string to represent a hierarchy of refactoring types (e.g. "refactor.extract.constant"). TS could also use this scheme. Or we could hardcode the hierarchy:
enum RequestedRefactoringType {
    Refactor,
    Refactor_Extract,
    Refactor_Extract_Constant,
    Refactor_Extract_Function,
    ...
}

I think the hierarchal refactorings for tsserver is something we should add for the reasons mentioned here. @minestarks

Can we close this since we added the refactor trigger reason, or does this item track the hierarchical refactoring as well?

From discussions, the TS team did seem interested in the hierarchical part of this proposal, so let's make sure we still track that, either with this issue or by opening a new one


Minor config improvements

Coverage remained the same at 94.69% when pulling c124ea2027cbcbac2de9eb917198954817c85a14 on falexandrou:config-changes into 4f8b64a6d8e4959e2aa95e8bdba3729f34a1134d on hirakiuc:master.

Sorry, I missed checking this pull request...
I already added access_token config in #106. 🙇
Please remove access_token config and fix conflicts. 🙏

Coverage increased (+0.004%) to 94.709% when pulling 12f0123d1c0c1e476ec7e43c8bf26a1f5676481b on falexandrou:config-changes into 5207b290afea5367c3868b8a115f92a1b589b133 on hirakiuc:master.

I fixed the conflict and LGTM.
Thanks!


Update resources link to eShopOnContainers repo

Summary
Update resources link to eShopOnContainers repo
Fixes #19245

@nishanil why did you merge the same PR created at a few hours ago, but skip this one?

@nishanil This PR is getting stale (and has conflicts now). Can you give it a review and check?

@Marusyk Apologies for overlooking this PR. And, Thank You for helping us out here.
We had internally triaged this bug and @sughosneo was working on this along with the others that were prioritized to go out that week. Closing this PR now.

@Marusyk Apologies for overlooking this PR. And, Thank You for helping us out here.
We had internally triaged this bug and @sughosneo was working on this along with the others that were prioritized to go out that week. Closing this PR now.

Thank you @nishanil for clarifying the details.
Yes @Marusyk, As a part of other changes, I have also fixed this item in a single PR without seeing the existing one of yours. Sorry about that. But keep sharing your feedback as issues or PRs. We will be happy to review and merge the changes in future. Thanks again for your help.


AOT compilation

Hi, I was just wondering if there had been any more thoughts on supporting AOT kernel compilation to allow execution outside of Python? Referencing https://github.com/openai/triton/issues/175

We are waiting on a rewrite to be done
See: https://github.com/openai/triton/pull/490#issuecomment-1238752684

And what will AoT compilation generate, a C/C++ API plus source/.so?

Great news, is there some branch/PR we can track the progress of this?

@ptillet I am very keen to have a go at using this feature whatever state the code currently is in, even if it is only the unit test you mentioned previously (have a time sensitive project which could benefit from AOT functionality)

We have a prototype that works with an old version of Triton. You might be able to hack it for your needs?
https://github.com/openai/triton/pull/490

We have a prototype that works with an old version of Triton. You might be able to hack it for your needs? #490

Great thanks @gaxler, will give it a go! For the main feature is there any WIP branch that can be tracked or is it separate from the main repo?

@gaxler should there be a correlation between the triton BLOCK_SIZE defined in the kernel definition, and the gX, gY, gZ defined in GridWarps when calling the kernel?

You mean add grid size constrains at compile-time?
In general I avoided dealing with anything related to kernel launches in the draft PR, its all just placeholders to make it run

Great thanks! I now have it working but have noticed the performance is much worse than the JIT triton equivalent. From the profile trace I see large gaps between the triton kernel and the preceeding/successive kernels.
I am aware you are not actively maintaining this but was just wondering if this was expected or had any hints? I am not that familiar with PTX but understand it is JIT compiled so was wondering if it was not being cached or something like that.

sorry that you have to bump into all those things. this is just a POC and in no way optimized.
thanks for profiling the generated code!!
probably the worst thing for the C code performance is the PTX. it gets compiled to binary every time you call a kernel. this will be replaced by a cubin.
another overhead might be the dispatch for different input sizes. not sure how significant it is for overall performance.
perhaps you can use several cuda streams to bypass those issues?

If I know my target hardware apriori is there any downside/gotchas to me dumping the ptx code to a file and compiling down to cubin and loading that instead?

Assuming the _tr... is a triton JITFunction for JIT and the launch function from the generated C code for AOT.
I think you are correct.
The JITFunction does the module and function loading before it calls the launch code. For the generated C code each call loads the module and the CUFunction.
Thanks for doing this, this will be helpful when thinking about optimizing the generated code!

Tried caching the loaded CUFunction and things are now looking very close to JIT performance (only 5-10% slower now) 🙂

Got a new prototype together, maybe this can help in some way: https://github.com/openai/triton/pull/1056

Thanks, will check it out

Do you know how close it is to being merged? (just trying to gauge whether I should wait - or working from the branch)

It's pretty close but there are other things that have priority over merging it. So branch will be better. I'm happy to help, it will be great to get user feedback

@gaxler what is the relationship between this branch and aot.py on master? Will they both continue to exist after this branch is complete?


Add to profiler datatypes

I came across these data types in https://github.com/capitalone/DataProfiler that we could add to the profiler.

BAN
^(?(?=[\w- ]{17})0-9A-Za-z[ ])
UUID
^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$
HASH
https://www.regextester.com/26
SHA1
\b[0-9a-f]{5,40}\b
SHA256
https://www.regextester.com/95011
IPV6
https://stackoverflow.com/questions/53497/regular-expression-that-matches-valid-ipv6-addresses
MAC_ADDRESS
^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$
DRIVER LICENSE
https://stackoverflow.com/questions/61129744/regex-pattern-for-driving-licence-validation
\b[A-Z](?:\d[- ]*){14}\b


Title: Broken button to subscribe to the newsletter

Description:
The "Subscribe" button in the bottom of the PetMe website is not working. When clicked, after filling the form, it does not trigger any action as expected. Instead, nothing happens and users are unable to subscribe to the newsletter.
To Reproduce
Steps to reproduce the behavior:
1)Go to the PetMe website: 'https://akshitagupta15june.github.io/PetMe/#'
Click on the "Subscribe" button in the bottom of the website.
Observe that the button does not trigger any action, and users are unable to subscribe to the newsletter.
Expected behavior
Clicking on the "Subscribe" button in the bottom of the website should trigger an action that leads users to sign up to receive the PetMe newsletter.
Impact:
This issue prevents users from subscribing to the PetMe newsletter, which can impact the website's ability to communicate with its audience and keep them informed about updates and news related to the website.

kindly assign this issue to me.

There is no js for the subscribe button. there is no link attached to it

If this issue is still open,please assign it to me,i can fix it

@akshitagupta15june I can work on this issue by using EMAIL.JS. With the help of which a user can subscribe and he/she will get the newsletter using Email.

hey if this issue is still open can you assign it to  me


Claimed Furnaces randomly unclaiming

*Detailed description of the issue:

Claimed smelters, furnaces, blast furnaces, etc. are intermittently lost when players relog or the server restarts.
Jobs version:
<IP_ADDRESS>
Server Type (Spigot/Paper/etc):
Paper
Server Version (using /ver):
1.18.1-124

Should be fixed with latest builds


Allow taker to pass in extend private key for wallet

This key will be used to derive the internal wallet according to (Bip84)[https://github.com/bitcoin/bips/blob/master/bip-0084.mediawiki].
This will allow us to easier start a taker with a wallet with funds.

you can easily generate a xprv key with wagyu:
wagyu bitcoin hd -d "m/84'/0'/0'" -n testnet


Setting retain_graph to False in adversarial_patch_pytorch attack

In adversarial_attacks_pytorch.py line 191 : loss.backward(retain_graph=True)
However retain graph being true retains the graph at every backward call and causes a high RAM usage. (processing 6 frames takes 40 GB if max tiers = 10). This heavily impacts our tracking defense as our defense improves significantly with a larger number of frames. Setting retain_graph = False allows us to use 20 frames for our defense, i.e. allows us to set batch_size = 20
We request retain_graph to be set to False :  loss.backward(retain_graph=False)
We reviewed a number of online posts, such as these  1, 2, 3, 4 and they suggest retrain_graph would typically be set to False, unless setting to True is required for specific purposes. They suggest that a new graph is created every backward call, so we do not need to retain the graph.
We tested this by setting retain_graph to False and running the tracking scenarios. The scenarios ran without any issues and and we did not observe any changes in the metrics (e.g. HOTA). Therefore, we would like to request retrain_graph to be set to False.

Hi @mphute Thank you very much for raising this issue! I think to to remember that we had been discussing this setting in an earlier issue. Let me check the older issues to see if there was a reason for this setting and com back to you here.
Are you able to train successful patches loss.backward(retain_graph=False) without your defense being applied?

Yes, when I ran my experiments I saw the adversarial attack functions similarly in the "undefended" case too.


[BUG] Another LUA ERROR

🔵 Describe the bug:
Message: Interface/AddOns/GSE/API/StringFunctions.lua:60: attempt to index local 'str' (a nil value)
Time: Tue Jun 25 20:28:28 2024
Count: 1
Stack: Interface/AddOns/GSE/API/StringFunctions.lua:60: attempt to index local 'str' (a nil value)
[string "@Interface/AddOns/GSE/API/StringFunctions.lua"]:60: in function SplitMeIntolines'
[string "@Interface/AddOns/GSE/API/Storage.lua"]:895: in function CompileMacroText'
[string "@Interface/AddOns/GSE/API/Storage.lua"]:943: in function ManageMacros'
[string "@Interface/AddOns/GSE/API/Storage.lua"]:174: in function ReloadSequences'
[string "@Interface/AddOns/GSE/API/Events.lua"]:289: in function ?'
[string "@Interface/AddOns/GSE/Lib/CallbackHandler-1.0/CallbackHandler-1.0.lua"]:109: in function <.../GSE/Lib/CallbackHandler-1.0/CallbackHandler-1.0.lua:109>
[string "=[C]"]: ?
[string "@Interface/AddOns/GSE/Lib/CallbackHandler-1.0/CallbackHandler-1.0.lua"]:19: in function <.../GSE/Lib/CallbackHandler-1.0/CallbackHandler-1.0.lua:15>
[string "@Interface/AddOns/GSE/Lib/CallbackHandler-1.0/CallbackHandler-1.0.lua"]:54: in function Fire'
[string "@Interface/AddOns/GSE/Lib/AceEvent-3.0/AceEvent-3.0.lua"]:120: in function <...terface/AddOns/GSE/Lib/AceEvent-3.0/AceEvent-3.0.lua:119>
[string "=[C]"]: ?
[string "=[C]"]: in function LoadConfig'
...
[string "@Interface/AddOns/Blizzard_PlayerSpells/ClassTalents/Blizzard_ClassTalentsFrame.lua"]:138: in function <...erSpells/ClassTalents/Blizzard_ClassTalentsFrame.lua:135>
[string "=[C]"]: in function Show'
[string "@Interface/AddOns/Blizzard_UIParentPanelManager/Mainline/UIParentPanelManager.lua"]:483: in function SetUIPanel'
[string "@Interface/AddOns/Blizzard_UIParentPanelManager/Mainline/UIParentPanelManager.lua"]:290: in function ShowUIPanel'
[string "@Interface/AddOns/Blizzard_UIParentPanelManager/Mainline/UIParentPanelManager.lua"]:169: in function <...ParentPanelManager/Mainline/UIParentPanelManager.lua:164>
[string "=[C]"]: in function SetAttribute'
[string "@Interface/AddOns/Blizzard_UIParentPanelManager/Mainline/UIParentPanelManager.lua"]:861: in function ShowUIPanel'
[string "@Interface/AddOns/Blizzard_FrameXMLUtil/Mainline/PlayerSpellsUtil.lua"]:125: in function TogglePlayerSpellsFrame'
[string "@Interface/AddOns/Blizzard_FrameXMLUtil/Mainline/PlayerSpellsUtil.lua"]:147: in function ToggleClassTalentOrSpecFrame'
[string "TOGGLETALENTS"]:1: in function <[string "TOGGLETALENTS"]:1>
Locals: str = nil
t =  {
}
(temporary) = nil
(temporary) = nil
(temporary) = nil
(temporary) = nil
(*temporary) = "attempt to index local 'str' (a nil value)"
GSE =  {
modules =  {
}
MediaPath = "Interface\Addons\GSE\Media"
inArena = false
GUIRecordFrame =  {
}
inDungeon = false
UsedSequences =  {
}
UnsavedOptions =  {
}
Patron = true
TranslatorAvailable = true
TranslatorLanguageTables =  {
}
name = "GSE"
GUIImportFrame =  {
}
PrintAvailable = true
L =  {
}
defaultModuleState = true
GameMode = 11
Utils = true
ProfileStop = 7846461.214500
AdditionalLanguagesAvailable = false
inHeroic = false
GUICompressFrame =  {
}
GUICacheFrame =  {
}
GUITransmissionFrame =  {
}
WagoAnalytics =  {
}
inRaid = false
MacroPreviewFrame =  {
}
BooleanVariables =  {
}
OutputQueue =  {
}
CurrentGCD = 0
DebugOutput = ""
Static =  {
}
PVPFlag = false
inMythicPlus = false
inParty = false
Library =  {
}
RecorderActive = false
inScenario = false
VersionString = "3.2.00-beta1-11-g19e7387-PatronBuild"
GUICompareFrame =  {
}
baseName = "GSE"
SequenceDebugOutput = ""
GUIMacroFrame =  {
}
GUIVariableFrame =  {
}
OOCQueue =  {
}
enabledState = true
GUIDebugFrame =  {
}
GUIExportframe =  {
}
inTimeWalking = false
OOCTimer =  {
}
LDB = true
GCDLDB =  {
}
defaultModuleLibraries =  {
}
StandardAddInPacks =  {
}
GUI
🔵 To reproduce: (Steps to reproduce the behavior)

Go to '...'
Click on '....'
Scroll down to '....'
See error

🔵 The error:
Paste the error message in this blockquote.

🔵 Screenshots:
If applicable, add screenshots to help explain your problem.
🔵 Expected behavior:
A clear and concise description of what you expected to happen.
🔵 GSE.lua file:
Please provide your GSE.lua file or the export string for the specific macro that is causing an issue.

To find the GSE.lua file:

First, make sure you have enabled the "File Name Extensions" checkbox in Explorer (View tab)
Browse to C:\path\to\wow\_retail_\WTF\Account\YourAccountName\SavedVariables\ or /path/to/WoW/_retail_/WTF/Account/YourAccountName/SavedVariables
Copy the GSE.lua file to your Desktop
Rename this copy to GSA.lua.txt
Then just drag it here in your message

Note: If you do not include your GSE.lua file you are wasting your time and mine. 90% of the time, the first thing I will ask for is "Can you please upload your GSE.lua file?".  This file is needed with the error to work out Why that error occured for you.

🔵 Desktop (please complete the following information):

OS: [e.g. Windows 10 64bit]
Game Version [e.g. classic/bcc/retail (9.1.0)]

🔵 GSE Version:

Version: [e.g. 3.x.y]
Downloaded From: [e.g. https://www.curseforge.com]

🔵 Additional context:
Add any other context about the problem here.

Hey Gau - I need to know a lot mroe info than you have provided.
The error is a start but I need your GSE.lua file and also a description of what you were doing at the time to trigger this.  Without that I have no where to go

I updated to the latest version Alpha Version of GSE of Patron Build. When I log on Beta and log on my monk, this Lua Error pop up.

Can you please give me your GSE.lua file - without it i cant replicate what you are experiencing

To find the GSE.lua file:
First, make sure you have enabled the "File Name Extensions" checkbox in Explorer (View tab)
Browse to C:\path\to\wow_retail_\WTF\Account\YourAccountName\SavedVariables\ or /path/to/WoW/retail/WTF/Account/YourAccountName/SavedVariables
Copy the GSE.lua file to your Desktop
Rename this copy to GSA.lua.txt
Then just drag it here in your message

Here you go,
GSE.lua.txt
GSE.bak.txt

Cheers - on it

that the wrong one - thats your character one - i need the account one from WTF\Account\YourAccountName\SavedVariables\ as instead of the one from WTF\Account\YourAccountName_CHARACTERNAME_\SavedVariables\

hang on managed to cause the bug another way
give it 5 mins and try the next beta

try GSE-3.2.00-beta1-14-g4f03a93-PatronBuild.zip

Here,
GSE.lua.txt
GSE.bak.txt


[Search] The default usage with remote data is unsecure (XSS)

Steps
Read the documentation on the search() module, and start using it with remote data, copying code samples from the documentation.
Expected Result

The code and behaviour of the search() module should be fully secure.
The code should be fully immune to Cross-Site Scripting (XSS).
There should be a section dedicated to the secure usage of the .search() module.

Actual Result

No mention is made of security in the documentation.
No warning about sanitizing user-supplied data nor 3rd-party data is made.
Provided code samples are inherently unsecure, because they implicitly trust the data provided by third parties.

Most notably:
https://semantic-ui.com/modules/search.html#/examples
All the code using url: '//api.github.com/search/repositories?q={query}'  implicitly trust the source of the data and rely on github to have sanitized user data for us. If github is ever vulnerable, Semantic-UI users consulting the documentation would be exposed.
What's worse, the provided code samples is offered as if it were an acceptable, secure use of the search() module. Users will very likely copy the code and use it with their own data sources, either with another 3rd party (other than github) which might be less security aware than hopefully github is, or with their own, unsanitized data provided by their own users (stored Cross-Site Scripting).
Basically, any of the fields used from the server response, are used raw, unsanitized:
https://semantic-ui.com/modules/search.html#server-responses
so if the content of the fields title or description of each result item are not sanitized server side, then the default, documented use of search() module is inherently insecure and vulnerable to XSS.

This is another, more specific manifestation of the same underlying problem:
[Dropdown] Security Vulnerability with data-text #5376

The bigger issue is that there is no coverage of security anywhere in the documentation:
[Security] Documentation on the secure usage of Semantic-UI #6570

We implemented data sanitizing and added a security page to the docs
https://fomantic-ui.com/modules/search.html#/security