ScanQuery supports multi column orderBy queries

Fixes #12958.

Description

Add normal column sorting
Add Inline DataSource Sorting
Fix #13152 inline empty data sorting
Sorter interface extraction for future improvement
By default, the data in the segment is sorted by QueueBasedMultiColumnSorter,Other sorters can be used through context parameters
Merge results using QueueBasedMultiColumnSorter
Reference  Sort style of __time , adding priority queue strategy and  n-way merge strategy

Key changed/added classes in this PR

OrderByQueryRunner
OrderBySequence
QueueBasedMultiColumnSorter
TreeMultisetBasedOrderByQueryRunner
TreeMultisetBasedSorterSequence
TreeMultisetBasedMulticolumnSorter
ScanQueryRunnerFactory
MultiColumnSorter
QueueBasedMultiColumnSorter
ScanQueryQueryToolChest

This PR has:

[x] been self-reviewed.

[x] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[x] been tested in a test Druid cluster.

@599166320, thanks for the update! I'll take another look soon.
I noticed that this is a new PR that replaces the existing, closed one. As it turns out, you can just reuse the same PR by pushing (or force-pushing) new commits. Reusing the same PR allows all comments to be in one place. If the PR is closed, you can reopen it and be back in business. Let's use this PR from now on for this project. Please reach out if you need help: doing this PR stuff can be tricky the first time you do it.

Note to reviewers: please see prior history in PR #13031.

@599166320, thanks for your explanations. Is it fair to state these as the requirements you want to satisfy?

Allow sorting of columns other than the pre-defined sort order.
Sorting requires materializing the sort keys. Materialize only the sort keys: once the data is sorted, use the row IDs (rids) within the sort keys to fetch the data. This approach minimizes the memory footprint of the sort as it does not buffer the non-key columns.
Optimize a limit operation: sort the entire set of rows that match the filter criteria, but fetch on-key columns only for those rows within the limit.

These are all great goals! The goal of minimizing the memory footprint is a worth one.
The above approach is similar to how an RDBMS uses a B-Tree index to read off index keys, then uses the rid within the key to fetch matching rows. (RDBMS systems usually add another layer of optimization. If the query requests only the sort keys, then skip the step to read the underlying rows. This is called an "index-only query" by some of us; other names also exist.)

I previously commented on the use of a priority queue to do sorting. I was concerned about the cost of sorting n rows using a priority queue. A quicksort runs in O(n log n) time.
According to the docs, the insert time for MinMaxPriorityQueue is O(log n). Since we insert n elements total, the total cost is O(n log n), the same as for quicksort. Given this, the difference in performance will be in the constant overhead of the two methods, plus any memory overhead differences.
These questions can only be worked out by doing performance testing. In the interests of moving ahead, the priority queue implementation is fine for now. According to the docs, when used with a limit, the cost of removing the last element (to allow a new, lower-value element) is constant time, so that should be fine.

@599166320, thanks for the update! I'll take another look soon.
I noticed that this is a new PR that replaces the existing, closed one. As it turns out, you can just reuse the same PR by pushing (or force-pushing) new commits. Reusing the same PR allows all comments to be in one place and preserves tags and other status. If the PR is closed, you can reopen it and be back in business. Let's use this PR from now on for this project. I've copied tags from your prior PR to this one, and included a link to the prior PR so we can see our earlier comments.
Please reach out if you need help: doing this PR stuff can be tricky the first time you do it.

Thanks for your tips and help

@paul-rogers Recently, I often see an integration-tests error in Travis. It seems to me that it has nothing to do with this PR. It may be caused by instability of other external resources.

@599166320, one more thought on your PR. As I work on mine, I sometimes realize that more work needs to be done. Rather than close the PR (and reopen it), I just mark the PR as a draft. This tells reviewers to pause until you mark the PR as ready for review again.
I apologize for those IT failures. The team has been trying to fix these flaky tests as they waste many hours of time. There are two things you can do. First, if you see a flaky test, please create or update an issue. There are a bunch of issues already open, so look for those first. (Search for "Flaky".) Else, create a new one. Take a look at a few of the existing issues to see what info to provide.
Second, you can as a committer to rerun those particular tests. Sometimes we get lucky and the second run passes. This is a crazy way to run a railroad, but it is the best we have until we fix the particular test.

@599166320, I took an in-depth look at the code, including downloading your branch and stepping through the logic. The good news is that your unit test worked the first time, allowing me to see your code in action. Thanks!
Let's start by clarifying the approach that this PR takes. We discussed various approaches. I think there was some confusion (probably on my part) about the goal.
On the earlier PR, I thought the code was doing this:

Use the cursor to materialize all rows into a priority queue, which sorts them using multiple columns.
Deliver n rows of output, where n is the limit applied to the query.

Your comments suggested you had a fancier (more efficient) approach:

Use the cursor to materialize just the sort keys into a priority queue.
Use the priority queue to sort the keys. For each key set, preserve the row ID (rid) of the underlying data row.
Make a second pass to read the full row (including non-key fields) in the sort order, using the rid to locate the underlying cursor row.

In doing a detailed review, I found:

The cursor actually has no way to do the "fancy" approach: cursors are sequential; they don't provide random access.
The implementation actually materializes rows, sorts them, and limits them, just as I originally assumed.

Is this an accurate assessment? If so, I think we can simplify the PR down to a few of your core improvements.

In the past, I remember that the Cursor interface of Druid has an advance(int offset), which allows skip to the row of the specified offset. Is this a random access? However, the new version of Druid removes the advance(int offset).
The reasons why I used the factory approach are as follows:
I think druid implements column storage, and each column is independent. In the process of query, if you can skip unnecessary columns, skip unnecessary data reading, and deserialize at each stage, you can reduce the consumption of memory, cpu, and other resources, thus speeding up the query.
Back to the specific use case of order by limit:
If there is a datasource with a, b, c, d, e, f, g or more columns, I will execute a query as follows:
select * from datasource order by a limit 100

When executing this sql, you can first load column a into memory, use cursor to traverse the corresponding BaseObjectColumnValueSelector of column a, sorting,  get the top100 and the corresponding offset, and then construct a Map<Int, List<Comparable>>offsetOrderByColumnValuesMap
Then construct the corresponding BaseObjectColumnValueSelector according to a, b, c, d, e, f, g, and use cursor to traverse the corresponding BaseObjectColumnValueSelector of a, b, c, d, e, f, g columns.
In the process of traversing the BaseObjectColumnValueSelector, determine Whether the current offset already exists in the offsetOrderByColumnValuesMap. If so, read the data of row corresponding to the offset. Otherwise, skip reading.
The above is the sorting inside the segment. Then, there is a merge stage in Historical. The intermediate results of multiple top100 will be merged using priority queues (the bottom layer of n-way merge strategy is also merged based on priority queues) to generate the top100 of historical level.
Finally, the broker arrives. The broker uses the priority queue to calculate the final top100 and returns it to the client.
This is my past idea. According to your review, you think this idea is not feasible at present?Next，I should remove the implementation related to sorting in the segment (delete TopKOffsetSequence) and replace it with the implementation of ListBasedSorterSequence, TreeMultisetBasedSorterSequence, and TreeSetBasedSorterSequence.

@599166320, to follow-up a bit: see the summary above: the one that tries to summarize the approach. If that is correct, then the simplest solution is:

In the ScanQueryRunnerFactory,  detect the sort case which the cursor cannot handle. (Is it only multi-column? Or, is it also that some columns don't provide a built-in sort order?)
If a sort is needed:

Insert your sorting sequence (see below)
Remove the sort from the scan query (or at least from the arguments passed down to the cursor)

If a limit is needed:

Provide a limit to the sort sequence
Remove the limit from the scan query (or at least rom the limit passed down to the cursor)

Build the cursor in the usual way (with sort and limit removed)

This gives three possible outcomes:
sort <-- cursor                (sort without limit)
sort-with-limit <-- cursor     (sort with limit)
cursor-with-limit              (default case)

Finally, it turns out that there is an existing class that can help with the sorting : StableLimitingSorter. You provide the comparator and the limit, it does the rest. The key trick is that scan has two row types: you'll need a different comparator for each type.

@599166320, to follow-up a bit: see the summary above: the one that tries to summarize the approach. If that is correct, then the simplest solution is:

In the ScanQueryRunnerFactory,  detect the sort case which the cursor cannot handle. (Is it only multi-column? Or, is it also that some columns don't provide a built-in sort order?)

If a sort is needed:

Insert your sorting sequence (see below)
Remove the sort from the scan query (or at least from the arguments passed down to the cursor)

If a limit is needed:

Provide a limit to the sort sequence
Remove the limit from the scan query (or at least rom the limit passed down to the cursor)

Build the cursor in the usual way (with sort and limit removed)

This gives three possible outcomes:
sort <-- cursor                (sort without limit)
sort-with-limit <-- cursor     (sort with limit)
cursor-with-limit              (default case)

Finally, it turns out that there is an existing class that can help with the sorting : StableLimitingSorter. You provide the comparator and the limit, it does the rest. The key trick is that scan has two row types: you'll need a different comparator for each type.

Let me briefly summarize your simplest solution. When traversing the internal data of the segment, if it is an ordinary order by query, modify the order by parameter and limit parameter in the scanQuery object to prevent them from being transmitted to cursor. Let him just traverse the required data and sort it in the merge phase?

Let me briefly summarize your simplest solution. When traversing the internal data of the segment, if it is an ordinary order by query, modify the order by parameter and limit parameter in the scanQuery object to prevent them from being transmitted to cursor. Let him just traverse the required data and sort it in the merge phase?

That's about right. However, as you point out, there are two (actually three) "phases":

The scan-then-sort phase. This requires the full result set, even with a limit, so we know the "top n" items after sorting.
The Historical-side merges of the segment "scans" on that node so the Broker receives an ordered list.
The Broker-side merge of the results from data nodes.

In my comments, I addressed only the scan-then-sort phase (to keep comments simple.)
The sort phase needs a sort algorithm: it makes more sense to buffer the rows in an array and sort the array, then it does to build a huge priority queue as we read rows. We can do performance tests, but my gut says a sort will be faster.
The two merge phases have a limited queue size: the number of incoming sequences. (The number of segments to merge in a historical, the number of historicals to merge on the Broker.) A priority queue is a good solution for the merge cases.
The same comparator (structure) can be used in all three cases. The various comments in that area suggest that, once the path works, we should focus on optimizing the comparator since it is an O(n log n) inner loop. The optimization is not a design issue, just a fine-tuning issue once stuff basically works.

In the past, I remember that the Cursor interface of Druid has an advance(int offset), which allows skip to the row of the specified offset. Is this a random access? However, the new version of Druid removes the advance(int offset).

Your idea here is sound. If the cursor had a seek() operation, then you could further optimize the sort step as follows:

Run the cursor once, reading just the sort keys. Create that combined (key1, key2, ... rid) tuple.
Sort the key tuples.
Run the cursor a second time. seek(rid) to the position of each row. Then, read the full row into the final tuple to be sent downstream. (Extra credit for copying key values from the key tuple to the final tuple to avoid a segment read.)

The above would be the "gold standard" solution. Alas, as you note, the Cursor class does not provide a seek() method. (Even a advance() method won't work, that is just a sequential scan skipping records. The result of the sort will be a semi-random list or rids.) So, the next-best alternative is that scan-and-sort process we discussed above.
So, perhaps as a "phase 1", we get the crude-but-effective mechanism to work (along with the merges). Then, as a "phase 2", we go back and ask, is it possible to seek within segments? If so, can we modify the Cursor interface (or define a SeekableCursor) to allow random access?
It may be that segments can allow random access, but no other data sources. For example, one cannot seek into a Parquet file: the data encoding is such that a row group column has to be read sequentially. CSV and JSON are seekable in theory, but based on the byte offset, not the rid. And so on.

@599166320, nice improvements! We can perhaps simplify the code even further.
Scan query is a bit confusing. The ScanQueryEngine produces batches of rows (the ScanResultValue), but to sort want individual rows. To mimic the current code, we have to put those rows back into batches for the rest of the query stack. Let's take these one-by-one.
First, let's remind ourselves that, if we sort (even with a limit) we need all the rows. So, we do want to call the thing that runs the engine. The question is, what is the format of the data and what do we do with it?
For better or worse, ScanQueryEngine takes the rows from the cursor and wraps them in ScanResultValue. Worse, for our purposes, ScanQueryEngine maps the rows to the two different scan query result formats. What we really want is single rows in a single, known format. Second best is a batch of rows in a single, known format. That format should be the "compact list" since we'll be storing them in memory.
So, our first task is to convince the ScanQueryEngine to give us what we want. We could modify the engine to return rows. But, that code is a bit complex, so we might want to use what we have, at least for now.
You are already rewriting the query to strip off the sort and limit when we do your custom sort. We can also force the format to RESULT_FORMAT_COMPACTED_LIST. This will give us, returned from ScanQueryEngine.process a sequence of ScanQueryValues.
But, we want rows. So, the next step is to unpack the ScanQueryValue objects into individual rows, and store those in an array, ready for sorting. Your code does this (though see comments.)
Then, we an apply the sort you've developed. Here, we can know that we have rows in compact list form, so each sort key can be translated to a position so we're comparing row1[posn], row2[posn]. That's about as fast as it can get.
Now we've got a sorted array. But, we want it potentially limited. So, our effective row count is min(rowCount, limit).
We now have a (perhaps limited) sorted list. The final step is to recreate the ScanResultValue objects, potentially converting to the inefficient RESULT_FORMAT_LIST if configured. The size of each of the ScanResultValue objects is, I believe, controlled by the batch value in the query.
Perhaps all of this can be handled by a single class based on SimpleSequence().

make() reads all the rows, creates the array, sorts it, and returns our "batching iterator".
The "batching iterator" next() provides the next batchSize rows converted to a ScanResultValue, stopping when we hit the effective row count computed above.

With that, ScanQueryRunnerFactory.ScanQueryRunner:

Checks if a custom sort is needed using the function you've provided.
If not, just returns the sequence from ScanQueryEngine.process as today.
Else

Modifies the query as we discussed.
Calls ScanQueryEngine.process
Hands the returned sequence to the sorter sequence described above
Returns the sorter sequence

All this said, there is a bit of code that is pretty close already: `ScanQueryRunnerFactory.stableLimitingSort(). That methods returns a sequence which unpacks events, and sorts them. It is not quite what we want because it uses a priority queue (awkward for large numbers of items, great for merging) and returns rows one-by-one (we want a batch.) Still, it provides some good examples. (You've probably already looked at it since your implementation is similar.)

@599166320, you've taken on a helpful improvement in a complex area. Would be great if we could be in the same room and use a whiteboard to achieve a common understanding of how Druid works in this area, and the design of your improvement. The next best thing is for me to state the assumptions which I have, so we can see if we have that shared understanding.
We need sorting to fit into the existing scan query structure. Here's my understanding of the gist of a scan query "logical plan", from the root (client) down to the leaf (scan):
1. Project scan rows into SQL rows.
2. Unbatch `ScanResultValue` batches into rows.
3. Broker: merge results from h historicals into a single sequence
4. Historical: merge results from s segments into a single sequence
5. Segment: merge results from c cursors into a single sequence
6. Cursor: read r rows into b batches each of size < ~20K rows, so that b = ceil(r / 20K).

If our query has no sort order, the results are unordered. In SQL, this means that the order is non-deterministic. So, the easy way to do the above merges is just a first-in-first out concatenation.
In Druid, segments are always sorted by time. So, if our query is ordered by `time, then the logical plan would be:
3. Broker: ordered merge of results from h historicals: merging the rows from `ScanResultValue` batches to create new batches
4. Historical: as above, for the s segments
5. Segment: concatenate the results from the c cursor since, I believe, the cursors are already time-ordered
6. Cursor: read r rows which are time ordered

One more thing to note here is that Druid allows sorting of segments by more than just __time. We can order (I believe) a segment by, say, (__time, a, b, c). In this case, the cursor should be able to sort by any prefix of the segment sort order. That is, it should accept an ordering of (__time), (__time, a), (__time, a, b) or (__time, a, b, c). (I have to check the code to verify). If this is true, then the function in this PR that checks if we need an "extra" sort has to take this information into consideration, which means the decision has to occur segment-by-segment since each segment may have a different sort order.
To get even more into the weeds, the cursor should also be able to order by the descending version of any prefix, such as (__time DESC, a DESC, b DESC). The cursor cannot, however, order by a mixture of ASC/DESC that does not match the segment sort order. So, our example segment cannot return rows in an order of, say, (__time, a DESC). We would need the extra sort in this case. (Again, I need to verify: perhaps Druid has a trick, but most systems don't.)

The other thing to discuss is the ordered merge steps. As it turns out, the need to do the ordered merge is independent of how we do the sort. It doesn't matter if the cursor did the sort for us, or if we added a custom sort. In both cases, we have to generate a merge comparator that has the sort keys, with ASC/DESC sort sense, in the proper order.
Merging requires two implementations: one for the "list" (i.e. list of map) format, the other for the "compact list" (i.e. list of array) format. We can assume that the compact-list rows for any one segment have the same set of columns, and that map-based rows have the same keys. That is, the set of column names will be whatever the query requests. The order of columns in the compact-list form should (we should check) be driven by the order that columns appear in the query.
But, we must anticipate that the types of columns from different segments may differ. Druid allows column c to be a long in one segment, a string in another. I believe Druid has some rules for reconciling these type conflicts. This means that the comparator has to know how to compare 10 and "foo". There may already be implementations for this in the existing comparators: we should check.

The other thing to discuss is the ordered merge steps. As it turns out, the need to do the ordered merge is independent of how we do the sort. It doesn't matter if the cursor did the sort for us, or if we added a custom sort. In both cases, we have to generate a merge comparator that has the sort keys, with ASC/DESC sort sense, in the proper order.
Merging requires two implementations: one for the "list" (i.e. list of map) format, the other for the "compact list" (i.e. list of array) format. We can assume that the compact-list rows for any one segment have the same set of columns, and that map-based rows have the same keys. That is, the set of column names will be whatever the query requests. The order of columns in the compact-list form should (we should check) be driven by the order that columns appear in the query.
But, we must anticipate that the types of columns from different segments may differ. Druid allows column c to be a long in one segment, a string in another. I believe Druid has some rules for reconciling these type conflicts. This means that the comparator has to know how to compare 10 and "foo". There may already be implementations for this in the existing comparators: we should check.

Different segments do have different data types in the same column. Let me see how to solve this problem.

@paul-rogers
Maybe we don't need to consider the data type. You can use CAST function to convert to a uniform type, as shown in the following example:
SELECT
  a,b
FROM table
WHERE __time >= CURRENT_TIMESTAMP - INTERVAL '1' DAY
ORDER BY CAST(a AS DOUBLE) DESC,CAST(b AS DOUBLE) ASC
LIMIT 10

The above query will report the following error:
Error: Unknown exception

java.lang.Long cannot be cast to java.lang.Integer

java.lang.ClassCastException

The CAST function can be used to solve the problem of type conversion, such as the following query:
SELECT
  a,b
FROM table
WHERE __time >= CURRENT_TIMESTAMP - INTERVAL '1' DAY
ORDER BY CAST(a AS DOUBLE) DESC,CAST(b AS DOUBLE) ASC
LIMIT 10

When type conversion exceptions occur, we can prompt users to use CAST to solve their sorting problems. What do you think?

@599166320, you asked about the "special path" when the sort key is a prefix of the segment sort order. If this optimization does not exist today, it need not be done as part of this PR. It can be done as a follow-on refinement. I'd have to poke around in the scan query (and cursor, and storage adapter) code to see how this is handled in other query types. We don't have to get distracted here with that task.
Thanks for summarizing the tasks. You noted:

Let me briefly summarize the problems you mentioned above. Based on this PR, I have two things to do next:
Code implementation of special path(Segment by segment decision)
Solve the problem of different data types in the same column of different segments
Another thing: use Calcite (operator and planner based) to improve the query engine, which is what you plan to do later.

This sounds right. The Calcite planner part is outside the scope of this PR (though whatever we do will use the sorter you create here.) The special path can be outside this PR if we don't already have a way to check this case, as noted above.
That leaves several broad tasks:

Decide the level at which to sort. (Per-batch, per-cursor, or per-segment.)
Finish up the sorter for this case. The sorter can assume all values for a column have the same type since we're in a single segment. Handle the batching issues discussed earlier.
Finish up the merge code that is aware of the sort keys. This code should exist somewhere if we do support sorting other than just by __time in any query type. (The code may be specific to some other row format, however.)
Deal with the mixed-type issue we noted. If the above merge code exists, then it should already have a solution.
Test with a data set that spans multiple segments to ensure all the levels listed earlier work as expected.

On the mixed-type issue:

You can use CAST function to convert to a uniform type.

Indeed. This is the solution that the Catalog project proposes. Rather than do the cast in each and every query, tell the catalog the preferred type and the cast will be done automagically.
So, for the purposes of this PR, we can require that all column values are of a single type. Check if the type varies, and if so, throw an exception. The user works around the issue with the cast (or, later, with a catalog entry.) If Druid has some magic, that can be added as a follow-up.
One way to check that the schemas are identical is to check if the schema of the next batch to arrive at the merger is the same as the current batch. And, when we emit values from the merge, we use that same schema. Of course, we actually only care about the types of the sort key columns, so we an be a bit more lenient and only compare the key column types.
A side note about the "type conflict magic". In other projects, we failed to find the magic. If you have VARCHAR and BIGINT which type wins? What if you saw 10K of the VARCHAR rows before the first BIGINT row? Or visa-versa? Given this, the solution that requires the user to break the tie is reasonable.

@paul-rogers
Now these tasks should have been completed, please help me check if I am missing something?
1.Decide the level at which to sort. (Per-batch, per-cursor, or per-segment.)
ScanQueryOrderByRunner and ScanQueryOrderBySequence should have done each batch-level sorting and segment-level sorting.
2.Finish up the sorter for this case.
QueueBasedSorter should have the characteristics of the sorter you describe.
3.Finish up the merge code
ScanQueryOrderByMergeRowIterator is applied to both historical-level and broker-level merges
4.Test with a data set that spans multiple segments & cursors to ensure all the levels listed earlier work as expected.
This test class ScanQueryResultOrderingTest should already satisfy the test needs you describe

@599166320, I had a chance to discuss this PR with @imply-cheddar, one of the Druid veterans. His suggestion is to build on the design you started with in the previous PR, using the simplifications you've developed in this one. Specifically:

The sort code should reside in ScanQueryEngine:  since "that is the thing that implements the scan algorithm", including the custom sorting that you are adding.
Use a priority queue, as you did originally, not a sort algorithm as I suggested. That is, emphasize the small-results and limit use cases. Druid is not as often used for large result sets, so the benefit of a sort-the-list approach are not as important as I'd initially thought.
Use the existing merges at the Historical and Broker level. Or, has he pointed out, there is only one merge, used twice. Here, it is just a matter of defining an Ordering with the proper comparisons. Sounds like you've already handled this task.

In the scan engine, he suggests that you:

Read the individual rows.
Put them into a limited-sized priority queue.
Create batches only when you pull them out of the priority queue.

The result is that ScanQuery.process produces a Sequence of ScanQueryValue object as today, only now they are sorted.
He also notes that we have several existing priority-queue based merge implementations you can borrow. It looks like you have, in fact, already done that.
We discussed the idea of pushing sorts into the cursor. @imply-cheddar, notes that it is uncommon to have sorts of the form (__time, a, b). So, it is perfectly fine to assume, as you have done, that if the sort has multiple columns, or is a single column other than __time, that the code use your sort. No need to worry about the occasional odd multi-column sort that starts with __time.
Since all the code is in ScanQueryEngine, no changes are needed to ScanQueryRunnerFactory. Basically, the logic we'd been discussing to put there will shift into ScanQueryEngine instead.
These all sound like fine improvements that helps this feature be more "Druid-like." Perhaps @imply-cheddar will chime in with additional suggestions.
My own suggestion is that, to limit code complexity, create two paths in ScanQueryEngine: the existing one and the custom sort one. Otherwise, trying to do them both in the same method could be a bit messy.
Thanks for the notes earlier about the other areas. Sounds like we're in good shape there.
Once you are ready, I'll take another look.

Thanks @paul-rogers for the write up of our discussion.  I do still think that some changes will be warranted in the QueryRunnerFactory just fewer of them.  Specific to QueryRunnerFactory the code before this PR does a lot of stuff to organize the QueryRunners in time-interval order in order to short-circuit logic.  That logic only makes sense when the sort starts with time.  The key, though, is that the merge logic can assume that it is working with results from the segments that are merged.
To Paul's point, when we've decided which algorithm to use, we can be in a code path that only does that one thing.  So, one suggestion would be to maybe add a new method: ScanQueryEngine.processWithMultiColumnSort.  This method will assume that it was called because you want a multi-column sort and would then return a set of things that are sorted "properly".
The primary complexity of my suggestion of building batches of pre-sorted rows is that if the query actually does want to scan all of the rows of the segment (i.e. there is no limit applied and the filters match millions), you will end up doing multiple passes over the segment (each pass would generate the next set of values).  This is an acceptable trade-off as it's okay for these types of queries to be slow (i.e. if you have a data set sorted by X and need it sorted by Y and it's billions of rows, sheer physics is going to dictate that the query is not necessarily going to be the fastest thing).
In order to implement this, you will need to be able to keep track of the previous end point.  You will also need to "break ties" by using the rowId so that if there are multiple rows that share the same values for the sort keys, you can deterministically know which ones to include in which batch.
Hopefully this all makes sense...

@imply-cheddar suggested some additional optimizations, though these can come later. Recall that an earlier cut of the PR attempted to do the sort in several phases to minimize the memory footprint:

Read just the sort keys into "rows" used for sorting
Sort the key-only rows
Use random seeks into the cursor to fetch the full set of columns when returning rows

The idea was to read only those columns under the limit. @imply-cheddar explained that this is possible, just not using Cursor. It can be done using QueryableIndex which provides access to the individual columns and would allow random reads.
The other suggestion was to exploit indexes for string columns. The notion here is that column indexes are stored sorted. Each index entry consists of a (dictionaryId, string) pair. Since the dictionaries are sorted, the dictionaryId are in the same order as the keys. A sort can thus compare dictionaryIds rather than the actual strings, which should increase performance.
The sort code can either store only the dictionaryIds in the sort key, sort the data, and fetch the string later, or it can store both the dictionaryId and string for each row, but discard the dictionaryId when returning the row downstream.
It is fine if a first cut at the sort just stores the strings. Because of the dictionary, the strings will be "interned", there will be only one copy of each distinct value. This means that the rows stored in the priority queue will take somewhat less memory than they would if we stored copies of each string value per row.
Again, a suggestion is to get the basics working first. Then, once we can declare victory on that, there can be an optional second PR that uses the above tricks to further optimize the sort.

@paul-rogers @imply-cheddar
Thank you very much for your constructive ideas. Our thinking in this area has been further improved. I'm making some code adjustments based on your ideas, which should be completed soon.

@paul-rogers Sorry, I have been busy with other things recently.
Now, I have preliminarily completed the improvement of dictionary Id sorting and rowId random access.
If you have time, please review it for me to see what can be improved.

Let's ignore the problem reported by Travis CI first, because it has nothing to do with this pr.
I put the two versions of sorting on the test cluster to run the same sorting in a table with 300 million rows of data, and found that the performance of the improved version of sorting (based on dictionary sorting + based on rowId random access) is 10 times faster than the original.


Data as JSON even for transformations

Currently the dataAsJson functionality only works for range queries without any functions (aggregations, transformations, analyses). We should provide this feature also for queries that include functions.

We should unify the data as json representation.

fl:dataAsJson is now available for both, function query and range queries.


Update nestjs/axios to v1 and axios itself

Is your feature request related to a problem? Please describe.
We have a new version of axios & nestjs/axios and we're blocked by this library with updating our project ;)
Describe the solution you'd like
Updating dependencies ;)
Describe alternatives you've considered
There are no alternatives, I'd like to stay with yours package ;>
Additional context
No extra context, it's just about updating deps (I'm not sure if this should be a major update)

Hi @Dzixxx, peer deps versions updated in v3.2.0

Awesome! 😍 closing issue then, thank U! 🚀


LookupEditor CascadeFrom used on Many to Many entity relationships

The CascadeFrom functionality it's great for one to many relationships, but in the case of many to many it's complicated: the solution that I found was to create a custom LookupScript where I could include columns from the relation-table and the entity table, and control the IdField, and TextField of the returned items.

This is also how i do it, if want to work with such lookup editors. I don't want to make LookupEditor more complicated any further. For such matters, users should better write custom editors. Of course i should document it and add more samples.

Can you use the CascadeFrom for one to many relationships?   how make it?
I need your help, really thank you!

For that case you could use the tutorial from the guide

yep,I have read the guide : https://volkanceylan.gitbooks.io/serenity-guide/content/howto/how_to_setup_cascaded_editors.html
is this page??
I follow the guide ,code step by step.  but it doesn't work?
@florin-nitulescu

Can you give a sample on how your entities rows look? The master and the details.

BAreasRow.docx
CatcherInfoForm.docx
the BAreasRow.cs  is Country Province  City  data resource。
the CatcherInfoForm.cs is I want to show the cascade EditColumn like  the following picture

the following picture is my sqlserver data

the Country Province City is all in the table...
@florin-nitulescu


Contribution counter does not increase

Describe the bug
The thread counter is not incremented.

Which site: https://phanpy.rollenspiel.social
Which site version: 2<IP_ADDRESS>a6030c
Which instance: https://rollenspiel.social

Expected behavior
The counter should start from 1 and count up as many posts as there are.
Screenshots
https://rollenspiel.social/@Igni/112607984951868590

Desktop (please complete the following information):

OS: Fedora 40
Browser Firefox
Version 126.0.1 (64-bit)

Hmmm… the rest of the posts are replies to the 1st post, instead of chained replies.

Hmmm… the rest of the posts are replies to the 1st post, instead of chained replies.

How are threads formed? I thought it was by creating a post, and then replying to it with subsequent posts, over and over, until the last on the "thread". Isn't that the case?

@Fastidious yes, it's a reply of a reply of a reply (and goes on). Above use-case if what I call "forked" threads; if there are two replies by same author under the author's 1st post, there's no order on which one is 2nd or 3rd.
Possible solution would be sort by posted time, but again, we wouldn't be able to guess the real intentions here — latest might not be the 3rd and it gets worse if there are multiple replies under multiple replies under the post (and goes on) 😩

@Tealk in case you haven’t gotten a notification, this is an user error. See reply above.

yes, i have read it, but i think i can't educate the users for it.


Events not behaving as intended

Please note that this issue is on the tech-page branch
Notice in the video how the tech description text only fades in once? This is wrong, it should fade in every time.
https://github.com/valkyrienyanko/RaccoonsRiseUp/assets/6277739/3dd108d9-9e3b-4551-9b22-d9e455691e71
This is not the only problem that has been happening related to this.
The issue started with the uncertainty of whether or not this event should go in UITech or UITechNode. Currently it is in UITech. Note that UITechNode are all the nodes I am clicking on in the video.
https://github.com/valkyrienyanko/RaccoonsRiseUp/blob/10db2e16280f38f2a7e25d6374eb0b72b3ea6793/Scripts/UI/UITech.cs#L7
Ideally this event should go in UITechNode and UITech should subscribe to it, but it is the opposite right now. The reason I did this is so I can do the following in UITech
https://github.com/valkyrienyanko/RaccoonsRiseUp/blob/10db2e16280f38f2a7e25d6374eb0b72b3ea6793/Scripts/UI/UITech.cs#L48-L52
There has to be a way to do the above while keeping NoLongerHoldingDownTech event in UITechNode.

Latest commits resolve this issue


Add exec in the service template

After running procsd create the services were not starting and I was receiving:
/bin/bash: -c: option requires an argument

It turns out that an argument is missing after the service file is generated. An exec should be added:
ExecStart=/bin/bash -lc 'exec $BINDIR/gunicorn app.wsgi --log-file=- -b <IP_ADDRESS>:$PORT --log-level DEBUG'

It also the same in Foreman. procsd does not add the exec to the script.

I just manually added exec to my processes in the Procfile to get it running


Ability to Set Backlight (Dynamic Range, Backlight, Off)

I noticed that setting the backlight to Dynamic Range helps the image on my cameras quite a bit at night. But during the day having the backlight on causes the camera image to be washed out. It would be great to be able to control these settings with this addon so these settings could be handled by automations in HA. I have some RLC-410s and a couple RLC-810a's and the situation is the same on both cameras but more pronounced on the 810a. Thanks for the great addon!

Added in version 0.15!

So there is a service now, set_backlight. And also backlight_state is present as attribute on the camera entity.


accurate-color-for-dark-mode

Very great Idea @BytM3
Can you add an asset of the site preview here.. so I can merge afterwards

👇You prefer it on this ?

on all over the site


CI: Add gpg key from obs repository

packaging keys needed to be added in order to install the dependencies
from obs repository.
Fixes #87
Signed-off-by: Geronimo Orozco<EMAIL_ADDRESS>

lgtm


Documentation fix

What type of PR is this?
/kind documentation
What this PR does / why we need it:
Fixes the documentation.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Made some changes that will improve the quality of document and added some essential articles wherever required

I agree with @divya-mohan0209 and would refer you to the comment on this previous PR regarding improving content over personal grammar preferences. We welcome future contributions with the above in mind, thanks!
/close


Moved the Base64Encode() and Base64Decode() functions to be static members of a Convert class within the Azure::Core namespace.

/azp run cpp - storage


Won't Print layerGroup of drawn polygons, but will draw layerGroup of markers

When I print a layerGroup of markers your package works great but when trying to print a layerGroup of drawn polygons I get the error: Uncaught TypeError: Cannot read property 'min' of undefined

Hello,
Can you create some sort of example to understand and debug this issue? I will really help.

Hi,
I'm having the same issue. Please find an example attached.
leaflet print_bug_polyline.zip
It's working fine when the polyline is added to the layer group before the layer group is added to the map, but not after. (swap lines 59 and 60)
Thank you

I've somehow fixed it by not adding layergroup and featuregroup to the new overlay map by replacing the _setupPrintMap by :
_setupPrintMap: function (id, options, printLayer, printObjects, panes) {
        options.zoomControl = false;
        var overlayMap = L.map(id, options);

        if (printLayer) {
            printLayer.addTo(overlayMap);
        }

        panes.forEach(function (p) {
            overlayMap.createPane(p.name, p.container);
        });
        var self = this;
        var addType = function (type) {
            var closePopupsOnPrint = self.options.closePopupsOnPrint;
            printObjects[type] = printObjects[type].map(function (pLayer) {
                var clone = L.Control.BrowserPrint.Utils.cloneLayer(pLayer);

                if (clone) {
                    /* Workaround for apropriate handling of popups. */
                    if (pLayer instanceof L.Popup) {
                        if (!pLayer.isOpen) {
                            pLayer.isOpen = function () {
                                return self._isOpen;
                            };
                        }
                        if (pLayer.isOpen() && !closePopupsOnPrint) {
                            clone.openOn(overlayMap);
                        }
                    } else {
                        clone.addTo(overlayMap);
                    }

                    if (pLayer instanceof L.Layer) {
                        var tooltip = pLayer.getTooltip();
                        if (tooltip) {
                            clone.bindTooltip(tooltip.getContent(), tooltip.options);
                            if (pLayer.isTooltipOpen()) {
                                clone.openTooltip(tooltip.getLatLng());
                            }
                        }
                    }

                    return clone;
                }
            });
        };

        for (var type in printObjects) {
            if (type === "L.LayerGroup" || type === "L.FeatureGroup") continue;
            else {
                addType(type);
            }
        }

        return {map: overlayMap, objects: printObjects};
    },

Hey there.
Nice catch!
Fix applied successfully. Please get latest. v0.6.11 from npm or github.
Regards,
Igor


subsys: usb: Increase mass storage stack size when SD stack is enabled

SD stack requires a larger stack size than the default value for the
mass storage stack thread. Increase the default stack size to 768 when
the SD stack is enabled.
Fixes #49057
Signed-off-by: Daniel DeGrasse<EMAIL_ADDRESS>

@nelarsen. Can you give this fix a try and confirm it works for you.

I confirm that the fix solves the problem. I did a few tests and found out, that a stack size of 650 is sufficient, while 640 is not. So I propose to change the value to 650. (786 was probably anyway a small typo; my original suggestion was 768). Thank you for processing my issue so fast.


[AIRFLOW-2206] Remove unsupported args from JdbcOperator doc

Make sure you have checked all steps below.
JIRA

[x] My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-2206

Description

[x] Here are some details about my PR, including screenshots of any UI changes:

JdbcOperator's docstring has unsupported arguments and
a wrongly named argument. This PR fixes them.
Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:

This PR doesn't need testing since it's documentation fix. I ran docs/build.sh locally and confirmed that the API reference is generated as expected.
Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

[x] Passes git diff upstream/master -u -- "*.py" | flake8 --diff

Codecov Report

Merging #3122 into master will increase coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3122      +/-   ##
==========================================
+ Coverage   72.97%   73.05%   +0.07%
==========================================
  Files         180      180
  Lines       12595    12595
==========================================
+ Hits         9191     9201      +10
+ Misses       3404     3394      -10

Impacted Files
Coverage Δ

airflow/operators/jdbc_operator.py
0% <ø> (ø)
:arrow_up:

airflow/models.py
87.23% <0%> (-0.05%)
:arrow_down:

airflow/jobs.py
80.08% <0%> (+0.44%)
:arrow_up:

airflow/utils/helpers.py
54.02% <0%> (+2.87%)
:arrow_up:

airflow/task/task_runner/bash_task_runner.py
100% <0%> (+6.66%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 64100d2...bee4a80. Read the comment docs.


Add support for datalists

Description
This PR is based upon https://github.com/Altinn/app-lib-dotnet/pull/67 in order to run Sonar Cloud.
Related Issue(s)

#{issue number}

Verification

[ ] Your code builds clean without any errors or warnings
[ ] Manual testing done (required)
[ ] Relevant automated test added (if you find this hard, leave it and we'll help out)
[ ] All tests run green

Documentation

[ ] User documentation is updated with a separate linked PR in altinn-studio-docs. (if applicable)

@caroliss created this as a copy of your PR in order to run Sonar Cloud code analysis and test coverage.

@caroliss created this as a copy of your PR in order to run Sonar Cloud code analysis and test coverage.

@RonnyB71 Ok, I have updated the other branch with tests (among other things) now, should I update this branch as well?

Closing, only used for SonarCloud and duplicate of https://github.com/Altinn/app-lib-dotnet/pull/67


Hi,  reCAPTCHA flow on ios 13 is not working on lastest sdk.

[READ] Step 1: Are you in the right place?

For issues or feature requests related to the code in this repository
file a Github issue.

If this is a feature request make sure the issue title starts with "FR:".

For general technical questions, post a question on StackOverflow
with the firebase tag.
For general Firebase discussion, use the firebase-talk
google group.
For help troubleshooting your application that does not fall under one
of the above categories, reach out to the personalized
Firebase support channel.

[REQUIRED] Step 2: Describe your environment

Xcode version: _____
Firebase SDK version: _____
Firebase Component: _____ (Auth, Core, Database, Firestore, Messaging, Storage, etc)
Component version: _____

[REQUIRED] Step 3: Describe the problem
Steps to reproduce:
What happened? How can we make the problem occur?
This could be a description, log/console output, etc.
Relevant Code:
// TODO(you): code here to reproduce the problem

Hi @AbubakerMajeed, thanks for filing a report. However, I couldn't identify what's causing the issue here. To replicate, please provide the following information required from the template:

Firebase SDK version used / Podfile.lock file
Detailed steps to reproduce the issue
Screenshot / complete error logs (if any)
Relevant code / MCVE that I can run locally

Firebase sdk : Firebase/Auth (6.11.0)
ReCaptcha is not working, go to simulator, try using phone auth , it will take user to safariviewcontroller, then after ReCaptcha autoverifcation. screen stays there.
PhoneAuthProvider.provider(auth: Auth.auth())
PhoneAuthProvider.provider().verifyPhoneNumber(phoneNumber, uiDelegate: nil) { (result, error) in
if let _ = error {
completion(false, nil , nil )
return
}
completion(true , phoneNumber , result)
}

@AbubakerMajeed, the code you've shared is not enough to reproduce the issue. Please upload a sample project that exhibits the error you're describing.

@morganchen12  this is exactly the code, when you implement firebase otp service, you need to verify number first. above sample code is doing same. but problem is after ReCaptcha verification, screen stay there not take user back to application.

Hi @AbubakerMajeed Upon investigating, I was able to replicate the issue which it didn't automatically take back to the app. However, based on the documentation, I added the following code in SceneDelegate.swift to handle redirect URL and the issue has gone:
func scene(_ scene: UIScene, openURLContexts URLContexts: Set<UIOpenURLContext>) {
for urlContext in URLContexts {
     let url = urlContext.url
     Auth.auth().canHandle(url)
     }
}

You may try to add it in your SceneDelegate.swift file and see if it works on your end.

But i am not using swift UI, my app support ios 11 to onwards. This issue can be duplicated on ios12 too.

@AbubakerMajeed Could you try to check if you have a SceneDelegate.swift in your project file? If so, you may add the code snippet I provided here and see if the issue persists.

Did you try on simulator? recapctha process is not working in any case.

My project is running from last 4 years. and it start with xcode older version. I do added "SceneDelegate.swift" of you request. Although it specifically requires me available check. but its not working in simulator. It stuck on white screen. As shown above.

Hi @AbubakerMajeed thanks for clarifying. Yes, I'm using a simulator (iOS 12 and 13). I tried running the Auth quickstart for iOS which doesn't have a SceneDelegate.swift file, but I can't seem to get the same issue. It would be great if you can share a minimal repro of your app that I can run locally to reproduce the issue.

strange it quick easy to produce, I will share a repo with you today. Although i shifted to another otp sms solution. but it great if we have a fix.

Hi @AbubakerMajeed apologies, but I can't reproduce the behavior you encountered so I asked for an MCVE in order to replicate it on my end and determine what's causing it.

testfirebase.zip
here is demo code, just run app and click on "SignIn with defaultnumber".

Thanks for sharing an MCVE @AbubakerMajeed.
I tried running your app and the reCAPTCHA didn't show. However, when I tried to change the GoogleService-Info.plist file with the copy from my project, the reCAPTCHA worked as intended. I'll consult this internally to see what's causing the issue.

@AbubakerMajeed May I know if you have tried to re-download and update your GoogleService-Info.plist file?

this is latest download googleservicer-info plist. For above code(recaptch point) please run on any simulator, not on device.

Hi @AbubakerMajeed, apologies for the late response. May I know if you're not using Google Identity Toolkit?

No, i am not using identity toolkit. Using only firebase phoneauth.

Thanks for the info @AbubakerMajeed. I've forwarded this to the Auth team, and will let you know once I have an update.

any update on this?

I had similar issue, try using VPN. Firebase seems to have some limitations to users trying to spam from same IP, default sign up quota is 100 times/hr, but not sure about default login quota for same device with same IP

The conversation looks to be dropped here. Sorry about that. Is this still an issue?

no solved; being ignore... move to another service...

For anyone running into a similar issue, please try the steps mentioned above:

Implement the SceneDelegate method to handle URL redirects
Re-download your GoogleService-Info.plist file

If both of these steps fail, please file an issue to Firebase Support.
Given the limited number of reports and our inability to reproduce the issue with the sample project from @AbubakerMajeed and our own quickstart, I'm going to tentatively conclude that this was a configuration issue and does not require SDK code changes, but again if you're encountering this issue please file a ticket at support and we'll take a look.


Can I get the ensemble model?

Hi @juliandewit ,
First of all, thank you for your hard work and for releasing the codes. Correct me if I'm wrong but, You said that you trained two models, one is trained on LUNA data set only and the second was trained on LUNA + manual labeling of stage1 DSB2017 data set. The model with manual training was worst than the model trained with LUNA only, but the combination of these two models give better result. I already downloaded your trained model but I don't think I see the ensemble model for model1 (LUNA only) + model2 (LUNA + manual labeling). Do you think you can provide me with the model or maybe give me the script to ensemble them? Thank you for your help

Hello.
Thanks.
the ensembling happens in the "step 4....py" scripty.
I gather the outputs of all networks en put them in a GBM model.


Windows build fails

I'm following the steps in the README file and after I run cake.bat it only outputs
Build succeeded.
Build took 0:00:00.xxx.

and nothing else. No new files are generated after this, and if I use the -l --debug=stack flag it raises an IndexError, apparently there is no build target specified:
(py27) C:\UserFiles\Documents\Code\Libraries\cppcoro>cake.bat -l --debug=stack

Unhandled Task Exception:
  File "C:\UserFiles\Documents\Code\Libraries\cppcoro\tools\cake\src\cake\engine.py", line 455, in _wrapper
    return func()
  File "C:\UserFiles\Documents\Code\Libraries\cppcoro\tools\cake\src\cake\runner.py", line 464, in <lambda>
    task = engine.createTask(lambda s=scripts: listTargets(scripts))
  File "C:\UserFiles\Documents\Code\Libraries\cppcoro\tools\cake\src\cake\runner.py", line 439, in listTargets
    path = scripts[0].path
IndexError: list index out of range
Build failed with 1 errors.
Build took 0:00:00.186.

The Python executable is from an activated Anaconda Python 2.7.16 environment, and Visual Studio version is 16.2.0.
What could be wrong?
Thanks!

I have the same issue

Have the same issue too

I have the same issue but with extra output saying "Windows 10 SDK not found" even though it is installed on my computer


Setting src of HTML Audio element hangs UI

I'm experiencing a dramatic UI hang when the call is established and I attempt to attach the media stream to an HTML audio element:
const mediaElement = document.getElementById('remoteAudio');
mediaElement.srcObject = remoteMediaStream;
mediaElement.play();

The UI freezes for anywhere between 5 and 10 seconds, and then resumes just fine. It occurs due to the mediaElement.srcObject ... line - If I remove this, then it doesn't freeze. When I profile the activity in Chrome, it just shows a "Long Task" with no additional details other than that it's a Microtask.
Looking through the code, this is how it's done on the simple-user phone. Why does this hang occur, and is there a way to mitigate?

I have not seen this happen. This is a pretty standard way of attaching media. Can you see if it happens outside of using SIP.js? I think this is probably an issue with your browser and not SIP.js

Closing due to inactivity.

Closing due to inactivity.


Fixes api gen command to work on M1

Summary
Fixes the command make gen-api-v1 to work on M1 Mac.
Context
The command was not working because the docker image swaggerapi/swagger-codegen-cli was not built for arm64 platform.
Closes #468
Implementation overview

I built a new image for both platforms by doing the following:
git clone https://github.com/swagger-api/swagger-codegen/ && cd swagger-codegen
git checkout v3.0.41
./run-in-docker.sh mvn clean package
cd modules/swagger-codegen-cli
docker buildx build \
  --platform=linux/amd64,linux/arm64 \
  -f Dockerfile \
  --push \
  -t "bcalza/swagger-codegen-cli:3.0.41" \
  -t "bcalza/swagger-codegen-cli:latest" \
  .

Also, I've put the cleanup commands after code generation to be run inside the docker container and not outside because the sed command in Mac is different from the one running in Linux.

Check the docker hub repo.

We should create a repository called swagger-codegen-cli in the textile org. I didn't have permission, that's why I uploaded to bcalza.

The make command looks good. It works fine for me now on M1.


chore: fix broken examples for anybody using pnpm >= v8

Anybody on pnpm >= 8 will be unable to run most of the examples. This is because pnpm v8 changed the default behavior of auto-install-peers from false to true. This results in multiple versions of react getting installed and used (18.2.0, and 18.3.0-canary-587ae49d7-20230720), which leads to the error in the image below. It seems<EMAIL_ADDRESS>actually lists this canary react version as a peer dep...

Several options to resolve this:
**1. Explicitly configure pnpm auto-install-peers=false, so that pnpm v8 behavior doesn't change vs v7. **
This is the change in this PR atm, BUT unfortunately it doesn't really solve the problem.
This is because if npm is not able to read any env variables used in the npmrc file it IGNORES the entire file! https://github.com/pnpm/pnpm/issues/5914. Thus if the end user doesn't have the NPM_TOKEN env variable set when they run pnpm i, they'll still end up with the default auto-install-peers=true.
This might be why maintainers don't end up with this problem (if ya'll are on pnpm v8 and happen to have the NPM_TOKEN) env variable set.
2. Set engines.pnpm in the root package.json to force pnpm 7 or bail out with an error
{
  "engines": {
    "pnpm": "^7"
  },
}

3. Set react and react-dom in the root package.json overrides to point to 18.3.0-canary-587ae49d7-20230720, since a key dep (use-sync-external-store) lists this as a peer dep.
This might be the simplest and would cover all users without requiring changes on their end to their pnpm version or env variables.

Up to maintainers... which approach do ya'll want to go with? (or is there one I haven't though of?). I'll update the PR to reflect the decision.
SIde note, looks like maybe pnpm i hasn't been run in a while? Running it resulted in lots of cleanup (including removing old examples, etc) in the lock file, hence the large diff.

Fixed already


Support for hyperv_drm kernel module under Hyper-v in Windows

Hyprland Version
  Tested on NIXOS unstable + stable verison 23.11

Bug or Regression?
Bug
Description
I would like to see hyprland support under Hyper-v with hyperv_drm,
Hyprland works okay with the old dated kernel module "hyperv_fb", but now it is outdated because it suffers from frame rates drop, hyperv_drm works like a charm and to support this protocol is a game changer for hyprland users, who have high end GPU and want to have windows as host machine.
I really hope this scope would get serious attention from the maintainers, because the users could be good enough.
How to reproduce
using NixOS
boot = {
blacklistedKernelModules = [ "hyperv_fb" ]; # force to use hyperv_drm, because hyper_fb had low frame rates (see github issue)
initrd.kernelModules = [
"hv_vmbus"
"hv_storvsc"
];
virtualisation.hypervGuest = {
enable = true;
};
};
Crash reports, logs, images, videos
No response

I don't think it's up to us to support but rather wlroots.
At any rate, what's the point?

The difference is huge between the two drivers, and the only option for users at the moment is Gnome.
I opened an issue for wlroots devs, for the people who are interested following the issue here is the link:
https://gitlab.freedesktop.org/wlroots/wlroots/-/issues/3842

It would be immensely helpful if you could provide a version number or commit this broke. Try following this guide to bisect in order to find the version/commit this broke for hyprland/wlroots.
Unfortunately I don't have a windows machine handy but if you can find the breaking commit this should be pretty easy to fix (: I'm happy to take a look if you can find the breaking commit.

Thank you guys for taking this serious, adding the line "debug:disable_logs = false" has no effect what so ever,
it is worth to mention that SwayWM is working fine, so I am not sure if it is a wlroots bug anymore.
And I am now having difficulties getting the Hyprland to build in debug mode, as NixOS fails to build under the flake, which actually should be also maintained officially by you, not sure if I should open another issue for it.
this is my flake config:
{ config, pkgs, lib, ... }: {   nix.settings = {     substituters = [       "https://hyprland.cachix.org"     ];     trusted-public-keys = [       "hyprland.cachix.org-1:a7pgxzMz7+chwVL3/pzj6jIBMioiJM7ypFP8PwtkuGc="     ];   };   programs.hyprland = {     enable = true;   };   environment.systemPackages = with pkgs ; [     (pkgs.hyprland.override { # or inputs.hyprland.packages.${pkgs.system}.hyprland         debug = true;     })   ];   services.displayManager.sessionPackages = [ pkgs.hyprland ]; }
My flake.nix is:
`{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
nixpkgs-stable.url = "github:nixos/nixpkgs/nixos-23.11";
hyprland.url = "github:hyprwm/Hyprland";
};
outputs = inputs@{
self,
nixpkgs,
nixpkgs-stable,
hyprland,
...
}: {
nixosConfigurations = {
tinkynixy = nixpkgs.lib.nixosSystem rec {
system = "x86_64-linux";
specialArgs = {
pkgs-stable = import nixpkgs-stable {
inherit system inputs;
config.allowUnfree = true;
};
#pkgs-fd40cef8d = import nixpkgs-fd40cef8d {
#  inherit system;
#  config.allowUnfree = true;
#};
};
    modules = [
      hyprland.nixosModules.default

      ./configuration.nix
    ];
  };
};

};
}`

@fufexan for nix

I am not fan of windows products either, but I have to say, Hyper-V's performance is really the best, and I am not feeling any frame rate drops whatsoever, otherwise I'm forced to go back to Sway after months and years configuring my Hyperland, I hope there would be a solution... Maybe if I found the time I would contribute to do a patch myself.

Change the hyprland input url to the one in the pinned issue.

@fufexan  I am trying to change the commit,  inputs.hyprland.url = "github:hyprwm/Hyprland/{here the commits}";
but I keep build the "hyprland-debug-0.39.1" version, and here is how I build Hyprland
environment.systemPackages = with pkgs ; [
(pkgs.hyprland.override {
debug = true;
})
];
Can you guide me how to build another version of hyperland-debug in flakes?

That's not the way to do it.
Change the input to inputs.hyprland.url = "git+https://github.com/hyprwm/hyprland?submodules=1";. Optionally append &ref=<commit>.
In your config, set
programs.hyprland.package = inputs.hyprland.packages.${pkgs.system}.hyprland.override { debug = true; };

@fufexan I was able to build, thanks to you, but I'm not sure logs keep complaining about  not enabling logs
I added these two lines at the beggining
debug:enable_stdout_logs = true
debug:disable_logs = false
it seems to me, the drm is not supported
Hyprland Crash Report
I tried so hard, and got so far...
Hyprland received signal 6(ABRT)
Version: 7173f0c9e722b52eebcc231a118df738c8e32013
Tag:
System info:
System name: Linux
Node name: user
Release: 6.8.6-zen1
Version: #1-NixOS ZEN SMP PREEMPT_DYNAMIC Tue Jan  1 00:00:00 UTC 1980
GPU:
os-release:
ANSI_COLOR="1;34"
BUG_REPORT_URL="https://github.com/NixOS/nixpkgs/issues"
BUILD_ID="24.05.20240515.33d1e75"
DOCUMENTATION_URL="https://nixos.org/learn.html"
HOME_URL="https://nixos.org/"
ID=nixos
IMAGE_ID=""
IMAGE_VERSION=""
LOGO="nix-snowflake"
NAME=NixOS
PRETTY_NAME="NixOS 24.05 (Uakari)"
SUPPORT_URL="https://nixos.org/community.html"
VERSION="24.05 (Uakari)"
VERSION_CODENAME=uakari
VERSION_ID="24.05"
Backtrace:
# | /run/current-system/sw/bin/Hyprland(_Z12getBacktracev+0x48) [0x6976e8]
getBacktrace()
??:?
#1 | /run/current-system/sw/bin/Hyprland(_ZN13CrashReporter18createAndSaveCrashEi+0x998) [0x60cea8]
CrashReporter::createAndSaveCrash(int)
??:?
#2 | /run/current-system/sw/bin/Hyprland(_Z25handleUnrecoverableSignali+0x5f) [0x5966ef]
handleUnrecoverableSignal(int)
??:?
#3 | /nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31/lib/libc.so.6(+0x3ff30) [0x74ced2252f30]
??
??:0
#4 | /nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31/lib/libc.so.6(+0x8fefc) [0x74ced22a2efc]
??
??:0
#5 | /nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31/lib/libc.so.6(gsignal+0x16) [0x74ced2252e86]
??
??:0
#6 | /nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31/lib/libc.so.6(abort+0xd7) [0x74ced223b935]
??
??:0
#7 | /nix/store/xq4l4d5yva871mbqf99k1d40pqir490c-gcc-13.2.0-lib/lib/libstdc++.so.6(+0xacc0b) [0x74ced24acc0b]
??
??:0
#8 | /nix/store/xq4l4d5yva871mbqf99k1d40pqir490c-gcc-13.2.0-lib/lib/libstdc++.so.6(+0xbc21a) [0x74ced24bc21a]
??
??:0
#9 | /nix/store/xq4l4d5yva871mbqf99k1d40pqir490c-gcc-13.2.0-lib/lib/libstdc++.so.6(+0xbc285) [0x74ced24bc285]
??
??:0
#1 | /nix/store/xq4l4d5yva871mbqf99k1d40pqir490c-gcc-13.2.0-lib/lib/libstdc++.so.6(+0xbc4d7) [0x74ced24bc4d7]
??
??:0
#11 | /run/current-system/sw/bin/Hyprland() [0x50741d]
throwError(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) [clone .cold]
??:?
#12 | /run/current-system/sw/bin/Hyprland(_ZN11CCompositor10initServerEv+0x2ca) [0x5ada5a]
CCompositor::initServer()
??:?
#13 | /run/current-system/sw/bin/Hyprland(main+0x7f9) [0x552c49]
main
??:?
#14 | /nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31/lib/libc.so.6(+0x2a10e) [0x74ced223d10e]
??
??:0
#15 | /nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31/lib/libc.so.6(__libc_start_main+0x89) [0x74ced223d1c9]
??
??:0
#16 | /run/current-system/sw/bin/Hyprland(_start+0x25) [0x596555]
_start
??:?
Log tail:
[LOG] Creating the EventLoopManager!
[LOG] Creating the HookSystem!
[LOG] Creating the ProtocolManager!

[LOG] TextInputV1Manager started successfully!
[LOG] GlobalShortcutsManager started successfully!
[LOG] ScreencopyProtocolManager started successfully!
[LOG] Creating the SeatManager!
[LOG] Creating the KeybindManager!
[LOG] Creating the AnimationManager!
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 13.98µs. Estimated average calc time: 0.02µs.
[LOG] Creating the ConfigManager!
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 7.57µs. Estimated average calc time: 0.02µs.
[LOG] Using config: /home/user/.cache/wal/hyprland.conf
[INFO] !!!!HEY YOU, YES YOU!!!!: further logs to stdout / logfile are disabled by default. BEFORE SENDING THIS LOG, ENABLE THEM. Use debug:disable_logs = false to do so: https://wiki.hyprland.org/Configuring/Variables/#debug
[LOG] Creating the CHyprError!
[LOG] Creating the LayoutManager!
[LOG] Creating the TokenManager!
[LOG] [hookSystem] New hook event registered: preConfigReload
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 6.69µs. Estimated average calc time: 0.02µs.
[LOG] Using config: /home/user/.cache/wal/hyprland.conf
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 430.01µs. Estimated average calc time: 0.04µs.
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 6.28µs. Estimated average calc time: 0.02µs.
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 6.28µs. Estimated average calc time: 0.02µs.
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 6.32µs. Estimated average calc time: 0.02µs.
[LOG] Created a bezier curve, baked 255 points, mem usage: 4.08kB, time to bake: 6.32µs. Estimated average calc time: 0.02µs.
[LOG] Creating the PointerManager!
[render/egl.c:473] Failed to get DRM device: Invalid argument
[render/egl.c:319] Software rendering detected, please use the WLR_RENDERER_ALLOW_SOFTWARE environment variable to proceed
[render/egl.c:571] Failed to initialize EGL context
[render/gles2/renderer.c:503] Could not initialize EGL
[render/egl.c:473] Failed to get DRM device: Invalid argument
[render/egl.c:319] Software rendering detected, please use the WLR_RENDERER_ALLOW_SOFTWARE environment variable to proceed
[render/egl.c:571] Failed to initialize EGL context
[render/gles2/renderer.c:503] Could not initialize EGL
[CRITICAL] m_sWLRRenderer was NULL! This usually means wlroots could not find a GPU or enountered some issues.
[CRITICAL] Critical error thrown: wlr_gles2_renderer_create_with_drm_fd() failed!


interactions_endpoint_url

interactions_endpoint_url: The specified interactions endpoint URL could not be verified.

Please provide more details - also feel free to ask for help in the Discord server


Different version in metadata

When installing the package with "pip install spaudiopy", the error occurs:
ERROR: Requested spaudiopy from https://files.pythonhosted.org/packages/45/d3/bc5d9e57b8b8a339b582dc47e058979403f771ffe368ebfc72585b64df7b/spaudiopy-0.1.3.tar.gz#sha256=0a35b290e11f148f4e9b0834748fbfe9e2c5b9f8054a31dd0814be3902511095 h
as different version in metadata: '-tail-.is.not.recognized.as.an.internal.or.external.command-operable.program.or.batch.file.-dirty'

Can confirm, same thing happened for me with a quick test to install 0.1.3 via pip on Windows.
WARNING: Discarding https://files.pythonhosted.org/packages/45/d3/bc5d9e57b8b8a339b582dc47e058979403f771ffe368ebfc72585b64df7b/spaudiopy-0.1.3.tar.gz#sha256=0a35b290e11f148f4e9b0834748fbfe9e2c5b9f8054a31dd0814be3902511095 (from https://pypi.org/simple/spaudiopy/) (requires-python:>=3.6). Requested spaudiopy from https://files.pythonhosted.org/packages/45/d3/bc5d9e57b8b8a339b582dc47e058979403f771ffe368ebfc72585b64df7b/spaudiopy-0.1.3.tar.gz#sha256=0a35b290e11f148f4e9b0834748fbfe9e2c5b9f8054a31dd0814be3902511095 has inconsistent version: filename has '0.1.3', but metadata has '-tail-.is.not.recognized.as.an.internal.or.external.command-operable.program.or.batch.file.-dirty'

Uhh, that's not good...
Thanks for reporting!
I will release a new version very soon which will include the things I am currently working on.
I'll test the release better next time!
In the meantime, pip can also install directly from GitHub:
pip install git+https://github.com/chris-hld/spaudiopy.git@master


SMTP (and other) notifications mechanism

Attempt to abstract out notification mechanism into separate part of the service. This PR is currently just for progress tracking purpose. Most commits will be squashed/changed later. Hope to receive comments when it's more or less ready.
Update: Spent some time trying to change as little as possible, but it seems that notifications do need some refactoring (for example interface for integration of dangling notifications.Notification type). So started from that.
Task list (checked ones are ready for review):

[x] Unify notifications in core, listener and service packages
[x] Implement notifier mechanism
[x] Implement SMTP notifier
[x] Implement header/body generator for notifications
[x] Notification interface type seems unnecessary
[x] Testing

Coverage remained the same at 27.134% when pulling b447e5cd3614934967ef06f91f2edc63268c093f on yurizhykin:smtp-notifications into 0fd5e3859605a1e3d8374a9ab17be48aa031a5be on OpenBazaar:master.

Spent some time trying to change as little as possible, but it seems that notifications do need some refactoring (for example interface for integration of dangling notifications.Notification type). So started from that.

Coverage decreased (-0.2%) to 27.219% when pulling 7820bce2024ca7db48684e24ac8b742c8f443905 on yurizhykin:smtp-notifications into 9d62bcd73ac1fcac0a2a9a00bf008aa5a6995fa0 on OpenBazaar:master.

Coverage decreased (-0.2%) to 27.229% when pulling 2fb913757e8606b38ca646be2967998869e69c08 on yurizhykin:smtp-notifications into 9d62bcd73ac1fcac0a2a9a00bf008aa5a6995fa0 on OpenBazaar:master.

Coverage decreased (-0.5%) to 27.189% when pulling 0db52b1a3ae4a45bc7638d4c7d7f314cd50fb24d on yurizhykin:smtp-notifications into a242aaf1434db2cb65e1dad3de00a0169a1b97f4 on OpenBazaar:master.

@yurizhykin can we resolve the conflicts so we can move forwarrd on this? ping @cpacia

@hoffmabc , sure. Just rebased on 'master'. Can be merged now, and I will add tests in another PR. But I think, review would be useful, since my commits change some core parts.

Yes of course. Thank you for the rebase.

Coverage decreased (-1.0%) to 27.419% when pulling a8f61ca489787e6b56e930331e9ea3634eefa8aa on yurizhykin:smtp-notifications into b12dd986680134edd304497d0c7d9350973ea5b1 on OpenBazaar:master.

Final touch: added template for notification emails. Also added ServerPort field to SMTPSettings struct, as most popular services seem to break "port 25" convention. Tested in MailHog, everything seems alright.
QA test is not done yet: spent whole evening trying to read from python's subprocess.Popen.stdout on smtpd.DebuggingServer and failed. In REPL I observe that first read() blocks indefinitely, and when it is interrupted from keyboard, subsequent one actually returns expected data. Have no idea what it is. Have anyone seen something like that before?
Anyway, QA test can be rewritten to use MailHog and its JSON API but it seems like a lot of overhead to me.

Coverage decreased (-0.8%) to 27.419% when pulling 9740fc384fce9b223feaac13e2d98f86c2c358f4 on yurizhykin:smtp-notifications into e773625b33865b9db78f6ceeb6873a57f5b9afeb on OpenBazaar:master.

Coverage decreased (-0.8%) to 27.419% when pulling 8ba047285c39f70282045a3ee1a867791213d1ea on yurizhykin:smtp-notifications into e773625b33865b9db78f6ceeb6873a57f5b9afeb on OpenBazaar:master.

Looks pretty good to me. Would just probably remove the status from the describe switch.
Other than that I just need to test locally.


[TC-319] Need Localized Traffic Router

Jeff has a POC, and an idea in his head.
From https://github.com/Comcast/traffic_control/issues/1771
Author: Jan van Doorn
JIRA Link: https://issues.apache.org/jira/browse/TC-319

@elsloo has this done. Closing issue.


⚠ Bofrost.lu has degraded performance

In dc22e67, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1493 ms

Resolved: Bofrost.lu performance has improved in 5f806bb after 9 minutes.


[Doc Issue][Update correct code sample]

Docs Build status updates of commit 8699116:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/messaging-extensions/how-to/action-commands/define-action-command.md
:white_check_mark:Succeeded
View

msteams-platform/messaging-extensions/how-to/search-commands/define-search-command.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/messaging-extensions/how-to/action-commands/define-action-command.md
88
29
link
:white_check_mark:

msteams-platform/messaging-extensions/how-to/search-commands/define-search-command.md
89
12
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Docs Build status updates of commit 9bb16af:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/messaging-extensions/how-to/action-commands/define-action-command.md
:white_check_mark:Succeeded
View

msteams-platform/messaging-extensions/how-to/search-commands/define-search-command.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Error message for #[async] on a for

Consider
let (tx, rx) = mpsc::channel(1);
let process_bootstrap = async_block! {
    let msg = create_msg();
    #[async]
    for foo in rx {
        let msg = msg.clone();
        sink = await!(sink.send((msg, foo)))?;
    }
   Ok(())
};

Where the sink has io::Error as Error type, and rx obviously has () as Error type. This gives the very unhelpful message:
error[E0277]: the trait bound `std::io::Error: std::convert::From<()>` is not satisfied
   --> src/net/server.rs:142:33
    |
142 |           let process_bootstrap = async_block! {
    |  _________________________________^
143 | |             let msg = create_msg()
144 | |             #[async]
145 | |             for foo in rx {
...   |
150 | |             Ok(())
151 | |         };
    | |_________^ the trait `std::convert::From<()>` is not implemented for `std::io::Error`
    |

where the arrows point just around the block. Is it somehow possible to let the error actually point to the #[async] for loop?

Secondly, note that
#[async]
for foo in rx.map_err(|_| -> io::Error { unreachable!() }) {
}

works perfectly, but is quite painful to write. But that's not the real issue here, I guess :-)


print space in results file

results file is being written with the parameter section and statistics section squished together - it needs a space between them.
results file is written with the function write_sim_results_file in main_tools/write_files.py.
It should have the form
parameter_header stats_header
parameter_values stats_values

An example of how it is now:
A	daf	B	AB_t	AN	AN_tSegS_D1_CGI	Sing_D1_CGI	Dupl_D1_CGI	TajD_D1_CGI	SegS_D1_ASC	Sing_D1_ASC	Dupl_D1_ASC	Pi_D1_ASC	TajD_D1_ASC	SegS_S1_ASC	Sing_S1_ASC	Dupl_S1_ASC	Pi_S1_ASC	TajD_S1_ASC	FST_D1_S1_ASC
44499.7180488	0.0264139586625	40008.4616861	(1600:4100)	10000.0	2113.439056124372	1724	590	-0.408906346485	165	27	11	57.4052631579	0.970678370999	158	4	1	49.1563206578	2.31756616044	0.0379303444817

You can see that AN_tSegS_D1_CGI is smushed together. there should be a space.

difficulty - easy

@agladstein I pushed and think its fixed now.


SWDEV-327563 - hipStreamPerThrdTsts fix develop to staging

SWDEV-327563 - hipStreamPerThrdTsts fix develop to staging
Issues seen while merging to staging

@

@mangupta  - This fixes the most recent Windows build failure in develop to staging build
"SWDEV-299127 - Merge 'develop' into 'amd-staging'"


canreach does not work in instanced areas like pestcontrol

Wrong place to put this but your Rs2NPC interact function does
Microbot.getMouse().clickFast(Random.random(0, Microbot.getClient().getCanvasWidth()), Random.random(0, Microbot.getClient().getCanvasHeight()));
which just clicks a random point on the ui

Wrong place to put this but your Rs2NPC interact function does Microbot.getMouse().clickFast(Random.random(0, Microbot.getClient().getCanvasWidth()), Random.random(0, Microbot.getClient().getCanvasHeight()));
which just clicks a random point on the ui
        Point point = Perspective.localToCanvas(Microbot.getClient(), npc.getLocalLocation(), 0);
        Microbot.getMouse().clickFast(point);

seems to work better. If you'd like I can put a PR for that and add a little randomization to the coords
Something is off with the VirtualMouse. It has issues with clicking npcs even when the coords are correct.

Is this still happening? Npc interactions seem fine on my side

Has been fixed with the latest version of the new web walker


refactor(randwords): remove

This was unused, and the word list needed better filtering to be viable.

Why is base 64 a problem?
On Wed, Apr 3, 2024 at 10:53 AM Miroslav Shubernetskiy <
@.***> wrote:

@.**** commented on this pull request.
cc @drraid https://github.com/drraid you had some ideas how to generate
better password for co-sign vs base64. Not sure if cleaning up random words
to use for password generator in chalk would make more sense
—
Reply to this email directly, view it on GitHub
https://github.com/crashappsec/nimutils/pull/64#pullrequestreview-1977083904,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABELGQNPU3WYC6EIVOT36Z3Y3QJVLAVCNFSM6AAAAABD7WH5RGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTSNZXGA4DGOJQGQ
.
You are receiving this because your review was requested.Message ID:
@.***>

I think @drraid comment came out of the chalk messaging:
https://github.com/crashappsec/chalk/blob/66a5bb4b979e9b94f881abf929363db164cba975/src/attestation/embed.nim#L24-L28
Write this down. In future chalk commands, you will need
to provide it via CHALK_PASSWORD environment variable.

writing down base64 is not ideal I think vs a collection of words so it was mostly a UX thing, not a security issue but he can comment better on his thoughts

It would take 5 minutes to migrate to a smaller wordlist with an RFC behind
it.  Might make sense to do; I’m stuck in meetings but maybe later
afternoon will hit you up on slack.
On Wed, Apr 3, 2024 at 10:58 AM Miroslav Shubernetskiy <
@.***> wrote:

I think @drraid https://github.com/drraid comment came out of the chalk
messaging:
https://github.com/crashappsec/chalk/blob/66a5bb4b979e9b94f881abf929363db164cba975/src/attestation/embed.nim#L24-L28
Write this down. In future chalk commands, you will need
to provide it via CHALK_PASSWORD environment variable.
writing down base64 is not ideal I think vs a collection of words so it
was mostly a UX thing, not a security issue but he can comment better on
his thoughts
—
Reply to this email directly, view it on GitHub
https://github.com/crashappsec/nimutils/pull/64#issuecomment-2034855938,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABELGQKVARG5XXH6VAMNIFLY3QKKFAVCNFSM6AAAAABD7WH5RGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZUHA2TKOJTHA
.
You are receiving this because your review was requested.Message ID:
@.***>


DOCKER-422 add ALRESCO_ and SHARE_ env props

As said on #docker, I believe there is a additional problem since even after changing all the SHARE_ and ALFRESCO_ var to GLOBAL_ format, the email templates are still rendered incorrectly. However, we can tackle that outside of this PR.

As said on #docker, I believe there is a additional problem since even after changing all the SHARE_ and ALFRESCO_ var to GLOBAL_ format, the email templates are still rendered incorrectly. However, we can tackle that outside of this PR.

@WimCrols Can you confirm that the issue with your email templates came from another source?


Evolv EScribe Suite 2_SP12: uninstall script

This script unloads launchctl job. This job has specific name for each installation and we can't set this name to "uninstall launchctl" field.
After making all changes to the cask:

[ ] brew cask audit --download {{cask_file}} is error-free.
[ ] brew cask style --fix {{cask_file}} reports no offenses.
[x] The commit message includes the cask’s name and version.

I will fix it.

Thanks  :-)


python3Packages.mypy: 0.770 -> 0.780

Motivation for this change
A new version has been released upstream: https://mypy-lang.blogspot.com/2020/06/mypy-0780-released.html
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Thank you but this change is already in staging-next and should land in master in a couple of days.

Ah, I missed that. I’ll just wait for staging-next then, thanks!


⚠ Exoticaz has degraded performance

In a5e5f68, Exoticaz ($EZ) experienced degraded performance:

HTTP code: 200
Response time: 9020 ms

Resolved: Exoticaz performance has improved in 08f8825.


Release 0.2

This release adds support for user provided timezones. The user inputs an offset from UTC and this is used to set the real time clock with that offset. The real time clock is now written.
Checklist:

All new PRs  into the development branch have a completed checklist.
A release draft is created with binary files.

Checked the PR checklists and drafted the release.


Make queue help messages dismissable

Or add something to preferences to hide them, whatever works.

While I'm in the area, I might as well pick this one up at the same time.

No longer needs a schema change following preferences refactor

PageMain is the main page of the tool, listing all the queues and the requests within them. As with all pages, the default method called in that class when the page is loaded is main().

Irrelevant but interesting information
Within `main()`, there's a call to `setupStatusSections`, `setupHospitalQueue`, and `setupJobQueue`. These functions setup the different queues for display on the main page.
These functions add to a large array listing all of each status section's configuration, including whether a "show all" button is displayed, whether the section is a special type, and of course the help text and requests within that section

The queues are displayed via the template mainpage/mainpage.tpl, and inside there we call the template mainpage/accordiongroup.tpl to show each section in turn. This is precisely where the queue help text is displayed.
https://github.com/enwikipedia-acc/waca/blob/74230da6092f54b7b3cc7da82d82d86658b60f78/templates/mainpage/accordiongroup.tpl#L19-L21
We should add a preference - let's say showQueueHelp. You can pull the current value of a preference via a PreferencesManager instance and the getPreference method. For example, see how this is used when closing a request to decide whether to trigger a welcome user job:
https://github.com/enwikipedia-acc/waca/blob/74230da6092f54b7b3cc7da82d82d86658b60f78/includes/Pages/RequestAction/PageCloseRequest.php#L246-L259
If the preference is null, we should default to true (showing the help messages). Something like ->getPreference(...) ?? true should work well here. You can pass the value of the preference through to the template via an assign(...) call, then modify the template to add the conditional check to show/hide the queue help message.
We also need to make it possible for users to change the setting. PagePreferences and the template preferences/prefs.tpl are where this should happen. As this is a boolean setting, you can use the existing skipJsAbort preference as an example.
One thing the existing preferences page does not do well is allow a default value to be specified. It may be worth adding an optional parameter to PagePreferences::assignPreference to allow a default value to be passed in.
Note: $defaultGlobal determines if this is a global preference or not. As queue help will be different per-domain, we should set this to false.


Incompatible with other mod's monster spawning cages

General Info

[ ] I would like to propose new content
[ ] I would like to propose changes
[ ] I would like to propose another thing

Suggestion Description
Can it be compatible with other monster spawning cages

such as https://www.curseforge.com/minecraft/mc-mods/dungeons-towers This mod，Perhaps there are other mods that do the same。 allowing players to add non vanilla monster cage compatibility features like they do in their configuration files?

The linked mod has no source available. Also what is the incompatibility as you never explained how it's incompatible? What happens?


Remove redundancy in cmake file

Assume that there is always the top level cmake file, dont try to have 'standalone' cmakefile in each sub-directory.

I have looked at this pretty deeply now.   I'm now not convinced that this is the right thing.   The annoying thing about the current structure, is that it always wants to install SystemCLanguage and won't install the "header-only" interface libraries (i.e. the tlm_extensions) locally (always wants to put it in CMAKE_INSTALL_PREFIX).
I think we should have another (in my case, more informed) discussion on whether CMakeLists.txt files that each can be "top level" may be a pretty good solution for "pick and choose" consumers.


Documentation: Fixed-point?

The README documents that this crate provides "Decimal fixed-point arithmetics", but the exponent is variable at run-time. Therefore, it seems like this crate actually provides an implementation of Decimal floating point, as explained in Decimal floating point for example.
If so, the README should be fixed. Otherwise, could you please explain why you consider this crate a fixed-point Decimal implementation?

Historically, this crate startet with a fixed-point type in the strong sense of fixed by handling the number of fractional digits as a type parameter, thus providing a range of types Decimal with 0 <= N <= some maximum (see https://github.com/mamrhein/rust-fixed-point-decimal, especially the explanations in its README).
The current implementation is somewhat a hybrid. Yes, its not really "fixed-point" because Decimal can represent values with different numbers of fractional digits. On the other hand, it does not provide an explicit exponent as the decimal floating point types specified by the IEEE and it does not follow the requirements defined by the IEEE. Its representation uses an integer and a scaling factor in the form 10⁻ⁿ, just like most "fixed-point" implementaions, but the scaling factor is variable within a limited range.
In the README I wanted to avoid any association with the IEEE floating point types and point out the different representation, so decided to stick to the original formulation.
If you think it is really misleading I would simply delete the word "fixed-point".


Compile python requirements

Automated changes by create-pull-request GitHub action

This wasn't compiled from latest requirements so gonna close and let this happen next week. I only added a stubs package, but still just seems like the simplest practice if this deviates from main


Add a command \ModelicaLogo

I just had a few minutes and added a \ModelicaLogo command. I'm not sure, if you like it, hence the pull request. It shouldn't break anything else.

It doesn't feel very natural to me to embed it like that, so I rather would keep the template as it is .

I agree with @bernhard-thiele here. It would be "over doing" the template.


Change shebang for groovy

#!/usr/bin/env groovy is more general and searches the right executable for groovy for you on many systems.
This makes the Jenkinsfile more compatible with different systems.

I have been doing this out of habit, so I consider this a best practice, your thoughts @abayer?

@rtyler thanks for replying so fast. I made some research into it and found that the official groovy page is recommending the longer version:
Official Groovy Docs
Stack Overflow's answer suggests that the longer version will search the right path for you. Since the paths may be different on many different systems, it would be more general:

@byteknacker the Groovy documentation is good enough for me

Thanks for the merge!


Error Output: PHP Fatal error:  Cannot instantiate interface PhpParser\Parser

Hi everyone,
I am new on laravel and php world, so probably this is going to be a quick thing.
I am getting this error on compose update:
> pre-update-cmd: php artisan clear-compiled
Loading composer repositories with package information
Updating dependencies (including require-dev)
Dependency resolution completed in 1.038 seconds
Analyzed 7717 packages to resolve dependencies
Analyzed 120469 rules to resolve dependencies
Nothing to install or update
Dependency resolution completed in 0.003 seconds
Generating autoload files
> post-update-cmd: php artisan optimize
Generating optimized class loader
Compiling common classes
PHP Fatal error:  Cannot instantiate interface PhpParser\Parser in /var/www/et-web/releases/1462787485/vendor/laravel/framework/src/Illuminate/Foundation/Console/OptimizeCommand.php on line 115

  [Symfony\Component\Debug\Exception\FatalErrorException]
  Cannot instantiate interface PhpParser\Parser

Script php artisan optimize handling the post-update-cmd event returned with an error

  [RuntimeException]
  Error Output: PHP Fatal error:  Cannot instantiate interface PhpParser\Parser in /var/www/et-web/releases/1462787485/vendor/laravel/framework/src/Illuminate/Foundation/Console/OptimizeCommand.php on line 115

Exception trace:
 () at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:209
 Composer\EventDispatcher\EventDispatcher->doDispatch() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:94
 Composer\EventDispatcher\EventDispatcher->dispatchScript() at phar:///usr/local/bin/composer/src/Composer/Installer.php:337
 Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/UpdateCommand.php:173
 Composer\Command\UpdateCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:259
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:844
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:192
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:181
 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:123
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:99
 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:43
 require() at /usr/local/bin/composer:24

Do you have any idea what happened? What does the Optimizer does?
Thanks for helping,

Can I see your composer.json file please?

@xeviknal Can you provide your composer.json like @GrahamCampbell has asked?


dwarfish link broken?

The RaptorJIT example section in the manual references a "dwarfish" tool, the link for which goes nowhere.

(perhaps it should refer to https://github.com/studio/studio/tree/master/backend/dwarfish as that's the only thing I found...)


Highlight code sections

Is your feature request related to a problem? Please describe.
There may be multiple places i repeat a piece of code in one query - for example i could write:
TO_NUMBER(TO_CHAR(SYSDATE-1, 'YYYYMMDD'))
In multiple places, but if i want to change the date range, i would likely have to change it in all places in occours.
Describe the solution you'd like
It would be nice if you could have a piece of code remain consistently highlighted (with a background color of my choice). For example, in this case, i could 'pin' the TO_NUMBER(TO_CHAR(SYSDATE portion of the script, so it would appear highlighted everywhere in the SQL file, just so this is easy to find throughout the file.
Something like this:

Describe alternatives you've considered
NA
Additional context
NA

You can already find and even replace everything at once with Find/Replace feature using Ctrl+F
Are you suggesting this feature because it will be more convenient?

You can already find and even replace everything at once with Find/Replace feature using Ctrl+F
Are you suggesting this feature because it will be more convenient?

Yes, i am specifically looking for functionality to highlight or call out important or significant, blocks or lines of code. This is different functionality than the find/replace function.

@WolfX2 So just to clarify, you suggest a feature that will highlight all instances of a specific part of code. Just highlight with a chosen background color, without any changes to text. And this is to help navigating in a long script. Correct?

@WolfX2 So just to clarify, you suggest a feature that will highlight all instances of a specific part of code. Just highlight with a chosen background color, without any changes to text. And this is to help navigating in a long script. Correct?
Thats correct. You could compare this to the well known functionality that exists in notepad++ where you can style all, or one occurrence of a code block in the context menu, screenshot attached.