Query.newPager

Lets build a query paging mechanism so developers can get an unbounded number of rows without harming our precious databases.
The pager interface gives you a page of rows. You pass a session to each call and you can use the same Pager instance across different transactions! Each call to nextPage() advances the internal offset.
interface Pager<T> {
  /** Returns null when there are no more pages. */
  fun nextPage(session: Session): Page<T>?
}

data class Page<T>(
  val contents: List<T>,

  /** Null if there are no more results. Pass this in a subsequent call to [Query.newPager]. */
  val nextOffset: Offset?
)

/**
 * Offset into a query. Typically this is a string-encoded ID like 5, or a composite containing a
 * date followed by an ID, like "2019-08-16T:10:26:51Z/5".
 *
 * Don't put PII in here. Clients see these in URLs and API calls, and can manipulate them. This
 * could also potentially leak database growth information!
 */
data class Offset(
  val offset: String
)

Create a pager from a query. Note that you don’t need to be in a transaction to create a pager!
interface Query<T> {
  ...

  /** Use a null [initialOffset] to start at the beginning. */
  fun newPager(paginator: Paginator<T>, initialOffset: Offset? = null): Pager<List<T>
}

It uses a pagination mechanism that turns rows into offsets and offsets into query constraints.
/**
 * Breaks a query into pages. Each page break is delimited by a string that is the offset of the
 * last row of the preceding page.
 */
interface Paginator<T> {
  /** Figure out what the offset of [row] is, and encode that as a string. */
  fun offset(row: T): Offset

  /**
   * Adjust [query] so that is in paging order and offset by [offset] (if non-null).
   * Most implementations will add an `ORDER BY` clause. They should also add
   * a `WHERE column > offset` constraint if the offset is non-null.
   */
  fun offset(query : Query<T>, offset: Offset?)
}

/** Pages through entities by descending ID. */
val ID_DESC_PAGINATOR: Paginator<DbEntity<*>> = TODO()

/** Pages through entities by ascending ID. */
val ID_ASC_PAGINATOR: Paginator<DbEntity<*>> = TODO()

FYI @lkerford I don’t need you to do this, but I remember we talked about it last year.


lodash needs to be downgraded to be compatible with angular-patternfly

lodash should be downgraded to v3.9.0 to be compatible with "angular-patternfly": "~2.3.2".
It was upgraded in https://github.com/ManageIQ/manageiq-ui-self_service/pull/51/commits/d2813dfe0a14478d5299942d360cf34d87805f4a for the .sumBy call.
The .sumBy call has now been replaced with .sum which is supported by lodash v3.9.0.

@himdel Please review. Thanks.

verified in UI :+1:

Yup, looks good :) :+1:
We just need to remember to convert it back when finally moving to lodash v4.


format of ALL does not include XML output

The XML output is not included when specify --format "ALL" on the command line, instead only the HTML and VULN outputs are produced.
The help states the following which implies ALL would include XML too.
-f,--format <format> The output format to write to (XML, HTML, VULN, ALL). The default is HTML.
Thoughts?
Ta, Mark

Scratch this one.


Empty storage when loading a new version of CoMPAS

After opening a new release of CoMPAS in the webbrowser, the webbrowser storage still contains the old version and tries to execute it.
This problem is fixed by deleting the cache manually.
To reproduce:
Release and deploy a new version of CoMPAS
Open CoMPAS in the web browser; the old version get still served
Acceptance criteria:
The user does not got errors/weired behaviour

This is an openSCD issue: https://github.com/openscd/open-scd/issues/1468


⚠ Telly has degraded performance

In 474da2d, Telly ($TLY) experienced degraded performance:

HTTP code: 200
Response time: 1112 ms

Resolved: Telly performance has improved in eb0036c.


A route of grid line

Hi.
When I used grid line in table, I found a strange behavior.
It is an example.
https://jsfiddle.net/s9q264nv/
When I draw a line from 'A' to 'D' or 'E', the arrow made an unnecessary loop path.
Is it based on the technical specifications?
Should I use not 'grid' but 'straight'?

Hi @K-actus, thank you for the comment.
That loop is necessary path, and that is correct behavior because LeaderLine tries to keep a measurable size of grid.
You can adjust the path via options if you need.
For example: https://jsfiddle.net/hp7vzqbo/
Note that I don't know a line which you want. Is the example above your expected result?

Hi, @anseki .
Yes, this is it.
Thank you for great answering.
I close this issue.

:smile:


Report on actionable emit hotspots

After looking at lots of traces it appears that JS emit is nearly always proportional (with a small constant) to file size (even with down-levelling), so flagging JS files that took a long time to emit rarely does anything beyond identifying large input files.
Declaration emit, on the other hand, is much less directly related to input size.  So far, the most important factor we've found for disproportionately slow declaration emit is the presence of a lot of import expressions in the output.  This PR attempts to flag such instances by searching slow-to-emit declaration files (if present) for import expressions.
Fixes #3 (until we find some other cause of disproportionate slowdown).

If this functionality helps you, please upvote https://github.com/microsoft/TypeScript/issues/44044 and, if possible, provide a repro.


Disallow backup deletion when it is used for restoring a volume

Longhorn/longhorn#5458

@Mergifyio backport v1.4.x

@Mergifyio backport v1.3.x


Strip trailing whitespace from hosts added to the whitelist

Upon accidentally adding a host with trailing whitespace this morning, thanks to a failed copy/paste, I realised that the whitelist let it through without stripping the whitespace or erroring. Make it strip the whitespace from the hostname, so that people and Bouncer don't get confused or match on the wrong thing.

The tests pass here now, thanks to Jenny yesterday. :sunny:


Add tutorials and discussion to documentation

Is your feature request related to a problem? Please describe.
The reference documentation is comprehensive but the docs are still lacking in two areas:

How-to guides: recipes to achieve very specific outcomes: use mignore, use the selective sync API, etc.
Discussion: There is already a decent discussion of the sync logic, the config and state files and the logging setup. Other areas are lacking.

Describe the solution you'd like
Expand the documentation, especially by writing how-to guides.

User facing docs have been improved in the new website.


SW-3937 Delete reports when project is deleted

When a project is deleted, it no longer makes sense for people to submit
project-level reports for them, but we want to keep any reports that have already
been submitted.

Merge activity

Dec 12, 6:56 PM: @@sgrimm started a stack merge that includes this pull request via Graphite.


Implement SharedString in rbx-binary

This PR seeks to implement support for the SSTR chunk and the SharedString datatype in rbx-binary.
Because this isn't as trivial as the other datatypes, I thought having a PR to actually keep track of it would be good, so I'm opening this as a draft ahead of time.
As of me making this PR, we need to:

[x] Implement SharedString in text_deserializer.rs
[x] Implement SharedString in deserializer.rs
[ ] Implement SharedString in serializer.rs
[x] Add actual tests

Accidentally includes the test file in that last PR, expect the CI to start failing from here on out. 🥴

I'm not too sure about the format we're going with for serializing SharedString, but it's usable for testing and isn't noisy visually, so I think it's fine. Interested if anyone has opinions on it.
Keep in mind we only get one implementation of Serializer for SharedString, so it'll end up being global.

I see that you cherrypicked the SharedString fix commit from master, but I think that this is just delaying your merge conflicts as those changes are now showing up as part of this PR too!
I would recommend backing up to b26b76b and then doing a normal merge (git merge master), which should let you resolve the conflicts fairly easily and commit them.

That was a good call. Was going to wait until the end (since a bunch of PRs were coming in at once) but I suppose we're probably mostly done for now.

Alright so as it turns out, serializing SharedString isn't a trivial task.
We have to write the SSTR chunk prior to writing PROP chunks, but as a result we don't actually know what SharedStrings exist in the DOM up until then. The only solutions I can think of at the moment are to either change how we're deserializing so that SSTR can come after PROP chunks or to keep track of SharedStrings somehow when they're added to the DOM.
Neither one is great, since the first one involves a second pass over the file (rbx_xml does this but XML trees and binary chunks are different stories) and the second involves either some overhead or an invasive change to WeakDom to keep track of SharedStrings.
I think the second one is more practical long-term (SharedStrings being first-class citizens solves the problem for rbx_xml too) but it has some implications for the future of the project so I don't want to just do it without consulting someone first and discussing the pros and cons (it's also late and I don't want to do it tonight 😛)

fwiw, rbx_xml doesn't take a second pass over the file while deserializing. It keeps a list of rewrites to apply, which is used for both referents and shared strings. The second pass then just drains those arrays and applies the properties to the tree, using information that's now complete as the entire file has been traversed.
We should be able to apply the same technique here.

Right, sorry, I wasn't very thorough in investigating rbx_xml's deserialization since it wasn't super important to this case. I just noticed that it looked to be doing that, so I thought it was worth mentioning above.
This is about serializing and not deserializing though, so it may have been misleading to bring up. That's my bad.
This is different than rbx_xml since we can't write the SSTR chunk after the fact; it has to be before PROP chunks that use it. We can probably get away with just keeping track of the various information and then writing any PROP chunks that use SharedStrings after the SSTR chunk, though I'm not sure if Studio will load a file that does that. We can give it a go though.

I think I understand the problem now. I'll restate it to confirm. 😅
We need to write the SSTR chunk before the shared strings contained within can be referenced by PROP chunks. However, we don't know which shared strings we need to write until we've traversed all the properties.
I think we already have a good place to do this. It happens in a single step, traverses all instance properties, and happens before we emit any part of the file. We can extend BinarySerializer::collect_type_infos to also collect any new SharedString values and assign them an ID.
Does that solution sound workable? I prefer solutions that generate files that are closer to what Roblox Studio outputs whenever we can, and this should let us be nearly identical.

Hah. I knew that deriving Hash would be a bad idea. I'll push a fix for that in a bit.
As for the build failure... It turns out that UnionOperations have both formFactorRaw and FormFactor serialized. Parts don't. That's... neat. I'm not sure if that's what's causing it to fail (I hope not) but it could be an issue. Feel like taking a look @LPGhatguy?

Alright upon further inspection it looks like it is actually that. We use different error text in serialize_properties, which is convenient, since it let me actually identify that it isn't that.
The only other place we throw UnsupportedPropType is in collect_type_info.
Conveniently not related to this PR which is nice (I didn't touch that part of the code!), but it does need to be resolved before we can merge this, and it's really only through luck that it didn't come up until now.
But hey, at least we learned something about UnionOperation today!


Add deps required to run lint task

Here is a quick PR to fix the lint task, which is missing some dependencies. Just FYI, I'm also still getting a port conflict when trying to run the dev task on a clean install. Will continue to investigate that and report back when I understand the problem better.

Hey, @cayblood! Thanks for the PR! We didn't give our linter the proper attention yet. It's not on the CI, and I don't even run it in my workflow. I'll work on this issue soon, and then we can check if we still need to update the deps. I'll leave your PR open in the meantime. Thank you again!

Hey, @cayblood! I just merged #208, which configured our linter. Can you try again and see if you still need to add other deps to run the linter? Since it's running successfully on the CI, I believe you should be good to go ;)
I'm closing this PR, but feel free to open a new one if you catch something. Thank you!


ContextMenuPortal: useClickOutside hook?

Is your feature request related to a problem? Please describe.
the opening and closing of the context menu is janky
Describe the solution you'd like
use the useClickOutside hook provided by mantine in ContextMenuPortal instead of the window events
maybe also add custom events/controlling of the behavior too?

What do you mean by "janky"?
The window events in ContextMenuPortal.tsx have nothing to do with useClickOutside. The useClickOutside hook simply does what its name implies, while the window event subscriptions you're referring to are there to ensure the menu is closed when the user presses Escape or the windows is resized (i.e. when a mobile device is rotated).
Initially I was using the useClickOutside hook to control the menu closing, but I had to walk away from it when I added the submenus feature. If you're curious why, when implementing submenus, best practice says you need to apply a delay before closing a submenu, to give the user's pointer enough time to travel from the trigger to the submenu area.
So, to be honest, I don't really understand what's this about. Is this something you consider a bug (again, what do you mean by "janky") or...?

laconic < explicit < verbose 😀
In other words, there's a huge difference between laconic and verbose and I'd always go with "explicit".
Yes, I see what you mean now.

In other contexts such as operating system apps for example, when you right click on somewhere else while a context menu is open, it closes the context menu then reopens one where you right clicked. Right now mantine-contextmenu treats right clicks outside the context menu the same way it does left clicks.

As a matter of fact, a few versions back I was relying on useClickOutside and the behavior was the one you describe. But @Songkeys came up with this perfectly reasonable suggestion (and a PR), and I found the argument reasonable.
Good catch on submenus not working on mobile, I'll have a look and try to come up with a fix soon.

why doesn't mantine-contextmenu use the Mantine Menu component?

AFAIK, the Mantine Menu component needs to create some DOM elements in order to implement its functionality, which wouldn't be acceptable in certain situations - such as when using a context-menu on table rows - see Mantine DataTable for an example.

Thanks for chiming in, @Songkeys. TBH, I think it's good enough as it is. It's the same behavior I'm seeing in my current browsers (Brave, Firefox and Edge), and in Manjaro Linux with KDE (don't have a mac or windows to test on at the moment).
When you have an open context-menu and right-click on something else, the currently open context menu closes, but you don't get a new one opening.
@TheAlan404 - I'll be coming up with a fix for the submenu bug on mobile in a few minutes, thanks again for noticing.

The bug with submenus not showing on mobile devices is fixed in V7.3.1.
Unrelated to this, I've noticed there is, however, a potential improvement to be made: at the moment, the corresponding submenu is shown when the user is hovering over the parent item, with a small (configurable) delay. However, I think we could instantly show the submenu without a delay if/when the user clicks on on the parent item...
Feel free to have a look if you have the time and the mood (I don't at the moment; it's 10:40 PM here and I'm going to call it a day :-).