[KYUUBI #3666][Subtask] Support {OWNER} variable for queries run on CatalogV2

Why are the changes needed?
Subtask of #3607
How was this patch tested?

[x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

cc @bowenliang123 @yaooqinn

LGTM.

@zhouyifan279  it is possible to move ut from iceberg catalog to v2 jdbc catalog with in mem db for less file footprint for testing?

@zhouyifan279 it is possible to move ut from iceberg catalog to v2 jdbc catalog with in mem db for less file footprint for testing?

v2 jdbc catalog table does not have a owner property.

v2 jdbc catalog table does not have a owner property.
OK, let's leave it there.

Codecov Report

Merging #3674 (e33551c) into master (bc3fd3a) will increase coverage by 0.05%.
The diff coverage is 94.44%.

@@             Coverage Diff              @@
##             master    #3674      +/-   ##
============================================
+ Coverage     52.05%   52.10%   +0.05%
  Complexity       13       13
============================================
  Files           485      485
  Lines         27260    27274      +14
  Branches       3804     3804
============================================
+ Hits          14189    14211      +22
+ Misses        11700    11689      -11
- Partials       1371     1374       +3

Impacted Files
Coverage Δ

.../kyuubi/plugin/spark/authz/PrivilegesBuilder.scala
71.38% <85.71%> (+0.04%)
:arrow_up:

...he/kyuubi/plugin/spark/authz/util/AuthZUtils.scala
61.70% <100.00%> (+2.61%)
:arrow_up:

.../apache/kyuubi/plugin/spark/authz/v2Commands.scala
90.95% <100.00%> (+0.32%)
:arrow_up:

...rg/apache/kyuubi/ctl/cmd/log/LogBatchCommand.scala
78.00% <0.00%> (-2.00%)
:arrow_down:

...pache/kyuubi/engine/YarnApplicationOperation.scala
62.96% <0.00%> (-1.86%)
:arrow_down:

...he/kyuubi/ha/client/etcd/EtcdDiscoveryClient.scala
66.25% <0.00%> (-0.63%)
:arrow_down:

...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala
80.74% <0.00%> (-0.63%)
:arrow_down:

...in/scala/org/apache/kyuubi/config/KyuubiConf.scala
97.29% <0.00%> (-0.08%)
:arrow_down:

...g/apache/kyuubi/operation/BatchJobSubmission.scala
74.64% <0.00%> (ø)

...main/java/org/apache/kyuubi/client/RestClient.java
86.20% <0.00%> (+3.44%)
:arrow_up:

... and 2 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Thanks, merging to master


Won't start on an offline PC

barcodrod.io installed properly on an offline Win11 machine from the github-downloaded install file but after clicking to start the app (no matter if immediately after installation or via Start menu) nothing happens. Only when the machine is connected to the internet (having provided the credentials in some other way), barcodrod.io opens up.
Does it need to fetch dependencies first?
The use case is to connect an offline laptop via webcam with a wifi network where the credentials come from a printed QR code. The built-in Win11 camera app can decode the QR code but needs a store app to handle it further and doesn't allow to copy the string (and therefore is totally useless).
I think this use case is important for barcodrod.io to shine as Win11 (with its camera app) doesn't help and your app could be the savior here.

Hi @llvs, thanks for the request. The GitHub hosted installer .msix is actually scraped from the MSFT Store. The main reason I went this route is because the Store package has the benefit of using the MSFT Store's code signing certificate, which I believe should be trusted on just about all Windows 10/11 machines. From a security perspective, I'm hesitant to publish an offline installer because it would be unsigned. Windows doesn't allow MSIX installations unless they're code-signed (without disabling security controls, using PowerShell, and enabling Developer Mode).
Since I don't plan on purchasing one for this project since it's FOSS, it would be difficult to offer a .MSI or .EXE setup installer available securely since it would be unsigned.
To your issue- I suspect that since this is the Store version of the app, it's failing to launch for one of two reasons:
Either it may need to contact Microsoft Store to acquire a license, or it's failing to install the Windows App Runtime which may need to be pulled from the Store. If you're able to either install the WinAppRuntime manually and/or launch the store while connected online once, I think it should work offline after that. FWIW, app usage can work completely offline. It's the install process that's a bit tricky.
I will investigate ways to package the .MSIX differently so it contains all relevant dependency packages for offline installer scenarios. In the meantime, the code is available so feel free to try those workarounds or build/modify locally for offline scenarios if you'd like.

Hi @llvs
I believe I've found the root cause here, and it's because the .MSIX hosted in GitHub is the exact same as the Store version. Because it's the Microsoft-signed, store version, it requires an internet connection on installation for license acquisition through the Microsoft Store After first installation and launch, it should be able to be used fully offline. There's two options to solve this:

Release an unsigned version that needs to be sideloaded via PowerShell (not ideal- Edge/Defender will likely see it as unsafe). However, I could post checksums for users to self-verify.
I could acquire an offline, volume-licensed version of the Store buil through Winget, and provide that along with the .MSIX here. Only issue is these offline, volume-licensed versions are technically licensed against enterprises (you need to authenticate with Entra ID as an admin to download). I can acquire this just fine using my developer tenant, but I need to make sure that's not breaking any Terms & Conditions in the Microsoft Store.

TLDR; Two potential paths to fix, both with caveats.


Testenv

This reformats the project as a downloadable package as per our discussion last group meeting. This is important for the tests that I will implement among other things. The new files are configuration files.
The game code is the same (other than some linting and import fixes).
Sorry to wait so long to make a pull request, there are a lot of changes. I am happy to explain anything on Teams if you need.
Ignore "do not merge" messages. Everything needs to be merged. The commit messages are old.

Changed my mind, lol. Not ready yet.


Feature/add modal before create account

PivotalTracker ticket: #158365629 Add modal before adding a new account

Screenshots attached

@ror-shubham please, rebase this branch on top of develop branch

@raymundjacobs please, check this PR


[RFC] .basis Texture Format

What do you guys think about the newly announced Basis Universal Texture Format?
It seems like it would solve the problem with GPU Texture Size vs Disk Texture Size.
Should this new format be supported and implemented in PIXI or trough a plugin?
Links:
https://github.com/BinomialLLC/basis_universal
https://opensource.googleblog.com/2019/05/google-and-binomial-partner-to-open.html

We were waiting for it for a long time. Finally. @GoodBoyDigital is looking into it.

Basis is pretty cool, glad they are open sourcing it.

It seems like it would solve the problem with GPU Texture Size vs Disk Texture Size.

Basis files are often smaller than a PNG equivalent (using default setting, without mips), but that isn't the best feature IMO.
The biggest benefit of basis is the ability to transcode it to whatever GPU compression is supported on the machine you are on. That way you don't have to build your texture into 5 different formats for each different platform it might land on, you just build it to basis and transcode it to the platform specific format the GPU on your host machine supports.

Yeah, the benefit of not having to serve multiple texture formats for different platforms is also a great advantage.

@stale Don't close.

I believe @eXponenta has already done this 👍
Where did this code land in the end?

Pretty sure it’s here:
https://github.com/pixijs/pixi-compressed-textures
Not sure if there’s a release but @ivanpopelyshev  should be able to answer that.
Closing as this seems to be addressed. Further questions or issues about BASIS should be left on the compressed textures repo.

https://pixijs.io/examples/#/plugin-compressed-textures/basis.js


Feature Clarification: reading hyperlinks

In the file https://www.dropbox.com/s/az0zsl67ilykjjg/example.pptx?dl=0  - there are two shapes with hyperlinks: one to a slide and one to a URL.  Can either be read by python-pptx?   It's a little unclear between http://python-pptx.readthedocs.org/en/latest/dev/analysis/features/txt-hyperlink.html and https://groups.google.com/forum/#!topic/python-pptx/et-q2xbb9GI if the code is in the repo?
(Also, if there is a hacky way to just read a slide link, that would be fine)
Thanks.

Hi Joe, as I recall, this feature is somewhere in partial implementation status, probably most suitable for writing a hyperlink rather than reading.
A Run object has a read-only hyperlink property that provides a _Hyperlink object. _Hyperlink.address will be None if there is no actual hyperlink (hlinkClick element) present. So you should be able to detect hyperlinks by testing the address property. If you get something it will be the address, at least in the case of a URL. Not sure exactly what you get if it's an internal link to a different slide.
So something like this air code I imagine:
shape = ... shape you want to look in
for paragraph in shape.text_frame.paragraphs:
    for run in paragraph.runs:
        hyperlink = run.hyperlink
        if hyperlink.address is not None:
            print(hyperlink.address)

Let us know how you go and where it falls short if it does, you can probably get further by going down to the lxml layer from hyperlink to get more. Once you have some partially working code we can help point you in the right direction.

Hi,
I did some checking - I'm actually looking for hyperlinks on the shapes rather than the text boxes (I genuinely didn't know that the text boxes could be links...) and my rough code:
'''
for shape in slide.shapes:
hyperlink = shape.hyperlink
if hyperlink.address is not None:
print(hyperlink.address)
'''
tells me that there shapes lack the same attribute - does this mean that python-pptx doesn't support them? Will be very sad if so... :(

So I've explored a little more, part of this will be a bug report (which I can happily spin out into its own issue)
Given the code
#!/usr/bin/python
from pptx import Presentation

prs = Presentation("testSuite/example.pptx")

for slide in prs.slides:
    for shape in slide.shapes:
        try:
            text = ""
            for paragraph in shape.text_frame.paragraphs:
                for run in paragraph.runs:
                    text += run.text.encode('ascii',
                                            'ignore')
                    hyperlink = run.hyperlink
                    if hyperlink.address is not None:
                        print("Address", hyperlink.address)
            print text
        except NotImplementedError:
            pass

and the presentation : https://www.dropbox.com/s/kyzyzr1vgaceye0/example.pptx?dl=0
The urls that are part of text rather than a shape are identified correctly. As long as they are URLS! If they are are hyperlinks to another part of the file we have
Traceback (most recent call last):
  File "./grab_text.py", line 120, in <module>
    if hyperlink.address is not None:
  File "/Library/Python/2.7/site-packages/python_pptx-0.5.7-py2.7.egg/pptx/text/text.py", line 415, in address
    return self.part.target_ref(self._hlinkClick.rId)
  File "/Library/Python/2.7/site-packages/python_pptx-0.5.7-py2.7.egg/pptx/opc/package.py", line 320, in target_ref
    rel = self.rels[rId]
KeyError: ''

Because the rId is blank at that point.
For debugging purposes, I altered the target_ref function as follows
def target_ref(self, rId):
315       try:
316         """
317         Return URL contained in target ref of relationship identified by
318         *rId*.
319         """
320         rel = self.rels[rId]
321         print "Dictionary"
322         for key in self.rels:
323            print key, 'corresponds to', self.rels[key].target_ref
324         return rel.target_ref
325       except KeyError:
326         return "No to look up with"

Obviously it now swallows the exception just to keep me moving, but it also prints out the content of the dictionary:
Dictionary
rId5 corresponds to http://google.com
rId4 corresponds to https://www.dropbox.com/home
rId3 corresponds to http://joereddington
rId2 corresponds to slide1.xml
rId1 corresponds to ../slideLayouts/slideLayout6.xml

Which shows that the hyperlinks to slides are definitely in there, as are the hyperlinks from shapes.
So, the thing I'm now curious about is: is the 'Keyerror' issue a bug? Also - given I think I can hack my way to the rels link information, is there a way of say, getting the xml that corrasponds to a shape so I can hack my way to the link?
Thanks in advance - I'm aware we're both dealing with this sort of thing in our spare time :)
J

Hi Mark, can I bump this a little bit? I'm aware you are busy with other things at the moment but I wanted to try and stay on the radar...

Added in v0.5.8.


[TACHYON-1093] Code formatting cleanup

See PR #1683 (comments by haoyuan)

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/5652/
Test PASSed.

Thanks for the format cleanup. Since you're on this, could you also remove the comment "  // (andrea) why doesn't this method return a file.TachyonFile instance instead?" in TachyonFileSystemCore?

Sure, commende removed. However, out of curiosity, do you have an answer to the question in my comment? Cheers!

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/5662/
Test PASSed.

Yeah, actually I am with you on that: we should always return tachyonFile instead of fileId. If you dont mind, could you open a PR to change the return type of create and loadMetadata?

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/5668/
Test PASSed.

LGTM +1

LGTM


feat(keybindings): execute keybinding starting with : as ex command

A simple change that allows keybindings to invoke ex commands directly, and could address quite a few of the requests in #1423. The downside of doing this instead of implementing proper commands is of course that they won't be discoverable through the command palette, but this doesn't prevent proper commands to be added either, so I think it might be a nice convenient stop-gap solution to unblock people at least.
Example:
  {key: "kk", command: ":split", when: "editorTextFocus"}

Let me know if you think this is a viable approach @bryphe, then I'll add a few integration tests before it's merged.
Fixes #807

Thanks for implementing this, @glennsl ! Approach looks great to me.

I think it might be a nice convenient stop-gap solution to unblock people at least.

I agree - and definitely fits in to the 0.6.0 milestone work. 👍
This fixes https://github.com/onivim/oni2/issues/807 as well

Oh wow, exactly what you proposed. And here I was thinking I had novel ideas! Probably had that one stored away in the subconscious somewhere :D I'll add some integration tests then.

I'm unfortunately not able to run any integration tests locally because I get a sanitation error:
==63020==ERROR: AddressSanitizer failed to deallocate 0x2000 (8192) bytes at address 0x6250001e0100
==63020==AddressSanitizer CHECK failed: /build/gcc/src/gcc/libsanitizer/sanitizer_common/sanitizer_posix.cpp:61 "(("unable to unmap" && 0)) != (0)" (0x0, 0x0)

So I'm just pushing and hoping they'll work...

/azp run

CI is being very difficult, but it's just Validate Windows Release that times out now.

Thanks for adding those tests, @glennsl - sorry about the CI difficulties. I bumped the Windows machine and it's green now. Looks great!

Oh, and with license key bounty - this would definitely be a qualifier for it. LMK if you want an extra license key, @glennsl

Thanks! But no need to worry about bounties on my account 🙂


(EAI-201): Step-back prompting for MongoDB AI Chatbot

Jira: https://jira.mongodb.org/browse/EAI-201
Changes

Generate search query with step-back prompting
Separate LLM calls to generate search query and extract metadata for better results
Bundle all context for LLM in the user message

Notes

[ ] Will need to update the production Azure OpenAI ChatGPT API version to use the 0610 model w/ function calling. will need to get our deployment updated to have more tokens/minute using that version as well.

Amazing...

hopefully 🙏


Update jansi to 2.1.1

Updates org.fusesource.jansi:jansi from 1.18 to 2.1.1.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.fusesource.jansi", artifactId = "jansi" } ]

labels: library-update

Superseded by #725.

Superseded by #725.


Powershell Syntax

Can we see the backticks removed and splat these cmdlet calls?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 97d342da-1086-5136-71d7-445bef390373
Version Independent ID: 69234e98-d84c-8cca-6f6b-65baae35ecab
Content: Renew an Azure Application Gateway certificate
Content Source: articles/application-gateway/renew-certificates.md
Service: application-gateway
GitHub Login: @greg-lindsay
Microsoft Alias: greglin

@jkav58,
Thank you for your feedback. The backticks are added to make the commands easy to read and understand. This holds true for all the PowerShell documents in Application Gateway.
We will now proceed to close this issue out. Please tag me in your response if you have any questions. Thank you!

@ChaitanyaNaykodi-MSFT
Yes, this debate has been ongoing for some time (for example https://powershell.org/2013/07/powershell-great-debate-backticks/) so I guess it is a personal style choice.


Styling as part of the components

According to the Angular Style Guide, components should include their own css. Global styles shouldn't be used: https://angular.io/guide/styleguide#extract-templates-and-styles-to-their-own-files
If the styles are part of the global style.css, then they have to be loaded always when the site loads. They cannot be loaded as part of a lazy module.
I can see the component files already have a .css file attached, so changing things around shouldn't be that much of a problem, although it will break existing projects that includes the current style.css file.

I consider this approach is not suitable to the picker development


XP-261 move tests to tests dir

References
https://xainag.atlassian.net/browse/XP-261
Summary
Move tests to its own dir
Are there any open tasks/blockers for the ticket?
None

Reviewer checklist
Reviewer agreement:

Reviewers assign themselves at the start of the review.
Reviewers do not commit or merge the merge request.
Reviewers have to check and mark items in the checklist.

Merge request checklist

[x] Conforms to the merge request title naming XP-XXX <a description in imperative form>.
[x] Each commit conforms to the naming convention XP-XXX <a description in imperative form>.
[x] Linked the ticket in the merge request title or the references section.
[x] Added an informative merge request summary.

Code checklist

[x] Conforms to the branch naming XP-XXX-<a_small_stub>.
[x] Passed scope checks.
[x] Added or updated tests if needed.
[x] Added or updated code documentation if needed.
[x] Conforms to Google docstring style.
[x] Conforms to XAIN structlog style.

closing, created a new PR against development: https://github.com/xainag/xain-fl/pull/197


Problem with RadioButtons

On the latest commit of the development version (#9350667), Radio Buttons doesn't seem to be working; the value doesn't seem to update:

Wheras for the Dropdown widget, it does:

Environment: OS X 10.11, Chrome, using this Conda environment:
environment.txt

Eventually fixed this - I think it may have been version clash occurring with files stored in the
~/Library/Jupyter directory

Awesome. Thanks I could not reproduce.


API resources can be accessed using id_token

Description
Hi Team.
When executing an OpenID authorization_code OAuth2 flow to generate an access token it returns both an access_token and an id_token.
The problem comes when the user is able to use both the id_token and the access_token in the Authorization header to consume the API resources of the APIM.
According to the OpenID Connect Core [1] and RFC 6749 Oauth2 Framework [2]:
ID Tokens (OpenID Connect Core Spec, Sections 1.1, 2, <IP_ADDRESS>) are used to authenticate the user and contain claims about the user's identity and authentication event.
Access Tokens (OAuth 2.0 Spec, Sections 1.4, 7, 3.3, 5.1) are used to authorize access to protected resources and contain information about the granted permissions and scope.
Therefore, an id_token should not be used as an access_token and should be used only to authenticate the user.
[1] - https://openid.net/specs/openid-connect-core-1_0.html
[2] - https://datatracker.ietf.org/doc/html/rfc6749
Thanks,
Janaka.
Steps to Reproduce
Call authorize endpoint:
GET https:///oauth2/authorize?response_type=code&client_id=xxxx&state=state&scope=openid&redirect_uri=xxxx
Call token endpoint:
POST https:///oauth2/token
grant_type=authorization_code&code=xxxx&redirect_uri=xxxx&client_id=xxx&client_secret=xxxx
Response of the token endpoint:
{ "access_token":"xxxxx",
"scope":"openid",
"id_token":"xxx",
"token_type":"Bearer",
"expires_in":3600
}
Invoke the API resource with id_token
Affected Component
APIM
Version
4.1.0
Environment Details (with versions)
No response
Relevant Log Output
No response
Related Issues
No response
Suggested Labels
wso2am-4.1.0

Hi @lasanthaS ,
Please share the public fix and close this issue.
Regards,
Tharani.


Claudiol machine sensor updates

Updates for manuela-tst-all namespace.
Updated the manifests for messaging-kafka and messaging.

Too much tekton gunk, let's do this a different way


Update configure-teams-live-events.md

Updating to include Ramp

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@cichur PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


feat: add semantic resource attributes in node sdk config

Which problem is this PR solving?
Fixes #2799
configure service name directly in the node SDK config
Short description of the changes
add optional serviceName option in node SDK constructor, which behind the scenes creates another resource that is merged with any one that may have been passed in. this occurs after autoDetectResources so that the in code config set service name trumps any env set service name
Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)
[x] This change requires a documentation update

How Has This Been Tested?

[x] unit tests

Checklist:

[x] Followed the style guidelines of this project
[x] Unit tests have been added
[x] Documentation has been updated

OK, now back to original proposal of just adding a serviceName config option 👍


Add generic conditional container plugin.

See #325 for explanation
Not intended as merge worthy, but just something to help visualize the idea. Let me know if it's worth refining.

This is a great start, @gnfzdz! I'd like to assist on implementing a number of initial condition providers, writing up documentation, and adding unit tests. I can pick this up after #313 is reviewed and merged, as it will dramatically affect how the unit tests are implemented (and will make it possible to test the condition providers across all supported operating systems).

@kurtmckee Thanks for the positive feedback. I can definitely see how this would work better with cross platform support. Thanks for the offer to help?
@anishathalye Have you had a chance to review the proposal? I did release this as a separate plugin, but it would feel a lot cleaner if at least the abstract base Condition was available in the core Dotbot project.


Projects can not be opened from vrc-get-gui due to Unity license error

Description of the bug / バグの説明
When I open my Unity project from vpm-get-gui, Unity responds "License Error" although I own a valid license.
I can open projects from Unity Hub without the error.
Steps to Reproduce / 再現手順
Click "Open Unity" button
Expected Behavior / 想定される挙動
Projects can be opened
Screenshots / スクリーンショット

Software Version / バージョン情報
v0.1.0-beta.15
OS
macOS 14.3.1
Additional context / 補足
I imagine that this is my environmental issue, but I don't know why this happens.
If this were simply my Unity license related issue, I could not open projects from Unity Hub either, I think.
That's why I open this issue, so in case of NTF, close this issue.

Can you open the unity from command line?

So, what do you mean? Do you mean vpm check unity?

No.
We are able to launch unity with /Applications/Unity/Hub/Editor/<version>/Unity.app/Contents/MacOS/Unity -projectPath /path/to/project and vrc-get-gui internally use this to launch unity so could you try this manually?

I have a different path so I executed /Applications/2022.3.6f1/Unity.app/Contents/MacOS/Unity -projectPath /path/to/project only to fail with the same error.

Now I know this is my environmental issue rather than vrc-get-gui issue, but I have no idea why I suddenly had this error.

According to the installed path, it looks you have installed unity without unity hub. Did you installed the unity with Unity Hub or changed unity installation path in Unity Hub?

it looks you have installed unity without unity hub

No, I installed Unity with Unity Hub and I did not change the path either.

I've solved this problem on my own. I found Unity license file (Unity_lic.ulf) did not exist at '/Library/Application\ Support/Unity' so I added my personal license with Unity Hub again, which enabled me to open my project with command (and vrc-get-gui); though I do not know why my license file disappeared at the directory nor it was vrc-get-gui related issue (I guess not). I apologise for opening this issue which is not estimably related to your project.
Now you can close this case with NTF.

I think it's better to detect the situation and suggest user to re-activate unity license so I keep this issue open.


Conditional Logic Guidance

Description:
Create public documentation and guidance for using conditional logic.
Acceptance Criteria:

[ ] Glossary of terms for the new product features @anikbrazeau @samsadasivan
[ ] Outline a strategy for communicating new features @anikbrazeau
[ ] Documentation covers how to set up and use conditional logic
[ ] Include examples and best practices
[ ] Documentation is published as a Guidance page

Conditional Logic Feature doc: https://docs.google.com/document/d/1nF5YW6lLQMLGdyljwxtKqUH4FoYaebw71w_Nv0Kt2Ew/edit#heading=h.ru24wrgw4m14

https://github.com/cds-snc/platform-forms-client/pull/3818

Closing as covered in other new broken down cards


Excessive taskhost process detection

Generates an alert if there are an excessive number of taskhost processes running on a machine in an hour window.  This behavior was seen with Meterpreter.

This PR was created by Automated Detection Testing Service :robot:
Tested detections:
Excessive number of taskhost processes :white_check_mark:

diskUsage: 450560
runDuration: 1.308s
scanCount: 46


Link API references in topics to API ref docs

Blocked by #26

@guardrex Why is this blocked? Even without the automation, we have API refs in our docs already, right?

We don't for API. I only held off because you mentioned that you were going to change the way API docs are built (i.e., not just assemblies pushed to the repo). Therefore, I guessed (perhaps incorrectly) that the links to API topics might change. Right now, Blazor API mentions are merely code fenced. If you'd like them linked now, no problem ... I'll take care of it.

Ok ... sounds good. I'll find a few free minutes this weekend to update the references.

It's not urgent or high priority. Feel free to spend those extra minutes on yourself :smile:


Add issues

As the maintainer of a collaborative repository, it's essential to keep Issues up-to-date to help the student community understand how they can contribute. By creating a mix of issues, from simple tasks like updating documentation to more complex ones like developing new features, you can cater to contributors of all skill levels. The more actively you engage, the more chances there are for meaningful collaboration.
Recommendations:

Regularly create issues of varying difficulty and label them accordingly:  wob24, Level 1, Level 2, Level 3 etc.
Scoring for the leaderboard is as follows:

Each PR earns one point.
Level 1 Issues: 10 points.
Level 2 Issues: 25 points.
Level 3 Issues: 45 points.

Include documentation-related issues to provide an easier entry point for beginners to explore open-source.
Even if you plan on solving an issue yourself, generate it to maintain the repository’s activity.
Commit and contribute frequently to prevent the repo from becoming inactive.

Reference: https://github.com/Durgesh4993/CryptoX/issues

Hi Harish,
I saw your website and i really felt the UI is not good. I can make the UI and UX better for you.
If you permit I can give it a fresh look !

Hi Harish,
I saw your website and i really felt the UI is not good. I can make the UI and UX better for you.
If you permit I can give it a fresh look !

You need to raise an issue


🚀 Feature: Requesting that the chunk file size be a configurable option

🔖 Feature description
Requesting that the chunk file size be a configurable option.
Right now, I think it's fixed at 10MB.
I would like to increase it to something like 50MB.
🎤 Pitch
The reason for this request is to hopefully speed up the upload speed.

Added in the newest version. You can find the setting in the share tab of the admin settings.

That is very quick; I'll test it now.
Thanks a lot.

Increasing it, I always got this error during large file uploads:
413 Payload Too Large
I tried 32, 25, and 20MB chunk sizes so far.
Cloudflare allows 100MB; my Nginx allows 1GB.

On the front-end app, I got this log:
node[2416818]:
API response for /api/shares/1gb/files/e9d69354-515d-455d-a608-2468044ce8b5 exceeds 4MB.
API Routes are meant to respond quickly. https://nextjs.org/docs/messages/api-routes-response-size-limit

So I lower it down to 4MB to prevent the error above.

Oh I have found the issue. The limit for the body parser gets set in the bootstrap function of the backend which only gets called when the application starts. So if you change the chunk size the changes will only be applied when you restart the application.
I'll release a fix later today until then you have to restart the application. Thanks for your feedback!

On the front-end app, I got this log:
node[2416818]:
API response for /api/shares/1gb/files/e9d69354-515d-455d-a608-2468044ce8b5 exceeds 4MB.
API Routes are meant to respond quickly. https://nextjs.org/docs/messages/api-routes-response-size-limit

I think this error log or warning happens when a large file is downloaded, like this URI path for a 1GB file.
Not related to file uploads, I guess.

So the error occurs when you want to download a file?

So, does the error occur when you try to download a file?

Yes correct.


yopass-ng: init at 0.0.14

Yopass, selfhosted secret sharing service, as convenient, all-in-one embedded, multilingual, hardened single binary.

[x] nixos native integration via service.yopass-ng... (allows to host yopass native, without docker!) prepared, via separate commit

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[x] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 352574 run on x86_64-linux 1

1 package built:

yopass-ng

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/2101


Let's crypt init fix

Let's crypt init. script fix. Without it certificate may become self-signed.

@olenagerasimova , please review.


add support for kaslr on ps4 3.55, 4.00 and 4.01

#4 hardcode some offsets of symbols

I have non-merged changes to support versioned offsets for locations without symbols/more recent firmware. This would conflict with that so I'll have to merge them at some point...


[Feature] Support PERCENT_RANK window function

What type of PR is this:

[ ] BugFix
[x] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[x] Doc
[ ] Tool

Which issues of this PR fixes:

Fixes #13021 #19632
Problem Summary(Required):

The PERCENT_RANK() function calculates the relative rank of a row within a result set as a percentage. It returns the percentage of partition values less than the value in the current row, excluding the highest value. The return values range from 0 to 1. This function is useful for percentile calculations and analyzing data distribution.
The PERCENT_RANK() function is calculated using the following formula, where rank represents the row rank and rows represents the number of partition rows:
(rank - 1) / (rows - 1)

It can not been used with the windowing clause. And for the implementation, it uses RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW.
NOTE that, PERCENT_RANK need the boundary of partition to calculate its value. For these functions, we must wait util the partition finished. Therefore, it uses _process_by_partition_for_unbounded_preceding_range_frame_materializing.
The PERCENT_RANK window function is only supported by the pipeline engine. BE will report the NotSupported error, when non-pipeline engine executes PERCENT_RANK in BE.
Checklist:

[x] I have added test cases for my bug fix or my new feature
[x] This pr will affect users' behaviors
[x] This pr needs user documentation (for new or modified features or behaviors)
[x] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[ ] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

[FE PR Coverage Check]
:heart_eyes: pass : 9 / 9 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/analysis/AnalyticExpr.java
2
2
100.00%
[]

:large_blue_circle:
com/starrocks/catalog/FunctionSet.java
3
3
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/WindowTransformer.java
2
2
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/AnalyticAnalyzer.java
2
2
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 9 / 9 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/analysis/AnalyticExpr.java
2
2
100.00%
[]

:large_blue_circle:
com/starrocks/catalog/FunctionSet.java
3
3
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/WindowTransformer.java
2
2
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/AnalyticAnalyzer.java
2
2
100.00%
[]

@Mergifyio backport branch-3.1


Simplify project creation and recreate the default setup example.

Also avoids a TOCTOU issue when creating projects.
@mlange-42 I wonder if we should change the file extensions now that we are limited to Markdown: The document is usually almost-valid Markdown as well even if has unresolved transclusions and such. Hence it could make sense to drop README.md.md and just use README.md resp. <file> instead of <file>.md as the main file?

I was already thinking about simplifying the project as created by create, and also about getting rid of the .md.md. Now that features are well explained in the book, I also don't see the need for including a lot of features in it.
The problem with the double extension, however, is that the file: feature would be required for the entrypoint. It would thus already completely rely on a more advanced feature.
I think we now have two very different approaches how to structure a project. The "old" one (main.rs.md -> main.rs) is useful for small one-file projects, like a single algorithm, while the "new" one, for larget projects, would use a file README.md as the main document and do everything with file: or links and transclusions (the latter generating file: blocks).
So the question is which approach to chose for project setup.

I was already thinking about simplifying the project as created by create, and also about getting rid of the .md.md. Now that features are well explained in the book, I also don't see the need for including a lot of features in it.
The problem with the double extension, however, is that the file: feature would be required for the entrypoint. It would thus already completely rely on a more advanced feature.
I think we now have two very different approaches how to structure a project. The "old" one (main.rs.md -> main.rs) is useful for small one-file projects, like a single algorithm, while the "new" one, for larget projects, would use a file README.md as the main document and do everything with file: or links and transclusions (the latter generating file: blocks).
So the question is which approach to chose for project setup.

Further, I was asking myself whether we should remove all the comments from the Yarner.toml tempate, now that all options are explained in the book. It looks quite cluttered and confusing. It may, however, still be useful to provide that information to users without requiring them to open the website. Not really sure...

Further, I was asking myself whether we should remove all the comments from the Yarner.toml tempate, now that all options are explained in the book. It looks quite cluttered and confusing. It may, however, still be useful to provide that information to users without requiring them to open the website. Not really sure...

So the question is which approach to chose for project setup.

I think we should stick we the README.md as the main file.

The problem with the double extension, however, is that the file: feature would be required for the entrypoint. It would thus already completely rely on a more advanced feature.

We could try to automatically detect the case where the document source already ends in .md just not remove the suffix in that case? If someone really needs .md being removed, they could use file:. Then again, this kind of implicit assumption might make the whole thing harder to understand?

It may, however, still be useful to provide that information to users without requiring them to open the website.

I think this is usually the motivation for including comments. So I'd we should either provide a really minimal configuration template without comments or defaults, or stick to the full blown one we currently have. Personally, I prefer the minimal version but I like the defaults to update automatically, but this a bit of a cultural schism AFAIK.

So the question is which approach to chose for project setup.

I think we should stick we the README.md as the main file.

The problem with the double extension, however, is that the file: feature would be required for the entrypoint. It would thus already completely rely on a more advanced feature.

We could try to automatically detect the case where the document source already ends in .md just not remove the suffix in that case? If someone really needs .md being removed, they could use file:. Then again, this kind of implicit assumption might make the whole thing harder to understand?

It may, however, still be useful to provide that information to users without requiring them to open the website.

I think this is usually the motivation for including comments. So I'd we should either provide a really minimal configuration template without comments or defaults, or stick to the full blown one we currently have. Personally, I prefer the minimal version but I like the defaults to update automatically, but this a bit of a cultural schism AFAIK.

We could try to automatically detect the case where the document source already ends in .md just not remove the suffix in that case?

Not sure if I understand correctly. Do you mean detect it during project creation, or during the build process?
It you are referring to the build process, shouldn't we detect the cases where there is no "double extension" and write to an .md file in these cases?
Otherwise, if our refer to project creation, it should be "where the document source already ends in .md just not add the suffix in that case"
Or rather, we need both then?

Personally, I prefer the minimal version but I like the defaults to update automatically...

Not really sure what you mean here

We could try to automatically detect the case where the document source already ends in .md just not remove the suffix in that case?

Not sure if I understand correctly. Do you mean detect it during project creation, or during the build process?
It you are referring to the build process, shouldn't we detect the cases where there is no "double extension" and write to an .md file in these cases?
Otherwise, if our refer to project creation, it should be "where the document source already ends in .md just not add the suffix in that case"
Or rather, we need both then?

Personally, I prefer the minimal version but I like the defaults to update automatically...

Not really sure what you mean here

It you are referring to the build process, shouldn't we detect the cases where there is no "double extension" and write to an .md file in these cases?

I was referring to the build process. We currently "print" foo.bar.md.md into docs/foo.bar.md AFAIU. I would suggest to relax for Markdown files only and "print" foo.bar.md into docs/foo.bar.md as well.

Not really sure what you mean here

For example, all keys under [parser] have defaults and hence could be omitted from Yarner.toml. This would imply that if these defaults ever change, the user will automatically start working with the new defaults. This could be considered useful but also problematic as it might break the current document.

It you are referring to the build process, shouldn't we detect the cases where there is no "double extension" and write to an .md file in these cases?

I was referring to the build process. We currently "print" foo.bar.md.md into docs/foo.bar.md AFAIU. I would suggest to relax for Markdown files only and "print" foo.bar.md into docs/foo.bar.md as well.

Not really sure what you mean here

For example, all keys under [parser] have defaults and hence could be omitted from Yarner.toml. This would imply that if these defaults ever change, the user will automatically start working with the new defaults. This could be considered useful but also problematic as it might break the current document.

I was referring to the build process. We currently "print" foo.bar.md.md into docs/foo.bar.md AFAIU. I would suggest to relax for Markdown files only and "print" foo.bar.md into docs/foo.bar.md as well.

Now, I am very confused: We are actually writing docs/README.md.md? Why is that?

I was referring to the build process. We currently "print" foo.bar.md.md into docs/foo.bar.md AFAIU. I would suggest to relax for Markdown files only and "print" foo.bar.md into docs/foo.bar.md as well.

Now, I am very confused: We are actually writing docs/README.md.md? Why is that?

We currently "print" foo.bar.md.md into docs/foo.bar.md AFAIU.

No. From a file file.ext.md, with an unnamed code block, we generate:

docs/file.ext.md
code/file.ext

This could be considered useful but also problematic as it might break the current document.

I would definitely prefer to leave the default in there.

We currently "print" foo.bar.md.md into docs/foo.bar.md AFAIU.

No. From a file file.ext.md, with an unnamed code block, we generate:

docs/file.ext.md
code/file.ext

This could be considered useful but also problematic as it might break the current document.

I would definitely prefer to leave the default in there.

Now, I am very confused: We are actually writing docs/README.md.md? Why is that?

I was a bit too slow... 😄
It is for consistency only, and should become clear with another "first" extension as in my above example.

Now, I am very confused: We are actually writing docs/README.md.md? Why is that?

I was a bit too slow... 😄
It is for consistency only, and should become clear with another "first" extension as in my above example.

Now, I am very confused: We are actually writing docs/README.md.md? Why is that?

I think my confusion stems from writing "documentation", i.e. "README.md" into the "code" output. Maybe we should restructure the example to use "README.md" as the document source, creating "docs/README.md" as the documentation and "code/Cargo.toml" and "code/src/main.rs" as the code output?
Thinking a bit further, does the "code" and "docs" really make sense for Markdown-based projects? Do we want to distribute those separately? Writing a "code/README.md" as-is this was no literate programming project at all?

Now, I am very confused: We are actually writing docs/README.md.md? Why is that?

I think my confusion stems from writing "documentation", i.e. "README.md" into the "code" output. Maybe we should restructure the example to use "README.md" as the document source, creating "docs/README.md" as the documentation and "code/Cargo.toml" and "code/src/main.rs" as the code output?
Thinking a bit further, does the "code" and "docs" really make sense for Markdown-based projects? Do we want to distribute those separately? Writing a "code/README.md" as-is this was no literate programming project at all?

Maybe we should restructure the example to use "README.md" as the document source, creating "docs/README.md" as the documentation and "code/Cargo.toml" and "code/src/main.rs" as the code output?

Yes, I think this is the best option. Although it "skips" the most simple usage pattern of an unnamed code block as entrypoint.

Thinking a bit further, does the "code" and "docs" split really make sense for Markdown-based projects? Do we want to distribute those separately? Writing a "code/README.md" as-if this was no literate programming project at all?

An interesting consideration. I think currently the best approach when documentation output is required (as with, e.g., transclusions) and the sources can't be used directly (as we did in the ECS tutorial) is like this:

Have Markdown sources in a sub-folder of the project, e.g. lp. And then:
root = "lp"
code_dir = "../code" (may even be ignored be gitignore)
doc_dir = "..", I.e. the output README.md would be at the toplevelof the project

project
  |-- lp (root)
  |    `-- README.md (source)
  |-- code (code output)
  |    |-- Cargo.toml
  |    `-- src
  |         `-- main.rs
  |-- README.md (doc output)
  `-- Yarner.toml

It would also be possible to have code_dir and code_dir point to the same directory, which would result in the behaviour you suggested. But I am not sure if that could result in any conflicts. But the above example could, too.
I am away now for an hour or so...

Maybe we should restructure the example to use "README.md" as the document source, creating "docs/README.md" as the documentation and "code/Cargo.toml" and "code/src/main.rs" as the code output?

Yes, I think this is the best option. Although it "skips" the most simple usage pattern of an unnamed code block as entrypoint.

Thinking a bit further, does the "code" and "docs" split really make sense for Markdown-based projects? Do we want to distribute those separately? Writing a "code/README.md" as-if this was no literate programming project at all?

An interesting consideration. I think currently the best approach when documentation output is required (as with, e.g., transclusions) and the sources can't be used directly (as we did in the ECS tutorial) is like this:

Have Markdown sources in a sub-folder of the project, e.g. lp. And then:
root = "lp"
code_dir = "../code" (may even be ignored be gitignore)
doc_dir = "..", I.e. the output README.md would be at the toplevelof the project

project
  |-- lp (root)
  |    `-- README.md (source)
  |-- code (code output)
  |    |-- Cargo.toml
  |    `-- src
  |         `-- main.rs
  |-- README.md (doc output)
  `-- Yarner.toml

It would also be possible to have code_dir and code_dir point to the same directory, which would result in the behaviour you suggested. But I am not sure if that could result in any conflicts. But the above example could, too.
I am away now for an hour or so...

Yes, I think this is the best option. Although it "skips" the most simple usage pattern of an unnamed code block as entrypoint.

Will do. But shouldn't we just remove the "main_file" argument then as the name "README.md" is pretty much set then. "yarner create" would also be a bit simple to get started?

I think currently the best approach when documentation output is required.

I suggest we stick to modifying the default project structure as above and defer this discussion to the book?

Yes, I think this is the best option. Although it "skips" the most simple usage pattern of an unnamed code block as entrypoint.

Will do. But shouldn't we just remove the "main_file" argument then as the name "README.md" is pretty much set then. "yarner create" would also be a bit simple to get started?

I think currently the best approach when documentation output is required.

I suggest we stick to modifying the default project structure as above and defer this discussion to the book?

Will do. But shouldn't we just remove the "main_file" argument then as the name "README.md" is pretty much set then.

You mean the CLI argument? If users want a different main file (e.g. main.rs.md), they would have to rename the file and change the files option in the toml.
But I agree to go this route. 👍

Will do. But shouldn't we just remove the "main_file" argument then as the name "README.md" is pretty much set then.

You mean the CLI argument? If users want a different main file (e.g. main.rs.md), they would have to rename the file and change the files option in the toml.
But I agree to go this route. 👍

Or the main_file CLI argument could also be optional and default to README (without extension then?)

Or the main_file CLI argument could also be optional and default to README (without extension then?)

Or the main_file CLI argument could also be optional and default to README (without extension then?)

I would leave it out completely as the contents of the README will be useless for a "main.rs.md" which I think will be more confusing than just replacing the it manually.

Or the main_file CLI argument could also be optional and default to README (without extension then?)

I would leave it out completely as the contents of the README will be useless for a "main.rs.md" which I think will be more confusing than just replacing the it manually.

But then, when it really takes no parameters, I would prefer to indeed rename create to init.

But then, when it really takes no parameters, I would prefer to indeed rename create to init.

Ok, I tried to update as discussed. Please have a look. I guess we also need to update the guide... :-\

Ok, I tried to update as discussed. Please have a look. I guess we also need to update the guide... :-\

On second thought, I will back out the last commit as this implies much changes in other places than I am willing to commit to right now... So this stays limited to refactoring and only discussing on file creation needs to rechecked against the latest version.

On second thought, I will back out the last commit as this implies much changes in other places than I am willing to commit to right now... So this stays limited to refactoring and only discussing on file creation needs to rechecked against the latest version.

So the massive changes go to another MR as I understand? Fine! For what changed here: approved!

So the massive changes go to another MR as I understand? Fine! For what changed here: approved!


Database performance

The startup time can be greatly improved by creating an index on the LCSC part number:
CREATE UNIQUE INDEX parts_lcsc_part_index ON parts ('LCSC Part');

Before:

After:

(populate_footprint_list() from 2552ms to 137ms)
Another nice touch would be to open the database only once when the plugin starts up instead of whenever needed.
P.S.: I used VizTracer for the profiling:
--- a/plugin.py
+++ b/plugin.py
@@ -2,6 +2,7 @@

 import os

+from viztracer import VizTracer
 from pcbnew import ActionPlugin  # pylint: disable=import-error

 from .mainwindow import JLCPCBTools
@@ -25,6 +26,12 @@ class JLCPCBPlugin(ActionPlugin):

     def Run(self):
         """Overwrite Run"""
+        tracer = VizTracer()
+        tracer.start()
+
         dialog = JLCPCBTools(None)
         dialog.Center()
         dialog.Show()
+
+        tracer.stop()
+        tracer.save("/tmp/jlcpcb_profile.html")

As always, PRs are highly appreciated 😁
I simply lack the knowledge to improve stuff like that 😅

Some inspiration:

You do a lot of string concatenation for query building, like
query += f' ORDER BY "{self.order_by}" COLLATE naturalsort {self.order_dir}'

That might be bearable in this case (plugin for desktop use), but introduces SQL injection vulnerabilities.

For proper escaping always use the SQL library's parameter handling (cursor.execute(sql, [param_1, param_2])).

Is there a reason to not merge all plugin databases (mappings, parts, rotations) into a single one? “Parts” might make some sense to keep separate for easier backups.

I couldn't find any migration handling to update the database(s) for existing users. A common solution is to store the version number of the database structure inside of the database, typically in a table named “schema”. A detailed description of the concept can be found here. There are sophisticated libraries, but it can also be kept pretty simple.
Otherwise, how would you know when to call CREATE INDEX to upgrade existing databases? You could call CREATE INDEX IF NOT EXISTS on every startup, but that's cheating.  😉

HTH

As an example – and just for fun –, I gave ChatGPT a little spin:

Write a Python class to handle versioned migrations in a SQLite database. The class should be given a database connection and a list of versioned migrations (like [{'version': 1, 'sql': 'CREATE TABLE...'}, {'version': 2, 'sql': 'ALTER TABLE...'}]) on initialisation and have a method to do the actual migration. That migation method should accept a version identifier to migrate to as parameter. Also include a method to setup a database table for schema tracking when necessary, make it include timestamps. Rollback is not required, only forward migration.

Here's an updated version of the Python class that handles versioned migrations in a SQLite database, considering the updated requirements:
 import sqlite3
 from datetime import datetime

 class DatabaseMigrator:
     def __init__(self, conn, migrations):
         self.conn = conn
         self.migrations = migrations

     def create_schema_table(self):
         cursor = self.conn.cursor()
         cursor.execute("CREATE TABLE IF NOT EXISTS schema_versions (version INTEGER PRIMARY KEY, timestamp TIMESTAMP)")
         self.conn.commit()

     def migrate(self, target_version):
         cursor = self.conn.cursor()

         current_version = self.get_current_version()
         if current_version >= target_version:
             print(f"Database already at version {target_version}")
             return

         for migration in self.migrations[current_version:target_version]:
             try:
                 cursor.execute(migration["sql"])
                 current_version += 1
             except Exception as e:
                 print(f"Error migrating to version {current_version+1}: {str(e)}")
                 return

         timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
         cursor.execute("INSERT OR REPLACE INTO schema_versions (version, timestamp) VALUES (?, ?)", (current_version, timestamp))
         self.conn.commit()

         print(f"Database migrated to version {current_version}")

     def get_current_version(self):
         cursor = self.conn.cursor()
         cursor.execute("SELECT MAX(version) FROM schema_versions")
         result = cursor.fetchone()
         if result[0] is None:
             return 0
         return result[0]

Usage Example:
 # Assume the SQLite database file 'mydatabase.db' already exists.
 conn = sqlite3.connect('mydatabase.db')

 migrations = [
     {'version': 1, 'sql': '''
         CREATE TABLE IF NOT EXISTS users (
             id INTEGER PRIMARY KEY AUTOINCREMENT,
             name TEXT,
             email TEXT
         )
     '''},
     {'version': 2, 'sql': '''
         CREATE TABLE IF NOT EXISTS products (
             id INTEGER PRIMARY KEY AUTOINCREMENT,
             name TEXT,
             price REAL
         )
     '''}
 ]

 migrator = DatabaseMigrator(conn, migrations)

 # Create the schema_versions table if it doesn't exist
 migrator.create_schema_table()

 # Migrate to the latest version
 target_version = len(migrations)
 migrator.migrate(target_version)

Note: In this updated implementation, each migration is now represented as a dictionary with keys "version" and "sql". The "version" key represents the version identifier, and
"sql" key holds the SQL statement for that migration. The migrate() method has been updated to use these keys to execute the SQL statements in the migrations. Additionally, a
timestamp is recorded in the schema_versions table for each migration for tracking purposes.

Good, but use the "version" id stored in the list of migrations to find the migrations to apply.

Apologies for the oversight. Here's the updated implementation of the migrate() method that uses the "version" id stored in the list of migrations:
 def migrate(self, target_version):
     cursor = self.conn.cursor()

     current_version = self.get_current_version()
     if current_version >= target_version:
         print(f"Database already at version {target_version}")
         return

     for migration in self.migrations:
         version = migration["version"]
         if current_version < version <= target_version:
             try:
                 cursor.execute(migration["sql"])
                 current_version = version
             except Exception as e:
                 print(f"Error migrating to version {version}: {str(e)}")
                 return

     timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
     cursor.execute("INSERT OR REPLACE INTO schema_versions (version, timestamp) VALUES (?, ?)", (current_version, timestamp))
     self.conn.commit()

     print(f"Database migrated to version {current_version}")

Now, the migrate() method iterates over the list of migrations and checks if the version falls within the range of current version (exclusive) to target version (inclusive). If
so, it executes the corresponding SQL statement and updates the current version accordingly.
The rest of the class and usage remain the same.
Please let me know if you have any further questions!

@serpent213 I've got a PR that uses fts5 to improve search performance across all fields. I was also thinking, as you mentioned, that it made sense to keep the db open once its opened but haven't looked at that yet.

@serpent213

P.P.S.: Easy to apply without touching the source:
% cd $PLUGIN_DIR/jlcpcb
% sqlite3 parts.db
SQLite version 3.39.5 2022-10-14 20:58:05
Enter ".help" for usage hints.
sqlite> CREATE UNIQUE INDEX parts_lcsc_part_index ON parts ('LCSC Part');
sqlite> .quit

So this could be integrated into the DB creation in the GH action, right?
https://github.com/Bouni/kicad-jlcpcb-tools/blob/main/jlcparts_db_convert.py

@Bouni

So this could be integrated into the DB creation in the GH action, right?

Not sure what GH means. But yes, a first step would be to just execute CREATE INDEX once after CREATE TABLE. Or even call CREATE INDEX IF NOT EXISTS during startup to update existing DBs (will take a few seconds), being a no-op during subsequent runs.
I'm pretty busy right now, maybe I can integrate the migration mechanism one day.  🙂

@serpent213 I've just tested CREATE UNIQUE INDEX parts_lcsc_part_index ON parts ('LCSC Part'); and the plugin startup ist almost instantly compared to several seconds without.
The only downside is that @chmorgan proposal for virtual tables does not allow the creation of indexes. at least not like @serpent213 suggested.
I'm not experienced in DB tech at all, so I have to rely on you guys 😅
At the moment I cannot judge if its better to stay with a normal table and use indexes or use virtual tables to improve search speed.
Ideally we would benchmark this with VisTracer like @serpent213 showed in a previous post!

I did notice the startup can take a bit and correct that the index on the virtual table isn't the case.
Is that the only index helping startup time? I've seen that library.categories() can take several seconds.
We could make a table of categories and subcategories as a part of creating the database. That would avoid having to select them each time and resolve the issue of being unable to add indexes to the virtual table. It's maybe not as clean but imo once you switch to the fts5 changes it's hard to go back in terms of search capability and performance.
@serpent213 thoughts? And have you seen the fts5 PR?

@serpent213 thoughts? And have you seen the fts5 PR?

Read the PR, but didn't play with it, yet. Will come back to you, guys.  🙂

Had some deeper insight into the code and the GH action thingy.  😎
With #348 I had mixed feelings. When unpacking the download it crashed once, worked then on second try. Startup time is extremely high on my system, more than 34s in comparison to the viztrace numbers above. Search was very glitchy, regularly it would hang and show the beachball of death with no high CPU load or anything.  🤷🏼♂
OTOH, we had our good moments. For some periods of time I was able feel the benefits of its real-time search abilities.  🙂
I was testing with KiCAD 7.0.6-0 including Python 3.9.13.
Also the fts5-DB is 5.8 GiB here, that might be difficult to handle for some users.
To make it work, I can imagine the following:

Make it optional, a checkbox in the settings.
Create fts5-DB on demand from original parts-DB.
Maybe optimise fts5-DB schema for just full text search.
Profit!

In general, I think, the next steps should be to implement a migration mechanism like sketched out above and clean up the code. That would also bring some more little speed improvements. Is there a reason for the separate rotations and mappings DBs?

Hey @serpent213!
Have you had a chance to use this plugin for a large number of parts mappings? I put the fts5 search in place for a single board and it saved me a ton of time, of course its perfect and I can't believe you found any criticisms...  ;-)
This is good feedback. Let me see if I can't address at least a few of these over the next few days, in particular the startup time. I'll also look at putting the search into a background thread, that would avoid the UI hanging up when the search is a complex one.
Would you know what you were searching for that was causing the UI to hang up? It was previously pretty glitchy but after the tokenizer switch to trigram and not searching on empty keywords I haven't seen many, or any cases of it hanging up here.

Well, it is really a showstopper on my 2017 MacBook 4 x 2,8 GHz – not the hottest gear, but should still have enough oomph to run a parts database like this.  😉
The main problem is, my guess, that the new fts table performs very badly for general purpose (non-fts) access schemes like on startup, therefore it should only be used to index the text that is to be searched and contain a reference back to the traditional parts table.

I’m guessing it can do better. I’ll be profiling this weekend and see how much I can squeeze out of it. On the size topic if there are columns we don’t ever need or use it would help to drop them from the import even if not in the fts5 case.

@serpent213 jlcparts_db_convert.py is the script run by the Github Action which converts the data provided by jlcparts and converts that into this plugins format in a sqlite database. This database file is stored as chunks of < 100MB on Github (no larger files allowed).
When a user starts the plugin for the first time or updates the database these chunks are downloaded and merged into a normal db file again.
rotations and mappings where back in the day part of the project db, which turned out to be a bad idea because for every project the user had to do the rotations an mappings again. But the could be absolutely two tables in one db file ...
This plugin has huge room for improvements as you see 😅


[WFLY-10542] log thread stack trace which is processing timer task in debug level to diagnose timer overlap

https://issues.jboss.org/browse/WFLY-10542
[WFLY-10542] log thread stack trace which is processing timer task in debug level to diagnose timer overlap.

@stuartwdouglas Can you check this one?

@soul2zimate please address James's comment and rebase

I have rebased this one.
The powermock framework is here to mock static method. Unfortunately, powermock 1.7.3 is incompatible with Mockito > 2.8.47 (WFLY is on 2.18.0), it's advised to use version 2, version 2.0.0-beta.11 was released on github, but version 2.0.0-beta.5 is the only public available version to download on https://repo.maven.apache.org/maven2/org/powermock/powermock-api-mockito2/
I think we can hold this until there is GA version released.


[Spec Deviation] Actions allowed in default values and module level declarations

In the spec, only expressions are allowed in module-level declarations, default values for fields in records and objects and in default values for function parameters.
The spec says the following regarding the module-level declarations:

Listener declaration

listener-decl :=
metadata
[public] listener [type-descriptor] variable-name = expression ;

Module variable declaration

module-var-decl :=
metadata
[final]
typed-binding-pattern = expression ;

Module constant declaration

module-const-decl :=
metadata
[public] const [type-descriptor] identifier = const-expr ;

But currently the following are valid:
Foo mF = new("module_level");

string mV1 = mF->test(); // Should be compile error

int mV2 = start bar(); // Should be compile error

public function main() {
}

function bar() returns int {
    return 10;
}

type Foo client object {

    string name;

    function __init(string name) {
        self.name = name;
    }

    remote function test(string s = "") returns string {
        return s == "" ? self.name : self.name + "->" + s;
    }

    remote function getInt() returns int => 100;
};

The same is true regarding using actions for default values of params and record/object fields since the spec defines default values as follows:

default-value := = expression

function actionsInParams(string p1 = mF->test()) { // Should be compile error
    io:println(p1);
}

type Person record {|
    string name;
    int age = mF->getInt(); // Should be compile error
|};

Errors are now logged as expected.
The following changes were done to the original sample:

the type of mV2 was changed from int to future<int>
parentheses were introduced in the ternary expression in the body of the test method

Foo mF = new("module_level");

string mV1 = mF->test();

future<int> mV2 = start bar();
public function main() {
}

function bar() returns int {
    return 10;
}

client class Foo  {

    string name;

    function init(string name) {
        self.name = name;
    }

    remote function test(string s = "") returns string {
        return s == "" ? self.name : (self.name + "->" + s);
    }

    remote function getInt() returns int => 100;
}

$ ballerina run qwe.bal
Compiling source
        qwe.bal
error: .::qwe.bal:3:14: action invocation as an expression not allowed here
error: .::qwe.bal:5:19: action invocation as an expression not allowed here

Errors are also logged for
function actionsInParams(string p1 = mF->test()) { // action invocation as an expression not allowed here
}

type Person record {|
    string name;
    int age = mF->getInt(); // action invocation as an expression not allowed here
|};


Dunno how to fix this, what ive done wrong?

This is my first time trying Titan
what ive done wrong here?

Yea you need to fix your accounts.json. Checkout https://jsonlint.com/, paste it into there and look for errors. Then fix them.

kisak

nope, testing out with 11 accounts
me and my friends cant do it too.

I'm running on Windows 10 build 15063 x64

Case closed, its fixed, I should not change accounts into #01.
thanks


Adding support for R code

This looks like a great package! I was wondering if it would be able to support R code indentation, or at least general indentation where I want to take something like
df <- data.frame(x = c(1,2,3), myvar = c(2,3,4))
to something where the myvar aligns right under the x, i.e. the arguments all align to the right of the first '(' character

This package aligns between lines. It can be improve to do alignment on a single line but I really don't have time to work on it.
If you want to make your input on this subject you're welcome.


How to remove sidebar from static pages?

Hi,
I am able to see sidebar in each static page. How can i remove it from only static page?

Did you get answer to this ? , I'm in the same situation

Hi Marrax,
I didn't get reply for it. I override spree_application file and removed sidebar code from it.
I have commented this line in spree_application layout file.
        <%#= render :partial => 'spree/shared/sidebar' if content_for? :sidebar %>

Regards,

Here is the code to remove the sidebar:
app/overrides/my_override.rb
Deface::Override.new(:virtual_path => "spree/shared/_sidebar",
:name => "remove_sidebar",
:remove => "aside#sidebar")


Faster Power

fixes incorrect DrawType for APCs due to deserialization and misuse of a virtual impl's field in a ctor
uses SnapGridComponent to select nearby wires instead of entity list (only marginally faster if entity queries are using DynamicTree)
TheoreticalLoad double dipping property setter, reduced to one update
minor refactoring
Benefits heavily from https://github.com/space-wizards/RobustToolbox/pull/951 changes to entity intersection queries
Lag-free wire cutting w/ both

commented on this with one of my other accounts, woops


Initial version of ramp implementation for discussion.

This change includes the ramp nodedef, ramp gradient nodedef and subsequent nodegraph implementations for these two nodedefs. The ramp node includes support for both linear and smoothstep interpolation. The ramp also supports up to 30 control points.
Feel free to reach out with any questions. Looking forward to the discussion!

Looks good! Great improvement over ramplr and ramptb shaders.
It supports a constant and linear interpolation, and I wonder what it would take to also support a smooth interpolation (eg, bezier).
Also, in one of the MtlX meetings we talked about a user friendly UI that could tie all the interval and color inputs of the shader into a single gradient-looking color bar. I think there was a mention of a new UI hint in MaterialX. How would that be expressed (ie, what would that hint be)?

Looks good! Great improvement over ramplr and ramptb shaders.
It supports a constant and linear interpolation, and I wonder what it would take to also support a smooth interpolation (eg, bezier).
Also, in one of the MtlX meetings we talked about a user friendly UI that could tie all the interval and color inputs of the shader into a single gradient-looking color bar. I think there was a mention of a new UI hint in MaterialX. How would that be expressed (ie, what would that hint be)?

Actually it currently supports linear and smoothstep interpolation. Doesn't support constant yet, although that would be very nice to add!

This is looking good to me too - I'm looking forward to a MaterialX world where we have more fully featured ramps :)
I do have one meta-observation/thought.  Aside from debating the number of inputs, I think the node definition interface for ND_ramp looks pretty good and I don't think is controversial at all.  (though I do think there may also be a place for a sister ramp node in the future that just takes in a varying array - as we discussed in the TSC meeting).  But, ND_ramp_gradient seems very specific to the implementation of ND_ramp, and I wonder if it is really useful as a standard library node itself?  It's almost as if we need a way to tag node definitions as "private"  - or perhaps there is some other mechanism (node graphs?) that could enshrine the logic internal to the implementation of the node.
Also, as raised in the TSC I wonder if we will at some point have a need for a higher level of continuity than can be offered by piece-wise inspection of the ramp entries, which would mean this approach of chaining these "private" nodes together might need to evolve in the future.

This is looking good to me too - I'm looking forward to a MaterialX world where we have more fully featured ramps :)
I do have one meta-observation/thought. Aside from debating the number of inputs, I think the node definition interface for ND_ramp looks pretty good and I don't think is controversial at all. (though I do think there may also be a place for a sister ramp node in the future that just takes in a varying array - as we discussed in the TSC meeting). But, ND_ramp_gradient seems very specific to the implementation of ND_ramp, and I wonder if it is really useful as a standard library node itself? It's almost as if we need a way to tag node definitions as "private" - or perhaps there is some other mechanism (node graphs?) that could enshrine the logic internal to the implementation of the node.
Also, as raised in the TSC I wonder if we will at some point have a need for a higher level of continuity than can be offered by piece-wise inspection of the ramp entries, which would mean this approach of chaining these "private" nodes together might need to evolve in the future.

Thanks for the feedback! In the original version of the ramp I wrote up, I didn't have the gradient node. When we decided to expand the number of control points this seemed way easier by introducing this node. I 100% love your idea of being able to somehow have private nodes that can be used in circumstances like this. Perhaps in editors for example this way they could be easily flagged as nodes you shouldn't be able to create. I think the only reason someone would want access to this node would be if they wanted to implement their own ramp that supported a different number of control points.

Yeah I totally see the benefit of having the node available to allow for varying the number of inputs really easily, or potentially allowing others to create their own ramp implementations with different number of inputs.
I'm not sure the "private" node idea implementation can just be left to the UI though, otherwise we run the risk of different DCCs respecting it or not.  I do wonder if its currently possible (or we could make changes to support this) that we could just define a nodegraph in stdlib_ng.mtlx for the ramp gradient, and then just use instances of this nodegraph in the implementation of the ramp node.  This would let us leverage the re-use, without committing the project to maintain the ramp gradient node as part of the public specification.
We should also remember to not left perfect be the enemy of good here!  Making a better ramp is going really benefit the whole MaterialX community

Your instancing nodegraph idea is interesting, just don't know if this is on anyone's radar. I just want to avoid having 30 copies of the nodegraph in the definition (thus the nodedef for now).

This is coming along nicely @feldstj, I love the new interpolation!
Here are a few thoughts:
I think just informing in the docs that ramp_gradient is a building-block for ramp, and can probably be hidden from average artist workflows, is plenty?
Also this node only has a color4 signature, but most of the color nodes tend to default to color3? Should color3 be available, along with float?
This node might also have some expectations documented around the UI. Because different application often handle ramp controls and node networks differently, it might be good to have some hints for the implementors. For example, whether or not all of these inputs must to be shown if they aren't in use by the ramp widget? Or whether or not intervals must be evenly spaced?
I know it may seem obvious, and maybe the "up to" mention in the docs is fine, but I think it'd be good to remove as much ambiguity as possible.
Especially because transporting materials in .mtlx or .usd between DCCs is/will be increasingly common, so having enough signatures and/or conventions/expectations to edit in various places seems important. For example, should a DCC reading a .mtlx/.usd network always interpret ND_ramp-->ND_convert_vector4_float as a float ramp, even if a user creates those nodes manually?
Definitely agree with @ld-kerley that we don't want perfect the enemy of good enough, but when we created a ramp node similar to this, the UI/interchange aspects were what tripped everyone up.

Thanks for the feedback @crydalch, appreciate it!
Adding docs to the ramp_gradient node that suggest that it is simply a building block for the ramp node certainly makes sense and seems easy enough to do. Did you want this in the MaterialX document doc string, or just in the MaterialX specification documentation?
Your point about supporting color3 and float versions of the ramp node makes sense as well. My only concern is that it may take some time to implement as creating the initial version did take some time and unfortunately this isn't the only project I'm working on at the moment.
With respect to the ramp docs that try to clarify what ramp implementors need to do, I'm mixed on this topic. On the one hand, I think it's true that ports like num_intervals don't need to be exposed to users of a ramp they are more needed internally to the ramp implementor. That said, I'm worried that most people care about ramp docs for usage information, not so much about implementation information. To me this feels like it should live elsewhere although I'm not quite sure where.

Adding docs to the ramp_gradient node that suggest that it is simply a building block for the ramp node certainly makes sense and seems easy enough to do. Did you want this in the MaterialX document doc string, or just in the MaterialX specification documentation?

Yeah I think the spec doc makes sense? Might be a good point to ask @jstone-lucasfilm about this sort of detail. Should it go in the spec, or just as a note in the node's documentation?

Your point about supporting color3 and float versions of the ramp node makes sense as well. My only concern is that it may take some time to implement as creating the initial version did take some time and unfortunately this isn't the only project I'm working on at the moment.

I think this should be easy/straightforward conceptually: node signatures can actually use an existing signature in their nodegraph. I remember hand-editing to do this, but that might have been before the MtlX Node Editor was working? This approach means the color3/float nodegraphs can actually use the color4 signature, and convert/extract as expected; so that should simplify the implementation of other signatures? I have been trying to find an example, I'm positive I've made one but I haven't found it so far :(

I really like this feature proposal, @feldstj, and my one thought is that we may want to limit this initial implementation to 10 control points, keeping the door open to an array-based node with an unlimited number of control points in the future.
That would then give artists the best of both worlds, where the fixed graph logic of this node handles the vast majority of cases, and a future array-based node handles less common cases requiring scores or hundreds of control points.
What are your thoughts?

Hi @jstone-lucasfilm, I'm torn on this. On the one hand, 10 control points is simpler and a nice round number. On the other hand Nikola had me increase the original 10 count to 30 because he thought it was important to provide a larger number of control points to users. Yes, the array-based approach will provide a larger number of control points, but the problem is still that those control points can't be connected to such as with textures as is possible with this version. I'm certainly open to it, but we should make sure we're ok with the consequences.

These are good points, @feldstj, though I'll offer two counterpoints that may lend support to the idea of starting with 10 control points:

Starting with 10 control points in an initial release of ramp, it's always possible to increase the number in future releases, providing full backwards compatibility for older documents.  The reverse approach would not be possible if we commit to 30 control points in the initial release.
Artists can use your ramp_gradient helper node to construct custom ramp nodes supporting any number of control points that they wish, and they can bundle these with their assets to make the functionality universally accessible.

I'm also with @jstone-lucasfilm with starting out at 10 for all his great reasons, but also because I'm a little concerned about the generated code size as well.  The 30 input version is likely to be one of the larger nodegraph based nodes we have in MaterialX, and in my previous production experience, once an artist is using a ramp for something procedural, its not uncommon to find them using many ramps combined together in "interesting" ways, and so that large network size could potentially multiply rapidly in heavy production cases.
Also thinking forward to a future world where we might have ramp nodes that have fixed inputs that allow connections like these, but also varying input ramps (potentially not allowing input connections), I wonder a little about node definition naming.  Maybe we want to call this something like ND_ramp_fixed, or perhaps even ND_ramp_fixed_10 allowing for performance critical applications to opt between a 10 knot ramp and a potential 30 knot ramp in the future too.  That would allow for the varying ramp to be named something like ND_ramp_varying in the future.

@ld-kerley I'm very open to recommendations for the name of this new node, and we might look to existing implementations of this concept in industry tools for guidance.
For the differentiation between fixed and varying versions of this node, we may not need to include fixed and varying in the node name itself, as long as the interfaces of the different versions are clear and distinct.
For example, this version of ramp takes a series of colorN and intervalN inputs, while a varying ramp would likely have single colorarray and intervalarray inputs, in which case the inputs to a ramp node would be sufficient to determine which variant was intended (excepting the trivial case of a ramp node with no inputs, in which case our precedence rules would take over).

Hey All,
I'm going to prefix this with the fact that I'm relatively new to MaterialX, and be making some assumptions (which I'll state).
My first assumption: There is a current limitation (and one I can see as being "impossible" due to the differing supported shading languages) that we can't have a node graph that can have an array of connections, with an array of indices indicating which knot they relate to?
A question with the current proposal, is what would happen if I had the values (red, green) connected to the first(color1) and last input(color30), with intervals of 0 and 0.5, would the 2nd input be ignored because the 2nd input has an interval of 1.0 (therefore greater than color30). Such that I'd end up with a ramp from red -> white from 0 -> 1.0 intervals?  This could make UI tricky to understand is my current thoughts, although I'm happy to have intrepreted the graph incorrectly - again not the best at reading MaterialX yet!
The other note, is that typically there's two ramps I've seen, float and Color, again multiple Color (4 or 3 values, float or double), I'd hope that we'd have differently named MatX nodegraphs for each.  I like the above idea of naming them along the lines of NMD_ramp_color3f_fixed30, NMD_ramp_color3d_fixed30 ,NMD_ramp_color4f_fixed10, etc...  This way we can easily add change how ramps are handled in a non-destructive manner.
To relate this back to something I have planned for a USD proposal around Ramps in shaders was to describe them as such:
float[] inputs:<someValue>:knot:position (or interval if you prefer)
token[] inputs:<someValue>:knot:interpolations (per knot interpolation values) - could also be indexed? linear if not specified or clamp settings across future knots, unsure.
rel[] <someValue>:knot:connections // a sparse connection list of connected attributes
int[] <someValue>:knot:connections:indices // indicies to match the connections to a particular index in the positions/intervals
color3f[] (or other array type) inputs:<someValue>:knot:values // default values for the connected values would be stored here as well, such that if the relationship fails, we have a value to use.

A bit of a brain dump, but trying to make sure we can also align as to how this information will be stored in the future to avoid having to revisit this again.
Cheers,
James

@JamesPedFoundry For the array-based version of the ramp node, this should eventually be possible in MaterialX using inputs of type color3array and floatarray, though we'll need a bit more work in our shader generation system to support this across languages.  We're thinking of this as a future, complementary partner to the fixed-count ramp node that @feldstj has proposed for the current version of MaterialX.

@jstone-lucasfilm thanks for confirmation around future array inputs, makes a lot of sense to me.
I guess then my only change request would be the name of this node, making it specifically named, such that other ramp types are available, and it is clear which should be used when, and we don't limit ourselves for the future case where the array based nodes might exist.

@feldstj @ld-kerley
Now that MaterialX 1.39.1 is released, I wanted to follow up on this proposal, to see what additional steps we will need before this can be merged.
If it meets the approval of our Autodesk colleagues, I'd recommend limiting this node to 10 control points, motivated by the discussions above.
Additionally, a few folks have suggested changing the name of this node, though I'll node that ramp could have both graph-based and array-based versions, as long as the inputs to these two nodes have distinct and clear names.  For example, the graph-based version could used inputs named color1, interval1, color2, interval2, and so forth, while the array-based version could use inputs named colorarray and intervalarray.
Because these interfaces have no overlap, it would always remain clear which version of ramp is intended by an artist, so we would not strictly need two different node names.

@jstone-lucasfilm
I'll try to follow up with some Autodesk colleagues about the 10 control points. I think the other thing that was requested was to have float and color3 versions of the ramp. So something like ND_ramp_float, ND_ramp_color3 and ND_ramp_color4.

@jstone-lucasfilm
Spoke with Nikola and Jerry. Nikola still seems to really want 30 control points. Jerry mentioned you may have discussed this at the ASWF meeting this week. Any conclusions?

I still think that we might be happy in the long run If we were to name the fixed size ramp and the variable ramp case differently.  We do still have some outstanding issues regarding ambiguous node resolution, if the inputs are under specified, which does manifest as concrete difficulties if the MaterialX document is converted to OpenUSD.
I'm not sure that there is a strong disadvantage to having separate node categories for the different ramp types.  One other consideration might be that the different ramp types may possibly end up providing different implementation behavior?
Finally, if this all turns out to be a mistake, it would be a lot easier to write an upgrade that renamed a number of different ramp node names, to a single name, than to reverse  this decision in the opposite direction.

To confirm, my concerns on the naming of just a "ramp" is less for between array or interval based ramps, but more the difference between a color3, color4 and float types of ramp shader.
Also how this would then fit into something like USD where it wouldn't necessarily support arbitrarily having its inputs be either a color3, color4 or float, it accepts one or the other in the schema definition.

@JamesPedFoundry Just as a point of comparison, I'll note that MaterialX traditionally uses a single name for all type variations of a single node category, including ramp-related nodes such as remap, range, and smoothstep:
https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/documents/Specification/MaterialX.Specification.md#adjustment-nodes
@ld-kerley If we were to propose a naming distinction between fixed-length and variable-length ramp nodes, what naming convention would you suggest?

I don't have super strong opinions on what the names should be, but perhaps...

<ramp_fixed>
<ramp_varying> or <ramp_array>

I also wonder if for the fixed version we end up having multiple nodes with differing number of inputs if we might want

<ramp_fixed_10>
<ramp_fixed_30>
etc....

This leaves the future door open for a clean move to a unified <ramp> in the future, without favoring either flavor.

@feldstj I'd love to include this initial feature in MaterialX 1.39.2 if that's possible, and I'm currently thinking that we might start with a node named ramp that supports 10 control points.  This would give us the flexibility to refine names and increase the number of control points in the future, but would not initially commit us to these choices.
What are your thoughts?

Hi @jstone-lucasfilm - That's exciting! I'm totally fine with that and so is Orn. We can always increase the number of points later. That said, I'm still waiting to hear back from Nikola about this.

Hey @jstone-lucasfilm, All
We are super exciting regrading Ramp implementation in Standard Library.
Customers are craving for standardized implementation, and they’ve already been enjoying Ramp in LookdevX for a while. The feedback so far is very positive, our initial 30-point integration seems to cover intermediate complexity well, addressing many of the needs commonly seen in production.
To deliver the most value early on, it makes sense to focus on a highly useful Ramp node with strong longevity. Maybe is good to mention that our implementation already allows input nodes to connect and be driven by textures, which is a big win for flexibility.
While a dynamic Ramp node with unlimited points would be a fantastic enhancement, enabling connections to its inputs will be proven as more complex challenge, which will extend significantly lifespan of the proposed 30-point Ramp.
...thoughts?
Thanks
Nikola

Hey @jstone-lucasfilm
Thank you again for your thoughts. I’m aligned with you on advancing the Ramp node. @feldstj will prepare the node with 10 control points for the next steps. As always, please keep us updated on the best time to reevaluate the number of embedded points.
Thanks
Nikola


Scriptable tick fonts

Closes https://github.com/chartjs/Chart.js/issues/2442

It would be more efficient to use one callback for font. But thats a bit bigger change.
Something to consider before alpha through.

There's no perf hit because there are so few labels - this is run only after autoSkip. I agree resolving each option individually seems slightly wasteful, but I was trying to follow the pattern from _resolveDataElementOptions and it seems like that's what's done there


Ssc terasawa

I have implemented a function Tk3D_SSC_Terasawa22 to compute super-sample covariance into the tk3d.py.

I am going to add a unit test and a benchmark test later.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

4 of 62 (6.45%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-1.1%) to 96.379%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pyccl/tk3d.py
4
62
6.45%

Totals

Change from base Build<PHONE_NUMBER>:
-1.1%

Covered Lines:
4844

Relevant Lines:
5026

💛  - Coveralls

The changes in this PR is now included in
https://github.com/LSSTDESC/CCL/pull/1134


Add support for two factor authentication

Some github users have 2FA. Github3.py supports this, see here for an example:
http://github3.readthedocs.io/en/develop/examples/two_factor_auth.html

I think this is done


[Bug]: The link to google colab from the tutorials on docs.pybamm.org does not work.

PyBaMM Version
v24.1
Python Version
NA
Describe the bug
The link to google colab from the tutorials on docs.pybamm.org does not work.
It gives the error:
malformed GitHub path: missing 'blob' before branch name: pybamm-team/PyBaMM/tree/v24.1/docs/source/examples/notebooks/getting_started/tutorial-1-how-to-run-a-model.ipynb
Steps to Reproduce
From
https://docs.pybamm.org/en/stable/source/examples/notebooks/getting_started/tutorial-1-how-to-run-a-model.html#
clink on
Open this notebook in Google Colab
Relevant log output
No response

Thanks for opening this @jonchapman1, all the Colab badges are currently directing to bad URLs, missing blob before branch name in v24.1, looking into the fix


Sunspot.commit on demand

I found that when writing tests, I would often need to do
Sunspot.commit

After any operation that would require solr indexing. In production I don't need to do that since I let sunspot decide when to do the commit. So I wrote something like this
if Reservation.searchable?
  class Reservation < ActiveRecord::Base

    class << self
      #prior to doing any operations in solr we want to make sure
      #that the data is persisted

      alias_method :orig_search, :search

      def search(options = {}, &block)
        Sunspot.commit
        orig_search options, &block
      end

    end

  end
end

This helped to speed up my tests significantly. Woud this be a good candidate for this gem? There is probably a better way to make it more generic but I'm not sure how

Making Sunspot.commit would not be a good idea because I think the dev needs to be in control of when that happens. For example, I've used helper methods like these:
def commit_and_assert_solr_index_count(count)
  Sunspot.commit; assert_solr_index_count(count)
end

def assert_solr_index_count(count)
  SearchableProfile.solr_search {}.total.should eq(count)
end

Maybe these can help your pain:
def with_commit
  yield
  Sunspot.commit
end

with_commit { Model.create }

########################

module SunspotHelpers
  def with_index(&block)
    block.instance_eval { Sunspot.commit }
    before(&block)
  end
end

RSpec.configuration.extend SunspotHelpers

describe 'foo' do
  with_index do
    Model.create
  end
end

########################

specify 'the :solr key will create a before block that commits', :solr do
end

RSpec.configuration.treat_symbols_as_metadata_keys_with_true_values = true
RSpec.configuration.before(:each, :solr) { Sunspot.commit }