[fpga] connect external clk to pin

Related to #17491.
Background
For manufacturing tests, we need to be able to drive LC transitions via the LC JTAG TAP, which requires driving the external clk.
Problem
On CW310 FPGA, the external clk is tied off to 0, so we currently cannot develop said manufacturing tests on the FPGA.
Tasks

[ ] Investigate what frequency to drive an "external clk" FPGA pin such that another PLL can drive another 48 or 96 MHz clk, as this is what is required?
[x] Investigate what pin the external clk can be routed too such that it can reach the HyperDebug PWM pin through the swizzle board (if #17491 is feasible); (ideally IOC6 as this is the [pin used for the external clk on the ASIC]) - IOC6 should work
[ ] Modify FPGA RTL / XDC file to drive external clock pin with another PLL.

CC: @a-will

From current discussions, we're looking to model an observable change from an "internal" oscillator to external clocks. For this, we would look to connect the raw output from the IOC6 pad to a PLL in the FPGA, then either (1) choose an output that matches the main PLL's CW310-provided input and mux them or (2) choose outputs that match the main PLL's outputs and mux those.
(1) may only allow for emulating the all_clocks_byp_req, whereas (2) could handle doing the feature where it is just the I/O clocks. (2) may use BUFGMUX instances we don't have, though.
We would need to remap the IOC6 function at USERIOA-18 to a clock-capable pin and respin the hyperdebug's swizzle board. Candidates:

USERIOA-32 (currently unmapped)
USERIOA-11 (currently IOB5 - UART)
USERIOA-33 (currently IOR9 - FLASH WP)
USERIOB-14 (currently IOC0 - SW STRAP0)
USERIOB-26 (currently unmapped)
USERIOB-30 (currently unmapped)
USERIOB-32 (currently unmapped)

At the most basic level, we really just want the FPGA OT instance not to freeze up when we select ext_clk, since right now its just tied off to 0.
This will allow us to develop manufacturing tests on the FPGA.
I could see a potential case where we want to use hyp + ASIC together and drive ext_clk, but in that case, might be better to have ext_clk driven by something other than hyp, e.g. something with tigher tolerances like a function generator?
So maybe in the interim it might be better to just tie the ext_clk in the CW310 top-level to the same (or a similar) PLL that drives the main clk? This would then render #17491 un-needed.
Thoughts?
CC: @msfschaffner

At the most basic level, we really just want the FPGA OT instance not to freeze up when we select ext_clk, since right now its just tied off to 0.
This will allow us to develop manufacturing tests on the FPGA.
I could see a potential case where we want to use hyp + ASIC together and drive ext_clk, but in that case, might be better to have ext_clk driven by something other than hyp, e.g. something with tigher tolerances like a function generator?
So maybe in the interim it might be better to just tie the ext_clk in the CW310 top-level to the same (or a similar) PLL that drives the main clk? This would then render #17491 un-needed.
Thoughts? CC: @msfschaffner

It turns out I goofed and got the hierarchy wrong, but yeah, from a high level, that makes sense. It actually would be something like a modification in ast_clks_byp.sv to funnel the clk_osc group through if AST_BYPASS_CLK is defined.

The AST files claim to be auto-generated, though. Is there a proper way to go about editing them?

@a-will has found a working solution in #17553, but since the AST RTL model in the open source is autogenerated, we have contacted @Jacob-Levy about integrating these changes into the AST RTL model template and regenerating the RTL.

We have a way to control ast clocks from external source, did you try to use it instead?
Drive the following 3 ast ports:
clk_ast_ext_i
ext_freq_is_96m_i
all_clk_byp_req_i

If you read the issue, the problem is that clk_ast_ext_i is currently tied off to zero on the FPGA toplevel. We don't want to tie this to an actual external FPGA pin and drive it externally say with a function generator because that would entail more infrastructure to add to CI. @a-will was trying to tie the external clk to the same clocks generated within the FPGA in #17553, so that when our tests switch to the ext clock the FPGA doesn't die. Does that make sense?

@timothytrippel
I understand why using a real ext clock would impose difficulties on CI.
Can you explain why @a-will solution is easier than connecting the internal FPGA PLL output to clk_ast_ext_i and use the existing clock bypass dividers ?
thanks

@arnonsha
Those dividers would yield clocks that are too fast for the FPGA, and it is simply more complex for timing analysis to have a separate input path. The dividers are also a very poor implementation on the FPGA, adding yet more LUT logic on the clock paths.
On top of that, the ratios between clocks must be different vs. the ASIC. USB must continue to be fed 48 MHz and the AON clock must still be around 200-250 kHz, but the main and I/O clocks must be significantly less than 96 MHz / 24 MHz to meet timing.
For these reasons, it is sensible to bypass the clock generation from clk_ast_ext_i.

Thanks @a-will . That makes sense.
We will fix in the AST template accordingly.

thanks @arnonsha ! Please close this issue when it is complete.


Setup Faild

Microsoft PowerToys version
0.61.1
Running as admin

[X] Yes

Area(s) with issue?
Installer
Steps to reproduce
Hi team,
I wanted to install PowerToys on the company laptop, so I suspect the error.
But since I have admin rights, I thought that this shouldn't be a problem.
Unfortunately, I still got an error
0x800421f6 - Unknown error
Maybe you can still help me. Sorry guys I'm trying to figure out how to properly post the log here, can anyone help me? In the editor there is only add code but it can look like it does now and it doesn't look good.
[49C4:45C8][2022-08-05T07:16:32]i001: Burn v<IP_ADDRESS>26, Windows v10.0 (Build 19042: Service Pack 0), path: C:\Users\tgerman\AppData\Local\Temp\{182C1253-2270-4066-BDB0-20DE26A50DAA}\.cr\PowerToysSetup-0.61.1-x64.exe [49C4:45C8][2022-08-05T07:16:32]i000: Initializing string variable 'InstallFolder' to value '[ProgramFiles64Folder]PowerToys' [49C4:45C8][2022-08-05T07:16:32]i000: Initializing string variable 'MsiLogFolder' to value '[LocalAppDataFolder]\Microsoft\PowerToys\' [49C4:45C8][2022-08-05T07:16:32]i000: Initializing version variable 'DetectedPowerToysVersion' to value '<IP_ADDRESS>' [49C4:45C8][2022-08-05T07:16:32]i000: Initializing version variable 'TargetPowerToysVersion' to value '0.61.1' [49C4:45C8][2022-08-05T07:16:32]i000: Initializing version variable 'DetectedWindowsBuildNumber' to value '0' [49C4:45C8][2022-08-05T07:16:32]i009: Command Line: '-burn.clean.room=C:\Users\tgerman\Downloads\PowerToysSetup-0.61.1-x64.exe -burn.filehandle.attached=536 -burn.filehandle.self=560' [49C4:45C8][2022-08-05T07:16:32]i000: Setting string variable 'WixBundleOriginalSource' to value 'C:\Users\tgerman\Downloads\PowerToysSetup-0.61.1-x64.exe' [49C4:45C8][2022-08-05T07:16:32]i000: Setting string variable 'WixBundleOriginalSourceFolder' to value 'C:\Users\tgerman\Downloads\' [49C4:45C8][2022-08-05T07:16:32]i000: Setting string variable 'WixBundleLog' to value 'C:\Users\tgerman\AppData\Local\Temp\powertoys-bootstrapper-msi-0.61.1_20220805071632.log' [49C4:45C8][2022-08-05T07:16:32]i000: Setting string variable 'WixBundleName' to value 'PowerToys (Preview) x64' [49C4:45C8][2022-08-05T07:16:32]i000: Setting string variable 'WixBundleManufacturer' to value 'Microsoft Corporation' [49C4:3D30][2022-08-05T07:16:32]i000: Setting numeric variable 'WixStdBALanguageId' to value 1033 [49C4:3D30][2022-08-05T07:16:32]i000: Setting version variable 'WixBundleFileVersion' to value '<IP_ADDRESS>' [49C4:45C8][2022-08-05T07:16:32]i100: Detect begin, 4 packages [49C4:45C8][2022-08-05T07:16:32]i000: Product or related product not found: {42B84BF7-5FBF-473B-9C8B-049DC16F7708} [49C4:45C8][2022-08-05T07:16:32]i000: Setting version variable 'DetectedPowerToysVersion' to value '<IP_ADDRESS>' [49C4:45C8][2022-08-05T07:16:32]i000: Setting string variable 'DetectedWindowsBuildNumber' to value '19042' [49C4:45C8][2022-08-05T07:16:32]i000: Setting numeric variable 'HasDotnet607' to value 1 [49C4:45C8][2022-08-05T07:16:32]i000: Registry key not found. Key = 'Software\Microsoft\EdgeUpdate\Clients\{F3017226-FE2A-4295-8BDF-00C3A9A7E4C5}' [49C4:45C8][2022-08-05T07:16:32]i000: Setting numeric variable 'HasWebView2PerUser' to value 0 [49C4:45C8][2022-08-05T07:16:32]i000: Registry key not found. Key = 'SOFTWARE\WOW6432Node\Microsoft\EdgeUpdate\Clients\{F3017226-FE2A-4295-8BDF-00C3A9A7E4C5}' [49C4:45C8][2022-08-05T07:16:32]i000: Setting numeric variable 'HasWebView2PerMachine' to value 0 [49C4:45C8][2022-08-05T07:16:32]i000: Setting string variable 'DetectedVCRedistVersion' to value '28' [49C4:45C8][2022-08-05T07:16:33]i052: Condition 'HasDotnet607' evaluates to true. [49C4:45C8][2022-08-05T07:16:33]i052: Condition 'HasWebView2PerMachine OR HasWebView2PerUser' evaluates to false. [49C4:45C8][2022-08-05T07:16:33]i052: Condition 'DetectedVCRedistVersion >= 32' evaluates to false. [49C4:45C8][2022-08-05T07:16:33]i101: Detected package: DotnetRuntime6, state: Present, cached: None [49C4:45C8][2022-08-05T07:16:33]i101: Detected package: WebView2, state: Absent, cached: None [49C4:45C8][2022-08-05T07:16:33]i101: Detected package: VCRedist, state: Absent, cached: None [49C4:45C8][2022-08-05T07:16:33]i101: Detected package: PowerToysSetup_0.61.1_x64.msi, state: Absent, cached: None [49C4:45C8][2022-08-05T07:16:33]i052: Condition 'TargetPowerToysVersion >= DetectedPowerToysVersion OR WixBundleInstalled' evaluates to true. [49C4:45C8][2022-08-05T07:16:33]i052: Condition 'DetectedWindowsBuildNumber >= 19041 OR WixBundleInstalled' evaluates to true. [49C4:45C8][2022-08-05T07:16:33]i199: Detect complete, result: 0x0 [49C4:3D30][2022-08-05T07:16:39]i000: Setting numeric variable 'EulaAcceptCheckbox' to value 1 [49C4:45C8][2022-08-05T07:16:39]i200: Plan begin, 4 packages, action: Install [49C4:45C8][2022-08-05T07:16:39]w321: Skipping dependency registration on package with no dependency providers: DotnetRuntime6 [49C4:45C8][2022-08-05T07:16:39]w321: Skipping dependency registration on package with no dependency providers: WebView2 [49C4:45C8][2022-08-05T07:16:39]i000: Setting string variable 'WixBundleLog_WebView2' to value 'C:\Users\tgerman\AppData\Local\Temp\powertoys-bootstrapper-msi-0.61.1_20220805071632_000_WebView2.log' [49C4:45C8][2022-08-05T07:16:39]w321: Skipping dependency registration on package with no dependency providers: VCRedist [49C4:45C8][2022-08-05T07:16:39]i000: Setting string variable 'WixBundleLog_VCRedist' to value 'C:\Users\tgerman\AppData\Local\Temp\powertoys-bootstrapper-msi-0.61.1_20220805071632_001_VCRedist.log' [49C4:45C8][2022-08-05T07:16:39]i000: Setting string variable 'WixBundleRollbackLog_PowerToysSetup_0.61.1_x64.msi' to value 'C:\Users\tgerman\AppData\Local\Temp\powertoys-bootstrapper-msi-0.61.1_20220805071632_002_PowerToysSetup_0.61.1_x64.msi_rollback.log' [49C4:45C8][2022-08-05T07:16:39]i000: Setting string variable 'WixBundleLog_PowerToysSetup_0.61.1_x64.msi' to value 'C:\Users\tgerman\AppData\Local\Temp\powertoys-bootstrapper-msi-0.61.1_20220805071632_002_PowerToysSetup_0.61.1_x64.msi.log' [49C4:45C8][2022-08-05T07:16:39]i201: Planned package: DotnetRuntime6, state: Present, default requested: Present, ba requested: Present, execute: None, rollback: None, cache: No, uncache: No, dependency: None [49C4:45C8][2022-08-05T07:16:39]i201: Planned package: WebView2, state: Absent, default requested: Present, ba requested: Present, execute: Install, rollback: None, cache: Yes, uncache: No, dependency: None [49C4:45C8][2022-08-05T07:16:39]i201: Planned package: VCRedist, state: Absent, default requested: Present, ba requested: Present, execute: Install, rollback: None, cache: Yes, uncache: No, dependency: None [49C4:45C8][2022-08-05T07:16:39]i201: Planned package: PowerToysSetup_0.61.1_x64.msi, state: Absent, default requested: Present, ba requested: Present, execute: Install, rollback: Uninstall, cache: Yes, uncache: No, dependency: Register [49C4:45C8][2022-08-05T07:16:39]i299: Plan complete, result: 0x0 [49C4:45C8][2022-08-05T07:16:39]i300: Apply begin [49C4:45C8][2022-08-05T07:16:39]i010: Launching elevated engine process. [49C4:45C8][2022-08-05T07:16:46]i011: Launched elevated engine process. [49C4:45C8][2022-08-05T07:16:46]i012: Connected to elevated engine. [1C58:07AC][2022-08-05T07:16:47]i358: Pausing automatic updates. [1C58:07AC][2022-08-05T07:16:47]i359: Paused automatic updates. [1C58:07AC][2022-08-05T07:16:47]i360: Creating a system restore point. [1C58:07AC][2022-08-05T07:16:47]w363: Could not create system restore point, error: 0x80070005. Continuing... [1C58:07AC][2022-08-05T07:16:47]i370: Session begin, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{65d1bc7a-8b73-4352-b61e-5c72c806594e}, options: 0x7, disable resume: No [1C58:07AC][2022-08-05T07:16:47]i000: Caching bundle from: 'C:\Users\tgerman\AppData\Local\Temp\{52EA0206-7641-42E1-8BD1-FC8D3FCF6223}\.be\PowerToysSetup-0.61.1-x64.exe' to: 'C:\ProgramData\Package Cache\{65d1bc7a-8b73-4352-b61e-5c72c806594e}\PowerToysSetup-0.61.1-x64.exe' [1C58:07AC][2022-08-05T07:16:47]i320: Registering bundle dependency provider: {65d1bc7a-8b73-4352-b61e-5c72c806594e}, version: <IP_ADDRESS> [1C58:07AC][2022-08-05T07:16:47]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{65d1bc7a-8b73-4352-b61e-5c72c806594e}, resume: Active, restart initiated: No, disable resume: No [1C58:1C54][2022-08-05T07:16:47]i305: Verified acquired payload: WebView2 at path: C:\ProgramData\Package Cache\.unverified\WebView2, moving to: C:\ProgramData\Package Cache\F7119E3AB3D5CB3ADCD9A57F0EC227F783695819\MicrosoftEdgeWebview2Setup.exe. [49C4:0C10][2022-08-05T07:16:47]w343: Prompt for source of package: VCRedist, payload: VCRedist, path: C:\Users\tgerman\Downloads\VCRedist-14.32.31332.exe [49C4:0C10][2022-08-05T07:16:47]i338: Acquiring package: VCRedist, payload: VCRedist, download from: https://download.visualstudio.microsoft.com/download/pr/ed95ef9e-da02-4735-9064-bd1f7f69b6ed/CE6593A1520591E7DEA2B93FD03116E3FC3B3821A0525322B0A430FAA6B3C0B4/VC_redist.x64.exe [1C58:1C54][2022-08-05T07:16:51]i305: Verified acquired payload: VCRedist at path: C:\ProgramData\Package Cache\.unverified\VCRedist, moving to: C:\ProgramData\Package Cache\D4F9181E70E3F1AA6C8EDFFCC15B3C3D4BABE36B\VCRedist-14.32.31332.exe. [1C58:1C54][2022-08-05T07:16:51]i305: Verified acquired payload: PowerToysSetup_0.61.1_x64.msi at path: C:\ProgramData\Package Cache\.unverified\PowerToysSetup_0.61.1_x64.msi, moving to: C:\ProgramData\Package Cache\{9110B562-D4E7-45E7-BA08-9D9B6746BED0}v0.61.1\PowerToysSetup-0.61.1-x64.msi. [1C58:07AC][2022-08-05T07:16:51]i301: Applying execute package: WebView2, action: Install, path: C:\ProgramData\Package Cache\F7119E3AB3D5CB3ADCD9A57F0EC227F783695819\MicrosoftEdgeWebview2Setup.exe, arguments: '"C:\ProgramData\Package Cache\F7119E3AB3D5CB3ADCD9A57F0EC227F783695819\MicrosoftEdgeWebview2Setup.exe" /silent /install' [1C58:07AC][2022-08-05T07:17:36]e000: Error 0x800421f6: Process returned error: 0x800421f6 [1C58:07AC][2022-08-05T07:17:36]e000: Error 0x800421f6: Failed to execute EXE package. [49C4:45C8][2022-08-05T07:17:36]e000: Error 0x800421f6: Failed to configure per-machine EXE package. [49C4:45C8][2022-08-05T07:17:36]i319: Applied execute package: WebView2, result: 0x800421f6, restart: None [49C4:45C8][2022-08-05T07:17:36]e000: Error 0x800421f6: Failed to execute EXE package. [1C58:07AC][2022-08-05T07:17:36]i351: Removing cached package: WebView2, from path: C:\ProgramData\Package Cache\F7119E3AB3D5CB3ADCD9A57F0EC227F783695819\ [1C58:07AC][2022-08-05T07:17:36]i372: Session end, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{65d1bc7a-8b73-4352-b61e-5c72c806594e}, resume: None, restart: None, disable resume: No [1C58:07AC][2022-08-05T07:17:36]i330: Removed bundle dependency provider: {65d1bc7a-8b73-4352-b61e-5c72c806594e} [1C58:07AC][2022-08-05T07:17:36]i352: Removing cached bundle: {65d1bc7a-8b73-4352-b61e-5c72c806594e}, from path: C:\ProgramData\Package Cache\{65d1bc7a-8b73-4352-b61e-5c72c806594e}\ [1C58:07AC][2022-08-05T07:17:36]i371: Updating session, registration key: SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{65d1bc7a-8b73-4352-b61e-5c72c806594e}, resume: None, restart initiated: No, disable resume: No [49C4:45C8][2022-08-05T07:17:36]i399: Apply complete, result: 0x800421f6, restart: None, ba requested restart:  No
✔ Expected Behavior
to install it
❌ Actual Behavior
It was not installed
Other Software
No response

Same error code as #18736

Thanks thas works!

Thanks thas works!

can this issue be closed?

Sorry yes you can close it
yashpalgoyal1304 @.***> schrieb am Sa., 17. Sept. 2022,
16:14:

Thanks thas works!
can this issue be closed?
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/PowerToys/issues/19780#issuecomment-1250079164,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AH6GEM6VFHY6BGDXB2T5J73V6XG43ANCNFSM55U2VPSQ
.
You are receiving this because you authored the thread.Message ID:
@.***>


Failed step - Publish to GitHub Pages

When publishing js-ipfs-api today:
 ❯ Publish documentation
   ✔ Clean ./docs
   ✔ Generating documentation
   ✖ Publish to GitHub Pages
     → Failed to get remote.origin.url (task must either be run in a…
   Publish to npm/FormidableLabs/radium/tree/master/docs/api#radium fFailed to get remote.origin.url (task must either be run in a git repository with a configured origin remote or must be configured with the "repo" option).
Error: Failed to get remote.origin.url (task must either be run in a git repository with a configured origin remote or must be configured with the "repo" option).
    at /Users/alan/Code/protocol-labs/js-ipfs-api/node_modules/gh-pages/lib/git.js:215:13
    at process._tickCallback (internal/process/next_tick.js:68:7)

@ipfs/dx can you take a look?

This has worked successfully since. I'll open a new issue if I see it again.


Windows support

Relates to #63

@anidel thanks for submitting the PR :) Could you rebase on master and resolve the current conflicts before we merge it in? Would also still need to test it properly.


Bump Dependencies #9610911852

Bumps dependencies for product-is. Link : https://github.com/wso2/product-is/actions/runs/9610911852

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Dotenv for docker compose

Motivation and context

related issues
#7311

When I use CVAT with Docker Compose, I find it necessary to remember which version of CVAT I am using. However, I suggest maintaining CVAT versions and any customizations in a .env file, as it could reduce the number of properties that need to be remembered.
In the current commit, only CVAT VERSION and HOST are included, but I think it might be beneficial to gradually expand this functionality.
Please let me know your thoughts.
Thank you.
How has this been tested?

Checklist

[X] I submit my changes into the develop branch
[ ] I have created a changelog fragment
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[X] I have linked related issues (see GitHub docs)
#7311
[ ] I have increased versions of npm packages if it is necessary
(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)

License

[X] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

Hello,
Thank you for the contribution.
Unfortunately now we are not ready to accept the pull request.


Fix GTK for Windows links in README

Fixes issue #216

Thanks! Once we confirmed my note, I'll merge.

OK I updated the PR like we discussed, should be ready to merge.

Didn't you want to add a howto as well ?

Ah sorry, didn't see your comment on #218 until now. Yeah I con do it in this PR as well.

Then ping me when it's done please, github doesn't notify when a PR has been updated.

OK I added a first version of the HowTo. A few points to note:

Should we keep the old instructions or should I just remove them?
I only tested in a 32Bit Windows VM but wrote the HowTo for 64Bit since I guessed it's more common.
I also added a note about installing rust for the GNU ABI and not installing "Linker and plattform libraries".

Should we keep the old instructions or should I just remove them?

It makes sense to remove them but maybe later after we have some feedback about the msys2 way.

I only tested in a 32Bit Windows VM but wrote the HowTo for 64Bit since I guessed it's more common.

I can test this on 64-bit later. BTW you don't need a local VM, appveyor allows rdp access.

I can test this on 64-bit later. BTW you don't need a local VM, appveyor allows rdp access.

Ah that's awesome I didn't now that! :smile: This can safe me a lot of time while trying to get this to build with AppVeyor!

@gkoz I tried to address all your points. Tell me if I should squash the commits or if there are any other issues.

Please keep the first commit and squash other in the second one please.

@GuillaumeGomez Squashed it. Is it OK like that?

@rnestler: Perfect! But it's @gkoz who will merge when he thinks everything's ok.

Some grammar nits...

Some grammar nits...

Yeah sorry about that. I'm not a native english speaker and my grammar is poor :cry:

Looks good :+1:

Seems good to me too. Once you have fixed my last notes, we merge.

@GuillaumeGomez Done :)

Thanks a lot for your contribution @rnestler! I merge.

Happy to help! Thanks for your work on gtk-rs!


Installation Fails on Delphi 11 Professional

Using the Skia4Delphi Setup.exe installation method, the setup wizard halts with the following error...

As I have Delphi 11 Professional, my machine has no FMX Linux capabilities :-(  so it looks like the installation program assumes Delphi Exterprise / Architect Edition is installed.
The same error occurs on several Delphi 11 installs - in VMs and on a physical machine.

@Mr-KultiVator Thank you very much to report the problem!
I fixed the problem and updated Skia4Delphi_3.0.0_Setup.exe
Can you download and test again?

Now that's what I call a quick fix!
Everything working great in both FMX and VCL.
Amazing work - huge thanks for creating this impressive library and sharing this with the Delphi Community.


Uncaught app exception Traceback

2024-04-01 22:40:14 | INFO | "./app\services\video.py:179": generate_video - using font: C:/Users/hourp/Downloads/MoneyPrinterTurbo-V1/resource/fonts/STHeitiLight.ttc
2024-04-01 22:40:14.482 Uncaught app exception
Traceback (most recent call last):
File "C:\Users\hourp\anaconda3\envs\MoneyPrinterTurbo\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 542, in _run_script
exec(code, module.dict)
File "C:\Users\hourp\Downloads\MoneyPrinterTurbo-V1\webui\Main.py", line 376, in
tm.start(task_id=task_id, params=params)
File "C:\Users\hourp\Downloads\MoneyPrinterTurbo-V1\app\services\task.py", line 158, in start
video.generate_video(video_path=combined_video_path,
File "C:\Users\hourp\Downloads\MoneyPrinterTurbo-V1\app\services\video.py", line 214, in generate_video
sub = SubtitlesClip(subtitles=subtitle_path, make_textclip=generator, encoding='utf-8')
File "C:\Users\hourp\anaconda3\envs\MoneyPrinterTurbo\lib\site-packages\moviepy\video\tools\subtitles.py", line 69, in init
self.duration = max([tb for ((ta, tb), txt) in self.subtitles])
File "C:\Users\hourp\anaconda3\envs\MoneyPrinterTurbo\lib\site-packages\moviepy\video\tools\subtitles.py", line 69, in
self.duration = max([tb for ((ta, tb), txt) in self.subtitles])
TypeError: cannot unpack non-iterable NoneType object

Thank you for your feedback.
Could you please package and upload the subtitle.srt and audio.mp3 files from the task output directory here?
This will help me locate and resolve the issue.
Thank you.


Add arch flag to assembler to fix build on M1 mac.

Fixes https://github.com/godotengine/godot/pull/59167#issuecomment-1172047037

You could possibly add the same to the iOS build, as I ran into the same issue there while testing the PR.
I then excluded iOS from the list but it could likely be added back:
https://github.com/godotengine/godot/blob/110248cd57b5abb82c216d4a117fd7696a593931/core/SCsub#L132L135
That would only impact the simulator build though and maybe it's best if it's not used there, so we don't have unexpected differences between real hardware and simulator. (But fixing ASFLAGS would still be good IMO.)

Add the same for iOS, and enabled huf_decompress_amd64.S build for iOS.
Note for cherry-picking: iOS SDK versions are different in the 3.x branch (10.0 for both device and simulator).

Thanks!

Cherry-picked for 3.6.


Feature/optional flags

Adds option for disabling showing the flags based on what @mlopezFC commented on #4

... we could provide an option for disabling showing the flags. The component is still useful without flags.

Summary by CodeRabbit

New Features

Introduced flag display alongside locale names in the LocaleComboBox for enhanced user experience.
Added a demo component allowing users to toggle flag visibility in locale selections.
Updated routing and structure in demo components for better navigation and functionality.

Bug Fixes

Adjusted rendering logic to ensure correct flag display based on user settings.

rebased with main to resolve merge conflicts
fixed code formatting
fixed the Rendering Demo
improved BaseLocaleComboBoxDemo's code


SConscript(variant_dir) and VariantDir() inconsistant

This issue was originally created at: 2008-07-11 20:44:17.
This issue was reported by: gregnoel.
gregnoel said at 2008-07-11 20:44:17

The sequence
VariantDir('build', '.')
Sconscript(whatever...)
should be equivalent to
SConscript(whatever..., variant_dir='build', src_dir='.')
and also to
SConscript(whatever..., variant_dir='build')
It isn't.  The first case works fine.  The second and third cases fail with the
message "Source directory cannot be under variant directory."
The description of VariantDir() documents this restriction; the description of
SConstruct(variant_dir) is silent on the point.  (This same pattern occurs with
older versions of the man page where it's documented under BuildDir().)
I believe that all of these cases should work identically.  My preference is
that the situation be allowed.

gregnoel said at 2008-07-21 23:27:39

Bug party triage.

garyo said at 2008-08-20 04:21:48

Bug party triage: move to 1.x, not a regression in 1.0.

gregnoel said at 2008-12-26 13:27:32

Adjust triage of issues.

stevenknight said at 2009-11-10 18:00:19

stevenknight => issues@scons

ajf58 said at 2014-07-07 12:26:26

This looks OK to me.
$ ls
a.cpp b.cpp SConscript SConscript
$ cat SConstruct
VariantDir('build_1', '.')
SConscript('build_1/SConscript')
SConscript('SConscript', variant_dir='build_2', src_dir='.')
SConscript('SConscript', variant_dir='build_3')
$ cat SConscript
StaticLibrary(target='mylib', source=Glob('*.cpp'))
$ scons -Q
scons: building associated VariantDir targets: build_1 build_2 build_3
g++ -o build_3/b.o -c build_3/b.cpp
ar rc build_3/libmylib.a build_3/b.o
ranlib build_3/libmylib.a
g++ -o build_2/b.o -c build_2/b.cpp
ar rc build_2/libmylib.a build_2/b.o
ranlib build_2/libmylib.a
g++ -o build_1/b.o -c build_1/b.cpp
ar rc build_1/libmylib.a build_1/b.o
ranlib build_1/libmylib.a
I suggest this is marked as resolved (fixed).

dirkbaechle said at 2014-07-12 03:52:00

I'm not convinced, yet. It would be awesome if someone could do the detective
work, and find out which commit changed things and makes the originally failing
cases work now.
Do our test cases properly cover these scenarios?
Needs further investigation, in my opinion.

I just rediscovered this same problem.  It still exists in scons version 4.0.1

@garyemiller - what was your usage (exactly)?

Well, the SConscript is 2700+ lines, so you prolly do not want to know "exactly".

SConscript('buildtmp/SConscript', must_exit=True, duplicate=1)

I'll see if I can distill a working example.

Sorry, I tried to replicate what I thought I saw and failed.  And re-reading this issues original description that is not what I thought I was looking at.  Sorry to waste your time.

Maybe drop in on the discord channel and chat about it?  It is true that variant_dir in SConscript call is different from VariantDir and (at the risk of offending @bdbaddog) I find some of the "visibility" aspects of variants pretty irritating.


Need to reverse the OneDrive-Uninstaller script

Curious as to whether a script was created to reverse the affect of this script on a Windows 10 workstation.

Hi,
I haven't written a script to reverse the process, but you can download the OneDrive app from the Microsoft website to reinstall it.


razed:waterway=stream should not be expected as an area

Describe the bug
warning tells that razed:waterway=stream should be an area
In most cases waterway:stream are lines: (17 M line / 41 k area features)
To Reproduce
[Steps to reproduce the behavior:
1.got to way https://www.openstreetmap.org/way/1109163077)
remove area=no
Expected behavior
no warning for this feature

This is most likely a bug in iD itself, and not caused by the tagging presets. I'll move the issue to the other repository.


Transform fingerprint collisions due to setting fixed random seed

Describe the bug
The transform fingerprinting logic relies on the random library for random bits when the function is not hashable (e.g. bound methods as used in trl: https://github.com/huggingface/trl/blob/main/trl/trainer/dpo_trainer.py#L356). This causes collisions when the training code sets a fixed random seed, which is common practice: https://github.com/huggingface/alignment-handbook/blob/main/recipes/zephyr-7b-beta/sft/config_full.yaml#L45.
This results in fingerprint collisions which leads to silently loading incorrect cache files corresponding to completely different datasets.
Steps to reproduce the bug
n/a
Expected behavior
Use uuid v4 instead of random.getrandbits()
Environment info
datasets main branch

I've opened a PR with a fix.

I don't think the PR fixes the root cause, since it still relies on the random library which will often have its seed fixed. I think the builtin uuid.uuid4() is a better choice: https://docs.python.org/3/library/uuid.html


Add flag evaluation emitter

There's some things that need tweaking:

It’s sending evaluation for all flags when it receives them from the API, rather than when they are evaluated with hasFeature/getValue. We want to record when the client application actually evaluates a specific flag. Then we know which flags are being evaluated and which aren't.

It needs to use async storage so that it sends correct data if there is bad network e.g. on mobile. It should record data if the client is offline and then post when network is available.

It’s posting every single evaluation individually rather than batching them into a single call.


Run LXD with the adm group to let the runner user interact with LXD

@roosterfish I initially wanted to switch to using canonical/setup-lxd but then realized we use a "specialized" init to use ZFS here. So I ended up just taking the group trick.


silero-models and silero-vad combined lead to ImportError

If using both silero-models and silero-vad combined in a function call, only either the models or vad call works, while the second leads to an ImportError:
ImportError: cannot import name 'get_speech_ts'
I assume not being aware of something trivial here, but couldn't figure out how to solve this until now. Any ideas?

Can you please post a code snippet?

It seems to be related to local cache and torch.hub.load() when loading both models in the same subprocess.
Quoting PyTorch documentation:
"A known limitation that worth mentioning here is user CANNOT load two different branches of the same repo in the same python process. It’s just like installing two packages with the same name in Python, which is not good. Cache might join the party and give you surprises if you actually try that. Of course it’s totally fine to load them in separate processes."
https://pytorch.org/docs/stable/hub.html
Current implementation:
-> call in loop
outputs = speech_to_text_extraction._speech_to_text(path+country+i, country_v1)
timestamps = speech_to_timestamps_extraction._speech_to_timestamps(path+country+i)
-> function 1
def _speech_to_timestamps(filepath):
# load model and utils from github if not previously done so
# force_reload=True to ensure newest version
model2, utils2 = torch.hub.load(repo_or_dir='snakers4/silero-vad',
                              model='silero_vad')

#  util functions
(get_speech_ts,
 _, read_audio,
 _, _, _) = utils2

# read .wav file
wav = read_audio(filepath)

# extract timestamps
speech_timestamps = get_speech_ts(wav, model2,
                                  num_steps=4)

# print and return speech timestamps
print(speech_timestamps)
return speech_timestamps

-> function 2
def _speech_to_text(filepath, language):
# silero STT model
# set device to cpu
device = torch.device('cpu')

# load model from silero repo
model, decoder, utils = torch.hub.load(repo_or_dir='snakers4/silero-models', model='silero_stt', language=language)

# util functions
(read_batch, split_into_batches,
 read_audio, prepare_model_input) = utils

# test file on STT silero model
file = glob(filepath)

batches = split_into_batches(file, batch_size=100)
input = prepare_model_input(read_batch(batches[0]),
                            device=device)

# generate prediction from model
output = model(input)
# return entire prediction as string
print(decoder(output[0]))
return decoder(output[0])

Currently trying to implement the second call into a new subprocess as a workaround.
If you know a better solution I'd greatly appreciate it though!

Hm, interesting, I suppose repos are different but the name is the same, that's why it has a problem (I believe it can be checked by following torch hub cache paths)?
I will verify a bit later, but some plain solutions out of my head:

Use force_reload=True, though it is suboptimal, maybe you will have to load in one function;
Use suprocess;
Architecture wise it makes more sense to separate vad and stt into different workers / entities / consumers;

Alright, thank you

It pulls repos into different folders for sure
files_dir = torch.hub.get_dir()
!ls $files_dir

>> snakers4_silero-models_master  snakers4_silero-vad_master

So, I believe I am narrowing on the culprit
I have tried various torch hub models
And omitting the dependencies and other code bulk, this can be made to work
stt_model, decoder, stt_utils = torch.hub.load(repo_or_dir='snakers4/silero-models',
                                               model='silero_stt',
                                               language='en',
                                               device=device)

model = torch.hub.load('huggingface/pytorch-transformers', 'model', 'bert-base-uncased')

but this cannot
stt_model, decoder, stt_utils = torch.hub.load(repo_or_dir='snakers4/silero-models',
                                               model='silero_stt',
                                               language='en',
                                               device=device)

vad_model, utils_vad = torch.hub.load(repo_or_dir='snakers4/silero-vad',
                                      model='silero_vad')

Using force_reload=True does not really help, and cache is correctly stored in two different folders

Found the culprit, looks like all torch.hub models are treated as the same package / namespace. So you cannot have duplicate module names there.
This minimal example solves the problem
import torch

device = torch.device('cpu')  # gpu also works, but our models are fast enough for CPU
files_dir = torch.hub.get_dir()

!ls $files_dir

hubconf = """dependencies = ["torch"]
import torch
from utils2 import init_jit_model

def silero_vad(**kwargs):
    hub_dir = torch.hub.get_dir()
    model = init_jit_model(model_path=f"{hub_dir}/snakers4_silero-vad_master/files/model.jit")
    return model
"""

!cp $files_dir/snakers4_silero-vad_master/utils.py $files_dir/snakers4_silero-vad_master/utils2.py
!echo '$hubconf' > $files_dir/snakers4_silero-vad_master/hubconf.py
!cat $files_dir/snakers4_silero-vad_master/hubconf.py

vad_model = torch.hub.load(repo_or_dir='snakers4/silero-vad',
                           model='silero_vad')

stt_model, decoder, stt_utils = torch.hub.load(repo_or_dir='snakers4/silero-models',
                                               model='silero_stt',
                                               language='en',
                                               device=device)

@samueldomdey
After a small fix in the silero-vad this works just fine (notice that I reloaded cached files intentionally here):
import torch
device = torch.device('cpu')

stt_model, decoder, stt_utils = torch.hub.load(repo_or_dir='snakers4/silero-models',
                                               model='silero_stt',
                                               language='en',
                                               device=device,
                                               force_reload=True)

vad_model, utils_vad = torch.hub.load(repo_or_dir='snakers4/silero-vad',
                                      model='silero_vad',
                                      force_reload=True)

Architecture wise it makes more sense to separate vad and stt into different workers / entities / consumers;

Also please note that architecture-wise VAD is fast enough to operate on 1 thread, while STT is not.
So you for test purposes it is of course fine to keep them together, but in ideal world they are better kept in separate workers.

Please confirm the fix and close

You are correct, it does work now.
Vielen Dank for looking into it and fixing it!


Build is failing with ELIFECYCLE

If somehow lib/binding directory is changed to a file, build process fails with:
Failed to execute '/usr/bin/node /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/home/aea/aecad/node_modules/node-occ/lib/binding/occ.node --module_name=occ --module_path=/home/aea/aecad/node_modules/node-occ/lib/binding --napi_version=3 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v57' (1)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

This case has occurred on two distinct computers.

I can't build it

something the matter


Fix typo.

This PR fixes a Python 2.7 incompatibility. We used to test for the existence of 'self.tests' as a measure of whether tests were successfully instantiated. However, that member only exists in Python 3.
Testing for 'self.test' instead will work for Python 2 and Python 3.

I think it would be good to add a couple tests exercising python -m numba.runtests -l. Perhaps by spawning subprocesses.

Hmm, while I agree in principle about testing even the testing infrastructure itself, I think this is quite hard. Such failures always creep in in corner-cases that are hard to test for. (It would be different if we were in the unit testing business, though at that point I'd rather throw out unittest and start from scratch, based on the code I have seen...)

I agree, but at least testing the very basic functionality of test selection ensures we don't regress unwillingly there.

Yes, that's possible. We may compare the reported numbers of tests to be in a certain range (given that among platforms and Python versions there is quite a bit of variability, so we can't hard-code numbers).

For the record, this makes some CUDA Jenkins builders run the whole test suite, instead of only the CUDA suite.

OK, so I'm convinced now that we do have to add some tests. This is starting to be ridiculous...
Thanks for testing this !

I have added a few test infrastructure tests (see python numba.tests.test.py to run them in isolation.
I also took the opportunity to add a new parameter ('topleveldir'), so now the numba.testing module can be used with other projects).

Why doesn't numba/tests/test.py have a regular test name, such as numba/tests/test_runtests.py? Is it run as part of the test suite (including on Travis CI, for example)?

Good question. Originally I hadn't planned to integrate it, but thought that we may want to keep it separate (part of the problem right now is that there is no automated way to check whether a test was run and succeeded or wasn't detected at all). Do you have an opinion on the matter ? I'm fine with either: I can just rename it to something more meaningful or move it elsewhere entirely so it isn't run as part of the default test suite.

(OK, I simply renamed it to test_runtests.py, as I assume that's what you'd prefer.)

The upstream CI builders apparently run the tests from a directory other than the project root, which adds another parameter that isn't easy to check automatically for. (Another argument to reduce the variability in all our test invocations !)
This is now also fixed with the last checkin.


Using code for my own plugin.

Hey m0bilebtw
Would i be allowed to use the SoundEngine class and the Sound enum for my own plugin?
I also would like to use a cutted of one of the c engineer sounds.
Thanks already

Code can be used as per the LICENSE file (using BSD 2-Clause "Simplified" License for runelite plugin hub) - I'd suggest getting explicit permission from C Engineer for use of voice clips, as I did for this plugin.


AD identity search and Pull Request (Created/Closed) results are incorrect.

@ostreifel
Referencing issue #17  and the following workaround for Active Directory (AD) group members:
https://github.com/ostreifel/vsts-contributions/commit/603c161e3496b073787e240ea0eb4a0629f22d37
Is there anyway to present AD group members as an suggestion in the activity filter?  It would help tremendously if we did not have to type out the entire user's name or email.
With the workaround previously committed, the created and closed Pull request counts are incorrect when the filter is applied. As a test, when I added 2 of my AD group members to the team as an explicitly mentioned member, the PR created and closed counts' were correct.
Please help if you can, I would appreciate this.

@ostreifel Providing more information about the incorrect pull request counts and commit #603c161:
There are currently 80 repositories and over 50 teams in our Azure Devops Projects.
When I enter in a user's email as a workaround for the name suggestion (we use Active Directory), I see that every team member in the activity filter has 344 pull requests.
Looking further, every pull request for all repositories (since last November) that our team has committed to in our project. When I remove all team members from the activity filter and leave only 1 I still see 344 created pull requests (same count for closed pull request filter)
I temporarily added an existing (very active) team member (referenced by the AD member group) to the team config and he was suggested when I searched for his identity...after selecting his name, he then displays 54 pull requests for the single repository he has been working in. This is correct.
I think incorporating the members from Active Directory into the suggestions from an identity search would solve 2 issues - the created and closed PR contributions and the team member suggestion box (activity filter).

There appears to be an api that almost does this but the client libraries are not currently working - see attached commit for details

The problem is that the client does not seem to consider that the graph apis are hosted on a different domain from the project. Even if the client did consider that the CORS header is not being set in those apis so the browser would just block the request. I can't find any examples of this api being used on github either.
To unblock this feature the CORS header in the Graph Apis and graph api client libraries need to be updated by the Azure Devops team.

@ostreifel
The api you have added as part of #be33f3e  (scripts/data/identities/getGraphIdentities.ts), is there another API that you could reference? Else, do you have to raise this as a feature request in the ms developer community for Azure Devops or is there a better approach?

This has actually been an issue I've had as an extension author for a while now. Using a teams as a workaround for identities mostly works - it just doesn't expand groups. The only identity api's for extensions that I know of are the teams apis.
There is technically an identity picker api for extensions but its so buggy/undocumented that its not worth using. A product change to make this supported would be most likely be to make that picker work.

@ostreifel Would you create a feature request for the product change and provide a link please? I would like to see a follow-up on behalf of the product team.

@ostreifel  Were you able to add a feature request in the developer community?  Has anyone from the product team reached out about this issue?

@ostreifel Were you able to add a feature request in the developer community? Has anyone from the product team reached out about this issue?


what is the function of "send ping request every 5 minitus?

why should we send ping request to amazon every 5 minitus,and before we sendAudioRequst we must openDownChannel,why?

The answer to both your question is given in this link - https://developer.amazon.com/public/solutions/alexa/alexa-voice-service/docs/managing-an-http-2-connection#ping-timeout
Read - Downchannel Lifecycle and Ping and Timeout


Do not return err on symlink eval

Currently the only way to hit this error is to remove the block of code in daemon/start.go#setHostConfig which creates the paths for binds.
What's weird is the error returned when doing this is an Lstat: No such file or directory, but docker run/docker create gives either:

If --name is specified, says that --name is already taken and it's not
If no --name, it says it can't find the image

Also, this could potentially be hit if someone fires up a daemon and there is a container which has a bind mount set for a host path which does not exist.
I'm going to add some unit tests for volumes separately.

Ping @crosbymichael

LGTM

LGTM


Catalog service to use cached git clone

We would like to ship a pre-cloned version of the catalog repo as part of the rancher server container.
If the rancher-catalog-service detects a cloned version, it should try and pull an update down to the server.
If the upstream catalog url can not be reached or cloned, then it should serve up the cached version shipped with the container.

I added cached copy to build process.
I run rancher-catalog-service -validate -catalogUrl <default> as part of the container build process.
It gets cloned into /var/lib/cattle/DATA
I use IPtables to block outbound 443.
when the container starts up I get:
INFO[0000] Starting Rancher Catalog service
INFO[2016-03-21T04:35:55Z] Using catalog library=https://github.com/rancher/rancher-catalog.git
INFO[2016-03-21T04:35:55Z] Using catalog community=https://github.com/rancher/community-catalog.git
INFO[2016-03-21T04:35:55Z] Cloning the catalog from git URL https://github.com/rancher/rancher-catalog.git
Cloning into './DATA/library'...
INFO[0000] Setting log level                             logLevel=info
INFO[0000] Starting go-machine-service...                gitcommit=8bc2633
INFO[0000] Updating docker-machine-drivers from cattle.
INFO[0000] Starting rancher-compose-executor             version=v0.7.2
fatal: unable to access 'https://github.com/rancher/rancher-catalog.git/': Failed to connect to github.com port 443: Connection refused
ERRO[2016-03-21T04:35:55Z] Failed to clone the catalog from git err: exit status 128
INFO[2016-03-21T04:35:55Z] Cloning the catalog from git URL https://github.com/rancher/community-catalog.git
Cloning into './DATA/community'...
INFO[0000] Initializing event router                     workerCount=10
fatal: unable to access 'https://github.com/rancher/community-catalog.git/': Failed to connect to github.com port 443: Connection refused

When I click on the catalog in the UI no items show up.
I would expect that the catalog items still show up. The /var/lib/cattle/DATA directory is destroyed.
This is using the latest v0.6.0 version.

These are the steps I did:

docker run -it rancher/server:v1.0.0-rc1 bash
root@c5cb3b4b7db9:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  s6-statics  sbin  service  srv  sys  tmp  usr  var

====> As seen here no "DATA" directory seen existing

So when catalog-service is started, it clones the repo's newly

root@c5cb3b4b7db9:/# rancher-catalog-service -validate -debug -catalogUrl library=https://github.com/rancher/rancher-catalog.git,community=https://github.com/rancher/community-catalog.git
INFO[0000] Starting Rancher Catalog service
DEBU[2016-03-22T19:48:59Z] url %slibrary=https://github.com/rancher/rancher-catalog.git,community=https://github.com/rancher/community-catalog.git
INFO[2016-03-22T19:48:59Z] Using catalog library=https://github.com/rancher/rancher-catalog.git
INFO[2016-03-22T19:48:59Z] Using catalog community=https://github.com/rancher/community-catalog.git
DEBU[2016-03-22T19:48:59Z] Catalog library does not exist, proceeding to clone the repo :
INFO[2016-03-22T19:48:59Z] Cloning the catalog from git URL https://github.com/rancher/rancher-catalog.git
Cloning into './DATA/library'...
remote: Counting objects: 1038, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 1038 (delta 0), reused 0 (delta 0), pack-reused 1036
Receiving objects: 100% (1038/1038), 380.62 KiB | 0 bytes/s, done.
Resolving deltas: 100% (473/473), done.
Checking connectivity... done.
DEBU[2016-03-22T19:49:01Z] Reading metadata folder for template:cloudflare, path: DATA/library/templates/cloudflare
DEBU[2016-03-22T19:49:01Z] Reading metadata folder for template:convoy-gluster, path: DATA/library/templates/convoy-gluster
DEBU[2016-03-22T19:49:01Z] Reading metadata folder for template:convoy-nfs, path: DATA/library/templates/convoy-nfs
DEBU[2016-03-22T19:49:01Z] Reading metadata folder for template:dnsimple, path: DATA/library/templates/dnsimple
DEBU[2016-03-22T19:49:01Z] Reading metadata folder for template:logstash, path: DATA/library/templates/logstash
DEBU[2016-03-22T19:49:01Z] Reading metadata folder for template:pointhq, path: DATA/library/templates/pointhq
DEBU[2016-03-22T19:49:01Z] Reading metadata folder for template:route53, path: DATA/library/templates/route53
INFO[2016-03-22T19:49:01Z] Catalog loaded without errors

Stop the rancher-catalog-service

Start back again

root@c5cb3b4b7db9:/# rancher-catalog-service -validate -debug -catalogUrl library=https://github.com/rancher/rancher-catalog.git,community=https://github.com/rancher/community-catalog.git
INFO[0000] Starting Rancher Catalog service
DEBU[2016-03-22T19:49:47Z] url %slibrary=https://github.com/rancher/rancher-catalog.git,community=https://github.com/rancher/community-catalog.git
INFO[2016-03-22T19:49:47Z] Using catalog library=https://github.com/rancher/rancher-catalog.git
INFO[2016-03-22T19:49:47Z] Using catalog community=https://github.com/rancher/community-catalog.git
DEBU[2016-03-22T19:49:47Z] Catalog library already exists with same repo url, pulling updates
DEBU[2016-03-22T19:49:47Z] Pulling the catalog library from the repo to sync any new changes to ./DATA/library
DEBU[2016-03-22T19:49:47Z] Reading metadata folder for template:cloudflare, path: DATA/library/templates/cloudflare
DEBU[2016-03-22T19:49:47Z] Reading metadata folder for template:convoy-gluster, path: DATA/library/templates/convoy-gluster
DEBU[2016-03-22T19:49:47Z] Reading metadata folder for template:convoy-nfs, path: DATA/library/templates/convoy-nfs
DEBU[2016-03-22T19:49:47Z] Reading metadata folder for template:dnsimple, path: DATA/library/templates/dnsimple
DEBU[2016-03-22T19:49:47Z] Reading metadata folder for template:logstash, path: DATA/library/templates/logstash
DEBU[2016-03-22T19:49:47Z] Reading metadata folder for template:pointhq, path: DATA/library/templates/pointhq
DEBU[2016-03-22T19:49:47Z] Reading metadata folder for template:route53, path: DATA/library/templates/route53
INFO[2016-03-22T19:49:47Z] Catalog loaded without errors

@cloudnautique I see the service using the cached repo if it is available under ./DATA//  where ./DATA should be available at the level where the service is started.

@prachidamle, we had a packaging issue with the Rancher server container. Will be testing again in the next RC.

@prachidamle , it is using the cached copy in RC2.
The log levels are ERROR, can they be WARN. This is normal/defined behavior for hosts behind proxies or Airgap networks.

Can we do exponential back off up to 1 check/day or something like that so this isn't spamming logs for updates that will never succeed on airgapped networks?  And reset the back off on a reload request from the UI.

@cloudnautique I can change the log levels
@vincent99 can we use the parameter "refreshInterval" for this use case? This interval can be set to 24hrs and the service will not pull often. I am wondering if for air gapped usecases, catalog service "refreshInterval" could be set higher.
Btw this parameter can be changed from cattle settings and that will reboot the catalog service

@cloudnautique Under v1.1.4, I've mounted a host path under /var/lib/cattle/DATA/<catalogName> which is not a git repository, but since the catalog service fails to git pull, it is unable to use it.


NIFI-12515 Removed deprecated Client Auth property from AMQP processors

Summary
NIFI-12515
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[x] Apache NiFi Jira issue created

Pull Request Tracking

[x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[x] Pull Request based on current revision of the main branch
[x] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 21

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

Do we want to leverage the migrateProperties feature so that it'd be transparent for the users when upgrading to 2.0 and not make the processor invalid?

Thanks for the heads-up @pvillard31!
Added migrateProperties().


Quick Question: Does MRTK3 break existing MRTK2 projects?

Hi!
I have my project all setup and working with MRTK2 and I like the new prefabs. Does adding the new MRTK3 break the existing project?
Best regards!

For now, we don't support intermingling MRTK3 features with MRTK2 projects, with the exception of data binding. Stay tuned for additional info + guidance regarding migrating existing MRTK2 projects to MRTK3.

Thanks, for now all I want to do is to import new buttons and hand menu prefabs 👍🍡
Best regards!


Mock in Container - Error

When I try to start the API for testing in docker container, I get the following error:
Failed to start API Mock endpoint UnknownEndpoint: Inaccessible host: localhost. This service may not be available in the us-fake-1' region.

Hi @aexei what do you mean when you say "testing in docker container"? The CLI does not use docker for mock. Can you provide more details around what resources you have in your project and detailed repro steps that led to this error?

Hi @edwardfoyle,
For our CI/CD process, we use Gitlab Pipelines and have implemented various tests (e2e, intergration, ...). For this I want to mock the API within a Gitlab CI/CD job and then run the tests against the API. Gitlab uses one Docker container per job. Initializing Amplify and getting it ready works without problem, but when I run amplify mock I get the error.
I know that in Gitlab localhost is blocked, so I created a custom Docker container. This is then executed inside the job. Unfortunately, nothing changes.  If I run the container on my Mac, everything runs without a problem.
Gitlab Job:
integration:
  stage: test
  services:
    - docker:19.03-dind
  image: docker:latest
  script:
    - docker run
        --rm
        -v ${PWD}:/data
        --env AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
        --env AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
        --env AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
        notionug/docker-aws-amplify
        sh /data/.scripts/ci/runTestIntegration.sh

Dockerfile.txt
init_project_ci.txt
runTestIntegration.txt

@edwardfoyle
I have found a solution. Apparently one part (I suspect DynamoDB) does not work on Alpine Linux. I have adjusted my DOCKERFILE to amazon linux 2 and everything works


Android App Crashes When Submitting Exposure Keys (CRITICAL)

The Android version of the app crashes when trying to share exposure keys.
Platform: Android
Build: 13
Error:
AndroidRuntime: FATAL EXCEPTION: mqt_native_modules
Process: gov.ny.health.proximity, PID: 19935
AndroidRuntime: java.lang.NullPointerException: Attempt to invoke virtual method 'byte[] java.lang.String.getBytes()' on a null object reference
AndroidRuntime: 	at android.util.Base64.decode(Base64.java:119)
AndroidRuntime: 	at com.rajivshah.safetynet.RNGoogleSafetyNetModule.stringToBytes(Unknown Source:1)
AndroidRuntime: 	at com.rajivshah.safetynet.RNGoogleSafetyNetModule.sendAttestationRequest(Unknown Source:0)
AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
AndroidRuntime: 	at com.facebook.react.bridge.JavaMethodWrapper.invoke(Unknown Source:149)
AndroidRuntime: 	at com.facebook.react.bridge.JavaModuleWrapper.invoke(Unknown Source:21)
AndroidRuntime: 	at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
AndroidRuntime: 	at android.os.Handler.handleCallback(Handler.java:883)
AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:100)
AndroidRuntime: 	at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(Unknown Source:0)
AndroidRuntime: 	at android.os.Looper.loop(Looper.java:214)
AndroidRuntime: 	at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(Unknown Source:37)
AndroidRuntime: 	at java.lang.Thread.run(Thread.java:919)

Steps to reproduce:

Complete the onboarding normally
Go to the "My COVID Alerts" tab
Tap on "What to do if you test positive COVID-19"
Tap on "Share your Close Contact Codes"
Enter any code
Tap the "Next" button
App will either crash at this point or after you allow Android to share your keys (depending on if you have allowed this successfully before)

Looking at the app code it seems like we're failing in Safetynet verification.
The only call is at https://github.com/project-vagabond/covid-green-app/blob/nys/src/services/api/index.ts#L35
Given that code the only was we would see a null passed to the base64 function is if the nonce is null.
From the upstream RN wrapper library: https://github.com/rajivshah3/react-native-google-safetynet/blob/master/android/src/main/java/com/rajivshah/safetynet/RNGoogleSafetyNetModule.java#L84
This is a bit of a mystery since to view these screens you must have completed registration with the backend api which provides you with the nonce and a token. Somehow we're getting into a state where the app is trying to verify itself against the server but doesn't have a nonce after registration has completed and initial Safetynet verification has passed.

@dharding @segfault I'll take a look at this and come up with all the details when I'll be able to get hold of some codes to tests the whole flow. Will pick something else in the meantime. /cc @arybka777

Issue was: any error on the server-side was treated as being successful and we were calling a safetynet verification  with an invalin none (i think). All of this was address by simply enabling ssl pinning (https://github.com/project-vagabond/covid-green-app/pull/88).
Issue should be solved in build 15.

@floridemai have you been able to confirm if this issue is addressed?

Are we logging the error on the client side?

@dharding Yes, I think so. I tried doing the  same thing as before but got an "code is expired" and could not replicate the issue.

Are we logging the error on the client side?

@segfault We do log errors on the client side, but in this case it was a native error, and our try/catch blocks didn't have any effect.

I meant the upstream request fail not the SafetyNet RN api call. That call was passed null and somewhere prior there was a call or fetch from something that can and should catch/log the issue. Either we got back a bad value from the backend api or we pulled a bad value. Looking through the full exposure upload workflow it seems we might have stored a bad value for the uploadToken.
Reading through there code there appear to be a few ways for this to fail. https://github.com/project-vagabond/covid-green-app/blob/nys/src/components/views/upload-keys.tsx#L53

Needs to be validated in build 15+

@dharding was unable to submit a number (got error invalid code).

@dharding I was able to submit codes on both iOS and android. @fiacc tried on android on build 15 and worked for him as well. Note that  the codes were generated  with https://push.prd.ctprox.health.ny.gov/notify/positive. Let us know if you still can't submit them. Thanks

@floridemai I will take a look

I was able to enter a positive test case code and then agree to share my keys without a crash. Closing.


Adds needsReassignment to balancer

Adds a new method to the balancer that can be used to efficiently detect when a tablet needs reassignment based on configuration changes in balancer.
fixes #3590

I created this as a draft PR because the implementation is not complete.  It compiles but has never been tested.  Will finish the implementation based on feedback.


D365 Localized database

Hello,
is there localized database for NA and D365? Current one is W1 on Azure that comes with image.
Thanks!

Not NA, but we used to have a US localized database. In July Updated we had to release though on W1 due to internal build process changes.
For more details see #367.

How can we develop US market specific solution if no US database available? (Deposits/Checks/NA Bank Recs functionality and etc.)

We are working on returning the US database back.

Thanks


Add svg support

Could this possibly be done with an extension?

Please expand a little on what you mean with svg support.
DOM element creation is handled by the core. And there is currently no way the core can create namespaced elements (easily fixable).
I've seen other virtual DOM libraries that automatically adds the correct namespace to svg elements.

Yeah this is basically all i mean, being able to create namespaced elements.
Maybe a top level "namespace" or "ns" option to be put on a vnode's data.  Then the logic for creating this element (in core) would be use this to determine whether or not to use createElementNS or createElement.  At that point it should be simple to create hooks (in an extension) to automatically add the svg namespace to certain elements.  Would that work?

As far as I know only svg and MathML require namespaces.
Yes! That would work.

If a ns property is supplied on a vnode's data it is created with createElmentNs (this is similar to virtual-dom).
A svg module adds the correct namespace automatically to elements if their tag name is "svg".

As far as I know only svg and MathML require namespaces.

When you are creating them through the DOM API you also need to apply it to all their children to make it works.
https://jsfiddle.net/1byvqbqd/

@jails Thank you for the heads up. That is unfortunate.

(Probably) a simpler idea is to have a separate factory function svg
function svg(...args) {
  var vnode = h(...args);
  vnode.data.ns = 'http://www.w3.org/2000/svg'
  return vnode;
}

then use it like the h function
var svg = require('snabbdom/svg')

var vnode = svg('svg', [ svg('circle', { ... }) ]);

The issue is inside the {...} i'm not that aware of SVG but i'm not sure if the props module would do it since all SVG examples i've seen so far relies on setAttribute. In snabbdom-jsx examples i used an attributes module that's a mere copy of the props module

@yelouafi That is a slendid idea! Such a simple solution for a problem that would otherwise require work inside the core (I have seen how other virtual DOM libraries handles this and it is an annoying complexity).

In snabbdom-jsx examples i used an attributes module that's a mere copy of the props module

I'd love a PR with that module! :)

I'd love a PR with that module! :)

i'll be glad to do it. We just have a small issue to fix : how to deal with removed attributes. There are 2 ways to define it
1- an attribute is removed when it's not physically present in the object (either in the sens of the in operator or hasOwnProperty method)
2- an attribute is removed when it has some falsy value (null, undefined, false)
The 1st is more consistent but sometimes the 2nd option is more suitable especially in the case of Boolean attributes like disabled for example : a falsy value  means remove the attribute and not just set it to false.
Or perhaps should we adopt a mixed approach : take the 1st option as the general rule, with exception for all known HTML boolean attributes where we opt for the 2nd solution.

FYI i pushed the PR with the mixed approach (special handling for boolean attributes). i think i'd better to move the discussion on the PR

Hi. Just wonder if SVG support has been implemented.  Virtual-dom supports svg in a similar fashion as described here.

@zywind It was never added to Snabbdom. I'll add in the helper function that @yelouafi suggested.

Thanks a lot. Having SVG support would be tremendously useful.

It seems like this would be pretty easy, this is where its done in in virtual-domhttps://github.com/Matt-Esch/virtual-dom/blob/master/virtual-hyperscript/svg.js
It seems as though it would just require a function which sets the namespace (for convenience), and then one could just use the attributes module as usual. Is there anything I'm missing?

I've now added an svg helper to the repository.
Usage is like this:
var vnode = h('div', [
  svg({attrs: {width: 100, height: 100}}, [
    h('circle', {attrs: {cx: 50, cy: 50, r: 40, stroke: 'green', 'stroke-width': 4, fill: 'yellow'}})
  ])
]);

Instead of using h('svg', ... use svg(...). The proper namespace will then be added to the svg element and all of its child elements.
To me this seems like a solution that is both simple in implementation and convenient in usage.
I'd like to hear opinions. This is not released yet.

:+1: It's the same thing that virtual-dom uses. Personally, I would have loved for h to just auto-detect svg tags, but I think that it'd be easy enough for someone to just make a wrapper for that use case if needed.

That looks like a simple solution to me! Do you think there will be any noticeable performance issues performing the recursion over the children like that?
Virtual-dom does is slightly differently (without the recursion), by using svg('svg', {}, [ svg('circle', {}, [])]). Where svg() adds the namespace to the supplied tagName, not all of its children.
If the performance is negligible I like the proposed solution much more from a usability standpoint.

@paldepind why the recursion in the children ? if we call svg in the children instead of h we wont need the recursion ? did i miss something ?
var vnode = h('div', [
  svg({attrs: {width: 100, height: 100}}, [
    svg('circle', {attrs: {cx: 50, cy: 50, r: 40, stroke: 'green', 'stroke-width': 4, fill: 'yellow'}})
  ])
]);

Do you think there will be any noticeable performance issues performing the recursion over the children like that?

No, not really. Recursion isn't expensive. Snabbdom and other virtual DOM libraries uses recursion to construct the DOM. As long as you don't hit the stack limit you should be fine.

@paldepind why the recursion in the children ? if we call svg in the children instead of h we wont need the recursion ? did i miss something ?

That is only for convenience.
As far as I can tell there are three options:

Users should use the svg helper on all svg elements–which gives the simplest implementation
Users should only use the svg helper on the svg element itself. The current implementation.
h detects svg elements and working with svg is seamless.

Instead of using recursion in the svg helper the core of Snabbdom could implement namespace inheriting. I'm aware that other virtual DOM libraries does that.
I'm currently thinking that going for user convenience as @RangerMauve suggests would be optimal. This also means that svg would automatically work with snabbdom-jsx as well.

I'm currently thinking that going for user convenience as @RangerMauve suggests would be optimal. This also means that svg would automatically work with snabbdom-jsx as well.

If you think the perf. costs aren't expensive then i don't see a particular problem; although my personal thinking is that using an explicit svg factory for each node makes the code intent more explicit, actually i think of svg like a partial application of h.
From the JSX point; snabbdom-jsx doesn't actually use the h function but constructs the vnodes directly; there is another factory svg; but given that we have to put a JSX pragma at the top of the source file we can't actually mix both html and svg content. the second approach seems to solve that issue : although the approach can be directly implemented inside snabbdom-jsx. Another solution is to use the h factory inside snabbdom-jsx and switch to svg whenever we encounter and svg tag.

Though the current implementation works fine, I would go either with 1 or 3.

(1) Users should use the svg helper on all svg elements–which gives the simplest implementation

Good: the most explicit approach and simplest implementation.
Bad: one extra thing to think about?

(3) h detects svg elements and working with svg is seamless.

Good: it just works…
Bad: adding to core?
Hope this helps.
(Hi there, I'm new here. I'm planning to refactor a side project from React to snabbdom and it needs svg :smile:)

I don't really think that a separate svg helper makes the code more explicit in a way that really matters.
Consider this
var vnode = h('div', [
  svg({attrs: {width: 100, height: 100}}, [
    svg('circle', {attrs: {cx: 50, cy: 50, r: 40, stroke: 'green', 'stroke-width': 4, fill: 'yellow'}})
  ])
]);

Versus this
var vnode = h('div', [
  h('svg', {attrs: {width: 100, height: 100}}, [
    h('circle', {attrs: {cx: 50, cy: 50, r: 40, stroke: 'green', 'stroke-width': 4, fill: 'yellow'}})
  ])
]);

I think they're equally easy understand.
@acstll

Bad: adding to core?

It can also be implemented inside h without touching core.

I've added svg support to h and released a new version with the feature.
I choose that approach because having svg "just work" seemed to be the simplest and most predictable approach with regards to the API. This also avoids having to reimplement stuff in the svg helper that is already is in the h helper.
Unless there is any objections I will close this issue.

Ok for me. I think i'll do the same thing in snabbdom-jsx

Awesome! I think that solution will be easiest going forward. It'd be cool if virtual-dom did the same thing, but oh well.

Loving it (both API and implementation).

I'm happy you like it :)


How to build the libEsptouch_v<IP_ADDRESS>.a in the Lib ESPTouch library.

Thank for controbutors for this repo. It's work well for me. If there is a readme, it will be better for other users. And I 'm new to IOS, I want to know how to make the static lib libEsptouch_v<IP_ADDRESS>.a.

I've update README.md.


use correct label to identify heketi resources

Closes: #136
Signed-off-by: Raghavendra Talur<EMAIL_ADDRESS>

This change is

Even for deploy-heketi this should be taken care in the same model.

Even glusterfs template should be . :)

Hmmm.. we have a problem. @raghavendra-talur @MohamedAshiqrh  The abort will fail in pure kube deployment.

@humblec yeah
there are no template labels there.

then we have to do
${CLI} get dc,svc,routes heketi

@raghavendra-talur  Could we make it only oc specific with  a conditional  if [[ "${CLI}" == *oc* ]]; then

@humblec @MohamedAshiqrh @ramkrsna sent the updated PR, please review.

LGTM.

LGTM


feat(UICustomizationService):Add a customization service

This is just the UI Customization Service, without any use of said service.   See the documentation for details, as well as related PR's for examples of how this is used.

@sedghi - for the first use of the service, how about we do a simple global customization - allow adding extra routes via customization, and then for the modes, show the overlay (which should be ready Monday).
For the naming, I'm trying to avoid name collisions and conflicts with Type versus instance versus registration object.  According to what I understand of the code styling  TypeNames are upper camel case, instanceValues are lower camel case.  Constants are also upper camel case, but can't have the same name as the TypeName,  so that ends up with:
uiCustomizationService for an instanceValue
UICustomizationService for the class for the isntanceValue
UICustomizationRegistration for the registration object.
No, I'm not going to change everything to suite, nor am I going to make PubSubInterface used throughout, but where we have things that disagree with our stated naming conventions, I'd prefer to go back to the naming conventions.  That means I will add the single serve in the lower camel case naming, and leave the other services to be updated at some future point.  Does that mean I think the other ones SHOULD be updated - ABSOLUTELY they should get done, but when the time is appropriate.

Hey there @sedghi @wayfarer3130, I'm just updating the #3009 PR to fix merge issues with this, I don't think much of this relates and I'll probably just fix the merge conflicts and call it a day.
However while digging into the documentation this change I noticed my local environment doesn't seem to have the same types as the examples:
import { Types } from '@ohif/ui';

// UIContextMenu is not found
const customContextMenu: Types.UIContextMenu

I also checked the deployed package @ohif/ui, but as there is no version 3 released I don't imagine this applys?
Are these types coming soon in another branch? Or maybe an example for docs?
I'd be very interested in hearing what the OHIF way is with types, I've been dreaming of building some types/intelisense for the config object.

Hey guys, I noticed a few types are not resolving in the CustomizationService:
configuration: UICustomizationConfiguration;

and also:
protected setConfigGlobalCustomization(
    configuration: AppConfigCustomization
  ):

I can't find the UICustomizationConfiguration and AppConfigCustomization declarations anywhere - are they lost during merge or what?


Feature dyn Array::as_any_mut + Default implementation of arrays.

The new into_mut works like a charm. <3
But its quite hacky to make work with dyn Array.
We then need to use as_any() -> clone and drop the owned dyn Array to drop the ref count again.
Here an excerpt of what we do in polars:
        // Depending on the state of the underlying arrow array we
        // might be able to get a `MutablePrimitiveArray`
        //
        // This is only possible if the reference count of the array and its buffers are 1
        // So the logic below is needed to keep the reference count 1 if it is

        // First we must obtain an owned version of the array
        let arr = self.downcast_iter().next().unwrap();

        // increments 1
        let arr = arr.clone();

        // now we drop our owned ArrayRefs so that
        // decrements 1
        {
            self.chunks.clear();
        }

What I'd ideally like as an api would be:

let owned = std::mem::take(array.as_any_mut().downcast_mut::<PrimitiveArray<T>>())?;
owned.into_mut()

Then a Default implementation could be an empty array from that same type. This is similar to standard data collections in std.
Is this something we could add to the Array trait? I am willing to make a PR.

This would be great and was the exact issue I ran into and mentioned here. Thanks for sharing your solution, I will give this a try.

I think that we have some challenges with the mutable versions since they are statically typed, while the arrays are dynamically typed.
Maybe it is time to introduce dynamically-typed mutable arrays that mimic 1-to-1 the current arrays?


xorg.xorgserver: apply CVE patches

Motivation for this change
https://lists.x.org/archives/xorg-announce/2021-December/003122.html
Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

I just assume that the patches will work as expected when applied to our xorg-server version(s).  That sounds likely, as they're minimalist and it doesn't evolve fast anymore.

List of commits:

https://gitlab.freedesktop.org/xorg/xserver/-/commit/e56f61c79fc3cee26d83cda0f84ae56d5979f768
https://gitlab.freedesktop.org/xorg/xserver/-/commit/b5196750099ae6ae582e1f46bd0a6dad29550e02
https://gitlab.freedesktop.org/xorg/xserver/-/commit/6c4c53010772e3cb4cb8acd54950c8eec9c00d21
https://gitlab.freedesktop.org/xorg/xserver/-/commit/ebce7e2d80e7c80e1dda60f2f0bc886f1106ba60

Nothing concerning or seemingly breaking for nixpkgs, just some improved bounds checks.

@OfBorg build xorg.xorgserver
(Maybe the builds would start eventually, but I'm impatient.)

I didn't realize that the two darwin platforms are using such an old version (1.18.4).  I don't feel like backporting them, so I assume I'll change this to be linux-only (no commonPatches).  There are probably more vulnerabilities in the old version already anyway, but certainly feel free to fix them (in a separate PR).

I didn't realize that the two darwin platforms are using such an old version (1.18.4). I don't feel like backporting these fixes, so I assume I'll change this PR to be linux-only (no commonPatches). There are probably more vulnerabilities in the old version already anyway, but certainly feel free to fix them (in a separate PR).

Since 2017! https://github.com/NixOS/nixpkgs/commit/10d48574eb7a501db9b6eab0f726db1c07c8e971
@NixOS/darwin-maintainers

Oh, even by my commit.  My darwin-laziness is still the same.

Note that the version darwin uses is the same one linux will use if abiCompat is set to 1.18, we might want to deprecate / remove that configuration option if we're not backporting security patches to it and it shouldn't be used.

abiCompat 1.18 isn't in use in NixPkgs, but I see 1.19 still set by amdgpu-pro (assuming that module still works).

@corngood: still maintaining amdgpu-pro?

No, not really.  Because of amdgpu/mesa I haven't needed it in years.  I'll take a look at the latest release just in case it's easier to get running than the last time I tried.

If you suspect that it might not generally be useful anymore, we could e.g. mark it as insecure/broken and see if someone complains – and drop it some months later otherwise.

@vcunat I like the idea of supporting it in nixpkgs.  it's a hacky pain in the ass, but it's a pretty cool example of how we can make the most uncooperative software work.  It's also an alternate implementation of opengl/vulkan/etc for some pretty popular hardware, which seems like it could be valuable.
Last time I gave up because it didn't work on any of the LTS kernels, but this one seems to work on 5.10.  WIP here #151019


brew-cask: depend on Ruby 2.0.

Nicer to do this at Homebrew time so it can e.g. install the Ruby rather than throwing an error at runtime.

Lovely!  Thanks!

This change pulled in ruby 2.2.2 for me when I updated to brew-cask 0.54 on Yosemite, where the system ruby already is 2.0.0:
ruby --version
ruby 2.0.0p481 (2014-05-08 revision 45883) [universal.x86_64-darwin14]

This is undesired behaviour when a ruby version manager ist used. For example it changes the system ruby for rbenv from Apple’s ruby 2.0.0 to homebrew’s ruby 2.2.2 (when /usr/local/bin appears before /usr/bin in your path).
Since you only want to depend on ruby 2.0.0 brew-cask should not install a newer ruby but use the system ruby on Yosemite and Mavericks, where ruby 2.0.0 is already preinstalled.

This will use whatever Ruby is first in your PATH.


Questions/observations on requirements regarding to "links"

Recommendation 3 "/rec/types-schemas/type-links" , "The "links" array in any JSON-FG feature in the JSON document SHOULD include at least one link object where the "rel" member is set to "type"."

so in a FeatureCollection of many Feature, each Feature would have this link object ? A bit of bloat if many features
the spec doesn't seem to explain what type of content is pointed by the link. I guess on purpose, but this doesn't look like something you can use for an automated process, but more for human consumption

Recommendation 4 "/rec/types-schemas/schema-link-geojson"  .  The recmomandation says "If the JSON document is a GeoJSON feature" or "If the JSON document is a GeoJSON feature collection" whereas "Requirement 19 | /req/types-schemas/schema-link-jsonfg" mentions JSON-FG feature / feature-collections. I don't really get the point of the distinction. The standard addresses JSON-FG feature/feature-collections, and JSON-FG feature/feature-collections are GeoJSON one too, as being an extension.  While the phrasing is correct, the different language between Rec 4 and Req 19 is still a bit confusing. Either change the mention of GeoJSON to JSON-FG, or add a note saying the distinction here is mostly to be pedantic and that a JSON-FG feature[collection] is a GeoJSON one.

We may get several links with rel=describedby and type=application/schema+json. From a consumer point of view, how do I know which one is the most "precise" ? Or am I (I = automater consumer perspective, like a GDAL driver) supposed to open each one and infer that from their content ?

Recommendation 5  "/rec/types-schemas/schema-link-properties" doesn't really help if you have a document with several FeatureTypes. How do I know where to find the schema of a given FeatureType ? We'd ideally need something explicitly relating a value of FeatureType with the corresponding schema. As far as I know a single JSON schema cannot cleanly describe different feature types, contrary to a .xsd where in a single one you can have distinct <xs:element name="myFeatureType" type="my:myFeatureTypeType" substitutionGroup="gml:AbstractFeature"/> elements
In the building.json example (which has only one single FeatureType), I see that the hint is given in the "title"

      {
         "href": "https://example.org/data/v1/collections/buildings/schemas/item?type=jsonfg",
         "rel": "describedby",
         "type": "application/schema+json",
         "title": "JSON Schema of building features"
      }

but that's not very helpful from an automated consumer perspective. Should there be a specific JSON object whose keys would be the featureType and the value the array of links that apply to it ?
So overall, while .gml + .xsd is something that can be exploited "easily" (at least conceptually) from an automated reader point of view, JSON-FG links/schemas seem harder. (hum, not totally sure about what I've just written: I was forgetting for a second how painful making sense of .xsd can be :-))

Thank you for the comments/questions, @rouault.

Recommendation 3 "/rec/types-schemas/type-links" , "The "links" array in any JSON-FG feature in the JSON document SHOULD include at least one link object where the "rel" member is set to "type"."

so in a FeatureCollection of many Feature, each Feature would have this link object ? A bit of bloat if many features

Good point. I guess framing this as a general recommendation is not a good idea. Using those link only makes sense, if there is a resource available that defines the feature type and provides additional information and if the supplier knows that users will benefit from this information.
The main point of adding text about links to a semantic type was to point out the existence of the IANA link relation type and before inventing something new the existing link relation should be used.
For homogenous feature collections, we might also just put a single link in the feature collection instead of the same link in each feature.
We should update the text to make this clearer.

the spec doesn't seem to explain what type of content is pointed by the link. I guess on purpose, but this doesn't look like something you can use for an automated process, but more for human consumption

Correct, at least in general. A community that maintains a feature concept or feature type register could be more specific.

Recommendation 4 "/rec/types-schemas/schema-link-geojson" . The recmomandation says "If the JSON document is a GeoJSON feature" or "If the JSON document is a GeoJSON feature collection" whereas "Requirement 19 | /req/types-schemas/schema-link-jsonfg" mentions JSON-FG feature / feature-collections. I don't really get the point of the distinction. The standard addresses JSON-FG feature/feature-collections, and JSON-FG feature/feature-collections are GeoJSON one too, as being an extension. While the phrasing is correct, the different language between Rec 4 and Req 19 is still a bit confusing. Either change the mention of GeoJSON to JSON-FG, or add a note saying the distinction here is mostly to be pedantic and that a JSON-FG feature[collection] is a GeoJSON one.

I see your point and I agree that we should change the reference in Recommendation 4 from "GeoJSON" to "JSON-FG".

We may get several links with rel=describedby and type=application/schema+json. From a consumer point of view, how do I know which one is the most "precise" ? Or am I (I = automater consumer perspective, like a GDAL driver) supposed to open each one and infer that from their content ?

Technically, multiple such links imply an allOf wrapper around the referenced schemas. It depends on the information that the client is looking for, which is the most important one. I guess for a GDAL driver it would be information to identify the file as a JSON-FG document and information about the feature properties.
For the first aspect we have recently (not included in 0.1) introduced conformsTo (link) which should be better than trying to "sniff" information from a JSON Schema.
For the second aspect it would be the properties schema from Recommendation 5 ("/rec/types-schemas/schema-link-properties").
In OGC API Features we have recently made the decision to specify the schema of a collection with an encoding-agnostic JSON Schema that describes the logical model, not the GeoJSON or JSON-FG encoding. That schema then also describes the feature properties in other encodings like GML, CSV, FlatGeobuf, etc. If an API also wants to link to schemas for validation purposes, they can do that in accordance with the rules for the specific language (e.g., in the JSON Schema case, a link with rel=describedby and type=application/schema+json).
I think we should revisit the schema aspects in JSON-FG also with that in mind.

Recommendation 5 "/rec/types-schemas/schema-link-properties" doesn't really help if you have a document with several FeatureTypes. How do I know where to find the schema of a given FeatureType ? We'd ideally need something explicitly relating a value of FeatureType with the corresponding schema. As far as I know a single JSON schema cannot cleanly describe different feature types, contrary to a .xsd where in a single one you can have distinct <xs:element name="myFeatureType" type="my:myFeatureTypeType" substitutionGroup="gml:AbstractFeature"/> elements
In the building.json example (which has only one single FeatureType), I see that the hint is given in the "title", [...] but that's not very helpful from an automated consumer perspective. Should there be a specific JSON object whose keys would be the featureType and the value the array of links that apply to it ?

Well, one approach to define a single schema for multiple feature types. A pattern could be:
{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "oneOf": [
    {"$ref": "#/$defs/building" },
    {"$ref": "#/$defs/parcel" },
    {"$ref": "#/$defs/etc" }
  ],
  "$defs": {
  ...
  }
}

This is still different from XML in the sense that XML has a element tag to identify the XSD definition to validate against and no such thing exists in JSON. To avoid ambiguities, we need to include the type name in the content - which is what the featureType member is about. This would be our own convention, and we need to add rules how to detect the schema fragment in the schema for a feature type, but I do not see another way, because JSON simply does not offer any standard mechanism.
One conclusion from all this is that we need to work more on the schema aspects to make it as easy and reliable as possible for clients like GDAL to extract schema information.

hum, not totally sure about what I've just written: I was forgetting for a second how painful making sense of .xsd can be :-)

Indeed. But JSON Schema also can be quite complex and weird; and hard to parse. In OGC API Features we have a requirements and recommendations for a profile of JSON Schema that should be expressive enough, but relatively straightforward to parse (currently specified for queryables, but the idea is to apply this to schemas in general; link, see /req/queryables/get-queryables-response and /rec/queryables/queryables-schema). In addition, the idea is to add additional keywords to declare properties as the feature id or the primary geometry property, etc.
As I said above, there is currently work going on in the API context and we need to make sure we approach this in a consistent way across the different standards, including JSON-FG.

Meeting 2023-08-21: The current thinking in the discussion was as described below. We will discuss again in the next meeting in two weeks.

There are two reasons for using a schema: understanding the properties that a feature has / may have and validation of an instance. The first reason is considered to be the more important one.
OGC API Features has decided to use a encoding-agnostic feature schema to describe the schema of a feature type - with additional information to tag special roles of properties, e.g., to identify the primary geometry, in order to be unambiguous how the properties are mapped to a feature encoding, e.g., GeoJSON or JSON-FG. That schema is represented as a JSON Schema, even though it will typically not be used for validation. The reason for using JSON Schema is the fact that JSON provides a straightforward way to express objects, arrays and values, has tool support and the fact that API developers are typically familiar with it (use in OpenAPI etc.). JSON-FG should consider to follow this approach, too, but need to think how we can declare the feature schema for the feature types in the document.
With respect link(s) to JSON Schemas for validation (rel='describedby') as well as for the link to a semantic type (ref='type'), we should remove normative language and simply point out that these standard link relation types can be used, if such links are relevant for the users of the data.
The fact that a document is a JSON-FG document can be determined from the conformsTo array.


VAE - Training

I'm having a hard time recreating your results. I'm trying to retrain the VAE from scratch: LR 1e-4, Adam, 512 embedding size. The validation error seems to be leveling off, and I don't think brute forcing to 150K epochs would solve this issue.
Would it be possible to share your loss function curves?

Need to turn debug mode off

I meet the same problem. Could you please share the correct loss curve?

My problem ended up being that there is a debug flag which the author's used for iterating quickly. The debug mode only loads ~100 examples. Also, the batch size needs to be increased to 256 for VAE. Make sure you are using stage1 config.
In the dataloader, there is also a caching function which checks for the existence of a ./tmp folder and loads a .pkl file instead of loading in the entire dataset again.

Thanks! It really helps! But my commit loss looks strange. Does yours look the same?

Yes

@palmex @aixiaodewugege I am so sorry about your troubles caused by "this debug flag". If you have any other questions, feel free to ask. I'm more than happy to help.

@palmex @aixiaodewugege I am so sorry about your troubles caused by "this debug flag". If you have any other questions, feel free to ask. I'm more than happy to help.

"Hello, I'm a newcomer to VQ-VAEs and am currently working on training a face motion VQ-VAE using your code. However, I've noticed that the validation loss curve appears to be indicating overfitting. Could you please offer some advice to address this issue?"

@aixiaodewugege Based on the loss-feature/train and loss-feature/val curves, there is clear overfitting. The commit loss reflects the efficiency of the codebook, which seems somewhat correct.
One question could help: how does this feature differ from the parameters of FLAME?
A general solution to address this problem is to apply data augmentation and use dropout with masking on both input features and loss during training.

@ChenFengYe
Thank you for your reply! It was really helpful. The FLAME model represents the vertex loss. After adding dropout, the results look better, but I still notice a gap between the training and validation loss. How can I minimize this gap?
Additionally, could you clarify what you mean by data augmentation?

@aixiaodewugege Data augmentation could help to minimize this gap. The specific process could be, for example,  RTS (rotation, translation, scale) on training data (like vertex, XYZ). You can augment your data by following your target tasks.