Some Facebook plugin URLs are unsupported

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.09.27. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2016.09.27

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
> youtube-dl.exe "https://www.facebook.com/v2.3/plugins/video.php?allowfullscreen=true&app_id=&channel=http%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fx
d_arbiter%2Fr%2FP5DLcu0KGJB.js%3Fversion%3D42%23cb%3Df1d45e6da%26domain%3Dwww.mycarforum.com%26origin%3Dhttp%253A%252F%252Fwww.mycarforum.com%252Ff1e33442e4%26r
elation%3Dparent.parent&container_width=883&href=http%3A%2F%2Fwww.mycarforum.com%2FTheStraitsTimes%2Fvideos%2F10153797435237115%2F%3Ftype%3D3&locale=en_US&sdk=j
oey" -v
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['https://www.facebook.com/v2.3/plugins/video.php?allowfullscreen=true&app_id=&channel=http%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2
Fxd_arbiter%2Fr%2FP5DLcu0KGJB.js%3Fversion%3D42%23cb%3Df1d45e6da%26domain%3Dwww.mycarforum.com%26origin%3Dhttp%253A%252F%252Fwww.mycarforum.com%252Ff1e33442e4%2
6relation%3Dparent.parent&container_width=883&href=http%3A%2F%2Fwww.mycarforum.com%2FTheStraitsTimes%2Fvideos%2F10153797435237115%2F%3Ftype%3D3&locale=en_US&sdk
=joey', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2016.09.27
[debug] Python version 3.4.4 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-74113-gcdb0225, ffprobe N-74113-gcdb0225
[debug] Proxy map: {}
[generic] video.php?allowfullscreen=true&app_id=&channel=http://staticxx.facebook.com/connect/xd_arbiter/r/P5DLcu0KGJB.js?version=42#cb=f1d45e6da&domain=www.myc
arforum.com&origin=http%3A%2F%2Fwww.mycarforum.com%2Ff1e33442e4&relation=parent.parent&container_width=883&href=http://www.mycarforum: Requesting header
WARNING: Falling back on generic information extractor.
[generic] video.php?allowfullscreen=true&app_id=&channel=http://staticxx.facebook.com/connect/xd_arbiter/r/P5DLcu0KGJB.js?version=42#cb=f1d45e6da&domain=www.myc
arforum.com&origin=http%3A%2F%2Fwww.mycarforum.com%2Ff1e33442e4&relation=parent.parent&container_width=883&href=http://www.mycarforum: Downloading webpage
[generic] video.php?allowfullscreen=true&app_id=&channel=http://staticxx.facebook.com/connect/xd_arbiter/r/P5DLcu0KGJB.js?version=42#cb=f1d45e6da&domain=www.myc
arforum.com&origin=http%3A%2F%2Fwww.mycarforum.com%2Ff1e33442e4&relation=parent.parent&container_width=883&href=http://www.mycarforum: Extracting information
ERROR: Unsupported URL: https://www.facebook.com/v2.3/plugins/video.php?allowfullscreen=true&app_id=&channel=http%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fxd_a
rbiter%2Fr%2FP5DLcu0KGJB.js%3Fversion%3D42%23cb%3Df1d45e6da%26domain%3Dwww.mycarforum.com%26origin%3Dhttp%253A%252F%252Fwww.mycarforum.com%252Ff1e33442e4%26rela
tion%3Dparent.parent&container_width=883&href=http%3A%2F%2Fwww.mycarforum.com%2FTheStraitsTimes%2Fvideos%2F10153797435237115%2F%3Ftype%3D3&locale=en_US&sdk=joey

Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpaa64pv68\build\youtube_dl\YoutubeDL.py", line 694, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpaa64pv68\build\youtube_dl\extractor\common.py", line 355, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpaa64pv68\build\youtube_dl\extractor\generic.py", line 2424, in _real_extract
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.facebook.com/v2.3/plugins/video.php?allowfullscreen=true&app_id=&channel=http%3A%2F%2Fstaticxx.f
acebook.com%2Fconnect%2Fxd_arbiter%2Fr%2FP5DLcu0KGJB.js%3Fversion%3D42%23cb%3Df1d45e6da%26domain%3Dwww.mycarforum.com%26origin%3Dhttp%253A%252F%252Fwww.mycarfor
um.com%252Ff1e33442e4%26relation%3Dparent.parent&container_width=883&href=http%3A%2F%2Fwww.mycarforum.com%2FTheStraitsTimes%2Fvideos%2F10153797435237115%2F%3Fty
pe%3D3&locale=en_US&sdk=joey
...
<end of log>

Description of your issue, suggested solution and other information
Hi,
When I tried downloading videos from the Facebook urls beginning with "https://www.facebook.com/v2.3/plugins/video.php", the Unsupported URL error is returned.
Eg.:

https://www.facebook.com/v2.3/plugins/video.php?allowfullscreen=true&app_id=&channel=http%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fxd_arbiter%2Fr%2FP5DLcu0KGJB.js%3Fversion%3D42%23cb%3Df1d45e6da%26domain%3Dwww.mycarforum.com%26origin%3Dhttp%253A%252F%252Fwww.mycarforum.com%252Ff1e33442e4%26relation%3Dparent.parent&container_width=883&href=http%3A%2F%2Fwww.mycarforum.com%2FTheStraitsTimes%2Fvideos%2F10153797435237115%2F%3Ftype%3D3&locale=en_US&sdk=joey
https://www.facebook.com/v2.3/plugins/video.php?allowfullscreen=true&app_id=&channel=http%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fxd_arbiter%2Fr%2FP5DLcu0KGJB.js%3Fversion%3D42%23cb%3Df61fb0db%26domain%3Dwww.mycarforum.com%26origin%3Dhttp%253A%252F%252Fwww.mycarforum.com%252Ff77f158a4%26relation%3Dparent.parent&container_width=861&href=http%3A%2F%2Fwww.mycarforum.com%2FBehChiaLor%2Fvideos%2F1779898258932927%2F%3Ftype%3D3&locale=en_US&sdk=joey
https://www.facebook.com/v2.3/plugins/video.php?allowfullscreen=true&app_id=&channel=http%3A%2F%2Fstaticxx.facebook.com%2Fconnect%2Fxd_arbiter%2Fr%2FP5DLcu0KGJB.js%3Fversion%3D42%23cb%3Df1dbf9be7%26domain%3Dwww.mycarforum.com%26origin%3Dhttp%253A%252F%252Fwww.mycarforum.com%252Ff77f158a4%26relation%3Dparent.parent&container_width=883&href=http%3A%2F%2Fwww.mycarforum.com%2FBehChiaLor%2Fvideos%2F1779959948926758%2F%3Ftype%3D3&locale=en_US&sdk=joey

Note the youtube-dl is able to download videos from Facebook urls beginning with
"https://www.facebook.com/plugins/video.php"
Thanks.

This also does not work:
https://www.facebook.com/groups/ateistiskselskab/permalink/10154930137678856/
It thinks you need to log in - which is clearly not true.

@ole-tange That's a different bug. Could you open a new issue?

Is this the same as mine problem?
fun~$ youtube-dl --verbose 'https://www.facebook.com/reel/1463302304499951'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.facebook.com/reel/1463302304499951']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.11.3 (CPython) - Linux-6.3.9-1-default-x86_64-with-glibc2.37
[debug] exe versions: ffmpeg 6.0, ffprobe 6.0
[debug] Proxy map: {}
[generic]<PHONE_NUMBER>499951: Requesting header
WARNING: Falling back on generic information extractor.
[generic]<PHONE_NUMBER>499951: Downloading webpage
[generic]<PHONE_NUMBER>499951: Extracting information
[generic]<PHONE_NUMBER>499951: Requesting header
WARNING: Falling back on generic information extractor.
[generic]<PHONE_NUMBER>499951: Downloading webpage
[generic]<PHONE_NUMBER>499951: Extracting information
[redirect] Following redirect to https://www.facebook.com/plugins/video.php?height=1920&width=1080&href=%2F100013522218132%2Fvideos%2Fmassive-us-aircraft-carrier-in-waters-of-split-croatia-with-70-planes-that-can-r%2F1463302304499951%2F&_fb_noscript=1
[generic] video: Requesting header
WARNING: Falling back on generic information extractor.
[generic] video: Downloading webpage
[generic] video: Extracting information
ERROR: Unsupported URL: https://www.facebook.com/plugins/video.php?height=1920&width=1080&href=%2F100013522218132%2Fvideos%2Fmassive-us-aircraft-carrier-in-waters-of-split-croatia-with-70-planes-that-can-r%2F1463302304499951%2F&_fb_noscript=1
Traceback (most recent call last):
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/youtube-dl/youtube_dl/extractor/generic.py", line 3489, in _real_extract
    raise UnsupportedError(url)
youtube_dl.utils.UnsupportedError: Unsupported URL: https://www.facebook.com/plugins/video.php?height=1920&width=1080&href=%2F100013522218132%2Fvideos%2Fmassive-us-aircraft-carrier-in-waters-of-split-croatia-with-70-planes-that-can-r%2F1463302304499951%2F&_fb_noscript=1

fun~$

@dirkf Thank you, that really makes youtube-dl working for me again! Passed it to the openSUSE package.


Consider replacing request with got.

It would significantly reduce the size of the dependency tree.

Or https://github.com/tomas/needle

@maxogden
request is now deprecated.


Sluggable: can't use "+" as the separator

I'm using the extensions through the StofDoctrineExtensionsBundle. I'm trying to change the default separator to "+":
/**
 * @var string $slug
 * @Gedmo\Slug(fields={"label"}, separator="+")
 * @ORM\Column(length=130, unique=true)
 * @Expose
 */
protected $slug;

It doesn't work, I get the following error:
[Symfony\Component\Debug\Exception\ContextErrorException]
Warning: preg_match(): Compilation failed: nothing to repeat at offset 24

When trying to escape the "+" with "\+", I don't have this error any-more but in the database I've got the following:
my\+slug

Same thing with a double escape "\". Any idea?

The bug only happen when using the uniq constraint, it happens at this line;
    if (!$recursing) {
        // filter similar slugs
        foreach ($result as $key => $similar) {
            $config['separator'] = preg_quote($config['separator']);
            // Here
            if (!preg_match("@{$preferedSlug}($|{$config['separator']}[\d]+$)@smi", $similar[$config['slug']])) {
                unset($result[$key]);
            }
        }
    }


[Humble Actor] Is being listed as a skill

[Humble Actor] is a chat nickname Cara uses in one of the earther group chats, not a skill or class.

Nvm, realized you're just pulling data from the wiki. edited the wiki.

@TheSofties Thanks! I'll make sure to update the database for future updates.
There also seems to be a reference of [Humble Actor] here to Wesle as a [Humble Actor] which might be a class or skill. It's somewhat unclear. Either way I'll add it to my list of ambiguous names.


Namespaces

Please add support for import of namespace. Currently it imports a class but it is not visible without importing namespace.

Adding namespace support is very tricky, because this requires an advanced parsing of the source files.
Besides that, please provide a sample project with an expectation example.

No sample, no further actions.

Hello @pmneo, I have the following usecase
I use React for a TS project and while minifying the expressions React.createElement is quite long, I created first a class named R to include some some shortcuts to the React methods and classes, but then I needed to use types and had to add namespace to be able to export types, example of the namespace:
import * as React from 'react';
import * as ReactDOM from 'react-dom';

export namespace R {

	export type Element = React.ReactElement<any>;
	export type MouseEvent = React.MouseEvent<any>;
	export type TouchEvent = React.TouchEvent<any>;

	export const clone = React.cloneElement;
	export const create = React.createElement;
	export const find = ReactDOM.findDOMNode;
	export const render = ReactDOM.render;
	export const unmount = ReactDOM.unmountComponentAtNode;

	export const Component = class <P, S> extends React.PureComponent<P, S> { };
	export const Stateless = class <P> extends R.Component<P, null> { };
	export const Base = class extends R.Stateless<null> { };
}


[Feature]: Allow metadata to be saved for appsmith users in appsmith db which can be used by the application developers

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
Customer has a heavy API which queries mutliple datasources to collect all the metadata about the user and returns this metadata. If this metadata can be stored in appsmith, then the application doesn't need to call this very heavy API on each page load.
Why should this be worked on?
As above ^

Often, information about Appsmith users exists in systems outside of Appsmith. With user attributes, you can bring this information into Appsmith and store it on the user–across all of your Appsmith apps.
You can create and add attributes to users to enable custom experiences in Appsmith. Admins can create and apply attributes to any Appsmith user. When building Appsmith apps, you can reference attributes through the appsmith.user or a similar object to control functionality.


Refactoring - Create ProjectIconSelection Component

Once PR #120 landed we will have two location where we're showing the project icon selection.
One in CreateProjectWizard and one inside of ProjectConfigurationModal
It would be good to refactor the code duplication into a new ProjectIconSelection Component. So we have only one code location for the selection.
The component will be used like following:
    <ProjectIconSelection
        selectedIcon={?string}
        showRandomSubset={boolean | number}
        onSelectIcon={function} />

showRandomSubset can be called with true to show all icons or a number e.g. 10 so it will only show 10 icons. The code for the limited icon subset is available in 'CreateProjectWizard` component.
It would be also good to have a story for this in Storybook to show the default usage.
Screenshot of selection area from Project creation modal:

I would love to give this a shot. Can I take it?

@raerpo go for it! That'd be much appreciated =)

Thanks @raerpo!


Emitting of error event on task doesn't work

const runCronScheduler = () => {
  const task = cron.schedule('*/1 * * * *', async () => {
    console.log('running a task every minute');
    const result = {
      updates: null
    }
     // provocate an error
    // const newList = await getCurrentNeonProposals()
    // const oldList = await getSavedProposals()
    result.updates = new UpdatesMatcher(newList, oldList)
    ...
    })
  }, {
    runOnInit: true
  })
// try to handle it
  task.on('task-failed', (error) => {
    console.log('CRIT_ERR Node Cron task was failed with error: ', error)
  })
}

Hi, guys. I check your source and try to handle and error from created task, but it is not working.
Can you tip me, please?

You need to do task._task.on('task-failed') - altough task-failed and task-finished should 100% be emitted by ScheduledTask and not the underlying Task


[QUERY] Does the App Authentication library support .NET Core (for requesting SQL Database) ?

Query/Question
In .NET Framework, I am using App Authentication library with Microsoft.Data.SqlClient Active Directory Interactive authentication mode to authenticate and query an Azure SQL Database without any code change as described in this blog post.
Active Directory Interactive authentication mode is now available for .NET Core in Microsoft.Data.SqlClient but I don't know what is the support of App Authentication library for .NET Core. It there a mechanism similar to what is done in .NET Framework to make authentication work simply by setting some configuration and a connection string with "Authentication=Active Directory Interactive" instead of having to manually retrieve the access token and setting it in the SqlConnection.
See related issue  #https://github.com/dotnet/SqlClient/issues/735
Environment:

Name and version of the Library package used: Microsoft.Azure.Services.AppAuthentication 1.5.0
Hosting platform or OS and .NET runtime version (dotnet --info output for .NET Core projects): Windows 10 .NET Core 3.1.402
IDE and version : Visual Studio 16.7.4

Thank you for your feedback.  Tagging and routing to the team member best able to assist.

@TechWatching Unfortunately, we do not yet support the SqlAppAuthenticationProvider scenario for .NET Core. We have a feature request for this but at this time I can't give a timeline for when this will be available.


CI updates

Codecov Report

Merging #50 (60b36e3) into main (51910ed) will decrease coverage by 1.37%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##            main     #50      +/-   ##
========================================
- Coverage   6.79%   5.41%   -1.38%
========================================
  Files          4       4
  Lines        162     203      +41
========================================
  Hits          11      11
- Misses       151     192      +41

Impacted Files
Coverage Δ

src/key.rs
7.74% <0.00%> (-2.94%)
:arrow_down:

src/util.rs
0.00% <0.00%> (ø)

src/format.rs
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51910ed...60b36e3. Read the comment docs.


Temporary "Spy-node" banlist

Long-lived connections from Bitcoin clients that only listen to transaction INVs but never request transactions caused problem. While this problem is fixed in the most recent releases (23.2, 24.1, and 25.0), some network participants (e.g. mining pools running custom patches) might not be able to upgrade immediately. A solution is to ban the entities making these long-lived connections. Some of the entities are likely "spy-nodes" while others are "network monitoring tools" by researchers.
There's a special and temporary version of this banlist that bans these entities for 90 days. It can be found in this branch.
Banlists:

banlist_cli.txt
banlist_gui.txt
banlist_plain.txt

This banlist was originally constructed as a short-term mitigation against the DoS described in https://bitcoincore.org/en/2024/10/08/disclose-large-inv-to-send/ - in case someone (e.g. a miner) can't upgrade reasonably fast due to e.g. custom patches.
Many of these IPs are still active spy-nodes.


pull request for e2e test at 01/17/2017 19:56:08

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Close json code block properly in README.md

Pull Request
What does this PR do?
Fixes #177

PR checklist
Please check if your PR fulfills the following requirements:

[x] Does this PR fix an existing issue?
[x] Have you read the contributing guidelines?
[x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

bors merge

bors merge

bors merge

bors retry

bors ping

@ShakyaCsun can you rebase this branch? I had some issue with our bors automation :/


feat: OCI repository support for Helm charts

Codecov Report

Merging #207 (4150535) into main (d0a1b30) will decrease coverage by 1.67%.
The diff coverage is 28.57%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #207      +/-   ##
==========================================
- Coverage   56.33%   54.66%   -1.67%
==========================================
  Files           9        9
  Lines         742      761      +19
==========================================
- Hits          418      416       -2
- Misses        293      310      +17
- Partials       31       35       +4

Files
Coverage Δ

api/v1alpha1/validatorconfig_types.go
100.00% <ø> (ø)

pkg/validationresult/validation_result.go
93.54% <100.00%> (ø)

internal/controller/validatorconfig_controller.go
70.38% <25.92%> (-7.19%)
:arrow_down:

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d0a1b30...4150535. Read the comment docs.


feature: automation for creating credential definitions

Summary
What it says in the title
Notes

Added a new docker-compose file that creates a local agent with a ledger and genesis transaction so that new schemas can be registered
This needs a bit of review for error cases, but we can tidy that up in a follow up PR

It seems that the clippy check failed. make lint or cargo clippy can check it locally.


AeroDyn14 removal

This PR is not ready for merging (regression tests must be dealt with first).  This PR supercedes #1429 (that PR sat for too long).
Feature or improvement description
The AeroDyn14 module was a carry-over from FAST v7 to FAST v8 as a stop-gap while AeroDyn15 was under development.  As such AeroDyn14 was never converted to the FAST Modular Framework; this created many code workarounds in ElastoDyn and in the openfast-library to get it to work.  At this point AeroDyn15 is far more comprehensive than AeroDyn14 so very few users ever use it. Therefore the plan for many years has been to remove it entirely, and now we finally got around to it.
Also includes minor fixes for AD15 input files:

Updated all AD15 input files to include the UA_Mod comments instead of the deprecated AFAeroMod comments
Updated all AD15 input files to include all lines (some lines are optional)

Related issue, if one exists
#527 -- bug with AD14 tower
#93 -- deprecating AD14
https://github.com/OpenFAST/r-test/issues/43
Impacted areas of the software
There are some significant improvements to the glue code and ElastoDyn with the removal of AD14.
Additional supporting information
The regression tests that previously used AeroDyn14 have been updated to AeroDyn15, but have not been tuned correctly for the DBEMT parameters (in particular the Tau time constants).  Some of these cases also show instabilities and had been turned off as a result.  We need to decide which cases we will simply remove, and which we will keep.
Test results, if applicable
Regression tests with AD14 converted to AD15.

This PR also removes PitchAxis input column from the ED blade file (backwards compatible with older format).


Source code links from doc pages are broken.

Starting in, for example:
https://googlecloudplatform.github.io/gcloud-python/#/docs/master/gcloud/logging/metric/metric
or
https://googlecloudplatform.github.io/gcloud-python/#/docs/master/gcloud/client/client
All the links labeled "Source code"  seem to be broken. ("Not the page you're looking for")
Here's one of the bad links:
https://github.com/GoogleCloudPlatform/gcloud-python/blob/master//home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/json-docs/lib/python2.7/site-packages/gcloud/client.py/gcloud/client.py#L120

Thanks for reporting @sharbison3
@daspecster Can you point us in the right direction?

Yeah, I think I know what's wrong. I have to strip out everything except the lib path and I try splitting it on gcloud  but that's been flakey between environments(local vs travis).

Should be fixed from #2052. Thanks for reporting @sharbison3!


Add default systemPropertyVariables for failsafe and surefire plugins

This is to avoid having to configure those in all the quarkiverse extensions and be closer to the quarkus-parent ones.

Why this content isn't in the project's pom.xml?
AFAIK they would be overridden if this is also defined in the project's pom.xml (eg. when                   <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>)

Why this content isn't in the project's pom.xml?
AFAIK they would be overridden if this is also defined in the project's pom.xml (eg. when                   <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>)

@gastaldi

AFAIK they would be overridden if this is also defined in the project's pom.xml

You mean something like this in a profile?
https://github.com/quarkiverse/quarkiverse-mybatis/blob/master/integration-test/pom.xml#L104
No, AFAICT it is all merged.

@gastaldi

AFAIK they would be overridden if this is also defined in the project's pom.xml

You mean something like this in a profile?
https://github.com/quarkiverse/quarkiverse-mybatis/blob/master/integration-test/pom.xml#L104
No, AFAICT it is all merged.

Why this content isn't in the project's pom.xml?

These properties are needed for Quarkus tests.

AFAIK they would be overridden if this is also defined in the project's pom.xml (eg. when  <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path> is set for the integration tests)

I don't think so.

Why this content isn't in the project's pom.xml?

These properties are needed for Quarkus tests.

AFAIK they would be overridden if this is also defined in the project's pom.xml (eg. when  <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path> is set for the integration tests)

I don't think so.

You merge quicker than I can review. 😉
I was about to nitpick about groupId being set for surefire but not for failsafe, but as long as it works...

You merge quicker than I can review. 😉
I was about to nitpick about groupId being set for surefire but not for failsafe, but as long as it works...

You merge quicker than I can review. 😉
I was about to nitpick about groupId being set for surefire but not for failsafe, but as long as it works...

aw snap, I will provide another PR, I like it clean :)

You merge quicker than I can review. 😉
I was about to nitpick about groupId being set for surefire but not for failsafe, but as long as it works...

aw snap, I will provide another PR, I like it clean :)

@famod https://github.com/quarkiverse/quarkiverse-parent/pull/14

@famod https://github.com/quarkiverse/quarkiverse-parent/pull/14


Reduce the size of Box.module.css

The CSS for the Box component was previously ~2,800 lines of code. This PR reduces it to 135 lines.
The result of this is a 22% reduction in the size of our main CSS bundle, and an 18% reduction in the size of our main JavaScript bundle.

Size Before
Size After
Size Change
% Change

main.css
629 KB
497 KB
-132 KB
-22%

index.js
429 KB
351 KB
-78 KB
-18%

total
1058 KB
848 KB
-210 KB
-20%

There is a trade-off in that the rendered HTML markup for Box may now be slightly longer due to the addition of inline CSS variables when compared to the best-case scenario from the previous implementation. When compared to the worst-case scenario, however, it's about half of the size.

<div class="Box-module__Box-narrow-paddingBlockStart--8__lH0Nx Box-module__Box-regular-paddingBlockStart--12__gOq9H Box-module__Box-wide-paddingBlockStart--16__f5Dfj Box-module__Box-narrow-paddingInlineEnd--8__Rt9bb Box-module__Box-regular-paddingInlineEnd--12__X02EG Box-module__Box-wide-paddingInlineEnd--16__K4CHM Box-module__Box-narrow-paddingBlockEnd--8__y1kaa Box-module__Box-regular-paddingBlockEnd--12__uroYu Box-module__Box-wide-paddingBlockEnd--16__yvgLa Box-module__Box-narrow-paddingInlineStart--8__WQHFH Box-module__Box-regular-paddingInlineStart--12__MFP_O Box-module__Box-wide-paddingInlineStart--16__vDqYc Box-module__Box-narrow-marginBlockStart--8__i685k Box-module__Box-regular-marginBlockStart--12__HLits Box-module__Box-wide-marginBlockStart--16__R8o04 Box-module__Box-narrow-marginInlineEnd--8__jhqqu Box-module__Box-regular-marginInlineEnd--12__VJwpo Box-module__Box-wide-marginInlineEnd--16__JaWXe Box-module__Box-narrow-marginBlockEnd--8__kUGhn Box-module__Box-regular-marginBlockEnd--12__kpsJs Box-module__Box-wide-marginBlockEnd--16__LqPgv Box-module__Box-narrow-marginInlineStart--8__Md4jm Box-module__Box-regular-marginInlineStart--12__LVaCT Box-module__Box-wide-marginInlineStart--16__mUIjr">

<div class="Box-module__Box__iXAWK" style="--box-npbs: var(--base-size-80); --box-rpbs: var(--base-size-80); --box-wpbs: var(--base-size-80); --box-npbe: var(--base-size-80); --box-rpbe: var(--base-size-80); --box-wpbe: var(--base-size-80); --box-npis: var(--base-size-80); --box-rpis: var(--base-size-80); --box-wpis: var(--base-size-80); --box-npie: var(--base-size-80); --box-rpie: var(--base-size-80); --box-wpie: var(--base-size-80); --box-nmbs: 0; --box-rmbs: 0; --box-wmbs: 0; --box-nmbe: 0; --box-rmbe: 0; --box-wmbe: 0; --box-nmis: 0; --box-rmis: 0; --box-wmis: 0; --box-nmie: 0; --box-rmie: 0; --box-wmie: 0;">

Short variable names have been chosen to reduce the impact of this. For example, --box-npie refers to a Box component on a Narrow viewport and sets the Padding-Inline-End property.
Backwards compatibility
I'm pretty confident that the underlying logic hasn't changed and the VRT snapshots suggest the same. However please do let me know if you think there's something I've missed.
Tests
The spacing variables have quite a few tests associated with them — 1008 to be exact. The test suite takes advantage of Jest's .each() function to test all of the variables, for all of the spacing props, against all of the available options (none, condensed, 4, 8, etc). It may seem like overkill, but the benefit of this approach is that it gives really clear messages when something breaks.

For example, if a test fails you will see:
when marginBlockStart > equals 8 > --box-rmbs is not being set to var(--base-size-8)

You can see the test in question here
The benefit of this approach over the established pattern of running a for/of loop inside of the test is that each assertion is part of its own test, which isn't the case when using a for/of loop. When an assertion fails you'll see exactly which test fails, as shown above. If we were to use a for/of loop, when an assertion fails the whole test will fail, and we won't get the same level of detail in the test output.
Test performance
Running tests is pretty fast. Running all 1008 tests takes <3.5 seconds on my machine. Looking at recent runs of the CI / Install and run tests Action it looks like the impact of these additional tests is pretty negligible.
The number of tests could be reduced by reducing the number of spacing props we're testing against. Instead of testing against all of our valid base scale values (4, 8, 12, 16, etc) we could just pick one number to test along with our string spacings ('none', 'condensed', 'normal', 'spacious'). This would bring the number of tests down to 240, however it would only reduce the time taken to run the tests by ~10%.
What should reviewers focus on?

Happy with the implementation?
Happy with the tests?
Can you find any visual regressions, or reasons that the underlying logic might have changed?

Contributor checklist:

[x] All new and existing CI checks pass
[x] Tests prove that the feature works and covers both happy and unhappy paths
[x] Any drop in coverage, breaking changes or regressions have been documented above
[x] New visual snapshots have been generated / updated for any UI changes
[x] All developer debugging and non-functional logging has been removed
[x] Related issues have been referenced in the PR description

Reviewer checklist:

[ ] Check that pull request and proposed changes adhere to our contribution guidelines and code of conduct
[ ] Check that tests prove the feature works and covers both happy and unhappy paths
[ ] Check that there aren't other open Pull Requests for the same update/change

Thanks for the review @rezrah, appreciate you taking the time to dig into it.
One thing I forgot to note in the PR description is that there's already precedent for a similar pattern within the Section component.
https://github.com/primer/brand/blob/main/packages/react/src/Section/Section.tsx#L123-L133
https://github.com/primer/brand/blob/main/packages/react/src/Section/Section.module.css#L8-L15

It's a bit less pronounced here as we (hopefully 😅) don't have to worry about consumers nesting Sections within Sections, and so we don't have to reset unused values as I do with Box. We're also likely to only have a few Section components on a page — compared to a lot of Box components — so there's less pressure to reduce the size of the CSS variable names.

@rezrah Just to note, I dug into this a bit more and managed to find a better solution to resetting the CSS variables, and it turns out we don't need @property at all as I previously thought 🎉
Take a look at 6f6574de8b7b7d7ba38daf093bfa965a6253af6a, it's actually more straightforward than I initially thought.
This change removes the need for all of the --box-npie: 0; --box-rpie: 0; resets and reduces the size of the markup


Add GetUser, GetUsers and UpdateUser endpoints

Got one more for you to review 😅

@NikiforovAll got one more for you to review 😅

Sure, happy to see it

Well done!


uORB::Subscription subscribe directly to uORB device node object

Continuation of https://github.com/PX4/Firmware/pull/11176.
This is a minimal version of https://github.com/PX4/Firmware/pull/11176 with the core uORB changes. The larger changes to mavlink, logger, flighttasks, etc can follow incrementally.
Changes

existing uORB::Subscription moved to uORB::SubscriptionPolled

uORB::SubscriptionBase -> uORB::SubscriptionBasePolled
uORB::SubscriptionNode -> uORB::SubscriptionPolledNode
uORB::Subscription -> uORB::SubscriptionPolled

The new uORB::Subscription subscribes directly to the underlying uORB::DeviceNode, bypasses file descriptors. It's faster, uses less memory, and gets us past NuttX file descriptor limits.
Longer term uORB::SubscriptionPolled will be phased out entirely.

tested on pixhawk 4 v5
indoor test
Modes Tested
Takeoff: Good
Stabilized Mode: Good.

Pre-flight Procedure:
Flight Test:
Arm and Take off in the stabilized mode ✓
Throttle Test ✓
Pitch, Roll, Yaw ✓
Land ✓
Notes:
Took off in stabilized mode for approximately 5 min, No issues noted good flight in general. The vehicle behaved as expected.
Flight log: 1
https://review.px4.io/plot_app?log=3d557ab4-1ca7-4bee-a095-8f72371533cf
Flight log: 2
https://review.px4.io/plot_app?log=637bfdf6-7aa9-4ef5-bf97-f9f1ffca0d52
Indoor flight test on Pixhawk 2 Cube V3
Modes Tested
Takeoff: Good
Stabilized Mode: Good.
Pre-flight Procedure:
Flight Test:
Arm and Take off in the stabilized mode ✓
Throttle Test ✓
Pitch, Roll, Yaw ✓
Land ✓
Notes:
Took off in stabilized mode for approximately 5 min, No issues noted good flight in general. The vehicle behaved as expected.
Flight log: 1
https://review.px4.io/plot_app?log=25566deb-76b0-460b-b6b1-5b6109f7d975
Flight log : 2
https://review.px4.io/plot_app?log=19995186-fdae-4faa-b52f-7bc7ab64cffb

The main motivation for this change is saving memory (see https://github.com/PX4/Firmware/pull/11176#issue-243209888 for details), but the performance improvements in uORB are also going to be worthwhile overall.
Here's a comparison of the common check and copy operations.
C API (orb_check(), orb_copy(), etc)
orb_check vehicle_local_position: 1000 events, 2746us elapsed, 2.75us avg, min 2us max 4us 0.438us rms
orb_stat vehicle_local_position: 1000 events, 2319us elapsed, 2.32us avg, min 2us max 3us 0.466us rms
orb_copy vehicle_local_position: 1000 events, 3928us elapsed, 3.93us avg, min 3us max 5us 0.324us rms

C++ API
uORB::Subscription orb_check vehicle_local_position: 1000 events, 907us elapsed, 0.91us avg, min 1us max 2us 0.301us rms
uORB::Subscription orb_stat vehicle_local_position: 1000 events, 1092us elapsed, 1.09us avg, min 1us max 2us 0.289us rms
uORB::Subscription orb_copy vehicle_local_position: 1000 events, 2530us elapsed, 2.53us avg, min 2us max 3us 0.499us rms

On a typical setup these operations are called tens of thousands of times every second, so shaving off microseconds from each orb check and copy will actually be a considerable cpu usage improvement (in aggregate).

Awesome! Just for the sake of curiosity: Coukd you share the output of top for one representative setup?

Awesome! Just for the sake of curiosity: Coukd you share the output of top for one representative setup?

We'll need to move the high rate modules and logger to uORB::Subscription to see most of the benefit.
Here's the current comparison below where the main difference is a small decrease in cpu usage for each Mavlink instance (notice the FDs per task). This isn't a direct comparison though, because for safety I also restored the original MavlinkOrbSubscription semantics where the data is always copied even if there's no timestamp update (the PR is doing more orb copy work yet still faster).
https://github.com/PX4/Firmware/blob/88127380e7b61df030a270765d6570f80d01afaa/src/modules/mavlink/mavlink_orb_subscription.h#L61
Memory-wise the huge win will be after logger and a few other heavy subscribers are updated. That's when we'll be able to drop the NuttX file descriptors significantly and overall have an extra 15-20 kB of memory to play with.
Master
 PID COMMAND                   CPU(ms) CPU(%)  USED/STACK PRIO(BASE) STATE FD
   0 Idle Task                   20693 16.990   284/  748   0 (  0)  READY  3
   1 hpwork                        328  0.404   800/ 1780 249 (249)  READY 12
   2 lpwork                        104  0.161   648/ 1780  50 ( 50)  w:sig  8
   3 init                         1129  0.000  2040/ 2604 100 (100)  w:sem  3
   4 wq:manager                      2  0.000   760/ 1172 244 (244)  w:sem  7
1561 top                           481  3.883  1256/ 1684 255 (255)  RUN    3
  17 dataman                        84  0.000   752/ 1180  90 ( 90)  w:sem  4
  21 wq:I2C1                       708  0.728   880/ 1196 248 (248)  READY  7
  24 wq:lp_default                 100  0.161   496/ 1196 205 (205)  w:sem  7
  27 wq:hp_default                  19  0.000   432/ 1196 244 (244)  w:sem  7
 138 wq:SPI1                     25778 26.213   696/ 1196 254 (254)  READY  7
 143 wq:I2C2                         0  0.000   360/ 1196 247 (247)  w:sem  7
 148 wq:I2C3                       492  0.485   704/ 1196 246 (246)  READY  7
 151 wq:SPI5                         0  0.000   360/ 1196 250 (250)  w:sem  7
 165 wq:SPI4                       405  0.404   528/ 1196 251 (251)  READY  7
 182 sensors                      3767  3.964  1352/ 1964 238 (238)  w:sem 17
 186 commander                     844  0.809  1480/ 3212 140 (140)  READY 29
 188 commander_low_prio              2  0.000   576/ 2996  50 ( 50)  w:sem 29
 200 mavlink_if0                  6749  6.877  1760/ 2532 100 (100)  READY 34
 201 mavlink_rcv_if0               512  0.404  1544/ 2836 175 (175)  w:sem 34
 307 gps                           250  0.242   992/ 1516 209 (209)  w:sem  5
 363 mavlink_if1                  5137  5.339  1768/ 2484 100 (100)  READY 36
 364 mavlink_rcv_if1               393  0.404  1176/ 2836 175 (175)  w:sem 36
 382 mavlink_if2                  5273  5.339  1632/ 2492 100 (100)  READY 36
 383 mavlink_rcv_if2               460  0.485  1120/ 2836 175 (175)  w:sem 36
 422 px4io                        3674  3.802   976/ 1484 241 (241)  w:sem 11
 434 fmu                           520  0.566   760/ 1324 241 (241)  w:sem  7
 848 ekf2                        11185 11.974  4576/ 6572 239 (239)  READY 20
 861 mc_att_control               4707  5.177   928/ 1660 240 (240)  w:sem 18
 877 mc_pos_control                365  0.404   904/ 1860 239 (239)  w:sem 10
 910 navigator                     116  0.080   896/ 1764 105 (105)  w:sem 16

Processes: 31 total, 12 running, 19 sleeping, max FDs: 54
CPU usage: 78.32% tasks, 4.69% sched, 16.99% idle
DMA Memory: 5120 total, 1024 used 1024 peak
Uptime: 100.517s total, 20.693s idle

PR
 PID COMMAND                   CPU(ms) CPU(%)  USED/STACK PRIO(BASE) STATE FD
   0 Idle Task                   26888 19.174   284/  748   0 (  0)  READY  3
   1 hpwork                        403  0.323   768/ 1780 249 (249)  w:sig 12
   2 lpwork                        126  0.161   656/ 1780  50 ( 50)  w:sig  8
   3 init                         1120  0.000  2048/ 2604 100 (100)  w:sem  3
   4 wq:manager                      2  0.000   792/ 1172 244 (244)  w:sem  7
1319 top                          2960  3.883  1264/ 1684 255 (255)  RUN    3
  17 dataman                        84  0.000   752/ 1180  90 ( 90)  w:sem  4
  21 wq:I2C1                       888  0.728   872/ 1196 248 (248)  READY  7
  24 wq:lp_default                 126  0.161   496/ 1196 205 (205)  w:sem  7
  27 wq:hp_default                  25  0.000   432/ 1196 244 (244)  w:sem  7
 138 wq:SPI1                     31942 26.375   696/ 1196 254 (254)  READY  7
 143 wq:I2C2                         0  0.000   360/ 1196 247 (247)  w:sem  7
 148 wq:I2C3                       619  0.485   704/ 1196 246 (246)  READY  7
 151 wq:SPI5                         0  0.000   360/ 1196 250 (250)  w:sem  7
 165 wq:SPI4                       510  0.404   528/ 1196 251 (251)  READY  7
 182 sensors                      4593  3.883  1320/ 1964 238 (238)  w:sem 17
 186 commander                    1008  0.809  1472/ 3212 140 (140)  READY 21
 188 commander_low_prio              3  0.000   656/ 2996  50 ( 50)  READY 21
 200 mavlink_if0                  5782  6.229  1776/ 2532 100 (100)  READY  4
 201 mavlink_rcv_if0               545  0.404  1528/ 2836 175 (175)  w:sem  4
 307 gps                           293  0.242   992/ 1516 209 (209)  w:sem  5
 363 mavlink_if1                  6100  4.611  1648/ 2484 100 (100)  READY  4
 364 mavlink_rcv_if1               532  0.404  1424/ 2836 175 (175)  w:sem  4
 382 mavlink_if2                  6303  4.773  1648/ 2492 100 (100)  w:sig  4
 383 mavlink_rcv_if2               592  0.485  1424/ 2836 175 (175)  w:sem  4
 422 px4io                        4319  3.721   936/ 1484 241 (241)  w:sem 11
 434 fmu                           677  0.566   760/ 1324 241 (241)  w:sem  7
 848 ekf2                        14018 11.893  4576/ 6572 239 (239)  w:sem 20
 861 mc_att_control               6119  5.339   928/ 1660 240 (240)  READY 18
 877 mc_pos_control                461  0.404   936/ 1860 239 (239)  w:sem 10
 910 navigator                     188  0.080   896/ 1764 105 (105)  w:sem 14

Processes: 31 total, 11 running, 20 sleeping, max FDs: 54
CPU usage: 76.38% tasks, 4.45% sched, 19.17% idle
DMA Memory: 5120 total, 1024 used 1024 peak
Uptime: 124.074s total, 26.889s idle

The class API docs don't make it clear when to use which class. Can you clarify this? Especially that Subscription should generally be used over the Polled classes.
Can you rename SubscriptionPolled to SubscriptionPollable? I find Polled confusing, since there is no poll involved.

The naming is better, but I'd actually like to kill this off entirely as soon as possible (likely once FlightTasks is migrated). Then it's only LPE using it for polling, which we can either work around with the C API, or add a blocking update method to the new uORB::Subscription.

Did you check the need for virtual for the Subscription classes? If not required, we should remove that overhead early on.

uORB::SubscriptionInterval overrides update, but let's revisit after logger has been migrated and see what we really need. I'd still like to get to a template uORB::Subscription (without the internal data).

I believe I've responded to all comments. @julianoes @bkueng can you give it another pass and test on your side? @PX4/testflights please test this one again.

Tested on Pixhawk 4 and 4 mini v5
Notes:
Loaded PR and plugged in battery to the vehicle, vehicle kept rebooting in a loop. This issue was found only on v5 Firmware.
Tested on Pixhawk Pro v4
Flight Card 1
Modes Tested:

Position Mode: Good.
Altitude Mode: Good.
Stabilized Mode: Good.

Procedure:
Arm in position mode, check take off behavior (Vehicle should stay on the ground as long as the throttle stick is at the middle or below, and start to ascend smoothly as long as throttle stick as above the middle.), the land and disarm.
Same procedure was repeated with Altitude and Stabilized mode.
Notes:
No issues were noted, good flight in general.
Flight Card 2
Modes Tested

Mission Plan Mode (Automated): Good.
RTL (Return To Land): Good.

Procedure
Arm and Take off in mission plan mode then make sure that vehicle follows all waypoints as shown in QGC, once completed all waypoint see landing behavior.
Note:
No issues were noted, good flight in general.
Flight Card 3
Modes Tested

Position Mode: Good.
Mission Plan Mode (Automated): Good.
RTL (Return To Land): Good.

Procedure
Arm and Take off in position mode, after flying for approximately one minute, switched to mission plan mode then make sure that vehicle follows all waypoints as shown in QGC, once completed all waypoint see landing behavior.
Note:
No issues were noted, good flight in general.
Flight Card 4
Modes Tested

Position Mode: Good.
Failsafe RTL (Return To Land): Not returning after RC shutdown.

Note:
No issues were noted, good flight in general.
Logs:

Flight Card 1: https://review.px4.io/plot_app?log=05cf5291-a7c8-46f3-a087-bc95db5009ba
Flight Card 2: https://review.px4.io/plot_app?log=4dbdde54-48bd-4d71-b19e-61fddfbe7cd1
Flight Card 3: https://review.px4.io/plot_app?log=0e465c96-a9bb-4502-83c6-9c20dfaafb35
Flight Card 4: https://review.px4.io/plot_app?log=20f90564-33eb-4262-9289-6d20dd3cef05

Tested on Pixhawk 2 Cube V3
Flight Card 1
Modes Tested:
Position Mode: Good.
Altitude Mode: Good.
Stabilized Mode: Good.
Procedure:
Arm in position mode, check take off behavior (Vehicle should stay on the ground as long as the throttle stick is at the middle or below, and start to ascend smoothly as long as throttle stick as above the middle.), the land and disarm.
Same procedure was repeated with Altitude and Stabilized mode.
Notes:
No issues were noted, good flight in general.
Flight Card 2
Modes Tested
Mission Plan Mode (Automated): Good.
RTL (Return To Land): Good.
Procedure
Arm and Take off in mission plan mode then make sure that vehicle follows all waypoints as shown in QGC, once completed all waypoint see landing behavior.
Note:
No issues were noted, good flight in general.
Flight Card 3
Modes Tested
Position Mode: Good.
Mission Plan Mode (Automated): Good.
RTL (Return To Land): Good.
Procedure
Arm and Take off in position mode, after flying for approximately one minute, switched to mission plan mode then make sure that vehicle follows all waypoints as shown in QGC, once completed all waypoint see landing behavior.
Note:
No issues were noted, good flight in general.
Flight Card 4
Modes Tested
Position Mode: Good.
Failsafe RTL (Return To Land): Not returning after RC shutdown.

Note:
No issues were noted, good flight in general.
Logs:

Flight card 1: https://review.px4.io/plot_app?log=87614b7d-3dde-45ee-918d-c0394e2637bf

Flight card 2: https://review.px4.io/plot_app?log=8b6f7eaf-8aa7-4f1f-9523-21aa545376f4

Flight card 3: https://review.px4.io/plot_app?log=6d47cac3-e254-48a7-a1d8-aa623c25c79d

Thanks @Junkim3DR @jorge789

Notes:
Loaded PR and plugged in battery to the vehicle, vehicle kept rebooting in a loop. This issue was found only on v5 Firmware.

Can you share this exact setup? Is it fine with current master? My pixhawk 4 on this PR seems to be working.

@dagar after the pr we upload master and it worked without issues

@dagar after the pr we upload master and it worked without issues

Can you share the details of that setup and the parameters? I'm unable to reproduce locally on a Pixhawk 4 or 4 mini.

In the last refactor I consolidated the copy logic to be shared by read(), copy(), and copy_and_get_timestamp(). The common code was missing a null check here (data isn't allocated until the first publication).
https://github.com/PX4/Firmware/pull/12040/files#diff-6933463aec16725fb6cb34d9b949df62R169
@dannyfpv please try again.

Tested on Pixhawk 4 v5 (Update)
Modes Tested:

Position Mode: no issue
Altitude Mode: no issue
Stabilized Mode: no issue
Mission mode: no issue

Notes:
No issues were noted, good flight in general.
Logs:

https://review.px4.io/plot_app?log=16115d93-31e0-47de-a0de-012414627bb0
https://review.px4.io/plot_app?log=7138dc8d-f164-4e5b-b2ec-5afd344c3fc1

As discussed on the dev call this week, let's give this another round of thorough testing, including a setup with heavy mavlink usage and logging over mavlink. The majority of the new uORB::Subscription usage is in the mavlink module.

Rebased on master to resolve minor FlightTasks merge conflict.

Commit before this PR:

This PR:

Something strange I noticed, this firmware is missing the MPC_COL_PREV_D in QGC. Tested on 2 different ground stations, via wifi and direct USB. Maybe it is missing some major merged components from ~October last year? If those are missing it will be affecting my tests.

Ok, there was a bug with the system, this is updated info
Commit before this PR:

This PR:

Mavlink status during testing:

Testing looks good, let's get this into master and keep going. There's a lot more to be done.
Thanks everyone!


ModuleNotFoundError: No module named 'point_e'

After installing requirement.txt, I encountered this "ModuleNotFoundError: No module named 'point_e'". Where can I get this package? Thank you for your help.

hi，
you can remove that line and install the pointe manually follow the instruction below:https://github.com/openai/point-e
---- Replied Message ----
| From | @.> |
| Date | 12/13/2023 03:05 |
| To | @.> |
| Cc | @.***> |
| Subject | [EnVision-Research/LucidDreamer] ModuleNotFoundError: No module named 'point_e' (Issue #20) |
After installing requirement.txt, I encountered this "ModuleNotFoundError: No module named 'point_e'". Where can I get this package? Thank you for your help.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>


Staff customization for MIDI guitar

Hi, Rémi, congratulations for your great work! Everything is working fine here, WIndows 10. I teach both keyboard and guitar (online lessons and video recordings using OBS), and your software is a great solution for displaying the notes on the staff in real-time. I use Roland-A88 keyboard controller and Roland GI-20 as guitar MIDI interface. The problem is that guitar notation uses a single staff with treble clef, and the notes sound one octave lower than written. Can this feature be implemented, so you can switch between the keyboard grand staff and guitar treble clef single staff? It would be nice if the lower ledger lines could go as low as low D, for when you have to drop the low E 6th string down to D. Also, guitar notation demands a considerable amount of upper ledger lines. Thank you very much!

Hello @lgmusic
Yes, this is a great idea to be able to customize the staff, being a Piano or another clef (bass or treble, or maybe something else).
Could you provide me some resources about Guitar notation (or even other instruments) so that i can learn more about the specifications ?
Even hand drawn figures, or some paint mockups could be helpful.
Concerning the upper ledger lines, if i understand right, on guitar, all notes are shifted up in notation ? Like, a D6 on guitar (the highest note on a 6 string neck i suppose) would be noted like a D7 on piano ?

I think the whole range on piano is able to be rendered on the grand staff, so i don't see the issue with upper lines.
Would it be very annoying to not being able to switch between the staff configurations only in the settings page (like it is currently with the key signature ) ?
I will plan this feature for the next minor version, with a Circle of Fifth to change key more easily in the chord display module.
Thanks !

Thank you very much for your very quick reply, Rémi! Yes, both guitar and bass are octave transposing instruments. If you play on a piano any note written on a guitar or bass score it sounds one octave higher (because it uses a single staff, in order to avoid a ridiculous amount of lower ledger lines):

No, it doesn't need to have a switch, as long as you could select the configuration in the beginning of a section (lesson, recording, etc...)
Of course I'm very interested in collaborating with all input I can possibly offer regarding notation matters. Even through some chat or private messages, if needed. I have also some considerations regarding chord symbol notations. In my case, the ability to customize them is a must, according to the specific situation. I found an "abandoned" software which allows you to customize most symbols by editing a single txt file:

But I guess this could be the subject for another request.
Thank you so much!

@lgmusic OK, those are the perfect figures for me to correctly understand your needs.
I'll work on this asap.
--
Concerning the chords notation, this is mostly choices from https://github.com/tonaljs/tonal library that MIDI Jar depends on.
I tweak it a bit, for displaying purpose and fixing some incoherences (like some chords being formatted as 'M6' while others where 'maj7'...).
Its dictionnary is kinda incomplete i think, i personally noticed a missing maj11 chord, and a lot of chords were not named. My knowledge of music is too limited to suggest changes atm (this is why i developed MIDI Jar, to learn).
You can find the dictionary here: https://github.com/tonaljs/tonal/blob/main/packages/chord-type/data.ts
As you can see, some chords have aliases, but there is no consistency in the order of the alias, so i cannot for instance choose to display all chords with +/-/Δ/°.
But ideally i would like to allow choosing its preferred notation (displaying either maj9 or Δ9).
I'm thinking about improving tonal, but i'm focusing on new features in MIDI Jar first.

ArTiSTiX, thanks again.
Chord naming is a complex and subjective matter. it involves a lot of inconsistent and sometimes discordant theories, and each chord in each composition should be a particular case, subject to many considerations and variables. Many chords are ambivalent in the context of the tonality. That's one of those things that differentiates art from science. Also, different countries name them differently. My country, Brazil, for instance, has a peculiar way to write chord symbols, which I like, but I ended up adopting the North-American way to make it more accessible to different people, and also not to struggle with notation software. But even in US there are many variations (maj7, Ma7, Δ, as you exemplified) and sometimes lots of misconceptions. I think that available programs still have a long way to develop in that sense (Chordie and Midiculous, for instance), and should maybe offer more customization. Sometimes all those aliases make it too confusing. Sometimes it's nice to have them, though. Being able to select when to show aliases, or sometimes not even showing certain chords at all would be a cool feature. If you are dealing with more basic harmony, for instance, you wouldn't want all those complex symbols appearing and confusing a student. Besides that there's the issue of enharmonics: both programs I mentioned display an E Major chord 1st inversion as E/Ab, so they are not yet able to tell that a Major Third in the bass should always be spelled as a Major Third, not as a Diminished Fourth. In short, I'm just saying that there's still a lot to develop in this area, and it would be a great improvement in music learning tools.
I really appreciate your efforts and availability, Rémi. Keep up your great work! (Nice turntable and LP collection in your virtual music room, btw ;D)

Hello @lgmusic
Sorry i took my time to release this feature.
You can now change staff transposition and choose the clef.
I didn't want to do it specifically for guitar, so you can transpose by any semitones.
Note: only notes that can be transposed to a midi note will be written, it was easier for me to implement it that way. So if you play a C2 and transpose -4 octaves, you won't get any notes.
Also, i tried to do my best to position the staves so that the whole range of a piano can still be rendered, but this is prone to some overflowing. If this is an issue, don't hesitate to create an issue specifically for usecases where it happens, i would like to not create a bigger rendering surface for the staff notation, it will make the layout too complex to handle.


STYLE: Use the itkNameOfTestExecutableMacro macro in tests

Use the itkNameOfTestExecutableMacro macro to get the test name and
prevent the test from crashing when argv[0] is null.
PR Checklist

[X] No API changes were made (or the changes have been approved)
[X] No major design changes were made (or the changes have been approved)

A few notes:

Would probably be best to rename all av (and ac) to argv and argc for the sake of consistency.
I noticed that in these files failure return codes are many times set to -1.
Did not improve neither the above nor the style of the missing parameter message.

These may be best addressed in separate PRs as time permits.

/azp ITK.macOS.Python

As in #2940 itkPyBufferMemoryLeakTest is failing. Looks like a recently introduced change is causing it.

Failing test is unrelated to this. itkPyBufferMemoryLeakTest


py: Further refactoring of docker client

Signed-off-by: Ryan Northey<EMAIL_ADDRESS>

Codecov Report

Merging #130 (0d655c1) into main (1c887e7) will increase coverage by 2.10%.
The diff coverage is 40.00%.

@@            Coverage Diff             @@
##             main     #130      +/-   ##
==========================================
+ Coverage   32.75%   34.86%   +2.10%
==========================================
  Files          73       79       +6
  Lines        2009     2074      +65
==========================================
+ Hits          658      723      +65
  Misses       1351     1351

Impacted Files
Coverage Δ

playground/control/attribs/request/network.py
42.55% <ø> (ø)

playground/control/connectors/docker/networks.py
28.00% <0.00%> (ø)

playground/control/connectors/docker/proxies.py
33.33% <0.00%> (ø)

playground/control/connectors/docker/services.py
32.65% <0.00%> (ø)

playground/control/connectors/docker/images.py
35.71% <35.71%> (ø)

playground/control/connectors/docker/volumes.py
40.62% <40.62%> (ø)

playground/control/connectors/docker/client.py
44.61% <100.00%> (+9.92%)
:arrow_up:

tests/py/request.py
100.00% <0.00%> (ø)

tests/py/run.py
100.00% <0.00%> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c887e7...0d655c1. Read the comment docs.


iOS13 not work. in the iPad

WVJB Bug Report
Thanks for reporting an issue with WebViewJavascriptBridge.
Do these 4 things and I will fix your problem!

Go to https://github.com/marcuswestin/WebViewJavascriptBridge and click Fork.
Clone your fork, cd into it and run make test. All tests should pass!
Edit Tests/WebViewJavascriptBridgeTests/BridgeTests.m and create a new, failing test which demostrates your issue.
Create a pull request for https://github.com/marcuswestin/WebViewJavascriptBridge

That's it!
I will take it from there and promise that I'll fix your problem ASAP.
If you don't do this then I can't help you!
And I probably won't :)
Cheers,
@marcuswestin

Occurs the same problem, anybody figure out how to solve it?
ios13 in iphone is no problem, but ipad not work

Can you provide the iPad model and version you tested?

我遇见过这个问题，问题发生在iPadOs13.1.2的ipad上。
因为前端js代码注入，是根据userAgent判断设备为iphone, ipad, Android，然后注入不同的代码。但是iPadOS13.1.2的设备获取到的userAgent没有ipad的特性，导致注入失败或者注入了Android的那部分。
不知道能不能帮到你

Any update here? Looking for a solution

[(WKWebView *)_webView setCustomUserAgent:customUserAgent];

iOS 13新增了WKWebpagePreferences这个api，在iPadOS上默认会使用WKContentModeDesktop，这就导致自定义的UserAgent会失效，变成mac端的UserAgent，只需要将WKWebpagePreferences设置为WKContentModeMobile，UserAgent就这很正常了；
（https://www.jianshu.com/p/ae4755a2069b）

iOS13 has added the API of WKWebpagePreferences. WKContentModeDesktop will be used by default on iPadOS. You just need to set WKWebpagePreferences to WKContentModeMobile.And then the UserAgent is perfectly normal.
（https://www.jianshu.com/p/ae4755a2069b）

        let configuration = WKWebViewConfiguration()
        configuration.preferences.javaScriptEnabled = true
        if #available(iOS 13.0, *) {
            configuration.defaultWebpagePreferences.preferredContentMode = .mobile
        }

        let w = WKWebView(frame: .zero, configuration: configuration)


Change input parameters to do federal register calls and pull document date from xml file

This may be failing the build because we removed the API base in order to run the script without the regulations-core, while the vagrant bootstrap was being developed.
settings.py >> line 2.

I think this is a really good idea. @khandelwal @micahsaul: mind giving a second set of eyes?

I can take a look at this. Is there a description of the problem it's trying to solve somewhere?

ok, other than a couple minor comments - I think this is good.

Going to merge this and will update docs in a later PR.


Feat/UI improvement

A WIP PR for #892

Summary

added video column in dashboard with some UI improvements overall

Checklist

[x] My code follows the style guidelines of OpenAdapt
[x] I have performed a self-review of my code
[ ] If applicable, I have added tests to prove my fix is functional/effective
[x] I have linted my code locally prior to submission
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation (e.g. README.md, requirements.txt)
[ ] New and existing unit tests pass locally with my changes

How can your code be run and tested?

 python -m openadapt.app.dashboard.run

Thank you @Animesh404 ! Can you please upload a short video demonstrating the screenshot animation/playback functionality we discussed?

https://github.com/user-attachments/assets/1380c9a9-2c4a-4657-9e83-58930ee875b0


Bug when we abruptly close the back layer

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Create a Backdrop Scaffold with app bar and all
in back layer add a couple of list tile , one of them just opens up a webview or redirects you to playstore ,or something like that,
In front layer add a list view with elements such that occupy atleast double the screen
open backlayer through Icon Button in actions, instead of closing the backlayer by tapping that icon button close it by pressing back key , the listview wouldnt scroll .
also reproduceble when you open one of the listtile in back layer and then come back to app

Expected behavior
List View should be working
Screenshots
If applicable, add screenshots to help explain your problem.
Smartphone (please complete the following information):

Device: Pixel 3 ,redmi note 7 pro
-OS android

@amanv8060 could you share your code or a simplified version of it, please?

PlatformException(error, No Activity found to handle Intent { act=android.intent.action.VIEW dat=market://details?id=com.ophilia.allcppprograms (has extras) }, null) i think the issue is due to url launcher plugin , Due to anonymous closure , it is happening i guess , I think we could close this here


The authorization code has already been received

Unhandled Exception: Bad state: The authorization code has already been received
How to avoid this issue

Any solution on this??


fix safeAreaInsets for iOS 11

Now you also should take into account safeAreaInsets

@romansorochak can you merge this?


Stop referring to the alloc crate as unstable

The alloc crate was stabilized in 1.36.0 https://blog.rust-lang.org/2019/07/04/Rust-1.36.0.html#whats-in-1360-stable

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jamesmunns (or someone else) soon.
Please see the contribution instructions for more information.


email/numeric and maxLenght doesn't work

['required', 'email', 'maxLength:199']
['required', 'maxLength:199', 'numeric']
If i put email/numeric and maxLength in the rules its not working...
Can we get a fast fix for this?

Use max for numbers, not maxLength.
Max length is for the number of characters.

What about the first one for email?


403

Non-fatal Exception: com.google.firebase.d: An internal error has occurred. [

Error 403 (Forbidden)!!1

403. That’s an error.
Your client does not have permission to get URL /identitytoolkit/v3/relyingparty/sendVerificationCode from this server.  That’s all we know. ]
       at com.google.firebase.auth.api.internal.zzdv.zza(zzdv.java:18)
       at com.google.firebase.auth.api.internal.zzex.zza(zzex.java:3)
       at com.google.firebase.auth.api.internal.zzfa.run(zzfa.java:4)
       at android.os.Handler.handleCallback(Handler.java:883)
       at android.os.Handler.dispatchMessage(Handler.java:100)
       at android.os.Looper.loop(Looper.java:214)
       at android.app.ActivityThread.main(ActivityThread.java:7403)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:935)

Device
Brand: Xiaomi Model: Mi A2 Orientation: Vertical Free RAM: 1.36 GB Free disk space: 2.32 GB operating system Version: 10 Orientation: Vertical Root Access: No Crash Date: Feb 17 2020, 23:50:00
Device
Brand: samsung Model: Galaxy J3 (2017) Orientation: Vertical Free RAM: 401.89 MB Free disk space: 580.48 MB operating system Version: 9 Orientation: Vertical Root Access: No Crash Date: Feb 17 2020, 20:29:00
Device
Brand: samsung Model: Galaxy A50 Orientation: Vertical Free RAM: 2.35 GB Free disk space: 95.8 GB operating system Version: 9 Orientation: Vertical Root Access: No Crash Date: Feb 17 2020, 00:32:00

Thanks for reporting this, @AlexeyGodyaev. Could you share a minimal, reproducible example of your project as well as the steps you've taken before encountering this issue? This way, I can try to reproduce the issue on my end and hopefully find a solution.


FallbackComponent and beforeSend do not work

I'm trying to render error page using FallbackComponent. I created simple application and if you click button the error appears in browser console and Bugsnag UI, but I can not see rendered FallbackComponent and beforeSend does not firing on button click. I just can click the button again and again and see new errors in console (means Bugsnag catch it and re-render tree)

import bugsnag from '@bugsnag/js'
import bugsnagReact from '@bugsnag/plugin-react'
import React from 'react'
import ReactDOM from 'react-dom'

const bugsnagClient = bugsnag({
  apiKey: MY_API_KEY,
  notifyReleaseStages: ['dev'],
  releaseStage: 'dev'
})

bugsnagClient.use(bugsnagReact, React)
const ErrorBoundary = bugsnagClient.getPlugin('react')

const FallbackComponent = () =>
ERROR!!!

ReactDOM.render(
  <ErrorBoundary
    beforeSend={(report) => { console.log({ report })}}
    FallbackComponent={FallbackComponent}
  >

      Test
      <button onClick={() => { throw new Error('Test error') }}>
        throw error
      </button>

  </ErrorBoundary>,
  document.getElementById('root')
)

@bugsnag/js: 6.2.0,
@bugsnag/plugin-react: 6.2.0,
react: 16.6.0

Anything I'm doing wrong? Any ideas why it happening?

Errors in button press callbacks are not caught by error boundaries. Error boundaries only catch errors during the render() of a component.
This isn't an issue with Bugsnag, it's just a nuance of how errors in React work. See the react example in this repo for reference.
If you need any more support getting set up, we'll gladly help over at<EMAIL_ADDRESS>Cheers!

@bengourley how do you explain then that he does see it in the Bugsnag UI even though beforeSend is not called?

Bugsnag reports errors that happen outside of React's error boundaries too, but it would only run the beforeSend callback configured in the error boundary if the error boundary caught it. Hope that makes sense!


Probot-stale seems to not be working

@bkeepers It appears that the stale bot isn't working. GitHub's APP server and API status says it is all good, but when I click the link listed at the top of the repo, I get a 404 Error:
https://github.com/apps/probot-stale
Based on my settings in my stale config file, there are about 4 days of issues that should have been marked as stale, but have not received the stale label or a bot comment. There are also issues open that were marked as stale 11 days ago, but have not been closed or updated in any way (we're using the default setting of 7 for the daysUntilClose option).

Hey @rallytime

when I click the link listed at the top of the repo, I get a 404 Error:
https://github.com/apps/probot-stale

Sorry, I renamed the app this morning to be consistent with the other Probot apps that are being shipped and forgot to update the link in the repo. It's been updated now: https://github.com/apps/stale

Based on my settings in my stale config file, there are about 4 days of issues that should have been marked as stale, but have not received the stale label or a bot comment.

Can you link to the repo that you're having issues with? It looks like it's working on some repositories:

https://github.com/exercism/ruby/issues/666#issuecomment-320968776
https://github.com/exercism/ecmascript/issues/303

@bkeepers Oh, yes. Here's the list of stale issues that have been marked recently as stale, but not closed: https://github.com/saltstack/salt/issues/13681
And here is the stale config: https://github.com/saltstack/salt/blob/develop/.github/stale.yml
We are slowly updating our daysUntilStale value, which is rather large, so we don't close a ton of issues all at once. It appears that the bot ran when that the new value was changed from 1145 to 1130 in https://github.com/saltstack/salt/commit/64ae7e4475552f12722e0e6e9638bc17bb78897a 11 days ago, but looks like it hasn't done much since then.

Ah I just realized I only linked a single issue. Here's the list of issues that were marked as stale, but not closed: https://github.com/saltstack/salt/issues?utf8=✓&q=is%3Aissue is%3Aopen label%3Astale

@rallytime it doesn't look like the stale app is installed on the @saltstack org. Does it show that it is when you visit https://github.com/apps/stale/installations/new?

@bkeepers Yes, it does show up there with Configure before the > for the SaltStack organization. It also shows up in the "Installed GitHub Apps" section of the SaltStack organization settings. We're also using it on another repo in the org: https://github.com/saltstack/salt-bootstrap
The salt-bootstrap repo isn't as active as the salt repo, but the last activity we had there was in this issue: https://github.com/saltstack/salt-bootstrap/issues/880. I don't think there should have been anything else active for the stale bot since then, though.
Here is a screenshot of the org's settings for the Probot: Stale installation:

@rallytime ok, I've figured out the issue (/integration/installations API now redirects to /app/installations and loses pagination information). It's going to require an update to the node-github client, which might take a day or so.
Thanks for reporting the issue, and thanks for your patience.  On the bright side, this app has >100 installations now. Unfortunately, it only works for the 100 most recent.

@bkeepers No problem! I'm glad you were able to discover the issue and that is also fantastic that there are >100 installations. 🎉

@rallytime This is finally resolved. Thanks for your patience while I worked through days of debugging trying to uncover the issue.

Great! Thanks @bkeepers!


Create MCG api and implement initial controller

As part of the PR:

Create MultiCloudGateway api scaffold
Define MultiCloudGateway api
Implement MultiCloudGateway Initial controller
Resolve go deps for noobaa

Signed-off-by: Nitin Goyal<EMAIL_ADDRESS>

Closing this as we are no longer planning to implement separate MCG


Add Intezer SOAR integration

Please ensure your pull request (PR) adheres to the following guidelines:

Please refer to our contributing documentation for any questions on submitting a pull request, link: Contribution Guide

Pull Request Checklist
Please check if your PR fulfills the following requirements:

[ ] Testing of all the changes has been performed (for bug fixes / features)
[ ] The readme.html has been reviewed and added / updated if needed (for bug fixes / features)
[ ] Use the following format for the PR description: <App Name>: <PR Type> - <PR Description>
[ ] Provide release notes as part of the PR submission which describe high level points about the changes for the upcoming GA release.
[ ] Verify all checks are passing.
[ ] Do NOT use the next branch of the forked repo. Create separate feature branch for raising the PR.
[ ] Do NOT submit updates to dependencies unless it fixes an issue.

Pull Request Type
Please check the type of change your PR introduces:

[ ] New App
[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation
[ ] Other (please describe):

Security Considerations (REQUIRED)

[ ] If you are exposing any endpoints using a REST handler,
please document them in the readme.html.
[ ] If this is a new connector or you are adding new actions

[ ] Please document in the readme.html all methods (eg, OAuth) used to authenticate
with the service that the connector is integrating with.
[ ] If any actions are unable to run on SOAR Cloud, please document this in the readme.html.

[ ] Are you introducing any new cryptography modules? If yes, please elaborate their purpose:
[ ] Are you are accessing the file system? If yes, please verify that you are only accessing paths returned through
the Vault API.
[ ] Are you are marking code to be ignored by Semgrep with nosemgrep?
If yes, please provide justification in an additional comment next to the ignored code.

Release Notes (REQUIRED)

Provide release notes as part of the PR submission which describe high level points about the changes for the upcoming GA release.

What is the current behavior? (OPTIONAL)

Describe the current behavior that you are modifying.

What is the new behavior? (OPTIONAL)

Describe the behavior or changes that are being added by this PR.

Other information (OPTIONAL)

Any other information that is important to this PR such as screenshots of how the component looks before and after the change.

Pay close attention to (OPTIONAL)

Any specific code change or test case points which must be addressed/reviewed at the time of GA release.

Screenshots (if relevant)

Thanks for contributing!

Hey @splunk-soar-connectors-bot ,
I get the error repo_name_has_expected_app_id But I'm not sure why.
I have an app id and also a name in my JSON file.
Can you please explain what I should do to pass this test?
Thanks :)

Hello @guysl10, Could you please provide the write access of your forked repo to me so that I can make code changes related to the development standards?
Also, Could you please remove the app_versions folder and change the app_version to 1.0.0 as this is the initial release of the app?

Hey, @achandya-crest, I gave you write access to the project and fixed what you asked.
Thanks.

Hello @guysl10
I have updated the code and according to developer standards I have removed the requests module from requirements.txt and the module's .whl files as well because the requests module is present on Splunk SOAR instances.
Could you please test these changes from your side before we proceed with the release?

@achandya-crest Looks good. Let's proceed.


Harden the CSP

For information, see https://web.dev/csp-xss/

Codecov Report

Merging #185 (f54c87c) into main (ed639b9) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #185   +/-   ##
=======================================
  Coverage   68.86%   68.86%
=======================================
  Files          57       57
  Lines         546      546
  Branches      136      136
=======================================
  Hits          376      376
  Misses        162      162
  Partials        8        8

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed639b9...f54c87c. Read the comment docs.


Using internal monorepo packages without transpilation in stacks code

Setup
Example monorepo setup:
my-project/
|-- apps/
|   +-- backend/
|       |-- .build/
|           +-- index.js # built using the SST CLI, namely Stacks.build from @serverless-stack/core
|       |-- src/
|       |-- stacks/
|           +-- index.ts
|       |-- package.json
|       +-- sst.json
|-- node_modules/
|-- packages/
|   +-- core/
|       |-- index.ts
|       +-- package.json
|-- package.json
+-- yarn.lock

my-project/apps/backend/package.json
{
  // ...
  "dependencies": {
    "@my-project/core": "*"
  }
}

my-project/packages/core/index.ts
export const SOME_CONSTANT = 42

my-project/apps/backend/stacks/index.ts
import { App } from '@serverless-stack/resources'
import { SOME_CONSTANT } from '@my-project/core'

export default function main(app: App) {
  console.log(SOME_CONSTANT)

  // ...
}

Problem
When you run your stacks with sst start, the following file will be built:
my-project/apps/backend/.build/index.js
// ...
import { SOME_CONSTANT } from '@my-project/core'; // This is a problem, because @my-project/core is written in TS and not transpiled at any point.
                                                  // Throws an error in runtime: TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /path/to/my-project/packages/core/index.ts
//...

Of course this is because the stacks code, when being built using esbuild, marks all "dependencies" from package.json as external.
Solution
The easy way
Add a configuration option to mark dependencies as "non-external", so they are built and bundled by esbuild. This is what is used by nextjs and its next-transpile-modules.
The harder way
Find out, the package is internal, as it points to a TS file ("main" field in<EMAIL_ADDRESS>symlinked to my-project/packages/core/package.json by yarn, is set to "*.ts").
The hard part may be, that this can be potentially slow.
PR
Is a PR for this welcome? Which approach should I take?

Thanks for the detailed writeup!
Does it work if you import it using a relative path?

If the package uses relative imports, it would work, yes.
However, in our setup, we ban relative imports (in apps and in packages!) and rely on the package name.
For example, in my-project/packages/core/index.ts we would import using import ... from '@my-project/core/other', not import ... from './other'. (self referencing)
This is because we really like importing relative to the root, not relative to the file that has the import statement. In the above example, @my-project/core can be thought of as a Typescript alias for the root of the package.

Just to chime in and I might be wrong but the way NPM handles workspaces (monorepos) is that all "workspaces" get symlinked (not sure how it is with yarn). Thus there's no need to add a local package as a dependency. It already exists as an import reference of import ... from "my-other-package" based on the name in package.json. Thus esbuild should include it if you remove as a dependency and just import it (again for NPM not yarn).

@AshHeskes yarn also symlinks, so there is no need to list it there either.
However, there are two problems with this approach:

You are not explicit as to which packages you are using (and which packages should be be bundled in app.asar in Electron for example).
The import/no-extraneous-dependencies ESLint rule, if used, would fail.

I'm having the same issue. Each micro-service is a different package and is exposed as Stack. The only solution I have in mind is to build all of the packages and then run sst start, but I don't think it will scale well.
Any ideas?

If you break things up into multiple packages you must build those packages separately.  Node cannot import TS directly from another package, it expects built code.
This is why we strongly advise against creating a lot of package boundaries.

@thdxr What do you think about exposing a whitelist for packages that should be built, similar to next-transpile-modules?

@thdxr Before I switched to sst, I built my entry package with esbuild, which transpiled all of the other TS packages. What am I missing here?
If it helps anyone, for now, I settled on the following setup:

tsup for building/watching.
turborepo for concurrent builds and caching.

The issue is that is a hack around the way the module system in node and typescript works.
We're unlikely to support non-standard things like this because it breaks compatibility with other tools - we've already seen this in other cases where some non-standard code in our codebase breaks under pnpm.
To reiterate this isn't an SST issue. In the node/TS world when importing code from another package it must already be built. So you'd have to setup your own build process to build your stacks code into js first, which you can do through turborepo
We advise against this because there's no advantage to breaking things up into packages like this and you lose a bunch of typescript features like symbol renaming. We recommend using typescript path aliases to simulate a different package

We advise against this because there's no advantage to breaking things up into packages like this

I agree, however in our use-case we import packages that should be packages (like our internal helpers).
We don't want to have to build the package just for SST, if for our frontend, backend, and app we don't have to do that.
Building packages makes us lose some features like instant feedback loop, jump to definition etc.
Currently, we simply don't list the package in package.json, but that seems to be a hack. An API like next-transpile-modules seems to work much better, as you can opt-in to building specific packages on the fly.

By the way, turborepo's yarn create turbo uses this setup by default, ie. their "ui" package is never built (doesn't even have builder set up nor "build" script in package.json). Both "web" and "docs" apps transpile the "ui" package on the fly with
experimental: {
  transpilePackages: ["ui"],
},

in next.config.js.

Is there any progress on this?
I found out the hard way that this actually works out of the box for the "Runtime" or Lambda code, but it does not work for the stacks code. So for example if I set a dependency to a package in package.json and the package is local, then the
import { Example } from '@localPackage/example';

works just fine for code within Runtime, but For the hell of it I dont know how to make it work for code that lies within stacks.
I tried adding different tsconfig.json files from within both, tried referencing the package "manually" with the tsconfig paths: {}   option, but I keep on getting the following error: Anyone knows how to get around this?
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for ...

 Do you have a really large stacks codebase?

I have some code that is reused between source and stacks. That code is a Config-style (@myapp/config) package where we keep all the application config that is reused on both BE and FE applications. Some of that config also creates AWS resources on demand.
What we've done to get around this is to split the config part related to stacks out of the Config package for now, but it would be nice if this gets addressed at some point.

Have you tried this with sst2? I think it might just work given some of the other restructuring we did

@paymonsattar what's your use case for this? How come you have a separate package for your stacks code?

@paymonsattar what's your use case for this? How come you have a separate package for your stacks code?

We are using SST extensively in our company. We have a common project that is used by several other projects. In one of our projects, a private api built using SST, that utilises the common package we get the following error. This error only appears on projects that use SST.
`node:internal/errors:490
ErrorCaptureStackTrace(err);
^
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/paymon/project/backend/common/src/index.ts
at new NodeError (node:internal/errors:399:5)
at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:79:11)
at defaultGetFormat (node:internal/modules/esm/get_format:121:38)
at defaultLoad (node:internal/modules/esm/load:81:20)
at nextLoad (node:internal/modules/esm/loader:163:28)
at ESMLoader.load (node:internal/modules/esm/loader:605:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
at new ModuleJob (node:internal/modules/esm/module_job:64:26)
at #createModuleJob (node:internal/modules/esm/loader:480:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34) {
code: 'ERR_UNKNOWN_FILE_EXTENSION'`
This is referring to the "main": "src/index.ts" in common/package.json. Replacing that with "main": "dist/index.js",   "types": "dist/index.d.ts" does fix the issue. But obviously, it is just a temp fix.

Hey @paymonsattar can you create a minimal reproduction - I think I should be able to fix this

@thdxr  this is much needed, specially in the monorepo type setup

Well our starters are all monorepo by default. But this is for folks that are managing their own constructs. Is this what you are doing?

We've also just run into this issue. Firstly, let me say that SST is amazing, and we're rapidly adopting it at @vital-software. Thanks for making an awesome thing 😀 🎉
In our nx monorepo, each stack lives in a separate app directory, so that nx can figure out which stacks to test and deploy via nx affected. This works really well, and we use it to create ephemeral stacks for our PR integration tests.
We've reached the point where it makes sense to start extracting some common instrumentation config (logging and monitoring constructs, and some sensible defaults for lambda functions) that can be shared between stacks.
Successful workarounds so far:

Build our common package first, and import the built artefacts into our apps
Use a relative import in sst.config.ts files to import our shared constructs from the nx libs directory (../../libs/).

For now, I think we're going to use relative imports. I couldn't get path aliases working with TypeScript/ESM, but that's probably PEBKAC 😅

Ah I see thanks for sharing the details. This might be something we need to figure out. @thdxr thoughts?

@jayair That would be great our company is using netlify at the moment for deployments, but we're playing around SST haven't found a good guide on NX + SST (I'd be curious to hear how @JoshBarr figured that out. 😄

Hmm we don't have a guide but I came across this message about some NX related question today on Discord: https://discord.com/channels/983865673656705025/1027265626085019769/1118172312445919272

Pattern that requires this: https://turbo.build/blog/you-might-not-need-typescript-project-references

For anybody hitting this issue, I just went ahead and use typescript aliases instead of using workspace imports in my package.json, and it worked for me.
It's a little frustrating because there's a bit of a miss-match between what the turborepo docs recommend here: and this setup, but I don't think it's a very serious problem.
Just to help people who may be googling the error I had, I'm going to paste the error message here, because mine was a little different to those I've seen in the thread, but the root cause was the same:
✖  Build failed apps/lambdas/src/migrate.handler
   Could not resolve<EMAIL_ADDRESS>   node_modules/.pnpm/@babel+core<EMAIL_ADDRESS>   98 │             const packageJson = require("@babel/preset-typescript/package.json");
   No loader is configured for ".node" files<EMAIL_ADDRESS><EMAIL_ADDRESS>   13 │ const Native = require("./fsevents.node");

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble


MP, various transporters /substrate specific

It would be useful if the substrate-specific descriptions of transmembrane transport could be grouped under
substrate-specific transmembrane transporter activity
Currently many are, but many
phosphate transmembrane transporter activity
modified amino acid transmembrane transporter activity
salt transmembrane transporter activity
organic phosphonate transmembrane transporter activity
galactosamine transmembrane transporter activity
iron chelate transmembrane transporter activity
organic hydroxy compound transmembrane transporter activity
vitamin transmembrane transporter activity
(or there are redundant relations which could be removed)
Reported by: ValWood
Original Ticket: geneontology/ontology-requests/11676

Hi @ukemi
I merged 'substrate-specific transmembrane transporter activity' into 'transmembrane transporter activity'; was this not correct ?
https://github.com/geneontology/go-ontology/issues/14491
Pascale

This goes back to the discussion that we had at a GOC meeting a while ago where we thought it was important to distinguish between transporters that could transport more than one cargo versus transporters that were specific to a certain type of cargo.   You will have to investigate that. #11840. If I recall, @huaiyumi  felt strongly about how we represent these and has expertise in the field. Perhaps work with him to make sure everything is ok.

@paolaroncaglia  asked me earlier to close,  but this request predates  @ukemi comment. Still want me to close?

Actually, now that I look at the direct children of transmembrane transporter, I think it might help curators to group the  substrate specific axis of classification under a single term. Mainly to help curators when browsing for the correct term....
So the list  directly under transmembrane transport would be:
export across plasma membrane
mitochondrial transmembrane transport
import across plasma membrane
vacuolar transmembrane transport
substrate-specific transmembrane transporter activity (but this grouping terms could also be inorganic /organic compound transmembrane transport), either would be good. We probably don't need both.

I did not know substrate-specific meant this:
"we thought it was important to distinguish between transporters that could transport more than one cargo versus transporters that were specific to a certain type of cargo"
I don't know how we would know this? we only know what has been assayed. If a transporter transports >1 compound we annotate to multiple activities. Why do we need to capture  in the ontology that a transporter only transports one thing? We don't do this for enzymes? It seems i) outside the scope of GO, and ii) impossible to know from most expereiments....

@ValWood
Actually, the ticket I suggested might be closed was a different one :-)
https://github.com/geneontology/go-ontology/issues/12317
I have nothing to do with the present one I'm afraid.


Update rsat-install-configure.md

Adjusted number of days between certificate recycle to 60 which is the accurate number of days.

:x: Organization membership check failed
Members of the MicrosoftDocs organization should use the private repo. Please move your changes to https://github.com/MicrosoftDocs/Dynamics-365-Operations, and close this pull request.


No longer works for Node 11+

Due to the use of an outdated version of ajv (uses 5.2.2 and latest is 8.2.0) that still uses the deprecated url.parse method (deprecated since node 11) this module will never work if you use Node 11+.

If anyone runs into the same issue.
I've taken it upon myself to update and rewrite the module as jform-react


Cannot parse inline named argument types

This code
let foo = (first: ((~one: 'one, ~two: 'two) => 'three)) => first(~one=1);

refmts too
let foo = (first: (~one: 'one, ~two: 'two) => 'three) => first(~one=1);

Which I believe should be valid - but no longer parses because it detects a tuple.

This is not a bug in current master. I think @IwanKaramazow fixed it recently.


Add support for reading authoritativeSource

Need to add support to read authoritativeSource (in camel case) as mentioned here -> https://github.com/geolexica/jekyll-geolexica/issues/12

Am I missing some context here? Why do we want to use "authoritativeSource" instead of the "sources -> type: authoritative" now?

@ronaldtse This ticket was created to handle authoritativeSource key present in metaverse-glossary, but that was handled in Jekyll-geolexica.
So should I close this ticket?

Yes, thanks!


'should.and' failing with<EMAIL_ADDRESS>

This spec-code
var methodTest = sinon.stub();

methodTest('y');

methodTest.should.have.been.calledOnce
    .and.calledWith('y');

causes
✖ validates each validation
      Chrome 43.0.2357 (Mac OS X 10.9.5)
    TypeError: Cannot read property 'and' of undefined

with<EMAIL_ADDRESS>version.
--
I've created simple project for bug repro:
https://github.com/AlexKVal/karma-test
master branch uses<EMAIL_ADDRESS>and causes test fail
karma13 https://travis-ci.org/AlexKVal/karma-test/builds/71240835

https://github.com/AlexKVal/karma-test/tree/karma12 branch uses<EMAIL_ADDRESS>version and works fine
karma12 https://travis-ci.org/AlexKVal/karma-test/builds/71240954

Thanks for the report, will try to take look as soon as I can

It looks like the underlying issue is the same as #1499. There is a conflict between the karma-chai and the karma-sinon-chai plugin. They both include a version of chai, and in 0.12 the addition from karma-sinon-chai simply won, but for some reason this broken now. You can work around it for now by removing karma-chai, which isn't needed anyway.

Thank you @Dignifiedquire for the hint about karma-chai. :cherries:
Indeed it helps with tests. :sparkle:
Now all is left to migrate react-bootstrap to<EMAIL_ADDRESS>is https://github.com/karma-runner/karma/issues/1497, which as I can see is also caused by the new file loading mechanic.

Fix was released in 0.13.1

Thank you :tada: :cherries:


Update ja-JP.json

Translations corresponding to the added parts have been made. Thank you for the attractive features!

Hehehe, I can't thank you enough for your supports,
I'll add this translation right away!


Guarantee service ID and backend ordering across restarts

This PR is deliberately kept simple and non-intrusive to allow for easy backporting.
Both the service ID and the ordering of the service backends were inconsistent across restarts which had the potential to disrupt connections across when the agent was restarted.
Fixes: #5744

This change is

test-me-please

test-me-please

Rebased after merging #6072

test-me-please


Organize better tests

Coverage remained the same at 100.0% when pulling 404e9716c923d8b3b1dc707582298613375b7fc4 on change-rspec into f50e774d119136439c93593f879e2443475caab4 on master.


Expose ClipboardAction

This is an excellent library! :boat:
Would be great if one could interface directly with ClipboardAction. With a SPA action-handling etc. is already taken care of by the framework.

Hey @sandstrom, thanks!
Here are some alternatives for React, Angular, etc: https://github.com/zenorocha/clipboard.js/wiki#have-you-built-a-clipboardjs-wrapper
Duplicated from #66

@zenorocha thanks!
I ended up using vanilla js (in this situation I only needed support for  evergreen browsers). Copied below in case it helps someone.
var el = document.querySelector('#myEl');
el.focus();
el.setSelectionRange(0, el.value.length);

var result = document.execCommand('copy');
if (result) {
  # notify here...
  document.activeElement.blur(); # clear focus
}

Awesome ;)


Moved DynamoDB code examples into DynamoDB directory

Issue #, if available:
Most of the DynamodB code examples were in individual directories.
Start of resolving issue #400
Description of changes:
Moved all of the main.rs files into dynamodb/src/bin.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Updated.


Fix writing value for hand-tracking V2.0 to AndroidManifest.xml

Due to a typo, the value for the meta-data tag for the hand-tracking-version was never set. It should be set in the "value" attribute, not in the "name" attribute.
Corresponding excerpt from the AndroidManifest.xml:
        <meta-data
            android:name="xr_hand_tracking_version_name"
            android:value="xr_hand_tracking_version_value"/>

Compatible with 3.x

Thanks!

Cherry-picked for 3.6.

Cherry-picked for 3.5.2.


WIP: Updated lint checks and README.md.

r? @inetic
(maidsafe_highfive has picked a reviewer for you, use r? to override)

r? @rossmuir

This is replaced by https://github.com/maidsafe/crust/pull/352


Handle shears in RenderLinearGradient

@jonathanolson found a bug that a shear matrix ( eg. [ [ 1, 1, 0 ], [0, 1, 0], [0, 0, 1]] ), was not being handled int the split function of RenderLinearGradient.

@jonathanolson and I worked on a solution together. I believe this can be closed now.


Possible  goroutine  leak?

//  the following  code is from  pipe.go
func (s *Session) WaitTimeout(timeout time.Duration) (err error) {
select {
case <-time.After(timeout):
s.Kill(syscall.SIGKILL)
return ErrExecTimeout
case err = <-Go(s.Wait):
return err
}
}
func Go(f func() error) chan error {
ch := make(chan error)
go func() {
ch <- f()
}()
return ch
}
if timeout happends,  go func() {ch <- f() }(), will never return, since
A receive from an unbuffered channel happens before the send on that channel completes.

That sounds reasonable