Question: Too many arguments for dataclass

Question

OS: macOS 10.14
Python version: 3.7.0
Pydantic version 0.14

Hi. I decided to try using pydantic instead of JSON Schemas for validating input / output data in my aiohttp web application, and so far run into strange mypy issue. Let consider next code of adding new project,
from aiohttp import web
from pydantic.dataclasses import dataclass

@dataclass
class AddProject:

    """Add new user project."""

    name: str
    slug: Optional[str]
    description: Optional[str]

@login_required
async def add_project(request: web.Request) -> web.Response:
    """Add new user project."""
    data = AddProject(**await api_request(request))
    user_id = await get_user_id(request)

    async with request.app[APP_DB_KEY].acquire() as conn:
        project = await create_project(
            conn,
            user_id=user_id,
            name=data.name,
            slug=data.slug,
            description=data.description)

    return api_response(project.dict(), status=201)

It works as expected in runtime, but mypy failed with,
error: Too many arguments for "AddProject"

I tried to satisfy mypy by changing AddProject instantiation to,
    raw_data = await api_request(request)
    data = AddProject(
        name=raw_data.get('name'),
        slug=raw_data.get('slug'),
        description=raw_data.get('description'))

but that made mypy failed with,
error: Unexpected keyword argument "name" for "AddProject"
error: Unexpected keyword argument "slug" for "AddProject"
error: Unexpected keyword argument "description" for "AddProject"

I'm a bit curious, as behind that mypy seems to work properly with AddProject, for example, it knows its attributes and fail in case of using missed ones, like data.does_not_exist.

Any suggestions on how to make mypy happy? Or this is an expected behaviour, which fixed only by # type: ignore comment?
ps. api_request coroutine returns Dict[Any, Any] if that necessary

Afraid I'm not sure. Does it give the same error if you use a vanilla dataclass?

@samuelcolvin
Unfortunately not. It works well with,
from dataclasses import dataclass as py_dataclass

@py_dataclass
class PyAddProject:

    """Add new user project."""

    name: str
    slug: Optional[str]
    description: Optional[str]

mypy succeed, where previously failed with Too many arguments error.

@samuelcolvin
Thanks for your answer. As of now I'm OK with # type: ignore comments, so feel free to close this as wontfix

thanks for understanding, I'll leave it open to make it easier for others to find and in case someone else has a solution.


Update paths to css file in readme

I was playing around with this project and noticed that a URL returned a 404.

Codecov Report

Merging #162 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #162   +/-   ##
=======================================
  Coverage   54.23%   54.23%
=======================================
  Files          31       31
  Lines         684      684
  Branches      152      152
=======================================
  Hits          371      371
  Misses        311      311
  Partials        2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cade8e3...767c551. Read the comment docs.

Just came across 5fb16758, looks like the path will be right as soon as the next release is available on npm. 👍


🐛 Bug Report: v1.30.4 IdentityAuthInjectorFetchMiddleware fetch request timeout when 14k user entities are loaded into the system

📜 Description
After updating Backstage from 1.29.2 to 1.30.4 and running with limited set of plugins the Backstage /api/catalog queries and  auth /refresh calls (using Guest module) will initially load but once the catalog is called again every fetch request will timeout.
Began to experiment by commenting out the package/backend imported plugins and removing any plugins from package/app routing and from entity details pages and tabs.
I removed loading the user catalog entry point so that only 9k~ entities were loaded into the system and running with local default configurations with yarn dev.
The site responded without issue and had no issues with any requests to the catalog API or with the IdentityAuthInjectorFetchMiddleware and PermissionClient.
It seems that once the number of user entities reaches a certain threshold Backstage begins to fail making any request and is functionally frozen. Even ctrl-c on the terminal to stop the server is non-responsive.
I've been spelunking the Backstage source in v1.30.4 and see that most of the code around where it's failing hasn't been touched in some time. Still working through but prior to removing backend-common and other libraries for the 1.30.4 migration this problem did not exist.
This behavior exists in both in-memory local testing scenarios and when deployed with Postgres / Redis resources. Starting to smell like a bad lookup / blocked promise / or maybe related to a throttling library.
👍 Expected behavior
Backstage site should not timeout on every request after loading with 21k total entities, which includes 14k user entities. Running locally with in-memory should not perform this poorly.
Seeing the same behavior in our deployed beta environment which is backed by Postgres database and Redis.
👎 Actual Behavior with Screenshots

👟 Reproduction steps
Point Backstage at a catalog of 500+ teams, 14k users, few thousand resources, and a few thousand services / systems / and a dozen or so domains.
Performance degrades to the point where the size is not usable and functionally blocked until request timeouts unblock it.
📃 Provide the context for the Bug.
Unable to use Backstage. Site is non-responsive when loading a catalog of static entity definitions. Was functionality without issue in 1.29.2 but post-migration to 1.30.4 the site has become crippled loading the same catalog.

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
No, but I'm happy to collaborate on a PR with someone else

Running with debug logging picking up:
[1] 2024-09-13T00:32:47.283Z rootHttpRouter error Request failed with status 500 Unable to generate legacy token for communication with the 'catalog' plugin. You will typically encounter this error when attempting to call a plugin that does not exist, or is deployed with an old version of Backstage; caused by Error: Unable to generate legacy token, no legacy keys are configured in 'backend.auth.keys' or 'backend.auth.externalAccess' type=errorHandler cause=Error: Unable to generate legacy token, no legacy keys are configured in 'backend.auth.keys' or 'backend.auth.externalAccess' name=Error stack=Error: Unable to generate legacy token for communication with the 'catalog' plugin. You will typically encounter this error when attempting to call a plugin that does not exist, or is deployed with an old version of Backstage; caused by Error: Unable to generate legacy token, no legacy keys are configured in 'backend.auth.keys' or 'backend.auth.externalAccess' [1]     at <anonymous> (/Users/coderrob/go/src/backstage-collab-portal/node_modules/@backstage/backend-defaults/src/entrypoints/auth/DefaultAuthService.ts:175:17)

Bumped to
{
  "version": "1.31.0-next.2"
}

Still happening.
To see whether this exists in current Backstage create app I just created a fresh Backstage application, made sure version 1.30.4 was used, and then added a app-config.local.yaml to point the catalog to the catalog static data on my system.
It's having the same behavior. It bombs trying to fetch any data. It works the first time and then all subsequent requests just bomb or timeout.
Again, this behavior is not something that has existed in the versions prior to 1.30.x.

New app screens

Gave up on 1.30.x versions and downgraded to 1.29.2. Used package.json resolutions to attempt to find the correct versions of 1.30.x packages but the freezing behavior continued and without additional warnings. Used backstage-cli bump to downgrade to 1.29.2 and things are back to functioning without freezing / failing.

We've walked through the changes in 1.30 and what you've reported here, and so far nothing stands out 🤔. Main thing in the changelog is really the node-fetch bump, is it enough to downgrade node-fetch to get this working?
Would you be able to provide a repro repo or similar? In that case we can get a bit more hands on in helping out to get this fixed.
I'm also wondering to what extends you've pinned down when exactly this lockup seems to happen, as in is it always after a specific request or scheduled job? For example if you curl the backend separately, is there a specific request that the frontend makes that triggers this?

@Rugvip I took enough notes on the state of things that I should be able to take the codebase, strip down entirely, and try to emulate.
The way I was able to reproduce the behavior was:
yarn dev:debug (LOG_LEVEL=debug yarn dev)
.... wait till all the catalog data loads and then I see the timestamp shown for the next refresh ....

Once the catalog was loaded I'd click to login with the "guest" provider.
The site would load our placeholder "overview" page (not using the customizable homepage yet).
I'd then click on the "Catalog" from the left-side nav bar.
The catalog would load fine with the 2~ catalog items and it'd do so quickly <500ms
Then I'd click on the "Teams" left-side nav bar
(for reference)

(teams)

(users)

I'd watch the network tab during the clicks to see what was called / where it stalled. I'd see the 404 on the cookies check and it'd behave well clicking catalog or clicking teams. It was after the first loading of data it seemed to trigger an auth /refresh and the fetch would block the site.
Other times it'd be just clicking the catalog link or teams link and I'd see the by-query call being made and it'd block and the entire site becomes unresponsive. Even trying to ctrl-c the running service would not respond.
After you ctrl-c you'll see some unhandled rejections within a second or two and a number of "Shutdown hook failed, Error: IPC request 'DevDataStore.save' with ID..." messages.
In the cases of the frozen fetch it's completely non-responsive.
The behavior made it feel like it was working for a request or two and then just locked.
Grabbed some screenshots during the freeze and the endpoints that were usually impacted were:
/by-query (IdentityAuthInjectorFetchMiddleware)
/refresh (ProxiedSignInIdentify.esm.js)
/entities?filter=kind%3Dcomponent (IdentityAuthInjectorFetchMiddleware)
I set Chrome to stop at unhandled and handled exceptions to see if I could work backwards on what was causing it and it seemed to be within @backstage/core-components.

The one thing I was starting to poke around on was the dependencies of "backend-common" and "backend-app-api" hat dependencies on "plugin-auth-node" with two different minor revisions. 0.5.x vs 0.4.x.

I started to see the backend-test-utils was dependent on the 0.4.x version and is commonly used in backend plugins.

I needed to get our beta environment out of a bad state so I used the Backstage cli to bump down to version 1.29.2 at that point and was able to get it back to operating normally.
The "backend-test-utils" is still "sus" as my kids say.
We definitely have a vested interest in discovering the cause / solution since it looks like the upgrade would be blocking. :)
I have some priority to-dos but I'll find some time to create a sharable reproduction.
From what I tried...

auditing each of the Backstage dependencies taking "info" lists before each / after each action
stripping backend index.ts to only import core backstage plugins for catalog, auth, search, permissions, and techdocs
assigning fixed versions using package.json resolutions in packages/app and packages/backend

from the info list tried older versions / newer versions

used guest auth, used with OIDC and no guest
stripped down loading of our catalog data to subset of groups, users, domains, systems, and other services/resources
removed yarn.lock
used bump with "main"
used bump with "next"
removed the "node-fetch" packages from plugins
running backstage-cli migrate commands (just in case...)
running backstage-cli repo fix / clean / list-deprecations (to see if anything new popped up)
using git clean -xdf to ensure no cruft was leftover in the repo to cause oddities before any changes / testing

I didn't try explicitly calling the Backend api to see if I could manually trigger the problem. I have a CLI tool that was built to do data-integrity checks and some self-healing for our Backstage data.
I have a stripped down version of Backstage there running 1.29.2 I can bump to 1.30.4 and have it pointed at the catalog. I saw similar behaviors there and it seemed to be related to the number of users loaded into the catalog that triggered the condition.

 You could create a fake provider that inserts tons of entities to simulate the error in there.

Sorry for the delay. We're dealing with a large layoff that impacted our already tiny team.
We stabilized on 1.29.2 but search is working, then not working, then working, and then not working. Flipping back and forth between HTTP 500 MissingIndexError and working without problems. It's completely erratic.
One second...

The next...

See that 1.31.1 was released. Going to try bumping from 1.29.2 to 1.31.1 first to see if that resolves the problems seen with 1.30.x.

After a git clean -xdf, yarn, and a bump to 1.31.1. I cleared cache / did a hard empty / reload, and tried with incognito.
Did the yarn dev with debug enabled. Refreshed the page... annnnnnd stuck.

Sat like this for a few minutes... did the inspector with network tab ^^

... after a minute or so I clicked refresh again. This time it loaded the page but then immediately blocked again.

Going to give it a few minutes, but seems like the behaviors still exist.
I'll go about stripping down to a shareable version bit by bit ensuring the bad behaviors persist between each change.

Ok, the requests resolved after 4.2 minutes.
I noticed in the console right at the point when things resolved a whole bunch of @backstage/plugin-search-backend-module-techdocs failures started flashing into the server output:

After that it started listing a bunch of paginated requests to the catalog entities.

The catalog was fully loaded. I clicked on an entry and saw the details page. Everything was quick.
Clicked on "APIs" .... and back to blocked

OK, interestingly it took another 4~ minutes for the request to finish and then another flood of techdocs.

Thank you for your quick response everybody.
Could you try enabling pagination on the catalog index page and see if that helps?
Here's how to do it:
https://backstage.io/docs/next/features/software-catalog/catalog-customization/#pagination

Hmm, I think it's enabled by default already https://github.com/backstage/backstage/blob/v1.31.1/packages/app/src/App.tsx#L122

Hi @deejay1, that's just the example app for this repo and not what gets used when generating a Backstage instance using npx @backstage/create-app@latest - that would be what's here: https://github.com/backstage/backstage/blob/56becbfb19b85361c43707a6553615fa1e16a360/packages/create-app/templates/default-app/packages/app/src/App.tsx#L67

Hey all I was able to use create-app on version 1.31.1 and seems like I can reproduce it with our data. Running on a completely stripped down version.
Going to see about making some data scrubbers to keep the behaviors in data I can share for external testing.

@freben could use your thoughts... i had some time for debugging into the setup and found something that matched the behaviors I was seeing.
When I run the site locally I stepped through and I found a dependency that looks like it's potentially related.
https://github.com/backstage/backstage/blob/2e406568d20749de27af7b2854b99c254dd824c2/plugins/catalog/src/apis/EntityPresentationApi/DefaultEntityPresentationApi.ts#L370
This DefaultEntityPresentationApi depends on dataloader version ^2.0.0. In the yarn.lock the resolution is set to using ^2.2.2.
Backstage libraries that depended dataloader on ^2.0.0:

@backstage/plugin-catalog
@backstage/plugin-permission-backend
@backstage/plugin-search-backend
....

https://github.com/graphql/dataloader/releases shows that some updates in 2.2.2 changed some behaviors related to how it batches and how it fails. It looks like previous to 2.2.2 the call to .load() could throw an error, which is fine since the code handles it, but previously it could launch a task unintentionally.
Wanted to call that out as something I saw checking yarn.lock differences to versions defined in package.json. Blocking batches and the state "cache" implementation in the DefaultEntityPresentationApi seemed suspect for potential "block" points.
What i see happening is the first call on the page loading for "catalog" goes to:

plugin-catalog-react/hooks/useEntityListProvider
CatalogClient => DefaultApi.client.ts => getEntities(...

. . . once this loads several queries fire off to request the catalog client for the same data with filters applied for the catalog page filters . . .

plugin-catalog-react/hooks/useOwnedEntitiesCount
CatalogClient => DefaultApi.client.ts => getEntitiesByQuery(...
UserListPicket: plugins/catalog-react/src/components/UserListPicker/useAllEntitiesCount.ts
CatalogClient => DefaultApi.client.ts => getEntitiesByQuery(...
UserListPicker: plugins/catalog-react/src/components/UserListPicker/useStarredEntitiesCount.ts
CatalogClient => DefaultApi.client.ts => getEntitiesByQuery(...
EntityAutocompletePicker: plugin-catalog-react/components/EntityAutocompletePicker.tsx
CatalogClient => DefaultApi.client.ts => getEntityFacets(...

The first page request of data seems to be fine. The data is present and interactable. There are still several "pending" promises that return loading true from useAsync for the filter data requests.
The thing I noticed was that requests 2 through 5 consistently sit at pending state all get "unblocked" for a consistent delay. The duration of the pending state delay seems relative to the amount of data it's trying to load.
It is made much worse if you go from the "components" (2k~) around 4-7 seconds locked to "users" (14k~) the delay has been pretty consistent around 1.4 minutes (with no debugger attached.)

I was doing some checks again the behavior and the timings I was seeing on loading 14,500 users with a latency of around 200-500ms for round trip calls. The 20 items default "limit" on items and the 100 max requests with 50ms setTimeout on batch delay milliseconds. If you average it out takes could take about 5~ minutes, which is what we're seeing in our cloud hosted integration environment.
I could be way off but I saw recentish (in < 2 months) work in that area around creating a "state" of entities along with an update to only return a limited set of fields for entities.
I may have been chasing a red herring but it seems like something is blocking on batching requests to the backend API and it seems to match really closely to the default timings i saw in DefaultEntityPresentationApiOptions.
viewer.zip
Attaching the stripped down viewer I've been using to test behaviors with.
I'm sorry it's taking so long, but I'm still pending on getting a reproduction of our data that is sanitized / sharable. I'll see if i can spare some personal time this weekend to get something together.

. . once this loads several queries fire off to request the catalog client for the same data with filters applied for the catalog page filters (should they be using the entity list instead of making catalog api calls for the same data w/ filters?) . . .

this is correct as those api calls are used to resolve the number of "owned/total/starred" entities.
I feel like there is something else keeping your backend busy. Probably the request that keeps the backend busy isn't even initiated from the frontend.

Well it COULD be that some of those filter calls turn out to be so expensive that they hold up backend execution.

but I guess the load will be at the database level not in node. I don't think we do something crazy that holds up resources for seconds

Hi,
Have you tried profiling the backend according to: https://backstage.io/docs/tooling/local-dev/profiling ?
We also have quite a big catalog with 25k+ entities and found some issues both in backstage and in our own code.
I think this happens because something is eating all the CPU and the event loop gets stuck. Do you have any info about resource usage from both the backend and the database?

@adamj29 whaaaaaat O_o. Are you absolutely sure? 😅. We're staring at that piece of code and can't conceive any way in which that change could lead to a performance degredation.

ran the same test again today and the performance is nearly identical, must have been something else 🙃

I wonder if https://github.com/backstage/backstage/pull/27167 made this better.

@CiscoRob is it possible to check how much data those by-refs calls are returning to the frontend?I have the feeling the issue might be caused by https://github.com/backstage/backstage/pull/25366

After upgrading to version 1.31.1, I encountered timeout errors like @CiscoRob mentioned in his screenshots while loading the home page and hitting endpoints, causing Backstage to load entities slowly. Initially, I considered reverting to an older Backstage version, but instead, I rolled back specific dependencies related to @backstage/core-app-api, particularly IdentityAuthInjectorFetchMiddleware. I also downgraded related dependencies like @backstage/app-defaults, @backstage/plugin-user-settings, and others, as well as @backstage/plugin-catalog-backend, which manages API endpoints. Additionally, I reverted versions of dependencies that interact with plugin-catalog-backend.
After these changes, Backstage now loads faster and no longer encounters timeout issues.

For anyone reading along here, I think one thing to look into, is your packages/app/src/App.tsx file, where the catalog route should look like
    <Route
      path="/catalog"
      element={<CatalogIndexPage pagination={{ mode: 'offset', limit: 20 }} />}
    />

or
    <Route
      path="/catalog"
      element={<CatalogIndexPage pagination={{ mode: 'cursor', limit: 20 }} />}
    />

but absolutely NOT like
    <Route
      path="/catalog"
      element={<CatalogIndexPage />}
    />

Client-side tables are an absolute killer of performance and they risk loading the entire world upfront.

How this will fix the api endpoints getting timed out? @freben

I wonder if this is duplicate with #27628 or at least share some of the same symptoms 🤔

@pshenbagadelip hypothesis is, frontend doing such massively heavy requests that it actually hurts the backend / database, in scenarios where either the service and/or the database are sorely underpowered

In my case, When the server is running using RDS. It performs better and we don't see the timeouts issue of backend api endpoints whenever the catalog home page is loaded. But still it's slow compared to the previous version 1.28.0(our previous versions)
But when i deployed a server without RDS, i could see the timeout issues in the network tab for the backend apis. If i hit any apis it throws 5XX and the ecs task gets killed and restarted.
reducing the plugin-catalog-backend dependency to older version all the backend apis are working fine without RDS server. But it's not compatible when i tried to deploy with RDS.
`/app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1680
backstage
15 November 2024 at 20:33 (UTC+5:30)
throw new errors.ForwardedError(
backstage
15 November 2024 at 20:33 (UTC+5:30)
^
backstage
15 November 2024 at 20:33 (UTC+5:30)
ForwardedError: Plugin 'catalog' startup failed; caused by Error: The migration directory is corrupt, the following files are missing:<PHONE_NUMBER>0511_alter_target_in_locations.js
backstage
15 November 2024 at 20:33 (UTC+5:30)
at /app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1680:19
backstage
15 November 2024 at 20:33 (UTC+5:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async /app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1679:11
backstage
15 November 2024 at 20:33 (UTC+5:30)
... 3 lines matching cause stack trace ...
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async BackstageBackend.start (/app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1762:5) {
backstage
15 November 2024 at 20:33 (UTC+5:30)
cause: Error: The migration directory is corrupt, the following files are missing:<PHONE_NUMBER>0511_alter_target_in_locations.js
backstage
15 November 2024 at 20:33 (UTC+5:30)
at validateMigrationList (/app/node_modules/knex/lib/migrations/migrate/Migrator.js:567:11)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at Migrator.latest (/app/node_modules/knex/lib/migrations/migrate/Migrator.js:69:7)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async applyDatabaseMigrations (/app/node_modules/@backstage/plugin-catalog-backend/dist/cjs/CatalogBuilder-km2G8Nmv.cjs.js:1481:3)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async CatalogBuilder.build (/app/node_modules/@backstage/plugin-catalog-backend/dist/cjs/CatalogBuilder-km2G8Nmv.cjs.js:6950:7)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async Object.init [as func] (/app/node_modules/@backstage/plugin-catalog-backend/dist/alpha.cjs.js:223:46)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async /app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1679:11
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async Promise.all (index 2)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async #doStart (/app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1633:5)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async BackendInitializer.start (/app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1562:5)
backstage
15 November 2024 at 20:33 (UTC+5:30)
at async BackstageBackend.start (/app/node_modules/@backstage/backend-defaults/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1762:5)
backstage
15 November 2024 at 20:33 (UTC+5:30)
}
backstage
15 November 2024 at 20:33 (UTC+5:30)
Node.js v20.18.0

ForwardedError: Plugin 'catalog' startup failed; caused by Error: The migration directory is corrupt, the following files are missing:<PHONE_NUMBER>0511_alter_target_in_locations.js
backstage

yeah unfortunately you need to downgrade your database manually, as this isn't currently automated.

Just reverted the following 2 dependencies and the timeout issues of backend apis are resolved
"@backstage/plugin-catalog-backend": "~1.23.0",
"@backstage/plugin-catalog-backend-module-logs": "~0.0.4"

I've just deleted the<PHONE_NUMBER>0511_alter_target_in_locations .js file from rds and the server started working fine. that particular migration file will do the conversion of target column data type from String to text. A text data type can store anything right, so we just have the text data type with old plugin-catalog-backend  dependency version.
backstage_plugin_catalog=>
backstage_plugin_catalog=>
backstage_plugin_catalog=> SELECT * FROM knex_migrations WHERE name = '20241003170511_alter_target_in_locations.js';
 id |                    name                     | batch |       migration_time
----+---------------------------------------------+-------+----------------------------
 31 |<PHONE_NUMBER>0511_alter_target_in_locations.js |     4 | 2024-10-22 10:25:58.842+00
(1 row)

backstage_plugin_catalog=>
backstage_plugin_catalog=>
backstage_plugin_catalog=> DELETE FROM knex_migrations WHERE name = '20241003170511_alter_target_in_locations.js';
DELETE 1
backstage_plugin_catalog=>

@drodil I think you're correct, rolling back the catalog backend plugin in the above response seems to have fixed the issue - so I would say there's something in there? Is it actually the server side sorting or no? 😅

Hi @benjdlambert ,
I checked the migration mentioned in the above comment, and it seems to change the locations.target from string to text.
  await knex.schema.alterTable('locations', table => {
    table.text('target').alter();
  });

Why this then affects the performance, I have no clue :D My only guess is the DefaultLocationStore:getLocationByEntity getting too slow due to this change as it uses the target in the where query and the column is not indexed.  🤔  That's a long shot though 😄

@drodil not sure about that though. Unless you run knex migrate:down manually, it's not going to switch back to the previous code for the database. So I don't think it's in the migrations. @pshenbagadelip only mentions running an older version of the plugin, but it would still be using the latest migrated database.

Any update on this issue?


usage with docker desktop k8s

How to use this with docker desktop k8s?

Closing as we no longer maintain this repo.

Closing as we no longer maintain this repo.


ENSCORESW-2933: new CCDS parser with test case

Requirements

Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
Review the contributing guidelines for this repository; remember in particular:

do not modify code without testing for regression
provide simple unit tests to test the changes
if you change the schema you must patch the test databases as well, see Updating the schema
the PR must not fail unit testing

Description
Using one or more sentences, describe in detail the proposed changes.
Fresh implementation of CCDS parsing
Use case
Describe the problem. Please provide an example representing the motivation behind the need for having these changes in place.
Retrieves CCDS data from the core database and adds it as direct xref into the xref database
This replaces the functionality from the CCDSParser in the xref_mapping directory. Rather than requiring a dedicated CCDS database, the data is retrieved from the core database directly. It is Perl critic compliant and matches the standards as seen in other parsers in the repository
Benefits
If applicable, describe the advantages the changes will have.
CCDS data is added to the xref pipeline
Possible Drawbacks
If applicable, describe any possible undesirable consequence of the changes.
Testing
Have you added/modified unit tests to test the changes?
Yes
If so, do the tests pass/fail?
Tests pass
Have you run the entire test suite and no regression was detected?
Yes

Travis fails for other reasons unrelated to this PR.


upgrade crates-index-diff & migrate to gix- crate for gitoxide

https://github.com/Byron/crates-index-diff-rs/releases/tag/v16.0.0
in the release above you see also that git-repository is now gix

r? @Byron


Support the ScanIndexForward option in QUERY

Is your feature request related to a problem? Please describe.
It seems the UI doesn't have the ScanIndexForward option to reverse the index traversal in QUERY: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#API_Query_RequestSyntax.
Describe the solution you'd like
Support the ScanIndexForward option in QUERY.
Describe alternatives you've considered
None.
Additional context
None.

@chehsunliu, Working on adding ScanIndexForward. Here's the progress so far. Will include in the new release.

Sort Ascending (default true)
Sort Descending

BTW, I really like your UI. Very easy to use and also modern! Thanks!

I'm glad you like the UI. Thank you, I appreciate your kind words!


Feature/more generic functions

Possible new generic functions for cornflow core and cornflow client.
I do have several functions, endpoints and model which I tend to reuse between projects. So it may be better to have them directly in cornflow.
For the moment I did put all the functions in core in a "new_functions" directory.
The functions which are being considered usefull should later be moved to where it makes more sense.

@marioncottard what do we do with this issue?


Full screen support

Can you please provide an example of how to support full screen video playing for both Android and iOS?

@logo17  do you want youtube video player or a simple player like av player and exoplayer?

yeah , youtube video player if possible.

i think full screen already works on ios and android. did you check the code and built the app?


feat: enable long-lived grpc clients whilst supporting per request timeouts

feat: new base grpc client that supports adding timeouts per request
This can be used as a inner helper which provides common functionality of creating a new client
connection and setting specific timeouts for each request.
This allows us to use long-lived connections with keep-alives.

refactor(pool): use of the new generic gRPC client

refactor: keep the pool gRPC server handlers cancellable
The specific service implementations will be responsible for making it otherwise. This gives finer
grained control to decide how to implement each handler.

fix: spawn core pool create/destroy to avoid early drop on timeout
The existing code does not handle being dropped very well, as it will leave things in a
potentially "bad" state.
TODO: handle serive handlers drops gracefully. This will be tricky as eg ETCD client is async and so
we can't really flush things to ETCD in a nice way on drop.
For now, try to avoid dropping the futures when the client times out by spawning it.
NOTE: the future can still be dropped on reactor shutdown.

test: add grpc request timeout test
This validates that timeouts by modifying the grpc timeout header works.

bors try


Update to latest version

There has been major updates in mongodb and various other dependencies which leads this template unusable if someone is setting up with latest version

https://github.com/sc0rp10n-py/Next.js-Mongodb-Authentication-App/tree/new-version
I tried to fix it to the newer version
but handler.post is not running

only 8 and 9 are printed not 1 2 3 4 5 6 7

@tsaxena4k

Hey @sc0rp10n-py sorry for replying back late, yes you are right there's been some major updates in MongoDb and I also haven't gotten a chance to rework on this. Thank you so much to report this and for trying to fix it. I am free this weekend if you want to collab and work on this it will be great.

@tsaxena4k yes sure you can see my repo, i started the fix but the request is getting stuck on handler.post
i couldn't figure out what might be the issue.

did you manage to fix it?


Run master branch nightly using Travis cron

What
Run all tests on daily basis instead of each time something is merged on master.
Change travis test condition + add cron to travis config (todo after the PR is merged):

Once this PR is merged we can schedule a cron at midnight CET time.
Motivation
Running all test at each merge on master is costly and have low value.
Running it once a day should be enough.
Docs: https://docs.travis-ci.com/user/cron-jobs/

Travis cron is already schedule at midnight CET.

Removed logic for running validation on master since it might make the meaning of a "success build" of master. We wouldn't know if a success build on master mean validation or validation+tests. => https://github.com/DataDog/integrations-core/pull/3988/commits/6ac4c74424f1b3777125593de66104e68f9e37dd


Expand auto-import to all package.json dependencies

This PR creates an auxiliary program in the language service layer that contains node_modules packages that are specified as a direct dependency (including devDependencies peerDependencies) in a package.json file visible to the project’s root directory and not already present in the main program. This program is then supplied to the auto-imports machinery as a provider of additional modules that can be used for auto-import. Effectively, this means that packages that ship their own types should be available for auto-import immediately after npm installing them, lifting a long-despised limitation of “you have to write out the import manually once in your project before auto-imports will work, unless it’s in @types, because @types is special.”
Performance
I compared a few operations against master on the output of @angular/cli (with routing). (I chose this as a test because it’s a realistic scenario where a user’s package.json will contain more dependencies with self-shipped types than are actually used in the program. For example, @angular/forms is a listed dependency but isn’t used in the boilerplate.) Measurements are an average of three trials run in VS Code 1.46.0-insider (b1ef2bf) with extensions disabled.

master
PR
Diff

Startup time (ms)
2729
2636
-92 ms (-3%)

Type “P”*

updateGraph
18
15
-4 ms (-20%)

completionInfo
102
115
+13 ms (+13%)

Backspace, Type “P”**

completionInfo
44
38
-6 ms (-13%)

Comment out import***

getApplicableRefactors
44
62
+18 ms (+42%)

* This triggers completions, including import suggestions for PatternValidator from @angular/forms in the PR trials, a suggestion that is not included in the master trials.
** This triggers completions again without disrupting the auto-import cache populated in the previous action.
*** This changes the structure of the primary program, which triggers an update of the auto-import provider program. The time it takes to do so is included in the subsequent getApplicableRefactors measurement.
Observations:

Editing operations that don’t change the structure of the program are unaffected (looking at updateGraph time)
When generating completions, there’s an added cost associated with looking through more modules and generating auto-import suggestions for them. The penalty isn’t huge, though, and seems to be worth it for the added utility.
When something happens that triggers an update of the auto-import provider, there’s an associated cost that scales with the number of typed dependencies in package.json files that are not already in the main program (i.e., the size of the auto-import provider program). Because the Angular CLI generates boilerplate that has more unused typed dependencies than most real codebases (normally when you install a dependency, you import it shortly thereafter), I suspect that it will be rare for users to see much more delay than the 18 ms measured here.

Limitations

Only package.jsons found in the project’s currentDirectory and upward are considered. So in an unusual project structure like
tsconfig.json
project/
  node_modules/
  package.json
  index.ts

the project/package.json file won’t be found.

This doesn’t currently resolve to JavaScript files—that is, to get auto imports, the package has to have .d.ts files. Processing plain JS for auto-imports could be explored if demand were high, but when I tried it, most of the added packages were CLI tools and plugins, not things people want to import from, so it was a performance hit for little (or even negative) utility.

Memory considerations
The auxiliary program is configured to be as light as possible, but in a project references editing scenario, it’s possible to have many projects open at once. Each project has its own language service, so each open project will typically have one of these auxiliary programs. Some possibilities for mitigating high memory usage in these scenarios: (updated: decided to exclude devDependencies for UX reasons)

The auxiliary program won’t be created if there are no packages listed that aren’t already in the main program, but an analysis of around 500 popular open source TypeScript repos shows that this rarely happens, largely due to devDependencies that ship their own types but are never used in the program (prominent example: typescript). Excluding devDependencies significantly increases the chances that creation an auxiliary program can be skipped, at the cost of those devDependencies being unavailable for auto-import (which can be useful when writing build scripts and tests).
The auxiliary program isn’t created until there’s an open file that belongs¹ to the project. This prevents us from creating these programs as we do a cross-project find-all-references.
If we detect that memory usage is a problem, we could dispose or stop creating these programs. I think @sheetalkamat has looked into this in the past.
We could either hard-code into TypeScript or indicate in third-party projects that a package should opt out of being eagerly auto-importable. In one real-world project I tested, the contents of the auxiliary programs were limited to typescript, typescript-eslint, ts-jest, jest, and prettier, none of which are commonly imported.

However, at this point I’m not convinced that any of these mitigations are necessary. The size of these programs is usually in the ballpark of a browser tab (around 50 MB from one early test). But we have avenues to explore if needed.
Fixes #37812
Fixes #36042

¹ “file that belongs to the project” here means “file whose default project is the project in question,” in contrast to “file that is contained by the project”

@typescript-bot pack this

After further thought, I do think devDependencies should be excluded by default (we could add a configuration option to include them if enough people really want them). Including them by default means that virtually every project written in TypeScript would get auto-imports for the compiler, which nobody wants. If you do want to import the compiler API, typescript is either going to be in your "dependencies" or "peerDependencies"—and we should include peer dependencies by default.

@typescript-bot pack this

For anyone else buddy testing this, here's a list of packages that include their own types: https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/notNeededPackages.json

The default is exclude-dev, since all will contain a lot of noise for most users (e.g., anyone who has typescript as a devDependency would get auto-imports from the compiler). In fact, I’m not even sure that all needs to be exposed in VS Code’s UI, but it was trivial to add on our side.

After further thought, I do think devDependencies should be excluded by default (we could add a configuration option to include them if enough people really want them). Including them by default means that virtually every project written in TypeScript would get auto-imports for the compiler, which nobody wants. If you do want to import the compiler API, typescript is either going to be in your "dependencies" or "peerDependencies"—and we should include peer dependencies by default.

sometimes we do want to auto import in devDependencies, for example, some test only package. is there some better way to avoid the "typescript" problem? maybe some kind of filter?

Once you manually import your devDependency once, it will be available for auto-import throughout the rest of your project. So... it's kind of already a reverse filter that you configure implicitly as you code.

Is there a way to specify which package.json to use for this?
I have a monorepo, structured like this:

apps

nextjs

packages

components

My package.json inside of apps/nextjs has all of my dependencies, such as react, etc.
However, VSCode only checks the root package.json of my monorepo, which has zero dependencies inside of it. As a result, none of the monorepo packages get autoimport.
Is there a way to set the setting to specify a packageJsonPath or something like that?
  "typescript.preferences.includePackageJsonAutoImports": "on",
  "typescript.preferences.packageJsonPath": "./apps/nextjs",

@nandorojo are you primarily a TypeScript or JavaScript user? (Or more to the point, is this particular project written in TypeScript?) The expected structure is that each package of a monorepo should have a tsconfig.json file. As a JS user, you can still have a tsconfig.json or jsconfig.json file there, and even just putting {} as its contents should be enough for package.json auto-imports to work. Note that we have an open issue for investigating inferring this structure for JS users without any tsconfig/jsconfig files, but it’s complicated: #45558.

@andrewbranch really appreciate the fast reply.
I'm a TS user exclusively. I tried adding a tsconfig.json to each packages/* folder as well, but no dice on autoimports.
Interestingly, autoimport does get solved if I duplicate my dependencies from apps/nextjs/package.json → root package.json's peerDependencies. But it's a definite antipattern to put dependencies in the root of a workspace like that.
I don't mind having a tsconfig in each packages/* and apps/* folder, but I don't think this solves it for me.
I am able to autoimport my monorepo packages (@beatgig/components does autoimport). But the dependencies from apps/nextjs, such as react, still don't autoimport inside of packages/*.
Let me know if you need me to clarify anything there, it might be a bit hard to follow.

Are you able to either share the project you’re working in or reproduce this in a little example? The only possible explanation I can think of besides this being a bug is that if we discover too many unused dependencies, we disable the package.json auto imports:

This would happen if you have a bunch of dependencies listed in a package.json but the project in question has not already imported them. You can try toggling this setting to on and see if that makes a difference. Otherwise, go ahead and open a new issue with repro steps. Thanks!

I did already set that to on inside of .vs-code/settings.json`, actually, but unfortunately it didn't help. I don't think I can share the current repo since it's private, but I'll try to reproduce it in one you can use and will post an issue. Thank you again!

@andrewbranch I have a repro here: https://github.com/nandorojo/vscode-monorepo-autoimport-repro
It's very minimal (these are all the dependencies):

Do you need me to open an issue, or is this sufficient? I added a few comments in the only two ts files that should give you what you need.

This is working as intended—react is not listed in any package.json visible to packages/components/index.ts. We filter out transitive dependencies from auto-imports because importing from packages you don’t explicitly depend on is fragile—see #30033 and #38768. You don’t need to add react to the root package.json, but it should be in packages/components/package.json. I believe yarn/npm/lerna will all hoist this to the root node_modules so the dependency isn’t duplicated, so there’s no reason not to specify it as a dependency.

I see, thanks for clarifying.
It works fine if I manually import it, though, so it's as if the autoimport doesn't match the actual behavior. Both the Next.js app and the type checker work without explicitly adding these dependencies to each package.
My hesitation with adding it to packages/components/package.json is that I'll have tons of duplicate items in each package.json when it isn't actually required, either at runtime or for types. I would only add these dependencies to each package to get auto-import to work.
Is there a way to possibly extend the dependencies from apps/nextjs? (My guess is there isn't, but just in case...)

I believe yarn/npm/lerna will all hoist this to the root node_modules so the dependency isn’t duplicated, so there’s no reason not to specify it as a dependency.

I agree that practically speaking, since they won't duplicate, there's no real downside to this.
But in a large monorepo with over a dozen packages/* folders and tons of dependencies, it's a pretty rough DX to copy every dependency into so many places, given that it's all only to get the auto-imports to work.
If there are no intentions to allow transitive dependencies (even behind a flag) for autoimports, then I'll just add each dependency to each subpackage. My guess is, you've thought about the downsides more than I have. But I thought I would leave my thoughts on the DX.

I did try explicitly adding these to my monorepo packages, as recommended, and it does solve the autoimports.
However, to prevent duplicate dependencies, I have to fix the versions in packages/components/package.json to the same exact ones in apps/nextjs/package.json. If I don't do this, then my lockfile will reflect an additional dependency.
I was hoping this would result in no changes to my lockfile, but it seems it does:
{
  "name": "@repo/components",
  "version": "0.0.0",
  "main": "index.ts",
  "devDependencies": {
    "moti": "*",
  }
}

For example, moti now has 0.16.1 released. Originally, apps/nextjs had version 0.16.0. But now that I added it to @repo/components, I now have both versions installed:

I can put versions there explicitly instead to avoid duplication, but it makes updating a dependency a bit of a hassle. I now have to increment the version in 15+ folders each time I want to update.
I know this is intended, but I figured it would be useful to share the downside. Thanks again!

I think package/workspace managers have tools for doing these kind of bulk install operations. I also think that if you’re not publishing these packages individually, but rather the monorepo structure is just for code organization, there’s no great downside to putting common dependencies in the root package.json. And if you are publishing each package individually, then I think it is an antipattern to import transitive dependencies. There are people who vehemently disagree with me on that point in #38768, but I’m pretty firm on it.

I also think that if you’re not publishing these packages individually, but rather the monorepo structure is just for code organization, there’s no great downside to putting common dependencies in the root package.json.

Interesting, good to know. I'm only concerned for the times that I'm using this to organize shared packages across our internal apps. When it comes to open source monorepos I publish, I do in fact copy over the dependencies in each package. But for an app with code organization, I don't know why importing transitive dependencies would really be an issue. I suppose I'll just put the deps in the root package.json, and hopefully that works fine across the apps themselves.

I think package/workspace managers have tools for doing these kind of bulk install operations.

Maybe lerna does this, I've only used it for publishable packages so I'm not sure. If there aren't downsides to using the root package on internal projects, I might just do that. But yarn workspaces warns against it whenever you install something on the fly, so I figured it wasn't a good pattern.


[Translator] Filter dumped locales

Currently, the translation dumper dumps all translations for all locales. However, in some projects, we have a list of locales we support and we don't care about others (which might still have keys defined for the third-party translations).
It would be great to filter the list of dumped translations based on the kernel.enabled_locales parameter defined in FrameworkBundle.

Hi stof,
Maybe I'm wrong, but in a project with framework.enabled_locales set, I don't seem to have all locales dumped, only the ones defined in this configuration key.

Hi, this feature is already presents, it uses the TranslatorBagInterface#getCatalogues() as source of truth (https://github.com/symfony/ux/blob/262db2b22730ba8a5c628055e7d83e6c8a9f0c06/src/Translator/src/CacheWarmer/TranslationsCacheWarmer.php#L39), but indeed there is not functional tests ATM.
I can try to add one asap.

Tests mentionned in https://github.com/symfony/ux/issues/1226#issuecomment-1776938061 still need to be added I guess


feat: Introduce changed-files GitHub action.

Remarks:

Add "changed-files" action to CI workflow
Configure action to output "definitons//.yaml" files
Add lines to shell script to print changed files.

The idea here is to add just enough to test functionality.
IF the external script is able to print the env ALL_CHANGED_FILES during
a merge-commit, rebase-commit, and squash-merge then we proceed. Meaning
another PR will follow-up to finish integration.
Jira: hacbs-692
Signed-off-by: Jon Disnard<EMAIL_ADDRESS>

I am a little confused as to why the docs show as part of this PR, but the changes to tkn_push-bundles.sh and tekton_bundle_push.yaml lgtm

Ah, Good catch!
That was probably caused by git rebase origin/main on my local branch, which is a habit of mine before pushing new changes. I ran another git rebase origin/main and it had merge conflicts, that I resolved. Hopefully that fixed it.

As this is still a testing PR, one thing you can do is to modify the workflow so it doesn't require a push to main. That way you can fully test it within this PR without having to merge it.

Good idea!
So I tried that, and it worked.
I modified a few bundle definitions in trivial ways, and let the action run on this feature branch.
You can see the result here:
https://github.com/hacbs-release/release-bundles/runs/6902732290?check_suite_focus=true
(This result might be garbage collected in the future)
The output log shows my test strings:
...
definitions/hello-release/hello-release.yaml was changed
definitions/hello-world/hello-world.yaml was changed
...

So I went ahead and just removed those extra testing lines from the script, and restored the branch to it's previous state to minimize the diff.
PTAL

Nicely done. I’ll go through it tomorrow and add my review then.

The docs appear here as the branch is not rebased (it has conflicts).
Lets not merge test stuff into main if that's not required though. The change is ok and I like that you (Jon) are using an action to detect changed files (if it works for your workflow), but let's keep it limited to this change whenever possible. If it's actually necessary to merge it to main, ask for a review and any of us will approve it.

This change is the first-part of a two-part effort to re-allow "merge commits" to the main branch. Previously our CI was silently failing due to the use of merge commits, rather than squash merge. The action makes it easier, but it's possible to have done the same kind of things directly in the workflow, but I'd rather reduce complexity.
Once this PR is merged, we can update the script in another PR, and finish-up.
PTAL

I think we can use this same PR (changing its scope) for applying all the necessary changes to get the CI fixed.
I’m also ok with merging this one and work in a separate one. In this case I’d update the PR description and commit as you removed some stuff. Also, if you go for this option, you can create a branch on top of this one to not get stuck until it’s reviewed and rebase it once the PR gets merged.

I think we can use this same PR (changing its scope) for applying all the necessary changes to get the CI fixed.
I’m also ok with merging this one and work in a separate one. In this case I’d update the PR description and commit as you removed some stuff. Also, if you go for this option, you can create a branch on top of this one to not get stuck until it’s reviewed and rebase it once the PR gets merged.

Okay, I've updated the PR with a few minimal changes to the script.
I'll also update the PR to reflect the change.
Thanks!

The PR lgtm. I see a change in the <> in the README.md file, but I think that is just a rebase issue.

Thank!
Yeah I'm not 100% clear why there was a rebase issue, but I DO rebase my feature branches overtop main on the regular. :man_shrugging:


Independent get_analytics method

📥 Pull Request
References #437 by using the latest commits.
📘 Description

Adds a separate get_analytics method to enable its usage independently of the session lifecycle
Adds helper methods for the get_analytics method - _format_duration and _format_token_cost for formatting needs
Refactored the response fetching code into the get_response method to return a Response object

🧪 Testing

Tested for both single and multiple session(s) with the correct formatting and calculations
Ensured that no additional requests are made using the get_response method

When I was writing the documentation, I figured that the current implementation was not effective, since it required a Response object from the user to get the analytics.
Since it is one of the attributes that a user doesn't need to know about, I refactored the code. The code is more readable and modular, and allows the user an easy way to fetch the analytics.


Make diff aware of Versioned Resources

If I switch a config map over to being a versioned resource using the kapp.k14s.io/versioned annotation
When I do a diff I get a 'delete' + "create" diff showing every old line removed and very new line added.
Ideally the diff system would understand that this is a versioned resource and diff the old version vs the new version so I can see the actual changes

I just discovered it only does this for the first time when you convert to a versioned resource.


Definition of variables for email

Description
I want to set the variables so that the boxes "Require Email Conformation to Register" and "Enable Email Notifications" are checked automatically.
I defined in a .env file in docker the following:
GITEA__service__REGISTER_EMAIL_CONFIRM="true" -> Is "Require Email Confirmation to Register" ?
GITEA__service__ENABLE_NOTIFY_MAIL="true" -> Is "Enable Email Notifications" ?
Screenshots
Checkboxes:

Gitea Version
Docker image: gitea/gitea:1.17.3-rootless
Can you reproduce the bug on the Gitea demo site?
No
Operating System
Host Docker: Ubuntu 22.04
Browser Version
Mozilla Firefox 106.0.1 (64 Bits) e Google Chrome 106.0.5249.103 (64 Bits)

To understand you better
This is on the install page?
If yes, then why not just use a default app.ini and then there is no install step needed
If no, I don't know what you mean with this
I'm not by any means a member of the team or something, I'm just curios to know the problem


Downloading dependency: sync 1.0.0-BETA-5.1 error

Realm version: 2.1.2
Xcode version: 8.2.1
iOS/OSX version: 10.12.2
CocoaPods version: 1.2.0.beta.3

"pod install" integration error.

Thank you for getting in touch with us!
Are you located in Mainland China? Sometimes our users from there have trouble downloading the prebuilt binary files. Please take a look at this thread for some potential workarounds. If you have any more questions, please post again to this ticket. Thanks!

@jpsim
Yes, I am a Chinese……

I'm sorry for the inconvenience. Please refer to https://github.com/realm/realm-cocoa/issues/2713 for more information.


Fix tabs font size. On small screen, tabs text was tiny.

@Joxit Thank you! Great work!

You are welcome :)


[Enhancement]:  Chrome_Dinosaur_Game scoring section updatition

Is your enhancement request related to a problem? Please describe.➕
Game enhancement:

scoring section will be in the middle position.

Describe the solution you'd like
Solution steps:

update the js file

Select program in which you are contributing
GSSoC23
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @samratghosh291 !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

/assign

Hey @samratghosh291 ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


Do not allow to connect to an IRPMon server instance with different "bitness"

Actually, the client disconnects (fails its initialization) if it detect that the server "bitness" differs. Such an approach permits us to make the client independent of server "bitness" in the future, without changing the server side. Whether such a future modification is wise necessary or desirable, is another question.


Your Project

Hi G5 team,
I am curious about your project. May I know which classical mechanics problem you guys are working on?

@s4il3sh We are going to work on projectile motion with air resistance. What about you?

We are working on finding the spring constant and the energy stored in it.

Sounds, good.


Does this work with the latest Cocoapods v1.x?

After upgrading to Cocoapods v1.x this doesn't seem to work anymore.

[!] Unknown command: open
Did you mean: spec?

Hi @lemonkey,
Probably not. If you have a fix a PR would be awesome.

Open


[SPARK-38760][PYTHON][SQL][SS] Add DataFrame.observe(str) for Structured Streaming in PySpark

What changes were proposed in this pull request?
This PR adds the support of Structured Streaming at DataFrame.observe in PySpark. This is the same support with the Scala side.
Why are the changes needed?
For SS users in PySpark to easily monitor their streaming queries.
Does this PR introduce any user-facing change?
Yes. After this PR, PySpark users can do:
from pyspark.sql.functions import count, col, sum, lit
from pyspark.sql.streaming import StreamingQueryListener

class MyListener(StreamingQueryListener):
    def onQueryStarted(self, event):
        pass
    def onQueryProgress(self, event):
        row = event.progress.observedMetrics.get("metric")
        # Do something with the metric, e.g.) row.cnt and row.sum
        print("Count: %s" % row.cnt)
    def onQueryTerminated(self, event):
        pass

spark.streams.addListener(MyListener())

df = spark.readStream.format("rate").option("rowsPerSecond", 10).load()
observed_df = df.observe("metric", count(lit(1)).alias("cnt"), sum(col("value")).alias("sum"))
observed_df.writeStream.format("noop").queryName("test").start()

How was this patch tested?
Manually tested with the example above, and unit test was added.

cc @HeartSaVioR @alex-balikov @hvanhovell @zsxwing, this PR is ready for a look. Please take a look when you find some time.

cc @belifer too

Thanks guys.
Merged to master.


State Pruning: add option to keep n best finalized blocks

This issue is a result of a discussion on substrate stackexchange https://substrate.stackexchange.com/questions/6994/state-pruning-archive-canonical-vs-archive-finalized/
Problem description
Currently in substrate there is a possibility to set state pruning to archive-canonical or number. In docs it states, that these values will cause to:

archive-canonical - Keep only the state of finalized blocks.
number - Keep the state of the last number of finalized blocks.

That said, when digging deeper there is a fundamental difference between canonical and finalized blocks. It is described in https://github.com/paritytech/substrate/blob/master/client/db/src/lib.rs
//! # Canonicality vs. Finality
//!
//! Finality indicates that a block will not be reverted, according to the consensus algorithm,
//! while canonicality indicates that the block may be reverted, but we will be unable to do so,
//! having discarded heavy state that will allow a chain reorganization.
//!
//! Finality implies canonicality but not vice-versa.

Currently state pruning works in such a way, that if best block is over 4096+n blocks on top of best finalized block, state of yet unfinalized blocks will be pruned by function force_delayed_canonicalize.
Because of that, it seems like there is no way to turn on pruning and always have access to state of current top n finalized block. For example in case of big finalization stall.
This has proven problematic for me, when adding pruning compatibility to our node. Inside our node binary, calling runtime_api for the state of best finalized block returned error. State was already pruned because of finalization stall. What I need is to access state of last finalized block, even if the node crashed/exited. This is needed for our finalization gadget.
Also if our node needs access to state of last finalized block, running it with pruning could irreversibly move DB to a state at which node would no longer be able to work. This could happen whenever there is a big finalization stall, and would require restoring DB to some previous state and syncing it once again.
Request
Could we add functionality to keep state of top n finalized blocks while pruning?
Also right now description of pruning arguments in CLI is misleading and can cause some errors due to not knowing, how pruning works under the hood. It states that finalized blocks are kept, when it should state that canonicalized blocks are kept.
Possible solution
When reading substrate code, I'm not sure what is the reason for calling force_delayed_canonicalize function. I understand that it is to make DB smaller, faster, and be sure it never exceeds 4096 states. Still, seems to me like not calling it should not break anything in case of finalization stall, and it should still be faster, then archive DB.
So if the default state of a node is without any finalization stall, maybe call to this function could be just removed? This would give a much stronger and useful guarantee of keeping top finalized blocks vs top canonicalized blocks.
Alternatively, keep n finalized CLI argument could be added and when it is provided, force_delayed_canonicalize would not be called. Something similar to how StateDb::last_canonicalized returns LastCanonicalized::NotCanonicalizing for --state-pruning archive here:
https://github.com/paritytech/substrate/blob/129fee774a6d185d117a57fd1e81b3d0d05ad747/client/db/src/lib.rs#L1407

The main reason for limiting the length of uncanonicalized chains is that it requires O(n) memory upkeep. Current implementation stores some data in memory for each unfinalized/uncanonicalized block. Substrate is not really designed for long forks. A 4096 block re-org is problematic on quite a few levels.
The particular issue with unvailable state is that state-db counts blocks from the last canoncialized rather than last finalized when checking against the pruning window size.  Default pruning window size is 256. So after there are 4096+256 forced canonicalizations, the pruning mechanism would discard the best finalized state. This should be easy enough to fix.
You can work around by seeting a bigger pruning window though. It is not limited to 4096 blocks and can be arbitrary large.
BTW substrate has a mechanism to prevent long infinalized chains. If finality stalls for some reason, block production should slow down exponentially. See here: https://github.com/paritytech/substrate/blob/414caffad1ce634352676d1884942282a8ec6ebb/client/consensus/slots/src/lib.rs#L166-L168

The main reason for limiting the length of uncanonicalized chains is that it requires O(n) memory upkeep. Current implementation stores some data in memory for each unfinalized/uncanonicalized block. Substrate is not really designed for long forks. A 4096 block re-org is problematic on quite a few levels.

But there is no force canonicalization for ArchiveAll DB. I'm not sure what is the difference here, and why in this case we can deal with O(max_stall_difference+n) memory upkeep. Is this memory used to keep blocks that are to be pruned? If so, then we do not really have a choice here if we want to always have access to last finalized state and we need to keep O(max_stall_difference+n) entries.

The particular issue with unvailable state is that state-db counts blocks from the last canoncialized rather than last finalized when checking against the pruning window size. Default pruning window size is 256. So after there are 4096+256 forced canonicalizations, the pruning mechanism would discard the best finalized state. This should be easy enough to fix.

Not sure if I understood this correctly. Do you mean this check?
https://github.com/paritytech/substrate/blob/129fee774a6d185d117a57fd1e81b3d0d05ad747/client/state-db/src/lib.rs#L409
If so, then state-db would need to distinguish canonicalized from finalized blocks, right? And this would also require keeping O(max_stall_difference+n) entries in DeathRowQueue.

I'm not sure increasing pruning window is a good idea. This would only make the problem less probable, not fix it. Also the DB would need to be bigger.
We are using backoff strategy but it does not guarantee not importing new blocks, when the gap is too big. So this problem can still appear when syncing, or when we have other nodes who are malicious in the network and do not respect this strategy.

BTW substrate has a mechanism to prevent long infinalized chains. If finality stalls for some reason, block production should slow down exponentially. See here:
https://github.com/paritytech/substrate/blob/414caffad1ce634352676d1884942282a8ec6ebb/client/consensus/slots/src/lib.rs#L166-L168

Thanks for the response. I just wanted to jump in with a related question that was bugging for quite some time already.
As @maciejnems mentioned, this is not a protocol-level defense. If there is even a single validator that replaces should_backoff with a function returning false every time, then this becomes an attack vector. Consider the following scenario:

There is a finalization stall at block n. Suppose also that a short fork happened: two variants A and A' of block number n+1.
Because there is a validator (or a few of them) that is not respecting the backoff strategy (there is nothing which prevents them from doing that), you cannot guarantee that the number of not finalized blocks is smaller than some bound, in this case 4096 is relevant. Suppose that the long branch is built upon the block A.
When the gap is very large, say >5000 blocks, GRANDPA comes back to life and finalizes A', which would cause >4096 to be reorged -- which does not even seem possible because of canonicalization? (At least for nodes that use pruning). Not sure what would even happen in this case.

Do you consider this scenario unlikely? If so, which part: 1, 2, 3? From my expertise in consensus protocols, I can confidently say that 3) is actually quite likely (maybe around ~20% odds for finalizing the "wrong fork" if the forks are there in the described configuration). The reason it can happen is, intuitively, because GRANDPA can get stuck (for instance >1/3 validators become unavailable for some reason) just after the validators commit to A' but before A' is finalized. But after it unstucks, A' will be finalized no matter what. The fact that this can happen is a general property of protocols that are safe under asynchrony, which GRANDPA is an example of.

@andresilva Could you comment on the above perhaps?

One more thing I noticed is that when using ParityDB, there seems to be no problem with in memory cache, right? References are kept in DeathRowQueue::DbBacked.


including default value for ADD argument in webcrypto aes-gcm algorithm

Solves https://github.com/paulmillr/noble-ciphers/issues/33

We shouldn't pass uint8array(0). Instead, delete this property from the object when it's not present


Ignore CAPI/CAPA env var image overrides for 4.12 and later

What this PR does / why we need it:
Starting with 4.12, the CAPI/CAPA images we use should come from the release payload. In an MCE deployment, we should ignore env vars set by MCE for those images, but only if the release payload is >= 4.12.
Checklist

[x] Subject and description added to both, commit and PR.

/lgtm


Duplicate symbols preventing launch on iOS

Hey,
for days and hours I've been trying to fix this issue now, but I've run out of things to try to be honest. I try to flutter run but it gives me this output:
Launching lib/main.dart on iPad in debug mode...

Automatically signing iOS for device deployment using specified development team in Xcode project: XXXXXXX
Running pod install...                                              5.0s
Running Xcode build...
 └─Compiling, linking and signing...                         3.5s
Xcode build done.                                           15.3s
Failed to build iOS app
Error output from Xcode build:
↳
    ** BUILD FAILED **

Xcode's output:
↳
    7 warnings generated.
    In file included from /Users/username/Documents/Flutter/flutter/.pub-cache/hosted/pub.dartlang.org/firebase_crashlytics-0.2.1/ios/Classes/FLTFirebaseCrashlyticsPlugin.m:7:
    /Users/username/Desktop/projectgit/project_flutter/ios/Pods/Headers/Public/Firebase/Firebase.h:75:10: warning: "FirebaseAnalytics.framework is not included in your target. Please add `Firebase/Analytics` to your Podfile or add FirebaseAnalytics.framework to
    your project to ensure Firebase Messaging works as intended." [-W#warnings]
            #warning "FirebaseAnalytics.framework is not included in your target. Please add \
             ^
    1 warning generated.
    duplicate symbol '_AnnotateRWLockDestroy' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_ValgrindSlowdown' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateEnableRaceDetection' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateIgnoreWritesBegin' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateIgnoreReadsBegin' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateRWLockCreate' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateThreadName' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateBenignRace' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_RunningOnValgrind' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateIgnoreWritesEnd' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateIgnoreReadsEnd' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateMemoryIsUninitialized' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateMemoryIsInitialized' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateBenignRaceSized' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateRWLockReleased' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    duplicate symbol '_AnnotateRWLockAcquired' in:
        /Users/username/Desktop/projectgit/tflite_flutter_plugin/ios/TensorFlowLiteC.framework/TensorFlowLiteC(dynamic_annotations_c9bf866fe89c02b98f86dda9d34be0c4.o)
        /Users/username/Desktop/projectgit/project_flutter/build/ios/Debug-iphoneos/abseil.framework/abseil(dynamic_annotations.o)
    ld: 16 duplicate symbols for architecture arm64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    note: Using new build system
    note: Building targets in parallel
    note: Planning build
    note: Constructing build description
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'GoogleUtilities' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'nanopb' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'Reachability' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'Protobuf' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'PromisesObjC' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'GoogleDataTransport' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'webview_flutter' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'shared_preferences' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'rate_my_app' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'permission_handler' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'path_provider' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'device_info' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'keyboard_visibility' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'connectivity' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'GoogleDataTransportCCTSupport' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseCoreDiagnostics' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseCore' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseInstallations' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseInstanceID' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseCrashlytics' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseMessaging' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'firebase_core' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'firebase_messaging' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'firebase_crashlytics' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseCoreDiagnosticsInterop' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseFirestore' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'Flutter' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'FirebaseAnalyticsInterop' from project 'Pods')
    warning: The iOS deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. (in target 'Firebase' from project 'Pods')

Could not build the precompiled application for the device.

Error launching application on iPad

My flutter doctor output:
[✓] Flutter (Channel stable, 1.22.0, on Mac OS X 10.15.6 19G71a, locale en-GB)
    • Flutter version 1.22.0 at /Users/username/Documents/Flutter/flutter
    • Framework revision d408d302e2 (6 days ago), 2020-09-29 11:49:17 -0700
    • Engine revision 5babba6c4d
    • Dart version 2.10.0

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from:
      https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK
      components.
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup
      for detailed instructions).
      If the Android SDK has been installed to a custom location, set
      ANDROID_SDK_ROOT to that location.
      You may also want to add it to your PATH environment variable.

[✓] Xcode - develop for iOS and macOS (Xcode 12.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.0.1, Build version 12A7300
    • CocoaPods version 1.9.3

[!] Android Studio (not installed)
    • Android Studio not found; download from
      https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/macos#android-setup
      for detailed instructions).

[✓] Connected device (1 available)
    • iPad (mobile) • 6b98f33877e0cab838d2ffafbc6aec62c7e4b98b • ios •
      iOS 14.0

! Doctor found issues in 2 categories.

I've tried literally every cleaning thing (flutter clean, pod deintegrate + pod setup, xcode clean project, ...) and nothing helped, so it doesn't seem to be corrupt build files. I also tried the solutions proposed in #18, but no luck there either.
I'd be very grateful for any ideas on how to resolve this.

Just to make sure that it really is this plugin causing the issue, I removed the dependency and its usages in the code. The build works just fine then.
It used to work for me, so I'm not sure what's causing the issues all of the sudden. I've tried downgrading to tflite_flutter 0.3.0 but that still leads to the same error. Another thing that's changed in the time since it last worked is that I updated to iOS 14 and xcode 12. I guess that's the most likely culprit as everything is working fine on Android.

It used to work fine for me.
Do you mean that after having tflite_flutter and all firebase deps it used to work?
If that is the case then issue might of build conflicts. If you are uploading code to github, try cloning your Repository to a new folder and try building from there.

Good point, it definitely used to work with Firebase cloud messaging. I'll try to remove everything else from Firebase and see if I can build it again then. Thanks for the suggestion!

Hmm, just making a fresh clone didn't help. But now I removed everything (even firebase cloud messaging) from firebase and got it to run :) Does that mean this plugin is simply not compatible with firebase plugins?
Thanks a lot already for your help!

Well, no real fix, I just removed all traces of firebase projects and then added FCM again. FCM and tflite are working together now, crashlytics isn't though, so I'll try a different error reporting system.
Thanks again Amish!

Few errors might be because tflite_flutter declares iOS version to be 8.0 https://github.com/am15h/tflite_flutter_plugin/blob/master/ios/tflite_flutter.podspec#L18. While some firebase plugins require 10.0. You can try changing
s.platform = :ios, '8.0' to s.platform = :ios, '10.0'
and
s.ios.deployment_target = '9.0' to s.ios.deployment_target = '10.0'
in https://github.com/am15h/tflite_flutter_plugin/blob/master/ios/tflite_flutter.podspec. See if that works.

Thanks for the tip, maybe I'll try it later! I've already set up Sentry instead of Crashlytics now, which seems to be working well with less complex setup than firebase :)

I need Crashlytics. Removing firebase plugins isn't a fix. That's rubbish


How to list related content below a page

I've added a field category to the metatabs. How can I list all articles in the same folder which have the same value for that field? Something like item.thisChapter.folderContent does not seem to exist.

Not sure if I totally understand your requirement: If you are on a "folder" page that contains sub-page, then you have access to all sub-pages with item.folderContent. You can list the pages of that folder like that (implemented in several themes):
	{% for element in item.folderContent %}

		{% set post = getPageMeta(settings, element) %}

{{ post.meta.title }}

{{ post.meta.description }}

	{% endfor %}

You can also query the category of the post with an if statement there. Not sure if that is enough for your case.
For a real category or tag feature, you have to create a plugin that creates an index file with all categories/tags and all pages, so it is a bit more complicated. Such a plugin is on my roadmap, but I won't find time in near future/this year.
Hope that helps?

Thanks alot. For a folder I already know how to get the subpages. I want to access the folderContents when I'm viewing a single file.

Ah, ok, makes sense. This.chapter has no folder content, it is only for paging and adding the folder-content would blow up the object in many cases.
But it should still be possible with the navigation and the getPageList-method. I have to check it when I find time, but it should work like this:
  {% set pagelist = getPageList(navigation, urlToFolder, base_url) %}

urlToFolder should be something like item.thisChapter.urlRel
Will check it in the next days and answer here.

Hi and thanks again. That was very helpful. I already found the Function getPageList in the Twig docu but it seems to behave different in Typemill. This code seems to work:
{% if item.thisChapter and metatabs.meta.category1 is not null %}
	{% set pagelist = getPageList(navigation, item.thisChapter.urlRel, base_url) %}

Related content for {{ metatabs.meta.category1 }}

	{% for element in pagelist.folderContent %}
		{% set post = getPageMeta(settings, element) %}
		{% if post.meta.category1 == metatabs.meta.category1 or post.meta.category2 == metatabs.meta.category1 %}

			{{ post.meta.title }}

		{% endif %}
	{% endfor %}

{% endif %}

glad you found a solution!


iOS sdk cleanup

Cleaned up some configuration problems in with the CMakeLists. Seems they were never checking for the libraries, instead relying on the platform to just have them and failing to build when unavailable.

@swift-ci Please test

Make some of the changes then do a smoke test! 👯♂

@swift-ci Please smoke test

Looks good except for the open issue around CLANG_TOOL_DRIVER_BUILD, which someone should actually investigate.

I believe the macOS failures are also happening on master, so I'm not going to retrigger just yet.

@swift-ci Please smoke test


[red-knot] Support for TOML configs in Markdown tests

Summary
This adds support for specifying the target Python version from a Markdown test. It is a somewhat limited ad-hoc solution, but designed to be future-compatible. TOML blocks can be added to arbitrary sections in the Markdown block. They have the following format:
```toml
[tool.knot.environment]
target-version = "3.13"
```

So far, there is nothing else that can be configured, but it should be straightforward to extend this to things like a custom typeshed path.
This is in preparation for the statically-known branches feature where we are going to have to specify the target version for lots of tests.
Test Plan

New Markdown test that fails without the explicitly specified target-version.
Manually tested various error paths when specifying a wrong target-version field.
Made sure that running tests is as fast as before.

I'd remove the tool.knot prefix for a more concise configuration similar to the format we'd use when configuring Red Knot in a knot.toml

The documentation in https://github.com/astral-sh/ruff/tree/main/crates/red_knot_test#configuration suggested that we want these blocks to contain a pyproject.toml-like config with a [tool.knot] section. I'm personally happy with both solutions, I don't mind the additional tool.knot. prefix too much. Maybe we want to support both eventually (using file=knot.toml or file=pyproject.toml?)

The documentation in main/crates/red_knot_test#configuration suggested that we want these blocks to contain a pyproject.toml-like config with a [tool.knot] section. I'm personally happy with both solutions, I don't mind the additional tool.knot. prefix too much. Maybe we want to support both eventually (using file=knot.toml or file=pyproject.toml?)

I'd prefer to default to a knot.toml configuration and only use the pyproject.toml layout when file=pyrpoject.toml (if there's even a use case for it because Carl argued in his proposal that end-to-end tests might be a better fit outside of mdtests and resolving options from the pyproject.toml (like requires-python) seems more like an end-to-end test case

Okay, I collapsed it down to a knot.toml-like configuration for now and adapted the documentation. If someone wants to see the original version, the first commit is still there.

I think it shouldn't be too hard to make this hierarchical as mentioned in the design doc

You're right. I didn't realize that we could simply change the Python version using Program::get(…).set_target_version(…).to(…) (thanks Micha), and didn't want to re-create the salsa DB for every test, because that resulted in a 3x slowdown.
Multiple (hierarchical) configurations are now fully supported. I verified that the whole test suite still runs in the same time as before.


Modify setup-java in ci workflow for v3

Following the update of setup-java to v3 the distribution is now required.
It's throw an error
Run actions/setup-java@v3
  with:
    java-version: 1.8
    java-package: jdk
    check-latest: false
    server-id: github
    server-username: GITHUB_ACTOR
    server-password: GITHUB_TOKEN
    overwrite-settings: true
    job-status: success
    token: ***
Error: Input required and not supplied: distribution

bors merge


Add key prop to each rendered item

The compontent needs updating to address the following warning:

Each child in an array or iterator should have a unique "key" prop. Check the render method of ReactReorderable. See http://fb.me/react-warning-keys for more information.

Ups..forgot the key property. The issue is fixed now!


Print asked question in the window EVEN if it is not sent yet

I just lost a long question due to some outage with Copilot servers. Once pressing , the question should ALWAYS be sent into the window. This allows for editing, history and is the UX I expect when I send a text somewhere.
Currently, I just send my text into nothingness and lost it :). Of course, I had clipboard'ed parts of the question, but not all. It's a bit silly to have to CTRL + A and CTRL + C before every time I use this feature out of fear to not lose everything I typed ;).

Sorry for the trouble, I'm hoping things are more stable now. It likely would have showed up in input history at least (up arrow).
And it's supposed to work in the way you describe, so I'm not sure what went wrong so that the request was not rendered.


[HOLD for payment 2023-06-29] [$250] Migrate AttachmentModal.js to function component

Class Component Migration
Filenames

[ ] src/components/AttachmentModal.js

type: React.PureComponent
has state values: true
has refs: false
has context: false
lifecycle methods: none

Task

We currently have some class components in our codebase that we would like to refactor to a function component.
Here's a link with some general advice on how to refactor a class component to a function component: https://react.dev/reference/react/Component#alternatives
If you need additional guidance, please ask in #expensify-open-source
Test for any regressions and verify that there are no breaking changes

Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01a7e287fa36d40c6a
Upwork Job ID:<PHONE_NUMBER>973076992
Last Price Increase: 2023-06-03

⚠ Could not update price automatically because there is no linked Upwork Job ID. The BZ team member will need to update the price manually in Upwork.

@yuwenmemon I can take this one if you don't have time.

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~01af33a55f4988a74b

I would love to take this

@yuwenmemon @marcochavezf @rushatgabhane @marcaaron
I think that no proposals is necessary for this issue, as it's straightforward.
can you assign me?

I want to work on this issue

Proposal
Please re-state the problem that we are trying to solve in this issue.
AttachmentModal.js is a class component, it needs to be converted to a function component according to modern React practices.
What is the root cause of that problem?
AttachmentModal.js was originally created as a class component.
What changes do you think we should make in order to solve the problem?
Replace render() function to return JSX format, convert this.state to useState hook variables, and conform to function component rules and practices.
What alternative solutions did you explore? (Optional)
x

I would like to take this

@yuwenmemon I can take this one if you don't have time.

@yuwenmemon you forgot to assign this issue to me

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/usmanawan

@rushatgabhane Looks like we have a lot of interest here. LMK who should be assigned, and I'll do that!

@rushatgabhane Looks like we have a lot of interest here. LMK who should be assigned, and I'll do that!

I want this issue

Thanks @marcochavezf
I applied to this job in Upwork.
I will try to finish this asap and let you know when PR is ready.

@marcochavezf  I didn't get an offer on upwork

@rayane-djouah as an fyi we may not always send an upwork offer as soon as you are assigned to the issue. But we'll always send one prior to when we need to issue payment.
I just sent you a $250 offer in Upwork though!

cc @marcochavezf @rushatgabhane for reviewing the PR

Assigned @allroundexperts as they will be reviewing PR here.

PR is still in review

@allroundexperts offer sent for $250!

This PR was merged more than 9 business days after assignment. But C+ approved on the 7th business day. The time after that was for our internal engineer to review. So we won't apply a penalty here.

All set to issue payment here!

@rayane-djouah $250 sent and contract ended!

@allroundexperts $250 sent and contract ended!

Upwork job closed.

Issue complete, and all payment issued. Closing as this is all set. Thanks everyone!