Search Engine Postgres Can't Index Big Records of Entitities

We were switching to Postgres Search Engine from LunrJS Search Engine because LunrJS is consuming high memory usage, we had around 40k entities across kind and previously LunrJS search engine can index all of this. After switching to Postgres Search Engine we realized that only ~24k records were indexed, thus resulting in limited search results found.
Expected Behavior

Postgres Search Engine should be able to index all the entity records.
Current Behavior

Postgres Search Engine can only able to index around ~24k records.
Possible Solution

Maybe some enhancement in how to get the entities or how to index big records. For the longer term, maybe need to separate this process out from backend service that serves the API, doing some queueing when indexing the documents.
https://github.com/backstage/backstage/blob/9c8ea7e24e49a090050c227288db950217e76ed9/plugins/search-backend-node/src/IndexBuilder.ts#L115
In here it's calling get all entities 2 times one for Catalog and one for TechDocs. Imagine it's querying a 40k records in single call and iterate it 2 times.
https://github.com/backstage/backstage/blob/2a320c21b4aadd1b7739d76da8d783b62b48f585/plugins/catalog-backend/src/search/DefaultCatalogCollator.ts#L94
https://github.com/backstage/backstage/blob/9255e143099beaabbea637c1c51c69b7d6bfe0ef/plugins/search-backend-module-pg/src/PgSearchEngine/PgSearchEngine.ts#L66
https://github.com/backstage/backstage/blob/2a320c21b4aadd1b7739d76da8d783b62b48f585/plugins/techdocs-backend/src/search/DefaultTechDocsCollator.ts#L98
A temporary solution for me was limiting total documents we index, but this is not a desirable outcome.
Steps to Reproduce

Import more than 30k entities into the catalog
Use postgres as the backend search engine
Wait some time to let the engine index the catalog
See table documents in db backstage_plugin_search

Context

Using the default LunrJS search engine was not suitable for production since it's using high memory usage for big datasets, moving to the Postgres search engine was a gradual step before moving to ElasticSearch. The indexing should can handle big datasets for this search engine.
Your Environment

OS:   Darwin 20.6.0 - darwin/x64
node: v14.18.1
yarn: 1.22.17
cli:  0.11.0 (installed)

Dependencies:
  @backstage/app-defaults                                  0.1.4
  @backstage/backend-common                                0.10.3
  @backstage/backend-tasks                                 0.1.3
  @backstage/catalog-client                                0.5.4
  @backstage/catalog-model                                 0.9.9
  @backstage/cli-common                                    0.1.6
  @backstage/cli                                           0.11.0
  @backstage/config-loader                                 0.9.2
  @backstage/config                                        0.1.12
  @backstage/core-app-api                                  0.4.0
  @backstage/core-components                               0.8.4
  @backstage/core-plugin-api                               0.5.0
  @backstage/errors                                        0.2.0
  @backstage/integration-react                             0.1.18
  @backstage/integration                                   0.7.1
  @backstage/plugin-api-docs                               0.6.22
  @backstage/plugin-app-backend                            0.3.21
  @backstage/plugin-auth-backend                           0.6.2
  @backstage/plugin-catalog-backend-module-ldap            0.3.9
  @backstage/plugin-catalog-backend                        0.20.0
  @backstage/plugin-catalog-common                         0.1.0
  @backstage/plugin-catalog-graph                          0.2.6
  @backstage/plugin-catalog-import                         0.7.9
  @backstage/plugin-catalog-react                          0.6.11
  @backstage/plugin-catalog                                0.7.8
  @backstage/plugin-explore-react                          0.0.10
  @backstage/plugin-explore                                0.3.25
  @backstage/plugin-kubernetes-backend                     0.4.3
  @backstage/plugin-kubernetes-common                      0.2.1
  @backstage/plugin-kubernetes                             0.5.4
  @backstage/plugin-org                                    0.3.34
  @backstage/plugin-pagerduty                              0.3.22
  @backstage/plugin-permission-common                      0.3.1
  @backstage/plugin-permission-node                        0.3.0
  @backstage/plugin-permission-react                       0.2.2
  @backstage/plugin-proxy-backend                          0.2.15
  @backstage/plugin-scaffolder-backend-module-cookiecutter 0.1.8
  @backstage/plugin-scaffolder-backend                     0.15.20
  @backstage/plugin-scaffolder-common                      0.1.2
  @backstage/plugin-scaffolder                             0.11.18
  @backstage/plugin-search-backend-module-pg               0.2.3
  @backstage/plugin-search-backend-node                    0.4.4
  @backstage/plugin-search-backend                         0.3.0
  @backstage/plugin-search                                 0.5.5
  @backstage/plugin-shortcuts                              0.1.19
  @backstage/plugin-tech-radar                             0.5.2
  @backstage/plugin-techdocs-backend                       0.12.3
  @backstage/plugin-techdocs                               0.12.14
  @backstage/plugin-user-settings                          0.3.16
  @backstage/search-common                                 0.2.1
  @backstage/techdocs-common                               0.11.3
  @backstage/test-utils                                    0.2.2
  @backstage/theme                                         0.2.14
  @backstage/types                                         0.1.1
  @backstage/version-bridge                                0.1.1

Thank you for merging this PR! https://github.com/backstage/backstage/pull/7239
I think it still not solving my problem here. I found interesting logs:
2022-03-11T08:24:19.499Z search error Collating documents for software-catalog failed: error: delete from "documents" where "type" = $1 and "hash" not in (select "hash" from "documents_to_insert") - canceling statement due to statement timeout type=plugin
...
2022-03-14T10:29:22.045Z search info Collating documents for software-catalog succeeded type=plugin
...
2022-03-14T10:30:41.144Z search error Collating documents for techdocs failed: error: delete from "documents" where "type" = $1 and "hash" not in (select "hash" from "documents_to_insert") - canceling statement due to statement timeout type=plugin

Sometimes it works, sometimes it's not. I think this is what causes the entities is not indexed properly.
I do check my postgres setting parameter statement_timeout, previously I had setting in 15s, and increased it to 60s, but no luck. The error still happening.
Any hints or clues to find out the possible cause of this problem? Is anyone know how to enable the raw query logging in backstage?

There are a lot of updates in the search module related, from the initial issue I created. Here is my updated dependencies used.
OS:   Darwin 20.6.0 - darwin/x64
node: v14.18.1
yarn: 1.22.17
cli:  0.15.3 (installed)

Dependencies:
  @backstage/app-defaults                                  0.2.0
  @backstage/backend-common                                0.12.1
  @backstage/backend-tasks                                 0.1.10
  @backstage/catalog-client                                0.8.0
  @backstage/catalog-model                                 0.11.0, 0.12.1
  @backstage/cli-common                                    0.1.8
  @backstage/cli                                           0.15.1
  @backstage/config-loader                                 0.9.6
  @backstage/config                                        0.1.15
  @backstage/core-app-api                                  0.6.0
  @backstage/core-components                               0.9.0, 0.9.1
  @backstage/core-plugin-api                               0.8.0
  @backstage/errors                                        0.2.2
  @backstage/integration-react                             0.1.24
  @backstage/integration                                   0.8.0
  @backstage/plugin-analytics-module-ga                    0.1.13
  @backstage/plugin-api-docs                               0.8.1
  @backstage/plugin-app-backend                            0.3.28
  @backstage/plugin-auth-backend                           0.12.0
  @backstage/plugin-auth-node                              0.1.4
  @backstage/plugin-catalog-backend-module-ldap            0.3.15
  @backstage/plugin-catalog-backend                        0.23.1
  @backstage/plugin-catalog-common                         0.2.1
  @backstage/plugin-catalog-graph                          0.2.13
  @backstage/plugin-catalog-import                         0.8.4
  @backstage/plugin-catalog-react                          0.8.1
  @backstage/plugin-catalog                                0.9.1
  @backstage/plugin-explore-react                          0.0.14
  @backstage/plugin-explore                                0.3.32
  @backstage/plugin-kubernetes-backend                     0.4.11
  @backstage/plugin-kubernetes-common                      0.2.6
  @backstage/plugin-kubernetes                             0.6.1
  @backstage/plugin-org                                    0.5.1
  @backstage/plugin-pagerduty                              0.3.28
  @backstage/plugin-permission-common                      0.5.2
  @backstage/plugin-permission-node                        0.5.3
  @backstage/plugin-permission-react                       0.3.3
  @backstage/plugin-proxy-backend                          0.2.22
  @backstage/plugin-scaffolder-backend-module-cookiecutter 0.2.3
  @backstage/plugin-scaffolder-backend                     0.17.3
  @backstage/plugin-scaffolder-common                      0.2.3
  @backstage/plugin-scaffolder                             0.14.0
  @backstage/plugin-search-backend-module-pg               0.3.0
  @backstage/plugin-search-backend-node                    0.5.0
  @backstage/plugin-search-backend                         0.4.6
  @backstage/plugin-search                                 0.7.2
  @backstage/plugin-shortcuts                              0.2.2
  @backstage/plugin-tech-radar                             0.5.8
  @backstage/plugin-techdocs-backend                       0.14.1
  @backstage/plugin-techdocs                               0.15.0
  @backstage/plugin-user-settings                          0.4.0
  @backstage/release-manifests                             0.0.2
  @backstage/search-common                                 0.3.0
  @backstage/techdocs-common                               0.11.11
  @backstage/test-utils                                    0.3.0
  @backstage/theme                                         0.2.15
  @backstage/types                                         0.1.3
  @backstage/version-bridge                                0.1.2

After looking again on the log timestamps here:
2022-03-11T06:24:08.886Z search info Collating documents for software-catalog via DefaultCatalogCollatorFactory type=plugin
2022-03-11T06:24:08.886Z search info Collating documents for techdocs via DefaultTechDocsCollatorFactory type=plugin
...
2022-03-11T06:24:19.107Z search info Collating documents for software-catalog succeeded type=plugin
...
2022-03-11T06:25:51.729Z search error Collating documents for techdocs failed: error: delete from "documents" where "type" = $1 and "hash" not in (select "hash" from "documents_to_insert") - canceling statement due to statement timeout type=plugin

The collating process for software-catalog is taking 11s, and for techdocs is taking 1m 43s. That's why increasing the statement_timeout in postgres is not sufficient if it was set for 1 minute. I did increase the statement timeout again and the techdocs are collated successfully.
Looking at this problem, I wonder why the insert process is considered as one statement, is it because we are inserting using one transaction?
Is there a way to improve the insert process, since I might hit this problem again with the increased indexed items.
FYI, I am using Postgres 12.7, in the docs they said

The timeout is measured from the time a command arrives at the server until it is completed by the server. If multiple SQL statements appear in a single simple-Query message, the timeout is applied to each statement separately. (PostgreSQL versions before 13 usually treated the timeout as applying to the whole query string.) In extended query protocol, the timeout starts running when any query-related message (Parse, Bind, Execute, Describe) arrives, and it is canceled by completion of an Execute or Sync message.

Is this also related with the current issue I am facing?
cc: @iamEAP

Hmm...  I'm not well-versed on PG transactions, and especially not how knex handles them.  What I'm not sure about is how PG treats the multiple calls we make via knex (including temp table creation, copying, and deleting) vis-a-vis this line from the docs you linked: If multiple SQL statements appear in a single simple-Query message, the timeout is applied to each statement separately.
One possible avenue for improvement might be to expose the batch size as an option when constructing the search engine.  For instance, you'll see that it will insert 100 rows at a time, currently, but it's not configurable yet.  The small number there could be the bottleneck on sufficiently large indices.
Also cc'ing @ottosichert who is goalie this week.

@dehamzah Are you still held up by this?

We still blocked with this, we are trying to upgrade our Postgres version to 14 that might be solve our problem here, we will share our results here after the testing is done.

We have confirmed that after upgrading our Postgres version to 14 the search indexing work seamlessly, now it didn't count the whole indexing process in the transaction as one statement which leads to timeout when using a small statement timeout config.
I think we can close this issue, it was the Postgres engine issue.


Ignite 4395 Implement communication backpressure per policy - SYSTEM or PUBLIC

The ticket is resolved, closing PR.
Please close PRs for your resolved tickets and other non-needed PRs.
Alternatively, ask a committer

to use GitHub UI (committer can associate accounts here: https://gitbox.apache.org/setup/ )
or ./scripts/apply_pull_request.sh from Apache Ignite source.
or mention in commit message 'Fixes #(a number of PR)'.

These methods will automatically close PR.
Feel free to reopen this PR if it is anyway actual.


Fix AVM1 SetVariable handling of property paths

Builds on top of #2337.
r? @yurydelendik

/botio test

Thank you for the patch. Looks good.

Reviewed 19 of 19 files at r1.
Review status: :shipit: all files reviewed at latest revision, all discussions resolved, all commit checks successful.

Comments from the review on Reviewable.io


Add avatar to AccountMeta

smol change. couple of new rules need avatar cids. currently have to fetch from public api before the rule processes in a separate request which is meh, since we have the info here already.

I think I was originally worried about redis cache size?
all the AccountMeta gets serialized/deserialized using some magic in the redis caching library we use. maybe we should take just the avatar CID as a string and include that? how much metadata about the avatar do you need for your rule? eg, might want banner also if looking for impersonation signals.
or, in the other direction, just include the entire profile view?

ah, didn't realize that. grabbing just the cid would be fine and i have a little function for that already. let me move that over to this pr.


Gateway network selection ignores "--internal" and "--ipv6"

I've noticed that when connected to multiple (user-defined) networks, NAT and gateway are always configured for the network with the (alphabetically) first network name. This seems a bit random, but I guess it works.
However, in this selection process, internal networks (created with --internal) are not excluded from this list. This means if you're connecting a container to an internal network, internet connectivity for that container will be lost, if the internal network happens to come before other (non-internal) networks it's connected to.
Here's an example showing the issue:
docker run --rm --name=test alpine ping google.com
# ping keeps going (container is connected to network "bridge")
docker network create --internal aaa
docker network create --internal zzz
docker network connect zzz test
# after this, the ping still keeps on going ("zzz" comes after "bridge")
docker network disconnect zzz test
docker network connect aaa test
# the ping now stops! ("aaa" comes before "bridge")
docker network disconnect aaa test
# ping works again

Seems to me this behaviour should not depend on what happens to be the name of the internal network in relation to the names of the other connected networks.
A possible solution would be to only consider non-internal networks for becoming "primary", i.e. select the alphabetically first non-internal network and use it for NAT/gateway. For a container only connected to an internal network, this means no gateway will be configured at all, but this makes sense because why configure a gateway if it won't work.
Similarly, IPv6 should have a separate selection, where it selects the first non-internal network that has IPv6 enabled. Currently, this breaks as well:
docker network create --ipv6 --subnet=fd00:aaa::/48 aaa
docker network create bbb
docker network create --ipv6 --subnet=fd00:ccc::/48 ccc

# a container connected to "aaa" and "bbb" has an IPv6 default gateway
# a container connected to "bbb" and "ccc" does not, even though "aaa" and "ccc" are the same

Thanks for reporting this. It's a bug.
Regarding the default gateway pick up logic it is now in the documentation
When a container is connected to multiple
networks, its external connectivity is provided via the first non-internal
network, in lexical order.

Ah, so that's already the intended case (selecting only non-internal), perfect! Wasn't aware of this in the documentation, sorry.
What about the IPv6 case, a bug as well? Or is it currently impossible to have 2 different networks selected for external connectivity, 1 for IPv4, 1 for IPv6?

The mixed v4/v6 gw network is currently a limitation. The needed code to support that is not there.


Added space between "accrate tracking." and "Access it"

Fixed this space that was bugging me!

Thank you so much ! I'm making a new release as soon as possible. You can follow the instructions on the readme to "build" it yourself!

Check pirate of aseprite on slack too! #pirates-of-aseprite


RaisedButton is deprecated

RaisedButton is deprecated , consider migrate to null-safety

Version 1.0.0 just published that uses null-safety
Thanks!


ApexCharts v4

are there any news about version 4, mentioned here?
https://github.com/apexcharts/apexcharts.js/issues/412
where svgjs in 3.0 or 3.1 is used?

and would it be possible to disable svgjs integration if my webpage already includes it?

also asked in this issue: https://github.com/apexcharts/apexcharts.js/issues/357

would it also possible to include an unbundeled version with v4?
means the "src" but with workin relative imports, so it could directly be used as es6 modules in browser?
(means only the ".js" extension need to be added to the imports)


Scheme design

Create a scheme file that will create a MySQL database and tables design.

![image](https://user-images.githubusercontent.com/81537606/200093704-b72ad060-a2cb-4f5f-98a6-34409f4fe355.png)


Remove data_source as a project dimension

This PR is the outcome of today's conversation. I don't like it very much. I think a better approach would be to remove DATA_SOURCE as a member of DimensionType. data_source is already a field in DatasetConfigModel, which may meet all requirements.

Closing because we will remove data_source from DimensionType.


[8.x] Allow test assertions to be made about email contents (rendered Mailables)

Currently, Laravel allows to make assertions about response contents, eg:
    /**
     * @test
     */
    public function homepage_welcomes_users()
    {
        $this->get('/')->assertSee('Welcome');
    }

You can make assertions about API JSON output as well. However, there is no easy way to test email contents - MailFake is not equipped for it. Developers have several options:

Extending Mailable class to "hijack" render() method
Extending Mailer class to extend render() or send() methods
Making assertions about data variables, rather than rendered output

With this pull request, I'm proposing that MailFake class is extended to allow developers to test their outgoing emails:
    /**
     * @test
     */
    public function email_confirmation_is_correct()
    {
        Mail::fake();

        event(new TestEvent());

        Mail::assertSent(TestMail::class, function ($mail) {
            $this->assertStringContainsString('shipped', $mail->render()['html']);
            $this->assertStringContainsString('shipped', $mail->render()['text']);
            $this->assertEmpty($mail->render()['raw']);

            return true;
        });
    }

Extra remarks:

Returning an array from MailFake's render() method is not ideal (Mailer class returns a string), but this is a path of least resistance, with minimum changes required to the existing classes or the API
There is a small amount of code repetition between MailFake and Mailer classes, namely parseView and renderView methods. From what I understand MailFake class (as well as other fake facades) is supposed to be as stand-alone as possible, so sharing a trait with Mailer didn't feel like a very good idea

Would love to hear more opinions, I'm sure there must be more developers out there in the wild who wanted to test their email contents at some stage!

I've always been against combining testing that an email was sent and testing how it is rendered. You should write a separate test that creates the Mailable instance and calls ->render() on it and then you can inspect it however you want.

@taylorotwell two issues with calling render() on Mailable are:

Have to write unit tests for Mailable passing data variables manually etc, can't do it during integration testing (which many developers prefer)
Mailable's render() always returns HTML, there is no way to access other variants (plain-text or raw)

However, the syntax felt wrong to me too. I'll try your proposed fluent API (instead of "array API") and make another PR, thanks!


[bug]: When FormDescription is not used there is an id still applied to aria-describedby causing Arc Toolkit errrors

Describe the bug
The FormDescription component can be used to add hint text to a form control. It will automatically have a generated id applied and is associated with the form control via aria-describedby. However, in cases where no hint text is required, and therefore the FormDescription component is not composed in, the id is still generated and added to the controls aria-describedby. I'm not sure how much of an issue this is (or if an easy solution exists), but it throws accessibility errors with Arc Toolkit (although Axe doesn't seem to error). Does having aria-describedby referencing non-existent elements really matter?
Affected component/components
Form
How to reproduce

Create a form example with no Form.Description component
Run Arc Toolkit to see the accessibility error relating to aria-describedby referencing non-existent id.

Codesandbox/StackBlitz link
No response
Logs
No response
System Info
N/A

Before submitting

[X] I've made research efforts and searched the documentation
[X] I've searched for existing issues

Couple of possible workarounds I think would be to pass a ref down through context to detect when Description is mounted, or avoid Description altogether and inject hint text into the form label avoiding the need for aria, and getting the benefit of increase hit area.


Use definitionsFullyResolved to look up consumes/produces + tests

@whitlockjc This includes the changes from #101 plus I added tests.
For the tests, I basically duplicated the existing tests for operation.js but they are now run on both swagger.yaml and swagger-relative-refs.yaml.
Let me know if you'd prefer a different approach 👍
Closes #101, fixes #92

Merged.


Audit of existing experimental patterns against checklist (Markdown template) to determine work required for completion

Goal
Audit existing experimental patterns and determine the work required for completion
Experimental patterns

[ ] Cards
[ ] Chatbots
[ ] Create flows
[ ] Dashboard widgets
[ ] Delete pattern
[ ] Export pattern
[ ] Generate and API key
[ ] Import pattern
[ ] Login pattern
[ ] Navigation pattern
[ ] Order summary template
[ ] Remove pattern

Closing.


Fix mono/gac location

I made a mistake with my last commit to this file. This corrects it. Thanks!

Tagged as 4.1.9.  Do you need a new FSharp.Compiler.Tools?

I already have this built with my fix included, so no urgency I think. Still need to test. Thanks!


[color] update color page overview tab image

In general should probably display disabled as its own state and not in every example.

We took this image out for our v11 release, so this has been resolved.


How to moderate a meeting

We should provide guidance for leads on why moderated meetings are more inclusive and improve participation. Then provide guidance on:

how to select a moderator (probably should not be someone very active in the conversation if possible)
rotate the role so it's not always the same person
how to "raise your hand", queue up people to speak
Coach the moderator on interrupting someone who is talking too much, or out of turn
Identifying people who want to speak, e.g. someone who is unmuting their mic/video
Repeating and answering questions in chat
etc

Started on this work in a google doc for simple edits before creating a PR. Feedback would be appreciated.
https://docs.google.com/document/d/1-bd04bqgahnI1PUTqBvhKBj5zRi05QK7kiClpGRNg1U/edit

@jberkus please assign me to this issue.

where would we host this file on the repo?

@Riaankl I would say here: https://contribute.cncf.io/maintainers/community/
There are similar docs in that directory.

I've done my first pass of review. Thanks @Riaankl , great content!

I've done my first pass of review. Thanks @Riaankl , great content!

Thanks for the review @aliok
I think I have all your review points covered. Let me know what you think.

Yes, you've got them all covered, thanks. Let's wait for a few more days for other reviews and then I can do a final review again


Parameters undefined

Describe the bug
Pulled the latest commit over the top of the fork and get parameters undefined
To Reproduce
Steps to reproduce the behavior:
check logs of consumer after docker compose up -d
Expected behavior
A clear and concise description of what you expected to happen.
processing torrents
Logs
Provide logs for all containers when this issue occurs.
If the logs are short, make sure to triple backtick them, or use https://pastebin.com/
2024-02-02T17:04:56.307005479Z     sql: "SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND t.relkind = 'r' and t.relname = 'files' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;",
2024-02-02T17:04:56.307009346Z     parameters: undefined
2024-02-02T17:04:56.307012292Z   },
2024-02-02T17:04:56.307015269Z   original: error: cache lookup failed for attribute 1 of relation 30294
2024-02-02T17:04:56.307019499Z       at Parser.parseErrorMessage (/app/node_modules/pg-protocol/dist/parser.js:287:98)
2024-02-02T17:04:56.307022693Z       at Parser.handlePacket (/app/node_modules/pg-protocol/dist/parser.js:126:29)
2024-02-02T17:04:56.307025727Z       at Parser.parse (/app/node_modules/pg-protocol/dist/parser.js:39:38)
2024-02-02T17:04:56.307029442Z       at Socket. (/app/node_modules/pg-protocol/dist/index.js:11:42)
2024-02-02T17:04:56.307032601Z       at Socket.emit (node:events:518:28)
2024-02-02T17:04:56.307035555Z       at addChunk (node:internal/streams/readable:559:12)
2024-02-02T17:04:56.307038513Z       at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
2024-02-02T17:04:56.307041551Z       at Readable.push (node:internal/streams/readable:390:5)
2024-02-02T17:04:56.307044518Z       at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
2024-02-02T17:04:56.307047525Z     length: 116,
2024-02-02T17:04:56.307050462Z     severity: 'ERROR',
2024-02-02T17:04:56.307053388Z     code: 'XX000',
2024-02-02T17:04:56.307056325Z     detail: undefined,
2024-02-02T17:04:56.307059256Z     hint: undefined,
2024-02-02T17:04:56.307062306Z     position: undefined,
2024-02-02T17:04:56.307068626Z     internalPosition: undefined,
2024-02-02T17:04:56.307071636Z     internalQuery: undefined,
2024-02-02T17:04:56.307074542Z     where: undefined,
2024-02-02T17:04:56.307077436Z     schema: undefined,
2024-02-02T17:04:56.307080355Z     table: undefined,
Hardware:

OS and distro: [e.g. Raspberry Pi OS, Ubuntu, Rocky]
Server: [e.g. VM, Baremetal, Pi]
docker
Additional context
Add any other context about the problem here.

dropped in new volumes in case schema wise had changed and does it on a complete new install

Can you confirm no changes after cloning the new repo?

I hadn't although please ignore, local issue and have resolved, thanks for the reply! :)


Create integration test suite and migrate appropriate tests from unit test suite #607

Integration testing is only available for components at the moment not mixin's.

Review status: 0 of 2 files reviewed at latest revision, all discussions resolved, some commit checks failed.

Comments from the review on Reviewable.io

Correct, though the {{sl-tooltip}} component mixes in the sl-tooltip-enabled mixin, the functionality of which should be tested in the Integration test.

Review status: 0 of 2 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

tests/integration/components/sl-tooltip-test.js, line 14 [r1] (raw file):
This should not be tested in an Integration test, per https://github.com/softlayer/ember-style-guide/issues/35

tests/unit/components/sl-tooltip-test.js, line 22 [r1] (raw file):
This test should be re-implemented, asserting the default value of the tagName property, per https://github.com/softlayer/ember-style-guide/issues/35

Comments from the review on Reviewable.io

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

Comments from the review on Reviewable.io

Review status: all files reviewed at latest revision, 3 unresolved discussions, some commit checks failed.

tests/unit/components/sl-tooltip-test.js, line 11 [r1] (raw file):
Is it required to pass an object to this.subject() in order for this assertion to pass?  If not, please remove it.

Comments from the review on Reviewable.io

Review status: 0 of 2 files reviewed at latest revision, 3 unresolved discussions, some commit checks pending.

tests/integration/components/sl-tooltip-test.js, line 14 [r1] (raw file):
Done.

tests/unit/components/sl-tooltip-test.js, line 11 [r1] (raw file):
It is required, errors out if title is not set and if its not a string.

tests/unit/components/sl-tooltip-test.js, line 22 [r1] (raw file):
Done.

Comments from the review on Reviewable.io

new tests added for dataTrigger (https://github.com/softlayer/sl-ember-components/blob/EmberCli-1.13.8/addon/mixins/sl-tooltip-enabled.js#L30-L38) since it's mixin functionality.

Review status: 0 of 2 files reviewed at latest revision, 3 unresolved discussions, some commit checks pending.

Comments from the review on Reviewable.io

Reviewed 1 of 2 files at r2.
Review status: 1 of 2 files reviewed at latest revision, 5 unresolved discussions, some commit checks failed.

tests/integration/components/sl-tooltip-test.js, line 41 [r2] (raw file):
Does this have any impact on the test?

tests/integration/components/sl-tooltip-test.js, line 64 [r2] (raw file):
Does this have any impact on the test?

tests/integration/components/sl-tooltip-test.js, line 87 [r2] (raw file):
Does this have any impact on the test?

tests/integration/components/sl-tooltip-test.js, line 110 [r2] (raw file):
Does this have any impact on the test?

tests/integration/components/sl-tooltip-test.js, line 124 [r2] (raw file):
If the element was given focus, which caused the popover to appear, what is making it go away before the click is called on the element?  Same idea with the mouse enter event.  The point is that there is nothing ensuring that the reason the 2nd and 3rd assertions are passing is not because of the first assertions setup.
This has also gotten me thinking about not only this test, but the other individual tests for hover, click, etc.  Seems we are tying ourselves specifically to the rendered output of the implementation rather than just spying on the setup of the expected values.  For example, I'm assuming aria-describedby is coming from the popover element itself from Bootstrap's implementation?

Comments from the review on Reviewable.io

Reviewed 1 of 2 files at r2.
Review status: all files reviewed at latest revision, 5 unresolved discussions, some commit checks failed.

Comments from the review on Reviewable.io

Please merge the latest changes to EmberCLI-1.13.8 into this issue as you continue to work on it.

Review status: all files reviewed at latest revision, 5 unresolved discussions, some commit checks failed.

Comments from the review on Reviewable.io

Review status: 1 of 3 files reviewed at latest revision, 5 unresolved discussions, some commit checks pending.

tests/integration/components/sl-tooltip-test.js, line 41 [r2] (raw file):
no, removed.

tests/integration/components/sl-tooltip-test.js, line 64 [r2] (raw file):
no, removed.

tests/integration/components/sl-tooltip-test.js, line 87 [r2] (raw file):
no, removed.

tests/integration/components/sl-tooltip-test.js, line 110 [r2] (raw file):
no, removed.

tests/integration/components/sl-tooltip-test.js, line 124 [r2] (raw file):
aria-described is the direct output of bootstrap's tool tip , aside from this test I believe the rest were pretty effective at testing the direct behavior of our ember wrapped bootstrap component. I've removed these tests and placed them in the mixin unit test file.

Comments from the review on Reviewable.io

Reviewed 2 of 2 files at r4.
Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.

Comments from the review on Reviewable.io


Add possibility to configure termsOfService in info object of OpenAPI document

For open APIs it would be needed to publish both open data license and terms of use like this in the info object of OpenAPI document:
"info": {
  "license" : {
      "name" : "Open data CC 4.0 licence.",
      "url" : "some url to opendata licence"
   },
   "termsOfService": "some url to terms of use for the open API service"
}

Currently termsOfService cannot be configured on Hakunapi. This support should be added at least in the conf parser and other places where Open API document is generated:
https://github.com/nlsfi/hakunapi/blob/main/src/hakunapi-core/src/main/java/fi/nls/hakunapi/core/config/HakunaConfigParser.java

I'll add this, however it will only appear in the /api response. No real place for it in / landing page response schema.

Thanks, adding this to the /api response should be enough.


Error when used on a project containing Cython PYX files

I have multiple PYX files that use Cython syntax for defining functions and variables in a C-like way.
I have this function definition: cdef void my_func(str msg):
Pipreqs sees the cdef void part of the function declaration as a syntax error and exits:
  File "<unknown>", line 5
    cdef void my_func(str msg):
         ^
SyntaxError: invalid syntax

I could just ignore any PYX files from being parsed by pipreqs, but I don't want to have any missing dependencies for the project by doing so.

Ignore this issue, it was due to user error
The file in question had a .py extension instead of the correct .pyx extension.


Fixes #27 Strategy succeeds even if user is not found

Does this have any side-effects?
OAuth2 authentication package does this already.

The tests expect the function to not return an error, anyone care to explain why this was the chosen functionality?

Some of the tests are failing. I thought there was a reason why it is doing it that way, but @ekryski might know.

@daffl Yes because sometimes you want to just create a JWT without anything in the payload (an anonymous JWT) or you may want to encode everything you need to make authorization decisions into the payload of a JWT that has a shorter TTL (for more scalable apps).
Thanks for the PR @Metamist but I don't think we can merge this. However, I did leave a comment on the issue with a few options for you. https://github.com/feathersjs/feathers-authentication-jwt/issues/27#issuecomment-317565578

The problem is that this will return a valid JWT on any error, including database or syntax errors. I'm pretty sure that is the same problem @zusamann is having in his demo application as well.

I would think it should only do this on a NotFound error.

@daffl Agreed. Which is related to #14. @Metamist if you wanted to update the PR to accommodate that and add a test then we can re-open this PR.

Actually @daffl I don't think I'm having such an error. The custom auth service I've written first finds the user with that username, and only then proceeds, so I'm sure it's not a case of user not being found. Secondly I don't think there are any syntax errors either, the service pretty much works for all cases correctly (correct username/pass, incorrect username/ incorrect pass).
The token seems to be completely valid when returned. And works perfectly fine with the rest client. Only in the websocket client once authenticate call has been made, the rest of the calls fail due to the error Auth token not found.

Though in the repository I've shared there is no custom auth service in the latest commits as I was trying to simplify the logic and see if the error still persists, which it does.


One-key Triple Thread👻

This is an issue for Quickstart of NPi to leave comments! 👻🥰
You can leave a supportive message to NPi here.

This is such an inspiring project! Keep up the great work! 🌟

A supportive message acknowledging the efforts and contributions of the team 🌟 Keep up the great work!

Great work on this project! Really excited to see where it goes. Count me in for support!

Leave a comment on issue #27.

Once in a digital forest, algorithms weaved patterns unseen. Among them, lived a wise old AI called Ophelia. Every day, she solved the forest’s problems, from predicting rainfall to optimizing resource allocation. One day, a young, curious code snippet named Nia approached her. "Teach me, Ophelia," Nia said. With patience, Ophelia guided Nia, showing how even the smallest lines of code carried immense power. Years passed and Nia grew wise and skilled, ready to weave her own patterns. The digital forest thrived, a testament to the knowledge passed from one line of code to the next, across generations.

@all_actions This action was triggered by Agent #27

All actions on this issue were completed by AI.


Update stable version in repo from 1.1.1 to 1.2.0

Think about update stable version to 1.2.0
Version: stable=1.1.1 (709 days old) => latest=1.2.0 (21 days old)
Installs: stable=243 (55.35%), latest=22 (5.01%), total=439
Click to use developer portal
Click to edit
Do not close this issue manually as a new issue will be created if condition for update still exists.
Please drop a comment if any reason exists which blocks updating to version 1.2.0 at this time.
Note: This is an automatically generated message and not personally authored by bluefox!
@mcm1957 for evidence

This issue seems to be outdated.
This issue suggests to update the stable version of this adapter to 1.2.0 but the current stable version is already 1.2.0.
So this issue should be closed.
@mcm1957 for evidence


Update format of anaconda-project yaml files

The existing format used in the anaconda-project yamls was designed to work as an environment yaml as well as a project specification. Unfortunately, this format no longer works properly with anaconda-project lock for newer versions of anaconda-project:  https://github.com/Anaconda-Platform/anaconda-project/issues/320
All new anaconda-project yamls should use the backwards compatible format suggested in that issue and old project yamls would ideally be updated too to avoid confusion when updating projects and their lock files.

The suggested format has been tested for the boids project here which means boids can be used as our reference when updating the project files everywhere else.


added wireframe and mockups

##Coauthors
Ryan
Anwar
Juliane
Description
Added the wireframe and mockups
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have removed unnecessary comments/console logs from my code
[ ] I have made corresponding changes to the documentation if necessary (optional)
[x] My changes generate no new warnings
[x] I have checked my code and corrected any misspellings
[x] No duplicate code left within changed files
[x] Size of pull request kept to a minimum
[x] Pull request description clearly describes changes made & motivations for said changes

 Save wireframes and mockups in FE repo in a "design" file for future reference


Change init_prefix_explicit to use self.resource_prefix.

Missed 1 use of prefix= that should have been migrated to resource_prefix= to avoid depreciation warnings.

:shipit:


Extend Rekognition support to include get_face_search and start_face_seach

Description
Extends Rekognition support to include get_face_search and start_face_seach
Background Context
Follows on from this work:
https://github.com/spulec/moto/pull/4957
refs:
https://docs.aws.amazon.com/rekognition/latest/dg/what-is.html
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/rekognition.html
More specifically:
https://docs.aws.amazon.com/rekognition/latest/APIReference/API_GetFaceSearch.html
https://docs.aws.amazon.com/rekognition/latest/APIReference/API_StartFaceSearch.html
Related issue:
https://github.com/spulec/moto/issues/4071

Fantastic!
Thanks for reviewing and merging...


Use installed packages in ci

Nice catch :-)

Wouldn't need catching if I did it right the first time!


Update tutorial-access-private-cloud.md

Corrected VMware product names.

@rvandenbedem : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"


Apply appropriate responses to non-api generated errors (ie loadbalancers)

Lightkube raises httpx.HTTPStatusError when the client receives an error that doesn't contain any json content.  Lightkube has determined that in this condition the api-server isn't returning the error per-se -- but perhaps a proxy or load-balancer is in the middle of the transaction between the client and the kube-apiserver
In the event of these types of errors, we likely should report a different log message, but treat both exceptions identically.

LGTM. Nevertheless, I have a suggestion that I would like to know what you think.
It seems to me that the way we would have to handle these exceptions from the charm would leave us more coupled to our dependencies. The following image best illustrates my concern.

What if we create an exception to better handle it within the charm to reduce the coupling?

LGTM. Nevertheless, I have a suggestion that I would like to know what you think.
It seems to me that the way we would have to handle these exceptions from the charm would leave us more coupled to our dependencies. The following image best illustrates my concern.
What if we create an exception to better handle it within the charm to reduce the coupling?

you're in my head @mateoflorido i was afraid of the bigness of the change. However -- given the nature of the bug and the fact we've been afforded a bit more time to fix -- i think you're approach is spot on!  I'll start making those adjustments


Unable to allocate more than 2GB of RAM (while some wasm binaries are able to use up to 4GB or RAM)

💡 Feature description
I'm currently working on bindings that manipulate large amounts of data, I've tried a lot of things but can't get the binaries generated by wasm-pack to allocate memory beyond 2GB (while chrome and node are able to run such binaries in theory).
Emscripten seems to have supported this for a while https://v8.dev/blog/4gb-wasm-memory#trying-it-out
I could not find any relevant documentation or issues, so I'm opening one, if there are resources on the topic with a solution I'm a taker.
Cheers

Facing simillar issue


This closes #1794, add new GetBaseColor function

PR Details

Description

Related Issue

Motivation and Context

How Has This Been Tested

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (9a68553) 99.06% compared to head (a510e72) 99.06%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1798   +/-   ##
=======================================
  Coverage   99.06%   99.06%
=======================================
  Files          32       32
  Lines       28609    28616    +7
=======================================
+ Hits        28342    28349    +7
  Misses        177      177
  Partials       90       90

Flag
Coverage Δ

unittests
99.06% <100.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


refactor(caller): unify callers

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

16 of 16 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 98.896%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
627

Relevant Lines:
634

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

16 of 16 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 98.896%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
627

Relevant Lines:
634

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

16 of 16 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 98.896%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
627

Relevant Lines:
634

💛  - Coveralls


Error Python

When i try open builder.bat an error appears "Python"

Can you send me screenshot of:

The error itself.
The command prompt result where python && python --version

2. where python && python --version

Can you send me screenshot of:

The error itself.
The command prompt result where python && python --version

is working

Did you fix it? If so, please write the your side of fix here.


Using custom types for scalars?

I'm serializing some decimal values from database into JSON. In order not to risk loosing precision of database data, I've implemented a C++ class for storing values in fixed decimal form. I'm trying to pass my class as NumberFloatType template parameter to basic_json but it looks like parsing of floating point values is hard coded in lexer::strtof overloads. Are there any "customization points" which could be used to support deserialization of strings into custom basic types like NumberFloatType?

I'm afraid this is not possible in the moment, and I am not sure how to realize this right now.


impure.nix: add crossSystem as arg

Motivation for this change
This lets you pass in "--arg crossSystem ..." to  like in #25215.
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

Should be fixed

Thanks!


Ability to build SPIRV-Reflect libraries only.

Hi there,
It would be really convenient if it was possible to build SPIRV reflect as a static or shared library so multiple things could link against it at the same time without having to deal with symbol conflicts.
Additionally, it would be useful to be able to configure the CMakeLists to only build said library and optionally turn off the main executable and stripper util.
If this is something you're happy for me to do, it probably shouldn't be a massive hassle.
Many thanks,
Alister

Thanks for the Feature Request. I will get to this as soon as I can.

I added some build options to toggle building of the various targets in https://github.com/KhronosGroup/SPIRV-Reflect/commit/05b6ec33bf50a4126c9bea4608250927c7cc18dd.
I was only able to add a build target for a static library. The shared library is going to take some amount of non-trivial work to get all the export declarations in place. I'll have to find a larger chunk of time for that.
To build only the static library, add these CMake command line arguments:
-DSPIRV_REFLECT_STATIC_LIB=1 -DSPIRV_REFLECT_EXECUTABLE=0 -DSPIRV_REFLECT_EXAMPLES=0 -DSPIRV_REFLECT_STRIPPER=0


Remove ignore directives from closing backtick blocks

CC #122

lol I suppose my search-and-replace was a bit hasty

I wish I could find a way to make mdbook test stop running rust code blocks as tests


[docs] Add a note (or an article section) about new scroll functionality to the "How to Scroll Web Pages Using TestCafe" article

How to Scroll Web Pages Using TestCafe.
New scroll functionality: Scroll Actions.
Reference: we have already added such a note here: Scroll Elements Into View.

https://github.com/DevExpress/testcafe/issues/6185

The topic is updated.


A Question on Evaluation

Hello, I am a Chinese student.
I have downloaded two solo videos（2P83WJXifEs and 3d1b4UH43-E）from 'val.csv' to evaluate the performance of the model. Finally, loss is 0.5479. The effect of each speech separation is very unsatisfactory. Why is that? hope to get your reply.
P.s. I have download the trained model weights for evaluation by:
> ./ scripts / download_trained_model.sh
and I Evaluate the trained model performance by:
> ./ scripts / eval_MUSIC.sh

You are using two videos of the same instrument I guess.

You are using two videos of the same instrument I guess.
Thank you. I changed two different instruments and the effect was improved.

@GFENGG I am a Chinese student too. if you have time, could I ask some questions about this project from you ...my email is<EMAIL_ADDRESS>Thank you very much....


The blob container doesn't prompt for refresh when there happens some changes

Storage Explorer Version: 1.9.0
Branch: master
Build: 20190809.2
Platform/OS: Linux Ubuntu 19.04/Windows 10/macOS High Sierra
Architecture: ia32/x64
Regression From: Previous release 1.9.0
Steps to reproduce:

Expand anyone account -> 'Blob Containers'.
Open one blob container which contains one blob at least.
Try to delete one blob.
Check whether there prompts for refresh or not.

Expect Experience:
There prompts for refresh after deleting the blob successfully.
Actual Experience:
The blob container doesn't prompt for refresh after deleting the blob successfully.
More info:
This issue also reproduce for ADLS Gen2 blob containers when there happens some changes.

This issue has been fixed on the latest master build 20190813.12. So closed it.


[ALLUXIO-2490] change name of method 'setTtlForFileWithFreeOperationTest()'

https://alluxio.atlassian.net/browse/ALLUXIO-2490
According to the Alluxio unit test style guide, test names should not contain the "Test" suffix.
Consequently, the FileSystemMasterTest#setTtlForFileWithFreeOperationTest() should be renamed to setTtlForFileWithFreeOperation().

Can one of the admins verify this patch?

ok to test

add to whitelist

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13948/
Test PASSed.

Merged build finished. Test PASSed.

This problem don't has to be fixed in master branch, Jira issue has been closed. I will close this PR. Thx.


Not generating empty structs.

I came across the problem, where a struct that is defined after an empty one would have its values shifted in the runtime for some reason (building with Xcode 7.1). For instance, I have a code:
...
struct ImagesAssets {
// bunch of stuff. works well
...
} ImagesAssets;
struct Strings {
} Strings;
struct OtherAssets {
   __unsafe_unretained NSString *first; // Will have a value of second
   __unsafe_unretained NSString *second; // Will have a null value, if it's the last attribute. Otherwise will have value of third
} OtherAssets;
...

By eliminating empty Strings struct I managed to fix the behavior.
(I closed my original pull request and made the very same one from a different branch)

Thanks for your PR! I'd propose, that empty sections could be rather filtered in #enumerate_sections instead of passing an additional parameter. Should make the code easier to read.

I'll have a look, when I have a chance. Could you meanwhile take a look at my branch named xcres-0.5. I added support for sounds and datasets in xcassets, started working on analysis of xibs and storyboards. And as a result, I changed the architecture quite significantly, abstracting the collection types (xcassets, bundle, loose files) from the resource types (image, sound, xib, etc). I would really like to merge it back to you at some point. But that's a big change...

Thanks!
Wow, this spunds super amazing! I'd love to add this features and am open to architectural improvements needed to make this happen.
Could you make a PR, so that I can review and comment on the changes?

I did what you asked for this PR.

I squashed some commits together and merged that by #26. Thanks again! :+1:


Stop loading

Im building an always on web app and when I call .stop() it's still rendering the stream(im using a radio stream) no matter what I do. Is there a way to stop streaming?

+1 for this one!


Remove unnecessary postinstall step

prebuild 4.5.0 is here 🚀
https://github.com/mafintosh/prebuild/pull/135 makes the postinstall step unnecessary which means apm rebuild will work now.
We should ship 1.1.0 after this.
Fixes #518 and may help with #507

@rgbkrk Do we need to bump prebuild in zeromq and jmp to ensure apm update installs prebuild 4.5.0?
We currently require prebuild: ^4.4.0 there.

^4.4.0 will cover >= 4.4.0 < 5.0.0, which means that new users (or fresh installs) will get the latest prebuild. To help current users, we'll need to bump the prebuild requirement.


An exercise for the reader - HTTP Archive files

Firstly - awesome job on this project. I was thinking the other week of writing something similar.
My use case was reading in http archive files in swift.
(I ended up abandoning using Decodable because it was taking too long to get the data structure right and used SwiftyJSON instead. )
https://github.com/johndpope/HAR-Reader
Presumably you've used Charles proxy software on mac?
https://gist.github.com/jjarava/5f26806bba75d052c41a
It's possible to record stuff using this tool - and then save the file as a HAR file.
This format hasn't really caught on - but there's some software around that could leverage this stuff - and save everyone a bunch of time.
https://github.com/AliSoftware/OHHTTPStubs/issues/204
This HTTP ARchive has a specification - and it's probably the most complicated json file you're ever going to see. It includes all the communications from an API - nested in yet more json. The file format is .har and there are plenty around - if you have a taste to try this - could be a nice hack.
I could use your help on this -
https://github.com/grpc/grpc-swift/issues/2

Thanks for the suggestion.


I am encountering this error. Kindly help me out.

Traceback (most recent call last):
File "C:/Users/user/PycharmProjects/Panoramic-Image-Stitching-using-invariant-features-master/panorama.py", line 2, in
import imutils
File "C:\Users\user\PycharmProjects\untitled\venv\lib\site-packages\imutils_init_.py", line 8, in
from .convenience import translate
File "C:\Users\user\PycharmProjects\untitled\venv\lib\site-packages\imutils\convenience.py", line 6, in
import cv2
ModuleNotFoundError: No module named 'cv2'
Process finished with exit code 1

Please install OpenCV. If installed then you have to reinstall. It may happen that it has not correctly installed.

I have installed opencv 3.4.4. But i am not getting. Please help me.

@Sarala1  , I have used opencv version 3.3.0.

@Sarala1 We have updated the repo to use latest libraries. You can now give a shot!


Enhancement: accept unknown options for aad user add

Since we are not supporting all user properties right now, I suggest that we allow unknown options for aad user add just like we do for aad user set.

Add a remark to the docs that tells people that they can use unknown options and add a link to the Graph docs page listing all user properties. Let's add the same remark for aad user set.

Hi Milan, If this is ready to be worked on, can I take it?

Yes sure! All yours!


so many cudafree api called between nccl allgather when larger batchsize  and zero2 config is enabled during training

as the timeline shows below,

model parameters: 3.9B
zero2 config:
"zero_optimization": {
"stage": 2,
"contiguous_gradients": false,
"overlap_comm": true,
"reduce_scatter": false,
"reduce_bucket_size": 250000000,
"allgather_bucket_size": 500000000
},

@gongjingcs, this is probably due to the gradient partitioning during the backward pass. After reduction, each rank only keeps a partition of the gradients and frees up others belonging to other ranks.
Is this causing an issue in someway on your end?

Is this causing an issue in someway on your end?

Yes，because in this case cudafree costs more than 2 seconds. In addition, I’m very curious that using a smaller bsz and the same zero configuration does not have this problem. Looking forward to your reply

I am still unsure of what could be causing such slow cudafrees. Also, I am quite surprised that smaller bsz makes a difference. Can you try the following?

Try zero-1 which will disable the gradient partitioning logic. I think zero-1 should fit your 3.9B model.
Enable contiguous_gradients with zero-2. One possible cause of high frees is memory fragmentation, which contiguous_gradients is supposed to prevent.

I am still unsure of what could be causing such slow cudafrees. Also, I am quite surprised that smaller bsz makes a difference. Can you try the following?

Try zero-1 which will disable the gradient partitioning logic. I think zero-1 should fit your 3.9B model.
Enable contiguous_gradients with zero-2. One possible cause of high frees is memory fragmentation, which contiguous_gradients is supposed to prevent.

1、 yes，of course，zero-1 fit my models
2、Enable contiguous_gradients with zero-2 also cause such slow cuda frees.
One thing that needs to be pointed out is that it seems that this happens only when the V100 GPU memory is almost full.

One thing that needs to be pointed out is that it seems that this happens only when the V100 GPU memory is almost full.

This is perhaps the correct clue. This could be because higher batch size creates memory pressure that could cause the memory allocator to react in a way that results in this behavior. The two that come to mind are:

The memory allocator is defragmenting memory to satisfy the large buffer requests of allgather.

The memory allocator is exploiting the memory frees of gradient partitioning to trigger memory compaction?
.
Does this make sense? If so, then perhaps we can test these hypotheses in a couple of ways.

Do you observe similar high number of cudafrees with zero-1, assuming you can fit the large batch size? Since zero-1 does not have gradient partitioning, this could rule out hypothesis (2).

Reduce memory pressure for large batch size by reducing both bucket sizes to 5e7 and disabling overlap_comm.

Further reduce memory pressure by offloading optimizer to cpu.

Can you please try these suggestions?

One thing that needs to be pointed out is that it seems that this happens only when the V100 GPU memory is almost full.

This is perhaps the correct clue. This could be because higher batch size creates memory pressure that could cause the memory allocator to react in a way that results in this behavior. The two that come to mind are:

The memory allocator is defragmenting memory to satisfy the large buffer requests of allgather.
The memory allocator is exploiting the memory frees of gradient partitioning to trigger memory compaction?
.
Does this make sense? If so, then perhaps we can test these hypotheses in the following ways.

Do you observe similar high number of cudafrees with zero-1, assuming you can fit the large batch size? Since zero-1 does not have gradient partitioning, this could rule out hypothesis (2).
. Reduce memory pressure for large batch size by reducing both bucket sizes to 5e7 and disabling overlap_comm.
. Further reduce memory pressure by offloading optimizer to cpu.

Can you please try these suggestions?

we do not observe similar  high number cudafrees with zero-1.

Thanks for sharing that information. So perhaps hypothesis 2 is also involved.

Thanks for sharing that information. So perhaps hypothesis 2 is also involved.

Sorry for the trouble. I found that the zero stage1 has this problem without gradient partition enabled.
So we can conclude that the reason is that torch memory allocator is defragmenting memory not deepspeed memory management mechanism?
looking forward to your reply!

Thanks for sharing the update regarding zero-1 behavior. So it does seem like a memory pressure issue. Did you try offloading the optimizer state to cpu and running the same large batch size? Even though the perf might be hurt, I think this will provide further evidence of memory pressure being the root cause. Thanks!


Makefile not detected when filename is not capitalized

When I installed this extension, it would not detect my makefile because the file name was written lowercase, like "makefile". When I renamed it to "Makefile" with a capital M, it worked perfectly fine. I could still run make commands normally when it was lowercase, so I feel like this should be considered a bug (if make can run the makefile just fine then this extension should be able to as well). This might only be a problem on Linux (and WSL2) since Windows is typically case-insensitive..

Hey @leggettc18, thanks for reporting the issue.
You are completely right, here we have the name Makefile with a capital M, and it is used everywhere in the extension to match the file name.
Also, the official Docs that make is not case-sensitive related to the file name.
I will investigate and check how VSCode will allow me to watch for files not case-sensitive.

Also, I do not have a "custom" make parser, I use make cmd to check what is available to work with.
The problem seems to be that the extension is not "activating" due to the capital M, and not trying to get the tasks to list.

Yeah I figured it just wasn't seeing the makefile at all due to the lack of capital M, given that it was working properly when I changed the filename to a capital M. Thanks for looking into it! Not a big deal all-in-all, just figured it was either easy to fix or if not it should be documented somewhere.

Hey @leggettc18, I just released a new version v1.15.0 and it will allow you to have case-insensitive file names for the default Makefile file name.
please let me know if it works for you.

Hey, I'm not sure whether it would work or not, I renamed an existing one back to lowercase from uppercase and all the tasks are still there and there's probably a naming collision but I can't figure out how to get rid of the old tasks.

Or maybe there's a separate issue with the openFile command provided by the extension because the file that shows up in the makefile tasks section in the explorer can't be opened if I right click on it and it still shows the uppercase filename.

Ok, I edited the name of a make task in my lowercase-named makefile and it updated on the task list. So I think the detection of the file works but something's wrong with the openFile command.

If I may offer a suggestion, I've never coded a VSCode extension before but I noticed that in the src/shared/config.ts file it appears that it references a command called openMakefile for the VSCode openMakefile command but I cannot find an openMakefile command anywhere else in the extension's source code. I do see an openHostfile though, is there perhaps a desync in naming conventions going on here or am I just misunderstanding how vscode extensions work?

@leggettc18 thanks for the debugging session.
I will investigate the naming issue you just mentioned, but I'm pretty sure it will be fixed in the next release, as I'm creating a new option to allow you to define any name to your makefiles, as the documentation says, Make allows it.
About the capital/lowercase, this one seems to be fixed as well in my current branch.

Ok. It's definitely fixed in terms of parsing the makefile and creating tasks whether the makefile is capitalized or not, so thanks for fixing that.

Hey @leggettc18 I just released a new version 2.0.2.
It is now possible to customize any number of names for your Makefiles by adding the names to the config make-task-provider.makefileNames.
With this release, a "dynamic" file watcher will watch for any file change that matches the names you set and refresh the available tasks.
This release should fix all the mentioned issues.

Can confirm, everything that didn't work before is now working! Thanks for seeing it through.

Thank you very much @leggettc18 for spending your time debugging and testing the extension.
That is the spirit that moves the open-source projects forward, and also, that is why VSCode and its extensions are so great.


Slider carousel nowrap or rewind behavior

I have a page topper slider on autoplay, but the rewind effect when the slider starts over is distracting as it slides left though all the slides already seen.
Is there a way to make the slides wrap, so after the last slide, it continues to auto play to the right, more like a carousel?
Here's a quick clip showing the default slider behavior I see now with two test slides: https://www.dropbox.com/s/juab27hic4ablyq/swiffy-slider-carousel-question.mp4?dl=0

Yes, this is one of the limitations of Swiffy Slider due to how it works.
To do infinite scroll, javascript is required to move the first slide as the next or place slides on top of each other and animate them (and that would require js to handle touch sliding) - this slider is created to avoid all of that. The main feature of Swiffy Slider is to support touch based scroll with almost zero performance impact using the device scrolling mechanism.
So since the slides are grid columns in a scrollable div placed next to each other, they cannot blend into each other (as that would require the slides are placed on top of each other) and the slides cannot go infinite rounds as that again would require slides on top of each other or changing the order of slides as you slide right.
The only solution I have for that is to repeat slide 1+2 a number of times so it looks like infinite (users will not look at the frontpage for ages anyways) - the markup overhead would be minimal, and the images are already loaded. Sounds like a clunky solution, but I guarantee that it is faster and takes less performance that using a slider that can do that using 25+kb of js.


feat: Optional results_aggregation.py regression parameter

The script currently requires the regression parameter to be specified. This forces the user to look up the name of an operation, even if the outcome of the regression analysis is not relevant.
Please make this parameter optional.

Implemented.


[DatePicker] Add missing periods at end of some descriptions.

#24173
I found a few files that were missing periods in the description among the pickers components.

[x] I have followed (at least) the PR section of the contributing guide.

@mbrookes Thanks for taking a look!

Sorry, I approved this prematurely. Could you make the changes in the type definition files instead, and run `yarn proptypes && yarn docs:api'? Thanks!

I actually ended up pulling the changes you made directly to the branch and ran yarn proptypes && yarn docs:api on top of that.


Getting r instead of relevance

The docs describe the some functions return an object with the attribute relevance as in
http://highlightjs.readthedocs.io/en/latest/api.html#highlight-name-value-ignore-illegals-continuation
But using the last version of highlight.js seems to return r instead of relevance
http://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.4.0/highlight.min.js
If you search the minified file you can verify this.
You will find language:, value: and top: but you won't find relevance:

Thanks for reporting this. It is now fixed in master

This is a really old wart of our custom compression scheme, and because of that I suspect we may break some code in the wild that depends on .r in a compressed build rather than .relevance. I'll revert it for 9.5.0 since it would've been a breaking change.
Going forward I see two ways of dealing with the problem:

document .r for compressed builds
remove compression for the identifier relevance altogether

The former is easier and I'd say pretty reasonable. And I don't like the hack with ['relevance'] in some places but not others as it feels too fragile…
Thoughts?

I don't have enough context to know if this is still a relevant issue or not? Anyone else have any ideas?

remove compression for the identifier relevance altogether

Easy enough, doing that.


workers

Hi I'm noticing that the first generation of an pdf when a server is started is a lot slower then the rest.
I have set numberOfWorkers to `2, does that start the workers directly or will the workers not start until there is an conversion?
Or is there any other way of starting workers so that they are running when the first request comes?

I have set numberOfWorkers to `2, does that start the workers directly or will the workers not start until there is an conversion?

they start when the first conversion comes.

Or is there any other way of starting workers so that they are running when the first request comes?

right now, there is no way to start the workers directly without triggering a conversion, it can be a nice addition.
for now you can run an empty conversion right when you create the phantom-html-to-pdf instance if you want to have the workers ready before your first request comes.

I've updated readme to include warmup section.
I believe that running empty conversion is obvious solution. I don't plan to implement an automatic "warm up". If someone wants to give it some time and implement it, I will gladly merge the PR thought.


Release built toolchains

Once we confirm https://github.com/vkottler/toolchains/issues/1, this is the next step.
After this, we would add a download mechanism for projects that want to use these toolchains.

That's complete: https://github.com/vkottler/toolchains/releases/tag/0.1.0!


UTC to Local fails after subsequent converts...

Hi!
first of all, congrats for such amazing framework! :+1:
What i'm trying to do is:

I have a huge (around 90) date inputs on page
I have UTC times on Database
I want to show the local time to the user, instead of the UTC time

Problem:
First UTC to local conversion, works good, (I'm in spain, so +2h to UTC) BUT, after that first one, the subsequent dates add +1h instead of +2h...
So this is an example of the dates

And here is the code (alerts are commented):

I'm sure i'm doing something wrong, but I've being looking for into all Docs but nothing like utc flushing, or forcing format again...

Spain has a standard offset of +1, and switches to +2 for daylight saving time.  Looking at the dates in question, it is converting properly.
However, do keep in mind #831, which may convert some of those older values incorrectly.


Authenticated User Timeline Workbook - Parse Error

The Authenticated User Timeline Workbook does not escape UserId in Session Timeline sec.
KQL snippet:
set query_take_max_records=10001;set truncationmaxsize=67108864;
let monthName = (month:int) {
case(
month == 1, 'January',
month == 2, 'February',
month == 3, 'March',
month == 4, 'April',
month == 5, 'May',
month == 6, 'June',
month == 7, 'July',
month == 8, 'August',
month == 9, 'September',
month == 10, 'October',
month == 11, 'November',
month == 12, 'December',
''
)
};
union customEvents, pageViews
| where timestamp > ago(1d)
| where user_AuthenticatedId == 'domain\testuser' // needs to be escaped
| where name in ('') or '' in ('*')
| summarize Activities = count(), Start = min(timestamp) by SessionId = session_Id
| order by Start desc
| project ['Session Start'] = strcat('⌚ ', monthName(monthofyear(Start)), ' ', dayofmonth(Start), ', ', tolower(format_datetime(Start, 'h:mm tt'))), Activities, SessionId
//'⌚'
Is the expectation that the source data is responsible for escaping?
If not, is there an accepted pattern or utility in other Workbooks?
Happy to submit a PR based on feedback.

@mskailian i think this is an appinsights template, just needs to have that parameter use :escape in the query like {parameterName:escape} ?


Add a few more API headers

Follows @hkaiser's lead and adds headers for exceptions, a few utilities, and algorithms.
It may be that we have to not just forward names to the hpx namespace, but put them there in the first place to get the documentation to look reasonable, but that's a separate issue.

The hpx/future.hpp header is going to be a bit problematic because of hpx::promise. We might have to deprecate the hpx::promise alias first before we change it to hpx::lcos::local::promise. I'll have to check if deprecation attributes work on just an alias or if it will apply it to the aliased name as well.

I think is ready to go now. The API headers are not complete by any means, but this is step in that direction.


[zero-copy] Implement missing primitives

May need more testing, but these should be all the primitives in main but not zero-copy except the deprecated seq

Can you add them to the prelude as well?

Thanks for the PR!


org-agenda error

when open emacs, and SPC o A a,
condition-case: Error in a Doom startup hook: recentf-mode, (error "Defining as dynamic an already lexical var")

Hi! This issue was automatically deleted because it does not meet the minimum requirements for a bug report. There are a couple reasons why this may have happened. Please read the following carefully, because repeatedly failing to meet these standards is a bannable offense.

It ignores the issue template, which asks that you include vital information, such as:

Steps to reproduce the issue.
A link to your personal config (if possible).
Your M-x doom/info.

Your description of the issue is too vague and/or under-researched.

Avoid phrases like “it doesn’t work” or “X stopped working” without explaining what "working" and "doesn't work" mean to you. Better to explain what happened and what you expected to happen, instead.
Link to your personal config. Please don’t paste the contents of your config files in your post – these only pollute the search results. Use services like pastebin or gist, or better: github.
Screenshots and gifs go a long way to communicate an issue. Try to include them, if possible.

You've mentioned errors without including error messages or a backtrace. If you can't get either, include the last 15-20 lines of the *Messages* buffer after the error occurred (SPChe or C-he will open this buffer).

Once you are ready, feel free to open another issue, but please do not ignore the issue template when you do. If you must omit parts of it, please offer an explanation why.
I apologize if this is an inconvenience, but please understand it is a lot of work to support and maintain this project on my own. I depend heavily on you to meet me half way on issues.
Thanks for using Doom!


Real URLs, Preserved Hashes w/ pageResolver

Since this would be a departure from how this plugin works (at least, it would break the current functionality) I don't expect this change to be implemented by you necessarily, however, I'd love to know where to start should I maintain that change for our project.
Summary
I want to link to pages with hash marks, without removing the hash marks, but while benefiting from pageResolver;
Such that all of the following are true:

Input
Default
Currently
Expected

hello world
#/page/hello_world
/hello+world
/hello+world

food
#/page/food
/food
/food

foods
#/page/foods
/food
/food

#help
(fails)
(fails)
... href="permalink#help" ...

drinks#unhealthy
(fails)
/drinks (fails)
/drinks#unhealthy

category:food
#/page/category (fails)
/food (fails)
/Category:food

The Ask
I either need to:

Modify the signature of pageResolver to allow changing the href and label (breaking); or
Have another config function called after pageResolver which allows changing the label
Plugin supporting hash URLs specifically (unlikely, large breaking changes based on default function.)

If there was an option to parse the link before/after pageResolver matching, I would be able to apply this logic myself.
Detail
Our project works closer to WikiPedia, with the following rules on URL generation:
alphabet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ+
separators:  _^-~()<>+.,:;/\\|!&*
Slug is sourced from the document's "Title", with the following transformations:

accented characters are replaced with their latin/arabic equivalents
separators are replaced with +, series of + are squashed into one +
any leading or trailing + are removed
if reserved, /page/${slug}, otherwise, /${slug}

Then when it comes to linking, we have some WikiPedia-esque conveniences:

Every singular word slug is also available at its equivalent plural
Every plural word slug is also available at its equivalents singular
Links with hashes should allow you to link to sections (:boom:)

Using the options provided by the plugin, we could get rather close to realizing this:
    .use(remarkWikiLink, {
      hrefTemplate: (permalink: string) => `/page/${permalink}`,
      permalinks:   getAllDocs().map((x) => x.slug),
      pageResolver: (pageName: string) => {
        const slug = pageName.replace(/ /g, '+').toLowerCase()
        const possible = [slug]
        if (pluralize.isPlural(slug)) {
          possible.push(pluralize.singular(slug))
        } else {
          possible.push(pluralize.plural(slug))
        }

        return possible
      }
    }

However, if I want 3 to work, I need some logic around how the link itself is parsed.
I tried to do this with pageResolver, but this removes the hash from the link.
        const parts = pageName.split('#', 2)
        // part[0] is the actual permalink / page slug
        // part.length > 1 && part[1] is a section hash
        const slug = parts[0].replace(/ /g, '_').toLowerCase()

        // ...

Sorry for the delay. I see what you're saying -- you need the hash fragment for further processing, but to match against permalinks, the page returned by pageResolver mustn't have the hash (or the permalinks list must include hashes, which is unwieldy). And then the matched permalink of course doesn't include the hash.
Here's my proposed solution:
Instead of having pageResolver return a list of permalinks that are matched with permalinks, have pageResolver actually resolve the page and return an object, whose type looks something like this:
{
  id: string,  // Previously `permalink`
  exists: boolean,
  href: string | undefined,
  data: any | undefined
}

TBD on the exact key names, but the idea is that you would do the actual matching yourself in pageResolver (ensuring that exists is set). You could also attach whatever data you wanted, including the hash, for retrieval from the AST later. You could also define the href there instead of using hrefTemplate.
Less relevant to you, but we could retain backward compatibility by using hrefTemplate and permalinks conditionally on the return type of pageResolver.
Would that solve your problem?

{
  id: string,  // Previously `permalink`
  exists: boolean,
  href: string | undefined,
  data: any | undefined
}

Would that solve your problem?

Hmm, then I'd be sending out something like:
[[Food#Important]]

[[Food#References]]

[[Food#nonexist]]

[[Food#nonexist:Alias Always Wins]]

Hypothetical usage of the type you suggested
const allDocs = getAllDocs() // TODO: I need to cache this
// const allDocs = ['food', 'bar', 'baz', 'food#important']

pageResolver: (pageName: string) => {
  const slug = pageName.replace(/ /g, '+').toLowerCase()
  const base = {
    data: null, // what consumes this?
    href: slug,
    id: slug,
  }

  // 4ace13:
  // add the slug and it's plural or singular (whichever is missing)
  // this will fail for unknown hashes, that's okay, see below
  const possible = [{ ...base, exists: allDocs.any(x => x == slug) }]
  if (pluralize.isPlural(slug)) {
    const tmp = pluralize.singular(slug)
    possible.push({ ... base, href: tmp, id: tmp, exists: allDocs.any(x => x == tmp) })
  } else {
    const tmp = pluralize.plural(slug)
    possible.push({ ... base, href: tmp, id: tmp, exists: allDocs.any(x => x == tmp) })
  }

  // if ( slug contains # )
  // ... now what? I can't re-write the label.

  return possible
}

Output I want to make
Important Foods

References (Food)

Nonexist (Food)

Alias Always Wins

Hmm.. I think I didn't understand how to use the example, I realize after trying to think up a usage ^

I really tried to think about how I would end up using this, and hit a bit of a visualization wall.
The pieces of input data that exist are what?
type Input = {
  name:   string
  alias?: string
}

And the output data seems to be something like:
type Output = {
  className: string    // or w/e adds `internal` and/or `new`
  label:     string // doesn't seem I can see/change this atm
  permalink: string
}

What I'm having trouble understanding is how I would match the permalink while masking the label.


Cannot read property '3' of undefined

Hi, i try to use this library but obtain this error message "Cannot read property '3' of undefined".
I am implementing as follows:
In javascript
_cqr = function construirQR(){
var idCajero = $("#idCajero").val();
var idTransaccion = $("#idTransaccion").val();
var template = $("#huellaTemp").val();
var valor = idCajero + "%" + idTransaccion + "%" + template;
var qrcode = new QRCode(document.getElementById("qrCode"), {
text: valor,
width: 300,
height: 300,
colorDark : "#000000",
colorLight : "#ffffff",
correctLevel : QRCode.CorrectLevel.H
});
}

I just tested your code sample replacing jQuery calls with placeholder strings and it works fine, so I'm closing this. If you still have this issue (sorry for the delay btw), feel free to reopen with more details.


Logical error: 'Server 2 client 9 already registered in dispatcher'.

Description
2023-11-16T08:08:44.5759742Z 2023.11.16 07:56:43.746655 [ 125 ] <Trace> ForwardingConnectionHandler: Forwarding handler socket readable
2023-11-16T08:08:44.5760136Z 2023.11.16 07:56:43.746676 [ 125 ] <Trace> ForwardingConnectionHandler: forwarding handler socket available
2023-11-16T08:08:44.5760696Z 2023.11.16 07:56:43.746689 [ 125 ] <Trace> ForwardingConnectionHandler: Try handle new package
2023-11-16T08:08:44.5761128Z 2023.11.16 07:56:43.746708 [ 125 ] <Trace> ForwardingConnectionHandler: Receive Handshake
2023-11-16T08:08:44.5761516Z 2023.11.16 07:56:43.746725 [ 125 ] <Trace> ForwardingConnectionHandler: forwarding handler socket available
2023-11-16T08:08:44.5762034Z 2023.11.16 07:56:43.746898 [ 125 ] <Fatal> : Logical error: 'Server 2 client 9 already registered in dispatcher'.
2023-11-16T08:08:44.5762288Z 2023.11.16 07:56:43.747143 [ 11 ] <Trace> BaseDaemon: Received signal 6

How to reproduce
found in https://github.com/JDRaftKeeper/RaftKeeper/pull/106/checks?check_run_id=18735304047

Fixed by #118


Feedforward capability

Need to build a function to load a previously trained model and run data through, putting the results in a file. Would be best to write this as taking input of a standard form, so that I can test it on binary inputs using existing functions, but later on write some sort of function that will do fits IO

Would be best for this function to be in a different .py file so that feedforward is separated from training. Probably use metagraph saver here so that only one copy of the code defining the graph exists.

MWE finished - need to clean up and make truly feedforward. List of features needed:

Fits IO
Ability to interpolate input data onto the wavelength grid used for training

Closing since it's only the fits io part that remains to build.


release: v1.9.4

Summary

[!IMPORTANT]
I think we should fix #4308 before releasing because it is an important regression.
I will not delay the release too much because it fixes another important regression regarding Unicode handling. If we don't manage to fix #4308 soon, then I will revert the PR that caused this regression.

[x] Add a changelog entry for every Pull Request that lacks one.
You can filter merged PRs that don't update the changelog.
Read our guidelines for editing the changelog.

[x] Based on the changelog, determine which version number to use.
See our versioning guide for more details.

[x] Rename Unreleased to <version> (iso-date) in the changelog.

[x] Update version in Biome's package.json if applicable.

[ ] ~Update to the same version in all crates if you publish crates. (Cargo.toml and crates/**/Cargo.toml)~

[x] Linter rules have a version metadata directly defined in their implementation.
This field is set to next for newly created rules.
This field must be updated to the new version.

[ ] Update the website with the new version number:
BIOME_VERSION=<version> just gen-web.
This will also copy the configuration schema in the right place.

[ ] Once the PR is merged, the CI will trigger the Release: * workflows. Once these workflows finish compiling the final artefact, they need to be approved manually.

@Conaclos The PR https://github.com/biomejs/biome/pull/4301 wasn't supposed to be merged, because it's a new feature. We should revert it, and merge it once we finished with bug fixes.

The PR https://github.com/biomejs/biome/pull/4301 wasn't supposed to be merged, because it's a new feature. We should revert it, and merge it once we finished with bug fixes.

You are right. I reverted the change in the PR. I think we can merge it today :)


refactor(): rework of ocibuilder library and cli

Replaces io.ReadCloser output with channels.
Client for Buildah and Docker
Single OCIBuilder builder.go logic which drives the ocibuilder
All interface with the same client
Command builder for Buildah exec

This is ready to review - ton of changes in this one, restructures a bunch of the codebase


Add SharePoint Loader

Added a loader (SharePointLoader) that can pull documents (pdf, docx, doc) from the SharePoint Document Library.
Added a Base Loader (O365BaseLoader) to be used for all Loaders that use O365 Package
Code refactoring on OneDriveLoader to use the new O365BaseLoader.

@netoferraz Thank you for the contribution! As a heads up I'm in the process of adding a few abstractions to the document flow to decouple loading from parsing code.
General strategy is here:
https://github.com/hwchase17/langchain/pull/2833#issuecomment-1509097756
TLDR;  If you're able to implement a BlobLoader (interface and file system implementation shown below), it'll make it easier for users to add arbitrary parsers on top the loading interface.
class BlobLoader(ABC):
    """Abstract interface for blob loaders implementation.

    Implementer should be able to load raw content from a storage system according
    to some criteria and return the raw content lazily as a stream of blobs.
    """

    @abstractmethod
    def yield_blobs(
        self,
    ) -> Iterable[Blob]:
        """A lazy loader for raw data represented by LangChain's Blob object.

        Returns:
            A generator over blobs
        """

Example:
https://github.com/hwchase17/langchain/blob/master/langchain/document_loaders/blob_loaders/file_system.py#L39

@netoferraz Thank you for the contribution! As a heads up I'm in the process of adding a few abstractions to the document flow to decouple loading from parsing code.
General strategy is here: #2833 (comment)
TLDR; If you're able to implement a BlobLoader (interface and file system implementation shown below), it'll make it easier for users to add arbitrary parsers on top the loading interface.
class BlobLoader(ABC):
    """Abstract interface for blob loaders implementation.

    Implementer should be able to load raw content from a storage system according
    to some criteria and return the raw content lazily as a stream of blobs.
    """

    @abstractmethod
    def yield_blobs(
        self,
    ) -> Iterable[Blob]:
        """A lazy loader for raw data represented by LangChain's Blob object.

        Returns:
            A generator over blobs
        """

Implementation for local file system:
https://github.com/hwchase17/langchain/blob/master/langchain/document_loaders/blob_loaders/file_system.py#L39

Hi @eyurtsev ! I'll try to bring those concepts to this loader and implement it.

@netoferraz Thank you for the contribution! As a heads up I'm in the process of adding a few abstractions to the document flow to decouple loading from parsing code.
General strategy is here: #2833 (comment)
TLDR; If you're able to implement a BlobLoader (interface and file system implementation shown below), it'll make it easier for users to add arbitrary parsers on top the loading interface.
class BlobLoader(ABC):
    """Abstract interface for blob loaders implementation.

    Implementer should be able to load raw content from a storage system according
    to some criteria and return the raw content lazily as a stream of blobs.
    """

    @abstractmethod
    def yield_blobs(
        self,
    ) -> Iterable[Blob]:
        """A lazy loader for raw data represented by LangChain's Blob object.

        Returns:
            A generator over blobs
        """

Implementation for local file system:
https://github.com/hwchase17/langchain/blob/master/langchain/document_loaders/blob_loaders/file_system.py#L39

Hi @eyurtsev ! I'll try to bring those concepts to this loader and implement it.

Hey @eyurtsev and @hwchase17. Finally, I had a chance to work again on this PR.  I decoupled the loading and parser process using FileSystemBlobLoader and  BaseBlobParser. Could you guys review it ?

Are you able to solve the issue? =]

Did anyone have a chance to look at this? Would love to have this merged :-)

Any thoughts on resurrecting this so it can be merged? I'd like to send a PR to improve some of the auth handling and didn't want to step on this because I'd also like Sharepoint support. Thanks!

yes! working to revive

Any thoughts on resurrecting this so it can be merged? I'd like to send a PR to improve some of the auth handling and didn't want to step on this because I'd also like Sharepoint support. Thanks!

It would be great if we can move forward with this PR! I'm not sure if the maintainers have any intentions to accept this work.

updated, would love one more review @netoferraz and @eyurtsev!

@netoferraz :wave:  thank you for the contribution! I left a few comments on the PR, overall looks good to me, so okay merging as is. (cc @baskaryan )
There's a BlobLoader abstraction in the codebase that would fit the requirements here pretty well with an implementation for the file system called FileSystemBlobLoader that can be replicated here. The way it would look would be to declare something like
O365BlobLoader, it will take a bunch of attribtues in the init like auth, and filters, and yield blobs.
Then one could compose it with GenericLoader to apply any sort of parser to content that can be fetched from O365
Not a requirement for merging this PR as we can re-use the existing code at a later point. :)

@netoferraz 👋 thank you for the contribution! I left a few comments on the PR, overall looks good to me, so okay merging as is. (cc @baskaryan )
There's a BlobLoader abstraction in the codebase that would fit the requirements here pretty well with an implementation for the file system called FileSystemBlobLoader that can be replicated here. The way it would look would be to declare something like
O365BlobLoader, it will take a bunch of attribtues in the init like auth, and filters, and yield blobs.
Then one could compose it with GenericLoader to apply any sort of parser to content that can be fetched from O365
Not a requirement for merging this PR as we can re-use the existing code at a later point. :)

Thank you, @eyurtsev ! @baskaryan If you understand that we need to do some additional work based on the @eyurtsev review, let me know, ok? Otherwise, seems we could move to approve this work.


Added same-release argument

Fixes https://github.com/jenkins-x/jx-release-version/issues/21
Added same-release argument if set work only with versions which Major and Minor versions are the same with base project version.
Like we have only 7.2.1 and 7.1.0 tags but our pom is 7.1.0-SNAPSHOT output will be 7.1.1
If no tag found with the same Major and Minor versions our pom is 7.1.0-SNAPSHOT output will be 7.1.0

Fixes #21
Added same-release argument if set work only with versions which Major and Minor versions are the same with base project version.
Like we have only 7.2.1 and 7.1.0 tags but our pom is 7.1.0-SNAPSHOT output will be 7.1.1
If no tag found with the same Major and Minor versions our pom is 7.1.0-SNAPSHOT output will be 7.1.0

I'm not sure I understand. If your repository has the tag 7.1.0, wouldn't your pom be 7.1.1-SNAPSHOT with the changes that would be part of 7.1.1? Are you planning to publish the same 7.1.0 version twice with a different set of changes?

@fiunchinho thanks for the feedback.. We need to be able to release both 7.0.x and 7.1.x at the same time, and that requires that go function to pick the right tags and increment the right numbers for each job.
We need to make sure that when 7.0.x is being released it doesn't pick 7.1.x as the latest and tag 7.1.x for the 7.0.x build/release process.
Hopefully that makes sense.

The idea is that we need to support two versions 7.0.x and 7.1.x in pom file for 7.0.x will be 7.0.0-SNAPSHOT for 7.1.0-SNAPSHOT for 7.1.x if commit inside branch 7.0.x will increment version like had tag 7.0.5 will be 7.0.6 even if tags with 7.1.1 exists

/ok-to-test

Use case makes sense, but I didn't understand the functionality from the command help, so perhaps improve that a bit to explain better? You could include the use case as often explaining the problem you are solving means it's much easier to grasp the solution!

@pmuir thanks for your comment I added additional info to documentation, can you please tell me is it enough?

@fiunchinho updated as proposed

/test this