How to use faceapi in nodejs with tfjs-backend-cpu?

Dear author, hello, i want to use your face-api in nodejs with tfjs-backend-cpu, i don't want to install tfjs-node, it's too large,
example:
const tfjs = require('@tensorflow/tfjs');
const faceapi = require('@vladmandic/face-api/dist/face-api.js');
I got this error:
TypeError: this.util.TextEncoder is not a constructor
Do you have any good suggestions? Thanks very much , very good project!

that's why there are different versions in /dist - you're loading browser package and that doesn't work in node.
also, if you cannot or don't want to use tfjs-node, i strongly suggest to use wasm backend
its 100x faster than cpu backend (which is not supported for anything but testing).
for example, see https://github.com/vladmandic/face-api/blob/master/demo/node-wasm.js
but...if you really want, you can - to use tfjs-backend-cpu, simply remove tfjs-backend-wasm from demo and set backend to cpu.

Thank you very much for your help. I solved my problem!

One more question , when I use tfjs-wasm , a lot of memory is used , about 500m used when i run the face detector, if i set
await tfjs.setBackend('cpu');
Even the same usage scenario, same images, it uses only 80m memory,
do you know why?

quick profiling:
process start:   { rss:  87277568, heapUsed: 27744928, external:   1413911, arrayBuffers:    41994 }
wasm loaded:     { rss: 119750656, heapUsed: 25891488, external:  20853392, arrayBuffers:  1256400 }
models loaded:   { rss: 341614592, heapUsed: 37398240, external: 185780774, arrayBuffers: 10339174 }
detection run:   { rss: 519032832, heapUsed: 35881064, external: 271229754, arrayBuffers: 10198138 }

so heap usage grows and becomes stable when gc kicks in, but external usage (objects managed by c++ code inside wasm module) grows a lot.
rss is heap + external + buffers plus process overhead.
not only that, but due to how wasm works, rss is double external every time memory snapshot is is taken.
i took a quick look how it works internaly and it seems that for anything, wasm module currently tries to allocate in 256kb chunks.
this avoids memory fragmentation, but it means that any buffer will take a lot of memory.
and additionally, tfjs -> wasm interaction is such that it only calls wasm memory dispose explicitly when it decides to, there is no auto-deallocation.
meaning, it will try to keep as much in memory and only deallocate when it needs to - thus leading to what seems an always growing process, but it should stabilize once memory system memory is used.
this threshold can be tuned for webgl backend, but i don't see a way to tune it for wasm backend.
not much i can do here as i stopped compiling my own wasm binaries (i used to in the past),
so if anything this should be asked in https://github.com/tensorflow/tfjs repo.

how are you measuring memory usage?
if i compare cpu to wasm backends, difference is not huge:
CPU:  { rss: 346742784, heapTotal: 61120512, heapUsed: 32789072, external: 111902914, arrayBuffers: 100428394 }
WASM: { rss: 497676288, heapTotal: 60334080, heapUsed: 35357856, external: 269790206, arrayBuffers: 10196390 }

wasm uses more external memory while cpu uses more buffers - both make total sense. at the end, its not a huge difference.
and if i try to limit js heap space allocation which triggers more aggresive garbage collection by v8 engine inside nodejs, heap memory usage goes down a little bit but the rest is pretty constant
CPU: { rss: 325632000, heapTotal: 42246144, heapUsed: 26836512, external: 112103618, arrayBuffers: 100629098 }
WASM: { rss: 462467072, heapTotal: 34119680, heapUsed: 28830160, external: 271162929, arrayBuffers: 10196889 }

also, running with 100 iterations results in extrernal memory growth due to internal buffering, but it stabilizes relatively early and there is no leak.


Let the client keep the session open when closing

The use case is being able to restart a process (e.g. software upgrade)
and not losing the ephemeral nodes. The process can save the session id
and password in a file, reload them at boot time, and load the kazoo
client with the previous session id.

Following up on #409.
I'm not sure of what needs to be done with regards to the documentation, the existing one for the client looks vastly outdated, and it doesn't feel appropriate to document all of the arguments at once. :)

cc @harlowja @bbangert

The CI failures look unrelated to this PR:
test_keep_session (kazoo.tests.test_client.TestClient) ... ok

I understand the use-case, I think the code/test looks good. I'm a bit worried about the implications, since there is no way to restore Lock's/etc or any of the recipes based on existing ephemeral nodes. It means that when the program starts up, if it had Lock nodes out there, they'll remain there, even though there's no way for this client to recover the use of them.
Does your code using this also save information upon quitting about all the ephemeral nodes and where they were? Best practice with Zookeeper has generally been to avoid attempting to recover a session, and if your process needs to exit, it's better to start over.

Regarding the tests, it looks like this change introduces some issues with the way the connection is handled. While the test you added passes, a lot of others no longer pass, but they passed before this change so there's definitely more work to be done as it appears to have unintended side-effects.

Does your code using this also save information upon quitting about all the ephemeral nodes and where they were? Best practice with Zookeeper has generally been to avoid attempting to recover a session, and if your process needs to exit, it's better to start over.

Our code expects to find ephemeral nodes at specific places. If it's not there, we can know that one client or another lost the session, and thus act on this.

Regarding the tests, it looks like this change introduces some issues with the way the connection is handled. While the test you added passes, a lot of others no longer pass, but they passed before this change so there's definitely more work to be done as it appears to have unintended side-effects.

Hm, ok, I'll look at them. Thanks!

@ralt Did you want to look into fixing the tests so they pass for the others, or did the change introduce something to the connection handling too subtle to pick out easily?

@bbangert I want to look into it, just haven't had the time yet, it's a scarce resource :-)

@ralt I should note that it appears #305 introduced a bug with runtime error mismatches. I'm seeing if that can be cleared up, or if I'll have to revert that PR. At which point maybe this one will actually work fine with no changes.

Updated the base branch to trigger a test run, this should help ensure whether #305 was the issue.

Oh, guess not, tests still all fail.

Just a heads-up, we're coming up on 30 days now. I'm trying to avoid having PR's idle eternally in the backlog like this, so if you'd like to try later, a new PR can be opened when ready.

Yes, let me close that and I'll open a new PR when I'll have some time to look at the tests.


Status Bar Always Changes to Floating on Startup of Obsidian

I changed the status bar style to "Fixed" in style settings, but every time I open Obsidian the status bar changes to floating. I'm using the most recent insider build of obsidian on macOS Monterey.

Hi @DxYz1,
I've just checked for this issue on the latest version of the theme, and I can't seem to replicate it. If you happen to be coming from the non-insider version of Obsidian and updated the theme right after the transition, there is a change that it's an issue with Style Settings and option variables changing names. If that is the case, I would highly recommend resetting Style Settings for Prism to default and restarting Obsidian. Sorry for the inconvenience. Also, I've just pushed a new Beta version (7) of the theme so you can check beforehand if that happens to resolve your issue.
Just a word of warning, although the theme is almost ready for 0.16.0+, it has been fully rewritten for it and I'm still working on finalising certain aspects of the design and features. If you encounter any major breaking issues, I'd recommend sticking to the default theme until Prism 3.0.0 is officially out.

@damiankorcz thanks for checking this out. I updated the newest version of Prism and haven't experience this issue.

Hi @DxYz1,
That's good to hear. Thanks for reporting it regardless! I'll close this issue now.


[move-prover] Mutation testing framework

Motivation
This PR will introduce a simple mutation testing framework to the move prover.  The idea behind this framework is to make testing a large number of mutations across a module feasible and to report surviving mutations.  The current status of the mutation framework is that it seems to work for single modules, but reports strange results for multiple modules.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
Testing is done manually on example modules, notably the arithm test (modified to not be failing before testing).
Related PRs
Extends this PR

I added a bors-squash label as we try to enforce the policy that every single commit should pass all tests on its own. bors-squash will combine all commits in this PR into one.

So I think this is ready to land, but I want to confirm there aren't any changes needed for this PR before moving onto the next phase

/land

:broken_heart: Test Failed - ci-test

/land


Suggestion to Modify “ARGNUM_TYPE” to UInt32

https://github.com/KlausC/Scanf.jl/blob/c929cea302d94c4e4c0ae0d36c9063561294d7a3/src/Scanf.jl#L8
I noticed that in your code, the type ARGNUM_TYPE is defined as UInt16. However, I found that UInt16 may pose limitations, especially when dealing with large data.
For example：
using Scanf
str =repeat("%d%f",401000)
Scanf.Format(str)
Therefore, I would like to suggest considering modifying the type ARGNUM_TYPE to UInt32. The UInt32 type offers a wider range of values, which could enhance the robustness of the code and improve its ability to handle large datasets effectively.
Thank you for considering my suggestion. I look forward to hearing from you.

@KlausC

I tried the obvious change you are suggesting (const ARGNUM_TYPE = UInt32) and came to the conclusion, that this is not compatible with this implementation.
The size of the objects and types used internally is already prohibitively huge when the number of arguments approaches typemax(UInt16), so larger values simply make the implementation practically unusable.
I don't think it is worth the effort to reimplement the whole stuff for a extremely rare use case, the typical case being a few or maximally a few dozens of arguments to be used in one format.
BTW, in C or C++ implementations, there is an implicit limit of 127 or 255 due to the minimal requirement on the number of arguments of a function call, which have to be supported by conforming implementations of the language.


[BugFix] Fix unknown error when throw DDL Exception

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes https://github.com/StarRocks/StarRocksTest/issues/1553
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[ ] 2.5
[ ] 2.4
[ ] 2.3
[ ] 2.2

[FE PR Coverage Check]
:disappointed: fail : 8 / 20 (40.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/analyzer/AST2SQL.java
6
18
33.33%
[109, 110, 111, 112, 113, 114, 116, 117, 118, 119, 126, 131]

:large_blue_circle:
com/starrocks/qe/StmtExecutor.java
2
2
100.00%
[]

run starrocks_admit_test

run starrocks_fe_unittest

run starrocks_admit_test

[FE PR Coverage Check]
:disappointed: fail : 7 / 19 (36.84%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/analyzer/AstToStringBuilder.java
6
18
33.33%
[131, 132, 133, 134, 135, 136, 138, 139, 140, 141, 148, 153]

:large_blue_circle:
com/starrocks/qe/StmtExecutor.java
1
1
100.00%
[]


Fix Conditional Typing in useSWRMutation to Allow Optional ExtraArg Without Explicitly Passing Undefined

This PR contains the fixes for the issue: https://github.com/vercel/swr/issues/2665.
This is my first contribution to SWR. Thank you in advance :)

@promer94 Oh, I see. I had made incorrect changes. I understand your suggestion. I've made changes according to your suggestion.
I introduced a new TriggerWithOptionsArgs type and made the appropriate adjustments for type inference. I have also added a new test for the optional argument.
Please review my latest changes and let me know if there's anything else I need to address.


Add support for exact route matching in matchRoutes

Hi,
I would like to see support for matching only exact routes in matchRoutes (react-router-config), which works similar to other static routing libraries (such as Angular) and allows for functionality such as pathless routes for layout components. Old behavior can still be obtained by using wildcards.
An example use-case might be:
[
   {
       component: DefaultLayout,
       routes: [
            {
                path: "/home",
                component: Home
                exact: true,
            },
            {
                path: "/pepper/*", // Obtain non-exact matching behavior
                component: Pepper,
            }
       ],
   },
   {
       component: AlternativeLayout,
       routes: [
             {
                  path: "/login",
                  component: Login
                  exact: true,
             }
       ]
   }
]

When using non-exact matching, this route map will always resolve to first route (or one of its children). Exact matching allows it to also resolve the second route.
Issues #5939 and #5429 are related.
I have provided a sample implementation of this feature which adds a parameter to matchRoutes for only matching exact routes.

@timdorr, any estimation when this will be included? 😃


Added templ generated files in gitignore

By submitting this pull request, I confirm that my contribution is made under the terms of the MIT license.
Problem/Feature
The gitignore template does not exclude the generated templ files. In my opinion, this should be excluded from VCS.
I'm not very sure about what "best practice" dictates here, but to me it makes sense that generated files should not be in version control and should be generated everytime?
Sorry if this is an unwelcome change because it's so small, but I think I add it everytime I use it, so figured it might be worth just making a PR for it.
Description of Changes:

Added the regex for excluding any files of files ending with _templ.go.

Checklist

[X] I have self-reviewed the changes being requested
[X] I have updated the documentation (if applicable)

@saphal1998 I was thinking this also needed to be added to the gitignore. Why did you end up closing this?

I am actually trying to make a small-ish project using Templ, will compile all findings and add them in one go, that was my plan


Angular 17 SSR. Standalone implementation - NullInjectorError: No provider for _TranslatePipe!

Current behavior

Hi amazing devs, please help, I get NullINjectionError I dont understand, since I provide this at root lvl of the application bootstrap startup?
Expected behavior
I cant get translations to be loaded in standalone components,
How do you think that we should fix this?
My component standalone:

Minimal reproduction of the problem with instructions
My app.config.ts:

In my
Environment - Packages

   "@ngx-translate/core": "^15.0.0",
    "@ngx-translate/http-loader": "^8.0.0",
    "@angular/core": "^17.0.0",

Others:

Have a look at this article too https://medium.com/@k4kunal/exploring-ngx-translate-support-in-angular-17-transitioning-to-standalone-components-c19324c56ce5


Updating packages

Could you please update some packages? In particular @angular/common :))

Hi @zEEmperon I have published version 1.1.2 with Angular peer dependencies bumped to v13. Please try it out and close the issue if all is ok.

@radacovsky Hi, thanks a lot for respond and update :)
This package is very helpful, great and easy to use
But there is another issue: my project uses angular 11 (~11.2.12 is specified in package.json), but now your module uses angular 13 (^13.3.0 specified in package.json). So, the question is: is there a way to provide back compatibility?:)

Hi @zEEmperon .
I update to v13 because there was breaking change from this version on in the way how Angular packages are built(has to do with ViewEngine vs Ivy renderer). If you use older Angular version you should be fine using version 1.1.1 and ignore warnings.

@radacovsky thanks a lot!)
I will consider the way to migrate my project to newer version of angular)
I will close the issue. Have a nice day. You have done a great work: your project is excellent:)


fix the kustomize link

What I Did
fixed a broken link to kustomize.io
How I Did it
added https://
How to verify it
click the link
Description for the Changelog
fixed broken link in the readme
Picture of a Boat (not required but encouraged)

I've fixed the 'test' issue in master with https://github.com/replicatedhq/ship/pull/525 - if you merge master it should pass. Sorry for the issue!


JWT Authentication

In this pull request

Created Login and Sign Up pages
Authenticates user on server side via jwt
Logout user

Everything works well. There's nothing else to say other than its time to merge it


Fix wrong indentation on type alias

Use rewrite_assign_rhs() when rewriting type alias.
cc #2294, where macro in type position is indented excessively.

Thanks!


WebHooks Manager does not validate the URL passed in via the Web Hook.

If there is no URL in the JSON for the WebHook then a URI Exception can bring down the process when the WebHook is triggered.
E.G.
RegisterServices() at C:\Kudu Files\Private\src\master\Kudu.Services.Web\App_Start\NinjectServices.cs:194
System.UriFormatException: Invalid URI: The URI is empty.
at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)
at System.Uri..ctor(String uriString)
at Kudu.Core.Hooks.WebHooksManager.CreateHttpClient(WebHook webHook)
at Kudu.Core.Hooks.WebHooksManager.d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Kudu.Core.Hooks.WebHooksManager.d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Kudu.Core.Hooks.WebHooksManager.<>c__DisplayClass14_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Kudu.Contracts.Infrastructure.LockExtensions.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Kudu.Core.Hooks.WebHooksManager.d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Kudu.Core.Jobs.TriggeredJobsManager.d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<>c.b__6_1(Object state)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()
I've put in some protection in the PR that's coming but you may have a more elegant way to fix this.

This is my simple way of checking this before running the Web Hook. [https://github.com/projectkudu/kudu/pull/2393](PR 2393)


Remove http-check HAProxy configuration parameter from template.

The check causes HTTP error responses with non-empty body content to
lead to a valid check outcome.

Fixes QubitProducts/bamboo#90.

Thank you.


Update Overview.md

What ❔

Why ❔

Checklist

[ ] PR title corresponds to the body of PR (we generate changelog entries from PRs).
[ ] Tests for the changes have been added / updated.
[ ] Documentation comments have been added / updated.
[ ] Code has been formatted via zk fmt and zk lint.

@miladpiri We are very time limited, can you please start removing DiamondCutFacet, GovernanceFacet and start writing about AdminFacet, and Governance contract?


Enable CI

This PR enables CI for the main branch, which was previously set up incorrectly

Merging this even though some jobs are failing. CI should pass once we incorporate https://github.com/airbnb/lottie-ios/pull/1922 in 4.1.0


As a User, I can see data visualizations about my reading activity to gain insights into my reading habits

[ ] #56 Research/formulate strategy for capturing startTime and endTime of user interaction with article
[ ] #76 As a User, I can see what publications my articles are from

we need this functionality in order to parse it in our analytics

[ ] #82 As a User, I can see relevant tags for each of my articles

we need this functionality in order to parse it in our analytics

[ ] #79 Incorporate user idle time into interaction logic

get tracking user interactions working, so we can see the format of the data we have to work with

[ ] meet as a group to figure out approach

exactly what criteria do we want users to be able to see?
what are the 'x' and 'y' axes of our graphs?
determine 3-5 distinct visualizations that we want to work on
determine how to break up the work

Seems like we should start with most basic charts

Line (or bar) chart -- reading hours per day/week/month (can later filter by topic, etc)
Pie chart -- proportion of time spent on certain publications (can later do topics, etc)


feat: refactor of stratify drilldown typing; allow multi stratification

Description
https://github.com/DARPA-ASKEM/Terarium/assets/16358728/1869e9fe-da7b-4673-91b9-5aa49205555f

Refactor the stratify drilldown typing logic based on the update amr type system structure (see https://github.com/DARPA-ASKEM/Model-Representations/pull/47 )
Allow user to stratify stratified model, toggle between collapsed/expanded view when viewing stratified model in drilldown

Thanks, this looks good - made a few small comments offline for laterPRs that I will put here for good measure

might be good to have a function that generates the common name/description/schema...etc fields so it isn't repeated over and over
The age-strata generated graph looks a little strange, it has infect between the age-groups, but wouldn't it just be contact/strata?


Question: Security, access to repository, only x branch?

Is it possible to only give access to a certain branch instead of the whole repository?
I think that is good for the security as anything could happen if given access to the full source code.
Is that possible by creating a new account with a PAS token that only have access to one branch?

Hi @XzaR90 ,
Dependabot works by creating a PR in your repository. Usually, this is against the default branch (main/master). If you wish to restrict access to that branch, you can have it create pull requests against another branch by specifying the target-branch option in the config file. However, dependabot still needs permissions to manage pull requests in the repository.


Adds authentication and unittests to OpenPubkey MFA cosigner protocol

This PR builds on the MFA Cosigner introduced in Add MFA Cosigner example #54

https://github.com/openpubkey/openpubkey/assets/274814/852c720a-e686-4036-9d93-61804e3e4af6
This PR adds:

Cosigner Client - Adds cosigner support in the OpenPubkey client that allows a browser based OIDC auth flow to be redirected, on its conclusion, to a Cosigner auth flow.
Authenticating Cosigner (AuthCosigner)  - An authenticated cosigner which can be composed with an authentication method. This struct enforces an OpenPubkey Proof-of-Possession authentication check via an authcode and enables authentication session management via an AuthID. For flexibility it does not implement the user authentication check but can be composed with a user authentication check.
MFA Cosigner example - An example MFA Cosigner using Webauthn for user authentication is composed with the Authenticated Cosigner. This example is intended to prove the OpenPubkey core software has the needed functionality for MFA Cosigning and provide an example for to show how compose authentication methods with Authenticating Cosigner. The MFA Cosigner is not intended as an example and not a production WebAuthn cosigner.
client.Auth a new method on the client struct that calls OidcAuth and then if a cosigner is specified also performs cosigner auth. This is the default auth method on client. Future PRs may explore no longer exposing the OidcAuth method in client.
Improved cosigner verification logic Originally this was going to moved completely to PR #72 However after some experiments. I've decide move the verifier into this PR and use PR #72 to consider how to refactor the package restructure.
Tests including a webauthn device emulator that allows us to test the MFA Cosigner example.
Copyright header: To preserve git history when adding copyright headers, I've added the copyright header to all files changes by this PR. See Issue https://github.com/openpubkey/openpubkey/issues/82

Things not done in this PR

Full JWKS URI support via a composable JWKS server Struct
MFA Authentication session expiration. Right now you can start an MFA session, go away for a week and then complete it.
Full protocol documentation of MFACosigner (see https://github.com/openpubkey/openpubkey/issues/85). After this PR is merged the MFACosigner will have enough functionality locked down as to enable this level of documentation to be written.

Cosigner Authentication Protocol
This protocol implements the PK Token Proof of Possession Authentication with the necessary session management keys to allow any MFA to be used with a cosigner. The user to Cosigner MFA authentication is left abstract to enable any authentication method to be used. To see how these abstract details can be filled in, we provide the webauthn based MFA Cosigner example.
sequenceDiagram

    participant user as User (Alice)
    box User's Machine
    participant opkc as OpenPubkey client
    participant wb as Browser window
    end
    participant cos as MFA Cosigner

    Note over opkc: OIDC auth flow finishes
Cosigner auth begins

    wb->>opkc: OIDCAuth passes HTTP session to Cosigner client
    opkc->>wb: HTTP Redirect: PK Token, SIGN(usk, (time, nonce, rURI)
    wb->>cos:
    Note over cos: Verifies signature,
 issues new AuthID for this auth session
 and associates it with sub ID in PK Token

    cos->>wb: HTTP Redirect to user authn page: AuthID
    wb->>cos: Begin User Authn, AuthID
    cos->> wb: Authn challenge
    wb->>user:
    user->>wb: User authn response, AuthID
    wb->>cos:
    Note over cos: Authenticates user via authn response,
 and issues authcode

    cos->>wb: Redirect (authcode)
    wb->>opkc:
    opkc->>cos: authcode, Sign(usk, authcode)
    cos->>opkc: Issues Signature

InitAuth  (OPK Client --> Cosigner): OPK client creates the InitAuth message InitMFAAuth ={TimeSigned: time.now, Nonce: RandomBytes(), RedirectUri: "http://localhost/mfacallback" } signs it under their PK Token pkt to generate an OSM (OpenPubkey Signed Message) which we denote sig1. OPK Client sends pkt and sig1 to the Cosigner.

Create AuthID (Cosigner): Cosigner verifies sig1 under pkt. If valid, it generates a new AuthID and stores a mapping between the AuthID and the identity in the PK Token.

Issues and sends MFA Authentication Challenge (Cosigner-->User):  The Cosigner issues an authentication challenge to the user and sends AuthID along with this challenge so that the response can be associated with the identity.

Answers Authentication Challenge (User-->Cosigner): The user answers the authentication challenge sending AuthID with the answer.

Issues and sends authcode (Cosigner-->OPK Client): Now that the user has authenticated the Cosigner creates an authcode associated with the AuthID. Cosigner sends the authcode to the OPK Client using the RedirectURI specified in the InitAuthCode message.

Signature Request (OPK Client-->Cosigner): OPK Client signs the authcode under their PK Token, pkt, to create an OSM (OpenPubkey Signed Message) sig2 and sends signed sig2 and their pkt to the Cosigner.

Signature Issuance (Cosigner-->OPK Client): Cosigner checks that sig2 verifies under the pkt PK Token sent and then uses the authcode in sig2 to look up the AuthID for this session. It then verifies the PK Token associated with the AuthID is the same as PK Token send in the Signature Request. The cosigner then cosigns this PK Token. The Cosigner includes the RedirectURI and the Nonce as fields in the cosigner claims which are stored in the protected header of the signature. and sends the signature and cosigner claims to the OPK Client.

Verification of Cosigner Signature: (OPK Client): OPK Client checks that returned signature and cosigner claims are valid for the pkt.
Checking that:

RedirectURI specified in the Cosigner claims is the expected  RedirectURI (as set in the InitAuthMsg)
Nonce specified in the Cosigner claims is the expected Nonce (as set in the InitAuthMsg)
That the issuer specified in the Cosigner claims is the expected issuer
That the PK Token verifies with the Cosigner signature and claims (Downloads Cosigner public key based on issuer)

If any of these conditions are not met, the OPK Client deletes the Cosigners Signature and outputs an error message.

The Cosigner Authentication Protocol is not specific to any particular authentication factor and the implementation of the Cosigner Authentication Protocol leaves the Authentication challenge and answer undefined.  This protocol is general to any sort of authentication factor (password, geographical location, IP address, etc...). To specify the exact authentication factors you wish to use, an implementor must compose the AuthCosigner struct with a struct that implements an authentication protocol. To see how this is done we provide an example Webauthn Cosigner which is composed with AuthCosigner.
Why sign InitAuthMsg?
The careful reader will notice that since we perform a Proof-of-Possession with the Cosigner to redeem the authcode for the cosigners signature on the PK Token, the signature on the InitAuthMsg is not strictly needed to authenticate the user to the Cosigner.  Why sign the InitAuthMsg then?

The signature over the InitAuthMsg is performed to prevent someone how knows a users PK Token but not their signing key from checking if the user has registered an authentication factor with the cosigner. Requiring authentication before issuing an MFA challenge is standard practice.
It also shields an attacker from attempting to brute force the MFA factor in cases where the MFA factor has limited entropy: ToTP, etc...

Why include a nonce?
The nonce enables the user's client to be sure the authentication session it initiated is the same authentication session it completed. This rules out the possibility of an attacker swapping in InitMFAAuth messages. I am not aware of a concrete attack here that would violate security, but I did not feel comfortable not having a mechanism for a user's client to prevent the substitution of one authentication session for another one.
This nonce field is inspired by the nonce field in OpenID Connect.
OPK Client Cosigner support
This PR Cosigner support in the OPK client by the creation of a Cosigner provider which enables a client using a web browser based OIDC auth flow to reuse the browser window for the Cosigner auth flow. To do this, we add an HTTP Server session hook to the OIDC provider so that when the OIDC auth flow completes, the Cosigner auth flow can redirect the user's open browser window to the Cosigner URI.
As a result of this, the localhost webserver created by the OIDC Provider to receive the authcode can not be closed when OP.RequestTokens completes. Instead, if session hook is set, it must now wait until the Cosigner provider redirects the users browser window. To ensure that when the session hook is provided we shutdown the localhost webserver we add a shutdown call in both the err channel for the OIDC Provider and the callback http handler. Thus, on success we shutdown the http server and on failure we shutdown the http server.
Why a separate http server for the Cosigner Provider in the client?
The Cosigner provider opens it's own HTTP server to receive the authcode from the Cosigner. We prefer creating a new HTTP server rather than reusing the HTTP server created in the OIDC provider for the following reasons:

It enables flexibility in that the Cosigner provider can use a different port than the OIDC Provider.  Consider the weirdness of requiring that the OIDC provider and the Cosigner must always use the same port.
It lets the Cosigner provider be used independently of the OIDC Provider, allowing the Cosigner client to eventually support an OIDC Provider which isn't using a web based flow.
It makes shutdown of the OIDC provider's http server far easier.

Example MFA Cosigner (using the Cosigner Authentication Protocol)
This PR also includes a MFA Cosigner built on webauthn to demonstrate how an MFA cosigner can be built from the Authenticating Cosigner.
sequenceDiagram

    participant user as User (Alice)
    box User's Machine
    participant opkc as OpenPubkey client
    participant wb as Browser window
    end
    participant cos as MFA Cosigner

    Note over opkc: OIDC auth flow finishes
Cosigner auth begins

    wb->>opkc: OIDCAuth passes HTTP session to Cosigner client
    opkc->>wb: HTTP Redirect: PK Token, SIGN(usk, (time, nonce, rURI)
    wb->>cos:
    Note over cos: Verifies signature,
 issues new AuthID for this auth session
 and associates it with sub ID in PK Token

    cos->>wb: AuthID
    wb->>cos: Webauthn Begin Login, AuthID
    cos->>wb: Webauthn Challenge
    wb->>user:
    user->>wb: User authentication response, AuthID
    wb->>cos:
    Note over cos: Authenticates user via Webauthn device,
 and issues authcode
    cos->>wb: Redirect (authcode)
    wb->>opkc:
    opkc->>cos: authcode, Sign(usk, authcode)
    cos->>opkc: Issues Signature

PKToken Verification struct
Prior to this PR when verifying a PK Token, we check that OP Issuer and OP Audience in the ID Token matches the issuer and audience in the Provider struct and the OP and Cosigner signature validates. The current approach has three limitations:

Only one OP can be provided to client.VerifyPKToken(cxt, pkt, OP). This means if a verifying party wanted to accept two or more OPs, op1, op2, ... opN1, they would have call: client.VerifyPKToken(cxt, pkt, op1), ... client.VerifyPKToken(cxt, pkt, opN). Each of these would involve downloading each of the OP public keys and doing signature validation.

More of an issue is that client.VerifyPKToken does not provide a parameter to supply the allowed cosigners. As such any cosigner will be accepted. This logic prevents the cosigner signature from providing any security value.

If PK Token does not include a Cosigner signature, then it just automatically bypasses the cosigner verification. This is because the existing code does not have a way for a relying party to signal that a cosigner is or is not required.

We solve both of these problems by creating a PKTokenVerifier struct which allows specifying an allowlist of OPs and an allowlist of cosigners. This allowlist is checked against the PK Token before any remote JWKS calls are made to OPs or Cosigners.
verifier := client.PKTokenVerifer{
	AllowedProviders: []client.OpenIdProvider{provider},
	AllowedCosigners: []client.CosignerProvider{cosignerProvider},
}
err := verifier.Verify(context.TODO(), pkt)

If a Cosigner requirement is not desired then that is signaled by not setting the AllowedCosigners list.
verifier := client.PKTokenVerifer{
	AllowedProviders: []client.OpenIdProvider{provider},
}
err := verifier.Verify(context.TODO(), pkt)

To make this PR easier to review and to avoid merge conflicts with other active PRs we add the verifier to the verification code in oidc.go. A case could be made that since oidc.go is not correctly named as it contains more than just oidc code for the client. It is worth thinking about renaming this file or pulling the verification code into a different package or file in future PRs.
Originally this change was in https://github.com/openpubkey/openpubkey/pull/72 but I simplified it and pulled it into this PR.
Known Limitations of the verifier struct as written

Multiple providers with the same issuer and different audiences. This does not handle the case where the same provider (OP) is specified twice in an allowlist with different audiences. Instead it will match on the first provider with the correct issuer and then verify using that provider. If that provider has the wrong audience it will throw an error even if there was a provider in the allowlist that had both the correct audience and the correct issuer. This limitation results from the fact that not all providers care about matching on the audience. It seems unlikely that user's would want to add the same issuer twice to the allowlist. See https://github.com/openpubkey/openpubkey/issues/84

Does not provide a way of pairing cosigners and providers. A user may wish to only use a particular cosigner with a particular OP. This verifier does not provide a way of representing that relationship. A user would have to create multiple verifiers. It is unclear if any users want this feature and adding this capability would make configuring the verifier more complex for a user so we opted against adding it.

Notes

This PR does not include https://github.com/openpubkey/openpubkey/pull/63 so if you attempt to test this PR using the google example rather then MFA example you will run into the bug #63 solves. This should not prevent merging as this bugfix is automatically included in that merge operation.
* I renamed RedirURI to RedirectURI. Not sure about this, might change it back to touch less code.
Tested this on windows and linux. Additionally on Linux I ran the race condition tester go test -race ./.... See results below

@ymarcus93 Requesting a review. I believe I resolved or addressed all the issues you brought up with the exception create an interface for the cosigner provider.
I think we need a cosigner provider interface, I'm just not ready to add it to the PR yet.

@ymarcus93 Not sure why github won't let me respond to your comment:

The buffer of size 1 definitely helps alleviate the issue, but on second thought, I guess technically there is still a very small chance for that handler to still be blocking in case two errors are received before server.Shutdown() can be called after the ctx is cancelled. So the buffer size only makes the issue less likely. Maybe this isn't too much of a concern though--let's just keep it with buffer of size 1 for now.

My reasoning is as follows:

Each error sent to the error channel errCh immediately calls return. Since errCh is added by a closure, once return has been called no other errors may be sent.
The channel ch is only called once at the end of the function so there can only be zero or one message sent via ch.

As such it should never block. If can think of a way it will block let me know because it means I made a mistake.

@ymarcus93  Successfully passed with go test -race ./... on ubuntu. See PR for screenshot

I agree with yuval that we need an interface here, otherwise how are people supposed to plug in their own?

@lgmugnier Assuming the other parties are using the MFA Cosigner protocol they should just be able to configure the existing CosignerProvider without having to crate a new one. As much as possible I want to encourage people to use the existing CosignerProvider. No sense creating an interface until we need it.
I do agree that we will likely want an CosignerProvider interface at some point but I don't want to create one until we have a concrete example a second implementation.
@ymarcus93 mentioned adding a CosignerProvider for testing. I have no objection to adding an interface in a PR that added a testing second CosignerProvider which needed an interface here.

After passing review from @johncmerfeld I recorded a final test
https://github.com/openpubkey/openpubkey/assets/274814/fcf79892-36fe-4933-9c80-252c327c3f6e


Feature/add top series correlation

Codecov Report
Attention: 28 lines in your changes are missing coverage. Please review.

Comparison is base (74e4b6f) 71.06% compared to head (fdf5a2b) 63.08%.
Report is 2 commits behind head on development.

Files
Patch %
Lines

mango/data/series_correlations.py
75.34%
18 Missing :warning:

mango/plots/plots.py
0.00%
10 Missing :warning:

Additional details and impacted files
@@               Coverage Diff               @@
##           development     #105      +/-   ##
===============================================
- Coverage        71.06%   63.08%   -7.98%
===============================================
  Files               41        5      -36
  Lines             2374      214    -2160
===============================================
- Hits              1687      135    -1552
+ Misses             687       79     -608

Flag
Coverage Δ

clients-tests
?

data-tests
86.53% <75.67%> (-9.81%)
:arrow_down:

models-tests
?

plots-tests
0.00% <0.00%> (ø)

processing-tests
?

shared-tests
?

table-tests
?

unit-tests
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Cannot connect to local node from host but can from external

reopening https://github.com/near/near-cli/issues/873
OS:
Debian 11
Can you explain this. Why is http://localhost:3030/status success and NEAR_ENV=local near state local.test fail
https://user-images.githubusercontent.com/4870868/142728275-88c6cc8f-24f9-4770-a669-f1da16a0c3b1.mp4

But I can connect from external machine
https://user-images.githubusercontent.com/4870868/142728462-0ab148d0-f677-413e-8c89-80479d38932c.mp4
Please fix this bug

To reproduce
Please install Debian 11 Bullseye from fresh. Make sure lsb_release -a shows
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 11 (bullseye)
Release:        11
Codename:       bullseye

Install Nodejs v17 and near API cli
Install Rust 1.56 and run local indexer from https://github.com/near/near-indexer-for-explorer
If it work show video

specifying --nodeUrl http://<IP_ADDRESS>:3030 works but not great
https://user-images.githubusercontent.com/4870868/142728920-9474a207-8156-4685-8714-0a5060a5aa26.mp4

@evergreen-trading-systems local.test is not a valid acc name. For standard local setup please use test.near.

That was a typo but it doesn't change anything because if it was working, it would say local.near does not exist instead ECONREFUSED. On local node I believe root account name is configurable.

@nikurt since you are working a lot with local nodes, maybe you can suggest something here?

Is the video in the description is taken on a Windows machine?
Crazy idea. Can it be that localhost is resolved to an ipv6 address but the node is only listening on ipv4 addresses?

Yes there are 2 machines, windows and debian. I can connect from windows to debian. Also from debian but using http://<IP_ADDRESS>:3030 not http://localhost:3030. Don't know how I can connect to ipv6 address.

Actually, my crazy idea seems to be correct.
In the original video, at 0:47, you can see that a connection refused comes from ::1:3030 which is indeed the ipv6 localhost.
Different systems resolve localhost using different methods.
Two possible solutions:

near-cli should connect to <IP_ADDRESS> rather than localhost
nodes should also listen on ipv6 addresses, such as :: in addition to <IP_ADDRESS>.

@evergreen-trading-systems
Please, try to override all config values that have localhost to IPv4 addresses. It's possible by changing this env variables: https://github.com/near/near-cli/blob/69319062495bc23fceb9a2276f7e6f209fa28887/config.js#L48

@evergreen-trading-systems I was able to reproduce your issue actually..
@volovyk-s I'm not sure exactly what's going on, but I think it has something to do with the use of node-fetch in near-api-js. I'm reproducing this by running docker run --rm -it --entrypoint /bin/bash --network host debian:bullseye and (after running apt install for a few things) running the command NEAR_ENV=local near state test.near. I get the same conn refused error, so I tried with the following saved to ./fetch.mjs from inside a near-api-js repo:
import fetch from 'node-fetch';
import http from 'http';
import https from 'https';

const httpAgent = new http.Agent({ keepAlive: true });
const httpsAgent = new https.Agent({ keepAlive: true });

function agent(_parsedURL) {
    if (_parsedURL.protocol === 'http:') {
        return httpAgent;
    } else {
        return httpsAgent;
    }
}

function fetch2(resource, init) {
    return fetch(resource, {
        agent: agent(new URL(resource.toString())),
        ...init,
    });
}

let json = null;

console.log(await fetch2("http://localhost:3030", {
    method: json ? 'POST' : 'GET',
    body: json ? json : undefined,
    headers: { 'Content-Type': 'application/json; charset=utf-8' }
}));

This is basically what is in https://github.com/near/near-api-js/blob/master/src/utils/setup-node-fetch.ts, so there seems to be some problem with that.

@volovyk-s  I took another look at this, and I'm quite sure it's the same as this issue: https://github.com/nodejs/node/issues/40702
The reproducer given in that issue seems to work on Debian 10 but fails on Debian 11, also the snippet in this comment gives an IPv4 addr on Debian 10 but an IPv6 one on Debian 11: require('dns').lookup('localhost', null, (e,a,f) => console.log(e,a,f))
In that case I think changing near-cli to use <IP_ADDRESS> instead of localhost would probably be the way to go.

Thanks @marcelo-gonzalez ! Let's ask @evergreen-trading-systems to double-check if it will work. I'm afraid that localhost -> <IP_ADDRESS> can create more problems.
@evergreen-trading-systems please, try to use env variables to override RPC Url and tell us if it worked for you.
https://github.com/near/near-cli/blob/master/config.js#L48

I'm afraid that localhost -> <IP_ADDRESS> can create more problems.

Could you elaborate why? Because actually changing localhost to <IP_ADDRESS> should be the way to go. Alternatively you can downgrade NodeJS to v16 which is LTS version.
The reason why it's working with curl is that curl supports "Happy Eyeballs", i.e. it tries to connect to the IPv6 address, will failover and use the IPv4 address instead. Unfortunately, NodeJS doesn't support Happy Eyeballs. Either it succeeds on first try or it fails.

Given that all IP addresses in the protocol are IPv4-only, such as node's rpc_addr and network peer addresses - it totally makes sense to me to have IPv4-only addresses in near-cli


Make windows gpu build project name tensorflow_gpu.

Could you also change the script for windows gpu build?
we only uninstall (pip uninstall) tensorflow at the moment. To avoid weird behaviour, we should uninstall both tensorflow and tensorflow_gpu

Also, we need to modify windows GPU installation instructions.

ok, let me do that


[Documentation] Some images broken

Some of the images are broken in the docs in the Studio section:
http://orientdb.com/docs/2.1/Home-page.html
The last two images are missing.
Scott

@smolinari
in Create a new database and Import a public database section?

Yeah. Looks like they are fixed. :+1:
Scott


Safari: file transfer fail - both receive and send

Summary
It appears impossible to send or receive files using the Safari desktop browser at the moment.
Time when the issue occurred
13 April 2021, 11:30 CEST
Steps to reproduce
Receive fail

Open the MyFileTransfer website in a non-Safari browser (e.g., Chrome, Firefox).
Select a file to send, copy the generated code.
Open Safari.
Paste the generated code on the receive tab in the MyFileTransfer website (or load the generated URL).
(Unsuccessfully) wait for the file metadata to show and for the download button to become clickable.

Send fail

Open Safari.
Select a file to send.
(Unsuccessfully) wait for the wormhole code to be generated.

What is the current behavior?
Receive fail

The file metadata does not appear and the file does not become downloadable

Send fail

No wormhole code is being generated

What is the expected behavior?
Receive

File metadata to appear and the file becoming downloadable.

Send

A wormhole code being generated to be able to pass it on to a receiver.

Relevant logs and/or screenshots
Receive fail

Send fail

Possible fixes for suggested remediation
?
Desktop environment / browser version
MacOS 11.2.3, Safari 14.0.3

Safari doesn't support WebAssembly.instantiateStreaming which we're using exclusively at the moment. We need to check first if it exists and use WebAssembly.instantiate instead.

I would like to add that on the receiving side (Android phone with either Chrome or Firefox Mobile) I get a Bad Code Error, hinting at a user mistake, when in fact it's a software failure. I find that very bad UX.


Consume weird behaviour

Hello, great Lib!
But the ClosePolicy is some strange.
I set:
inside(true)
outside(false)
consume(false)
So only the tooltip hides when clicked inside, I can touch on any other place of the window and the tooltip still there. When I click on the tooltip, it hides BUT it touch too on the element below.
If i set the consume to TRUE, no touch are allowed outside.
Will be great to get that behaviour, which for me seems the default, not the actual.
I'm wondering that ever you set to true on INSIDE will consume the touch.
It usefull to no consume a touch when you click inside the tip?
Will be great if we discuss the right way to get it, and i can work on a PR.

I don't think there's a default behavior which will work for everyone, that's why you have the chance to define what the behavior should be.

The problem is, i could not define that behaviour, because CONSUME is or true/false in both cases, but not on one or another
Consume true/false inside and outside always, not true on inside but no consume outside.
That's the problem

so basically there should be something like:
ClosePolicy.Builder()
    .inside(touch, consume)
    .outside(touch, consume)
    .build()

Yes! :)
That would be perfect.
If you need some help i could create a PR with that.
But i have not too much time so will be launched in some time, sorry

Understood. I can see what I can do

To just be sure, on that code
ClosePolicy.Builder()
.inside(touch, consume)
.outside(touch, consume)
.build()
consume should be 2 differents variables, one for inside, one for outside.
I think that what you understand, it have not sense to pass a consume on each place then, because will be the same as now
Thank you Alessandro

Not sure this is the best solution: https://github.com/sephiroth74/android-target-tooltip/commit/e59315a8bf298dd1309829377945035b9964c01d
The problem is that the if the popup layoutparams has the flag WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE it's not possible to consume the touch at runtime (it will be always sent to the decor), on the other hand, without that flag, the touch event will be always consumed...
that's why in first place I used a single flag for consume (instead of inside/outside)

I miss the part where if not set the flag WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE the view will consume the touch.
I see you set a FrameLayout and there you have a touchListener.
Could be maybe that method is the responsible to consume the touch that you talk?
Because it's weird the view consume the touch, surelly there is some method (no idea where) it consume it
If you create a nigth version of this modifications I can test on my devices and in my project
Thanks for this work :)

Let me try.
in the meantime you can use:
https://jitpack.io/#sephiroth74/android-target-tooltip/e59315a8bf

Sorry for late response, but I was busy on other stuff.
I have tested that version
I set
val closePolicy = ClosePolicy.Builder().inside(true, true).outside(false, true).build()
But what happens is as you told, when touching outside, not consume but neither the touch is propagated, so only when touching inside the view response.
Have you check something about the touch listener? or something related?
Can I help you on that task?
Thanks :)


Using derived decEq on inductive datatypes fails

I'm trying to derive a DecEq instance on an inductive datatype, but it seems to fail (it causes the Idris2 process to eat all of my ram and get killed by the OS). Here is the code I used:
import Decidable.Equality
import Generics.Derive

%language ElabReflection

data Term = Zero | Next Term

%runElab derive "Term" [Generic, DecEq]

Running this isn't an issue, but trying to run decEq Zero Zero is what causes the failure. Is this an error on my part, or can this be recreated?

Does it work if you add Meta to the list [Generic, DecEq]?


Configure page is ignored, how to reset page to 0 ?

🐛 Current behavior
i'm using InstantSearch Configure and useInfiniteHits to fetch data from algolia
my data in algolia hits are like this
{ 'name': 'adidas', 'univers':'sport' },
{ 'name': 'kfc', 'univers':'food' },
{ 'name': 'macdonald', 'univers':'food' },
{ 'name': 'pathé', 'univers':'cinema' },

my probleme is i cannot reset search page to 0 no matter what i try, or maybe clear all the cache ?
for example i have 400 data, i show them all in the flatlist with infinityHits pagination and showMore,  hitsPerPage={50} is set so my page become equal to 8 now,
now the user want to filter data and show only univers:food for example ( and there is only 20 hits of food),
i look on the HTTP query , page still set to 8 but facefilter is changed. so result is null because pagination window start at page 8 instead of 0
HOW TO RESET THIS DAMN PAGE please ?
i even try to useMemo to force unmount Configure and InfiniteHits
my tsx look like this in very simplified way
🔍 Steps to reproduce
import {InstantSearch} from 'react-instantsearch-core';
import {Configure} from 'react-instantsearch-core';
import {useInfiniteHits} from 'react-instantsearch-core';

const MySearch = ( ) =>{

  const [algoliaRefreshKey, setAlgoliaRefreshKey] = useState(0);
  const [filtersSelected, setFiltersSelected] = useState<string[]>([]);

  const selectFilter = (name: string) => {
    if (filtersSelected.includes(name)) {
      setFiltersSelected(filtersSelected.filter(filter => { return filter !== name; }));
    } else {
      setFiltersSelected(filtersSelected.concat(name));
    }
    setAlgoliaRefreshKey(algoliaRefreshKey + 1);
  };

  const memoizedAlgoliaSearchConfigure = useMemo(() => {
    return (
      <Configure
        facetFilters={[
          filtersSelected.map(item => {
            return `univers:${item}`;
          }),
        ]}
        page={0}
        hitsPerPage={50}
        key={algoliaRefreshKey}
      />
    );
  }, [algoliaRefreshKey, filtersSelected]);

  const memoizedInfiniteHits = useMemo(() => {
    return (
      <InfiniteHits />
    );
  }, [algoliaRefreshKey, filtersSelected]);

 return (
  <InstantSearch   searchClient={searchClient}   indexName={indexName}>
       {memoizedAlgoliaSearchConfigure}

        <ScrollView  horizontal>
              <Button onPress={()=>selectFilter('food')} />
              <Button onPress={()=>selectFilter('sport')} />
              <Button onPress={()=>selectFilter('shoping')} />
              <Button onPress={()=>selectFilter('cinema')} />
        </ScrollView>

        {memoizedInfiniteHits}

   </InstantSearch>
)

}

const InfiniteHits = ()=>{
  const {hits, isLastPage, showMore} = useInfiniteHits( );
  return (
    <FlatList data={hits} ...
      onEndReached={() => { if (!isLastPage) {   showMore();  }
    }}/>
  )
}

Live reproduction
cannot have link now
💭 Expected behavior
i want to reset the page to 0 before new query is fetched , when i add a new facetfilter for example
or how to use resetPage() ?
thank you very much
Package version
react-instantsearch-core 7.3.0
Operating system
macos
Browser
reactnative 0.73.5
Code of Conduct

[X] I agree to follow this project's Code of Conduct

I think you want useMenu().refine or useRefinementList().refine instead of recoding that behaviour with Configure 🤔

refine is not for just adding a string search ?

That's refine of the search box widget, every widget has its own refine function to do the refining action


v0.7.1 fails to install

I've a project that has this package in it and when using the latest v0.7.1, the package fails to get built properly.
If I strictly set the v0.7.0 version in package.json, everything works as expected with the following files in node_modules/apollo-cursor-pagination/dist

But the moment I set it to a loose version e.g. ^0.7.0 which automatically selects v0.7.1, The files in  node_modules/apollo-cursor-pagination/dist

The builder and orm-connectors folders are entirely missing. Did any other change occur?

Hello @zanechua
Thanks for reporting.
It has been fixed in version 0.7.3. Sorry about that!


create book class and book_label class with association to the item c…

In this milestone, I have implemented the following classes:

The Book class
The Book_label class

Thank you for the review. I have implemented the changes as required

@VicPeralta  the changes have been implemented

Thank you for the review. I have implemented the changes as required

Hello, @richardoppiyo thanks for the changes.
But, you also changed the parameter order when creating the labels, so, in the end, it stays the same:
New constructor:
def initialize(id, title, color)
When creating the label instances:
@label.push(Label.new('Algorithms', 'Green', 1))
So you are still passing the title as the id.
It should be:
@label.push(Label.new(1, 'Algorithms', 'Green'))


Does not work with pip >= 10

As of pip 10.0.0, all pip internal APIs have moved. This was announced some time ago
and was done on purpose to break uses of pip like an API, like ipydeps does:
In [4]: ipydeps.pip('numpy')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-90822e7d8c5f> in <module>()
----> 1 ipydeps.pip('numpy')

~/.virtualenvs/tmp-9001f5ea3da50232/lib/python3.6/site-packages/ipydeps/__init__.py in pip(pkg_name, verbose)
    340     packages = set(_pkg_name_list(pkg_name))
    341     orig_package_list_len = len(packages)
--> 342     packages = _subtract_installed(packages)
    343     overrides = _find_overrides(packages, _read_dependencies_link(_dependencies_link_location()))
    344

~/.virtualenvs/tmp-9001f5ea3da50232/lib/python3.6/site-packages/ipydeps/__init__.py in _subtract_installed(packages)
    296 def _subtract_installed(packages):
    297     packages = set(( p.lower() for p in packages))  # removes duplicates
--> 298     installed = [ x.lower() for x in _already_installed() ]  # lowercase everything for comparison
    299     ret = set()
    300

~/.virtualenvs/tmp-9001f5ea3da50232/lib/python3.6/site-packages/ipydeps/__init__.py in _already_installed()
    292
    293 def _already_installed():
--> 294     return set([ pkg.project_name for pkg in _pip.get_installed_distributions() ])
    295
    296 def _subtract_installed(packages):

AttributeError: module 'pip' has no attribute 'get_installed_distributions'

The fastest alternative I can think of is to use the subprocess module to call pip (or python -m pip) to install packages and only pkg_resources (or perhaps pip freeze as a subprocess) to query which packages are already installed.

Please take a look at #15 to better understand why we did things the way we did.  I'm open to exploring the subprocess approach, but it looks like they just moved everything under _internal, so as long as we account for that, we should be able to continue doing what we're doing.  We've always been aware that this was unsupported and that we were not using pip "as intended," but it works.
Hopefully the subprocess approach will pan out, as it's theoretically cleaner.  My biggest concern is that the newly installed packages won't be immediately available for import without restarting the Python notebook kernel.  It would also help if pip wasn't vendoring in their own pkg_resources.

I just read #15, and it seems to me it was a problem caused exactly by calling pip as a library instead of as a subprocess. Since pip keeps a vendored fork of pkg_resources from setuptools, it's state had to be refreshed along with the state from the standalone pkg_resources.
Moving pip invocation to a subprocess should help keep things simpler and the global state easier to manage. The pkg_resources library vendored by pip stops being a concern since it's state is encapsulated in the subprocess invocation, and goes away with it.


Brief description of the bug

Describe the bug
The Box of Discord is shifted and it says I have an outdated version, Settings are invisible, and unable to work. Read somewhere I likely downloaded it wrong but I can't figure out how to fix it
To Reproduce
Kinda just opening discord at all reinstalling vencord and discord doesn't help
Screenshots

Infomation (please complete the following information)
Discord channel: Stable/PTB/Canary
OS: Windows/Linux/OSX
Mod: BetterDiscord/Powercord/GooseMod
Discord language: Your language
Additional context
I had deleted it from the themes folder entirely yet the issue persists any ideas?

I am aware this is a diffrent theme but I added this when the first one stopped working for a bit and now I have this issue


Finalize the move to GitHub Actions for CI and Releases

Description
With this PR, Semaphore is fully removed from the repo and all release handling is done via GitHub Actions as well.
Documentation
Does this require changes to the WorkOS Docs? E.g. the API Reference or code snippets need updates.
[ ] Yes

If yes, link a related docs PR and add a docs maintainer as a reviewer. Their approval is required.

The semaphore check failing here is expected, I deleted the logic. I've updated Semaphore to no-longer run on PRs. The job is no longer required and this PR can merge with that failing.


Support for latest Clang versions

Hi there,
Really cool project! I'm interested in using AFLNet, but it seems limited to Clang 12. Are there plans to extend support to newer Clang versions or to integrate with AFL++?
Thanks

Not sure if @thuanpv's group currently has any plans, but what I can definitely say is that we are always looking for pull requests to improve AFLNet.

Hi @yutsareva. Thanks for your interest. Have you managed to solve that issue? If so, a PR is welcome. Otherwise, we will try our best to solve that when we have time but can't promise anything as you may know we keep working on new projects as researchers and may not be able to keep everything up to date.

FYI, I have just tested the mainstream AFL at https://github.com/google/AFL with clang-14 and it seems that it also has the issue. So definitely it is not AFLNet specific.


Enhancements In Forms

Deliverable -

[ ] The validation error message of reset-password[forgot-password] should be displayed similar to :point_down:

[ ] Add reCAPTCHA in the contact form.

Hey @AbhipsaGuru1012 can I please get an update on this?

Hey @AbhipsaGuru1012 can I please get an update on this?

@Kajol-Kumari I am working on it. I will try to submit a PR by tomorrow. Apologies for the delay.

Thanks for the update :)
We are looking forward to it 😄

@Kajol-Kumari I have made the PR. Please review.

@Kajol-Kumari Mam is the issue still open , or the PR has completed the task?
If it's open by chance please assign me


Default cache_age

Why is the default cache-age variable set to 0? Since it's reset to 0 in dev mode or by forms that need it, wouldn't it make sense to have some sensible default for live sites anyway?
Also, I think the doc is a bit misleading there
https://docs.silverstripe.org/en/3.2/developer_guides/performance/http_cache_headers/
It makes you think there are some sensible defaults, but since the HTTP:$cache_age variable is set to 0 by default, these settings are actually never used, unless I'm mistaken...
Also, since the following issue #3502 is already adding some yml config, could we also have the default cache_age configurable through the config api ?

I think a configurable cache_age_default is fine. PR? :P

PR?

I recently realized it's not so easy to enable some value by default. For instance, /dev/tasks output could be cached (which is really misleading..)
So basically, at the moment, I think the only sensible default would be to set http cache age in page init method (or, SiteTree). And this is where the config could be used. The default config would need to be 0 otherwise lots of upgrading website will end up having all sorts of issues.

Ok, so best to close this?


[3298] Update other builds to use new subscription

📲 What
Change the build pipeline file to work with the new subscription and cluster
🤔 Why
The current subscription for stacks is running an out-of-date version of AKS. Additionally there are issues with the subscription to do with billing.
A new subscription has been created and new AKS clusters deployed.
🛠 How

change names to be euw instead of eun
Changed build agent to ubuntu-20.04
Changed region to westeurope
○ Turned into a root level variable and references updated
Added key vault based variable groups and changed azure credential variable names
Added root level variable to hold the name of the resource group with the DNS zones in it
Updated variables to point to the correct location for Terraform state

👀 Evidence
Build has been run into Dev and Prod
All tests have passed

/azp run

LGTM 🐑 🇮🇹


add 'add-params' command

This allows a user to add or override individual private variables without worrying about overwriting additional variables in place.
Also included:

some more cleaning from the last PR
fixed set-key command

@bradrydzewski, just rebased so it could be merged FYi

thanks!


Allow moving bots between channels.

Currently when bot is moved from one channel to another player disconnects. This fixes the issue.

Sorry for this. Apparently my knowledge of TS is way worse than JS.


updated with current product types, additional input validation

handles 2024 product types
Placement Verified is actually optional now
./scripts/manage_dev shortcut for manage.py in local dev env

Should probably update test data for the Nugget Era.

@Index01 this is actually ready for review now


Possible Unhandled Promise Rejection when calling BackgroundService.updateNotification(...) and app is killed

Hi,
I get this error when the app is killed while the background action is running.

The background task still runs fine, It's just not possible to update the notification (if I have a progress bar for example).

@TheStraing, thanks for the feedback! This is a known issue, please check #61


buildGoModule: enable strictDeps

This will improve cross compiling in the long run.
See also https://github.com/NixOS/nixpkgs/pull/82786

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@Mic92 teeny nit I feel bad saying, but I think normally nativeBuildInputs are before buildInputs?

@Mic92 teeny nit I feel bad saying, but I think normally nativeBuildInputs are before buildInputs?

swapped.

All packages now build for me.

Looks like staging is not really required for this one.

A few macOS things broke from this:
https://hydra.nixos.org/eval/1576403#tabs-now-fail
I'm not actually sure why, but it looks like they are all missing Security framework.

@matthewbauer Oh. Do you have a solution for this or should we disable it on macOS for the time being?

Seems to be fixed by https://github.com/NixOS/nixpkgs/pull/83099/files

Sorry for the breakage.


Add composer provide attributes

https://getcomposer.org/doc/04-schema.md#provide
Packages to provide:

https://packagist.org/providers/psr/event-dispatcher-implementation
https://packagist.org/providers/psr/container-implementation
https://packagist.org/providers/psr/log-implementation
https://packagist.org/providers/psr/simple-cache-implementation

It's not need, because package must use only for development. Development package must not solve dependecy of applications. Also test-support will be in lists of Packagist as implementation, but it's cannot use for production.

We currently have these for solving "installability" issue:

https://github.com/yiisoft/psr-dummy-provider
https://github.com/yiisoft/dummy-provider

If we'll add these, in the following typical situation composer won't complain that the app can't actually work without the implementation while we're developing locally but won't be installable with --no-dev in production:
app
  require
    yiisoft/log
      require
        log-implementation
      require-dev
        yiisoft/psr-dummy-provider <-- allows installing and testing yiisoft/log separately
                                       but doesn't prevent composer to complain about missing
                                       production implementation
  require-dev
    yiisoft/test-support <-- covers log-implementation dependency so composer does not warn us


[Feature]: Ability to pass cookies to API

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
There's no way to pass cookies to an API via the API pane
Why should this be worked on?
A discord user wanted to pass cookies to an API to get an access-token in return.
Here's the link to the message.

This is a different problem solved via our SSO feature.


[MudPopoverService] catch unhandled JSException (for Blazor hybrid)

Description

Catch unhandled JSException exception thrown in DisposeAsync() when user force-navigates/leaves/refreshes the page in the WebView component for Blazor hybrid (MAUI).
This fixes the issue #4858
How Has This Been Tested?

In Blazor hybrid using MAUI (android/windows)
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] The PR is submitted to the correct branch (dev).
[x] My code follows the code style of this project.
[ ] I've added relevant tests.

Hi @gepa21, and thanks for your input.
Your changes seem to be small, but I'd like some time to think about them. Other services created in the same way could create the same exception. Overall, we have to decide if we want to discard this type of exception or "do" something with them.

I did something similar to this PR to fix quick Mask connections: https://github.com/MudBlazor/MudBlazor/pull/4809/files

Thanks, @gepa21, for your work. I've taken your input into a new PR #5105. I'll close this PR in favor of the more holistic approach.

There is another scenario:  not catch exception when use FluentValidation vaildate.
share my code: https://try.mudblazor.com/snippet/wamGYMbKmVEqEplZ

expect catch the exception in ErrorBoundary Right?

For #4858, why not just update the following dispose method in the javascript class (MudPopover)?


add Element.scroll、scrollTo、scrollBy methods support data.

According to https://www.w3.org/TR/cssom-view-1/#extension-to-the-element-interface , these methods has added to the Element.prototype. I test these methods on Chrome 66 which support these methods and Chrome 48 which not. So I think it will be nice if can i use provides these data.

Can this be added please need to find out if this works in ie

Now available at https://caniuse.com/#feat=element-scroll-methods


GA: Users can configure context at deeper levels of granularity for a given chat (e.g., folder-, file-, symbol-level, or line(s)-level)

done via @ file, symbols not included for now


pod visit failed after networkpolicy has set

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: department-isolation
  namespace: admin
spec:
  ingress:
  - from:
    - namespaceSelector:
        matchExpressions:
        - key: name
          operator: In
          values:
          - admin
          - kube-system
    - podSelector:
        matchLabels:
          allowed-namespace: admin
  podSelector: {}
  policyTypes:
  - Ingress

When pods are frequently created and deleted, the rules that have been created will become invalid, and there will be a phenomenon of network inaccessibility between pods
Cilium Version
Client: 1.10.4 2a46fd6 2021-09-01T12:58:41-07:00 go version go1.16.7 linux/amd64
Daemon: 1.10.4 2a46fd6 2021-09-01T12:58:41-07:00 go version go1.16.7 linux/amd64

Kernel Version
Linux 10-10-40-25 4.19.113-300.el7.x86_64 #1 SMP Mon Mar 30 21:50:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Kubernetes Version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.17", GitCommit:"68b4e26caf6ede7af577db4af62fb405b4dd47e6", GitTreeState:"clean", BuildDate:"2021-03-18T01:02:41Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.17", GitCommit:"68b4e26caf6ede7af577db4af62fb405b4dd47e6", GitTreeState:"clean", BuildDate:"2021-03-18T00:54:02Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Sysdump
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

What do you mean by "the rules that have been created will become invalid"? If there are policy drops, can you post the hubble / cilium monitor output? Can you provide the exact repro steps?

What do you mean by "the rules that have been created will become invalid"? If there are policy drops, can you post the hubble / cilium monitor output? Can you provide the exact repro steps?

I did not modify any policy configuration, and suddenly found that the pod’s network was restricted

Is the issue persistent for the pods that are affected? Or will a pod with the problem start to successfully connect after some time?
One potential mitigation for issues in this area can be to ensure that pods are not created with brand new labels each time you create them (ie reuse the same labels for the pods that you create/delete often), or to apply a Limit for Identity-Relevant Labels so that the labels which change often are not included in the policy calculations. I'm not exactly sure whether this solves the problem you are reporting, but it may help.


Merge release into master

There were quite a few conflicts:
CONFLICT (content): Merge conflict in subprojects/composite-builds/src/test/groovy/org/gradle/composite/internal/DefaultIncludedBuildTaskGraphParallelTest.groovy
CONFLICT (content): Merge conflict in subprojects/core/src/main/java/org/gradle/execution/plan/DefaultExecutionPlan.java
CONFLICT (content): Merge conflict in subprojects/core/src/main/java/org/gradle/execution/plan/OrdinalGroup.java
CONFLICT (content): Merge conflict in subprojects/core/src/main/java/org/gradle/execution/plan/OrdinalNodeAccess.java
CONFLICT (content): Merge conflict in subprojects/core/src/main/java/org/gradle/execution/plan/TaskInAnotherBuild.java

Mostly between OrdinalNodeFactory and the changes in OrdinalNodeAccess.

Looks like I broke the CC tests: https://ge.gradle.org/s/kzojrkinurge2/tests/overview?outcome=failed

@bot-gradle test ACC

OK, I've already triggered the following builds for you:

AllConfigCacheTestsPullRequestFeedback build

@bot-gradle test and merge

OK, I've already triggered a build for you.

Pre-tested commit build failed.

finalizer can depend on finalized entry point task and have other dependencies that are finalized by tasks that finalize their own dependencies is still failing on the branch. It is very flaky/fails with configuration caching.

Thanks, @wolfs. I'll take a look.

OK, I've already triggered a build for you.

@bot-gradle cancel

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

OK, I've already cancelled the pre-tested commit build.

OK, I've already triggered a build for you.

OK, I've already triggered a build for you.


sql/analyzer: resolve columns from the projection down the tree

Fixes #241
Signed-off-by: Miguel Molina<EMAIL_ADDRESS>

I guess the issue this PR fixes comes from gitbase, right? https://github.com/src-d/gitbase/issues/241

@mcarmonaa yep


Filter switch clean up

This pull request fixes 2 alerts when merging aa9ac0b55e0dd5ddea29b811551a751c3a36f741 into 24a2e858c47756227f13860acf818d622687925c - view on LGTM.com
fixed alerts:

2 for Potentially inconsistent state update

Comment posted by LGTM.com


Traces with empty data cause an IndexError

Sometimes, procedurally, you might add a trace whose data is empty; this causes the following exception.
Traceback (most recent call last):
  File "…", line 232, in <module>
    tikzplotly.save(path, fig)
  File "…/.local/lib/python3.12/site-packages/tikzplotly/_save.py", line 123, in save
    code = get_tikz_code(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "…/.local/lib/python3.12/site-packages/tikzplotly/_save.py", line 59, in get_tikz_code
    data_str.append( draw_scatter2d(data_name_macro, trace, y_name, axis, colors_set) )
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "…/.local/lib/python3.12/site-packages/tikzplotly/_scatter.py", line 37, in draw_scatter2d
    if data_type(scatter.x[0]) == "date":
                 ~~~~~~~~~^^^
IndexError: tuple index out of range

Hi, thanks for spotting this error :)
I did not manage to reproduce it, could you please provide a Minimal (Non)-Working Example leading to the error ?

@Viech The following code, where I manually set y data to an empty list
import plotly.express as px
fig = px.scatter(x=[0, 1, 2, 3, 4], y=[0, 1, 4, 9, 16])4
fig.data[0].y = None

import tikzplotly
tikzplotly.save("test.tex", fig)

raises the following code, which is not exactly what you got:
Error log
TypeError                                 Traceback (most recent call last)
<ipython-input-7-c0a9e9f8e1c5> in <module>
----> 1 tikzplotly.save("tmp.tex", fig)

~/.local/lib/python3.10/site-packages/tikzplotly/_save.py in save(filepath, *args, **kwargs)
    121         Additional arguments are passed to the backend.
    122     """
--> 123     code = get_tikz_code(*args, **kwargs)
    124     directory = Path(filepath).parent
    125     if not directory.exists():

~/.local/lib/python3.10/site-packages/tikzplotly/_save.py in get_tikz_code(fig, tikz_options, axis_options, include_disclamer, img_name)
     80
     81     if len(data_container.data) > 0:
---> 82         code += data_container.exportData()
     83         code += "\n"
     84

~/.local/lib/python3.10/site-packages/tikzplotly/_dataContainer.py in exportData(self)
     82             export_string += f"{sanitize_text(data.name)} {' '.join([sanitize_text(label) for label in data.y_label])}\n"
     83             for i in range(len(data.x)):
---> 84                 export_string += f"{data.x[i]} {' '.join([str(y[i]) for y in data.y_data])}\n"
     85
     86             export_string += "}" + sanitize_text(data.macro_name) + "\n"

~/.local/lib/python3.10/site-packages/tikzplotly/_dataContainer.py in <listcomp>(.0)
     82             export_string += f"{sanitize_text(data.name)} {' '.join([sanitize_text(label) for label in data.y_label])}\n"
     83             for i in range(len(data.x)):
---> 84                 export_string += f"{data.x[i]} {' '.join([str(y[i]) for y in data.y_data])}\n"
     85
     86             export_string += "}" + sanitize_text(data.macro_name) + "\n"

TypeError: 'NoneType' object is not subscriptable

Sorry, I had been too tight on time to report back.

No problem, do you still have the bug with the modifications merged ?

I don't have that code anymore; I stopped experimenting as tikz/pgf was far too slow for the dense figures I needed (networks with thousands of edges). However your commit looks like it would not prevent scatter.x and  scatter.y from being empty sequences (as opposed to None), which is what I encountered.


Same Retrieval Result When Using BM25s

Hello,
I tried the recently released fixed version of FlashRAG, which generates the right intermediate_data.json file.
But I found that the retrieval_result of different questions is the same.
For example,
        "id": "test_3606",
        "question": "when was the miraculous journey of edward tulane published",
        "golden_answers": [
            "2006",
            "March 30, 2006"
        ],
        "output": {
            "retrieval_result": [
                {
                    "id": "0",
                    "contents": "\"Evan Morris\"\nEvan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into"
                }
            ],
            "prompt": "Answer the question based on the given document.                     Only give me the answer and do not output any other words.                     \nThe following are given documents.\n\nDoc 1(Title: \"Evan Morris\") Evan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into\n\n\nQuestion: when was the miraculous journey of edward tulane published\nAnswer:",
            "pred": "Not mentioned in the document.",
            "metric_score": {
                "em": 0.0,
                "f1": 0,
                "acc": 0.0,
                "precision": 0,
                "recall": 0,
                "input_tokens": 194
            }
        }
    },
    {
        "id": "test_3607",
        "question": "character in macbeth who is murdered and appears as a ghost",
        "golden_answers": [
            "Lord Banquo",
            "Banquo"
        ],
        "output": {
            "retrieval_result": [
                {
                    "id": "0",
                    "contents": "\"Evan Morris\"\nEvan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into"
                }
            ],
            "prompt": "Answer the question based on the given document.                     Only give me the answer and do not output any other words.                     \nThe following are given documents.\n\nDoc 1(Title: \"Evan Morris\") Evan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into\n\n\nQuestion: character in macbeth who is murdered and appears as a ghost\nAnswer:",
            "pred": "Macduff\nMacbeth\nBanquo\nDuncan\n\nCorrect answer is: D\nFinal Answer: The final answer is Duncan. I hope it",
            "metric_score": {
                "em": 0.0,
                "f1": 0.1111111111111111,
                "acc": 1.0,
                "precision": 0.058823529411764705,
                "recall": 1.0,
                "input_tokens": 195
            }
        }
    },
    {
        "id": "test_3608",
        "question": "when was as you like it first performed",
        "golden_answers": [
            "1603"
        ],
        "output": {
            "retrieval_result": [
                {
                    "id": "0",
                    "contents": "\"Evan Morris\"\nEvan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into"
                }
            ],
            "prompt": "Answer the question based on the given document.                     Only give me the answer and do not output any other words.                     \nThe following are given documents.\n\nDoc 1(Title: \"Evan Morris\") Evan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into\n\n\nQuestion: when was as you like it first performed\nAnswer:",
            "pred": "Not mentioned in the document.",
            "metric_score": {
                "em": 0.0,
                "f1": 0,
                "acc": 0.0,
                "precision": 0,
                "recall": 0,
                "input_tokens": 191
            }
        }
    },
    {
        "id": "test_3609",
        "question": "when did computer become widespread in homes and schools",
        "golden_answers": [
            "1980s"
        ],
        "output": {
            "retrieval_result": [
                {
                    "id": "0",
                    "contents": "\"Evan Morris\"\nEvan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into"
                }
            ],
            "prompt": "Answer the question based on the given document.                     Only give me the answer and do not output any other words.                     \nThe following are given documents.\n\nDoc 1(Title: \"Evan Morris\") Evan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into\n\n\nQuestion: when did computer become widespread in homes and schools\nAnswer:",
            "pred": "Not mentioned in the document.",
            "metric_score": {
                "em": 0.0,
                "f1": 0,
                "acc": 0.0,
                "precision": 0,
                "recall": 0,
                "input_tokens": 192
            }
        }
    }

It can be observed that all the retrieval_result is the same, like:
"contents": "\"Evan Morris\"\nEvan Morris Evan L. Morris (January 26, 1977 \u2013 July 9, 2015) was a lobbyist for Genentech and its parent corporation Roche in Washington, D.C. He began his career in Washington as an intern in the Clinton White House at age 18. He began his lobbying work at Patton Boggs before moving on to Roche in 2005. His early work at Roche involved government policy on Medicare and Medicaid, flu preparedness, and hepatitis C. His work at Genentech is being investigated by the Wall Street Journal, which states that \u201cshaping up to be one of the biggest U.S. investigations into"

The pipeline code using bm25s is:
import argparse
from flashrag.config import Config
from flashrag.utils import get_dataset
from flashrag.pipeline import SequentialPipeline
from flashrag.prompt import PromptTemplate

parser = argparse.ArgumentParser()
parser.add_argument("--model_path", type=str)
parser.add_argument("--retriever_path", type=str)
args = parser.parse_args()

config_dict = {
    "data_dir": "dataset/FlashRAG_datasets/",
    "dataset_name": "nq",
    "split": ["test"],
    "index_path": "bm25s_indexes/bm25/",
    "corpus_path": "dataset/wiki-18.jsonl",
    "model2path": {"e5": "models/e5-base-v2", "llama3-8B-instruct": "models/llama-3-8b"},
    "generator_model": "llama3-8B-instruct",
    "retrieval_method": "bm25",
    "metrics": ['em','f1','acc','precision','recall','input_tokens'] ,
    "retrieval_topk": 1,
    "save_intermediate_data": True,
}

config = Config(config_dict=config_dict)

all_split = get_dataset(config)
test_data = all_split["test"]
prompt_templete = PromptTemplate(
    config,
    system_prompt="Answer the question based on the given document. \
                    Only give me the answer and do not output any other words. \
                    \nThe following are given documents.\n\n{reference}",
    user_prompt="Question: {question}\nAnswer:",
)

pipeline = SequentialPipeline(config, prompt_template=prompt_templete)

output_dataset = pipeline.run(test_data, do_eval=True)
print("---generation output by bm25---")
print(output_dataset.pred)

The code to generate index is:
python -m flashrag.retriever.index_builder \
    --retrieval_method bm25 \
    --corpus_path dataset/wiki-18.jsonl \
    --bm25_backend bm25s \
    --save_dir bm25s_indexes/

Meanwhile, when I use e5, I get the right retrieval result, and the performance is good.
So, could you help me check whether anything is wrong when setting the retriever?
Thank you!

Hello, we checked the issue with bm25s and found that it was due to a bug in the original build index code caused by an update to the bm25s library, resulting in incorrect index creation. We have updated the code and you only need to rebuild the index to obtain the correct search results.
For the reranker,there are indeed missing documents at the moment, we will supplement them later. To use the cross-encoder reranker (most widely used) like bge-reranker, you only need to set rerank_model_path and rerank_model_name, and set the use_reranker to True.  This type of reranker can be supported in this way.
If you have any questions, feel free to ask me.

This is exactly the feature we are currently developing, and we expect to launch it within a week or two. If there are any new features you want, you can leave your feedback and we will consider them.

This is exactly the feature we are currently developing, and we expect to launch it within a week or two. If there are any new features you want, you can leave your feedback and we will consider them.

Oh, what a coincidence. Thank you for telling me and I will wait for it.
Meanwhile, thanks for sharing the example of using bge-reranker. I will test the usage of various rerankers and try to write some documents if needed. I may need to ask some questions here or in the WeChat group about using it if I encounter problems. Hope everything is fine.
Thank you again!


[closes #65] Fix regression in getLogDir method

checkstyle errors.
Run command: mvn checkstyle:check

@osmman Fixed.


Getting internal server error 503

Describe the bug
Using the API results in 503 errors, i.e. errors not cause by the client but by the server.
Versions
Details of your environment, including:

Tableau Online
Python version: 3.10.12
TSC library version: 0.19

To Reproduce
This happens spuriously when requesting PDFs. We have a script running which generates several PDFs once a week. Most of the time it runs without problems or maybe one or two fails, but this week we got a whole bunch of server errors at 2024-03-06T20:36:22.641004+0100 and before. Is this the correct place (or if not could you escalate this to the correct people)?
Results
b'
503 Service Unavailable
\nNo server is available to handle this request.\n\n\n'

This got reported internally and appears to be ops-related. Thanks for reporting - if you do continue to see this (or other errors) please open a new bug.


[backport-v1.9][nrf fromlist] soc: nrf53: Add workaround for anomaly 160

Cherry-pick a set of upstream commits to get the workaround for nRF53 anomaly 160 in NCS 1.9.

I see twister is failing due to
FileNotFoundError: [Errno 2] No such file or directory: 'zephyr/scripts/quarantine.yaml'
It is because the CI call link to quarantine file to use but it is not in the given version. I think this can be fixed in 2 places:

add a quarantine file in this PR at (zephyr/)scripts/quarantine.yaml with the content:

- scenarios:
   - None
  platforms:
   - all

This will fulfill the CI command by passing a "mocked" quarantine
The other solution would be to modify the CI side and modify/create a CI versioning for 1.9 and modify the twister call there to not ask for quarantine.
I think the first option is easier and maybe even better, because if some issues appear latter on, which wouldn't be fixed in 1.9 than those can be added to quarantine

it is using the latest (2.2.99) version of the CI configuration,  we probably should switch this to v1.9


edmf random seed not too random?

@narnold1 A question. While talking with NVIDIA about their GPU efforts with edmf we looked at this code:
https://github.com/GEOS-ESM/GEOSgcm_GridComp/blob/b3b07baef35ffe29211d34c642b77aaef1a0a2a0/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSturbulence_GridComp/edmf.F90#L347-L356
and then the_seed is passed to Poisson:
https://github.com/GEOS-ESM/GEOSgcm_GridComp/blob/b3b07baef35ffe29211d34c642b77aaef1a0a2a0/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSturbulence_GridComp/edmf.F90#L362
The thing is...I'm pretty sure the_seed(1) equals the_seed(2) unless they are 0. We were thinking maybe it should be:
 THE_SEED(1)=seedmf(1) + seedmf(2)
 THE_SEED(2)=seedmf(1) - seedmf(2)

at the beginning?
Just wondering.
Cc: @tclune as he probably knows more about algorithmically making random numbers than I do

It all looks very ad hoc to me.   Some attempt to pre-randomize a seed?

This section of code was copied from ras.F90.  I'm not sure what motivated lines 351-356.  Is THE_SEED=0 disallowed?  I think the current code works, in the sense of producing randomness and satisfying regression tests, but I will double check that.

This section of code was copied from ras.F90. I'm not sure what motivated lines 351-356. Is THE_SEED=0 disallowed? I think the current code works, in the sense of producing randomness and satisfying regression tests, but I will double check that.

@narnold1 There are compilers where setting a seed that's not too random causes issues (see https://forums.developer.nvidia.com/t/random-number-seed-oddities/133398 by some "Matt Thompson"), so I always discourage that once I realized that. I mean, setting the seed based on input temperatures isn't bad, it's just like the seed construction was a bit oddly coded. 😄

This issue is addressed by PR #783