Added benchmark for early removal of unused raster cache entris mecha…

benchmark for https://github.com/flutter/engine/pull/34627
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I signed the CLA.
[ ] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[ ] I added new tests to check the change I am making, or this PR is test-exempt.
[x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

I went ahead and added the autosubmit label so this can get into the tree to start gathering baseline data.


Layer in partner auth

Paolo: once you review, will hold off deploying until we are together to verify in production post deploy

Lgtm. We should go through the steps of creating a token for quiet tomorrow.


Revert "Bump redis from 4.1.4 to 4.2.1"

Reverts davidrunger/david_runger#2968

See https://github.com/davidrunger/david_runger/pull/2949

I have just ticked on this box in the dependabot config for ruby updates in this repo:

Hopefully that will prevent dependabot from making this same version bump again.


fix(Page.getMetrics): do not report a few metrics

A few of the reported metrics in the Page.getMetrics are already
reported with lifecycle events.
This patch excludes these metrics for now.

Oh god. Why did you @aslushnikov remove this metrics?
Can you point how to now get FirstMeaningfulPaint DomContentLoaded NavigationStart times without:
await page._client.send('Performance.enable');
const metrics = await page._client.send('Performance.getMetrics');

?

Oh god. Why did you @aslushnikov remove this metrics?
Can you point how to now get FirstMeaningfulPaint DomContentLoaded NavigationStart times

@Everettss you can get DOMContentLoaded and NavigationStart from window.performance DOM api. But, we'll bring them back!
There is too much movement in the First*Paint metrics, so we're hesitant exposing them at this point. What are your usecases for them?

@aslushnikov

There is too much movement in the First*Paint metrics, so we're hesitant exposing them at this point. What are your usecases for them?

Like... measuring performance?


UsePrePareContractWrite will stuck with Phaser to cooperate with PHASER task and get stuck in the interface

Is there an existing issue for this?

[X] I have searched the existing issues

Package Version
0.6.4
Current Behavior
No response
Expected Behavior
No response
Steps To Reproduce
No response
Link to Minimal Reproducible Example (CodeSandbox, StackBlitz, etc.)
No response
Anything else?
No response

Thanks for reporting. This issue is temporarily closed due to lack of information. Please provide a minimal reproducible example and reopen this issue. Other information is also helpful, like framework (e.g. Next.js), browser, etc.


Feature Request: Add the ability to create and manage Git pull requests from VS Code

Hi, I have been using VS Code lately and the one thing that really frustrates me  about the Git integrations is  not being able to manage git Pull Requests from VS Code. I wanted to kindly ask whether you could add the ability to create and manage git pull requests from VS Code. This will greatly simply workflows that require pull requests without having to leave VS Code

+1 I also need this

Pull requests are a GitHub concept, not git. This would make a fine extension, would you like to try it?

You could use this one vscode-github


[Gally]: master <- dev

Automatically created by Git-Ally

:tada: This PR is included in version 2.10.25 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Bug - Date javascript object convert to {} on $meteor.collection(..).save

file angular-meteor-utils.js line 28
if( !angular.isObject(data) || data instanceof File) { return data; }
replace to
if( !angular.isObject(data) || data instanceof File || data instanceof Date) { return data; }

Thanks @rgnevashev and thanks @cantoine , I've added tests to make sure we won't make the same mistake again


Template option

Hello there,
I'm migrating my application to use ES6 and, long story short, I needed Notification to deal with templates directly injected instead of be read from file.
I tried to write tests but selenium doesn't want to run on my pc, something about selenium jar file missing from the node_modules directory. If you could help me fix this I could even write some test, otherwise I'd be glad if you could merge this in!
Thank you very much for the library in the first place, is brilliant!

it seems like travis is having my same problem here :)


adding CLI tutorial link to cli-python / toc.yml

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/TOC.yml
:white_check_mark:Succeeded
View (azure-cli-2017-03-09-profile) View (azure-cli-latest)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@barlanmsft Please clone the repo and work off a branch. You are editing more than one file, and inline-edit PRs are meant for small changes to a single file. If there are requested changes, I also don't want to duplicate them across PRs.

You can test word wrap by expanding the browser window to maximum size on a 15" monitor @ 1080p (possibly with UI scaling turned on), which I think is a reasonable size to test with. Even on my much larger monitor, there is still word wrap in the ToC with the current name.
I also don't think that by adding "with quizzes" users understand what they're getting. The nomenclature needs to be standardized across all of the TC initiative, which may or may not involve changing the name of the subsection where the links to content appear. I don't have a good suggestion myself, please check with the TC team to find out what they want to call tutorials which have been modified in this way.


update zh-Hans

Wait, I didn’t even realize the lines were gone below
"COLOR_OPTIONS_2" = "Tint Color";
"COLOR_OPTIONS_3" = "SystemBlue Color";
"COLOR_OPTIONS_4" = "Progreess Bar Colour";

i’m confused 🙀

Now there are four options for color. I am also very confused, hahaha. The biggest problem with the v5 version now is that the playback will crash.

Wait, I didn’t even realize the lines were gone below
"COLOR_OPTIONS_2" = "Tint Color";
"COLOR_OPTIONS_3" = "SystemBlue Color";
"COLOR_OPTIONS_4" = "Progreess Bar Colour";

i’m confused 🙀

Now there are four options for color. I am also very confused, hahaha. The biggest problem with the v5 version now is that the playback will crash.

Lemme guess, did you tap on the OP Button under the video player while the video is playing? because most likely it will definitely crash or does it just crash immediately? then I would recommend changing some of the options.

No, just enable PIP. It crashes when playing any video.

PiP might be broken in YouTube Reborn if that’s the case.

But I have no problem using your v4.2.6, but I am very confused.


Build Fails on Mac OS 12.4

Downloading and doing cmake --build build after the configure command provides a memset_s undeclared error.
/Users/Greeley/Workspace/Cppspace/Culinoire/build/subprojects/Source/wxWidgets_external/src/unix/utilsunx.cpp:229:5: error: use of undeclared identifier 'memset_s'
    memset_s(v, n, 0, n);
    ^
1 error generated.
make[5]: *** [libs/base/CMakeFiles/wxbase.dir/__/__/__/__/src/unix/utilsunx.cpp.o] Error 1
make[4]: *** [libs/base/CMakeFiles/wxbase.dir/all] Error 2
make[3]: *** [all] Error 2

Downloading wxWidgets and performing cd ~/wxWidgets && mkdir buildOSX && cd buildOSX && ./configure --enable-debug && make -j3 && cd samples/minimal && make && open minimal.app builds and launches minimal.app just fine.
But even after building it that way I can't seem to figure out how to use it that way.

Second error is caused by the fact I forgot to put include(${wxWidgets_USE_FILE}) in the src/CMakeLists.txt.
And that is caused by the fact I absolutely loathe video tutorials, my attention span just cannot handle them.
Anyway; excellent work, great tutorial in all.
Thank you.


Added profile hover dropdown for profile and logout options #1153

Issue Number: #1153
Added a dropdown that appears when hovering over the profile picture.
Dropdown contains options for "Profile" and "Logout".
tested the changes for responsiveness and usability.
Before:

After:

@KrishnaPrakhya Your pull request has been successfully merged! Don't stop here—continue contributing by addressing more issues and aiming to become a top contributor. You can also contribute in my another Repository BuddyTrail which is also under Hacktoberfest
Great contribution to HACKTOBERFEST'24! 🎉


error: expected ident, found #

error: expected ident, found #
   --> /home/robbat2/.cargo/registry/src/github.com-1ecc6299db9ec823/bitflags-1.0.1/src/lib.rs:423:29
    |
423 |                               #[allow(deprecated)]
    |                               ^
    |
   ::: /home/robbat2/.cargo/registry/src/github.com-1ecc6299db9ec823/bitflags-1.0.1/src/example_generated.rs
    |
4   | / bitflags! {
5   | |     /// This is the same `Flags` struct defined in the [crate level example](../index.html#example).
6   | |     /// Note that this struct is just for documentation purposes only, it must not be used outside
7   | |     /// this crate.
...   |
13  | |     }
14  | | }
    | |_- in this macro invocation

Build failed, waiting for other jobs to finish...
error: failed to compile `s3lsio v0.1.21`, intermediate artifacts can be found at `/tmp/cargo-install.7Y1euTb11RKT`

Caused by:
  Could not compile `bitflags`.

I have the same error.
OS: Ubuntu 16.04
Cargo Version: cargo 0.23.0 (61fa02415 2017-11-22)
Compiler Version: rustc 1.22.1 (05e2e1c41 2017-11-22)
It happens both with the stable and the nightly toolchains.

Bitflags 1.0.1 requires rustc >=1.20.0. I believe that error is from rustc 1.18.0 or 1.19.0.

@dtolnay This is still happening to me with rustc >= 1.20.0.

@naccib do the following commands reproduce the error for you?
git clone https://github.com/rust-lang-nursery/bitflags
cd bitflags
rustup update 1.20.0
cargo +1.20.0 build

Separately, same question for +1.21.0 and +1.22.1.

It works with 1.20.0 and I found why it does not works in my case.
When installing from my project, I still get the error.
Running sudo cargo install with the --verbose flag, it adds the following information:
  process didn't exit successfully: `rustc --crate-name bitflags /home/naccib/.cargo/registry/src/github.com-1ecc6299db9ec823/bitflags-1.0.1/src/lib.rs --crate-type lib --emit=dep-info,link -C opt-level=3 --cfg feature="default" --cfg feature="example_generated" -C metadata=f9ca8c19181c7f4f -C extra-filename=-f9ca8c19181c7f4f --out-dir /home/naccib/play/rgba-sound/target/release/deps -L dependency=/home/naccib/play/rgba-sound/target/release/deps --cap-lints allow` (exit code: 101)

The thing is, rustc version is 1.20.0, but sudo rustc version is 1.17.0.
I will try to fix this.

Sounds good. I am closing this issue because we don't plan to support <1.20.0.


Add Programmer Dvorak

I know you have normal dvorak but not programmer dvorak which is what I generally use on my pc. It would be great if this layout was added to the keyboard.
I will probably work on this myself once I get the time unless someone else.
https://www.kaufmann.no/roland/dvorak/

I'm working on it and it's almost done.

I'm done! I've sent a pull request

@spff did via https://github.com/klausw/hackerskeyboard/pull/598


update example mode output file(s) to our format

In looking at this file, I noticed that we'll need to make some minor updates to get it to match the file format that we've landed on. The main thing I noticed is that we will need an output_type column with values "sample", and then we could basically rename the current sample column to output_id.

fixed.


Fix: Fix the breadcrumb

Pull request checklist
Make sure you:

[ ] Signed the Contributor License Agreement
[ ] Followed the commit message guidelines

Short description of the change(s)
Fix #407

@qzhou1607 you have a couple comments

@molant Done.


⚠ TechNews 科技新報 has degraded performance

In c4445dd, TechNews 科技新報 (https://technews.tw/) experienced degraded performance:

HTTP code: 200
Response time: 1411 ms

Resolved: TechNews 科技新報 performance has improved in 32e8b4e.


Question: How to send up properties on metrics such that I can split on them in the portal

Hey Team,
I'm doing something pretty simple in code:
        client.trackMetric({
            name: 'complete_time',
            value: endTime.valueOf() - startTime.valueOf(),
            properties: { notificationServiceEventType: type }, // `type` is a string based enum with 3 possible values
        });

However, in the portal, I dont see how to split on this?

Is there something I'm doing wrong, or is this not possible?
Thanks,
Sam

Hey @SamJarmanPP for this to work I believe you will need to enable custom metrics dimension alerting. It will only work on dimensions that are ingested after you checked the checkbox.

That did it, thanks! :)


Revert adding phone-utils to Attestations wrapper

Description
Including phone-utils as a dependency greatly increases the contractkit bundle size

Thank you for this

Can't remove phone-utils from the devDependencies because it's being used in a test:https://github.com/celo-org/celo-monorepo/blob/1d73f149d2005a9c8894841776ae20ed19003925/packages/sdk/contractkit/src/wrappers/Attestations.test.ts#L2

phone utils used in a test so keep as devDep


⚠ Mobitopup has degraded performance

In 53fba2e, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 6295 ms

Resolved: Mobitopup performance has improved in 5130790.


There is any API to update the (max,min, disable, ...) options of the datepicker

After initialize a new pickadate how can i update the (max, min, disable, ...) options values.
Cheers

so is it possible without re-initializing?


pam.d typo for unlock_time

Describe the Issue
Controls have option nlock_time instead of unlock_time. This means the option unlock_time will not be set successfully.
Expected Behavior
Tasks should set unlock_time per STIG
Actual Behavior
system-auth and password-auth get set with nlock_time
      - name: "MEDIUM | RHEL-08-020014 | PATCH | RHEL 8 must automatically lock an account when three unsuccessful logon attempts occur during a 15-minute time period. | Set preauth"
        lineinfile:
            path: "/etc/pam.d/{{ item }}"
            regexp: '^auth\s+required pam_faillock.so preauth'
            line: "auth       required pam_faillock.so preauth dir={{ rhel8stig_pam_faillock.dir }} silent audit deny={{ rhel8stig_pam_faillock.attempts }}{{ (rhel8stig_pam_faillock.fail_for_root) | ternary(' even_deny_root ',' ') }}fail_interval={{ rhel8stig_pam_faillock.interval }} nlock_time={{ rhel8stig_pam_faillock.unlock_time }}"
            insertafter: '^auth'

Control(s) Affected
RHEL-08-020014
RHEL-08-020016
RHEL-08-020018
RHEL-08-020020
Environment:

Ansible Version: 2.9.27
Host Python Version: 3.6.8
Ansible Server Python Version: 3.6.8

Additional Notes
This is probably correct for most users of this because RHEL-08-020022 ends up setting it back correctly, because it has the same line. If someone has that disabled though, it will not be set correctly. It is also an idempotence issue.
Possible Solution
Fix typo nlock_time to be unlock_time. nlock_time is listed twice in each of the controls listed.

Closing since this has been merged in


Update for Swift 3 and Xcode 8 b3

It also seems to have improved compile duration by a few seconds
although this was based off of counting in my head.

😂

👍

sure 👍


Documentation for object Field corrected

The object Field has no property "prettyName" but "label".

Thanks for contributing


Integrate NativeBase theming

There is a LOT of inline styling done in the app (i.e. setting the style prop of a text component to change its font size). This is a big issue because not only does it make the code harder to read, it also makes our UI less consistent and harder to reuse. NativeBase (our UI library) does have a solution for this called theming, but up until now it has not been a priority. You can see NativeBase's default theme on their website for a basic example. There are two big priorities for our theme work. We need to update the color scheme and create a bunch of variants for common components so we can reuse styles and change them all in one place, though it would be nice to support dark mode too (not a priority though).

We will also want to set up some prefab variants of common components, a few that come to mind are:

Headers
Text blocks (underline, italic, emphasis, etc.)
Links vs buttons
Common layout configurations
Image boxes

I have created a branch with a basis for you to build on, see src/theme on #156. It's mostly just the default theme copied into the project so you can work with it easily, there are a few properties I didn't stub out but feel free to add them if you like.

Attached are the components I am working on (completed and todo) as well as their original idea from the design and page implementation(s):

AboutItem

Page(s): About Us
Reference:

Breadcrumbs

Page(s): All Pages (except Home Page)
Reference:

Button

Page(s): idk
Reference:

FAQItem

Page(s): FAQ
Reference:

FeedBlog

Page(s): Explorer
Reference:

FeedInstagram

Page(s): Explorer
Reference:

FeedTwitter

Page(s): Explorer
Reference: None

JumbotronGeneral

Page(s): Scoreboard
Reference:

JumbotronWelcome

Page(s): Notifications / Profile
Reference:

Notification

Page(s): Notifications / TeamDashboard
Reference:

Scoreboard

Page(s): Scoreboard
Notes: Need to install LeagueSpartan font for numbering
Reference:

SponsorBlock (SponsorCard)

Page(s): Sponsors
Note(s): This is completed but this needs to be implemented as just SponsorCard so the data will be wrapped in the actual Sponsor's Page
Reference:

TeamInformation

Page(s): TeamDashboard
Reference:


Fix CUDA dependencies.

This is mostly to fix the CUDA dependency issues we're currently getting on the CI pipelines. We will live dangerously on four #master dependencies!
] add CuArrays#master CUDAnative#master CUDAdrv#master NNlib#master

@christophernhill Builds are passing again so things might work on binder now.


Top the 'About' page

I haven't found a better way how to make the About page the default one instead of the Posts.
The only way is to keep the date in the about.md newer than in the other content pages.

it's possible we would had to tinker with the template.


docs: remove redundant "--set ipam.node=kubernetes" from GKE GSG

File install/kubernetes/cilium/templates/cilium-configmap.yaml contains the following:
{{- if .Values.gke.enabled }}
  ipam: "kubernetes"
  tunnel: "disabled"
  enable-endpoint-routes: "true"
  enable-local-node-route: "false"
{{- end }}

In other words, when gke.enabled is set to true for Helm, the IPAM is automatically set to kubernetes. Therefore, it is redundant to pass it to the Helm command line. Let's remove it from the GSG.

To which Cilium versions should we backport? Has .gke.enabled always had this meaning?

It looks like v1.8 has it, although in a different file (install/kubernetes/cilium/charts/config/templates/configmap.yaml). Although backported to v1.7 at some point, the GSG option was removed in fd25d04436e1 (install: Remove IPAM setting) because apparently it doesn't exist on v1.7. There is instead global.k8s.requireIPv4PodCIDR, which is not present in the GSG for GKE.

FWIW this is taking the opposite approach from the one advocated previously by this line in the guide, per commit c1a0025f75b84be5e0b48fb31c8ce09df2343a85 (which advocates for explicitly configuring everywhere to make sure it's correctly configured instead of implicitly deriving the settings from a higher-layer option).

I don't understand why ipam: "kubernetes" wasn't removed from gke.enabled, when the equivalent for Azure/ENI were removed... /cc @tgraf

@pchaigno likely because Azure / ENI modes the IPAM is not handled by Cilium itself. In GKE it is handled by Cilium since we are not in chaining mode.

@aanm First part I agree. Chaining mode though is a separate issue entirely. The IPAM modes for Azure / ENI are specifically configured when Cilium is not in chaining mode, just like the standard GKE instructions.

@aanm I don't follow the logic there. In chaining mode, the chaining plugin is responsible for IPAM. But if you configure Cilium's ipam option to eni or azure, Cilium is responsible for IPAM, it just performs it via the cloud API.

@aanm I don't follow the logic there. In chaining mode, the chaining plugin is responsible for IPAM. But if you configure Cilium's ipam option to eni or azure, Cilium is responsible for IPAM, it just performs it via the cloud API.

@joestringer correct, I mixed the 2 things in my response your logic it also correct.

Ok so if the objective is to have the IPAM explicitly configured instead of taken from the defaults in the Helm charts, then I suppose we should simply close this PR. I will do so tomorrow, if nobody objects until then.


AMQ-4693 Adds Kerberos 5 authentication and connection encryption over TLS

Basing on RFC2712

Is there any hope/plan to accept this PR?

:+1: exciting to see this!

Can one of the admins verify this patch?


Playtesting obliterates edit mode

When you have a brush selected because you edited something, start playing the game, stop playing the game, edit mode is gone until you select another brush or alike. It's annoying and should be easy to fix.

This is one of those issues that users maybe don't always register at a conscious level, but a fix would go a long way to improve the "feel" of the workflow in general.


Added support description option in create_group method

In the description of the method create_group indicated that the parameter description supported, but the method signature does not support it.

I moved the code under the context. That's better?

@azomazo Looks good, thanks!


invalid_client error when using service account + domain wide delegation to authenticate

I am trying to authenticate into Google Ads using this API client. I am trying to authenticate using a service account + gsuite domain wide delegation. I have performed all of the steps in this documentation but am receiving an invalid_client error from Google. Below is the code I am running.
GoogleAdsClient client = new GoogleAdsClient();
client.Config.OAuth2Mode = OAuth2Flow.SERVICE_ACCOUNT;
client.Config.OAuth2PrnEmail =<EMAIL_ADDRESS>client.Config.OAuth2ClientId = "serviceaccountclientid";
client.Config.OAuth2SecretsJsonPath = HostingEnvironment.MapPath("~/service-account-keys-downloaded-from-gcp.json");
var svc = client.GetService(Services.V2.CustomerService);
var accounts = svc.ListAccessibleCustomers();

The code is supposed to authenticate on behalf of a user in a gsuite organization and list all of the accounts the user has access to. When I run the above code I get the following error.
Status(StatusCode=Unavailable, Detail="Getting metadata from plugin
failed with error: Exception occurred in metadata credentials plugin.
Google.Apis.Auth.OAuth2.Responses.TokenResponseException:
Error:"invalid_client", Description:"The OAuth client was not found.", Uri:""

It seems that no matter what email I try to authenticate with I still get the same error which makes me think that there is a fundamental issue with how I am trying to authenticate.
All of the necessary fields seem to map properly into the GoogleAdsClient Config except for the client secret. I am unable to pass a client secret into the request because the service account client (auto created when enabling domain wide delegation on a service account in GCP) does not come furnished with a client secret. The docs that I have mentioned don't seem to specify how you would obtain a client secret when authenticating with a Google service account.

Try setting a breakpoint on the line where you get the svc. See if (1) the json was loaded correctly into config and (2) that’s the correct client Id you whitelisted in your gsuite domain. That’s the primary suspect.

@AnashOommen Yes the config object in the svc is populated with all of the values that I would expect. Below are the config fields that are populated correctly to my knowledge, am I missing something?
OAuth2ClientId
OAuth2Mode
OAuth2PrivateKey
OAuth2PrnEmail
OAuth2ServiceAccountEmail
OAuth2Scope

@AnashOommen Is there anything else that I could potentially be missing?

@AnashOommen I was not able to get the SERVICE_ACCOUNT authentication working so I tried switching to APPLICATION auth and I am still experiencing the same invalid_client error. My code is below.
var settings = new Dictionary<string, string>
 {
      { "OAuth2Mode", "APPLICATION" },
      { "AuthorizationMethod", "OAuth2"},
      { "OAuth2RefreshToken", usergooglerefreshtoken },
      { "OAuth2ClientId", oauthclientid },
      { "OAuth2ClientSecret", oauthclientsecret },
      { "DeveloperToken", developertoken }
};
var config = new GoogleAdsConfig(settings);
GoogleAdsClient client = new GoogleAdsClient(config);
var svc = client.GetService(Services.V3.CustomerService);
var test = svc.ListAccessibleCustomers();

I have tried creating a new project in GCP and generating new client/secret values and I still get the same invalid_client error when I run the above code.

It seems to be working now, although I am not sure what I tweaked to get it working.

It seems to be working now, although I am not sure what I tweaked to get it working.

Hi Tyler, were you able to work with the service account or the application OAuth2?

@fsutanto88 I wasn't able to get the service account authentication working but I was able to get the standard application OAuth2 flow working.


Multiple Users, Will not remove roles.

The Bot ws implented on an existing server, roles are in place and auth system is working.
However the bot will not remove the roles of those who have existing roles prior to the bot being installed on the server. The discord users with the roles have not authed to the bot , but are seen by the bot as not being in the database.
The bot will not remove their roles.
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: Checking for users who have left corp/alliance.... [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [Elias] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [shwing shwing] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [scopehone] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [Pigzy] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [Leeja] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [Slazanger] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [Corperate] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [Rock3t Josh (Silv R)] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: User [Krisztian Sallai] not found in database. [] []
[2017-07-29 22:59:30] Dramiel.INFO: AuthCheck: Corp/alliance check complete. [] []
fly safe o7

The library this bot uses is depreciated. And I don't intend to rewrite the bot in a different language at this time.


Fix low wall overlay displaying incorrectly and remove LowWallOverlay entity

About the PR

Fixes a low wall bug that happened near arrivals, specifically with the low wall overlay putting layers where it shouldn't have been
Renames low wall overlay vars to be more inline with what they actually do
Removes low wall overlay entities, moves that responsbility to WindowComponent

NGL i am not sure about my changes that were required to get rid of the low wall overlay entity, and I think it might just be better to keep that entity even though I feel like it's CBT, since this feels like an equal if not greater amount of CBT. need input on that though
Fixes #1239
Fixes #2790
Screenshots

Changelog

:cl:

fix: Airlocks & low walls near arrivals don't look weird anymore

There's still some cursed situations where windows / walls are smoothing to stuff that isn't there (i.e. it's treating the low wall as one of its own).

There's still some cursed situations where windows / walls are smoothing to stuff that isn't there (i.e. it's treating the low wall as one of its own).

idk if there's anything I can do about this without drastically changing how low wall/window smoothing works

That's fine but it does mean it doesn't fix #2790 given there's still bugs with it.

The low wall overlay is another entity because it has to be rendered above the glass window itself. This PR breaks that.

The low wall overlay is another entity because it has to be rendered above the glass window itself. This PR breaks that.

i'm aware of that, this PR moves the rendering of the overlay to the window instead, you can obviously tell that the overlay still renders properly in the first image. i'm still unsure of if i want to do that at all, instead of just keeping the entity

removed the entity-related changes this now just fixes the one rendering bug that occurred


WG: Proposal for CNCF Accessibility (A11y) WG

Tasks
Charter Proposal

[ ] call for contributions
[ ] community feedback
[ ] present to TAG
[ ] TOC vote
[ ] email TAG Observability list
[ ] figure out the new process at the TOC level

@karayim can you join in for tomorrow's O11y TAG (Feb 6) meeting to walkthrough your proposal and discuss!

Hi @karayim are you available to join in for one of the June TAG meetings to give us an update from the a11y workgroup?

Closing this issue since there is no response from the initial proposer for this workgroup. Let's reopen when there is active interest.


Kong with Socket.io (namespace)

Summary
Is there any tutorial on how to integrate use socket.io with namespace
For example,
We have 2 socket server.
<IP_ADDRESS>:8082/customer
<IP_ADDRESS>:8083/merchant
Then we want to have this in kong
socket.mydomain.com/customer => <IP_ADDRESS>:8082/customer
socket.mydomain.com/merchant => <IP_ADDRESS>:8083/merchant
I tried and does not get it working. Can anyone share me how to achieve this?

What are the unusable representations? Perhaps you can provide some details?

So let say the client is trying connect to "socket.mydomain.com/customer". The way that client making a request in socket client is "ws://socket.mydomain.com/socket.io/?station_id=2&EIO=3&transport=websocket".
The client does not provide path "customer" as the namespace during the request. So that in the route configuration. We can only configure "socket.mydomain.com" to point to "<IP_ADDRESS>:8082".
For "socket.mydomain.com/merchant", we cannot do any configuration because "socket.mydomain.com" has been used by the "customer" name space.

Please refer to the image above.

maybe you can change socket.io default namespace?

maybe you can change socket.io default namespace?

@marlonfan, this is helpful. I try looking in socket.io document and have not find any option to do that. Can you share me how to do that?
Thanks,
Punleu

yep, you just change this namespace on your project
const socket = io('/your custom namespace');

Hi @marlonfan, I tried changing this and it does not work. Maybe, I can share a project later.

Dear contributor,
We're closing this issue as there hasn't been any update to it for a long time. If the issue is still relevant in the latest version, please feel free to reopen it. We're more than happy to revisit it again.
Please have a look at our pledge to the community for more information.
Sincerely,
Kong Gateway Team


Thread Exception when Downloading

Forgive me if this is a duplicate, but I don't believe I've seen it anywhere when passing through the issues.
Here's the error:
minamoto-kun.monogatari.[mh].c029: 100% |######################| ETA:  00:00:00

Exception in thread Thread-21:031:  60% |#############         | ETA:  00:00:09
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/nicolas/bin/manga_downloader/src/parsers/base.py", line 403, in run
    raise FatalError("Thread crashed while downloading chapter: %s" % str(exception))
FatalError: Thread crashed while downloading chapter: CRC check failed 0xe0e6dd4f != 0xfeb86693L

At the same time I also get the following error, which I am not sure if it is related:
Exception in thread Thread-27:033:  11% |##                    | ETA:  00:00:00
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/nicolas/bin/manga_downloader/src/parsers/base.py", line 403, in run
    raise FatalError("Thread crashed while downloading chapter: %s" % str(exception))
FatalError: Thread crashed while downloading chapter: Error -3 while decompressing: invalid distance too far back

The Python version I'm running this on is the default for my system (Debian Testing):
nicolas@pulse $ python --version
Python 2.7.12+

It also seems that although it gives this error it continues to download the next files in the background (not showing any progress), and also continues to have errors every now and then with other files.
I hope this is helpful.

Do you have the exact steps you followed before you found the exception? I've never seen it before, what site were you trying to download from? What manga and what chapters? Any particular arguments you passed on to the application?

First, I'm using the code from the following git commit: 13e4eaa4c23ce99a5e5c0ddffbb5e928a34aa991, since the code from the last release is old af.
As for what I did, here's what it is:
nicolas@pulse $ manga_downloader "minamoto-kun monogatari"

Program: Copyright (c) 2010. GPL v3 (http://www.gnu.org/licenses/gpl.html).
Icon:      Copyright (c) 2006. GNU Free Document License v1.2 (Author:Kasuga).
           http://ja.wikipedia.org/wiki/%E5%88%A9%E7%94%A8%E8%80%85:Kasuga

minamoto-kun monogatari
Which site?
(1) MangaFox
(2) MangaReader
(3) MangaPanda
(4) MangaHere
(5) EatManga
(6) Batoto
4
Beginning MangaHere check: minamoto-kun monogatari
(1) c001
(2) c002
(3) c003
...
(193) c193
(194) c194

Download which chapters?
190-194
...

On the last  ... is where it starts downloading and every now and then gives me the error with one of the downloads (it seems quite random to me). If I limit the number of threads to 1 it doesn't seem to give me any errors (which isn't surprising since this is a thread exception I'm getting, and I can't get one if the program doesn't use threads).
Just in case you were wondering, the manga_downlaoder command is a bash alias I made to the file, literally as follows:
alias manga_downloader="~/bin/manga_downloader/src/manga.py"

That is, I run the python file as an executable (since it has executable permissions and also starts of the file with #!/usr/bin/env python, which tells my shell what environment to use). Should I change my alias to use python3?

@jiaweihli can correct me if I'm wrong but while there is (was?) an effort to migrate manga_downloader to Python3 it is my understanding that the migration was never completed, Python2 continues to be the supported interpreter.
As for your bug, and prepare yourself for a long post, I'm not able to reproduce it.
You do mention that this happens randomly and by looking at your traces it doesn't look like this is related to thread handling code, but rather, something is happening in the underlying code; quite possibly related to a ZIP file getting corrupted somewhere and for some reason. Both CRC check failed 0xe0e6dd4f != 0xfeb86693L and Error -3 while decompressing: invalid distance too far back are the actual culprits of your problem.
Which brings me to the next point. I think you've found a bigger issue with the current code: we're not able to correctly debug the application even in edge cases because we're silencing all Exceptions in the DownloadChapterThread code.
The code that is raising this FatalErroris the following in base.py lines 391-403 of the DownloadChapterThread class:
	def run (self):
		try:
			self.siteParser.processChapter(self, self.chapter)
		except Exception as exception:
			# Assume semaphore has not been release
			# This assumption could be faulty if the error was thrown in the compression function
			# The worst case is that releasing the semaphore would allow one more thread to
			# begin downloading than it should
			#
			# If the semaphore was not released before the exception, it could cause deadlock
			chapterThreadSemaphore.release()
			self.isThreadFailed = True
			raise FatalError("Thread crashed while downloading chapter: %s" % str(exception))

If an Exception is raised in any part of the underlying code in self.siteParse.processChapter(self, self.chapter) it will be silenced by the code on line 403 where we raise FatalError.
While we may not be able to further troubleshoot your problem at least for now I believe we should tag your issue as a bug and proceed to open another issue to take care of the silenced exceptions, maybe then we'll be able to get more information on what is going on.
If nobody has any objection, I'll proceed to document that in another ticket.

I just remembered another issue that may have to do with this (I don't know how your file streaming works, so it's hard for me to say this is part of the problem). My connection to the internet from my room is rather iffy, sometimes it cuts off completely and unexpectedly and then goes back 100%. Could any slight cuts in the connection like this cause for the file stream to be corrupted or in general cause issues with an individual thread? As I said, if one thread has an error downloading it gives me this error, but the the other threads seem to continue.

My hunch is that you're seeing high packet loss, and that this is causing issues when creating an archive.

@jiaweihli, that seems like it would be it, but that shouldn't be a problem if these packets were transfered over TCP (which would be normal considering it's a file transfer). I'm not sure as to the workings of manga_downloader (mostly 'cause I don't know any real Python), but it should be able to use TCP.
Of course, this is all assuming that the issue is packet loss.

If your internet is cutting out intermittently, it's possible that the connection to the server is getting closed and you're unable to resume the original connection.
In days past when I had a wonky internet setup, I would sometimes end up with incomplete/truncated images.  I'm not sure if you're experiencing a similar problem, since I never ran into an issue building the archive.
Fixing #82 will help diagnose this issue.

@jiaweihli, I'm not getting any truncated images, just this thread exception. It seems to have stopped since I created an alias and now it only runs on one thread, but that isn't necessarily fixing the problem but rather avoiding it ('cause I want to download my next chapter of manga). So yeah, it's just the exception.

Fixed in #90

Whoops, not fixed - but should be easier to debug.

So for quite a while now I've been using only a single thread for downloading manga, which has worked just fine (with my shitty internet), however recently I got a PLC so I can have a much more consistent connection. I'll try using multi-threaded again and see if it works fine, which should also help to see where this is coming from.
Have a great New Year's!

@Deathsbreed
Thanks for keeping us updated.  I'm actually really curious what's wrong, this is a puzzler 🤔
And happy holidays!  🎉

This is still an issue, I think it may be the same issue anyways:
Thread crashed while downloading chapter: tokyo ghoul - 39
Exception in thread Thread-42:
Traceback (most recent call last):
  File "C:\Python27\lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\Users\James\Documents\_Github-Projects\manga_downloader\src\parsers\base.py", line 395, in run
    self.siteParser.processChapter(self, self.chapter)
  File "C:\Users\James\Documents\_Github-Projects\manga_downloader\src\parsers\base.py", line 274, in processChapter
    self.downloadChapter(downloadThread, max_pages, url, manga_chapter_prefix, current_chapter)
  File "C:\Users\James\Documents\_Github-Projects\manga_downloader\src\parsers\mangafox.py", line 161, in downloadChapter
    self.downloadImage(downloadThread, page, pageUrl, manga_chapter_prefix)
  File "C:\Users\James\Documents\_Github-Projects\manga_downloader\src\parsers\base.py", line 156, in downloadImage
    img_url = self.__class__.re_getImage.search(source_code).group(1)
TypeError: expected string or buffer

Workaround is to disable multi-threading as Deathsbreed suggested using -t numb optional cli argument with numb = 1.
e.g.
python manga.py -t 1 "naruto"


api: remove hard coded jwt authentication secret

Right now the JWT authentication secret is hard coded into the code (see links below):
https://github.com/shellhub-io/shellhub/blob/ee6d67a609310611a05f0639dd6c939c63cfc2f2/api/main.go#L443
https://github.com/shellhub-io/shellhub/blob/ee6d67a609310611a05f0639dd6c939c63cfc2f2/api/main.go#L466
Task: use an environment variable to get the JWT secret value replacing the hard coded value

@gustavosbarreto
Hi, I’ll take this up, however I‘m not sure it’s good that it only converts all to os.Getenv.  for example I don’t know which the tests of this package is used them. CI may be getting fail when it changes which doesn’t care environment variables.


Chinese _matchesInfo length

Describe the bug
_matchesInfo length seems like a byte length, so one Chinese character will return 3
same issue with cropLength
To Reproduce
import { MeiliSearch } from 'meilisearch';

const client = new MeiliSearch({
  host: 'http://<IP_ADDRESS>:7700',
  apiKey: 'hellomat2021',
});

const index = client.index('test');
const documents = [{ id: 1, title: '一二三四' }];
await index.addDocuments(documents);

const res = await index.search('一', { matches: true });
console.log(res);

_matchesInfo will return
"title": [
  {
    "start": 0,
    "length": 3
  }
]

Expected behavior
"title": [
  {
    "start": 0,
    "length": 1
  }
]

Hey @tyuan511,
Which version of Meilisearch do you use? Just to know the version of the tokenizer that returned those boundaries.
However, I'm pretty sure that the length is in bytes and not in character count, you can verify this on this website for example i.e. 一 is 3 bytes long.

I use Meilisearch v0.25.2.
Yes,  it's  bytes length . I just want to highlight content by the _matchesInfo, but it's hard to slice a string by byte  for javascript in browser.

Hi @tyuan511 👋
Why did you want to highlight from _matchesInfo and not _formatted?
Thanks!

I used to _formatted with cropLength, but cropLength is also in  bytes (e.g. cropLength 50 will only return 17 Chinese character).  I think I can use _formatted without cropLength, and slice the result by html em tag.
Thanks!

Yes, for article mixed with Chinese and English, it is difficult to control the length of the highlight.

I think Token object should have char_start and char_end  in meilisearch/tokenizer  for include these to _matchesInfo
https://github.com/meilisearch/tokenizer/blob/491071ab186d8d240dd105a4a29f04d4429a225d/src/token.rs#L28-L40
And Jieba also has unicode based start and end property, so this doesn't seem to be that hard to implementing.
https://github.com/messense/jieba-rs/blob/2a093182f254f5e96997f133bcda4f11186c0620/src/lib.rs#L186-L193

Hello everyone
Looks like there are multiples issues in this issue 🙂

Regarding _matchesInfo that returns bytes instead of chars -> this is expected, so not a bug. However this is something we are considering to change. Please check out this discussion, and if you have the time, you can let your feedback regarding your usage and your need. This will definitely help the product team making a decision :)
Regarding the bad cropping (cf @tyuan511 comment "cropLength is also in bytes (e.g. cropLength 50 will only return 17 Chinese character)") -> this is not something expected. Could you please @tyuan511 open another issue detailing the problem you observe with the cropping and how to reproduce it? We plan to stabilize the cropping part in the near future, so any feedback and bug reports are more than welcome 🙏
Regarding any bad highlighting -> we've just released the first release candidate of v0.26.0 that should fix the highlight with non-unicode char. You can check it using v0.26.0rc0. If you have an issue with v0.26.0rc0, please open another issue

I close this issue since the original topic is _matchesInfo and for every point I listed, the discussion should continue in another place (in the discussion or in another fresh new issue) 🙂


[Snyk] Security upgrade sharp from 0.28.1 to 0.29.3

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to fix one or more vulnerable packages in the `yarn` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

package.json
yarn.lock

Note for zero-installs users
If you are using the Yarn feature zero-installs that was introduced in Yarn V2, note that this PR does not update the .yarn/cache/ directory meaning this code cannot be pulled and immediately developed on as one would expect for a zero-install project - you will need to run yarn to update the contents of the ./yarn/cache directory.
If you are not using zero-install you can ignore this as your flow should likely be unchanged.
Vulnerabilities that will be fixed
With an upgrade:

Severity
Priority Score (*)
Issue
Breaking Change
Exploit Maturity

658/1000  Why? Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 5.3
Regular Expression Denial of Service (ReDoS) SNYK-JS-SEMVER-3247795
No
Proof of Concept

(*) Note that the real score may have changed since the PR was raised.
Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Regular Expression Denial of Service (ReDoS)

👷 Deploy Preview for glistening-gaufre-abd10e processing.

Name
Link

🔨 Latest commit
abedb5573e23c4c6cdd8ae2c1e1ad9c24895056f

🔍 Latest deploy log
https://app.netlify.com/sites/glistening-gaufre-abd10e/deploys/6491f67917a6510008e9001d


[FUZZ] Beaconfuzz_v2 crash-8aca1b5ce295f678fb67bd5ea8bb36c13a0d7f20 in struct_proposer_slashing

I've done and provided the following:

[x] Checked to see if any other [FUZZ] issue already refers to that crasher
[x] Attached the crashing input (either attached to the issue as a .zip or .gz, or as a link to a file sharing service)
[x] Noted the beacon-fuzz version or commit used.
[x] Provided crash output
[x] Noted the command or fuzzer used to generate the crash
[x] Name of the original crash file
[x] (Optional but optimal) Checked if the crash can be consistently replicated by re-running the input.

Info to Reproduce

Command run: e.g. make fuzz-all

Crasher file name: crash-8aca1b5ce295f678fb67bd5ea8bb36c13a0d7f20
crash-8aca1b5ce295f678fb67bd5ea8bb36c13a0d7f20.zip

Client exercised: Teku

Fuzzing engine used (if applicable):

Crash output and stacktrace
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last, using override)
/home/nimbus-eth2/vendor/nimbus-build-system/vendor/Nim/lib/system/excpt.nim(614) signalHandler
SIGABRT: Abnormal termination.
==1516179== ERROR: libFuzzer: fuzz target exited
    #0 0x55c3c5eec791  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xe45791)
    #1 0x55c3c89b7160  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3910160)
    #2 0x55c3c89cbecb  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3924ecb)
    #3 0x7f996f31ca26  (/lib/x86_64-linux-gnu/libc.so.6+0x49a26)
    #4 0x7f996f31cbdf  (/lib/x86_64-linux-gnu/libc.so.6+0x49bdf)
    #5 0x55c3c678039c  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x16d939c)
    #6 0x7f996f31920f  (/lib/x86_64-linux-gnu/libc.so.6+0x4620f)
    #7 0x7f996f31918a  (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
    #8 0x7f996f2f8858  (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
    #9 0x55c3c8a84916  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x39dd916)
    #10 0x55c3c8a6d455  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x39c6455)
    #11 0x55c3c89aa6a6  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x39036a6)
    #12 0x55c3c8a752d5  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x39ce2d5)
    #13 0x55c3c605bdf4  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xfb4df4)
    #14 0x55c3c605a4b9  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xfb34b9)
    #15 0x55c3c605bc94  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xfb4c94)
    #16 0x55c3c61289d4  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x10819d4)
    #17 0x55c3c6124b36  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x107db36)
    #18 0x55c3c600e102  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xf67102)
    #19 0x55c3c5f91681  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xeea681)
    #20 0x55c3c89aa6d0  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x39036d0)
    #21 0x55c3c89aa32f  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x390332f)
    #22 0x55c3c89cc32c  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x392532c)
    #23 0x55c3c899d439  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x38f6439)
    #24 0x55c3c89a7232  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0x3900232)
    #25 0x55c3c5e69346  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xdc2346)
    #26 0x7f996f2fa0b2  (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #27 0x55c3c5e694ed  (/home/beacon-fuzz/beaconfuzz_v2/fuzz/target/x86_64-unknown-linux-gnu/release/struct_proposer_slashing+0xdc24ed)

SUMMARY: libFuzzer: fuzz target exited
────────────────────────────────────────────────────────────────────────────────

Error: Fuzz target exited with exit code: 77

Your Environment

Fuzzer ran: beaconfuzz_v2
Version/Commit used: latest
Operating System and version: Ubuntu 20.04

FYI, that the same file and issue than https://github.com/sigp/beacon-fuzz/issues/82 but this time it's teku that return a different result than the others.
For analysis, here is a package containing the pre state, the proposerslashing and the post state generated by teku.
issue_90_proposer_slashing.zip
lighthouse reject this proposerslashing processing with the error: ProposalsIdentical
prysm reject the proposerslashing processing
nimbus reject the proposerslashing processing
nimbus process the proposerslashing processing
You can reproduce with:
../beaconfuzz_v2 debug beacon.ssz propslash.ssz proposerslashing

Confirmed to be the same type of discrepancy described in #82. For its process_proposer_slashings function, Teku compares the SignedBeaconBlockHeaders, instead of the BeaconBlockHeaders.

I just got a crash with the same filename in struct_attester_slashing,probably the same issue unsure if the PR also fixes it there.


authn/claims: Fix segfaults in AuthInfo.GetName() and AuthInfo.GetUID()

These methods currently segfault in case IdentityClaims is nil. I have added a nil check to avoid the segfaults.
I ran into this when using Unified Storage with the Cloud API Server which only passes an Access Token. The GetUID() method is used for creating deletion markers in US.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[WIP] ENH: Initial changes needed to get asv working with package/setup.py

I managed to make a minimal set of hacks to asv that allow for benchmarking with asv continuous  when the project setup.py is in a subdirectory instead of the project root. This is obviously a work in progress based on a hack for a single project with a single specified / hard-coded subdirectory for the (conda) build.
Perhaps over time & with some guidance I can help make this more general so that it may be suitable for asv itself and not just our custom fork.

Do you mind if we close this for the moment (in order to not clog the appveyor queue --- I don't have permissions to cancel jobs there) --- I think the approach to take will need some thinking how to do, and this PR is not yet very close to a solution.

@pv Agreed, this is likely to require some thought & I was a bit too excited opening the PR. Closing myself. The other project I work on will use the crude hack in its own fork for now.


Use the exact conversion factor from kilometers to miles.

This should be uncontroversial.  Possibly a few tests will fail.
But this can be easily fixed.

Merged in 1963a90f6358c12a6beb58d92b366021f3abf9c9, thanks.


Pretrained weights do not exist

Hello, I am trying to run inference with the Hact-net model, but it appears that the download link for the pretrained weights does not work.

Is there a download link?

I couldn't find one. I ended up just training it myself and got comparable results.

@purelyboba did you train a model using the graphs they provided or did you create your own from the previous_version dataset? I was able to replicate comparable results (e.g., 61% weighted F1) when training and testing on the graphs the authors provided in #1, but when training and testing on graphs I created locally I topped out at 57% weighted F1 score.


Move docs folder to CI generated web app and remove from downloadable package code

Is your feature request related to a problem? Please describe.
Extra docs folder not needed when downloading package dependency. This adds bloat to package code.
Describe the solution you'd like
Convert example pages into a web app that can be generated by webpack via a GitHub CI action. This would allow the docs folder to be git ignored and would remove extra bloat from the downloaded package files.
Describe alternatives you've considered
Alternatively, an external repo could be used for all documentation, this plugin is so small, it seems excessive. Plus, the example pages are useful when testing and developing code.

Done! I have used a separate branch for deploying documentation, which has a slightly different gitignore file (docs build files are not included).
This is now working with GitHub Pages.
Closing this ticket.


Complete documentation for gogen

I've been using your tool and find it very helpful. However, I noticed that the documentation for the gogen is incomplete.  For example, the method to bootstrap the generated go implementation.  Can you provide complete documentation for gogen with some examples?

Hi - thanks for your question.
You can find a smaller fully worked Modular PlusCal + Go example here: https://github.com/DistCompiler/pgo/blob/main/systems/loadbalancer/
The tests in that folder are one example of how to use the gogen output. Look specifically at distsys.NewMPCalContext and how it is used. The functions in distsys are documented using the standard Go docs format, and there should be descriptions for many of the common components. Do let us know if there's something unclear there.
I've also made a request for online Go docs, which should be available here eventually: https://pkg.go.dev/github.com/DistCompiler/pgo/distsys

Thanks! I can run the examples now.

Closing for now - reopen or make a new issue if you run into any trouble.


Add support for Bigtable

Could you please add support for Bigtable?
Support for the Bigtable Admin API can be added to the master branch as it does not require any additional work beyond just calling the gRPC APIs. We can start with the Bigtable Data API on a separate branch and merge to master once it's ready for general consumption; Bigtable protocol requires additional development beyond just calling the gRPC APIs.
Thanks!
/cc: @jgeewax, @sduskis

@mbrukman
We would be very happy to add support for Bigtable! The only issue of course is understanding how to prioritize it relative to our existing pipeline. I assume you are hoping for Q1 2017?

Bigtable protocol requires additional development beyond just calling the gRPC APIs.

Can you provide links and/or more detail on this? I see that google-cloud-python and google-cloud-node have recently added Bigtable packages in master. Can they serve as examples for us?

@quartzmo

The only issue of course is understanding how to prioritize it relative to our existing pipeline. I assume you are hoping for Q1 2017?

Yes, please, though I'm not going to object to anything happening in Q4 2016. :-)

Bigtable protocol requires additional development beyond just calling the gRPC APIs.

Can you provide links and/or more detail on this? I see that google-cloud-python and google-cloud-node have recently added Bigtable packages in master. Can they serve as examples for us?

The google-cloud-python repo is more complete than google-cloud-node, so I would use that as a model. That said, the Bigtable implementation in google-cloud-go is even better.
There are several things above and beyond gRPC support that are necessary:

reconstructing streaming replies from the server
batch read/write operations are not atomic, issuing a batch of ops may cause some to succeed while others fail; the caller needs to parse the response and re-issue the failed ones until some timeout or number of failures are exceeded, with some backoff
some performance optimizations were done in other languages, such as connection pooling, this is language-specific as to what needs to be done

@sduskis and @garye may have additional comments.

@quartzmo -- any updates on this? Can we get started with a set of generated classes from gRPC protobufs and proceed from there?

@swcloud @geigerj What do you think?

Hi @mbrukman -- I'm going to follow up with you internally about getting BigTable on the client library pipeline. At the moment, there is no ETA for support.

Thanks @geigerj!

Note: I'm going to leave this issue open to track the feature requests beyond gRPC, but this will be blocked on basic GAPIC support, which is tracked internally.

Is there any news on this? I really would be able to use BigTable :-)

@jochenseeber Sorry, no updates yet.

Closing, as this is being tracked internally. The ETA for support is currently 2017Q4.

Thanks for the update, @geigerj! We're all looking forward to it.

Looks like this was implemented in https://github.com/GoogleCloudPlatform/google-cloud-ruby/pull/1786 — thank you!

We will need to either reopen this issue, or create a new one.  The Cloud Bigtable team would like to flesh out a hand-written client.  The GAPIC client isn't enough by itself to make a usable client.

Is there an update on this?  Is there a new issue tracking further work, or will updates be posted here?

@geigerj Can we reopen this issue? It states:

Bigtable protocol requires additional development beyond just calling the gRPC APIs.

@quartzmo Sure.
@retnuh No updates so far, but we expect to begin work early next year. The new ETA is by the end of 2018Q1.

Thanks @geigerj !

Thanks for the update @geigerj !

We're still working on this.  We have plenty of code written, and we have some PRs (#1990 for an improved admin and #2001 for the data API).  We need to get them reviewed.  We'll still have a bit of engineer work for a fully reliable production grade client.  We'll do what we can to move that forward in Q2.

This is almost done.  We should be close to Alpha soon.

@sduskis Can I close this issue now, since the release is imminent, and the PR(s) is/are a better place to continue discussion? Because of its age, this issue is out of SLO.

Got an ETA on when google-cloud-bigtable will be available via RubyGems?

@sduskis Any ETA?


Method to get ID of RPC calling peer

This is fulfilling the same goal/API as #7995 though there are small differences.
There's no stack - I can't see any reason for one.
I named the function get_rpc_sender_id as I found get_rpc_id a bit ambiguous
The default value (when not called from an RPC) is zero, instead of the client's own ID.
There's no explicit error condition for the network peer not being initialised, that seems unnecessary + the function would just return 0 anyway.
Cheers :)

Referencing https://github.com/godotengine/godot/issues/7362 and https://github.com/godotengine/godot/issues/7853


Doc unclear on how to "translate" from tokio-core to tokio-proto

In the Example: a toy HTTP+TLS client using core doc, it says:

In this example we’ll only use tokio-core for now rather than tokio-proto, but the concepts should translate quite readily.

There's no further mention of tokio-proto on that page though, so it's not clear how to translate it to tokio-proto.

FWIW it all boils down to an implementation of Io. Most implementations of ServerProto are generic over Io and the BindTransport future allows you to do work with an I/O object before returning it, such as an SSL handshake.
Essentially the translation for TLS would look like:

Store a TlsAcceptor context in your protocol object
When bind_transport is called, use that to call connect_async
Return that all as a future

Hopefully this clears things up. If it doesn't, please open a new issue on http://github.com/tokio-rs/tokio-website (since this appears to be a website bug) and reference this one.


feat: Tailwind-based UI package

The Tailwind-based UI library.
Vite, in this case, builds the library code and produces simple JavaScript that takes React's createElement and applies the right classes to it. The styling responsibility for now is on the consumer's side. However, I exported one function for simple Tailwind configuration of consumer's tailwind.config.ts:
import { withPenumbra } from '@penumbra-zone/ui-tailwind/theme';

export default withPenumbra({
  content: [
    './app/**/*.{js,ts,jsx,tsx,mdx,css}',
    './src/**/*.{js,ts,jsx,tsx,mdx,css}',
  ],
});

Components list. Agenda: ✅ – implemented, 🟡 – not implemented (not needed for now), ❌ – i decided not to implement it.

⚠AccountSelector
✅AddressViewComponent
✅AssetIcon
✅AssetSelector
✅Button
❌ButtonGroup
✅Card
❌CharacterTransition
✅ConditionalWrap
✅CopyToClipboardButton
✅Density
✅Dialog
✅Display
✅DropdownMenu
🟡FormField
✅Grid
✅Icon
✅Identicon
❌IsAnimatingProvider
✅MenuItem
❌PenumbraUIProvider
✅Pill
✅Popover
✅Progress
🟡SegmentedControl
🟡Slider
🟡SwapInput
✅Table
✅Tabs
✅Text
✅TextInput
✅Toast
🟡Toggle
✅Tooltip
🟡ValueInput
✅ValueView
✅WalletBalance

@grod220 do you have the idea why the tests are failing in the minifront? Seems like it is triggered by your changes
https://github.com/penumbra-zone/web/actions/runs/11724483746/job/32658624212?pr=1891

The "multiple copies of react" issue sounds like you are bundling react with the tailwind package. Think that means you need to make it a peerDep and not a main dependency.


How to actually use EVP to do ED25519

I am attempting to use ED25519.  I found this page
https://www.openssl.org/docs/manmaster/man7/Ed25519.html
which mentions EVP_PKEY_ED25519 which does not appear in my copy of the openssl source that I downloaded earlier this evening.
I did ./config followed by make and it built without any errors.  I am looking at the evp.h file in the include/openssl directory.  Searching for 25519 gets no hits. The openssl app reports:  OpenSSL 1.1.0h-fips  27 Mar 2018
In particular, I want to write code to generate a random private key and then derive the corresponding public key, if someone can point me at a working example I'd be very appreciative.

Only X25519 is supported in 1.1.0
For Ed25519 you need the 1.1.1 version

No, X25519 is not supported in 1.1.0 EVP.
Closing this issue.

I wanted to suggest that it would be wise to add a section to the docs (such as the one I linked above) that says "since {version}" so that when encountering something you think you could use, you'd know what version of the library is required.

Rich, at least this command does work with 1.1.0 (and it uses EVP intenally):
openssl genpkey -algorithm X25519 -text

No, X25519 is not supported in 1.1.0 EVP.

X25519 is supported in 1.1.0 EVP but has some significant limitations. The support is better in 1.1.1. Ed25519 is only supported in 1.1.1.

If X25519 is supposed to be supported in 1.1.0 in EVP then that still wouldn't explain why there are NO references to 22519 in any constant in the evp.h file... I mean I would expect it to occur in some way in a constant related to X25519?

Actually the only constant you need is NID_X25519 which is defined in openssl/obj_mac.h.

@bernd-edlinger yeah, I confused EVP and EC ... Thanks.
I still think there's no reason to re-open this, but feel free to do so if I'm wrong on that (too).


error message "undefined"

Describe the bug
Sporadically I get the error message "undefined" in the ioBroker log.
To Reproduce
Sorry, I have no method to reproduce the error.
I think it is produced by the status or info requests.
Expected behavior
No error message :)
Screenshots & Logfiles
iobroker.log
Versions:

Adapter version: 1.3.4
JS-Controller version: 5.0.14
Node version: 18.17.1
Operating system: Ubuntu Linux

Additional context
None

Okay - strange.
thank you very much for the logfile .
Sometimes the polling of battery data fails and sometimes it works. When it fails it results in this undefined.
I need to take a deeper look into it.
Please be patient.

Okay - I think I didn't made myself clear enough.
When you change the config of the adapter it gets restarted by ioBroker to initialize newly. This is what I ment in my last post.
And when I read the log correctly this "undefined" error only happens on the first run after each restart, from the second run/poll intervall on it gets the battery state correctly.
And I asked you to proof and corfirm or deny this theory.

Yes your rights. The message appears every time on adapter start.


New Script Sample - Spo bulk publish syntex model

Hi
I have added a new sample below to bulk publish a Syntex model to many sites from a CSV.
Hope I've not missed anything out with the formatting - apologies if so!
Thanks
Leon

Hey @LeonArmston , really awesome sample - thank you!! 👏 This should be live shortly 😊


Compatibility inquiry - AnimDessin2 and Photoshop CC 2024 (v25.6.0)

I am writing to inquire about the compatibility of AnimDessin2 with Adobe Photoshop CC 2024, version 25.6.0. Your documentation states compatibility with CC 2020+, but I am encountering [describe the problem you are experiencing, e.g., difficulties installing the plugin, the plugin not functioning correctly, a specific error message]. Could you please confirm whether this version is officially supported?

Could you explain the problem, its not clear what the actual issue is?
Also provide screengrabs of possible errors or video


cannot import name 'run' (aio_compat.py)

I created a simple python function and deployed to a repo. A working colleague migrated to another repo. I cloned the code and tried to debug it again.
When I press F5 I got the following Error:

I tried to recreate the function and spend several hours but I was not able to find a solution.
I started visual studio code from the conda shell activating the desired environment. I verified the environment is set in vs code.
Any feedback would be appreciated.
Environment data

VS Code version: 1.45.1
Extension version (available under the Extensions sidebar): 2020.5.86398
OS and version: Windows 10 Professional
Python version (& distribution if applicable, e.g. Anaconda): Anaconda, Python 3.7.4 and 3.6.8
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda
Relevant/affected Python packages and their versions: asyncio 3.4.3
Relevant/affected Python-related VS Code extensions and their versions: Azure Functions

After installing the latest version of vscode-python extension I am not able to debug the functions any longer.

Yes, I am seeing the same issue. I am using the command line "func" tool to work around the issue.
Running from within code was working on Monday.
The functions I am trying to run are in this repo:
https://github.com/asrashley/dashpiff


Add support for asymmetric embedding models

Description
This PR adds support for asymmetric embedding models such as https://huggingface.co/intfloat/multilingual-e5-small to the neural-search plugin.
It builds on the work done in https://github.com/opensearch-project/ml-commons/issues/1799.
Asymmetric embedding models behave differently when embedding passages and queries. For that end, the model must "know" on inference time, what kind of data it is embedding.
The changes are:
1. src/main/java/org/opensearch/neuralsearch/processor/TextEmbeddingProcessor.java
The processor signals it is embedding passages, by passing the new AsymmetricTextEmbeddingParameters using the content type EmbeddingContentType.PASSAGE.
2. src/main/java/org/opensearch/neuralsearch/query/NeuralQueryBuilder.java
Analogously, the query builder uses EmbeddingContentType.QUERY.
3. src/main/java/org/opensearch/neuralsearch/ml/MLCommonsClientAccessor.java
Here is where most of the work was done. The class has been extended in a backwards-compatible way with inference methods that allow one to pass MLAlgoParams objects. Usage of AsymmetricTextEmbeddingParameters (which implements MLAlgoParams) is mandatory for asymmetric models. At the same time symmetric models do not accept them.
The only way to know whether a model is asymmetric or symmetric is by reading its model configuration (if the models' configuration contains a passage_prefix and/or a query_prefix, they are asymmetric, otherwise they are symmetric).
The src/main/java/org/opensearch/neuralsearch/ml/MLCommonsClientAccessor.java class deals with this, keeping the complexity in one place and not requiring any API change to the neural-search plugin (as proposed in #620). When calling the inference methods, clients (such as the TextEmbeddingProcessor) may pass the AsymmetricTextEmbeddingParameters object without caring if the model they are using is symmetric or asymmetric. The accessor class will first read the model's configuration (by calling the getModel API of the mlClient) and deal appropriately.
To avoid adding this extra roundtrip to every inference call, the asymmetry information is kept in a cache in memory.
Issues Resolved
#620
Check List

[x] New functionality includes testing.

[x] All tests pass

[ ] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@br3no can you add an entry in the changelog.

@br3no Thanks for raising the PR. I am wondering do we require this change? In MLCommons repository a generic MLInference processor is getting launched which is supposed to do the inference of any kind of model both during ingestion and search. RFC: https://github.com/opensearch-project/ml-commons/issues/2173
If that gets launched do we really need this feature?

@navneet1v I have been loosely following the discussions in the mentioned RFC. It's a large change that I don't expect to be stable soon – the PR is very much in flux. Also, I don't see the use-case of asymmetric embedding models being addressed.
This PR here is much smaller in comparison and is not in any way in conflict with the RFC work. If once the work on the ML Inference Processors is finished and the use-case is addressed there as well, we can deprecate and eventually remove the functionality again.
Until then, this PR offers users the chance to use more modern local embeddings. I'm eager to put this to spin, tbh.

Also, I don't see the use-case of asymmetric embedding models being addressed.

If that is the case I would recommend posting the same on the RFC to ensure that your use case is handled.
On the other hand, I do agree this is an interesting feature. I would like to get some eyes on this change mainly in terms of should this be added or not given a more generic processor is around the corner. As I am of my opinion is concerned the main reason of generic processor was to avoid creating new/updating processors to support new model types which is happening in this PR.
Thoughts? @jmazanec15 , @martin-gaievski , @vamshin , @vibrantvarun .
Let me add some PMs too for Opensearch-project to know their thoughts. @dylan-tong-aws

@navneet1v I have added a comment earlier today to the RFC (cf. https://github.com/opensearch-project/ml-commons/issues/2173#issuecomment-2078796787).
Sure, let's open the discussion and get some PMs into it.
I really don't mind leaving this out if the support is introduced in another PR in 2.14. I'm concerned https://github.com/opensearch-project/ml-commons/issues/2173 is a much larger effort, that won't be ready that quickly...
It's not about my contribution – I need the feature. 🙃

I really don't mind leaving this out if the support is introduced in another PR in 2.14. I'm concerned https://github.com/opensearch-project/ml-commons/issues/2173 is a much larger effort, that won't be ready that quickly...

I can see the feature is marked for 2.14 release of Opensearch. Let me add maintainers from ML team too. @mingshl , @ylwu-amzn

@mingshl @ylwu-amzn, I'd really like to have this feature in 2.14.
Do you think this use-case will be fully supported with https://github.com/opensearch-project/ml-commons/issues/2173? Cf. https://github.com/opensearch-project/ml-commons/issues/2173#issuecomment-2078796787
If not, I'd be happy to help this PR get merged as an interim solution! Let me know what you think!

@br3no ml inference processor is targeting at first supporting remote model only. How did you usually connect this model? is it local or remote?
if remote, can you please provide a SageMaker deployment code piece then I can quickly test it in 2.14 test cluster. Thanks

@mingshl sorry for taking so long to answer!
The use-case for now is to use a local, asymmetric model such as https://huggingface.co/intfloat/multilingual-e5-small.
This PR here is the last puzzle piece to allow one to use these kinds of model and should in principle also work with remote models. It makes sure that the neural-search plugin uses the correct inference parameters when embedding passages and queries with asymmetric models. Regardless of whether the model is local or remote, if you are using asymmetric models, you will need to provide this information anyway.
The thing is that asymmetric models need to know at inference time what exactly they are embedding. OpenSearch currently treats embedding models as symmetric, meaning that regardless of whether the text being embedded is a query or a passage, the embedding will be always the same. Asymmetric models require content "hints" to the text being embedded; the model exemplified above uses the string prefixes passage:  and query: . These models perform better than similarly sized symmetric models.
In https://github.com/opensearch-project/ml-commons/issues/1799 we have added the concept of asymmetric models into ml-commons, introducing the AsymmetricTextEmbeddingParameters class, used at inference time to signal if the text being embedded is a query or a passage. So this PR is only using this new infrastructure.
I would really be happy to get this merged as an interim solution until the ml inference processor fully supports this use-case.

I also vote for this PR in need for this functionality.

@br3no will it possible if you can contribute back in MLInference processor for local model support? Is that even an option?

I'd propose then to merge this PR now and then start the work to eventually replace this once the MLInference processor supports this use case...

The problem is once this is released it cannot be deprecated till a major version release. Hence I am bit hesitant to have this feature in neural search plugin.

This PR has a very small surface. It doesn't change any APIs. So I believe this is not something to worry about, actually. Once the MLInferece processor supports asymmetric models the neural-search plugin can be changed to start using it instead of what I built here. This would be not a breaking change, only an internal implementation detail.

Thanks @br3no , Can you show some examples of neural search query with asymmetric embedding model?  ML inference processor can support asymmetric embedding model. We are working on unifying the interface so we can also support local model in ML inference processor.  I think @navneet1v concern is valid. We should consider deprecation effort. Let's check how user experience looks like with this PR.

@ylwu-amzn as I said in the comment above, there is no API change in this PR. You would use the neural-search plugin in the same way it is used today.
What is implemented here?
The main work is done in MLCommonsClientAccessor. Whenever neural-search is embedding text (either in the TextEmbeddingProcessor or in the NeuralQueryBuilder) the plugin will ensure that if the embedding model is asymmetric, the parameters will be passed on correctly. More details can be found in the description and code.

@br3no , sorry that I didn't have enough time to read the details. Took a quick look, seems the code will specify the "QUERY" type when run query, and set  "PASSAGE" type when ingest. That seems no API change, cx can use same ingest pipeline and neural search, switching to asymmetric model is seamless. I think this is a good design, so it can support BWC and seems can be deprecated/migrated together with current text embedding APIs.   I'm good for merge this.

Exactly! Great!

@ylwu-amzn could you drive the review process forward then?

@ylwu-amzn could you drive the review process forward then?

Sorry , missed your comment. Asking neural-search maintainers if they have other concerns

Is there some progress or news regarding this PR? It is still a small thing without Api change which is the last puzzle piece to enable asymmetric embedding models.

I have no news. We've moved to prefixing the strings outside of OpenSearch.

@ylwu-amzn, @martin-gaievski, are there some things left that speaks against merging this functionality? It is a blocker right now, and it is a marginal change only that doesn't influence other functionalities.

I am going to pick this PR up right now, @br3no @reuschling sorry for the late reply. Let me check your code first.

@br3no I read through the context and I believe this is a valid use case and this PR provides user the flexibility to integrate the feature, thank you very much! But current the PR needs rebase with master and in case you're not monitoring the PR,  I've rebased your code with main and created this PR to your repo: https://github.com/br3no/neural-search/pull/1, please take time to check the PR, you can fix this PR or cherry-pick my commit to this PR, I would like to push this forward to make it released in 2.18 which is 5th Nov, thanks again.

@zane-neo great to hear this!
I have merged your PR. Is there anything else I need to do?

@zane-neo great to hear this!
I have merged your PR. Is there anything else I need to do?

@br3no The main blocker for now is the current PR has conflicts needs to be resolved, so I was saying there are two approaches to fix this:

You fix the conflicts by rebasing the main branch.
You cherry-pick my commit in the PR sent to your repo to your asymmetric-embeddings-620 branch and push the commit.
To be honest I prefer the first approach, but since you're merging my PR so you can continue with the left ones: cherry-pick my commit to the branch and push. Thanks!

@zane-neo I rebased the changes and adapted new tests that were affected by my changes. I think this is now good to go,

@zane-neo I rebased the changes and adapted new tests that were affected by my changes. I think this is now good to go.

Left several comments and please take a look, other part LGTM, thanks.

@zane-neo I have pushed a small refactoring of a test, as you suggested. Thanks!
I have also added comments to the issues you raised in the other threads.

@zane-neo I have pushed a small refactoring of a test, as you suggested. Thanks!
I have also added comments to the issues you raised in the other threads.

Thanks @br3no currently it LGTM, approved the PR, I'll push for another approval and then we can merge it. Sadly it's not possible to release it in 2.18 as the release flow has been started and code has been frozen, but I'm sure we can make it released in 2.19.

Hi @vibrantvarun ! Since this PR is targeted towards 2.19, we still have time to raise next PRs before code freeze date of 2.19 (Jan 28, 2025). I think we can leave the BWC tests for the next PR.
@br3no Can you discuss with @vibrantvarun on whether a DTO object is needed for this PR? I understand DTO object is a best practice, but I don't want it takes too long before merging this PR.

@br3no How are we ensuring BWC test compatibility?

This is a valid concern, I took a look on the code and it's could have BWC issue during seder. The latest version serializes the AsymmetricTextEmbeddingParameters, and if a node is deployed with a legacy version OS and when deserialization, it fetches the class from the internal cache map here, and since a legacy version doesn't have this class, the IllegalArgumentException will be thrown. @br3no Can you do a test on this case to double confirm if this is true? Thanks.

@zane-neo for some reason github ui glitching for me so I cannot put my answer under your comments

replacing map with cache object - it still worth it before we don't have many references to hashmap, besides API for our cache is very simple
nature of calls for requesting model metadata - that's good then, we can move forward without adding a retry logic then

I'll try to find time to work on this tomorrow. I'll replace the map with the OS cache and I'll try to have a look at the BWC tests.
Regarding the DTO, there's already #790 opened as a follow-up task. @yuye-aws, is this fine by you?

@br3no DTO object helps to make look code more readable and understandable.  Issue 790 is currently not prioritized and is in backlog. We don't know when will that issue will be prioritized. Therefore, I would request you to add it in this PR. It will not be a design level change or something completely changes the purpose of this PR.
cc: @martin-gaievski @yuye-aws @zhichao-aws

I'll try to find time to work on this tomorrow. I'll replace the map with the OS cache and I'll try to have a look at the BWC tests.
Regarding the DTO, there's already #790 opened as a follow-up task. @yuye-aws, is this fine by you?

It's fine for me as long as the DTO gets merged before the code freeze date of the 2.19.

@br3no DTO object helps to make look code more readable and understandable. Issue 790 is currently not prioritized and is in backlog. We don't know when will that issue will be prioritized. Therefore, I would request you to add it in this PR. It will not be a design level change or something completely changes the purpose of this PR.
cc: @martin-gaievski @yuye-aws @zhichao-aws

Let's label the issue 790 with 2.19 tag. The DTO is a small fix. If @br3no is too busy to create a PR, I can help him instead. As @reuschling has stated, this PR is the last puzzle for the asymmetric embedding feature. I don't want it be too long before we gets it merged.

@br3no @yuye-aws some of my comments still are not addressed, the major one is switching from hashmap to existing core class for cache. in regards to comments for using unbounded map - I do not agree with mentioned arguments. We need to have a limit of that cache size because the lifespan of the cluster may be long (e.g. in a managed OS service) and instance type for data node can be tiny, like t2 with 1Gb. Switching to existing cache component that provides that limit and keeps system stable is close to zero effort, why not to do this.

Seems that the cache class will evict outdated elements with LRU policy. @br3no Do you think it's fine to deprecate some model cache?

Sorry folks, I didn't have the time to look into the latest comments yet. It's been busy weeks. I'll try to have a look today and address all remaining points.

I have addressed all open issues with the exception of the BWC test. I'll try to have a look in the coming days.

Pinging @martin-gaievski to review the cache.

I have addressed all open issues with the exception of the BWC test. I'll try to have a look in the coming days.

For the BWC test, you just need to create two classes for TextEmbeddingProcessorIT.java. One for rolling upgrade and another for restart upgrade:

qa/rolling-upgrade/src/test/java/org/opensearch/neuralsearch/bwc/
qa/restart-upgrade/src/test/java/org/opensearch/neuralsearch/bwc/TextChunkingProcessorIT.java

@yuye-aws aren't the test cases covered completely by the HybridSearchIT classes? There the text embedding processor is created, documents are indexed and a hybrid query is executed.

I think I will need some assistance in understanding what to do exactly regarding the BWC tests. I'm not even sure I understand where to look for errors.
The error scenario you folks are seeing is:

a cluster is updated
a query is sent to a node running an older version of OS
the query contains an AsymmetricTextEmbeddingParameters
the legacy version cannot deserialize this class

Did I get this right? Or is it another scenario you are concerned with?

The class AsymmetricTextEmbeddingParameters has been part of ml-commons for about 3 months. This would then be a problem for all versions older than 2.14.

You can exclude versions older than 2.14 just like here.

@yuye-aws aren't the test cases covered completely by the HybridSearchIT classes? There the text embedding processor is created, documents are indexed and a hybrid query is executed.

Do you mean the BWC tests for HybridSearchIT? If so, you can fill in the gap between BWC tests and integrations tests for TextEmbeddingProcessorIT.

@yuye-aws
What I meant with this comment is that I don't see the need to implement a different BWC Test class for the change in this PR. The existing BWC tests (e.g. the HybridQueryIT, but others as well) test the complete process of creating an index, creating a pipeline for document embedding and issuing neural queries. There is no new process introduced with this PR that requires different BWC test code, from my point of view. Please correct me if I'm wrong.
BTW the rolling-upgrade BWC tests are failing on the main branch.
I ran
./gradlew :qa:rolling-upgrade:testRollingUpgrade -D'tests.bwc.version=2.18.0-SNAPSHOT'

without success.
This makes it hard to add new tests.

./gradlew :qa:rolling-upgrade:testRollingUpgrade

@vibrantvarun The comment makes sense to me. Can you check whether BWC tests are needed and help him?

@martin-gaievski I have addressed all your latest comments. Hope this PR can now be approved.

@br3no How are we ensuring BWC test compatibility?

This is a valid concern, I took a look on the code and it's could have BWC issue during seder. The latest version serializes the AsymmetricTextEmbeddingParameters, and if a node is deployed with a legacy version OS and when deserialization, it fetches the class from the internal cache map here, and since a legacy version doesn't have this class, the IllegalArgumentException will be thrown. @br3no Can you do a test on this case to double confirm if this is true? Thanks.

@br3no Please take a look on this comment, the seder is the only risk I see between old nodes and new nodes, and I don't think HybridSearchIT has covered asymmetric case as it only tested with text embedding model(symmetric model), they don't have the new configuration introduced in asymmetric model so no seder issue either. I would suggest you create a old cluster with two nodes(ml-node) and replace one nodes with latest code, and test two cases:

Send request to old node and make the old node dispatch the request to new node.
Send request to new node and make the new node dispatch request to old node.
You don't need to manually configure anything to enable the dispatch as ml-commons automatically dispatch the requests to different nodes in a round-robin fashion, by controlling which node receives the request and trigger request twice, one of the request will be dispatched to another node. If you don't see any seder exception then it's good to merge the PR, thanks.

@zane-neo so if I get this right, I should create a new test that uses the asymmetric model feature. This test should only run for OS versions >= 2.19. Is this right?

Ping.

@zane-neo so if I get this right, I should create a new test that uses the asymmetric model feature. This test should only run for OS versions >= 2.19. Is this right?
Your concern is about making sure future releases will not break compatibility with this feature.

@br3no That's right to use the asymmetric model feature for OS>=2.19, but the thing is a little bit different. you should test a mixed cluster with OS 2.19 and OS 2.18, since 2.18 has different code base of neural search, so a request to 2.19 node and then being dispatched to 2.18 node could encounter seder issues. My guess is you can test on this and based on the result:

No seder error, you can add BWC tests with OS >= 2.19
Has seder error, fix that and add BWC tests with OS >= 2.19

btw, if you're working on BWC for 2.19+ and want to check how the results are, better rebase on latest main, we've switched 2.18-snapshot to 2.18/2.19-snapshot. This branch will keep failing on running BWCs

@br3no the CI is in good shape now, you can work on BWC tests.

@zane-neo so if I get this right, I should create a new test that uses the asymmetric model feature. This test should only run for OS versions >= 2.19. Is this right?
Your concern is about making sure future releases will not break compatibility with this feature.

@br3no That's right to use the asymmetric model feature for OS>=2.19, but the thing is a little bit different. you should test a mixed cluster with OS 2.19 and OS 2.18, since 2.18 has different code base of neural search, so a request to 2.19 node and then being dispatched to 2.18 node could encounter seder issues. My guess is you can test on this and based on the result:

No seder error, you can add BWC tests with OS >= 2.19
Has seder error, fix that and add BWC tests with OS >= 2.19

@br3no I tested on this and I didn't find seder issues for OS version >= 2.13 so there's no concern on this, you can work on creating BWC tests for OS version >= 2.19, thanks.

Sorry folks, I didn't have time lately to invest here. I'll try to do this in the holiday season.

Sorry folks, I didn't have time lately to invest here. I'll try to do this in the holiday season.

Hi @br3no,  @brianf-aws is planning on adding a tutorial for using asymmetric local model with ml inference processors during ingest and search. https://github.com/opensearch-project/ml-commons/pull/3258 you can watch for the new updates and post comments through the pr.

@mingshl does this make this PR here obsolete?

@mingshl does this make this PR here obsolete?

Hey Breno, I did researching and thanks to your contribution (Asymmetric Embedding support) within ML-Commons its possible to get embeddings on demand using the ML Inference processor (MLinferenceSearchProcessor & MLInferenceIngestProcessor^). I didn't read fully through the comment thread but I believe what you are going after can be done.
Hopefully the tutorial (https://github.com/opensearch-project/ml-commons/pull/3258) being created matches your use case. If not can you let us know what might be missing so we can understand better?
^There is a bit of a caveat, (There is a slight code change required to allow Asymmetric Embeddings with MLInferenceIngest Processor please see here https://github.com/opensearch-project/ml-commons/pull/3281).


"file is not a database: , while compiling: select count(*) from sqlite_master; "

Hello!, works fine on most devices, but there are those who crash after 2-3 weeks of use. With exception "file is not a database: , while compiling: select count(*) from sqlite_master; "
how fix this error ?
use example:
  SafeHelperFactory.Options options = SafeHelperFactory.Options
            .builder()
            .setClearPassphrase(false)
            .build();

    val factory = SafeHelperFactory.fromUser(new SpannableStringBuilder("123456789xx"), options);
    return Room.databaseBuilder(application, AppDatabase.class, mDBName)
            .openHelperFactory(factory)
            .addMigrations(
                    MIGRATION_1_2,
                    MIGRATION_2_3,
            )
            .build();

As is noted on the project home page, this project is no longer under active development. You should no longer be using this library. SQLCipher for Android supports Room directly.
Beyond that, the most common reason for this message is an incorrect passphrase.

Beyond that, the most common reason for this message is an incorrect passphrase.

That's not right. There is a systematic error in SQLCipher for Android. ~ 2% of our 10 million user devices run into this error with a fix passphrase.

You should no longer be using this library

But as your wrote 👍


New Scraper: Huron City Council

Create a new scraper for Huron City Council
Jurisdiction: Huron
Website: http://cityofhuron.com/meeting-agenda-and-minutes/

Hello! May I take on this issue?

@RenTrieu hi im so sorry this is so late. if still interested, yes absolutely, would much appreciate your help. again, sos osrry i wasn't checking messages on these issues.