Deflake GRM's kubelet CSR approver integration test suite

How to categorize this PR?

/area dev-productivity
/kind flake
What this PR does / why we need it:
After #6784, we see some flaky behaviour of the new integration test for GRM's kubelet CSR approver controller. This PR fixes them and improves the test's robustness:
Before:
$ stress -ignore "unable to grab random port" -p 32 ./csrapprover.test -ginkgo.v
...
87 runs so far, 1 failures

Now:
$ stress -ignore "unable to grab random port" -p 32 ./csrapprover.test -ginkgo.v
...
567 runs so far, 0 failures

Which issue(s) this PR fixes:

https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/6719/pull-gardener-integration/1579816538729353216
https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/6719/pull-gardener-integration/1579819687879905280
https://prow.gardener.cloud/view/gs/gardener-prow/pr-logs/pull/gardener_gardener/6806/pull-gardener-integration/1579828221333999616

Release note:

NONE

/assign

/lgtm

/approve


below Tibetan syllables can't be converted.

I used pyewts to convert some text and here are the Tibetan syllables with errors.
dangs
zhwa
dwang
rma
tshe
phywa
dge
rgya
dwags
I don't have much time to look into it by myself. Will someone pls fix them?
Interestingly, it can convert left parenthesis but not the right one. For example,
(rtse mgron)
will be converted to
༼རྩེ་མགྲོན)

thanks! I'll be traveling in the next 10 days so I probably won't have time soon, but I'll look into this yes

I've fixed an exception and the right parentheses. There's definitely a bug with the wasur, but for
dangs
rma
tshe
dge
rgya

I'm getting
དངས
རྨ
ཚེ
དགེ
རྒྱ

which look ok to me... what result do you get?

I've just fixed the wasur issue

they are good now. great job!
On Fri, Nov 8, 2019 at 4:49 PM Elie Roux<EMAIL_ADDRESS>wrote:

I've just fixed the wasur issue
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Esukhia/pyewts/issues/5?email_source=notifications&email_token=ABD5ROMJDNCBZIAZBUTWKJLQSURZ3A5CNFSM4JKTBYU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDPFCHQ#issuecomment-551440670,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABD5ROOVN3II2S43UKEVDBLQSURZ3ANCNFSM4JKTBYUQ
.

I'm pretty sure this can be closed, please reopen if it isn't


ServiceRequestError: SSL Certificate Verification Failure with Azure Document Intelligence

Package Name - azure-ai-documentintelligence:
Package Version - 1.0.0b3:
**Operating System -  Windows 10 **:
Python Version - 3.10.4:

Describe the bug
I am encountering a ServiceRequestError when attempting to analyze a PDF document using the Azure Document Intelligence service. The error seems to be related to SSL certificate verification. Below is the full traceback for your reference:
ServiceRequestError Traceback (most recent call last)
...
ServiceRequestError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)
I want to disable the SSL Verification.  How do I do that using Document Intelligence Client
To Reproduce
Steps to reproduce the behavior:
import os
import re
import json
import base64
import logging
from io import BytesIO
from PyPDF2 import PdfWriter, PdfReader
from azure.core.credentials import AzureKeyCredential

from azure.ai.documentintelligence import DocumentIntelligenceClient
def get_text_from_pdf(fname):
    #try:
        fstream = open(fname, "rb")
        inputpdf = PdfReader(fstream)
        page_dict = {}

        document_intelligence_client = DocumentIntelligenceClient(endpoint=config['doc_intel_endpoint'],
                                                                  credential=AzureKeyCredential(config['doc_intel_key']),pipeline=pipeline)
        for i in range(len(inputpdf.pages)):
            output = PdfWriter()
            output.add_page(inputpdf.pages[i])
            response_bytes_stream = BytesIO()
            output.write(response_bytes_stream)

            poller = document_intelligence_client.begin_analyze_document(
                "prebuilt-layout",
                AnalyzeDocumentRequest(bytes_source=response_bytes_stream.getvalue()),
                output_content_format=ContentFormat.MARKDOWN
            )
            result: AnalyzeResult = poller.result()
            page_dict[i + 1] = result.content

        fstream.close()
        return page_dict
    # except Exception as e`
```

Install the required libraries using pip install azure-ai-documentintelligence pypdf2.
Run the provided code snippet with the given PDF file path.
Observe the ServiceRequestError during the document analysis process.
**Expected behavior**
It will give an SSL  Verification error.  It is mostly due to firewall at my companies VDI. I need help in disabling it.

**Additional context**

Thanks for reaching out.
Could you clarify what you mean by "I need help in disabling it. "?
Do you want to disable the ssl verification?

Hello Xiang Yan,
Thank you for your response. I am getting SSL verification error. So the way we say in request(verify=False), can we do something similar in Azure SDK for Document intelligence?
Regards,
Nirmal
From: Xiang Yan @.>
Sent: Monday, June 10, 2024 5:46 PM
To: Azure/azure-sdk-for-python @.>
Cc: Nirmalsing Patil @.>; Author @.>
Subject: Re: [Azure/azure-sdk-for-python] ServiceRequestError: SSL Certificate Verification Failure with Azure Document Intelligence (Issue #36003)
[External Message]
Thanks for reaching out.
Could you clarify what you mean by "I need help in disabling it. "?
Do you want to disable the ssl verification?
—
Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/Azure/azure-sdk-for-python/issues/36003*issuecomment-2159342446__;Iw!!PePb2utDNNs!tEOi9RpgvzA3PmSlDOgTAQUyplSZHrEJcV1rRkbJHeJmAz80AzfgV3DnTpwidsImnl6BhaloW_mlDWkm47mffgk7iw$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/BIZGKUYZHEJ7VIJKMSBFGTDZGYNB7AVCNFSM6AAAAABJCXV7OGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJZGM2DENBUGY__;!!PePb2utDNNs!tEOi9RpgvzA3PmSlDOgTAQUyplSZHrEJcV1rRkbJHeJmAz80AzfgV3DnTpwidsImnl6BhaloW_mlDWkm47lsG7fDBQ$.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited.

Thanks for the clarification.
You can add connection_verify=False when you create the client to disable the ssl verification. https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md#configurations
Please note that by setting that, you will not validate the certificate when communicating to the service. It will be insecure.

"/unresolve"
Now I am getting 404 Resource not found. I double checked the API key and the endpoint
region - eastus2
Code :

def get_text_from_pdf(fname):
    #try:
        fstream = open(fname, "rb")
        inputpdf = PdfReader(fstream)
        page_dict = {}

        document_intelligence_client = DocumentIntelligenceClient(endpoint=config['doc_intel_endpoint'],
                                                                  credential=AzureKeyCredential(config['doc_intel_key']),connection_verify =False)
        for i in range(len(inputpdf.pages)):
            output = PdfWriter()
            output.add_page(inputpdf.pages[i])
            response_bytes_stream = BytesIO()
            output.write(response_bytes_stream)

            poller = document_intelligence_client.begin_analyze_document(
                "prebuilt-document",
                AnalyzeDocumentRequest(bytes_source=response_bytes_stream.getvalue()),
                output_content_format=ContentFormat.MARKDOWN
            )
            result: AnalyzeResult = poller.result()
            page_dict[i + 1] = result.content

        fstream.close()
        return page_dict

Package Name - azure-ai-documentintelligence:
Package Version - 1.0.0b3:
**Operating System - Windows 10 **:
Python Version - 3.10.4:

"Now I am getting 404 Resource not found. "
This issue is different from the original one.
To make it more trackable, please open another one. (likely it needs help from a different dev)


Update ExoPlayer version

Updated ExoPlayer version

Fixed broken music file of example

Why will need to update the ExoPlayer version?

Sometime online streaming stop or can't able to play long audio file. To prevent this issue we will need to use the wake lock. Updated version of ExoPlayer take care of acquire/release the wake lock.
Release Note
More detail of issue

Hi, First of all, I really appreciate your help. Second of all, did you run tests to see if your update doesn't cause any errors?

@danielR2001  Yes I have run the all test case and it is working fine. I will share the video link.

Ok, as far as I can see the PR is great I will merge it as soon as I can (hopefully today).


Pending Tiles

https://github.com/rafaelkamv/Ark-Of-Lights/blob/93ee1bddc61607d090eb4642a702f05f07b45196/ArkOfLights/Content/Items/Placeables/Bars/SteelBar.cs#L13

will add more later

https://github.com/rafaelkamv/Ark-Of-Lights/blob/564dec86b826653a494a4a4e893ba209991515f8/ArkOfLights/Content/Items/Placeables/Bars/FoolsGold.cs#L15-L16
https://github.com/rafaelkamv/Ark-Of-Lights/blob/564dec86b826653a494a4a4e893ba209991515f8/ArkOfLights/Content/Items/Placeables/Bars/RoyalGold.cs#L14
(Royal gold Tiles doesn't works)


Add key rotation timer and key switch guard timer.

This PR fixes #732

Current coverage is 69.89% (diff: 62.79%)

Merging #751 into master will decrease coverage by 0.78%

@@             master       #751   diff @@
==========================================
  Files           106        105     -1
  Lines         15733      15759    +26
  Methods        2143       2145     +2
  Messages          0          0
  Branches       1873       1878     +5
==========================================
- Hits          11120      11014   -106
- Misses         4013       4137   +124
- Partials        600        608     +8

Powered by Codecov. Last update a2dc290...13755e7

LGTM


On Flutter Web CircularProgressIndicator() takes 100% of CPU

If I always show CircularProgressIndicator() on the web it creates an infinite number of verbose.
please check this video:
https://imgur.com/r0UPFAy

Hi @abuanwar072
Can you please provide your flutter doctor -v, and a minimal complete reproducible code sample?
Thank you

This appears to be a debug build, so the verbose diagnostics and high CPU usage is expected
@abuanwar072 could you please test if the issue persists in release or profile mode?

@PixelToast Thank you so much 😊, It works properly on the release or profile mode.
If anyone faces this same problem please run flutter run --profile


Fix Incorrect Pulse Length

As referenced in https://github.com/Zepheus/ESP8266-rf-mqtt-bridge/issues/1


test: test ci

@fospring I have fixed CI here: https://github.com/near/near-sdk-rs/pull/1186/commits/4f3785733427016b9c3e3c157620459832efd20e


behavior_path_planner needs to lock planner_data_ mutex before calling getTurnSignal

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I'm convinced that this is not my fault but a bug.

Description
This line is outside the mutex_pd_ lock guard:
https://github.com/autowarefoundation/autoware.universe/blob/17f1b956a397b1661471ad820cb53219aaeaaa54/planning/behavior_path_planner/src/behavior_path_planner_node.cpp#L645
This means the planner_data_ content may be modified when the getTurnSignal function is trying to access it.
For example I have modified the route_handler::getClosestLaneletWithinRoute to crash if the route handler is not ready:
bool RouteHandler::getClosestLaneletWithinRoute(const Pose & search_pose, lanelet::ConstLanelet * closest_lanelet) const
{
  if (!is_handler_ready_) {
    // TODO(vrichard) clean
    std::cerr << "Trying to call getClosestLaneletWithinRoute() but handler is not ready" << std::endl;
    throw std::runtime_error("debug");
    return false;
  }
  std::optional<lanelet::ConstLanelet> optional_lanelet = lanelet_route_.getClosestLaneletWithinRoute(search_pose);
  if (!optional_lanelet) {
    return false;
  }
  *closest_lanelet = *optional_lanelet;
  return true;
}

Then I run the planning simulator and set the goal pose with Rviz, many times. After many tries (it's all about timing), the exception is thrown:
[New Thread 0x7fff808ab700 (LWP 114166)]
Trying to call getClosestLaneletWithinRoute() but handler is not ready
terminate called after throwing an instance of 'std::runtime_error'
  what():  debug
[Thread 0x7fff808ab700 (LWP 114166) exited]

Thread 17 "component_conta" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffce7fc700 (LWP 108608)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7947859 in __GI_abort () at abort.c:79
#2  0x00007ffff7bd0911 in  () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff7bdc38c in  () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff7bdc3f7 in  () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff7bdc6a9 in  () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007fffec0f8d68 in route_handler::RouteHandler::getClosestLaneletWithinRoute(geometry_msgs::msg::Pose_<std::allocator<void> > const&, lanelet::ConstLanelet*) const [clone .cold] () at /home/sig/autoware/install/route_handler/lib/libroute_handler.so
#7  0x00007fffec5acc49 in behavior_path_planner::util::calcLaneletPathAroundPose(std::shared_ptr<route_handler::RouteHandler>, geometry_msgs::msg::Pose_<std::allocator<void> > const&, double, double) () at /home/sig/autoware/install/behavior_path_planner/lib/libbehavior_path_planner_node.so
#8  0x00007fffec6542ff in behavior_path_planner::TurnSignalDecider::getTurnSignal(std::shared_ptr<behavior_path_planner::PlannerData const> const&, autoware_auto_planning_msgs::msg::PathWithLaneId_<std::allocator<void> > const&, behavior_path_planner::TurnSignalInfo const&) () at /home/sig/autoware/install/behavior_path_planner/lib/libbehavior_path_planner_node.so
#9  0x00007fffec4fdca2 in behavior_path_planner::BehaviorPathPlannerNode::run() () at /home/sig/autoware/install/behavior_path_planner/lib/libbehavior_path_planner_node.so
#10 0x00007fffec50ab77 in rclcpp::GenericTimer<std::_Bind<void (behavior_path_planner::BehaviorPathPlannerNode::*(behavior_path_planner::BehaviorPathPlannerNode*))()>, (void*)0>::execute_callback() () at /home/sig/autoware/install/behavior_path_planner/lib/libbehavior_path_planner_node.so
#11 0x00007ffff7e6ef7d in rclcpp::Executor::execute_any_executable(rclcpp::AnyExecutable&) () at /opt/ros/galactic/lib/librclcpp.so
#12 0x00007ffff7e754bb in rclcpp::executors::MultiThreadedExecutor::run(unsigned long) () at /opt/ros/galactic/lib/librclcpp.so
#13 0x00007ffff7c08de4 in  () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#14 0x00007ffff7883609 in start_thread (arg=<optimized out>) at pthread_create.c:477
#15 0x00007ffff7a44133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)

By the time I print is_handler_ready_ in gdb it is already back to true.
Expected behavior
planner data read/write requires a mutex lock.
Actual behavior
route_handler may be read/written simulataneously.
Steps to reproduce
add a printf or an exception in the route_handler function above, and set the goal pose many times in the planning simulator.
Versions
No response
Possible causes
No response
Additional context
No response

@purewater0901 Could you check this issue?

@purewater0901 is assigned for this issue

Submit PR.
@VRichardJP Thank you for your reporting.

same problem than https://github.com/autowarefoundation/autoware.universe/issues/2495


[API-8] Add AbsoluteSortPosition annotations

Gonna close this for now as AbsoluteSortPosition is quite borked in eventimplgen. Need to spend the time to fix it.


[WIP] Feature: Add ECR presubmit testing.

:gift: This leverages OIDC federation to enable presubmit testing against ECR.
/kind feature

Codecov Report

Merging #934 (e0eb352) into main (76f46cc) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #934   +/-   ##
=======================================
  Coverage   51.21%   51.21%
=======================================
  Files          44       44
  Lines        3419     3419
=======================================
  Hits         1751     1751
  Misses       1439     1439
  Partials      229      229

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


pull request for e2e test at 12/14/2016 22:15:11

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


fix windows build

Checklist

[x] Used a fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

there are 2 builds for windows and each has its own checksum.
using conda selectors we can have a different checksum for each.
in addition to this it seems cran is rebuilds these binary packages and the checksum changes every so often, but there not much we can do about that..

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ready to merge !

@rvalieris Thanks!

Nice!


[RFC] Implement pruning for neural sparse search

Background
Neural Sparse is a semantic search method which is built on native Lucene inverted index. The documents and queries are encoded into sparse vectors, where the entry represents the token and their corresponding semantic weight.
Since the model expands the tokens with semantic weights during the encoding process, the number of tokens in the sparse vectors is often greater than the original raw text. Additionally, the token weights in the sparse vectors exhibit a significant long-tail distribution, where tokens with lower semantic importance occupy a large portion of the storage space. In the experiments of this blog, we found that the index sizes produced by the two modes of neural sparse search are 4.7 times and 6.8 times larger than the BM25 inverted index.
Pruning can effectively alleviate this problem. During the process of ingestion and search, we prune the sparse vectors according to different strategies, removing tokens with relatively small weights. Research has shown that even simple pruning strategies can significantly reduce index size while preserving most of the search accuracy[1]. This can help users achieve a better balance between search accuracy and cost.
What are we going to do?

Implement pruning at sparse_encoding ingestion processor. Users can configure the pruning strategy when create the processor, and the processor will prune the sparse vectors before write to index.
Implement pruning at neural_sparse query clause. Users can configure the pruning strategy when search with neural_sparse query. The query builder will prune the query before search on index.
Give a quantitative analysis of the pruning strategies. Users can refer to the analysis results to configure the pruning strategy, thereby achieving the desired balance between accuracy and cost.

Pruning strategy
We propose to implent these 4 pruning strategies:
Pruning by weight threshold
For this method, given a threshold T, all tokens whose weight is smaller than T will be pruned.
def prune_by_weight_threshold(sparse_vector, threshold):
    pruned_vector = {}
    for token, weight in sparse_vector.items():
        if weight >= threshold:
            pruned_vector[token] = weight
    return pruned_vector

Pruning by ratio with max weight
For this method, given a sparse vector X, we first find the max weight of X, and calculate the weight ratio of every token and the max weight. If the ratio is smaller than threshold T, it will be pruned.
def prune_by_ratio_with_max_weight(sparse_vector, threshold):
    pruned_vector = {}
    max_weight = max(sparse_vector.values())
    for token, weight in sparse_vector.items():
        ratio = weight / max_weight
        if ratio >= threshold:
            pruned_vector[token] = weight
    return pruned_vector

Pruning by Top-K
For this method, given a sparse vector S, we first sort the tokens based on their weights, from large to small. And we only keep the tokens with Top-K values.
def prune_by_top_k(sparse_vector, k):
    sorted_vector = sorted(sparse_vector.items(), key=lambda x: x[1], reverse=True)
    pruned_vector = dict(sorted_vector[:k])
    return pruned_vector

Pruning by alpha-mass[2]
For this method, given a sparse vector S, we first sort the tokens based on their weights, from large to small. We iterate on the vector entries and record the accumulated values, until the ratio of accumulated values and sum of all values is larger than threshold T. And the non-iterated entries are dropped.
def prune_by_alpha_mass(sparse_vector, threshold):
    sorted_vector = sorted(sparse_vector.items(), key=lambda x: x[1], reverse=True)
    pruned_vector = {}
    accumulated_mass = 0
    total_mass = sum(sparse_vector.values())
    for token, weight in sorted_vector:
        accumulated_mass += weight
        if accumulated_mass / total_mass >= threshold:
            break
        pruned_vector[token] = weight
    return pruned_vector

API
To create an ingest processor with pruning:
PUT /_ingest/pipeline/sparse-pipeline
{
    "description": "Calling sparse model to generate expanded tokens",
    "processors": [
        {
            "sparse_encoding": {
                "model_id": "fousVokBjnSupmOha8aN",
                "field_map": {
                    "body": "body_sparse"
                },
                "pruning_config":{
                    "pruning_type": "alpha_mass",
                    "threshold": 0.8
                }
            }
        }
    ]
}

To search with pruning:
GET /test-index/_search
{
    "query": {
        "neural_sparse": {
            "body_sparse": {
                "query_text": "i be going",
                "model_id": "fousVokBjnSupmOha8aN",
                "pruning_config":{
                    "pruning_type": "alpha_mass",
                    "threshold": 0.8
                }
            }
        }
    }
}

References
[1]: A Static Pruning Study on Sparse Neural Retrievers
[2]: Efficient Inverted Indexes for Approximate Retrieval over
Learned Sparse Representations

To make the ingest processor work for raw sparse vectors ingestion, one prerequiste is to implement this issue: #793 . We can add a parameter to configure whether call model inference for raw vectors ingestion

Another tricky part is the combination with 2-phase search. My thought on the proper behavior is we first prune, then split the queries to two phase. Please feel free to put more comments about this.

@zhichao-aws Can you add some context on what is pruning?

Can you add some context on what is pruning?

In the context of sparse vector representations, pruning is a technique used to reduce the size of the sparse vectors by removing or "pruning" tokens that have relatively low semantic weights or importance.
In neural sparse search, documents and queries are encoded into sparse vectors, where each entry represents a token and its corresponding semantic weight. For example: "hello world" -> {"hello": 1.1, "world":1.2, "hi": 0.9, "planet": 0.1, "greeting": 0.5, "earth":0.15} (just for example, not real encoding result). For pruning purpose, we can remove planet and earth, to reduce the storage and increase search speed.
By applying pruning strategies, users can achieve a balance between search accuracy and storage costs, as research has shown that even simple pruning strategies can significantly reduce index size while preserving most of the search accuracy.

I conducted experiments to test the impact of pruning on ingestion of search. POC code: https://github.com/zhichao-aws/neural-search/tree/pruning. Benchmark code: https://github.com/zhichao-aws/neural-search/tree/prune_test.
In conclusion, we can save about 60% index size with a trade-off of ~1% search relevance by applying the pruning during ingestion (works for both doc-only and bi-encoder).
doc-only:

bi-encoder:

We then test the impact on search relevance by pruning the sparse vector at retrieval time. We test on the BEIR datasets that  have sufficient amount of documents, and use the index that are pruned to ~40% index size. We test these pruning methods and the 2-phase search.
In conclusion, the search-time pruning can boost the search time, but the search relevance get dropped very fast when we prune more tokens. In conclusion, 2-phase search is a more robust method that is both fast and precise. (especially bi-encoder)

Is there a GH issue for this RFC?

Is there a GH issue for this RFC?

No, we don't have an issue for it

To make the ingest processor work for raw sparse vectors ingestion, one prerequiste is to implement this issue: #793 . We can add a parameter to configure whether call model inference for raw vectors ingestion

Please help me to understand why #793 is prerequisite? In my understanding #793 is about not calling model inference again during document update when there is no changes on the original text for the embedding.

To make the ingest processor work for raw sparse vectors ingestion, one prerequiste is to implement this issue: #793 . We can add a parameter to configure whether call model inference for raw vectors ingestion

Please help me to understand why #793 is prerequisite? In my understanding #793 is about not calling model inference again during document update when there is no changes on the original text for the embedding.

If we want to support prune for raw sparse vector ingestion, it's a prerequisite. Because currently we always do model inference and it will update the sparse vector field, even it is already provided.
But from my understanding this is not a top-priority requirement, because if the sparse vectors are managed outside OpenSearch, the prune logics can also be applied direcly on them. For the first launch of the feature, we can go with current framework.

Thank you for your response. From my understanding, we call the model to generate embeddings during a document update because we're using a neural search ingest pipeline. In that case, if pruning is configured in the ingest pipeline, it would also be applied, correct?

Thank you for your response. From my understanding, we call the model to generate embeddings during a document update because we're using a neural search ingest pipeline. In that case, if pruning is configured in the ingest pipeline, it would also be applied, correct?

Yes, it will be applied

When searching, applying prune to 2-phase search has superseded applying it to neural sparse query body. Therefore, enhancing the existing 2-phase search pipeline makes more sense to me.
To maintain compatibility with existing APIs, the overall API will look like:
# ingestion pipeline
PUT /_ingest/pipeline/sparse-pipeline
{
    "description": "Calling sparse model to generate expanded tokens",
    "processors": [
        {
            "sparse_encoding": {
                "model_id": "fousVokBjnSupmOha8aN",
                "pruning_type": "alpha_mass",
                "pruning_ratio": 0.8,
                "field_map": {
                    "body": "body_sparse"
                },
            }
        }
    ]
}

# two phase pipeline

PUT /_search/pipeline/neural_search_pipeline
{
"request_processors": [
{
"neural_sparse_two_phase_processor": {
"tag": "neural-sparse",
"description": "Creates a two-phase processor for neural sparse search.",
"pruning_type": "alpha_mass",
"pruning_ratio": 0.8,
}
}
]
}


[BUG] Could not resolve "#imports" in product-detail-page example (stackblitz only)

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
project does not run, for example https://stackblitz.com/github/shopware/frontends/tree/main/examples/product-detail-page?file=vite.config.js
Expected Behavior
I'm able to run the project
Steps To Reproduce

go to https://stackblitz.com/github/shopware/frontends/tree/main/examples/product-detail-page?file=vite.config.js
watch the console to see the errors

Environment
- OS:
- Node:
- pnpm:

Anything else?
No response

@mkucmus I tried to create a custom Vue application with shopware frontends, but I run into the same errors as the "#imports" imports could not be resolved in different files from "@shopware-pwa/composable-next". Will there be a bug release soon?


Replace PDF icon in diary page to an oss friendly one

The current icon for the PDF (I believe it's in this file) is very similar to the one from Adobe (p 6).
Adobe isn't exactly your open-source friendly company.
There are other icons that represent PDF, some even with just the letters PDF, which is also clearer than the PDF symbol.
What do you guys think?
This one looks like a good candidate
https://thenounproject.com/icon/pdf-2201522/

Hi Leo
I see your concerns. I actually thought I had only used icons from https://materialdesignicons.com. These have a harmless license.
Unfortunately, I can no longer find the corresponding PDF icon on the website.
When I find time again, I can replace it.
PS: Code contributions are welcome.


[stable/cerebro] Add ability to disable livenessProbe/readinessProbe.

Also add myself as maintainer.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [stable/mychartname])

Up.

/lgtm


Add image support

Allows attaching images to roadmap items
Adds faux Cloudinary integration by allowing users to enter their Cloudinary credentials on /settings
If Cloudinary creds are not present in the org, image upload features are hidden

@lottiecoxon Now we can do "album art" for each of our in-progress features!