Error in Leiden Algorithm in create_base_entity_graph

Describe the issue
I got an empty network when doing the Leiden clustering algorithm as follows:
{"type": "error", "data": "Error executing verb "cluster_graph" in create_base_entity_graph: EmptyNetworkError", "stack": ... leiden.EmptyNetworkError: EmptyNetworkError\n", "source": "EmptyNetworkError", "details": null}
When opening my parquet files for each step in pandas, there is only an entity_graph column with an incomplete graphml URL. I saw on other posts that there should also be a clustered_graph column, but there is none for me. When I look in the cache directory however, both entity_extraction and summarize_descriptions have valid JSON results, so I'm not sure how exactly the graph became empty. My data is a set of .txt files of US Congressional hearings, and I previously used the prompt autotune feature to customize prompts to my data.
Steps to reproduce
joint-20240710T193325Z-001.zip
To generate results, I simply ran the init command followed by !python -m graphrag.prompt_tune --root ./ragtest --domain "US congress hearings" and then !python -m graphrag.index --verbose --root ./ragtest
GraphRAG Config Used
encoding_model: cl100k_base
skip_workflows: []
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_chat # or azure_openai_chat
model: gpt-4-turbo-preview
model_supports_json: true # recommended if this is available for your model.
max_tokens: 4000
request_timeout: 180.0
api_base: https://.openai.azure.com
api_version: 2024-02-15-preview
organization: <organization_id>
deployment_name: <azure_model_deployment_name>
tokens_per_minute: 150_000 # set a leaky bucket throttle
requests_per_minute: 10_000 # set a leaky bucket throttle
max_retries: 10
max_retry_wait: 10.0
sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
concurrent_requests: 25 # the number of parallel inflight requests that may be made
parallelization:
stagger: 0.3
num_threads: 50 # the number of threads to use for parallel processing
async_mode: threaded # or asyncio
embeddings:
parallelization: override the global parallelization settings for embeddings
async_mode: threaded # or asyncio
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_embedding # or azure_openai_embedding
model: text-embedding-3-small
# api_base: https://.openai.azure.com
# api_version: 2024-02-15-preview
# organization: <organization_id>
# deployment_name: <azure_model_deployment_name>
# tokens_per_minute: 150_000 # set a leaky bucket throttle
# requests_per_minute: 10_000 # set a leaky bucket throttle
# max_retries: 10
# max_retry_wait: 10.0
# sleep_on_rate_limit_recommendation: true # whether to sleep when azure suggests wait-times
# concurrent_requests: 25 # the number of parallel inflight requests that may be made
# batch_size: 16 # the number of documents to send in a single request
# batch_max_tokens: 8191 # the maximum number of tokens to send in a single request
# target: required # or optional
chunks:
size: 300
overlap: 100
group_by_columns: [id] # by default, we don't allow chunks to cross documents
input:
type: file # or blob
file_type: text # or csv
base_dir: "input"
file_encoding: utf-8
file_pattern: ".*\.txt$"
cache:
type: file # or blob
base_dir: "cache"
connection_string: <azure_blob_storage_connection_string>
container_name: <azure_blob_storage_container_name>
storage:
type: file # or blob
base_dir: "output/${timestamp}/artifacts"
connection_string: <azure_blob_storage_connection_string>
container_name: <azure_blob_storage_container_name>
reporting:
type: file # or console, blob
base_dir: "output/${timestamp}/reports"
connection_string: <azure_blob_storage_connection_string>
container_name: <azure_blob_storage_container_name>
entity_extraction:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
prompt: "prompts/entity_extraction.txt"
entity_types: [organization,person,geo,event]
max_gleanings: 0
summarize_descriptions:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
prompt: "prompts/summarize_descriptions.txt"
max_length: 500
claim_extraction:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
enabled: true
prompt: "prompts/claim_extraction.txt"
description: "Any claims or facts that could be relevant to information discovery."
max_gleanings: 0
community_report:
llm: override the global llm settings for this task
parallelization: override the global parallelization settings for this task
async_mode: override the global async_mode settings for this task
prompt: "prompts/community_report.txt"
max_length: 2000
max_input_length: 8000
cluster_graph:
max_cluster_size: 10
embed_graph:
enabled: false # if true, will generate node2vec embeddings for nodes
num_walks: 10
walk_length: 40
window_size: 2
iterations: 3
random_seed: 597832
umap:
enabled: false # if true, will generate UMAP embeddings for nodes
snapshots:
graphml: false
raw_entities: false
top_level_nodes: false
local_search:
text_unit_prop: 0.5
community_prop: 0.1
conversation_history_max_turns: 5
top_k_mapped_entities: 10
top_k_relationships: 10
max_tokens: 12000
global_search:
max_tokens: 12000
data_max_tokens: 12000
map_max_tokens: 1000
reduce_max_tokens: 2000
concurrency: 32
Logs and screenshots
Logs.json
{"type": "error", "data": "Error executing verb "cluster_graph" in create_base_entity_graph: EmptyNetworkError", "stack": "Traceback (most recent call last):\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb\n    result = node.verb.func(**verb_args)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 61, in cluster_graph\n    results = output_df[column].apply(lambda graph: run_layout(strategy, graph))\n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/series.py", line 4924, in apply\n    ).apply()\n      ^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/apply.py", line 1427, in apply\n    return self.apply_standard()\n           ^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/apply.py", line 1507, in apply_standard\n    mapped = obj._map_values(\n             ^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/base.py", line 921, in _map_values\n    return algorithms.map_array(arr, mapper, na_action=na_action, convert=convert)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/algorithms.py", line 1743, in map_array\n    return lib.map_infer(values, mapper, convert=convert)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "lib.pyx", line 2972, in pandas._libs.lib.map_infer\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 61, in \n    results = output_df[column].apply(lambda graph: run_layout(strategy, graph))\n                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 167, in run_layout\n    clusters = run_leiden(graph, strategy)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/strategies/leiden.py", line 26, in run\n    node_id_to_community_map = _compute_leiden_communities(\n                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/strategies/leiden.py", line 61, in _compute_leiden_communities\n    community_mapping = hierarchical_leiden(\n                        ^^^^^^^^^^^^^^^^^^^^\n  File "<@beartype(graspologic.partition.leiden.hierarchical_leiden) at 0x32b439d00>", line 304, in hierarchical_leiden\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graspologic/partition/leiden.py", line 588, in hierarchical_leiden\n    hierarchical_clusters_native = gn.hierarchical_leiden(\n                                   ^^^^^^^^^^^^^^^^^^^^^^^\nleiden.EmptyNetworkError: EmptyNetworkError\n", "source": "EmptyNetworkError", "details": null}
{"type": "error", "data": "Error running pipeline!", "stack": "Traceback (most recent call last):\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/run.py", line 323, in run_pipeline\n    result = await workflow.run(context, callbacks)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 369, in run\n    timing = await self._execute_verb(node, context, callbacks)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb\n    result = node.verb.func(**verb_args)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 61, in cluster_graph\n    results = output_df[column].apply(lambda graph: run_layout(strategy, graph))\n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/series.py", line 4924, in apply\n    ).apply()\n      ^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/apply.py", line 1427, in apply\n    return self.apply_standard()\n           ^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/apply.py", line 1507, in apply_standard\n    mapped = obj._map_values(\n             ^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/base.py", line 921, in _map_values\n    return algorithms.map_array(arr, mapper, na_action=na_action, convert=convert)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/pandas/core/algorithms.py", line 1743, in map_array\n    return lib.map_infer(values, mapper, convert=convert)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "lib.pyx", line 2972, in pandas._libs.lib.map_infer\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 61, in \n    results = output_df[column].apply(lambda graph: run_layout(strategy, graph))\n                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 167, in run_layout\n    clusters = run_leiden(graph, strategy)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/strategies/leiden.py", line 26, in run\n    node_id_to_community_map = _compute_leiden_communities(\n                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/strategies/leiden.py", line 61, in _compute_leiden_communities\n    community_mapping = hierarchical_leiden(\n                        ^^^^^^^^^^^^^^^^^^^^\n  File "<@beartype(graspologic.partition.leiden.hierarchical_leiden) at 0x32b439d00>", line 304, in hierarchical_leiden\n  File "/Users/amansingh/anaconda3/lib/python3.11/site-packages/graspologic/partition/leiden.py", line 588, in hierarchical_leiden\n    hierarchical_clusters_native = gn.hierarchical_leiden(\n                                   ^^^^^^^^^^^^^^^^^^^^^^^\nleiden.EmptyNetworkError: EmptyNetworkError\n", "source": "EmptyNetworkError", "details": null}
Additional Information

GraphRAG Version: 0.1.1
Operating System: mac OS Sonoma 14.4.1
Python Version: 3.11.9
Related Issues:

HI @as1078
It seems entity extraction process failed and yield an empty graph.
Could you please share your log file?

Sure, since my log file is too large to upload, I have uploaded a portion of it here It seems that all other errors besides the clustering one were rate limit errors, which I thought were dealt with by GraphRAG through waiting before submitting another API request. I excluded the clustering errors that were put in above.
logs.json

I have this error too. I noticed that my generated prompts were missing a ) in the entity extraction.

Just noticed I had the same issue. Thanks!

On less performant models like the phi-3 https://github.com/microsoft/graphrag/pull/503 was able to repair the json. I did not test with prompt rewrite.

I have this error too. I noticed that my generated prompts were missing a ) in the entity extraction.

I still have the same error after I fix my generated prompts for entity extraction, do anyone know what might be the cause?

Faced the same issue. Found a potential fix. Putting it here just in case it is useful for somebody in the future, or if someone can identify why is this causing the error.
in the file graphrag/prompt_tune/prompt/entity_relationship.py
line no 25
3. Return output in {language} as a single list of all the entities and relationships identified in steps 1 and 2. Use **{{record_delimiter}}** as the list delimiter.
removing the asterisks(*) on either side of the {{record_delimiter}} fixes the prompt generation and the error during indexing for me:
3. Return output in {language} as a single list of all the entities and relationships identified in steps 1 and 2. Use {{record_delimiter}} as the list delimiter.

I meet this error on graphrag v0.2.1

Same in graphrag v0.2.0

Can anyone who is still facing the error try this change:
https://github.com/microsoft/graphrag/issues/515#issuecomment-2273538639
and confirm if the prompts generated now work fine or was it just a one time case for me

Faced the same issue. Found a potential fix. Putting it here just in case it is useful for somebody in the future, or if someone can identify why is this causing the error. in the file graphrag/prompt_tune/prompt/entity_relationship.py line no 25 3. Return output in {language} as a single list of all the entities and relationships identified in steps 1 and 2. Use **{{record_delimiter}}** as the list delimiter.
removing the asterisks(*) on either side of the {{record_delimiter}} fixes the prompt generation and the error during indexing for me: 3. Return output in {language} as a single list of all the entities and relationships identified in steps 1 and 2. Use {{record_delimiter}} as the list delimiter.

I didn't modify entity_relationship.py itself, but this worked for me in the auto-generated prompts.

Can anyone who is still facing the error try this change: #515 (comment) and confirm if the prompts generated now work fine or was it just a one time case for me

it doesn't work for me. In my graphrag version, ****has been removed in prompt and still having this issue


Add features to Tree cover loss widget.

Add min and max date.
Add all dates on the selector.

Add canopy density values on the selector.

Add:

Change data with date intervals.
Change data with canopy values.
Disabled and enabled dates.

Include percentage value.

Add background loading.


Sentry webpack plugin generates two artifacts bundles with uploadLegacySourcemaps option

Environment
sentry/webpack-plugin 2.10.0
node v18.12.1
Self hosted sentry version 23.8.0
Sentry react SDK: 7.80.0
Windows 10
Steps to Reproduce
I set up webpack plugin to upload my sourcemaps:
sentryWebpackPlugin({
    org: '<org>',
    project: '<project>',
    authToken: '<token>',
    url: '<url>',
    release: {
          name: 'test3',
          uploadLegacySourcemaps: {
              paths: [path.join(__dirname, 'Static/Bundles/js/')],
              urlPrefix: '~/Static/Bundles/js/'
          }
      },
      sourcemaps: { deleteFilesAfterUpload: '**/*.js.map' },
      debug: true
})

Expected/Actual Result
Its generating two bundles in sentry, one with Static/Bundles/js/ prefix and one without. I think that this is the reason, why the errors stacktrace does not get "prettyfied" — the debug_meta in the event points to the bundle without url prefix

Logs
Logs after build also contain several bundles reports
Source Map Upload Report
  Minified Scripts
    ~/05dbc6fd-2bc2-4308-986b-937d17df65be-0.js (no sourcemap ref, debug id 05dbc6fd-2bc2-4308-986b-937d17df65be)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/05dbc6fd-2bc2-4308-986b-937d17df65be-0.js)
    ~/11e33d51-5453-4516-9a55-99bf4fddd080-13.js (sourcemap at full-cert-info-page-b276916a2ad0cb320383.js.map, debug id 11e33d51-5453-4516-9a55-99bf4fddd080)
    ~/15f85526-5f39-4390-b889-3d230f523fcf-24.js (sourcemap at polyfill-c4060570eed3c8216f6f.js.map, debug id 15f85526-5f39-4390-b889-3d230f523fcf)
    ~/32e614d7-c2ac-4080-bf51-527424f95f9a-12.js (sourcemap at error-page-21f5ec10393dfb4042a4.js.map, debug id 32e614d7-c2ac-4080-bf51-527424f95f9a)
    ~/360e0e50-2b5f-4f14-b6af-b47a99d1d357-29.js (sourcemap at wizard-page-8649fa080da7ee8eafe3.js.map, debug id 360e0e50-2b5f-4f14-b6af-b47a99d1d357)
    ~/387800ce-c165-4d2c-8914-ef604996c21d-5.js (sourcemap at download-mydss-app-page-a1ffdc43d9de937cf16b.js.map, debug id 387800ce-c165-4d2c-8914-ef604996c21d)
    ~/39b35a04-ebe6-44b9-9a67-a1f410c341ed-20.js (no sourcemap ref, debug id 39b35a04-ebe6-44b9-9a67-a1f410c341ed)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/39b35a04-ebe6-44b9-9a67-a1f410c341ed-20.js)
    ~/45921710-a50f-4006-8489-8cb47e4eb891-15.js (sourcemap at list-page-ae3c91362aae7f20a29c.js.map, debug id 45921710-a50f-4006-8489-8cb47e4eb891)
    ~/491f06ed-7cd0-470d-9312-926683e70289-9.js (sourcemap at heic2any-dd255d7bf3cd15be65f4.js.map, debug id 491f06ed-7cd0-470d-9312-926683e70289)
    ~/4aee82fc-5436-4880-8903-4f9d0476562a-25.js (sourcemap at readonly-form-page-32a020046242e3cdf9a2.js.map, debug id 4aee82fc-5436-4880-8903-4f9d0476562a)
    ~/4b57be1e-8712-4f45-844d-b6cdb271d007-1.js (sourcemap at check-session-756756c267ca24c43de8.js.map, debug id 4b57be1e-8712-4f45-844d-b6cdb271d007)
    ~/5716f833-e669-484f-a1cc-b09cc5f7d73d-4.js (sourcemap at download-mobile-signature-app-page-dcf38d05c592e293cb84.js.map, debug id 5716f833-e669-484f-a1cc-b09cc
5f7d73d)
    ~/60b30516-7dbc-4537-a553-f7578438b98a-11.js (sourcemap at error-handler-aa1235590ab03cfccf53.js.map, debug id 60b30516-7dbc-4537-a553-f7578438b98a)
    ~/69076acc-e9a3-423a-9faf-216ff101e168-14.js (sourcemap at invalid-browser-page-fd7ec231df5e1d7c5012.js.map, debug id 69076acc-e9a3-423a-9faf-216ff101e168)
    ~/77011b16-899d-4814-89d0-5811ba0476eb-16.js (sourcemap at login-by-token-page-2fc4370e05b5a626bb1e.js.map, debug id 77011b16-899d-4814-89d0-5811ba0476eb)
    ~/9232ae4e-7b0d-4a82-85d1-108c6837fc45-19.js (sourcemap at page-with-text-from-window-3e8f6a1f3775f980f619.js.map, debug id 9232ae4e-7b0d-4a82-85d1-108c6837fc45
)
    ~/95d5a823-fbfd-4f6c-a7a2-3f258fbf1cdf-28.js (sourcemap at vendors-dad2a179a20ea42fa8a0.js.map, debug id 95d5a823-fbfd-4f6c-a7a2-3f258fbf1cdf)
    ~/a1a9dd35-f8c0-4a42-a244-1f71bdd2d4d7-7.js (sourcemap at dss-info-page-ce21b6ab27e152f540fd.js.map, debug id a1a9dd35-f8c0-4a42-a244-1f71bdd2d4d7)
    ~/ae4ff70d-fb0d-4b75-81ab-804d362ae866-6.js (sourcemap at dss-activation-page-7ff62b986f49deb1cce0.js.map, debug id ae4ff70d-fb0d-4b75-81ab-804d362ae866)
    ~/ae8385f1-25f3-4106-afdf-01705f640213-2.js (sourcemap at common-components-4ba55aad8713ac01e789.js.map, debug id ae8385f1-25f3-4106-afdf-01705f640213)
    ~/b0745f11-14f5-4598-b3da-4078f1210c5e-21.js (no sourcemap ref, debug id b0745f11-14f5-4598-b3da-4078f1210c5e)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/b0745f11-14f5-4598-b3da-4078f1210c5e-21.js)
    ~/bc19330a-e35f-431d-a565-bb12597cc17c-27.js (no sourcemap ref, debug id bc19330a-e35f-431d-a565-bb12597cc17c)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/bc19330a-e35f-431d-a565-bb12597cc17c-27.js)
    ~/bda19e9b-f012-43f5-b232-f3cb99d2d158-26.js (sourcemap at runtime-62f9ef326a5d3bae0cb3.js.map, debug id bda19e9b-f012-43f5-b232-f3cb99d2d158)
    ~/c27b4579-a93e-4f1f-bd70-40f4dc645aeb-3.js (sourcemap at core-580fa937c40abb194431.js.map, debug id c27b4579-a93e-4f1f-bd70-40f4dc645aeb)
    ~/c5700fcc-4d42-4f85-9d14-ccdadd34eeab-23.js (no sourcemap ref, debug id c5700fcc-4d42-4f85-9d14-ccdadd34eeab)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/c5700fcc-4d42-4f85-9d14-ccdadd34eeab-23.js)
    ~/cc1e95f8-f594-480f-825b-1652bf13493e-22.js (no sourcemap ref, debug id cc1e95f8-f594-480f-825b-1652bf13493e)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/cc1e95f8-f594-480f-825b-1652bf13493e-22.js)
    ~/cdf03558-3bb2-4d25-8b97-b3c40c1fb3f8-10.js (sourcemap at enter-8b27127e99b41fd5ddeb.js.map, debug id cdf03558-3bb2-4d25-8b97-b3c40c1fb3f8)
    ~/daf79fa8-be74-4184-9cd4-da3300397bc3-8.js (sourcemap at file-type-7f95dd38a67cb8950b2c.js.map, debug id daf79fa8-be74-4184-9cd4-da3300397bc3)
    ~/e141203a-0c4e-4231-94ab-93529c2cba4f-17.js (sourcemap at modules-b9f5b21ac15377849caf.js.map, debug id e141203a-0c4e-4231-94ab-93529c2cba4f)
    ~/fb1d3dbb-50ca-4dbe-943e-749ecc820bee-18.js (no sourcemap ref, debug id fb1d3dbb-50ca-4dbe-943e-749ecc820bee)
      - warning: could not determine a source map reference (Could not auto-detect referenced sourcemap for ~/fb1d3dbb-50ca-4dbe-943e-749ecc820bee-18.js)
  Source Maps
    ~/11e33d51-5453-4516-9a55-99bf4fddd080-13.js.map (debug id 11e33d51-5453-4516-9a55-99bf4fddd080)
    ~/15f85526-5f39-4390-b889-3d230f523fcf-24.js.map (debug id 15f85526-5f39-4390-b889-3d230f523fcf)
    ~/32e614d7-c2ac-4080-bf51-527424f95f9a-12.js.map (debug id 32e614d7-c2ac-4080-bf51-527424f95f9a)
    ~/360e0e50-2b5f-4f14-b6af-b47a99d1d357-29.js.map (debug id 360e0e50-2b5f-4f14-b6af-b47a99d1d357)
    ~/387800ce-c165-4d2c-8914-ef604996c21d-5.js.map (debug id 387800ce-c165-4d2c-8914-ef604996c21d)
    ~/45921710-a50f-4006-8489-8cb47e4eb891-15.js.map (debug id 45921710-a50f-4006-8489-8cb47e4eb891)
    ~/491f06ed-7cd0-470d-9312-926683e70289-9.js.map (debug id 491f06ed-7cd0-470d-9312-926683e70289)
    ~/4aee82fc-5436-4880-8903-4f9d0476562a-25.js.map (debug id 4aee82fc-5436-4880-8903-4f9d0476562a)
    ~/4b57be1e-8712-4f45-844d-b6cdb271d007-1.js.map (debug id 4b57be1e-8712-4f45-844d-b6cdb271d007)
    ~/5716f833-e669-484f-a1cc-b09cc5f7d73d-4.js.map (debug id 5716f833-e669-484f-a1cc-b09cc5f7d73d)
    ~/60b30516-7dbc-4537-a553-f7578438b98a-11.js.map (debug id 60b30516-7dbc-4537-a553-f7578438b98a)
    ~/69076acc-e9a3-423a-9faf-216ff101e168-14.js.map (debug id 69076acc-e9a3-423a-9faf-216ff101e168)
    ~/77011b16-899d-4814-89d0-5811ba0476eb-16.js.map (debug id 77011b16-899d-4814-89d0-5811ba0476eb)
    ~/9232ae4e-7b0d-4a82-85d1-108c6837fc45-19.js.map (debug id 9232ae4e-7b0d-4a82-85d1-108c6837fc45)
    ~/95d5a823-fbfd-4f6c-a7a2-3f258fbf1cdf-28.js.map (debug id 95d5a823-fbfd-4f6c-a7a2-3f258fbf1cdf)
    ~/a1a9dd35-f8c0-4a42-a244-1f71bdd2d4d7-7.js.map (debug id a1a9dd35-f8c0-4a42-a244-1f71bdd2d4d7)
    ~/ae4ff70d-fb0d-4b75-81ab-804d362ae866-6.js.map (debug id ae4ff70d-fb0d-4b75-81ab-804d362ae866)
    ~/ae8385f1-25f3-4106-afdf-01705f640213-2.js.map (debug id ae8385f1-25f3-4106-afdf-01705f640213)
    ~/bda19e9b-f012-43f5-b232-f3cb99d2d158-26.js.map (debug id bda19e9b-f012-43f5-b232-f3cb99d2d158)
    ~/c27b4579-a93e-4f1f-bd70-40f4dc645aeb-3.js.map (debug id c27b4579-a93e-4f1f-bd70-40f4dc645aeb)
    ~/cdf03558-3bb2-4d25-8b97-b3c40c1fb3f8-10.js.map (debug id cdf03558-3bb2-4d25-8b97-b3c40c1fb3f8)
    ~/daf79fa8-be74-4184-9cd4-da3300397bc3-8.js.map (debug id daf79fa8-be74-4184-9cd4-da3300397bc3)
> Uploading completed in 0.762s
> Uploaded files to Sentry
> Processing completed in 0.071s
> File upload complete (processing pending on server)
> Organization: ***
> Project: ***
> Release: test3
> Dist: None
> Upload type: artifact bundle

Source Map Upload Report
  Scripts
    ~/Static/Bundles/js/base-09ae2ebe8fdb53e1b9a4.js
    ~/Static/Bundles/js/old-ie-dd0ca898f591a55791fa.js
    ~/Static/Bundles/js/pages~Forbidden-6b7bc3e7650a93e925dd.js
    ~/Static/Bundles/js/pages~IE-d9cc54337a4d8c31552e.js
    ~/Static/Bundles/js/pages~OldIE-70cdafc8b487ba00d73c.js
    ~/Static/Bundles/js/pages~OldOS-9c1b70ec108ba4a73be9.js
    ~/Static/Bundles/js/static-page-0751dbad1bdcfe7b536f.js
  Minified Scripts
    ~/Static/Bundles/js/check-session-756756c267ca24c43de8.js (sourcemap at check-session-756756c267ca24c43de8.js.map)
    ~/Static/Bundles/js/common-components-4ba55aad8713ac01e789.js (sourcemap at common-components-4ba55aad8713ac01e789.js.map)
    ~/Static/Bundles/js/core-580fa937c40abb194431.js (sourcemap at core-580fa937c40abb194431.js.map)
    ~/Static/Bundles/js/download-mobile-signature-app-page-dcf38d05c592e293cb84.js (sourcemap at download-mobile-signature-app-page-dcf38d05c592e293cb84.js.map)
    ~/Static/Bundles/js/download-mydss-app-page-a1ffdc43d9de937cf16b.js (sourcemap at download-mydss-app-page-a1ffdc43d9de937cf16b.js.map)
    ~/Static/Bundles/js/dss-activation-page-7ff62b986f49deb1cce0.js (sourcemap at dss-activation-page-7ff62b986f49deb1cce0.js.map)
    ~/Static/Bundles/js/dss-info-page-ce21b6ab27e152f540fd.js (sourcemap at dss-info-page-ce21b6ab27e152f540fd.js.map)
    ~/Static/Bundles/js/dynamic/file-type-7f95dd38a67cb8950b2c.js (sourcemap at file-type-7f95dd38a67cb8950b2c.js.map)
    ~/Static/Bundles/js/dynamic/heic2any-dd255d7bf3cd15be65f4.js (sourcemap at heic2any-dd255d7bf3cd15be65f4.js.map)
    ~/Static/Bundles/js/enter-8b27127e99b41fd5ddeb.js (sourcemap at enter-8b27127e99b41fd5ddeb.js.map)
    ~/Static/Bundles/js/error-handler-aa1235590ab03cfccf53.js (sourcemap at error-handler-aa1235590ab03cfccf53.js.map)
    ~/Static/Bundles/js/error-page-21f5ec10393dfb4042a4.js (sourcemap at error-page-21f5ec10393dfb4042a4.js.map)
    ~/Static/Bundles/js/full-cert-info-page-b276916a2ad0cb320383.js (sourcemap at full-cert-info-page-b276916a2ad0cb320383.js.map)
    ~/Static/Bundles/js/invalid-browser-page-fd7ec231df5e1d7c5012.js (sourcemap at invalid-browser-page-fd7ec231df5e1d7c5012.js.map)
    ~/Static/Bundles/js/list-page-ae3c91362aae7f20a29c.js (sourcemap at list-page-ae3c91362aae7f20a29c.js.map)
    ~/Static/Bundles/js/login-by-token-page-2fc4370e05b5a626bb1e.js (sourcemap at login-by-token-page-2fc4370e05b5a626bb1e.js.map)
    ~/Static/Bundles/js/modules-b9f5b21ac15377849caf.js (sourcemap at modules-b9f5b21ac15377849caf.js.map)
    ~/Static/Bundles/js/page-with-text-from-window-3e8f6a1f3775f980f619.js (sourcemap at page-with-text-from-window-3e8f6a1f3775f980f619.js.map)
    ~/Static/Bundles/js/polyfill-c4060570eed3c8216f6f.js (sourcemap at polyfill-c4060570eed3c8216f6f.js.map)
    ~/Static/Bundles/js/readonly-form-page-32a020046242e3cdf9a2.js (sourcemap at readonly-form-page-32a020046242e3cdf9a2.js.map)
    ~/Static/Bundles/js/runtime-62f9ef326a5d3bae0cb3.js (sourcemap at runtime-62f9ef326a5d3bae0cb3.js.map)
    ~/Static/Bundles/js/vendors-dad2a179a20ea42fa8a0.js (sourcemap at vendors-dad2a179a20ea42fa8a0.js.map)
    ~/Static/Bundles/js/wizard-page-8649fa080da7ee8eafe3.js (sourcemap at wizard-page-8649fa080da7ee8eafe3.js.map)
  Source Maps
    ~/Static/Bundles/js/check-session-756756c267ca24c43de8.js.map
    ~/Static/Bundles/js/common-components-4ba55aad8713ac01e789.js.map
    ~/Static/Bundles/js/core-580fa937c40abb194431.js.map
    ~/Static/Bundles/js/download-mobile-signature-app-page-dcf38d05c592e293cb84.js.map
    ~/Static/Bundles/js/download-mydss-app-page-a1ffdc43d9de937cf16b.js.map
    ~/Static/Bundles/js/dss-activation-page-7ff62b986f49deb1cce0.js.map
    ~/Static/Bundles/js/dss-info-page-ce21b6ab27e152f540fd.js.map
    ~/Static/Bundles/js/dynamic/file-type-7f95dd38a67cb8950b2c.js.map
    ~/Static/Bundles/js/dynamic/heic2any-dd255d7bf3cd15be65f4.js.map
    ~/Static/Bundles/js/enter-8b27127e99b41fd5ddeb.js.map
    ~/Static/Bundles/js/error-handler-aa1235590ab03cfccf53.js.map
    ~/Static/Bundles/js/error-page-21f5ec10393dfb4042a4.js.map
    ~/Static/Bundles/js/full-cert-info-page-b276916a2ad0cb320383.js.map
    ~/Static/Bundles/js/invalid-browser-page-fd7ec231df5e1d7c5012.js.map
    ~/Static/Bundles/js/list-page-ae3c91362aae7f20a29c.js.map
    ~/Static/Bundles/js/login-by-token-page-2fc4370e05b5a626bb1e.js.map
    ~/Static/Bundles/js/modules-b9f5b21ac15377849caf.js.map
    ~/Static/Bundles/js/page-with-text-from-window-3e8f6a1f3775f980f619.js.map
    ~/Static/Bundles/js/polyfill-c4060570eed3c8216f6f.js.map
    ~/Static/Bundles/js/readonly-form-page-32a020046242e3cdf9a2.js.map
    ~/Static/Bundles/js/runtime-62f9ef326a5d3bae0cb3.js.map
    ~/Static/Bundles/js/vendors-dad2a179a20ea42fa8a0.js.map
    ~/Static/Bundles/js/wizard-page-8649fa080da7ee8eafe3.js.map

Hi, can you check in your setup that webpack isn't executed twice, cause I am pretty sure we're only uploading once/creating one bundle? Thanks.
https://github.com/getsentry/sentry-javascript-bundler-plugins/blob/88c345151752a3ff3017497665aab2a9254e53a6/packages/bundler-plugin-core/src/plugins/release-management.ts#L63-L81

When I remove the uploadLegacySourcemaps  I get only one bundle as expected, so I guess it is not executed twice...

Maybe, I do not need this option at all? But without it my stacktrace is still unminifyed. Trying to run sentry-cli resolve ends with missing context line error
sentry-cli   --url=URL --auth-token=TOKEN --log-level=debug  sourcemaps explain --release=test4 --org=ORG --project=PROJECT cd800fc87654457084f02ebcd2057850

✔ Fetched data for event: cd800fc87654457084f02ebcd2057850
✔ Event has release name: test4
✔ Event has a valid exception present
✔ Event has a valid stacktrace present
⚠ Exception is already source mapped and first resolved frame points to:

> [missing context line]

I tired ommiting the uploadLegacySourcemaps option all together, changed sentry libraries versions as in this thread but uminifying still does not work :(
sentryWebpackPlugin({
    org: '<org>',
    project: '<project>',
    authToken: '<token>',
    url: '<url>',
    release: {
        name: 'test6',
        // uploadLegacySourcemaps: {
        //     paths: [path.join(__dirname, 'Static/Bundles/js/')],
        //     urlPrefix: '~/Static/Bundles/js/'
        // }
   },
   sourcemaps: {
       // deleteFilesAfterUpload: '**/RA/**/*.js.map',
       assets: '**/RA/**/*.js*'
  },
  debug: true
})

The only difference is that is issue became handled, although I don't know, what does it mean

With this config I also get one bundle uploaded in sentry, the issue gets unhandled, stacktrace not unminifying
sentryWebpackPlugin({
    org: '<org>',
    project: '<project>',
    authToken: '<token>',
    url: '<url>',
    release: {
        name: 'test7',
        uploadLegacySourcemaps: {
            paths: ['**/RA/**/js/**'],
            urlPrefix: '~/Static/Bundles/js/'
        }
    },
    sourcemaps: {
        // deleteFilesAfterUpload: '**/RA/**/*.js.map',
        assets: '**/RA/**/*.js*'
    },
    debug: true
})

You're on an old self hosted version that might be buggy. Please upgrade to the newest one!

Is there a minimal not-buggy version? Will it work on lower version, 22.10.0? Right now we are using 22.10.0 for prod sentry and 23.8.0 is for testing\dev

Is there any way to fix it on given version of Sentry? I tried plugin 1.20.1, but still no result
new SentryWebpackPlugin({
        org: '***',
        project: '***',
        authToken: '***',
        url: '***',
        include: ['**/RA/**/js', '**/RA/**/js/**'],
        urlPrefix: '~/Static/Bundles/js/',
        debug: true,
        release: 'test10'
    })

Unfortunately there are a million things that can go wrong and it is super hard to debug self hosted just from some screenshots. I recommend upgrading everything to the latest version and we can work from there.

I was having the same issue. I fixed it by passing a nonexistent glob to the sourceMap asset. I think whenever uploadLegacySourcemaps is there, sentry tries to upload both the default sourcemaps and the legacy ones. Passing a nonexisting glob terminates the default sourcemap upload. Though I am purely guessing here.
sentryEsbuildPlugin({
  org: <org>,
  project: <project>,
  authToken: <authToken>,
  sourcemaps: {
    assets: "./**/*.nonexistent", // try to trick sentry into not uploading default sourcemaps
  },
  release: {
    cleanArtifacts: true,
    name: <releaseName>,
    uploadLegacySourcemaps: {
      paths: ["./build"],
      urlPrefix: "~/static",
    },
  }
})

@roymrinmoy good find this is likely it.

Sadly didn't help. Will try again when our self-hosted sentry gets updated

The workaround with fake sourcemaps did help with the two artifacts bundles being uploaded, but the stacktrace is still unminifyed. So I guess I'll wait till our team update sentry itself, and then willl try again

Hi! We updated out Sentry version, so now I've got the following setup:

Sentry 24.4.1
@sentry/react 7.112.1
@sentry/webpack-plugin 2.16.1

Code for initing the react library is the following:
init({
    release: 'test27-04-24_2',
    dist: 'distName',
    dsn: '<dsn>'
});

This is the webpack plugin:
sentryWebpackPlugin({
   org: '<org>',
   project: '<project>',
   authToken: '<token>',
   url: '<url>',
   release: {
          name: 'test27-04-24_2',
           dist: 'distName'
           // note: if I uncomment the code below, I get +1 artifact bundle
           // uploadLegacySourcemaps: {
           //     paths: ['**/RA/**/js', '**/RA/**/js/**'],
           //     urlPrefix: '~/Static/Bundles/js/'
           // }
    },
    sourcemaps: {
        // deleteFilesAfterUpload: '**/RA/**/*.js.map',
        assets: ['**/RA/**/js', '**/RA/**/js/**']
    },
    debug: true
})

The result, I'm getting at Sentry — only one artifact bundle, happy checkboxes, that I have configured the source maps, but stacktrace is still minifyed. If I uncomment the uploadLegacySourcemaps option, I get a success checkbox on the "Releases" tab, but still — the stacktrace is minifyed.
What am I missing?
Below are screenshots of Issue stacktrace, debug_meta, and uploaded sourcemaps with debug_id matching the event:

@Ollisteka Can you check the logs for the symbolicator service of your self hosted instance? Thanks.

Hello again! Thanks for the hint about symbolicator — there were errors Failed to create cache directory due to Permission denied. DevOps gave the rights to the service, and uminifying started to work!
The sourcemaps are uploading fine (only one artifact bundle in the source maps tab, instead of two) and unminifying also works with the following setup:
"@sentry/webpack-plugin": "2.16.1",
"@sentry/react": "7.112.1",
"webpack": "5.66.0",

Self hosted Sentry v24.4.1

sentryWebpackPlugin({
    org: '<org>',
    project: '<project>',
    authToken: '<token>',
    url: '<url>',
    release: {
        name: env.ReleaseName,
        dist: env.CUSTOMIZATION
    },
    sourcemaps: {
        assets: ['**/serviceName/**/js', '**/serviceName/**/js/**']
    },
    debug: true
})

So I'm finally closing the issue, thanks for the help!


Task from all running containers seams to get registered

Task from all running containers seams to get registered even when
ofelia daemon -f label=com.docker.compose.project=${COMPOSE_PROJECT_NAME}
is used

Yep, I've noticed the same issue. When I get time I'll look into it. thank you for reporting this!

The fix is released in v0.6.5: https://github.com/netresearch/ofelia/releases/tag/v0.6.5


Can Firebase Security Rules be manages from separate files in Source Control

This question came up during technical planning. I wanted each of the 5 backend groups of serverless functions to manage their own firebase rules. This would make it more maintainable over the long run. This ticket is here in case anyone following the project want to help me answer or do the experiment for me. The way we would do the experiment is as follows.

Setup two firebase folders on your machine connected to the same Firebase project.
Init the security rules and make them different in both of those folders
Deploy project ones rules, check that they are there.
Deploy project two's rules, check if projects ones rules are still available.

This is most likely going to fail and the rules will be replaced by the second deploy. The next step of the experiment is to spend maximum an hour to see if there's a way that we can setup an npm script that will get all the rules from different folders, combine it and then deploy that. I'm not skilled in the npm management / setup but when I get to this point I'll probably check for it.
If that's not possible then we'll most likely have to maintain all the firebase security rules in one massive file which will be a maintenance nightmare. Comments will become our friend in there with clear separation between the groups and their functions.
If anyone will take this on please comment in here and i'll change the label to in progress, otherwise I'll keep it open until I get to this point.

Hi! I hope this could help https://stackoverflow.com/questions/35052400/concatenate-files-with-npm-as-build-tool

Hi! I hope this could help https://stackoverflow.com/questions/35052400/concatenate-files-with-npm-as-build-tool

Hi! I haven't tested it but you might want to read this.
https://stackoverflow.com/questions/35052400/concatenate-files-with-npm-as-build-tool

Hi! I haven't tested it but you might want to read this.
https://stackoverflow.com/questions/35052400/concatenate-files-with-npm-as-build-tool

What about creating multiple firebase projects? Might be too complicated but it’s posible.
https://firebase.google.com/docs/projects/multiprojects

What about creating multiple firebase projects? Might be too complicated but it’s posible.
https://firebase.google.com/docs/projects/multiprojects

@DevSkiss thanks. That looks like what I'm looking for.

@DevSkiss thanks. That looks like what I'm looking for.

@jorgemndoza I use multiple firebase projects for different environments. I like the suggestion but maintaining 5 firebase projects with takes too much of a hit to maintenance capability for what you're getting back. I can simply just have 1 firestore rules file. Plus you also can't connect the Flutter app to multiple firebase projects at the same time which means we can't use that approach anyway.

@jorgemndoza I use multiple firebase projects for different environments. I like the suggestion but maintaining 5 firebase projects with takes too much of a hit to maintenance capability for what you're getting back. I can simply just have 1 firestore rules file. Plus you also can't connect the Flutter app to multiple firebase projects at the same time which means we can't use that approach anyway.

I like @DevSkiss idea too.
@FilledStacks for sure. 5 firestore projects to maintain is going to be complicated.
I found this while searching the web.
https://github.com/lupas/firestore-rulez

I like @DevSkiss idea too.
@FilledStacks for sure. 5 firestore projects to maintain is going to be complicated.
I found this while searching the web.
https://github.com/lupas/firestore-rulez


Allow setting ExternalTrafficPolicy for k8s dataplane proxy service

Problem Statement
For cloud providers like azure and gcp, they dont support proxy protocol and the only way to setup client ip preservation is by setting ExternalTrafficPolicy to local in the lb service spec. Currently KGO don't expose setting this for the dataplane service
...
Proposed Solution
Extend ServiceOptions to include more k8s service overrides
Additional Information
https://techcommunity.microsoft.com/t5/fasttrack-for-azure/how-client-source-ip-preservation-works-for-loadbalancer/ba-p/3033722

@sentinelleader #241 contains the draft of API change.

#241 has been merged. It's already available in nightly and will land in 1.3


Restore numpy 2 build

Comment:
This is a note to self about getting the numpy 2.0 builds back up and running for this repo.
The numpy 2.0 builds were automatically proposed in https://github.com/conda-forge/pygeoprocessing-feedstock/pull/28
The release build for https://github.com/conda-forge/pygeoprocessing-feedstock/pull/29 had failed builds that look related to the numpy 2.0 builds.
I reverted the numpy 2.0 builds in https://github.com/conda-forge/pygeoprocessing-feedstock/pull/31
Once the 2.4.4 build is merged in, it'd be great to try out the numpy 2.0 build once again.

This was resolved in #33 .
Also note the specific comment that includes how I got the CI stuff to re-render correctly: https://github.com/conda-forge/pygeoprocessing-feedstock/pull/33#issuecomment-2403439938


Billing options template messed up

@Cig-group , thank you for the reply.
This issue fixed by this commit


MobileService create table/API/new job gives exception

Right click mobile service -> Create Table -> specify the table name and click create gives exception. This issue was reported by @DonGlo

This is something temporary i guess. This was working fine before. May be API flakiness??

I can replicate this every time on my machine.

This issue is reproducible when the user creates a new mobile service from the plug in and tries to  create table/API/new job in the created mobile service.

Refresh will allow to operate, it is dependent of the provisioning time. This is a low bug.

We need to research if there is a way of querying the status of the mobile service.

Let's check with the mobile services team as to what would be a reliable way to get status of readiness of a newly created mobile service.

The issue still exist for newly created mobile services , and after considerable amount of times one can create table , jobs and API's .

deprecated functionality


Atlassian confluence audit function app changes reverted from PR 4539

Required items, please complete
Changes regarding IcM Incident 349338069 : [CRI] <2211150030000166 > -
Change(s):

See guidance below

Reason for Change(s):

See guidance below

Version Updated:

Required only for Detections/Analytic Rule templates
See guidance below

Testing Completed:

See guidance below

Checked that the validations are passing and have addressed any issues that are present:

See guidance below

Guidance <- remove section before submitting

Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:
Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR.  Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process.  Providing associated Issues that this resolves also easily connects the reason.

Change(s):

Updated syntax for XYZ.yaml

Reason for Change(s):

New schema used for XYZ.yaml
Resolves ISSUE #1234

Version updated:

Yes
Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly.  If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally.
https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:

Guidance for Detection checks
General contribution guidance
PR validation troubleshooting

@NikTripathi : Please resolve the conflicts. Thanks

Hi @NikTripathi,
Do we need these changes. If yes please get it reviewed and merged, else close it please.
Thanks

Got confirmation.


Set PriorityClass for kube-flannel, kube-dns, kube-dns-autoscaler, metrics-server to system-node-critical

What kind of request is this (question/bug/enhancement/feature request):
bug
Steps to reproduce (least amount of steps as possible):
Manually patch and add priorityclasses for kube-flannel, kube-dns, kube-dns-autoscaler, metrics-server to system-node-critical deployments:
kubectl patch daemonset kube-flannel --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-node-critical"}}}}'

kubectl patch deployment kube-dns --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-cluster-critical"}}}}'

kubectl patch deployment kube-dns-autoscaler --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-cluster-critical"}}}}'

kubectl patch deployment metrics-server --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-cluster-critical"}}}}'

Run kube-flannel update in a cluster upgrade.
Result:
The Priority Class settings get lost on a cluster upgrade because the daemonset is overwritten with default settings.
Other details that may be helpful:
The goal is to set a PriorityClass for kube-flannel, kube-dns, kube-dns-autoscaler, metrics-server, or flag them as "system-cluster-critical" in Rancher UI which will not be removed or overwritten on update or upgrade.
Using PriorityClass has the side effect that if not set for "kube-flannel, kube-dns, kube-dns-autoscaler and metrics-server" they get a lower priority, which can cause critical issues.
Manually patched and added PriorityClass for kube-dns, kube-dns-autoscaler, kube-flannel, and metrics-server these deployments work until an update is deployed. It should not remove their PriorityClass setting.
Perhaps, to set Rancher to use the Kubernetes "system-cluster-critical" value for the affected workloads.
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): 2.4.8
Installation option (single install/HA): HA

gz#13214

Todo:

Determine what is the current state of upstream and decide if this is the new default or if this needs to be made configurable
Implement the choice from above
Test if the setting is configured
Add docs on the new default/configurable setting

The point is the customer is using priorityClassName for his cluster to prioritizes pods. The side effect is that kube-flannel, kube-dns and kube-dns-autoscaler has by default no  priorityClassName.
The could be too slow, the cluster would crash.
Therefore the customer is patching. On update, the patches get lost.

Please share a status update. Is planed ETA for Rancher v2.5.5?

@asemen2608 Can you provide more information on how to reproduce this issue? I did the following and no success:

Run rancher v2.4.8 from docker image
Create cluster k8s version v1.15.12-rancher2-7 and use kube-flannel
Patch flannel and metrics-server from kubectl as following:

kubectl patch deployment metrics-server --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-cluster-critical"}}}}' -n kube-system
kubectl patch daemonset kube-flannel --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-node-critical"}}}}' -n kube-system

Verify priority class was set

kubectl get deployment metrics-server -n kube-system -o yaml
kubectl get daemonset kube-flannel -n kube-system -o yaml

Increase Recurring etcd Snapshot Interval to 13 in the UI's Cluster Edit page to trigger an update
Check priority class in kube-flannel and metrics-server (Template is different and kept changes from patch)
Upgrade cluster to k8s v1.16.15-rancher1-3
Check priority class in kube-flannel and metrics-server (Template is different and kept changes from patch)

@asemen2608 Can you provide more information on how to reproduce this issue? I did the following and no success:

Run rancher v2.4.8 from docker image
Create cluster k8s version v1.15.12-rancher2-7 and use kube-flannel
Patch flannel and metrics-server from kubectl as following:

kubectl patch deployment metrics-server --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-cluster-critical"}}}}' -n kube-system
kubectl patch daemonset kube-flannel --patch '{"spec": {"template": {"spec": {"priorityClassName": "system-node-critical"}}}}' -n kube-system

Verify priority class was set

kubectl get deployment metrics-server -n kube-system -o yaml
kubectl get daemonset kube-flannel -n kube-system -o yaml

Increase Recurring etcd Snapshot Interval to 13 in the UI's Cluster Edit page to trigger an update
Check priority class in kube-flannel and metrics-server (Template is different and kept changes from patch)
Upgrade cluster to k8s v1.16.15-rancher1-3
Check priority class in kube-flannel and metrics-server (Template is different and kept changes from patch)

PriorityClass could be set in rkeConfig.
For example:

https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/
https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/

dns:
provider: coredns
priorityClassName: system-cluster-critical
network:
plugin: flannel
priorityClassName: system-node-critical

PriorityClass could be set in rkeConfig.
For example:

https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/
https://rancher.com/docs/rke/latest/en/config-options/add-ons/network-plugins/

dns:
provider: coredns
priorityClassName: system-cluster-critical
network:
plugin: flannel
priorityClassName: system-node-critical

This is currently available to test in 2.5-head and the master PRs are slowly geting in. We should start testing with 2.5-head first

Here is what will be tested:

rancher v2.5-head (4cb2ebe259c073a2ea0611736731b98280ca6951)
configure priority_class_name via cluster yaml file through rancher UI (when provisioning cluster)
format is <service_name>_priority_class_name: <priority_class> i.e. metrics_server_priority_class_name: system-node-critical which is a parameter under options, except for metrics_server and ingress
check these resources by running kubectl get <type> -n <namespace> -o yaml | grep priorityClassName.
find the type and namespace by navigating to the system project of your cluster

what has been tested and is working correctly:

metrics_server
nginx_ingress_controller + default_http_backend
canal
kube-flannel
calico_kube_controllers

tested, but not configuring:

coredns_autoscaler  yaml persists, but resource <service_name>_priority_class_name does not exist
coredns -- yaml persists, but resource <service_name>_priority_class_name does not update

(fixes will be posted later by @PennyScissors for above issues)
still need to test:
kubedns, nodelocal, calico

part of this (dns) is still not working correctly and appears to be a UI issue.
when passing this through the yaml:
dns:
  options:
    coredns_priority_class_name: system-node-critical

the UI converts this to camelCase instead of the values used in the yaml

All other addons are able to have a <service_name>_priority_class_name set through the yaml.

Just wanted to clear up a few things:
This issue was filed for specific addons, but the configurable priority class name option has been to all of these so that we don't need to update them in the future.

calico
canal
flannel
coredns
kubedns
nodelocaldns
metrics-server
weave
ingress

The coredns issue is not in the backend. The backend work is complete, but to fix the issue itself this UI issue must be resolved first. At a high level, the problem is the UI is changing addon_name_priority_class_name to addonNamePriorityClassName and it breaks because the backend expects the former (underscore_case) for dns.options specifically.
All other addons can be tested as they are not affected, but DNS can only be tested using the API UI or RKE standalone until the UI issue is fixed.

now able to save DNS config type correctly tested on 2.5-head (03b8153) and master-head(007e298)
all of the addons from the above comment have been tested and work properly with <addon_name>_priority_class_name set in the yaml when provisioning the cluster.

documentation request here: https://github.com/rancher/docs/issues/3122


Fix "Add It" Button Functionality on Index Page

Issue Description
The "Add It" button on the index page is currently not functioning as expected. Clicking the button does not trigger the intended action.
Tasks:

Debug the issue to identify why the button is not working.
Verify that the button is properly linked to its event handler or functionality.
Ensure the required JavaScript, backend API, or route is correctly implemented and connected.
Test the button to confirm it performs the expected action after fixing the issue.

Additional Notes:

Provide a detailed explanation in the comments or PR description about what caused the issue and how it was resolved.
Test the fix across different browsers and devices to ensure consistency.

Images for reference:

i like to work on this issue could you assign this issue to me

@prajwal2431 ok I will assign you tomorrow


'+' at the second-last place in tab row should be removed.

Description of the new feature/enhancement
'+' button at the second-last place in tab row is quite redundant and can be removed. It just duplicates default profile, which can be also accessed from arrow-down menu. Whenever I go to open a new tab, I have particular profile in mind, and it tends to be non-default one (which, typically, would be already running in the window)

Triaged under deliverable #6685.


Downgrade flask to fix issue with google colab

Closes #152
Google colab has a conflict with Flask 2.0.0
Here is a link to a working Colab Notebook

Is there a way to do this without downgrading Flask?


Document admin CLI docker, add generate-mnemonic command

https://api3dao.atlassian.net/browse/AN-277
I am not sure where to put these changes - since this is a feature for v0.3 not v0.2.

These changes will integrate nicely with the current doc flow. I will make a small adjustment to the DockerImageVersions component to hide future features from v0.2 where they do not exist.


Pipe slot definitions through the render tree at construction to improve perf

Currently YuiTemplateRenderer sets up slot definitions in the update() call, and YuiRenderContext.resolveBinding has to evaluate them each frame. The slot definitions are static though, so they can be piped through when the render tree is constructed, removing the need for YuiTemplateRenderer and thus getting a performance gain whenever one is used.
Doing so will also obviate the need for fragments, as templates with no slots will no longer have a performance cost.

(fixed when I rewrote the render system)


[Feature] Let REPL file loading share environment with eval

Currently, file loading using dynamic-enter!. But it creates a new namespace.

Oops, I realize the problem is coming from I didn't load it to eval scope but program scope...

NOTE: 0.1.0 won't take care of this.


Proposal: Observable Streams

Node.js's built in streams have awkward semantics which do not fit well into a functional programming paradigm. Observable Streams have much better semantics and at least in theory should be able to accurately capture the behavior of Node's streams. Could have a nice symmetry with promises. If carefully implemented, should only have a small constant additional overhead compared to a naïve interpretation of the bindings.
Streams are used fairly heavily throughout the node APIs, so it's important that they have a sound foundation.

I working on a general implementation of observable streams for reason, however my underlying goal is to provide a performant isomorphic RX implementation and currently do not plan to support any type of interop to the proposed tc3p observable spec. You can checkout https://github.com/bordoley/rx-reason if you are interested.

Closing, as it's probably best done in a separate library. We've also become sensitive to bundle size in Repromise, e.g. https://travis-ci.org/aantron/repromise/jobs/583078509#L244


Contribute to hnpwa and todomvc

Contribute to hnpwa.com and todomvc.com maybe? Most of JS View library listed there. Making moon more well known by community.

Yes, I've been thinking about making an hnpwa, and I'm currently working on a PR for todomvc, it is already available, just not official.


update rpc starter version of dependencie to 5.3.2

Pull Request Test Coverage Report for Build 175

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 50.959%

Totals

Change from base Build 173:
-0.1%

Covered Lines:
930

Relevant Lines:
1825

💛  - Coveralls


a relative mouse problem

Hi, i was using vmulti on my remote desktop app. and when i used api <vmulti_update_relative_mouse> to move mouse position relatively(like x-axis move 10 pixels), the actual movement is greater than 10 pixels. is that a bug or that's the design? thanks.

Hi! I assume it's OS affects cursor movement. So, cursor movements are affected by mouse movement + system settings. Did you try to change mouse move speed in system settings? Does it affect your test?

@ihor-drachuk when I turn off enhanced pointer precision and set the mouse move speed to normal, it works well. thanks.


NPM update and (boring) HU lang

Description
I have created a very last (boring, sorry) HU lang update
Under development I have noticed some npm outdate, but I have modified very safe update only:

qrcode updated to 1.5.0: this solves 7 vulnerable (dependent) problem. Now npm audit fix does not display anyone.
bree updated to 7.1, because earlier version in (very) unsupported. This version works on server side (not in browser), I have checked.

Type of change
Please delete options that are not relevant.

NPM update (safe only, now all displayed by ncu)
Translation update

Checklist

[x] My code follows the style guidelines of this project
[x] I ran ESLint and other linters for modified files
[x] I have performed a self-review of my own code and test it
[ ] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[ ] My code needed automated testing. I have added them (this is optional task)

I recommend setting the dependencies in package.json back to ~, since louislam did like the risk of minor version having breaking changes.

I recommend setting the dependencies in package.json back to ~, since louislam didn't like the risk of minor version having breaking changes.

Understand, next time I will. Thx!


Remove unused zos.json file

Pull Request Test Coverage Report for Build 3014

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 2999:
0.0%

Covered Lines:
666

Relevant Lines:
666

💛  - Coveralls


From and To addresses with leading spaces

We found that two of our systems are unable to send emails to our service that uses smtpd as the smtp handler.  Tracking it down to the errors for 'invalid FROM parameter' and 'invalid TO parameter'.  Found that updating the regex to allow for a single space handles this and then trimmed the addresses as they were added.  I was hoping this could be evaluated and added so that we don't have to use a custom fork.  Had issues making a PR so adding the code changes here.  Thank you.
26:rcptToRE   = regexp.MustCompile(`[Tt][Oo]:\s?<(.+)>`)
27:mailFromRE = regexp.MustCompile(`[Ff][Rr][Oo][Mm]:\s?<(.*)>(\s(.*))?`) // Delivery Status Notifications are sent with "MAIL FROM:<>"
294:from = strings.TrimSpace(match[1])
334:to = append(to, strings.TrimSpace(match[1]))

Just wanted to point out that your two systems are not adhering to the specification.
You should probably be fixing it at that end.

I agree with @Ruxton that it is better to fix this in the sending code.
However, since you provided a patch and asked nicely, I've made the regex changes and added test coverage. Double spaces will still be rejected. Trimming the strings is unnecessary since the space is not part of the subpattern.

I figured it was an issue with the software since it's the only two out of many that have issues but unfortunately we don't have control over how they send out email since they are purchased software packages.  I appreciate the update.


Feature Request: automatically print expression on variable assignment (similar to pry)

pry (ruby REPL) has a feature where the resulting expression is printed during a variable assignment.
Here's a screengrab for clarity:

I find this useful as I'd often like to see each result when sending existing code (containing variable assignments) to ptpython from vim. Currently, It seems that one has to retype the assigned variable to view results, which is a bit slow.
Is this an existing feature? If not, might you consider a PR? I imagine it'd be preferred as an optionally configurable feature.

Interesting idea!
Unfortunately, right now, I won't have time for this. First finishing prompt_toolkit 2.0 and making sure everything (including ptptyhon) runs on 2.0. Then we can start again on new features.


Arizen on Ubuntu suggestion and bug.

Dangerus!
Creating a new wallet with the name of already existing wallet name overwrite .awd file and create a new entry in users.arizen file, after that it is not possible to log in to this wallet.
Bug:
Import wallet on Ubuntu from .awd/.uawd not working, timestamp on wallet.awd file it's changing, but size stay the same, (sqlite-3.8.2-1ubuntu2.1)
Suggestion:
Possibility of importing a wallet just after start program, right now is only possible after create a new wallet, then login into it and import.

Thank you for testing! We are working on it!

fixed in #118
@LukasBures check mac
@neduchal check linux

@Tigger1 test it please too

"dangerous" fixed for v1.0.1
"bug" - try it once again please in v1.0.1
"suggestion" - importing functionality was improved in v1.0.1


support for 3d scatter plot if both axes are numerical values

we are not adding additional plot support endemic to pandas. you can use matplotlib, seaborn, and bokeh to extend

FYI, only matplotlib has built-in support for 3d plots out of listed, but not aware of pandas in its API.

you can simply pass in the data like any other matplotlib function.
http://pandas.pydata.org/pandas-docs/stable/visualization.html#plotting-directly-with-matplotlib


PROV: Fix EC OSSL_FUNC_keymgmt_match() to work in the FIPS provider

In the FIPS provider, calling EC_GROUP_cmp() with NULL for the BN_CTX
argument is forbidden.  Since that's what ec_match() does, it simply
cannot work in the FIPS provider.  Therefore, we allocate a BN_CTX
with the library context asssociated with one of the input keys
(doesn't matter which) and use that.

But the cmp will pick it up later if it needs it..

Yup

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set.  Please review the updates and set the label manually.

Merged
16486f6332410d0d9e8f2606abb970d32b0572d3 PROV: Fix EC OSSL_FUNC_keymgmt_match() to work in the FIPS provider


migrate to Django 1.8

the pinned Django version is currently the RC of 1.7, with 1.8's release it is probably time to address that (if this takes longer than expected we should at least move to 1.7.7)
Right now I have a branch @ https://github.com/opencivicdata/python-opencivicdata-django/tree/django1.8

fixes the loaddivisions command to use the new management command argparse-based framework
migrates djorm_pgarray to the new django.contrib.postgres

Three outstanding questions on the ArrayField front:

what does this do with an existing DB?  do the fields still work (the migration had to be hand-edited)
do we query the pgarray field anywhere?
was adding blank=True the right thing to do on all ArrayFields?  It looks like pgarray didn't check for blank but django.contrib.postgres does

@paultag latest commit tries to use 1.8 but doesn't make these changes

Ach, duh, right, we had a release of Django

I'm implementing some filter adjustment things in imago that will query array fields, but I don't think the syntax is different between django.contrib.postgres and  djorm_pgarray. At least field__contains=['foo'] and field__contained_by=['foo'] are the same.
And here is some unsolicited input:
I used ArrayField in a project and setdefault=[] so that I could assume I could iterate over an array and not need to check for null. I don't think ArrayField allows blank=True, so setting an empty default seemed like the best bet.
There is also a  django.contrib.postgres UUIDField that could possibly replace uuidfield.UUIDField.

@paultag @boblannon
if we migrate off UUIDField and ArrayField I think we'd need to delete the migrations and start fresh, there's not a clean way to do this since the UUID is the PK.  would that be problematic for either one or is now the time to do so.
migration to django's internal JSONfield would be a separate issue, still looking into that but I'd actually probably suggest dropping the existing JSONfield for now and  then re-adding it when we have a working JSONB implementation.

Resetting the DB sounds OK; what's the underlying storage type of UUIDField
and the Django UUID stuff?
On Tue, May 19, 2015 at 12:31 PM, James Turk<EMAIL_ADDRESS>wrote:

@paultag https://github.com/paultag @boblannon
https://github.com/boblannon
if we migrate off UUIDField and ArrayField I think we'd need to delete the
migrations and start fresh, there's not a clean way to do this since the
UUID is the PK. would that be problematic for either one or is now the time
to do so.
migration to django's internal JSONfield would be a separate issue, still
looking into that but I'd actually probably suggest dropping the existing
JSONfield for now and then re-adding it when we have a working JSONB
implementation.
—
Reply to this email directly or view it on GitHub
https://github.com/opencivicdata/python-opencivicdata-django/issues/31#issuecomment-103580746
.

--
Paul Tagliamonte
Software Developer | Sunlight Foundation

old was a varchar(32) new is the postgres UUID type
https://github.com/opencivicdata/python-opencivicdata-django/tree/django1.8-nouuid

thoughts on how we want to handle the JSON type?

Sounds like we'll have to rebuild the DB; might as well do it once --
howabout we trash, go full UUID, drop jsonfield totally, and switch to
jsonb? We might need a backport but I'm sure that we can find one
On Tue, May 19, 2015 at 1:24 PM, James Turk<EMAIL_ADDRESS>wrote:

thoughts on how we want to handle the JSON type?
—
Reply to this email directly or view it on GitHub
https://github.com/opencivicdata/python-opencivicdata-django/issues/31#issuecomment-103601498
.

--
Paul Tagliamonte
Software Developer | Sunlight Foundation

ready for review in https://github.com/opencivicdata/python-opencivicdata-django/pull/42

one better, 0.7.x will go straight to 1.9


Discussion for a new capability for RegionDeviceCount

Problem description
Thinking about our API and UC I'm wondering if we shouldn't think about adding a new capability to send notification when the volume of devices go up or below a threshold.
We will offer a deviceCount-subscription endpoint (or other better name) to allow developer to subscribe for these events (number of devices above threshold, number of devices below a threshold), providing this threshold value and providing a sink url where notification must be send.
When situation occurs we trigger this event.
I saw UCs where depending on the volume of connected devices you want to propose/trigger to user some offers to get  always a good connectivity.
What do you think?
Possible evolution
Add a second yaml to manage deviceCount-subscription
Alternative solution

Additional context
Do we need to go through the API backlog WG?

@bigludo7 I think this is a great proposal. Adding subscription functionality can meet more scenario requirements. Some of the scenarios I can think of now, such as hosting large-scale conferences or events, organizers can use this API to monitor the number of devices in the activity area. When the number of devices approaches or falls below the set threshold, notifications can be sent to event organizers so that they can adjust the activity process or provide personnel guidance in a timely manner.

Do we need to go through the API backlog WG?
In my understanding, changing yaml should not require notifying WG, right？
@ShutingQing What do you think?

@bigludo7 👍 I think this is a really great proposal that i can see potential value. Developers will have this need to be notified in which area, users are increase or decrease in a fast speed.
If we add such a function, should we make it an independent API? RegionDeviceCount is applied for those who request for a specific area. A new API with device num  changes can applied for those who subscribe the service.

Thanks @ShutingQing and @chinaunicomyangfan for your positive feedback.
We need a separate yaml following CAMARA subscription ruling (here) so I guess we need to trigger a request to the API backlog WG.
Our plan could be:

Let's tackle first the API name change with the backlog team (@chinaunicomyangfan manages this)
once done we can trigger this new feature to the backlog team (I can make a PR in backlog for this and getting your support)
we propose to work on this API in our project

suggestion for a UC. A targeted advertisement on in some area (video screens). Less people - show cheaper ads, more people show more expensive ads.  Filters are useful in this scenario as well.

Agree with Gregory.
Another use case I come across is, for Uber/Lyft/Gaode Map/Bolt taxi APPs, they can call the API, combined with GPS, to provide more accurate prediction of people. For some users, they allow the authentification of GPS only when "allow GSP when using the APP only", under such scenario, they can have one more input for predicting potential users, and so as to arrange the drivers.
As well for some food delivery APPs, they can call the API and provide an option for restaurants a reference, where to choose the restaurant location.


Post selection animation is clunky

The spinner that takes over the window after an app is launched but before the app has a window doesn't look great and is jarring.

Has been removed entirely


[semantic-arc-opts] When performing load [copy] -> load_borrow on classes, do not ignore forwarding uses.

Otherwise in cases like the test case attached, we will not properly look
through the destroy.
In a subsequent commit, I am going to do some more fixes here by adding memory
use verification around begin_borrow. I have wanted to enable this for a minute,
but this seems like a good time to do so given that would have caught this.
rdar://58289320

EDIT! I am doing a smaller cherry-picking fix. I am still using this PR to not confuse the automation. See message below in conversation!

@swift-ci test

Build failed
Swift Test OS X Platform
Git Sha - eb0e96943779ca499166a906a613b17b291437ae

Thanks for tracking this down!

@swift-ci test os x platform

I started to play with this a bit locally, and I found a smaller simpler change to cherry-pick. I am going to first commit that.

I tried to cherry-pick this change and I found it to be a bit difficult since the LiveRange infrastructure doesn't exist on 5.2. I am about to push an even smaller fix that only uses things in that older code to make it slightly easier to cherry-pick. In a subsequent PR, I am going to re-add the LiveRange code.

@swift-ci test

@swift-ci test

@swift-ci test

@swift-ci test

@swift-ci test

@swift-ci test

@swift-ci test

Build failed
Swift Test OS X Platform
Git Sha - 23f36a059a15f47b2a622bf16cb89557a97aa18c

Java failure...

@swift-ci test os x platform


updated FILE type browse view and added "where" advanced BREAD option

updated FILE type browse view
added advanced option "where" for select_dropdown
sample usage
{   "relationship": {       "key": "id",       "label": "name",       "page_slug": "admin/users",       "where": [           "role_id",           2       ]   } }

Great improvement!


Add additional {{dictkey}} option

Currently there is a {{key}} and {{value}} options to allow admins to supply both the preference key title as well as the corresponding value (documented on the wiki here).  However, in working on a Printer manifest (https://help.apple.com/deployment/mdm/#/mdm9514788C) I've come across a new situation that doesn't work with the current implementation of the above.
Every printer in the Printer payload is configured as an array of dictionaries within a dictionary where the array of dictionaries key name is the printer name.  This then has a number of contained subkey preferences.  The desired profile result with one printer looks like this:
<key>UserPrinterList</key>
<dict>
	<key>HP LaserJet M402dn</key>
	<dict>
		<key>DeviceURI</key>
		<string>ippusb://HP%20LaserJet%20M402dn%20(BEBF27)._ipp._tcp.local./?uuid=504XXXX1-42A2-3Y36-3131-fc3fdZZZZbf27</string>
		<key>DisplayName</key>
		<string>HP LaserJet M402dn</string>
		<key>Location</key>
		<string>mymac</string>
		<key>Model</key>
		<string>HP LaserJet M402dn</string>
		<key>PPDURL</key>
		<string>file://localhost/System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Resources/Generic.ppd</string>
		<key>PrinterLocked</key>
		<false/>
	</dict>
</dict>

As a result, while the dictionary needs a user-supplied key name, it does not need a value as it is a dictionary.  See my attempted manifest XML below, which does not produce any options in the GUI:
<dict>
	<key>pfm_description</key>
	<string>The printers available to a user. Locked printers require an administrator password</string>
	<key>pfm_name</key>
	<string>UserPrinterList</string>
	<key>pfm_subkeys</key>
	<array>
		<dict>
			<key>pfm_description</key>
			<string>Exact printer (cups) name</string>
			<key>pfm_name</key>
			<string>{{key}}</string>
			<key>pfm_subkeys</key>
			<array>
				<dict>
					<key>pfm_name</key>
					<string>DeviceURI</string>
					<key>pfm_title</key>
					<string>DeviceURI</string>
					<key>pfm_type</key>
					<string>string</string>
				</dict>
				<dict>
					<key>pfm_name</key>
					<string>DisplayName</string>
					<key>pfm_title</key>
					<string>Display Name</string>
					<key>pfm_type</key>
					<string>string</string>
				</dict>
				<dict>
					<key>pfm_name</key>
					<string>Location</string>
					<key>pfm_title</key>
					<string>Location</string>
					<key>pfm_type</key>
					<string>string</string>
				</dict>
				<dict>
					<key>pfm_name</key>
					<string>Model</string>
					<key>pfm_title</key>
					<string>Model</string>
					<key>pfm_type</key>
					<string>string</string>
				</dict>
				<dict>
					<key>pfm_format</key>
					<string>^file\:\/\/(.*)$</string>
					<key>pfm_name</key>
					<string>PPDURL</string>
					<key>pfm_title</key>
					<string>PPD File Path</string>
					<key>pfm_type</key>
					<string>string</string>
				</dict>
				<dict>
					<key>pfm_name</key>
					<string>PrinterLocked</string>
					<key>pfm_title</key>
					<string>Lock</string>
					<key>pfm_type</key>
					<string>boolean</string>
				</dict>
			</array>
			<key>pfm_title</key>
			<string>PrinterListDictionary</string>
			<key>pfm_type</key>
			<string>dictionary</string>
		</dict>
	</array>
	<key>pfm_title</key>
	<string>Printer List</string>
	<key>pfm_type</key>
	<string>dictionary</string>
</dict>

When I replace {{key}} with {{value}} in the above XML, I achieve somewhat of a desired subkey preference results, however the dictionary key name is listed as {{value}} (and cannot be changed) and additional printers cannot be added.

Perhaps adding a {{dictkey}} placeholder would best address this scenario?
This would also require that in the dictionary interface the first column be a text field in order to supply the desired array of dictionaries key name.

I see I made an error in my original report and that with the pfm_name of {{key}} the pfm_type must be string.  However, this causes the dictionary to appear as a single item, rather than listing the subkeys.

In that case, not certain a separate {{dictkey}} placeholder is necessary, but just an extension of the existing implementation.


redirecting to blank screen while selecting or searching for some values

[x] location - https://tgbooks.vercel.app/purchase/purchase_order
procedure- select purchase module
- select purchase order
- click on create new purchase order
- select items and select the vendor from dropdown

location- https://tgbooks.vercel.app/purchase/bills/create
procedure- select purchase module
-select bills and click on new bills
-click on bulk update line items
-click on update customers
-select a customer from the drop-down

issue fixed !

issue still exist ! @zahidmuhammed

please recheck this issue @zahidmuhammed

location - bulk update under accountant module

issue fixed !

[ ]edit option in invoices templates under templates @saranya13-tech @zahidmuhammed

issue fixed !


Stock

Hi. I'm wondering whether it would make sense to also provide info about stock. And is such info, in principle, available?

Stock and other financial products is something that we would definitely consider implementing, but we would need to define a common data model at the core library first (/cc @raulmarcosl)
Do you have stocks on BBVA?


jinaai/jina-embeddings-v2-base-* not working

When using the worker with the image runpod/worker-infinity-embedding:stable-cuda12.1.0, we see this error:
The transformation of the model "JinaBertModel" to BetterTransformer failed

According to https://github.com/michaelfeil/infinity/issues/115#issuecomment-1967237474 we should be able to solve this by setting these env variables:

INFINITY_DISABLE_OPTIMUM: TRUE
INFINITY_DISABLE_COMPILE : TRUE

But this is not working, we still see an error:
2024-08-05T13:35:35.781469998Z {"requestId": "6a91cfa6-db93-4c04-83c2-3ae5e3a8a52f-e1", "message": "Started.", "level": "INFO"}
2024-08-05T13:35:36.794152731Z {"requestId": "6a91cfa6-db93-4c04-83c2-3ae5e3a8a52f-e1", "message": "Failed to return job results. | 400, message='Bad Request', url=URL('https://api.runpod.ai/v2/4nv0a16wv8ef1p/job-done/1az42tjeq0sk40/6a91cfa6-db93-4c04-83c2-3ae5e3a8a52f-e1?gpu=NVIDIA+RTX+A4500&isStream=false')", "level": "ERROR"}
2024-08-05T13:35:36.794207183Z {"requestId": "6a91cfa6-db93-4c04-83c2-3ae5e3a8a52f-e1", "message": "Finished.", "level": "INFO"}

Request
{
  "input": {
    "model": "jina-embeddings-v2-base-de",
    "input": "Hello World"
  }
}

Output
{
  "delayTime": 1125,
  "executionTime": 1049,
  "id": "8a3de0e1-6b43-41e4-a4af-5ad1473463a1-e1",
  "status": "COMPLETED"
}

So it looks like everything is completed, but there is no expected output (the embeddings).
OpenAI-compatible API
The behavior is the same when using the OpenAI-compatible API: It doesn't work, just provides the same output as above.

@michaelfeil do you maybe have another idea on how to get this sorted?

@TimPietrusky The output you posted are not really descriptive for the problem that occures.
The environment variables that are currently usable are not up to date. Here are all the functions that generate env variables in infinity. They are however just generating the defaults, I think there is nothing to do here.
https://github.com/michaelfeil/infinity/blob/main/libs/infinity_emb/infinity_emb/env.py

@michaelfeil thanks for your quick response.
Sorry for the unusable error messages, this is what we get in our UI. I hopefully get access to more in depth logging or find someone how can actually help out here.

infinity-emb[all,onnxruntime-gpu]==0.0.35

Which version do you recommend? Should we try the 0.0.53?

Yeah, maybe 0.0.53 fixes this? Can you try running infinity from this and 0.0.35 & see if it works?
Then try and run the image from runpod locally.
if both work, check the UI for additional messages, not the other way around. :)

@michaelfeil awesome, thank you! Will do.

@pandyamarut please let us know when you had time to update this 🙏 Then I can do the testing.

Thanks to @pandyamarut we have updated the version in the worker runpod/worker-infinity-text-embedding:0.0.1-cuda12.1.0, but it will still not produce the desired outcome when using the same request / env vars as before:
2024-08-15T10:07:37.654523432Z INFO     2024-08-15 10:07:37,650 datasets INFO: PyTorch version     config.py:59
2024-08-15T10:07:37.654545312Z          2.5.0.dev20240618+cu121 available.
2024-08-15T10:07:38.125792094Z INFO     2024-08-15 10:07:38,124 infinity_emb INFO:           select_model.py:57
2024-08-15T10:07:38.125805034Z          model=`jinaai/jina-embeddings-v2-base-de` selected,
2024-08-15T10:07:38.125806844Z          using engine=`torch` and device=`None`
2024-08-15T10:07:38.384517276Z INFO     2024-08-15 10:07:38,381                      SentenceTransformer.py:189
2024-08-15T10:07:38.384544466Z          sentence_transformers.SentenceTransformer
2024-08-15T10:07:38.384548407Z          INFO: Use pytorch device_name: cuda
2024-08-15T10:07:38.387090089Z INFO     2024-08-15 10:07:38,384                      SentenceTransformer.py:197
2024-08-15T10:07:38.387112810Z          sentence_transformers.SentenceTransformer
2024-08-15T10:07:38.387116550Z          INFO: Load pretrained SentenceTransformer:
2024-08-15T10:07:38.387119610Z          jinaai/jina-embeddings-v2-base-de
2024-08-15T10:07:44.040260125Z WARNING  2024-08-15 10:07:44,038 infinity_emb WARNING:        acceleration.py:35
2024-08-15T10:07:44.040299926Z          DEPRECATED `INFINITY_DISABLE_OPTIMUM` - setting
2024-08-15T10:07:44.040303436Z          optimizations via
2024-08-15T10:07:44.040305626Z          BetterTransformer,INFINITY_DISABLE_OPTIMUM is no
2024-08-15T10:07:44.040307746Z          longer supported, please use the CLI / ENV for that.
2024-08-15T10:07:44.041460234Z INFO     2024-08-15 10:07:44,040 infinity_emb INFO:   sentence_transformer.py:81
2024-08-15T10:07:44.041485354Z          Switching to half() precision (cuda: fp16).
2024-08-15T10:07:44.059055655Z --- Starting Serverless Worker |  Version 1.7.0 ---
2024-08-15T10:07:45.357533272Z {"requestId": "sync-9ba78228-2771-4781-a1b8-16aac1718974-e1", "message": "Started.", "level": "INFO"}
2024-08-15T10:07:45.360670254Z INFO     2024-08-15 10:07:45,358 infinity_emb INFO:         batch_handler.py:321
2024-08-15T10:07:45.360695094Z          creating batching engine
2024-08-15T10:07:45.363612029Z INFO     2024-08-15 10:07:45,360 infinity_emb INFO: ready   batch_handler.py:384
2024-08-15T10:07:45.363629230Z          to batch requests.
2024-08-15T10:07:46.110630606Z {"requestId": "sync-9ba78228-2771-4781-a1b8-16aac1718974-e1", "message": "Failed to return job results. | 400, message='Bad Request', url='https://api.runpod.ai/v2/wsdp02o8uipf7h/job-done/4vmqpadg317i92/sync-9ba78228-2771-4781-a1b8-16aac1718974-e1?gpu=NVIDIA+RTX+4000+Ada+Generation&isStream=false'", "level": "ERROR"}
2024-08-15T10:07:46.110678748Z {"requestId": "sync-9ba78228-2771-4781-a1b8-16aac1718974-e1", "message": "Finished.", "level": "INFO"}
2024-08-15T10:07:46.634466067Z {"requestId": "79345661-90dd-42ef-a3bc-5a2b82f70d66-e1", "message": "Started.", "level": "INFO"}
2024-08-15T10:07:46.735420699Z {"requestId": "79345661-90dd-42ef-a3bc-5a2b82f70d66-e1", "message": "Failed to return job results. | 400, message='Bad Request', url='https://api.runpod.ai/v2/wsdp02o8uipf7h/job-done/4vmqpadg317i92/79345661-90dd-42ef-a3bc-5a2b82f70d66-e1?gpu=NVIDIA+RTX+4000+Ada+Generation&isStream=false'", "level": "ERROR"}
2024-08-15T10:07:46.735438659Z {"requestId": "79345661-90dd-42ef-a3bc-5a2b82f70d66-e1", "message": "Finished.", "level": "INFO"}

I guess this also doesn't help with finding anything. What could be the next steps here to debug what is going on @pandyamarut? Or maybe you have another idea @michaelfeil?

Talked with @pandyamarut: We will try to debug what is going on here!

i also got the same problem in here seems like the error has relation with the size of the embed data that server try to send back to the client. because i tried to hit an endpoint with with different length of list that contain sentence, when i limit the length of the list its just work fine. i've been test it with intfloat/multilingual-e5-large and also nvidia/NV-Embed-v2 embedding model.
and here is the data that i use for testing https://www.kaggle.com/datasets/rtatman/questionanswer-dataset i use the question data only.


chore: release v0.2.10 (next v0.0.6)

#473  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @hi-ogawa and the rest of your teammates on  Graphite


Version 0.48.4 missing from npm

npm show react-native version returns 0.48.3  although version 0.48.4 is released

This version hasn't been released yet. Latest is 0.48.3 as I said in the other issue.
We are aware of an issue that prevents latest build from being published and are working on a fix.

Release is out now
Duplicate of #16022


Best Practices/Guidelines sections(s) or document

The current ED includes many recommendations for applications that might be better suited for either a Best Practices document or Guidelines sections(s) in this document.
We should consider these alternatives and move some of the current recommendations.

🙏


Fix subscription event. Add gift subs, raid, and command events.

Twitch Integration v2:
General code improvements, shorter methods
Use usernotice command for subscriptions
Support gifted subscriptions
Add channel raid event
Add command event (line starts with ! or ?)

Documentation PR: https://github.com/missionpinball/mpf-docs/pull/328

+1 for the test. Thanks!


Remove version: FineprintSoftware.pdfFactory.Pro version 9.10

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

Close with reason: Duplicate;


no Application Gateway option in the steps

I cannot find Application Gateway (which is available on the portal currently) option in the steps - the content hasn't been updated yet?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e0ac55b8-8be8-1f86-e3b1-3fe3f6c340c1
Version Independent ID: c26d3fc8-b62c-4895-3687-2fc7364c6383
Content: Quickstart - Create a virtual machine scale set in the Azure portal
Content Source: articles/virtual-machine-scale-sets/quick-create-portal.md
Service: virtual-machine-scale-sets
GitHub Login: @cynthn
Microsoft Alias: cynthn

Thanks for the feedback! We are currently investigating and will update you shortly.

@kazshi Thanks for bringing this to our attention. I have assigned the issue to the content author to evaluate and update the doc as appropriate.

@kazshi thanks for the feedback. The screenshots were updated today. I'll close this issue now. Please feel free to reopen it if this doesn't fix your issue.
#please-close


Added an opensource python calender project

Hi there, in this PR, i am added a calender project in python..
So please merge the PR,
Thanks,
VIVEK K J,
www.vivekkj.me

This Repo accept only .cpp file


Reconnect when token changes on successful connection

If the websocket connection succeeds, but the token is wrong, changing
the token in the UI did not succeed (because we are already connected
and nothing was triggering a reconnect attempt with the new config).

This makes sense. Looks good! Thanks @johnswanson


fix win32 event objects

quick merging this one. I discovered it when trying to use event objects to make abort_or_stop more reliable


Cheminf, ligand-based virtual screening: Acpc

@wjurkowski Feel free to reopen if you are still interested in adding these tools!


PDS java.io.IOException: Store of sourcecode.zip to s3 failed

Problem
Uploading files to a local object storage does not work. I tested it using the GoSec+PDS-Solution and a local object storage (SeaweedFS). The exceptions are:
 com.mercedesbenz.sechub.commons.model.SecHubRuntimeException: Was not able to upload job data because of IO problems.
	at com.mercedesbenz.sechub.pds.job.PDSFileUploadJobService.handleUploadAndProblems(PDSFileUploadJobService.java:116)
	at com.mercedesbenz.sechub.pds.job.PDSFileUploadJobService.upload(PDSFileUploadJobService.java:84)
	at com.mercedesbenz.sechub.pds.job.PDSFileUploadJobService$$FastClassBySpringCGLIB$$6e4418a6.invoke(<generated>)
…
Caused by: java.io.IOException: Store of sourcecode.zip to s3 failed
	at com.mercedesbenz.sechub.storage.s3.aws.AwsS3JobStorage.store(AwsS3JobStorage.java:66)
	at com.mercedesbenz.sechub.pds.job.PDSFileUploadJobService.startUpload(PDSFileUploadJobService.java:184)
	at com.mercedesbenz.sechub.pds.job.PDSFileUploadJobService.handleUploadAndProblems(PDSFileUploadJobService.java:95)
…
Caused by: com.amazonaws.ResetException: Failed to reset the input stream;  If the request involves an input stream, the maximum stream buffer size can be configured via request.getRequestClientOptions().setReadLimit(int)
…
Caused by: java.io.IOException: Resetting to invalid mark
	at java.base/java.io.BufferedInputStream.reset(BufferedInputStream.java:454)
	at com.amazonaws.internal.SdkBufferedInputStream.reset(SdkBufferedInputStream.java:106)
	at com.amazonaws.internal.SdkFilterInputStream.reset(SdkFilterInputStream.java:120)
	at com.amazonaws.event.ProgressInputStream.reset(ProgressInputStream.java:168)

Uploading files to PDS with PDS versions 0.28.0 and less works well. Uploading files to PDS with PDS versions 0.29.0, 0.30.0 and 0.31.0 does not work.
This might be related to #1471.
NOTE: This bug only affects source uploads. Binary uploads work well.

@Jeeppler
Please assign me to this issue. I'll examine it on my return.

@zigfridus I assigned you.

Hello @Jeeppler
As I understand you are working on the #1471 that could fix this issue #1501 now.  Thus I need your advice what should I do. Do I have to wait till you complete your work on #1471?
Thanks

@zigfridus I am not sure, as this is a bug #1471 might solve it or might not solve it. My advice for you would be to work on a different issue, as the issue #1471 is currently my highest priority. As part of this issue a number of other things need to be improved as well.
My recommendation for you would be to work on #1506. This issue is an important step towards creating a web UI (the epic is: #11). We generate the OpenAPI client for Java, but it is not used until now. We have no idea if the Java client library is easy to use or does work well, therefore it would be nice to have a test/demo app.

We fixed it with #1471.


CDAP-7442 Require Ambari 2.2+

Require Ambari 2.2+ to support empty-value-valid tag.

Reopening against correct branch


New D210 check for white spaces.

This is a proposal for checking whitespaces in docstrings.

[x] number
[x] error message

First of all, thank you so much for investing in this. It's greatly appreciated.
I'm currently on my phone, so I only skimmed the changes, but I have a few issues:

Why did you choose the D290 error code? Always use the next available one in the appropriate category.
The check name should be more specific. The entire D2xx category is about whitespace issues, so it's not enough to call it that.

You should also update the README file with the new error.

[ ] number

Why did you choose the D290 error code? Always use the next available one in the appropriate category.

I'm not sure about the number and error message. Do you have any proposal?

@Nurdok based on README, I've changed the error code to 210. Is it ok?

D210 is okay, but please also address my other comments.

Can you please provide the message?

@Nurdok thanks for your comments and time. :shipit:

I left one more comment in the actual code - but notice that the build is now failing on your branch. Make sure your tests pass.

@Nurdok please note that I've also added sudo: false to .travis.yml in order to use container based infrastructure (see http://docs.travis-ci.com/user/workers/container-based-infrastructure/).

@Nurdok sorry for so many iterations. Please let me know when you release it on PyPI so I can remove it from my package https://github.com/jirikuncar/invenio-kwalitee/blob/master/invenio_kwalitee/kwalitee.py#L82-L93. Thanks

@jirikuncar Merged. It will probably take a few weeks before I make another release. In any case, thank you so much for investing the time and effort into this. I really can't thank you enough!


DEGA Parachutes + Compat

Requester: Mirrow
Link: http://steamcommunity.com/sharedfiles/filedetails/?id=929073462
http://steamcommunity.com/sharedfiles/filedetails/?id=894224113
Reason:
None given

We won't be adding this one. We're going down the backpack on chest route instead.


fix: align conditionals in identityPostgres

Which problem does the PR fix?
closes #1651
In the identity deployment, there is an environment variable named IDENTITY_DATABASE_PASSWORD which is conditionally applied if identityPostgresql.enabled is true, or if an externalDatabase is configured. However, the referenced secret may not be created if identityPostgresql is enabled, because the secret is only created if an externalDatabase is configured.

What's in this PR?
the postgresql secret is now created in the case that identityPostgresql is enabled.

Checklist
Please make sure to follow our Contributing Guide.

Before opening the PR:

[x] In the repo's root dir, run make go.update-golden-only.
[x] There is no other open pull request for the same update/change.
[x] Tests for charts are added (if needed).
[ ] In-repo documentation are updated (if needed).

After opening the PR:

[x] Did you sign our CLA (Contributor License Agreement)? It will show once you open the PR.
[x] Did all checks/tests pass in the PR?

I learned that the postgresql helm chart creates this secret, so there is no need for identity to create the postgres secret itself. I confirmed with @Ben-Sheppard that this was a mistake and not some intended functional purpose.  I also found that Chart.yaml gets interpretted before any of the templates get read, which means the conditions inside the Chart.yaml  are not accounting for the z_compatibility_layer file.

As a result, I changed the implementation of this PR to remove the identity secret, and to include extra conditions in Chart.yaml.  This time, I made sure to check values.yaml to ensure that by default  the compatibility mode options are null by default, which is important because of how the comma operator works in helm chart conditionals.


[HUDI-6717] Fix downgrade handler for 0.14.0

Change Logs
Since the log block version (due to delete block change) has been upgraded in 0.14.0, the delete blocks can not be read in 0.13.0 or earlier.
Similarly the addition of record level index field in metadata table leads to column drop error on downgrade. The Jira aims to fix the downgrade handler to trigger compaction and delete metadata table if user wishes to downgrade from version six (0.14.0) to version 5 (0.13.0).
Impact
NA
Risk level (write none, low medium or high below)
low
Documentation Update
NA
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

@nsivabalan @codope Please review.

Could you reuse the same JIRA ticket already filed instead of creating a new one: HUDI-6621?


Tech Design : Menu Theming

Issue Description
Adding active state background color theming variable in terra-menu.
Issue Type

[ ] New Feature
[ ] Enhancement
[ ] Bug
[X] Other

Tech Design
      &:active {
        background-color: var(--terra-menu-item-active-color, #ebf6fd);
      }

Can you be a bit more specific on this tech design? Are you only adding a theming variable into the terra-menu component? Or are you also doing the theming itself?

@ChaseSinclair i think this is just meant to add the ability to theme one property, but that could be a good point to add.

@ChaseSinclair I am just adding the theming variable to the menu item

+1 on tech design.

+1

Not going forward with Menu changes in core due to unavailability of updated callouts. Hence closing the issue.


🚧Setup project Social Preview

I have done the items below . . .

[X] I have updated the title by replacing the '<title>' section.

Description
Do some research on setting up a project's Social Preview and set it up
Refer to image below:

Acceptance Criteria
This issue is finished when:

[ ] Social preview setup.

ToDo Items

[x] Draft pull request created and linked to this issue
[X] Priority label added to issue (low priority, medium priority, or high priority)
[X] Issue linked to the proper project
[X] Issue linked to proper milestone

Issue Dependencies
No response
Related Work
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

We need to create an image and set it in the GitHub settings.  No PR needed.


feat(tier4_auto_msgs_converter): remove unnecessary code

Signed-off-by: yutaka<EMAIL_ADDRESS>Related Links

Description
Since new drivable area is defined by left and right bound, I removed the drivable_area.

Remarks

Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[x] Code is properly formatted
[x] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[ ] Commits are properly organized and messages are according to the guideline
[ ] Code is properly formatted
[ ] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[ ] All open points are addressed and tracked via issues or tickets
[ ] Write release notes

CI Checks

Build and test for PR: Required to pass before the merge.
Check spelling: NOT required to pass before the merge. It is up to the reviewer(s). See here if you want to add some words to the spell check dictionary.

@yukkysaito I'll add a document about the new drivable area later with another PR. Is it okay with you?