[Question]: BAAI/bge-small-en-v1.5 Fails to Load for HuggingFaceEmbedding in LlamaIndex (Stuck Without Progress)

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
I am trying to use the BAAI/bge-small-en-v1.5 model as the embedding model with LlamaIndex's HuggingFaceEmbedding integration. The model loads successfully via Hugging Face when used independently, but when I integrate it into LlamaIndex, the process stalls indefinitely during the initialization or embedding generation phase.
Here is the code snipet I used:
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
Load FinBERT as the embedding model
embed_model = HuggingFaceEmbedding(
model_name="BAAI/bge-small-en-v1.5",
device="cuda" if torch.cuda.is_available() else "cpu"
)
from llama_index.core import PropertyGraphIndex
index = PropertyGraphIndex(
nodes=nodes,
kg_extractors=[kg_extractor],
property_graph_store=graph_store,
embed_model=embed_model,
show_progress=True,
)

Are you uploading a lot of data? It can be very slow especially without gpu

Are you uploading a lot of data? It can be very slow especially without gpu
I use the data from sample notebook docs/docs/examples/cookbooks/GraphRAG_v2.ipynb. I adjust the code to use HuggingFace llm and embedding models instead of Open AI
I mean this data from the sample notebook in this Llamaindex github repo. I dont think it take up to 10 minutes without any progress with these amount of data and my laptop is macbook M2 Pro. I'm running this notebook by cpu
import pandas as pd
from llama_index.core import Document

news = pd.read_csv(
"https://raw.githubusercontent.com/tomasonjo/blog-datasets/main/news_articles.csv"
)[:50]
news.head()

Can you give some minimum code to reproduce? It works fine for me tbh (also on an M2)
Also works in google colab 🤔

Here is the entire code I used to run on my macbook Vscode
-- coding: utf-8 --
"""GraphRAG_v2_with_Neo4j.ipynb
Automatically generated by Colab.
Original file is located at
https://colab.research.google.com/drive/1qfnVg6xW4cOniWjPQCn8Zkhg7sU5AoPo
GraphRAG Implementation with LlamaIndex - V2
GraphRAG (Graphs + Retrieval Augmented Generation) combines the strengths of Retrieval Augmented Generation (RAG) and Query-Focused Summarization (QFS) to effectively handle complex queries over large text datasets. While RAG excels in fetching precise information, it struggles with broader queries that require thematic understanding, a challenge that QFS addresses but cannot scale well. GraphRAG integrates these approaches to offer responsive and thorough querying capabilities across extensive, diverse text corpora.
This notebook provides guidance on constructing the GraphRAG pipeline using the LlamaIndex PropertyGraph abstractions using Neo4J.
This notebook updates the GraphRAG pipeline to v2. If you haven’t checked v1 yet, you can find it here. Following are the updates to the existing implementation:

Integrate with Neo4J Graph database.
Embedding based retrieval.

Installation
graspologic is used to use hierarchical_leiden for building communities.
"""
!pip install llama-index llama-index-graph-stores-neo4j graspologic numpy==1.24.4 scipy==1.12.0 future
"""## Load Data
We will use a sample news article dataset retrieved from Diffbot, which Tomaz has conveniently made available on GitHub for easy access.
The dataset contains 2,500 samples; for ease of experimentation, we will use 50 of these samples, which include the title and text of news articles.
"""
import pandas as pd
from llama_index.core import Document
news = pd.read_csv(
"https://raw.githubusercontent.com/tomasonjo/blog-datasets/main/news_articles.csv"
)[:50]
news.head()
"""Prepare documents as required by LlamaIndex"""
documents = [
Document(text=f"{row['title']}: {row['text']}")
for i, row in news.iterrows()
]
"""## Setup API Key and LLM"""
import os
os.environ["OPENAI_API_KEY"] = "sk-.."
from llama_index.llms.openai import OpenAI
llm = OpenAI(model="gpt-4")
!pip install llama-index
!pip install llama-index-llms-huggingface
!pip install llama-index-embeddings-huggingface
!pip install llama-index-embeddings-huggingface-api
from llama_index.llms.huggingface import HuggingFaceLLM
from transformers import AutoTokenizer
import torch
hf_token = ""
Initialize the Hugging Face LLM
llm = HuggingFaceLLM(
model_name="instruction-pretrain/finance-Llama3-8B",
model_kwargs={
"token": hf_token,
"torch_dtype": torch.bfloat16,
},
generate_kwargs={
"do_sample": True,
"temperature": 0.6,
"top_p": 0.9,
},
tokenizer_name="instruction-pretrain/finance-Llama3-8B",
tokenizer_kwargs={"token": hf_token},
stopping_ids=[
AutoTokenizer.from_pretrained("instruction-pretrain/finance-Llama3-8B", token=hf_token).eos_token_id
],
)
"""## GraphRAGExtractor
The GraphRAGExtractor class is designed to extract triples (subject-relation-object) from text and enrich them by adding descriptions for entities and relationships to their properties using an LLM.
This functionality is similar to that of the SimpleLLMPathExtractor, but includes additional enhancements to handle entity, relationship descriptions. For guidance on implementation, you may look at similar existing extractors.
Here's a breakdown of its functionality:
Key Components:

llm: The language model used for extraction.
extract_prompt: A prompt template used to guide the LLM in extracting information.
parse_fn: A function to parse the LLM's output into structured data.
max_paths_per_chunk: Limits the number of triples extracted per text chunk.
num_workers: For parallel processing of multiple text nodes.

Main Methods:

__call__: The entry point for processing a list of text nodes.
acall: An asynchronous version of call for improved performance.
_aextract: The core method that processes each individual node.

Extraction Process:
For each input node (chunk of text):

It sends the text to the LLM along with the extraction prompt.
The LLM's response is parsed to extract entities, relationships, descriptions for entities and relations.
Entities are converted into EntityNode objects. Entity description is stored in metadata
Relationships are converted into Relation objects. Relationship description is stored in metadata.
These are added to the node's metadata under KG_NODES_KEY and KG_RELATIONS_KEY.

NOTE: In the current implementation, we are using only relationship descriptions. In the next implementation, we will utilize entity descriptions during the retrieval stage.
"""
import asyncio
import nest_asyncio
nest_asyncio.apply()
from typing import Any, List, Callable, Optional, Union, Dict
from IPython.display import Markdown, display
from llama_index.core.async_utils import run_jobs
from llama_index.core.indices.property_graph.utils import (
default_parse_triplets_fn,
)
from llama_index.core.graph_stores.types import (
EntityNode,
KG_NODES_KEY,
KG_RELATIONS_KEY,
Relation,
)
from llama_index.core.llms.llm import LLM
from llama_index.core.prompts import PromptTemplate
from llama_index.core.prompts.default_prompts import (
DEFAULT_KG_TRIPLET_EXTRACT_PROMPT,
)
from llama_index.core.schema import TransformComponent, BaseNode
from llama_index.core.bridge.pydantic import BaseModel, Field
class GraphRAGExtractor(TransformComponent):
"""Extract triples from a graph.
Uses an LLM and a simple prompt + output parsing to extract paths (i.e. triples) and entity, relation descriptions from text.

Args:
    llm (LLM):
        The language model to use.
    extract_prompt (Union[str, PromptTemplate]):
        The prompt to use for extracting triples.
    parse_fn (callable):
        A function to parse the output of the language model.
    num_workers (int):
        The number of workers to use for parallel processing.
    max_paths_per_chunk (int):
        The maximum number of paths to extract per chunk.
"""

llm: LLM
extract_prompt: PromptTemplate
parse_fn: Callable
num_workers: int
max_paths_per_chunk: int

def __init__(
    self,
    llm: Optional[LLM] = None,
    extract_prompt: Optional[Union[str, PromptTemplate]] = None,
    parse_fn: Callable = default_parse_triplets_fn,
    max_paths_per_chunk: int = 10,
    num_workers: int = 4,
) -> None:
    """Init params."""
    from llama_index.core import Settings

    if isinstance(extract_prompt, str):
        extract_prompt = PromptTemplate(extract_prompt)

    super().__init__(
        llm=llm or Settings.llm,
        extract_prompt=extract_prompt or DEFAULT_KG_TRIPLET_EXTRACT_PROMPT,
        parse_fn=parse_fn,
        num_workers=num_workers,
        max_paths_per_chunk=max_paths_per_chunk,
    )

@classmethod
def class_name(cls) -> str:
    return "GraphExtractor"

def __call__(
    self, nodes: List[BaseNode], show_progress: bool = False, **kwargs: Any
) -> List[BaseNode]:
    """Extract triples from nodes."""
    return asyncio.run(
        self.acall(nodes, show_progress=show_progress, **kwargs)
    )

async def _aextract(self, node: BaseNode) -> BaseNode:
    """Extract triples from a node."""
    assert hasattr(node, "text")

    text = node.get_content(metadata_mode="llm")
    try:
        llm_response = await self.llm.apredict(
            self.extract_prompt,
            text=text,
            max_knowledge_triplets=self.max_paths_per_chunk,
        )
        entities, entities_relationship = self.parse_fn(llm_response)
    except ValueError:
        entities = []
        entities_relationship = []

    existing_nodes = node.metadata.pop(KG_NODES_KEY, [])
    existing_relations = node.metadata.pop(KG_RELATIONS_KEY, [])
    entity_metadata = node.metadata.copy()
    for entity, entity_type, description in entities:
        entity_metadata["entity_description"] = description
        entity_node = EntityNode(
            name=entity, label=entity_type, properties=entity_metadata
        )
        existing_nodes.append(entity_node)

    relation_metadata = node.metadata.copy()
    for triple in entities_relationship:
        subj, obj, rel, description = triple
        relation_metadata["relationship_description"] = description
        rel_node = Relation(
            label=rel,
            source_id=subj,
            target_id=obj,
            properties=relation_metadata,
        )

        existing_relations.append(rel_node)

    node.metadata[KG_NODES_KEY] = existing_nodes
    node.metadata[KG_RELATIONS_KEY] = existing_relations
    return node

async def acall(
    self, nodes: List[BaseNode], show_progress: bool = False, **kwargs: Any
) -> List[BaseNode]:
    """Extract triples from nodes async."""
    jobs = []
    for node in nodes:
        jobs.append(self._aextract(node))

    return await run_jobs(
        jobs,
        workers=self.num_workers,
        show_progress=show_progress,
        desc="Extracting paths from text",
    )

"""## GraphRAGStore
The GraphRAGStore class is an extension of the Neo4jPropertyGraphStoreclass, designed to implement GraphRAG pipeline. Here's a breakdown of its key components and functions:
The class uses community detection algorithms to group related nodes in the graph and then it generates summaries for each community using an LLM.
Key Methods:
build_communities():

Converts the internal graph representation to a NetworkX graph.

Applies the hierarchical Leiden algorithm for community detection.

Collects detailed information about each community.

Generates summaries for each community.

generate_community_summary(text):

Uses LLM to generate a summary of the relationships in a community.
The summary includes entity names and a synthesis of relationship descriptions.

_create_nx_graph():

Converts the internal graph representation to a NetworkX graph for community detection.

_collect_community_info(nx_graph, clusters):

Collects detailed information about each node based on its community.
Creates a string representation of each relationship within a community.

_summarize_communities(community_info):

Generates and stores summaries for each community using LLM.

get_community_summaries():

Returns the community summaries by building them if not already done.
"""

import re
import networkx as nx
from graspologic.partition import hierarchical_leiden
from collections import defaultdict
from llama_index.core.llms import ChatMessage
from llama_index.graph_stores.neo4j import Neo4jPropertyGraphStore
class GraphRAGStore(Neo4jPropertyGraphStore):
community_summary = {}
entity_info = None
max_cluster_size = 5
def __init__(self, username, password, url, llm):
    # Pass Neo4j connection parameters to the parent class
    super().__init__(username=username, password=password, url=url)
    # Store the LLM instance
    self.llm = llm

def generate_community_summary(self, text):
    """Generate summary for a given text using an LLM."""
    prompt = (
        "You are provided with a set of relationships from a knowledge graph, each represented as "
        "entity1->entity2->relation->relationship_description. Your task is to create a summary of these "
        "relationships. The summary should include the names of the entities involved and a concise synthesis "
        "of the relationship descriptions. The goal is to capture the most critical and relevant details that "
        "highlight the nature and significance of each relationship. Ensure that the summary is coherent and "
        "integrates the information in a way that emphasizes the key aspects of the relationships.\n\n"
        f"{text}"
    )
    # Use the LLM's complete method to generate the summary
    response = self.llm.complete(prompt)
    return response.text.strip()

def build_communities(self):
    """Builds communities from the graph and summarizes them."""
    nx_graph = self._create_nx_graph()
    community_hierarchical_clusters = hierarchical_leiden(
        nx_graph, max_cluster_size=self.max_cluster_size
    )
    self.entity_info, community_info = self._collect_community_info(
        nx_graph, community_hierarchical_clusters
    )
    self._summarize_communities(community_info)

def _create_nx_graph(self):
    """Converts internal graph representation to NetworkX graph."""
    nx_graph = nx.Graph()
    triplets = self.get_triplets()
    for entity1, relation, entity2 in triplets:
        nx_graph.add_node(entity1.name)
        nx_graph.add_node(entity2.name)
        nx_graph.add_edge(
            relation.source_id,
            relation.target_id,
            relationship=relation.label,
            description=relation.properties["relationship_description"],
        )
    return nx_graph

def _collect_community_info(self, nx_graph, clusters):
    """
    Collect information for each node based on their community,
    allowing entities to belong to multiple clusters.
    """
    entity_info = defaultdict(set)
    community_info = defaultdict(list)

    for item in clusters:
        node = item.node
        cluster_id = item.cluster

        # Update entity_info
        entity_info[node].add(cluster_id)

        for neighbor in nx_graph.neighbors(node):
            edge_data = nx_graph.get_edge_data(node, neighbor)
            if edge_data:
                detail = f"{node} -> {neighbor} -> {edge_data['relationship']} -> {edge_data['description']}"
                community_info[cluster_id].append(detail)

    # Convert sets to lists for easier serialization if needed
    entity_info = {k: list(v) for k, v in entity_info.items()}

    return dict(entity_info), dict(community_info)

def _summarize_communities(self, community_info):
    """Generate and store summaries for each community."""
    for community_id, details in community_info.items():
        details_text = (
            "\n".join(details) + "."
        )  # Ensure it ends with a period
        self.community_summary[
            community_id
        ] = self.generate_community_summary(details_text)

def get_community_summaries(self):
    """Returns the community summaries, building them if not already done."""
    if not self.community_summary:
        self.build_communities()
    return self.community_summary

"""## GraphRAGQueryEngine
The GraphRAGQueryEngine class is a custom query engine designed to process queries using the GraphRAG approach. It leverages the community summaries generated by the GraphRAGStore to answer user queries. Here's a breakdown of its functionality:
Main Components:
graph_store: An instance of GraphRAGStore, which contains the community summaries.
llm: A Language Model (LLM) used for generating and aggregating answers.
Key Methods:
custom_query(query_str: str)

This is the main entry point for processing a query. It retrieves community summaries, generates answers from each summary, and then aggregates these answers into a final response.

generate_answer_from_summary(community_summary, query):

Generates an answer for the query based on a single community summary.
Uses the LLM to interpret the community summary in the context of the query.

aggregate_answers(community_answers):

Combines individual answers from different communities into a coherent final response.
Uses the LLM to synthesize multiple perspectives into a single, concise answer.

Query Processing Flow:

Retrieve community summaries from the graph store.
For each community summary, generate a specific answer to the query.
Aggregate all community-specific answers into a final, coherent response.

Example usage:
query_engine = GraphRAGQueryEngine(graph_store=graph_store, llm=llm)

response = query_engine.query("query")

"""
from llama_index.core.query_engine import CustomQueryEngine
from llama_index.core.llms import LLM
from llama_index.core import PropertyGraphIndex
import re
class GraphRAGQueryEngine(CustomQueryEngine):
graph_store: GraphRAGStore
index: PropertyGraphIndex
llm: LLM
similarity_top_k: int = 20
def __init__(self, model, tokenizer):
    self.model = model
    self.tokenizer = tokenizer

def custom_query(self, query_str: str) -> str:
    """Process all community summaries to generate answers to a specific query."""

    entities = self.get_entities(query_str, self.similarity_top_k)

    community_ids = self.retrieve_entity_communities(
        self.graph_store.entity_info, entities
    )
    community_summaries = self.graph_store.get_community_summaries()
    community_answers = [
        self.generate_answer_from_summary(community_summary, query_str)
        for id, community_summary in community_summaries.items()
        if id in community_ids
    ]

    final_answer = self.aggregate_answers(community_answers)
    return final_answer

def get_entities(self, query_str, similarity_top_k):
    nodes_retrieved = self.index.as_retriever(
        similarity_top_k=similarity_top_k
    ).retrieve(query_str)

    enitites = set()
    pattern = (
        r"^(\w+(?:\s+\w+)*)\s*->\s*([a-zA-Z\s]+?)\s*->\s*(\w+(?:\s+\w+)*)$"
    )

    for node in nodes_retrieved:
        matches = re.findall(
            pattern, node.text, re.MULTILINE | re.IGNORECASE
        )

        for match in matches:
            subject = match[0]
            obj = match[2]
            enitites.add(subject)
            enitites.add(obj)

    return list(enitites)

def retrieve_entity_communities(self, entity_info, entities):
    """
    Retrieve cluster information for given entities, allowing for multiple clusters per entity.

    Args:
    entity_info (dict): Dictionary mapping entities to their cluster IDs (list).
    entities (list): List of entity names to retrieve information for.

    Returns:
    List of community or cluster IDs to which an entity belongs.
    """
    community_ids = []

    for entity in entities:
        if entity in entity_info:
            community_ids.extend(entity_info[entity])

    return list(set(community_ids))

def generate_answer_from_summary(self, community_summary, query):
    """Generate an answer from a community summary based on a given query using LLM."""
    prompt = (
        f"Given the community summary: {community_summary}, "
        f"how would you answer the following query? Query: {query}"
    )
    messages = [
        ChatMessage(role="system", content=prompt),
        ChatMessage(
            role="user",
            content="I need an answer based on the above information.",
        ),
    ]
    response = self.llm.chat(messages)
    return response.message.strip()  # No need for regex cleaning

def aggregate_answers(self, community_answers):
    """Aggregate individual community answers into a final, coherent response."""
    # intermediate_text = " ".join(community_answers)
    prompt = "Combine the following intermediate answers into a final, concise response."
    messages = [
        ChatMessage(role="system", content=prompt),
        ChatMessage(
            role="user",
            content=f"Intermediate answers: {community_answers}",
        ),
    ]
    final_response = self.llm.chat(messages)
    return final_response.message.strip()  # No need for regex cleaning

"""##  Build End to End GraphRAG Pipeline
Now that we have defined all the necessary components, let’s construct the GraphRAG pipeline:

Create nodes/chunks from the text.
Build a PropertyGraphIndex using GraphRAGExtractor and GraphRAGStore.
Construct communities and generate a summary for each community using the graph built above.
Create a GraphRAGQueryEngine and begin querying.

Create nodes/ chunks from the text.
"""
from llama_index.core.node_parser import SentenceSplitter
splitter = SentenceSplitter(
chunk_size=1024,
chunk_overlap=20,
)
nodes = splitter.get_nodes_from_documents(documents)
len(nodes)
"""### Build ProperGraphIndex using GraphRAGExtractor and GraphRAGStore"""
KG_TRIPLET_EXTRACT_TMPL = """
-Goal-
Given a text document, identify all entities and their entity types from the text and all relationships among the identified entities.
Given the text, extract up to {max_knowledge_triplets} entity-relation triplets.
-Steps-

Identify all entities. For each identified entity, extract the following information:

entity_name: Name of the entity, capitalized
entity_type: Type of the entity
entity_description: Comprehensive description of the entity's attributes and activities
Format each entity as ("entity"$$$$<entity_name>$$$$<entity_type>$$$$<entity_description>)

From the entities identified in step 1, identify all pairs of (source_entity, target_entity) that are clearly related to each other.
For each pair of related entities, extract the following information:

source_entity: name of the source entity, as identified in step 1
target_entity: name of the target entity, as identified in step 1
relation: relationship between source_entity and target_entity
relationship_description: explanation as to why you think the source entity and the target entity are related to each other

Format each relationship as ("relationship"$$$$<source_entity>$$$$<target_entity>$$$$$$$$<relationship_description>)

When finished, output.

-Real Data-
######################
text: {text}
######################
output:"""
entity_pattern = r'("entity"$$$$"(.+?)"$$$$"(.+?)"$$$$"(.+?)")'
relationship_pattern = r'("relationship"$$$$"(.+?)"$$$$"(.+?)"$$$$"(.+?)"$$$$"(.+?)")'
def parse_fn(response_str: str) -> Any:
entities = re.findall(entity_pattern, response_str)
relationships = re.findall(relationship_pattern, response_str)
return entities, relationships
kg_extractor = GraphRAGExtractor(
llm=llm,
extract_prompt=KG_TRIPLET_EXTRACT_TMPL,
max_paths_per_chunk=2,
parse_fn=parse_fn,
)
"""## Docker Setup And Neo4J setup
To launch Neo4j locally, first ensure you have docker installed. Then, you can launch the database with the following docker command.
docker run \
    -p 7474:7474 -p 7687:7687 \
    -v $PWD/data:/data -v $PWD/plugins:/plugins \
    --name neo4j-apoc \
    -e NEO4J_apoc_export_file_enabled=true \
    -e NEO4J_apoc_import_file_enabled=true \
    -e NEO4J_apoc_import_file_use__neo4j__config=true \
    -e NEO4JLABS_PLUGINS=\[\"apoc\"\] \
    neo4j:latest

From here, you can open the db at http://localhost:7474/. On this page, you will be asked to sign in. Use the default username/password of neo4j and neo4j.
Once you login for the first time, you will be asked to change the password.
"""
from llama_index.graph_stores.neo4j import Neo4jPropertyGraphStore
Note: used to be Neo4jPGStore
graph_store = GraphRAGStore(
username="neo4j", password="", url="bolt://localhost:7687", llm=llm
)
Test connection
print("Neo4j connection established:", graph_store)
"""### Custom Embed FinBERT Model"""
import torch
print("CUDA available:", torch.cuda.is_available())
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
Load FinBERT as the embedding model
embed_model = HuggingFaceEmbedding(
model_name="BAAI/bge-small-en-v1.5",
device="cuda" if torch.cuda.is_available() else "cpu"
)
from llama_index.core import PropertyGraphIndex
index = PropertyGraphIndex(
nodes=nodes,
kg_extractors=[kg_extractor],
property_graph_store=graph_store,
embed_model=embed_model,
show_progress=True,
)
index.property_graph_store.get_triplets()[10]
index.property_graph_store.get_triplets()[10][0].properties
index.property_graph_store.get_triplets()[10][1].properties
"""### Build communities
This will create communities and summary for each community.
"""
index.property_graph_store.build_communities()
"""### Create QueryEngine"""
query_engine = GraphRAGQueryEngine(
graph_store=index.property_graph_store,
llm=llm,
index=index,
similarity_top_k=10,
)
"""### Querying"""
response = query_engine.query(
"What are the main news discussed in the document?"
)
display(Markdown(f"{response.response}"))
response = query_engine.query("What are the main news in energy sector?")
display(Markdown(f"{response.response}"))

Do you need any further information from me ?


Global key is getting reused in navigator

This is happening when I pop the route and then push a new route which uses as same global key in the old route.
Here is a simple example:
import 'package:flutter/material.dart';

void main() {
  runApp(new MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(
      title: 'Flutter Demo',
      home: new MyHomePage(),
    );
  }
}

class MyHomePage extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return new GestureDetector(
      onTap:() {
        Navigator.push(context, new MaterialPageRoute(builder: (_) => new Company(0)));
      },
      child: new Text('go to company'));
  }
}

class Company extends StatelessWidget {
  static final _scaffoldKey = new GlobalKey<ScaffoldState>();
  final int companyId;
  Company(this.companyId);

  @override
  Widget build(BuildContext context) {
    return new Scaffold(
      key: _scaffoldKey,
      body: new Column(children: [
        new Text('Company $companyId'),
        new GestureDetector(
          onTap:() {
            Navigator.pop(context);
            Navigator.push(context,
              new MaterialPageRoute(builder: (_) => new Company(companyId+1)));
          },
          child: new Text('next company')),
      ]
    ));
  }
}

Clicks "go to company" and then clicks "next company", you will see
I/flutter ( 8026): The following assertion was thrown building Company():
I/flutter ( 8026): Multiple widgets used the same GlobalKey.
I/flutter ( 8026): The key [LabeledGlobalKey<ScaffoldState>#33415169] was used by multiple widgets. The parents of
I/flutter ( 8026): those widgets were different widgets that both had the following description:
I/flutter ( 8026):   Company()
I/flutter ( 8026): A GlobalKey can only be specified on one widget at a time in the widget tree.

It seems that the popped route did not remove the global key from the widget tree.

@Hixie would know.

When you pop a route, it animates away. While this is happening, the old route and the new route are both on the screen at the same time, and cannot share a global key.

Thanks @Hixie !
The issue is resolved if you make _scaffoldKey non-static. I was wondering is this a proper way to assign a global key to the StatelessWidget or State<T> for StatefulWidget?
class CompanyState extends State<Company> {
   final _scaffoldKey = new GlobalKey<ScaffoldState>();
   /// use _scaffoldKey to show snackbar...
}

Usually I just skip using the key and use the Builder pattern instead. See the second code snippet here:
https://docs.flutter.io/flutter/material/Scaffold/of.html

Thanks! Builder pattern looks good!


Start win standalone setup

Fixes #343.
Incomplete list of things that still need to be done:

[ ] Put things on PATH.
[ ] Figure out how to handle julialauncher -> julia.
[ ] Add uninstall option to standard Windows hooks.
[ ] Add to start menu.
[ ] Create install script.
[ ] Fix juliainstaller on Windows.

Closing this, https://github.com/JuliaLang/juliaup/pull/636 hopefully will address the root problem.


Log streaming service has been terminated happens a lot

What happened?
When using devspace dev, a lot of times the message Log streaming service has been terminated appears. Indeed, displaying of the logs in the terminal stops. Perhaps not a bug, but I was wondering what is happening here? And is there any way to fix it?
What did you expect to happen instead?
That logging of the project is not interrupted / terminated on the cli.
How can we reproduce the bug? (as minimally and precisely as possible)
This mainly happens when I have two devspace dev projects running simultaneously.
Local Environment:

DevSpace Version: 4.14.0
Operating System: linux
Deployment method: kubectl apply

Kubernetes Cluster:

Cloud Provider: other
Kubernetes Version:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:47:41Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

/kind bug

@hvt thanks for reporting this issue! The error message occurs if devspace somehow loses connection to the pods the logs are streamed from. Losing the connection could have several reasons:

The pod is restarting for some reason (you can verify this via kubectl get pods)
Some kubernetes api servers have a timeout for idle connections and will kill the connection if this timeout is reached
There is an issue with the network which causes the connection loss

Having multiple devspace dev running shouldn't cause any issues with log streaming usually, are these projects running in the same namespace or have mutual deployments?

The projects are running in the same namespace.
Mmm, I will see what kubectl get pods gives, around the time this is happening. Will get back once it appears again :].

Hey @FabianKramm, this indeed happens when the pod is getting terminated and restarted because of failing liveness or readiness checks (your first point) because hey, we're developing and stuff breaks ;], or when I have a really long lasting devspace dev session that probably triggers a timeout at the Kubernetes API (your second point).
Is there anything that can be done about this? I.e. find the new deployed pod? Or is there missing metadata since the pod is not explicitly started by devspace itself, but by Kubernetes itself?

@hvt I'll fix this in the next release and devspace will reopen the logs

Closing due to inactivity and the version used. If you are still facing this issue with our latest DevSpace version, please open a new issue.


Add type definitions

Would love to use this in my TS project.

I do not support TypeScript, it is not open-source friendly for the JavaScript community nor GitHub.

@niftylettuce I'd love to hear your reasons why. Not criticizing or disagreeing, just want to hear your reasons for academic curiosity

IMO TypeScript is horrible and the community doesn't need yet another CoffeeScript, the main reason being the latter.

Sorry, no.
TypeScript is a completely separate project that brings only objective benefits over JavaScript, and is growing rapidly. Pure JavaScript development has already been deprecated by a lot of people.
You can choose to not maintain your project out of laziness, but that excuse is simply not valid.

@sweepyoface this isn't my project so I can't speak for @niftylettuce , but your tone is not appropriate for a constructive discussion. This is a JavaScript project, the maintainers have zero obligation to support typescript. And for every typescript purist, theres plenty of people that think TS is bloated and not worth the effort.
Here's a thought: would you or your company consider paying for TS bindings?

@sweepyoface I'm a veteran in the community, and from one look at your GitHub profile you truly don't have any open source experience, unless you have aliases I don't know about.  Please do not submit issues to my projects about TypeScript again, thank you though.
And no, I'm not lazy.  I am co-founder and CTO of a company and I still find time to maintain hundreds of packages.


NuGet package for strongly typed DacFx

Please, create NuGet package for Strongly Typed Model!

Thanks Mihail for your interest in the project. I agree this is a good idea and we are working on getting this done. I don't have a firm date yet as we are still working on getting a signed version to include in the NuGet package.
Stay tuned.
Lonny

Thank you, Lonny!
I can wait. Let me know how will news!

Any update on this?

Hi Erik and Mihail. We do not have a firm ETA for resolving this issue at present. Nugetization support is on our backlog but we've been prioritizing some other work ahead of this. We will keep this thread updated if and when this changes.
Thanks,
Kevin

Hi, any progress on this?
It would also be really useful to have several of the SQLServer related assemblies that are typically used in VS extensions/tools as a dependency in nuget for easy access on build servers similar to how VisualStudioExtensibility is doing nuget packages.

Is it any help if we get it signed and package it up?
Are there any technical problems to doing it?
The main blocker I see (unless i'm missing something) is that the DacFx (and script dom) is redistributable only in msi form - is this right? Can it be changed?
If we had a nuget of this and the dependencies then it would make our life so much easier and I can spend some time on it.
ed

We hope to produce a nuget package once the dependencies (SQLDOM in particular) are not installed into the GAC by default. We're actively working on this but exact timing is TBD.
Note that DacFx is redistributable by copying the DLLs + License.txt file. The important part is to include the license file as that’s what makes it legal to redistribute. As of right now, ScriptDom (and SqlSysClrTypes) are GAC’d which is why we recommend they be installed via MSI. However we are actively working to get a version bumped non-GAC’d version available so that you could reliably build and use that version in applications. Otherwise, you risk an earlier version being in the GAC which could mess up your code.

Cool thanks for clarifying!  /slopes away to add a license.txt to a few packages :)

The nuget version of DacFx is out now - would it be possible to get an appveyor build setup which would publish to nuget.org or something?

Getting this built on Appveyor and pushed to Nuget now would be awesome!

Almost 3 years in, any chance this will happen soon?

I ended up just pulling on the project in my solution, but please, could you just state: this is not going to happen, or provide an ETA?

This TSqlTypedModel has been around for 4 years now and you are still distributing TSqlModel as a default?
Its easier to select metadata from the information schema or sys tables and hold that data in custom written classes than it is to use the TSqlModel.
It's an enormous disservice to much of the work you all have done for SSDT and SQL server by continuing to push that very-hard-to-use model. Please either include this into DacFx or a separate package (that is consumable by sql projects and can replace the SQL tt stuff).

Any update on this?


[$250] Tag - App crashes when saving 0 as custom tag name

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.83-0
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4631162
Issue reported by: Applause - Internal Team
Action Performed:

Go to staging.new.expensify.com
Go to workspace settings > Tags
Click Settings > Custom tag name
Enter 0 and save it

Expected Result:
App will not crash
Actual Result:
App crashes when saving 0 as custom tag name
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/cde0b3b5-8921-4c2f-b121-832e001b4c73
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01aa311cffaf18adc9
Upwork Job ID:<PHONE_NUMBER>377724186
Last Price Increase: 2024-06-24
Automatic offers:

ZhenjaHorbach | Reviewer | 102869898
bernhardoj | Contributor | 102869900

Issue OwnerCurrent Issue Owner: @danieldoglas

@slafortune  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

We think that this bug might be related to #wave-collect - Release 1

Proposal
Please re-state the problem that we are trying to solve in this issue.
App crash when trying to save tag name as 0.
What is the root cause of that problem?
When we save the tag name as 0, the BE responds with an invalid error. If it fails, we set an errors object to the policy tag list collection.
https://github.com/Expensify/App/blob/f0b635585aa32d32ba395f23d009218f3abfe29e/src/libs/actions/Policy/Tag.ts#L557-L563
So, it becomes:
{
errors: {...},
TagName: {...},
}

Then, it crashes in this line of code
https://github.com/Expensify/App/blob/f0b635585aa32d32ba395f23d009218f3abfe29e/src/pages/workspace/tags/WorkspaceTagsSettingsPage.tsx#L36
The code gets the values of the object and then extracts the tags property from them. TagName has the tags property, but errors doesn't have the tags property because it's not a tag.
What changes do you think we should make in order to solve the problem?

If we see the offline with feedback here,
https://github.com/Expensify/App/blob/f0b635585aa32d32ba395f23d009218f3abfe29e/src/pages/workspace/tags/WorkspaceTagsSettingsPage.tsx#L76-L78
we can see that the component expects the error property to be added inside the tag object, just like what we did with the pendingAction.
https://github.com/Expensify/App/blob/f0b635585aa32d32ba395f23d009218f3abfe29e/src/libs/actions/Policy/Tag.ts#L538
So, we need to update the failure data to
[newName]: null,
[oldName]: {
    ...oldPolicyTags,
    errors: ErrorUtils.getMicroSecondOnyxError('workspace.tags.genericFailureMessage'),
},

Last thing we need to do is to clear the error when pressing the offline with feedback close (X) button.

Proposal
Please re-state the problem that we are trying to solve in this issue.
Naming or renaming policy tag list setting with name as '0' crashes the app with failure.
What is the root cause of that problem?
Error in console.
src\pages\workspace\tags\WorkspaceTagsSettingsPage.tsx
Uncaught TypeError: Cannot convert undefined or null to object
    at Function.values (<anonymous>)
    at eval (WorkspaceTagsSettingsPage.tsx:35:1)
    at Array.flatMap (<anonymous>)
    at WorkspaceTagsSettingsPage (WorkspaceTagsSettingsPage.tsx:33:138)
    at renderWithHooks (react-dom.development.js:16175:18)
    at updateFunctionComponent (react-dom.development.js:20382:20)
    at beginWork (react-dom.development.js:22425:16)
    at HTMLUnknownElement.callCallback (react-dom.development.js:4161:14)
    at Object.invokeGuardedCallbackDev (react-dom.development.js:4210:16)
    at invokeGuardedCallback (react-dom.development.js:4274:31)

App calls backend API RenamePolicyTaglist with valid oldName property and newName property as '0'. Backend responds back with error.
{
    "code": 666,
    "jsonCode": 402,
    "type": "Expensify\\Error\\InvalidParameterError",
    "UUID": "4456F600-4114-4B54-A7BD-4C3431656E18",
    "message": "Invalid input parameter",
    "title": "",
    "data": [],
    "htmlMessage": "",
    "onyxData": [],
    "requestID": "893aeca77e751c99-AMS"
}

While processing the response after converting policyTags to flatMap, which expects tag names inside it (instead of generic error that comes back from API), the code fails to fetch tags.
var hasEnabledOptions = OptionsListUtils.hasEnabledOptions(Object.values(policyTags !== null && policyTags !== void 0 ? policyTags : {}).flatMap(function (_ref2) {
    var tags = _ref2.tags;
    return Object.values(tags);
  }));

Code expects property tags inside the response. This works fine in case of a successful response as there is tags property inside tagsListData . But in case of errors, though response code is HTTP 200, tags property is completely missing.
As the code always assumes the presence of tags property, this fails in case of errors.
What changes do you think we should make in order to solve the problem?
Assuming that we always get a successful response is incorrect. Frontend code must be able to handle error responses in a graceful way.
Suggest following approaches for this

Check that tags property is not undefined. If that is missing, fallback for error response and fetch and display appropriate error as a red error text under tagList setting name field, based on the error message returned from backend. Display options for error need UI/UX input.
Alternatively, accept only valid tag names from frontend using regex. Also as the field is pre-populated with value Tag, do not allow changing the pre-populated text, instead forcefully accept only appending to this value. Need product input on this.

What alternative solutions did you explore? (Optional)
Backend to return tags as empty property in API response and add error as a different property, rather than replacing tags property.

Interesting, checking this on our side before assigning - I'm not sure if we should allow 0 as a tag name.

We should allow 0 on our side. I'm working on a fix for that.

@danieldoglas I think we can fix the FE first to handle the errors properly, then fix the BE to allow 0, otherwise, the app will crash again every time there is an edit tag error. What do you think?

@danieldoglas / @ZhenjaHorbach : is someone working on this on FE? Or can we consider this proposal?

@ZhenjaHorbach : I agree that this. failureData for renamePolicyTag must be fixed. I belive that in addition to this there needs to be safe response handling done in frontend.

I agree we need to fix the front end to deal with this as well.

PR is ready
cc: @ZhenjaHorbach

Backend changes implemented, waiting to be merged/deployed

This is kind of done?
@slafortune not sure why it didn't create the comments, the app part it's already in production!

Thanks for the heads up @danieldoglas - so are we just in the 7 day holding period for regressions? Which would be 7/17

@ZhenjaHorbach - C+ Role -  has been paid $250 via UpWorks
@bernhardoj - Contributor Role - is due $250 via NewDot

$250 approved for @bernhardoj


Fresh install is failing due to GDAL after #287

Following the latest merge with the pinned xclim and xarray (#287), RAVEN fails to install. It seems gdal is not installing or failing:
After:

make clean
make install

I get the following errors:
Building wheel for gdal (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/ets/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-fwt5rejb/gdal/setup.py'"'"'; file='"'"'/tmp/pip-install-fwt5rejb/gdal/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-9fu1l896 --python-tag cp37
cwd: /tmp/pip-install-fwt5rejb/gdal/
Complete output (250 lines):
running bdist_wheel
[...]
ERROR: Failed building wheel for gdal
And also this after it tries to roll back the uninstall:
ERROR: Command errored out with exit status 1: /home/ets/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-fwt5rejb/gdal/setup.py'"'"'; file='"'"'/tmp/pip-install-fwt5rejb/gdal/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-wd4gi007/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.

@richardarsenault I'm nearly convinced that this is related to another issue I raised not long ago (#284). Can you try the following and let us know if it still isn't working?

make env_clean
make conda_env
make install
make start

GDAL from PyPI (and other GIS packages) won't build if you don't have the accompanying C libraries (at appropriate versions) installed, hence why the conda install approach is a much better option.

Tried it, and still fails at make install, same problem.

I am on branch bias_correction (4fbb8df026e7f61d881b37be9bce91736aca033f), I have a similar problem, gdal problem on make develop after a clean conda env create.
gdal in environment.yml and requirements.txt do not match, that might be the reason.
environment.yml: gdal~=2.4: https://github.com/Ouranosinc/raven/blob/4fbb8df026e7f61d881b37be9bce91736aca033f/environment.yml#L28
requirements.txt: gdal==3.0.4: https://github.com/Ouranosinc/raven/blob/4fbb8df026e7f61d881b37be9bce91736aca033f/requirements.txt#L45

Oh wait, this branch bias_correction is not merged to branch master yet.  On master gdal in both files are still in sync so not sure what happened.

I'm not sure why it's failing locally for you (seems to be working for me), The RtD build from master was failing, and I have a feeling it could be because I added xclim to the environment.yml dependencies. Will explore more today.

Sorry, just to be clear, I was able to reproduce the gdal problem on branch bias_correction, I did not  try to reproduce on master since I was working on bias_correction.

Yep, all good. I was responding to @richardarsenault. I'm trying to reproduce on master.

Just removed my Raven environment and ran all commands listed (plus conda activate raven after make conda_env) and all tests are passing. Master is currently still using the older GDAL (~=2.4). Still searching.

@richardarsenault I can't seem to reproduce this error. If GDAL is being properly installed via Anaconda, it shouldn't even try to build a wheel. Is it possible that the make conda_env command is silently failing before it tries to install the remaining libraries with pip?

I'll try and see what I can do, I'll do a few tests with what you have said above in mind. Thanks for the support!

OK I have tried multiple things, the latest is as follows:
1- git checkout master
2- git pull
3- make env_clean
4- make conda_env
5- conda activate raven
6- make install
And it fails with the GDAL problem. Anything else I should be looking at?

OK I solved it!
My problem is that (I think) I had a few environments active one over the other, as when I tried "conda deactivate" it cycled through my last envs and eventually hit the "raven" environment again, which was the first one in that chain... not sure how to explain it but anyways, from base conda env I repeated the process and everything worked.
Thanks a lot for the support and sorry for the troubles!

All good. Glad to see that it's working!


Bluesky Custom Handle Trouble: Deleted Account Issue

Steps to Reproduce
I previously had an account with the handle @vagasprajr.com.br, but I deleted it. Now, I'm trying to use the same domain for a new account's custom handle, but Bluesky keeps using the old TXT record settings.
Here's how to reproduce it:

Go to: https://bsky.app/settings
Click: Change Handle
Clickck: I have my own domain
Put your domain
Update the DNS TXT in the host provider
Click: Verify DNS Record

Attachments
No response
What platform(s) does this occur on?
Web (Desktop)
Device Info
No response
What version of the app are you using?
Build version: 1.93.0; Bundle info: 5f449e3 (prod); Bundle date: 24111700; Platform: web
Additional Information
No response

When did you delete the old account and when did you update the DNS record?
Looking at the screenshot, you added the record with a TTL of 14400 so it can take up to 4 hours for that to update, as with any DNS record. That's not an issue when adding a new record that didn't exist before, but it is an issue when you're just changing an existing record.
Try again in 4 hours and I'd assume it'll work then. For the future, if you plan to modify a DNS entry, you should reduce its TTL a while before you do that (so, in this case, 4 hours before the change) to make sure it propagates faster. This is not an issue with Bluesky, it's just how DNS caching works.

Also if you test on https://toolbox.googleapps.com/apps/dig/#TXT/ the proper value is returned from DNS.

I am also experiencing this issue, and since my hosting for my website is just for username purposes, i dont have access to the web panel to perform verification through a .well-known route.
the bsky.app backend is not updating its cached DNS value quick enough, or the new value is not allowed to replace the old value.

Same here, TXT record was removed 4 days ago (I removed it hoping this would trigger an update), but the system is still returning the old entry.
How long are you caching these values?

I am getting the exact same issue with a deleted account and a new account with the .TXT record for the original domain used on the deleted account.
I am curious how long the cache is as well. I think it is hung up on that side too.

Can confirm that this is still happening for me as well, with the TXT record seemingly being cached by Bsky indefinitely (at 13 days now).

Same problem. Any update? DNS caching is not the issue here.

Hi! same problem here, I had a DNS entry for an account I delete later, then I create a new account, delete the old TXT register, create a new one, try to set it up in bsky, but 24hours later BSky keep the old TXT reg. With an external TXT reg search tools I could confirm the DNS is propagated.
With other subdomains, bsky correctly get the new TXT rec, but not for the original changed.
Cheers!

My DNS had a TTL for the DNS of one day and I was able to create the user 24 hours later once it expired.

Even with the file: https://vagasprajr.com.br/.well-known/atproto-did I am not able to use my domain as handle.

I have exactly the same issue with some of my accounts that I use for dev. The TTL on the records was 300 seconds (5 minutes). The accounts were deleted from the PDS, they haven't existed for weeks. I can't use the domain on a new account anymore as it's stuck reporting a DID Mismatch on the old DID.

also having this issue

Same issue here.

Im having this issue, any workaround?

Update: just had a reply on the support ticket saying:

We've escalated your concern to our team, and they have manually refreshed your DID.

And indeed I was now able to change to my domain.

@tiferrei Whom did you contact and what did you write exactly? My ticket got a reply saying that my DNS setup is wrong. Thanks in advance.

I just emailed<EMAIL_ADDRESS>from the email address I have in my account, with the body:

I previously had an account with a custom domain, that i then deleted. I wanted to create a new account and use this same domain but on the new account. However even after changing the DNS challenge (already some days ago), the “change handle” page still claims to be getting the old DID. Could this cache be released on your side please?

Seemed to be fixed globally! I was able to change my domain handle

I’m still having an issue.

On Dec 10, 2024, at 7:58 AM, Calum H. @.***> wrote:
Seemed to be fixed globally! I was able to change my domain handle
—
Reply to this email directly, view it on GitHub https://github.com/bluesky-social/social-app/issues/6493#issuecomment-2531567562, or unsubscribe https://github.com/notifications/unsubscribe-auth/BI4RZCCMGA5TZAFGEZV5HR32E3QPBAVCNFSM6AAAAABR7RAQP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMZRGU3DONJWGI.
You are receiving this because you commented.

Workaround: Use this script to force bluesky to change your handle.
It seems the frontend has some caching, but when you change your handle via the API it works.
https://gist.github.com/IMB11/b215dfc92d827c03d05f4981308cfd12

Workaround: Use this script to force bluesky to change your handle.
It seems the frontend has some caching, but when you change your handle via the API it works.
https://gist.github.com/IMB11/b215dfc92d827c03d05f4981308cfd12

this worked for me, thanks !


How do I rotate scene node? [question]

Hello. I have a TopoShape, I display it on scene, and try to transform obtained SceneNode:
            Matrix4 transfrm = Matrix4.makeRotationAxis(new Vector3(0, 1, 1), (float) Math.PI/2);
            SceneNode tool0 = mRenderView.ShowShape(myTopoShape, Vector3.Blue);
            tool0.SetTransform(transfrm);
            tool0.RequstUpdate();

But my scene element is not affected. What am I missing?

For performance reason, you need to update the render control  explicitly.
            render.GetContext().UpdateWorld();             render.RequestDraw();

Graphics_Motion.cs also features such way for it:
Creating a shape and storing CylinderNode for later use:
            GrayMaterial = MeshPhongMaterial.Create("phong.color");
            GrayMaterial.SetUniform("diffuse", Uniform.Create(new Vector3(0.8f)));

            var cylinder = ShapeBuilder.MakeCylinder(GP.XOY(), 10, 60, 0);
            CylinderNode = BrepSceneNode.Create(cylinder, GrayMaterial, null);

and then when this cylinder is moved:
            CylinderNode.SetTransform(Matrix4.makeTranslation(0, 0, nCurrentHeight));
            CylinderNode.RequstUpdate();

            render.GetContext().UpdateWorld();
            render.RequestDraw();


Don't stop the music

Automatically play similar music at the end of a queue

@MarvinSchenkel please mark this PR as ready for review once you are ready for a final review.

I will. Still need some final tweaks + the UI bit. Will try to finalize it this week.


HashMap JoinSemiLattice/MeetSemiLattice instances look wrong

Maybe I'm missing something, but shouldn't the join/meet semilattice instances for HashMap k v be analogous to Map k v? That is:
instance (Eq k, Hashable k, JoinSemiLattice v) => JoinSemiLattice (HM.HashMap k v) where
  (\/) = HM.unionWith (\/)

(and similarly for Meet)

If nothing else the present instance for JoinSemiLattice is wrong since HM.union is not commutative.

They most likely are, and I'm :+1: for making them the similar to Map instances. Also HashSet to be similar to Set. And we need to add tests for all of them. At least as with Map.


38 book records not migrated

38 records that are not migrated nor in the  migration report.
Ex. https://cds.cern.ch/record/1215503/export/xm?ln=en
Ex. https://cds.cern.ch/record/339866/export/hm?ln=en
However they have the correct indicator.
Do you understand why they are not in the dump file? They can be added after production.

This ticket was created after the last migration on Sandbox.
For the migration on the Production site, those records were correctly migrated.


Wrong response code

On volumes.js, line 36 .create:
return this.client.post('/volumes', attributes, 202, 'volume', callback);

which causes an error, needs to be changed to
return this.client.post('/volumes', attributes, 201, 'volume', callback);

Whoops! Thanks @raaspazasu, I'll get this fixed and an updated release out tomorrow.


Support new per-CF-statistics log entry

Once https://github.com/speedb-io/speedb/pull/801 is merged, the log parser needs to be updated accordingly.
It should support both the existing display and the new one and automatically detect / handle either one.

plz add a picture of how it looks

How what looks?
The artefacts should remain unmodified. It's just a matter of correctly parsing both an existing log and a log that has a separate dump per cf.

comment was intended for https://github.com/speedb-io/log-parser/pull/58


Why does following a link in the output window toggle auto-scrolling

Issue Type: Bug
When my CMake build runs it writes output into the "Output" pane. When there is an error, the output pane allows me to Cmd-click the file name to open the file. This is so useful - especially the fact that it works for any output and takes you to the right line! Thanks so much.
The issue I'm reporting is a minor aggravation for me. I don't understand why following a link from the Output pane in the manor described above should automatically turn the auto scroll setting (represented by a padlock icon) off. I never want the former action to cause the latter and I can't imagine why anyone would. Perhaps there is a good reason but just filing this in case it's simply a bug that no one has noticed. Thanks again!
VS Code version: Code 1.56.1 (e713fe9b05fc24facbec8f34fb1017133858842b, 2021-05-06T10:08:24.325Z)
OS version: Darwin x64 20.4.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz (12 x 2200)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
15, 12, 11

Memory (System)
16.00GB (0.63GB free)

Process Argv
../RDFox-console --crash-reporter-id 2a71d04f-cf99-42e6-a8d6-edf3509d111d

Screen Reader
no

VM
0%

Extensions (13)

Extension
Author (truncated)
Version

azure-pipelines
ms-
1.183.0

jupyter
ms-
2021.6.832593372

remote-containers
ms-
0.177.2

remote-ssh
ms-
0.65.4

remote-ssh-edit
ms-
0.65.4

remote-wsl
ms-
0.56.2

vscode-remote-extensionpack
ms-
0.21.0

azure-account
ms-
0.9.8

cmake-tools
ms-
1.7.2

cpptools
ms-
1.3.1

cmake
twx
0.0.17

vscode-lldb
vad
1.6.3

vscodeintellicode
Vis
1.2.13

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
pythonvspyt700cf:30270857
pythonvspyt602:30300191
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyt639:30300192
pythontb:30283811
pythonvspyt551cf:30291415
vspre833cf:30267465
pythonptprofiler:30281270
vshan820:30294714
pythondataviewer:30285071
vscus158cf:30286554

This is because of the feature smartScroll in output view. This is enabled by default.
Smart scrolling allows you to lock scrolling automatically when you click in the output view and unlocks when you click in the last line.

You can disable it if wanted using the setting output.smartScroll.enabled


Change ArrayBase.ptr to NonNull type

Rust 1.25 introduces a NonNull type, which represents a pointer that must always be non-null. It would be nice to switch the ArrayBase.ptr field to type NonNull<S::Elem> to indicate that the pointer must always be non-null.
However, I'm not quite sure about the implications of NonNull<T> being "covariant over T". After reading this page in the nomicon, I think this is okay for Array, ArrayView, ArrayViewMut, and ArcArray, but I'm not sure that it's okay in general for arbitrary S: Data.

Sure, using NonNull should be good for us, don't see a problem with that. We need to remember to highlight the non-null requirement in ArrayView::from_shape_ptr. I'll have a think about if we already de facto require that pointer to be null (it certainly is non-null in all the safe ways of making an ArrayView).

NonNull was already listed over in #414, but I updated it to link to this issue as the specific one for that point.

(There is no “arbitrary S: Data” yet, because the Data trait is documented to be for our own purpose and we control all its implementations.)

There is no “arbitrary S: Data” yet

Specifically, I'm thinking about the raw pointer array views we discussed in #388, for which NonNull might not make sense. There may also be other Data types that we want to add in the future.


Drowning in warnings in chisel3

Running GCD in chiselTests gives a bunch of errors of the following forms.
%Warning-STMTDLY: /var/folders/ls/0xl3wjy949b2b9j36yn36v3c0000gn/T/GCDTester2434714804424790707/GCDTester1526173926997725650.v:31: Unsupported: Ignoring delay on this delayed statement.
%Warning-WIDTH: /var/folders/ls/0xl3wjy949b2b9j36yn36v3c0000gn/T/GCDTester2434714804424790707/GCDTester1526173926997725650.v:27: Operator EQ expects 32 bits on the RHS, but RHS's CONST '1'h0' generates 1 bits.

For problems in the chisel-tutorials, there can be dozens or more of these lines and they dwarf the actual output.
Question:
Are these errors important, if so, can they be fixed.
Otherwise, can we suppress them, new chiselers will be unnerved and overwhelmed with this output

They're benign.  Ultimately, the right thing to do is to emit Verilog that
doesn't generate warnings, but for now suppressing them should be OK.

%Warning-STMTDLY: /var/folders/ls/0xl3wjy949b2b9j36yn36v3c0000gn/T/GCDTester2434714804424790707/GCDTester1526173926997725650.v:31: Unsupported: Ignoring delay on this delayed statement.
%Warning-WIDTH: /var/folders/ls/0xl3wjy949b2b9j36yn36v3c0000gn/T/GCDTester2434714804424790707/GCDTester1526173926997725650.v:27: Operator EQ expects 32 bits on the RHS, but RHS's CONST '1'h0' generates 1 bits.
For problems in the chisel-tutorials, there can be dozens or more of these
lines and they dwarf the actual output.
Question:
Are these errors important, if so, can they be fixed.
Otherwise, can we suppress them, new chiselers will be unnerved and
overwhelmed with this output
—
Reply to this email directly or view it on GitHub
https://github.com/ucb-bar/chisel3/issues/115.

I believe @azidar and I have fixed these with recent FIRRTL PRs.


⚠ Bofrost.be has degraded performance

In c283657, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1587 ms

Resolved: Bofrost.be performance has improved in 9b353ad after 22 minutes.


'height' key must be a positive Integer

Error executing vim.schedule lua callback: /usr/share/nvim/runtime/lua/vim/lsp/util.lua:1537: 'height' key must be a p
ositive Integer
stack traceback:
        [C]: in function 'nvim_open_win'
        /usr/share/nvim/runtime/lua/vim/lsp/util.lua:1537: in function 'open_floating_preview'
        ...packer/opt/lsp_signature.nvim/lua/lsp_signature/init.lua:458: in function 'handler'
        /usr/share/nvim/runtime/lua/vim/lsp.lua:1173: in function ''
        vim/_editor.lua: in function <vim/_editor.lua:0>```

Got this erorr when trying to complete something on second line of any file  with lsp
quick replication of error if you wish :
open python file nvim test.py
write print() - should work normally on line one
write print() - again and it should proceed to give an error shown above.

i am having the same issue too

Failed to reproduce

I push a check for this, not sure if it will works for you

I push a check for this, not sure if it will works for you

Cheers, il check it out now :)

Update : Seems like it has fixed the issue, thanks for the quick fix :)


Sub state detail

Unclear if your data source(s) provide this precision, but it would be awesome to see sub state (region) date. Example: county level reporting for the state of California. See how Bing is doing it — I got it to work for the state of Florida https://www.bing.com/covid

we do have some county reported data, if you click on the "Stats" table and select "US Counties" you'll be able to select the U.S. State to filter, and see all the reported info we have. It's no historic, but pulled once daily.
The view for Florida on that site is similar to a link we've seen where the data probably came from: https://fdoh.maps.arcgis.com/apps/opsdashboard/index.html?fbclid=IwAR2lYjGL7yHakqtGP1clRws6BsO8mCLTRpRiQIxgbvZa7Mr3PfMu6OgTScE#/8d0de33f260d444c852a615dc7837c86
hoping that more of this data becomes available. what would you suggest?

we do offer county-level data now under https://virustrack.live/stats?filter=US_Counties and as always we're looking for more data, always.
Going to close this for now.


"unexpected EOF" error when using concatenated text on net_dns_record table

Describe the bug
Using a LEFT JOIN on the net_dns_record table (from the net plugin), while doing a csv-based lookup on a text file results in an "unexpcted EOF" error
Warning: executeQueries: query 1 of 1 failed: unexpected EOF
Steampipe version (steampipe -v)
steampipe version 0.15.2
To reproduce

Create a simple csv file in the relevant folder where the csv plugin reads the files, called "domains.csv" with the following contents

domain
steampipe.io

In this example, look for the dmarc record in DNS using the following SQL query

SELECT
    Q.domain,
    N.value
FROM
    domains Q
LEFT JOIN
    net_dns_record N
    ON N.domain = '_dmarc.' || Q.domain
    AND N.type = 'TXT'

Expected behavior
I am expecting the LEFT JOIN to respect the concatenation statement, and do a lookup of the TXT record in DNS of _dmarc.steampipe.io (as per the CSV file).
Additional context
In another example, I was able to pass a simple SELET 'steampipe.io' as domain query, which worked fine.  It would appear the combination of using net_dns_record and csv in the join with the concatenation is a problem.  A direct join without the concatenation works fine.

Hi @massyn, thanks for trying out the Net plugin!
I tried the above mentioned steps and was able to reproduce that error. Will take a look at this issue.
In the meantime can you try the below query and check if this works?
-- list domains from domains.csv
with domain_list as (
  select
    concat('_dmarc.', domain) as domain
  from
    domains
  order by
    domain
)
select
  domain,
  value
from
  net_dns_record
where
  domain in (
    select domain from domain_list
  )

The workaround script you've provided is working correctly.  I'll use this in the mean time while you're working on the bugfix.

The same bug exists in the net_http_request module too.

Hi, i tried using the net_http_request table and I got a simular error as mentioned above, but even with the workouround I got the exact same error
with service_url_list as (
  SELECT concat('http://url?name=', name) as service_url FROM kubernetes_deployment
)
select
  k.name,
  k.namespace,
  r.url,
  i -> 'field' ->> 'name' as "Name",
from
  kubernetes_deployment as k,
  net_http_request as r,
  jsonb_array_elements(r.response_body::jsonb -> 'items') as i
where
   r.url in (select service_url::text from service_url_list);

PS.: URLs are normalized because they're internal but the request works just fine
Also, the select concact query works like a charm, but not when using inside of the where clause
Is there any other way that i could get this working before the patch?

The main "workaround" is trying to trick the postgres planner into do the subquery first rather than magically collapsing your query to combine them together. This will prevent it attempting a full table scan of the steampipe table that requires a qual.
One trick that sometimes works (and is used above) is to put an order by in the first query, e.g.:
with service_url_list as (
  SELECT concat('http://url?name=', name) as service_url FROM kubernetes_deployment order by service_url
)

Any chance that helps?
You can also better understand the plans by using explain analyze select ..., it should show the order of execution and the quals it will expect.
Sorry we don't have a better answer for now, hoping the workaround nails it!

@e-gineer thanks for the advice, it really worked
I didn't really understood the workaround before your explanation as I am not a real SQL person (just getting my feet wet)
Thanks for the help


Changing path from a RefluxJS Store

Hi, In my application I use RefluxJS, and one of my Stores need to figure out the path and than change it. Reflux lets you to use mixins within Sotres just like React Classes, so I've tried to add the Navigation Mixin - all the methods there.
Later in one of the Stores function I run this.transitionTo("Foo"); but I get an error "Uncaught TypeError: Cannot read property 'router' of undefined".
Any Ideas how to work this?

this.transitionTo is probably accessing the router property through contexts (this.context.router) which do not exists in RefluxJS stores.

@srph Any idea how can I change the path from the store ?

I don't know anything about, but the Navigation mixin looks up things from context. Since this is a question and not an issue I'm closing, but feel free to post some more of your code and I can take a look.

I understand that using the Navigation mixin like this wont work because of the context as you said. Nevertheless I need help to figure out how one changes a path from a different javascript module... There must be a way after all : )
Here is an example of my Code:
// routes.js
//////////////////////////////////////////////////////////
var Router  = require('react-router');
var Route   = Router.Route;

var App       = require('./app')
var Comp    = require('./components/comp');

var routes = (
	<Route path='/' handler={App}>
		<Route name='Foo' path='/foo' handler={Comp}/>
	</Route>
);

module.exports = routes;

// main.js
//////////////////////////////////////////////////////////
var React   = require('react');
var routes  = require('./Routes');

var appElement = document.createElement("div");

Router.run(routes, Router.HistoryLocation, function(Root, state) {
	Actions.pathChange({path: state.path});
	React.render(<Root params={state.params} query={state.query} />, appElement);
});

// comp.js
//////////////////////////////////////////////////////////
var React          = require("react");
var Reflux         = require("reflux");
var Actions        = require("../actions/actions.js");
var SomeStore  = require("../stores/some-store.js");

var Comp = React.createClass({

    render: function() {
        return (

Hello World

        );
    }
});

module.exports = Comp;

// store.js
//////////////////////////////////////////////////////////
var SomeStore = Reflux.createStore({

	onSomeAction: function() {
		// CHANGE ROUTER PATH HERE TO /foo
	}

});

module.exports = SomeStore;

Any help will be appreciated !!!


Pseudoelements not searchable

When searching for signature numbers, the pseudoelements created in the scss don't appear in Chrome. You can find them in Firefox, but not Chrome. I'll look at this when I'm in the lab tomorrow.

Sounds like not searchable in Edge either

Usually we use pseudo-elements for content which is part of the output style, and is not actually significant or part of the content. For anything which is part of the actual content that people will realistically want to search for, we should output literal characters from the XSLT. So it's a question of identifying any pseudo-elements which really shouldn't be pseudo-elements.

This code is generated at line 243 of xhtml5_templates_tei_module.xsl. Could be replaced with physical text. But the phrase "Page beginning" would then become part of the searchable/indexable text. Thinking back, I believe the reason we did this was a) because sig nums are often missing or wrong, and if we drop in literal text, they'll appear to be part of the document whether or not they're really there, and b) they're all indexed in the TOC anyway.


ll.an.js not working in live.ppooll

this is essential for some patches where actname is requested after loading.
we need a strategy to know if we are in live or not.

its working now, but:
i changed the code as for outputting the path as soon as there is a named object "act".
this is not ideal, but i could not get there to retrieve the attributes of the object, the patcher-name.
this should be [actmaker *], only then we can be sure that it is the object the script is looking for.
(like it is now, there might be a subpatch that somebody gave the scripting_name "act".

today 11olsen posted the solution by chance;
.getboxattr("boxatoms")


Understanding the terrain munge

I was taking it out for a spin just now but there hasn't been much success. Maybe it's something I've misunderstood, but then again the error message I get leaves practically everything up for imagination.
Error munging RhenVar1.tmtrl: The request is not supported.
Essentially the terrain doesn't get munged at all. I pretty much copied the config file from the example but I'll stamp it here just in case I screwed it up.
Global:
   UseEnvironmentMapping: no

   UseZEStaticLighting: yes

   sRGBDiffuseMaps: yes

   BumpMappingType: Normal Mapping

   Rendertype: normal_ext

   DiffuseColor: [1.0, 1.0, 1.0]
   SpecularColor: [1.0, 1.0, 1.0]

Materials:
   Rhn1_far:
      DiffuseMap: Rhn1_far_diffuse
      NormalMap: Rhn1_far_normal
      GlossMap: Rhn1_far_specular
      AOMap: $null_ao
      HeightMap: $null_heightmap
      HeightScale: 0.05
      SpecularExponent: 128.0

   Rhn1_main_2:
      DiffuseMap: Rhn1_main_2_diffuse
      NormalMap: Rhn1_main_2_normal
      GlossMap: Rhn1_main_2_specular
      AOMap: $null_ao
      HeightMap: $null_heightmap
      HeightScale: 0.05
      SpecularExponent: 128.0

   Rhn1_main_2a:
      DiffuseMap: Rhn1_main_2a_diffuse
      NormalMap: Rhn1_main_2a_normal
      GlossMap: Rhn1_main_2a_specular
      AOMap: $null_ao
      HeightMap: $null_heightmap
      HeightScale: 0.05
      SpecularExponent: 128.0

   Rhn1_main_4:
      DiffuseMap: Rhn1_main_4_diffuse
      NormalMap: Rhn1_main_4_normal
      GlossMap: Rhn1_main_4_specular
      AOMap: $null_ao
      HeightMap: $null_heightmap
      HeightScale: 0.05
      SpecularExponent: 128.0

   Rhn1_main_5:
      DiffuseMap: Rhn1_main_5_diffuse
      NormalMap: Rhn1_main_5_normal
      GlossMap: Rhn1_main_5_specular
      AOMap: $null_ao
      HeightMap: $null_heightmap
      HeightScale: 0.05
      SpecularExponent: 128.0

   Rhn1_main_8:
      DiffuseMap: Rhn1_main_8_diffuse
      NormalMap: Rhn1_main_8_normal
      GlossMap: Rhn1_main_8_specular
      AOMap: $null_ao
      HeightMap: $null_heightmap
      HeightScale: 0.05
      SpecularExponent: 128.0

So there are a few things I need clarified as well. I noticed that gloss maps for terrain are now separate from the normal map in the case of normal_ext, what sort of format does it expect for it? Is BC4 sufficient for it?
Also I was curious about the whole environment mapping thing. It doesn't actually have anything to do with this, but do we have any way to actually create cube textures?
Furthermore, just so this isn't mere idle questioning there was a minor inconvenience I came across. If the terrain material file has any extra notes commented out with #, it causes an error and fails the munge.

First a thing I forgot to do when posting the release (there's always something...) was mention that the order of sp_texture_munge and material_munge need to be switched so that sp_texture_munge is called first. I've now updated the modified munge_world.bat accordingly.
So that is likely to be causing problems, you're config file looks fine so see if making that change and then munging fixes things.

So there are a few things I need clarified as well. I noticed that gloss maps for terrain are now separate from the normal map in the case of normal_ext, what sort of format does it expect for it? Is BC4 sufficient for it?

Yes it is. But when a texture is used for terrain it'll get assembled and packed with other textures into a texture array so that SP doesn't need to split up terrain geometry based on what textures it uses.
This is why the gloss map is provided separately, because it will actually end up being packed along side the normal map material_munge just does it for you.
Because of the processing done on terrain textures I actually recommend setting the Uncompressed value for textures that will only be used for terrain. Otherwise you'll be wasting you're CPUs time compressing them with sp_texture_munge, only for material_munge to decompress them so it can process them and then compress them itself.
You can find more info on this in the second half of the Intro in the docs.

Also I was curious about the whole environment mapping thing. It doesn't actually have anything to do with this, but do we have any way to actually create cube textures?

Yes and no. Nothing ideal at least. I think the game is supposed to have a console command for it but I've never had much luck with getting it to work. You can also use something like XSI to do it, which in theory should give great results but requires you to be able to setup your map as a scene in it.

Furthermore, just so this isn't mere idle questioning there was a minor inconvenience I came across. If the terrain material file has any extra notes commented out with #, it causes an error and fails the munge.

Are you sure? The library is used to load the .tmtrl files as every other config.

Hm. I tried with the new munge_world.bat but it still hands out the same error even with a clean munge.

Are you sure? The library is used to load the .tmtrl files as every other config.

Actually I missed a key word in that particular error message before. It seems to be annoyed by a tab, as it says
Error munging RhenVar1.tmtrl: yaml-cpp: error at line 7, column 1: illegal tab when looking for indentation
I had it set up like this.
   sRGBDiffuseMaps: yes

	# options: Parallax Occlusion Mapping, Parallax Offset Mapping, Normal Mapping
   BumpMappingType: "Parallax Occlusion Mapping"

When I had one note just like it further down, it was fine, oddly enough. I wonder why it caused an error here. Oh well, the solution is simple. Just avoid tabs.

I think I've fixed what was causing the cryptic error message in the latest release. It seamed to be related to improper handling of compressed textures in material_munge. They should now be handled correctly, however I still recommend adding Uncompressed: yes to your *.tex files if they're only used by the terrain materials.
Hopefully you can play around with it successfully now.


The find method seems to be case-sensitive when using postgres

Waterline version: 0.11.0
sails-postgresql version: 0.11.3
Node version: 7.6.0
NPM version: 4.1.2
Operating system: Ubuntu 16.04

After experiencing some issues I just tried to write a very simple test case:
const g1 = {name: 'Test'};
await Group.create(g1);
const res1 = await Group.findOneByName('test');
should.exist(res1);

And the result is a failure (res1 is undefined).
According to the docs all the queries should be case-insensitive, so what's going on here? Is there a workaround?

Still waiting, thanks.


Characteristic setValue - Node.js 8.0.0

Hiya!
After upgrading to NodeJS 8.0.0 the characteristic.setValue function seems to be broken.
Whenever I set values of the characteristic it looks like the value is not being updated in any Homekit app. The first one or two calls do work though.
Downgrading to Node.js 7.xx or even lower seems to fix the problem.
I am using nfarina/homebridge to test this.
Thank you in advance.

I can confirm this issue is resolved when updating to the latest version.
https://github.com/nodejs/node/issues/13391


Long running SSoT job status never changes to complete

Environment

Nautobot version (Docker tag too if applicable): 2.3.2
Python version: 3.11
Database platform, version: PG15
Middleware(s): N/A

Steps to Reproduce

Create a SSoT job that runs for longer than 10 minutes
You'll see "Sync Complete" in the logs once the SSoT part has been completed however the job will stay in the "running" state and you'll notice many UserWarning: Field "model_flags" in <model> has conflict with protected namespace "model_". messages from the Celery Worker container multiple times in what appears like a loop.

Expected Behavior
The job runs, completes and the status updates to 'Completed'

Observed Behavior
You'll see "Sync Complete" in the logs once the SSoT part has been completed however the job will stay in the "running" state and you'll notice many UserWarning: Field "model_flags" in <model> has conflict with protected namespace "model_". messages from the celery-worker container which repeats multiple times.
After a long while you'll see that the job results finally update to Failed and job results will show something like:
{
    "exc_message": [
        "Worker exited prematurely: signal 9 (SIGKILL) Job: 2."
    ],
    "exc_module": "billiard.exceptions",
    "exc_type": "WorkerLostError"
}

This doesn't occur on shorter jobs, say 1 or 2 mins.
I upgraded from 2.2.7 to 2.3.2 and also upgraded SSOT from 2.7.0 to 3.1.0 and have since noticed this issue.

FWIW, I've seen the logs on celery-worker (those saying UserWarning: Field "model_flags" in <model> has conflict with protected namespace "model_".) just keep looping suggesting something is restarting all the time, until the "time_limit" (hard limit) is reached.
I ended up seeing this in the GUI
{
    "exc_message": [
        6000.0
    ],
    "exc_module": "billiard.exceptions",
    "exc_type": "TimeLimitExceeded"
}

and this in the celery-worker container:
[2024-09-09 14:38:41,046: ERROR/MainProcess] Task handler raised error: TimeLimitExceeded(6000.0)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/billiard/pool.py", line 683, in on_hard_timeout
    raise TimeLimitExceeded(job._timeout)
billiard.einfo.ExceptionWithTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/billiard/pool.py", line 683, in on_hard_timeout
    raise TimeLimitExceeded(job._timeout)
billiard.exceptions.TimeLimitExceeded: TimeLimitExceeded(6000.0,)
"""
[2024-09-09 14:38:41,051: ERROR/MainProcess] Hard time limit (6000.0s) exceeded for nautobot.extras.jobs.run_job[37dcf37d-d8b1-4333-abc9-37c9f8f10c3a]
[2024-09-09 14:38:41,220: ERROR/MainProcess] Process 'ForkPoolWorker-8' pid:78 exited with 'signal 9 (SIGKILL)'

The actual SSoT Sync completed in 17 mins:
2024-09-09 12:58:41.699858	Running job
<stuff>
2024-09-09 13:16:18.723874	Sync complete
2024-09-09 13:16:18.728409	Sync Time: 0:17:25.959060

I think this is a match for the issue we were diagnosing over Slack. Basically:

The run() method for a Job is wrapped in a web_request_context context manager. After run() returns ("Sync complete"), the context manager loops over all ObjectChange records created during run()

For each such ObjectChange, it calls enqueue_job_hooks(object_change)

enqueue_job_hooks() calls (since #5825) get_jobs(reload=True), which reloads every Job registered to the system.

So if you have a large number of ObjectChange records, a large number of registered jobs, and/or Jobs which take a long time to re-import, this can quickly compound into a tremendous amount of wasted processing time.
Fortunately this should be pretty easy to fix - we should just need to move that get_jobs(reload=True) out of enqueue_job_hooks() and just call it once directly from web_request_context() so that it runs only once per Job instead of once per ObjectChange per Job.


No attribute error: triton.language.constexpr in conda env

How To Reproduce

Dockerfile

FROM continuumio/miniconda:latest

WORKDIR /root
RUN conda create -y -n triton python=3.8
RUN bash -c ". activate triton && conda install pytorch cudatoolkit=11.3 -c pytorch && pip install --upgrade triton"

The output of conda list

docker run --mount type=bind,src=$(pwd),dst=/root/  --rm -it triton:latest bash -c ". activate triton && conda list"
# packages in environment at /opt/conda/envs/triton:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
_openmp_mutex             5.1                       1_gnu
blas                      1.0                         mkl
ca-certificates           2022.4.26            h06a4308_0
certifi                   2<IP_ADDRESS>      py38h06a4308_0
cudatoolkit               11.3.1               h2bc3f7f_2
filelock                  3.7.1                    pypi_0    pypi
intel-openmp              2022.0.1          h06a4308_3633
ld_impl_linux-64          2.38                 h1181459_1
libffi                    3.3                  he6710b0_2
libgcc-ng                 11.2.0               h1234567_1
libgomp                   11.2.0               h1234567_1
libstdcxx-ng              11.2.0               h1234567_1
libuv                     1.40.0               h7b6447c_0
mkl                       2022.0.1           h06a4308_117
ncurses                   6.3                  h7f8727e_2
openssl                   1.1.1o               h7f8727e_0
pip                       21.2.4           py38h06a4308_0
python                    3.8.13               h12debd9_0
pytorch                   1.11.0          py3.8_cuda11.3_cudnn8.2.0_0    pytorch
pytorch-mutex             1.0                        cuda    pytorch
readline                  8.1.2                h7f8727e_1
setuptools                61.2.0           py38h06a4308_0
sqlite                    3.38.3               hc218d9a_0
tk                        8.6.12               h1ccaba5_0
triton                    1.1.1                    pypi_0    pypi
typing_extensions         4.1.1              pyh06a4308_0
wheel                     0.37.1             pyhd3eb1b0_0
xz                        5.2.5                h7f8727e_1
zlib                      1.2.12               h7f8727e_2

Run the following python source.

import torch
import triton
import triton.language as tl

@torch.jit.script
def naive_softmax(x):
    # read  MN elements ; write M  elements
    x_max = x.max(dim=1)[0]
    # read MN + M elements ; write MN elements
    z = x - x_max[:, None]
    # read  MN elements ; write MN elements
    numerator = torch.exp(z)
    # read  MN elements ; write M  elements
    denominator = numerator.sum(dim=1)
    # read MN + M elements ; write MN elements
    ret = numerator / denominator[:, None]
    # in total: read 5MN + 2M elements ; wrote 3MN + 2M elements
    return ret

@triton.jit
def softmax_kernel(
    output_ptr, input_ptr, input_row_stride, output_row_stride, n_cols,
    BLOCK_SIZE: tl.constexpr
):
    # The rows of the softmax are independent, so we parallelize across those
    row_idx = tl.program_id(0)
    # The stride represents how much we need to increase the pointer to advance 1 row
    row_start_ptr = input_ptr + row_idx * input_row_stride
    # The block size is the next power of two greater than n_cols, so we can fit each
    # row in a single block
    col_offsets = tl.arange(0, BLOCK_SIZE)
    input_ptrs = row_start_ptr + col_offsets
    # Load the row into SRAM, using a mask since BLOCK_SIZE may be > than n_cols
    row = tl.load(input_ptrs, mask=col_offsets < n_cols, other=-float("inf"))
    # Substract maximum for numerical stability
    row_minus_max = row - tl.max(row, axis=0)
    # Note that exponentials in Triton are fast but approximate (i.e., think __expf in CUDA)
    numerator = tl.exp(row_minus_max)
    denominator = tl.sum(numerator, axis=0)
    softmax_output = numerator / denominator
    # Write back output to DRAM
    output_row_start_ptr = output_ptr + row_idx * output_row_stride
    output_ptrs = output_row_start_ptr + col_offsets
    tl.store(output_ptrs, softmax_output, mask=col_offsets < n_cols)

def last_dim_softmax(x):
    n_rows, n_cols = x.shape
    # The block size is the smallest power of two greater than the number of columns in `x`
    BLOCK_SIZE = triton.next_power_of_2(n_cols)
    # Another trick we can use is to ask the compiler to use more threads per row by
    # increasing the number of warps (`num_warps`) over which each row is distributed.
    # You will see in the next tutorial how to auto-tune this value in a more natural
    # way so you don’t have to come up with manual heuristics yourself.
    num_warps = 4
    if BLOCK_SIZE >= 2048:
        num_warps = 8
    if BLOCK_SIZE >= 4096:
        num_warps = 16
    # Allocate output
    y = torch.empty_like(x)
    # Enqueue kernel. The 1D launch grid is simple: we have one kernel instance per row o
    # f the input matrix
    softmax_kernel[(n_rows,)](
        y,
        x,
        x.stride(0),
        y.stride(0),
        n_cols,
        num_warps=num_warps,
        BLOCK_SIZE=BLOCK_SIZE,
    )
    return y

torch.manual_seed(0)
pp=last_dim_softmax(torch.randn(16, 781, device="cuda:0"))

Error messages

/opt/conda/envs/triton/lib/python3.8/site-packages/torch/_masked/__init__.py:223: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at  /opt/conda/conda-bld/pytorch_1646755903507/work/torch/csrc/utils/tensor_numpy.cpp:68.)
  example_input = torch.tensor([[-3, -2, -1], [0, 1, 2]])
Traceback (most recent call last):
  File "run.py", line 42, in <module>
    BLOCK_SIZE: tl.constexpr
AttributeError: module 'triton.language' has no attribute 'constexpr'

Expected Behavior

Reproducing tutorial softmax results.

Hi @atsushi3110 did you manage to resolve this? I'm having the same issue so it would be great if you could share your solution :slightly_smiling_face:


ja: Make docs/tutorials/stateless-application/expose-external-ip-address.md follow v1.18 of the original text

This is a Feature Request

What would you like to be added
Update docs/tutorials/stateless-application/expose-external-ip-address.md to follow v1.18 of the original (English) text.
Why is this needed
content/ja/docs/tutorials/stateless-application/expose-external-ip-address.md is outdated.
Comments
/language ja
/good-first-issue
File to update
https://github.com/kubernetes/website/blob/dev-1.18-ja.1/content/ja/docs/tutorials/stateless-application/expose-external-ip-address.md
Original
https://github.com/kubernetes/website/blob/fb6364d/content/en/docs/tutorials/stateless-application/expose-external-ip-address.md
diff between translated and v1.18
https://gist.github.com/734f5c99caf5623ae524f93e3a73dda9
Note
Currently, we use dev-1.18-ja.1 branch. Please open a PR targeting the branch.
ref
https://kubernetes.io/ja/docs/contribute/localization/
If you have a question, feel free to ask us at slack.k8s.io #kubernetes-docs-ja channel.

/milestone 1.18-ja.1

/assign

/close

あれcloseできないのか
@inductor @kakts mergeしていただいたので、クローズお願いします

/close


Minor issue: Internal Server Error in registerInfra when description is null

Context:
We are trying to automate the installation of Chaos Infra in different namespaces using Kyverno. When playing around with the API to figure out how the registration flow works, I ran into this issue.
What happened:
When the description field in registerInfra call is empty as the docs say it's not required, the request fails with 500 Internal Server Error.
Logs:
runtime error: invalid memory address or nil pointer dereference

goroutine 363924 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
	/usr/local/go/src/runtime/debug/stack.go:16 +0x19
github.com/99designs/gqlgen/graphql.DefaultRecover({0x7fb0d1b441f8?, 0x30f8ec7a68?}, {0x1bdbf20?, 0x3038ee0?})
	/go/pkg/mod/github.com/99designs/gqlgen@v0.11.3/graphql/recovery.go:16 +0x71
github.com/litmuschaos/litmus/chaoscenter/graphql/server/graph/generated.(*executionContext)._Mutation_registerInfra.func1()
	/gql-server/graph/generated/generated.go:19355 +0x71
panic({0x1bdbf20, 0x3038ee0})
	/usr/local/go/src/runtime/panic.go:884 +0x213
github.com/litmuschaos/litmus/chaoscenter/graphql/server/pkg/chaos_infrastructure.(*infraService).RegisterInfra(0xc0006a42a0, {0x21e22c0, 0xc000d0bce0}, {0xc0002ca1d2, 0x24}, {{0xc000d050f0, 0xf}, {0xc000d050e0, 0x8}, {0xc000d05100, ...}, ...})
	/gql-server/pkg/chaos_infrastructure/service.go:135 +0x97b
github.com/litmuschaos/litmus/chaoscenter/graphql/server/graph.(*mutationResolver).RegisterInfra(0xc00051e380, {0x21e22c0, 0xc000d0bce0}, {0xc0002ca1d2, 0x24}, {{0xc000d050f0, 0xf}, {0xc000d050e0, 0x8}, {0xc000d05100, ...}, ...})
	/gql-server/graph/chaos_infrastructure.resolvers.go:36 +0x255
github.com/litmuschaos/litmus/chaoscenter/graphql/server/graph/generated.(*executionContext)._Mutation_registerInfra.func2({0x21e22c0?, 0xc000d0bce0})
	/gql-server/graph/generated/generated.go:19376 +0x1de
github.com/99designs/gqlgen/graphql/executor.processExtensions.func3({0x21e22c0?, 0xc000d0bce0?}, 0xc000d0bce0?)
	/go/pkg/mod/github.com/99designs/gqlgen@v0.11.3/graphql/executor/extensions.go:62 +0x26
github.com/litmuschaos/litmus/chaoscenter/graphql/server/graph/generated.(*executionContext)._Mutation_registerInfra(0xc000afff50, {0x21e22c0, 0xc000d0bcb0}, {0xc000d0e400?, {0xc004565180?, 0x1?, 0x1?}})
	/gql-server/graph/generated/generated.go:19374 +0x2e3
github.com/litmuschaos/litmus/chaoscenter/graphql/server/graph/generated.(*executionContext)._Mutation(0xc000afff50, {0x21e22c0, 0xc000d0bc80}, {0xc000aff920, 0x1, 0x1})
	/gql-server/graph/generated/generated.go:31765 +0xfd7
github.com/litmuschaos/litmus/chaoscenter/graphql/server/graph/generated.(*executableSchema).Exec.func2({0x21e22c0?, 0xc000d0bc80?})
	/gql-server/graph/generated/generated.go:4326 +0x5b
github.com/99designs/gqlgen/graphql/executor.(*Executor).DispatchOperation.func1.1.1({0x21e22c0, 0xc000d0bc80})
	/go/pkg/mod/github.com/99designs/gqlgen@v0.11.3/graphql/executor/executor.go:105 +0x2b
github.com/99designs/gqlgen/graphql/executor.processExtensions.func2({0x21e22c0?, 0xc000d0bc80?}, 0x1b2f860?)
	/go/pkg/mod/github.com/99designs/gqlgen@v0.11.3/graphql/executor/extensions.go:59 +0x26
github.com/99designs/gqlgen/graphql/executor.(*Executor).DispatchOperation.func1.1({0x21e22c0, 0xc000d0bc20})
	/go/pkg/mod/github.com/99designs/gqlgen@v0.11.3/graphql/executor/executor.go:104 +0x110
github.com/99designs/gqlgen/graphql/handler/transport.POST.Do({}, {0x7fb0d180aa98, 0xc000664400}, 0xc000f2e900, {0x21d4ef0, 0xc0004d4630})
	/go/pkg/mod/github.com/99designs/gqlgen@v0.11.3/graphql/handler/transport/http_post.go:52 +0x308
github.com/99designs/gqlgen/graphql/handler.(*Server).ServeHTTP(0xc000526600, {0x7fb0d180aa98, 0xc000664400}, 0xc000f2e900)
	/go/pkg/mod/github.com/99designs/gqlgen@v0.11.3/graphql/handler/server.go:115 +0x273
github.com/litmuschaos/litmus/chaoscenter/graphql/server/pkg/authorization.Middleware.func1(0xc000664400)
	/gql-server/pkg/authorization/middleware.go:39 +0x2a5
github.com/gin-gonic/gin.(*Context).Next(...)
	/go/pkg/mod/github.com/gin-gonic/gin@v1.9.0/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc000664400)
	/go/pkg/mod/github.com/gin-gonic/gin@v1.9.0/recovery.go:102 +0x82
github.com/gin-gonic/gin.(*Context).Next(...)
	/go/pkg/mod/github.com/gin-gonic/gin@v1.9.0/context.go:174
github.com/litmuschaos/litmus/chaoscenter/graphql/server/api/middleware.StructuredLogger.func1(0xc000664400)
	/gql-server/api/middleware/logger.go:26 +0x9a
github.com/gin-gonic/gin.(*Context).Next(...)
	/go/pkg/mod/github.com/gin-gonic/gin@v1.9.0/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc000456340, 0xc000664400)
	/go/pkg/mod/github.com/gin-gonic/gin@v1.9.0/gin.go:620 +0x66b
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc000456340, {0x21e1130?, 0xc000e7a000}, 0xc000860900)
	/go/pkg/mod/github.com/gin-gonic/gin@v1.9.0/gin.go:576 +0x1dd
net/http.serverHandler.ServeHTTP({0x21d3100?}, {0x21e1130, 0xc000e7a000}, 0xc000860900)
	/usr/local/go/src/net/http/server.go:2936 +0x316
net/http.(*conn).serve(0xc0009c6000, {0x21e22c0, 0xc00027b3e0})
	/usr/local/go/src/net/http/server.go:1995 +0x612
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3089 +0x5ed

This works if you set the description as an empty string
What you expected to happen:
The request should be successful, or the documentation should say description is a mandatory field.
How to reproduce it (as minimally and precisely as possible):
mutation registerInfra($req: RegisterInfraRequest!) {
  registerInfra(
    projectID: "<PROJECT_ID>"
    request: $req
  ) {
    manifest
    token
    infraID
    name
  }
}

{
  "req":{
    "environmentID": "testing1",
    "name": "testing-cluster2",
    "infrastructureType": "Kubernetes",
    "infraScope": "namespace",
    "platformName": "Kubernetes",
    "infraNamespace": "sre-enablement",
    "infraNsExists": true,
    "infraSaExists": false,
    "serviceAccount": "sre-enablement-chaos",
    "skipSsl": false
  }
}

Setting description to empty solves the problem, so it's not a big deal, but putting this out there so if someone is trying to use API, they don't waste 2 hours trying to figure out why the API is not working as expected.

Will take a look at this issue, thanks for reporting @smitthakkar96


https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled

Request type

[ ] Please close this issue, I accidentally submitted it without adding any details
[ ] New documentation
[X] Correction or update

Details
It is Promise.allSettled not Promise.prototype.allSettled.

It looks to be documented in the correct place: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
Where are you seeing the issue?

Looks like it was fixed already ...


Workaround IBM's J9 JVM getSupportedCipherSuites() returning SSL_ prefix cipher names

Motivation:
IBM's J9 JVM utilizes a custom cipher naming scheme with SSL_ prefix [1] instead of the TLS_ prefix defined by TLS RFCs and the JSSE cihper suite names [2]. IBM's documentation says that the SSL_ prefix are "interchangeable" with cipher names with the TLS_ prefix [1]. To work around this issue we parse the supported cipher list and see an SSL_ prefix we can also add the same cipher with the TLS_ prefix. For more details see a discussion on IBM's forums [3] and IBM's issue tracker [4].
[1] https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/com.ibm.java.security.component.80.doc/security-component/jsse2Docs/ciphersuites.html
[2] http://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#ciphersuites
[3] https://www.ibm.com/developerworks/community/forums/html/topic?id=9b5a56a9-fa46-4031-b33b-df91e28d77c2
[4] https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=71770
Modifications:

When parsing the supported cipher list to get the supported ciphers and we encounter a SSL_ prefix we should also add a TLS_ prefix cipher.
Remove SSL_ prefix ciphers from Http2SecurityUtil.

Result:
Work around for IBM JVM's custom naming scheme covers more cases for supported cipher suites.

4.1 (449befa00) 4.0 (4be25ca59)

@CodingFabian - Please validate this give expected behavior. If not we can address issues in a followup PR.

I can't validate this.

If one of your clients (https://github.com/netty/netty/issues/6437#issuecomment-282273125 https://github.com/netty/netty/issues/6437#issuecomment-283604788) experiences problems please open another issue.