no entities shown: Failed to create community graph project: No projected graph named 'None' exists in current database 'neo4j'

I installed LLM Graph Builder App and running Backend and Frontend separately (dev environment). I added an article from Wiki: King Kong (1933 film) - Wikipedia. The log in backend show proper entities extraction. However, when I Preview Graph, there is only "Document & Chunk" and no entities shown.
Errors appear:
2024-12-14 11:17:08,451 - Failed to create community graph project: No projected graph named 'None' exists in current database 'neo4j'
2024-12-14 11:17:08,451 - Failed to create communities: No projected graph named 'None' exists in current database 'neo4j'**
Here is the full log:
[INFO]{'fileName': 'King_Kong_(1933_film)', 'nodeCount': 116, 'relationshipCount': 232, 'total_processing_time': 7.33, 'status': 'Completed', 'model': 'diffbot', 'success_count': 1, 'chunkNodeCount': 116, 'chunkRelCount': 232, 'entityNodeCount': 0, 'entityEntityRelCount': 0, 'communityNodeCount': 0, 'communityRelCount': 0, 'db_url': 'bolt://localhost:7687', 'api_name': 'extract', 'source_url': None, 'wiki_query': 'King_Kong_(1933_film)', 'source_type': 'Wikipedia', 'logging_time': '2024-12-14 19:17:06 UTC', 'elapsed_api_time': '7.34', 'userName': 'neo4j', 'database': 'neo4j', 'aws_access_key_id': None, 'gcs_bucket_name': None, 'gcs_bucket_folder': None, 'gcs_blob_filename': None, 'gcs_project_id': None, 'allowedNodes': 'Document,Session,Actor,Director,Genre,Movie,Person,User', 'allowedRelationship': 'SIMILAR,ACTED_IN,DIRECTED,IN_GENRE,RATED', 'language': 'en', 'retry_condition': 'delete_entities_and_start_from_beginning'}
2024-12-14 11:17:06,151 - extraction completed in 7.34 seconds for file name King_Kong_(1933_film)
INFO: <IP_ADDRESS>:49623 - "POST /extract HTTP/1.1" 200 OK
INFO: <IP_ADDRESS>:49629 - "GET /update_extract_status/King_Kong_%281933_film%29?url=bolt://localhost:7687&userName=neo4j&password=cGFzc3dvcmQ=&database=neo4j HTTP/1.1" 200 OK
2024-12-14 11:17:06,424 - SSE Client disconnected
2024-12-14 11:17:06,484 - update KNN graph
2024-12-14 11:17:06,534 - Updated KNN Graph
2024-12-14 11:17:06,536 - Use pytorch device_name: mps
2024-12-14 11:17:06,536 - Load pretrained SentenceTransformer: all-MiniLM-L6-v2
2024-12-14 11:17:08,124 - Embedding: Using Langchain HuggingFaceEmbeddings , Dimension:384
2024-12-14 11:17:08,124 - Starting the process of creating full-text indexes.
2024-12-14 11:17:08,129 - Database connectivity verified.
2024-12-14 11:17:08,129 - Creating a full-text index for type 'entities'.
2024-12-14 11:17:08,134 - Dropped existing index (if any) in 0.00 seconds.
2024-12-14 11:17:08,168 - Fetched labels in 0.03 seconds.
2024-12-14 11:17:08,173 - Created full-text index in 0.01 seconds.
2024-12-14 11:17:08,178 - Process completed in 0.05 seconds.
2024-12-14 11:17:08,178 - Full-text index for type 'entities' created successfully.
2024-12-14 11:17:08,178 - Creating a full-text index for type 'hybrid'.
2024-12-14 11:17:08,184 - Dropped existing index (if any) in 0.01 seconds.
2024-12-14 11:17:08,200 - Created full-text index in 0.02 seconds.
2024-12-14 11:17:08,206 - Process completed in 0.03 seconds.
2024-12-14 11:17:08,206 - Full-text index for type 'hybrid' created successfully.
2024-12-14 11:17:08,206 - Creating a vector index for type 'vector'.
2024-12-14 11:17:08,206 - Starting the process to create vector index.
2024-12-14 11:17:08,212 - Dropped existing index (if any) in 0.01 seconds.
2024-12-14 11:17:08,232 - Created vector index in 0.02 seconds.
2024-12-14 11:17:08,236 - Vector index for chunk created successfully.
2024-12-14 11:17:08,236 - Driver closed successfully.
2024-12-14 11:17:08,237 - Full-text and vector index creation process completed.
2024-12-14 11:17:08,237 - Full Text index created
2024-12-14 11:17:08,330 - Successfully created GDS driver.
2024-12-14 11:17:08,330 - Starting to clear communities.
2024-12-14 11:17:08,330 - Dropping communities...
2024-12-14 11:17:08,346 - Communities dropped successfully
2024-12-14 11:17:08,346 - Dropping community property from entities...
2024-12-14 11:17:08,358 - Community property dropped successfully
2024-12-14 11:17:08,395 - Creating new graph project 'communities'.
2024-12-14 11:17:08,435 - Graph projection 'None' created successfully with None nodes and None relationships.
2024-12-14 11:17:08,451 - Failed to create community graph project: No projected graph named 'None' exists in current database 'neo4j'
2024-12-14 11:17:08,451 - Failed to create communities: No projected graph named 'None' exists in current database 'neo4j'**
2024-12-14 11:17:08,451 - created communities
2024-12-14 11:17:08,456 - updating node and relationship count
INFO: <IP_ADDRESS>:49623 - "POST /post_processing HTTP/1.1" 200 OK
2024-12-14 11:17:36,910 - Starting graph query process
2024-12-14 11:17:36,910 - Attempting to connect to the Neo4j database at bolt://localhost:7687
2024-12-14 11:17:36,911 - Connection successful
2024-12-14 11:17:36,911 - Executing query for documents: ['King_Kong_(1933_film)']
2024-12-14 11:17:37,116 - no of nodes : 51
2024-12-14 11:17:37,116 - no of relations : 100
2024-12-14 11:17:37,116 - Query process completed successfully
2024-12-14 11:17:37,116 - Closing connection for graph_query api
[INFO]{'api_name': 'graph_query', 'db_url': 'bolt://localhost:7687', 'userName': 'neo4j', 'database': 'neo4j', 'document_names': '["King_Kong_(1933_film)"]', 'logging_time': '2024-12-14 19:17:37 UTC', 'elapsed_api_time': '0.21'}
INFO: <IP_ADDRESS>:49702 - "POST /graph_query HTTP/1.1" 200 OK

Thanks kaustubh-darekar.
Here are the details:
The LLM model you tried - Diffbot. I don't subscribe to ChatGPT or similar service. I tried to use Ollama without success. I put a note in https://github.com/neo4j-labs/llm-graph-builder/issues/678 about my issue
The DB instance (Is it GDS instance or aura DB instance). - I am using Neo4j Desktop 1.6.1
Full logs trace - could you tell me where can I get the full logs?

logs.txt
Please find attached logs of backend terminal for successful extraction when using diffbot model for given wikipedia page.
the actual extraction logs will be just above the logs you shared.
it will start with statement something like
2024-12-19 05:57:29,121 - Total Pages from Wikipedia = 24
and ends with statement like
2024-12-19 06:00:59,434 - extraction completed in 250.84 seconds for file name King_Kong_(1933_film)
Also note currently community creation is supported with openai model specifically openai_gpt_4o.
if you dont have subscription you can disable communitites through
home page --> graph enhancement --> post processing jobs --> uncheck enable communities

this will skip community creation still you will be able to create graph with chunks & entities.

I uncheck enable communities and still cannot get entities using Diffbot LLM. Could you please let me know what I need to change?

can you also share the extraction logs.
it will start with statement something like
2024-12-19 05:57:29,121 - Total Pages from Wikipedia = 24
and ends with statement like
2024-12-19 06:00:59,434 - extraction completed in 250.84 seconds for file name King_Kong_(1933_film)
sample logs i have shared in earlier message

Here is the log:
INFO:     <IP_ADDRESS>:63605 - "POST /graph_query HTTP/1.1" 200 OK
2024-12-18 23:43:55,285 - Schema result from DB: [{'labels': ['Document', 'Session'], 'relationshipTypes': ['SIMILAR']}]
[INFO]{'api_name': 'schema', 'db_url': 'bolt://localhost:7687', 'userName': 'neo4j', 'database': 'neo4j', 'logging_time': '2024-12-19 07:43:55 UTC', 'elapsed_api_time': '0.10'}
INFO:     <IP_ADDRESS>:63724 - "POST /schema HTTP/1.1" 200 OK
2024-12-18 23:44:01,267 - Schema result from DB: [{'labels': ['Document', 'Session'], 'relationshipTypes': ['SIMILAR']}]
[INFO]{'api_name': 'schema', 'db_url': 'bolt://localhost:7687', 'userName': 'neo4j', 'database': 'neo4j', 'logging_time': '2024-12-19 07:44:01 UTC', 'elapsed_api_time': '0.04'}
INFO:     <IP_ADDRESS>:63729 - "POST /schema HTTP/1.1" 200 OK
2024-12-18 23:44:11,444 - <src.entities.source_node.sourceNode object at 0x3367fedb0>
2024-12-18 23:44:11,445 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'status': 'Ready to Reprocess', 'nodeCount': 0, 'relationshipCount': 0, 'is_cancelled': False, 'processed_chunk': 0, 'retry_condition': 'delete_entities_and_start_from_beginning'}}
2024-12-18 23:44:11,445 - Update source node properties
[INFO]{'api_name': 'retry_processing', 'db_url': 'bolt://localhost:7687', 'userName': 'neo4j', 'database': 'neo4j', 'file_name': 'King_Kong_(1933_film)', 'retry_condition': 'delete_entities_and_start_from_beginning', 'logging_time': '2024-12-19 07:44:11 UTC', 'elapsed_api_time': '0.21'}
INFO:     <IP_ADDRESS>:63734 - "POST /retry_processing HTTP/1.1" 200 OK
2024-12-18 23:44:14,923 - Time taken database connection: 0.00 seconds
2024-12-18 23:44:14,975 - Retry : start_from_beginning with chunks 116
2024-12-18 23:44:14,975 - Time taken to create list chunkids with chunk document: 0.05 seconds
2024-12-18 23:44:14,998 - Time taken to get the current status of document node: 0.02 seconds
2024-12-18 23:44:14,998 - King_Kong_(1933_film)
2024-12-18 23:44:14,998 - <src.entities.source_node.sourceNode object at 0x336cfc6e0>
2024-12-18 23:44:14,998 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'status': 'Processing', 'model': 'diffbot', 'total_chunks': 116, 'processed_chunk': 0}}
2024-12-18 23:44:14,998 - Update source node properties
2024-12-18 23:44:15,010 - updating node and relationship count
2024-12-18 23:44:15,188 - Time taken to update the document source node: 0.19 seconds
2024-12-18 23:44:15,188 - Update the status as Processing
2024-12-18 23:44:15,188 - Selected Chunks upto: 20
2024-12-18 23:44:15,190 - Value of is_cancelled : False
2024-12-18 23:44:15,190 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-18 23:44:15,190 - update embedding and vector index for chunks
2024-12-18 23:44:15,348 - create vector index on chunk embedding
2024-12-18 23:44:15,385 - create vector index on chunk embedding
2024-12-18 23:44:15,401 - create vector index on chunk embedding
2024-12-18 23:44:15,423 - create vector index on chunk embedding
2024-12-18 23:44:15,440 - create vector index on chunk embedding
2024-12-18 23:44:15,454 - create vector index on chunk embedding
2024-12-18 23:44:15,475 - create vector index on chunk embedding
2024-12-18 23:44:15,487 - create vector index on chunk embedding
2024-12-18 23:44:15,498 - create vector index on chunk embedding
2024-12-18 23:44:15,517 - create vector index on chunk embedding
2024-12-18 23:44:15,535 - create vector index on chunk embedding
2024-12-18 23:44:15,560 - create vector index on chunk embedding
2024-12-18 23:44:15,572 - create vector index on chunk embedding
2024-12-18 23:44:15,588 - create vector index on chunk embedding
2024-12-18 23:44:15,607 - create vector index on chunk embedding
2024-12-18 23:44:15,624 - create vector index on chunk embedding
2024-12-18 23:44:15,639 - create vector index on chunk embedding
2024-12-18 23:44:15,653 - create vector index on chunk embedding
2024-12-18 23:44:15,668 - create vector index on chunk embedding
2024-12-18 23:44:15,686 - create vector index on chunk embedding
2024-12-18 23:44:15,757 - Time taken to update embedding in chunk node: 0.57 seconds
2024-12-18 23:44:15,757 - Get graph document list from models
2024-12-18 23:44:15,757 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-18 23:44:15,757 - Model created - Model Version: diffbot
2024-12-18 23:44:15,757 - Combining 6 chunks before sending request to LLM
2024-12-18 23:44:16,323 - Time taken to extract enitities from LLM Graph Builder: 0.57 seconds
2024-12-18 23:44:16,540 - Time taken to save graph document in neo4j: 0.22 seconds
2024-12-18 23:44:16,540 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-18 23:44:16,540 - Create HAS_ENTITY relationship between chunks and entities
2024-12-18 23:44:16,540 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-18 23:44:16,540 - Time taken 20 chunks processed upto 20 completed in 1.35 seconds for file name King_Kong_(1933_film)
2024-12-18 23:44:16,540 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 18, 23, 44, 16, 540702), 'processingTime': 1.62, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 20}}
2024-12-18 23:44:16,540 - Update source node properties
2024-12-18 23:44:16,548 - updating node and relationship count
2024-12-18 23:44:16,559 - Selected Chunks upto: 40
2024-12-18 23:44:16,561 - Value of is_cancelled : False
2024-12-18 23:44:16,561 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-18 23:44:16,561 - update embedding and vector index for chunks
2024-12-18 23:44:16,583 - create vector index on chunk embedding
2024-12-18 23:44:16,601 - create vector index on chunk embedding
2024-12-18 23:44:16,618 - create vector index on chunk embedding
2024-12-18 23:44:16,638 - create vector index on chunk embedding
2024-12-18 23:44:16,654 - create vector index on chunk embedding
2024-12-18 23:44:16,675 - create vector index on chunk embedding
2024-12-18 23:44:16,692 - create vector index on chunk embedding
2024-12-18 23:44:16,711 - create vector index on chunk embedding
2024-12-18 23:44:16,726 - create vector index on chunk embedding
2024-12-18 23:44:16,740 - create vector index on chunk embedding
2024-12-18 23:44:16,761 - create vector index on chunk embedding
2024-12-18 23:44:16,784 - create vector index on chunk embedding
2024-12-18 23:44:16,803 - create vector index on chunk embedding
2024-12-18 23:44:16,819 - create vector index on chunk embedding
2024-12-18 23:44:16,829 - create vector index on chunk embedding
2024-12-18 23:44:16,847 - create vector index on chunk embedding
2024-12-18 23:44:16,862 - create vector index on chunk embedding
2024-12-18 23:44:16,879 - create vector index on chunk embedding
2024-12-18 23:44:16,917 - create vector index on chunk embedding
2024-12-18 23:44:16,947 - create vector index on chunk embedding
2024-12-18 23:44:17,000 - Time taken to update embedding in chunk node: 0.44 seconds
2024-12-18 23:44:17,000 - Get graph document list from models
2024-12-18 23:44:17,000 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-18 23:44:17,000 - Model created - Model Version: diffbot
2024-12-18 23:44:17,000 - Combining 6 chunks before sending request to LLM
2024-12-18 23:44:17,381 - Time taken to extract enitities from LLM Graph Builder: 0.38 seconds
2024-12-18 23:44:17,400 - Time taken to save graph document in neo4j: 0.02 seconds
2024-12-18 23:44:17,400 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-18 23:44:17,400 - Create HAS_ENTITY relationship between chunks and entities
2024-12-18 23:44:17,400 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-18 23:44:17,400 - Time taken 20 chunks processed upto 40 completed in 0.84 seconds for file name King_Kong_(1933_film)
2024-12-18 23:44:17,400 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 18, 23, 44, 17, 400611), 'processingTime': 2.48, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 40}}
2024-12-18 23:44:17,400 - Update source node properties
2024-12-18 23:44:17,407 - updating node and relationship count
2024-12-18 23:44:17,419 - Selected Chunks upto: 60
2024-12-18 23:44:17,421 - Value of is_cancelled : False
2024-12-18 23:44:17,421 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-18 23:44:17,421 - update embedding and vector index for chunks
2024-12-18 23:44:17,466 - create vector index on chunk embedding
2024-12-18 23:44:17,485 - create vector index on chunk embedding
2024-12-18 23:44:17,506 - create vector index on chunk embedding
2024-12-18 23:44:17,520 - create vector index on chunk embedding
2024-12-18 23:44:17,563 - create vector index on chunk embedding
2024-12-18 23:44:17,657 - create vector index on chunk embedding
2024-12-18 23:44:17,681 - create vector index on chunk embedding
2024-12-18 23:44:17,705 - create vector index on chunk embedding
2024-12-18 23:44:17,729 - create vector index on chunk embedding
2024-12-18 23:44:17,754 - create vector index on chunk embedding
2024-12-18 23:44:17,775 - create vector index on chunk embedding
2024-12-18 23:44:17,792 - create vector index on chunk embedding
2024-12-18 23:44:17,818 - create vector index on chunk embedding
2024-12-18 23:44:17,840 - create vector index on chunk embedding
2024-12-18 23:44:17,861 - create vector index on chunk embedding
2024-12-18 23:44:17,879 - create vector index on chunk embedding
2024-12-18 23:44:17,896 - create vector index on chunk embedding
2024-12-18 23:44:17,909 - create vector index on chunk embedding
2024-12-18 23:44:17,926 - create vector index on chunk embedding
2024-12-18 23:44:17,946 - create vector index on chunk embedding
2024-12-18 23:44:17,981 - Time taken to update embedding in chunk node: 0.56 seconds
2024-12-18 23:44:17,981 - Get graph document list from models
2024-12-18 23:44:17,982 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-18 23:44:17,982 - Model created - Model Version: diffbot
2024-12-18 23:44:17,982 - Combining 6 chunks before sending request to LLM
2024-12-18 23:44:18,359 - Time taken to extract enitities from LLM Graph Builder: 0.38 seconds
2024-12-18 23:44:18,431 - Time taken to save graph document in neo4j: 0.07 seconds
2024-12-18 23:44:18,431 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-18 23:44:18,431 - Create HAS_ENTITY relationship between chunks and entities
2024-12-18 23:44:18,431 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-18 23:44:18,431 - Time taken 20 chunks processed upto 60 completed in 1.01 seconds for file name King_Kong_(1933_film)
2024-12-18 23:44:18,431 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 18, 23, 44, 18, 431936), 'processingTime': 3.51, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 60}}
2024-12-18 23:44:18,432 - Update source node properties
2024-12-18 23:44:18,441 - updating node and relationship count
2024-12-18 23:44:18,449 - Selected Chunks upto: 80
2024-12-18 23:44:18,450 - Value of is_cancelled : False
2024-12-18 23:44:18,450 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-18 23:44:18,450 - update embedding and vector index for chunks
2024-12-18 23:44:18,468 - create vector index on chunk embedding
2024-12-18 23:44:18,491 - create vector index on chunk embedding
2024-12-18 23:44:18,509 - create vector index on chunk embedding
2024-12-18 23:44:18,537 - create vector index on chunk embedding
2024-12-18 23:44:18,578 - create vector index on chunk embedding
2024-12-18 23:44:18,595 - create vector index on chunk embedding
2024-12-18 23:44:18,610 - create vector index on chunk embedding
2024-12-18 23:44:18,628 - create vector index on chunk embedding
2024-12-18 23:44:18,643 - create vector index on chunk embedding
2024-12-18 23:44:18,659 - create vector index on chunk embedding
2024-12-18 23:44:18,671 - create vector index on chunk embedding
2024-12-18 23:44:18,690 - create vector index on chunk embedding
2024-12-18 23:44:18,706 - create vector index on chunk embedding
2024-12-18 23:44:18,725 - create vector index on chunk embedding
2024-12-18 23:44:18,742 - create vector index on chunk embedding
2024-12-18 23:44:18,759 - create vector index on chunk embedding
2024-12-18 23:44:18,773 - create vector index on chunk embedding
2024-12-18 23:44:18,790 - create vector index on chunk embedding
2024-12-18 23:44:18,804 - create vector index on chunk embedding
2024-12-18 23:44:18,818 - create vector index on chunk embedding
2024-12-18 23:44:18,855 - Time taken to update embedding in chunk node: 0.40 seconds
2024-12-18 23:44:18,855 - Get graph document list from models
2024-12-18 23:44:18,855 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-18 23:44:18,855 - Model created - Model Version: diffbot
2024-12-18 23:44:18,855 - Combining 6 chunks before sending request to LLM
2024-12-18 23:44:19,310 - Time taken to extract enitities from LLM Graph Builder: 0.46 seconds
2024-12-18 23:44:19,328 - Time taken to save graph document in neo4j: 0.02 seconds
2024-12-18 23:44:19,329 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-18 23:44:19,329 - Create HAS_ENTITY relationship between chunks and entities
2024-12-18 23:44:19,329 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-18 23:44:19,329 - Time taken 20 chunks processed upto 80 completed in 0.88 seconds for file name King_Kong_(1933_film)
2024-12-18 23:44:19,329 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 18, 23, 44, 19, 329091), 'processingTime': 4.41, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 80}}
2024-12-18 23:44:19,329 - Update source node properties
2024-12-18 23:44:19,341 - updating node and relationship count
2024-12-18 23:44:19,359 - Selected Chunks upto: 100
2024-12-18 23:44:19,361 - Value of is_cancelled : False
2024-12-18 23:44:19,361 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-18 23:44:19,361 - update embedding and vector index for chunks
2024-12-18 23:44:19,380 - create vector index on chunk embedding
2024-12-18 23:44:19,398 - create vector index on chunk embedding
2024-12-18 23:44:19,410 - create vector index on chunk embedding
2024-12-18 23:44:19,421 - create vector index on chunk embedding
2024-12-18 23:44:19,437 - create vector index on chunk embedding
2024-12-18 23:44:19,455 - create vector index on chunk embedding
2024-12-18 23:44:19,469 - create vector index on chunk embedding
2024-12-18 23:44:19,487 - create vector index on chunk embedding
2024-12-18 23:44:19,502 - create vector index on chunk embedding
2024-12-18 23:44:19,518 - create vector index on chunk embedding
2024-12-18 23:44:19,536 - create vector index on chunk embedding
2024-12-18 23:44:19,550 - create vector index on chunk embedding
2024-12-18 23:44:19,563 - create vector index on chunk embedding
2024-12-18 23:44:19,578 - create vector index on chunk embedding
2024-12-18 23:44:19,595 - create vector index on chunk embedding
2024-12-18 23:44:19,611 - create vector index on chunk embedding
2024-12-18 23:44:19,626 - create vector index on chunk embedding
2024-12-18 23:44:19,645 - create vector index on chunk embedding
2024-12-18 23:44:19,666 - create vector index on chunk embedding
2024-12-18 23:44:19,684 - create vector index on chunk embedding
2024-12-18 23:44:19,725 - Time taken to update embedding in chunk node: 0.36 seconds
2024-12-18 23:44:19,725 - Get graph document list from models
2024-12-18 23:44:19,725 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-18 23:44:19,725 - Model created - Model Version: diffbot
2024-12-18 23:44:19,725 - Combining 6 chunks before sending request to LLM
2024-12-18 23:44:20,111 - Time taken to extract enitities from LLM Graph Builder: 0.39 seconds
2024-12-18 23:44:20,128 - Time taken to save graph document in neo4j: 0.02 seconds
2024-12-18 23:44:20,128 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-18 23:44:20,128 - Create HAS_ENTITY relationship between chunks and entities
2024-12-18 23:44:20,128 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-18 23:44:20,128 - Time taken 20 chunks processed upto 100 completed in 0.77 seconds for file name King_Kong_(1933_film)
2024-12-18 23:44:20,128 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 18, 23, 44, 20, 128502), 'processingTime': 5.21, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 100}}
2024-12-18 23:44:20,128 - Update source node properties
2024-12-18 23:44:20,137 - updating node and relationship count
2024-12-18 23:44:20,147 - Selected Chunks upto: 120
2024-12-18 23:44:20,148 - Value of is_cancelled : False
2024-12-18 23:44:20,148 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-18 23:44:20,148 - update embedding and vector index for chunks
2024-12-18 23:44:20,170 - create vector index on chunk embedding
2024-12-18 23:44:20,183 - create vector index on chunk embedding
2024-12-18 23:44:20,201 - create vector index on chunk embedding
2024-12-18 23:44:20,216 - create vector index on chunk embedding
2024-12-18 23:44:20,234 - create vector index on chunk embedding
2024-12-18 23:44:20,257 - create vector index on chunk embedding
2024-12-18 23:44:20,316 - create vector index on chunk embedding
2024-12-18 23:44:20,340 - create vector index on chunk embedding
2024-12-18 23:44:20,359 - create vector index on chunk embedding
2024-12-18 23:44:20,388 - create vector index on chunk embedding
2024-12-18 23:44:20,427 - create vector index on chunk embedding
2024-12-18 23:44:20,445 - create vector index on chunk embedding
2024-12-18 23:44:20,464 - create vector index on chunk embedding
2024-12-18 23:44:20,476 - create vector index on chunk embedding
2024-12-18 23:44:20,494 - create vector index on chunk embedding
2024-12-18 23:44:20,512 - create vector index on chunk embedding
2024-12-18 23:44:20,556 - Time taken to update embedding in chunk node: 0.41 seconds
2024-12-18 23:44:20,556 - Get graph document list from models
2024-12-18 23:44:20,556 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-18 23:44:20,556 - Model created - Model Version: diffbot
2024-12-18 23:44:20,556 - Combining 6 chunks before sending request to LLM
2024-12-18 23:44:20,835 - Time taken to extract enitities from LLM Graph Builder: 0.28 seconds
2024-12-18 23:44:20,845 - Time taken to save graph document in neo4j: 0.01 seconds
2024-12-18 23:44:20,845 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-18 23:44:20,845 - Create HAS_ENTITY relationship between chunks and entities
2024-12-18 23:44:20,845 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-18 23:44:20,845 - Time taken 20 chunks processed upto 116 completed in 0.70 seconds for file name King_Kong_(1933_film)
2024-12-18 23:44:20,845 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 18, 23, 44, 20, 845954), 'processingTime': 5.93, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 116}}
2024-12-18 23:44:20,845 - Update source node properties
2024-12-18 23:44:20,853 - updating node and relationship count
2024-12-18 23:44:20,871 - Job Status at the end : Completed
2024-12-18 23:44:20,871 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'status': 'Completed', 'processingTime': 5.95}}
2024-12-18 23:44:20,871 - Update source node properties
2024-12-18 23:44:20,878 - updating node and relationship count
2024-12-18 23:44:20,885 - Updated the nodeCount and relCount properties in Document node
2024-12-18 23:44:20,885 - file:King_Kong_(1933_film) extraction has been completed
2024-12-18 23:44:20,885 - Time taken to processing source function completed in 5.97 seconds for file name King_Kong_(1933_film)
2024-12-18 23:44:20,886 - Going for counting nodes and relationships in extract
2024-12-18 23:44:20,899 - updating node and relationship count
2024-12-18 23:44:20,908 - Nodes and Relationship Counts updated
2024-12-18 23:44:20,908 - counting completed in 0.02
[INFO]{'fileName': 'King_Kong_(1933_film)', 'nodeCount': 116, 'relationshipCount': 232, 'total_processing_time': 5.95, 'status': 'Completed', 'model': 'diffbot', 'success_count': 1, 'chunkNodeCount': 116, 'chunkRelCount': 232, 'entityNodeCount': 0, 'entityEntityRelCount': 0, 'communityNodeCount': 0, 'communityRelCount': 0, 'db_url': 'bolt://localhost:7687', 'api_name': 'extract', 'source_url': None, 'wiki_query': 'King_Kong_(1933_film)', 'source_type': 'Wikipedia', 'logging_time': '2024-12-19 07:44:20 UTC', 'elapsed_api_time': '5.97', 'userName': 'neo4j', 'database': 'neo4j', 'aws_access_key_id': None, 'gcs_bucket_name': None, 'gcs_bucket_folder': None, 'gcs_blob_filename': None, 'gcs_project_id': None, 'allowedNodes': 'Document,Session,Actor,Director,Genre,Movie,Person,User', 'allowedRelationship': 'SIMILAR,ACTED_IN,DIRECTED,IN_GENRE,RATED', 'language': 'en', 'retry_condition': 'delete_entities_and_start_from_beginning'}
2024-12-18 23:44:20,909 - extraction completed in 5.97 seconds for file name King_Kong_(1933_film)
INFO:     <IP_ADDRESS>:63741 - "POST /extract HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:63747 - "GET /update_extract_status/King_Kong_%281933_film%29?url=bolt://localhost:7687&userName=neo4j&password=cGFzc3dvcmQ=&database=neo4j HTTP/1.1" 200 OK
2024-12-18 23:44:21,618 -  SSE Client disconnected
2024-12-18 23:44:21,645 - update KNN graph
2024-12-18 23:44:21,704 - Updated KNN Graph
2024-12-18 23:44:21,706 - Use pytorch device_name: mps
2024-12-18 23:44:21,706 - Load pretrained SentenceTransformer: all-MiniLM-L6-v2
2024-12-18 23:44:23,015 - Embedding: Using Langchain HuggingFaceEmbeddings , Dimension:384
2024-12-18 23:44:23,015 - Starting the process of creating full-text indexes.
2024-12-18 23:44:23,018 - Database connectivity verified.
2024-12-18 23:44:23,018 - Creating a full-text index for type 'entities'.
2024-12-18 23:44:23,021 - Dropped existing index (if any) in 0.00 seconds.
2024-12-18 23:44:23,065 - Fetched labels in 0.04 seconds.
2024-12-18 23:44:23,071 - Created full-text index in 0.01 seconds.
2024-12-18 23:44:23,080 - Process completed in 0.06 seconds.
2024-12-18 23:44:23,080 - Full-text index for type 'entities' created successfully.
2024-12-18 23:44:23,080 - Creating a full-text index for type 'hybrid'.
2024-12-18 23:44:23,082 - Dropped existing index (if any) in 0.00 seconds.
2024-12-18 23:44:23,128 - Created full-text index in 0.05 seconds.
2024-12-18 23:44:23,133 - Process completed in 0.05 seconds.
2024-12-18 23:44:23,133 - Full-text index for type 'hybrid' created successfully.
2024-12-18 23:44:23,133 - Creating a vector index for type 'vector'.
2024-12-18 23:44:23,133 - Starting the process to create vector index.
2024-12-18 23:44:23,137 - Dropped existing index (if any) in 0.00 seconds.
2024-12-18 23:44:23,148 - Created vector index in 0.01 seconds.
2024-12-18 23:44:23,152 - Vector index for chunk created successfully.
2024-12-18 23:44:23,152 - Driver closed successfully.
2024-12-18 23:44:23,152 - Full-text and vector index creation process completed.
2024-12-18 23:44:23,153 - Full Text index created
INFO:     <IP_ADDRESS>:63741 - "POST /post_processing HTTP/1.1" 200 OK
2024-12-18 23:44:25,549 - Starting graph query process
2024-12-18 23:44:25,549 - Attempting to connect to the Neo4j database at bolt://localhost:7687
2024-12-18 23:44:25,550 - Connection successful
2024-12-18 23:44:25,550 - Executing query for documents: ['King_Kong_(1933_film)']
2024-12-18 23:44:25,710 - no of nodes : 51
2024-12-18 23:44:25,710 - no of relations : 100
2024-12-18 23:44:25,710 - Query process completed successfully
2024-12-18 23:44:25,710 - Closing connection for graph_query api
[INFO]{'api_name': 'graph_query', 'db_url': 'bolt://localhost:7687', 'userName': 'neo4j', 'database': 'neo4j', 'document_names': '["King_Kong_(1933_film)"]', 'logging_time': '2024-12-19 07:44:25 UTC', 'elapsed_api_time': '0.16'}
INFO:     <IP_ADDRESS>:63790 - "POST /graph_query HTTP/1.1" 200 OK
2024-12-19 00:43:27,390 - <src.entities.source_node.sourceNode object at 0x336ab51f0>
2024-12-19 00:43:27,391 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'status': 'Ready to Reprocess', 'nodeCount': 0, 'relationshipCount': 0, 'is_cancelled': False, 'processed_chunk': 0, 'retry_condition': 'delete_entities_and_start_from_beginning'}}
2024-12-19 00:43:27,391 - Update source node properties
[INFO]{'api_name': 'retry_processing', 'db_url': 'bolt://localhost:7687', 'userName': 'neo4j', 'database': 'neo4j', 'file_name': 'King_Kong_(1933_film)', 'retry_condition': 'delete_entities_and_start_from_beginning', 'logging_time': '2024-12-19 08:43:27 UTC', 'elapsed_api_time': '0.26'}
INFO:     <IP_ADDRESS>:49969 - "POST /retry_processing HTTP/1.1" 200 OK
2024-12-19 00:43:38,582 - Time taken database connection: 0.01 seconds
2024-12-19 00:43:38,702 - Retry : start_from_beginning with chunks 116
2024-12-19 00:43:38,702 - Time taken to create list chunkids with chunk document: 0.12 seconds
2024-12-19 00:43:38,729 - Time taken to get the current status of document node: 0.03 seconds
2024-12-19 00:43:38,729 - King_Kong_(1933_film)
2024-12-19 00:43:38,729 - <src.entities.source_node.sourceNode object at 0x336cc5b20>
2024-12-19 00:43:38,729 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'status': 'Processing', 'model': 'diffbot', 'total_chunks': 116, 'processed_chunk': 0}}
2024-12-19 00:43:38,729 - Update source node properties
2024-12-19 00:43:38,741 - updating node and relationship count
2024-12-19 00:43:38,887 - Time taken to update the document source node: 0.16 seconds
2024-12-19 00:43:38,887 - Update the status as Processing
2024-12-19 00:43:38,887 - Selected Chunks upto: 20
2024-12-19 00:43:38,890 - Value of is_cancelled : False
2024-12-19 00:43:38,890 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-19 00:43:38,890 - update embedding and vector index for chunks
2024-12-19 00:43:39,842 - create vector index on chunk embedding
2024-12-19 00:43:39,962 - create vector index on chunk embedding
2024-12-19 00:43:40,040 - create vector index on chunk embedding
2024-12-19 00:43:40,128 - create vector index on chunk embedding
2024-12-19 00:43:40,172 - create vector index on chunk embedding
2024-12-19 00:43:40,192 - create vector index on chunk embedding
2024-12-19 00:43:40,231 - create vector index on chunk embedding
2024-12-19 00:43:40,257 - create vector index on chunk embedding
2024-12-19 00:43:40,276 - create vector index on chunk embedding
2024-12-19 00:43:40,326 - create vector index on chunk embedding
2024-12-19 00:43:40,353 - create vector index on chunk embedding
2024-12-19 00:43:40,394 - create vector index on chunk embedding
2024-12-19 00:43:40,409 - create vector index on chunk embedding
2024-12-19 00:43:40,436 - create vector index on chunk embedding
2024-12-19 00:43:40,479 - create vector index on chunk embedding
2024-12-19 00:43:40,496 - create vector index on chunk embedding
2024-12-19 00:43:40,509 - create vector index on chunk embedding
2024-12-19 00:43:40,526 - create vector index on chunk embedding
2024-12-19 00:43:40,546 - create vector index on chunk embedding
2024-12-19 00:43:40,569 - create vector index on chunk embedding
2024-12-19 00:43:40,686 - Time taken to update embedding in chunk node: 1.80 seconds
2024-12-19 00:43:40,686 - Get graph document list from models
2024-12-19 00:43:40,686 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-19 00:43:40,686 - Model created - Model Version: diffbot
2024-12-19 00:43:40,686 - Combining 6 chunks before sending request to LLM
2024-12-19 00:43:41,157 - Time taken to extract enitities from LLM Graph Builder: 0.47 seconds
2024-12-19 00:43:41,736 - Time taken to save graph document in neo4j: 0.58 seconds
2024-12-19 00:43:41,736 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-19 00:43:41,736 - Create HAS_ENTITY relationship between chunks and entities
2024-12-19 00:43:41,736 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-19 00:43:41,736 - Time taken 20 chunks processed upto 20 completed in 2.85 seconds for file name King_Kong_(1933_film)
2024-12-19 00:43:41,737 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 19, 0, 43, 41, 736705), 'processingTime': 3.16, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 20}}
2024-12-19 00:43:41,737 - Update source node properties
2024-12-19 00:43:41,745 - updating node and relationship count
2024-12-19 00:43:41,752 - Selected Chunks upto: 40
2024-12-19 00:43:41,754 - Value of is_cancelled : False
2024-12-19 00:43:41,755 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-19 00:43:41,755 - update embedding and vector index for chunks
2024-12-19 00:43:41,805 - create vector index on chunk embedding
2024-12-19 00:43:41,819 - create vector index on chunk embedding
2024-12-19 00:43:41,829 - create vector index on chunk embedding
2024-12-19 00:43:41,844 - create vector index on chunk embedding
2024-12-19 00:43:41,869 - create vector index on chunk embedding
2024-12-19 00:43:41,897 - create vector index on chunk embedding
2024-12-19 00:43:41,916 - create vector index on chunk embedding
2024-12-19 00:43:41,945 - create vector index on chunk embedding
2024-12-19 00:43:41,957 - create vector index on chunk embedding
2024-12-19 00:43:41,968 - create vector index on chunk embedding
2024-12-19 00:43:41,994 - create vector index on chunk embedding
2024-12-19 00:43:42,025 - create vector index on chunk embedding
2024-12-19 00:43:42,045 - create vector index on chunk embedding
2024-12-19 00:43:42,074 - create vector index on chunk embedding
2024-12-19 00:43:42,088 - create vector index on chunk embedding
2024-12-19 00:43:42,120 - create vector index on chunk embedding
2024-12-19 00:43:42,143 - create vector index on chunk embedding
2024-12-19 00:43:42,164 - create vector index on chunk embedding
2024-12-19 00:43:42,199 - create vector index on chunk embedding
2024-12-19 00:43:42,231 - create vector index on chunk embedding
2024-12-19 00:43:42,279 - Time taken to update embedding in chunk node: 0.53 seconds
2024-12-19 00:43:42,279 - Get graph document list from models
2024-12-19 00:43:42,279 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-19 00:43:42,279 - Model created - Model Version: diffbot
2024-12-19 00:43:42,279 - Combining 6 chunks before sending request to LLM
2024-12-19 00:43:42,926 - Time taken to extract enitities from LLM Graph Builder: 0.65 seconds
2024-12-19 00:43:42,943 - Time taken to save graph document in neo4j: 0.02 seconds
2024-12-19 00:43:42,943 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-19 00:43:42,943 - Create HAS_ENTITY relationship between chunks and entities
2024-12-19 00:43:42,943 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-19 00:43:42,943 - Time taken 20 chunks processed upto 40 completed in 1.19 seconds for file name King_Kong_(1933_film)
2024-12-19 00:43:42,943 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 19, 0, 43, 42, 943268), 'processingTime': 4.37, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 40}}
2024-12-19 00:43:42,943 - Update source node properties
2024-12-19 00:43:42,958 - updating node and relationship count
2024-12-19 00:43:42,975 - Selected Chunks upto: 60
2024-12-19 00:43:42,979 - Value of is_cancelled : False
2024-12-19 00:43:42,980 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-19 00:43:42,980 - update embedding and vector index for chunks
2024-12-19 00:43:43,029 - create vector index on chunk embedding
2024-12-19 00:43:43,049 - create vector index on chunk embedding
2024-12-19 00:43:43,077 - create vector index on chunk embedding
2024-12-19 00:43:43,091 - create vector index on chunk embedding
2024-12-19 00:43:43,106 - create vector index on chunk embedding
2024-12-19 00:43:43,145 - create vector index on chunk embedding
2024-12-19 00:43:43,159 - create vector index on chunk embedding
2024-12-19 00:43:43,171 - create vector index on chunk embedding
2024-12-19 00:43:43,188 - create vector index on chunk embedding
2024-12-19 00:43:43,205 - create vector index on chunk embedding
2024-12-19 00:43:43,230 - create vector index on chunk embedding
2024-12-19 00:43:43,245 - create vector index on chunk embedding
2024-12-19 00:43:43,258 - create vector index on chunk embedding
2024-12-19 00:43:43,270 - create vector index on chunk embedding
2024-12-19 00:43:43,293 - create vector index on chunk embedding
2024-12-19 00:43:43,326 - create vector index on chunk embedding
2024-12-19 00:43:43,341 - create vector index on chunk embedding
2024-12-19 00:43:43,352 - create vector index on chunk embedding
2024-12-19 00:43:43,367 - create vector index on chunk embedding
2024-12-19 00:43:43,391 - create vector index on chunk embedding
2024-12-19 00:43:43,432 - Time taken to update embedding in chunk node: 0.45 seconds
2024-12-19 00:43:43,432 - Get graph document list from models
2024-12-19 00:43:43,432 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-19 00:43:43,432 - Model created - Model Version: diffbot
2024-12-19 00:43:43,432 - Combining 6 chunks before sending request to LLM
2024-12-19 00:43:45,095 - Time taken to extract enitities from LLM Graph Builder: 1.66 seconds
2024-12-19 00:43:45,236 - Time taken to save graph document in neo4j: 0.14 seconds
2024-12-19 00:43:45,236 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-19 00:43:45,236 - Create HAS_ENTITY relationship between chunks and entities
2024-12-19 00:43:45,236 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-19 00:43:45,236 - Time taken 20 chunks processed upto 60 completed in 2.26 seconds for file name King_Kong_(1933_film)
2024-12-19 00:43:45,236 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 19, 0, 43, 45, 236687), 'processingTime': 6.66, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 60}}
2024-12-19 00:43:45,236 - Update source node properties
2024-12-19 00:43:45,246 - updating node and relationship count
2024-12-19 00:43:45,258 - Selected Chunks upto: 80
2024-12-19 00:43:45,260 - Value of is_cancelled : False
2024-12-19 00:43:45,260 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-19 00:43:45,260 - update embedding and vector index for chunks
2024-12-19 00:43:45,350 - create vector index on chunk embedding
2024-12-19 00:43:45,417 - create vector index on chunk embedding
2024-12-19 00:43:45,442 - create vector index on chunk embedding
2024-12-19 00:43:45,476 - create vector index on chunk embedding
2024-12-19 00:43:45,564 - create vector index on chunk embedding
2024-12-19 00:43:45,594 - create vector index on chunk embedding
2024-12-19 00:43:45,617 - create vector index on chunk embedding
2024-12-19 00:43:45,638 - create vector index on chunk embedding
2024-12-19 00:43:45,656 - create vector index on chunk embedding
2024-12-19 00:43:45,683 - create vector index on chunk embedding
2024-12-19 00:43:45,698 - create vector index on chunk embedding
2024-12-19 00:43:45,720 - create vector index on chunk embedding
2024-12-19 00:43:45,736 - create vector index on chunk embedding
2024-12-19 00:43:45,753 - create vector index on chunk embedding
2024-12-19 00:43:45,778 - create vector index on chunk embedding
2024-12-19 00:43:45,791 - create vector index on chunk embedding
2024-12-19 00:43:45,802 - create vector index on chunk embedding
2024-12-19 00:43:45,819 - create vector index on chunk embedding
2024-12-19 00:43:45,831 - create vector index on chunk embedding
2024-12-19 00:43:45,853 - create vector index on chunk embedding
2024-12-19 00:43:45,891 - Time taken to update embedding in chunk node: 0.63 seconds
2024-12-19 00:43:45,892 - Get graph document list from models
2024-12-19 00:43:45,892 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-19 00:43:45,892 - Model created - Model Version: diffbot
2024-12-19 00:43:45,892 - Combining 6 chunks before sending request to LLM
2024-12-19 00:43:46,530 - Time taken to extract enitities from LLM Graph Builder: 0.64 seconds
2024-12-19 00:43:46,548 - Time taken to save graph document in neo4j: 0.02 seconds
2024-12-19 00:43:46,548 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-19 00:43:46,548 - Create HAS_ENTITY relationship between chunks and entities
2024-12-19 00:43:46,548 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-19 00:43:46,548 - Time taken 20 chunks processed upto 80 completed in 1.29 seconds for file name King_Kong_(1933_film)
2024-12-19 00:43:46,548 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 19, 0, 43, 46, 548604), 'processingTime': 7.97, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 80}}
2024-12-19 00:43:46,548 - Update source node properties
2024-12-19 00:43:46,556 - updating node and relationship count
2024-12-19 00:43:46,567 - Selected Chunks upto: 100
2024-12-19 00:43:46,569 - Value of is_cancelled : False
2024-12-19 00:43:46,569 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-19 00:43:46,569 - update embedding and vector index for chunks
2024-12-19 00:43:46,590 - create vector index on chunk embedding
2024-12-19 00:43:46,610 - create vector index on chunk embedding
2024-12-19 00:43:46,623 - create vector index on chunk embedding
2024-12-19 00:43:46,641 - create vector index on chunk embedding
2024-12-19 00:43:46,662 - create vector index on chunk embedding
2024-12-19 00:43:46,679 - create vector index on chunk embedding
2024-12-19 00:43:46,691 - create vector index on chunk embedding
2024-12-19 00:43:46,709 - create vector index on chunk embedding
2024-12-19 00:43:46,722 - create vector index on chunk embedding
2024-12-19 00:43:46,746 - create vector index on chunk embedding
2024-12-19 00:43:46,780 - create vector index on chunk embedding
2024-12-19 00:43:46,795 - create vector index on chunk embedding
2024-12-19 00:43:46,819 - create vector index on chunk embedding
2024-12-19 00:43:46,842 - create vector index on chunk embedding
2024-12-19 00:43:46,859 - create vector index on chunk embedding
2024-12-19 00:43:46,875 - create vector index on chunk embedding
2024-12-19 00:43:46,893 - create vector index on chunk embedding
2024-12-19 00:43:46,916 - create vector index on chunk embedding
2024-12-19 00:43:46,939 - create vector index on chunk embedding
2024-12-19 00:43:46,964 - create vector index on chunk embedding
2024-12-19 00:43:47,008 - Time taken to update embedding in chunk node: 0.44 seconds
2024-12-19 00:43:47,008 - Get graph document list from models
2024-12-19 00:43:47,008 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-19 00:43:47,008 - Model created - Model Version: diffbot
2024-12-19 00:43:47,008 - Combining 6 chunks before sending request to LLM
2024-12-19 00:43:47,405 - Time taken to extract enitities from LLM Graph Builder: 0.40 seconds
2024-12-19 00:43:47,445 - Time taken to save graph document in neo4j: 0.04 seconds
2024-12-19 00:43:47,445 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-19 00:43:47,445 - Create HAS_ENTITY relationship between chunks and entities
2024-12-19 00:43:47,445 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-19 00:43:47,445 - Time taken 20 chunks processed upto 100 completed in 0.88 seconds for file name King_Kong_(1933_film)
2024-12-19 00:43:47,445 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 19, 0, 43, 47, 445804), 'processingTime': 8.87, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 100}}
2024-12-19 00:43:47,445 - Update source node properties
2024-12-19 00:43:47,461 - updating node and relationship count
2024-12-19 00:43:47,473 - Selected Chunks upto: 120
2024-12-19 00:43:47,475 - Value of is_cancelled : False
2024-12-19 00:43:47,475 - embedding model:model_name='all-MiniLM-L6-v2' cache_folder=None model_kwargs={} encode_kwargs={} multi_process=False show_progress=False and dimesion:384
2024-12-19 00:43:47,475 - update embedding and vector index for chunks
2024-12-19 00:43:47,494 - create vector index on chunk embedding
2024-12-19 00:43:47,506 - create vector index on chunk embedding
2024-12-19 00:43:47,521 - create vector index on chunk embedding
2024-12-19 00:43:47,534 - create vector index on chunk embedding
2024-12-19 00:43:47,559 - create vector index on chunk embedding
2024-12-19 00:43:47,580 - create vector index on chunk embedding
2024-12-19 00:43:47,600 - create vector index on chunk embedding
2024-12-19 00:43:47,617 - create vector index on chunk embedding
2024-12-19 00:43:47,636 - create vector index on chunk embedding
2024-12-19 00:43:47,650 - create vector index on chunk embedding
2024-12-19 00:43:47,670 - create vector index on chunk embedding
2024-12-19 00:43:47,682 - create vector index on chunk embedding
2024-12-19 00:43:47,709 - create vector index on chunk embedding
2024-12-19 00:43:47,720 - create vector index on chunk embedding
2024-12-19 00:43:47,735 - create vector index on chunk embedding
2024-12-19 00:43:47,756 - create vector index on chunk embedding
2024-12-19 00:43:47,800 - Time taken to update embedding in chunk node: 0.33 seconds
2024-12-19 00:43:47,800 - Get graph document list from models
2024-12-19 00:43:47,800 - Model: LLM_MODEL_CONFIG_diffbot
2024-12-19 00:43:47,800 - Model created - Model Version: diffbot
2024-12-19 00:43:47,800 - Combining 6 chunks before sending request to LLM
2024-12-19 00:43:48,083 - Time taken to extract enitities from LLM Graph Builder: 0.28 seconds
2024-12-19 00:43:48,095 - Time taken to save graph document in neo4j: 0.01 seconds
2024-12-19 00:43:48,095 - creating list of chunks and graph documents in get_chunk_and_graphDocument func
2024-12-19 00:43:48,095 - Create HAS_ENTITY relationship between chunks and entities
2024-12-19 00:43:48,095 - Time taken to create relationship between chunk and entities: 0.00 seconds
2024-12-19 00:43:48,095 - Time taken 20 chunks processed upto 116 completed in 0.62 seconds for file name King_Kong_(1933_film)
2024-12-19 00:43:48,095 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'updatedAt': datetime.datetime(2024, 12, 19, 0, 43, 48, 95714), 'processingTime': 9.52, 'nodeCount': 0, 'relationshipCount': 0, 'processed_chunk': 116}}
2024-12-19 00:43:48,095 - Update source node properties
2024-12-19 00:43:48,102 - updating node and relationship count
2024-12-19 00:43:48,123 - Job Status at the end : Completed
2024-12-19 00:43:48,123 - Base Param value 1 : {'props': {'fileName': 'King_Kong_(1933_film)', 'status': 'Completed', 'processingTime': 9.55}}
2024-12-19 00:43:48,123 - Update source node properties
2024-12-19 00:43:48,131 - updating node and relationship count
2024-12-19 00:43:48,139 - Updated the nodeCount and relCount properties in Document node
2024-12-19 00:43:48,139 - file:King_Kong_(1933_film) extraction has been completed
2024-12-19 00:43:48,139 - Time taken to processing source function completed in 9.56 seconds for file name King_Kong_(1933_film)
2024-12-19 00:43:48,139 - Going for counting nodes and relationships in extract
2024-12-19 00:43:48,147 - updating node and relationship count
2024-12-19 00:43:48,155 - Nodes and Relationship Counts updated
2024-12-19 00:43:48,155 - counting completed in 0.02
[INFO]{'fileName': 'King_Kong_(1933_film)', 'nodeCount': 116, 'relationshipCount': 232, 'total_processing_time': 9.55, 'status': 'Completed', 'model': 'diffbot', 'success_count': 1, 'chunkNodeCount': 116, 'chunkRelCount': 232, 'entityNodeCount': 0, 'entityEntityRelCount': 0, 'communityNodeCount': 0, 'communityRelCount': 0, 'db_url': 'bolt://localhost:7687', 'api_name': 'extract', 'source_url': None, 'wiki_query': 'King_Kong_(1933_film)', 'source_type': 'Wikipedia', 'logging_time': '2024-12-19 08:43:48 UTC', 'elapsed_api_time': '9.59', 'userName': 'neo4j', 'database': 'neo4j', 'aws_access_key_id': None, 'gcs_bucket_name': None, 'gcs_bucket_folder': None, 'gcs_blob_filename': None, 'gcs_project_id': None, 'allowedNodes': 'Document,Session,Actor,Director,Genre,Movie,Person,User', 'allowedRelationship': 'SIMILAR,ACTED_IN,DIRECTED,IN_GENRE,RATED', 'language': 'en', 'retry_condition': 'delete_entities_and_start_from_beginning'}
2024-12-19 00:43:48,155 - extraction completed in 9.59 seconds for file name King_Kong_(1933_film)
INFO:     <IP_ADDRESS>:49974 - "POST /extract HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:49978 - "GET /update_extract_status/King_Kong_%281933_film%29?url=bolt://localhost:7687&userName=neo4j&password=cGFzc3dvcmQ=&database=neo4j HTTP/1.1" 200 OK
2024-12-19 00:43:48,480 -  SSE Client disconnected
2024-12-19 00:43:48,507 - update KNN graph
2024-12-19 00:43:48,642 - Updated KNN Graph
2024-12-19 00:43:48,647 - Use pytorch device_name: mps
2024-12-19 00:43:48,647 - Load pretrained SentenceTransformer: all-MiniLM-L6-v2
2024-12-19 00:43:49,964 - Embedding: Using Langchain HuggingFaceEmbeddings , Dimension:384
2024-12-19 00:43:49,965 - Starting the process of creating full-text indexes.
2024-12-19 00:43:49,978 - Database connectivity verified.
2024-12-19 00:43:49,978 - Creating a full-text index for type 'entities'.
2024-12-19 00:43:49,984 - Dropped existing index (if any) in 0.01 seconds.
2024-12-19 00:43:50,044 - Fetched labels in 0.06 seconds.
2024-12-19 00:43:50,047 - Created full-text index in 0.00 seconds.
2024-12-19 00:43:50,055 - Process completed in 0.08 seconds.
2024-12-19 00:43:50,055 - Full-text index for type 'entities' created successfully.
2024-12-19 00:43:50,055 - Creating a full-text index for type 'hybrid'.
2024-12-19 00:43:50,056 - Dropped existing index (if any) in 0.00 seconds.
2024-12-19 00:43:50,074 - Created full-text index in 0.02 seconds.
2024-12-19 00:43:50,080 - Process completed in 0.03 seconds.
2024-12-19 00:43:50,080 - Full-text index for type 'hybrid' created successfully.
2024-12-19 00:43:50,080 - Creating a vector index for type 'vector'.
2024-12-19 00:43:50,080 - Starting the process to create vector index.
2024-12-19 00:43:50,084 - Dropped existing index (if any) in 0.00 seconds.
2024-12-19 00:43:50,148 - Created vector index in 0.06 seconds.
2024-12-19 00:43:50,152 - Vector index for chunk created successfully.
2024-12-19 00:43:50,152 - Driver closed successfully.
2024-12-19 00:43:50,152 - Full-text and vector index creation process completed.
2024-12-19 00:43:50,153 - Full Text index created
INFO:     <IP_ADDRESS>:49974 - "POST /post_processing HTTP/1.1" 200 OK
2024-12-19 00:44:07,265 - Starting graph query process
2024-12-19 00:44:07,265 - Attempting to connect to the Neo4j database at bolt://localhost:7687
2024-12-19 00:44:07,266 - Connection successful
2024-12-19 00:44:07,266 - Executing query for documents: ['King_Kong_(1933_film)']
2024-12-19 00:44:07,479 - no of nodes : 51
2024-12-19 00:44:07,479 - no of relations : 100
2024-12-19 00:44:07,479 - Query process completed successfully
2024-12-19 00:44:07,479 - Closing connection for graph_query api
[INFO]{'api_name': 'graph_query', 'db_url': 'bolt://localhost:7687', 'userName': 'neo4j', 'database': 'neo4j', 'document_names': '["King_Kong_(1933_film)"]', 'logging_time': '2024-12-19 08:44:07 UTC', 'elapsed_api_time': '0.21'}
INFO:     <IP_ADDRESS>:50036 - "POST /graph_query HTTP/1.1" 200 OK

Thanks for sharing logs.
From logs it shows the entity creation has failed without any specific error , which is strange.
need to investigate this further.
meanwhile if you find any observation related to this issue, do let us know.

I insert an api key for diffbot got from https://app.diffbot.com/custom/ . Entities show up!!. However, it took 5 mins to generate graph for a 75 page PDF documents.


Why not develop this on Kitware's gitlab instance?

Is there any reason for not trying to upstream this?
There are discussions about having lua as a scripting language here: https://gitlab.kitware.com/cmake/cmake/issues/19863
cmakesl looks way more familiar to c/c++ users.

First of, thanks for pointing the Lua issue out.
Well, that's quite a long story. I'll try to summarize this up.

When I started thinking about implementing CMakeSL I knew that I can't just go to the CMake's gitlab and create issue like "I'd like to create a new language for CMake. It'll be like C++ and it'll be great". I wanted to start discussion with CMake developers when I have a good POC.
Like five months ago I 'released' a POC that can build a simple hello world application. I posted it on r/cpp, r/cmake and on CMake mailing list. There were discussions on reddit, but no response on the mailing list, so I thought that CMake devs are still to sceptical. I decided to continue project and implement it to the state where it can prove its usefulness. I decided to do this in two phases:
2.1 Implement it to the state that it's able to build abseil-cpp project.
2.2 Implement it to the state that it's able to build LLVM + Clang. After this, it'd be clear whether CMakeSL helps with maintaining big project with a lot of CMake code. I wanted to start a discussion after this.
Recently CMakeSL became advanced enough to build itself. Still not able to build abseil-cpp, though.
You created the issue.

To answer your question. Why it's not at the CMake gitlab? Simply because I don't know whether CMake developers think that CMakeSL is a thing and they want in the project. Additionally, I'm not sure whether we (me and everyone else) have enough knowledge to predict whether CMakeSL would help with writing CMakeLists etc. I mean I believie it would. But you know, I'm just a simple C++ dev.
Nevetheless, we'll find out. I'm going to add a comment about CMakeSL to the Lua issue, clean up CMakeSL repo (README, docs, examples etc.) and create an issue to discuss merging CMakeSL into CMake's codebase.

Thank you. I recently been given developer rights on CMake, and managed to get PCH and Unity builds in CMake 3.16, so I can say that one can get things into CMake.
You need to document, add tests, integrate into existing machinery.
If cmakesl can somehow reuse clang-format and clang-tidy it would be very, very nice. CMake has a great backwards compatibility story, but it doesn't have tools to migrate old code to new code. CMakesl could have this with a tool like clang-tidy.
On the other hand people want to have something declarative, but this won't happen by itself. Somebody needs to do the work. We all agree that CMake language scripting is a pain, all those uppercase strings which need to pass semantic value, and you always have to look into documentation to get things right.
If you have a way to have both scripting languages coexist, and a way to migrate from one to another, I don't think people will be against it.

Thanks for the tips.
clang-format already works. All *.cmsl files in this repo are formatted with it (with a style got from the .clang-format file).
Also, doxygen works with *.cmsl files under doc/ folder.
I haven't tried clang-tidy, though.
About declarative language. I too think that it's a good approach for simple stuff. But, I've talked with a couple of people and they say, that declarative language doesn't work so well in a really big project, when you need to do complicated stuff. You'd need to mix it with an imperative language at some point.
About mixing old CMake and CMakeSL, that's exactly what happens in this repo. As I said in the previous post, CMakeSL is able to build itself. googletest is used as a testing framework. It's cloned and its sources are directly in the repo, under external/googletest. Of course, its CMakeLists doesn't support CMakeSL, old CMakeLists.txt are present. In this repo, in root CMakeLists.cmsl, add_subdirectory("external/googletest") is called. So from CMakeSL script, you can add another subdirectory with and old CMakeLists.txt. The call is here.
It'll work the other way, which is, in an old CMakeLists.txt, adding subdirectory with a CMakeList.cmsl, but I've accidentally removed it (it was late, don't ask).
So basically you can migrate your project to CMakeSL in small steps, directory after directory.

That looks great. Another thing that CMakeSL can improve, is overloading. With CMake you can overload a macro and a function and call the _ name, which is the old name. But you can do it only once.
You cannot have find_package redefined in vcpkg, and then again in your own script.
Another point would be performance wise. No idea how CMakeSL performs. I guess I could give it a try at some point 😄

Thanks, there is a lot of stuff to introduce. We'll see whether there will be a chance to implement it (:
I've created the issue. It's here, if anyone is interested: https://gitlab.kitware.com/cmake/cmake/issues/19889


 The code compiles, but in the linking step, it fails with the below message, plus a bunch of undefined symbols:
ld: warning: ignoring file /Users/.../therustlib.a, building for macOS-arm64 but attempting to link with file built for macOS-x86_64

I've tried messing with Rust_CARGO_TARGET, but every time the build fails on a mismatch or another. I've even tried putting both the targets in that variable, but it doesn't seem like it's something that the corrosion accepts.
I've tried detecting the arch that we're currently compiling to with stuff like CMAKE_SYSTEM_PROCESSOR, but this seems to only be ever set to x86_64.

As far as I know cargo/rust doesn't support compiling fat binaries (unless this changed recently)

A possible solution I can see is compiling both versions separately, and then merging them with lipo. I think that's what the xcode generator is doing.
I've tried doing that manually for a test, and hardcoding the .a path in cmake, and it seems to have worked. Maybe corrosion can automate this?

s compiling both versions separately, and then merging them with lipo.

What is the scope here? Does this work for all rust artifacts corrosion supports, i.e. for staticlibs AND for cdylibs AND for bin targets? Your example seems to imply you only tested with staticlibs.
In principle it would be possible for corrosion to support this, but i'd have to give some more thought on how much refactoring is required to allow this. Currently, the assumption of "one target triple" is pretty baked in.
If you just want a quick solution for your own project, then you could probably just configure two new sub-cmake projects from your main cmake project, and add a rule in the main cmake project to use lipo to merge the artifacts from the sub cmake project.

Initially I just tested with staticlibs, but just now I tested with a new hello world project with bin, staticlib and cdylib by building them twice and invoking lipo on them and it seems to be working on all the artifact types. sh for cdylib:
cargo b --target x86_64-apple-darwin &&
cargo b --target aarch64-apple-darwin &&
lipo -create target/x86_64-apple-darwin/debug/libhello_world.dylib target/aarch64-apple-darwin/debug/libhello_world.dylib -output libhello_world_fat.dylib

file output:
% file libhello_world_fat.dylib
libhello_world_fat.dylib: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64:Mach-O 64-bit dynamically linked shared library arm64]
libhello_world_fat.dylib (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64
libhello_world_fat.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64

The syntax for lipo seems to be identical in all 3 cases, I just changed the input and output filenames and it seems to pick up what they are automagically.
So yes, it could work with all already supported crate types.
About your quick solution, could you please elaborate more please? What I understood right now was that I should do 2x corrosion_import_crate, and then do the lipo step myself, which sounded like a good workaround, but importing the same crate twice doesn't seem to be supported because "a target with the same name already exists".

Thanks for testing!

What I understood right now was that I should do 2x corrosion_import_crate

No that doesn't work, because corrosion currently can only import each target once.
What I had in mind was something like
add_custom_command(OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild/mylib.a"
     COMMAND ${CMAKE_COMMAND} -S "${CMAKE_CURRENT_SOURCE_DIR}/rust"
                 -B "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild"
                 -DRust_CARGO_TARGET=....
     COMMAND ${CMAKE_COMMAND} --build "${CMAKE_CURRENT_BINARY_DIR}/arm64-subbuild"
)

This custom command would be added once for each target and assumes that "${CMAKE_CURRENT_SOURCE_DIR}/rust" contains a CMakeLists.txt (with its own project() ) which adds corrosion and calls corrosion_import_crate().
In your main CMake files you could then add another custom command which DEPENDs on the output files of the commands, and calls lipo on them.

Found out you can also pass --target multiple times to cargo, and it will know how to build multiple arches at a time. In parallel even!

Nice find - do you happen to know starting with which Rust version this works?

Looks like 1.64.


fix: [#4414] JS Linux Functional Test pipeline failing

Fixes #4414
Description
This PR fixes the errors in the JS Linux Functional Test pipeline. It also applies some prettier fixes to the DirectLine tests.
The error TypeError: Cannot read property 'text' of undefined was fixed by upgrading the botSku from F0 to S1, as the bot was throwing an error related to the max transactions being exceeded.
We also had an authorization issue that was solved by upgrading the bot's Node version from 10 to 16.
Specific Changes

Updated the functionaltestbot/template/linux/template.json to upgrade the Node version to 16 and the botSku default value to S1.
Updated prettier issues in testing/functional/tests/directLine.test.js.

Testing
This image shows the testing pipeline successfully executed after the changes:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage remained the same at 84.656%

Files with Coverage Reduction
New Missed Lines
%

libraries/botbuilder-lg/src/templateExtensions.ts
1
84.48%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
20002

Relevant Lines:
22391

💛  - Coveralls


Fix null in path names

Fix null in path names.
Related to PR 9 of pappl-retrofit.

Thanks for the pull request, but you can do it in a much simpler way. Simply add one line near the beginning of the function (~ line 3066)
if (root == NULL)
  root = "";

instead of all these ifs, especially also with all the { ... } when there is only a single statement.


added PV collections

added PV pages and subpages for open data collections

Hi @Dorothyrono, @gmilcinski asked me to connect with you about reviewing this request to add Planetary Variables here, can you review, please?

on it

Should we merge it ?
or you can merge  it if you have the rights

@Dorothyrono I don't have the rights to merge it, but yes, I think this is good to go


README: Add Ruby library

wheretz is created and maintained by me. It might be not the smartest implementation, but it works and is used by some.
(Thanks for the work on this tool, BTW!)

Thanks for submitting!


Add instructions for acceptance tests

Lets make @rtyler happy! :smile:

This doesn't look like beer...

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests


The use_docker example fails on nokogiri install

Describe the bug
I'm using the example listed under the example/use_docker directory. I'm using ruby version 2.5.8 for executing this code. However, every time I run sls package in the example directory, I get an error with nokogiri. Any help or guidance in this would be awesome.

Serverless: Excluding development dependencies...
Serverless: Clearing previous build ruby layer build
Serverless: Installing gem using docker bundler
Serverless: [DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'test development'`, and stop using this flag
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory:
/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri
/var/lang/bin/ruby -I /var/lang/lib/ruby/site_ruby/2.5.0 -r
./siteconf20201030-1-7135cl.rb extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
Using mini_portile version 2.4.0
checking for gzdopen() in -lz... yes
checking for iconv... yes
************************************************************************
IMPORTANT NOTICE:

Building Nokogiri with a packaged version of libxml2-2.9.10
with the following patches applied:
        - 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
        - 0002-Remove-script-macro-support.patch
        - 0003-Update-entities-to-remove-handling-of-ssi.patch
        - 0004-libxml2.la-is-in-top_builddir.patch
        - 0005-Fix-infinite-loop-in-xmlStringLenDecodeEntities.patch

Team Nokogiri will keep on doing their best to provide security
updates in a timely manner, but if this is a concern for you and want
to use the system library instead; abort this installation process and
reinstall nokogiri as follows:

    gem install nokogiri -- --use-system-libraries
        [--with-xml2-config=/path/to/xml2-config]
        [--with-xslt-config=/path/to/xslt-config]

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

Note, however, that nokogiri is not fully compatible with arbitrary
versions of libxml2 provided by OS/package vendors.
************************************************************************
Extracting libxml2-2.9.10.tar.gz into
tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10... ERROR, review
'/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10/extract.log'
to see what happened.   Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/var/lang/bin/$(RUBY_BASE_NAME)
        --help
        --clean
        --use-system-libraries
        --enable-static
        --disable-static
        --with-zlib-dir
        --without-zlib-dir
        --with-zlib-include
        --without-zlib-include=${zlib-dir}/include
        --with-zlib-lib
        --without-zlib-lib=${zlib-dir}/lib
        --enable-cross-build
        --disable-cross-build
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:402:in
`block in execute': Failed to complete extract task (RuntimeError)
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
`chdir'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
`execute'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:367:in
`extract_file'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:62:in
`block in extract'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
`each'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
`extract'
from
/var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:151:in
`cook'
        from extconf.rb:365:in `block (2 levels) in process_recipe'
        from extconf.rb:257:in `block in chdir_for_build'
        from extconf.rb:256:in `chdir'
        from extconf.rb:256:in `chdir_for_build'
        from extconf.rb:364:in `block in process_recipe'
        from extconf.rb:262:in `tap'
        from extconf.rb:262:in `process_recipe'
        from extconf.rb:557:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can
be found here:

/var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in
/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10 for inspection.
Results logged to
/var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/gem_make.out

An error occurred while installing nokogiri (1.10.10), and Bundler cannot
continue.
Make sure that `gem install nokogiri -v '1.10.10' --source
'https://rubygems.org/'` succeeds before bundling.

In Gemfile:
  nokogiri

  Error --------------------------------------------------

  Error: [DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set without 'test development'`, and stop using this flag
  Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

  current directory:
  /var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri
  /var/lang/bin/ruby -I /var/lang/lib/ruby/site_ruby/2.5.0 -r
  ./siteconf20201030-1-7135cl.rb extconf.rb
  checking if the C compiler accepts ... yes
  Building nokogiri using packaged libraries.
  Using mini_portile version 2.4.0
  checking for gzdopen() in -lz... yes
  checking for iconv... yes
  ************************************************************************
  IMPORTANT NOTICE:

  Building Nokogiri with a packaged version of libxml2-2.9.10
  with the following patches applied:
        - 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
        - 0002-Remove-script-macro-support.patch
        - 0003-Update-entities-to-remove-handling-of-ssi.patch
        - 0004-libxml2.la-is-in-top_builddir.patch
        - 0005-Fix-infinite-loop-in-xmlStringLenDecodeEntities.patch

  Team Nokogiri will keep on doing their best to provide security
  updates in a timely manner, but if this is a concern for you and want
  to use the system library instead; abort this installation process and
  reinstall nokogiri as follows:

      gem install nokogiri -- --use-system-libraries
          [--with-xml2-config=/path/to/xml2-config]
          [--with-xslt-config=/path/to/xslt-config]

  If you are using Bundler, tell it to use the option:

      bundle config build.nokogiri --use-system-libraries
      bundle install

  Note, however, that nokogiri is not fully compatible with arbitrary
  versions of libxml2 provided by OS/package vendors.
  ************************************************************************
  Extracting libxml2-2.9.10.tar.gz into
  tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10... ERROR, review
  '/var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10/ext/nokogiri/tmp/x86_64-pc-linux-gnu/ports/libxml2/2.9.10/extract.log'
  to see what happened.   Check the mkmf.log file for more details.  You may
  need configuration options.

  Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/var/lang/bin/$(RUBY_BASE_NAME)
        --help
        --clean
        --use-system-libraries
        --enable-static
        --disable-static
        --with-zlib-dir
        --without-zlib-dir
        --with-zlib-include
        --without-zlib-include=${zlib-dir}/include
        --with-zlib-lib
        --without-zlib-lib=${zlib-dir}/lib
        --enable-cross-build
        --disable-cross-build
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:402:in
  `block in execute': Failed to complete extract task (RuntimeError)
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
  `chdir'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:373:in
  `execute'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:367:in
  `extract_file'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:62:in
  `block in extract'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
  `each'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:60:in
  `extract'
  from
  /var/gem_build/build/ruby/2.5.0/gems/mini_portile2-2.4.0/lib/mini_portile2/mini_portile.rb:151:in
  `cook'
        from extconf.rb:365:in `block (2 levels) in process_recipe'
        from extconf.rb:257:in `block in chdir_for_build'
        from extconf.rb:256:in `chdir'
        from extconf.rb:256:in `chdir_for_build'
        from extconf.rb:364:in `block in process_recipe'
        from extconf.rb:262:in `tap'
        from extconf.rb:262:in `process_recipe'
        from extconf.rb:557:in `<main>'

  To see why this extension failed to compile, please check the mkmf.log which can
  be found here:

  /var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/mkmf.log

  extconf failed, exit code 1

  Gem files will remain installed in
  /var/gem_build/build/ruby/2.5.0/gems/nokogiri-1.10.10 for inspection.
  Results logged to
  /var/gem_build/build/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/nokogiri-1.10.10/gem_make.out

  An error occurred while installing nokogiri (1.10.10), and Bundler cannot
  continue.
  Make sure that `gem install nokogiri -v '1.10.10' --source
  'https://rubygems.org/'` succeeds before bundling.

  In Gemfile:
    nokogiri

      at runCommand (/Users/arjun/open_source/serverless-ruby-layer/examples/use_docker/node_modules/serverless-ruby-layer/lib/bundle.js:23:11)
      at docker (/Users/arjun/open_source/serverless-ruby-layer/examples/use_docker/node_modules/serverless-ruby-layer/lib/bundle.js:29:12)
      at ServerlessRubyBundler.bundleInstall (/Users/arjun/open_source/serverless-ruby-layer/examples/use_docker/node_modules/serverless-ruby-layer/lib/bundle.js:100:5)
  From previous event:
      at PluginManager.invoke (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:483:22)
      at /usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:518:24
  From previous event:
      at PluginManager.run (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:518:8)
      at /usr/local/lib/node_modules/serverless/lib/Serverless.js:136:33
  From previous event:
      at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:123:74)
      at /usr/local/lib/node_modules/serverless/scripts/serverless.js:54:26
      at processImmediate (internal/timers.js:444:21)
      at process.topLevelDomainCallback (domain.js:134:15)
  From previous event:
      at Object.<anonymous> (/usr/local/lib/node_modules/serverless/scripts/serverless.js:54:4)
      at Module._compile (internal/modules/cjs/loader.js:1128:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
      at Module.load (internal/modules/cjs/loader.js:983:32)
      at Function.Module._load (internal/modules/cjs/loader.js:891:14)
      at Module.require (internal/modules/cjs/loader.js:1023:19)
      at require (internal/modules/cjs/helpers.js:72:18)
      at Object.<anonymous> (/usr/local/lib/node_modules/serverless/bin/serverless.js:41:1)
      at Module._compile (internal/modules/cjs/loader.js:1128:30)
      at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
      at Module.load (internal/modules/cjs/loader.js:983:32)
      at Function.Module._load (internal/modules/cjs/loader.js:891:14)
      at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
      at internal/main/run_main_module.js:17:47

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              13.3.0
     Framework Version:         1.78.1
     Plugin Version:            3.7.0
     SDK Version:               2.3.1
     Components Version:        2.34.3

Example Config files
I am only running the example folder.
Desktop (please complete the following information):

OS: [e.g. Mac OS]
NodeVersion v13.3.0
Ruby version 2.5.8
Bundler version 1.17.2

Thanks, Anand for reporting this issue. Is it a very old docker image? Can you pull the latest and try again,

Hey @navarasu thanks for replying. I tried doing what you suggested, but I'm still having the same issue.
➜  use_docker git:(master) ✗ docker image pull lambci/lambda:build-ruby2.5
build-ruby2.5: Pulling from lambci/lambda
Digest: sha256:26c1d3de7621d2bad3f439c45723f3604f0651d9c018aa64d269ef6326baa33a
Status: Image is up to date for lambci/lambda:build-ruby2.5
docker.io/lambci/lambda:build-ruby2.5

Interesting. Not sure why it is not working on your machine with the same docker image.
This is my docker version.
➜  ~ docker version
Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 16:58:31 2020
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Not sure the docker version is the issue. Try updating your docker as well.
Also, try removing the image (docker image rm -f lambci/lambda:build-ruby2.5) and run sls deploy again

Still getting the same error after I do that :-/ What version of serverless should I be using? I've tried 1.7 and also 2.9.0

@yodarjun , I tried reproducing the same in different machines. It works in all cases.
Alternatively, you can configure your own docker file. Refer pg gem example here. Similarly, you can add the necessary native library to fix nokogiri issue.

@yodarjun I think this might be an issue related to volume mount but not sure. Please update to the latest version 1.4.0 and verify again.


security/acme-client: Added support for ZONE DNS API

@manhofmann Merged (with minor adjustments), thanks!


fix: back button to fit content instead of full width

Description
Previously, the back button on the datasource listing page was full width and enabled back clicking for the entire surrounding area. The button has now been changed to accommodate the content's width.
Fixes #21823
Media

A video or a GIF is preferred. when using Loom, don’t embed because it looks like it’s a GIF. instead, just link to the video

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Chore (housekeeping or task changes that don't impact user perception)
This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions, so we can reproduce.
Please also list any relevant details for your test configuration.
Delete anything that is not important

Manual
Jest
Cypress

Test Plan

Add Testsmith test cases links that relate to this PR

Issues raised during DP testing

Link issues raised during DP testing for better visiblity and tracking (copy link from comments dropped on this PR)

Checklist:
Dev activity

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] PR is being merged under a feature flag

QA activity:

[ ] Test plan has been approved by relevant developers
[ ] Test plan has been peer reviewed by QA
[ ] Cypress test cases have been added and approved by either SDET or manual QA
[ ] Organized project review call with relevant stakeholders after Round 1/2 of QA
[ ] Added Test Plan Approved label after reveiwing all Cypress test

/ok-to-test sha=c190e2a

/build-deploy-preview

/ok-to-test sha=a6d844d

/ok-to-test sha=91dd8a8


[Automation] Generate Fluent Lite from Swagger networkcloud#package-2024-06-01-preview

[Automation] Generate Fluent Lite from Swagger networkcloud#package-2024-06-01-preview

API change check
APIView has identified API level changes in this PR and created following API reviews.
com.azure.resourcemanager:azure-resourcemanager-networkcloud


Don't use directives

You should remove the usage of directives and use composable for click away... for example: https://vueuse.org/core/onClickOutside/
They aren't working in SSR apps, and require custom transformers in build tools.
Vue documentation says:

We recommend to prefer using components as the abstraction mechanism instead of directives.

Done in v1.4.0


Performance concerns

pion/logging has been really useful for me, and using a lot for debugging.
@Sean-Der expressed his concern about its performance, and I'd agree, that is a concern...
For example, if you have this statement:
	logger.Debug("received data %s", pkt.String())

It looks alright, and it is just a debug log that only prints out when it is enabled with PION_LOG_DEBUG...
BUT! What if pkt.String() was an expensive routine? This is evaluated no matter what, consuming CPU time in production as well... Unfortunately, go has no compile-time options like C has (#ifdef etc.)
One obvious solution would be to use a callback like this:
	logger.Debug(func (p *LoggerPad) {
		p.Format("received data %s", pkt.String())
	})

The Debug function can be designed in a way that the callback function passed in it won't be called if it is not set to DEBUG level. You will need to type a lot more but it saves a lot of CPU time!

I miss #ifdefs so much..
IMHO I think we are mixing things, one thing is the logging level and the other one is how optimized is your build. For example, you may want to set log level to DEBUG in prod to be able to collect info about a problem, but not necessarily enable this big expensive computation that you don't need. Or, maybe you have an big expensive computation in case of an error (that you log as ERROR) but you choose to enable it only for non release builds.

In summary I think if we are going to perform some expensive operation that is not related to the actual functionality we should special case it, no matter the log level.

Maybe a bit OT but also related to performance. I've been playing with zerolog (https://github.com/rs/zerolog) and although we can adapt or factoring to zerolog, it doesn't really take advantage of the full zerolog perfomance just because the API is so different, for example, we do things like:

logger.Debugf("datachannel opened, id: %d", channel.ID())

while zerolog will:
logger.Debug().Int("datachannelID", channel.ID()).Msg("datachannel opened")

Since we avoid varargs we can keep args on stack which is really useful. I'm wondering if we should move to a zerolog like interface instead, because zerolog -> traditional log will still work, but not the other way around.

In the future we will move to https://pkg.go.dev/golang.org/x/exp/slog
The Go team has fixed/addressed the concerns in this thread https://pkg.go.dev/golang.org/x/exp/slog#hdr-Performance_considerations


add alpn support

Closes #85

Unlike my previous PR, this does not provide the alpn protocol list in the client hello. Is this intended?

This does not send the ALPN list by default. That was intended. It just adds support to detect if it was negotiated.


All events?

Any chance to get a list of all events?
How to read data from response? You do not explain it. Suddenly in your docs on('data' appears and without any explanation as well. I suppose there are gazillion events you just do not cover in docs. How may we know them?

The response is an instance of http.IncomingMessage which in turn implements stream.Readable which in turn emits data event.


Declaring the with-pdf plugin fails if GObjects is not present

My use case is as follow:

some users have an environment with GObjects (on Linux and on the CI) and want to generate the PDF
some users have an environment without GObjects (on Windows, and they cannot install it even if it's possible) and don't need to generate the PDF

But as the title says, declaring the with-pdf plugin in the mkdocs.yml fails if GObjects is not present. So it would be great if the GObjects import was done only when the PDF is generated.
Thanks

Related:
https://stackoverflow.com/questions/69067530/how-to-solve-weasyprint-error-message-gobject-2-0-0-error-0x7e-message
https://doc.courtbouillon.org/weasyprint/stable/first_steps.html#windows ("Only Windows 11 64-bit is supported")


Release Spring Vault 1.1.1

~Depends on Spring Framework 4.3.13~

Release required for Spring Cloud Edgware, releasing without waiting for Spring Framework 4.3.14