Update dependency azure-mgmt-authorization to v4

This PR contains the following updates:

Package
Type
Update
Change

azure-mgmt-authorization
dependencies
major
2.0.0 -> 4.0.0

Release Notes

Azure/azure-sdk-for-python (azure-mgmt-authorization)
v4.0.0
4.0.0 (2023-07-21)
Features Added

Added operation RoleAssignmentScheduleRequestsOperations.validate
Added operation RoleEligibilityScheduleRequestsOperations.validate
Model AlertConfiguration has a new parameter alert_definition
Model AlertConfigurationProperties has a new parameter alert_definition
Model AlertOperationResult has a new parameter created_date_time
Model AlertOperationResult has a new parameter last_action_date_time
Model AlertOperationResult has a new parameter resource_location
Model AlertOperationResult has a new parameter status_detail
Model AzureRolesAssignedOutsidePimAlertConfigurationProperties has a new parameter alert_definition
Model DenyAssignment has a new parameter condition
Model DenyAssignment has a new parameter condition_version
Model DenyAssignment has a new parameter created_by
Model DenyAssignment has a new parameter created_on
Model DenyAssignment has a new parameter updated_by
Model DenyAssignment has a new parameter updated_on
Model DuplicateRoleCreatedAlertConfigurationProperties has a new parameter alert_definition
Model Permission has a new parameter condition
Model Permission has a new parameter condition_version
Model RoleDefinition has a new parameter created_by
Model RoleDefinition has a new parameter created_on
Model RoleDefinition has a new parameter updated_by
Model RoleDefinition has a new parameter updated_on
Model TooManyOwnersAssignedToResourceAlertConfigurationProperties has a new parameter alert_definition
Model TooManyPermanentOwnersAssignedToResourceAlertConfigurationProperties has a new parameter alert_definition

Breaking Changes

Removed operation AlertOperationOperations.list_for_scope

v3.0.0
3.0.0 (2022-10-11)
Features Added

Added operation AccessReviewInstancesOperations.create
Added operation group AccessReviewHistoryDefinitionInstanceOperations
Added operation group AccessReviewHistoryDefinitionInstancesOperations
Added operation group AccessReviewHistoryDefinitionOperations
Added operation group AccessReviewHistoryDefinitionsOperations
Added operation group AccessReviewInstanceContactedReviewersOperations
Added operation group AlertConfigurationsOperations
Added operation group AlertDefinitionsOperations
Added operation group AlertIncidentsOperations
Added operation group AlertOperationOperations
Added operation group AlertsOperations
Added operation group ScopeAccessReviewDefaultSettingsOperations
Added operation group ScopeAccessReviewHistoryDefinitionInstanceOperations
Added operation group ScopeAccessReviewHistoryDefinitionInstancesOperations
Added operation group ScopeAccessReviewHistoryDefinitionOperations
Added operation group ScopeAccessReviewHistoryDefinitionsOperations
Added operation group ScopeAccessReviewInstanceContactedReviewersOperations
Added operation group ScopeAccessReviewInstanceDecisionsOperations
Added operation group ScopeAccessReviewInstanceOperations
Added operation group ScopeAccessReviewInstancesOperations
Added operation group ScopeAccessReviewScheduleDefinitionsOperations
Added operation group TenantLevelAccessReviewInstanceContactedReviewersOperations
Model AccessReviewDecision has a new parameter insights
Model AccessReviewDecision has a new parameter membership_types
Model AccessReviewDecisionProperties has a new parameter insights
Model AccessReviewDecisionProperties has a new parameter membership_types
Model AccessReviewDefaultSettings has a new parameter recommendation_look_back_duration
Model AccessReviewInstance has a new parameter backup_reviewers
Model AccessReviewInstance has a new parameter reviewers
Model AccessReviewInstance has a new parameter reviewers_type
Model AccessReviewScheduleDefinition has a new parameter exclude_resource_id
Model AccessReviewScheduleDefinition has a new parameter exclude_role_definition_id
Model AccessReviewScheduleDefinition has a new parameter expand_nested_memberships
Model AccessReviewScheduleDefinition has a new parameter include_access_below_resource
Model AccessReviewScheduleDefinition has a new parameter include_inherited_access
Model AccessReviewScheduleDefinition has a new parameter recommendation_look_back_duration
Model AccessReviewScheduleDefinitionProperties has a new parameter exclude_resource_id
Model AccessReviewScheduleDefinitionProperties has a new parameter exclude_role_definition_id
Model AccessReviewScheduleDefinitionProperties has a new parameter expand_nested_memberships
Model AccessReviewScheduleDefinitionProperties has a new parameter include_access_below_resource
Model AccessReviewScheduleDefinitionProperties has a new parameter include_inherited_access
Model AccessReviewScheduleDefinitionProperties has a new parameter recommendation_look_back_duration
Model AccessReviewScheduleSettings has a new parameter recommendation_look_back_duration
Model DenyAssignmentPermission has a new parameter condition
Model DenyAssignmentPermission has a new parameter condition_version

Breaking Changes

Operation RoleAssignmentsOperations.list_for_scope has a new parameter skip_token
Removed operation RoleAssignmentsOperations.validate
Removed operation RoleAssignmentsOperations.validate_by_id

Configuration
📅 Schedule: Branch creation - "before 4am" (UTC), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

⚠ Artifact update problem
Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.
♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

any of the package files in this branch needs updating, or
the branch becomes conflicted, or
you click the rebase/retry checkbox if found above, or
you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:
File name: poetry.lock
Updating dependencies
Resolving dependencies...

Because no versions of azure-mgmt-core match >1.3.2,<1.4.0 || >1.4.0,<2.0.0
 and azure-mgmt-core (1.3.2) depends on azure-core (>=1.24.0,<2.0.0), azure-mgmt-core (>=1.3.2,<1.4.0 || >1.4.0,<2.0.0) requires azure-core (>=1.24.0,<2.0.0).
And because azure-mgmt-core (1.4.0) depends on azure-core (>=1.26.2,<2.0.0)
 and azure-mgmt-authorization (4.0.0) depends on azure-mgmt-core (>=1.3.2,<2.0.0), azure-mgmt-authorization (4.0.0) requires azure-core (>=1.24.0,<2.0.0).
So, because infra-thunder depends on both azure-core (1.19.0) and azure-mgmt-authorization (4.0.0), version solving failed.


[SUPPORT] Large keyspace upsert results in OOM

Describe the problem you faced
Hi Folks. I'm trying to get some advice here on how to better deal with a large upsert dataset.
The data has a very wide key space and no great/obvious partitionable columns. What I'm aiming for is a record key with roughly 1.5 billion unique values. Any one of those records could be updated at any time. The pipeline so far is a pyspark based streaming application. The eventual source will be a kafka topic, but at the moment it's using a datagenerator set to emit 10000 rows per second (specifically it's the dbldatagen python module). The sink is an S3 bucket with a MOR hudi table. The spark cluster is hosted under kubernetes on AWS. I'm using the latest hudi beta, the latest production spark (3.5.2), hive metastore 3.x and zookeeper for locking (latest production version). The executors are currently set to 10 at 8GB ram and 1 core each. And while I could feasibly increase the ram on the executors, I had previously done this (from 4 to 8) and it didn't buy me much. I seem to be topped out at roughly 500 million records (and 500 million distinct values on the record key).
The issue is that while I can get the pipeline to run, eventually it gets to a point where it simply OOM's on the executors during a union on the upsert. I've tried numerous memory / parallelism tuning options to no avail.
Is there a way to make this work without infinitely scaling memory? Is a wide keyspace like this a dead end? I did try partitioning by applying an md5 to the record key and using the 1st two characters. That gave me 256 unique more-or-less evenly distributed partitions. But that led to drastically increased batch times.
Record level indexing is enabled. That seems to help things on the write path. On the read path I've noticed that when data skipping is enabled queries actually take significantly longer. So perhaps my index has grown unmanageable? Maybe this is partially responsible for the OOM's on the write side.
To Reproduce
Steps to reproduce the behavior:
hudi config:
{
    "hoodie.insert.shuffle.parallelism": "500",
    "hoodie.upsert.shuffle.parallelism": "500",
    "hoodie.delete.shuffle.parallelism": "500",
    "hoodie.bulkinsert.shuffle.parallelism": "500",
    "hoodie.clean.commits.retained": 5,
    "hoodie.clean.policy": "KEEP_LATEST_COMMITS",
    "hoodie.clean.fileversions.retained": "1",
    "hoodie.keep.min.commits": "10",
    "hoodie.datasource.compaction.async.enable": "true",
    "hoodie.logfile.data.block.max.size": 8388608,
    "hoodie.logfile.max.size": 8388608,
    "hoodie.datasource.hive_sync.db": "default",
    "hoodie.datasource.hive_sync.enable": true,
    "hoodie.datasource.hive_sync.ignore_exceptions": true,
    "hoodie.datasource.hive_sync.jdbcurl": "jdbc:hive2://hive-metastore.default:10000",
    "hoodie.datasource.hive_sync.metastore.uris": "thrift://hive-metastore.default:9083",
    "hoodie.datasource.hive_sync.mode": "hms",
    "hoodie.datasource.hive_sync.partition_extractor_class": "org.apache.hudi.hive.NonPartitionedExtractor",
    "hoodie.datasource.hive_sync.password": "",
    "hoodie.datasource.hive_sync.recreate_table_on_error": true,
    "hoodie.datasource.hive_sync.skip_ro_suffix": true,
    "hoodie.datasource.hive_sync.table": "hudi_sink",
    "hoodie.datasource.hive_sync.username": "hive",
    "hoodie.archive.async": true,
    "hoodie.clean.async.enabled": true,
    "hoodie.clean.automatic": true,
    "hoodie.cleaner.policy.failed.writes": "LAZY",
    "hoodie.clustering.async.enabled": false,
    "hoodie.clustering.async.max.commits": 0,
    "hoodie.clustering.execution.strategy.class": "org.apache.hudi.client.clustering.run.strategy.SparkSortAndSizeExecutionStrategy",
    "hoodie.clustering.inline": true,
    "hoodie.clustering.inline.max.commits": 0,
    "hoodie.clustering.plan.strategy.class": "org.apache.hudi.client.clustering.plan.strategy.SparkSizeBasedClusteringPlanStrategy",
    "hoodie.clustering.plan.strategy.sort.columns": "name",
    "hoodie.clustering.preserve.commit.metadata": true,
    "hoodie.clustering.rollback.pending.replacecommit.on.conflict": true,
    "hoodie.clustering.updates.strategy": "org.apache.hudi.client.clustering.update.strategy.SparkAllowUpdateStrategy",
    "hoodie.compact.inline.max.delta.commits": 2,
    "hoodie.datasource.meta.sync.enable": "true",
    "hoodie.datasource.read.incr.fallback.fulltablescan.enable": true,
    "hoodie.datasource.read.use.new.parquet.file.format": true,
    "hoodie.datasource.write.hive_style_partitioning": "true",
    "hoodie.datasource.write.operation": "upsert",
    "hoodie.datasource.write.precombine.field": "timestamp",
    "hoodie.datasource.write.record.merger.impls": "org.apache.hudi.HoodieSparkRecordMerger",
    "hoodie.datasource.write.recordkey.field": "name",
    "hoodie.datasource.write.row.writer.enable": true,
    "hoodie.datasource.write.streaming.checkpoint.identifier": "datagen-writer1",
    "hoodie.datasource.write.table.name": "hudi_sink",
    "hoodie.datasource.write.table.type": "MERGE_ON_READ",
    "hoodie.enable.data.skipping": true,
    "hoodie.index.type": "RECORD_INDEX",
    "hoodie.logfile.data.block.format": "parquet",
    "hoodie.merge.use.record.positions": true,
    "hoodie.metadata.auto.initialize": true,
    "hoodie.metadata.enable": true,
    "hoodie.metadata.clean.async": true,
    "hoodie.metadata.index.async": false,
    "hoodie.metadata.index.column.stats.columns": "name",
    "hoodie.metadata.index.column.stats.column.list": "name",
    "hoodie.metadata.index.column.stats.enable": true,
    "hoodie.metadata.record.index.enable": true,
    "hoodie.parquet.compression.codec": "snappy",
    "hoodie.record.index.use.caching": true,
    "hoodie.table.name": "hudi_sink",
    "hoodie.table.services.enabled": true,
    "hoodie.write.concurrency.early.conflict.detection.enable": true,
    "hoodie.write.concurrency.mode": "optimistic_concurrency_control",
    "hoodie.write.executor.type": "DISRUPTOR",
    "hoodie.write.lock.provider": "org.apache.hudi.client.transaction.lock.ZookeeperBasedLockProvider",
    "hoodie.write.lock.zookeeper.base_path": "/hudi/locl/table",
    "hoodie.write.lock.zookeeper.lock_key": "hudi_sink",
    "hoodie.write.lock.zookeeper.port": "2181",
    "hoodie.write.lock.zookeeper.url": "zk-cs.default",
    "checkpointLocation": "s3a://XXXXX/hudi_checkpoints",
    "path": "s3a://XXXXX/hudi_out_data"
}

spark config:
spark.checkpoint.compress true
spark.driver.maxResultSize 2g
spark.dynamicAllocation.shuffleTracking.enabled true
spark.eventLog.dir s3a://XXXXX/spark_logs/
spark.eventLog.enabled true
spark.eventLog.rolling.enabled true
spark.eventLog.rolling.maxFileSize 20m
spark.executor.memoryOverhead 2g
spark.hadoop.fs.s3a.aws.credentials.provider com.amazonaws.auth.WebIdentityTokenCredentialsProvider
spark.hadoop.fs.s3a.fast.upload true
spark.hadoop.fs.s3a.impl org.apache.hadoop.fs.s3a.S3AFileSystem
spark.hadoop.fs.s3a.path.style.access true
spark.hadoop.hive.metastore.warehouse.dir s3a://XXXXX/warehouse
spark.hadoop.parquet.avro.write-old-list-structure false
spark.history.fs.logDirectory s3a://XXXXX/spark_logs/
spark.hive.metastore.warehouse.dir s3a://XXXXX/warehouse
spark.io.compression.codec snappy
spark.kryo.registrator org.apache.spark.HoodieSparkKryoRegistrar
spark.memory.fraction 0.2
spark.memory.storageFraction 0.2
spark.rdd.compress true
spark.serializer org.apache.spark.serializer.KryoSerializer
spark.shuffle.service.enabled false # leave disabled as kubernetes doesn't support exrernal shuffle service
spark.speculation false
spark.streaming.stopGracefullyOnShutdown true
spark.sql.catalog.spark_catalog org.apache.spark.sql.hudi.catalog.HoodieCatalog
spark.sql.catalogImplementation hive
spark.sql.extensions org.apache.spark.sql.hudi.HoodieSparkSessionExtension
spark.sql.hive.convertMetastoreParquet false
spark.sql.hive.metastore.jars path
spark.sql.hive.metastore.jars.path file:///opt/apache-hive-3.1.3-bin/lib/*.jar
spark.sql.hive.metastore.version 3.1.3
spark.sql.parquet.enableVectorizedReader true
spark.sql.shuffle.partitions 500
spark.sql.streaming.stateStore.compression.codec snappy
spark.sql.streaming.stateStore.providerClass org.apache.spark.sql.execution.streaming.state.RocksDBStateStoreProvider
spark.sql.warehouse.dir s3a://XXXXX/warehouse
spark.ui.prometheus.enabled true
#
# S3 optimization settings (see https://spark.apache.org/docs/latest/cloud-integration.html):
#
spark.hadoop.fs.s3a.committer.name directory
spark.hadoop.parquet.enable.summary-metadata false
spark.sql.hive.metastorePartitionPruning true
spark.sql.orc.cache.stripe.details.size 10000
spark.sql.orc.filterPushdown true
spark.sql.orc.splits.include.file.footer true
spark.sql.parquet.filterPushdown true
spark.sql.parquet.mergeSchema false
spark.sql.parquet.output.committer.class org.apache.spark.internal.io.cloud.BindingParquetOutputCommitter
spark.sql.sources.commitProtocolClass org.apache.spark.internal.io.cloud.PathOutputCommitProtocol
spark.sql.streaming.checkpointFileManagerClass org.apache.spark.internal.io.cloud.AbortableStreamBasedCheckpointFileManager

Expected behavior
Pipeline should run without OOM
Environment Description

Hudi version : 1.0.0-beta2

Spark version : 3.5.2

Hive version : 3.1.3

Hadoop version : 3.3.4

Storage (HDFS/S3/GCS..) : S3

Running on Docker? (yes/no) : yes

Additional context
Add any other context about the problem here.
Stacktrace
│ 24/09/18 18:10:29 WARN TaskSetManager: Lost task 5.0 in stage 68.0 (TID 3753) (<IP_ADDRESS> executor 8): java.lang.OutOfMemoryError: Java heap space                         │
│     at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:64)                                                                                │
│     at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:363)                                                                                │
│     at org.apache.spark.storage.memory.SerializedValuesHolder.$anonfun$allocator$1(MemoryStore.scala:715)                                                                    │
│     at org.apache.spark.storage.memory.SerializedValuesHolder.$anonfun$allocator$1$adapted(MemoryStore.scala:715)                                                            │
│     at org.apache.spark.storage.memory.SerializedValuesHolder$$Lambda$1216/0x00007995a8a1fa28.apply(Unknown Source)                                                          │
│     at org.apache.spark.util.io.ChunkedByteBufferOutputStream.allocateNewChunkIfNeeded(ChunkedByteBufferOutputStream.scala:87)                                               │
│     at org.apache.spark.util.io.ChunkedByteBufferOutputStream.write(ChunkedByteBufferOutputStream.scala:75)                                                                  │
│     at org.apache.spark.storage.memory.RedirectableOutputStream.write(MemoryStore.scala:809)                                                                                │
│     at net.jpountz.lz4.LZ4BlockOutputStream.flushBufferedData(LZ4BlockOutputStream.java:225)                                                                                │
│     at net.jpountz.lz4.LZ4BlockOutputStream.write(LZ4BlockOutputStream.java:178)                                                                                │
│     at com.esotericsoftware.kryo.io.Output.flush(Output.java:185)                                                                                │
│     at com.esotericsoftware.kryo.io.Output.require(Output.java:164)                                                                                │
│     at com.esotericsoftware.kryo.io.UnsafeOutput.writeLong(UnsafeOutput.java:124)                                                                                │
│     at com.esotericsoftware.kryo.io.UnsafeOutput.writeLong(UnsafeOutput.java:160)                                                                                │
│     at com.esotericsoftware.kryo.serializers.DefaultSerializers$LongSerializer.write(DefaultSerializers.java:151)                                                            │
│     at com.esotericsoftware.kryo.serializers.DefaultSerializers$LongSerializer.write(DefaultSerializers.java:145)                                                            │
│     at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)                                                                                │
│     at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:86)                                                                                │
│     at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)                                                                                │
│     at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:575)                                                                                │
│     at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:79)                                                                                │
│     at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)                                                                                │
│     at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)                                                                                │
│     at org.apache.hudi.common.model.HoodieAvroRecord.writeRecordPayload(HoodieAvroRecord.java:225)                                                                           │
│     at org.apache.hudi.common.model.HoodieAvroRecord.writeRecordPayload(HoodieAvroRecord.java:48)                                                                            │
│     at org.apache.hudi.common.model.HoodieRecord.write(HoodieRecord.java:365)                                                                                │
│     at com.esotericsoftware.kryo.serializers.DefaultSerializers$KryoSerializableSerializer.write(DefaultSerializers.java:514)                                                │
│     at com.esotericsoftware.kryo.serializers.DefaultSerializers$KryoSerializableSerializer.write(DefaultSerializers.java:512)                                                │
│     at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)                                                                                │
│     at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:278)                                                                             │
│     at org.apache.spark.storage.memory.SerializedValuesHolder.storeValue(MemoryStore.scala:729)                                                                              │
│     at org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:224)

Any help / tips would be greatly appreciated!

I guess the huge resource cost comes from the rewrite of the parquets, because each update touches almost all the file groups, it will trigger a whole table rewrite.
If we have some range partitioning strategy that can limit the scope of updates instead of whole table, would be helpful.

I can certainly attempt partitioning again, but doesn't that just exacerbate the file group problem? My last attempt at partitioning made the batches take waaaaaaaay too long.
If the writing to many files groups truly is the problem, then why does the process go OOM instead of simply taking longer? Is seems that this is something that could be parallelized and sped up. But in my trial increasing cores did not help. Is there a way I can configure this to not use as much ram?

Is the OOM happening in Driver or executor? The driver could takes large amout of memory for the fs view file index, the solution to ease it to enable the rockdb-based fs view. The executor could OOM when the parquet and delta log records take memory, that would depend on your parallelism of the executors.

The OOM is happening on the executors. What would you recommend for parallelism? It's already set at 500.

@meatheadmike Can you please share the spark event logs ?

sparklog.txt
Here you go. I captured the most recent run. I stopped it before the driver completely quit, but you can see where the executors start OOMing.
From the UI perspective it's happening here:
Job Group: SparkUpsertPreppedDeltaCommitActionExecutor
Building workload profile:hudi_sink_metadata
[(kill)](http://localhost:4040/stages/stage/kill/?id=77)[start at NativeMethodAccessorImpl.java:0](http://localhost:4040/stages/stage/?id=77&attempt=1)+details
RDD: [UnionRDD](http://localhost:4040/storage/rdd/?id=162)[UnionRDD](http://localhost:4040/storage/rdd/?id=180)

Update: Bumping driver and executors to 16GB of memory allowed me to get to 2.5 billion rows. So the OOM is possible to overcome with more memory although I'd certainly prefer to run with less memory and more executors.

@meatheadmike Were you able to check on the same? Do you still face the issue?

Thanks for the update @meatheadmike .


Fix for case, when there are no icons

Fix for case, when there are no icons, and count is displayed, as '1 - 0 of 0 '

Thanks for fix case!


Milestone 4 - Subtask 3: Finetune model with Q/A format dataset

Potential Data Format that gives courses to take and reasoning. Will be replace with LPI Dataset courses/infoy
input: I want to learn about llm and how to finetune them. I'm intermediate  and i want to build a rag pipeline
output: Given your intermediate level and interest in building a RAG pipeline using LLMs, I recommend the following courses and modules:

Course/Module
Source
Level
Duration (Estimate)
Keywords
Reason

Introduction to Large Language Models (LLMs)
Various online courses (e.g., Coursera, edX)
Beginner/Intermediate
Varies (4-8 weeks)
LLMs, transformers, attention mechanisms, language modeling, tokenization
Provides foundational knowledge of LLMs, their architecture, and capabilities. Essential before tackling fine-tuning or RAG.

Natural Language Processing (NLP) Fundamentals
Various online courses (e.g., Stanford NLP)
Intermediate
Varies (6-10 weeks)
NLP, text preprocessing, word embeddings, sentiment analysis, named entity recognition
Necessary for understanding how LLMs process and understand text. Many RAG techniques rely on NLP for data preprocessing and query understanding.

Fine-tuning LLMs
Hugging Face Course, Papers with Code
Intermediate/Advanced
Varies (2-4 weeks)
Fine-tuning, transfer learning, hyperparameter tuning, model evaluation
Teaches you how to adapt pre-trained LLMs to specific tasks, crucial for building a high-performing RAG system. Learn techniques like prompt engineering.

Retrieval Augmented Generation (RAG) Techniques
Research Papers, Blogs, Tutorials
Advanced
Varies (Ongoing Study)
RAG, vector databases, embedding generation, knowledge retrieval, question answering
Focuses on the architecture and implementation of RAG pipelines. You'll learn to select and integrate components like vector databases (e.g., Pinecone, Weaviate) and retrieval methods.

Python for Data Science (if needed)
DataCamp, Codecademy, Fast.ai
Intermediate
Varies (2-4 weeks)
Python, pandas, numpy, scikit-learn
Reinforce your Python skills for data manipulation and model building within your RAG pipeline.

Vector Databases (if needed)
Pinecone, Weaviate documentation
Intermediate
Varies (1-2 weeks)
Vector databases, similarity search, indexing, scalability
Understanding vector databases is crucial for efficient knowledge retrieval in a RAG pipeline. Learn about different databases and their strengths.

Potential Data Format that gives courses to take and reasoning. Will be replace with LPI Dataset
input: I want to learn about llm and how to finetune them. I'm intermediate  and i want to build a rag pipeline
output: Given your intermediate level and interest in building a RAG pipeline using LLMs, I recommend the following courses and modules:

Course/Module
Source
Level
Duration (Estimate)
Keywords
Reason

Introduction to Large Language Models (LLMs)
Various online courses (e.g., Coursera, edX)
Beginner/Intermediate
Varies (4-8 weeks)
LLMs, transformers, attention mechanisms, language modeling, tokenization
Provides foundational knowledge of LLMs, their architecture, and capabilities. Essential before tackling fine-tuning or RAG.

Natural Language Processing (NLP) Fundamentals
Various online courses (e.g., Stanford NLP)
Intermediate
Varies (6-10 weeks)
NLP, text preprocessing, word embeddings, sentiment analysis, named entity recognition
Necessary for understanding how LLMs process and understand text. Many RAG techniques rely on NLP for data preprocessing and query understanding.

Fine-tuning LLMs
Hugging Face Course, Papers with Code
Intermediate/Advanced
Varies (2-4 weeks)
Fine-tuning, transfer learning, hyperparameter tuning, model evaluation
Teaches you how to adapt pre-trained LLMs to specific tasks, crucial for building a high-performing RAG system. Learn techniques like prompt engineering.

Retrieval Augmented Generation (RAG) Techniques
Research Papers, Blogs, Tutorials
Advanced
Varies (Ongoing Study)
RAG, vector databases, embedding generation, knowledge retrieval, question answering
Focuses on the architecture and implementation of RAG pipelines. You'll learn to select and integrate components like vector databases (e.g., Pinecone, Weaviate) and retrieval methods.

Python for Data Science (if needed)
DataCamp, Codecademy, Fast.ai
Intermediate
Varies (2-4 weeks)
Python, pandas, numpy, scikit-learn
Reinforce your Python skills for data manipulation and model building within your RAG pipeline.

Vector Databases (if needed)
Pinecone, Weaviate documentation
Intermediate
Varies (1-2 weeks)
Vector databases, similarity search, indexing, scalability
Understanding vector databases is crucial for efficient knowledge retrieval in a RAG pipeline. Learn about different databases and their strengths.

Description moved to #108 (comment)

As discussed on Discord, you create a new dataset based on the LPI dataset and use that for your training and other purposes.
LPI dataset is still needed for our other purposes, plus it holds metadata about the courses which is added information (if can or need to be used).
Can you pls amend your wordings to remove any confusions to others who are also referring or reading this issue.

@Tacoman99 just moved the ticket to In Progress and also added it to the LPI project


Npm Run Dev not working with laravel 6

$ npm run dev

@ dev C:\xampp\htdocs\laravel6
npm run development

'npm' is not recognized as an internal or external command,
operable program or batch file.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ dev: npm run development
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:

probably problem with your npm installation, try downloading and installing again. https://nodejs.org/en/ or add the path C:\Program Files\nodejs to your OS Environment Variables.

Solved By Running
npm run dev
as administrator in cmd

Sounds like your environment variables were only set for the admin user account when you installed it.
You can change it for your normal user account by going to Control Panel > User Accounts then clicking "Change my environment variables" on the left-hand side.


working suggestion box for ayahs but i want it disabled on the surah …

…page; the css is inline and the look is not so good, i still need to make a css file; i also need to feed in surah name suggestions somehow;
@mmahalwy i'd like your feedback, if you can check this out and tell me what needs to happen

@mmahalwy both branches (here and in -api) are in a "done" stage and ready for review now. the only outstanding issue I see for the time being is that I want the autocomplete component to not be included if we're on the surah page. how do I do that?

I'd wait till #221 is done.

For some reason eslint is not running, otherwise it'd be very mad at you. We need to keep the spacing consistent. So, for example, someFn( function() {} ); is not allowed. should be someFn(function() {});. This is also the case for if statements.


chore!: cherry pick revert dynamic commits

Overview
leaving as a draft until we also get the commit that reverts overrides
Checklist

[x] New and updated code has appropriate documentation
[x] New and updated code has new and/or updated testing
[x] Required CI checks are passing
[x] Visual proof for any user facing features like CLI or documentation updates
[ ] Linked issues closed with keywords

Codecov Report

:exclamation: No coverage uploaded for pull request base (v1.x@4bbcbcb). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             v1.x    #2081   +/-   ##
=======================================
  Coverage        ?   21.60%
=======================================
  Files           ?      126
  Lines           ?    14252
  Branches        ?        0
=======================================
  Hits            ?     3079
  Misses          ?    10881
  Partials        ?      292


Support for language subdirectories

Hello,
Firstly I would like to say that your app is working great and the JS load on our site has decreased a lot thanks to you!
So for the problem. We have a multilingual site, where a subdirectory url style is used, so like domain.com/fr/. The scripts blocked unfortunaly show up on those sites while being successfully blocked on the regular non-language subdirectory version (except for the search app which is successfully blocked on both versions). I don't know how complex this would be to implement but it surely would help us :)
Thanks again for your great work!

Hi @andreasgloss
Thank you for the suggestion.
I see the use case here.
Technically this should be doable as the multilingual route can be detected.
Please let me see if this is something that can go live in the next verson


[Entity] separate jwks

At this moment in the entity configuration model we have a single attribute called jwks, that's a list (array) on many jwk.
this is isued for both federation and oidc core operation.
this is not good. We'll implement the best practice to have the oidc core jwk separated from the oidc fed one.
the jwks attribute will be separated in
jwks_core
jwks_fed
to distinguish their scopes.
nothing prevent to share the same jwk for bot oidc fed and core, but by default it won't be so

Addes some warning messages on invalid operations on disalinged jwks
https://github.com/italia/spid-cie-oidc-django/commit/c4722adebbe07e16b326ca48a08a35fa7bdea27e
https://github.com/italia/spid-cie-oidc-django/commit/5e14c320a1c8f0041267935940bda78af5d099cf

Done in v0.7.0.
https://github.com/italia/spid-cie-oidc-django/releases/tag/v0.7.0


Update indiv-link.component.html DOM text reinterpreted as HTML

By using innerText, it will avoid the risk of HTML injection, as these properties automatically escape any HTML special characters in the provided text. This helps prevent cross-site scripting (XSS) vulnerabilities by treating the input as plain text rather than interpreted HTML.

Hi @alifradityar Could You Please Review This PR
Thanks

Hi @alifradityar Could You Please Review This PR
Thanks

Hi @alifradityar Could You Please Review This PR
Thanks


Fix validation rules for task creation

Your checklist for this pull request
🚨Please review the guidelines for contributing to this repository.

[ ] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). Don't request your master!
[ ] Make sure you are making a pull request against the dev branch. Also you should start your branch off dev branch.
[ ] Check you have included issue number in your pr (e.g #23).
[ ] Check you have added tests that prove your fix is effective or that your feature works.
[ ] Check your code additions will fail neither code linting checks nor unit test.
[ ] Check you added any notable changes to the CHANGELOG.md file.

Description
Please describe your pull request.
💔Thank you!

Codecov Report

Merging #406 into dev will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                dev     #406   +/-   ##
=========================================
  Coverage     82.31%   82.31%
  Complexity      321      321
=========================================
  Files           106      106
  Lines          1018     1018
=========================================
  Hits            838      838
  Misses          180      180

Impacted Files
Coverage Δ
Complexity Δ

app/Http/Requests/ValidateTaskCreation.php
100% <ø> (ø)
2 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 342f5c3...6dadd1f. Read the comment docs.

@djaca  the previous validation rules are correct and what is intended so closing this pr.

Try to create task without assigned_to... Task is created, but can not open created task...

Then the bug is on frontend, it needs to be fixed there

assigned_to field is current nullable, but must me required because in repository you have 'assigned_to'       => $data['assigned_to'],

okay, then it should be $data['assigned_to'] ?? null

And notes, it forces to be string, but it should be optional...


Don't support change of height

Moved from https://github.com/mui-org/material-ui/pull/27439#discussion_r684940510
First I just wanted to move the derived state to render time when I realized that this is just controlled vs uncontrolled components. Mixing these concepts in a single component makes it harder to reason about than necessary.
It seems like we can just use the default* pattern instead until we understand why a MasonryItem needs to have a controlled height while responding to resizes. It's not obvious which value should win.
If people do want to change the initial height due to a different item (e.g. image) being displayed then they can use key to get the previous behavior. This also seems more intuitive assuming you do understand the key prop in React.

This totally makes sense. I agree.


avx2 doesn't work, segmentation fault, sse4 works, ( intel & amd )

tried this on both i7 6700 and ryzen7 1700 both systems with more than 32gb free ram..
compiled from source on ubuntu linux 64 bit,
Using AVX2 core.
Creating plots for nonces 2048 to 4096 (0 GB) using 2 MB memory and 8 threads
Pre-allocating space for file (536870912 bytes)...
Done pre-allocating space.
*** stack smashing detected ***: ./plot64 terminated
*** stack smashing detected ***: ./plot64 terminated
Segmentation fault (core dumped)

tested with 1 thread, x threads

sample with sse :  (  -s 2048  -t 8 -n 32768  -m 4096 )

Using SSE4 core.
Creating plots for nonces 2048 to 34816 (8 GB) using 1024 MB memory and 8 threads
Pre-allocating space for file<PHONE_NUMBER> bytes)...
Done pre-allocating space.
37 Percent done. 11553 nonces/minute, 0:02

same parameters avx2

Using AVX2 core.
Creating plots for nonces 2048 to 34816 (8 GB) using 1024 MB memory and 8 threads
Pre-allocating space for file<PHONE_NUMBER> bytes)...
Done pre-allocating space.
*** stack smashing detected ***: ./plot64 terminated
Segmentation fault (core dumped)

also note that mjminer works with avx2, (~19k nonces/min ) is this project abandoned and mjminer should be used instead ? should the readme be updated for those who are looking for an avx2 plotter ?
Have a nice day ,
Francesco

worksforme


"repeat" unknown to LilyPondParser

repeat is not included in the lilypond parser, which causes it to fail

create wrapper class for the parser

helps to fix issues like repeat
centralized parsing / easy maintanance


Problem proccessing PDF: [PDFALTO_CONVERSION_FAILURE]

Hey everyone, I've been having this issue when I try to process a PDF using the Java API. I'm using macOS monterey:
org.grobid.core.exceptions.GrobidException: [PDFALTO_CONVERSION_FAILURE] PDF to XML conversion failed on pdf file /xxx/Wang_paperAVE2008.pdf
at org.grobid.core.document.DocumentSource.processPdfaltoThreadMode(DocumentSource.java:209)
at org.grobid.core.document.DocumentSource.pdfalto(DocumentSource.java:155)
at org.grobid.core.document.DocumentSource.fromPdf(DocumentSource.java:64)
at org.grobid.core.document.DocumentSource.fromPdf(DocumentSource.java:50)
at org.grobid.core.document.DocumentSource.fromPdf(DocumentSource.java:42)
at org.grobid.core.engines.CitationParser.processingReferenceSection(CitationParser.java:396)
at org.grobid.core.engines.Engine.processReferences(Engine.java:237)
I'm also trying on the pdf in the grobid-example repo (Wang_paperAVE2008.pdf). The code is pretty much the basic usage:
(...)
  Engine engine = GrobidFactory.getInstance().createEngine();

  BiblioItem result = new BiblioItem();

  String tei = engine.processHeader(pdfPath, 0, result);

Do you know what I should do?
Thank you!

I get the same exception on Win10.

Hello !
@NorbertSandor Windows is not supported anymore by Grobid, the binaries for pdfalto are missing.
@cerulij it's likely that the binaries for pdfalto shipped with Grobid are not compatible with the recent macOS monterey
@lfoppiano what do you think about the macOS monterey issue?

I tested grobid with MacOs Monterey and it works fine.
@cerulij could you share the whole example or the project so that I could have a look? At first glance might be that the grobid home is not found perhaps?

@NorbertSandor just pointing to the documentation about the non-support of Windows https://grobid.readthedocs.io/en/latest/Troubleshooting/#windows-related-issues
Using docker is a convenient way to have a Grobid service working on windows, see https://grobid.readthedocs.io/en/latest/Grobid-docker/

Ran into similar issue when generating training data on Monterey even on files that has been working in the past. It did work when I built and ran within the CRF Docker image so think this is MacOS specific
Versions:
Grobid: 0.7.2-SNAPSHOT
MacOS: 12.3.1 (21E258)
Java: java version "1.8.0_40-ea" Java(TM) SE Runtime Environment (build 1.8.0_40-ea-b23)
/Users/NAME/grobid-training/new/Asano_et_al-2020-Cochrane_Database_of_Systematic_Reviews.pdf
apr 25, 2022 1:53:37 EM org.grobid.core.engines.Engine batchCreateTraining
ALLVARLIG: An error occured while processing the following pdf: /Users/NAME/grobid-training/new/Asano_et_al-2020-Cochrane_Database_of_Systematic_Reviews.pdf
org.grobid.core.exceptions.GrobidException: [GENERAL] An exception occurred while running Grobid training data generation for full text.
	at org.grobid.core.engines.FullTextParser.createTraining(FullTextParser.java:1452)
	at org.grobid.core.engines.Engine.createTraining(Engine.java:462)
	at org.grobid.core.engines.Engine.batchCreateTraining(Engine.java:566)
	at org.grobid.core.engines.ProcessEngine.createTraining(ProcessEngine.java:379)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.grobid.core.utilities.Utilities.launchMethod(Utilities.java:339)
	at org.grobid.core.main.batch.GrobidMain.main(GrobidMain.java:189)
Caused by: org.grobid.core.exceptions.GrobidException: [PDFALTO_CONVERSION_FAILURE] PDF to XML conversion failed on pdf file /Users/NAME/grobid-training/new/Asano_et_al-2020-Cochrane_Database_of_Systematic_Reviews.pdf
	at org.grobid.core.document.DocumentSource.processPdfaltoThreadMode(DocumentSource.java:209)
	at org.grobid.core.document.DocumentSource.pdfalto(DocumentSource.java:155)
	at org.grobid.core.document.DocumentSource.fromPdf(DocumentSource.java:64)
	at org.grobid.core.engines.FullTextParser.createTraining(FullTextParser.java:1001)
	... 9 more

@karatekaneen I will have a look

It seems that the grobid server works fine. In such case pdfalto is ran in "standalone server mode" and it works. When you run the process to generate the training data, pdfalto runs in "threaded mode" and, for some obscure reason, the pdfalto process fails.
I blame macOs :-)
Even I tested pdfalto from the command line and it works...
As a last resort I figured out that the problem is the ulimit stuff ran before pdfalto:
(base) Lucas-MacBook-Pro:libs lfoppiano$ ulimit -Sv 6242304 && /Users/lfoppiano/development/projects/grobid/grobid-home/pdfalto/mac-64/pdfalto -fullFontName -noLineNumbers -noImage  -annotation  -outline  -filesLimit 2000  '/Users/lfoppiano/Downloads/grobid/in/dys041.pdf' /Users/lfoppiano/Downloads/grobid/out/bao.xml
bash: ulimit: virtual memory: cannot modify limit: Invalid argument

Furthermore, it's possible to fix this by adding
                if (!SystemUtils.IS_OS_WINDOWS && !SystemUtils.IS_OS_MAC) {

in line 149 of DocumentSource.java. However the ulimit will not changed and the memory limit configuration will not be applied for mac Os.
The second option is to run the training data generation in server mode.
@kermitt2 I can push accordingly with your preference.

I think this was with PR #926. @cerulij , @karatekaneen  could you please check it out and if everything works, close this issue? 👍


gax: add fake OperationFuture

to help test Spanner

Codecov Report

Merging #530 into master will decrease coverage by 0.11%.
The diff coverage is 50%.

@@             Coverage Diff              @@
##             master     #530      +/-   ##
============================================
- Coverage     72.46%   72.34%   -0.12%
- Complexity      863      864       +1
============================================
  Files           170      171       +1
  Lines          3929     3949      +20
  Branches        305      305
============================================
+ Hits           2847     2857      +10
- Misses          938      948      +10
  Partials        144      144

Impacted Files
Coverage Δ
Complexity Δ

...m/google/api/gax/longrunning/OperationFutures.java
50% <50%> (ø)
1 <1> (?)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b936b7...5287532. Read the comment docs.

@garrettjonesgoogle PTAL


feat: Build with Alpine for static Linux binaries

This uses an Alpine Linux container to build truly static Linux binaries that do not depend on glibc at runtime.
Alpine uses musl instead of glibc, which enables this fully static build.  With musl compiler wrappers in Ubuntu, you still end up importing glibc references through libstdc++.  And since x265 uses C++, we can't eliminate these in Ubuntu without dropping HEVC support. Using Alpine, with its musl-native environment, allows us to avoid glibc and build fully static Linux binaries.
Closes #28

Here's a full CI run with the modified workflow: https://github.com/shaka-project/static-ffmpeg-binaries/actions/runs/11263885133


Generated Files Are Not Gitignored In Some Generators

Generated files such as .log, .rpt, .odb in flow/, work/, and simulations/run/ are not gitignored for the following generators:

LDO: No files are gitignored
Temperature  Sensor: simulations/run/ is not gitignored.

I was going through the documentation and came across this while running the LDO generator.
I am currently learning about the tool and hoping to contribute in the future so may I work on a fix for this?

I also found that some generated files which were not gitignored are tracked. Should I also untrack these files?
Example: Generated verilog files in ldo-gen and dcdc-gen

LDO_CONTROLLER.v
DCDC_BUFFER.sv

@harshkhandeparkar I don't think excluding the folders and files you mentioned is a good idea. Intended users are designers and usually these files should be under version control

@saicharan0112 The folders that are generated such as logs/ results/ will change everytime the generator is run. So I think those files should not be pushed to github because anyway when running the flow again the files will be created again.

@harshkhandeparkar I don't think excluding the folders and files you mentioned is a good idea. Intended users are designers and as far as I know, usually they want these files under version control

There are templates (which are pushed) for generating these files and they are generated everytime the generator is run. Also these files change slightly depending on the configuration. Also there are nearly a hundred files that are generated so I don't think pushing these is a good idea.


chore: update release process docs

Description

Updated the release process in the README.

:tada: This PR is included in version 1.8.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[InstagramBridge] Display usernames and hashtags as links.

A little modification to InstagramBridge.php. The usernames and hashtags in post are now links to instagram.

I was unable to apply this patch to my working area for some reason. But looks fine.


NIFI-8948 Upgraded Spring to 5.3.9 and Spring Security to 5.5.1

Description of PR
NIFI-8948 Upgrades Spring Framework from 5.3.8 to 5.3.9 and upgrades Spring Security from 5.4.6 to 5.5.1 across all modules. Additional updates include upgrading Spring Boot from 2.5.1 to 2.5.2 in Registry, and upgrading Spring Data Redis from 2.5.1 to 2.5.3.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[X] Has your PR been rebased against the latest commit within the target branch (typically main)?

[X] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on JDK 8?
[X] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

Thanks @exceptionfactory


STORM-919 Gathering worker and supervisor process information (CPU/Memory)

Changes to the worker and supervisor heartbeats to carry the system stats ( CPU util % and JVM memory stats in KB ) .
Changes to the SystemBolt.java to include CPU metrics.
Tested the changes by running on the local mode ( Mac )

It might be too late for this particular PR, but it makes sense to separate human changes and generated code into two separate commits. Otherwise it's really hard to review the code.

Thanks @lazyval . I shall keep that in mind going forward. For review we can omit all those under the "generated" directory.

I will revert the changes to Thrift generated files and update the pull request. Also I have updated details on how we plan to use these metrics @ https://issues.apache.org/jira/browse/STORM-919.

*** PLEASE DO NOT MERGE THIS YET ****
More information @ https://issues.apache.org/jira/browse/STORM-919.

@bourneagain Any update on this?


prejoin not working for redshift

The version of dbt being used: 1.1.1
The version of datavault4dbt being used: 1.2.0
Issue in file: datavault4dbt --> macros --> staging --> redshift --> stage.sql
issue in line: 231
issue: below code is used:
left join {{ source(vals['src_name']|string, vals['src_table']) }} as pj_{{loop.index}} on lcte.{{ vals['this_column_name'] }} = pj_{{loop.index}}.{{ vals['ref_column_name'] }}
However, redshift doesn't allow this kind of join. datavault4dbt.multikey should be used to do it properly.

Hey,
thanks for raising this issue, we are working on a patch for the adapter at the moment, I'll make sure to put this on the agenda. The patch is expected to be released by the 19.01.2024. If you have any further issues or questions we are happy to help

Hey,
in the newest version of datavault4dbt this issue was fixed
https://github.com/ScalefreeCOM/datavault4dbt/releases/tag/v1.2.2


De-index all publicly visible supporting pages

This content has been replaced by new policies and should no longer appear in
search.
Trello: https://trello.com/c/jhNb9FPl/262-supporting-pages-should-be-removed-from-site-search

:+1:


fix: modal icon was in wrong position.

also,  title and description defaults being set by mistake
before:

after:

:tada: This PR is included in version 0.3.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 0.3.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Rustdesk client not available on aarch64-darwin cause xdotool not on this platform

Describe the bug
As mentioned in the title. The rustdesk application seems not available on aarch64-darwin platform cause xdotool is not available.
Steps To Reproduce
Steps to reproduce the behavior:

Add rustdesk to configuration.
Try to install
Error:

error: Package ‘xdotool-3.20211022.1’ in /nix/store/kh5mw1hsalj27ha1wfvksljxyaikcmyb-source/pkgs/tools/X11/xdotool/default.nix:25 is not available on the requested hostPlatform:

Expected behavior
Application is installable
Additional context
Using this on Macboot Pro M1 Pro
Notify maintainers
@ocfox
@Leixb
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.3.0, macOS 14.3`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.18.1`
 - channels(root): `"nixpkgs"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

Add a :+1: reaction to issues you find important.

Currently, libsciter is only packaged for linux, so fixing the issue with xdotool would not fix the problem.
However, rustdesk is moving away from libsciter and using flutter. I would advise you to try rustdesk-flutter from #286129 and see if it works on darwin.


Pylance Report Issue

 Environment data

Language Server version: 2024.11.102
OS and version: linux x64
Python version (and distribution if applicable, e.g. Anaconda):
python.analysis.indexing: true
python.analysis.typeCheckingMode: off

 Code Snippet

XXX

 Repro Steps

XXX

 Expected behavior
XXX
 Actual behavior
XXX
 Logs

XXX

Thanks for the issue. Unfortunately, this is not enough information for us to investigate this issue.
If this continues to be a problem would you please reopen the issue and add the information as described here:
https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#filing-an-issue

My apologies, this issue was a miss click. System seems functional regardless now despite the notification I got. I look forward to any contribution with any REAL issues I might have in the future.


Update friendbot query param addr to newer terminology account_id

Currently, a friendbot endpoint requires an account id in it's addr query param. This param should be account_id to be consistent with the rest of horizon's endpoints.
Current: /friendbot?addr=GAAAAAAAAAAA
Proposed: /friendbot?account_id=GAAAAAAAAAAA

As we discussed in person, this seems like more churn than is useful for a simple testing facility.


Fix tracking on related posts

In order to accomplish this I had to "re-insert" the StreamViewController into the responder.  Easily done by adding a properTarget(forAction:,withSender:) to PostbarController.  This method starts the responder chain search at StreamViewController instead of PostbarController.
The PostbarController had a bit of schizophrenia going on whether to use cell or indexPath, so I refactored the code to prefer cell.  This also simplified the StreamHeader/FooterCell code.
Finishes #142723269

I liked the refactor until I had to fix the specs - the indexPath(for: cell) call doesn't work so well when you can't control the UICollectionView.visibleCells.  Instead, I added a fakeIndexPath, which is settable from the specs.

Ha, maybe thats why we went with indexPath in the first place.


chore: update homepage in gemspec to shakacode organization

Update the organization in the GitHub URL in the gemspec file to shakacode.

@justin808 May you share the information to include in the gemspec?

@G-Rath please merge if this is ready.


cmd/server: remove stopwords after detecting an SDN name's language

Fixes: https://github.com/moov-io/watchman/issues/176
This change uses:

https://godoc.org/github.com/abadojack/whatlanggo
https://godoc.org/github.com/pariz/gountries

Codecov Report

Merging #200 into master will increase coverage by 0.47%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #200      +/-   ##
==========================================
+ Coverage    66.9%   67.37%   +0.47%
==========================================
  Files          27       28       +1
  Lines        2698     2737      +39
==========================================
+ Hits         1805     1844      +39
  Misses        710      710
  Partials      183      183


Add interface for reading sample rate of acceleration.

It could be valuable to add an interface for reading rate of acceleration. This would be valuable for creating libraries that use an accelerometer. For example, with timing data I could begin calculating the position of the device based on acceleration.

What would be a good agnostic format for the sample_rate method? It could be exposed as a u32 representing hz.

@JosephBGerber a raw u32 or a Hertz newtype maybe? (guess you went with an f32 on your PR?)

I feel that an f32 is a good choice due to the existence of fractional frequencies and the usage in further calculations.

Resolved by #53


05/02/2018 23:50 : Create Pull Request for branch prTesting_2018050223 -> master

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestCon.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Update react-a11y-img-has-alt - Img element inside a link must not use alt text that duplicates the text content of the link

Img element inside a link must not use alt text that duplicates the text content of the link. http://www.w3.org/TR/WCAG20-TECHS/H2

Per #876, this repository is no longer accepting pull requests. ☠


Update on Api

I finally eeked out some time today to deploy the code for endpoint updates.
They are as follows:
http://<IP_ADDRESS>/api/v1/trollies -> Gets status and current stops for all known vehicles.
http://<IP_ADDRESS>/api/v1/trollies/:id/stops -> Gets current stops for vehicle with given :id
I haven't but any security on theses new endpoints for testing purposes.  Additionally the phonetester trolley (:id 5) hits all stops 24/7.  Again testing purposes.
The scheduling is setup so that we should be able to have a default schedule for each day of the week for each trolley.  If a different one off / special schedule is needed that is also handled.
Scheduling data is stored in a json column expecting data in the following format:
[ { "startTime": "12:00:00", "endTime": "16:00:00", "stops": [  1, 2  ]   } , { "startTime": "20:00:00", "endTime": "24:00:00", "stops": [  3, 4, 5 ]   } ]

Thank you @danfields!
On Jul 19, 2015 10:05 PM, "Dan Fields"<EMAIL_ADDRESS>wrote:

I finally eeked out some time today to deploy the code for endpoint
updates.
They are as follows:
http://<IP_ADDRESS>/api/v1/trollies -> Gets status and current stops
for all known vehicles.
http://<IP_ADDRESS>/api/v1/trollies/:id/stops -> Gets current stops for
vehicle with given :id
I haven't but any security on theses new endpoints for testing purposes.
Additionally the phonetester trolley (:id 5) hits all stops 24/7. Again
testing purposes.
The scheduling is setup so that we should be able to have a default
schedule for each day of the week for each trolley. If a different one off
/ special schedule is needed that is also handled.
Scheduling data is stored in a json column expecting data in the following
format:
[ { "startTime": "12:00:00", "endTime": "16:00:00", "stops": [ 1, 2 ] } ,
{ "startTime": "20:00:00", "endTime": "24:00:00", "stops": [ 3, 4, 5 ] } ]
—
Reply to this email directly or view it on GitHub
https://github.com/codeforgreenville/trolley-tracker/issues/40.


Configuration program arguments no longer being passed in

Hello, first off amazing plugin, thanks for developing this!
It seems that since a recent update (maybe after I upgraded to Rider 2024.1?), the Program Arguments for a build configuration are no longer being passed through (in addition to any custom args provided by this plugin). The result in my specific situation is that when attempting to run or debug an Unreal project, the specific .uproject path is not given, and so it simply launches the editor without any project.

Cheers!

Hello!
With the new version, I decided to change the approach to replacing arguments, since there is no universal solution for patching arguments of different types of configurations.
As for C++ configurations I'm thinking of bringing back old approach soon, since this is the only type of configuration that allows you to patch arguments without pain.
Thanks!


FID polyfill is flawed (on desktop)

The FID polyfill will do the following, more or less:

Register a set of event listeners for specific types

The first one to fire becomes the FID event

However, if pointerdown is specifically received first:

we first wait for pointerup or pointercancel event to fire

The problem is, before pointerup can fire, if we observe any of the other events, we will resolve the FID timing and clean up, leaving the pointerdown FID data hanging.
Well, mousedown is often automatically generated inside the pointerdown event, and so is guarenteed to fire before pointerup.  (On touch devices mousedown comes after pointerup)
And, any time pointerdown has little input delay, but might have some event processing, we will not just pick the wrong event, but also report a large FID value when it might have been small.

Example, inject this onto a page before interacting:
function block(ms) {
  const target = performance.now() + ms;
  while (performance.now() < target);
}

['pointerdown', 'pointerup', 'pointercancel', 'mousedown'].forEach(type => {
  document.body.addEventListener(type, (event) => {
    const start = performance.now();
    console.log('Event:', type, "InputDelay:", start - event.timeStamp);

    block(100);

    const end = performance.now();
    console.log('Event:', type, "ProcessingTime:", end - start);

    requestAnimationFrame(() => {
        setTimeout(() => {
            const paintEnd = performance.now();
            console.log('Event:', type, 'ToNextPaint(ish)', paintEnd - end);
        });
    });
  });
});

Now, inject the FID polyfill.  (I just paste a compiled version into console)
Compare entry event type and value to the real PO value (if you have one).

Note: we use the polyfill for bfcache activations.
Two options I see to resolve this:

Stop measuring other events after we have pointerdown.

If you get pointercancel, just reset the polyfill
It is probably impossible for real input to sneak in between pointerdown and cancel (on mobile)

Add a second buffer for non-pointerdown events, which is used only while pointerdown is active

If you hit pointercancel, report the other value

FID polyfill is flawed (on desktop)

Interesting...BTW, you can blame Tim Dresser for this, he originally wrote this logic :)

Two options I see to resolve this:

We've removed onFID() entirely (including the polyfill) in v5, so I think we've actually already done option 3, which is to just stop using this code altogether—unless I'm missing something?

Oh I thought it was used for non fid reasons, like first event timing event.
Mostly the motivation here was platforms that don't support event timing,
though.
Cheers.
On Thu, Nov 7, 2024, 02:54 Barry Pollard @.***> wrote:

Closed #564 https://github.com/GoogleChrome/web-vitals/issues/564 as
completed.
—
Reply to this email directly, view it on GitHub
https://github.com/GoogleChrome/web-vitals/issues/564#event-15172966861,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADTZKTZFYMAMW3Z7ZCXQODZ7MMC7AVCNFSM6AAAAABRKKLJ72VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJVGE3TEOJWGY4DMMI
.
You are receiving this because you authored the thread.Message ID:
@.***>


How to modify playlist width in the default layout

I always make foobar2000 fullscreen on a 3840x1100 screen. There appears to be a minor issue: the cover art is floating right with the Georgia-ReBORN theme.

I wonder if there is a way to change the width ratio of the cover art and the playlist. Or centering the cover art is also okay with me.
Thank you very much.

Hi @Cerallin,
you can only make the playlist full width when you right click somewhere in the playlist panel
for context menu > Change layout to full. To go back: right click again > Change layout to normal.
For the album art, you have the following options:
In top menu Options > Player controls > Album art > When player size is not proportional.
In top menu Options > Player controls > Album art > When player size is maximized/fullscreen.
-TT

Thank you for your reply. And I'd like to know if the playlist width is configured by the JavaScript files. If so, I may help add this feature. However, I've never developed foobar200 themes before. It would be nice if you could tell me how to do the modification.

Yes, it's controlled via JavaScript. Actually I'm finishing full code documentation for fellow developers to navigate and understand the code and infrastructure better. Thanks for the offer but I'll this myself, it would be faster than to explain all the stuff.

@Cerallin,
I have implemented the panel auto width feature which will adjust its panel size based on the player size width.
Everythings works fine but I can not test it in fullscreen/maximized because I do not own such monitor ( QHD/WQHD or other super wide panorama res monitor ). Can you test it for me when I send you the files?
-TT

@TT-ReBORN Sure I'm glad to do the tests.

Here you go: https://filepost.io/d/TwbtUg6dML
What you need to test is simple:
First activate the new auto panel width via top menu:
Options > Player controls > Panel > Width > Use auto panel width.
Then you need to change the default album art position via top menu:
Options > Options > Player controls > Album art > When player size is not proportional > Align album art left.
Now with your mouse cursor go to the edge of the player and manually enlarge the player size width and see how the
Playlist, Library, Biography and Lyrics panels will auto adjust the width size.
Also test this with the other two album art align options:
Options > Player controls > Album art > When player size is not proportional > Align album art left (margin).
Options > Player controls > Album art > When player size is not proportional > Align album art center.
This should all work for you as I could test this myself. But now you should test and try with maximized/fullscreen.
Maximized player state is:
When you double click on top menu bar, this will maximize, when you double click again it will resume your last player size.
Fullscreen you already know ( fullscreen button at the top right corner ).
Basically you only need to test and try all the align album art options in maximized and fullscreen mode
and see if the panel auto width does work properly in these modes.

If it does not work, you need to fix it please.
pref.panelAutoWidth is the setting we use. I guess you would need to adjust the position in gr-main-functions.js:
in line ~ 1978, function resizeArtwork(resetDiscArtPosition) {
		albumArtSize.x = // * When player size is not proportional, album art is aligned via setting 'pref.albumArtAlign' in Default layout and is centered in Artwork layout */
			pref.layout === 'default' ?
				displayPlaylist || displayLibrary ?
					UIHacks.FullScreen || UIHacks.MainWindowState === WindowState.Maximized ? ww * 0.5 - albumArtSize.w :
					pref.albumArtAlign === 'left' ? 0 :
					pref.albumArtAlign === 'leftMargin' ? ww / wh > 1.8 ? SCALE(40) : 0 :
					pref.albumArtAlign === 'center' ? Math.floor(xCenter - 0.5 * albumArtSize.w) :
					ww * 0.5 - albumArtSize.w :
				Math.floor(xCenter - 0.5 * albumArtSize.w) :
			pref.layout === 'artwork' ? !displayPlaylist || pref.displayLyrics ? ww * 0.5 - albumArtSize.w * 0.5 : ww : 0;

Tell me how it goes, thanks for the help =)
-Tom

What I have quickly noticed, I forgot to add this extra condition in the above code posted:
replace:
pref.albumArtAlign === 'leftMargin' ? ww / wh > 1.8 ? SCALE(40) : 0 :
to:
pref.albumArtAlign === 'leftMargin' ? pref.panelWidthAuto || (ww / wh > 1.8) ? SCALE(40) : 0 :

Aligning album art left or left works fine. I'm not sure if align center works as you expected: It appears the same as aligning left (margin), and the only difference is the margin size. I guess there should be a margin between the cover art and the playlist.
The pref.albumArtAlign seems not to be working in UIHacks.FullScreen and UIHacks.MainWindowState. I replaced
UIHacks.FullScreen || UIHacks.MainWindowState === WindowState.Maximized ? ww * 0.5 - albumArtSize.w :

with
UIHacks.FullScreen || UIHacks.MainWindowState === WindowState.Maximized ? 0 :

to see how it looks, and the result is shown below.

Here is the fixed code, this should work:
A little bit above the code starting from if (displayPlaylist || displayLibrary) {  replace with this:
		if (displayPlaylist || displayLibrary) {
			xCenter =
				pref.layout === 'artwork' ? 0 :
				UIHacks.FullScreen && !pref.panelWidthAuto ? RES_4K ? 0.261 * ww : 0.23 * ww :
				UIHacks.MainWindowState === WindowState.Maximized && !pref.panelWidthAuto ? RES_4K ? 0.267 * ww : 0.24 * ww :
				xCenter = 0.25 * ww;
		}
		else if (ww / wh < 1.40) { // When using a roughly 4:3 display the album art crowds, so move it slightly off center
			xCenter = 0.56 * ww; // TODO: check if this is still needed?
		}
		else {
			xCenter = 0.5 * ww;
			artOffCenter = false;
			if (albumScale === 0.75 * ww / albumArt.Width) {
				xCenter += 0.1 * ww;
				artOffCenter = true; // TODO: We should probably suppress labels in this case
			}
		}

		const noAlbumArtSize = wh - geo.topMenuHeight - geo.lowerBarHeight;
		albumArtSize.w = Math.floor(albumArt.Width * albumScale); // Width
		albumArtSize.h = Math.floor(albumArt.Height * albumScale); // Height
		albumArtSize.x = // * When player size is not proportional, album art is aligned via setting 'pref.albumArtAlign' in Default layout and is centered in Artwork layout */
			pref.layout === 'default' ?
				displayPlaylist || displayLibrary ?
					pref.albumArtAlign === 'left' ? 0 :
					pref.albumArtAlign === 'leftMargin' ? SCALE(40) :
					pref.albumArtAlign === 'center' ? Math.floor(xCenter - (0.5 * (noAlbumArtStub ? noAlbumArtSize : albumArtSize.w))) :
					Math.floor(ww * 0.5 - (noAlbumArtStub ? noAlbumArtSize : albumArtSize.w)) :
				Math.floor(xCenter - (0.5 * (noAlbumArtStub ? noAlbumArtSize : albumArtSize.w))) :
			pref.layout === 'artwork' ?
				!displayPlaylist || pref.displayLyrics ? Math.floor(ww * 0.5 - albumArtSize.w * 0.5) : ww : 0;

It should look like this:

Keep in mind album art centered is based on the whole width.
If everything is good, I'll commit to master.
-Tom

I've tested them, and they all worked fine. Thank you very much for your patience.

Implemented in https://github.com/TT-ReBORN/Georgia-ReBORN/commit/e907d6c0b30944c3e3021e044bcf711f01ee78c6.


Python/relative_import

Relative import the entire project.

Codecov Report
Patch coverage: 81.01% and no project coverage change.

Comparison is base (87f9241) 52.41% compared to head (012f9c6) 52.41%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #186   +/-   ##
=======================================
  Coverage   52.41%   52.41%
=======================================
  Files          36       36
  Lines        2902     2902
=======================================
  Hits         1521     1521
  Misses       1381     1381

Impacted Files
Coverage Δ

cashu/mint/app.py
0.00% <0.00%> (ø)

cashu/mint/main.py
0.00% <0.00%> (ø)

cashu/mint/router.py
0.00% <0.00%> (ø)

cashu/mint/startup.py
0.00% <0.00%> (ø)

cashu/wallet/cli/nostr.py
41.55% <87.50%> (ø)

cashu/mint/ledger.py
31.70% <92.85%> (ø)

cashu/core/base.py
95.38% <100.00%> (ø)

cashu/core/crypto.py
100.00% <100.00%> (ø)

cashu/core/helpers.py
35.71% <100.00%> (ø)

cashu/core/migrations.py
86.66% <100.00%> (ø)

... and 10 more

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


I'm not sure how confirmed this is BUT.

Whenever I terminate a contract or even switch between days (finish day 1, go to day 2), GoonMod likes to crash. It doesn't happen 100% of the time when going from Day 1 to 2 to 3, but there's a high chance.
Seems also whenever I terminate a contract, it crashes. I've fiddled around, loaded some personal scripts of mine alongside GoonMod, disabled them, loaded PocoHud, disabled it and no matter what, even if ONLY GoonMod is loaded, the game will crash when terminating a contract or exit to the main menu.

Can you upload, or send me, your crash logs after having a crash, so that I can see what your problem is. You'll find them in %appdata%/Local/PAYDAY 2/, and I'll need crash.txt and crashlog.txt

I don't ever get Crash logs from Payday. I only ever get he Mini-Dump files.

try validating game files

I have, everything is validated and up to date.

Nothing in the editor folder is modified by GoonMod, and especially nothing concerning loading level packages. Completely clean your entire Payday installation of mods, and then try installing them one by one until you can recreate the crash.

I did as you suggested. I loaded everything 1 by 1 except goonmod, not crash, then I loaded a freshly downloaded goonmod and it crashed.
I then went back and removed everything else, just kept goonmod, and it then crashed again.


Add Settings page in browser extension

Add the option to turn on/off the auto-checking of xpoc URIs

WIP on add-options-page branch

Fixed in #120