[SUPPORT]HoodieWriteHandle: Error writing record HoodieRecord{key=HoodieKey { recordKey=id308723 partitionPath=202210141643}, currentLocation='null', newLocation='null'}

Tips before filing an issue

Have you gone through our FAQs?

Join the mailing list to engage in conversations and get faster support at<EMAIL_ADDRESS>

If you have triaged this as a bug, then file an issue directly.

Describe the problem you faced
Spark write to hudi, Error occurred when configuring the following parameters.
All deltacommits after the first compaction failed.
To Reproduce
Steps to reproduce the behavior:
hoodie.datasource.write.operation=insert
hoodie.merge.allow.duplicate.on.inserts=true
hoodie.datasource.write.table.type=MERGE_ON_READ
hoodie.datasource.write.precombine.field=ts
hoodie.datasource.write.recordkey.field=id
hoodie.datasource.write.partitionpath.field=part
hoodie.table.name=ss_bucket_dsj_parquet_12_droppart
hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.SimpleKeyGenerator
hoodie.insert.shuffle.parallelism=8
hoodie.datasource.compaction.async.enable=true
hoodie.compact.inline.max.delta.commits=4
hoodie.index.type=BUCKET
hoodie.bucket.index.num.buckets=8
hoodie.bucket.index.hash.field=id
hoodie.storage.layout.partitioner.class=org.apache.hudi.table.action.commit.SparkBucketIndexPartitioner
hoodie.storage.layout.type=BUCKET
hoodie.metadata.enable=true
hoodie.embed.timeline.server=false
path=/tmp/hudi/ss_bucket_dsj_parquet_12_droppart
checkpointLocation=/tmp/hudi/ckp

hoodie.datasource.write.drop.partition.columns=true

Expected behavior
A clear and concise description of what you expected to happen.
Environment Description

Hudi version :
Hudi-0.12.0
Spark version :
Spark-3.1.1
Hive version :
None
Hadoop version :
Hadoop-3.3.0
Storage (HDFS/S3/GCS..) :
Hdfs
Running on Docker? (yes/no) :
No

.hoodie directory
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/.aux
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/.schema
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/.temp
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105540173.deltacommit
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105540173.deltacommit.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105540173.deltacommit.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105643842.deltacommit
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105643842.deltacommit.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105643842.deltacommit.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105654357.deltacommit
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105654357.deltacommit.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105654357.deltacommit.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105706366.deltacommit
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105706366.deltacommit.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105706366.deltacommit.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105715123.commit
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105715123.compaction.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105715123.compaction.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105728964.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105728964.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105728964.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105744714.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105744714.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105744714.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105753800.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105753800.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105753800.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105803146.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105803146.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105803146.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105811225.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105811225.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105811225.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105821027.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105821027.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105821027.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105830382.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105830382.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105830382.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105840358.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105840358.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105840358.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105850370.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105850370.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105850370.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105900443.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105900443.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105900443.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105910483.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105910483.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105910483.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105921875.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105921875.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105921875.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105931118.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105931118.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105931118.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105942142.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105942142.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105942142.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105950501.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105950501.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017105950501.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110003358.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110003358.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110003358.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110020881.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110020881.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110020881.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110030277.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110030277.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110030277.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110040543.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110040543.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110040543.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110050475.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110050475.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110050475.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110100662.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110100662.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110100662.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110110390.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110110390.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110110390.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110120524.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110120524.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110120524.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110132968.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110132968.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110132968.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110141260.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110141260.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110141260.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110150184.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110150184.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110150184.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110200144.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110200144.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110200144.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110210217.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110210217.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110210217.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110221109.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110221109.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110221109.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110230534.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110230534.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110230534.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110240149.deltacommit.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110240149.deltacommit.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110240212.rollback
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110240212.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110240212.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110250169.rollback.inflight
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/20221017110250169.rollback.requested
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/archived
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/hoodie.properties
/tmp/hudi/ss_bucket_dsj_parquet_12_droppart/.hoodie/metadata

Stacktrace
trace 1
22/10/17 11:04:08 ERROR HoodieWriteHandle: Error writing record HoodieRecord{key=HoodieKey { recordKey=id438436 partitionPath=part=202210171102}, currentLocation='null', newLocation='null'}
java.io.EOFException
        at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
        at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
        at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423)
        at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
        at org.apache.hudi.avro.HoodieAvroUtils.bytesToAvro(HoodieAvroUtils.java:158)
        at org.apache.hudi.avro.HoodieAvroUtils.bytesToAvro(HoodieAvroUtils.java:148)
        at org.apache.hudi.common.model.OverwriteWithLatestAvroPayload.getInsertValue(OverwriteWithLatestAvroPayload.java:75)
        at org.apache.hudi.common.model.HoodieRecordPayload.getInsertValue(HoodieRecordPayload.java:105)
        at org.apache.hudi.execution.HoodieLazyInsertIterable$HoodieInsertValueGenResult.<init>(HoodieLazyInsertIterable.java:90)
        at org.apache.hudi.execution.HoodieLazyInsertIterable.lambda$getTransformFunction$0(HoodieLazyInsertIterable.java:103)
        at org.apache.hudi.common.util.queue.BoundedInMemoryQueue.insertRecord(BoundedInMemoryQueue.java:190)
        at org.apache.hudi.common.util.queue.IteratorBasedQueueProducer.produce(IteratorBasedQueueProducer.java:46)
        at org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.lambda$null$0(BoundedInMemoryExecutor.java:106)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

trace 2
22/11/03 17:13:10 ERROR HoodieAppendHandle: Error writing record  HoodieRecord{key=HoodieKey { recordKey=id102047001 partitionPath=202211031711}, currentLocation='HoodieRecordLocation {instantTime=20221103171110071, fileId=00000001-5a03-4a5c-8fac-09ecaa8ab459-0}', newLocation='HoodieRecordLocation {instantTime=20221103171136887, fileId=00000001-5a03-4a5c-8fac-09ecaa8ab459-0}'}
java.io.EOFException
 at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
 at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
 at org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423)
 at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
 at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
 at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
 at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:179)
 at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
 at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:232)
 at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:222)
 at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
 at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
 at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
 at org.apache.hudi.avro.HoodieAvroUtils.bytesToAvro(HoodieAvroUtils.java:159)
 at org.apache.hudi.avro.HoodieAvroUtils.bytesToAvro(HoodieAvroUtils.java:149)
 at org.apache.hudi.common.model.OverwriteWithLatestAvroPayload.getInsertValue(OverwriteWithLatestAvroPayload.java:75)
 at org.apache.hudi.common.model.HoodieRecordPayload.getInsertValue(HoodieRecordPayload.java:119)
 at org.apache.hudi.io.HoodieAppendHandle.getInsertValue(HoodieAppendHandle.java:284)
 at org.apache.hudi.io.HoodieAppendHandle.getIndexedRecord(HoodieAppendHandle.java:237)
 at org.apache.hudi.io.HoodieAppendHandle.writeToBuffer(H

how do you insert your data?  is there any schema change?

@eric9204 how do you insert your data? is there any schema change?

@fengjian428 the schema of table has not changed,only this parameter is added hoodie.datasource.write.drop.partition.columns=true
Without this parameter, the program runs normally.

I don't think this config is supported across all flows. is this happening for an existing table (i.e. you had few commits with default value for this config and later you explicitly set to true)? or are you seeing this even for new table altogether?

@nsivabalan This is a new hudi table,no existing data.
All deltacommit after the first successful compaction will fail because of the above-mentioned error

create table hudi_mor_nonsavepoint (
  id int,
  name string,
  price double,
  ts long,
  par string
) using hudi
tblproperties (
  type = 'mor',
  primaryKey = 'id',
  preCombineField = 'ts',
  hoodie.datasource.write.operation='insert',
    hoodie.datasource.write.drop.partition.columns=true,
    hoodie.index.type='BUCKET',
hoodie.bucket.index.num.buckets=8,
hoodie.bucket.index.hash.field='id',
hoodie.storage.layout.partitioner.class='org.apache.hudi.table.action.commit.SparkBucketIndexPartitioner',
hoodie.storage.layout.type='BUCKET'
)
partitioned by (par)
location 'hdfs://xxx/hudi_mor_nonsavepoint';
 set hoodie.datasource.write.operation=insert;
set hoodie.merge.allow.duplicate.on.inserts=true;
 set hoodie.index.type=BUCKET;
 set hoodie.bucket.index.num.buckets=8;
 set hoodie.storage.layout.partitioner.class=org.apache.hudi.table.action.commit.SparkBucketIndexPartitioner;
set hoodie.storage.layout.type=BUCKET;
set hoodie.datasource.write.recordkey.field=id;
 set hoodie.bucket.index.hash.field=id;
insert into hudi_mor_nonsavepoint select 1, 'a1', 20, 1000,'p1';
insert into hudi_mor_nonsavepoint select 2, 'b1', 10, 344,'p1';

insert into hudi_mor_nonsavepoint select 1, 'a2', 20, 1001,'p1';
 set hoodie.compact.inline.max.delta.commits=0;
 call run_compaction(op => 'schedule', table => 'hudi_mor_nonsavepoint')
 call run_compaction(op => 'run', table => 'hudi_mor_nonsavepoint');

insert into hudi_mor_nonsavepoint select 2, 'b2', 10, 344,'p1';

I tried to reproduce but the commands above work fine @eric9204 is there anything I missed?

@fengjian428 I wrote hudi table with spark structured streaming, every deltacommit after the first compaction will be rollback because of the above-mentioned error.

create table hudi_mor_nonsavepoint (
  id int,
  name string,
  price double,
  ts long,
  par string
) using hudi
tblproperties (
  type = 'mor',
  primaryKey = 'id',
  preCombineField = 'ts',
  hoodie.datasource.write.operation='insert',
    hoodie.datasource.write.drop.partition.columns=true,
    hoodie.index.type='BUCKET',
hoodie.bucket.index.num.buckets=8,
hoodie.bucket.index.hash.field='id',
hoodie.storage.layout.partitioner.class='org.apache.hudi.table.action.commit.SparkBucketIndexPartitioner',
hoodie.storage.layout.type='BUCKET'
)
partitioned by (par)
location 'hdfs://xxx/hudi_mor_nonsavepoint';
 set hoodie.datasource.write.operation=insert;
set hoodie.merge.allow.duplicate.on.inserts=true;
 set hoodie.index.type=BUCKET;
 set hoodie.bucket.index.num.buckets=8;
 set hoodie.storage.layout.partitioner.class=org.apache.hudi.table.action.commit.SparkBucketIndexPartitioner;
set hoodie.storage.layout.type=BUCKET;
set hoodie.datasource.write.recordkey.field=id;
 set hoodie.bucket.index.hash.field=id;
insert into hudi_mor_nonsavepoint select 1, 'a1', 20, 1000,'p1';
insert into hudi_mor_nonsavepoint select 2, 'b1', 10, 344,'p1';

insert into hudi_mor_nonsavepoint select 1, 'a2', 20, 1001,'p1';
 set hoodie.compact.inline.max.delta.commits=0;
 call run_compaction(op => 'schedule', table => 'hudi_mor_nonsavepoint')
 call run_compaction(op => 'run', table => 'hudi_mor_nonsavepoint');

insert into hudi_mor_nonsavepoint select 2, 'b2', 10, 344,'p1';

I tried to reproduce but the commands above work fine @eric9204 is there anything I missed?

@fengjian428 I wrote hudi table with spark structured streaming, every deltacommit after the first compaction will be rollback because of the above-mentioned error.

ok,  can you also try my reproduce case to check if it works in your env?

I will let @fengjian428 follow up. but curious to know why are you setting max delta commits config value to 0. You might as well switch to using COW. but w/ spark structured streaming using MOR table, its not recommended to have very aggressive compaction.

@fengjian428 issue #6128 recurs and was not be solved, cannot reproduce your case in my env

@fengjian428 I have solved this issue #6128.
I have tested your reproduced case in my env。The same with you, spark sql commands work fine, and the result did not reproduce the problem that I found with spark structured streaming.

sorry, not sure I understand you. Do you mean to say, issue is resolved w/ resolution given in 6128. or is there any more pending?

sorry, not sure I understand you. Do you mean to say, issue is resolved w/ resolution given in 6128. or is there any more pending?

@nsivabalan I mean, I solved this issue 6128 in my env. Besides the solution given in 6128, it also needed some necessary work.
Add parameters spark.sql.hive.metastore.jars /$path/hive/lib/*  to spark.default.conf, it makes sure that spark can find
class org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat in hudi-hadoop-mr-bundle.jar when spark sql connect to hive standalone metastore.

dataFrame.writeStream
          .format("org.apache.hudi").options(hoodieConf)
          .outputMode(OutputMode.Append())
          .trigger(Trigger.ProcessingTime("10 seconds"))
          .start()
          .awaitTermination()

https://github.com/apache/hudi/pull/7063

hi @eric9204  I still cannot reproduce your case by using StructureStreamingTest, could you provide a UT for this case?

hi @eric9204 I still cannot reproduce your case by using StructureStreamingTest, could you provide a UT for this case?

@fengjian428 OK, I will provide a UT.
found a new situation, that if the hudi table is a non-partitioned table, this problem will not occur with structured streaming.

@fengjian428 This case can reproduce above issue
import java.io.EOFException

import org.apache.spark.SparkConf
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.streaming.StreamingQueryListener.{QueryProgressEvent, QueryStartedEvent, QueryTerminatedEvent}
import org.apache.spark.sql.streaming.{OutputMode, StreamingQueryListener, Trigger}

object RatePerMicroBatchSourceTest {
  def main(args: Array[String]): Unit = {
    val sparkConf = new SparkConf().set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
    val spark = SparkSession.builder().config(sparkConf).master("local[6]").getOrCreate()
    val dataFrame = spark.readStream
      .format("rate")
      .option("rowsPerBatch", 1)
      .option("numPartitions", 1)
      .option("startTimestamp", 0)
      .option("advanceMillisPerBatch", 1000)
      .load()
    dataFrame.createTempView("t")
    val resultDf = spark.sql(
      """
        |select timestamp,value,date_format(now(),'yyyyMMddHH') as part from t
        |""".stripMargin)
    try {
      runListener(spark)
      resultDf.writeStream
        .queryName("RateStreamSource")
        .format("hudi")
        .options(getHudiConfig)
        .outputMode(OutputMode.Append())
        .trigger(Trigger.ProcessingTime("10 seconds"))
        .start()
        .awaitTermination()
    } catch {
      case e: EOFException => println("eof" + e)
      case _: Exception => println("停止查询")
    }
  }

  def runListener(spark: SparkSession) = {
    spark.streams.addListener(new StreamingQueryListener() {
      override def onQueryStarted(queryStarted: QueryStartedEvent): Unit = {
        println("查询开始: " + queryStarted.id)
      }

      override def onQueryTerminated(queryTerminated: QueryTerminatedEvent): Unit = {
        println(queryTerminated.id)
      }

      override def onQueryProgress(queryProgress: QueryProgressEvent): Unit = {
        if (queryProgress.progress.batchId >= 4) spark.streams.active.foreach(_.stop())
        println("查询进度 batchId : " + queryProgress.progress.batchId)
      }
    })
    spark.streams.active.foreach(_.stop())
  }

  def getHudiConfig = {
    Map(
      "hoodie.datasource.write.operation" -> "insert",
      "hoodie.merge.allow.duplicate.on.inserts" -> "true",
      "hoodie.datasource.write.table.type" -> "MERGE_ON_READ",
      "hoodie.datasource.write.precombine.field" -> "timestamp",
      "hoodie.datasource.write.recordkey.field" -> "value",
      "hoodie.datasource.write.partitionpath.field" -> "part",
      "hoodie.table.name" -> "hudiTest",
      "hoodie.datasource.write.keygenerator.class" -> "org.apache.hudi.keygen.SimpleKeyGenerator",
      "hoodie.datasource.write.drop.partition.columns" -> "true",
      "hoodie.upsert.shuffle.parallelism" -> "1",
      "hoodie.insert.shuffle.parallelism" -> "1",
      "hoodie.datasource.compaction.async.enable" -> "false",
      "hoodie.compact.inline"->"true",
      "hoodie.compact.inline.max.delta.commits" -> "3",
      "hoodie.index.type" -> "BUCKET",
      "hoodie.bucket.index.num.buckets" -> "1",
      "hoodie.bucket.index.hash.field" -> "value",
      "hoodie.storage.layout.partitioner.class" -> "org.apache.hudi.table.action.commit.SparkBucketIndexPartitioner",
      "hoodie.storage.layout.type" -> "BUCKET",
      "hoodie.metadata.enable" -> "false",
      "hoodie.embed.timeline.server" -> "false",
      "path" -> "/tmp/hudi/hudiTest",
      "checkpointLocation" -> "/tmp/hudi/ckp"
    )
  }
}

@eric9204 : may I know if we can work w/ hoodie.datasource.write.drop.partition.columns=false only for your use-case. Or do you really looking to drop the partition columns. As called out earlier, we had some issues around dropping partition columns. we might need to spend time taking a detailed look at all flows and fix them.

if I am not wrong, you claim that you are not facing any issues if you set hoodie.datasource.write.drop.partition.columns= false right. so the issue is only when we set hoodie.datasource.write.drop.partition.columns=true?

if yes, I can file a jira and take a stab at triaging it based on priorities. let us know.

if yes, I can file a jira and take a stab at triaging it based on priorities. let us know.

@nsivabalan Yes, you are right. There aren't any issues when I set hoodie.datasource.write.drop.partition.columns= false.
This issue occurs when I set hoodie.datasource.write.drop.partition.columns= true.

if set hoodie.datasource.write.drop.partition.columns= true and sync meta to hive,data can be query by hive. otherwise,issue #6965 will occur.
So we need to fix this issue.

#7167

can you help to figure out why spark-SQL works fine but structure streaming cannot? @eric9204 I saw you make an API level change in #7167


[stable/graphite] Upgrading Graphite version to 1.1.4

1.1.4 is latest stable Graphite version, plus docker image 1.1.4-1 contains valuable fixes - see https://github.com/graphite-project/docker-graphite-statsd/releases/tag/v1.1.4-1
/cc @fabian-schlegel @monotek

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:

@deniszh please sign off your commits, see: https://www.helm.sh/blog/helm-dco/index.html

/ok-to-test
/assign

/approve

/lgtm


Workflow optimisation

turns out the speedup isn't very big (~3-4min), but this builds the arm images in parallel and deduplicates the building of documentation.

the workflow will fail until there's an artifact for it to grab from the docs repo


[BUG] Running many pods on a Windows node at the same will lead to failures of container

According to https://github.com/Azure/AKS/issues/3612, the first issue should belong to Windows Containers runtime.
Describe the bug
Running many pods on a Windows node at the same will lead to failures of the container.

Windows containerd created container but exited with failure repeatedly. E.g.,

time="2023-04-07T07:19:37.961009200Z" level=info msg=Span duration="575.2µs" endTime="2023-04-07 07:19:37.9615844 +0000 GMT m=+52.729818501" event="&TaskExit{ContainerID:dab1f8e69908b82004b7588053bb2e8b534d7f670ba47e4afebf95893b39d23e,ID:dab1f8e69908b82004b7588053bb2e8b534d7f670ba47e4afebf95893b39d23e,Pid:12876,ExitStatus:2147500037,ExitedAt:2023-04-07 07:19:25.9646369 +0000 GMT m=+40.733090601,XXX_unrecognized:[],}" name=publishEvent parentSpanID=1efed0bbef22d2cf spanID=f85b785afe091c68 startTime="2023-04-07 07:19:37.9610092 +0000 GMT m=+52.729243301" topic=/tasks/exit traceID=c472cfea53de4b4feaa1d5d276323654

To Reproduce
Refer to https://github.com/Azure/AKS/issues/3612
Expected behavior
Node has sufficient resources and should run these pods without the above issues.
Configuration:

Edition: Windows Server
Base Image being used: Windows Server Core
Container engine: Containerd
Container Engine version: containerd://1.6.14+azure

Additional context
Add any other context about the problem here.

Routing this to the correct team (#44584622)

This looks like an Azure CNI issue @ShiqianTao. Please route this issue to https://github.com/Azure/azure-container-networking/issues

Thank you, @fady-azmy-msft . The issue may be caused by 3 factors, the main issue is

https://github.com/Azure/AKS/issues/3612

There are different errors from different components, so I created 2 sub-issues:

https://github.com/microsoft/Windows-Containers/issues/351 (this one for Containers Runtime)
https://github.com/Azure/azure-container-networking/issues/1918 (another one for Azure CNI)

Some issues may need Containers Runtime's help, e.g., containers should only pull the same image once (a few seconds are enough), instead of all containers pulling their own images redundantly, which results in pulling images costing more than 10 minutes.
  Normal   Pulling                 27m                  kubelet            Pulling image "mcr.microsoft.com/dotnet/framework/samples:aspnetapp"
  Normal   Pulled                  13m                  kubelet            Successfully pulled image "mcr.microsoft.com/dotnet/framework/samples:aspnetapp" in 13m53.6124318s

Ah

I see. This looks like two different issues @ShiqianTao, is that right? If so, can you split this so we can deal with each separately?

Hi @fady-azmy-msft . Agree. I created another issue https://github.com/microsoft/Windows-Containers/issues/364, and updated this issue's description. Thank you.

@kiashok would you have thoughts on this?

Spoke with @sbangari and we believe this is an Azure CNI issue. Can you redirect this to the Azure CNI's github Issues · Azure/azure-container-networking (github.com). The issues raised here are actively monitored by the Azure CNI team.

Thank you, @fady-azmy-msft . I have created https://github.com/Azure/azure-container-networking/issues/1918


Caching NPM in Workflows

Previously the caching was occurring on the ~/.npm the cache is now directly caching node_modules
caching node_modules at any directory level.
To prevent restoring node_modules when the cache changed, the cache action is given no restore-keys in the cache Action. The id is added to the cache Action which is used at the NPM install step the cache is checked using id and skips install if the cache exists.

I reran the job for "Tests, Linter & Typechecking" to test the job with a cache within this PR


Update LAB_AK_01_Lab1_Ex4_Trainable_Classifiers.md

Module: 00
Lab/Demo: 00
Fixes # .
Changes proposed in this pull request:

Submitted by ParamRamki on behalf of MOC Labs Review


Move dbslayer, pathfinder, tcpurify to boneyard.

Corresponds to https://github.com/Homebrew/homebrew/pull/37014.

I believe coremod was renamed to libxmp-lite, but still exists: http://sourceforge.net/projects/xmp/files/libxmp/4.3.5/

@mistydemeo Got it. Fixed in the main PR.

Thank you!

Related Homebrew/homebrew PRs: https://github.com/Homebrew/homebrew/pull/37063 https://github.com/Homebrew/homebrew/pull/37056 https://github.com/Homebrew/homebrew/pull/37066 https://github.com/Homebrew/homebrew/pull/37069


Pass Props to individual plugins

Hey guys, is there anyway to pass props to plugins? Like I want to pass down a function to upload my image to s3 on an image upload plugin -- is there a way to do that?

@lightninglu10 have you found any solution?

You can write a proxy
import image from "../../src/plugins/image/plugin";

const {blockComponent} = image;

const Wrapper = (WrappedComponent) => {
  return class PropsProxy extends React.Component {
      render() {
        const props = Object.assign({}, this.props, {fn: () => 'My function'});
        return <WrappedComponent {...props}/>
      }
  }
}

image.blockComponent = Wrapper(blockComponent);`

@sohai yeah I have a solution.
So the block for plugins get blockProps. So if you create a fx that passes down props from blockprops, it will work.
For example, this is my plugin:
const createImagePlugin = (options) => ({
  title: constants.PLUGIN_NAME,
  type: constants.PLUGIN_TYPE,
  buttonComponent: Button,
  blockComponent: Block,
  ...options,
});

And in my Megadraft wrapper, I have:
  render() {
    var megadraftOptions = {
      uploadFile: this.props.upload,
      uploadCallback: this.uploadCallback,
    }
    var plugins = [SectionTitlePlugin, UploadImagePlugin(megadraftOptions), VideoUploadPlugin(megadraftOptions)]
    return (
      <MegadraftEditor
        customStyleMap={styleMap}
        readOnly={this.props.readOnly}
        editorState={this.props.editorState}
        onChange={this.props.onChange}
        plugins={plugins}
        actions={actions}
        entityInputs={entityInputs}
      />
    );
  }
}


Calyptos: zookeeper default port not required to be specified

https://github.com/eucalyptus/midokura-cookbook/blob/master/templates/default/zoo.cfg.erb#L28
Would be great if we could get this changed from<EMAIL_ADDRESS>to '@item', don't see any reason to alter the default port of 2181 at this early stage and in fact I don't see the midokura-cookbook picking up the port
Will help out the configuration yaml file if the default port isn't required

https://github.com/eucalyptus/midokura-cookbook/pull/15

merged


fix: tree erase reverse iterator return wrong iter

What kind of change does this PR introduce? (check at least one)

[x] Bugfix
[ ] Feature
[x] Code style update
[x] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

If yes, please describe the impact and migration path for existing applications:
The PR fulfills these requirements:

[x] It's submitted to the dev branch, not the main branch
[x] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[x] All tests are passing
[x] New/updated tests are included

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

Coverage increased (+0.1%) to 100.0% when pulling 1909132b47b205e54c19f24747dd803d2265bdbd on fix/tree_erase_iterator into ee4f37ce06bc9f5955598fe6782447e380ad6a14 on dev.


Allows different authentication processes by scope

This little change will allow different authentication processes per scope.
You can override authentication process and create another one based on scope extending class AuthServerDelegate and overriding getResourceOwner method.

I'm not sure what this PR intends to do, and it should include tests to demonstrate and validate that the changes have the expected behavior. That being said, at a quick glance, I don't think this solution is appropriate and the behavior is already available by following these instructions: https://aqueduct.io/docs/auth/auth_scopes/#defining-permissible-scope


Custom class props for styling

Hi,
There's some PR about same issue expect I've added class props in array notation. Also added this to documentation
Seems this repo is being active, so i think its good to give a PR.
Thanks

Thanks for the quick PR. I merged it in "alpha" branch for now. Will gather some more updates before generating a new release.


[bug] when the bot restarts, the repeatEveryInterval rule gets messed up because the startTime is not what we expect it to be

We knew this piece of code was fragile before, guess we need to fix it now that it's come up...

Now the event will be called as soon as it can be after the given period of time, you don't need to hit the start time exactly for it to be called at the correct times later


Importing CSV to Table is slow

Preflight Checklist

[X] I have installed the latest version of Storage Explorer.
[X] I have checked existing resources, including the troubleshooting guide and the release notes.
[X] I have searched for similar issues.

Storage Explorer Version
1.25.1
Regression From
1.20.0
Architecture
x64
Storage Explorer Build Number
20220809.7
Platform
Windows
OS Version
Windows 11
Bug Description
I have a simple CSV with 3 columns and 20,000 rows (PartitionKey,RowKey,Integer)
When I import using version 1.24.3 it imports about 2-3 rows per second.
When I import using version 1.20.0 it imports about 500 rows per second.
Azure details:
Performance: Standard
Replication: Read-access geo-redundant storage (RA-GRS)
Account kind: StorageV2 (General purpose v2)
same bug as https://github.com/microsoft/AzureStorageExplorer/issues/5961
This has been marked as done in 1.25 but issue still exists
Steps to Reproduce
Import using version 1.24.3 it imports about 2-3 rows per second.
Import using version 1.20.0 it imports about 500 rows per second.
Actual Experience
Import using version 1.24.3 it imports about 2-3 rows per second.
Import using version 1.20.0 it imports about 500 rows per second.
Expected Experience
No response
Additional Context
No response

@Ricky-G Do you know if Storage Explorer is generating lots of network requests? Can you tell me more about the data you're importing?
We use table transactions to send entities in batches to reduce the operation time and the number of requests that need to be sent. Transactions come with a few limitations, namely:

All entities in the transaction have to have the same partition key.
All entities in the transaction can appear only once.
Each transaction is limited to at most 100 entities.

Storage Explorer makes no assumption about the order of entities in the import CSV, and so it processes each row in the import CSV file in the order they appear. How your CSV file is written can affect the speed of the import operation in a number of ways:

If most entities don't have any partition keys in common (lots of different partitions), Storage Explorer has to generate smaller, more numerous batches, which slows the operation down.
Similarly, because rows are processed in order, if the entities in the input CSV file aren't roughly sorted by partition key, Storage Explorer will make smaller, more numerous batches.
Having a lot of repeating entities close together in your CSV file (rows with the same partition and row key) can also cause batch fragmentation.

@Ricky-G Have you had a chance to answer any of the questions I asked a few days ago about your data? We are planning on releasing a Storage Explorer update soon, and we'd like to include an improvement to table imports if possible. We can't do that without more information from you.

@Ricky-G Something I've also noticed is that after about 500 rows, the upload speed increases from about 2-3 rows per second to 20-30 rows per second. Do you also observe that change?

We have merged some changes that should help speed up the import operation during the column scan phase. We are still looking into performance improvements for the upload phase.
The largest factor in import speed still remains the structure of the data you are uploading. As mentioned above, minimizing the number of batches that need to be sent by minimizing the number of partitions and duplicate entities should speed things up considerably.
Improving performance from there is a challenge. We do a lot to make import reliable, and that comes with trade-offs. One such problem we face is what to do when encountering the same entity (a row with the same partition and row keys as another row) more than once in a CSV file. We determined the safest approach was to break them apart into sequentially processed batches; all version are uploaded in the order they are found. While more reliable, it means batches cannot be submitted in rapid-fire requests; we have to wait for success or failure before submitting the next batch.

@Ricky-G Since it's been some time since we've heard from you, we are going to close this item. As mentioned earlier, we've made some changes that should help improve performance. If you are still experiencing issues after considering our suggestions here, please respond with answers to our previous questions, and we can reopen the investigation.

I experience the same issue; I have quite a big CSV file of about 40k records. Which will take about 44 hours (rough estimate based on 15 records a minute. Not sure if this tool is meant for that though..
I will probably do this some other way, or by coding my own solution. But I was hoping this could be achieved pretty easily by this tool.
I'm playing around with setting the partition key to several (every 100) or all records. When I set them to all, the tool does not do anything and keeps loading at 0 imported and seems to crash eventually. At 100 I don't see any progress either.

Reopening this issue, because import performance is something we want to improve on.

After some in-depth analysis, I can share a few findings:

The CSV parsing code used in 1.20.0 is roughly 10 times faster than it is in recent versions. Fortunately, this is a relatively easy thing to swap out.
Using Node streams seems to run about twice as fast as the 1.20.0 implementation.
Other processes in table import (entity transformations, batching, network requests), do not appear to contribute as significantly to the performance of the operation. After subtracting the time spend parsing, there's little difference in performance between 1.20.0 and recent versions.

From these findings, here's what we'll do:

Swap out the current CSV parsing code for the Node streams implementation.
Reevaluate performance. If further improvements are necessary, we will dive deeper into the other import processes.


Unable to install "npm install --save ng2-bs3-modal"

while trying to installing i am getting this error
32028 error Windows_NT 6.1.7601 32029 error argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "--save" "ng2-bs3-modal" 32030 error node v4.4.0 32031 error npm  v2.14.20 32032 error code EBADPLATFORM 32033 error notsup Unsupported 32033 error notsup Not compatible with your operating system or architecture<EMAIL_ADDRESS>32033 error notsup Valid OS:    darwin 32033 error notsup Valid Arch:  any 32033 error notsup Actual OS:   win32 32033 error notsup Actual Arch: ia32 32034 verbose exit [ 1, true ] 32035 verbose unbuild node_modules\ng2-bs3-modal 32036 info preuninstall<EMAIL_ADDRESS>32037 info uninstall<EMAIL_ADDRESS>32038 verbose unbuild rmStuff<EMAIL_ADDRESS>from E:\wsJs\h5\popups\node_modules 32039 info postuninstall<EMAIL_ADDRESS>
While using -f install works. But run time exception is there.

@shankarAtjunglee what version of npm are you running?

@dougludlow  2.14.20

Ah, I've only tested with v4 and v5. From the error, fsevents is a dependency of karma, which is a dev dependency of ng2-bs3-modal, and shouldn't be installed. This appears to be an issue with npm.

I found this issue, but it looks like it's for v3: https://github.com/npm/npm/issues/11039

Found this issue as well. I shrink wrapped my packaged and it appears that npm2 is trying to install everything in the shrink wrap: https://github.com/npm/npm/issues/8707

This one as well: https://github.com/npm/npm/issues/2679

@shankarAtjunglee I've removed my shrinkwrap file and published a new release. Try version 0.4.4 and let me know how that goes.

@dougludlow last version in npm is still 0.4.3

@dougludlow
problem is still same. here is what i did

Created new Folder(test)
Created new Fresh package.json
run command "npm install --save ng2-bs3-modal"
still same error.

other details

OS windows7
Npm -v = 2.14.20

npm-debug.log
32011 verbose pkgid<EMAIL_ADDRESS>32012 verbose cwd E:\wsJs\h5\test 32013 error Windows_NT 6.1.7601 32014 error argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "--save" "ng2-bs3-modal" 32015 error node v4.4.0 32016 error npm  v2.14.20 32017 error code EBADPLATFORM 32018 error notsup Unsupported 32018 error notsup Not compatible with your operating system or architecture<EMAIL_ADDRESS>32018 error notsup Valid OS:    darwin 32018 error notsup Valid Arch:  any 32018 error notsup Actual OS:   win32 32018 error notsup Actual Arch: ia32 32019 verbose exit [ 1, true ] 32020 verbose unbuild node_modules\ng2-bs3-modal 32021 info preuninstall<EMAIL_ADDRESS>32022 info uninstall<EMAIL_ADDRESS>32023 verbose unbuild rmStuff<EMAIL_ADDRESS>from E:\wsJs\h5\test\node_modules 32024 info postuninstall<EMAIL_ADDRESS>

Sorry, I guess I forgot to publish... :tired_face: It should be there now.


feature: batch ray job submission

Today, to submit a ray job via the sdk the typical steps are:
Cluster.up()
Cluster.wait_ready()
DDPJobDefinition.submit()
Cluster.down()
The desired feature is to have one macro operation that combines the above steps, so that when the ray cluster is ready, which could be potentially hours later, the client does not have to be connected to submit the ray job. Basically submit ray cluster request + ray job definition simultaneously, in a fire and forget manner.
As an added value, this scenario is also useful when the ray cluster is obtained on a different OCP cluster. In that case, if the appwrapper  is self sufficient and includes ray job definition which is passed to the ray cluster at time of its creation, there will be no need to return back a ray cluster dashboard route to the submission OCP cluster, which may be tricky.

fyi @MichaelClifford @Maxusmusti

Does it mean to support Ray Jobs by CodeFlare?


Allow for custom publication status

Purpose
Sometimes one would like to add a publication that is not yet accepted or is otherwise in preparation. Currently, the date still appears for these. This PR allows the user to add a custom status to appear instead of the date. The date field remains useful for sorting the entries (e.g. if the "in progress" publications have a date set to today's date, they will appear at the top of the list as per issue #388).
Screenshots
Please ignore the custom logos and custom details from #596
custom_status = "Under Review: TAAC-D-18-00175"
Publications list (Simple)
No date here, so added the status to the end of the title.

Publications list (Detailed)

Publications list (APA)

Publications list (MLA)

Publication details

Note for the above, I will push a commit to remove the extra period.
Documentation
I have added the option to the archetype and the example site publications.

This seems to be related to #357.

@shaform, shoot, I had missed that one. Perhaps we could work toward a single united PR. I like the categorization of #357, but prefer to replace dates as in this one. Happy to collect feedback from folks.

I am very new to building sites, github pages etc. I have used the Hugo Academic Theme and I would like to implement this option. However, the files I have in my repository seem different. Any idea how I can implement this option in my version of the repository?
Thanks!


Cannot read file in screen initialization event.

Describe the bug
Try to read the file when the screen is initialized, but the GotText event did not trigger.

After tracking the code, it is found that the screenInitialized tag is set after the screen initialization event, which determines whether other events will be dispatched.

Is it designed like this? Or is a bug?
Affects

[ ] Designer
[ ] Blocks editor
[ ] Companion
[x] Compiled apps
[ ] Buildserver
[ ] Debugging
[ ] Other... (please describe)

Expected behavior

Steps to reproduce

We will need to determine the underlying cause of what you're seeing, but the code is correct here. If anything, the bug is likely in how permissions are handled. The file read operation happens on a different thread than the UI thread. Once the operation has finished the results are dispatched on the UI thread, which will have finished executing the blocks and set the initialized flag in the meantime.
Sent from my iPhone

On Nov 5, 2020, at 01:20, 皮皮蟹<EMAIL_ADDRESS>wrote:

Describe the bug
Try to read the file when the screen is initialized, but the get text event did not trigger.
After tracking the code, it is found that the screenInitialized tag is set after the screen initialization event, which determines whether other events will be dispatched.
Is it designed like this? Or is a bug?
Affects
Designer
Blocks editor
Companion
Compiled apps
Buildserver
Debugging
Other... (please describe)
Expected behavior
Steps to reproduce
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

My fault. The cause of the problem is that the full absolute path is not supported.
Can only be:  /Download/test.txt

After further digging, I think we still need to set the flag before the initialization event. Otherwise, the error event will not be triggered.


Wrong error message : reserved instead of uniqueSuffix

In slug service, line 268 to 275 :
    $method = $config['uniqueSuffix'];
    if ($method === null) {
        $suffix = $this->generateSuffix($slug, $separator, $list);
    } elseif (is_callable($method)) {
        $suffix = call_user_func($method, $slug, $separator, $list);
    } else {
        throw new \UnexpectedValueException('Sluggable "reserved" for ' . get_class($this->model) . ':' .     $attribute . ' is not null, an array, or a closure that returns null/array.');
    }

The exception thrown says reserved instead of uniqueSuffix.

Btw, is their a way to have uniqueness and prevent creation of doubles instead of adding a suffix ?

Fixed the typo a while back ... thanks!
If you are looking for a way to make a slug that already exists unique, that would be something you code yourself in the uniqueSuffix closure.  Just do anything you want in that function to make a new, unique slug.


Fix ts package

@JoshMarler confused about this PR, I think both of these commits are already on master, no?

Oops. Accident!


Fix 404 error on short URL in private blog

close #9578

updated check statements to disable RSS for private blog

[x] There's a clear use-case for this code change

[x] Commit message has a short title & references relevant issues

[x] The build will pass (run grunt validate)

What's exact failed issue? mysql?

Travis just randomly fails 😞 At times it gets rather busy and causes timeouts or failed DB connections. I've restarted the job for you.

Thank you @kevinansfield, checks are passed now :)

@tiendq Thanks :)

@tiendq One last question. I remember you have added a test to ensure a two letter blog post does not result in a 404, didn't you? 🤔

I have also added several tests for the check.

Yeah, but the original bug report is not covered :)
From #9578

Publish a post with e.g. 2 characters


Delivery for Coordinate Conversion Addin DotNet V3 (Support)

This build addresses the GitHub tickets mentioned below:

#532 - Coordinate Conversion ArcMap custom formatting not applied to MGRS, partially applied DDM
#531 - Coordinate Conversion changing symbol for locations loses selection and location highlight
#528 - Coordinate Conversion custom input format crashes Pro

\Solutions\DefenseSolutions\CoordinateConversion\Testing\CTLocalGovTeam-dev\39 was built to test these changes

This is still unmerged, waiting on #531


Huge memory requirements in ECG artifact removal

ECG artifact removal does not scale with BSTD signal length. It is quadratic in signal length.
For long signals (hours and more), the code does not finish in several hours or simply crashes by requesting too much memory:
...running cross-correlation on 8104 segments...
...ecg template 1 generated...
Error using bsxfun
Out of memory. Type "help memory" for your options.
Error in corr>corrPearson (line 297)
    x = bsxfun(@minus,x,sum(x,1)/n);  % Remove mean
Error in corr (line 212)
        coef = corrFun(rows,tail,x,y);
Error in perceive_ecg (line 49)
r = corr(corrdata,ecg.proc.template1').^2;
Error in perceive (line 521)
                            d.ecg{e} = perceive_ecg(raw(e,:));

Interesting, I would not have believed anyone would record/stream for hours. That must cause a huge battery drain for the patients? Would you be able to share such a file, so I can try to improve performance?

It would probably make sense to include ECG processing into the options. Run ECG, but also whether the original signal should be replaced by cleaned data could be an option there.


Fixed prettier errors

coverage: 93.925% (-0.004%) from 93.929%
when pulling d7fc0c6bc96135b9c23aa4f9a751b8aba3d42d17 on cody-fix-prettier-errors
into 5230a7ce13abdc14d288d85cba12f7c98e3ad5d9 on main.


Update dependent images upon base image new build

When the images on this repo are built in a new version, we should trigger other repositories that depend on these images to build a new version as well to everyone is up-to-date.

[x] https://github.com/kool-dev/docker-php-sqlsrv
[x] https://github.com/kool-dev/docker-php-oci8

This has been accomplished.


Add CONTRIBUTING.md and CODE_OF_CONDUCT.md

First off, I want to say that the website looks fantastic! It’s clean, feature-rich, and has so many useful functionalities that I personally find incredibly valuable. It's clear a lot of thought and effort has gone into building this project, and I really appreciate the work you've done.
While browsing the repo, I noticed that there isn’t a CONTRIBUTING.md or CODE_OF_CONDUCT.md file. These would help guide contributors and foster a welcoming, inclusive environment. I’d be happy to assist in drafting these documents if needed to help the project grow!
Best regards,
Juan Díaz

Thank you so much for your kind words! I'm glad to hear you're finding the site useful 😊. You're absolutely right, adding a CONTRIBUTING.md and a CODE_OF_CONDUCT.md file would be beneficial. If you're interested, I'd love to have your help in drafting these documents! 🚀

Great, what will be a good contact method? (normally an email)


Update grammar on configuration doc page.

I've updated a few of the sentences on the configuration page. Let me know if you want me to make any changes.

@IntuitDeveloperRelations Love to see if this is able to be merged! Thanks,


Add disk IO reporting

Disk I/O reporting metrics should be gathered and reported if possible.

Use https://www.npmjs.com/package/procfs-stats


Switch to personal fork of menubar until fix is merged

@keybase/react-hackers
Fixes:

Into:

I'll be PR'ing this change to menubar & electron-positioner so we can get off my fork eventually.

Let's make these keybase forks and not our own
On Thursday, December 10, 2015, Marco Munizaga<EMAIL_ADDRESS>wrote:

@keybase/react-hackers
https://github.com/orgs/keybase/teams/react-hackers
Fixes:
[image: 2015-12-01 at 5 34 pm]
https://cloud.githubusercontent.com/assets/594035/11732975/4cc5b5c6-9f5e-11e5-9d4a-dea113e3fd91.png
Into:
[image: screen shot 2015-12-10 at 4 35 23 pm]
https://cloud.githubusercontent.com/assets/594035/11732979/53bfb6ce-9f5e-11e5-89c0-1619d81e298c.png
I'll be PR'ing this change to menubar & electron-positioner so we can get
off my fork eventually.
You can view, comment on, or merge this pull request online at:
https://github.com/keybase/client/pull/1511
Commit Summary

Switch to personal fork of menubar until fix is merged

File Changes

M desktop/package.json
https://github.com/keybase/client/pull/1511/files#diff-0 (2)

Patch Links:

https://github.com/keybase/client/pull/1511.patch
https://github.com/keybase/client/pull/1511.diff

—
Reply to this email directly or view it on GitHub
https://github.com/keybase/client/pull/1511.

Changed menubar to be a keybase fork, electron-positioner is still a personal one just because I already made a PR from that fork

:+1:


scheduleLocalNotification fires immediately even though fireDate is of future

I am creating a scheduleLocalNotification request, which I intend to fire after 7 days and repeat every week, in the following way
but this notification is shown as soon as the code is executed, even though fireDate is of the future

`const request = {
        userInfo: { id: '1' },
        alertTitle: "TItle Text",
        alertBody: "body text.",
        fireDate: new Date(Date.now() + 604800000),
        repeatInterval: 'week',
      }

      PushNotificationIOS.scheduleLocalNotification(request);`

Use PushNotificationIOS.addNotificationRequest(notification) instead of PushNotificationIOS.scheduleLocalNotification(request) since scheduleLocalNotification in deprecated.
https://github.com/react-native-push-notification-ios/push-notification-ios#schedulelocalnotification

Use PushNotificationIOS.addNotificationRequest(notification) instead of PushNotificationIOS.scheduleLocalNotification(request) since scheduleLocalNotification in deprecated.
https://github.com/react-native-push-notification-ios/push-notification-ios#schedulelocalnotification

I updated to the latest version and addNotificationRequest does not fire at all if a fireDate is mentioned in the passed object. fires instantly when there is no fireDate object.

Use PushNotificationIOS.addNotificationRequest(notification) instead of PushNotificationIOS.scheduleLocalNotification(request) since scheduleLocalNotification in deprecated.
https://github.com/react-native-push-notification-ios/push-notification-ios#schedulelocalnotification

I updated to the latest version and addNotificationRequest does not fire at all if a fireDate is mentioned in the passed object. fires instantly when there is no fireDate object.

same issue as this. addNotificationRequest does nothing if the fire date is in the future but works if I don't set a fireDate

This might be helpful: https://github.com/react-native-push-notification-ios/push-notification-ios/issues/247


live-migration: fix bytes to Mbps conversion in log

What this PR does
Before this PR:
bytes to Mbps conversion is wrong.
After this PR:
bytes to Mbps conversion is correct.
Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to kubevirt-dev was considered

Release note

NONE

/lgtm

/hold
Please answer any questions/comments.

/unhold

/cherry-pick release-1.2


ci: configure codecov upload during PR

This PR

adds codecov to PR runs
updates make test to include recommended test flags

Notes
This change includes test flags for race and covermode based on a recommendation from Codecov. Looking into the flags, they both seem like good ideas and run fine locally.
How to test
Run make test locally.
Signed-off-by: Michael Beemer<EMAIL_ADDRESS>

It works! ^^^


Simplified eval throws exception on applyMap

Description
This occurs only in the simplified eval and not the common eval.
The expression and input vectors are provided for repro below.
Error Reproduction
No response
Relevant logs
E0124 03:26:16.537164  5920 Exceptions.h:68] Line: ../.././velox/functions/lib/SubscriptUtil.h:142, Function:applyMap, Expression:  Unsupported map key type for element_at: DOUBLE, Source: USER, ErrorCode: UNSUPPORTED
E0124 03:26:16.537281  5920 ExpressionVerifier.cpp:164] Simplified eval wasn't supposed to throw, but it did. Aborting.
E0124 03:26:16.537595  5920 FuzzerToolkit.cpp:131] Only simplified path threw exception:
I0124 03:26:16.538622  5920 ExpressionVerifier.cpp:295] Persisted input: --fuzzer_repro_path /tmp/fuzzer_repro/velox_expressionVerifier_8TkX8d --input_path /tmp/fuzzer_repro/velox_expressionVerifier_8TkX8d/input_vector --result_path /tmp/fuzzer_repro/velox_expressionVerifier_8TkX8d/result_vector --sql_path /tmp/fuzzer_repro/velox_expressionVerifier_8TkX8d/sql --lazy_column_list_path /tmp/fuzzer_repro/velox_expressionVerifier_8TkX8d/indices_of_lazy_columns --complex_constant_path /tmp/fuzzer_repro/velox_expressionVerifier_8TkX8d/complex_constants
terminate called after throwing an instance of 'facebook::velox::VeloxUserError'
  what():  Exception: VeloxUserError
Error Source: USER
Error Code: UNSUPPORTED
Reason: Unsupported map key type for element_at: DOUBLE
Retriable: False
Function: applyMap
File: ../.././velox/functions/lib/SubscriptUtil.h
Line: 142
Stack trace:
Stack trace has been disabled.Use --velox_exception_user_stacktrace=true to enable it.

*** Aborted at<PHONE_NUMBER> (Unix time, try 'date -d @1674530776') ***
*** Signal 6 (SIGABRT) (0x1720) received by PID 5920 (pthread TID 0x7f5ab6b1d300) (linux TID 5920) (maybe from PID 5920, UID 0) (code: -6), stack trace: ***
(error retrieving stack trace)
/bin/bash: line 1:  5920 Aborted                 (core dumped) _build/debug/velox/expression/tests/velox_expression_fuzzer_test --seed 15935 --lazy_vector_generation_ratio 0.2 --duration_sec 600 --enable_variadic_signatures --velox_fuzzer_enable_complex_types --velox_fuzzer_enable_column_reuse --velox_fuzzer_enable_expression_reuse --retry_with_try --logtostderr=1 --minloglevel=0 --repro_persist_path=/tmp/fuzzer_repro
FAIL: Expression run failed

FuzzerApplyMap.zip

This is probable the same issue that @oerling ran into: #3773

Turns out this is due to subscript function declaring incorrect signature: map(K,V), K -> V. This signature suggests that K can be any type, but in fact K can only be tinyint, smallint, integer, bigint or varchar. The fix is to provide more specific set of signatures.


DuckDBConnectionStringBuilder threw an exception

I used Nuget to add  DuckDB.NET.Data.Full (<IP_ADDRESS>) to my Blazor WebAssembly project.
I tried this code:
using var duckDBConnection = new DuckDBConnection("DataSource=:memory:");
duckDBConnection.Open();
As well as this code pointing to a local duckDB database:
using var duckDBConnection = new DuckDBConnection("DataSource=stocks.db");
duckDBConnection.Open();
Both methods give me this error on this part of the code:   duckDBConnection.Open();
'The type initializer for 'DuckDB.NET.Data.DuckDBConnectionStringBuilder' threw an exception.'
Is there anything I need to add or change to fix this issue. I tried a try{} catch{} to get more info:
   catch(Exception e) {
       // Print the full exception details, including inner exceptions
       Console.WriteLine($"Error: {e.Message}");
       if (e.InnerException != null)
       {
           Console.WriteLine($"Inner Exception: {e.InnerException.Message}");
       }
   }

But I was unable to get more info about the problem. Any help is appreciated.

'The type initializer for 'DuckDB.NET.Data.DuckDBConnectionStringBuilder' threw an exception.' means that the exception happened in the static constructor of DuckDBConnectionStringBuilder This is probably because of DuckDBConfigCount call to the DuckDB native library.
Even if you comment out that code and remove the static constructor completely, you will probably only delay the error. I'm not sure why that call is failing, but probably every native method call will fail because of the same reason. You could debug the library to see what exception is thrown when calling the native methods.
By the way, using the library from Blazor isn't supported yet. See https://github.com/Giorgi/DuckDB.NET/issues/169

Ok! Thanks for the quick response! :)

The full exceptions is
Unhandled exception rendering component: The type initializer for 'DuckDB.NET.Data.DuckDBConnectionStringBuilder' threw an exception.
System.TypeInitializationException: The type initializer for 'DuckDB.NET.Data.DuckDBConnectionStringBuilder' threw an exception.
 ---> System.DllNotFoundException: duckdb
   at DuckDB.NET.Data.DuckDBConnectionStringBuilder..cctor() insrc\DuckDB.NET\DuckDB.NET.Data\DuckDBConnectionStringBuilder.cs:line 25
   --- End of inner exception stack trace ---
   at DuckDB.NET.Data.DuckDBConnection.get_ParsedConnection() in S:\src\DuckDB.NET\DuckDB.NET.Data\DuckDBConnection.cs:line 29
   at DuckDB.NET.Data.DuckDBConnection.Open() in \src\DuckDB.NET\DuckDB.NET.Data\DuckDBConnection.cs:line 103

Closing this, Blazor support is tracked in #169


Adding import on app.d.ts once, instead of every single component

In the docs it said

[!IMPORTANT]
You need to import import "@macfja/svelte-persistent-runes" to prevent Typescript to complain about the unknown function $persist

but I found adding this on every component isn't very clean.
Instead, I suggest adding it once to /src/app.d.ts file.
I tested on my local project and it worked well.

By the way, maybe adding to a new file /src/global.d.ts is cleaner, because it actually doesn't have to do with the app itself, but the project.
What do you think?

Hi! This suggestion comes from discussing the issue with Claude AI, so please verify if this would work:
It suggested making $persist types automatically available by modifying index.ts (of the library) like this:
declare global {
    function $persist<T>(
        initial: T,
        key: string,
        options?: Partial<PersistentRunesOptions>,
    ): T;
}

// Make this file a module
+export {};

(https://github.com/MacFJA/svelte-persistent-runes/blob/main/src/index.ts)
The AI explained that adding export {} should make TypeScript treat it as a module while keeping the global declaration, potentially removing the need for users to add imports in app.d.ts.
However, since this comes from an AI, I'd appreciate if you could verify if this approach would actually work in practice. The goal is to make the types automatically available when the package is installed, working consistently across all IDEs.
What do you think about trying this approach?
Here's the technical explanation it provided me with:
In TypeScript, there's a key difference between script files and module files:

Script files (no imports/exports):

Their declarations automatically go into the global scope
But they can interfere with other files in unexpected ways
Each script file adds to the same global namespace

Module files (have imports/exports):

They have their own scope
Their declarations don't automatically pollute the global namespace
They're more predictable and better for libraries

By adding export {}, we're telling TypeScript "this is a module file" (even though we're not actually exporting anything useful). This is important because:

As a module, our global declarations are more intentional - they only affect the global scope when we explicitly use declare global
It prevents our file from being treated as a script file, which could merge with other globals unpredictably
When someone installs our package, TypeScript will properly process these declarations since they're part of a module

The reason it works better than requiring imports in app.d.ts is that npm/TypeScript will automatically find and process our module's type declarations when the package is installed, making the global declarations available without any extra setup.

It did some test, and "automatic" global declaration (global declaration with explicit import) only work for package in typeRoots or in @types/ namespace (which is the default typeRoots)
So changing the value of typeRoots (and you have to add ./node_modules/@types in the list) or asking to add/create an ambient module seem to be about the same to me.


LAS with MWER

Maybe we can close this PR before we finished the implementation of MWER with speech-transformer

Sure, we will resubmit this pr when we get improvements with MWER on speech-transformer.


Remove -i flag from go build commands to eliminate re-installs that give 502s

Description
It's unclear that we need the -i flag for our builds if we are using go.mod and go get correctly. This PR tests it out.

It appears to be a 1-2 minute improvement per step if I compare this build:
https://circleci.com/workflow-run/00bfe248-34ba-4ec1-8f5a-454789f429da
To one where I leave the -i flag alone:
https://circleci.com/workflow-run/b8237a8a-0e4a-416e-b033-6aa385c12a72
There are probably other factors here like load on the system that each container was running on in CircleCI.  But I'm thinking this is a good improvement.


Fix webpack

+1

+1 builds on ci seems to have failed because of a problem with saucelabs, can someone restart the tests on travis for this PR ?

+1 seems to be problem with Sauce Labs credentials. Tests are passing on local.

@tontonAurel I would suggest to rebase your changes on current master, which would trigger new CI build and then @ocombe would hopefully have easy choice to merge it.
Thanks!

Sorry that's my fault. I need to get back into this project sometimes but I don't really have the time right now.
Also CI is broken for PRs because of sauce labs (I don't know why, probably a config problem).

Thanks @ocombe!
p.s. It seems that sauce labs now require to set username and access_key as per docs https://docs.travis-ci.com/user/sauce-connect/#Setting-up-Sauce-Connect
Just for future reference not pushing ;) I'm sure that you have lots of other better things to do.

@ocombe can we release this merge request pretty please :)

+1, need a release :)

+1

I'll take a look at getting this merged correctly, if it hasn't already been handled. Stay tuned!


verbs crash with bad packet length on stampede

Original issue: https://charm.cs.illinois.edu/redmine/issues/803

The latest charm version (v6.6.0-317-g80fea48) crashes a ChaNGa run on stampede with:
Fatal error on PE 502> packet in the middle does not have expected length
An earlier version (v6.6.0-258-ge8e17df) works.

Original date: 2015-08-12 02:49:45

Bilge, please judge if its related to the broadcast change, and if not, reassign to Changa group.

Original date: 2015-08-12 14:21:40

It could be related to my recent broadcast change. (b1db2f25a931534c555aadd14740f8f6831bf9ae)
Tom, can you tell the parameters you're using for ChaNGa when you get this crash? I'll try to reproduce the issue.

Original date: 2015-08-12 19:02:13

I used "git bisect" to figure out which commit caused the problem.  It tells me:
login3.stampede(17)$ git bisect bad
b1db2f25a931534c555aadd14740f8f6831bf9ae is the first bad commit
To reproduce on Stampede:

In charm, "./build ChaNGa verbs-linux-x86_64 smp -j4 -O2"
In ChaNGa, "./configure --enable-bigkeys; make"
Run ChaNGa with the attached job script, and param file.  The data file is too large to attach, but can be found at:
ftp://ftp-hpcc.astro.washington.edu/pub/hpcc/bench/hrwh_sbc_gas.tbin

The files are also on Stampede in the directory /home1/00333/tg456090/work/hrwh_sbc_g

Original date: 2015-08-17 19:59:45

Thanks Tom, so it's my recent change is causing the problem. I have reproduced the crash and working on fixing it.

Original date: 2015-09-10 22:01:07

Any progress on this?  Having charm broken on the ibverbs platform is not good.

Original date: 2015-09-11 13:35:12

I'm going to look into this after my paper deadline this weekend.

Original date: 2015-09-14 17:56:25

Fix is implemented here: ~~https://charm.cs.illinois.edu/gerrit/#/c/827/~~ https://github.com/UIUC-PPL/charm/commit/7f5f80087cdd1e4fb075e274bfe629357cbf9364
I've tested and Changa works fine now. Tom, can you test it as well please?

Original date: 2015-09-17 19:48:15

The fix is merged.

Original date: 2015-11-11 03:05:23

Set status back to Merged so that we can distinguish whether the code was changed in some way, or the fix was elsewhere. Both are non-open states from Redmine's perspective.


Ghost Teleporting

It would be nice to teleport to players (not orbiting) and specific positions (like warp escape does).

I'll look into getting this done


move window modifier stop working

when I switch layout between bsp and float with skhd it can stop move window with modifier so I have to restart yabai and then moving windows available, I was trying to do debug but idk it doesn't happen when I'm tryna debug, I'm thinking its triggered when I off all windows with "cmd Q" so when I'm not close debug window it not happens
OS Sonoma 14.2.1
.yabairc

.skhdrc

I got it. Doesn't matter what key setup for window dragging, you can press button and start dragging but if you press cmd Q while dragging it's stop dragging at all before you don't restart yabai

Note to self:
g_mouse_state.window = NULL;
g_mouse_state.current_action = MOUSE_MODE_NONE;

never gets reset in mouse_up handler when the window is destroyed through cmd+w or cmd+q


Added Tibia Char Bazaar Auction endpoint.

@JorgeMag96 nice PR, good start of the bazaar!
I created a separate feature-branch (feature-char-bazaar) so we can continue working on.
One thing that I'm missing is the endpoint to list all auctions.. we'll add that in the new branch.
Also, the function to do ajax request, we could probably combine that function with the other request, since many things are "the same".
But I really like the clean structure of your code, thanks for the nice PR!


Please publish a new release

I would like to be able to pip install the latest version from PyPI with your awesome changes. :)

hi @evanhempel i would :heart:  to see a new release on pyppi as well. not trying to be pushy... is there anything i can do to help?

Hi guys, I've been heads down with work the last few months, but that should end next week.  I hope to get a new one out soon.

no worries, appreciate your work on this lib!

@evanhempel would it be possible for you to tag current master and push it to pypi? that would be awesome!


No event when I send int Id

Expected Behavior
Amplidue works with integer Id
Current Behavior
Amplidue don't  works with integer Id
Steps to Reproduce

Send event with int user_id = 99
There is no events in panel
Send event with user_id = "USER_99"
There is event in panel

Environment

SDK Version: v1.1.4
Python Version: 3.12.1
OS Info: Win11

https://github.com/rudderlabs/rudder-transformer/issues/127

@navff thank you for reaching out. We expect the min id length to be 5 characters by default. This can be overridden by updating min_id_length in the configuration to a lower value.
https://www.docs.developers.amplitude.com/data/sdks/python/#configuration


Remove hooks

They don't actually work currently, and provide little to no value (everything that can be done via a hook should just use the python.call operation).

Note to self: need to come up with examples/ways to achieve the same w/o hooks!

v1 is merged, will be released after a short pre-release period.


'rest' was not declared in this scope

void loop()
{
String response;
rest.get(path, &response);
Serial.println(response);
delay(1000);
}
from your example you are calling rest.get(), however, rest was not defined. Is this right usage?

example fixed


Code Length Overflow for QR

Brief Description: I tried to purchase this item (ob://9a791e0e2e4936754b103a6cc3cf22e77c8e507b/listing/913f8d10a9213102a5d1e730b3febff4cdc97486) and got a blank QR code.
Operating System (OS and version): OSX 10.11
OpenBazaar version (shown on About OpenBazaar page in menu): 1.0.7
Hardware:
Reproducible (Always / Almost Always / Sometimes / Rarely / Couldn't Reproduce):
Always
Steps to reproduce:

Visit ob://9a791e0e2e4936754b103a6cc3cf22e77c8e507b/listing/913f8d10a9213102a5d1e730b3febff4cdc97486
Purchase using moderated flow

Observed Behavior:
At QR screen you get:

Expected Behavior:
See QR for payment.
Additional info (links, images, etc go here):

purchase_contract
{
    "order_id": "71aa409bb143bc95c04e0e117cfbff85b5c662ab",
    "amount": 0.15,
    "payment_address": "2Mu4jQMi8RcvMKrpiPiCg8yYHdYCJCkve2i",
    "success": true
}

Closed by #1128


fixed: issue #378: There is a bug in the layout of the login/register…

Well detailed description of the change :
 - I worked on there is a bug in the layout of the login/register pages
Context of the change :

Why is this change required?
- because the loginSignup__heroImage and loginSignup__footer was overlap in UI during or at responsiveness level.

Does it solve a problem?

#378

Type of change :

[x] Bug fix
[ ] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Preview (Screenshots) :
 - Here at large screens, previously the loginSignup__heroImage could overlap into the loginSignup__header text.

- Here at Mid screens, previously the loginSignup__heroImage could still overlap into the loginSignup__header text as the loginSignup__footer overlaps beyond the loginSignup__body yet at the same time not fixed at the bottom of the page.

- Here at Small screens, previously still the loginSignup__heroImage could overlap into the loginSignup__header text as the loginSignup__footer overlaps beyond the loginSignup__body yet at the same time not fixed at the bottom of the page.

Checklist:

[x] I have read the CONTRIBUTING document.
[x] My code follows the STYLE_GUIDE of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] All new and existing tests passed.

Reviewers
@Mr-spShuvo, @divanov11

Good first RR.
However, never force pushed! It can causes merge conflicts.
This time I will merge it, but in future, force pushed will be ignored.

@Emmanuel-Dominic there is a bug in the footer (mobile view)


fix(tabs-extended): apply correct mobile gutters

Related Ticket(s)
Resolves #10298
Description
This PR aligns elements in the mobile version of the <dds-tabs-extended> component to the Carbon grid. It achieves this by replacing the custom accordion markup with <bx-accordion> web components, which has the correct mobile styling.
Changelog
Added

Adds "With Mixed Content" Story that demonstrates selected web components used within tabs.

Changed

Replaces custom accordion markup with @carbon/web-components' <bx-accordion> markup.
Makes tabs extend the full width available to them.

Removed

Removes styles that were used for custom markup.

Hey @jkaeser thanks for taking a stab at this!
I am thinking we may need a design spec for this to control some of the inconsistencies of the content that might be placed into the tabs extended component. Would AEM have a designer available to make a proposed design spec?

We'd want to ensure the spacing between the bottom of the tab and the beginning of the content is consistent, especially in a mixed content setting so the user isn't experiencing the shift.

Content item horizontal has 32px top spacing
Image has none (this makes it difficult to see the active tab state
Card group has some
Accordion has none

Same comment on consistent spacing within the smaller breakpoints with the accordion

vs

We will need to remove the accordion example, we would never recommend an accordion within a tab  or nested inside another accordion (mobile breakpoints). It's too many clicks for the user and hides the content too much.

You could replace it with a Content block or Content group as an example?

The cards don't seem to be rendering or behaving correctly

They are changing grid modes hanging in large vs not hanging in medium and small breakpoints

Desktop
- the content of the cards is all overlapping

Medium breakpoint

Cards are only spanning 3 columns
Expecting them to span 4 columns like card group

Current
Expecting

Small breakpoint

Cards are only spanning 3 columns
Expecting them to span 4 columns

Current
Expecting

We're going back to the drawing board based on @oliviaflory's feedback (thank you, Olivia 🙏). Closing this out with the understanding I may re-open it if this proves to be a good starting point.


Unit testing

Set up at least one unit test, along with CI/CD pipeline. Preferably using tox.

MyPy seems to fail on Tox via GHA. I think the MyPy version is the same as my local, so I don't know why. Need to investigate.

Managed to stop the tests from failing due to MyPy linting. Turns out numpy 1.21 changes the type requirements, which was the version installed on GHA, but I was using 1.20.3 locally.  I downgraded numpy in the dev build, but it seems unable to build wheels for Python 3.10. Perhaps an issue with how I specified the Python version in tox, but for the moment I will just test between Python 3.7 - 3.9.


[bug]: CLI add command breaks on comments in tailwind.config.ts

Describe the bug

      fontFamily: {
        sans: [
          // "var(--font-sans)",
          "var(--font-geist-sans)",
          ...fontFamily.sans,
        ],
        serif: ["var(--font-serif)", ...fontFamily.serif],
      },

Became
  		fontFamily: {
  			sans: [\\n          // "var(--font-sans)",\\n          "var(--font-geist-sans)",\\n          ...fontFamily.sans,\\n        ],\n  			serif: ["var(--font-serif)", ...fontFamily.serif]
  		},

Affected component/components
CLI
How to reproduce
Install something with a comment at the top of the fontfamily in tailwind config:
npx shadcn@latest add sidebar-01 sidebar-02 sidebar-03 sidebar-04 sidebar-05 sidebar-06 sidebar-07 sidebar-08 sidebar-09 sidebar-10 sidebar-11 sidebar-12 sidebar-13 sidebar-14 sidebar-15

https://github.com/lacymorrow/juicy-stack/pull/2/files#diff-655dc9e3d0aa561e3fa164bf48bd89cb0f5da65e0a567f8ebbf9dd791a0e7f40
Codesandbox/StackBlitz link
No response
Logs
No response
System Info
Latest CLI

Before submitting

[X] I've made research efforts and searched the documentation
[X] I've searched for existing issues

This also occurs with a fresh install of Remix and Shadcn. Just follow the official instructions to see

I think the only workaround is manually delete the \n               stuff for now.

it is coming as line breaks "\n" whenever installing a new component or block


feat(lidar_apollo_instance_segmentation, tensorrt_common): use onnx instead of caffe for apollo cnnseg

Description

🤖 Generated by Copilot at 3b4cb35
This pull request updates the lidar_apollo_instance_segmentation package and its dependencies to use the new TensorRT and CUDA libraries for faster and more robust lidar-based instance segmentation. It also improves the code quality, style, and consistency of the package by adding namespaces, include guards, license headers, and fixing typos and braces. It also simplifies the launch file and the .gitignore file by using the .onnx format for the neural network models.
Related links

Tests performed

Notes for reviewers

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

before(caffe)
Screencast from 04-10-2023 12:22:08 AM.webm
after(onnx)
Uploading Screencast from 04-10-2023 12:30:08 AM.webm…

Thank you for PR :+

@miursh @yukke42 Pls check this PR :+1:

@wep21
Thank you for PR. Regarding common/tensorrt_common, it looks good to me. Also, I confirmed that this PR's modification does not have negative effect for building/executing tensorrt_yolox

@yukkysaito You're the code owner of lidar_apollo_instance_segmentation, so I appreciate it if you could approve this PR.


CP-11583: DepthFirstSearchAlgorithm.Visit

From unknown CodePlex user on Saturday, 08 November 2008 01:42:22
depth parameter is not used.

Form unknown CodePlex user on Saturday, 08 November 2008 09:48:51
Resolved with changeset 24000.


Amazon Linux 2023 Support

Describe the Enhancement:
According to Chef Infra release notes, the latest stable version should support Amazon Linux 2022 already (18.x), but I can’t find a build for it.
It was introduced in PR #12437 and later removed in PR #13169
Amazon Linux 2022 is a release candidate and should be GA soon (hopefully)
https://docs.aws.amazon.com/linux/al2022/release-notes/relnotes-20230118.html
EDIT:
Amazon Linux 2023 is now GA and has replaced any reference to Amazon Linux 2022
https://docs.aws.amazon.com/linux/al2023/ug/release-cadence.html

Describe the Need:

Remove Amazon Linux 2022 as a supported OS from the documentation, or make it available.
Current Alternative

Can We Help You Implement This?:

Build succeeded, does that means it worked ?

@PrajaktaPurohit any updates ?

Looks like Amazon Linux 2022 is renamed to Amazon Linux 2023.
Amazon Linux 2023 is in preview release and is subject to change.
https://docs.aws.amazon.com/linux/al2023/ug/release-cadence.html

The build is successful, but it was not able to find the builders for Amazon Linux 2022. Still looking.

FYI, Amazon Linux 2023 went GA today: https://aws.amazon.com/about-aws/whats-new/2023/03/amazon-linux-2023/

Curious if there's any updates on this?

@PrajaktaPurohit any updates ? This was opened almost 3 months ago and no actions yet.

@PrajaktaPurohit Now 4 months since it was requested, a year after it was apparently supported and still no update ?

I think everyone here would highly appreciate if you provide ETA for AL2023 support. I can not install Chef on AL2023 and ETA for support would help to understand how to move further.

It is possible to work around this by installing libxcrypt-compat.
sudo yum install -y libxcrypt-compat

Hey folks, sorry for the delay on this one. We are currently in the process of prioritizing the addition of builders for Amazon Linux 2023 as a part of the PI planning process for the iteration beginning on the 1st of August. I will post an update based on the outcome of the planning by the end of July 23.

@PrajaktaPurohit - Any updates on this?

Are there any new updates on this issue? Thank you.

@PrajaktaPurohit are you able to comment on whether there are any workarounds for use with amazon Linux 2023?

Although we don't seem to have a request on the AL2023 GitHub project for chef, I can say that we are getting customer requests for Chef support for AL2023 through the non-GitHub channels available to customers.
Am happy to sync up out of band on possible options for those looking for Chef on Amazon Linux if that's preferred.

I am now using Ansible as Chef was taking too long.

Hey @stewartsmith 👋🏻
What would be the best way to reach out to you? Should we go through our AWS TAM?

Hey @stewartsmith 👋🏻
What would be the best way to reach out to you? Should we go through our AWS TAM?

TAMs are great because they're good at following up when devs and PMs are busy 😀
i.e. AWS customers should reach out to their TAM for best/most reliable results.
For the Chef product team, and chatting about how we can work together to help our mutual customers: you can either shot me an email, or connect through the normal AWS channels.

Hello! I'm trying to use Chef InSpec on Amazon Linux 2023 and came across the issue discussed here. Is there any update on compatibility or any recommended workaround that we can use in the meantime? The ability to effectively use Chef InSpec on Amazon Linux 2023 is crucial for our workflow. Thank you, any additional information would be greatly appreciated!

FYI https://docs.chef.io/release_notes_client/#18.4.12
We are now building packages for Amazon Linux 2023 on aarch64 and x86_64 platforms. #14228


How to add an external file with acronyms?

Hello, thank you for this plugin and it is a great help. However, my blog will keep adding acronyms and I am trying to figure out a better way to organize this. The gatsby-config file will not allow import statements so this is a bit tough to figure out. I have a dictionary.js file with an object of acronyms and I want to pass that object to the options instead of inline coding them in the gatsby-config file. Anyway around this?

You can use therequire keyword to import external files, as you can see here: https://github.com/PedroLamas/pedrolamas.com/blob/master/gatsby-config.js#L4

@PedroLamas thank you this worked!


Text in Basic Data Structures lesson is better clarified.

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Closes #40412

@GJ-16 You deleted the file instead of reverting your changes to the file.
To revert the changes to that file, do the followiing:
git checkout 3af3a2e6fe~1 -- tools/dashboard/one-off-scripts/prs-with-merge-conflicts.js
git add .
git commit -m "fix: reverted changes to prs-with-merge-conflicts.js"
git push origin dsa_a_text_changes

The above checks out the file in one commit earlier than your original first commit.  This effectively recreates the file as it was before any of your changes. Then you just add the file to be committed and create a new commit explaining you reverted the changes.
In the end you will have 3 commits but only show one file changed (the curriculum file).
Let me know if you run into any problems with the above commands.

Sorry, I messed somethings at my end. Will open new PR. Sorry again.

@RandellDawson I have some issue.
I didn't change any other file except curriculum/challenges/english/02-javascript-algorithms-and-data-structures/basic-data-structures/add-items-using-splice.md
Now I deleted the repo and forked again made change only to this file curriculum/challenges/english/02-javascript-algorithms-and-data-structures/basic-data-structures/add-items-using-splice.md then made a commit and push.
But still changes are made to this file (tools/dashboard/one-off-scripts/prs-with-merge-conflicts.js) as well.
Initially I thought I made some mistake while changing the file but I'm sure I ain't do any change to this file tools/dashboard/one-off-scripts/prs-with-merge-conflicts.js.
what I can do is just to edit and do a PR via Browser, but local development is not possible at moment.
Can you please help me with this?

@RandellDawson I have opened a new PR can you please check that #40492.
Thank you.