[SPARK-37035][SQL] Improve error message when use parquet vectorize reader

What changes were proposed in this pull request?
When we use parquet, found vectorized read won't show error message about failed read parquet file path, it cause different to find out which file have problem.
None-vectorized parquet reader
Exception: Encounter error while reading parquet files. One possible cause: Parquet column cannot be converted in the corresponding files. Details:
	at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.nextIterator(FileScanRDD.scala:193)
	at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:101)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
	at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:462)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:125)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.parquet.io.ParquetDecodingException: Can not read value at 1 in block 0 in file hdfs://R2/projects/data_notificationmart/dwd_traceid_sent_civ_first_di/tz_type=local/grass_region=TW/grass_date=2021-10-13/noti_type=AR/part-00013-22bdd509-4469-47f7-a37e-50fddd4266a7-c000.zstd.parquet
	at org.apache.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:251)
	at org.apache.parquet.hadoop.ParquetRecordReader.nextKeyValue(ParquetRecordReader.java:207)
	at org.apache.spark.sql.execution.datasources.RecordReaderIterator.hasNext(RecordReaderIterator.scala:39)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
	at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:101)
	at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.nextIterator(FileScanRDD.scala:181)
	... 15 more

Vectorize parquet reader
21/10/15 18:01:54 WARN TaskSetManager: Lost task 1881.0 in stage 16.0 (TID 10380, ip-10-130-169-140.idata-server.shopee.io, executor 168): TaskKilled (Stage cancelled)
: An error occurred while calling o362.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 963 in stage 17.0 failed 4 times, most recent failure: Lost task 963.3 in stage 17.0 (TID 10351, ip-10-130-75-201.idata-server.shopee.io, executor 99): java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainIntegerDictionary
	at org.apache.parquet.column.Dictionary.decodeToLong(Dictionary.java:49)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetDictionary.decodeToLong(ParquetDictionary.java:36)
	at org.apache.spark.sql.execution.vectorized.OnHeapColumnVector.getLong(OnHeapColumnVector.java:364)
	at org.apache.spark.sql.execution.vectorized.MutableColumnarRow.getLong(MutableColumnarRow.java:120)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.writeFields_0_0$(Unknown Source)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(Unknown Source)
	at org.apache.spark.sql.execution.FileSourceScanExec$$anonfun$doExecute$2$$anonfun$apply$2.apply(DataSourceScanExec.scala:351)
	at org.apache.spark.sql.execution.FileSourceScanExec$$anonfun$doExecute$2$$anonfun$apply$2.apply(DataSourceScanExec.scala:349)
	at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
	at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:463)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:125)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

This is caused in vectorized reader, it use Dictionary don't have file information,
public abstract class Dictionary {
    private final Encoding encoding;

    public Dictionary(Encoding encoding) {
        this.encoding = encoding;
    }

    public Encoding getEncoding() {
        return this.encoding;
    }

    public abstract int getMaxId();

    public Binary decodeToBinary(int id) {
        throw new UnsupportedOperationException(this.getClass().getName());
    }

    public int decodeToInt(int id) {
        throw new UnsupportedOperationException(this.getClass().getName());
    }

    public long decodeToLong(int id) {
        throw new UnsupportedOperationException(this.getClass().getName());
    }

    public float decodeToFloat(int id) {
        throw new UnsupportedOperationException(this.getClass().getName());
    }

    public double decodeToDouble(int id) {
        throw new UnsupportedOperationException(this.getClass().getName());
    }

    public boolean decodeToBoolean(int id) {
        throw new UnsupportedOperationException(this.getClass().getName());
    }
}

and spark have wrapper a ParquetDictionary so we can do this in ParquetDictionary, to add file information in error message.
Why are the changes needed?
Improve error message to help make sure which file have problem.
Does this PR introduce any user-facing change?
User can know which file failed to read when use parquet vectorized  reader
How was this patch tested?
WIP

Test build #144349 has started for PR 34308 at commit 1730662.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48827/

Test build #144349 has finished for PR 34308 at commit 1730662.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #144364 has finished for PR 34308 at commit 56407de.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

ping @dongjoon-hyun

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48910/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48910/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144437/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144489/

retest this please

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48964/

also ping @srowen

@sadikovi do you know to construct a parquet file with bad dictionary, for testing?

I guess this can be reproduced by setting read schema to be long when reading a int typed Parquet dictionary, since we don't support schema evolution for Parquet yet. Let me try to write a code snippet for it.

Here it is @AngersZhuuuu @cloud-fan
  test("XXX") {
    val data = (0 to 10).flatMap(n => Seq.fill(10)(n)).map(i => (i, i.toString))
    withParquetFile(data) { path =>
      val readSchema = StructType(Seq(StructField("_1", LongType)))
      spark.read.schema(readSchema).parquet(path).first()
    }
  }

I usually load the file in memory and flip bits or replace bytes of data in a byte range that corresponds to the dictionary page. Because everything else can be constant, this allows you to trigger various decoding issues from compression to thrift metadata to data page encoding for dictionary. I can provide code if needed.

 test("XXX") {
    val data = (0 to 10).flatMap(n => Seq.fill(10)(n)).map(i => (i, i.toString))
    withParquetFile(data) { path =>
      val readSchema = StructType(Seq(StructField("_1", LongType)))
      spark.read.schema(readSchema).parquet(path).first()
    }
  }

Tried this before, won't throw decode error.

Test build #144920 has started for PR 34308 at commit b4318f8.

I usually load the file in memory and flip bits or replace bytes of data in a byte range that corresponds to the dictionary page. Because everything else can be constant, this allows you to trigger various decoding issues from compression to thrift metadata to data page encoding for dictionary. I can provide code if needed.

Would be great that if you can provide such code that can reproduce the decoding error.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49391/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49391/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49391/

Test build #144920 has finished for PR 34308 at commit b4318f8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144920/

Oops sorry @AngersZhuuuu . Maybe we can try the following then:
  test("XXX") {
    val data = (0 to 10).flatMap(n => Seq.fill(10)(n)).map(i => (i, i.toString))
    withParquetFile(data) { dir =>
      val file = SpecificParquetRecordReaderBase.listDirectory(new File(dir)).get(0)
      val filePath = new Path(file)
      val reader = ParquetFileReader.open(HadoopInputFile.fromPath(filePath, new Configuration))
      try {
        val descriptor = reader.getFileMetaData.getSchema.getColumns.get(0)
        val pages = reader.readNextRowGroup().getPageReader(descriptor)

        val dictionaryPage = pages.readDictionaryPage()
        assert(dictionaryPage != null, "dictionaryPage shouldn't be null")
        val dictionary = dictionaryPage.getEncoding.initDictionary(descriptor, dictionaryPage)
        val parquetDictionary = new ParquetDictionary(dictionary, file, true)
        val msg = intercept[UnsupportedOperationException] {
          parquetDictionary.decodeToInt(0)
        }.getMessage
        assert(msg.contains("Decoding to Int is not supported"))
      } finally {
        reader.close()
      }
    }
  }

Oops sorry @AngersZhuuuu . Maybe we can try the following then:
  test("XXX") {
    val data = (0 to 10).flatMap(n => Seq.fill(10)(n)).map(i => (i, i.toString))
    withParquetFile(data) { dir =>
      val file = SpecificParquetRecordReaderBase.listDirectory(new File(dir)).get(0)
      val filePath = new Path(file)
      val reader = ParquetFileReader.open(HadoopInputFile.fromPath(filePath, new Configuration))
      try {
        val descriptor = reader.getFileMetaData.getSchema.getColumns.get(0)
        val pages = reader.readNextRowGroup().getPageReader(descriptor)

        val dictionaryPage = pages.readDictionaryPage()
        assert(dictionaryPage != null, "dictionaryPage shouldn't be null")
        val dictionary = dictionaryPage.getEncoding.initDictionary(descriptor, dictionaryPage)
        val parquetDictionary = new ParquetDictionary(dictionary, file, true)
        val msg = intercept[UnsupportedOperationException] {
          parquetDictionary.decodeToInt(0)
        }.getMessage
        assert(msg.contains("Decoding to Int is not supported"))
      } finally {
        reader.close()
      }
    }
  }

Thanks a lot, such way can trigger this an I have tried but failed to use these apis. Have updated and add you as the co-author.

Test build #144949 has started for PR 34308 at commit 5b88a26.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49421/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49421/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49421/

Test build #144949 has finished for PR 34308 at commit 5b88a26.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144949/

So what does the new error message look like? Can you provide a stacktrace?

Full stack
va.lang.UnsupportedOperationException: Decoding to Int is not supported by PlainIntegerDictionary while reading file /Users/yi.zhu/Documents/project/Angerszhuuuu/spark/target/tmp/spark-fb0360ae-32ce-4504-93c5-214167e7258a/part-00001-c8610250-45a7-4b80-ae56-6e7e0ee295bd-c000.snappy.parquet
	at org.apache.spark.sql.errors.QueryExecutionErrors$.unsupportedParquetDictionaryDecodingError(QueryExecutionErrors.scala:603)
	at org.apache.spark.sql.errors.QueryExecutionErrors.unsupportedParquetDictionaryDecodingError(QueryExecutionErrors.scala)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetDictionary.decodeToInt(ParquetDictionary.java:48)
at org.apache.spark.sql.execution.datasources.parquet.ParquetIOSuite.$anonfun$new$234(ParquetIOSuite.scala:1106)
	at scala.runtime.java8.JFunction0$mcI$sp.apply(JFunction0$mcI$sp.java:23)
	at org.scalatest.Assertions.intercept(Assertions.scala:749)
	at org.scalatest.Assertions.intercept$(Assertions.scala:746)
	at org.scalatest.funsuite.AnyFunSuite.intercept(AnyFunSuite.scala:1563)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetIOSuite.$anonfun$new$233(ParquetIOSuite.scala:1105)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetIOSuite.$anonfun$new$233$adapted(ParquetIOSuite.scala:1093)
	at org.apache.spark.sql.execution.datasources.FileBasedDataSourceTest.$anonfun$withDataSourceFile$1(FileBasedDataSourceTest.scala:70)
	at org.apache.spark.sql.execution.datasources.FileBasedDataSourceTest.$anonfun$withDataSourceFile$1$adapted(FileBasedDataSourceTest.scala:68)
	at org.apache.spark.sql.catalyst.plans.SQLHelper.withTempPath(SQLHelper.scala:69)
	at org.apache.spark.sql.catalyst.plans.SQLHelper.withTempPath$(SQLHelper.scala:66)
	at org.apache.spark.sql.QueryTest.withTempPath(QueryTest.scala:34)
	at org.apache.spark.sql.execution.datasources.FileBasedDataSourceTest.withDataSourceFile(FileBasedDataSourceTest.scala:68)
	at org.apache.spark.sql.execution.datasources.FileBasedDataSourceTest.withDataSourceFile$(FileBasedDataSourceTest.scala:65)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetIOSuite.withDataSourceFile(ParquetIOSuite.scala:59)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetTest.withParquetFile(ParquetTest.scala:65)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetTest.withParquetFile$(ParquetTest.scala:63)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetIOSuite.withParquetFile(ParquetIOSuite.scala:59)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetIOSuite.$anonfun$new$229(ParquetIOSuite.scala:1093)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
	at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
	at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
	at org.scalatest.Transformer.apply(Transformer.scala:22)
	at org.scalatest.Transformer.apply(Transformer.scala:20)
	at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
	at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:190)
	at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
	at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
	at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
	at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
	at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
	at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:62)
	at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
	at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
	at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:62)
	at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
	at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
	at scala.collection.immutable.List.foreach(List.scala:431)
	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
	at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
	at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
	at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
	at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
	at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1563)
	at org.scalatest.Suite.run(Suite.scala:1112)
	at org.scalatest.Suite.run$(Suite.scala:1094)
	at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1563)
	at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
	at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
	at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
	at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
	at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:62)
	at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
	at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
	at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
	at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:62)
	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)
	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)
	at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedOperationException: org.apache.parquet.column.values.dictionary.PlainValuesDictionary$PlainIntegerDictionary
	at org.apache.parquet.column.Dictionary.decodeToLong(Dictionary.java:49)
	at org.apache.spark.sql.execution.datasources.parquet.ParquetDictionary.decodeToInt(ParquetDictionary.java:43)
	... 64 more

Also, I think we should include the column name in the error message if possible, otherwise it might be difficult to debug when you have 300 dictionary encoded columns.

Done

Test build #144979 has started for PR 34308 at commit 0af1bd4.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49450/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49450/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49450/

Test build #144979 has finished for PR 34308 at commit 0af1bd4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144979/

ping @sunchao @cloud-fan

After deep check this cause, this exception should be caused we have parquet files with different schema, then  if we don't set mergeSchema, then it will directly use the first file's schema to read data, so first file is long type and next file is int type for one column, when read the second file, it will read long data but the column's descriptor is int type, then will throw such Unsupported decoding problem.
But such caused should have been denied by @sunchao 's this pr https://github.com/apache/spark/pull/32777
And such case will be denied when call ParquetVectorUpdaterFactory.getUpdator()  and will throw exception will file path.
[info]   Cause: org.apache.spark.sql.execution.QueryExecutionException: Parquet column cannot be converted in file file:///Users/yi.zhu/Documents/project/Angerszhuuuu/spark/target/tmp/spark-3eccc50d-9d9c-4970-9674-87de46ea1192/test-002.parquet/part-00000-4332031b-e514-4b95-b52a-e8d798c999e6-c000.parquet. Column: [a], Expected: bigint, Found: INT32
[info]   at org.apache.spark.sql.errors.QueryExecutionErrors$.unsupportedSchemaColumnConvertError(QueryExecutionErrors.scala:586)
[info]   at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.nextIterator(FileScanRDD.scala:172)
[info]   at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:93)

Thanks all for your help. @sunchao @cloud-fan @sadikovi .
Hope your confirm and then I will close this one.

ah, thanks for the confirmation! I think it proves that it's important to write end-to-end tests for errors, to avoid wasting time centralizing the error message, defining error class, etc. while the error can never be hit.
I think we can close it.

ah, thanks for the confirmation! I think it proves that it's important to write end-to-end tests for errors, to avoid wasting time centralizing the error message, defining error class, etc. while the error can never be hit.
I think we can close it.

Yea, it also proves that all the error we should find out the root cause. Then things can be easier.


The exported APK is unsigned, but the exported AAB is signed

Tested versions
4.2.1.stable
System information
Windows-11
Issue description
Problem:
When "use gradle build" is checked, use "keytool" to show that the file is signed when exporting to AAB format. But when
choosing to export to APK format, using "keytool" shows that the file has no signature and the file size is large. After decompressing the APK file, I found that the files in the "lib" folder were not compressed.
Steps to reproduce
Premise:
1, Configure the keystore properly, and check "package signed".
2, Regardless of whether "use gradle build" is checked or not, the real machine can run normally.
Minimal reproduction project (MRP)
Unable to provide source file.

me too


Fix wrong cli command in documentation to remove any docker volumes

Missing character "'" in line 24

This is good to go after you sign the silly CLA, thanks @dkoether .


Theta and CNVkit

Hello,
I am trying to get Theta hetcalls, and encountering a few errors. The first occurred in the export_theta function of cnvkit.py. The error was "unknown argument: path/file.cnr". Looking at the CNVKit manual, it seemed as tho CNVkit didn't need this file to export data formatted for theta input. So, I commented that out (see snippet 1, below),  and indeed a theta input file was generated, but this time I get the following error (see snippet 2, below). Looking at the empty dict in convert.SUBSET_BY_SUPPORT, and the functions in convert.py, it isn't clear to me how this dict gets populated. Any thoughts on this? Have I broken the code? And if so, why does CNVKit not recognize the .cnr file as a filetype it should be reading as an argument. Thanks in advance!

#original
#cmd = [_get_cmd(), "export", "theta", cns_file, cnr_file, "-o", tx_out_file]
#new
cmd = [_get_cmd(), "export", "theta", cns_file, "-o", tx_out_file]

convert.SUBSET_BY_SUPPORT["cnvkit"]]
KeyError: 'cnvkit'

David;
Sorry about the issues with theta support. We didn't have a lot of success with theta so haven't worked on this in a while and it appears to have gotten out of sync with changes in cnvkit. We'd have to investigate or happy to accept patches if you're able to dig into it more. Alternatively I'd suggest different approaches like TitanCNA for heterogeneity if you have tumor/normal samples. Thanks again for reporting this.

Hi Brad,
Thank you for the quick reply! I will dig in more and let you know if I
find an easy fix. Thanks for the tip on TitanCNA... I am going to test that
now. I already have bubbletree working, and just wanted to have two methods
for comparison sake... so, Titan might do the trick. Either way, I'll let
you know if I have luck with Theta.
Cheers!
On Wed, Aug 29, 2018 at 6:12 AM Brad Chapman<EMAIL_ADDRESS>wrote:

David;
Sorry about the issues with theta support. We didn't have a lot of success
with theta so haven't worked on this in a while and it appears to have
gotten out of sync with changes in cnvkit. We'd have to investigate or
happy to accept patches if you're able to dig into it more. Alternatively
I'd suggest different approaches like TitanCNA for heterogeneity if you
have tumor/normal samples. Thanks again for reporting this.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bcbio/bcbio-nextgen/issues/2494#issuecomment-416947234,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMUKdoeurmu3e8xN10aOYgoZ7_vzVxe_ks5uVpMigaJpZM4WQF7u
.

Thanks, we still haven't worked much on Theta, so closing this issue. Thank you!


Samples on Task.Yield

This documentation should deserve a sample or a link to a related article that contains it but on its own section of sample. documentation with no sample are really hard to adopt.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3239bb41-1ef3-f64b-4a45-1ae747a96643
Version Independent ID: 3b725311-f6b1-5321-be79-f46ff04b221c
Content: Task.Yield Method (System.Threading.Tasks)
Content Source: xml/System.Threading.Tasks/Task.xml
Product: .net
GitHub Login: @dotnet-bot
Microsoft Alias: dotnetcontent

Thanks for pointing out the absence of an example where one is needed, @JuanMejiaVelez. We've added it to our backlog.

What exactly would you expect from that sample? The Remarks section already shows you how to use the method: await Task.Yield();. I'm not sure how a longer example would help here.


Command name fuzzy matching for typo correction suggestions

If a command isn't found, attempt to provide a suggestion on the command they were actually trying to run

Closing because I forgot I wasn't done with it


Bugs with crithack

So I've used crithack for the last 1 hour and I've experienced some bugs.

Holding crithack while shooting with pistols & mini guns, causes your game to lag slightly. (Depends on the pc)
Using a bind to fix invisible players (record fix;stop) causes the bucket to reset back to 300
Pressing the crit key once causes it to force crits everytime, (doesn't happen while using aimbot) so farming crits is kinda harder.

Other than that, the crithack works great.

it doesn't work

wdym it doesn't work? I literally used it both in casual and local server and it works fine

I've not used this cheat for a while so Idk

The crithack recently got fixed and it works great

yeah uh its not working that well i was testing

just tested it, it does work, I also get some issues too.

are you sure on the last bug, ive done some testing with it but i dont use crithack a lot and it seemed fine with having normal crits when i wasnt holding the button

@xxmyn he means the bug where you press a key and it still does the thing after you release it until the thing it does has been done

That occurs case by case, and I fixed it on edge jump earlier which was the only case of it I know of.

It happens with DT, crits and warp (when 0 charge)

I just tested DT, warp and the crit key in a local server and had no issues with any of them.

Well it's happened to me regardless. Not an issue, but put DT on key, press your key but don't shoot, move around a bit then shoot while not holding your key. It will DT.

I've tested all of the issues you've described in the post, the only one I can recreate is the "demo record" one, this occurs in nullcore from memory and isn't really a problem imo as if you're having to use that ghetto bind to fix invis players you are going to have bigger issues than your crit bucket indicator being inaccurate.

The crit bucket is inaccurate anyway.

i thought i closed this

here's my config
https://anonfiles.com/FcD9M6R9x3/sexer_fed


fix: Error having URI hash, but no element in Dom

FIX DOMException: Document.querySelector: '#5' is not a valid selector vue-scrollactive.min.js:385
scrollToHashElement vue-scrollactive.min.js:385
mounted vue-scrollactive.min.js:257

Correction when not finding element in the dom and gives general error


Sylius 1.9 Symfony 5.3

Update to Sylius 1.9 from Sylius plugin skeleton

I don't see a change in the .github folder, with the workflow setup as in PluginSkeleton. Can you also have a look at that one @vicla31? Probably good to align the setup there also, to make sure we install dependencies etc. the same way.

Finalised in #184


Reminder emails for monthly leadership assessments are being sent to positions that don't have an advisor assigned

Describe the bug
An advisor has redeployed.  The outgoing advisor's profile becomes inactive and the advisor's former position is vacant.  However, because the Active Directory accounts are role-based (and not name-based), an incoming advisor assumes the outgoing advisor's AD account and email address.  The incoming advisor does not create an ANET profile and the outgoing advisor's position stays vacant.  However, at the first of the month, the incoming advisor receives reminder emails to assess his/her principal counterpart.
To Reproduce
Condition 1: An (outgoing) advisor has redeployed.
Condition 2: The outgoing advisor's profile is set to inactive.
Condition 3: The outgoing advisor's position is vacant, i.e. another advisor is not assigned to the outgoing advisor's former position.
Condition 4: The outgoing advisor's position had one or more principal's assigned.
Condition 5: The outgoing advisor's AD account is role-based.
Condition 6: An incoming advisor is assigned to the outgoing advisor's AD account and assumes the outgoing advisor's email address.
Condition 7: The incoming advisor does not create an ANET profile.
Condition 8: It is the first of the month.
Result/Bug: The incoming advisor receives a reminder email to assess their principal counterpart.  However the incoming advisor does not have an ANET profile and as such is not assigned to an advisor position.
Expected behavior
Incoming advisors that have not created an ANET profile, should not receive ANET leadership assessment reminder emails
Screenshots
None
Environment (please complete the following information):
Unknown
Additional context
This happened to two advisors.  Both of their predecessors had ANET profiles and were assigned to advisor positions.  For some reason, the incoming advisors did not create ANET profiles.
Good thing the ANET support email is displayed at the bottom of the reminder email.

It was a timing issue. At the moment the PendingAssessmentsNotificationWorker ran, the outgoing advisor was still assigned, so the email was sent to that email address. The incoming advisor, when opening that email account, saw that message in their INBOX.

This was not a bug.
The reminder email was sent 1 April.
The outgoing advisor's ANET profile was inactivated on 3 April.
The incoming advisor had already assumed the outgoing advisor's email account.


Pin mkdocstrings==0.13.2

What does this PR do?

Pin mkdocstrings to version 0.13.2
Motivation

Latest release 0.13.3 is broken: https://github.com/pawamoy/mkdocstrings/issues/154
And makes our docs CI job fail, eg: https://github.com/DataDog/integrations-core/pull/7661/checks?check_run_id=1165278158
Not clear yet how exactly the mkdocstrings bug should be resolved (though I pushed a PR: https://github.com/pawamoy/mkdocstrings/pull/155) and how quickly, so pinning to unblock our builds in the meantime.
Additional Notes

Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] PR title must be written as a CHANGELOG entry (see why)
[ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
[ ] PR must have changelog/ and integration/ labels attached

Closing, a bug fix has been released: https://pypi.org/project/mkdocstrings/0.13.4, docs job now succeeding again ✅


Error in docs of create_collections(): parentCollection instead of parent

Creating a subcollection as described in the docs fails with the following error message:

'code': 400, 'message': "'parent' property is now 'parentCollection'"

In the dict passed as the first input parameter to create_collections() one must use the key parentCollection and not parent.
I guess the docs should be updated accordingly.
Code example that fails:
zot = zotero.Zotero(library_id, library_type, api_key)
result = zot.create_collections([{'parent': parent_id, 'name': 'test'}])

Code example that runs successfully:
zot = zotero.Zotero(library_id, library_type, api_key)
result = zot.create_collections([{'parentCollection': parent_id, 'name': 'test'}])

Thanks!


fix(java): module "" not found error

Struct proxies did not trigger loading of their parent module when
passed as a parameter, which then causes the @jsii/kernel type checker
to be unable to locate the struct type definition for checking pruposes,
leading to a crash.
This updatates the JsiiSerializable serialization process to
proactively load the relevant modules (the call is idempotent and short
circuits if the module is known to already be loaded).
Fixes #1904

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Labeled with pr/do-not-merge as I still want to add a similar test in Python & .NET to confirm the bug does not affect them.

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: cb95336272bfa60e2f93dcf5114772a4a698f469
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: dfbf5da0c3a0758b14d76e67c0373ec2a5b54b95
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: aca81d023d3ce8aaacffbfd6e0f1109f96731d45
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: a288bb407a7cb9192cef4f5b3363e0939bfa90b4
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: 37e8623a9e383881408d064a4995517345e4e029
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Was this only triggered by nested structs for some reason?

@MrArnoldPalmer yeah - the bottom line is that Java does not issue a create call to the kernel upon initialization, which is where the load for the hosting module happens for other objects (the struct proxies are "pure data" so they don't appear in the JS process before they're actually sent across the boundary).
The nesting might not add to the problem (but I had it to reproduce the original issue discovered, and I reckon this makes it a slightly more hostile pattern - meaning I have more chances to catch more issues with that :D).

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: 9caa1b92a084dfeb294e8b929c2080e0417b4c04
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: 48a4a864aef1e437fb836a7a2b35f495653a0cb9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: de747c6a57c5bbc2603ea490098dfb95d9be374b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: 5e460c5bd8ad6794fc6c8329763aff21af3c20f5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Historical PDF layout: 1972 to 1978 layout

(Originally split from #1090.)
During this period ISO used a "prettier" layout that has a wide header.
This layout was in use from 1972 to 1979. In 1979, the new 1979 layout started being used from 1979-04-15, and both 1972 and 1979 layouts were used together from 1979-04-01 to 1979-06-15.
These are PDF samples from iTeh that provide a good demonstration of the layout.
ISO-1988-1975.pdf
ISO-1966-1973.pdf
ISO-2141-1972.pdf
ISO-1980-1977.pdf
ISO-2000-1978.pdf
ISO-4068-1978.pdf
ISO-4605-1978.pdf
History of layout transition
Phase out of old layout:

1979-04-01: ISO 1001, ISO 5698, ISO 56/I in old layout
1979-04-15: ISO 4967, ISO 4001/I, ISO 4002/I, ISO 3004/II, 4570/II, ISO 748 (corrected 04-15), 4967, 4109, 4330,  in old layout
1979-05-01: ISO 5264/III still in old layout
1979-05-15: ISO 1073/I (corrected in 1979-05-15) still in old layout
1979-06-01: nothing in old layout
1979-06-15: ISO 1073/II (corrected in 1979-06-15) still in old layout
1979-07-01: nothing in old layout
1979-07-15: nothing in old layout

Transition into new layout:

1979-04-01: nothing in new layout
1979-04-15: ISO 4335, ISO 5678, 5679, 4375, 4360, 668, 4087, 1813, 2635, 113/I, 4166, 4168, 4360, 668, 113/I, 2635, 1813 in new layout
1979-05-01: all except one in old layout
1979-05-15: all except one in old layout
1979-06-01: all in new layout
1979-06-15: all except one in old layout
1979-07-01: all in new layout
1979-07-01: all in new layout

Heading styles / fonts
ISO-5703-1979.pdf
Level 1 and 2:

Level 3 and 4:

From https://github.com/metanorma/metanorma-iso/issues/1090:

Cover title
1972 to 1978

(The closest free font I found was Boricua Regular.)
Proper font Univers:

The bottom tiny text in English/French doesn't change, it is Univers Pro 55 Roman. At one point the Russian/Cyrillic font changes from serif to sans-serif, but we don't care right now.

ISO XSLT updated.


error: call of overloaded ‘isinf(double&)’ / ‘isnan(double&)’ is ambiguous (GCC 5.x)

Hi Silvio,
I've attemped to build kASA with GCC 5.4.0, and I receive these two compiler errors (after having done cmake .. followed by make in the build directory, as per your instructions):
In file included from /path/to/kASA/source/modes/Compare.hpp:16:0,
                 from /path/to/kASA/source/main.cpp:13:
/path/to/kASA/source/modes/../utils/dToStr.h: In function ‘void dtoa_milo(double, std::__cxx11::string&)’:
/path/to/kASA/source/modes/../utils/dToStr.h:431:17: error: call of overloaded ‘isnan(double&)’ is ambiguous
  if (isnan(value)) {

[… other warnings redacted …]

In file included from /path/to/kASA/source/modes/Compare.hpp:16:0,
                 from /path/to/kASA/source/main.cpp:13:
/path/to/kASA/source/modes/../utils/dToStr.h:435:17: error: call of overloaded ‘isinf(double&)’ is ambiguous
  if (isinf(value)) {
                 ^

If you're wondering, I chose GCC 5.4.0 from among the modules on our cluster just because it had the required C++17 support, and was in the middle of the range of versions we had available, and that seemed "safe."
I should also note that kASA compiles without errors (two warnings about control reaches end of non-void function) with GCC 8.2.0. As this is a much more recent GCC version, I suppose the solution to the problem stated in this issue may well be "use a more recent version of GCC."
However, I wanted to record the error message here, for posterity, in case anyone else is stuck in an environment using an older GCC. Feel free to tag the issue "wontfix" and close it, as you see fit.

Hey Kevin,
thank you for bringing this to my attention! I really should be more sensitive to my namespaces :D
Anyway, it's fixed now and I've tested it on 5.4.0-1 which is also available on our cluster as well - without an issue now.
The warnings of control reaches end of non-void function come from the STXXL and are of no importance to kASA. Beyond that: C++17 is a nice-to-have but not really a requirement for it (just to be petty^^).
Closed because fixed.


python3Packages.qcs-api-client: 0.20.4 -> 0.20.5

Motivation for this change
Update to latest upstream release 0.20.5
Change log: https://github.com/rigetti/qcs-api-client-python/releases/tag/v0.20.5
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 148394 run on x86_64-linux 1

8 packages built:

python38Packages.cirq
python38Packages.cirq-rigetti
python38Packages.pyquil
python38Packages.qcs-api-client
python39Packages.cirq
python39Packages.cirq-rigetti
python39Packages.pyquil
python39Packages.qcs-api-client


log stage template

Closes #468.
Logs output during Live.end() like:
INFO:dvclive:To run experiments with DVC, add this to /Users/dave/Code/lstm_seq2seq/dvc.yaml:
stages:
  dvclive:
    cmd: <python my_code_file.py my_args>
    deps:
    - <my_code_file.py>
    outs:
    - params.yaml:
        cache: false
    - metrics.json:
        cache: false
    - results/plots:
        cache: false
    - model

Codecov Report
Patch coverage: 92.51% and project coverage change: +0.14 :tada:

Comparison is base (3308c68) 95.71% compared to head (67c2f0b) 95.86%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #497      +/-   ##
==========================================
+ Coverage   95.71%   95.86%   +0.14%
==========================================
  Files          37       39       +2
  Lines        2336     2466     +130
  Branches      196      204       +8
==========================================
+ Hits         2236     2364     +128
  Misses         61       61
- Partials       39       41       +2

Impacted Files
Coverage Δ

src/dvclive/lightning.py
91.07% <71.42%> (-2.93%)
:arrow_down:

src/dvclive/live.py
91.09% <72.22%> (-0.05%)
:arrow_down:

src/dvclive/dvc.py
93.33% <100.00%> (+3.04%)
:arrow_up:

src/dvclive/plots/metric.py
97.36% <100.00%> (-0.07%)
:arrow_down:

src/dvclive/serialize.py
90.32% <100.00%> (ø)

src/dvclive/utils.py
82.35% <100.00%> (-1.35%)
:arrow_down:

tests/conftest.py
93.93% <100.00%> (+5.70%)
:arrow_up:

tests/test_dvc.py
100.00% <100.00%> (ø)

tests/test_frameworks/test_lightning.py
97.95% <100.00%> (+0.87%)
:arrow_up:

tests/test_log_artifact.py
100.00% <100.00%> (ø)

... and 2 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

TODO: tests


feat: check timepoints defined time values

Summary:
This PR provides changes to check: timepoint define time values in stop_times.txt.
New notice

StopTimeTimepointWithoutTimesNotice

Expected behavior:
A WARNING  should be issued if a row from stop_times.txtis defined as a timepoint (i.e. timepoint == 1) and one of the time fields (stop_times.arrival_time or stop_time.departure_time is missing)
Please make sure these boxes are checked before submitting your pull request - thanks!

[x] Run the unit tests with gradle test to make sure you didn't break anything
[x] Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
[x] Linked all relevant issues
~[ ] Include screenshot(s) showing how this pull request works and fixes the issue(s)~

@MobilityData/transit-specs, here the rule is implemented as a WARNING however we believe that this should be an ERROR based on the past GTFS spec definition before timepoint was added. When timepoint was added the language wasn't as descriptive as it should have been to make it clear that this is an error. Here's the related GitHub discussion for a proposal to clarify this - https://github.com/google/transit/issues/61.
@barbeau

Thank you all for your review!
@aababilov
@barbeau
@maximearmstrong


setups cleanup (move to neorv32-setups repository)

This is a clean-up PR that removes setups and according links, workflows and documentation. All example setups have already been moved to new neorv32-setups repository.
Changes made in this PR:

removed setups folder; now located in neorv32-setups
removed rtl/templates folder; now located in neorv32-setups/examples/processor_templates
removed implementation-related workflows:

removed .github/generate-job-matrix.py
removed .github/workflows/Implementation.yml
removed implementation-part from .github/workflows/Windows.yml

added rtl/wrappers/ that contains the std_logic top entity and the AXI4-Lite top entity
added rtl/test_setups/: contains two minimalist test setups (one using the bootloader, one using pre-installed executable in IMEM ROM) that are used for the reworked "General Hardware Setup" section of the user guide
modified READMEs and documentation (data sheet & user guide) according to cleaned-up repository

TODO:

clean-up workflow triggers
...? 🤔

@stnolting, I cannot look thoroughly into this at the moment. I just want to point out that CI workflows don't need to be removed here. It is up to you to have CI in a single repo or split between the repos, regardless of where the code is located. I will write later provide examples of cross-triggering between repos and how to do it without cross-triggers.

I cannot look thoroughly into this at the moment.

No stress! The text here is just a check-list for me :wink:

I just want to point out that CI workflows don't need to be removed here. It is up to you to have CI in a single repo or split between the repos, regardless of where the code is located. I will write later provide examples of cross-triggering between repos and how to do it without cross-triggers.

I think it is cleaner to keep only the workflows here, that are actually used for the processor core. All implementation-specific workflow should go to neorv32-setups. Maybe we can add a trigger there so the workflow gets executed every time neorv32/rtl changes. But that is not really urgent right now.

I think it is cleaner to keep only the workflows here, that are actually used for the processor core. All implementation-specific workflow should go to neorv32-setups. Maybe we can add a trigger there so the workflow gets executed every time neorv32/rtl changes. But that is not really urgent right now.

The point is that you are proposing to move away all the workflows which test synthesis! So, we are not just reorganising, but losing feedback about a fundamental feature of the project (whether it is synthesisable or "a simulation model only").
We don't need to run the Implementation workflow here. We can just execute ghdl --synth here, and have the board specific implementation tests executed elsewhere. However, if the templates are either moved or removed (see #128) we run out of design tops to run the synthesis here.
With regard to triggers, please see gitter.im/hdl/community?at=6106c042c9f8852a971c798d. In this case, I believe the best strategy is to have a subset of the tests in this CI, which can be executed without external dependencies (except maybe the riscv-arch-tests submodule). Then, have several tests in the setups repository, executed against the submoduled NEORV32, as well as the 'master' branch. However, I do not recommend setting cross-triggers (due to the permission issues).

The point is that you are proposing to move away all the workflows which test synthesis! So, we are not just reorganising, but losing feedback about a fundamental feature of the project (whether it is synthesisable or "a simulation model only").

Right, I did not think about this enough. 🙈
We do need a synthesis target here! And I think we need a lot of "different" synthesiseses (plural "synthesis"). Maybe we could use your idea from #128 and run a synthesis for every "single-module-entity-wrapper". 🤔

With regard to triggers, please see gitter.im/hdl/community?at=6106c042c9f8852a971c798d. In this case, I believe the best strategy is to have a subset of the tests in this CI, which can be executed without external dependencies (except maybe the riscv-arch-tests submodule). Then, have several tests in the setups repository, executed against the submoduled NEORV32, as well as the 'master' branch. However, I do not recommend setting cross-triggers (due to the permission issues).

I saw your mention there but I did not have time to look through that yet.

Some ideas to get this resolved:

I like the idea of having explicit setups for "test setups" only - so not tied to any specific board/platform/toolchain. That is the purpose of the rtl/test_setups/
rtl/wrappers/ should contain wrappers for system integration. I know the term "wrapper" might be misleading, so maybe something like rtl/integration/ or rtl/connectivity/ would be better ("system" might also be misleading here).
Let's undo the removal of rtl/templates/ :+1: maybe we can name that processor_templates to be more specific. But we should rework the templates there. The Minimal* can be removed I think. If we use rtl/test_setups/ then these setups should be used for those minimal "hello world" examples. Additionally, we should clean-up the templates. Maybe we can use something like three templates there but I am not sure about the actual configuration.
As discussed earlier, let's use some names for those templates that do not provide any connotations at all. You had a great table somewhere (I need to check for that again). Maybe we can use a simplified concept and put that table into the rtl folder's README (not in the actual documentation due to maintaining :wink:)

I thought about this again... Maybe it is a little bit too hasty to make such a huge reorganization (moving things to a different repository). So let's keep the repository in its current state (for now)! But I really want to do a rework of the rtl folder:
With regard to the changes in this PR, I would like to split the rtl sources into 4 groups:

The CPU/Processor core files (a.k.a. the rtl/core/ directory)
System bridges: default top with different bus interfaces (like AXI)
Minimal SoC tops to be used as test setups for "first encounters"
Exemplary processor templates (-> pre-configured SoCs)

(maybe goup 3 and 4 could be combined)
This could look like this:

rtl/bridges/ top entity wrappers that provide different interface types/protocols

neorv32_top_axi4lite.vhd same as default top but with AXI4-lite instead of Wishbone
neorv32_top_stdlogic.vhd same as default top but using resolved entity signals
TODO: Intel Avalon bus, AXI4-Stream

rtl/core/ CPU and SoC core files (no README in here!)
rtl/templates/ pre-configured example SoCs

neorv32_mars.vhd
neorv32_jupiter.vhd
...

rtl/test_setups/ minimal test setups; intended to be used as top entity for "first encounters" (will be used in the User Guide); only providing GPIO.output ports (8-bit), clock, reset and UART (only the bootloader setup); simplified generics: internal memory size configurations, clock speed

neorv32_test_setup_approm.vhd use internal IMEM as ROM, pre-initialized with software application
neorv32_test_setup_bootloader.vhd  use internal IMEM as RAM, software application upload vie UART + bootloader

Regarding the templates: the current neorv32_ProcessorTop_Minimal*.vhd templates would be replaced by the new test setups. But what about the actual templates in rtl/templates/? So the question is what pre-configured SoCs do we want to provide?
Maybe combining rtl groups 3 (rtl/test_setups/) and 4 (rtl/templates/) would make things easier as both folders provide "SoCs" that could be used for the example setups?! :thinking:

I like the idea of having explicit setups for "test setups" only - so not tied to any specific board/platform/toolchain. That is the purpose of the rtl/test_setups/

👍🏼

rtl/wrappers/ should contain wrappers for system integration. I know the term "wrapper" might be misleading, so maybe something like rtl/integration/ or rtl/connectivity/ would be better ("system" might also be misleading here).

👍🏼
In any case, it would be desirable to have a definition/description in rtl/README.md which clarifies the content of each of the subdirs.

Let's undo the removal of rtl/templates/ 👍 maybe we can name that processor_templates to be more specific. But we should rework the templates there. The Minimal* can be removed I think. If we use rtl/test_setups/ then these setups should be used for those minimal "hello world" examples. Additionally, we should clean-up the templates. Maybe we can use something like three templates there but I am not sure about the actual configuration.

Agree. Let's have #136 and #137 merged first, and then we can rethink the current processor templates. We cannot remove Minimal because it is required in order to have meaningful output in the Fomu. However, we can rename/move/adapt it so that it's used for Fomu only.

As discussed earlier, let's use some names for those templates that do not provide any connotations at all. You had a great table ~somewhere (I need to check for that again)~ in Do we really need wrappers for the top entity? #128. Maybe we can use a simplified concept and put that table into the rtl folder's README (not in the actual documentation due to maintaining 😉)

That sounds nice! Actually, I just reviewed #136 and my last comment was heading in that direction: https://github.com/stnolting/neorv32/pull/136#discussion_r685071582.
Let's flatten the structure, rename processor and system to what you feel more comfortable with, and have it explained in rtl/README.md.

I thought about this again... Maybe it is a little bit too hasty to make such a huge reorganization (moving things to a different repository). So let's keep the repository in its current state (for now)! But I really want to do a rework of the rtl folder:

Strictly, I think the "Delayed" label was better than "Wontfix". However, it's ok to have this as wontfix, have the other PRs resolved, and we can later think about what to to put in the other repo.
I agree with reworking the rtl folder, because that's what needs to remain in this repo, regardless of other sources being moved elsewhere.

With regard to the changes in this PR, I would like to split the rtl sources into 4 groups:

The CPU/Processor core files (a.k.a. the rtl/core/ directory)
System bridges: default top with different bus interfaces (like AXI)
Minimal SoC tops to be used as test setups for "first encounters"
Exemplary processor templates (-> pre-configured SoCs)

This could look like this:

rtl/bridges/ top entity wrappers that provide different interface types/protocols

neorv32_top_axi4lite.vhd same as default top but with AXI4-lite instead of Wishbone
neorv32_top_stdlogic.vhd same as default top but using resolved entity signals

TODO: Intel Avalon bus, AXI4-Stream
rtl/core/ CPU and SoC core files (no README in here!)
rtl/templates/ pre-configured example SoCs

neorv32_mars.vhd
neorv32_jupiter.vhd
...

rtl/test_setups/ minimal test setups; intended to be used as top entity for "first encounters" (will be used in the User Guide); only providing GPIO.output ports (8-bit), clock, reset and UART (only the bootloader setup); simplified generics: internal memory size configurations, clock speed

neorv32_test_setup_approm.vhd use internal IMEM as ROM, pre-initialized with software application
neorv32_test_setup_bootloader.vhd  use internal IMEM as RAM, software application upload vie UART + bootloader

Ok. I should have read this before... So, we are exactly on the same page: have four subdirs in rtl.

Regarding the templates: the current neorv32_ProcessorTop_Minimal*.vhd templates would be replaced by the new test setups. But what about the actual templates in rtl/templates/? So the question is what pre-configured SoCs do we want to provide? But this is not an urgent question.

Agree with the not urgency. Let's solve #136 and #137, then we'll cleanup the Minimal example and we'll see what we are left with.

In any case, it would be desirable to have a definition/description in rtl/README.md which clarifies the content of each of the subdirs.

I totally agree. But let's do that in a new PR when we also split templates into processor and wrappers/connectivity/integration.

We cannot remove Minimal because it is required in order to have meaningful output in the Fomu. However, we can rename/move/adapt it so that it's used for Fomu only.

I think we could use neorv32_test_setup_approm.vhd from #136 instead of the "Minimal" setup. Sure, we would loose the fancy PWM drive, but we could still glue the RGB LED to the GPIO port.

Strictly, I think the "Delayed" label was better than "Wontfix". However, it's ok to have this as wontfix, have the other PRs resolved, and we can later think about what to to put in the other repo.

:smile: 👍

With pyVHDLModel/pyGHDL.dom, we might automatically generate the table that shows which peripherals/features are available in each "template", "system", "bridge", "name it as you want". So, we would not need to manually update it.

That would be awesome!!
If I dare to dream further: teroshdl-documenter (or something similar) to auomatically generate fancy figures of the top entities (showing the available signals and generics)... 😎 :smile:

-> #142, #137, #136


[3.x] Fix routing of InputEventScreenDrag events to Control nodes

Multitouch support enables multiple concurrent InputEventScreenTouch and InputEventScreenDrag events each differentiated by their index property.
An InputEventScreenDrag event typically follows another InputEventScreenDrag or InputEventScreenTouch event and thus should be routed to the target that last matched the previous touch event.
Prior to this fix, in a multitouch scenario only the target of the first touch event was tracked, causing all subsequent multitouch drag events to be routed to that invalid target. The PR tracks the touch targets using the touch event's index property, and use that data to properly route drag events based on their matching index property.
main version

cc @Sauermann

Alternative to #34383

@RandomShaper Ping on this PR. It's the same as https://github.com/godotengine/godot/pull/68632 for the 3.x branch.

Thanks!

Cherry-picked for 3.5.2

Cherry-picked for 3.5.2

Reverted with a204af90d540999771656290564b540736498392 to fix regression #70119. We can consider cherry-picking it again if/when the regression is fixed in 3.x.

I have noticed that if you have overlapping TextureButtons it is impossible to guarantee that the topmost button is the one that gets clicked. It depends on order in the tree but if you reorder them it still can affect the button below. This is maybe not a common problem, but in my case, making a jigsaw game it is a critical issue. Maybe this problem is related to this one? Just a heads up if you are looking around in this code.


Uncaught Error: [$injector:modulerr]

I am trying to use devise_token_auth and ng-token-auth to establish authentication in my rails/angular app.
When gem is installed with its dependencies, bower installation is completed I add ng-token-auth module to my modules in app.js.
I've included
"/js/angular/angular.js">
"/js/angular-cookie/angular-cookie.js">
"/js/ng-token-auth/dist/ng-token-auth.js">
in my index.html
As it is stated in title it simply throws the error
Uncaught Error: [$injector:modulerr] http://errors.angularjs.org/1.2.28/$injector/modulerr?p0=app&p1=Error%3A%20…Fassets%2Fvendor-d246c56eb280352b7a4d132de7860475.js%3Fbody%3D1%3A25%3A170)
Here is my question on SO:
http://stackoverflow.com/questions/29250184/uncaught-error-injectormodulerr-ng-token-auth
I hope you can tell what's the issue.

I'm not sure what the issue is, but I'm certain that it isn't caused by this module.
Are you using any other modules that may have unmet dependencies?

@lynndylanhurley, hey let me show you what modules I use:
 'ngRoute',  'ngAnimate',  'ui.bootstrap',  'easypiechart',  'ui.tree',  'ngMap',  'ngTagsInput',  'angular-loading-bar'
Please let me know, if I can provide any more of relevant info. Thanks a lot!

Hey, the issue seem to go away after I've tossed the included modules in index.html. Thanks!

edit ng-token-auth.js like this:
(function(){
'use strict';
angular.module('ng-token-auth', ['ipCookie']).provider('$auth', function() ...
})()


Move to use ASF's DockerHub

Requested INFA via JIRA INFRA-25940

waiting on INFA to action

hi @justinmclean
I already have a issue to track this #4445


AZ Tool Caching Service Principals

While trying to provision a new kubernetes cluster using acs I ran into an issue with a service principal from a different account was assigned to the cluster.
The SP that my kubernetes installation is using is in a different subscription that is in a different account.  I use two different accounts, a corporate account tied to my corporate credentials, and an MSDN account tied to my personal credentials. I have noticed occasionally that when I list the accounts using az account list that I will sometimes see accounts from both login credentials.
Yesterday when I created my new kubernetes cluster the first time I ran the command az acs ... it failed because it couldn't find the resource group but not until after it had done some SP provisioning. I fixed that and reran the command. My guess is that it somehow created another new SP in a different account/subscription than the one in which it created the actual cluster and associated the kubernetes config with the wrong SP.

I will sometimes see accounts from both login credentials.
This is correct behavior. If you use cli to login to 2 accounts, it will remember/cache all subscriptions from 2 accounts

created my new kubernetes cluster the first time I ran the command az acs ...

I assume to use  --service-principal argument to create service principal on the fly? Or you have logged in as a service principal and create acs?

it failed because it couldn't find the resource group but not until after it had done some SP provisioning.

You might have to ensure you are under right active subscription by using az account set

My guess is that it somehow created another new SP in a different account/subscription than the one in which it created the actual cluster and associated the kubernetes
SP is a tenant specific entity which is not tied to a subscription. A single tenant can managed multiple subscriptions. For you situation, your multiple subscriptions will belong to one of the 2 tenants, one from your corporate account, and the other from you MSDN one.  When you you use az account set to ensure you are in right subscription and tenant, the SP will be created under that (tenant, subscription) pair. As long as you create acs under subscriptions belonging to the same tenant, it should work.

I did not use --service-principal in the command at all. I was following the instructions here: https://docs.microsoft.com/en-us/azure/container-service/container-service-kubernetes-walkthrough
The SP was created in a subscription in a different tenant than where the cluster was created.
There does seem to be an issue with the initial subscription that is preventing me from successfully running ACS at all once that gets sorted out I will try and repro this again and provide additional information.

@brendandburns, I think this is a SP working with ACS, not used in general login

I believe that this is now fixed, there was a bug where SPs were not per subscription. They are now...
If this re-occurs please re-open.
Thanks!


Multiple migrations are run in a single transaction

If you have a migration 0001 with this:
export async function up(db: Kysely<unknown>): Promise<void> {
  await db.schema.createType("some_enum").asEnum(["value_1", "value_2"]).execute();
}

and a migration 0002 with this:
export async function up(db: Kysely<unknown>): Promise<void> {
  await sql`ALTER TYPE some_enum ADD VALUE 'value_3';`.execute(
    db
  );
}

and then some migration 0003 that uses some_enum with value value_3 then:

running migration 0001, 0002 and then at a later time 0003 will succeed
running all three migrations in one go will result in this error:

error: unsafe use of new value "value_3" of enum type some_enum

From what I understand from the code this is caused by migrateToLatest running all pending migrations in a single transaction.
Shouldn't each migration be independent and run in its own transaction?
IME this is the behaviour that most other migration systems adopt (e.g. Django migrations).
Despite this ^ I'm loving this library more and more every day, so thank you again for this magical tool. ❤

Having individual transaction for each migration would certainly make other things easier too. For example we could easily implement disabling transactions for individual migrations. There has been a feature request open for that forever.
However, there was a reason why it was implemented the way it is. I just can't remember that right now.
Changing the behaviour could also be a major breaking change and I don't know if we can do it without a massive deprecation period.

Oh true. For reference to others this is what I went with:
  const migrations = await migrator.getMigrations();
  const pendingMigrations = migrations.filter((m) => isUndefined(m.executedAt));

  if (isEmpty(pendingMigrations)) {
    console.info(`✅ Migrations are up to date!`);
  }

  for (const migration of pendingMigrations) {
    const { error, results } = await migrator.migrateTo(migration.name);

    results?.map((it) => {
      if (it.status === "Success") {
        console.info(
          `✅ Migration "${it.migrationName}" was executed successfully`
        );
      } else if (it.status === "Error") {
        console.error(`❌ Failed to execute migration "${it.migrationName}"`);
      }
    });

    if (error) {
      console.error("❌ Failed to migrate");
      console.error(error);
      process.exit(1);
    }
  }

Let's leave this open. It might make sense to have individual transactions.

Maybe we could ship both and allow enabling this? no breaking change..

Maybe. But enabling/disabling transaction for a single migration when we run a batch of migrations in one transaction would be horrible to implement. Basically we'd only allow that when the "transaction per migration" setting is enabled.

Haha, I just had a discord discussion with someone who explicitly wanted all migrations to run in the same transaction. So we probably need to make both possible.

Running all migrations in a single transaction is IMO the better option if the database engine supports it.
The reason is that if we update an application running in production, and we need to apply multiple migrations (say M1 and M2), then all need to succeed, or none.
If we apply them sequentially, there's a risk of the application entering an inconsistent state -> we apply M1, then M2, which fails, so we try to rollback M1, which also fails. A failing transaction makes the application code rollback to the previous deployment (or rather, not update to the new), but this puts it in a state where it expects the database schema of M0 (before M1), but it is really in M1.
But, it is apparent that not all statements can be run within a single transaction, so an escape hatch is needed.

The reason is that if we update an application running in production, and we need to apply multiple migrations (say M1 and M2), then all need to succeed, or none.

M1 and M2 are two separate atomic change sets that were created in different times, maybe by different developers.

@igalklebanov

How come?

It's literally in the original post - apparently, in Postgres, you can't add a new enum value and then use it within the same transaction.


Errors in downloading. help?

Errors in downloading.
any idea?

I experienced a similar problem, 403 error. My script was working fine before.  I think Dukas changed something on their end

I discovered the cause of the error, it is due to the datafeed URL changing. I have submitted a pull request. Temporarily, you can either edit it yourself or uninstall the old version from pip and install my one with the new datafeed URL.
pip install git+https://github.com/alvin-leong/duka.git


NoMethodError: undefined method `first' for nil:NilClass

View details in Rollbar: https://rollbar.com/trystant/BOW/items/323/

NoMethodError: undefined method `first' for nil:NilClass
  File "/Users/msnyon/code/EBWiki/app/controllers/articles_controller.rb", line 32, in create
  File<EMAIL_ADDRESS>line 4, in send_action
  File<EMAIL_ADDRESS>line 198, in process_action
  File<EMAIL_ADDRESS>line 10, in process_action
  File<EMAIL_ADDRESS>line 20, in block in process_action
  File<EMAIL_ADDRESS>line 117, in call
  File<EMAIL_ADDRESS>line 117, in call
  File<EMAIL_ADDRESS>line 555, in block (2 levels) in compile
  File<EMAIL_ADDRESS>line 505, in call
  File<EMAIL_ADDRESS>line 505, in call
  File<EMAIL_ADDRESS>line 92, in __run_callbacks__
  File<EMAIL_ADDRESS>line 778, in _run_process_action_callbacks
  File<EMAIL_ADDRESS>line 81, in run_callbacks
  File<EMAIL_ADDRESS>line 19, in process_action
  File<EMAIL_ADDRESS>line 29, in process_action
  File<EMAIL_ADDRESS>line 32, in block in process_action
  File<EMAIL_ADDRESS>line 164, in block in instrument
  File<EMAIL_ADDRESS>line 20, in instrument
  File<EMAIL_ADDRESS>line 164, in instrument
  File<EMAIL_ADDRESS>line 30, in process_action
  File<EMAIL_ADDRESS>line 250, in process_action
  File<EMAIL_ADDRESS>line 107, in process_action
  File<EMAIL_ADDRESS>line 18, in process_action
  File<EMAIL_ADDRESS>line 137, in process
  File<EMAIL_ADDRESS>line 30, in process
  File<EMAIL_ADDRESS>line 196, in dispatch
  File<EMAIL_ADDRESS>line 13, in dispatch
  File<EMAIL_ADDRESS>line 237, in block in action
  File<EMAIL_ADDRESS>line 74, in call
  File<EMAIL_ADDRESS>line 74, in dispatch
  File<EMAIL_ADDRESS>line 43, in serve
  File<EMAIL_ADDRESS>line 43, in block in serve
  File<EMAIL_ADDRESS>line 30, in each
  File<EMAIL_ADDRESS>line 30, in serve
  File<EMAIL_ADDRESS>line 817, in call
  File<EMAIL_ADDRESS>line 12, in call
  File<EMAIL_ADDRESS>line 12, in call
  File<EMAIL_ADDRESS>line 35, in block in call
  File<EMAIL_ADDRESS>line 34, in catch
  File<EMAIL_ADDRESS>line 34, in call
  File<EMAIL_ADDRESS>line 24, in call
  File<EMAIL_ADDRESS>line 38, in call
  File<EMAIL_ADDRESS>line 13, in call
  File<EMAIL_ADDRESS>line 27, in call
  File<EMAIL_ADDRESS>line 27, in call
  File<EMAIL_ADDRESS>line 260, in call
  File<EMAIL_ADDRESS>line 225, in context
  File<EMAIL_ADDRESS>line 220, in call
  File<EMAIL_ADDRESS>line 560, in call
  File<EMAIL_ADDRESS>line 36, in call
  File<EMAIL_ADDRESS>

This method no longer uses code that could cause this error. Closing now.


making log directory configurable during init

Proposed changes

the log directory configured in tedge config set logs.path
is now used in --init

Types of changes

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
[ ] Documentation Update (if none of the other choices apply)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue
#1070
Checklist

[x] I have read the CONTRIBUTING doc
[x] I have signed the CLA (in all commits with git commit -s)
[x] I ran cargo fmt as mentioned in CODING_GUIDELINES
[x] I used cargo clippy as mentioned in CODING_GUIDELINES
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments

Ah, and commit messages should use imperative mood:
So instead of
Making log directory configurable during init

you'd write
Make log directory configurable during init

:wink:

@matthiasbeyer thanks!

sorry whitespace police! @matthiasbeyer @albinsuresh


DFI not received

hello. i have been missing DFI for several days. have not arrived on my DefiChain wallet.
app_log.log
binary_log.log
app_config.zip

Hi @domedesign55 , may I know what OS you're using? Could you describe your user flow more? Are you on Mainnet / Testnet? Screenshots would also help! We need more details in order investigate how to reproduce this issue. Thank you.

hello thanks for the feedback.
OS X EL Capitan Version 10.11.6
I am on Mainnet.

@domedesign55 , well noted, thank you for the explanation. Let us investigate this, thank you for your patience!

thank you!
I have reinstalled the defi app.
synchronize has also worked.
862 defi have arrived!
dfi send also works from dfi app.
only from cake, dfi send still does not go to defi app
this is the only problem yet.


Support React 17?

There is currently a version bound on React being less than 17 in the «peer dependencies» section. Is it there for a reason? I want to use this library with React 17, what should I do?

There is currently a version bound on React being less than 17 in the «peer dependencies» section. Is it there for a reason? I want to use this library with React 17, what should I do?

why??? use JSX


Removed redundant argument being passed

I've removed a redundant argument that got passed to Zend\Loader\AutoloaderFactory::factory(). It defaults to the same value that got passed.

Ha, nice find. Thanks!


2023-10-06 is not a trade day, but it is in

China stock. 'XHSG'

https://github.com/gerrymanoim/exchange_calendars#calendar-is-missing-a-holiday-has-a-wrong-time-should-have-a-break-etc

Reopened just to leave up as advice of the possible inaccuracy in the calendar, and in case anyone else has the opportunity to PR the change.
PR would need to include a reference to a source confirming that this is a holiday.


Workspace - Member list blinks briefly after inviting member to the workspace

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4-36.0
Reproducible in staging?: Y
Reproducible in production?: N
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
Action Performed:

Go to staging.new.expensify.com
Go to Settings from the bottom
Select any workspace
Go to Members
Invite a member

Expected Result:
The member list will not blink
Actual Result:
The member list blinks briefly
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/c9ba34be-9e2e-4c36-80e2-b417df27a3ca
View all open jobs on GitHub

Proposal
Please re-state the problem that we are trying to solve in this issue.
The member list page blinks after adding a new member.
What is the root cause of that problem?
After adding a new member, we will navigate to the workspace members.
https://github.com/Expensify/App/blob/3ab4e6e12d9ff38e69493321f0d31f032481b373/src/pages/workspace/WorkspaceInviteMessagePage.js#L107
But we have a logic here to change the navigation type to PUSH if the destination is a central pane navigator (which is true for members page) and the current central pane is not a report screen, so we always push the members page to the stack, thus the page "blinks".
https://github.com/Expensify/App/blob/3ab4e6e12d9ff38e69493321f0d31f032481b373/src/libs/Navigation/linkTo.ts#L163-L179
What changes do you think we should make in order to solve the problem?

We should check whether the route destination is the same as the current central pane route instead of check whether it's a report screen or not. If it's the same, then don't push it.
topmostCentralPaneRoute.name !== action.payload.params.screen

@lanitochka17 Looks like you want to ping @dylanexpensify  instead of me

@DylanDylann I'm sorry, you're right

@dylanexpensify Is there anything special we have to do for issues relating to the wave? https://github.com/Expensify/App/issues/35692#issuecomment-1924459471

looking!

I think it'll be wave6, and low priority but we can open it up

🙏

I can't reproduce it in the latest main.
https://github.com/Expensify/App/assets/25520267/4ceb46c0-6b0e-44c9-b51d-5fec5ae11085

Thanks for the test @mollfpr , I think we may be all set here for now 👀


Localisation Issue EN <-> FR

ezeriToday at 12:28 PM
So there is a localisation error :

Discipline should be "Sang-Froid" linked to Wilpower
Cool should be  Calme linked to Presence

"SWFFG.SkillsNameDiscipline": "Discipline",
"SWFFG.SkillsNameCool": "Sang-froid",
should be :
"SWFFG.SkillsNameDiscipline": "Sang-froid",
"SWFFG.SkillsNameCool": "Calme",

Commited your fix into dev branch, it'll be in the next update. :)

@daezeri if you want french version of the SWFFG module you can find it on my fork, they don't want my improvement in their Work but it's totally functional


Can we add a --geo-bypass flag

Requested Feature
I'm suggesting for the countries where youtube.music is not supported we add an option to allow geo byoass
Possible implementation
No response

I don't think that's possible, YouTube's Music geo blocking is purely IP based. You can use a VPN or --audio youtube to bypass this.

I know that yt-dlp has this option:
--geo-bypass                    Bypass geographic restriction via faking
                                X-Forwarded-For HTTP header (default)

This is the link  of the README.

I wanted to work on this but I can't verify if my patch is working, is someone with restricted yt music access interested in helping me?

I have no restrictions. Maybe using a VPN in China might help

I have no restrictions. Maybe using a VPN in China might help

I couldn't find a free one, and I don't really want to spend money just to send 3 requests ://

Nono, don't worry about it. Let's wait and see if someone helps

I wanted to work on this but I can't verify if my patch is working, is someone with restricted yt music access interested in helping me?

Hi i can help here: my yt is location limited. We can chat on<EMAIL_ADDRESS>

I wanted to work on this but I can't verify if my patch is working, is someone with restricted yt music access interested in helping me?

Hi i can help here: my yt is location limited. We can chat on<EMAIL_ADDRESS>
Hey if you are still interested please contact me on discord xnetcat#8144


Switch new automation code to linear automation

Instead of supporting curved automation in AKNodeParameter, provide AKEvaluateAutomation which converts curved automation to linear automation. This keeps the DSP side simple and offers compatibility with Apple AUs.

There might be some pain on getting ADD back to its functionality now, but this the way forward, so maybe better to get it on the right track now than to wait. I still think my nomenclature suggestions should be considered before merge.

This should not break ADD functionality. I haven't touched the AKParameterAutomation class. I've only removed curved automation from the new automation code.

so how do i do a tapered curve with automation?

@ryanfrancesconi
let point = AKParameterAutomationPoint( ... )
let events = AKEvaluateAutomation(initialValue: 0, points: [point], resolution: 0.1)
node.$param.automate(events: events)

what does resolution do?

///   - resolution: Duration of each linear segment in seconds


[HUDI-1345] Remove Hbase and htrace relocation from utilities bundle

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[x] Has a corresponding JIRA in PR title & commit

[x] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2185 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #2185   +/-   ##
=========================================
  Coverage     53.62%   53.62%
  Complexity     2848     2848
=========================================
  Files           359      359
  Lines         16553    16553
  Branches       1780     1780
=========================================
  Hits           8876     8876
  Misses         6918     6918
  Partials        759      759

Flag
Coverage Δ
Complexity Δ

#hudicli
38.37% <ø> (ø)
193.00 <ø> (ø)

#hudiclient
100.00% <ø> (ø)
0.00 <ø> (ø)

#hudicommon
54.73% <ø> (ø)
1795.00 <ø> (ø)

#hudihadoopmr
33.05% <ø> (ø)
181.00 <ø> (ø)

#hudispark
65.42% <ø> (ø)
304.00 <ø> (ø)

#huditimelineservice
62.29% <ø> (ø)
50.00 <ø> (ø)

#hudiutilities
70.07% <ø> (ø)
325.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


CMOS battery monitoring

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description
I would like to add a function to raise the soundness of CMOS batteries to diagnostics.
Purpose
PC have a CMOS battery to store the state of the BIOS and to run the RTC. When the battery is exhausted, the contents of the BIOS will be erased and the RTC will be reset.
If it happens, the BIOS will be initialized at each boot.
The following problems occur.

The BIOS settings are erased.
It takes time to start the PC.
The RTC time is incorrect.
To avoid the above problems, provide a function to raise the CMOS battery status to diagnostics.

Possible approaches
The default policy is to get the result of "cat /proc/driver/rtc".
As an optional is to get the result of lm-sensors.
Depending on the PC, lm-sensors can be used to monitor the voltage of CMOS batteries.
The default operation is OK, but monitoring with lm-sensors allows you to replace the battery before it runs out completely.
It is recommended because you can avoid the risk of sudden battery exhaustion.
Definition of done
Being able to reflect the battery status in diagnostics.

This issue was fixed in https://github.com/autowarefoundation/autoware.universe/pull/1989. I'm closing this issue.


style can be shorter than inline attributes

if there are many inline styles, style version will become shorter than the second one with attributes:
[
`style="font-family:'Nunito';font-weight:700;font-style:italic;font-stretch:expanded;font-variant:small-caps;font-size:48px;fill:#fff;"`.length,
`font-family="'Nunito'" font-weight="700" font-style="italic" font-stretch="expanded" font-variant="small-caps" font-size="48" fill="#fff"`.length
]
// [134, 137]

Yet SVGO returns the version with attributes
Not easy to put that as a plugin, it could be done in JS2SVG I guess

It can be done as a plugin, with 'full' mode and compare the stringified length with style or with props
closing for now


Add Unison support

Adds support for Unison language.
The code was originally in Unison's official repo. I asked for author's permission in https://github.com/unisonweb/unison/issues/4202 to copy it over here.

This needs an additional test in src/testdir/test_filetype.vim

Hey @dkearns, @cvwillegen! I've addressed all your feedback - one commit per comment. Please let me know if I can do something else.

Thanks, @dkearns! Addressed this round as well.

This might take a week or so to be merged as the interim development model is still to be worked out.

Sure, no worries, take your time. Really hope it all settles well for Vim community, such a loss!

can you please squash into a single commit?

Hey @chrisbra! Sure, done!

Thank you. I'll merge it.


Don't require HTTPS Decryption

Currently FiddlerCert only works if HTTPS traffic decryption is enabled. This should not be a requirement as it should still be able to show the certificate chain in CONNECT tunnels.
@ericlaw1979 Is the SessionFlags.IsBlindTunnel the right way to check if a CONNECT tunnel is for an HTTPS connection that doesn't have traffic decryption? My "Should I show certificate" check looks like this right now:
if (oS.isHTTPS || (oS.BitFlags & SessionFlags.IsDecryptingTunnel) == SessionFlags.IsDecryptingTunnel || (oS.BitFlags & SessionFlags.IsBlindTunnel) == SessionFlags.IsBlindTunnel)
{
     //Do fascinating things
}

A tunnel is a Decrypting tunnel if it's using HTTPS. It's a BlindTunnel if Fiddler is just shuffling bytes back and forth-- note that beyond HTTPS, I'm pretty sure WebSocket traffic can be going through a BlindTunnel too (if you don't have message parsing enabled).
So, you can write things like this:
if (isAnyFlagSet(SessionFlags.IsWebSocketTunnel | SessionFlags.IsBlindTunnel | SessionFlags.IsDecryptingTunnel))

OK, my previous assumptions were wrong.
The issue I am running into is when "Capture HTTPS CONNECTs" is checked but "Decrypt HTTPS traffic" is not. It appears that in these circumstances, Fiddler does not fire the FiddlerApplication.OnValidateServerCertificate.
That makes sense - Fiddler shouldn't need to validate the certificate if it is just acting as a dumb pipe. It should be possible to capture the certificate from the Server Hello, however. Is there a way to do this in Fiddler?

<<It should be possible to capture the certificate from the Server Hello, however. Is there a way to do this in Fiddler?>>
Not presently, no. FiddlerCore has an OnReadResponseBuffer event but this isn't called by Fiddler for performance reasons.


Waybar is unstable when used on multiple monitors

I am using wayfire with waybar on 2 monitors, named EDP1 laptop, and HDMI monitor, i have noticed that wayfire crashes often on this setup without any errors or logs given as to reason what happened.
It happens mostly when some new icon is loaded.
my config is at https://github.com/MrKorner/DotFiles/blob/main/configs/waybar/config
And waybar never crashed so far under single monitor instance
EDIT: version is v0.9.5

Are using the master version?
Just release another version, 0.9.6, can you test with this version and reopen the issue, if your problem persist.


Add ability to add rootClasses to Form errors

Describe the feature
Form error messages can be changed like this: <formkit type="form" incomplete-message="Foo">,
but there is no elegant way to theme said error messages. Currently we are stuck with theming "li.formkit-message" using css, which is not very tailwind friendly.
It would be great to add classes to theses error messages, or even better, have the ability to globally add classes to them using rootClasses

There is already a way to add classes to each error message, you can use the generateClasses from @formkit/tailwindcss to write their classes like so:
generateClasses({
  text: {
    message: 'text-red-500 border border-red-500'
  }
});

I've found an elegant solution, thank you. For future devs, here's the code:
In your formkit.config.js:
`
import { generateClasses } from '@formkit/tailwindcss'
export default {
config: {
classes: generateClasses({
form: {
messages: 'class1 class2'
},
}
}
}
`


allow general pytrees in jvp

currently the primal and tangent args for  jax.jvp are limited to lists/tuples.
support for general pytrees will help in not having to do the tree-flatten/unflatten manually.
this is esp. useful for frameworks like haiku which use dict-like structures for params.
_jvp already does tree-flatten/unflatten but general trees are disallowed here (for reasons as yet unclear),
https://github.com/google/jax/blob/master/jax/_src/api.py#L1754

Hey @akssri!
Actually, pytrees are allowed with jvp. But the primals and tangents must be lists or tuples of pytrees. That just comes from the fact that Python functions can in general take multiple arguments, so that if for a function f you can evaluate f(*xs) then you can call jvp like jvp(f, xs, xs).
Here's an example of using pytrees with jvp:
In [1]: from jax import jvp

In [2]: def f(x, y):
   ...:     return x['hi'] + y['foo']
   ...:

In [3]: f({'hi': 1.}, {'foo': 2.})
Out[3]: 3.0

In [4]: jvp(f, [{'hi': 1.}, {'foo': 2.}], [{'hi': 3.}, {'foo': 4.}])
Out[4]: (DeviceArray(3., dtype=float32), DeviceArray(7., dtype=float32))

WDYT?

ah! this makes sense. thanks for clarifying @mattjj ! sorry for the trouble - i probably didn't pass the arguments properly when i was testing things out.

No trouble at all! We like answering questions :)


flycast: 1.3 -> 2.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/flyinghead/flycast/releases.
meta.description for flycast is: "A multi-platform Sega Dreamcast, Naomi and Atomiswave emulator"
meta.homepage for flycast is: "https://github.com/flyinghead/flycast"
meta.changelog for flycast is: "https://github.com/flyinghead/flycast/releases/tag/v2.0"
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2.0 with grep in /nix/store/swi5ipb15hjgnmfadgxkf7zwfisgmi20-flycast-2.0
found 2.0 in filename of file in /nix/store/swi5ipb15hjgnmfadgxkf7zwfisgmi20-flycast-2.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
flycast

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/swi5ipb15hjgnmfadgxkf7zwfisgmi20-flycast-2.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A flycast https://github.com/r-ryantm/nixpkgs/archive/661844351c6ab02c8deff596298cebdd752f83fc.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/swi5ipb15hjgnmfadgxkf7zwfisgmi20-flycast-2.0
ls -la /nix/store/swi5ipb15hjgnmfadgxkf7zwfisgmi20-flycast-2.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

flycast

Maintainer pings
cc @IvarWithoutBones for testing.


align columns

I've tested first example and it give this output:
PID TTY TIME CMD
60881 ttys000 0:00.19 -bash
61674 ttys000 0:00.17 vim README.md

I think it would be more usefull if the output would be
PID   TTY     TIME    CMD
60881 ttys000 0:00.19 -bash
61674 ttys000 0:00.17 vim README.md

it should count the length of each cell and head and pad right to the longest using spaces.

Thanks.  I'm assuming you saw this:
https://github.com/joyent/node-tab#controlling-field-width-and-alignment
Early feedback was that it was useful to be able to use cut and similar tools with specific column widths, so we opted with that approach for many of the tools that use this library.  We could add an 'auto' column width to do what you suggest.  That would have the downside that it couldn't be streaming: it would need to see all of the output rows before emitting even the header.  That might be appropriate for some situations.
In terms of workaround: as a user, you can get similar behavior using column -t (though it is not quite the same, since column doesn't know which cells are supposed to have whitespace in them).

{width: 'auto'} would be great for use case when user want to display nice looking table. I was thinking on using this in browser (it will be challenging because you don't have stdout), and don't care about streaming, because you display whole table at once.


Fix inconsistency in template mapping term

shg1:country -> shg1:country:value for consistency with other terms

Coverage remained the same at 92.216% when pulling d81e8cedc34ceaff8490e58e1e81b62da9cf51bc on template-typo-fix into 2323f1cb85ce2a5682608535ef5dc46eb24f91ac on master.


Mat/Pat : Bug with Submission

Extra submission for supervisors in the review and process tab of a form that was never submitted and is not in Submitted forms tab.
Video - https://bcgov.sharepoint.com/:v:/r/teams/03991/Shared Documents/Maternity and Parental Leave/Test Screenshots %26 Recordings/Screen Recording 2023-09-14 at 10.17.56 AM.mov?csf=1&web=1&e=kfjbEi
Another double submission, form for "ere", i validated multiple times as a supervisor because it was looping -
Private Zenhub Image

@fazil-ey fix one issue of task not update correctly in workflow. That fix double task in review tab.
Could you please review this now?
Let me know your finding if any on this ticket so we can add proper bug task. I need full form data(You can share basic details and I can collect from camunda) so I can review it's all fields that are present in workflow at right place.

Submitted multiple forms and was not able to replicate this issue even after changing the name in draft. We can close this ticket, but keep an eye out to see if this happens again

Okay to close


Changed HERON path variable in collect_tex.py

Changed heron_path in collect_tex.py so that the HERON module is found when compiling the documentation via make_docs.sh.
Pull Request Description
What issue does this change request address?
#184
What are the significant changes in functionality due to this change request?
No changes in functionality.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[ ] 1. Review all computer code.
[ ] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[ ] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 4. Automated Tests should pass.
[ ] 5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large tes.
[ ] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[ ] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[ ] 8. If an analytic test is changed/added, the the analytic documentation must be updated/added.
[ ] 9. If any test used as a basis for documentation examples have been changed, the associated documentation must be reviewed and assured the text matches the example.

@dylanjm looks like we were reviewing this at the same time! Sorry, @AnthoneyGriffith I'm a bit more picky 😛

Job CentOS 8 on 89700d6 : invalidated by @dgarrett622
failed in updating RAVEN


How to creat a splits file for other classes?

As the title shows, I want to test DeepSDF for some other classes. But I can not find any solutions about how to create a splits file such as sv2_sofas_train.json?
Thanks and Best!

Did you solve this problem? I have the same confusion……