[SPARK-11374][SQL] Support skip.header.line.count option for Hive Table

What changes were proposed in this pull request?
This PR adds skip.header.line.count option support for Hive tables (external and managed).

CREATE EXTERNAL TABLE or CREATE TABLE ... LOCATION uses external files as a data source for the tables. External files like CSV frequently contains one or more header lines as their own metadata. So, skip.header.line.count table property is used in order to exclude those metadata lines from data lines.
Managed tables also have the same situations to handle files loaded by LOAD DATA SQL commands.

SPARK-11374 is a long lasting issue since 1.5.1, and all current Spark versions have the same situation.
The following is an example for SPARK-11374.
CSV Data
$ cat /data/test.csv
c1,c2
1,a
2,b

Scala
scala> spark.read.option("header","true").csv("/data").show
+---+---+
| c1| c2|
+---+---+
|  1|  a|
|  2|  b|
+---+---+

SQL (External Table)
scala> sql("CREATE TABLE t1 (id INT, value VARCHAR(10)) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION '/data' TBLPROPERTIES('skip.header.line.count'='1')")
scala> sql("SELECT * FROM t1").show
+---+-----+
| id|value|
+---+-----+
|  1|    a|
|  2|    b|
+---+-----+

SQL (Managed Table)
scala> sql("CREATE TABLE t2 (id INT, value VARCHAR(10)) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' TBLPROPERTIES('skip.header.line.count'='1')")
scala> sql("LOAD DATA LOCAL INPATH '/data/test.csv' OVERWRITE INTO TABLE t2")
scala> sql("SELECT * FROM t2").show
+---+-----+
| id|value|
+---+-----+
|  1|    a|
|  2|    b|
+---+-----+

How was this patch tested?
Pass the Jenkins test with a new test case.

Test build #63759 has started for PR 14638 at commit 4465415.

Test build #63759 has finished for PR 14638 at commit 4465415.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63759/
Test FAILed.

Merged build finished. Test FAILed.

Test build #63760 has started for PR 14638 at commit 519b14a.

Is this a CSV specific issue? If yes, it seems wrong to have logic implemented for all hive serdes for this thing.

Actually, I described like that for simplicity. But, CSV is one of the typical case of this option. This issue is not a CSV specific issue. I think this is a general table properties as you see the following.
CREATE TABLE t1 (id INT, b VARCHAR(10))
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE LOCATION '/data'
TBLPROPERTIES('skip.header.line.count'='1')

Test build #63760 has finished for PR 14638 at commit 519b14a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63760/
Test PASSed.

Hi, @rxin . I update the PR description first.

Does this option make sense in Parquet or ORC?

I see. Thank you for guidance. Initially, I thought this table property is orthogonally skipping the starting rows of each file for all formats. And, I assumed that users will not use this property improperly.
But, what you mean is it has no meaning for columnar and vectorized formats. So, if a user give this table property for this Parquet or ORC, Spark need to ignore this.
If then, definitely, we should find some places for TEXT format only. BTW, do you have some proper location in your mind instead of the current hadoopRDD.mapPartitionsWithIndex?

Test build #64128 has started for PR 14638 at commit 345f7f3.

Test build #64128 has finished for PR 14638 at commit 345f7f3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64128/
Test PASSed.

Test build #64440 has started for PR 14638 at commit 3c9adb3.

Hi, @rxin . Until now, I couldn't find more suitable class in hive package. I added the logic to check the Table's InputFormat. Now, skip.header.line.count option is applied for the table using org.apache.hadoop.mapred.TextInputFormat only.

Test build #64440 has finished for PR 14638 at commit 3c9adb3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64440/
Test PASSed.

Test build #64569 has started for PR 14638 at commit 1e22b68.

Test build #64569 has finished for PR 14638 at commit 1e22b68.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64569/
Test PASSed.

Merged build finished. Test PASSed.

Test build #64872 has started for PR 14638 at commit 3857e32.

Test build #64872 has finished for PR 14638 at commit 3857e32.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64872/
Test PASSed.

Test build #64963 has started for PR 14638 at commit 2b1c99b.

Test build #64963 has finished for PR 14638 at commit 2b1c99b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64963/
Test PASSed.

Test build #65138 has started for PR 14638 at commit 63bcc08.

Test build #65138 has finished for PR 14638 at commit 63bcc08.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65138/
Test PASSed.

Test build #65251 has started for PR 14638 at commit 257708a.

Test build #65251 has finished for PR 14638 at commit 257708a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65251/
Test PASSed.

Test build #65429 has started for PR 14638 at commit eac4037.

Test build #65429 has finished for PR 14638 at commit eac4037.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65429/
Test PASSed.

Test build #65666 has started for PR 14638 at commit 12792e4.

Test build #65666 has finished for PR 14638 at commit 12792e4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65666/
Test PASSed.

Test build #66008 has started for PR 14638 at commit 51a83cf.

Test build #66008 has finished for PR 14638 at commit 51a83cf.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66008/
Test PASSed.

Rebased to the master.

Test build #66228 has started for PR 14638 at commit 74c3e81.

Test build #66228 has finished for PR 14638 at commit 74c3e81.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66228/
Test PASSed.

Test build #66493 has started for PR 14638 at commit ec85638.

Test build #66493 has finished for PR 14638 at commit ec85638.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66493/
Test PASSed.

Test build #66845 has started for PR 14638 at commit 0c4c06e.

Test build #66845 has finished for PR 14638 at commit 0c4c06e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66845/
Test PASSed.

Merged build finished. Test PASSed.

Test build #67426 has started for PR 14638 at commit 9010388.

Test build #67426 has finished for PR 14638 at commit 9010388.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67426/
Test PASSed.

Test build #68460 has started for PR 14638 at commit 2302447.

Test build #68460 has finished for PR 14638 at commit 2302447.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68460/
Test FAILed.

This is necessary, please add this to a release !

Thank you, @jamartinh . I hope so.

Retest this please.

Test build #68628 has started for PR 14638 at commit 2302447.

Thank you for review, @srowen !
Yes. The issue was the question here is whether the change belongs here or elsewhere, and I couldn't find better place.

Test build #68633 has started for PR 14638 at commit 3c06aa6.

Test build #68628 has finished for PR 14638 at commit 2302447.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68628/
Test PASSed.

Merged build finished. Test PASSed.

Test build #68633 has finished for PR 14638 at commit 3c06aa6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68633/
Test PASSed.

Retest this please

Test build #69334 has started for PR 14638 at commit 3c06aa6.

Test build #69334 has finished for PR 14638 at commit 3c06aa6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69334/
Test PASSed.

Test build #69362 has started for PR 14638 at commit 5355de9.

Test build #69362 has finished for PR 14638 at commit 5355de9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69362/
Test PASSed.

Thank you for approving, @srowen !

@dongjoon-hyun
can you explain why this is only a problem for external table but not managed tables?

@rxin .
I didn't say this is only a problem for external table.
I simply focused on SPARK-11374 use case here.
What use case are you considering when you mentioned managed tables?

The way the description is written implies that there is something special about external tables or tables with location specified. I'm just confused here.

Ah, I see.
Then, I'll try to update PR descriptions or testcases more.

@rxin .
I updated the title and description of this PR.
For the test case, I think the current test case covers enough the functionality.

Test build #69539 has started for PR 14638 at commit 6f602ba.

Test build #69539 has finished for PR 14638 at commit 6f602ba.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69539/
Test PASSed.

@dongjoon-hyun can you show a link to how Hive implements this? I'm just surprised that this is a general feature rather than something that is format specific.

I didn't read Hive code part for this. I referenced the Spark csv part code.
But, I'll search the corresponding one in Hive master branch tomorrow since you requested.

Hi, @rxin .
Here is the corresponding HIve implementation.
getNextRow

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java#L470-L474

          headerCount = Utilities.getHeaderCount(currDesc.getTableDesc());
          footerCount = Utilities.getFooterCount(currDesc.getTableDesc(), job);

          // Skip header lines.
          opNotEOF = Utilities.skipHeader(currRecReader, headerCount, key, value);

skipHeader

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L3385-L3393

  public static boolean skipHeader(RecordReader<WritableComparable, Writable> currRecReader,
      int headerCount, WritableComparable key, Writable value) throws IOException {
    while (headerCount > 0) {
      if (!currRecReader.next(key, value))
        return false;
      headerCount--;
    }
    return true;
  }

@rxin
Could you give me any advice for this issue?

Hi, @rxin .
Do you want me to prepare something based on InputFileBlockHolder of #16133 ?

Could you review this again please when you have sometime, @rxin ?

Hi, @rxin , @srowen , @yhuai .
If there is something for me to do more for this issue, please let me know.

Retest this please

Test build #69796 has started for PR 14638 at commit 6f602ba.

Test build #69796 has finished for PR 14638 at commit 6f602ba.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69796/
Test PASSed.

Thank you for review, @mridulm .
I updated the PR by using isAssignableFrom.

Test build #69816 has started for PR 14638 at commit 47ebd23.

Test build #69816 has finished for PR 14638 at commit 47ebd23.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69816/
Test FAILed.

Merged build finished. Test FAILed.

Retest this please

Test build #69830 has started for PR 14638 at commit 47ebd23.

Test build #69830 has finished for PR 14638 at commit 47ebd23.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69830/
Test PASSed.

In the original Hive JIRA of this feature: https://issues.apache.org/jira/browse/HIVE-5795
See the latest reply from Sergey Shelukhin.

This forces the entire input into a single split, which defeats the purpose of using Hive in the first place - might as well run the analysis on a local machine. I would not recommend anyone to use this feature except for experimentation. The headers/footers should be cleared as part of an ETL process.

I do not know how many Hive users are using them in the production system. This might not be recommended in the Hive side.

Hi, @gatorsmile .
Do you think it's valid comment for this implementation? Why 'this forces entire input into a single split'? We are not doing such a thing.
For me, it looks like a comment for footer. We didn't provide footer.
Also, the approach in this PR is based on Spark spark.read.option("header","true").csv("/data").show behavior, isn't it?

If Hive users are not using this feature, should we still use the table property as the interface to do it?
Asking users to change the table property looks hacky to me.

Why do you think Hive user are not using this feature?

That is my question. I do not know the answer. Based on the comment in the Hive JIRA, I am doubting how many Hive users are using it.

Ah, if that is the question, I knew the customers using this and requesting this. :)

Actually, it's a Spark issue SPARK-11374, isn't it?
So, the remaining question should be the Spark user do want this.
For this, I'll send an email to user and dev mailing list.
It's simple.

In the Hive document, this is not included. https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTable
The list does not include it:
TBLPROPERTIES ("comment"="table_comment")
TBLPROPERTIES ("hbase.table.name"="table_name") – see HBase Integration.
TBLPROPERTIES ("immutable"="true") or ("immutable"="false") in release 0.13.0+ (HIVE-6406) – see Inserting Data into Hive Tables from Queries.
TBLPROPERTIES ("orc.compress"="ZLIB") or ("orc.compress"="SNAPPY") or ("orc.compress"="NONE") and other ORC properties – see ORC Files.
TBLPROPERTIES ("transactional"="true") or ("transactional"="false") in release 0.14.0+, the default is "false" – see Hive Transactions.
TBLPROPERTIES ("NO_AUTO_COMPACTION"="true") or ("NO_AUTO_COMPACTION"="false"), the default is "false" – see Hive Transactions.
TBLPROPERTIES ("compactor.mapreduce.map.memory.mb"="mapper_memory") – see Hive Transactions.
TBLPROPERTIES ("compactorthreshold.hive.compactor.delta.num.threshold"="threshold_num") – see Hive Transactions.
TBLPROPERTIES ("compactorthreshold.hive.compactor.delta.pct.threshold"="threshold_pct") – see Hive Transactions.
TBLPROPERTIES ("auto.purge"="true") or ("auto.purge"="false") in release 1.2.0+ (HIVE-9118) – see Drop Table and Drop Partitions.
TBLPROPERTIES ("EXTERNAL"="TRUE") in release 0.6.0+ (HIVE-1329) – Change a managed table to an external table and vice versa for "FALSE".

More importantly, we have not officially allowed users to use TBLPROPERTIES in Spark. I do not know whether this is the right way to do it. cc @rxin

In DDL for creating data source tables, OPTIONS is being used for table-level parameters. In the near future, we need to combine DDL for data source tables and Hive serde tables. I am not sure whether OPTIONS is a better place for this purpose.

Also cc @cloud-fan @hvanhovell

Thank you for the investigation. Is the Hive table property list exhaustive?

That list is not exhaustive, but should we follow what we did for CSV, instead of using this hidden Hive feature? Should we use OPTIONS instead of TBLPROPERTIES?

Yes. Right, that is possible, but it's a different use case.
This PR supports the followings:

The existing Hive table with TBLPROPERTIES(skip.header.line.count=1) will show the same result in Hive & Spark.
Previously, Spark ignores that option. So, Spark shows incorrect result for that table.
And, vice versa, the table created by Spark will show the same result in Hive & Spark.

This is not a hidden Hive feature. It was

Reported by Daniel Haviv (Oct. 2015)
Asked by Stephane Maarek (Apr. 2016)
Asked by Rahul Jain (Sep. 2016)

Also, there is a StackOverflow question.

http://stackoverflow.com/questions/38566591/spark-sql-hivecontext-dont-ignore-header

I think this is really an uncommon hive feature, and users can work around it by creating an external data source table on the CSV files.
And another concern is, we can't simply call some Hive APIs, but have to reimplement this feature at Spark side(in the hive table reader). If there are more Hive features like this, we will complicate the hive table reader quite a bit.

I know votes and stars might not be an appropriate standard for the popularity of a feature but I found another stackoverflow question - http://stackoverflow.com/questions/15751999/hive-external-table-skip-first-row

@HyukjinKwon yea for Hive this feature is useful, as the only workaround is running a ETL program to remove the headers manually. However, in Spark SQL, our CSV data source already support skipping header lines, the problem is external hive serde table, and users can easily work around it by creating an external data source table on the CSV files.

Thank you for opinion, @HyukjinKwon and @cloud-fan.
BTW, our CSV data sources does not support multiple line headers so far.

Now, I think we have been considered almost pros and cons here. Thank you all for reviewing and giving opinions for this issue.
I value the committers' decision and feel the worry from @rxin , @gatorsmile , @cloud-fan .
The overall opinion seems to be it's not a proper Spark way for Spark users, right?

My humble comment is simply that an external Hive table still is a Hive
Table, and as such it should be transparent for whichever user how to work
with it, even for spark users like me. The expectation is that a query from
such table yield the same results as if it where queried from Hive. I have
seen people worried about this.
That is, it is very unexpected querying a table from Hive and obtain
different result when queried from Spark.
I still consider this functionality as a neccesary feature in order to
maintain consistency with Hive engine, otherwise spark hive datasource
will slowly be diverging from Hive. I feel this should not be the case.
Best,
Jose
El 9 dic. 2016 23:42, "Dongjoon Hyun"<EMAIL_ADDRESS>escribió:
Now, I think we have been considered almost pros and cons here. Thank you
all for reviewing and giving opinions for this issue.
I value the committers' decision and feel the worry from @rxin
https://github.com/rxin , @gatorsmile https://github.com/gatorsmile ,
@cloud-fan https://github.com/cloud-fan .
The overall opinion seems to be it's not a proper Spark way for Spark
users, right?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/spark/pull/14638#issuecomment-266143334, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADbWs0ABWRHyV8H1ZB_PdK087bCSEN13ks5rGdlagaJpZM4Jj_em
.

I am really on the fence about it. I can see the argument that it's useful to support, and not hard to support. But also that this is just one of many Hive features that one could support, and we have to support it separately, and it duplicates other Spark-specific mechanisms.
For what it's worth we did hear from a few users who said this would be useful. If we don't merge this, then we're saying that we believe there's a different and better way to do this via Spark CSV support. Is that true -- do we think that's not a big inconvenience?

Up to my knowledge, currently, CSV datasource does supports to skip a (only) single line with header option and also comment option to skip the lines started with the character given to comment whereas skip.header.line.count suggested here literally supports to skip multiple lines.
So, strictly they are not exactly duplicates but I believe it is true some ranges that skip.header.line.count supports can be worked around with the options, header and comment as explained by @dongjoon-hyun and @cloud-fan.
If we admit skipping a single line (or skipping lines started with comment) is a common case and enough to cover this feature, that would not be a big inconvenience but if users believe it is still useful to skip multiple lines, we would still need this.

@jamartinh I got your point. Hive tables can be read/wrote by both Hive and Spark. Ideally, we should respect all the features supported by Hive. That means, if the metadata of an existing Hive table already has such a TBLPROPERTIES, we should not simply ignore it.
However, when users want to add it in Spark side, what is the right interface? In the existing Spark SQL, table properties are not used for such purposes. When users specify the table-level properties/options for data source tables, we store them in the storage property. Thus, they will not be recognized by Hive. We might need to hold it until we figure out how to unify the SQL interfaces of Hive tables and the other data source tables. TBLPROPERTIES is not the right interface for us, at least now.

Thank you so much, @jamartinh , @srowen , @HyukjinKwon , and @gatorsmile .
We can distinguish the two existing problems separately here.
First, a) Spark returns incorrect result for an existing Hive table already with skip.header.line.count table property. This is the most common use case which this issue aimed to solve.
Second, more ridiculously, b) Spark can create a table with skip.header.line.count table property and only Hive returns the correct result from that table.
SPARK (Current master branch)
scala> sql("CREATE TABLE t2 (id INT, value VARCHAR(10)) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' TBLPROPERTIES('skip.header.line.count'='1')")

scala> sql("LOAD DATA LOCAL INPATH '/data/test.csv' OVERWRITE INTO TABLE t2")

scala> sql("SELECT * FROM t2").show
+----+-----+
|  id|value|
+----+-----+
|null|   c2|
|   1|    a|
|   2|    b|
+----+-----+

Hive
hive> select * from t2;
OK
1	a
2	b

@gatorsmile . I totally agree on the Apache Spark development direction.  But, IMO, TBLPROPERTIES or OPTION is not a proper issue in this PR. It's because this PR only updates TableReader.scala to support the existing table property, case a). For TBLPROPERTIES, I simply used that because it's already supported on Spark. I can update the PR description in order to focus on a) instead of b).
Someday later, Apache Spark may delete(or block) TBLPROPERTIES SQL syntax in favor of OPTION syntax. It's okay. It's just a kind of regression on purpose. No problem at all. However, even in that case, we had better read the Hive table with skip.header.line.count correctly.

Is there any high-level API that we can use to read hive tables? The current hive table reader is so low-level that we have to support features like skip.header.line.count, and I think it doesn't work well with non-file based hive tables.

Do you mean to replace the current whole TableReader.scala which was introduced in SPARK-1251 ? I guessed Spark chose this direct access approach for the performance issue at that time.
Yes. This option is targeting only TextInputFormats. For non-file based hive tables like Orc/Parquet, this option is ignored.
val isTextInputFormatTable = classOf[TextInputFormat].isAssignableFrom(hiveTable.getInputFormatClass)

Thank you all so much for reviewing and discussion on this issue since Aug 14th.
Since we focused this fully both here and in Spark mailing list and it has been 4 months already, I'll happily close this PR and SPARK-11374 as a WON'T FIX tomorrow (Dec. 14th).
All of your time is too precious to be stuck here any more. Apache Spark still have many things to do. Go Spark!

Sadly, it's a useful feature which i want.
i want to retreive HDFS tsv files via spark sql jdbc interface which defined as hive table

Also a useful feature for me.

Since this pr won't be accepted. My solution is:
# create table using spark-csv
CREATE TABLE tmp.cars
USING com.databricks.spark.csv
OPTIONS (path "/tmp/cars.csv", header "true", inferSchema "true");

# create table use CTAS: whether using parquet or orc or others depends on your conf
create table tmp.cars_transformed as select * from tmp.cars

# optional: drop the spark-csv table
drop table tmp.cars

@sadhen Your solution works well without any code change, right?

yes

@sadhen Using pySpark 2.4.0 and I still have this issue, in the repo of spark.csv it says "This functionality has been inlined in Apache Spark 2.x." so that means that tblproperties should work? or I have to modify something in your code to make it work in the new versions?

Nope, please don't expect that, @set92 . BTW, where did you see the message?

This functionality has been inlined in Apache Spark 2.x

@dongjoon-hyun in the README of spark-csv, which I read was one option to solve this problem pre2.0.

I'm not sure why you ask that question here. This is Apache Spark repo and Apache Spark community didn't write that sentence. Could you ask there, @set92 ? :)

It's a fair question as it was created by some of the same people as here, and we advertised it as merged into Spark. I don't know if 100% of the functionality was ported as is. But it's a question for user@, really.

Got it, @srowen .

We are having the same issue on our side as well, this patch would have been terribly useful to us.


Association on SQLite gives unsupported data type:

Your Question
Hi, I am getting an unsupported data type: <nil> while trying to retrieve associated data. I can see it in the database and it's probably me doing something incorrectly.
Can anyone help?
This is my command, if I remove the Association then everything seems to work so the "FirstOrCreate" is working.
	block := model.Block{
		Filename: "test.blk",
	}

	err := s.dbConnection.DB.FirstOrCreate(&block).Association("BlockVersion").Find(&block.BlockVersion)
	if err != nil {
		log.Fatalln("There was error :- ", err)
	}

Here are my models.
type BlockVersion struct {
	gorm.Model
	BlockID  uint
	Filesize int64
}

type Block struct {
	gorm.Model
	Filename     string
	BlockVersion []BlockVersion
}

Once I have it working, the BlockVersion may contain more than 1 record, so I was hoping to return "only" the latest one.
I have placed some images below to show what is currently in the database.
Expected answer
I would like for the last (latest) record of "BlockVersion" to be populated inside the "Block".
Block

BlockVersion

Oh,
I got it working.
Seems I might have been doing it wrong.
I am now using
	err = s.dbConnection.DB.Model(&block).Association("BlockVersion").Find(&block.BlockVersion)

This now works.
So doing this does not.
      s.dbConnection.DB.FirstOrCreate(&block) // Get main table first

      err := s.dbConnection.DB.FirstOrCreate(&block).Association("BlockVersion").Find(&block.BlockVersion)

I thought you could chain then?

After a Chain method, Finisher Method, GORM returns an initialized *gorm.DB instance, which is NOT safe to reuse anymore, you should use a New Session Method to mark the *gorm.DB as shareable.

https://gorm.io/docs/method_chaining.html#New-Session-Method


Do we need to build both "development" and "production" bundles?

npm run build currently produces both development and production bundles. However, this makes build time quite long and it produces many files. Can we just produce the production bundles, at least for the library target? Or do external users really need the development version as well?
Alternatively, is there a faster way to create bundles for multiple build modes? Right now webpack has to execute a separate config for each of these build modes/envs. But I would think that most of the steps across builds are the same, maybe it is just that the production builds need to be minified at the end?
https://github.com/hubmapconsortium/vitessce/blob/master/scripts/constants.js#L5
Related to #504

A better option may actually be to run the builds in parallel, if that is easy to do.

https://github.com/trivago/parallel-webpack


pull request for e2e test at 12/11/2016 05:15:02

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Don't treat handicap stones as initially legal moves for white

With non-free-placement hanidicap stones, _moves is empty until white's first move (because we're treating the board setup with the handicap stones as not-a-move), and so an intersection with a handicap stone on it is mistakenly treated as a legal move for white. White could in fact play at the occupied point!
This was visually showing up as a bug on the board where hovering over a handicap stone would remove the stone on mouseenter because of the way hoverValue checks for move legality.

Broken-ness before this fix:
Overwriting an occupied stone

Temporarily removing a stone that then reappears


[Shortcode] Set an "Expand" item open by default

Hello,
It would be nice to have hability to set "expand" item open by default.
By example:
{{%expand "Open by default?"  true %}}Yes!{{% /expand%}}

{{%expand "Open by default?"  false %}}No!{{% /expand%}}

Great idea ! Would you like to make a PR ?

I will try to do it for today.


fix: disallow references to undefined relations and misc

The changes herein disallow references in relation tuples to undefined relations in namespace configs. If a write is issued that references a relation that is undefined at that particular snapshot in time, an error is returned. Other changes include exponential retry-backoff for namespace monitoring and more appropriate error status responses.
closes #8
closes #14
closes #15

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@6adaf3d). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #17   +/-   ##
=========================================
  Coverage          ?   73.10%
=========================================
  Files             ?        7
  Lines             ?      989
  Branches          ?        0
=========================================
  Hits              ?      723
  Misses            ?      198
  Partials          ?       68

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6adaf3d...d32f1c1. Read the comment docs.


Add feature to skip non-image file extensions

Add a feature to skip any file with an extension that indicates it is not within a set of image filetypes that are currently supported by Pillow.

Hi @elisemercury,
This is the pull request for the feature we were speaking about in https://github.com/elisemercury/Duplicate-Image-Finder/issues/60.
The hard-coded tuple of filetypes currently supported by Pillow is correct to the best of my knowledge, but as you say, it's not dynamically updated.
On the other hand, it'd be trivial to update the tuple if and when new filetypes are supported (or indeed, removed from support). If the user has logging turned on, they'd be alerted to the skipped files,  thus would be in a position to raise an issue if something they thought should have been accepted was not.
Cheers,
Dan

Hi @UplandsDynamic,
Thanks a lot for this! I think this is a great idea and can definitely be useful as feature improvement. I like the idea of having the option to turn limit_extensions on and off so that users have the option to choose whether they want to use it.
Again, thanks a million for your contributions!
All the best,
Elise

HI @elisemercury,
I'm glad it was useful! Since I made the PR last night, I realised there's a slightly more efficient way of grabbing the extensions than the RegEx I used - using the pathlib module. So at some point soon I'll sync over your updated main, make the change, test and send a PR for the tweaked code.
Cheers,
Dan

Hi @UplandsDynamic,
Yes indeed, I thought the same. I'm currently working on an alternative to regex to make it more efficient. The update should go live soon. Thanks again!
Cheers,
Elise

@elisemercury Ah ok, I was just working on that now, but if you're already on the case I'll abandon sticks. I was thinking along the lines of ext = Path(f).suffix.lower().strip("."), which I've tested and does the job.
Out of interest, I was going to do an experiment to test the execution times between the two approaches, but just realised it's gone 10pm and I've not eaten dinner yet, so maybe another time 😆

@UplandsDynamic, I implemented it with os.path.splitext(f)[1] as I assumed it would be more accurate than 'manually' stripping of the end of the path. This meant including a . in front of the valid_extensions list, then it worked perfectly fine.
Again, thanks again for your contribution! The list seems very complete, so it seems like a very good solution to increase the performance a bit (happy to see the test results if you decide to do some!). It might make sense to set this parameter to default True in the future - let's see what the community says. :slightly_smiling_face:
Good night and thanks again,
Elise


Bug: Getting FailedPreStopHook even if preStop command executed

📜 Description
Added a script in preStop hook to gracefully shutdown the sidekiq. This script  got executed but still showing FailedPreStopHook event
👟 Reproduction steps

add below script in deployment template on Devtron

containerSpec:
lifecycle:
enabled: true
preStop:
exec:
command:
- /bin/sh
- -c
- bundle exec sidekiqctl stop tmp/pids/sidekiq/main_2.pid 30 >> /proc/1/fd/1 2>&1

Rebuild and re-depoly

👍 Expected behavior

sidekiq should stop gracefully
preStop success event should be these in events log

👎 Actual Behavior

Sidekiq is getting gracefully shutdown verified by checking pod logs
But getting FailedPreStopHook every time

☸ Kubernetes version
v1.28.9-eks-036c24b
Cloud provider

on premise

🌍 Browser
Chrome
🧱 Your Environment
No response
✅ Proposed Solution
No response
👀 Have you spent some time to check if this issue has been raised before?

[X] I checked and didn't find any similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

AB#9864

https://stackoverflow.com/questions/78537044/sidekiqctl-stop-halting-the-further-execution-of-commands-in-prestop-on-devtron


show-language plugin broken in Prism 1.6.0

The show-language plugin doesn't seem to work for me anymore after upgrading to latest version of Prism (v1.6.0). It's also broken on your website http://prismjs.com/plugins/show-language/ so I'm fairly sure it's not something wrong in my setup.
As a side note, the CSS example on the page linked above seems to be throwing a 404.

Scratch that - the languages do show up in your example, but the behaviour seems to have changed just on hover which is not great implementation to be honest. I'll check latest CSS code against what I have in my setup.


Login not working after last update

After January 28th update of the YouTube Vanced app and MicroG the login would not work anymore.
I'd enter Google account logins, it would seemingly go well but the account would not be added and also after that, the YouTube app wouldn't work normally (buttons "Explore", "Subscriptions", "Library" sometimes would not work at all when clicked etc).
I had to find previous APKs from APK Mirror and install them both for YouTube and MicroG for login to work again.
Check below for my current working versions.

I'm using Samsung Galaxy A71 with the following software

This is an upstream issue; so close this and go here:
https://github.com/microg/GmsCore/issues/1373
Here is the solution.

This is an upstream issue; so close this and go here:
https://github.com/microg/GmsCore/issues/1373
Here is the solution.

Thanks @gghhkm that fixed the issue 👍

Thanks @gghhkm that fixed the issue 👍


Changelog for 1.10.0

As title, let's release v1.10.0!

Don't mind me as I close this and open it for the correct branch, first. :unamused:

Codecov Report

Merging #8976 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #8976      +/-   ##
==========================================
- Coverage   41.28%   41.26%   -0.02%
==========================================
  Files         546      546
  Lines       71006    71006
==========================================
- Hits        29315    29302      -13
- Misses      37971    37984      +13
  Partials     3720     3720

Impacted Files
Coverage Δ

models/repo_mirror.go
2.27% <0%> (-11.37%)
:arrow_down:

modules/process/manager.go
74.32% <0%> (-4.06%)
:arrow_down:

services/mirror/mirror.go
18.7% <0%> (-1.08%)
:arrow_down:

models/repo_list.go
73.14% <0%> (-0.93%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 16a4315...a6af3f7. Read the comment docs.


Ganic incoming

#

What article on docs.github.com is affected?

What part(s) of the article would you like to see updated?

Additional information

https://github.com/github/docs/projects/1#card-49100661


user agent does not loads in python lesser than 3.6

>>> from random_user_agent.user_agent import UserAgent
>>> user_agent_rotator = UserAgent(software_names=['chrome'], operating_systems=['windows', 'linux'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/random_user_agent/user_agent.py", line 23, in __init__
    for user_agent in self.load_user_agents():
  File "/usr/local/lib/python3.5/dist-packages/random_user_agent/user_agent.py", line 49, in load_user_agents
    return json.loads(user_agents)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

Thanks @HusnainAslam


Wistia onPlay not firing

Current Behavior
If you have the same wistia video on multiple pages within your react application, only the first one will fire the onPlay (and other) events.
Expected Behavior
Each wistia video should fire events.
Steps to Reproduce

Embed a wistia video on a page with onPlay.
Embed the same wistia video on another page/popup with onPlay.

Environment

URL attempting to play: Any
Browser: Any
Operating system: Any

Other Information
The issue looks to be because Wistia.js is using the Wistia video key as the unique identifier for the array. So two of the same video will not end up with unique entries in the array.

I'm afraid I need more information to look into this. I can render multiple of the same Wistia video and onPlay is firing correctly for all of the videos. I can also toggle videos off and on, and onPlay still fires correctly.

Sorry about that. Our specific use case is we have an overview page with a Wistia video, then you can click into a detailed page with that has a different react component, but the same Wistia video (url). The onPlay event fires fine for the first video, but when you get to the second video, the onPlay does not fire. Similarly, if you start on the detail page (reload browser), the onPlay fires there, but not on the overview. Does that help?

Can you reproduce this in jsFiddle? My attempts to cause the issue to happen aren't working.

@CookPete I think @brandon-rentpingmedia is correct. If you look here: https://github.com/CookPete/react-player/blob/bb69b1df1972ca50feb97090c8ff35aab5203b3c/src/players/Wistia.js#L113 you can see that the key that is being used for the wistia video is the URL. So if you have two of the same players on the page they are fighting with each other in the virtual dom.

Annoyingly, this is just how the Wistia player works. You render a div with class of wistia_async_abcde12345 where abcde12345 is the ID of your video, then call _wq.push({ id: 'abcde12345', ... }) to embed the player.
I can't see a way to use an ID that isn't the video ID (like a random string) to embed a player in the same way. The docs even mention having multiple players with the same video but I can't see how this is possible. Using wistia_async_randomstring123 will cause a Media not found. error and won't let you call replaceWith('actualvideoid').
If someone is able to contact Wistia about this, feel free to report back here.


[benchmark] Test styleFunctionSx vs. @styled-system/css

[ ] I have followed (at least) the PR section of the contributing guide.

Let's merge than and compare per these benchmarks 👍


Thinner plate with original stabilizers

Does it make sense to laser a thinner plate (say, 1.5 mm) and reuse original stabilizers?

I guess that would be an option. I don't have the original parts on hand anymore but if I remember correctly ALPS plates are 1.2 mm in general. But If you want to use the original stabilizers, the dimensions and positions of the current cutouts are definitely not correct and would have to be edited.


[Feature] Integrate test result cells to home screen

Description
This PR prepares the result card feature that allows to see status information about tests on the home screen. Furthermore, I do some refactoring of the HomeViewController and the HomeInteractor. Note that the final feature is not done until we can integrate the missing cells ("thank you", "You are positive").
Checklist

[x] Test your changes as thoroughly as possible before you commit them. Preferably, automate your test by unit/integration tests.
[ ] If this PR comes from a fork, please Allow edits from maintainers
[x] Set a speaking title. Format: {task_name} (closes #{issue_number}). For example: Use logger (closes # 41)
[ ] Link your Pull Request to an issue (if applicable)
[ ] Create Work In Progress [WIP] pull requests only if you need clarification or an explicit review before you can continue your work item.
[x] Make sure that your PR is not introducing unnecessary reformatting (e.g., introduced by on-save hooks in your IDE)

Description

Could you please write a short comment in my comments for example "I fixed it", that I could see what you fixed and what not :)

Could you please write a short comment in my comments for example "I fixed it", that I could see what you fixed and what not :)

I added some comments to clear up what I changed. I am now waiting for your PR to be merged to master and will integrate your changes into this branch. When I am done, I will mark this PR ready for review and request another review round. Thanks for your feedback so far!


Actions on containers that are on a deactivated host hang forever

Version - V0.11.1
Steps to reproduce the problem:
Have few containers started on a host.
Deactivate the host.
Try to Start/Stop/Restart  containers that are on this host.
UI shows that the operation is in progress forever.
The state of the containers are stuck in "Starting"/"Stopping".
Expected Behavior:
We should not allow for these options to be presented to the user.
If they are presented to the user, API should error out gracefully stating that these operations are not allowed because of the state in which the host is in.

Inactive means only that new containers are not scheduled there.  Actions should still work.  They seem fine to me so I think this was addressed a while ago.

Tested with rancher-server vesrion - v0.52.0-rc3
Able to Start/Stop/Restart containers on host that are in "inactive" state.


Theme Creator assistance tools

📝 Provide a description of the new feature
Tools we can provide:
Parse the css to find every class referenced, check if those classes exist or not, if they don't, create a popup modal to inform the theme maintainer
Color.ini modification from within marketplace, just need to a simple color picker.
Color.ini stealer, allow for users to mix and match color.inis from other themes onto their own
➕ Additional Information
No response

Is this done?


Add more logs for troubleshooting

Purpose

Add more logs for troubleshooting

Codecov Report

Merging #778 into master will decrease coverage by 0.02%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #778      +/-   ##
==========================================
- Coverage   53.47%   53.45%   -0.03%
==========================================
  Files          56       56
  Lines        2214     2215       +1
  Branches      218      218
==========================================
  Hits         1184     1184
- Misses        979      980       +1
  Partials       51       51

Impacted Files
Coverage Δ

...ng/src/main/java/io/cellery/impl/RunTestSuite.java
1.7% <0%> (-0.01%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f306620...f2cc002. Read the comment docs.


Snowflake warehouse

You should modify the warehouse settings in your documentation. Unless there is some overwhelming need, you should always start with an XS warehouse and always with a 1 minute auto suspend. I don't know much (anything) about Sentinel but I was asked to setup a connection for our security team. I don't know how often these will be run, but if a query was run once per hour on the settings suggested in the documentation, it would cost around $500/mo. With XS and 1 minute suspend you are closer to $50-75/mo.
Also, nothing is mentioned about a network policy. I don't know much about Azure as I use AWS but I assume since it is a function there is probably a way of binding it to an individual IP address and adding it to a network policy. A lot of snowflake environments will have network policies in place to restrict by IP so having some documentation about that would be helpful.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d7e4570e-2fd9-f986-7d16-90a1b21f3ad0
Version Independent ID: 00e42ab1-9e59-1ecb-581a-36bff02831ae
Content: Snowflake (using Azure Functions) connector for Microsoft Sentinel
Content Source: articles/sentinel/data-connectors/snowflake.md
Service: microsoft-sentinel
GitHub Login: @cwatson-cat
Microsoft Alias: cwatson

@rwilliams-mpg
Thanks for your feedback! We will investigate and update as appropriate.

Since I posted this, I was told by my security department that the default polling interval is 5m. Snowflake charges per credit for compute time. 1 credit on an Business Critical account is $4 and a Small warehouse uses 2 credits an hour, minimum 1 min billing each time the warehouse starts. Since the system polls every 5 minutes and shuts down the warehouse after 5 minutes of inactivity, it would end up running 24/7. Based on the recommended setup here it would end up being $5952/mo in costs.
$4/credit * 2 credits/hour (Small) * 24/hrs * 31
To put this in perspective, we run a snowflake instance running 40 + Tableau dashboards, dozens of automated reports, 3 Data engineers, ~20 analysts, hundreds of ETL's, 1000+ models in the warehouse and our total bill is less than half that a month.

@rwilliams-mpg Thanks for the feedback and information! I'll pass this along to the program managers that manage the data connector instructions in the upstream public repo: https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/Snowflake. This content is autogenerated from the information about the data connector that's published there.

@cwatson-cat Thank for quick responses.
@rwilliams-mpg
if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

@rwilliams-mpg
We are going to close this thread as resolved but if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.


Actually installing DLLs via @[Link]

To facilitate load-time dynamic linking on Windows, Crystal added experimental compiler support to detect and load DLLs from more "convenient" locations via a delay-load helper, but it does not work on DLLs that import data. So let's try to actually install them instead.
To get a better idea about how other compiled languages handle this, here are some examples with SDL2:

C/C++: The major C compilers are not responsible for installing DLLs. SDL2 has a CMakeLists.txt though, so any project that consumes SDL2 via CMake ≥3.21 should be able to use install(TARGET_RUNTIME_DLLS).
Go: It doesn't install DLLs.
Haskell: It doesn't install DLLs. (It uses MinGW-based toolchains, but the point is the same.)
Rust: It doesn't install DLLs. Another example with SFML
Zig: The link step does this via installBinFile, which can install files other than DLLs too. Presumably, this link step is automatically executed when building projects that depend on the given SDL2 bindings.

I propose adding a new dll parameter to the @[Link] compiler annotation:
@[Link(dll: "foo.dll")]
@[Link(dll: ["foo1.dll", "foo2.dll"])]
lib Foo
end

dll must be a string literal or an array literal of them. Each string on a lib indicates the name of a DLL that should be searched and installed. When building on Windows and not cross-compiling, the compiler either hardlinks or copies the DLLs to the same directory as the built executable. The search order used by the compiler, completely independent from the runtime DLL search order, would be:

CRYSTAL_LIBRARY_PATH
Same directory as the compiler itself
%PATH%

There is no longer automatic DLL detection from import libraries, and all libraries must declare their DLLs manually. This is because we only want to install library DLLs, not system DLLs like kernel32.dll or vcruntime140.dll, but no detection scheme is ever going to handle that. Hence, this proposal effectively reverts all the delay-load stuffs added in recent Crystal versions.
Alternatively, if we go down the Zig route of supporting arbitrary file installs, we could spare a new annotation:
{% if flag?(:windows) %}
  {%
    search = Crystal::LIBRARY_PATH.split(";") +
      ... + # `Process.executable_path` of the compiler itself in the macro language
      env("PATH").split(";")
    dll_path = nil
    search.each do |dir|
      unless dll_path
        full = ... # `File.join(dir, "foo.dll")` in the macro language
        dll_path = full if file_exists?(full)
      end
    end
  %}
  @[Install({{ dll_path }})]
{% end %}
lib Foo
end

Of course, this further blurs the line between Crystal as a compiler and Crystal as a build system, in absence of an actual build system tailor-made for Crystal.

So how does your proposal work for system dlls?
I'm not sure I understand the alternative Zig-like proposal, why is the dll_path in charge of the lib user?
I'll invoke @oprypin to weigh in.

So how does your proposal work for system dlls?

It doesn't, and not all of those DLLs are even redistributable.

why is the dll_path in charge of the lib user?

This is not 100% necessary here, @[Install] can be made as terse as @[Link(dll: ...)]; the snippet is mainly to show the minimum missing pieces needed to express the same functionality in the current macro language.

I personally don't find it hard to just copy some files in my own way. Users of other programming languages also need to do it.
Automatic copying can be slightly unexpected but if it's opt-in then great! 👍
Can be a nice solution 🙂

There is no correlation between a DLL's name and its import library's name. Using stdlib's dependencies as example:

gc-dynamic.lib imports gc.dll
iconv-dynamic.lib imports libiconv.dll
ssl-dynamic.lib imports libssl-3-x64.dll
z-dynamic.lib imports zlib1.dll

Reconciling the two sets of names means either rebuilding the DLLs, or giving the import libraries different names from non-Windows platforms. IMO neither option is desirable. And not doing this means the DLL names must be supplied somewhere, which is what dll: does.

They will just use the same @[Link] annotation as today, right? This was my question.

Yes, the meaning of @[Link]'s positional parameter remains the same

One practical problem is @[Link] doesn't understand the dll: parameter from Crystal 1.0 to 1.10, because argument parsing is part of the compiler. So the annotations in stdlib may look like this:
@[Link("z")]
{% if flag?(:win32) && compare_versions(Crystal::VERSION, "1.11.0-dev") >= 0 %}
  @[Link(dll: "zlib1.dll")]
{% end %}
lib LibZ
end

or like this:
@[Link("z")]
{% if flag?(:win32) && flag?(:supports_link_dll) %}
  @[Link(dll: "zlib1.dll")]
{% end %}
lib LibZ
end

where supports_link_dll is a compile-time flag added in the same PR that implements dll:.

I don't think this practical problem is a big issue. Stdlib and compiler version are usually expected to be equal. So there's not much to worry about for stdlib, yet it's easy to integrate safe guards.
For user code, it's probably not much of an issue either since there aren't many Windows specific library bindings anyway.

There is another caveat: if the local compiler is statically built, even from a compiler with support for dll:, then no DLLs would be present under .build, which means the local compiler in turn won't find the DLLs in the original compiler unless that compiler is already in %PATH%.
I guess we could add %CRYSTAL% as a search path before %PATH%, since that is what the wrapper scripts use if a local compiler is absent?


Playing "Plaguespreader Zombie" create infinite loops when there's no monster in the discard pile

Thank you for creating this mod, I enjoy playing it.
The issue is similar to https://github.com/adambennett/StS-DuelistMod/issues/3, but I created new issue for easier tracking.

This will be addressed in the next build as Plaguespreader will be nerfed considerably!

Fixed with release of v2.483.0!


beer-song: Update tests to match the other tracks tests

verse(99) now produces text starting with "99 beers on the wall"
updated example.exs

Awesome! Thanks so much! 🎉


Simplify HelloWorld tutorial including generic client invocation

Modified plug-in.cpp/h to minimize number of required changes and make code more concise
Modified generic client to

allow multiple input string
decrease amount of output (by default decrypted JSON output and receipts are not shown)
set a default URI value

Updated related README.md files

Signed-off-by: EugeneYYY<EMAIL_ADDRESS>

Could you rebase and push updated commit

In order to save time, as Eugene is busy today, I opened new PR #113  for this and I am closing this.


Use of dollar sign in inline code producing odd results

When using the following in Gmail or Blackboard, the conversion is wonky due to the first line:
Markdown:
* Need `'$'` in constructor

* Constructor and `display` don't have parameters

* No `return` for `menuChoice`

* Method `display` should not call `menuChoice()`

* Input in `menuChoice` should use `input.next().charAt(0)`

The output:
Need `’ in constructor

Constructor and display don’t have parameters

No return for menuChoice

Method display should not call menuChoice()

Input in menuChoice should use input.next().charAt(0)

` in constructor

Constructor and 0.07960530809955746 don’t have parameters

No 0.967272506292158 for 0.849369396167897

Method 0.9170844589050049 should not call 0.31504472305241105

Input in 0.1455992251161714 should use 0.2595394013807545

It appears to be the $ that's throwing it off in the first line.  If I remove the first line entirely or change it to something like 'c' (replacing the $ with a regular character), it works like a peach.
This is feedback for a Java course I'm teaching, and one of the things the students need to do is initialize a char variable to a dollar sign character, so I'm trying to give them feedback to that effect, but getting the weird results.
I've tried escaping the $ character (though I didn't see anything about needing to do that anywhere) and that doesn't seem to work.  :)
If I've missed something obvious, sorry for the bandwidth.  I've been using Markdown and Markdown Here for years and this is the first really weird conversion issue I've stumbled across.
Thanks for the great work!
Any insights?

It turns out the problem is user input gets passed as the replacement argument to a String.replace() call, but the replacement argument isn't just a plain string -- it has some magic values, including "$`".
The workaround is: Where you have one of the magic dollar-sign values, use two dollar signs. So, "$$`", etc.
Fixed in rev 498d796. Will be in next release.
Thanks.

Somewhat funny: The same problem seems to affect the Mocha test runner. When the test I added fails it prints out:
AssertionError: expected '`' to equal '`expected '`' to equal '


Create a dynamic web page which generates student grade sheet using V…

…BScript
Please accept my pull request sir

Please accept my pull request sir

No thank you.


[Feature] Apache Airflow DAG to run the new noun_project.py script

Problem Description

In order to get the noun_project.py script (see [#525  ]) into production, we need to implement a new Apache Airflow DAG to run the script.
Solution Description

Implement such a dag. For an example, see src/cc_catalog_airflow/dags/flickr_workflow.py (note the use of the new DAG Factory). The DAG should be configured to run the main function from src/cc_catalog_airflow/dags/provider_api_scripts/noun_project.py (no parameter) once per month, with a 24-hour Dag Run timeout. Leave concurrency at its default value of 1.

Blocket until #525 is sorted


In fixed_quad, the correct axis to sum w*f over is the last one

Since w has shape (n,), the result of multiplication w*func(y, *args) will have the last dimension equal to n, and this is the dimension we want to sum over. There may be other dimensions if func is vector-valued; summing over axis=0 is incorrect in this case. The change enables using fixed_quad for integrating vector-valued functions, and has no effect on integration of scalar-valued functions.
Test example: simultaneous integration of x**0, x**1, ... x**9 on the interval [0,1].
from scipy.integrate import fixed_quad
import numpy as np
def func(x):
    p = np.arange(0, 10).reshape(-1, 1)
    return x.reshape(1, -1)**p
result = fixed_quad(func, 0, 1)

With the current version of SciPy, the result is (array([ 0.12429409,  0.31110693,  0.56833333,  0.96181703,  0.96341687]), None) which makes no sense. The different functions got added, and the output has 5 values because there was no summation over the 5 points of quadrature.
With the commit, the result will be  (array([ 1. ,  0.5,  0.33333333,  0.25,  0.2, 0.16666667,  0.14285714,  0.125,  0.11111111,  0.1]), None) which are the correct integrals of the 10 components of func.

merged in gh-6885


Integration of nvim-ufo

It would be great to add an integration to nvim-ufo
It defines the following highlight groups:

hi default UfoFoldedFg guifg=Normal.foreground
hi default UfoFoldedBg guibg=Folded.background
hi default link UfoPreviewSbar PmenuSbar
hi default link UfoPreviewThumb PmenuThumb
hi default link UfoPreviewWinBar UfoFoldedBg
hi default link UfoPreviewCursorLine Visual
hi default link UfoFoldedEllipsis Comment
hi default link UfoCursorFoldedLine CursorLine

UfoFoldedFg: Foreground for raw text of folded line.
UfoFoldedBg: Background of folded line.
UfoPreviewSbar: Scroll bar of preview window, only take effect if the border is missing right horizontal line, like border = 'none'.
UfoPreviewCursorLine: Highlight current line in preview window if it isn't the start of folded lines.
UfoPreviewWinBar: Virtual winBar of preview window.
UfoPreviewThumb: Thumb of preview window.
UfoFoldedEllipsis: Ellipsis at the end of folded line, invalid if fold_virt_text_handler is set.
UfoCursorFoldedLine: Highlight the folded line under the cursor

Any colors suggestion to make it blend with Cappuccin?
Thanks

Any color suggestions to make it blend with Cappuccin?

I personally like the default highlight groups the way it is:

Perhaps you want something more eye candy? There is not much to work with here.

Hum, I do not get any highlight of folded code. Any idea?

What's the output of :hi Folded? Perhaps if you're using transparent
background?
https://github.com/catppuccin/nvim/blob/044e19ad9f5e48fabd1d96771f814db5dd48008c/lua/catppuccin/groups/editor.lua#L18
On Tue, Sep 5, 2023 at 3:34 AM Vincent Tencé @.***>
wrote:

Hum, I do not get any highlight of folded code. Any idea?
—
Reply to this email directly, view it on GitHub
https://github.com/catppuccin/nvim/issues/577#issuecomment-1705682032,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANRPOB5MLJXN2B5UMJ47FU3XYY3M5ANCNFSM6AAAAAA4AZPCDA
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

Transparent background, that's it.
Thanks ☺

Hello, @nullchilly. I have a question regarding the recent modification.
When the transparent_background=true parameter is set, I expect a completely transparent setup. However, when the fold is not respected, too many folds can ruin the entire transparent background.
Would UfoFoldedEllipsis be a better solution for @testinfected's problem?
For example:
UfoFoldedEllipsis = { bg = colors.green, fg = colors.blue }

It looks like this:

Users have the ability to customize the style of nvim-ufo to their liking.

I hope https://github.com/catppuccin/nvim/commit/846388d137590e653390ce2f84fea5351a7516ac is good enough :)


Feature Request: Cmdlets to add/remove explorer context menu (MSIX does not support custom registry keys)

Summary of the new feature/enhancement

As a user of the MSIX package I want to have an easy way of adding the explorer context menu that the MSI currently offers. MSIX does not support custom registry keys, therefore cmdlets seem to be a natural way for this, similar to the Enable-PSRemoting cmdlet.

As a user of the MSI, I want to remove/add the context menu post-installation dynamically to my liking or cleanup leftover keys from old previews that might have been left-over due to an upgrade.

I don't have time for this right now but might consider this at some point in the future.
cc @SteveL-MSFT
Technical implementation
2 cmdlets:

New-PowerShellExplorerContextMenu
Remove-PowerShellExplorerContextMenu

They probably need to have an optional version and preview/rtm parameter in case someone wants to remove/add the context menu for an older version as there have been some issues with past MSI previews upgrades. The MSI should probably start using them as well (depending on how complex custom actions in upgrade or uninstall is)

Would we want to generalize this to allow future extensibility?  Perhaps Get/Set-ShellExtension (need a better noun as Shell can be confusing, but don't want it specific to Windows either).

If we did a generic one (no strong opinion on pros/cons except that it would result in more maintenance and might become more complex in the future), then Id think the proper place would be a module of its own on the PSGallery. In this case, would you be happy to ship such a module by default similar to PSReadline?

@PowerShell/powershell-committee reviewed this, we suggest starting this as a separate module to validate the design and get user feedback


Soft Assertion Query

Hi brillout,
We are using Cucumber Framework and test cases have been written in Feature Files.
Basically we want to achieve 3 objectives as below:
1.expected vs actual comparison syntax
2.There are 10 steps(lines)of assertion in the feature file and if 5th line fails, the execution should still continue and execute 6th to 10th steps.
3.In the report the failed step(i.e 5th Step) should be clearly marked as "Failed". Thus we can clearly make out 9 out of 10 steps passed and 1 failed.
Please let me know how can I achieve this?

Sounds like reassert is not what you are looking for


Update setup.py to accommodate Python3.13.0

Changes made

Update setup.py to pass in a local dictionary to exec to capture the locals() in _version.py for package version reporting.

Motivation

Python3.13.0 enforces stricter safety standards on locals() with PEP 667 (see here)

This causes the following error upon attempting to install with python3.13 -m pip install openai-whisper:

Collecting openai-whisper
  Using cached openai-whisper-20240930.tar.gz (800 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      <string>:5: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
      Traceback (most recent call last):
        File "/Users/jasonshipp/.pyenv/versions/3.13.0/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
          ~~~~^^
        File "/Users/jasonshipp/.pyenv/versions/3.13.0/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/jasonshipp/.pyenv/versions/3.13.0/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/private/var/folders/hk/8xz63_l16zb9snz57jx9hqgr0000gn/T/pip-build-env-l_19mrvs/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/hk/8xz63_l16zb9snz57jx9hqgr0000gn/T/pip-build-env-l_19mrvs/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
          self.run_setup()
          ~~~~~~~~~~~~~~^^
        File "/private/var/folders/hk/8xz63_l16zb9snz57jx9hqgr0000gn/T/pip-build-env-l_19mrvs/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 516, in run_setup
          super().run_setup(setup_script=setup_script)
          ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/hk/8xz63_l16zb9snz57jx9hqgr0000gn/T/pip-build-env-l_19mrvs/overlay/lib/python3.13/site-packages/setuptools/build_meta.py", line 318, in run_setup
          exec(code, locals())
          ~~~~^^^^^^^^^^^^^^^^
        File "<string>", line 21, in <module>
        File "<string>", line 11, in read_version
      KeyError: '__version__'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip

EXTRA NOTES

This does not enable openai-whisper to be pip installed with Python3.13, due to a dependency on the numba library. This issue and missing dependency can be tracked here

Would it be a more intuitive and durable solution to replace the deprecated pkg_resources with packaging?
The error message contained

DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html

Use of pkg_resources is deprecated in favor of importlib.resources, importlib.metadata and their backports (importlib_resources, importlib_metadata). Some useful APIs are also provided by packaging (e.g. requirements and version parsing). Users should refrain from new usage of pkg_resources and should work to port to importlib-based solutions.

Any luck with this?

Yes, #2435 merge would be great! But there's an issue with numba <-> llvmlite


Drop privileges to reduce impact of an exploit.

running spindump as root is dangerous, as there is a lot of analyzer code also running as root. I would suggest the approach of chmod +4755 spindump, and this patch. OpenBSD does this already with ping/ping6, except that it's using setresuid() & setresguid().
Comments/feedback welcome. I'm also looking at push the capture interface code earlier so that root privileges can be dropped.

We have not forgotten this, but working on other things first. This will also get done soon...

Working on this

@jariarkko how can I help ?

Thanks again.
This issue has now been fixed in commits 270d75a and 848079e, and will be closed.

Loganaden, it is mostly done (I think), but could you download the newest version and test that it is to your liking? That would be very helpful!

@jariarkko tested here. Thanks.

Thank you!


[bug]: Can not select port 443 in configuration

Description
When selecting a pot all ports lower the 4 numbers eg. 443 arre marked as not possible

Affected part
Which part of the application is impacted

[x] General UI
[x] General Options/Settings
[x] Connection settings
[ ] Task list
[ ] Notification
[ ] Pop-up
[ ] Context-menu
[ ] Content-script (inside the page)
[ ] Overlay

Screenshots or Mock up
If possible provide a screenshot or a mock-up of the issue.
Steps to Reproduce the Issue

try setting port 443

Hello, priviledge ports can now be used in credentials. A warning will be shown but it can be overriden.
Hopes that helps :)

If you enjoy the extension, please leave a review in the webstore, it helps with discovery :)


Enum-based supertype, codegen

This blocks #79 and #70

This should be after #59

Required part is split from #59 into #82

  ENTITY named_unit
    SUPERTYPE OF (ONEOF (si_unit,conversion_based_unit) ANDOR ONEOF (
        length_unit,plane_angle_unit));
      dimensions : dimensional_exponents;
  END_ENTITY; -- named_unit

and
  ENTITY b_spline_curve
    SUPERTYPE OF (ONEOF (uniform_curve,b_spline_curve_with_knots,
        quasi_uniform_curve,bezier_curve) ANDOR rational_b_spline_curve)
    SUBTYPE OF (bounded_curve);
      degree              : INTEGER;
      control_points_list : LIST [2:?] OF cartesian_point;
      curve_form          : b_spline_curve_form;
      closed_curve        : LOGICAL;
      self_intersect      : LOGICAL;
    DERIVE
      upper_index_on_control_points : INTEGER := SIZEOF(
                                         control_points_list) - 1;
      control_points                : ARRAY [0:
                                         upper_index_on_control_points] OF
                                          cartesian_point := list_to_array(
                                         control_points_list,0,
                                         upper_index_on_control_points);
    WHERE
      wr1: ('EXPLICIT_DRAUGHTING.UNIFORM_CURVE' IN TYPEOF(SELF)) OR (
               'EXPLICIT_DRAUGHTING.QUASI_UNIFORM_CURVE' IN TYPEOF(SELF))-
               OR ('EXPLICIT_DRAUGHTING.BEZIER_CURVE' IN TYPEOF(SELF)) OR (
               'EXPLICIT_DRAUGHTING.B_SPLINE_CURVE_WITH_KNOTS' IN TYPEOF(
               SELF));
  END_ENTITY; -- b_spline_curve

are not correctly handled due to ANDOR

  ENTITY approval_assignment
    ABSTRACT SUPERTYPE;
      assigned_approval : approval;
  END_ENTITY; -- approval_assignment

is also not handled correctly.

SUPERTYPE handling problem has been fixed in #86

SUBTYPE OF and SUPERTYPE OF specifications are not consistent:
  ENTITY annotation_fill_area
    SUBTYPE OF (geometric_representation_item);
      boundaries : SET [1:?] OF curve;
  END_ENTITY; -- annotation_fill_area

  ENTITY geometric_representation_item
    SUPERTYPE OF (ONEOF (point,direction,vector,placement,curve,
        geometric_set))
    SUBTYPE OF (representation_item);
    DERIVE
      dim : dimension_count := dimension_of(SELF);
    WHERE
      wr1: SIZEOF(QUERY ( using_rep <* using_representations(SELF) | (NOT-
               ('EXPLICIT_DRAUGHTING.GEOMETRIC_REPRESENTATION_CONTEXT' IN-
               TYPEOF(using_rep.context_of_items))) )) = 0;
  END_ENTITY; -- geometric_representation_item

We should use sub-super graph in #86 always not only for ABSTRACT SUPETYPE case.


Hello. when i run this code on cuda environment this error happend. would you please help me fixing it?

ValueError: Encountered tensor with size 640000 in dimension 0, but expected size 3200.
It seems that it's because of this code： z = conv(z, edge_index, edge_attr) in gat.py
It doesn't seem as simple as the official website suggests：For a GAT-based setup, simply replace the GCN model initializations with GAT.

I meet the same problem as you. How did you solve it？

ValueError: Encountered tensor with size 640000 in dimension 0, but expected size 3200. It seems that it's because of this code： z = conv(z, edge_index, edge_attr) in gat.py It doesn't seem as simple as the official website suggests：For a GAT-based setup, simply replace the GCN model initializations with GAT.

I met the same problem, did you solve it?

I solved this problem by modifying the pytorch version as described in #29

Thanks! I also solved it by modify the GAT and MessagePassing class with the prior PyG version (2.0.4).


modules: acpica: Fix CMakeLists.txt style

Fix indentation to use spaces (instead of tabs) and remove the unnecessary repetition of the condition inside endif().

This fixes the style to conform with what's used everywhere else in the Zephyr tree. It also fixes warnings in my editor whenever I look at this file :)


There are some program problems or bug.

When adding a picture as a reference picture, the parameters are abnormal.
For example submitted is:
https://cdn.discordapp.com/attachments/1097388640411320393/1112334804248186970/girl.jpg a girl
The output is:
https:cdndiscordappcomattachments10973886404113203931112334804248186970girljpg a girl --seed<PHONE_NUMBER> --ar 3:4 --q 1.0 --stylize 100 --v 5.1
Obviously, image links are filtered out of symbols.
When using Describe, the callback cannot be completed when the picture name contains spaces or is not in English.
Using the variant function, if Midjourney's settings enable Remix mode, the program will get stuck.
There is also the inability to view historical records, except for the works collected in the gallery.
When using the variant or upgrade function, the status prompt text is at the top. If I don’t look up, I always think that the program is stuck again.

Is resolved


Including template as placeholder in one file make that non-executable in another

block.ss

- template [%fileName%](text)
	- block root
		< div
			- block body
				{text}

body.ss

- include './block'

- template [%fileName%]() extends ['block']
	- block body
		Это тело
		- call self['block']('Это блок')

index.ss

- include './block' as placeholder
- include './body'

- template [%fileName%]() extends ['block']
	- block body
		- call self['body']()

If you run index.ss on execution you've got an error 'TypeError: self.block is not a function'. If you delete line 'as placeholder' in the 'index.ss', everything is okay.

Исправлено.


Pt 160014862 missing db table

Sporadically on CI we seem to not be able to find a database table in the test case aeu_requests. This test is a remainder from early days added to allert us when a started system had errors in its logs.
I cannot understand why this happens. Locally I have run this test with combinations of other tests in an attempt to reproduce it. No luck. I tried reproducing it on CI by pushing it a few times, no luck.
I wrote a completely new test that can replace the old one and also added some more info in the test case to help when the test fails again. In the long run, we can remove the test, but I would like to understand the cause of this.
Made use of this patch to cleanup aec_db a bit.

Happened again in recent master https://circleci.com/gh/aeternity/epoch/28827


ManageIQ.dynatreeReplacement - fix nonexistent function call - cmfeAddNodeChildren

cfmeAddNodeChildren is now called miqAddNodeChildren, but ManageIQ.dynatreeReplacement.replace would still attempt to call the old method..
(Discovered as part of #9019)

@martinpovolny darga label?


Fix code scanning alert - Reflected server-side cross-site scripting

Tracking issue for:

[ ] https://github.com/jamesvandyne/tanzawa/security/code-scanning/5

Fixed merged in #141. Closing.


Update documentation for Storage Connection String and add Azure Extension bundle

I tried running through the documentation and the Blob Trigger didn't work until I performed the steps listed.

Thanks for catching this! @yokawasa, i believe this is needed for all functions that use a blob trigger/input/output, right? Or any function that touches Azure Storage?

@lastcoolnameleft, the host.json in the root of the function app seems to have the bundles in
already, which should take effect for all functions under that function app. Did you clone the repo or started from scratch and maybe missed that config file?
If the latter, i believe we should definitely mention why the bundle is in there and link to the host.json file for clarity. We should surface this in the root README. All this should avoid suboptimal user experience. Can you please make this change?

Closing this PR as I realized it was PEBKAC.  (I ran func host start in the sub dir (e.g. blob-trigger-watermark-blob-out-binding) , which allows the function to start and creates a new host file. once I added the changes, then everything worked for me, but if I would have ran it in the correct directory, that would have fixed my root problem)
Sorry for the inconvenience.
That said, I think the documentation suggestions you stated would be useful.  Specifying which directory to run in specifically for my case.

Created feature request to Function Core tools to reflect issue I encountered due to inconsistent behavior:  https://github.com/Azure/azure-functions-core-tools/issues/1489


Reverting Firefox OIT Fix #4815

Description
Reverts code change for Firefox OIT.
Issue number and link
Issue #4815
https://github.com/CesiumGS/cesium/issues/4815
Testing plan

Author checklist

[x] I have submitted a Contributor License Agreement
[x] I have added my name to CONTRIBUTORS.md
[ ] I have updated CHANGES.md with a short summary of my change
[ ] I have added or updated unit tests to ensure consistent code coverage
[ ] I have updated the inline documentation, and included code examples where relevant
[x] I have performed a self-review of my code

@javagl Could you please review this PR?

@javagl I've reverted the changes to #5047 as well.

The current CI failure seems to only/mainly be at the linting stage. Doing an
npm run prettier
will probably remove some 'space' or newline somewhere, and that should fix it.
(There also is a test failure, but that seems pretty unrelated - that can be handled later if it is still there after the linting update)
@ggetz The "Review required" is new - I'm not sure whether I'm automatically expected to "Approve" this. But I'd do this as soon as CI passes (even though someone else will merge it)

The "Review required" is new - I'm not sure whether I'm automatically expected to "Approve" this. But I'd do this as soon as CI passes (even though someone else will merge it)

@javagl If you can do so, please do!

Thanks @squrki and @javagl!


completing support for NSG security rule; Network#subnets() as Map

Can one of the admins verify this patch?

Can one of the admins verify this patch?