Officially maintained Arrow2 branch

Which issue does this PR close?
Close arrow2 milestone https://github.com/apache/arrow-datafusion/milestone/3
Rationale for this change
Provide a complete arrow2 based datafusion implementation for full evaluation of the migration. This should give us a good feeling of the arrow2 API UX as well as a starting point for performance benchmarks within datafusion and downstream projects.
The goal is to merge this code into an official arrow2 branch in the short run, until we are comfortable doing the switch in master.
What changes are included in this PR?

Switched to arrow2
Enabled miri test (current miri failure is caused by https://github.com/rust-lang/miri/issues/602)

Here is a TPCH benchmark I ran on my Linux laptop (baseline 2008b1dc06d5030f572634c7f8f2ba48562fa636):

On avg, we are getting around 5% speed up across the board, with q5 at 11% and q12 at only 1% as the two outliners. If this performance gain can also be replicated in downstream projects, then I think it would be a strong case for us to do the arrow2 swtich. On top of this, we end up with a nice 1000+ lines of code reduction ;)
Are there any user-facing changes?
Yes, downstream consumer of datafusion will need to switch to arrow2 as well.

Thanks to @Igosuki and @yjshen , we are now down to a single parquet row group pruning related test failure. All other 855 tests are passing without error.

I found that the peak memory usage of this branch increases by 80% compared to the master branch。
sql : select avg(user_id) from parquet_event_1 group by user_name limit 10
test dataset ： total 450 million, 50 million users

branch
peak memory

master(d7e465 and 35d65fc)
10G

arrow2_merge
6G

@ic4y I'm confused with the numbers and "increases by 80% compared to the master branch"

@yjshen
Sorry, I made a typo, it has been corrected.
I want to express is the same query, using 6G memory on arrow but 10G memory on arrow2

Just use massif or heaptrack

@houqp What hardware/setup did you use to run the benchmark ? I'm actually getting way worse performance if running tpch using parquet

Thank you @houqp  and @Igosuki  -- I'll try and take a look at this later today or tomorrow. I will also start the discussion of "what does this mean for arrow-rs", which I expect may take some time to come to consensus on today

Ok adding BufReader gave 50% perf on parquet. https://github.com/houqp/arrow-datafusion/pull/19/commits/d8a184969bd2a88292158cbc704e0cb959b28ea6

@ic4y  @tustvold  mentioned to me in passing that one of the differences in memory usage might be related to parquet2 decoding entire pages at a time rather than reading them out in smaller batches. I am not sure if that matches your observations

Discussion of arrow-rs / arrow2 is here in case anyone missed it: https://github.com/apache/arrow-rs/issues/1176

Here are some quick benchmark results comparing the master branch and this PR, running on a threadripper desktop with 24 cores and an NVMe drive. I ran each query once only.
These are the commands that I ran:
cargo build --release
cd benchmarks
cargo run --release --bin tpch -- benchmark datafusion --iterations 1 --path /mnt/bigdata/tpch/sf100-24part-parquet --format parquet --query 13 --batch-size 4096 --partitions 24

Queries 1 and 6 were both at least 3x slower and query 12 was more than 2x slower.
I used the following commits:
Master: 1c39f5ce865e3e1225b4895196073be560a93e82
Arrow2: e53d165f018a54d47f80ff2a132f83cee363c79c
I will poke around a bit myself this weekend and see if I can debug this based on metrics to try and determine where the performance issues are.

Metrics for query 1 from master & arrow2.
Master
=== Physical plan with metrics ===
SortExec: [l_returnflag@0 ASC NULLS LAST,l_linestatus@1 ASC NULLS LAST], metrics=[output_rows=4, elapsed_compute=148.532µs]
  CoalescePartitionsExec, metrics=[output_rows=4, elapsed_compute=26.889µs]
    ProjectionExec: expr=[l_returnflag@0 as l_returnflag, l_linestatus@1 as l_linestatus, SUM(lineitem.l_quantity)@2 as sum_qty, SUM(lineitem.l_extendedprice)@3 as sum_base_price, SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount)@4 as sum_disc_price, SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount * Int64(1) + lineitem.l_tax)@5 as sum_charge, AVG(lineitem.l_quantity)@6 as avg_qty, AVG(lineitem.l_extendedprice)@7 as avg_price, AVG(lineitem.l_discount)@8 as avg_disc, COUNT(UInt8(1))@9 as count_order], metrics=[output_rows=4, elapsed_compute=162.216µs]
      HashAggregateExec: mode=FinalPartitioned, gby=[l_returnflag@0 as l_returnflag, l_linestatus@1 as l_linestatus], aggr=[SUM(lineitem.l_quantity), SUM(lineitem.l_extendedprice), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount Multiply Int64(1) Plus lineitem.l_tax), AVG(lineitem.l_quantity), AVG(lineitem.l_extendedprice), AVG(lineitem.l_discount), COUNT(UInt8(1))], metrics=[output_rows=4, elapsed_compute=836.092µs]
        CoalesceBatchesExec: target_batch_size=2048, metrics=[output_rows=96, elapsed_compute=2.467991ms]
          RepartitionExec: partitioning=Hash([Column { name: "l_returnflag", index: 0 }, Column { name: "l_linestatus", index: 1 }], 24), metrics=[fetch_time{inputPartition=0}=121.646159259s, repart_time{inputPartition=0}=1.068665ms, send_time{inputPartition=0}=181.052µs]
            HashAggregateExec: mode=Partial, gby=[l_returnflag@5 as l_returnflag, l_linestatus@6 as l_linestatus], aggr=[SUM(lineitem.l_quantity), SUM(lineitem.l_extendedprice), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount Multiply Int64(1) Plus lineitem.l_tax), AVG(lineitem.l_quantity), AVG(lineitem.l_extendedprice), AVG(lineitem.l_discount), COUNT(UInt8(1))], metrics=[output_rows=96, elapsed_compute=62.022452324s]
              ProjectionExec: expr=[l_extendedprice@1 * CAST(1 AS Float64) - l_discount@2 as BinaryExpr-*BinaryExpr--Column-lineitem.l_discountLiteral1Column-lineitem.l_extendedprice, l_quantity@0 as l_quantity, l_extendedprice@1 as l_extendedprice, l_discount@2 as l_discount, l_tax@3 as l_tax, l_returnflag@4 as l_returnflag, l_linestatus@5 as l_linestatus], metrics=[output_rows=591599326, elapsed_compute=2.333647686s]
                CoalesceBatchesExec: target_batch_size=2048, metrics=[output_rows=591599326, elapsed_compute=18.07630248s]
                  FilterExec: l_shipdate@6 <= 10471, metrics=[output_rows=591599326, elapsed_compute=14.438278954s]
                    ParquetExec: batch_size=4096, limit=None, partitions=[/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-5.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-10.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-11.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-13.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-6.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-14.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-20.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-2.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-22.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-19.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-0.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-16.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-21.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-23.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-4.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-17.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-9.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-1.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-18.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-8.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-12.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-15.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-7.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-3.parquet], metrics=[predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-9.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-22.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-23.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-18.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-1.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-11.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-3.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-5.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-23.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-11.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-1.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-14.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-3.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-9.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-21.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-18.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-6.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-7.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-21.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-19.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-10.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-5.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-17.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-15.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-12.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-8.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-16.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-12.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-6.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-16.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-10.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-4.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-20.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-0.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-7.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-2.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-8.parquet}=0, num_predicate_creation_errors=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-22.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-15.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-0.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-14.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-13.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-2.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-19.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-4.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-13.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-17.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-20.parquet}=0]

+--------------+--------------+------------+--------------------+--------------------+--------------------+--------------------+-------------------+----------------------+-------------+
| l_returnflag | l_linestatus | sum_qty    | sum_base_price     | sum_disc_price     | sum_charge         | avg_qty            | avg_price         | avg_disc             | count_order |
+--------------+--------------+------------+--------------------+--------------------+--------------------+--------------------+-------------------+----------------------+-------------+
| A            | F            |<PHONE_NUMBER> |<PHONE_NUMBER>243.051  |<PHONE_NUMBER>986.644  |<PHONE_NUMBER>226.674  | 25.49937018008533  | 38236.11640655464 | 0.05000224292310849  | 148047875   |
| N            | F            | 98553062   |<PHONE_NUMBER>85.98004 |<PHONE_NUMBER>65.03497 |<PHONE_NUMBER>32.7758  | 25.501556956882876 | 38237.19938880452 | 0.04998528433805398  | 3864590     |
| N            | O            |<PHONE_NUMBER> |<PHONE_NUMBER>0531.352 |<PHONE_NUMBER>4399.148 |<PHONE_NUMBER>4528.229 | 25.500009351223113 | 38237.22761127162 | 0.049997917972634566 | 291619606   |
| R            | F            |<PHONE_NUMBER> |<PHONE_NUMBER>938.198  |<PHONE_NUMBER>920.04   |<PHONE_NUMBER>982.476  | 25.500066311082758 | 38236.6972423322  | 0.05000130406955949  | 148067255   |
+--------------+--------------+------------+--------------------+--------------------+--------------------+--------------------+-------------------+----------------------+-------------+
Query 1 iteration 0 took 8165.9 ms
Query 1 avg time: 8165.94 ms

Arrow2 PR
=== Physical plan with metrics ===
SortExec: [l_returnflag@0 ASC NULLS LAST,l_linestatus@1 ASC NULLS LAST], metrics=[output_rows=4, elapsed_compute=90.895µs]
  CoalescePartitionsExec, metrics=[output_rows=4, elapsed_compute=16.752µs]
    ProjectionExec: expr=[l_returnflag@0 as l_returnflag, l_linestatus@1 as l_linestatus, SUM(lineitem.l_quantity)@2 as sum_qty, SUM(lineitem.l_extendedprice)@3 as sum_base_price, SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount)@4 as sum_disc_price, SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount * Int64(1) + lineitem.l_tax)@5 as sum_charge, AVG(lineitem.l_quantity)@6 as avg_qty, AVG(lineitem.l_extendedprice)@7 as avg_price, AVG(lineitem.l_discount)@8 as avg_disc, COUNT(UInt8(1))@9 as count_order], metrics=[output_rows=4, elapsed_compute=269.589µs]
      HashAggregateExec: mode=FinalPartitioned, gby=[l_returnflag@0 as l_returnflag, l_linestatus@1 as l_linestatus], aggr=[SUM(lineitem.l_quantity), SUM(lineitem.l_extendedprice), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount Multiply Int64(1) Plus lineitem.l_tax), AVG(lineitem.l_quantity), AVG(lineitem.l_extendedprice), AVG(lineitem.l_discount), COUNT(UInt8(1))], metrics=[output_rows=4, elapsed_compute=966.04µs]
        CoalesceBatchesExec: target_batch_size=2048, metrics=[output_rows=96, elapsed_compute=3.118624ms]
          RepartitionExec: partitioning=Hash([Column { name: "l_returnflag", index: 0 }, Column { name: "l_linestatus", index: 1 }], 24), metrics=[send_time{inputPartition=0}=202.529µs, fetch_time{inputPartition=0}=167.702662304s, repart_time{inputPartition=0}=881.476µs]
            HashAggregateExec: mode=Partial, gby=[l_returnflag@5 as l_returnflag, l_linestatus@6 as l_linestatus], aggr=[SUM(lineitem.l_quantity), SUM(lineitem.l_extendedprice), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount Multiply Int64(1) Plus lineitem.l_tax), AVG(lineitem.l_quantity), AVG(lineitem.l_extendedprice), AVG(lineitem.l_discount), COUNT(UInt8(1))], metrics=[output_rows=96, elapsed_compute=69.277706429s]
              ProjectionExec: expr=[l_extendedprice@1 * CAST(1 AS Float64) - l_discount@2 as BinaryExpr-*BinaryExpr--Column-lineitem.l_discountLiteral1Column-lineitem.l_extendedprice, l_quantity@0 as l_quantity, l_extendedprice@1 as l_extendedprice, l_discount@2 as l_discount, l_tax@3 as l_tax, l_returnflag@4 as l_returnflag, l_linestatus@5 as l_linestatus], metrics=[output_rows=591599326, elapsed_compute=1.948225184s]
                CoalesceBatchesExec: target_batch_size=2048, metrics=[output_rows=591599326, elapsed_compute=19.900026641s]
                  FilterExec: l_shipdate@6 <= 10471, metrics=[output_rows=591599326, elapsed_compute=17.25811968s]
                    ParquetExec: batch_size=4096, limit=None, partitions=[/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-5.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-10.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-11.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-13.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-6.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-14.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-20.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-2.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-22.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-19.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-0.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-16.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-21.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-23.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-4.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-17.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-9.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-1.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-18.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-8.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-12.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-15.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-7.parquet, /mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-3.parquet], metrics=[row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-23.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-21.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-2.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-11.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-0.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-3.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-0.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-3.parquet}=0, num_predicate_creation_errors=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-12.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-7.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-4.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-2.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-23.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-9.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-6.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-5.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-18.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-13.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-1.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-10.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-8.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-1.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-4.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-5.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-22.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-17.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-9.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-21.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-7.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-12.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-8.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-18.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-16.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-15.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-14.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-10.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-19.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-16.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-17.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-11.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-15.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-6.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-13.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-20.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-20.parquet}=0, predicate_evaluation_errors{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-14.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-19.parquet}=0, row_groups_pruned{filename=/mnt/bigdata/tpch/sf100-24part-parquet/lineitem/part-22.parquet}=0]

+--------------+--------------+------------+--------------------+--------------------+--------------------+--------------------+-------------------+---------------------+-------------+
| l_returnflag | l_linestatus | sum_qty    | sum_base_price     | sum_disc_price     | sum_charge         | avg_qty            | avg_price         | avg_disc            | count_order |
+--------------+--------------+------------+--------------------+--------------------+--------------------+--------------------+-------------------+---------------------+-------------+
| A            | F            |<PHONE_NUMBER> |<PHONE_NUMBER>243.05   |<PHONE_NUMBER>986.642  |<PHONE_NUMBER>226.676  | 25.49937018008533  | 38236.11640655463 | 0.05000224292310846 | 148047875   |
| N            | F            | 98553062   |<PHONE_NUMBER>85.98004 |<PHONE_NUMBER>65.0348  |<PHONE_NUMBER>32.77588 | 25.501556956882876 | 38237.19938880452 | 0.04998528433805397 | 3864590     |
| N            | O            |<PHONE_NUMBER> |<PHONE_NUMBER>0531.35  |<PHONE_NUMBER>4399.143 |<PHONE_NUMBER>4528.23  | 25.500009351223113 | 38237.22761127161 | 0.04999791797263452 | 291619606   |
| R            | F            |<PHONE_NUMBER> |<PHONE_NUMBER>938.199  |<PHONE_NUMBER>920.042  |<PHONE_NUMBER>982.476  | 25.500066311082758 | 38236.6972423322  | 0.05000130406955945 | 148067255   |
+--------------+--------------+------------+--------------------+--------------------+--------------------+--------------------+-------------------+---------------------+-------------+
Query 1 iteration 0 took 27109.6 ms
Query 1 avg time: 27109.65 ms

Thank you everyone for all the reviews and comments so far. @Igosuki and I have addressed most of them. Here are the two remaining todo items:

Get the parquet row group filter test to pass
Restore sql integration test migration. All those sql tests were migrated and passing previously, but those changes got lost when we merged the sql test refactoring from master.

I will keep working on this tomorrow. In the mean time, feel free to send PRs to my fork if you are interested in helping. After these two items are fixed, I will run another round of benchmark to double check the performance fix. It's quite interesting that I got the opposite performance test result initial even without that file buf fix :P I will dig into what's causing that as well.

Yes sorry about that, these were simply comments to Indicate that these
particular feature tests were not passing.
Le dim. 16 janv. 2022 à 09:52, QP Hou @.***> a écrit :

Thank you everyone for all the reviews and comments so far. @Igosuki
https://github.com/Igosuki and I have addressed most of them. Here are
the two remaining todo items:

Get the parquet row group filter test to pass
Restore sql integration test migration. All those sql tests were
migrated and passing previously, but those changes got lost when we merged
the sql test refactoring from master.

I will keep working on this tomorrow. In the mean time, feel free to send
PRs to my fork if you are interested in helping. After these two items are
fixed, I will run another round of benchmark to double check the
performance fix. It's quite interesting that I got the opposite performance
test result initial even without that file buf fix :P I will dig into
what's causing that as well.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/arrow-datafusion/pull/1556#issuecomment-1013835311,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADDFBSPYHZOIG4O2XOGWZDUWKBLXANCNFSM5L5P5AVQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

The parquet row group test failure turned out to be a red herring. The asserted expected result is actually not correct. I have filed a follow up issue at https://github.com/apache/arrow-datafusion/issues/1591. I changed the expected result in this branch to fix the test failure for now. What the predicate pruning logic returns in this branch is more correct than what we have in master, but still wrong. The proper fix is out of scope of arrow2 migration and tracked in #1591.
We are now passing all 856 unit tests. 2 more integration tests to fix, which are caused by difference in how arrow2 formats binary array.

I also noticed my benchmarks were ran with data generated from tpch-gen.sh, which only produces single partition CSV files. @andygrove could you share with me how you generated your sf100 dataset?

The parquet row group test failure turned out to be a red herring. The asserted expected result is actually not correct. I have filed a follow up issue at #1591.

I shared the same observation in https://github.com/houqp/arrow-datafusion/pull/16, but ignored the test at the time.

all datafusion unit and integration tests are passing now, down to a single test failure in datafusion-cli related to json display format.

oops, looks like the arrow2 branch got updated with latest commits from master, anyone mind if I revert it back to https://github.com/apache/arrow-datafusion/commit/2008b1dc06d5030f572634c7f8f2ba48562fa636 and handle master catch up in a follow up PR?

All tests are passing now, the MIRI check is failing due to an upstream tokio issue I believe. I will file some follow up issues tomorrow to track the remaining work needed for us to make the final call on master merge.

Thank you @jorgecarleitao @yjshen and @Igosuki for your hard work on the migration thus far :)

Wow! Milestone reached! Thanks for driving on this and making it happen @houqp 👍

Great work @houqp
Le jeu. 20 janv. 2022 à 08:27, Yijie Shen @.***> a
écrit :

Wow! Milestone reached! Thanks for driving on this and making it happen
@houqp https://github.com/houqp 👍
—
Reply to this email directly, view it on GitHub
https://github.com/apache/arrow-datafusion/pull/1556#issuecomment-1017190215,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADDFBWXQK6FFRP4FXB26JDUW62PLANCNFSM5L5P5AVQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

Thanks @houqp , epic work !

Query 1 iteration 0 took 27109.6 ms
Query 1 avg time: 27109.65 ms

@andygrove What tool did you use to get such a smooth CPU chart ?

Quick update on this, I have cleaned up the issues in the arrow2 milestone: https://github.com/apache/arrow-datafusion/milestone/3. The main remaining items are:

https://github.com/apache/arrow-datafusion/issues/1652
https://github.com/apache/arrow-datafusion/issues/1656
https://github.com/apache/arrow-datafusion/issues/1657

I will keep work on issues in the arrow2 milestone whenever I have capacity. If anyone of you are interested in helping, please feel free to comment on those issues or send PRs to the official arrow2 branch.


responders - rails 4.2

hi
is token auth ready for rails 4.2 (with responders gem moved out?)
i can't seem to get it to respond to :json ..
thanks

@matteolc - I haven't tested against rails 4.2 yet. I'll take a look ASAP.

Thanks much appreciated ;)

FYI
Here's my application controller:
class ApplicationController < ActionController::API
include ActionController::Caching
include ActionController::MimeResponds
def self.respond_to(*mimes)
include ActionController::RespondWith::ClassMethods
end
respond_to :json
include DeviseTokenAuth::Concerns::SetUserByToken
before_filter :authenticate_account!
end
I had to add
def self.respond_to(*mimes)
include ActionController::RespondWith::ClassMethods
end
to avoid the error: ActionController::RoutingError (The controller-level respond_to' feature has been extracted to the responders` gem.
I think it's related to warden.
Thanks

Update.
If i make an explicit json request, session controller responds with json:
curl http:/..../api/auth/sign_in.json --data "email=...&password=..."
Processing by DeviseTokenAuth::SessionsController#create as JSON
however, i must be missing something as i get straight 401 with no active record call:
Filter chain halted as :authenticate_account! rendered or redirected
Completed 401 Unauthorized in 1ms (Views: 0.2ms | ActiveRecord: 0.0ms)

Update
It must be related to request.headers / permitted parameters ..
Filter chain is halted in set_user_by_token at return false unless @token because request.headers['..'] is apparently empty / gets filtered
Started POST "/api/v1/auth/sign_in"
Parameters<EMAIL_ADDRESS>"password"=>"12345678"<EMAIL_ADDRESS>"access-token"=>"12345678"}
UID:
token:
client_id: default
Filter chain halted as :authenticate_account! rendered or redirected

So actually all that should be required to make it work when inheriting from ActionController::API is
class ApplicationController < ActionController::API
include ActionController::MimeResponds
def self.respond_to(*mimes)
include ActionController::RespondWith::ClassMethods
end
respond_to :json
end
(Not sure what wasn't working before :O)

weird..i got it to work, restarted the server, and got again halted before finding the user..
Filter chain halted as :authenticate_account! rendered or redirected
:O

@matteolc - I just updated to rails 4.2.
Can you please check out version 0.1.31.beta3 and let me know if it works for you?

@matteolc  - I just pushed version 0.1.31.beta6 and everything seems to be ok. Please re-open if you run into any issues.

Ok thanks!

@matteolc - make that 0.1.31.beta8 :smiley:


Update CONTRIBUTING.md with versioning info

As mentioned by @sainnhe in https://github.com/sainnhe/gruvbox-material-vscode/pull/123#issuecomment-913239430

Thank you!


Feedback for “NextJS Example”

NextJS example not working. Please check.

Same for the vite example 😕

Hi there, this is because typewind switched over to lightningcss for the intellisense previews of style declaration as that improves the speed alot as compared to using prettier. Unfortunately lightningcss is not compatible with stackblitz, which is what is being used for the examples :/ They are working on supporting it though I believe. In the meanwhile I'll switch them over to codesandbox, which should fix it

I just checked and it looks like Stackblitz has been updated to support these! The examples should be working now.


Add tx scale color to hotspot hover

change the font color to highlight tx scale reward score

moving this to #21


Reusable Payments and Client Id for PayPal Checksums

As per https://developers.paymill.com/guides/paypal/payments
The reusable payment parameters are missing from the API client:
-d "require_reusable_payment=true" \
-d "reusable_payment_description=Automatically pay invoices using this account."

As well adding the client parameter, and tests

Any idea when this will be merged? I need this for a recent project ...

Hello,
The require_reusable_payment & reusable_payment_description seem to have disappeared from the API, but the client parameter is definitely missing.
It is impossible to associate a Paypal Payment with a Client to create a Subscription otherwise.

@alexsegura   Have a look at   https://github.com/cyrrill/paymill-php
These issues have been fixed and merged there. You can use that repo while this "official" one gets fixed. Feel free to suggest (or make PRs) to that fork as well.

Thank you :+1:
I was about to make a pull request for the exact same thing.
I'm actually wondering if Paypal is really supported to pay Subscriptions.
I see whole parts of the documentation about Paypal & Subscriptions have been removed a few months ago: https://github.com/paymill/paymill-documentation/pull/45
When I see that I'm not really confident with implementing Paypal via Paymill...
Did you successfully implement Subscriptions with a Paypal Payment via Paymill?

It is supported, but it turned out that PayPal was requiring a hefty transaction history before authorizing the billing agreement option. (at least in my case) However, it appears this requirement may vary by country, so really the only way to know for sure is to try it and see if it works. For me, it worked in Test Mode perfectly, but upon switching to live mode in production, PayPal billing agreements were disallowed.


Image cropped in note view

Upper part of image is cropped.
Image:

Fixed, new layout.
Applied on next commit.


Added InfoCard component

Could you add a background color to the InfoCard story?

of course.

Please avoid using color variables directly. Instead, create separate variables in the _colors.scss file for each component. This will help us to change colors easily for each component in the future.

got it


[Sandbox] Kairos

Application contact emails
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Project Summary
Kairos is a meta-Linux distribution to build immutable edge Kubernetes.
Project Description
Kairos is an open-source project that simplifies edge, cloud, and bare-metal OS lifecycle management. It is a framework to build immutable Linux meta-distribution for edge Kubernetes, enabling users to customize or run bootable Kubernetes and OS images for edge devices with ease. Kairos offers features like immutability, security, container-based management, P2P mesh for self-coordinated deployments, and distro agnosticism. It allows users to create their own cloud on-premises, provision nodes using custom or Kairos images, and manage the entire cluster lifecycle with Kubernetes. Kairos aims to provide a unified, cloud-native approach to OS management.
With Kairos, users can effortlessly create Kubernetes clusters with Kubernetes distributions such as k3s and any Linux distribution of their choice.
Instead of the traditional upgrade process involving package managers, Kairos uses a common, generic process across all the distributions: A/B image swap using standard container images. Users can choose to use Kairos container images or build custom ones that can be based from upstream Linux distribution images.
Kairos also supports the creation of highly flexible and scalable multi-node, single clusters that can span multiple regions with k3s, providing enhanced adaptability to meet varying requirements. In order to orchestrate deployments Kairos doesn’t need an external service, k3s HA deployments are fully autonomous, removing friction of deploying Kubernetes at the edge.
Org repo URL (provide if all repos under the org are in scope of the application)
github.com/kairos-io
Project repo URL in scope of application
github.com/kairos-io/kairos
Additional repos in scope of the application
The organization with all the repositories is in scope of application
Website URL
https://kairos.io
Roadmap
https://github.com/orgs/kairos-io/projects/2
Roadmap context
In our office hours and community channels, we openly discuss the Roadmap, gathering input and prioritizing items through collaborative discussions. Our approach is strongly oriented towards fostering a thriving ecosystem, and we actively seek opportunities for interoperability with CNCF projects and other open-source initiatives. This commitment helps us better address the interests and needs of our community.
Contributing Guide
https://github.com/kairos-io/community/blob/main/CONTRIBUTING.md
Code of Conduct (CoC)
https://github.com/kairos-io/community/blob/main/CODE_OF_CONDUCT.md
Adopters
https://github.com/kairos-io/community/blob/main/ADOPTERS.md
Contributing or Sponsoring Org
https://spectrocloud.com
Maintainers file
https://github.com/kairos-io/community/blob/main/MAINTAINERS.md
IP Policy

[X] If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

[X] If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?
Kairos seeks to apply for CNCF to find a vendor-neutral home for its open-source project.
Kairos wants to establish itself as a cloud-native meta-Linux distribution tailored for Kubernetes. From its inception, Kairos was designed with cloud-native and container-first principles in mind.
By becoming a part of CNCF, Kairos aims to demonstrate its commitment to managing expansion in an open-source manner and ensure neutral governance for its growing community of users and contributors. The CNCF platform will serve as a catalyst for community adoption and encourage contributions from developers. Additionally, being a part of CNCF allows Kairos to learn from other projects and actively contribute back to the cloud-native community, aligning with the maintainer's ethos.
Benefit to the Landscape
The CNCF ecosystem would benefit from having a vendor-neutral project that applies cloud-native approaches to OS management.
It seems a natural solution that does not prefer any distribution or seek any sort of promotion or solution vendor-lock in.
The user is free to pick the OS of their choice, and just benefit from Immutability, A/B upgrades, where OS images are delivered as standard container images.
There are no similar projects to my knowledge already in the CNCF ecosystem targeting directly the OS management layer with a cloud-native, distro-agnostic first approach.
Cloud Native 'Fit'
The project is a natural fit for the CNCF TAG Runtime. There has been demand in creating a special interest group for operative systems (see: https://github.com/kairos-io/kairos/discussions/1660), and we are happy to collaborate with.
Cloud Native 'Integration'
When it comes to Kubernetes deployment, Kairos defaults to several components in the CNCF landscape:
K3s https://github.com/k3s-io/k3s
Kube-vip https://kube-vip.io/ (I think at the time of writing this it’s in the process, right?)
Cloud Native Overlap
As far as I know, there are no similar projects in the CNCF landscape
Similar projects
Outside of the CNCF landscape there are projects similar or that are tangentially related to Kairos.
To some extent, Kairos shares a similar philosophy to k3os (https://github.com/rancher/k3os) and linuxkit (https://github.com/linuxkit/linuxkit). It can be actually seen as a “mix” of the two with a truly distro-agnostic approach, with a strong focus on Kubernetes and maintenance. Kairos has roots on the Elemental toolkit project (https://github.com/rancher/elemental-toolkit), indeed it shares some of the feature sets, while it expands on security and advanced customization features.
Landscape
No, we are not listed in the Landscape
Business Product or Service to Project separation
Spectro Cloud does provide Edge solutions to run Kubernetes and uses Kairos as base to build custom OS images for Enterprise customers.
Project presentations
There is a media section on the Kairos website: https://kairos.io/docs/media/
The project was presented in various conferences:

FOSDEM 2023: https://www.youtube.com/watch?v=XD5nfMf59v4
CNCF webinar: https://www.youtube.com/watch?v=kiDQujibz2k
SCaLE: https://www.youtube.com/watch?v=GuxIiludGgA
KCD Amsterdam: https://community.cncf.io/events/details/cncf-kcd-netherlands-presents-kubernetes-community-days-amsterdam-2023/
KCD France: https://www.youtube.com/watch?v=46qcE5MC6-8&t=17s
KCD Czech: https://www.youtube.com/watch?v=dwS-dBAAuOk
Devconf.cz: https://www.youtube.com/watch?v=UmBxh5q_AF0

Project champions
No response
Additional information
We have an open issue to formally collect all the adopters and external stakeholders, however, there are companies like Spectro Cloud which is funding the project entirely and using it in production environments.
There are individuals from companies involved in the community and/or contributing directly to the project, such as Intel [1], Deutsche Bahn or Facebook.
1: We have published for instance a whitepaper with the Intel team addressing Security at the Edge in Kubernetes: https://kairos.io/blog/2023/04/18/kairos-is-now-part-of-the-secure-edge-native-architecture-by-spectro-cloud-and-intel/

@mudler It would be great if you could present Kairos at a TAG Runtime meeting.
@rajaskakodkar can help coordinate :)
cc @raravena80 @helayoty

I'd be happy to!

Part of Kairos is Linux images.  Does the project fork any Linux distribution?  Asking because that would affect licensing.

Part of Kairos is Linux images. Does the project fork any Linux distribution? Asking because that would affect licensing.

No, it does not fork any Linux Distribution, the process is explained here: https://kairos.io/docs/reference/build-from-scratch/

CNCF TAG-Runtime meeting recording: https://www.youtube.com/watch?v=kTG8FuKzGYE&t=563s

let's get a review from Tag Runtime after the presentation

Requesting an update from tag-runtime on the outcome of the meeting

Kairos builds Linux but does not include it.  As such, I wouldn't assume a licensing issue, although it might be nice to run that past the legal committee.

Revisited the notes and presentation for the project - looks good. I don't see any licensing issues here because the project code itself doesn't include code that is used by other distros (to create those specific other distros). However, I'm asking @mudler to confirm.  After confirmation, I think this is good for a vote.

Revisited the notes and presentation for the project - looks good. I don't see any licensing issues here because the project code itself doesn't include code that is used by other distros (to create those specific other distros). However, I'm asking @mudler to confirm. After confirmation, I think this is good for a vote.

Yes I can confirm, Kairos does not include code by other distros, it actually is composed of a set of standalone golang binaries (https://github.com/kairos-io/kairos-agent) and configuration files (for e.g. dracut configurations) that are packed in a container image, that we call Kairos framework. Those images can then be overlayed on top of container images from other distros, anyone can build their own images bu using any existing compatible OS as a base.

On hold for a TOC discussion around distributions more generally

I think this project is a good addition to CNCF. I have a question though. Could you clarify why it is a kubernetes distribution specific for edge? Does it slim down the Kubernetes in some way?

@cathyhongzhang indeed, k3s has a smaller footprint. One example would be its use of sqlite3 over etcd, which also helps for single node setups.
It's worth mentioning that while we focus mostly on the k3s provider (as we call them), we are open to more of them and there's already implementations for kubeadm and rke2.
cc @mudler

I think this project is a good addition to CNCF.

Thanks! we would be happy to donate Kairos to the CNCF

I have a question though. Could you clarify why it is a kubernetes distribution specific for edge? Does it slim down the Kubernetes in some way?

While Kairos can be extended and customize for various use-cases, it is tailored for Kubernetes as it comes tweaked torwards running Kubernetes workloads in an immutable environment, and it targets the edge as it comes pre-packaged for a support for various HW platforms, including for instance Raspberry, or Nvidia AGX Orin.
As also @mauromorales mentioned, Kairos abstracts away the Kubernetes distribution that can be used, and there are many supported alongside with k3s (kubeadm, rke2, microk8s, ...)
Hope it clarifies!

/vote-sandbox

/check-vote

Heya @mudler !
Welcome aboard! We're very excited to get you onboarded as a CNCF sandbox project! Here's the link to your onboarding checklist: https://github.com/cncf/toc/issues/1300
That issue is where you can communicate any questions or concerns you might have. Please don't hesitate to reach out!

Heya @mudler !
Welcome aboard! We're very excited to get you onboarded as a CNCF sandbox project! Here's the link to your onboarding checklist: cncf/toc#1300
That issue is where you can communicate any questions or concerns you might have. Please don't hesitate to reach out!

Thanks @jeefy !
I'll follow up from there for the next steps  - we are really thrilled to join the CNCF foundation!

With the onboarding issue open at: https://github.com/cncf/toc/issues/1300
I'm going to go ahead and close this out. 👍
Thanks!


在job开启failover功能后,只有在状态为running的时候宕机才真正执行失效转移

Fixes #ISSUSE_ID.
Changes proposed in this pull request:

ElasticJob will as a subproject of Apache ShardingSphere soon, we will change the package and artifactId.
I just close too old pull request for now.


[BugFix] Fix schedule tablet in recycle bin failed bug

Fixes
2023-06-21 19:42:49,307 INFO (tablet scheduler|30) [TabletScheduler.removeTabletCtx():1429] remove the tablet 2431373. because: db 2431321 not exist
2023-06-21 19:42:49,307 INFO (tablet scheduler|30) [TabletScheduler.removeTabletCtx():1429] remove the tablet 2431337. because: db 2431321 not exist
2023-06-21 19:42:49,307 INFO (tablet scheduler|30) [TabletScheduler.removeTabletCtx():1429] remove the tablet 2431363. because: db 2431321 not exist
2023-06-21 19:42:49,307 INFO (tablet scheduler|30) [TabletScheduler.removeTabletCtx():1429] remove the tablet 2431361. because: db 2431321 not exist

If the database is dropped, and moved to recycle bin, the database.writeLockAndCheckExist will return false. So we should not check the existence when schedule tablet, because the tablet in recycle bin should be scheduled too.
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[x] 3.0
[x] 2.5
[ ] 2.4

[FE PR Coverage Check]
:heart_eyes: pass : 8 / 8 (100.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/clone/TabletScheduler.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/clone/TabletSchedCtx.java
4
4
100.00%
[]

@Mergifyio backport branch-3.1

@Mergifyio backport branch-3.0

@Mergifyio backport branch-2.5


Webpack cannot handle string chunk.id

I'm submitting a bug report
Webpack version:
2.1.0-beta.27
Current behavior:
For this snippet from my plugin:
compilation.plugin("before-chunk-ids", function(chunks) {
            chunks.forEach(function(chunk) {
                if(chunk.id === null) {
                    chunk.id = chunk.name;
                }
                if(!chunk.ids) {
                    chunk.ids = [chunk.id];
                }
            });
        });

where chunk.name is a string it being ignored by webpack when using require.ensure():
require.ensure([], (require) => {
                cb(null, require('./components/test/index').default);
            },"test");

Webpack generates this:
Promise.resolve().then((function (require) {
                cb(null, __webpack_require__("fd4e592df95c42900fdf").default);
            }).bind(null, __webpack_require__)).catch(__webpack_require__.oe);

Expected/desired behavior:
Not ignored chunk and generated output:
__webpack_require__.e/ nsure /("test").then((function (require) {
                cb(null, __webpack_require__("fd4e592df95c42900fdf").default);
            }).bind(null, __webpack_require__)).catch(__webpack_require__.oe);

I believe the issue is that your require uses a (completely) constant string, so no automatic async context is created, as webpack wouldn't know where to start (or end) the context.
I'm not really familiar with require.ensure, but maybe giving the module name as the first argument would make it async.

After some investigation i think the root of problem is in these parts:
webpack/lib/dependencies/DepBlockHelpers.js
DepBlockHelpers.getDepBlockPromise = function(depBlock, outputOptions, requestShortener, name) {
	if(depBlock.chunks) {
		var chunks = depBlock.chunks.filter(function(chunk) {
			return !chunk.hasRuntime() && typeof chunk.id === "number"; // This filters out my chunk so it's being ignored after.
		});
...
};

webpack/lib/JsonpMainTemplatePlugin.js
JsonpMainTemplatePlugin.prototype.apply = function(mainTemplate) {
	mainTemplate.plugin("local-vars", function(source, chunk) {
		if(chunk.chunks.length > 0) {
			return this.asString([
				source,
				"",
				"// objects to store loaded and loading chunks",
				"var installedChunks = {",
				this.indent(
					chunk.ids.map(function(id) {
						return id + ": 0"; // Here, my chunk id comes with some illegal characters like "test.1", so id should be quoted here, otherwise it's an error.
					}).join(",\n")
				),
				"};"
			]);
		}
		return source;
	});
...

}

That's doable. We've done the same change for module ids. It just need to fixed in many places.
There are many benefits of this. i. e. id = chunkhash or id = chunkname.
Feel free to try a PR. It's basically replacing id with JSON.stringify(id) and typeof id === "number" with id !== null and so on...

@sokra hello, unfortunately I am not so familiar with webpack codebase.
Could you please clarify or point me where should replacing be done?

fixed by #4553


Please add support for pydantic-2

The package currently requires pydantic-1. Please add support for pydantic-2. FWICS it would require some code changes:
ImportError while loading conftest '/tmp/wtfis/tests/conftest.py'.
tests/conftest.py:8: in <module>
    from wtfis.models.greynoise import GreynoiseIp
wtfis/models/greynoise.py:18: in <module>
    class GreynoiseIpMap(BaseModel):
.venv/lib/python3.11/site-packages/pydantic/_internal/_model_construction.py:95: in __new__
    private_attributes = inspect_namespace(
.venv/lib/python3.11/site-packages/pydantic/_internal/_model_construction.py:279: in inspect_namespace
    raise TypeError("To define root models, use `pydantic.RootModel` rather than a field called '__root__'")
E   TypeError: To define root models, use `pydantic.RootModel` rather than a field called '__root__'

Thanks for the heads up! I will fix soon.

Latest version v0.6.2 is on pydantic 2.

Thank you!


Fix credential leak

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

This is the old track 1 which has been removed https://github.com/Azure/azure-sdk-for-net/pull/45142


[1.x] Regenerate contraband files

After contraband is updated, the contraband generated files need to be regenerated.

CI failure certainly false positive. Only change made in the PR is to update autogenerated contraband file


Add app version badge to UI

please add a version number  of the release to the user interface.
this will aid in quickly determining the deployed version of the app. this will become important as there will be 3 instances of this app deployed (dev, prod and AWS).

@PhillipsOwen closing this as completed


Persist logs

Is your feature request related to a problem? Please describe.
I'm using Organize in Docker, and I have it set in Portainer to just keep restarting so it can do file actions constantly. I noticed that the logs for Organize don't appear in Dozzle, and I can only look at the logs through Docker or Portainer. Is there any way to persist the logs somewhere so I can review what happened in case something unexpected happens?
Describe the solution you'd like
Ability to persist the logs.
Describe alternatives you've considered
I can look in Portainer but with my setup it keeps refreshing and scrolling me back up.

I now see the logs in Dozzle. Not sure how.


Gsijb/examples

Adding  /examples/nlp/ and initial py files

@jtbenson116 If you are ready with this PR, can you "re-request review" ?  The option should appear near my name at the top right...


Implemented get_contract_storage_proof

Implemented get_contract_storage_proof
Pull Request type

Please check the type of change your PR introduces:

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Testing
[ ] Other (please describe):

What is the current behavior?

Not implemented
Issue Number: #296
What is the new behavior?

Given params returns contract storage proof
Does this introduce a breaking change?

[ ] Yes
[X] No

Other information

Needs testing

@drspacemn this pr needs testing, i wasn't able to make sure given the right params it returns the right data but the implementation lgtm. I think we should close all rpc endpoints then add tests for all. wdyt ?

Also everytime it adds this no new line at end of file:

I am confused do we need a new line at EOF or not?

@drspacemn this pr needs testing, i wasn't able to make sure given the right params it returns the right data but the implementation lgtm. I think we should close all rpc endpoints then add tests for all. wdyt ?

yeah this sounds good to me.
As for new line at EOF yeah typically a good formatting practice


Support content attachments api

@gmlewis Done 👍🏼

Thank you, @gauntface!
Merging.

Fixes #1073.


TODO List

[x] Add line-search method to Newton methods
[ ] Add PJFNK method with the following options

[x] gmres

[ ] pcg

[x] bicgstab

[ ] minres

..... others
[x] Add tests with preconditioners
[ ] Add more non-linear tests with new preconditioner function args
[ ] Add equilibriate and dissect function option for using actual jacobian matrix
- https://www.mathworks.com/help/matlab/math/iterative-methods-for-linear-systems.html
- Make a NewtonKrylov method?

Add more preconditioning options and add tracking to linear residuals.

Issue with function handles for preconditioners is NOT resolved.


Set specific pytorch version for major platforms (linux, win32, darwin)

Current dependency setup failed on macOS with following error message:
 - Installing torch (2.3.1): Failed

  RuntimeError

  Unable to find installation candidates for torch (2.3.1)

  at ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/chooser.py:74 in choose_for
       70│
       71│             links.append(link)
       72│
       73│         if not links:
    →  74│             raise RuntimeError(f"Unable to find installation candidates for {package}")
       75│
       76│         # Get the best link
       77│         chosen = max(links, key=lambda link: self._sort_key(package, link))
       78│

Cannot install torch.

This is due to the fact that torch version 2.3.1 apparently does not exist for macOS currently. Therefore, we adjusted the version of torch in pyproject.yml to the range >=2.1.0, <2.3.0 for macOS.

Took me some time, but it seems the issue on my side occurred because PyTorch no longer supports macOS x86_64 (Intel Chips) from Release 2.3.0 on, as Apple stopped building Macs with Intel Chips. See https://dev-discuss.pytorch.org/t/pytorch-macos-x86-builds-deprecation-starting-january-2024/1690

Thanks a lot of your investigation! Can we define the version explicitly for x86_64, i.e. via sth like platform_machine == 'x86_64' in markers? see https://python-poetry.org/docs/dependency-specification/#using-environment-markers


Issue with ImageAI : Custom Prediction Model Training

Hello.
I'm training to do a prediction for a custom dataset of leaves.
Firstly, before training, I see something like:
input_2 (InputLayer)            (None, **224, 224,** 3)  0
Does this mean the dataset is supposed to be resized to (244,244)
Currently, it's just one type of leaf, so i set num_objects=1:
model_trainer.trainModel(num_objects=1, num_experiments=100,                           enhance_data=True, batch_size=32, show_network_summary=True)
Right now, it produces a value error:
ValueError: You are passing a target array of shape (32, 1) while using as loss categorical_crossentropy. categorical_crossentropy` expects targets to be binary matrices (1s and 0s) of shape (samples, classes). If your targets are integer classes, you can convert them to the expected format via:
from keras.utils import to_categorical
y_binary = to_categorical(y_int)

Alternatively, you can use the loss function sparse_categorical_crossentropy instead, which does expect integer targets.
`
Any ideas on how to rectify this?                                 ________________________________________________________________________________________________

Please refer to #15


json: cannot unmarshal number into Go struct field GoTrueClaims.exp of type int64

Bug report
I have created a JWT token and when I use it in supabase client with the session it throws an error. I tracked down the issue and its thrown in your library GoTrue, on setSession(accessToken: String, refreshToken: String).
The issue reported is the following:
invalid JWT: unable to parse or verify signature, json: cannot unmarshal number<PHONE_NUMBER>.2241859 into Go struct field GoTrueClaims.exp of type int64
I am creating the token like the following:
let jwtSigner = JWTSigner.hs256(key: pKey)
let exp = Date(timeIntervalSinceNow: 3600)
let header = Header(kid: SupabaseService.SupabaseAnoKey)
let claims = SupabaseClaims(userID: userID,
                                    exp:  exp)
var supJWT = JWT(header: header, claims: claims)

To Reproduce
Create a JWT token.
Add the token to the current supabase session  try await client.auth.setSession(accessToken: token, refreshToken: refreshToken)
Get the error.
invalid JWT: unable to parse or verify signature, json: cannot unmarshal number<PHONE_NUMBER>.2241859 into Go struct field GoTrueClaims.exp of type int64
Expected behavior
It should let me do the request since the token is valid.
System information

OS: [e.g. macOS, Windows]: macOS Monterey (12.5.1 (21G83))
Version of supabase-js: [e.g. 6.0.2]: Latest (points to master)
Version of Node.js: [e.g. 10.10.0]

Here's the workaround to fix it:
 let expDouble = Date(timeIntervalSinceNow: 3600)
let d: Double = expDouble.timeIntervalSince1970
let i = Int(d)
let exp = Date(timeIntervalSinceNow: Double(i))


Using browser's back button

If you click 'Back Button' on browser or (swipe your magic mouse to left) after you click any link that changes the view (eg. Click on a tag), latest view slides down and the container remains empty.

@yigitbacakoglu Thanks for the bug report. If anyone can fix this issue, please submit a PR 👍

will work on this ...

@kenny-hibino issue is caused by Turbolinks not playing nice with JS.
R

@rawdreeg fork it and fix it i will give you a vps to work in i love this project but i dont know anything about ruby i want to use it as and open source project team blog

@rawdreeg @kenny-hibino : This issue has been already reported to react-rails. Please see this github issue. and changed has been pushed into master branch of react-rails. So by updating react-rails gem  this issue will resolved. And I prefer not to remove turbolinks.

gread news :D
On 13 March 2017 at 14:17, AjayBarot<EMAIL_ADDRESS>wrote:

@rawdreeg https://github.com/rawdreeg @kenny-hibino
https://github.com/kenny-hibino : This issue has been already reported
to react-rails. Please see this github issue
https://github.com/reactjs/react-rails/issues/159. and changed has been
pushed into master branch of react-rails. So by updating react-rails gem
this issue will resolved. And I prefer not to remove turbolinks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kenny-hibino/stories/issues/35#issuecomment-286091308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEA5c9kCuQdxDNx_6g5e-VDq--xzuREPks5rlTPugaJpZM4KpTCg
.

@AjayBarot I merged your pull request! I'm still seeing the bug with the back navigation. Do you have bandwidth to investigate further?

@AjayBarot is there any news?

@strivemag : I created one pull request regarding this issue. and pull request still need to reviewed. https://github.com/kenny-hibino/stories/pull/45


Skip evicting when no evictable pod found on node

In lowNodeUtilization, only try to evict pods when we found evictable pods on node, otherwise, just skip and try next node.

/retest

/ok-to-test

Looks good to me, @lixiang233 could you just add a simple test case?

done. @damemi

/lgtm


Release 1.0 Gem

It looks like we tagged v1.0 a week ago (exciting!) - are there are blockers to releasing it to rubygems?  Do you need any support here @bkroeker ?
Unfortunately, we have engines / gemspecs that are consuming temporal tables, which cannot consume git urls, so our team needs a released gem.

Hmm, I had run the command to release it on rubygems, but apparently it didn't work.  I'll need to investigate.

Aha, apparently the command failed because rubygems requires MFA these days and I missed the error message.  Should be good to go now.


View are only working in 4.7 inch devices. Support other screen size.

It seems difficult to use auto layout. I'm having same trouble with my application.

@tiepviet  it's easy to work with auto layout. I just didn't have time to fix this bug. Hopefully this week I will fix.
Thanks

Thank you!

I do need this too

use snapkit.
10,000 github stars can't be wrong.
stars https://github.com/SnapKit/SnapKit
eg.
to make a view fill the screen.
self.collectionView.snp.remakeConstraints { (make) -> Void in
                make.left.top.right.height.equalToSuperview()
 }

How you fixed the issue?


a way to make stream-bytes produce multiple http chunks

Currently (unless I'm missing something) the chunk_size parameter on stream-bytes controls the block size used to write the data to the socket. But the entire data is still sent as one HTTP chunk i.e. there is only one total size followed by the data.
Would it be reasonable to have chunk_size control the size of the HTTP chunks? Or if not, maybe another parameter to control that?
I don't know Python so I'm not sure how feasible that is.
Thanks for a great tool.

The chunk_size parameter on stream-bytes actually works as it should. The problem with httpbin.org is that they use nginx as a reverse proxy, and apparently it merges all chunks into a single one. I'd suggest turning off proxy_buffering in nginx or exposing gunicorn directly.

@andreasf considering httpbin is hosted on heroku (last I checked) I doubt that's plausible.

It is not hosted on Heroku, but I will look into making this change with the team. It has a low probability of happening quickly though.

Thanks everyone!


Fix for issue 1

This refactors out the loading of a particular file, so that the same routine can be used whether we need to load multiple files or just one (as is the case in issue #1)

If your happy I am happy. This all looks good to me.


Allow passing in manifest data directly

Context
Currently, user has to provide a URL which contains the manifest. This creates friction and slows down development cycles, leading to weird workaround such as reusing an already existing manifest etc.
Suggestion

Allow url, iconUrl to be optional (currently the work with empty strings but we can make it more explicity)
Allow passing in metadata props to the bridge directly (in addition to the URL as being done today)

I'd suggest showing a warning if data was passed directly, but since there's no verification being done (i.e. you could pass in any existing manifest belonging to a different app) at the moment I'm not sure that's required.

Good idea.
Providing a URL to the manifest even leads to new issues as described in this thread https://github.com/ton-connect/sdk/issues/12
This can be avoided by simplifying the configuration.

@shaharyakir yes, agree
@siandreev it will also solve this step aswell https://github.com/ton-connect/sdk/issues/12
Thank you for well argument @shaharyakir

Hi @shaharyakir , could you provide more details on the frictions you are facing?
For local development you can place the manifest on the local server and run you app on LAN.
(for instance, if you use Vite, run a webserver with the --host option vite --host)

@inartin I agree with you that SDK should be simple. Mentioned libraries do not provide a connection to the wallet, and you do not need to change your manifest, you should upload it once. For me it seems like describing the manifest in json file, or TonConnect constructor are the same. However, there is an important advantage of the json manifest file -- wallets can show a dApp meta information even if the dApp is not connected yet

I agree that the JSON file has an advantage, but I'm not suggesting to replace it, only to provide an option to pass information through the constructor (or even omit metadata altogether) for prototyping purposes.
@siandreev I think that even vite --host creates friction we can avoid. It'd just be a better DX without mandating it.
See for example Step 4 in this tutorial -> https://ton-community.github.io/tutorials/03-client/
where we provide a temporary published manifest. Ideally, I'd like to be able to avoid that and add the manifest later.


v6 update: gRPC Diagnostics

Internal Review
Updates to reflect .NET 6, project templates and top level statements for:

Content
Server and Client diagnostic app samples

Thanks @parid0kht for your contribution to help improve this content.

@Rick-Anderson, thanks for the great suggestions!  Just as a reminder, I like to merge my own PR's and delete my own branches if I can.

I like to merge my own PR's

Why?

Then they are merged and the branches deleted when I am absolutely sure I am ready to do so.  Also kind of a let down to not do it myself honestly.  As a rule I don't merge and delete yours figuring you like to do your own as well, and it is not my PR merge and my branch to delete, but maybe I'm the only one who cares about that.


Multiple pnp:ComposedLook element support

Version 201705 of the schema only allows for a single <pnp:ComposedLook />. Part of my provisioning process involves creating several composed looks per site (same font/background, different .spcolor) and defaulting to the first one.
Proposing that composed looks get their own group, just like everything else:
<pnp:ComposedLooks>
  <pnp:ComposedLook Name="New Default" ColorFile="..." FontFile="..." BackgroundFile="..." Apply="true" />
  <pnp:ComposedLook Name="Other Option" ColorFile="..." FontFile="..." BackgroundFile="..." Apply="false" />
</pnp:ComposedLooks>

For anyone wanting to implement I had success doing this:
<pnp:ListInstance Title="Composed Looks" TemplateType="124" Url="_catalogs/design">
  <pnp:DataRows UpdateBehavior="Overwrite">
    <pnp:DataRow>
      <pnp:DataValue FieldName="Name">My Look</pnp:DataValue>
      <pnp:DataValue FieldName="ThemeUrl">{themecatalog}/15/mylook.spcolor</pnp:DataValue>
      <pnp:DataValue FieldName="DisplayOrder">5</pnp:DataValue>
      <pnp:DataValue FieldName="MasterPageUrl">{masterpagecatalog}/seattle.master</pnp:DataValue>
    </pnp:DataRow>
  </pnp:DataRow>
</pnp:ListInstance>

Hi, Thanks for your feedback. Good to see that you solved it. I will close this item.

You appear to have a pnp:DataRows tag closed with a </pnp:DataRow> tag?

Thank you for the suggested work around!


Added valid input message in all 2-D Shapes Inscribed calculator

Related Issue

Info about Issue or bug

fix #2976
Describe the changes you've made
Added valid input message in all 2-D Shapes Inscribed calculator
Checklist:

[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[x] My changes generate no new warnings.

Screenshots

@sairish2001 Resolved conflicts :)


Update nscplugin, sbt-scala-native, ... to 0.4.16

About this PR
📦 Updates

org.scala-native:nscplugin
org.scala-native:sbt-scala-native
org.scala-native:scala3lib
org.scala-native:scalalib
org.scala-native:test-interface

from 0.4.14 to 0.4.16
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-native" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scala-native" }
}]

labels: library-update, early-semver-minor, semver-spec-patch, commit-count:1

Superseded by #120.


Add HourlyTopicPartitionRecordGrouper

Hi.
I was searching for open source plugin for kafka connect that allows to send data to GC. This one came first in my google search.
I tried it and it seems to works nice, but does not allow partitioning by time. What we need is following folder structure
{{year}}/{{month}}/{{day}}/{{hour}}
So I tried to extend your code to support that. I am quite far from Java world, so it can definitely be improved.
However, what do u think about merging it into upstream?

Closing in favor of #34


Unpackaging Undefined Byte Array in Structure Editor results in undefined1 bytes, not undefined bytes

Describe the bug
In many programs, when I use the Structure Editor, and create an undefined[x] component and then unpackage it, it creates x elements of the DataType undefined. This works well for me, as I can then take some undefined byte, redefine it as a larger DataType, and it will cover the bytes after it, avoiding shifting anything after it down. However, in many programs now, this "Unpackage Component" operation results in bytes of type undefined1, which do not have the same properties. Is there a setting I changed somewhere accidentally that causes this?
To Reproduce
Steps to reproduce the behavior:

Open the Structure Editor
Create a component of DataType undefined[x] (where x is some integer)
Right-click the component and select "Unpackage Component"
Observe that the component is unpackaged into x bytes of DataType undefined1.

Expected behavior
The component is unpackaged into x bytes of DataType undefined.
Screenshots
N/A
Attachments
N/A
Environment (please complete the following information):

OS: macOS 14.5
Java Version: 22.0.1
Ghidra Version: 11.0.3
Ghidra Origin: GitHub Releases ZIP

Additional context
This has been happening to me more often than it doesn't, and I apologize for any lack of details. For some programs, it works as expected, but for many it doesn't. This also seems to affect the "Insert Undefined Byte" feature, as it will insert an undefined1 byte instead of an undefined byte in programs where this bug also occurs.

This is expected behavior for packed structures where use of undefined datatype is not permitted.
An undefined[*] datatype is considered a "defined" component type while the undefined type is not and is only permitted within non-packed structures to represent padding or unspecified components.   The undefined1 datatype is used in the place of undefined for packed structures when components are explicitly specified but the datatype is unknown.  You probably should avoid using undefined[*] and use undefined1[*] in its place when you are reserving defined regions.
You will notice that when transitioning from a non-packed to packed structure all undefined components are removed.

You probably should avoid using undefined[] and use undefined1[] in its place when you are reserving defined regions.
I am curious as to why this is recommended, as undefined1 bytes do not have the space-preserving behavior I described in my original issue.

Sorry, all I can say is that the undefined datatype serves a special purpose and an undefined-array does not.  Within a packed structure only "defined" components are supported and use of the "undefined" datatype is not.  The behavior of a packed and non-packed structure is quite different within the editor.  A packed structure is only intended to be used when the complete definition of the structure is known.

Good to know. Thank you!


GDScript constant folding for method call on const value base, implemented

Fix: #41673
extends Node2D

const KEY = "p@$$w0rD".hash()
const VAL_42 = [0, [1, 42]].back().max()

func _ready():
	print(KEY)
	print(VAL_42)

output
1842799034
42

This isn't a good solution and maybe a bit dangerous even. I think this might be okay for const methods but not for all of them. This also not checking if the arguments are constants themselves.
It definitely warrants some discussion about expected behavior.

This will need to be addressed, but I will set the milestone to 4.x, as this isn't a pressing issue, especially if there's a discussion to be had about the subject.


graphics: Use float32 instead of float64 at GeoM/ColorM APIs

Not determined yet...

Hello,
I was wondering why you decided to stick with float64. Many games don't really require full 64-bit precision, and it can be a hassle to convert from float32 to float64.
Since Ebiten converts float64 to float32 under the hood anyway, wouldn't it be more performant to simply accept float32? This would also be consistent with the vector.Path API, which only accepts float32.

Hello,
I was wondering why you decided to stick with float64. Many games don't really require full 64-bit precision, and it can be a hassle to convert from float32 to float64.
Since Ebiten converts float64 to float32 under the hood anyway, wouldn't it be more performant to simply accept float32? This would also be consistent with the vector.Path API, which only accepts float32.

While vector.Path's float32 values are passed to the low layer directly, GeoM's values can be calculated (e.g., multiplying) and can include numerical errors during the calculation.

While vector.Path's float32 values are passed to the low layer directly, GeoM's values can be calculated (e.g., multiplying) and can include numerical errors during the calculation.


bupstash: 0.10.0 -> 0.10.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/andrewchambers/bupstash/releases.
meta.description for bupstash is: "Easy and efficient encrypted backups"
meta.homepage for bupstash is: "https://bupstash.io"
meta.changelog for bupstash is: ""
Updates performed

Rust version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

/nix/store/9pvwbpvl49zn7k1ssk6wx2vikrfzrdxs-bupstash-0.10.2/bin/bupstash passed the binary check.

1 of 1 passed binary check by having a zero exit code.

1 of 1 passed binary check by having the new version present in output.

found 0.10.2 with grep in /nix/store/9pvwbpvl49zn7k1ssk6wx2vikrfzrdxs-bupstash-0.10.2

found 0.10.2 in filename of file in /nix/store/9pvwbpvl49zn7k1ssk6wx2vikrfzrdxs-bupstash-0.10.2

directory tree listing: https://gist.github.com/0abdff0489d080174a2a79bb7bee565a

du listing: https://gist.github.com/6ebc7f76afce0ec1041347d2dcc9b0b7

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
bupstash

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/9pvwbpvl49zn7k1ssk6wx2vikrfzrdxs-bupstash-0.10.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A bupstash https://github.com/r-ryantm/nixpkgs/archive/128d3d1ad79415cc8eacef62a424012f29150a2b.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/9pvwbpvl49zn7k1ssk6wx2vikrfzrdxs-bupstash-0.10.2
ls -la /nix/store/9pvwbpvl49zn7k1ssk6wx2vikrfzrdxs-bupstash-0.10.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

bupstash

Maintainer pings
cc @andrewchambers for testing.

lgtm


Where is [[PLAN]] for generation?

Hi, Jundong,
Appreciate your contribution to the community very much!
During checking your prompt, I didn't find where to fill the generated plan in your prompts:
For example, in AR-LSAT, the code here requires filling the generated plan into [[PLAN]], however, I didn't find the slot in your prompt (I only find [[CONTEXT]]).
Also, I think the in-context examples in solver.txt also do not include the plan part.
Could you help with a check if it is missing?
Thank you very much!
Best,
Fangkai

Hi Fangkai,
Thank you for raising the issue. I have already fixed the typo and it should work properly now.

Hi, thank you for your help very much!
I have checked the updated prompt, however, I found that there is still no [[PLAN]] in the in-context examples part, where the context part directly follows the instruction. Is this as expected?

Hi Fang Kai,
Yes it is expected. In the in-context examples, there is no [[PLAN]] as it would be sufficient to guide the model on solving the problem. The plan will be appended when the problem is being solved.


Support for media-queries-like options setting

Hello!
Do you think it is necessary/possible to implement width-dependent configuration?
For example, I have layout grid with 2 columns for wide screens and I set each even list item some delay to make left element in a row appear first and right element in a row appear next. However, my layout changes to 1 column, and there becomes no need to delay even items in my list.
Another words, it would be great to change behaviour
data-sr='enter left, move 24px, scale up 0%, delay 0.3s'
into
data-sr='enter left, move 24px, scale up 0%'
dynamically on page resize.

I think that makes perfect sense, especially with the use case you described: using delay only under certain conditions. I’m going to keep this in mind for future releases.
As far as achieving this right now in your current project, you’ve caught me at an odd time. You’re probably using v2.3, but the heavily refactored Scroll Reveal 3 drops soon… and it's primary feature is the new JavaScript API.
It's this API that I think will allow for something like this:

detect width (let's say 1000px)
configure elements (config with delay)
add listener for width changes
initialize ScrollReveal
on width change (let's say now 400px)
re-configure elements (config without delay)
initialize ScrollReveal

Now, I would say just grab the release candidate and start experimenting, but—this is why I say you’ve caught me an odd time—the current release candidate still has bugs when the same element is configured more than once. This is literally what I am in the middle of fixing in my development branch.
So...
This should be possible with an event handler bound to the window resize event, that checks the current width and uses the new JS API to set the appropriate configuration.
I just need a little more time to work on V3.

Hey @pawlikdoc, the latest release candidate now properly supports multiple reveal() calls on the same element set.
Are you still interested in getting this working?

Ok thanks. I don't know if i will use it in the future, great improvement though.


Multiple path conflict with secondary source in Gemfile

When :path option is used in multiple gems, #3414 issue still exists.
I added a failing spec here: https://github.com/eagletmt/bundler/commit/3ad56c8c34a1493b7ca9d91350ad05a8eaf6f0cd .

Thanks!

I can confirm that the following fails for me on a fresh RVM Ruby 1.9.3 install for an older project:
source 'https://rubygems.org'

gem 'rails'

source 'internal gem server' do
  gem 'internal_gem'
  gem 'in_development', path: '~/Desktop/in_development'
end

Moving the gem with :path option outside the source block does not resolve the issue either, it has to be commented out entirely.
On bundler 1.8.3 that was just released.

@eagletmt / @macfanatic would you be able to test out a fix?
https://github.com/TimMoore/bundler/tree/revert-local-rubygems
Clone that repo, check out the revert-local-rubygems branch, then run rake install to install the patched version.

I will work since it's virtually equivalent to my #3418 .

Bundler 1.8.4 has been released with this fix.


Enable QoS msg caching with SQLite cause service halting

Enable SQLite in nanomq.conf, then pressure broker with certain amount of QoS 1/2 messages, for a certain period then broker cannot accept new connections. This is probably due to I/O latency caused by SQLite. Will investigate it first.
Suggest user do not enable QoS message caching now, only use another SQLite option in nanomq_bridge.conf, which caches data when bridge connection offline, this is most commonly used feature.

Issue fixed by enabling a self-developed caching layer


TypeError: Cannot read properties of undefined (reading 'toggle')

Describe the bug
A clear and concise description of what the bug is.
revue-draggable.es.js:3 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'toggle')
at revue-draggable.es.js:3:7735
at Array.forEach ()
at b (revue-draggable.es.js:3:7679)
at revue-draggable.es.js:3:8602
at runtime-core.esm-bundler.js:2874:88
at callWithErrorHandling (runtime-core.esm-bundler.js:193:18)
at callWithAsyncErrorHandling (runtime-core.esm-bundler.js:201:17)
at hook.__weh.hook.__weh (runtime-core.esm-bundler.js:2854:19)
at flushPostFlushCbs (runtime-core.esm-bundler.js:369:40)
at flushJobs (runtime-core.esm-bundler.js:407:5)
To Reproduce
Steps to reproduce the behavior:

Install on latest Vue 3 + TS and Vite
See error

It works with
<template>

      <HelloWorld msg="Retro Overlay" />

</template>

but not
<Draggable>

I use a wrapper

  </Draggable>

error TS7016: Could not find a declaration file for module '@braks/revue-draggable'. <EMAIL_ADDRESS>implicitly has an 'any' type.
Probably something related to this, I get dist only but no index.js as other dependencies

Did you import the Draggable component?
The docs are built with Vite + Vue 3 and they use <Draggable> mostly and afaik the docs examples all still work 🤔
Do you, by chance, have a sandbox reproduction of this?

@brenoepics ping - any chance for a repro on this?

@brenoepics ping - otherwise I'll have to close the issue for now until I can get a repro for this :/

closed due to insufficient info, please re-open with a repro if necessary.


Address in Address Widget changable after a decline

When configured for synchronous authorization and simulating a soft decline (invalidPaymentMethod), the address is changable via the widget, which leads to an error.
This should not be possible, e.g. rendering the readonly version instead would prevent this from happening.
Daniel

was already fixed in #157
Seems to be a regression


feat: make qps observation duration configurable

The query traffic pattern can be bursty for some subgraphs. Indexers may desire to set a longer time period in that case.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 5 (0.0%)  changed or added relevant lines in 1 file are covered.
4 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.2%) to 12.862%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

autoagora/price_multiplier.py
0
5
0.0%

Files with Coverage Reduction
New Missed Lines
%

autoagora/main.py
1
0%

autoagora/price_multiplier.py
3
0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
40

Relevant Lines:
311

💛  - Coveralls

Thanks!
Some context around that value:

The optimizer for the price PDF has a natural tendency to average, because the optimizer Adam will take small steps towards the optimum, and also implements momentum.
Pretty cool interactive vizualization here, but in our case imagine that the function in the background changes a bit at every step because of variations in traffic, etc.
We observed that with the current 60s sampling, we get a satisfactory convergence in ~12hrs. Increasing that value too much will slow the process down further. Note that we are currently working on improving the convergence time.
We don't expect AA to be able to optimize on a subgraph that doesn't get a steady stream of queries (at least .1 qps). That's because we can't get enough market information within the characteristic time of fluctuations of the market price itself. Because we implemented a slight "pullback force" on the PDF mean and variance, we expect the price model to stay close to its initial position (fallback).

Here is an experiment with sparse queries. Note that "time" is in training steps, which is about 1m30s per step in production AA. We can indeed observe that the convergence (in PDF stddev terms) is not great, but the mean does stay somewhat close to optimal.
https://user-images.githubusercontent.com/33075224/200449224-52b32d4a-6891-403e-b2f6-ab6858f7b6d0.mp4
Again but with more reasonable noise (as we'd expect from longer sampling times:
https://user-images.githubusercontent.com/33075224/200450772-b41e5e96-a049-4c72-a5d5-fd4f22832f7e.mp4
Overall I'm fine with giving easy access to that parameter.
The only problem I can see here is that the setting is global to all subgraphs. Probably a better solution could be to sample over a constant number of queries (with a max time limit), instead of a constant amount of time.

PS: The initial values in the simulation videos above are not the same as in production AA.

Thanks for the context. Indeed the function performs well for most subgraphs despite the noise, but I felt the need to increase this value to cover some additional cases.  In a general, would be a good idea to allow the tuning of the parameters for specific subgraphs or do you want avoid any sort of specific intervention?
FYI, this is the pattern I am trying to optimize for. I realize this subgraph would be very tricky to train one because of its large peaks and very low and long valleys:

In this case, with a 60 seconds average (after 60 seconds gateway wait), the multipler oscillates by 10^5 and never converges


Allow people who have more than one ssh key to select one

Currently the installer just selects the first key available (probably based on order of creation).

All keys are used as of e24040587455e34e9586a0c4560d0d913c879b0e


Total Analytics for a work

We should be able to see FileSet total downloads and Fileset total views at the work level.
Does this need to be a graph or just a number?

High level graph in ScholarSphere
And a Download Button to get the graph data in csv

session_id, work_id, file_id, action

From the work page I can download a report of all activity on the work, including work views, files set views, and file set downloads.

dependent on #625

"work downloads" do not exist yet, that's why this is dependent on #625


Error in build when he generate the PTHash

he crash and make this error:
error[E0658]: use of unstable library feature 'int_log'
  --> /home/coco/.cargo/registry/src/github.com-1ecc6299db9ec823/quickphf_codegen-0.1.0/src/phf.rs:34:16
   |
34 |     let lg = n.ilog2() as f64;
   |                ^^^^^
   |
   = note: see issue #70887 <https://github.com/rust-lang/rust/issues/70887> for more information

For more information about this error, try `rustc --explain E0658`.
error: could not compile `quickphf_codegen` due to previous error

the rust version rustc 1.66.1 (90743e729 2023-01-10) (built from a source tarball)

Yeah, it looks like ilog2 was only added in 1.67. I just published a new version that should fix this, so you can either use a newer Rust version than 1.66, or use quickphf_codegen 0.1.1 which should work with the correct minimum rust version of 1.56.


Make it compatible with "laravel/scout": "^9.1",

The new default laravel/scout installation is using "^9.1" version. But Explorer still requiring ^8.0.
When you guys upgrade?

+1
I was wondering about the same

Feel free to open a PR 😄
Do you know if there are any breaking changes?

No but I'm guessing it's about adding new driver support for MeiliSearch

Ok! In that case adding 9 to the requirements  should be fine.

If anyone could verify that dev-master is working for them with the latest Scout I will tag a new release.

Yes, dev-master is working. Thanks

Tagged 2.1.0!


[GOBBLIN-332] Fetching Hive tokens in TokenUtils

Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[x] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-332] My Gobblin PR"

https://issues.apache.org/jira/browse/GOBBLIN-332

Description

[x] Here are some details about my PR, including screenshots (if applicable):

Tests

[x] My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

[ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

@ibuenros  Can you help review ? Thanks.

+1


I have a small error and I think it is related to ts since all the data is correct.

🔎 Search Terms
Type '"packTypes"' cannot be used to index type 'IProdSchemas[T]["unitTypes"][Y]'
🕗 Version & Regression Information

This changed between versions ______ and _______
This changed in commit or PR _______
This is the behavior in every version I tried, and I reviewed the FAQ for entries about _________
I was unable to test this on prior versions because _______

⏯ Playground Link
https://www.typescriptlang.org/play/?#code/JYOwLgpgTgZghgYwgAgJIAUoHsAmBlBACwgFs4BnZAbwChl7kA5AVzCmDgBtOAuauhoOYhgYACoBPAA4RyfWoMUMWbDtwD6AIywAPeQKWHkOTQGE4kAOZYoEvuVUhLBo4qmIA1pJlz+r-8ggrOxcnFq66uQQYHxi6J7esqiQJHjRANoAui4BikGqoeE66gCMABx8GAnSSSlZObkM+SEa2sUlACyV8QheNeTJpPWNhs1qYW3qAExT3dU+gyTDI3nB40WlAAxzvYkDddkrqwWtEQCcO30LBw3+AL63ig9Gzwyv9ADScFAkEgAitn0RmEoj2QICXx+-zw6Eini2Hks4JG7l2-WRR2QHm+vxw5Ck6hAEAA7nDeuocFhLOQEUi0D0rrUhodMfRsVC8QTyPCSptEepOMASB4bCRLntFstWezcfiyR5aeopNgAG5C4Di-qSlmswXC0VK1VCiCa67Mx4vC1vK2fHHQ2GTEgQTgK+C+BQrVGM9021wAIkdztdFHUmwAdABmUNhjqIv2mplLHWYgMRJ0u9Ru6NRkphgCs-KmYb5lnj9PmialKcDGaz4ajUTAZbiFf2pDSYCruXerh7Rkhvz+MPlpQLdI9KNbGKOMoknJHJTHAqFIp+CbbSd9Rln8+55MX-OVWDVJA15bRZs3rPoetXJENx+N6+1W+t3d9A-tC9j49f9C9YJ+NeO5ynuCqdPyt6is+NzXsgUE-A+J4mue3ovkcfaGJhSifkOsJgaUZSItOAQAeiQHSnau48kRlgUhA8DMJwMSoRKsErNhTyPD2DwPDQoCQLAiAoFUF6JhRyDEhAwCWIQLEOOwTguMwURQAyiR-LICBQPYjjOEIIjiDUaTfEQABCdjIApoD6QwZEyCZUDmZZE6KCQuAodZSmCD2MhQAAqoZfBBCQmjQC4CBYCQTrgLpim2fQKnQOpNRxTZNB8WANTIC2Ykbh2yAALzIAA8qeYAADyiWhKQADTIAA5H5gWiA1AB8yAAGQSc1QXIAAglAUBwBIVXoNALWdpkbUuN8YDAAgnCyGlTj1HxAnQPASD0hNhkSVJMlycFzCheFghzQtS0rQlyC9aIx2nVAGU0DQWUyDlLWJBVYjIBAOiQCAeJYhAEhYDA9LYPgRCkBQHXFR4INgxDuAEMQZDkOkYiZOkDUgkZPgNZkADcL1vSguWMt9v3-RAgOUAjoPgxgkOozD5D1QAmtTANA2In01N9bVw8gLgM0jzMo9D6OY9juOGXshPpBzsv2bIhMvRtQnbagADi0STQA6tJsmVeYVhYBI3O00DYtM5gktoxQ9WTYkVt0x9IhfWbEDWBIbX1Sl71-TzlAU17Fg+xbzvyzUQsSQgEfWICyDe77JMGaCqXIC7NTp3ZFZ8IHEAk3cQA
💻 Code
I'm applying a link to the playground above
🙁 Actual behavior
In our TPackType type, we cannot access the packTypes field
🙂 Expected behavior
In the TPackType type, we need to access this field as it is in each of our nested objects
Additional information about the issue
My team and I have tried many options and have not found an actual error.

Simpler repro to demonstrate what's happening
type Sample = {
    top: {
        fixed: {
            middle: {
                bottom: {
                    foo: string;
                }
            }
        }
    }
}
type MiddleKey<K extends keyof Sample> = keyof Sample[K]['fixed'];
type Deferred<K1 extends keyof Sample, K2 extends MiddleKey<K1>> = keyof Sample[K1]["fixed"][K2]["bottom"];

Deferred indexed access types don't enumerate all their possible inhabitants, so can't be indexed that way. You'd need to use someting like this:
type Sample = {
    top: {
        fixed: {
            middle: {
                bottom: {
                    foo: string;
                }
            }
        }
    }
}
type MiddleKey<K extends keyof Sample> = keyof Sample[K]['fixed'];
type ReadProp<T, K extends keyof any> = T extends Record<K, infer U> ? U : never;
type Deferred<K1 extends keyof Sample, K2 extends MiddleKey<K1>> = keyof ReadProp<Sample[K1]["fixed"][K2], "bottom">;
type S = Deferred<"top", "middle">;

Search Terms: Type '"packTypes"' cannot be used to index type 'IProdSchemas[T]["unitTypes"][Y]'

Aside: Why are you searching for code that would only ever appear in your codebase?

This can be reworked slightly to be more friendly to the compiler: Playground

Условия поиска: тип «packTypes» нельзя использовать для индексации типа «IProdSchemas[T]["unitTypes"][Y]»

Замечание: почему вы ищете код, который может появиться только в вашей кодовой базе?

I did not understand the essence of the question

I need this type to narrow down the IGetUnitWeight type


How does drf-yasg integration work when using JWT?

According to this piece of the Simple JWT documentation, the following code needs to be added. But I do not understand how this works for this library. Does anyone know how to fix this integration? Or does anyone know any documentation that can help me further?

I don't understand your question.


Feature/update GitHub action

Summary

Consider the GitHub Action as the only CI tools (deprecate travis setting)
Remove the pipenv setting because this repo aim to support multiple python version and pipenv might not a great choice (ref: Support multiple python_version · Issue #1050 · pypa/pipenv)

All CI failed due to the pytest issue

Awesome 👍


[TVMC] tune: Check if FILE exists (#10608)

Currently when a non-existing FILE is passed to 'tvmc tune' it throws
a traceback because a FileNotFoundError exception is not handled. Since
there is no need for such abrupt exit, and the trace can also confuse
users, this commit fixes it by checking if FILE indeed exists, kindly
informing the user about the non-existing FILE before exiting.
Signed-off-by: Gustavo Romero<EMAIL_ADDRESS>Thanks for contributing to TVM!   Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

@gromero please fix the merge conflict
@Mousius please take another look at this one