WIP: example solution for part of Epic 13525; invariant checking for implicit LP changes

Which issue does this PR close?
For discussion re: #13525
Rationale for this change
There are many implicit changes (not explicit API changes) which can result in unintended consequences when upgrading DF. This WIP is an example of handling one of these implicit changes: specifically, implicit LP changes between analyzer and optimizer passes which can result in side effects downstream (which is harder to debug).
It attempts to handle this issue by:

define the invariants
check the invariants for extensible interfaces (which may be user defined)
throw the error closer to the problem (rather than weird behavior later)

The example here addresses only the invariants for all analyzer and optimizer rules. The individual rules have their own invariants which should be checked there (altho perhaps we need to make this more explicit in the rule API? e.g. AnalyzerRule::check_invariants()).
What changes are included in this PR?
It provides a rough example of:

making the explicit contract (of what can and cannot change) in the analyzer vs optimizer passes
early failure after the user-defined modular component (e.g. AnalyzerRule or OptimizerRule)
it does two example checks:

for the one of the invariants (mentioned in the docs here and brought up in the Epic discussion)
for a union schema invariant which InfluxDB has hit a few times ourselves

Are these changes tested?
N/A. Is a WIP.
Altho the current checks do cause some existing tests to fail.
Are there any user-facing changes?
No.
Altho it hopefully surfaces errors sooner for user-defined analyzer and optimizer rules.

I love the idea of having additional testing like this! It's definitely out of my area of expertise but a few questions:

Should invariants be pluggable? For example, Influxdb might have additional invariants that another derivative of DF does not (or vice versa)?
Is the invariant testing overhead acceptable for production or should it there be the capability to disable it (enabled by default of course!) ?
Is there an equivalent set of invariants elsewhere in the core besides the LP (maybe physical plan? Just guessing here as I'm definitely no expert here) where this approach could be duplicated ?

I like the verification. The only concern is the overhead of doing it, especially for large plans. Every optimizer pass gets a new verification of all plan nodes. i hope one day we go towards more iterative optimization and then we should be able to verify plan invariants locally. Which would guarantee that the verification overhead is linearly proportional to number of modifications applied to the plan. This would require eg that get_type is a constant operation (https://github.com/apache/datafusion/issues/12604), since plan verification should also include that the types do match.

I think it would also be great if we could consider moving the invariant check directly into LogicalPlan -- for exmaple
LogicalPlan::check_invariants
This would make this function easier to discover and for others to understand what the invariants are.

The updates focus on dividing out LP invariants, vs analyzer pass checks, vs optimizer pass checks (including invariants listed as the analyzer's responsibility). I added in refernces links to invariants in the docs, in an attempt to delineate an invariant vs a valid plan. (But I'm muddy on this boundary.)
I'll make it configurable after I get feedback of whether this^^. 🙏🏼

Benchmarks in progress...

(existing/main check): each optimizer pass only does a check that the schema is not changed

We could also reduce the time on this one by making it compile time with Arc::ptr_eq, but that means (a) a stricter contract (metadata & nullability cannot change) and (b) it requires code changes to chase down the current pointer_eq  failures. 🤔

I propose that we add another full InvariantLevel::Executable between each optimization pass, that only runs in debug mode (since that will be a performance impact as @findepi mentioned).

Perhaps we can do this as a follow on PR

I can also make some of these check listed above into debug only, if the current performance change is unacceptable.

Yes, I think this is important for two reasons:

I think we can't introduce a 25% planning performance regression
Adidng a check to help development that will generate errors in plans that used to run to completion will likely be seen as a regression  for users

I am checking out the rest of this PR now as well

We no longer have any per-optimizer-pass validation in release mode. Instead, the checks run before and after ALL optimizer passes are completed.
When running in debug mode, the per-optimizer-pass will still occur. This will allow users of datafusion to more readily debug when their DF upgrades are failing. 🤞🏼
Here are the benchmarks. There is no change:

$ critcmp main  check_lp_invariants
group                                         check_lp_invariants                    main
-----                                         -------------------                    ----
logical_aggregate_with_join                   1.00  1001.9±10.39µs        ? ?/sec    1.01  1007.8±10.29µs        ? ?/sec
logical_select_all_from_1000                  1.00      4.8±0.04ms        ? ?/sec    1.00      4.8±0.04ms        ? ?/sec
logical_select_one_from_700                   1.01    753.9±7.37µs        ? ?/sec    1.00    743.5±5.70µs        ? ?/sec
logical_trivial_join_high_numbered_columns    1.00    716.8±5.68µs        ? ?/sec    1.00    718.2±4.64µs        ? ?/sec
logical_trivial_join_low_numbered_columns     1.00    699.3±5.84µs        ? ?/sec    1.00    699.6±3.40µs        ? ?/sec
physical_intersection                         1.07   1850.8±9.91µs        ? ?/sec    1.00  1726.6±15.43µs        ? ?/sec
physical_join_consider_sort                   1.17      2.9±0.01ms        ? ?/sec    1.00      2.5±0.01ms        ? ?/sec
physical_join_distinct                        1.00    688.2±5.06µs        ? ?/sec    1.00    689.2±7.02µs        ? ?/sec
physical_many_self_joins                      1.21     16.9±0.12ms        ? ?/sec    1.00     13.9±0.07ms        ? ?/sec
physical_plan_clickbench_all                  1.02    185.0±0.71ms        ? ?/sec    1.00    181.5±0.36ms        ? ?/sec
physical_plan_clickbench_q1                   1.01      2.6±0.02ms        ? ?/sec    1.00      2.6±0.01ms        ? ?/sec
physical_plan_clickbench_q10                  1.01      3.5±0.01ms        ? ?/sec    1.00      3.5±0.01ms        ? ?/sec
physical_plan_clickbench_q11                  1.02      3.6±0.02ms        ? ?/sec    1.00      3.5±0.01ms        ? ?/sec
physical_plan_clickbench_q12                  1.01      3.7±0.01ms        ? ?/sec    1.00      3.7±0.22ms        ? ?/sec
physical_plan_clickbench_q13                  1.02      3.4±0.01ms        ? ?/sec    1.00      3.3±0.01ms        ? ?/sec
physical_plan_clickbench_q14                  1.01      3.6±0.01ms        ? ?/sec    1.00      3.5±0.01ms        ? ?/sec
physical_plan_clickbench_q15                  1.02      3.5±0.01ms        ? ?/sec    1.00      3.4±0.01ms        ? ?/sec
physical_plan_clickbench_q16                  1.01      3.0±0.02ms        ? ?/sec    1.00      2.9±0.01ms        ? ?/sec
physical_plan_clickbench_q17                  1.01      3.1±0.04ms        ? ?/sec    1.00      3.0±0.01ms        ? ?/sec
physical_plan_clickbench_q18                  1.01      2.8±0.01ms        ? ?/sec    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q19                  1.01      3.5±0.01ms        ? ?/sec    1.00      3.4±0.01ms        ? ?/sec
physical_plan_clickbench_q2                   1.02      2.8±0.01ms        ? ?/sec    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q20                  1.02      2.7±0.01ms        ? ?/sec    1.00      2.6±0.01ms        ? ?/sec
physical_plan_clickbench_q21                  1.02      2.9±0.01ms        ? ?/sec    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q22                  1.01      3.7±0.01ms        ? ?/sec    1.00      3.6±0.01ms        ? ?/sec
physical_plan_clickbench_q23                  1.01      4.0±0.02ms        ? ?/sec    1.00      4.0±0.02ms        ? ?/sec
physical_plan_clickbench_q24                  1.02      4.8±0.02ms        ? ?/sec    1.00      4.7±0.01ms        ? ?/sec
physical_plan_clickbench_q25                  1.02      3.2±0.02ms        ? ?/sec    1.00      3.1±0.01ms        ? ?/sec
physical_plan_clickbench_q26                  1.02      2.9±0.01ms        ? ?/sec    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q27                  1.01      3.2±0.01ms        ? ?/sec    1.00      3.2±0.01ms        ? ?/sec
physical_plan_clickbench_q28                  1.01      3.8±0.01ms        ? ?/sec    1.00      3.7±0.02ms        ? ?/sec
physical_plan_clickbench_q29                  1.02      4.8±0.02ms        ? ?/sec    1.00      4.7±0.01ms        ? ?/sec
physical_plan_clickbench_q3                   1.01      2.8±0.01ms        ? ?/sec    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q30                  1.02     15.6±0.45ms        ? ?/sec    1.00     15.3±0.04ms        ? ?/sec
physical_plan_clickbench_q31                  1.01      3.9±0.02ms        ? ?/sec    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q32                  1.01      3.9±0.02ms        ? ?/sec    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q33                  1.01      3.5±0.01ms        ? ?/sec    1.00      3.4±0.01ms        ? ?/sec
physical_plan_clickbench_q34                  1.01      3.1±0.01ms        ? ?/sec    1.00      3.1±0.01ms        ? ?/sec
physical_plan_clickbench_q35                  1.01      3.2±0.01ms        ? ?/sec    1.00      3.2±0.01ms        ? ?/sec
physical_plan_clickbench_q36                  1.00      4.3±0.01ms        ? ?/sec    1.00      4.2±0.02ms        ? ?/sec
physical_plan_clickbench_q37                  1.01      4.3±0.01ms        ? ?/sec    1.00      4.3±0.02ms        ? ?/sec
physical_plan_clickbench_q38                  1.01      4.3±0.02ms        ? ?/sec    1.00      4.3±0.01ms        ? ?/sec
physical_plan_clickbench_q39                  1.01      3.9±0.01ms        ? ?/sec    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q4                   1.01      2.6±0.01ms        ? ?/sec    1.00      2.6±0.01ms        ? ?/sec
physical_plan_clickbench_q40                  1.01      4.4±0.02ms        ? ?/sec    1.00      4.3±0.02ms        ? ?/sec
physical_plan_clickbench_q41                  1.01      4.2±0.01ms        ? ?/sec    1.00      4.1±0.02ms        ? ?/sec
physical_plan_clickbench_q42                  1.02      4.1±0.01ms        ? ?/sec    1.00      4.0±0.01ms        ? ?/sec
physical_plan_clickbench_q43                  1.01      4.1±0.02ms        ? ?/sec    1.00      4.0±0.01ms        ? ?/sec
physical_plan_clickbench_q44                  1.01      2.8±0.01ms        ? ?/sec    1.00      2.7±0.01ms        ? ?/sec
physical_plan_clickbench_q45                  1.01      2.8±0.01ms        ? ?/sec    1.00      2.7±0.01ms        ? ?/sec
physical_plan_clickbench_q46                  1.01      3.3±0.01ms        ? ?/sec    1.00      3.3±0.01ms        ? ?/sec
physical_plan_clickbench_q47                  1.01      3.8±0.01ms        ? ?/sec    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q48                  1.02      4.3±0.02ms        ? ?/sec    1.00      4.3±0.01ms        ? ?/sec
physical_plan_clickbench_q49                  1.02      4.6±0.02ms        ? ?/sec    1.00      4.5±0.01ms        ? ?/sec
physical_plan_clickbench_q5                   1.01      2.8±0.01ms        ? ?/sec    1.00      2.7±0.01ms        ? ?/sec
physical_plan_clickbench_q6                   1.01      2.8±0.01ms        ? ?/sec    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q7                   1.01      3.3±0.01ms        ? ?/sec    1.00      3.3±0.01ms        ? ?/sec
physical_plan_clickbench_q8                   1.01      3.0±0.01ms        ? ?/sec    1.00      3.0±0.01ms        ? ?/sec
physical_plan_clickbench_q9                   1.01      3.3±0.01ms        ? ?/sec    1.00      3.3±0.02ms        ? ?/sec
physical_plan_tpcds_all                       1.01   1178.4±4.95ms        ? ?/sec    1.00   1161.8±5.74ms        ? ?/sec
physical_plan_tpch_all                        1.01     73.3±0.31ms        ? ?/sec    1.00     72.7±0.20ms        ? ?/sec
physical_plan_tpch_q1                         1.00      2.6±0.01ms        ? ?/sec    1.00      2.6±0.01ms        ? ?/sec
physical_plan_tpch_q10                        1.01      3.6±0.01ms        ? ?/sec    1.00      3.5±0.01ms        ? ?/sec
physical_plan_tpch_q11                        1.00      3.1±0.01ms        ? ?/sec    1.01      3.2±0.17ms        ? ?/sec
physical_plan_tpch_q12                        1.00      2.5±0.01ms        ? ?/sec    1.00      2.5±0.01ms        ? ?/sec
physical_plan_tpch_q13                        1.00   1831.4±9.43µs        ? ?/sec    1.00   1832.2±7.35µs        ? ?/sec
physical_plan_tpch_q14                        1.02      2.2±0.01ms        ? ?/sec    1.00      2.2±0.01ms        ? ?/sec
physical_plan_tpch_q16                        1.01      3.2±0.02ms        ? ?/sec    1.00      3.2±0.01ms        ? ?/sec
physical_plan_tpch_q17                        1.01      2.9±0.01ms        ? ?/sec    1.00      2.9±0.01ms        ? ?/sec
physical_plan_tpch_q18                        1.01      3.3±0.01ms        ? ?/sec    1.00      3.2±0.01ms        ? ?/sec
physical_plan_tpch_q19                        1.00      5.3±0.02ms        ? ?/sec    1.00      5.3±0.02ms        ? ?/sec
physical_plan_tpch_q2                         1.00      6.4±0.02ms        ? ?/sec    1.00      6.3±0.02ms        ? ?/sec
physical_plan_tpch_q20                        1.00      3.9±0.01ms        ? ?/sec    1.00      3.8±0.02ms        ? ?/sec
physical_plan_tpch_q21                        1.01      5.1±0.02ms        ? ?/sec    1.00      5.1±0.02ms        ? ?/sec
physical_plan_tpch_q22                        1.01      2.9±0.01ms        ? ?/sec    1.00      2.9±0.01ms        ? ?/sec
physical_plan_tpch_q3                         1.01      2.5±0.01ms        ? ?/sec    1.00      2.5±0.01ms        ? ?/sec
physical_plan_tpch_q4                         1.00      2.0±0.01ms        ? ?/sec    1.00   1993.7±9.04µs        ? ?/sec
physical_plan_tpch_q5                         1.02      3.6±0.14ms        ? ?/sec    1.00      3.5±0.01ms        ? ?/sec
physical_plan_tpch_q6                         1.01  1363.8±14.88µs        ? ?/sec    1.00   1349.8±7.07µs        ? ?/sec
physical_plan_tpch_q7                         1.01      4.7±0.02ms        ? ?/sec    1.00      4.7±0.01ms        ? ?/sec
physical_plan_tpch_q8                         1.01      5.6±0.01ms        ? ?/sec    1.00      5.6±0.02ms        ? ?/sec
physical_plan_tpch_q9                         1.01      4.3±0.01ms        ? ?/sec    1.00      4.3±0.01ms        ? ?/sec
physical_select_aggregates_from_200           1.01     24.4±0.06ms        ? ?/sec    1.00     24.3±0.05ms        ? ?/sec
physical_select_all_from_1000                 1.05     44.8±0.36ms        ? ?/sec    1.00     42.6±0.41ms        ? ?/sec
physical_select_one_from_700                  1.13      2.8±0.02ms        ? ?/sec    1.00      2.5±0.01ms        ? ?/sec
physical_theta_join_consider_sort             1.15      3.2±0.01ms        ? ?/sec    1.00      2.8±0.01ms        ? ?/sec
physical_unnest_to_join                       1.06      2.7±0.06ms        ? ?/sec    1.00      2.5±0.01ms        ? ?/sec
with_param_values_many_columns                1.03    189.2±0.92µs        ? ?/sec    1.00    183.2±0.56µs        ? ?/sec

Fixed the performance regression. It wasn't where we thought it was.
The problems was a recursive check (down the LP) of the check_fields within the assert_unique_field_names(). I've removed the recursive nature of this check.
Final numbers, no regression
$ critcmp main invariants__no_recursive_namecheck__schema_check_per_optimizer_pass
group                                         invariants__no_recursive_namecheck__schema_check_per_optimizer_pass    main
-----                                         -------------------------------------------------------------------    ----
logical_aggregate_with_join                   1.00    987.0±5.66µs        ? ?/sec                                    1.02  1007.8±10.29µs        ? ?/sec
logical_select_all_from_1000                  1.04      5.0±0.04ms        ? ?/sec                                    1.00      4.8±0.04ms        ? ?/sec
logical_select_one_from_700                   1.00    739.4±5.38µs        ? ?/sec                                    1.01    743.5±5.70µs        ? ?/sec
logical_trivial_join_high_numbered_columns    1.00    703.2±5.00µs        ? ?/sec                                    1.02    718.2±4.64µs        ? ?/sec
logical_trivial_join_low_numbered_columns     1.00    685.1±5.28µs        ? ?/sec                                    1.02    699.6±3.40µs        ? ?/sec
physical_intersection                         1.00  1726.0±10.72µs        ? ?/sec                                    1.00  1726.6±15.43µs        ? ?/sec
physical_join_consider_sort                   1.00      2.4±0.02ms        ? ?/sec                                    1.02      2.5±0.01ms        ? ?/sec
physical_join_distinct                        1.00    677.2±5.94µs        ? ?/sec                                    1.02    689.2±7.02µs        ? ?/sec
physical_many_self_joins                      1.00     13.6±0.07ms        ? ?/sec                                    1.02     13.9±0.07ms        ? ?/sec
physical_plan_clickbench_all                  1.00    181.7±0.39ms        ? ?/sec                                    1.00    181.5±0.36ms        ? ?/sec
physical_plan_clickbench_q1                   1.00      2.6±0.01ms        ? ?/sec                                    1.00      2.6±0.01ms        ? ?/sec
physical_plan_clickbench_q10                  1.00      3.5±0.01ms        ? ?/sec                                    1.00      3.5±0.01ms        ? ?/sec
physical_plan_clickbench_q11                  1.00      3.5±0.01ms        ? ?/sec                                    1.00      3.5±0.01ms        ? ?/sec
physical_plan_clickbench_q12                  1.00      3.6±0.01ms        ? ?/sec                                    1.01      3.7±0.22ms        ? ?/sec
physical_plan_clickbench_q13                  1.00      3.3±0.01ms        ? ?/sec                                    1.00      3.3±0.01ms        ? ?/sec
physical_plan_clickbench_q14                  1.00      3.5±0.01ms        ? ?/sec                                    1.00      3.5±0.01ms        ? ?/sec
physical_plan_clickbench_q15                  1.00      3.4±0.01ms        ? ?/sec                                    1.00      3.4±0.01ms        ? ?/sec
physical_plan_clickbench_q16                  1.00      3.0±0.01ms        ? ?/sec                                    1.00      2.9±0.01ms        ? ?/sec
physical_plan_clickbench_q17                  1.00      3.0±0.01ms        ? ?/sec                                    1.00      3.0±0.01ms        ? ?/sec
physical_plan_clickbench_q18                  1.01      2.8±0.01ms        ? ?/sec                                    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q19                  1.00      3.5±0.01ms        ? ?/sec                                    1.00      3.4±0.01ms        ? ?/sec
physical_plan_clickbench_q2                   1.00      2.8±0.01ms        ? ?/sec                                    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q20                  1.00      2.6±0.01ms        ? ?/sec                                    1.00      2.6±0.01ms        ? ?/sec
physical_plan_clickbench_q21                  1.01      2.8±0.01ms        ? ?/sec                                    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q22                  1.00      3.6±0.01ms        ? ?/sec                                    1.01      3.6±0.01ms        ? ?/sec
physical_plan_clickbench_q23                  1.00      4.0±0.01ms        ? ?/sec                                    1.00      4.0±0.02ms        ? ?/sec
physical_plan_clickbench_q24                  1.01      4.8±0.02ms        ? ?/sec                                    1.00      4.7±0.01ms        ? ?/sec
physical_plan_clickbench_q25                  1.01      3.2±0.02ms        ? ?/sec                                    1.00      3.1±0.01ms        ? ?/sec
physical_plan_clickbench_q26                  1.01      2.9±0.01ms        ? ?/sec                                    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q27                  1.00      3.2±0.01ms        ? ?/sec                                    1.00      3.2±0.01ms        ? ?/sec
physical_plan_clickbench_q28                  1.00      3.7±0.01ms        ? ?/sec                                    1.00      3.7±0.02ms        ? ?/sec
physical_plan_clickbench_q29                  1.00      4.7±0.02ms        ? ?/sec                                    1.00      4.7±0.01ms        ? ?/sec
physical_plan_clickbench_q3                   1.01      2.8±0.01ms        ? ?/sec                                    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q30                  1.01     15.5±0.41ms        ? ?/sec                                    1.00     15.3±0.04ms        ? ?/sec
physical_plan_clickbench_q31                  1.00      3.8±0.02ms        ? ?/sec                                    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q32                  1.00      3.8±0.01ms        ? ?/sec                                    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q33                  1.00      3.4±0.01ms        ? ?/sec                                    1.00      3.4±0.01ms        ? ?/sec
physical_plan_clickbench_q34                  1.00      3.1±0.01ms        ? ?/sec                                    1.00      3.1±0.01ms        ? ?/sec
physical_plan_clickbench_q35                  1.00      3.2±0.01ms        ? ?/sec                                    1.00      3.2±0.01ms        ? ?/sec
physical_plan_clickbench_q36                  1.00      4.2±0.01ms        ? ?/sec                                    1.00      4.2±0.02ms        ? ?/sec
physical_plan_clickbench_q37                  1.00      4.3±0.01ms        ? ?/sec                                    1.00      4.3±0.02ms        ? ?/sec
physical_plan_clickbench_q38                  1.00      4.3±0.02ms        ? ?/sec                                    1.00      4.3±0.01ms        ? ?/sec
physical_plan_clickbench_q39                  1.00      3.8±0.01ms        ? ?/sec                                    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q4                   1.00      2.6±0.01ms        ? ?/sec                                    1.00      2.6±0.01ms        ? ?/sec
physical_plan_clickbench_q40                  1.00      4.3±0.01ms        ? ?/sec                                    1.00      4.3±0.02ms        ? ?/sec
physical_plan_clickbench_q41                  1.00      4.1±0.01ms        ? ?/sec                                    1.00      4.1±0.02ms        ? ?/sec
physical_plan_clickbench_q42                  1.00      4.0±0.02ms        ? ?/sec                                    1.00      4.0±0.01ms        ? ?/sec
physical_plan_clickbench_q43                  1.00      4.0±0.01ms        ? ?/sec                                    1.00      4.0±0.01ms        ? ?/sec
physical_plan_clickbench_q44                  1.00      2.7±0.01ms        ? ?/sec                                    1.00      2.7±0.01ms        ? ?/sec
physical_plan_clickbench_q45                  1.00      2.8±0.01ms        ? ?/sec                                    1.00      2.7±0.01ms        ? ?/sec
physical_plan_clickbench_q46                  1.00      3.2±0.01ms        ? ?/sec                                    1.00      3.3±0.01ms        ? ?/sec
physical_plan_clickbench_q47                  1.00      3.8±0.01ms        ? ?/sec                                    1.00      3.8±0.01ms        ? ?/sec
physical_plan_clickbench_q48                  1.00      4.3±0.01ms        ? ?/sec                                    1.00      4.3±0.01ms        ? ?/sec
physical_plan_clickbench_q49                  1.00      4.5±0.02ms        ? ?/sec                                    1.00      4.5±0.01ms        ? ?/sec
physical_plan_clickbench_q5                   1.00      2.8±0.01ms        ? ?/sec                                    1.00      2.7±0.01ms        ? ?/sec
physical_plan_clickbench_q6                   1.00      2.8±0.01ms        ? ?/sec                                    1.00      2.8±0.01ms        ? ?/sec
physical_plan_clickbench_q7                   1.01      3.3±0.01ms        ? ?/sec                                    1.00      3.3±0.01ms        ? ?/sec
physical_plan_clickbench_q8                   1.00      3.0±0.01ms        ? ?/sec                                    1.00      3.0±0.01ms        ? ?/sec
physical_plan_clickbench_q9                   1.00      3.2±0.02ms        ? ?/sec                                    1.00      3.3±0.02ms        ? ?/sec
physical_plan_tpcds_all                       1.00   1160.9±2.22ms        ? ?/sec                                    1.00   1161.8±5.74ms        ? ?/sec
physical_plan_tpch_all                        1.00     72.5±0.23ms        ? ?/sec                                    1.00     72.7±0.20ms        ? ?/sec
physical_plan_tpch_q1                         1.00      2.6±0.01ms        ? ?/sec                                    1.00      2.6±0.01ms        ? ?/sec
physical_plan_tpch_q10                        1.00      3.5±0.01ms        ? ?/sec                                    1.00      3.5±0.01ms        ? ?/sec
physical_plan_tpch_q11                        1.00      3.1±0.01ms        ? ?/sec                                    1.01      3.2±0.17ms        ? ?/sec
physical_plan_tpch_q12                        1.00      2.5±0.01ms        ? ?/sec                                    1.00      2.5±0.01ms        ? ?/sec
physical_plan_tpch_q13                        1.00   1832.8±8.27µs        ? ?/sec                                    1.00   1832.2±7.35µs        ? ?/sec
physical_plan_tpch_q14                        1.00      2.2±0.01ms        ? ?/sec                                    1.00      2.2±0.01ms        ? ?/sec
physical_plan_tpch_q16                        1.00      3.2±0.01ms        ? ?/sec                                    1.00      3.2±0.01ms        ? ?/sec
physical_plan_tpch_q17                        1.00      2.9±0.01ms        ? ?/sec                                    1.00      2.9±0.01ms        ? ?/sec
physical_plan_tpch_q18                        1.00      3.2±0.01ms        ? ?/sec                                    1.00      3.2±0.01ms        ? ?/sec
physical_plan_tpch_q19                        1.00      5.3±0.01ms        ? ?/sec                                    1.00      5.3±0.02ms        ? ?/sec
physical_plan_tpch_q2                         1.01      6.4±0.03ms        ? ?/sec                                    1.00      6.3±0.02ms        ? ?/sec
physical_plan_tpch_q20                        1.00      3.9±0.01ms        ? ?/sec                                    1.00      3.8±0.02ms        ? ?/sec
physical_plan_tpch_q21                        1.00      5.1±0.01ms        ? ?/sec                                    1.00      5.1±0.02ms        ? ?/sec
physical_plan_tpch_q22                        1.00      2.9±0.01ms        ? ?/sec                                    1.00      2.9±0.01ms        ? ?/sec
physical_plan_tpch_q3                         1.00      2.5±0.01ms        ? ?/sec                                    1.00      2.5±0.01ms        ? ?/sec
physical_plan_tpch_q4                         1.00   1994.6±9.01µs        ? ?/sec                                    1.00   1993.7±9.04µs        ? ?/sec
physical_plan_tpch_q5                         1.01      3.6±0.02ms        ? ?/sec                                    1.00      3.5±0.01ms        ? ?/sec
physical_plan_tpch_q6                         1.00   1350.1±7.99µs        ? ?/sec                                    1.00   1349.8±7.07µs        ? ?/sec
physical_plan_tpch_q7                         1.01      4.7±0.02ms        ? ?/sec                                    1.00      4.7±0.01ms        ? ?/sec
physical_plan_tpch_q8                         1.01      5.6±0.03ms        ? ?/sec                                    1.00      5.6±0.02ms        ? ?/sec
physical_plan_tpch_q9                         1.00      4.3±0.02ms        ? ?/sec                                    1.00      4.3±0.01ms        ? ?/sec
physical_select_aggregates_from_200           1.00     24.3±0.09ms        ? ?/sec                                    1.00     24.3±0.05ms        ? ?/sec
physical_select_all_from_1000                 1.01     43.2±0.35ms        ? ?/sec                                    1.00     42.6±0.41ms        ? ?/sec
physical_select_one_from_700                  1.00      2.5±0.01ms        ? ?/sec                                    1.02      2.5±0.01ms        ? ?/sec
physical_theta_join_consider_sort             1.00      2.7±0.02ms        ? ?/sec                                    1.02      2.8±0.01ms        ? ?/sec
physical_unnest_to_join                       1.00      2.4±0.01ms        ? ?/sec                                    1.02      2.5±0.01ms        ? ?/sec
with_param_values_many_columns                1.00    183.9±0.62µs        ? ?/sec                                    1.00    183.2±0.56µs        ? ?/sec

The release vs debug mode only has a single difference in the checks. The debug mode will run a full InvariantLevel::Executable check per each optimizer pass (instead of after all pass).

My performance benchmarks show also show now difference ✅
++ critcmp main 13525_invariant-checking-for-implicit-LP-changes
group                                         13525_invariant-checking-for-implicit-LP-changes    main
-----                                         ------------------------------------------------    ----
logical_aggregate_with_join                   1.00  1498.7±38.07µs        ? ?/sec                 1.00  1494.0±18.89µs        ? ?/sec
logical_select_all_from_1000                  1.00      5.3±0.05ms        ? ?/sec                 1.00      5.3±0.05ms        ? ?/sec
logical_select_one_from_700                   1.00  1169.9±26.47µs        ? ?/sec                 1.01  1179.5±14.41µs        ? ?/sec
logical_trivial_join_high_numbered_columns    1.00  1154.9±13.48µs        ? ?/sec                 1.00  1158.7±16.27µs        ? ?/sec
logical_trivial_join_low_numbered_columns     1.00  1140.9±18.36µs        ? ?/sec                 1.00  1140.3±14.97µs        ? ?/sec
physical_intersection                         1.00      2.5±0.02ms        ? ?/sec                 1.00      2.5±0.02ms        ? ?/sec
physical_join_consider_sort                   1.00      3.3±0.02ms        ? ?/sec                 1.00      3.3±0.04ms        ? ?/sec
physical_join_distinct                        1.00  1130.5±14.83µs        ? ?/sec                 1.00  1133.2±15.97µs        ? ?/sec
physical_many_self_joins                      1.01     17.5±0.10ms        ? ?/sec                 1.00     17.4±0.08ms        ? ?/sec
physical_plan_clickbench_all                  1.00    227.8±1.70ms        ? ?/sec                 1.01    230.0±3.38ms        ? ?/sec

Thank you for the reviews.
I made the requested changes to the terminology and error messages.
.

Also, shall we file some follow on tickets for follow on work?

I have a task lists here for follow up items. I was planning to dig more into code to assess viability (& I can make tickets then too) as I start implementing.

Looks like Ci is failint a bit here


Update aes.c

Minor modification for support of systems with 16 bit addressing (uint8_t is 16bit). Used on Texas instruments C2000.

First, thank you for your interest in the project and for talking the time to create a patch and a pull request :)
I am reluctant to merge this one however, since the C99-type fixed width integers are required to be exactly that size and no larger.
If the Texas Instruments compiler makes a uint8_t wider than 1 byte, then I don't think it is C99-compliant.
Another story if using char, uint_least8_t or uint_fast8_t - but uint8_t is required to be one byte wide AFAIK.
Are you using the C2000 compiler with Code Composer Studio?

Hi,
yes, I'm using it with Code Composer Studio with C2000 compiler (ti-cgt-c2000_15.12.1.LTS) and linker.
I understand your remark, 16 bit chars might not be C99-compliant. They just lead to troubles.
The proposed modification helped to me so I wanted to contribute.


typo fix email email -> email

I just realized that there are two email in this message, I think this is a mistake right ? I had directly copied from @Betree 's message to the code.
This sign in email email was requested by <IP_ADDRESS>. If you did not request this email, please inform<EMAIL_ADDRESS>
Originally posted by @CanerSezgin in https://github.com/opencollective/opencollective/issues/2654#issuecomment-570076226
Typo Fix For: https://github.com/opencollective/opencollective-api/pull/2968
Issue: https://github.com/opencollective/opencollective/issues/2654

Thanks!


F# using keyword is not highlighted

VSCode Version: 0.10.11
OS Version: Windows 7

Steps to Reproduce:

Open an F# source/script file
Set the language to F#
The using keyword is not syntax-highlighted

Moved to https://github.com/ionide/ionide-fsharp/issues/213


feat: Kubeflow pipelines 2.0 update

Describe your changes
Updating wanna-ml to support kfp 2.x
Issue ticket number and link
Checklist before requesting a review

[x] I have performed a self-review of my code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] If it is a core feature, I have added thorough tests.

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (9dffe31) 74.42% compared to head (26df489) 74.53%.
Report is 4 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #103      +/-   ##
==========================================
+ Coverage   74.42%   74.53%   +0.10%
==========================================
  Files          57       57
  Lines        2839     2835       -4
==========================================
  Hits         2113     2113
+ Misses        726      722       -4

Files
Coverage Δ

src/wanna/cli/__main__.py
82.14% <100.00%> (ø)

...anna/components/kubeflow/get_or_create_endpoint.py
13.79% <ø> (ø)

.../wanna/components/kubeflow/upload_model_version.py
11.62% <100.00%> (+2.10%)
:arrow_up:

src/wanna/core/deployment/artifacts_push.py
100.00% <ø> (ø)

src/wanna/core/deployment/vertex_jobs.py
20.00% <ø> (ø)

src/wanna/core/services/docker.py
64.76% <ø> (ø)

src/wanna/core/services/path_utils.py
100.00% <ø> (ø)

src/wanna/core/services/pipeline.py
91.27% <100.00%> (+2.31%)
:arrow_up:

src/wanna/core/utils/time.py
100.00% <ø> (ø)

src/wanna/core/deployment/vertex_pipelines.py
59.45% <0.00%> (ø)

... and 2 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


I have a simple question

I know that the buffers are base64 that converted from some string.
But what the string is?the points?the matrixs?
Can you give me a detailed information?
thanks in advance.

Data URI (base64) is just one of possible ways of representing binary data. Other two are an URI to an external file and GLB container format. Buffer itself is just an array of bytes. Its meaning is defined by buffer views and accessors.

Thank you,lexaknyazev.
If i convert dae to gltf,which part of data in dae convert to buffers.That is my perplexity.

@Febfire Generally, mesh data, skins, and animation data ends up in the buffers, but everything else is in the JSON.
Take a look at the glTF Overview.  The first several columns across the top explain the fundamental organization of data inside glTF.

Closing this, assuming that the basic idea has been conveyed.


Data race in surveyor Recv during Init

Immediately after dialing 3 listening inproc Respondents, I call Send on my Surveyor socket, and then immediately call Recv.  Testing with -race:
$ go test -race -check.f TestNewResp
==================
WARNING: DATA RACE
Write at 0x00c420072950 by goroutine 36:
  github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos.(*socket).SetRecvError()
      /Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/core.go:167 +0x59
  github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/protocol/surveyor.(*surveyor).Init.func1()
      /Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/protocol/surveyor/surveyor.go:56 +0x8b

Previous read at 0x00c420072950 by goroutine 8:
  github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos.(*socket).RecvMsg()
      /Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/core.go:264 +0xf6
  github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos.(*socket).Recv()
      /Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/core.go:287 +0x50

(The reason I was running race tests is that I am randomly getting ErrProtoState on Recv.  This problem goes away when I add a short sleep between my Dial and my Send / Recv.  It is also minimized but not fully eliminated using runtime.Gosched() instead.)

Thanks.   I will investigate.
On Tue, Oct 25, 2016 at 9:08 PM, Bodie Solomon<EMAIL_ADDRESS>wrote:

Immediately after dialing 3 listening inproc Respondents, I call Send on
my Surveyor socket, and then immediately call Recv. Testing with -race:
$ go test -race -check.f TestNewResp
WARNING: DATA RACE
Write at 0x00c420072950 by goroutine 36:
github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos.(*socket).SetRecvError()
/Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/core.go:167 +0x59
github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/protocol/surveyor.(*surveyor).Init.func1()
/Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/protocol/surveyor/surveyor.go:56 +0x8b
Previous read at 0x00c420072950 by goroutine 8:
github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos.(*socket).RecvMsg()
/Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/core.go:264 +0xf6
github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos.(*socket).Recv()
/Users/jsolomon/meta/dev/go/src/github.com/synapse-garden/sg-proto/vendor/github.com/go-mangos/mangos/core.go:287 +0x50
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/go-mangos/mangos/issues/236, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABPDfeyt4fSejU9XtiLFqOnH7zKcmhLbks5q3tIjgaJpZM4KgvUT
.

Fwiw, the vendored ref is https://github.com/go-mangos/mangos/commit/c3916ba23b3aa721fd13b4dec916b2d15f4a2520.

I'll whip up a simplified reproduceable demo tonight if needed.

Yep, its a clear race.  Relatively easy to fix.  Do you have a test case for it?

I don't need the demo, but a test case we can integrate into the test suite would be nice.

I am not actually clear on what is causing the race so writing a test to demonstrate it will be "hard" :P

Fair enough -- if you could post your test program that would still be helpful.
That said, the race is caused by the attempt to access the Socket's recverr member without holding the Socket.Lock.
Reviewing the logic there last night, I found it to be a bit more convoluted than I'd like.  I will try to fix it later today.  (A quick fix will not be difficult.  Changing the logic to be a bit more straight-forward will take a little longer.)

go run -race
https://play.golang.org/p/VubnL5qoIx


deps: Upgrade graceful-fs dependency to the latest

graceful-fs used to monkey-patch node's core fs module.
This has been fixed in the version 4.
Related: https://github.com/nodejs/node/pull/2714

Thanks! Merged as a8b862e


Completing the non-Clifford capabilities [$800]

Bug bounty logistic details (click to expand)
To claim exclusive time to work on this bounty either post a comment here or message<EMAIL_ADDRESS>with:

your name
github username
(optional) a brief list of previous pertinent projects you have engaged in

Currently the project is claimed by no one until ....
If you want to, you can work on this project without making a claim, however claims are encouraged to give you and other contributors peace of mind. Whoever has made a claim takes precedence when solutions are considered.
You can always propose your own funded project, if you would like to contribute something of value that is not yet covered by an official bounty.

Project: Completing the non-Clifford capabilities [$800]
This library already has some nascent capabilities to represent non-Clifford states as a weighted sum of tableaux states. Such a representation is useful in settings where there is a small number of non-Clifford gates in a mostly Clifford circuit. The cost is still exponential in the number of non-Clifford gates, but manageable. We have simple state representation and gate application already done, expect is nearly implemented in this PR, and project! still needs to be implemented. To obtain this bounty, the aforementioned features need to be completed, well tested, and well documented.
Required skills: Understanding of the Stabilizer formalism and decompositions of arbitrary states in terms of Pauli channels
Reviewer: Stefan Krastanov
Duration: 2 months
Publication: In the next 2 years we plan to release a paper in a selective journal about this software. Contributing to this issue would deserve a co-authorship status on such a paper (if the contributor so desires)
Payout procedure:
The Funding for these bounties comes from the National Science Foundation and from the NSF Center for Quantum Networks. The payouts are managed by the NumFOCUS foundation and processed in bulk once every two months. If you live in a country in which NumFOCUS can make payments, you can participate in this bounty program.
Click here for more details about the bug bounty program.

I checked out Ted's paper some time ago, which seems very exciting. Your initial work on non-Clifford is also quite cool about generalized stabilizer.
Therefore, I will be delighted to work on this as that will teach me more about non-Clifford functionalities.
Also, there is a question I wanted to ask you: is non-Clifford the same as non-stabilizer/non-stabilizerness? In some papers, they use the term non-stabilizerness (when they refer to magic, they call it a non-stabilizerness)

Sounds good! Happy to reserve this bounty for you for the next two months.
Yes, the terms you mentioned are frequently used interchangeably.

Overall progress on Non-Clifford functionalities are divided into the following parts: 1) implementing the basic TODOs, 2) fixing bug in expect, 3) implement project 4) providing user rich experience by having implemented most APIs from #260
to push them asap after this issue 5) Overall documentation 6) scaffolding for future non-cliff functionality/APIs.

#346
#348
Only project! is left. I am working on it locally, while waiting for the above PRs to be reviewed.
Documentation: Improved the documentation in 1).  Future work will be to have a dedicated section in (stabilizer manual) stab_.md just like we have a detailed section for MixedDestablzier.
Opened an issue about md ⊗ S"X" in #339.
To fully utilize non-clifford functionality, there are important API that are necessary to work with sm which are in PR #260.  Atm, I have also completed 5 of them. So, they can the easily added to provide rich non-clifford functionality as soon as possible.  Thus, PR #345 implements  sm⊗sm, tHadamard*sm, pcT⊗P"X" , pcT*tId1 , P"-Y"*sm and copy(sm). Outputs of these tasks: https://github.com/QuantumSavory/QuantumClifford.jl/pull/316#issuecomment-2282218098
To realize embed for sm, #342 that provided embedding for Tableau and Stabilizer
Experimental Work on conj_destabs: https://github.com/QuantumSavory/QuantumClifford.jl/discussions/347
Also, we need for base.zero operators for MixedDeStabilizer , Destabilizer, MixedStabilizer,  they will be helpful for later use. Even size(md)[1], or size(md)[2] gives error atm. I have fixed them, but didn't push the PR yet.

@Fe-r-oz , what is the status of your work on this bounty? I see you have done a lot of quality of life improvements and additions to tests (certainly useful!). Among the actual main components of the work, I see you have finished expect (probably the most important completed part of the work).
It seems projectrand! is the main component left -- without it, the smaller "quality of life" improvements are not as useful. And it seems projectrand! has been the main "difficult" or "substantial" component. Could you focus on it so that we can wrap up the bounty? The work left after this is done would be much easier and more straightforward in comparison.

Note that this was reserved in July to be done by September. While you have been doing great work, I am not very comfortable keeping a bounty reserved while you are working on other bounties.

Thank you for the feedback. Currently, only projectrand! remains to be completed. I’ve updated the tests in line with your comments in PR #355 today, confirming that result1 or result2 yield a zero matrix in stabilizer state projections using GeneralizedStabilizer for single-qubit Pauli operators, which suggests that the approach is on the right track. The tests have also been updated to use random_stabilizer/pauli for this specific case.
Additionally, smaller work includes informative error message for the project! function today, drawing from your insights in #355. This updated message is reflected in PR #416 and incorporates both your suggestions from #355 and relevant details about pauli measurement from the associated paper with general documentation in #409. I hope we can integrate these improvements as well.
I plan to implement projectrand! by November 10th, 2024, by utilizing the density matrix update χ′, which is calculated using χ′=expect(p,genstab) and the corresponding probability defined by prob=(real(χ′)+1)/2. I hope to provide a valuable foundation via projectrand! for testing the consistency of multi-qubit, non-stabilizer operations by this date to facilitate future developments.

Hi Stefan, I wanted to provide you with an update on the bounty.
In #355, we demonstrate that multi-qubit non-stabilizer circuits, incorporating non-Clifford gates at randomized positions, can be probabilistically projected through repeated trials. These findings align with Ted's concluding observation that "no magic state will be simulatable through every stabilizer circuit" particularly given the assumption that BQP is larger than BPP. Ted also raised an unresolved question: "What conditions are both necessary and sufficient for determining the set of quantum states (magic states) that can be simulated through stabilizer circuits?"

In addition, as an example, attached is the .txt file showing the results of applying a randomized 3-qubit non-Clifford gate to a stabilizer circuit for multi-qubit projection. The results demonstrate that the post-measurement complex density matrices match the expected outcomes (printed as TRUE), supporting Ted's comment that we can simulate some 'magic' states using the stabilizer-defined basis. However, not all 'magic' states are simulatable in this way (printed as FALSE), as noted by Ted. These specific results have been reproduced via reproducible 2 from https://github.com/QuantumSavory/QuantumClifford.jl/pull/355#issuecomment-2465850317. PFA: 3_qubit_ncgate_results.txt.
Tests for a probabilistic non-stabilizer simulator have been added, evaluating up to 10 qubits with random non-Clifford gates at randomized positions. This message serves as a brief summary of the findings.
I look forward to hearing your thoughts. Thank you!

Future prospects: The paper Unbiased Simulation of Near-Clifford Quantum Circuits echoes similar insights as mentioned in Ted's one of concluding paragaph. Given that the dynamics of a general quantum system fall within the complexity class BQP, a scalable method for simulating universal quantum circuits on a classical computer is highly unlikely. This paper addresses the simulation of stabilizer circuits with non-stabilizer errors by representing these errors as quasiprobability distributions over stabilizer operations. The authors present a new method for simulating arbitrary quantum circuits that combines the efficiency of stabilizer propagation with Monte Carlo sampling via stabilizer channel decomposition.

Thank you for your feedback during the previous review.
Last week, I implemented the Pauli measurement algorithm for projectrand! in PR #427. I look forward to your review. Thanks!


Add serialize_as_any config flag + runtime support

Contributing to https://github.com/pydantic/pydantic/issues/6423
TODO:

[x] Add support for dataclasses
[x] Add support for TypedDict
[x] Add much more comprehensive testing
[ ] Investigate recursion guard that Adrian changed initially, write relevant test, investigate performance consequences - planning to chat with @davidhewitt about this later this week :)
[ ] Ask follow up questions about field_name = None
[ ] Resolve questions about extra usage, why we use mut in model.rs

Investigate recursion guard that Adrian changed initially, write relevant test, investigate performance consequences - planning to chat with @davidhewitt about this later this week :)

Did you remove that from this PR? Sounds like it's probably orthogonal and might be irrelevant with the recent changes samuel and I made.

Ask follow up questions about field_name = None

? :)

Resolve questions about extra usage, why we use mut in model.rs

If you mean for the extra, IIRC the mut is a temporary fudge to make it possible to use that extra with the recursion guard. We should be refactoring the serialisation state.

Resolutions from my conversation with @davidhewitt:

For now, we do need this at the model, dataclass, and field level, but notably, the field case is only for TypedDict cases
The recursion guard logic from Adrian's PR has been removed, as we think it's no longer necessary with more recent recursion guard changes
DH and I think that it could be beneficial to use a more generic type base class for the various model, dataclass, TypedDict serializers in the future, as there's definitely some replication that could be removed
The field_name stuff hasn't been changed, though it could be revisited in the future
The mut questions I had about extra stuff have also been resolved with my changes

I see that maybe that was addressed in other discussion with DH. I have reviewed and don't have any objections, I'd feel better if @davidhewitt also approved though.

So I'm generally happy with what's going on here. I think there are still cases that I'd like to at least ask about. For example, list[str] vs list[int] serialisation. The following test fails with warnings for passing str where int expected:
def test_serialize_as_any_list_types() -> None:
    serializer = SchemaSerializer(core_schema.list_schema(core_schema.int_schema()))

    assert serializer.to_python(['a', 'b', 'c'], serialize_as_any=True) == ['a', 'b', 'c']

The annotated code which I'd argue is equivalent doesn't warn:
from pydantic import TypeAdapter, SerializeAsAny

TypeAdapter(SerializeAsAny[list[int]]).dump_python(["a", "b", "c"])

So I'm generally happy with what's going on here. I think there are still cases that I'd like to at least ask about. For example, list[str] vs list[int] serialisation. The following test fails with warnings for passing str where int expected

I think it makes sense to still warn here in general. SerializeAsAny if I understand correctly is more about how we handle subclasses

The annotated code which I'd argue is equivalent doesn't warn

Hmm, well in this case, we're actually using an any schema, hence the lack of warning. I think we could address this separately in pydantic when we modify the schema for a type wrapped in SerializeAsAny...
@adriangb at one point suggested:

if we make SerializeAsAny generate it's schema as some sort of union of infer and use this static schema it should work for all cases. SerializeAsAny[SecretStr] would use the static schema unless you pass in a subclass of str that has a serializer and SerializeAsAny[SomeModel] would always use the instance's serializer.

I agree that SerializeAsAny is intended to be used for subclasses, but it does allow arbitrary type inference and my point above is that the annotation is still behaving differently to the runtime flag. I think users might find this subtle / surprising. It makes the behaviour something they have to discover empirically rather than easily reason about.
That said, I'm not entirely sure on the right solution here. I think the point is that I would like to close that gap in the long term. Perfect being the enemy of the good and all that, maybe the approach is to proceed with what we have here, and create a tracking issue with follow ups of where SerializeAsAny doesn't work like serialize_as_any=True and then we can improve inference later.

I came here to ask the same question @davidhewitt asked about list[int] and ['a'] (also worth checking ['1']).
Perhaps we can resolve the confusion by calling this flag / feature duck_type_serialization instead of serialize_as_any? I think logically then it follows that you "ask the thing how it wants to be serialized and if it doesn't know how to fall back to the schema". As opposed to "serialize as any" which means "use the same logic as if this were Any" which means "ask it how to serialize itself and if it doesn't know fall back to inferring how to serialize it from the type at runtime". The difference being that in the duck type case we fall back to the schema and in the serialize as any case we fall back to inferring and never use the schema.
Which does mean maybe we should have a DuckTypeSerialization[T] annotation.

Perhaps we can resolve the confusion by calling this flag / feature duck_type_serialization instead of serialize_as_any? I think logically then it follows that you "ask the thing how it wants to be serialized and if it doesn't know how to fall back to the schema". As opposed to "serialize as any" which means "use the same logic as if this were Any" which means "ask it how to serialize itself and if it doesn't know fall back to inferring how to serialize it from the type at runtime". The difference being that in the duck type case we fall back to the schema and in the serialize as any case we fall back to inferring and never use the schema.

Which does mean maybe we should have a DuckTypeSerialization[T] annotation.
@adriangb I think that this was really well said and carefully thought out. @davidhewitt, are you ok with this change? I think it allows us to sidestep the surprising inconsistencies with SerializeAsAny, while also moving forward with these changes!

I can see the purity in this argument but I worry it adds complexity to users by having "duck typed serialization" and "serialize as any" as separate concepts. Especially when the intended use for both of them is to serialize subtypes in full, and neither of them as implemented behave like what I'd expect from "duck typing".
I would much prefer we just lived with this being called serialize_as_any and we worked to unify it with the annotation.

@davidhewitt,
That makes sense, so perhaps we should proceed with this approach?

That said, I'm not entirely sure on the right solution here. I think the point is that I would like to close that gap in the long term. Perfect being the enemy of the good and all that, maybe the approach is to proceed with what we have here, and create a tracking issue with follow ups of where SerializeAsAny doesn't work like serialize_as_any=True and then we can improve inference later.

@sydney-runkle do you think we could make the annotation behave more like this?

@adriangb,
I think we would probably have to go the other way, that is, being more lax on the serialize_as_any runtime flag side of things. I've created this issue to track the discrepancies in behavior, and ultimately nudge us towards resolving those differences: https://github.com/pydantic/pydantic/issues/9049.

Just waiting on https://github.com/pydantic/pydantic-core/pull/1236 to fix the bug mentioned in the most recent test that I wrote


Add support for Python3.8

[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] I have searched the documentation and believe that my question is not covered.

Issue

After the update to python 3.8 poetry is broken for Arch Linux. The reason is that the vendored dependencies in
https://github.com/sdispater/poetry/releases/download/0.12.17/poetry-0.12.17-linux.tar.gz
don't include py3.8. (it includes py2.7 and py3.4 to py3.7 explicitly)

Version 0.12.* doesn't officially support python 3.8. Please upgrade to the latest preview version via poetry self:update --preview.
I will close this issue for now. If you have any problems please feel free to comment.

@finswimmer I understand how to upgrade poetry via the self updater, but this doesn't work when poetry is installed via the OS. As I said when I raised the issue:

After the update to python 3.8 poetry is broken for Arch Linux.
Well, it's fixed now in the Arch repo.

Furthermore, what you propose is a bit problematic. There is no non-prerelease that supports python 3.8, which means that the proposal of "updating" (to a pre-release) is just as :disappointed: as building from source.

@spapanik I don't understand. If I take a look at https://www.archlinux.org/packages/community/any/python-poetry/, it's been packaged for Python 3.8.
The release asset you mentioned is only used by the Poetry installer but is not, or should not at least, be used by OS packager.

It's fixed now, it wasn't when I opened the issue


Update js-yaml to 3.13.1

Security Issues identified by NPMJS

(Moderate) - Denial of Service  More Info (Patched in >=3.13.0)
(High) - Code Injection  More Info (Patched in >=3.13.1)

porting issue


disabling mod menus if not v1.0.7

This simply doesn't initialize mod menus unless game version starts with 1.0.7.
OWML works otherwise fine, except: mods that try to do menu stuff will produce errors, but it doesn't seem to crash the game.
It's not a permanent fix but better than what we have now.

Maybe we should just quit instead 🤷
In any case, we are able to do version check in-game, as long as we do it before menus are initialized.

I don't really think using in-code version is a good idea TBH


refined kde_distributions() w/ better error handling and addtl params

Description
This pull request updates the kde_distributions() function to enhance its capabilities and customization options for generating KDE and histogram distribution plots.
Key Changes

Alpha Transparency for Histogram Fill:

Added a fill_alpha parameter to control the transparency of the histogram bars' fill color.
The default value is 0.6. An exception is raised if fill=False and fill_alpha is specified.

Custom Font Sizes:

Introduced label_fontsize and tick_fontsize parameters to allow control over the font size of axis labels and tick marks independently.

Scientific Notation Toggle:

Added a disable_sci_notation parameter to enable or disable scientific notation on axes.

Improved Error Handling:

Added validation for the stat parameter to ensure that only valid options are accepted.
Added checks to ensure proper usage of fill_alpha and hist_edgecolor when fill is set to False.

General Enhancements:

Updated the function's docstring to reflect the new parameters and provide comprehensive guidance on its usage.

🙌


Add timeout option to the emr-serverless job run

The default timeout for jobs in the EMR serverless is 12 hours (https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/considerations.html). For most uses cases this timeout is sufficient, but not for all and especially for the spark structured streaming job, which is running all the time.
The solution proposal is to add the --emr-serverless-timeout flag to the run command and pass the value to the start_job_run method.

Merged


Update supported-smartphones.md

Samsung Galaxy A71: WiFi Beacon works.
Note: The device I tested had Android version 11.
The transmitter device was a Parrot Anafi drone.

Thanks for testing.
In general, with the latest version of the application, it should be possible to receive the Beacon signals on all devices running android 6 and higher (although with varying receive rate depending on the HW and SW). I need to add some generic comment about this to the table.

In general, with the latest version of the application, it should be possible to receive the Beacon signals on all devices running android 6 and higher (although with varying receive rate depending on the HW and SW).

I know. Will also test a few pre-11 devices soon ...
Do you think it's worth to measure and document rates as well?

I need to add some generic comment about this to the table.

It's mentioned in the ReadMe at least, which is sufficient to me.

Do you think it's worth to measure and document rates as well?

That is probably going into too much details. It seems to somehow depend on the transmitter implementation as well (at least based on the limited testing I have done here with the EPS32 beacon and a desktop hostapd beacon implementation I am working on.)


Use the output of git describe as the version string

Closes #151

@brianhelba, are there any pitfalls to accomplishing this feature this way?


Correct swarm init default-address-pool to default-addr-pool

What I did:
git grep -l default-address-pool  |\
xargs sed -i 's:default-address-pool:default-addr-pool:g'

References:
client version 18.09.3.
$ docker swarm init --help |grep -- '--default-addr.*-pool '
      --default-addr-pool ipNetSlice           default address pool in CIDR format (default [])

brush probably too broad (server 18.09.3):
$ dockerd --help |grep default-addr
      --default-address-pool pool-options       Default address pools for node specific local networks

Deploy preview for docsdocker ready!
Built with commit 785ce638bf6af1fa4ff81fd4e4b21acd6110049e
https://deploy-preview-8662--docsdocker.netlify.com

Updated to apply to docker swarm init docs only.

@DawnWood-Docker PTAL.

merge conflict

Right, this is on hold until I can work with @trapier to get the merge conflict resolved. It dropped on the priority list. Thanks.

Rebased and addressed merge conflict.

lgtm!


Upgrade to Tangram 0.9.x

Tangram 0.9 is here and it's better!
Let's upgrade to it at the next appropriate rev.

Yay! Any breaking API changes per our docs, or should this be a drop-in?

Fully backward-compatible, we should be able to just drop it in.


Question: what if we want to write parallel some app that consume the API that we are writing?

Hi!!
If I want to create in parallel an application that consume the API for the container(s) created, where to put this code? Maybe:

app

Containers

Users

Actions
Tasks
...
Consumers (or CLI's or whatever) // here is the container for the platform app API consumer

Android
AngularJS
Angular 2

src // module bundle source

JavaDesktop
VB.Net

Off course, with Angular 2 we have web, mobile and desktop apps with slight changes on the same code base (and some limitations too), I am writing the code on Angular 2. It's for demonstration purposes only...
This is with the purpose of make available the API app and the API consumer on the same project and having the latest fixes/features available on both sides. Even when this maybe means have a like Hello-API consumer core.
Or is a better idea to make these pieces separated?
Thanks!!

@llstarscreamll I used to have this structure:
- Project 1
  - Server (Laravel API code)
  - Clients
    - Web
      - Angular
    - Mobile
      - Android
      - iOS

Let me know what you think of it

Looks nice, and make use of git sparse checkout feature for develop/deploy the right folder on the right side.
Thank you so much!!


'Uncaught (in promise) null' error when querying profile with no apps key for a public file

When calling blockstack.getFile on a user with no apps key, 'Uncaught (in promise) null' throws, and nothing is return from the call. This breaks, for example, loops such as the following:
const filename = 'public_index.json'
var contacts = ['djh.id', 'pat.id', 'larry.id']
for(var i = 0; i < contacts.length; i++){
const publicOptions = {
username: contacts[i],
app: 'http://localhost:3000',
zoneFileLookupURL: 'https://core.blockstack.org/v1/names/'
}
var result = await blockstack.getFile(filename, { ...publicOptions })
console.log(result)
}
Because djh.id has public_index.json, the first call responds with the relevant file. However, pat.id breaks, and larry.id is never reached. Desired behavior: console.log(result) == null; this is the current behavior when querying for a file that does not exist, for a user with an apps key. The error is in storage/index.js; see pull request #375

Thanks for raising this issue. This is the same root issue as #432 -- namely, we want a better error message in this case. However, in your example, you'd still need to check for a promise rejection in this case.
I'm closing this issue in favor of #432.


Build string in conda package url for Spyder in the conda-lock file release asset does not match conda-forge

The installer workflow generates a conda-lock file for updating installer-based Spyder versions. The explicit url specification in the file has the form:
https://conda.anaconda.org/conda-forge/<platform>/spyder-<version>-<build-string>.conda

There are two issues with the url in the conda-lock file.

For non-stable releases (e.g. 6.0.0b2) label/spyder_dev/ is missing from the url.
The <build-string> portion does not match that of the package on conda-forge. This is because the <build-string> contains a hash that is automatically generated by conda-build based on the contents of meta.yaml in the recipe, and this file will differ slightly between our CI and conda-forge.

@spyder-ide/core-developers, we should discuss possible solutions.

And what could be a possible solution? I'm sorry but I don't understand how to address this problem.

The first issue, regarding the dev channel, is straight-forward and can be accommodated when we patch the conda-lock file.
I thought that the second issue would be more problematic. In order to have the same build string in the conda-lock file and on conda-forge I thought there were only 2 options: 1) explicitly set the build string; or 2) build the conda package from an un-patched meta.yaml file.
However, I have discovered that I'm wrong about how the hash info is compiled. The difference between our CI build string and the conda-forge build string results from conda-forge having an additional conda_build_config.yaml which is absent from our CI. conda-build searches several locations for this configuration file and merges all files discovered, so the configuration consists of more that just what is in our feedstock recipe.
I have narrowed down the relevant configuration spec to
python:
  - <major>.<minor>.* *_cython

This can be patched in the conda_build_config.yaml file in our feedstock when we build on CI.

Ok, that's really good 👍🏽


Backport fix for hanging on Node 0.12 to the 1.x version.

This was extremely useful for me, so I backported the fix from here to<EMAIL_ADDRESS>and pushed it to my fork on the fix-node-0.12 branch.
It would be awesome if this got published to npm as<EMAIL_ADDRESS>:)

Done and published as 1.2.5.


MAE and RMSE

I find that MAE and RMSE should use the number of observed pixels in labels as divisor. But you use the number (320x256) for evaluation.
Could you explain it? Thank you!

It's kind of you!
I still have some questions about it. L1 and RMSE  use ''np.mean'' (line 82 and 83)in your code, which means the divisor is "320x256". The number of observed pixels "N" you mentioned are only used for "delta_*"(line 85-89), instead of L1 and RMSE.
That's my confusion. Thank you!
------------------ 原始邮件 ------------------
发件人:                                                                                "tsunghan-mama/Depth-Completion"<EMAIL_ADDRESS>发送时间: 2020年11月10日(星期二) 晚上11:57
收件人<EMAIL_ADDRESS>抄送: "曹子东"<1097759190<EMAIL_ADDRESS>主题: Re: [tsunghan-mama/Depth-Completion] MAE and RMSE (#8)
Hi,
Yes as you've mentioned, we indeed use the number of observed pixels in evaluation.
Our implementation can be shown here.
We first calculate the number of unobserved pixels and then calculate # of all pixels - # of unobserved. (where # of all pixels = 320x256)
Feel free if you have any further questions.
Thanks!
Best Regards,
Tsung-Han Wu
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hi,
Thanks for your great findings!
It sounds that our implementation might be incorrect.
However, I don't think it would affect the comparison results between these depth completion methods.
Sincerely sorry to make your confused.
Best Regards,
Tsung-Han Wu


Add missing relationships (DigitalTwin)-[:HAS]->(Credential)

Created by: Bohumil Pospíšil May 14 2021 at 13:55 CET
Updated: Jul 05 2021 at 16:27 CET
ClickUp ID: IK-867
Modify cypher scripts:
12_add_missing_properties_or_relations.cypher
13_add_oauth2_credential.cypher
in import\data\v1.2.0.  The scripts will create new relationships HAS between DigitalTwin and Credential.
Missing relations are:
MATCH path = (dt:DigitalTwin)-[]->(c:Credential)
WHERE NOT (dt)-[:HAS]->(c)
RETURN path
Cypher query
CALL apoc.periodic.iterate(
"MATCH (dt:DigitalTwin)-[]->(c:Credential) WHERE NOT (dt)-[:HAS]->(c) RETURN dt, c",
"MERGE (dt)-[:HAS]->(c)",
{batchSize:10000});

Commented by: @Bohumil Pospíšil May 17 2021 at 11:24 CET
@yetizer I think that will be necessary to execute this query manually after installing the new version in the production environment too. If yes, please is there any common task for this purpose?

Commented by: @bayegan Jun 02 2021 at 21:43 CET
@yetizer @ Bohumil Pospíšil: can we add some acceptance criteria, as input to @kilimandzsaro and verification. cc @marvinkite @ Krisztian Marko

Commented by: @yetizer Jun 03 2021 at 09:12 CET
So far none of the cypher tasks have been reviewed by Natalia. Most of them can be tested when services will implement the logic that uses cyphers. Another part (much smaller) has to be manually applied to the database (updating existing data to be compatible with the new model)
This task is the later one - not sure what can be acceptance criteria here


Prevent formation of duplicate events

A lot of duplicate events show up in Datadog - events about same activity with only differing timestamps. Checking for duplication before pushing it to the OMG bus could be desirable.

Continuing the discussion from https://github.com/omisego/elixir-omg/issues/1426 -
Summing up the discussions with @unnawut -

A lot of duplicate events show up in datadog i.e. events about the same activity with only differing timestamp
Could potentially be an issue with the ethereum event listeners or the OMG bus from where the events are consumed.
Considering the number of event listeners combined ( around 10+10+4 ), it could be the entity which is producing these hundreds of duplicate events.
If that is the case, de-duplicating before publishing to DataDog might not be the best approach, since it is among the last consumers of the system.


Closes #112: Add WorldMapLegend class

Added WorldMapLegend class for future map legend development (Currently WorldMapLegend class can control the amount of locations to appear in the map legend, but that's about it. Hopefully in future builds WorldMapLegend will have more use).

Moving the renderLegend function into the WorldMapLegend class I changed the function to also take NW,NE,SE,SW making it 8 locations near the player. renderLegend now displays 6 locations.

renderLegend now uses a Set instead of an ArrayList, to get non-duplicate locations.

Since I am using a Set, I added the hashCode and equals function to WorldMapSymbol class. I never implemented a hashCode function before so I copied the one in creatures\ColoredString and tailored it to WorldMapSymbol. Hopefully it's correct.

As of the current build, the map legend adds a newline after the last two location letting you scroll on the map command (It's hard to see but you can scroll up and down, which I assume is not wanted):

I use == here instead of modules to fix this: if (i == NEW_LINE_VALUE) (Since module adds another new line).

I changed the locations text on the map legend to also be colored. This made the formatting easier for the 6 locations. If you don't like it I can try to change it back 👍 .
Using the longest location name "Ancient Dungeon Room" I was able to check if there was any problems with the formatting:

Seems good! May be a problem in the future if there are longer location names, but you can change the amount of locations displayed on the map legend using MAX_LEGEND_LOCATIONS.

Couldn't use forEach for set because there's a small chance you can get more then 6 locations (I use a for loop and a Iterator):

Referencing https://github.com/bernardosulzbach/dungeon/pull/423:

Some outstanding issues after what I did: the WorldMap is larger than it needs to be (we ignore the first and last line), the calculation for the player position is a bit... arbitrary. Should probably be done by WorldMap (which really knows where the player is in the matrix). Lastly, the redundancy generated by only using adjacent locations is an issue (see image below). You can move on with any of these improvements and I will be glad to review your work on them.

This PR fixes the redundancy but there may be a different way, using the classes WorldMapLegend and WorldMap together to produce a map legend.

I never implemented a hashCode function before so I copied the one in creatures\ColoredString and tailored it to WorldMapSymbol. Hopefully, it's correct.

It seems OK. The hashCode() should be a bit stronger.
IntelliJ IDEA can spit out "industry standard" equals() and hashCode() for most classes, which helps a lot. Effective Java has some explanation for what is "good enough" equals() and hashCode().

As of the current build, the map legend adds a newline after the last two locations letting you scroll on the map command.

I did not check this locally. It seems like something I did not want to do.

I changed the locations text on the map legend to also be colored. This made the formatting easier for the 6 locations.

It's OK for me.

Using the longest location name "Ancient Dungeon Room" I was able to check if there was any problems with the formatting.

As of right now. A better solution would be to cut the location name and append three dots to it if it was too long. However, this can be done in the future, when it becomes a problem.
Overall it seems quite good. I will properly fetch, test, tweak and merge this in the next few days.
Thank you for your good work.

OK, merged. It works well, and the code looks good.
Having too much (or even any) static around is sometimes a quality issue. This is the case in several old classes of this project. In this case, it's fine for now.
You can see how I changed the hash function you provided in https://github.com/bernardosulzbach/dungeon/commit/49271beb006dfec01b9fe91b219ec8a7e26e71fb. Doing it this way calls the following method eventually, which can handle several cases nicely and more "cleanly" than manually writing your own hashCode math. Feel free to replace manual hash calculations by this idiom in new classes or even in any classes you happen to work on.
    public static int hashCode(Object a[]) {
        if (a == null)
            return 0;

        int result = 1;

        for (Object element : a)
            result = 31 * result + (element == null ? 0 : element.hashCode());

        return result;
    }


ImportError: cannot import name 'patched_decoder' from 'src'

Hi,
I am trying to get it installed on Ubuntu 22.04 and followed the steps to create conda environment in GPU, installed all the pre-reqs and then when trying to import the timesfm, it gives following error:
(tfm_env) ubuntu@ip-10-218-241-180:~/timesfm$ ls
LICENSE    init.py  docs             environment_cpu.yml  pyproject.toml
README.md  datasets     environment.yml  experiments          src
(tfm_env) ubuntu@ip-10-218-241-180:~/timesfm$ ls -ltr
total 52
-rw-rw-r-- 1 ubuntu ubuntu   231 May 11 06:28 environment_cpu.yml
-rw-rw-r-- 1 ubuntu ubuntu   318 May 11 06:28 environment.yml
drwxrwxr-x 2 ubuntu ubuntu  4096 May 11 06:28 docs
drwxrwxr-x 2 ubuntu ubuntu  4096 May 11 06:28 datasets
-rw-rw-r-- 1 ubuntu ubuntu   763 May 11 06:28 init.py
-rw-rw-r-- 1 ubuntu ubuntu  6207 May 11 06:28 README.md
-rw-rw-r-- 1 ubuntu ubuntu 11358 May 11 06:28 LICENSE
-rw-rw-r-- 1 ubuntu ubuntu   696 May 11 06:28 pyproject.toml
drwxrwxr-x 4 ubuntu ubuntu  4096 May 11 06:28 experiments
drwxrwxr-x 4 ubuntu ubuntu  4096 May 11 06:33 src
(tfm_env) ubuntu@ip-10-218-241-180:~/timesfm$ python
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import timesfm
Traceback (most recent call last):
File "", line 1, in
File "/home/ubuntu/timesfm/src/timesfm.py", line 38, in
from src import patched_decoder
ImportError: cannot import name 'patched_decoder' from 'src' (/home/ubuntu/.local/lib/python3.10/site-packages/src/init.py)

simple:
$ cd /timesfm/src/
$ python
>>> import timesfm
...

Or
cd /timesfm/
pip install -e .

simple:
$ cd /timesfm/src/
$ python
>>> import timesfm
...

Or
cd /timesfm/
pip install -e .

Yes tried that already. Same error.

same exact issue here

You must run the python file with the conda environment. You cannot import timesfm elsewhere because it is only installed to the conda environment.
conda activate tfm_env

You must run the python file with the conda environment. You cannot import timesfm elsewhere because it is only installed to the conda environment.
conda activate tfm_env

The above issue is happening in conda too and without conda too.

Okay I see, I had the same issue. I believe it goes away when you append the timesfm folder directly to the python code and run with conda. Add the following lines:
import sys
sys.path.append('/home/kasm-user/Documents/timesfm') # The clone dir

Okay I see, I had the same issue. I believe it goes away when you append the timesfm folder directly to the python code and run with conda. Add the following lines:
import sys
sys.path.append('/home/kasm-user/Documents/timesfm') # The clone dir

(tfm_env) ubuntu@ip-10-218-241-180:~/timesfm$ python3
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import sys
sys.path.append('/home/ubuntu/timesfm')
import timesfm
Traceback (most recent call last):
File "", line 1, in
File "/home/ubuntu/timesfm/src/timesfm.py", line 38, in
from src import patched_decoder
ImportError: cannot import name 'patched_decoder' from 'src' (/home/ubuntu/.local/lib/python3.10/site-packages/src/init.py)

KeyboardInterrupt

Still doesnt work.

I encountered the same issue. The simplest solution is to create your first Python file in the root directory of the timesfm repository. After setting up, activating, and installing your Conda environment, you can run the file from there. Whilst the syspath solution should have worked, for some reason it didnt for me.

Sorry which first Python file?

Sorry which first Python file?

remove what I said from timesfm.py. You should be importing timesfm into a python script outside of the folder. If you get the patched_decoder error, add the lines I said to the start.

For me I solve this error by just running the py file from shell didn't try any other IDE still pretty weird

Still doesn't work. I think I will wait for a stable and proper version

Still doesn't work. I think I will wait for a stable and proper version

I think I found a fix-all. Navigate to the timesfm directory, then to the src folder and edit the timesfm.py code. Inject the following lines:
import sys
sys.path.append(path.dirname(path.abspath(__file__)))

So the final import structure for patched_decoder should look like this:
from praxis.layers import transformers

import sys
sys.path.append(path.dirname(path.abspath(__file__)))

import patched_decoder

from . import patched_decoder
This fixed it

No it doesn't fix it. Still same error. This model should never have been released in this buggy state. At least do the basic testing before releasing it. Very disappointing. Not wasting any more time on this.

No it doesn't fix it. Still same error. This model should never have been released in this buggy state. At least do the basic testing before releasing it. Very disappointing. Not wasting any more time on this.

The model works for me.
You probably don't know how to import the package correctly.

No it doesn't fix it. Still same error. This model should never have been released in this buggy state. At least do the basic testing before releasing it. Very disappointing. Not wasting any more time on this.

The model works for me. You probably don't know how to import the package correctly.

No it doesn't work, and I know how to import stuff. Are you one of the developer? Anyway instead of judging me, would have been better if you have shown your whole code here which 'works'.

Still doesn't work. I think I will wait for a stable and proper version

I think I found a fix-all. Navigate to the timesfm directory, then to the src folder and edit the timesfm.py code. Inject the following lines:

import sys

sys.path.append(path.dirname(path.abspath(__file__)))

So the final import structure for patched_decoder should look like this:

from praxis.layers import transformers

import sys

sys.path.append(path.dirname(path.abspath(__file__)))

import patched_decoder

This fixes the issue. Tested on multiple environments.


Incorrect /play Responses

E.g. if the song fails (e.g. super long video), incorrect message is shown.

/play song: https://youtube.com/watch?v=X0RK2jz5HOI
Fixed in V3.


Exception when computing Alt Comp Report

When computing the alt comp report for the Peres study I ran into the following exception:

The issue is that the observation number is NaN.
From the documentation:

I think there is an existing issue on this. I’m trying to fix it now. Let’s hope I have hair left once I do.

On Jul 6, 2022, at 10:01 AM, codymccoy @.***> wrote:

When computing the alt comp report for the Peres study I ran into the following exception:
The issue is that the observation number is NaN.
From the documentation:
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were assigned.

woops this is the existing issue!

the work on this issue has started with the alternative compute, next follows alternative comparison report, where we are producing results for dam cat - asset cat - impact area combinations that have a histogram of zero damage


Inherit scores for "Don't know" and "Don't care"

Fixes #32
Inherited keys may take slightly more time to update.
Performance for non-inheriting keys is fine.

The bottleneck is PieChart::render for all updated summaries.

It's only re-rendering the visible DOM, which more or less fixes issue #43.


S3 performance: add in-place (mount) vs copy support to brokers

This issue is primarily intended to address performance of jobs utilizing small portions (<25%) of extremely large files (10+ GiB). When using an S3 broker it presently will pull the entire file regardless of the job requirement.
This change will allow Scale to support S3 processing via a host mounted file system. The file system implementation must be POSIX compliant, at least as far as read is concerned. Testing will be done using goofys and s3fs.
The anticipated code changes required are:

Update Job Type schema to allow input_data file type objects to specify a member boolean with key 'partial'. A value of True indicates that file may be processed in-place (mounted), while False indicates that it should be copied into the input data volume.
Add support to the S3 Broker to support Docker volume mounting of the host mount of the S3 bucket for input_data files with 'partial' memeber set as True.
Add the ability for a workspace to define the host path that contains the mount of an S3 bucket. This path may be omitted which will cause the broker to always fall-back to download behavior.

Documentation Updates:

Host and NFS brokers should mention that they automatically support partial file reading. S3 broker supports partial file reading if an S3 mount is set up on each host (at least briefly explain how this is set up).

Default value for partial should be set to false


Rendering problems on Linux

I have been using PapyrusCS on my laptop (windows 10) and it renders fine. Now I am trying to run it on a server (in a Docker container) and I get strange looking rendering:

Same world rendered on Windows looks like this:

An update on this. The rendering problems described above where using the pre-built binaries for Linux. I took the source and built it instead and now it renders correctly. It also renders much, much faster.....at least an order of magnitude faster. So maybe there is an issue with the pre-built binaries?

Closing this for now, since I found a solution.


FadingActionBar with FragmentTabHost

Can I use FadingActionBar with FragmentTabHost?
I have FragmentTabHost with three Fragments and each has ListView. Above Fragment tabs, I have header layout which I want to parallax hide on ListView scroll.

Yes. It can be possible by placing header layout (where you want to parallax effect) within parent fragment which in your case is "FragmentTabHost". then you can simply make effect using this:
FadingActionBarHelper mFadingHelper = new FadingActionBarHelper()
.actionBarBackground(R.drawable.ab_background_light)
.headerLayout(R.layout.header)
.contentLayout(R.layout.fragment_tab_host)
.lightActionBar(true);
Note: make sure your three fragments have listview are in fragment_tab_host java class. even a better approach to use a viewpager within FragmentTabHost to hold all three fragment (List fragments).

Thanks for your response. I am not getting one thing here. What do you mean by "Note: make sure your three listview fragments are in fragment_tab_host's java class."
Here is my setup...
mTabHost = (FragmentTabHost) findViewById(android.R.id.tabhost);
mTabHost.setup(getActivity(), getChildFragmentManager(), R.id.tab_fragments_holder);
mTabHost.addTab(tabSpec, Fragment1.class, args);
mTabHost.addTab(tabSpec, Fragment2.class, args);
mTabHost.addTab(tabSpec, Fragment3.class, args);
each fragment has different layout file.. like fragment_layout1.xml, fragment_layout2.xml
is this setup correct?

I am still stuck on this. Any help would be great help.


As a potential first-time user of GEOPM I would like a simple workflow to try out the software

Is your feature request related to a problem? Please describe.
When someone tells me about an interesting software project I haven't used, I sometimes try to install and run the software to try it out. It is difficult for a new user of GEOPM to discover the correct steps to use GEOPM for the first time.
In this scenario, I am a user who is familiar with basic HPC software concepts. I am capable of building and installing simple software packages, but I might not care to go into all of the details of build configuration and system configuration. I just want to "kick the tires" and try out the software.
The following are 3 workflows I might follow to attempt to discover what I can do with GEOPM on my system:
Workflow 1: Browse the documentation:

Start at Documentation home
Click Getting Started. It doesn't really say how to get started, but points to either publications or use cases. Use cases seems closer to what I'm trying to do.
Nothing about getting started. Click Next.
I reached the publications page. Click Next.
Click "Guide to Installation". It shows how to install the service easily from package repositories! I also learn that this is only part of GEOPM, and the other part (the HPC runtime) must be built from source if I want a version that is compatible with the GEOPM service.

Workflow 2: Search the documentation:

Start at Documentation home, then search "install"
Top result is geopmpy(7), which explains how to install the python wrappers for the runtime components. The wrapped components are not installed.
The second and third results are probably better fits (Service installation, and source builds).

Workflow 3: Start from GitHub

I found GEOPM's GitHub repo. There is no README.md section with copy/paste steps to build and run. But I see an autogen.sh, so I'll assume it's an autotools project.
I blindly guess that this will build a version of GEOPM with some viable default settings ./autogen.sh && ./configure --prefix=$HOME/tryoutgeopm && make -j && make install
That didn't work because I need to build the service first.

Describe the solution you'd like
I would like for each of the outlined workflows to reveal some copy/paste command sequence that results in a usable version of GEOPM. The closer this sequence is to step 1, the better.

I think all of the required details already exist in the documentation, but some reorganization would help with discoverability.


Send out announcement about Pages transition

In order to inform our customers about the start of the transition. We need to send out emails, add documentation the marketing site, add news post to cloud.gov, and add links to the comms from the app.
Acceptance Criteria

[x] Finalize draft via pages comms working doc and/or other doc
[x] Send out transition info in cg newsletter
[ ] Send email to current customer list
[ ] Add news post to cloud.gov about Pages
[ ] Add docs in the marketing site
[ ] Add links in the pages/federalist app

Security Considerations
Needed to complete SCR

Moving into ongoing since this will progress over the next month.

Completed and broken out into smaller issues.


Feature Request: ignoring selected portals, and not showing it on the map

Sometimes when showing the "Tidy Links" layer (max-links.user.js), the links went to a portal I didn't want to. So it will be very helpful if ignoring portals is available.

There's a pull request that adds this feature.


ui: query with multiple yields, cannot choose which yield to view as table

Using commit 8de4517.
Here is my sample query:
mybucket = "my_bucket"

from(bucket: mybucket)
  |> range(start: -15m)
  |> keep(columns:["_measurement"])
  |> filter(fn: (r) => r._measurement =~ /^h/)
  |> unique(column: "_measurement")
  |> group()
  |> yield(name: "h measurements")

from(bucket: mybucket)
  |> range(start: -15m)
  |> keep(columns:["_measurement"])
  |> filter(fn: (r) => r._measurement =~ /^q/)
  |> unique(column: "_measurement")
  |> group()
  |> yield(name: "q measurements")

The raw result of this, according to the chrome network inspector, is
#group,false,false,false
#datatype,string,long,string
#default,q measurements,,
,result,table,_measurement
,,0,qxxx

#group,false,false,false
#datatype,string,long,string
#default,h measurements,,
,result,table,_measurement
,,0,hxxx

When I render this result as a table view, there is a picker for the table but I can't choose other than whatever view happens to be displayed. Viewing the raw data displays correctly.

I think this is still an issue.


cuelang.org: "Create Documentation Issue" link broken

As of https://github.com/cue-lang/cuelang.org/tree/15f879b2ae82de366f61dbab033179848c8400ec.
For example on page https://cuelang.org/docs/about/#philosophy-and-principles.
Top right hand corner there are a sequence of links:

The "Create Documentation Issue" link is broken. It tries to link to cue-lang/cuelang.org to create an issue. But we have intentionally disabled issues in that repo in favour of tracking all issues in this repo, cue-lang/cue.
The fix here should be to link to the main issue tracker with a title prefix of cuelang.org: and the cuelang.org label (just like this issue).
Indeed the main repo needs an issue/bug template dedicated to reports from cuelang.org.

Given that we are moving along nicely with https://alpha.cuelang.org, I'm tempted to not fix this on the current (soon to be old) site. See https://github.com/cue-lang/cue/issues/2446

I found this bug because I was about to file it myself.  As a person doing a first kind of speculative poke around the cue docs, it is surprising to me to see this unfixed for ~8 months!

This is redundant as of the new https://cuelang.org where there is a link at the bottom of each page that does work: