Log records starts with ERROR: logging before flag.Parse:

K: 1.16.0
v0.0.11
now log looks like:
time="2019-10-02T14:23:12Z" level=debug msg="Applied config: {\"nodePathMap\":[{\"node\":\"DEFAULT_PATH_FOR_NON_LISTED_NODES\",\"paths\":[\"/opt/local-path-provisioner\"]}]}"
time="2019-10-02T14:23:12Z" level=debug msg="Provisioner started"
ERROR: logging before flag.Parse: I1002 14:23:12.817611       1 leaderelection.go:187] attempting to acquire leader lease  local-path-storage/rancher.io-local-path...
ERROR: logging before flag.Parse: I1002 14:23:12.894052       1 leaderelection.go:196] successfully acquired lease local-path-storage/rancher.io-local-path
ERROR: logging before flag.Parse: I1002 14:23:12.894163       1 event.go:221] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"local-path-storage", Name:"rancher.io-local-path", UID:"b8b35657-ea50-4a08-a1a2-e63087b44ffe", APIVersion:"v1", ResourceVersion:"663678", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' local-path-provisioner-56db8cbdb5-6lfgx_2a9d3a0c-e520-11e9-95fc-865370074a10 became leader
ERROR: logging before flag.Parse: I1002 14:23:12.894239       1 controller.go:572] Starting provisioner controller rancher.io/local-path_local-path-provisioner-56db8cbdb5-6lfgx_2a9d3a0c-e520-11e9-95fc-865370074a10!
ERROR: logging before flag.Parse: I1002 14:23:12.994573       1 controller.go:621] Started provisioner controller rancher.io/local-path_local-path-provisioner-56db8cbdb5-6lfgx_2a9d3a0c-e520-11e9-95fc-865370074a10!

same question

I have created a pull request, which should potentially fix this issue.

This issue was already fixed when adopting a newer version of client-go using klog instead of glog anymore.
Closing, if have any question, feel free to reopen.

This issue was already fixed when adopting a newer version of client-go using klog instead of glog anymore.
Closing, if have any question, feel free to reopen.


[SPARK-44571][SQL] Eliminate the Join by combine multiple Aggregates

What changes were proposed in this pull request?
Some queries contains multiple scalar subquery(aggregation without group by clause) and connected with join. The general form of joined aggregates that can be merged as follows:
<aggregation function> ::=
  SUM | AVG | MAX | ...

<aggregation subquery> ::=
  SELECT
    <aggregation function>(...)[ , <aggregation function>(...)[ , ...]]
  FROM [tab | query]

<joined aggregation > ::=
  SELECT *
  FROM (
    <aggregation subquery> [INNER | CROSS | LEFT | RIGHT | FULL OUTER] JOIN
    <aggregation subquery> [INNER | CROSS | LEFT | RIGHT | FULL OUTER] JOIN
      ...
    <aggregation subquery> [INNER | CROSS | LEFT | RIGHT | FULL OUTER] JOIN
    <aggregation subquery>
  )

For example,
SELECT *
FROM (SELECT
  avg(power) avg_power,
  count(power) count_power,
  count(DISTINCT power) count_distinct_power
FROM data
WHERE country = "USA"
  AND (id BETWEEN 1 AND 3
  OR city = "Berkeley"
  OR name = "Xiao")) B1,
  (SELECT
    avg(power) avg_power,
    count(power) count_power,
    count(DISTINCT power) count_distinct_power
  FROM data
  WHERE country = "China"
    AND (id BETWEEN 4 AND 5
    OR city = "Hangzhou"
    OR name = "Wenchen")) B2

We can optimize this SQL to the form shown below:
SELECT
  avg(power) avg_power FILTER (country = "USA" AND (id BETWEEN 1 AND 3 OR city = "Berkeley" OR name = "Xiao")),
  count(power) count_power FILTER (country = "USA" AND (id BETWEEN 1 AND 3 OR city = "Berkeley" OR name = "Xiao")),
  count(DISTINCT power) FILTER (country = "USA" AND (id BETWEEN 1 AND 3 OR city = "Berkeley" OR name = "Xiao")),
  avg(power) FILTER (country = "China" AND (id BETWEEN 4 AND 5 OR city = "Hangzhou" OR name = "Wenchen")),
  count(power) FILTER (country = "China" AND (id BETWEEN 4 AND 5 OR city = "Hangzhou" OR name = "Wenchen")),
  count(DISTINCT power) FILTER (country = "China" AND (id BETWEEN 4 AND 5 OR city = "Hangzhou" OR name = "Wenchen"))
FROM data
WHERE
(country = "USA"
  AND (id BETWEEN 1 AND 3
  OR city = "Berkeley"
  OR name = "Xiao")) OR
(country = "China"
    AND (id BETWEEN 4 AND 5
    OR city = "Hangzhou"
    OR name = "Wenchen"))

If we can merge the filters and aggregates, we can scan data source only once and eliminate the join so as avoid shuffle.
This PR also supports eliminate nested Join, please refer to: https://github.com/apache/spark/blob/master/sql/core/src/test/resources/tpcds/q28.sql
Obviously, this change will improve the performance.
This PR also reuse some functions come from MergeScalarSubqueries.
This PR also add some TreePattern for easy to check the cost of predicate.
Why are the changes needed?
Improve the performance for the case show above.
Does this PR introduce any user-facing change?
'No'.
New feature.
How was this patch tested?

new test cases
new micro benchmark.

Benchmark CombineJoinedAggregates:                     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
-------------------------------------------------------------------------------------------------------------------------------------
filter is not defined, CombineJoinedAggregates: false            730            819          69         28.7          34.8       1.0X
filter is not defined, CombineJoinedAggregates: true             618            632          14         33.9          29.5       1.2X
step is 1000000, CombineJoinedAggregates: false                  572            590          20         36.7          27.3       1.3X
step is 1000000, CombineJoinedAggregates: true)                  769            794          21         27.3          36.6       1.0X
step is 100000, CombineJoinedAggregates: false                   350            370          26         59.9          16.7       2.1X
step is 100000, CombineJoinedAggregates: true)                   231            241          10         90.7          11.0       3.2X
step is 10000, CombineJoinedAggregates: false                    314            340          26         66.8          15.0       2.3X
step is 10000, CombineJoinedAggregates: true)                    171            182           9        122.5           8.2       4.3X
step is 1000, CombineJoinedAggregates: false                     303            337          32         69.3          14.4       2.4X
step is 1000, CombineJoinedAggregates: true)                     162            171           9        129.4           7.7       4.5X
step is 100, CombineJoinedAggregates: false                      300            316          27         70.0          14.3       2.4X
step is 100, CombineJoinedAggregates: true)                      160            169           9        131.3           7.6       4.6X
step is 10, CombineJoinedAggregates: false                       297            325          33         70.6          14.2       2.5X
step is 10, CombineJoinedAggregates: true)                       170            203          36        123.5           8.1       4.3X
step is 1, CombineJoinedAggregates: false                        328            352          17         64.0          15.6       2.2X
step is 1, CombineJoinedAggregates: true)                        140            148           7        149.3           6.7       5.2X

Benchmark CombineJoinedAggregates:                     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
-------------------------------------------------------------------------------------------------------------------------------------
Tree node number < 1, CombineJoinedAggregates: true)             398            503         109         52.7          19.0       1.0X
Tree node number < 9, CombineJoinedAggregates: true)             394            432          31         53.2          18.8       1.0X
Tree node number < 19, CombineJoinedAggregates: true)            399            427          47         52.6          19.0       1.0X
Tree node number < 29, CombineJoinedAggregates: true)            434            479         100         48.3          20.7       0.9X
Tree node number < 39, CombineJoinedAggregates: true)            480            499          24         43.7          22.9       0.8X

manual test on TPC-DS
TPC-DS data size: 2TB.
This improvement is valid for TPC-DS q28 and no regression for other test cases.

TPC-DS Query
Before(Seconds)
After(Seconds)
Speedup(Percent)

q28
109.665
43.938
249.59%

According to the micro benchmark, this improvement is worse than before if the filter has almost no selectivity.

ping @cloud-fan @viirya cc @MaxGekk @gengliangwang

Hi @beliefer,
This is a nice optimization and awsome performance improvement to Q28, but I'm not sure the implementation is done the right way:

You probably know that MergeScalarSubqueries does a very similar plan merging in case of scalar subqueries so I feel this PR shouldn't reimplement the logic of MergeScalarSubqueries.tryMergePlans() but reuse the already available code and add the necessary improvements to it if needed. The plan merging code can also be extracted to a common place to share between the rules.
I guess probably the main reason why you reimplemented merge logic is to be able to propagate diferent filter conditions from Filter nodes up into Aggregate nodes as that is required for Q28, but currently that feature is missing from MergeScalarSubqueries.tryMergePlans(). IMO that improvement should be added to the existing code because Q9 could also benefit from that. Please note that I've already tried to add that in SPARK-40193 / https://github.com/apache/spark/pull/37630.
Unfortunately I think this feature can be tricky as there might be cases when merging queries can introduce prformance degradation. E.g. such bad case is when we merge SELECT sum(a) FROM t WHERE p = 1 and  SELECT sum(b) FROM t WHERE p = 2 into SELECT sum(a) FILTER (p = 1), sum(b) FILTER (p = 2) FROM t WHERE p = 1 OR p = 2 and p is a partitioning column as we need to process more, but we don't scan less data in the merged query. To avoid that I used a trick in https://github.com/apache/spark/pull/37630 to peek into the physical plan to check that only pushed data filters differ but partitioning and bucketing filters match. That trick made the implementation a bit complex, but we don't need to stick to that, we could also disable filter propagation during plan merging by default as this PR does.
Anyway, my point is that probably the feature should be added to a common place (MergeScalarSubqueries.tryMergePlans()).

So I would suggest narrowing the cope of SPARK-44571 / this PR to merge joined aggregate queries using the existing merge logic and SPARK-40193 to handle filter propagation during plan merging. I'also happy to revive https://github.com/apache/spark/pull/37630 if there is any intterrest.

@peter-toth Thank you for the review.
For your first point, we surely could reuse some function if there are some code is similar. If SPARK-40193 / https://github.com/apache/spark/pull/37630. could be merged first, I will reuse them. Otherwise, this PR merged first, I will create follow up PR for reuse.
For your second point, I'm OK to pass Filter nodes up into Aggregate nodes if the tryMergePlans can be reused. The reason about prformance degradation is Filter haven't better or higher selective. Please refer the benchmark in the description. So I added the config and the default value is false.
Your suggestion is welcome. But I think what's the chance to reuse them? If https://github.com/apache/spark/pull/37630 merged first, I will reuse them directly. Otherwise, I will create follow up PR to fix.

@cloud-fan The CI failure is unrelated.

Instead of showing an example query, can you define the general form of joined aggregates that can be merged?

BTW, in my https://github.com/apache/spark/pull/37630 I used a different heuristics to disable merging of aggregates with > different filter conditions. If the conditions contain any partitioning or bucketing columns then aggregates are not merged.

@peter-toth Could you tell me more? I can't found the treat for partitioning or bucketing columns.

BTW, in my #37630 I used a different heuristics to disable merging of aggregates with > different filter conditions. If the conditions contain any partitioning or bucketing columns then aggregates are not merged.

@peter-toth Could you tell me more? I can't found the treat for partitioning or bucketing columns.

That heuristics made the whole PR complex. You can follow the logic of ScanCheck object and the case case (CHECKING, FileSourceScanPlan(_, newScan), FileSourceScanPlan(_, cachedScan)) => (https://github.com/apache/spark/pull/37630/files#diff-3d3aa853c51c01216a7f7307219544a48e8c14fabe1817850e58739208bc406aR277-R290) in tryMergePlans(). That case actually peeks into the physical plan to check if only pushed-down data filters differ (partitioning and bucketing filters do match).
BTW, I'm not saying that it is the right heuristics to decide if we should merge aggregates with different filters, it is just the one I was able to come up with...
Anyways, expecting highly selective predicates seems a bit counter intuitive to me. And as I mentioned I'm also fine with disabling the feature with a config by default and let the users enable it for some of queries that benefit from it.

@beliefer, I've upodated my PR with comments to ellaborate on how it handles Filters: https://github.com/apache/spark/pull/37630/files#diff-3d3aa853c51c01216a7f7307219544a48e8c14fabe1817850e58739208bc406aR351-R524

         case (CHECKING, FileSourceScanPlan(_, newScan), FileSourceScanPlan(_, cachedScan)) =>
           val (newScanToCompare, cachedScanToCompare) =
             if (conf.getConf(SQLConf.PLAN_MERGE_IGNORE_PUSHED_PUSHED_DATA_FILTERS)) {
               (newScan.copy(dataFilters = Seq.empty), cachedScan.copy(dataFilters = Seq.empty))
             } else {
               (newScan, cachedScan)
             }
           if (newScanToCompare.canonicalized == cachedScanToCompare.canonicalized) {
             // Physical plan is mergeable, but we still need to finish the logical merge to
             // propagate the filters
             tryMergePlans(newPlan, cachedPlan, DONE)
           } else {
             None
           }

I think the above code is not needed. Generally, we concat the predicates with OR, the origin filters still could be pushed down to file sources.

         case (CHECKING, FileSourceScanPlan(_, newScan), FileSourceScanPlan(_, cachedScan)) =>
           val (newScanToCompare, cachedScanToCompare) =
             if (conf.getConf(SQLConf.PLAN_MERGE_IGNORE_PUSHED_PUSHED_DATA_FILTERS)) {
               (newScan.copy(dataFilters = Seq.empty), cachedScan.copy(dataFilters = Seq.empty))
             } else {
               (newScan, cachedScan)
             }
           if (newScanToCompare.canonicalized == cachedScanToCompare.canonicalized) {
             // Physical plan is mergeable, but we still need to finish the logical merge to
             // propagate the filters
             tryMergePlans(newPlan, cachedPlan, DONE)
           } else {
             None
           }

I think the above code is not needed. Generally, we concat the predicates with OR, the origin filters still could be pushed down to file sources.

Please comment on the other PR regarding the code of that PR.
But my point is that it doesn't matter how the filter looks like (is it an OR condition or not). I enabled merging if only FileSourceScanExec.dataFilters differ between the 2 scans. If FileSourceScanExec.partitionFilters or FileSourceScanExec.optionalBucketSet differ then merging is disabled because partitioning and bucketing filters can be more selective in terms what files to scan...

But my point is that it doesn't matter how the filter looks like (is it an OR condition or not). I enabled merging if only
FileSourceScanExec.dataFilters differ between the 2 scans. If FileSourceScanExec.partitionFilters or
FileSourceScanExec.optionalBucketSet differ then merging is disabled because partitioning and bucketing filters can be
more selective in terms what files to scan...

In theory, whether it is data filters or partition filters, there is a possibility of data overlap when connected filters with or.
Before merge the filters (e.g. p = 1, p = 2), assume each partition have one file, so we need to read two partition files.
After merge the two filters, we still need to read two partition files.
I think the overhead of scan partition files is the same. the different is the filter need to calculates more. e.g. p = 1 also need to treat the data come from p = 2.
So, personally, I think the overhead of calculate is similar, no matter which filter is.

But my point is that it doesn't matter how the filter looks like (is it an OR condition or not). I enabled merging if only
FileSourceScanExec.dataFilters differ between the 2 scans. If FileSourceScanExec.partitionFilters or
FileSourceScanExec.optionalBucketSet differ then merging is disabled because partitioning and bucketing filters can be
more selective in terms what files to scan...

In theory, whether it is data filters or partition filters, there is a possibility of data overlap when connected filters with or. Before merge the filters (e.g. p = 1, p = 2), assume each partition have one file, so we need to read two partition files. After merge the two filters, we still need to read two partition files. I think the overhead of scan partition files is the same. the different is the filter need to calculates more. e.g. p = 1 also need to treat the data come from p = 2. So, personally, I think the overhead of calculate is similar, no matter which filter is.
The main reason for filter merging is the amount of overlapping data. For example, F1 obtains 100 rows of data, and F2 obtains 50 rows of data. If the 100 rows and 50 rows completely overlap, this is the best situation. F1 on Aggregate1 still processes 100 rows of data, while F2 on Aggregate2 processes an additional 50 rows, resulting in a total of 100 rows of data. The worst case scenario is that the two do not overlap at all. So F1 on Aggregate1 needs to process an additional 50 rows, a total of 150 rows; F2 on Aggregate2 processes an additional 100 rows, totaling 150 rows.

Sorry @beliefer, I didn't explain all the reasoning behind my heuristics in https://github.com/apache/spark/pull/37630. I've updated https://github.com/apache/spark/pull/42223#discussion_r1282023520, please see the details there.

For merging func1(...) ... WHERE cond1 and func2(...) ... WHERE cond2, we got
func1(...) FILTER cond1, func2(...) FILTER cond2 ... WHERE cond1 OR cond2

Assuming there is no overlapped scan (so almost no benefit), and the table has N rows. Previously, both cond1 and cond2 get evaluated at most N times (the scan gets prunned). If they are partition filters, then they are evaluated 0 times. Now, they get evaluated at most 2N times. It's likely more than 2 times than before as we prune less data from the scan. The worst case is partition filters.  I don't think putting the predicates in a Project helps as the problem is from scan prunning.
Given that it's hard to estimate the overlapped scan size, the only idea I can think of is to estimate the cost of the predicate. We only do the merge if the predicate is cheap to evaluate: only contains simple comparison and the expression tree size is small.
We can define some patterns when the overlapped scan size must be large, e.g. one side has no filter, or the filters in two aggregates are the same. For these cases, we can always apply the merge.

I don't think putting the predicates in a Project helps as the problem is from scan prunning.

Sorry, my first comment about the extra project (https://github.com/apache/spark/pull/42223#discussion_r1283307075 was confusing. I got that argument, and I agreed with you (https://github.com/apache/spark/pull/42223#discussion_r1283361954).
But if we decided to merge the queries (based on any heuristics) then the extra project can help to avoid evaluating the filters 2 times in the merged query and actually can decrease data to be shuffled for the aggregation: https://github.com/apache/spark/pull/42223#discussion_r1284057348. IMO the extra project beween the filter and scan won't prevent the merged condition to be pushed-down to the scan so i don't see any drawbacks of it.

Given that it's hard to estimate the overlapped scan size, the only idea I can think of is to estimate the cost of the predicate. We only do the merge if the predicate is cheap to evaluate: only contains simple comparison and the expression tree size is small.

That makes sense to me and I don't have any better idea. Although I still think we can and should check it any filters are pushed down to scans in the original queries. If there is no pushed-down filters or pushed-down filters fully match then we are safe to merge as scans fully overlap. If there are non-matching pushed-down filters then we can use the suggestged expression is cheap heuristics.

@peter-toth I agree that the extra project can help if we decided to merge. However, the plan pattern becomes complicated. Without the extra project, the merged aggregate is still Aggregate -> Filter -> Scan. We can just define a rule for merging two aggregates, and it can incrementally merge all joined aggregates.  With the extra project, we need to define how to merge Aggregate -> Filter -> Project -> Scan + Aggregate -> Filter -> Project -> Scan, or Aggregate -> Filter -> Project -> Scan + Aggregate -> Filter -> Scan
I think a few extra boolean columns won't increase the shuffle size too much. When we have a partial aggregate logical plan, we can strip the added boolean columns right after partial aggregate, but before final aggregate, as the aggregate function filter is only evaluated in partial aggregate.

@peter-toth I agree that the extra project can help if we decided to merge. However, the plan pattern becomes complicated. Without the extra project, the merged aggregate is still Aggregate -> Filter -> Scan. We can just define a rule for merging two aggregates, and it can incrementally merge all joined aggregates. With the extra project, we need to define how to merge Aggregate -> Filter -> Project -> Scan + Aggregate -> Filter -> Project -> Scan, or Aggregate -> Filter -> Project -> Scan + Aggregate -> Filter -> Scan.

Those patterns are already covered in the current code: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/MergeScalarSubqueries.scala#L235-L255 and not touched in my aggregate merge PR: https://github.com/apache/spark/pull/37630/files#diff-3d3aa853c51c01216a7f7307219544a48e8c14fabe1817850e58739208bc406aR292-R318

Also note that, WHERE cond1 OR cond2 can be better than FROM (SELECT cond1 AS b_col1, cond2 AS b_col2) WHERE b_col1 OR b_col2 , because OR has shortcut and cond2 is not always evaluated for each input row. common subexpression elimination might be a better approach here, if we can make it cross the operator boundary (or whole-stage-codegen can only do it?)

I'm nost sure WHERE cond1 OR cond2 is better because up in the aggregate cond2 is always evaluated for all rows.

@peter-toth Setting aside the previous discussion, after all, the current implementation is the simplest pattern, which is also an advantage that cannot be ignored. If adding a project can bring performance improvement, we can follow up and improve.

@peter-toth Setting aside the previous discussion, after all, the current implementation is the simplest pattern, which is also an advantage that cannot be ignored. If adding a project can bring performance improvement, we can follow up and improve.

If we want to push this PR for some reason I'm not against it if the above mentioned correctness issue gets fixed (https://github.com/apache/spark/pull/42223#discussion_r1279051263).
But honestly, I don't like the idea that we duplicate the code of MergeScalarSubqueries.tryMergePlans() with some modifications into this new rule just because we don't want to fix agregate merging there. IMO if we all on the same page about the discussed solution in https://github.com/apache/spark/pull/42223#issuecomment-1665208447 then we should update MergeScalarSubqueries.tryMergePlans() (in https://github.com/apache/spark/pull/37630) and use the updated common merging logic in this PR.

If we want to push this PR for some reason I'm not against it if the above mentioned correctness issue gets fixed (https://github.com/apache/spark/pull/42223#discussion_r1279051263).

Fixed.

But honestly, I don't like the idea that we duplicate the code of MergeScalarSubqueries.tryMergePlans() with some
modifications into this new rule just because we don't want to fix agregate merging there. IMO if we all are on the same
page about the discussed solution in https://github.com/apache/spark/pull/42223#issuecomment-1665208447 then we
should update MergeScalarSubqueries.tryMergePlans() (in https://github.com/apache/spark/pull/37630) and use the
updated common merging logic in this PR.

I'm also don't like the idea as you said. But it seems the code is different between this PR and https://github.com/apache/spark/pull/37630. Maybe we should advance these two PRs separately until a shared opportunity is discovered, and then reconstruct both.

But honestly, I don't like the idea that we duplicate the code of MergeScalarSubqueries.tryMergePlans() with some
modifications into this new rule just because we don't want to fix agregate merging there. IMO if we all are on the same
page about the discussed solution in #42223 (comment) then we
should update MergeScalarSubqueries.tryMergePlans() (in #37630) and use the
updated common merging logic in this PR.

I'm also don't like the idea as you said. But it seems the code is different between this PR and #37630. Maybe we should advance these two PRs separately until a shared opportunity is discovered, and then reconstruct both.

IMO the codes are very similar and the basic algorightm is identical. Both methods are just traversing the plan and comparing nodes if/how they can be merged.
I'm off for a week, but once returned I can update https://github.com/apache/spark/pull/42223 with the discussed and maybe remove the physical scan equality check part. Although that part is useful as in some cases we don't need to use any heuristics to allow merging (https://github.com/apache/spark/pull/42223#issuecomment-1665208447), it makes my PR complex. Maybe we can add it back later...
BTW this PR still doesn't use the "condition expression cost is low" heuristics that we discussed. The current "isLikelySelective" doesn't make sense (https://github.com/apache/spark/pull/42223#discussion_r1281846237) and has nothing to do with expression cost.

I'm not sure if we can reuse https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/MergeScalarSubqueries.scala#L235-L255 directly. If the Project is not generated by this optimization, it might contain other expensive expressions and we should stop merging.

Although that part is useful as in some cases we don't need to use any heuristics to allow merging
(https://github.com/apache/spark/pull/42223#issuecomment-1665208447), it makes my PR complex. Maybe we can add it > back later...

I agree.

ping @cloud-fan @peter-toth Could we continue to carry the work a step forward ?

Sorry, last week was a bit hectic for me. I've already put together a change to https://github.com/apache/spark/pull/37630 to make it simpler and contain the discussed, but haven't got time to update the PR. I will do it soon.


Update WalletLink to 2.1.0 and updates the Coinbase Wallet icon

This PR updates WalletLink to the latest version (2.1.0) and updates the Coinbase Wallet icon
New icon:

@vishnumad Thanks for the PR! I am going to test and get this into develop branch tomorrow 👍🏿

Rebased off develop. Thanks for reviewing!


Support for the Auto Save feature of OSX

Most native OSX text editors take advantage of a OSX feature called "Auto Save" explained at https://support.apple.com/en-us/HT202255.
Does Electron offer support for that feature?
Thanks.

If you mean adding it in Atom, there is already some packages doing that, if you mean adding it in your Electron app, you can reference how those packages did that.

Thanks @zcbenz.
To be sure that I understand, do you mean that there could be a package that works with the native "Auto Save" APIs of OSX or that there could be a package that has the "Auto Save" functionality but doesn't use the APIs of OSX?

If you mean the autosavingDelay API of NSDocumentController, then it is not supported in Electron. To use it we have to adopt the whole document model into Electron, which is not likely to happen since it is too much work to provide JavaScript binding and there is no corresponding APIs on other platforms.
If you really want to use it the only choice is to write native Node module for it.

Thanks @zcbenz I appreciate the quick response.


Multi-TPU Training Support for Composer

🚀 Feature Request: Multi-TPU Training Support for Composer
Add native support for multi-TPU training workflows within the Composer library. This would enable efficient scaling of large language model training across multiple TPU pods (specifically targeting TPU v3-8 initially).
Motivation
I'm often frustrated by the complexity of setting up multi-TPU training for large language models. While the MosaicML documentation provides guidance (https://docs.mosaicml.com/projects/composer/en/stable/examples/TPU_Training_in_composer.html), integrating this into existing Composer workflows is not straightforward.
Having built-in multi-TPU support within Composer would greatly simplify the process and make large-scale model training more accessible.
[Optional] Implementation

Abstraction Layer:  Create a new abstraction layer within Composer to handle multi-TPU communication and data parallelism.
Configuration: Provide a simple configuration interface (e.g., YAML or Python-based) to specify the number of TPU devices, pod placement, and other relevant settings.
Integration with Algorithms: Ensure seamless integration with existing Composer training algorithms and optimizers.
Example Scripts: Include example scripts demonstrating how to train models like Mosaic BERT on multi-TPU setups.

Additional context

The primary use case is to facilitate the training of language models (like Mosaic BERT) on TPU v3-8 pods.
The goal is to make multi-TPU training as simple and intuitive as single-device training within Composer.

We don't regularly use TPUs, so unfortunately the support is a bit worse. Can you highlight a few concrete pain points that need to be addressed? What exactly fails with the library as-is?

We don't regularly use TPUs, so unfortunately the support is a bit worse. Can you highlight a few concrete pain points that need to be addressed? What exactly fails with the library as-is?

Hmmm, I saw your version using torch xla, but I'm curious if it is possible to use libraries for TPUs such as flax, jax. To be honest, I haven't experimented with them yet because I don't seem to really imagine where I should start.
It seems a bit annoying but can you show me how to edit your framework?

Composer is PyTorch built, so moving to JAX would be a big lift. You'd have to rewrite the distributed logic, which is a lot of effort. It's certainly possible to do both... but currently we are opinionated in requiring PyTorch

Hmm, thank you.
It seems to me that with my current capabilities, switching a framework from torch to jax is still a complicated process. I will try again on another occasion.


Exception mapping

Added RestServiceExceptionFacade as described in issue https://github.com/devonfw-sample/devon4quarkus-reference/issues/12 for exception handling. Business exceptions are thrown through ApplicationBusinessException and return status 400 (bad request).  Other runtime exceptions will result in a response with status 500 (internal server error).
Also did some refactoring of the rest service and the database migration script.

@hohwille  thanks for your review comments. I have changed some parts accordingly. Can you please take again a look and check my comments?

@GuentherJulian  please check for the conflicts. thanks

@GuentherJulian please check for the conflicts. thanks

@maybeec Conflicts resolved.

@hohwille ,waiting for your approval


2.0.0null safety版本，报错「Don't use one refreshController to multiple SmartRefresher,It will cause some unexpected bugs mostly in TabBarView」

包含底部多个bottomNavigationBar页面，其中一个页面使用SmartRefresher，初次进入改页面正常，从此页面跳到其他bottomNavigationBar页面，然后返回此页面报错

Don't use one refreshController to multiple SmartRefresher,It will cause some unexpected bugs mostly in TabBarView
'package:pull_to_refresh/src/smart_refresher.dart':
Failed assertion: line 608 pos 12: '_refresherState == null'

1.6.4-nullsafety.0版本没有此错误
此页面之前不含TabBarView组件
主要代码如下：
return SmartRefresher(
  enablePullDown: true,
  enablePullUp: true,
  header: UIRefresherHeader(),
  footer: UIRefresherFooter(count: items.length, total: total),
  controller: controller.refreshController,
  onRefresh: controller.onRefresh,
  onLoading: controller.onLoading,
  child: items.length == 0
      ? UIRefresherEmpty()
      : ListView.builder(
    itemBuilder: (ctx, i) => ListItem(items[i]),
    itemCount: items.length,
  ),
);

any solution for this same problem with me after upgrade to flutter 2

i use provider pattern and i am using one controller before i upgrade to flutter 2 every thing run as expected but now after upgrade it's throw this error

@devmfouad Can you give an example code to reproduce(no third party library)?I have no time to check how to reproduce this,hope you can understand.

this issue example code doesn't show more details,so I don't know what the situation cause this.

I think this issue occurred when we refresh the whole widget. You can try using setstate{} to reproduce this issue.

same error when using PageView.builder and SmartRefresher after upgrade flutter 2.0

PageView.builder(
itemBuilder: (context, position) {
return return SmartRefresher();
},
itemCount: itemCount,
)

If you are using PageView or TabBarView crash this.You may use one RefreshController to mutiple SmartRefresher I have said before.just like this.
this is a wrong doing
  RefreshController _refreshController = RefreshController();

 ...

return PageView.builder(
      itemBuilder: (context, position) {
         return SmartRefresher(
           controller: _refreshController[position],
           child: Container(color: Colors.white,),
         );
      },
      itemCount: 4,
    );

you need to fix code to work
 List<RefreshController> _refreshController = [RefreshController(),RefreshController(),RefreshController(),RefreshController()];

return PageView.builder(
      itemBuilder: (context, position) {
         return SmartRefresher(
           controller: _refreshController[position],
           child: Container(color: Colors.white,),
         );
      },
      itemCount: 4,
    );

noted with thanks, this will be better?
return PageView.builder(
      itemBuilder: (context, position) {
         final RefreshController _refreshController = RefreshController(initialRefresh: false);
         return SmartRefresher(
           controller: _refreshController[position],
           child: Container(color: Colors.white,),
         );
      },
      itemCount: 4,
);

If you are using PageView or TabBarView crash this.You may use one RefreshController to mutiple SmartRefresher I have said before.just like this.
this is a wrong doing
  RefreshController _refreshController = RefreshController();

 ...

return PageView.builder(
      itemBuilder: (context, position) {
         return SmartRefresher(
           controller: _refreshController[position],
           child: Container(color: Colors.white,),
         );
      },
      itemCount: 4,
    );

you need to fix code to work
 List<RefreshController> _refreshController = [RefreshController(),RefreshController(),RefreshController(),RefreshController()];

return PageView.builder(
      itemBuilder: (context, position) {
         return SmartRefresher(
           controller: _refreshController[position],
           child: Container(color: Colors.white,),
         );
      },
      itemCount: 4,
    );

thanks this fix my problem
return PageView.builder(       itemBuilder: (context, position) {          final RefreshController _refreshController = RefreshController(initialRefresh: false);          return SmartRefresher(            controller: _refreshController[position],            child: Container(color: Colors.white,),          );       },       itemCount: 4, );

@peng8350   I had the same problem

I only have one page of the BottomBar that uses the SmartreFresher


remove support for EOT fonts

KaTeX supports all major browsers, including Chrome, Safari, Firefox, Opera, Edge, and IE 9 - IE 11.

WOFF is support of IE9+ and none of the other browsers support EOT so I'd like to suggest getting rid of EOT fonts.  Does anyone have any concerns about removing this font format?

Removal makes sense to me.

Reopening until second PR lands.

Thanks.  I didn't realize that commits to other repos could automatically close issues.


Changing source URL doesn't play subsequent videa

Bug

When I switch the video source from one valid url to another, the second video fails to play.  This also happens for all other subsequent videos.  In the code below, the user press a button in the parent component that causes step to change.
Platform
Which player are you experiencing the problem on:

Android however I haven't tested iOS yet

Environment info

Library version: 6.0.0-alpha.7

I also notice this bug on 5.2.1
Device: LG V50S ThinQ

Steps To Reproduce

Play first video
User action that changes the URL
Second video renders the first frame but fails to play even though the onLoad callback fires
...

Expected behaviour

The second video should play

Reproducible sample code

import { captureException } from "@sentry/react-native"
import { memo, useCallback, useMemo } from "react"
import { Text, View } from "react-native"
import Config from "react-native-config"
import Video from "react-native-video"

export type TutorialStep = {
  title: string
  subtext: string
  videoName: string
}
export type TutorialSlideProps = {
  step: TutorialStep
}

export const TutorialSlide = memo(function TutorialSlide({ step }: TutorialSlideProps) {
  const { title, subtext, videoName } = step

  const styles = createStyles(isLandscape())

  return (
    <View style={styles.tutorialCard}>
      <View style={styles.titleContainer}>
        <Text style={styles.titleText}>{title}</Text>
      </View>
      <View style={styles.videoContainer}>
        <Video
          style={styles.video}
          repeat
          source={{ uri: `${Config.PUBLIC_ASSETS_BASE_URL}/mobile-tutorial/${videoName}.mp4` }}
          bufferConfig={{
            minBufferMs: 2500,
            maxBufferMs: 5000,
            bufferForPlaybackAfterRebufferMs: 2500, // if this is greater than minBufferMs, the app will crash
            bufferForPlaybackMs: 2500,
          }}
          onLoad={() => console.log("BBenji tutorial loaded")}
        />
        <Text style={styles.subText}>{subtext}</Text>
      </View>
    </View>
  )
})

Video sample
If possible, include a link to the video that has the problem that can be streamed or downloaded from.

https://dd2cgqlmnwvp5.cloudfront.net/mobile-tutorial/dance.mp4
https://dd2cgqlmnwvp5.cloudfront.net/mobile-tutorial/camera.mp4

I am surprised by this issue, the sample basic app on the master branch in the repo acts like your sample...
Can you triple check and compare with the sample app please ?

will do!  Thanks so much for the quick response!

@MetaBenji any update ?


Make evil-snipe-replace-evil a bool/var?

I created a pull request for an evil-snipe contribution layer in Spacemacs (their tagline should be "vim done right"): https://github.com/syl20bnr/spacemacs/pull/712
To make evil-snipe integrate better with Spacemacs, it would be nice if (evil-snipe-replace-evil) could be set/unset like a regular variable. Then replace-evil could be the default in the evil-snipe layer, but users would be able to turn it off in their .spacemacs-file.
If my talk of layers is confusing you, please see: https://github.com/syl20bnr/spacemacs#configuration-layers

Can do. I'll let you know once I have.

Alright, I've implemented the variable evil-snipe-override-evil. By default it's disabled.


fix: correct R icon color from green to R-langs blue

see https://www.r-project.org/logo/Rlogo.svg.
The official color is a gradient, but I picked a spot in the middle.

In order to correctly generate light colors variants you need to run make colors (vim-colortemplate must be installed)

Thanks! Fixed the commit.


Include type config

Mongoid.include_type_for_serialization controls whether the _type
field is included when the model is serialized.
I guess that this is what you referred to in #1625

That was fast - thanks.


groupBy partial object creates issues with downstream usage.

Here's a sample usage of groupBy:
it("test", () => {
  type Foo = { id: number; color: string };
  const byColor = groupBy(({ color }) => color, [
    { color: "red", id: 1 },
    { color: "blue", id: 2 },
    { color: "red", id: 3 },
  ] as Foo[]);
  Object.values(byColor).map((grouping: Foo[]) => {
    console.log(grouping);
  });
});

which triggers this error:

The map parameter needs to be supplied with Foo[] | undefined to work correctly, even though that's not the right type.
Removing the Partial wrapping the return type would fix this DX issue... I'm wondering what is the rationale for providing a partial record type?

The suggestion to add the Partial<> was made and merged in this MR: https://github.com/ramda/types/pull/45
Your issue has already been brought up and resolved. Please see here: https://github.com/ramda/types/issues/91

Oh, sorry I didn't search for groupBy in the issues before posting! My bad, and thanks for the response. I'll move any subsequent questions there.


Update the gh-dxp PR template

📝 Description
This changes the look and feel of the gh-dxp PR template. It also auto-selects tests and lint.
Limits: This does not notice if no tests exists (add in a future PR). It also ignores documentation changes.
🔗 Issue ID(s): TDX-625
📋 Checklist

✅ Lint checks passed on local machine.
✅ Unit tests passed on local machine.

LGTM - I am especially a fan of the reduction in number of questions :)))


Spellcheck / markdown linting for docs

Hi all,
Love the Marten project, and wanted to see if there was an interest in some additional docs CI.
With the NUnit docs project, we utilize GitHub actions to run markdownlint for formatting and cSpell for spellcheck with all of our docs.
Dropping this here in case you'd find it useful. I could implement when I have time or someone else could probably make quick work of it by adding additional jobs using the steps we take; then it would just be a matter of any ignore rules, corrections, etc. in the PR.

@mysticmind

PS: We just did a big release of 4.0.0 today so taking a breather at this point.

Congratulations! 🎉 🚀
I'm looking forward to really attempting to use Marten and post-v4 is a great opportunity to do so.
I'll submit a PR when I'm able.


(PE-36947) Allow for Puppet::Util::Puppetdb to flush config

This provides a hook for flushing the Puppetdb classes @config variable. This in turn means the next time Puppetdb.config is callled it will reload the configuration file.
This becomes important if in some command using this utility, we have accessed the Puppetdb.config, then go on to update puppet's puppetdb.conf with new information, and need to then take additional operations with Puppetdb.config that should be based on the new configuration state.

Oops, need to rebase this against 7.x.

I kicked the jenkins test. Query timeout tests continue to fail on macos11.


Add files via upload

fixes #1056
color blast game in html,css and js.

Make this game in react because it's react based repository.


Breaks hot restart

This package appears to break hot restart in flutter. It hangs and does nothing when I attempt to.

Same issue here. When we implemented it not only breaks hot restart, but also app start up is slow with blank screen for like 10 seconds.

@zuzpk I have same problem my app stuck on startapp screen too ios ver are you using meta network too ?

@medalawi I have tried with a fresh project with AppLovin library only and result is same. Stuck on startapp screen (blank white)

@thomasmso Use this plugin example app, run example/lib/main.dart. SDK initialized successfully for the first time.

Syncing files to device Nokia 3 4...
Restarted application in 1,562ms.
I/flutter (15054): Initializing SDK...

Which platform? It looks like you were testing on Nokia?

@thomasmso Android 11

@zuzpk Did you find some work around?

Future<void> initializePlugin() async {
  final isInitialized = await AppLovinMAX.isInitialized();
  print(
      isInitialized); //returns true when hot restart. returns false when start debugging.
  if (isInitialized == null || !isInitialized) {
    final configuration = await AppLovinMAX.initialize(
        'sdkKey');
    if (configuration != null) {
      model.watchingAds =
          false; //notifyListener method to update a button UI.
    }
  } else {
    /*
    isInitialized may be true, but the SDK is actually not initialized at all,
    So i explicitly did an initialize again on hot restart, but this method will await forever.
    */
    final configuration = await AppLovinMAX.initialize(
        'sdkKey');
    model.watchingAds = false; //this never runs.
  }
}

The above code is for debugging purposes. I had put this code on initState() so as not to 'break hot restart'. If you put it in main() and hot restart,  await AppLovinMAX.initialize('sdkKey') would make the app unresponsive.
Workaround? I have no idea. Hope someone can provide a fix for this!

#77

Live in v2.4.2, thanks to @amondnet for looking into it!


Disable known bad unit tests until they are fixed

:+1: after travis

@fjy we can't put "fix #2185" in the commit, unless we actually fix the problem. Please reopen #2185

@fjy a more appropriate commit message would be "temporarily ignore test until #2185 is fixed"


MIDI Catalog Object

Create a MIDI catalog object
It should be smarted about when a MIDI channel isn't named, more concise

Not needed after refactor


Pipeline files

Coverage decreased (-0.1%) to 74.177% when pulling 3af175dbfaf6e49ed7c5597399abf8735f26fe68 on pipeline-files into c2af1d1e670bea714370b102f61178d61d04ec78 on master.

Coverage decreased (-0.06%) to 74.218% when pulling 74ee69f664823ae724c9d248e6182ccdd04d1a03 on pipeline-files into c2af1d1e670bea714370b102f61178d61d04ec78 on master.


the app is missing the tests

To my understanding, the waited elm-test mentioned in tests/README.md has been released.
Even a small example of how to do tests would be useful.
A more complex example that integrates with travis-ci would be even more helpful.

Nope, just haven't circled back to this code base in a long time. 😅

To set clear expectations, I'm looking at my book writing, OSS projects, and speaking schedule for 2018, and I'm oversubscribed on those alone, even if I did nothing else with my free time this year. With that in mind, the most time I'm going to be able to devote to this repo in 2018 is to update it for new Elm language releases (e.g. 0.19) and do quick fixes for some of the bugs.
I'd love to have bandwidth to add some elm-test and especially elm-html-test tests for this, but I'm choosing to prioritize writing, OSS, and speaking over that in 2018, which means I'll have bandwidth to add tests in 2019 at the earliest.


Xcode 9 not rendering in Interface builder

UI is not rendered at all in interface builder XCode 9

Thanks for reporting, this is a known issue since Xcode 8. Unfortunately, I haven't found a solution yet.


Tips for unit testing

Any advice on how to unit-test the ViewModel of a dialog? E.g. if I have (simplified):
@autoinject
export class EditNoteDialog {

	public noteId: number;
	public note: string;

	constructor(public controller: DialogController, private backend: Backend) {
	}

	activate(data: { noteId: number, note: string }) {
		this.noteId = data.noteId;
		this.note = data.note;
	}

	submit() {
		if (this.isValid()) {
			this.backend.put("notes/" + this.noteId, { note: this.note })
				.then((resp: { note: string }) => this.controller.ok(resp.note));
		}
	}

	isValid() {
		if (this.note.length > 16383) {
			return false;
		}
		return true;
	}
}

with template
<template>
	<ai-dialog class="note-dialog">
		<ai-dialog-body>

Edit note

			<textarea attach-focus="true" value.bind="note"></textarea>
		</ai-dialog-body>

		<ai-dialog-footer>
			<button click.trigger="controller.cancel()">Cancel</button>
			<button click.trigger="submit()">Save</button>
		</ai-dialog-footer>
	</ai-dialog>
</template>

I'd like my unit test to be pretty high level, something like:
describe("EditNoteDialog", () => {
  it("should take a noteId and note", () => { ... });
  it("should do nothing if canceled", () => { ... });
  it("should do nothing if the entered note is too long", () => { ... });
  it("should submit the note to the backend", () => { ... });
  it("doesn't handle errors from the backend", () => {});
  it("should return the new note if the backend request succeeded", () => { ... });
});

To test on this level, I can't manually construct an instance of the ViewModel. But there isn't much documentation I could find that deals with how to interact with the framework from unit tests. Any pointers would be greatly appreciated.

I don't know about testing with AutoInject but I think you'll need to use the container and pass in instances of all of your dependencies to manually construct it.
Something like this (pseudo-code):
import {Container} from 'aurelia-dependency-injection';
import {EditNoteDialog} from 'src/edit-note-dialog';

describe('MyViewModel', () => {
  let container;
  let instance;
  beforeEach(() => {
    container = new Container().makeGlobal();
    container.get(DialogController);
    container.get(Backend);
    let instance = container.get(EditNoteDialog);
  });
});

If you need more help with testing and can supply errors please consider asking in our official gitter or on StackOverflow.com (be sure to have a repro or specifically what errors are preventing you from testing it)


⚠ API - Lens has degraded performance

In f8ecf5c, API - Lens ($LENS_API_URL) experienced degraded performance:

HTTP code: 503
Response time: 332 ms

Resolved: API - Lens performance has improved in 0866ad8 after 8 minutes.


[FEATURE] Publish binaries for multiple platforms with every release

Is your feature request related to a problem? Please describe.
I am comparing toml-bombadil to other dotfile managers such as chezmoi and yadm, and one primary requirement for me is the ability to easily and reliably use the tool on a new machine without taking too long to install. yadm provides this by being written in Bash, while chezmoi provides binaries for most target platforms directly on GitHub, plus scripts in bash and powershell to detect the platform, download the correct binary, checksum it, and run it.
Conversely, toml-bombadil seems to require either building from source (which is slow, and by poor luck I immediately encountered a bug) or using one of the supported packages managers.
Describe the solution you'd like
Ideally, toml-bombadil would have a solution like chezmoi's, allowing users to quickly install the software on any machine using a short and reasonably memorable shell command.
However, even without providing scripts like this, it would be valuable for standalone executables for a variety of platforms to be included in the assets for each GitHub release.
Describe alternatives you've considered
If executables are available on GitHub, then a Cargo-binstall configuration file could also be provided, which would enable quickly installing toml-bombadil. This is not much better than building from source, though, because it still requires installing Rust, Cargo, and binistall itself.
Additional context
I realize there are some complications and expense involved in setting up the necessary CI to build executables for multiple platforms (especially Apple OSes), so I understand that this is not a trivial request!

Build for arm, x86, and macos are now available (see: https://github.com/oknozor/toml-bombadil/releases/tag/4.0.0).
A PR to add a cargo binstall configuration is welcome :)


fix DockerSuite.TestRunOOMExitCode

Should not error like this:
----------------------------------------------------------------------
FAIL: docker_cli_run_test.go:2868: DockerSuite.TestRunOOMExitCode

docker_cli_run_test.go:2881:
    c.Assert(err, check.IsNil)
... value *errors.errorString = &errors.errorString{s:"wrong exit code for OOM container: expected 137, got 1 (output: \"WARNING: Your kernel does not support memory limit capabilities. Limitation discarded.\\nsh: out of memory\\n\")"} ("wrong exit code for OOM container: expected 137, got 1 (output: \"WARNING: Your kernel does not support memory limit capabilities. Limitation discarded.\\nsh: out of memory\\n\")")

Hi!
Please read this important information about creating issues.
If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
This is an automated, informational response.
Thank you.
For more information about reporting issues, see https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
uname -a:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

#dibs


ICM: RecreateDB=true doesn't work be cause of existing connection

The existing connection seems to come from the icm-as deployment which seems to start earlier than the DROP of the database.
Either we need a kill of all existing connections or the icm-as deployment waits (initContainer?) for the mssql-Container.

fixed in image


Making _ a symbol constituent.

Comments indicate that this shouldn't work in XEmacs, but if we only
make it a symbol constituent in GNU Emacs, this should still work with XEmacs.

Can you give me a quick rundown on what _ as a symbol constituent means for users of go-mode? I've looked into it years ago when I wrote go-mode, and iirc the gist was "it's better", but I don't remember why.

Hello! "" as symbol constituent means that "a_b_c" is a symbol, but the words consist of "a", "b" and "c".  When "" is a word character, the word is "a_b_c" as well. This allows the user to navigate between parts of the symbol with backward/forward-word.
However, my change seems to introduce errors in syntax highlighting (probably related to the comment in the syntax table definition), which I'll have to debug. I'll close this for now.

Fixed the font-locking rules to add symbol characters to all regexes. This fixes font-locking for identifiers containing underscores.

Now that you mention it, this change might also break other emacs packages that copied our regexps for matching identifiers. I'll have to think about this.

This shouldn't influence other packages at all. The [:word:] class
depends on the current syntax table anyway, and the regexes won't work
in modes that have a different syntax table from go-mode. If the regex
is copied, this won't influence anything either, since _ is still a
word character.
Of course, I can't guarantee anything^^. I'd like to mention though
that not having _ as a symbol character differs significantly from
standard emacs mode (for example c-mode).

This shouldn't influence other packages at all

The go-eldoc package, for example, copied and slightly modified our regexps, which means they still use [:word:][:multibyte:]. Changing _ to be a symbol constituent instead of a word character would cause them the same problems that it caused for our syntax highlighting.

If go-eldoc uses the go-mode syntax table, this would break their copy
of the regexes. However, I'd suspect that this is an easy fix. Do you
know of any other modes that use the go-mode syntax table and have copied the regexes from go-mode?
I'll try and have a look at go-eldoc tomorrow and see if I can fix it
there as well.


[Quality Assurance] Zulip App let user send message to the stream that they're not subscribed

When using Zulip, and fill the "Stream" textbox with the stream name that's we're not subscribed. Zulip let user sends the message to that stream.
How to reproduce:

Go to Zulip and login to a server.
Then type the stream name that's exist but we're not subscribed yet in the "Stream" textbox. Here my Zulip account isn't subscribed to "test" stream.
Type the topic and message.
Then send the message.

Zulip app will returns Message sent! , and I can confirm that message is sent to that stream by accessing https://chat.zulip.org/#narrow/stream/test

Meanwhile when I try it in Zulip web:

Thanks for reporting this! :)

@kunall17 if this issue is not assigned to anyone then assign this to me, I would love to work on this.

@vishwesh3 :+1:

This is fixes in PR  https://github.com/zulip/zulip-android/pull/259 and can be closed.

Thanks @vishwesh3, closing :)


Make initialization settings take effect in SubViewport

SubViewport has some initialization settings of its own, but these settings are not synchronized to the RenderingServer when initialized, which causes some problems.
Split from #62854.
This is a prerequisite to resolve #55471, still need to modify update_mode manually, or add as a child to SubViewportContainer (update_mode is automatically set to UPDATE_ALWAYS).
Currently, UPDATE_WHEN_VISIBLE has no meaning for SubViewport. Presumably for visible, RenderingServer has different rules, viewport needs to have a visible rect to be considered visible, while SubViewport has only size and no position, so it is not considered visible even if it is set to be an active viewport (RS::viewport_set_active).

Superseded by #68358.


Passing options to chokidar

There seem to be two ways to do this: (1) via something like a opts.chokidar, or (2) piecemeal options like usePolling, interval and ignored.

via something like a opts.chokidar:

Pros:

no maintenance on the part of watchify since chokidar just takes a whole object blindly.
it's mostly isolated from browserify options since opts.chokidar is unique.

Cons:

it's a mouthful via the command line (e.g. --chokidar [ --usePolling --ignored '**/node_modules/**' ] vs --usePolling --ignoreWatch '**/node_modules/**').

piecemeal options:

Pros:

concise.
implementation agnostic (there are alternatives to chokidar like sane which can piggyback on watchman).

Cons:

clashes with existing browserify options (the entire opts object is passed to module-deps, browser-pack, etc.).
no options autonomy since options would have to be picked off the opts object.

Regardless of what way is decided, it is clear that having a way to specify usePolling is very important for NFS watching to work (see https://github.com/substack/watchify/issues/120)

@zertosh I like #1, but using the --chokidar flag is not really intuitive for someone quickly glancing for an option for watchify to ignore globs.
I think something along the lines of the following would be optimal:

Translate a simple --ignoreWatch like option to a default chokidar opts hash.
For fine-grained control one can look into documentation and use --chokidar excplicitly.

How's that sound?

Has anyone raised the issue of wanting to pass other options to chokidar other than usePolling? If not then maybe that makes the most sense as it is easy to understand and as @zertosh says implementation agnostic.

I came here to open an issue on this; seems I don't have to. :)
My (h|n)asty workaround is a contrib/polling-watchify.js with:
var chokidar = require('watchify/node_modules/chokidar');
var origWatch = chokidar.watch;

// Monkey-patch chokidar.watch() to use polling:
// @see https://github.com/substack/watchify/blob/2.4.0/index.js#L83
// @see https://github.com/substack/watchify/blob/2.4.0/index.js#L98
chokidar.watch = function(file, opts) {
    // @see https://github.com/paulmillr/chokidar#performance
    opts.usePolling = true;
    return origWatch.call(chokidar, file, opts);
};

// Proxy to the actual watchify "binary":
require('watchify/bin/cmd');

Also in the case of having to use polling I would assume it'd be hugely beneficial to be able to exclude certain paths, say, node_modules from being actively watched. At least in my setup node_modules content never changes unless I also restart the watchify process.

Translate a simple --ignoreWatch like option to a default chokidar opts hash.
For fine-grained control one can look into documentation and use --chokidar excplicitly.

@tarsolya That sounds good, but the issue then becomes deciding which flag overrides which, or how to merge them. That's not a technical challenge but a UX one - one whose outcome will make someone unhappy for sure.
I'm leading more towards mapping:

ignoreWatch -> ignored and,
usePolling: true to {usePolling: true, interval: [defaultValue]}, and
usePolling: integer to {usePolling: true, interval: integer}

And take the remaining options on case-by-case basis. I'm generally apprehensive about expanding API surface area - anywhere.

Has anyone raised the issue of wanting to pass other options to chokidar other than usePolling?

@jonbretman: Nope just that and only because it's a solution to NFS watching - which is kinda common since everyone wants to run and develop their apps in containers. Passing ignored to chokidar is a cheap solution to a more common request than NFS watching - which is to ignore files.

My (h|n)asty workaround is a contrib/polling-watchify.js with:

@jareware O_o wow heh. Thank you for sharing that snippet - it's clear that we need to support passing the polling option. Quick tip: since you're monkey-patching chokidar already, you can also pass in ignored to ignore paths - that'll still create chokidar instances but not watchers (that's the expensive bit).

I'll come up with something in the next few days. I want to cut another 2.x release that resolves (1) listeners leak in cmd.js and (2) EPERM on windows, before introducing a 3.x release that bumps chokidar to 1.0.0-rc4 and allows passing (some) options (in some form).

@zertosh Fair enough and I'm with you with keeping the API surface as tight as possible. I think what you proposed covers the majority of ignore use cases very well. And the custom ones ... well, they will have to be made custom anyway.
:+1:

@zertosh Ha, thanks for the tip! Also really looking forward to dropping the monkey-patching nonsense with an upcoming release. ;)

Just published 3.0.0 (see https://github.com/substack/watchify/pull/170). It has the options as I described them above.

Wicked, thanks!


[UX/UI] Search page design updates

During testing, it was mentioned:
Just low priority design quibbles:
(1) Mosaic mode has about double the space between items than is necessary.
Screenshot: https://drive.google.com/file/d/14GhRyk4qFnaJ5dGV52cIJmeOB7uTHrNG/view?usp=drive_link
(2) Inconsistent capitalization on filter buttons (Lets use titlecase)
(3) There is extra margin to the right of the text is the "add filter" button
Screenshot: https://drive.google.com/file/d/14NQ1mnItCBcN29_2PFwP6_9bPVP4y-mJ/view?usp=drive_link

@wrenaria this should be deployed to dev now.


[BUG] ToSKBitmap() doesn't work properly

ToSKBitmap() causes artifacts and displays at the incorrect size in Windows Forms.
I'm able to generate the correct bitmaps with my own method below:
        public static SKBitmap BitmapToSKBitmap(Bitmap bitmap)
        {
            var info = new SKImageInfo(bitmap.Width, bitmap.Height);
            var skiaBitmap = new SKBitmap(info);

            for (int i = 0; i < bitmap.Width; i++)
            {
                for (int j = 0; j < bitmap.Height; j++)
                {
                    Color color = bitmap.GetPixel(i, j);
                    skiaBitmap.SetPixel(i, j, new SKColor(color.R, color.G, color.B, color.A));
                }
            }

            return skiaBitmap;
        }

Hello,
I have the same kind of problem with ToSKBitmap function when I use bitmaps with transparency. The problem can be reproduce easily with this sample : https://github.com/Odirb/BugSkiaSharpWinform
I have fixed a problem inside ToSKPixmap function, and I will send a pull request.
Regards.

@mattleibow Hi! I try to load image by using:
UIImage img = UIImage.FromFile("./me.png");
SKBitmap bitmap = new SKBitmap(300,300, false);
bitmap = img.ToSKBitmap();
What i getting?

What i do wrong?
Thanks

@Mondonno make sure you install the SkiaSharp.Views package as well. The normal SkiaSharp package is just the .NET Standard library. For platform specifics, it is found inside the .Views packages.


Docker-compose for each supported CDMS

Each supported CDMS should have Dockerfile and docker-compose.yml to allow the quick creation of a test database instance for integration testing using their original database technology.

CDMS
Database technology
SQL DDL

CliDE
PostgreSQL 13
opencdms-test-data/schemas/clide/clidedb_schema.sql

Climsoft
MariaDB 10.1
opencdms-test-data/schemas/climsoft/v4/v4.1.1/climsoft_v4_all.sql

MCH
MySQL 5.1
opencdms-test-data/schemas/mch/Create_MCH_English_basic_tables.sql

MIDAS
Oracle XE 11g
opencdms-test-data/schemas/midas/midas_core_ddl.sql

OpenCDMS / WMDR
TimescaleDB with PostGIS extension
N/A  (DDL must ensure PostGIS is enabled)

For example: docker-compose.yml
midas_oracle:
    container_name: midas_oracle
    image: oracleinanutshell/oracle-xe-11g
    environment:
      - ORACLE_ALLOW_REMOTE=true
    ports:
      - 1521:1521
      - 5500:5500

Allowing us to then use a simple docker-compose up -d
Instead of manually copying the DDL into the container (as shown below) and then restoring this to the database instance, we should git clone –depth 1 –branch main this public repository from within the Dockerfile and then automate the restore process.

get container id with docker ps
Copy the DDL file in /tmp/ of the container using

docker cp ./met-office-midas/ddl/2020.12.04\ dumpMIDAS_ddl.sql 885756c1abad:/tmp/

get into the container using docker exec -it midas_oracle bash
sqlplus and use system/oracle
then run @'/tmp/2020.12.04 dumpMIDAS_ddl.sql' to execute the DDL file.

Although building MySQL 5.1 succeeded, attempts to build Oracle XE failed and we switched to using oracleinanutshell/oracle-xe-11g. We should make a copy of the oracle-xe-11g image, and other essential publicly available images, to ensure that we do not lose access to them in the future.

Closed by #12


Path is not getting expanded properly.

jail-task-driver is seeing:
"{Path:/ local/..
but the config is set:
Path    = "${NOMAD_TASK_DIR}/.."
I think if we set:
https://github.com/cneira/jail-task-driver/blob/2eec69fd464544c062f82352bc1a6d3aea3d15e2/driver/jail.go#L88
for the default to be TaskConfig.AllocDir if Path is unset, that would fix my issue.
I tried that:
diff --git a/driver/jail.go b/driver/jail.go
index 1d4480a..37ebf88 100644
--- a/driver/jail.go
+++ b/driver/jail.go
@@ -82,7 +82,11 @@ func (d *Driver) initializeContainer(cfg *drivers.TaskConfig, taskConfig TaskCon

        jailparams["name"] = fmt.Sprintf("%s-%s", cfg.Name, cfg.AllocID)
        jailparams["host.hostname"] = fmt.Sprintf("%s-%s", cfg.Name, cfg.AllocID)
-       jailparams["path"] = taskConfig.Path
+       if len(taskConfig.Path) > 0 {
+               jailparams["path"] = taskConfig.Path
+       } else {
+               jailparams["path"] = taskConfig.AllocDir
+       }

        if len(taskConfig.Jid) > 1 {
                jailparams["jid"] = taskConfig.Jid

but I get a
driver/jail.go:88:34: taskConfig.AllocDir undefined (type TaskConfig has no field or method AllocDir)
but the docs seem to say it exists:
https://godoc.org/github.com/hashicorp/nomad/plugins/drivers#TaskConfig
I don't know enough go to understand why AllocDir doesn't seem to exist..

Thanks!, I'll take a look.

This should be fixed by commit  https://github.com/cneira/jail-task-driver/commit/ede44c0fe7fa7543117445b8d3271749eebee659

So, cfg.AllocDir is not the root dir of the jail....
AllocDir =  /tmp/NomadClient654652155/bc6bee4a-3482-8779-3993-9ef8d809cfc0/

$ ls -lsa /tmp/NomadClient654652155/bc6bee4a-3482-8779-3993-9ef8d809cfc0/
total 18
1 drwxr-xr-x   4 root    wheel    4 Jul 10 20:41 .
9 drwx--x--x   7 root    wheel    7 Jul 10 20:41 ..
1 drwxrwxrwx   5 nobody  nobody   5 Jul 10 20:41 alloc
9 drwxr-xr-x  20 nobody  nobody  23 Jul 10 14:45 taskname

but it should equal  AllocDir + taskname
where taskname is the literal string from the task " " {} in the job file.
diff --git a/driver/jail.go b/driver/jail.go
index c045a18..3f29e6c 100644
--- a/driver/jail.go
+++ b/driver/jail.go
@@ -14,6 +14,7 @@ import (
        "github.com/hashicorp/nomad/plugins/drivers"
        "os/exec"
        "strings"
+       "path/filepath"
 )

 func simple_uuid() (string, error) {
@@ -102,7 +103,7 @@ func (d *Driver) initializeContainer(cfg *drivers.TaskConfig, taskConfig TaskCon
        if len(taskConfig.Path) > 0 {
                jailparams["path"] = taskConfig.Path
        } else {
-               jailparams["path"] = cfg.AllocDir
+               jailparams["path"] = filepath.Join(cfg.AllocDir, cfg.Name)
        }

        if len(taskConfig.Jid) > 1 {

This diff fixes that problem. I'll try to send a PR at some point.


Please read the readme once again, or look at this package for more info.


Failed to install a specific docker engine version in Centos

I prefer to install version 1.9.1 docker engine in centos, however, it seems only latest version can be installed as default, any suggestions?

Same issue +1
To install docker v1.9.1 in centos, the steps I tried are
step1: customized the docker installation script to specify the version of docker-engine as v1.9.1
step2: uploaded the above script to https://xxx/docker-install/docker-install.sh
step3: to test the script, run the command curl -sSL http://***/docker-install.sh | sh - manually. And it works as expected, docker-engine v1.9.1 has been installed in centos.
step4: set --engine-install-url value as https://xxx/docker-install/docker-install.sh when creating machine with docker-machine
However, the latest version of docker-engine had been installed in centos using docker-machine. It seems --engine-install-url does not work in centos.
The command used is
docker-machine create --driver generic --generic-ip-address xx.xx.xxx.xxx --generic-ssh-user ec2-user --generic-ssh-key ./mykey/mykey.pem --engine-install-url "https://xxx/docker-install/docker-install.sh" test
Any ideas on this?


feat(appbar): add inverse context to app bar and header

Codecov Report

Merging #220 (b1d3479) into dev (d226415) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev     #220      +/-   ##
==========================================
- Coverage   95.54%   95.53%   -0.02%
==========================================
  Files        1482     1482
  Lines        6442     6445       +3
  Branches     1035     1035
==========================================
+ Hits         6155     6157       +2
- Misses        137      138       +1
  Partials      150      150

Impacted Files
Coverage Δ

...s/react-magma-dom/src/components/AppBar/AppBar.tsx
92.85% <100.00%> (+0.54%)
:arrow_up:

patterns/header/src/components/header/Header.tsx
87.50% <100.00%> (+1.78%)
:arrow_up:

...act-magma-dom/src/components/Dropdown/Dropdown.tsx
80.88% <0.00%> (-1.48%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d226415...b1d3479. Read the comment docs.


Provide guidance and examples for different width pages

What
Provide guidance and examples to help users understand when to use full-width headers (like we use in the tech docs template, for example)
Why
The header in GOV.UK Frontend is capable of being used full-width, but we don't provide any examples of this in the Design System, nor do we have any guidance about when it's appropriate to do so.
Who needs to know about this
?

Notes from team discussion:

Common use case is admin interfaces
This work feels bigger than just the header, maybe we need guidance about how to set the width of your pages (this is what the weeks estimate is for)


#305 : Add android lib sample + Add support for android library plugin

(Untested)

Android gradle plugin comes into two flavors : com.android.application for normal applications, and com.android.library for libraries. Added support for the latter.

Thanks!


Is this project still alive?

Without going through the process of downloading/installing, etc - is this project still working / alive?  Any issues if I fork?

If so remember to publish on pipy to make everyone take advantage of your update.

@zmrow You may like to take a look at our fork. It is actively maintained/used. I will post to PyPI soon. @maximebf If you could grant me ownership on the current PyPi listing that would be great, otherwise I will have to choose a different PyPI name.

Hey everyone, we've made a new package on PyPI. Our organization is actively using this library, along with django-withings and we would happily accept PRs if you have any changes you need to make.


docs: highlight , correct typos and servers.md

Description

Enhancement: The code tags should appear bold everywhere in the docs
Typo: Found some typos in tutorials + missing of proper redirects.
Fix: The Codeblock in servers.md in not hightlighted correctly.

Related issue(s)
Fixes #2522

The spelling typos mentioned in this PR are already fixed LIVE.
The requested hard coding of  HTML tags is not within best practices nor does it follow our brand guidelines.

Closing this PR.


Add script to check plugin image changes

Signed-off-by: tarukumar<EMAIL_ADDRESS>This script handle the plugin changes in JL. No Luck getting plugin detail from notebook POD using OC command. Used Selenium instead.  Tried reaching out to Dev team(Václav (Vašek) Pavlín) and Inter team member. Will try to get in touch with Harshd from Dev team on how to get detail about plugin using cli.
PS: Some plugin links are not reachable with HTTP response 404(page not found)

Verified using CI PR-241. As mentioned above the TC is failed due to plugin url for ibm-s3 notebook in three images is not found. Ans script work as intended. There are multiple job failures due to known issue which is not caused by this scrip

t.

Ran new CI job PR-279
Script is working as designed


add cors support

Ohh, as we noticed, this CORS is required for web apps, for easy integration.
Thanks for the effort.


updated checkout and setup python action versions

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (32a51a5) 97.61% compared to head (a9d178e) 97.61%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master      #28   +/-   ##
=======================================
  Coverage   97.61%   97.61%
=======================================
  Files           2        2
  Lines          42       42
=======================================
  Hits           41       41
  Misses          1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Usage as a script tag

Hi
Is it possible to use this as a script tag on its own

This should already be possible. Take a look at the test file for sample usage.


Add option to set the default provider in gupm.json

Currently, when no provider is added, gupm stays the default provider. An option in gupm.json should allow user to set another provider as the default.
# gupm.json
{
  "cli" : {
     "defaultProvider": "npm"
   }
}

The option to set it only for certain tasks should also be available:

# gupm.json
{
  "cli" : {
     "make" : {
       "defaultProvider": "npm"
     }
   }
}

doing so, using g make will actually be equivalant to g make -p npm
Finally, a default provider for dependency retrieval:
# gupm.json
{
  "dependencies" : {
     "defaultProvider": "npm",
     "default": {
         .......
     }
   }
}

Code done -- need doc, and need to adapt default providers to use them on bootstrap


Expand documetantion about storage in Rahti

It expands a bit the current storage solution for Rahti. It talks about emptydir, pvcs and their types. Also it mentions Allas.

Looks good. In the longer run we should clear up the doucmentation a bit and have e.g. separate chapters for storage, networking, pods, etc. But for now it's better than it was, and has more info on storage which was the point.
Ok for me.


Collisions clear every time I save ldtk / Reimport

Not sure if this is intended or if there is already some intended script solution but this is annoying
~Thanks

Also to add tile physics to the tileset I have to delete the atlas and then reimport and then it lets me, but on default it just doesn't have the physics tab at all

The default import behaviour is that generated TileSets should preserve their data on reimporting. Physics Layers should be kept unless Force TileSet Reimport is ON, or something has gone wrong.
Can you test this in v2.0.1?

I was having the same issue until i turned Force TileSet Reimport off!

I am having this issue even though I've turned Force TileSet Reimport off! D:

Thank you, @dbrats. That solved the issue. I made a pull request with your fix.


tcgacompare reverse order of plot

Hi PoisonAlien,
Want to say your package maftools is really wonderful! You are doing such a great service to the scientific genomic research community.
A quick suggestion: the tcgacompare command that generates the tumor mutation burden plot is currently displaying tumors with the highest TMB starting on the left. This is the opposite of convention where the highest tumors usually start on the right, like here.

Is it possible to modify the command to reverse the order of plotting?
Thanks!

Hello,
Thank you for your kind words and suggestion. I have pushed a fix for this.
From now by default, cohorts are organised in increasing order (the way you wanted). However, you can set decreasing = TRUE for reversing it. Let me know if this helps.


making it possible to extend UCERF3_GridSourceGenerator

These changes enable us to extend UCERF3_GridSourceGenerator and write our own initialisation code in the new constructor.

@voj is there a reason that you need to extend it? This is a to-be-thrown-away (or heavily refactored) class, might be better in the short-term to just copy and modify it for your uses for now. Otherwise you'll be more affected if this class disappears or is heavily modified.

@voj is there a reason that you need to extend it? This is a to-be-thrown-away (or heavily refactored) class, might be better in the short-term to just copy and modify it for your uses for now. Otherwise you'll be more affected if this class disappears or is heavily modified.

@kevinmilner the idea was to duplicate as little code as possible. If you think copying it is safer than I'll do that. But I suppose when this is thrown out or heavily refactored we'll probably have to do the same anyway?

Closing this to follow @kevinmilner 's recommendation of duplicating the code instead


Analog clock

Closes: #411

Title : ANALOG CLOCK
Name: : MANKALA VAISHNAVI
Idenitfy yourself: : GSSOC

Describe the add-ons or changes you've made 📃
I have added analog clock into vannila js basic projects and updated readme
How Has This Been Tested? ⚙

Screenshots 📷

Type of change ☑

What sort of change have you made:

[x] New feature (non-breaking change which adds functionality)

Checklist: ☑

[x] My code follows the Contributing Guidelines & Code of Conduct of this project.
[x] This PR does not contain plagiarized content.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly wherever it was hard to understand.
[x] My changes generate no new warnings.

Note to reviewers 📄

@VaishnaviMankala19 properly link your issue with the pr


feat: add in B2B changes for 232

@AshRelateIQ please verify that package.json version has been incremented and package lock should be changed in this manner

@jarndt-ltm Acked. Bumped version up on package.json.


WsManListener: does not overwrite Hostname/CertificateThumbprint

Details of the scenario you tried and the problem that is occurring:
We are using this module in order to configure the WinRM HTTPS listener in an unorthodox manner-- utilizing a self-signed certificate with the same "Hostname" across all machines. This has worked well for us!
What I have discovered is that if we the HTTPS listener is already configured with a different certificate/hostname, the DSC resource will not overwrite the existing listener with the specified hostrname/thumbprint.
The DSC configuration that is using the resource (as detailed as possible):
In this case, we are using DSC within Chef-- but the same action occurs when using vanilla DSC:
  resource :WSManListener
  property :Transport, 'HTTPS'
  property :Ensure, 'Present'
  property :CertificateThumbprint, node['winrm']['certificate']['thumbprint']
  property :Hostname, 'Hostname'
end

In order to replicate:
Configure WinRM HTTPS listener with a self-signed certificate:
Remove-Item -Path WSMan:\Localhost\listener\listener* -Recurse

Set-Item WSMan:\localhost\MaxTimeoutms 1800000
Set-Item WSMan:\localhost\Service\Auth\Basic $true

$Cert = New-SelfSignedCertificate -CertstoreLocation Cert:\LocalMachine\My -DnsName "packer"
New-Item -Path WSMan:\LocalHost\Listener -Transport HTTPS -Address * -CertificateThumbPrint $Cert.Thumbprint -Force

# WinRM
write-output "Setting up WinRM"
write-host "(host) setting up WinRM"

cmd.exe /c winrm quickconfig -q
cmd.exe /c winrm set "winrm/config" '@{MaxTimeoutms="1800000"}'
cmd.exe /c winrm set "winrm/config/winrs" '@{MaxMemoryPerShellMB="1024"}'
cmd.exe /c winrm set "winrm/config/service" '@{AllowUnencrypted="true"}'
cmd.exe /c winrm set "winrm/config/client" '@{AllowUnencrypted="true"}'
cmd.exe /c winrm set "winrm/config/service/auth" '@{Basic="true"}'
cmd.exe /c winrm set "winrm/config/client/auth" '@{Basic="true"}'
cmd.exe /c winrm set "winrm/config/service/auth" '@{CredSSP="true"}'
cmd.exe /c winrm set "winrm/config/listener?Address=*+Transport=HTTPS" "@{Port=`"5986`";Hostname=`"packer`";CertificateThumbprint=`"$($Cert.Thumbprint)`"}"
cmd.exe /c netsh advfirewall firewall set rule group="remote administration" new enable=yes
cmd.exe /c netsh firewall add portopening TCP 5986 "Port 5986"
cmd.exe /c net stop winrm
cmd.exe /c sc config winrm start= auto
cmd.exe /c net start winrm

Then use DSC to try to overwrite those values:
{
    param
    (
        [Parameter()]
        [System.String[]]
        $NodeName = 'localhost'
    )

    Import-DscResource -Module WSManDsc

    Node $NodeName
    {
        WSManListener HTTPS
        {
            Transport             = 'HTTPS'
            Ensure                = 'Present'
            CertificateThumbprint = 'F2BE91E92AF040EF116E1CDC91D75C22F47D7BD6'
            Hostname              = 'Hostname'
        } # End of WSManListener Resource
    } # End of Node
} # End of Configuration

DSC will complete successfully and indicate everything is up to date, but will not overwrite the existing listener with the new values.
Version of the Operating System and PowerShell the DSC Target Node is running:
Server 2016, PowerShell 5.1
Version of the DSC module you're using, or 'dev' if you're using current dev branch:
<IP_ADDRESS>
Any help is greatly appreciated-- thanks for your excellent work on this module!

I have a version that fixes this and will create a PR.


TikTok as Data Source

For SumUp and the training of our AI model it is beneficial, if we can aggregate information from a multitude of different data sources.
For an overview of current data sources see here: Data Source Table
Done?

Required fields (orange) in Data Sources table have been filled out
Implementation is able to retrieve supported features that are not marked as unimplemented

Copied from #50:
- TikTok API

Official API: TikTok has an official API for accessing and managing content, user interactions, and more. It's mainly geared towards businesses and developers creating marketing and analytics tools.
Python Integration: Python libraries such as TikTokApi provide a way to interact with TikTok data. Note that these libraries often face challenges due to frequent changes in TikTok's internal API.


Improve config architecture

Having a convention for a config file could reduce the burden of manually setting config values in different contexts.
Also, memoizing at the module function level has already caused some issues in the test suite. Let's move away from that.

Playing devil's advocate, the current implementation of the config makes it simple to specify whatever is needed. E.g. being able to instantiate the Bunny connection yourself and tell Velveteen about it means Velveteen doesn't have to know those details.

The shared global config struck again with BunnyMock caching queues by name, which caused some data to leak out of tests (#11).

We hit this too, but with Bunny itself. 😁 We had some code that cleared the internal Bunny cache when we created the retry queue.
def create_retry_queue!(delay)
  clear_queues_cache!

  broker.queue #...
end

# When we create a new queue, `Bunny` stores its name in an internal
# cache. The problem is that as we are creating ephemeral queues that can
# expire shortly after they are created, this cached queue may not exist
# anymore when we try to publish a second message to it.
#
# Removing queues from the cache guarantees that `Bunny` will always try
# to check if they exist, and when they don't, it will create them for us.
def clear_queues_cache!
  broker.with_channel do |channel|
    channel.queues.clear
  end
end

I'm not sure if this is relevant here but I figured I'd chime in.

Yep, I ran into that as well! Bunny and BunnyMock have a slightly different interface, i.e. BunnyMock doesn't provide channel.queues, which caused some pain when testing. I worked around that by creating the retry queues manually (https://github.com/thoughtbot/velveteen/blob/master/lib/velveteen/error_handlers/exponential_backoff.rb#L62), which bypasses the cache. That approach does contribute to the config awkwardness though 😬 Perhaps a better solution would be to add channel.queues upstream to make the two consistent.


adding peer add endpoint

I would like to add something like this.  I am not sure the best approach.  It would be nice to have an option to make the peer get written to the config also.
This command would then add a peer while the application is running, not requiring a restart.
curl -d '{"WgPubKey": "pubkey", "WgAllowedIps": "fd00::0/8"}' http://localhost:7777/peer

Hi @casonadams , thanks for opening the PR.
We are working on adding a management layer to the system to improve peer management. Part of this layer will be to improve this aspect of adding new peers without needing to restart the service, but the way we invasion that will be different than what you proposed as it won't introduce a point of concern that is having a local HTTP endpoint on each client, which can increase the complexity of managing many peers as it might introduce security risk and the system is not managing firewalls and ACLs. Our initial idea is to keep file support without the need to restart and introduce a management system.
You can follow #39, #40 and #44 to see progress on that.

@mlsmaycon sounds great!  Any "first issues" I can take a stab at here?  I'd love to help this project along.

hey @casonadams
You could have a look at this one https://github.com/wiretrustee/wiretrustee/issues/43
We are enabling TLS for the gRpc services and Signal is one of them.
Yesterday we finished implementing Letsencrypt for the Management service (see https://github.com/wiretrustee/wiretrustee/pull/52 ) so you can use it as an example.
Thank you very much for being active :)


Use the same SVG images, in the default viewer, regardless of the CSS theme

As mentioned in https://github.com/mozilla/pdf.js/wiki/Frequently-Asked-Questions#faq-support, PDF.js version 2.6.347 is the last release with IE 11/Edge support.
Hence we should now be able to reduce unnecessary duplication in the default viewer image resources, note the files in the web/images/ folder with a -dark suffix, by using only one SVG-image for each icon and letting the background-color depend on the CSS theme instead.
For the gulp mozcentral build-target, the resulting web/images/ folder is reduced from 43 997 to 28 566 bytes (~35 percent).
Please note: I don't really know if this implementation is necessarily the best solution, but it seems to work well enough in e.g. Firefox Nightly and Google Chrome Beta as far as my testing goes.

/botio-linux preview

/botio-linux preview

Looks good to me. Thanks!


Document the harness manifest option.

Add documentation for the harness manifest option, introduced in a6589681351dc4a87b6fc268d47b7d608004b455.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @steveklabnik (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. The way Github handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see CONTRIBUTING.md for more information.

@bors: r+ 75d80d8
Thanks!

:hourglass: Testing commit 75d80d8 with merge b27cd47...

:sunny: Test successful - cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-32, cargo-win-64


added service-account impersonation

Description of your changes
Include the options to use service account impersonation instead of providing a service account key. This would be helpful in environments where creating service account keys is prohibited.
Fixes #319
I have:

[X] Read and followed Crossplane's [contribution process].
[X] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested
Tested with a local default application key which was a k8s secret that the google application default environment variable used for its value in the controller. This key's account was given Service Token Creator role on the impersonated account. This impersonated account was specified in the ProviderConfigRef resource and was successful in creating a bucket. In a real GKE cluster, a workload identity service account would be attached to the controller.
[contribution process]: https://git.io/fj2m9

Is there anything I need to do to get this to move forward?


Experimental: ODBC-based providers

Look at a base provider implementation that uses ODBC.

Related: System.Data.Odbc Namespace https://github.com/dotnet/corefx/issues/13035

Will take a look at this

@ajcvickers please re-triage and update https://github.com/aspnet/EntityFramework/wiki/Roadmap#ef-core-20 accordingly. I ported ODBC (https://github.com/dotnet/corefx/pull/15646) but very little progress was made on this.

@tinchou That is very interesting news - Are you able to share your ODBC port (a branch??)

@ErikEJ if you mean System.Data.Odbc namespace, it's been merged into dotnet/corefx. I linked to the PR in the previous comment, but here it is again: https://github.com/dotnet/corefx/pull/15646.
If you mean my work on creating an ODBC-based provider, I think I just got a couple of very basic select statements working. I was making it run on Oracle, but now I realize I should've started with SQLServer or Sqlite 😛.
Anyway, it's probably not worth sharing, even more considering that EFCore 2 has been changing and it most definitely won't compile by now, but let me know.

We are not planning to do this on the EF team, but once System.Data.Odbc is available in .NET Core, then somebody else could make use of it to do something like this.

@tinchou Based on the comment from @ajcvickers - are you able to share ? I would consider moving forward with coordinating an ODBC provider...

Here you go!
Tinchou.EntityFrameworkCore.Oracle.zip
Comments:

This is targeting the Full Framework since the API surface stayed the same and it is easier than dogfooding corefx. But if you want to do that, you'd need to follow these instructions since the ODBC package doesn't seem to exist in nuget or myget yet
This project was sitting next to and targeting the EF Core 2 source, but maybe you can point to a nightly feed instead
It says "Oracle" everywhere, but I hadn't reached the point of writing plsql code. Everything should be working with a sql-92 compatible DB with ODBC support - probably Sqlite is a good option since it matches the SQL from the Relational base provider very well.

@ajcvickers is this a u-turn from what was said on live.asp.net community standup, my understanding was odbc providers would be supported by the ef core team but not in 2.0 timeframe?

@tinchou Great, will you be OK with me hosting a "EntityFrameworkCore.ODBC" package? Can you recommend a SQLite ODBC provider? Sounds like SQL Server might be a better (easier) option?

@ErikEJ how about EntityFrameworkCore.Odbc? Anyway, keep in mind that you will have to use a specific SQL syntax. This means that a generic ODBC provider might not work with any specific database unless you override the methods that generate the SQL code.
The project I shared uses the Relational provider as a starting point, and so it will use whatever SQL it generates. It more closely resembles Sqlite's syntax as far as I know. For Sqlite, this driver worked fine in my tests: http://www.ch-werner.de/sqliteodbc/

This means that a generic ODBC provider might not work with any specific database unless you override the methods that generate the SQL code.

If that is the case, I do not really see the point...

The main use case for ODBC is databases that don't have a managed implementation. It abstracts away all the connection management and communication specifics, and you only focus on building a string of SQL code.
If I was going to write an EF provider for example for Oracle, I would:

Get a well supported database such as Sqlite working on top of ODBC
Switch the connection string to point to an Oracle instance
Override the SQL generation methods to use Oracle's syntax

Step 1 is generic and is a non-trivial task, but the SQL generation is always database-specific.

@ErikEJ This whole thing is a little confusing. 😸 The idea, as I understand it, that there is currently a barrier to entry on writing an EF Core provider for .NET Core which is that you have to have an ADO.NET provider for .NET Core to use at the lower level. If you've got a native one, then you're good to go. But this is not always the case, especially given that .NET Core is pretty new. But if there is an ODBC ADO.NET provider, then if EF can work with this, then its possible to build an EF provider for the database you are targeting but making use of the ODBC underneath. We were considering this approach for some well-known database engines, but with what we know now about the state of .NET Core ADO.NET providers we now see no need to take this approach at this time.

Thanks for clearing that up guys - I completely misunderstood the whole undertaking... 😕  - I will let it be...

@ajcvickers is/could there be any mileage on making public the POC work that was mentation only a few days ago on the live.asp.net as working? would really like to try and hook up this with the oracle odbc c++ provider for linux ;)   I do understand why it's moved across into a corefx task give that database connectivity via odbc etc. is a layer below EF and developers may not event want EF in the mix ;)

@grahamehorner I don't know what was specifically mentioned in the stand-up--I would follow up with whomever mentioned it. With regards to making the work we have done public, I believe it is all public already. @tinchou referenced the corefx work above and also provided a zip of the prototyping work in another comment. I don't think we have anything else.

@ajcvickers see https://www.youtube.com/watch?v=8jJVrYXSJvo&list=PL0M0zPgJ3HSftTAAHttA3JQU4vOjXFquF&index=0 or live.asp.net episode titled EF Core Team Apr 11, 2017

@grahamehorner The things @divega talked about at around the 18 minute mark are the things that are referenced above.

I have very little to add to @ajcvickers' explanation at https://github.com/aspnet/EntityFramework/issues/7432#issuecomment-294915210. I think that covers exactly what this item was about. But just as an update:

At this point the System.Data.Odbc package containing the code that @tinchou ported is unlikely to be included in the 2.0 release but should come shortly after. The main reason is that we are still need to take a few steps to produce the package and to include necessary native dependencies in the .NET Core test infrastructure, and this caught us as a point in which we are really trying to stabilize the builds and land the 2.0 release rather than focus on adding features to it.
Once the package is available I think System.Data.Odbc still represents a great opportunity for the OSS community and customers to build solutions that leverage the ODBC drivers available across all the various operating systems, in particular for those combinations in which a regular ADO.NET implementation isn't available yet. In other words, the fact that we closed this issue should not discourage others from trying this approach.