feat: emitting partial join results in HashJoinStream

Which issue does this PR close?

Closes #7848.
Rationale for this change

Ability to produce intermediate output, without accumulating "inner" batch until complete probe-side batch is joined, will help to avoid unpredictable memory consumption by HashJoinStream (e.g in case of implicit cross-joins) and to process such queries.
What changes are included in this PR?

HashJoinStream now manages current probe batch, and fetches next batch from probe-side input only if required (if previous one has been processed), and HashJoinStreamState -- structure for maintaining last matched indices (to resume matching while next iteration, if required), and last joined (matched, filtered and collision checked -- indices of record to be sent to output) probe indices -- required for adjusting probe-side before sending to output.
build_equal_condition_join_indices is able to skip previously joined pairs and stops matching indices in case potential (before applying join filters) output batch size reaches configuration.batch_size.
adjust_indices_by_join_type and subsequently called by it functions now accept Range argument in order to perform partial adjustment.
(as for now ) behaviour of SymmetricHashJoin remains untouched -- it still waits for the whole probe_batch being processed to produce the result.

Are these changes tested?

Tests for each type of join (both with and without filters) now run for multiple batch sizes + added tests for validation output/batch count for each join type and varying batch_size.
Additionally these modifications are covered by join_fuzz tests.
Are there any user-facing changes?

HashJoinExec default output now maintains only probe-side ordering (instead of preserving sort order for both build and probe sides) -- complies with calculate_join_output_ordering behaviour, but still is a change.

I will review it soon, can you share the performance changes with https://github.com/apache/arrow-datafusion/tree/main/benchmarks

@metesynnada, thanks! Results for tpch_mem (10 iterations, scale=1) and tpch (5 iterations, scale=10) are
--------------------
Benchmark tpch.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃     master ┃ hash_join_batch_size ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1     │  5415.03ms │            5425.11ms │    no change │
│ QQuery 2     │   857.37ms │             930.79ms │ 1.09x slower │
│ QQuery 3     │  2550.69ms │            2668.79ms │    no change │
│ QQuery 4     │  1329.48ms │            1394.45ms │    no change │
│ QQuery 5     │  3670.07ms │            3799.92ms │    no change │
│ QQuery 6     │  1226.23ms │            1232.87ms │    no change │
│ QQuery 7     │  7879.84ms │            8178.27ms │    no change │
│ QQuery 8     │  3723.55ms │            3748.62ms │    no change │
│ QQuery 9     │  6454.01ms │            6679.66ms │    no change │
│ QQuery 10    │  3747.42ms │            3792.84ms │    no change │
│ QQuery 11    │  1061.44ms │            1118.66ms │ 1.05x slower │
│ QQuery 12    │  1876.41ms │            1908.35ms │    no change │
│ QQuery 13    │  3787.24ms │            3937.90ms │    no change │
│ QQuery 14    │  1627.99ms │            1635.65ms │    no change │
│ QQuery 15    │  1648.28ms │            1655.15ms │    no change │
│ QQuery 16    │   931.55ms │             942.41ms │    no change │
│ QQuery 17    │ 14161.80ms │           14253.68ms │    no change │
│ QQuery 18    │ 21711.26ms │           23960.37ms │ 1.10x slower │
│ QQuery 19    │  3027.67ms │            3069.92ms │    no change │
│ QQuery 20    │  3825.12ms │            3925.90ms │    no change │
│ QQuery 21    │  9527.87ms │           10463.53ms │ 1.10x slower │
│ QQuery 22    │  1010.40ms │            1043.16ms │    no change │
└──────────────┴────────────┴──────────────────────┴──────────────┘
--------------------
Benchmark tpch_mem.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃   master ┃ hash_join_batch_size ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 393.17ms │             392.63ms │     no change │
│ QQuery 2     │  94.37ms │              99.65ms │  1.06x slower │
│ QQuery 3     │ 100.94ms │             105.51ms │     no change │
│ QQuery 4     │  69.70ms │              70.29ms │     no change │
│ QQuery 5     │ 227.66ms │             231.48ms │     no change │
│ QQuery 6     │  32.18ms │              30.46ms │ +1.06x faster │
│ QQuery 7     │ 461.49ms │             491.93ms │  1.07x slower │
│ QQuery 8     │ 159.03ms │             163.71ms │     no change │
│ QQuery 9     │ 337.22ms │             360.54ms │  1.07x slower │
│ QQuery 10    │ 190.43ms │             195.71ms │     no change │
│ QQuery 11    │ 111.48ms │             117.54ms │  1.05x slower │
│ QQuery 12    │  87.90ms │              88.86ms │     no change │
│ QQuery 13    │ 179.83ms │             187.62ms │     no change │
│ QQuery 14    │  35.45ms │              29.73ms │ +1.19x faster │
│ QQuery 15    │  35.02ms │              39.34ms │  1.12x slower │
│ QQuery 16    │  82.82ms │              89.14ms │  1.08x slower │
│ QQuery 17    │ 616.74ms │             583.82ms │ +1.06x faster │
│ QQuery 18    │ 723.74ms │             813.82ms │  1.12x slower │
│ QQuery 19    │  98.08ms │              97.18ms │     no change │
│ QQuery 20    │ 192.25ms │             192.46ms │     no change │
│ QQuery 21    │ 576.40ms │             609.72ms │  1.06x slower │
│ QQuery 22    │  59.03ms │              58.96ms │     no change │
└──────────────┴──────────┴──────────────────────┴───────────────┘

Thank you @alamb , but I don't think this PR worth benchmarking it -- after running current unit tests with different batch sizes, I've found that arguments for adjust_indices_by_join_type not really correct in this version + it works incorrect with join filters, so I suppose I need a bit more time to figure it out 😞
And yes, in this context, your suggestion about placing state management logic into separate JoinOutputBuilder struct is definitely worthwhile -- I'll look for an options of separating state from other join internals.

definitely worthwhile -- I'll look for an options of separating state from other join internals.

Thank you .
this particular issue has become a higher priority for us at InfluxData (because we are sometimes hitting it internally). I may spend some time working on it myself this upcoming week (e.g. maybe encapsulating the join output generation into a struct as a first step 🤔 )

@alamb, got it!
Anyway, current version works as expected, so I'll mark it as ready for review. Regarding logic / encapsulation -- I've moved all the index-tracking work to HashJoinStreamState, which provides methods to update it, and receive state for next iteration / range that should be adjusted (still not an output builder though, but as a good thing -- it doesn't affect current processing significantly).

Thanks @korowa  -- I'll review this PR later today.

Here are my benchmark results. My conclusion is that there is no significant performance change (the faster/slower is within the realm of noise). I will try running at a larger scale to see if anything shows up
--------------------
Benchmark tpch.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃ main_base ┃ hash_join_batch_size ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  928.44ms │             930.39ms │     no change │
│ QQuery 2     │  210.54ms │             222.96ms │  1.06x slower │
│ QQuery 3     │  454.08ms │             463.21ms │     no change │
│ QQuery 4     │  248.99ms │             280.77ms │  1.13x slower │
│ QQuery 5     │  589.44ms │             581.81ms │     no change │
│ QQuery 6     │  219.45ms │             223.18ms │     no change │
│ QQuery 7     │  925.05ms │            1063.70ms │  1.15x slower │
│ QQuery 8     │  675.09ms │             717.43ms │  1.06x slower │
│ QQuery 9     │ 1052.59ms │            1050.42ms │     no change │
│ QQuery 10    │  712.75ms │             688.61ms │     no change │
│ QQuery 11    │  212.84ms │             200.64ms │ +1.06x faster │
│ QQuery 12    │  362.82ms │             359.72ms │     no change │
│ QQuery 13    │  560.56ms │             580.54ms │     no change │
│ QQuery 14    │  322.74ms │             305.90ms │ +1.06x faster │
│ QQuery 15    │  232.62ms │             235.21ms │     no change │
│ QQuery 16    │  184.60ms │             188.35ms │     no change │
│ QQuery 17    │ 1218.95ms │            1166.47ms │     no change │
│ QQuery 18    │ 1741.70ms │            1742.98ms │     no change │
│ QQuery 19    │  566.74ms │             551.15ms │     no change │
│ QQuery 20    │  482.46ms │             456.34ms │ +1.06x faster │
│ QQuery 21    │ 1388.65ms │            1408.79ms │     no change │
│ QQuery 22    │  171.23ms │             165.01ms │     no change │
└──────────────┴───────────┴──────────────────────┴───────────────┘
--------------------
Benchmark tpch_mem.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃ main_base ┃ hash_join_batch_size ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  588.55ms │             574.48ms │     no change │
│ QQuery 2     │  154.73ms │             163.05ms │  1.05x slower │
│ QQuery 3     │  152.65ms │             154.83ms │     no change │
│ QQuery 4     │  109.42ms │             110.63ms │     no change │
│ QQuery 5     │  379.86ms │             391.93ms │     no change │
│ QQuery 6     │   39.26ms │              39.47ms │     no change │
│ QQuery 7     │  885.93ms │             898.37ms │     no change │
│ QQuery 8     │  277.20ms │             279.22ms │     no change │
│ QQuery 9     │  565.02ms │             548.77ms │     no change │
│ QQuery 10    │  311.39ms │             304.02ms │     no change │
│ QQuery 11    │  172.35ms │             172.79ms │     no change │
│ QQuery 12    │  144.86ms │             152.07ms │     no change │
│ QQuery 13    │  241.93ms │             253.64ms │     no change │
│ QQuery 14    │   48.83ms │              45.23ms │ +1.08x faster │
│ QQuery 15    │   45.07ms │              46.21ms │     no change │
│ QQuery 16    │  150.89ms │             160.37ms │  1.06x slower │
│ QQuery 17    │  751.06ms │             763.63ms │     no change │
│ QQuery 18    │ 1349.89ms │            1498.83ms │  1.11x slower │
│ QQuery 19    │  144.74ms │             150.76ms │     no change │
│ QQuery 20    │  305.43ms │             312.09ms │     no change │
│ QQuery 21    │ 1026.75ms │            1060.09ms │     no change │
│ QQuery 22    │   90.80ms │              87.23ms │     no change │
└──────────────┴───────────┴──────────────────────┴───────────────┘

Thank you so much @korowa  -- also if this PR is getting to large, maybe we can start with some smaller refactors (like pulling the existing output generation into a struct / stream without changing the logic, for example)
I can assist if that would help

@alamb I guess so -- seems like it's going to grow large enough, to make the review inconvenient and reach critical mass of quite an important changes 😞
One possible way is to split it into

FIFO JoinHashMap -- to preserve natural input order for build side
State management for HashJoinStream -- looks like it doesn't make much sense to build state machine only for matched/joined indices tracking, and instead it should be stream related
Applying current PR on top of the previous two

(all 3 assumes retaining SymmetricHashJoin logic as-is with code duplication if required).
First two parts could potentially be separate issues/PRs, and third is all about merging (or, more likely, rewriting from the scratch) so, I'd highly appreciate a help with any of the first two.
Any thoughts on this plan?

The SymmetricHashJoin is already implementing partial materialization of the sides, which means that its processes differ somewhat from those in your PR. It seems reasonable to keep the logic distinct in this case. However, you could potentially minimize repetition by employing helper functions or similar techniques.
I'll take another look at this, but could you provide a brief explanation of how you maintain the lexical order on the build side?

Hi @korowa, I can help on FIFO JoinHashMap if you wish, previously I implemented a version for the SymmetricHashJoin.

Any thoughts on this plan?

I think the plan sounds great. Is there anything I can do to help? E.g. could I try and implement one of the items in your list?

@metesynnada , @alamb, thank you!
To be honest, item#2 -- HashJoin refactoring seems like too massive task for me (at least in short/reasonable period of time) -- so, I would be awesome to get some help with it.
Regarding FIFO JoinHashMap, @metesynnada

brief explanation of how you maintain the lexical order on the build side?

That has given me a following thought:

the only two cases when DataFusion cares about build-side output ordering -- are inner joins 1 and 2
actually ordering in plan doesn't care about "natural" order (the order in which records were fetched from the source, and which actually is inverted/broken in this PR for build-side). The only important thing is lexical order -- which is still fine, cause HashJoinStream keeps maintaining it for probe-side.

Due to these couple of facts -- it doesn't seem to me that FIFO HashJoinMap is required, cause for inner joins both probe and build sides will keep their lexicographical order (if they both were ordered by join-key columns -- otherwise build-side order will anyway be distorted).
Example for current implementation would be like following (for build inner join probe using (key))
      Probe              Build                  Output
┌───────┬───────┐  ┌───────┬───────┐  ┌─────────┬─────────┬─────┐
│  idx  │  key  │  │  idx  │  key  │  │probe_idx│build_idx│ key │
├───────┼───────┤  ├───────┼───────┤  ├─────────┼─────────┼─────┤
│   0   │   1   │  │   0   │   1   │  │    0    │    1    │  1  │
│       │       │  │       │       │  │         │         │     │
│   1   │   2   │  │   1   │   1   │  │    0    │    0    │  1  │
│       │       │  │       │       │  │         │         │     │
│   2   │   3   │  │   2   │   2   │  │    1    │    3    │  2  │
│       │       │  │       │       │  │         │         │     │
│   3   │   4   │  │   3   │   2   │  │    1    │    2    │  2  │
│       │       │  │       │       │  │         │         │     │
│   4   │   5   │  │   4   │   5   │  │    4    │    4    │  5  │
└───────┴───────┘  └───────┴───────┘  └─────────┴─────────┴─────┘

index-based order of build side has changed, but lexicographically -- it is still correct.
So, I guess, there is no need for JoinHashMap modification, and all we need now -- is to use separate   build_equal_condition_join_indices  for SHJ.

If the message above is correct -- the only preconditions are HashJoinStream state management and moving retaining current build_equal_condition_join_indices version in SHJ.
@alamb could you please look at HashJoinStream state management (I've seen it on your list, so)? And I can handle separate implementation for SHJ and merging (I guess another build_equal_condition_join_indices for SHJ will be relevant in this PR).

I will look at this or again carefully tomorrow and try and offer some ideas

Hi @korowa, as far as I understand, you reverted the build side indices per probe side index.
Probe              Build                  Output
┌───────┬───────┐  ┌───────┬───────┐  ┌─────────┬─────────┬
│  idx  │  key  │  │  idx  │  key  │  │probe_idx│build_idx│
├───────┼───────┤  ├───────┼───────┤  ├─────────┼─────────┼
│   0   │   1   │  │   0   │   1   │  │    0    │    0    │
│       │       │  │       │       │  │         │         │
│   1   │   2   │  │   1   │   1   │  │    0    │    1    │
│       │       │  │       │       │  │         │         │
│   2   │   3   │  │   2   │   2   │  │    1    │    2    │
│       │       │  │       │       │  │         │         │
│   3   │   4   │  │   3   │   2   │  │    1    │    3    │
│       │       │  │       │       │  │         │         │
│   4   │   5   │  │   4   │   5   │  │    4    │    4    │
└───────┴───────┘  └───────┴───────┘  └─────────┴─────────┴─

Why do we construct the join result in a particular way? The order in which we construct the index is important because the build order is suffixed to the probe side order. For instance, if the build side order is a,b and the probe side order is x,y, then we expect the output order to be x,y,a,b.

@metesynnada thank you for you explanation! I've finally got it now and indeed, it has nothing in common with join operation.

Why don't we construct the join result in this way?
The main reason -- is that I've underestimated the importance of preserving build-side order and reused current implementation of update_hash / build_equal_condition_join_indices -- now I see that it's incorrect.

Regarding how to fix it prior to merging this PR -- I'm going to file separate issue regarding FIFO HashMap and allowing HashJoinExec to produce output in correct order (as in your example) while iterating in natural order (reverse iteration that we have now helps to maintain the order, but doesn't allow partial output without processing the whole batch).
I'll also mark this PR as WIP again, cause current behaviour is incorrect.
Thanks again for you patient explanations.

Thanks for the collaboration, @korowa! I will review it again when it is ready.

Update here is that with https://github.com/apache/arrow-datafusion/pull/8538 I think the idea in this PR can be ressurected

This PR is so exciting!

I hope I'll be able to rework all the logic from last commit in more handsome way during this week.

cc @alamb @metesynnada
I suppose this version to be ready for review. Could you, please, take a look when you have some time?
Just in case -- tpch_mem results for this version are
--------------------
Benchmark tpch_mem.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃   master ┃ hash_join_batch_size ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1     │ 219.32ms │             222.32ms │    no change │
│ QQuery 2     │  41.97ms │              42.19ms │    no change │
│ QQuery 3     │  98.34ms │              96.49ms │    no change │
│ QQuery 4     │  91.24ms │              93.54ms │    no change │
│ QQuery 5     │ 158.88ms │             159.20ms │    no change │
│ QQuery 6     │  25.88ms │              26.33ms │    no change │
│ QQuery 7     │ 349.07ms │             373.73ms │ 1.07x slower │
│ QQuery 8     │  69.36ms │              72.65ms │    no change │
│ QQuery 9     │ 158.26ms │             159.75ms │    no change │
│ QQuery 10    │ 160.81ms │             164.02ms │    no change │
│ QQuery 11    │  48.31ms │              47.87ms │    no change │
│ QQuery 12    │  73.05ms │              72.87ms │    no change │
│ QQuery 13    │ 113.89ms │             118.59ms │    no change │
│ QQuery 14    │  27.29ms │              30.08ms │ 1.10x slower │
│ QQuery 15    │  45.36ms │              46.13ms │    no change │
│ QQuery 16    │  55.31ms │              55.95ms │    no change │
│ QQuery 17    │ 156.56ms │             162.25ms │    no change │
│ QQuery 18    │ 476.42ms │             527.85ms │ 1.11x slower │
│ QQuery 19    │  64.40ms │              69.18ms │ 1.07x slower │
│ QQuery 20    │ 122.47ms │             127.80ms │    no change │
│ QQuery 21    │ 346.16ms │             361.06ms │    no change │
│ QQuery 22    │  42.12ms │              43.32ms │    no change │
└──────────────┴──────────┴──────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                   ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (master)                 │ 2944.46ms │
│ Total Time (hash_join_batch_size)   │ 3073.18ms │
│ Average Time (master)               │  133.84ms │
│ Average Time (hash_join_batch_size) │  139.69ms │
│ Queries Faster                      │         0 │
│ Queries Slower                      │         4 │
│ Queries with No Change              │        18 │
└─────────────────────────────────────┴───────────┘

Thanks @korowa - I will try to review this carefully tomorrow (though it is a holiday so I may not get to it until Tuesday)

I forgot to review this PR even though it was on my shortlist, will review it tomorrow.

My plan for this PR is to stare at it a bit more under the profiler and try and get back some of the lost performance so we can merge it

I reran the benchmarks any my conclusion is that now this branch shows no change in performance 🚀
--------------------
Benchmark tpch_mem.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃ main_base ┃ hash_join_batch_size ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  207.99ms │             211.40ms │     no change │
│ QQuery 2     │   45.81ms │              45.95ms │     no change │
│ QQuery 3     │   76.38ms │              76.74ms │     no change │
│ QQuery 4     │   72.99ms │              75.40ms │     no change │
│ QQuery 5     │  120.53ms │             121.80ms │     no change │
│ QQuery 6     │   16.81ms │              16.51ms │     no change │
│ QQuery 7     │  308.28ms │             317.97ms │     no change │
│ QQuery 8     │   82.58ms │              81.75ms │     no change │
│ QQuery 9     │  124.32ms │             125.46ms │     no change │
│ QQuery 10    │  152.78ms │             154.17ms │     no change │
│ QQuery 11    │   35.34ms │              34.39ms │     no change │
│ QQuery 12    │   71.18ms │              68.53ms │     no change │
│ QQuery 13    │   88.17ms │              82.83ms │ +1.06x faster │
│ QQuery 14    │   26.07ms │              26.46ms │     no change │
│ QQuery 15    │   62.80ms │              62.76ms │     no change │
│ QQuery 16    │   46.19ms │              45.84ms │     no change │
│ QQuery 17    │  148.89ms │             152.50ms │     no change │
│ QQuery 18    │  416.13ms │             435.87ms │     no change │
│ QQuery 19    │   62.67ms │              61.54ms │     no change │
│ QQuery 20    │  120.45ms │             113.52ms │ +1.06x faster │
│ QQuery 21    │  335.83ms │             366.62ms │  1.09x slower │
│ QQuery 22    │   31.35ms │              29.72ms │ +1.05x faster │
└──────────────┴───────────┴──────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                   ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (main_base)              │ 2653.54ms │
│ Total Time (hash_join_batch_size)   │ 2707.72ms │
│ Average Time (main_base)            │  120.62ms │
│ Average Time (hash_join_batch_size) │  123.08ms │
│ Queries Faster                      │         3 │
│ Queries Slower                      │         1 │
│ Queries with No Change              │        18 │
└─────────────────────────────────────┴───────────┘

Nice work @korowa

Actually, I should say "really nice work"

Hi @korowa, I think there is a good improvement here.
Do you think that we can easily adapt the same mechanism into nested loop join? This mechanism should be there as well, do you think we can make it without code duplication? If so, we can merge this PR and open an issue for nested loop join.

Hi @korowa, I think there is a good improvement here.
Do you think that we can easily adapt the same mechanism into nested loop join? This mechanism should be there as well, do you think we can make it without code duplication? If so, we can merge this PR and open an issue for nested loop join.

Good call @metesynnada  -- I have filed https://github.com/apache/arrow-datafusion/issues/8952 to track this issue

Thanks again everyone for their reviews and @korowa  for sticking with this. It is both really appreciated and a really nice PR


add emailsender and webmail

Related to #748
Email sender provide function to send plain test or html email using templates and high level functions to send specific emails as:

SendAccountVerificationEmail(to string, accountName string, token string) error

SendAccountResetPasswordEmail(to, string, accountName string, token string) error

SendAccountPasswordConfirmationEmail(to string, accountName string)

these email answers are managed by webmail module which receive the answer, execute the right appcelerator account functions and send back htrml page considering the account functions answers.
these module needs change on amp configuration side:

have configuration readable outside from cmd module (so move in amp/pkg/config)
add mail server setting in configuration
add webmail setting in configuration
separate server address from port and have haproxy understandable host format for the server address as for instance "local.amplifier.io instead of localhost or <IP_ADDRESS>, needed to allow an answer from the email client.
also related to #745

tests:

hack/amptools make -f Makefile.refactor.make cleanall install-deps
make install
make test

It exists also temporary commands to test emails manually:

amp pf sendVerificationEmail [targeted email] [accountName] [token]
amp pf sendResetPasswordEmail  [targeted email] [accountName] [token]

these commands will be deleted when accounts module will be merged
for theses emails, i used recommendations from:

https://postmarkapp.com/blog/password-reset-templates-best-practices
http://99rabbits.com/email-sign-up-confirmation/
https://reallygoodemails.com/category/inaugural/account/
https://reallygoodemails.com/category/password-reset/

do big pr, cut into smaller pieces


Update instructions for tutorials

The instructions for tutorials should be updated now that we use a new structure: https://docs.b-cubed.eu/dev-guide/tutorials/#creating-a-new-tutorial

Fixed in https://github.com/b-cubed-eu/documentation/pull/39


Fixed pattern replacements in Filter.php

Fixed bad pattern replacements. Eliminates warnings when used as PHP Sniffer extension in VSCode.

Description

If full Windows path is used as ignore filter in shouldIgnorePath() function, then conversion to PREG pattern was buggy.
Windows backslash separator should be converted into double backslash (escaped backslash) AND Linux slash separator should be converted into double backslash.
Suggested changelog entry

Add backslash into double backslash replacement for Windows.
Related issues/external references
PHP Sniffer extension for VSCode generates warnings on Windows without this fix.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

[ ] This change is only breaking for integrators, not for external standards or end-users.

[ ] Documentation improvement

PR checklist

[x] I have checked there is no other PR open for the same change.
[x] I have read the Contribution Guidelines.
[ ] I grant the project the right to include and distribute the code under the BSD-3-Clause license (and I have the right to grant these rights).
[ ] I have added tests to cover my changes.
[x] I have verified that the code complies with the projects coding standards.
[ ] [Required for new sniffs] I have added XML documentation for the sniff.

@arturm11 Thank you for this PR.
I've been able to reproduce the issue with PHPCS itself via the command line using something along the lines of:
phpcs --ignore=D:\path\to\PHP_CodeSniffer\src\Tokenizers\CSS.php

which results in a:
Warning: preg_match(): Compilation failed: unknown property after \P or \p at offset * in D:\path\to\PHP_CodeSniffer\src\Filters\Filter.php on line 278

And the change made in this PR fixes this. :+1:
Before I can accept the PR, could you please have a look at the third checklist item in the "PR checklist" ?
I'd also like to see this change safeguarded via tests. There are some tests for the Filters in the tests/Core/Filters directory already. These could probably be expanded.

@jrfnl Now I am not sure. Please analyze Ignoring Files and Folders section in the wiki. It is stated, that filter can be treated as regular expression. Backslash character is special character for RE. Therefore Windows paths shouldn't be accepted and backslash should be converted to slash in Windows path on input.

There is related bug in vscode-php-sniffer, so I close this request.
https://github.com/wongjn/vscode-php-sniffer/pull/73

@arturm11 Thanks for getting back to me. Hope that bug gets fixed soon.


Adding support for PGroonga operators

This code adds the following PGroonga operators to the BinaryOperator enum:
PGroongaFullTextSearch for the &@ operator
PGroongaEasyQuerySearch for the &@~ operator
PGroongaSimilarSearch for the &@* operator
PGroongaAdvancedSearch for the `&`` operator
PGroongaArrayKeywordSearch for the &@| operator
PGroongaArrayQuerySearch for the &@~| operator
Additionally, the fmt::Display implementation has been updated to handle these new operators appropriately.

The documentation for allowed characters in custom operators is here: https://www.postgresql.org/docs/current/sql-createoperator.html

what would you of having something like this in the tokenizer.rs ?
pub enum Token {
    EOF,
    Word(Word),
    Number(String, bool),
    Char(char),
    SingleQuotedString(String),
    DoubleQuotedString(String),
    CustomOperator(String),
}

impl<'a> Tokenizer<'a> {
    fn next_token(&mut self) -> Token {
        // Existing logic...

        // Detect and return a `CustomOperator` token if a custom operator is found.
        let start_pos = self.chars.clone();  // Save the starting position
        let mut operator = String::new();
        while let Some(&ch) = self.chars.peek() {
            // Check if the character is valid for a custom operator
            if "+-*/<>~=!@#%^&|`?".contains(ch) {
                operator.push(ch);
                self.chars.next();
            } else {
                break;
            }
        }

        if operator.len() > 1 && (operator.ends_with('+') || operator.ends_with('-')) {
            if !operator.contains('~') && !operator.contains('!') && !operator.contains('@') &&
               !operator.contains('#') && !operator.contains('%') && !operator.contains('^') &&
               !operator.contains('&') && !operator.contains('|') && !operator.contains('`') &&
               !operator.contains('?') {
                // Invalid operator according to the rules <https://www.postgresql.org/docs/current/sql-createoperator.html>
                self.chars = start_pos; // Rewind to the start position
                return Token::Char(self.chars.next().unwrap());  // Fallback to a single char token
            }
        }

        if !operator.is_empty() {
            return Token::CustomOperator(operator);
        }

        // Fallback to other token parsing logic
        // ...
    }
}

No, you would have to integrate with the existing operator tokenization logic not to break it.
It's a little bit technical, so if what you are interested in is just getting full text search to work for you, the easiest is probably for you to declare your own function that wraps pgroonga's custom operator, and can that function instead of the operator in your code to work around the current operator parsing bug.

@alamb , currently, the tokenizer is dialect-independent.
However, the same expression a&@b would be tokenized as identifier(a) operator(&@) identifier(b) in postgres and identifier(a) operator(&) identifier(@b) in Microsoft SQL Server.
Do you think it would be okay to add partial support for custom operators, ignoring the ones that end in @ and are not followed by a whitespace? Or would you require full support for differentiated tokenization before merging support for custom operators?


Changing only time does not trigger onChange event

Hi,
I noticed that when opening the picker, changing only the time of day, and closing it, no onChange event is generated. The input value and actual date are updated, but my event handler is not called. Did I miss something? (e.g. time-specific onChange event)
Thanks a lot for the project. Very useful.
Cheers!

did you tried oninput event?

@sijad oninput is not triggered when setting value via code.
@chmln Thanks! Works like a charm.

@on-input or @oninput not firing when only changing time. This is still an issue.


#439

#439 Prevent confusion shown in 439, while giving users a sane default which is probably what they want.

fixed by comment


Added Service Mesh tool @IstioMesh

Hello Awesome-Docker maintainers, - I believe we will start seeing more and more Service Mesh tools being added the next few months. We might consider adding a Service Mesh section to handle this. In the meantime, I have added it to Service Discovery as this is the closest section.
URL: https://github.com/istio/istio
Service Mesh tools aim to handle Service Discovery and a much more manageable abstraction. I would recommend everyone to read - https://istio.io/docs/concepts/what-is-istio/overview.html as this explains in great detail what Service Mesh is and the benefits.
By submitting this pull request I confirm I've read and complied with the below requirements.
Please read it multiple times. I spent a lot of time on these guidelines and most people miss a lot.

I have read and understood the contribution guidelines
yes
The project submitted conforms to the quality standards outlined in the contribution guidelines
Yes
Go to the contribution guidelines and read it again.

RULES (because you probably didn't read the guidelines):
Links will be tested with awesome_bot. Please make sure your URL to be listed is unique (do some search queries), otherwise put a link at the end of README.md file.

FORMAT

Use the following format: [Name](Link) - Description.
[Name] should be the title of the project
(Link) should be pointing to a GitHub repository, not to a website
Description. should end with a full stop/period
For GitHub repositories, add the name of the author with a link to its GitHub account after the description, like By [@author](GitHub account)

ORDER: The entries are in lexical order (English letter order)
LABEL (Optional): Add one of the following icons next to the project name like [Name](Link) :icon: - Description.

:heavy_dollar_sign: indicates if one needs a paying subscription to use the project.
:construction: indicates that the project is a work in progress/beta release.
:skull: indicates that the project is obsolete, but still worth mentioning it.

Hi @veggiemonk You were correct. I've been away too long and my fork was way outdated. I closed #465 and opened this one instead.

@vegasbrianc glad to see you back 😄


Assignment component

Please create something similar to the following. The goal of this task is to make a card that has the assignment title, due date, and short description.

I stole this component from brightspace so feel free to play around with the spacing.
Don't worry a ton about colors or styling, we will do a pass over to make everything look consistent afterwards.
You can also ignore the "dot" in the upper right corner.

@MathyouMB do I need to make a pull request to the main branch?

@MathyouMB do I need to make a pull request to the main branch?

You should be able to make a pull request to the main branch now if you're referring to how the layouts I showed in our last meeting were not merged in yet.


feat: add the ability to set custom container attributes

Description
Allows a user to add attributes to the container for a control
Summarize
Added inputContainerOptions to control.
Benefits
This is useful when integrating with third-party js solutions that expect a container to have certain attributes for the integration.

Tests fail.

Any new features added to master will need to be ported to ported to dev too.

Whats the difference between the two?

bs3 vs bs4 as in the readme.


Curriculum update needed on project_etch_a_sketch.md

The Odin's file, project_etch_a_sketch.md is updated. Please update the Kampus' file, checkout file here project_etch_a_sketch.md
Latest commits:

Add blank line below heading (additions: 1, deletions: 0) on Feb 5 2024, 17:33 UTC
Update lists per style guide (additions: 24, deletions: 24) on Feb 5 2024, 17:01 UTC

New commits have been made to the Odin's file. Please update the Kampus' file.
Latest commits:

fix: add 'the' for clarity (additions: 1, deletions: 1) on Feb 25 2024, 24:43 UTC
fix: update project etch-a-sketch 'extra credit' instructions (additions: 1, deletions: 1) on Feb 25 2024, 24:42 UTC

New commits have been made to the Odin's file. Please update the Kampus' file.
Latest commits:

Etch-a-Sketch: Adds a hint to the second optional credit assignment (#27633) (additions: 13, deletions: 11) on Mar 19 2024, 15:55 UTC

New commits have been made to the Odin's file. Please update the Kampus' file.
Latest commits:

Etch-a-Sketch Project: Fix extra credit typo  (#28012) (additions: 3, deletions: 1) on May 20 2024, 13:21 UTC


Visual debugger

Once the sdb tool is finished, it would be great to add a non-cli component. The thought would be to use JavaFX to provide a visual interface similar to Eclipse's standalone debugger, Nemiver (screenshot), or gede.
Could use ScalaFX to provide a nicer syntax for the UI. Would be nice to be able to distribute native applications using something like the sbt native packager. Could potentially use that instead of sbt-assembly for sdb as well.

This could potentially be another Google summer of code project unless I get to it before next summer.

Also, since JavaFX (and therefore ScalaFX) require JDK7+, there should be something like this in the project file for the visual debugger module that prevents earlier JDKs from being used.

Some discussion about syntax highlighting in JavaFX: http://stackoverflow.com/questions/19603457/what-is-javafx-equivalent-of-jsyntaxpane-for-making-a-code-editor


After VS CTP5 "Couldn't determine an appropriate version of runtime to run."

I've been playing around with the latest dev-builds for a MVC web application (the latest beta3-* builds at the moment). I installed the CTP5 of Visual Studio, since then I've been unable to get my projects working again. Whenever I try to run an application using IIS express, I immediately get the following yellow screen of death exception:
[InvalidOperationException: Couldn't determine an appropriate version of runtime to run. See http://go.microsoft.com/fwlink/?LinkId=517742 for more information.]
   System.Web.HttpRuntime.HostingInit(HostingEnvironmentFlags hostingFlags, PolicyLevel policyLevel, Exception appDomainCreationException) +303

[HttpException (0x80004005): Couldn't determine an appropriate version of runtime to run. See http://go.microsoft.com/fwlink/?LinkId=517742 for more information.]
   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +9920856
   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +101
   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +254

Unfortunately the provided link only redirects to the ASP.NET website. Google does not return anything, but it seems that others have this problem as well.
I run Windows 10 Preview (build 9926) and KRE-CLR-x<IP_ADDRESS>-beta3-10921.

I'm having exactly the same problem and I can't determine a work-around that works using IIS Express. Using the "k web" run target still works for me, though.

I'm having the same problem too.
I think the problem is that kvm renamed to dotnetsdk.
All nightly assemblies have a beta4-* version, and some targeted to .net 4.6 (v4.0.10 references).
I'm try to install dotnetsdk. All is ok, but VS 2015 CTP5 want %userprofile%.kre and packages named kre-*
problem is probably here:
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\AspNet

will be nice to get fixed aspnet tasks for VS15 CP5 ;-)

Thats the problem using the bleeding edge builds. Unfortunately there's no easy work around right now as things have been renamed.

OK, thanks for the update, @davidfowl. Good to know this is only a "temporary" issue with the current builds.

@Boglen How did you install dotnetsdk? I couldn't find an install script just like kvm/kre.

Looking forward to hear this solved

@ninety7 yesterday they roll back the changes, as I understand

waiting while dotnetsdk -> kvm renamed

Found solution:
I've deleted kvm and reinstalled from dev branch, install latest version of kre (beta4-11019).
Then i create hardlink from new .k directory to .kre (mklink /j .kre .k), create subdir runtimes hardlink to packages, rename. Duplicate package kre-clr-win-x<IP_ADDRESS>-beta4-11019 to kre-clr-x<IP_ADDRESS>-beta4-11019 (duplicate, not rename) and most impartant: place in wwwroot folder of your project web.config file

<system.web>

</system.web>

and debug in IISExpress work now.

Next problem is that VS15 CTP5 with last kre build, moved from .kre to .k. and renamed from kre-clr-x<IP_ADDRESS>-* kre-clr-win-x<IP_ADDRESS>-* :-) won't publish to filesystem:
Cannot find KRE package entityframework.commands in the folder:
Cannot find KRE package entityframework.commands in the folder: %userprofile%.kre\packages
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\AspNet\Microsoft.Web.AspNet.targets(108,5): Error : The ProjectK Runtime package needs to be installed. See output window for more details.

I have the same issue but am unclear how to move forward.
I run KRE version KRE-CLR-x<IP_ADDRESS>-beta2
This was working until I upgraded the KRE version to beta4 .. I have reinstalled KRE and reset the Visual Studio installation but am still getting the  error below. Am at a loss as to how to proceed.. any feedback will be appreciated.
Couldn't determine an appropriate version of runtime to run. See http://go.microsoft.com/fwlink/?LinkId=517742 for more information.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: Couldn't determine an appropriate version of runtime to run. See http://go.microsoft.com/fwlink/?LinkId=517742 for more information.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:

[InvalidOperationException: Couldn't determine an appropriate version of runtime to run. See http://go.microsoft.com/fwlink/?LinkId=517742 for more information.]
   System.Web.HttpRuntime.HostingInit(HostingEnvironmentFlags hostingFlags, PolicyLevel policyLevel, Exception appDomainCreationException) +506

[HttpException (0x80004005): Couldn't determine an appropriate version of runtime to run. See http://go.microsoft.com/fwlink/?LinkId=517742 for more information.]
   System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +9931632
   System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +101
   System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +191

To make it all work go back to beta2, visual studio ctp5 won't work with anything newer. There were a bunch of things that got renamed that will cause grief.
To go back, just set the default ailias back to beta2, kvm alias default 1.0.0-beta2.

Thanks for the reply @davidfowl
It helped me understand my issue a little better.
Unfortunately I am using  beta4 dlls in my project which leaves me in a very unpleasant state until this gets resolved. Fingers crossed we gets a fixed drop soon.

Beta4 work fine with ctp5. Create web.config in wwwroot, copy kre-clr-win* in kre-clr* and make hardline from runtimes to packages

@Boglen thanks for your reassurance that we can get it working, but it would be excellent if you could please be more specific about what people need to do, for example:

what should be the contents of web.config?
copy kre-clr-win* to where?
I don't understand what you mean by "hardline from runtimes to packages".

Thanks!

Open CMD.
cd %userprofile%
mklink /J .kre .k
then open c:%userprofile%.k\runtimes folder. Create duplicate of "kre-clr-win-x<IP_ADDRESS>-beta4-11037"
and rename to "kre-clr-x<IP_ADDRESS>-beta4-11037".
Copy c:%userprofile%.k\runtimes
To c:%userprofile%.k\packages
Open VS 2015 CTP5, open your project and define Target KRE version in properties to kre-clr-x<IP_ADDRESS>-beta4-11037.
Then open "wwwroot" folder in your project, create web.config file:

Profit!
bootstrapper-version - version of AspNet.Loader.dll

@Boglen I've done the steps. Magical version numbers at the time of writing are: 1.0.0-beta4-11117 and 1.0.0-beta4-11122 for the AspNet.Loader.dll.
Problem is the Visual Studio says in the Solution Explorer: References (Errors - see error list) in error list there is "Illegal characters in path."
I've deleted .kpm/packages it didn't hlep.

I got a new version using "kvm upgrade" yesterday, "11128" it works with the instructions above. What's more, I also got the Mvc debugging to work with this version.

@Ciantic yep. All you need to do - just rename the folders.  I think that CTP6 will coming soon and there will be corrected path

CTP6 came out today. Get it now.

I am getting same error saying that Couldn't determine an appropriate version of runtime to run when I try to execute iisexpress.exe. The command I ran:

"C:\Program Files\IIS Express\iisexpress.exe" /path:"D:\github\confidence\artifacts\apps\ConfyConf.Client.Web\wwwroot" /port:9090 /clr:v4.0

wwwroot folder I am targeting here is created by dnu publish and web.config is:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
	<appSettings>
		<add key="bootstrapper-version" value="1.0.0-beta4-11526" />
		<add key="runtime-path" value="..\approot\packages" />
		<add key="dnx-version" value="" />
		<add key="dnx-clr" value="" />
		<add key="dnx-app-base" value="..\approot\src\ConfyConf.Client.Web" />
	</appSettings>
</configuration>

I am also getting the below iisexpress.exe output:

I added the below system.web node to web.config but it didn't change anything:
	<system.web>
		<compilation debug="false" targetFramework="4.5.1" />
		<httpRuntime targetFramework="4.5.1" />
	</system.web>

I have the same issue as tugb

@davidfowl suggested to me in Home #452 (comment) to try dnu publish --runtime active to populate the dnx-version and dnx-clr of the generated Web.config. Does that work for you?


Bug 2093357: ACM-ICE: provide cluster claims to allow upgrades complete

Since ICE driver must be loaded before kubelet, we need the spokes to
get the image matching the running kernel, and this happens halfway
into the upgrade. Once the node is updated (and upgrade aint finished yet)
the new kernel is running and no updates are made towards ACM in the hub,
meaning the spoke will be waiting for an image that wont ever get built
until managedcluster in the hub is updated.
To fix this, we create a cluster claim looking at the desired image in the
spoke, allowing for fast builds before the upgrade even begins.

/hold

/unhold

/cc @cdvultur

/hold

/unhold

/lgtm


snack-sdk does not support SDK 51

Summary
The Expo Go iOS and Android apps now support SDK 51 exclusively, and cannot be downgraded. snack-sdk only supports SDK versions 48.0.0, 49.0.0, and 50.0.0 (see definition of SDKVersion in snack-content/build/sdks/types.d.ts). This means that if you download and install the latest version of Expo Go, you are locked out of running a snack generated via snack-sdk on your device.
Expected behaviour:

Create Snack via snack-sdk
Scan QR code with mobile device
Successfully run snack on device

Actual behaviour:

Create Snack via snack-sdk
Scan QR code with mobile device
Receive error message - Snack targets SDK 50 but installed Expo Go only supports SDK 51

What platform(s) does this occur on?
Android, iOS
SDK Version
51
Reproducible demo or steps to reproduce from a blank project

Create any Snack via snack-sdk
Install latest version of Expo Go on a mobile device
Scan QR code from snack

Actually it looks like this was addressed by https://github.com/expo/snack/pull/577 (thanks @byCedric!). Any idea when this could be published? I'm working on a project using snack-sdk that's now broken until the next update

Hi @biadars! We've just published<EMAIL_ADDRESS>(had to bump the major version due to incorrect nested dependency constraints). This supports SDK 51!
Let me know if anything doesn't work for you

That was super quick! I've upgraded to the latest version and everything's working again, thanks for the swift fix 🙏


Error: Getting requirements to build wheel did not run successfully

Hello,
I am struggling to install rnanorm. When I type "pip install rnanorm" in jupyter notebooks I get the following error. I have tried to install with python versions 3.8.3 and 3.10.9, but get the same error.
print(sys.version)
3.8.3 (default, Jul  2 2020, 17:30:36) [MSC v.1916 64 bit (AMD64)]

Collecting rnanorm
  Using cached rnanorm-1.5.1-py3-none-any.whl (11 kB)
Requirement already satisfied: numpy in c:\users\cathe\anaconda3\lib\site-packages (from rnanorm) (1.24.3)
Requirement already satisfied: pandas in c:\users\cathe\anaconda3\lib\site-packages (from rnanorm) (1.0.5)
Requirement already satisfied: scipy in c:\users\cathe\anaconda3\lib\site-packages (from rnanorm) (1.5.0)
Collecting pybedtools (from rnanorm)
  Using cached pybedtools-0.9.0.tar.gz (12.5 MB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: python-dateutil>=2.6.1 in c:\users\cathe\anaconda3\lib\site-packages (from pandas->rnanorm) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in c:\users\cathe\anaconda3\lib\site-packages (from pandas->rnanorm) (2020.1)
Requirement already satisfied: six in c:\users\cathe\anaconda3\lib\site-packages (from pybedtools->rnanorm) (1.15.0)
Collecting pysam (from pybedtools->rnanorm)
  Using cached pysam-0.21.0.tar.gz (4.1 MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
Note: you may need to restart the kernel to use updated packages.
  error: subprocess-exited-with-error

  Getting requirements to build wheel did not run successfully.
  exit code: 1

  [34 lines of output]
  # pysam: cython is available - using cythonize if necessary
  # pysam: htslib mode is shared
  # pysam: HTSLIB_CONFIGURE_OPTIONS=None
  '.' is not recognized as an internal or external command,
  operable program or batch file.
  '.' is not recognized as an internal or external command,
  operable program or batch file.
  # pysam: htslib configure options: None
  Traceback (most recent call last):
    File "C:\Users\cathe\anaconda3\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "C:\Users\cathe\anaconda3\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:\Users\cathe\anaconda3\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\cathe\AppData\Local\Temp\pip-build-env-6kaeyrrh\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "C:\Users\cathe\AppData\Local\Temp\pip-build-env-6kaeyrrh\overlay\Lib\site-packages\setuptools\build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "C:\Users\cathe\AppData\Local\Temp\pip-build-env-6kaeyrrh\overlay\Lib\site-packages\setuptools\build_meta.py", line 487, in run_setup
      super(_BuildMetaLegacyBackend,
    File "C:\Users\cathe\AppData\Local\Temp\pip-build-env-6kaeyrrh\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 383, in <module>
    File "<string>", line 79, in run_make_print_config
    File "C:\Users\cathe\anaconda3\lib\subprocess.py", line 411, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "C:\Users\cathe\anaconda3\lib\subprocess.py", line 489, in run
      with Popen(*popenargs, **kwargs) as process:
    File "C:\Users\cathe\anaconda3\lib\subprocess.py", line 854, in __init__
      self._execute_child(args, executable, preexec_fn, close_fds,
    File "C:\Users\cathe\anaconda3\lib\subprocess.py", line 1307, in _execute_child
      hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
  FileNotFoundError: [WinError 2] The system cannot find the file specified
  [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

Getting requirements to build wheel did not run successfully.
exit code: 1

See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

In addition, I would like to confirm. I have an output file from feature counts that looks like this. Will rnanorm be able convert this into TPM?
geneID  geneLength  read counts
ENSG00000121410 4006    3
ENSG00000268895 2793    3
ENSG00000148584 9603    11
ENSG00000175899 6384    1

Hi @CthangavUCI
Which OS are you using? Windows? If so, issues with installing of (py)bedtools are well known for Windows users.
Are you familiar with running things in Docker?
The other option would be to install the rnanorm 2.0.0, which is not yet published o Pypi. 2.0.0 does not depend on pybedtools.

In addition, I would like to confirm. I have an output file from feature counts that looks like this. Will rnanorm be able convert this into TPM? I do not have access to the original BAM file.

This is fine, do you have access to the GTF file used to obtain the expressions counts?

Thank you very much for your reply. Could you give me instructions on how
to access/install rnanorm 2.0.0? I can try to download docker if you have
advice on how to do that- I may try both methods and see which one works.
Also, just to confirm, I only have a featurecounts output file containing
the counts and feature lengths- is that sufficient for RNAnorm to calculate
TPMs?
THank you again for your help!
On Tue, Jun 13, 2023 at 1:14 AM Jure Zmrzlikar @.***>
wrote:

Hi @CthangavUCI
https://urldefense.com/v3/__https://github.com/CthangavUCI__;!!CzAuKJ42GuquVTTmVmPViYEvSg!ODBHCXrzVPlyoZvj6yfBa6O5zDD9vT6WTANEoQYosjyt-Kz_gQMMrG8hZ91IRnwKwktEuS15_9I4BEASu-7hfO3z$
Which OS are you using? Windows? If so, issues with installing of
(py)bedtools are well known
https://urldefense.com/v3/__https://www.biostars.org/p/9561321/__;!!CzAuKJ42GuquVTTmVmPViYEvSg!ODBHCXrzVPlyoZvj6yfBa6O5zDD9vT6WTANEoQYosjyt-Kz_gQMMrG8hZ91IRnwKwktEuS15_9I4BEASuzZoJExh$
for Windows users.
Are you familiar with running things in Docker?
The other option would be to install the rnanorm 2.0.0, which is not yet
published o Pypi. 2.0.0 does not depend on pybedtools.
—
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/genialis/RNAnorm/issues/18*issuecomment-1588780728__;Iw!!CzAuKJ42GuquVTTmVmPViYEvSg!ODBHCXrzVPlyoZvj6yfBa6O5zDD9vT6WTANEoQYosjyt-Kz_gQMMrG8hZ91IRnwKwktEuS15_9I4BEASu-ivbrgy$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ATZSYG6LJ655R7GZPVLC5XDXLAOPDANCNFSM6AAAAAAZD6JM74__;!!CzAuKJ42GuquVTTmVmPViYEvSg!ODBHCXrzVPlyoZvj6yfBa6O5zDD9vT6WTANEoQYosjyt-Kz_gQMMrG8hZ91IRnwKwktEuS15_9I4BEASu7kA47Kn$
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Chloe Thangavelu
PhD Candidate, University of California, Irvine
Department of Biological Chemistry

Hi @CthangavUCI
If you only have is FeatureCounts file than I suggest the following
Install rnanorm 2.0.0 release candidate with: pip install rnanorm==2.0.0.rc2
Warning: this is development version. It should work fine, but not 100% bulletproof.
If this works you can use the (1)command line or (2) Python

To normalize your expressions with TPM in command line use:
rnanorm tpm exp.csv --gene-lengths lengths.csv

The format of the exp.csv must be samples in rows and genes in columns, e.g.
,ENSG00000121410,ENSG00000268895,...
SAMPLE_1,31,54, ...
SAMPLE_2,26,34, ...
...

The format of the lengths.csv must be two columns, first gene IDs, second gene lengths, e.g.
gene_id,gene_length
ENSG00000121410,4006
ENSG00000268895,2793
ENSG00000148584,9603
ENSG00000175899,6384
...

To normalize your expressions with TPM in Python use:

import pandas as pd
from rnanorm import TPM

fc = pd.read_csv(<fc-file>, sep="\t", index_col=0)
gene_lengths = fc["geneLength"]
exp = fc[["read counts"]].T

exp_tpm = TPM(gene_lengths=gene_lengths).fit_transform(exp)

Thank you very much- option 2 worked in python for me. Is the code similar to get TMM normalization? Something like:
exp_tmm = TMM().fit_transform(exp)

Is the code similar to get TMM normalization? Something like:
exp_tmm = TMM().fit_transform(exp)

Exactly. All the methods have as similar interface as possible.


Enable serialization of the connection

Adds the SerializesConnections trait to the Connection class, enabling its serialization.
It is sometimes necessary to serialize the connection, as it is dispatched in common events such as MessageSent which may need storage. Listeners may wish to automatically queue the handling of such events by implementing ShouldQueue. This currently results in a "serialization of 'closure' is not allowed" error, as the connection object passed in the event is not serializable.
A workaround is to manually create the queue job in the listener's handler, by first parsing the connection object. However this sometimes isn't ideal and adds complexity.
In the context of Reverb, I think it's particularly important the handling of events are easily queued to prevent blocking of the event loop.

Thanks @richt222. Does this work with queued jobs immediately when adding this trait or is there still some work that needs to be done?

I dug into this some more and it's not actually possible to serialize the underlying connection, meaning you can't actually send any message to the connection once it's been unserialized. So, going to close this one since I don't think serializing it is providing much value.

Thanks @joedixon. Yep, I agree. I also spent a good while trying to figure out if the underlying connection could be serialized and I think, although possible, it would be a big task and ultimately prone to failures.
The only slight frustration is that common events such as MessageSent can't be automatically queued as the event includes the connection. So listeners either require an extra step to create the queued job, minus the connection, or they get handled synchronously, which can obviously block the event loop.

So, event though the SerializesConnections exist, it's not implemented on the Connection, which means that the listeners bound to Reverb events cannot be queued.
So, out of the box, there's no way to queue the listener for the MessageReceived event to parse/process the json payload and handle it however, without creating and manually dispatching a custom job from the "sync" listener.
Did I get this right?
Without diving too deep, is there a reason why the MessageReceived also has the Connection object passed along with stringified message?
Thanks :)


Org-Latex-Export-To-LaTeX Fail To Handle Code Blocks When Specifying Prog-Lang

What did you expect to happen?
Producing a proper pdf file with code block and highlighting.
What actually happened?
when exporting from org to latex, it does produce well formated tex file , however that's not how it goes for code blocks .
if we specified prog-language name after #+BEGIN_SRC , e.g:

And Export it to latex , it adds [] like below

which if we then compile it to pdf, it doesn't show the code block

Describe your attempts to resolve the issue
1 - if we remove [] manually , and then compile it . it produces the following correct pdf.
however it is stressful and requires removing [] everytime you export to latex.

2 - if we remove prog-language after #+BEGIN_SRC, it will show the code blocks, however without code highlighting since minted cannot determine which language it is thus will not not work .
Steps to reproduce

Open Emacs
Set The Following Lines in Ur Config

(require 'ox-latex)
(setq org-latex-listings 'minted)

Open Your Org File And Set.

#+LATEX_CLASS: article
#+LATEX_CLASS_OPTIONS: [letterpaper]
#+LATEX_HEADER: \usepackage{minted}
... snip...
#+BEGIN_SRC lisp
  (defun Fib (n)
    (if (< n 2) n (+ (Fib (- n 1)) (Fib (- n 2)))))
#+END_SRC

run org-latex-export-to-latex or org-latex-export-to-pdf and see

System Information
https://pastebin.com/EWwmfRYu

Please verify that this is a Doom-specific issue by attempting to reproduce with emacs -Q. If it is not, you'd be best off reporting this as a Org bug.

Apparently, this also can be reproduced with emacs -Q

Minted uses pygments which requires -shell-escape when compiling to pdf .
This Issue can be fixed with the following.
  (setq org-latex-pdf-process
        '("pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"
          "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"
          "pdflatex -shell-escape -interaction nonstopmode -output-directory %o %f"))


add: pnpm catalog

This PR will add pnpm catalog for deps that is reused through the repo. Making it easier to maintain. I will open this in draft mode so we can continue working on it. The alternative was using something like syncpack, but that would add another dep so I decided to go with pnpm catalogs instead.
Any suggestions on what more we could add? I have considered using catalog for the other shared deps in examples/*, like f.ex tailwindcss/typescript or postcss. Not sure if we want that, and also should we consider using a caret or tilde in any of the catalog versions?
Btw, I have tested building and deploying the examples apps with this PR, and it works as it should.

@sommeeeer Yeah we should add these 3 deps and bump typescript version as well.
And we don't want to use ^ or ~, we should always use exact where we can

Yeah we should add these 3 deps and bump typescript version as well.

i didnt add catalog to the open-next package, could however use it for @types/node and typescript. not sure what we want?


iOS does not shows the apps like gmail

Hi,
The plugin looks awesome and it saves me time.
I am facing a problem or maybe I am missing something with sharing a file with Gmail, It works in android but iOS does not show the list of apps in share extension.
is there something I need to do apart from installing the dependency in pubspec file?
The more option shows an empty list

can you try example app of this repository? It's strange this result.

I'm stuck in the same issue

@aravindhkumar23 can we see your code?
I ALMOST solved my problem.
void _sharePdf(String path) {
    final uri = Uri.file(path);
    SimpleShare.share(
        type: 'PDF',
        uri: path,
        title: "Share my file",
        msg: "My message");
  }

This is how I'm doing it now. The path is resolved and the file can be attached to an email. The only issue now is that the attached file doesn't have PDF extension even if the original file have it   :(

@giovapanasiti
I haven't used type as pdf, my file is filename.txt
this is how I'm doing will it accept text as a type?
final uri = Uri.file(path);
    SimpleShare.share(
        subject: "Log file",
        uri: path,
        title: "Titel",
        msg: "message");

from what I've seen it has the behaviour you described when it doesn't find the file. Try to keep an eye on the log to see if any error message shows up.

sorry, in this period i haven't iPhone device, i cant try


New theme: Dracula

I am using the Dracula theme for some tools and just saw that the Android app Fedilab implemented a Dracula theme as alternative to light, dark and other themes. That is awesome!
I would love to have that for To Don't and maybe other apps too.
Are you fine with that, @dracula?

Hey can i work on this issue?

Did you start with this already, @theftzoku?


Send magic link upon new user registration

I want to send magic link upon new user registration so the magic link can also act as a way to verify user email.
Currently my User model looks like this.
  devise :database_authenticatable, :registerable,
         :recoverable, :rememberable, :validatable,
         :trackable, :magic_link_authenticatable

As you can see, I don't use :confirmable, so currently the moment I create an account, I am automatically signed in. However, I want to change it so that when new user registers, they get a magic link too in order to log in.
How can I achieve it?

You should be able to send magic links after the account is creating by adding a callback like this to your model (untested):
after_commit -> () { send_magic_link }, on: :create

It's not clear to me from your message if you want to disable the automatic sign-in after the user creates an account, but if you do want that, you can create a custom RegistrationsController for your User model and override the sign_up method (https://stackoverflow.com/a/18545483/215168):
$ rails g devise:controllers users -c=registrations

# app/controllers/users/registrations_controller.rb
class Users::RegistrationsController < Devise::RegistrationsController
  def create
    super
  end

  protected

  def sign_up(resource_name, resource)
    true
  end
end

# config/routes.rb
devise_for :users,
  controllers: {
    registrations: "users/registrations",
    sessions: "devise/passwordless/sessions"
  }

Hello Abe! First of all, thank you so much for your prompt reply.
This is exactly what I wanted. I wanted to prevent users from being automatically signed in since I will be triggering the magic link, and only let them in if they click the link in the email.
Is there a simple way to know when they log in via the magic link and mark their email as "verified" in my DB? This would apply to either new users who sign up and log in via the magic link for the first time or old users who already have accounts prior to the magic link implementation.

Assuming you're using ActiveRecord and have a datetime field named :confirmed_at to track this, you could implement it like this (untested):
class User < ApplicationRecord
  # ...

  def after_magic_link_authentication
    update!(confirmed_at: Time.current) unless confirmed_at
  end
end

This uses the callback method after_magic_link_authentication which is invoked after every successful magic link authentication.

Closing this now as discussion seems to have gone dormant - feel free to reopen an issue or a discussion if you have more questions


Picamera2 with raspberry pi global shutter camera returns one color background image instead of the real image

Hello,
I have a raspberry pi global shutter camera  on a  raspberry pi 4 model B
I ran this code:
#!//usr/bin/python3

from picamera2 import Picamera2

picam2 = Picamera2()
config = picam2.create_still_configuration()
picam2.configure(config)

picam2.start()

np_array = picam2.capture_array()
print(np_array)
picam2.capture_file("demo.jpg")
picam2.stop()

And it produces this image:

yes it does, I am wondering, is having a lens mandatory ? because I don't have one

Yes, you need to get a lens of some sort, maybe like this https://thepihut.com/products/raspberry-pi-high-quality-camera-lens (other lenses and suppliers are available, this is just one example)

so, the problem comes from the fact that I don't have a lens ? thanks

Yes. All cameras need lenses.
An image sensor by itself will just capture all of the light that hits it. Lenses focus the light into a coherent image.
https://en.wikipedia.org/wiki/Camera_lens
The one exception would be if you create a pinhole camera, but even then the pinhole is considered a lens.
https://en.wikipedia.org/wiki/Pinhole_camera

Hello, I have put the lens, but it now returns a black image

Hi, could you please double-check that you've opened the lens aperture. Might it be possible to attach a small image of your Pi and the connected camera? Thanks.

Ok, I found the solution, it was a matter of setting the lens (more light, more focus)


64-bit OS Detection causing script to fail on ARM processors (Surface Pro X)

When you try to run the "(new-object Net.WebClient).DownloadString('https://bit.ly/LTPoSh') | iex;" command on a Surface Pro X (64-bit custom ARM CPU), it tries to switch to 64-bit PowerShell and creates an endless loop of spawned Powershell.exe processes.
Output looks like:
PS C:\windows\system32> (new-object Net.WebClient).DownloadString('https://bit.ly/LTPoSh') | iex;
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.

https://oofhours.com/2020/02/04/powershell-on-windows-10-arm64/ this site indicates that 64-bit tests should exclude the ARM processor architecture to avoid these issues.

Please test the branch at https://github.com/DarrenWhite99/LabTech-Powershell-Module/tree/ARM64-Dev .  Once we can verify that it works (without looping at least) under ARM64 I will complete the pull and bring the fixes to the main branch here.

Running unmodified, it still seems to loop, but now when I kill it at least the processes clean up without a task kill.
PS C:\windows\system32> (new-object Net.WebClient).DownloadString('https://raw.githubusercontent.com/LabtechConsulting/LabTech-Powershell-Module/4db32d746a3c1253eaa6cebbc7f834c402164a0e/LabTech.psm1') | iex WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands. WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands. WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands. WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
Copied into ISE and ran it, same results. When I change line 53 from Write-Debug to Write-Warning, I get:
`WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
WARNING: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
Program 'powershell.exe' failed to run: The filename or extension is too longAt line:131 char:9

    &"$pshell" -NonInteractive -NoProfile $myInvocation.MyCommand

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.

At line:131 char:9

    &"$pshell" -NonInteractive -NoProfile $myInvocation.MyCommand

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : ResourceUnavailable: (:) [], ApplicationFailedException
FullyQualifiedErrorId : NativeCommandFailed

WARNING: Exiting 64-bit session. Module will only remain loaded in native 64-bit PowerShell environment.`
And then a script break/exit. I tested enabling long paths in the registry and rebooting, but that had no effect.
If I change line 49 to:
If ($env:PROCESSOR_ARCHITEW6432 -match '64' -and [IntPtr]::Size -ne 8 -and $env:PROCESSOR_ARCHITEW6432 -ne 'ARM64')
It runs without error and I was able to execute the functions.

To get more information on the choices is it making, try this:
powershell.exe "$DebugPreference=$VerbosePreference='Continue'; iex (new-object Net.WebClient).DownloadString('https://raw.githubusercontent.com/DarrenWhite99/LabTech-Powershell-Module/ARM64-Dev/LabTech.psm1'); Get-LTServiceInfo"

$DebugPreference='Continue'; (new-object Net.WebClient).DownloadString('https://raw.githubusercontent.com/DarrenWhite99/LabTech-Powershell-Module/ARM64-Dev/LabTech.psm1') | iex; Get-LTServiceInfo
Results in an endless loop and a bunch of stuck processes:

WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit access.
DEBUG: Loading WOW64Redirection functions

I was able to run Install-LTService and Get-LTServiceInfo without any issues. Reinstall-LTservice seems to have an issue, but I tested and it is not specific to ARM.

PS C:\windows\system32> $DebugPreference='Continue'; (new-object Net.WebClient).DownloadString('https://raw.githubusercontent.com/DarrenWhite99/LabTech-Powershell-Module/ARM64-Dev/LabTech.psm1') | iex; Get-LTServiceInfo
WARNING: 32-bit PowerShell session detected on 64-bit OS. Attempting to launch 64-Bit session to process commands.
DEBUG: SYSNATIVE PATH REDIRECTION IS NOT AVAILABLE. Attempting to disable System32 path redirection for native 64bit
access.
DEBUG: Loading WOW64Redirection functions
DEBUG: Native 64-bit PowerShell environment unavailable in ARM64. Execution will continue.
DEBUG: Starting Get-LTProxy at line 3652
DEBUG: Line 3143: Decoding Server Password.
DEBUG: Line 3226: Attempting Decode for '{sanitized}' with Key 'Thank you for using LabTech.'
DEBUG: Line 3146: Decoding Agent Password.
DEBUG: Line 3226: Attempting Decode for
'{sanitized}' with Key '{sanitized}'
DEBUG: Line 3226: Attempting Decode for
'{sanitized}' with Key 'Thank you for
using LabTech.'
DEBUG: Line 3226: Attempting Decode for
'{sanitized}' with Key ''
DEBUG: Line 3266: Failed to Decode string:
'{sanitized}'
DEBUG: Line 3266: Failed to Decode string:
'{sanitized}'
DEBUG: Line 3266: Failed to Decode string:
'{sanitized}'
DEBUG: Line 3668: Setting ProxyServerURL to
DEBUG: Line 3676: Setting ProxyUsername to
DEBUG: Line 3683: Setting ProxyPassword to
DEBUG: Exiting Get-LTProxy at line 3697
DEBUG: Starting Get-LTServiceInfo at line 212
DEBUG: Exiting Get-LTServiceInfo at line 255
ProductCode            : {sanitized}
LocationID             : 1
Server Address         : https://{sanitized}
TrayPort               : 42000
VNCServiceName         : LabVNC
DefaultTempPath        : C:\windows\Temp
BasePath               : C:\windows\LTsvc
FullSystemFunction     : False


fix: do not override web3 in embark

At present, are we in a position to eliminate global web3 or does some provider bootstrapping still rely on it?

I understand.
On master branch and the beta.37 branch, for me in embark_demo the ENS part is broken. I'm not sure if one of our unmerged PRs fixes that problem, maybe this one does, haven't tried it yet; also not sure if you're aware.

@michaelsbradleyjr yes for me too, I am going to try to fix it now


Have 235 pages x 10 windows saved like 23500 websites, and its not saving new windows anymore

Have 235 pages x 10 windows saved like 23500 websites, and its not saving new windows anymore

I have the same issue


Styled radio buttons

Fixes #21
Before:

After:

You can also add the package to the list in readme.md

@ttomasz I added the package to list in README. Should I do something else :)?


fix: refactor txn types

Description
Depends on https://github.com/reown-com/yttrium/pull/98
How Has This Been Tested?
Not tested
Due Diligence

[ ] Breaking change
[ ] Requires a documentation update
[ ] Requires a e2e/integration test update

Fixing to fit the latest schema changes and initial tx nonce calculation in 112f5b91b3c8d8e539ab02bb7095cbbd90262dc4


Publish to crates.io on tags

Fix https://github.com/o1-labs/ocaml-gen/issues/16
Built on top of https://github.com/o1-labs/ocaml-gen/pull/17

Token has been added by @joseandro in CRATES_IO_TOKEN.


CZWeatherKit does not support historical data

Would love to see CZWeatherKit support querying historical weather data (cf http://openweathermap.org/history). I may add this for my own purposes if it is not already planned…

I wasn't planning on adding it. If you do pursue it on your own, I'd appreciate the pull request!

@quicklywilliam Just wanted to let you know that I'm currently working on adding this.

I've added support for this in the new release, any feedback is appreciated. Thanks.

Nice! Will check it out.


JSX Runtime problem

Hi guys,
It seems that there is a compatibility error with older react version (^16.14.0).
ERROR in ./node_modules/react-telephone/dist/react-telephone.modern.js 1:0-56
Module not found: Error: Can't resolve 'react/jsx-runtime' in 'C:\XXXXXX\node_modules\react-telephone\dist'
Did you mean 'jsx-runtime.js'?
BREAKING CHANGE: The request 'react/jsx-runtime' failed to resolve only because it was resolved as fully specified
(probably because the origin is strict EcmaScript Module, e. g. a module with javascript mimetype, a '*.mjs' file, or a '*.js' file where the ppackage.json contains '"type": "module"').
The extension in the request is mandatory for it to be fully specified.
Add the extension to the request.

Bonne journée à vous!

I will look at it this weekend, thank for reporting :) PR welcome

@romane-echino Have you tried a solution like that: https://github.com/react-dnd/react-dnd/issues/3423#issuecomment-1092621793

Closing due to inactivity


Add CI server

So we can automatically run tests when using pull requests.
Show on the README when the build is passing/failing by using a badge.
Consider using Travis or CircleCI.

Resolved in https://github.com/findmypast-oss/git-mob/compare/a167...e5846f93b0a4db145c07ee00161cc3e0fff966a4 + 1ca1eae816e365a3ec2e7ba30ea140619b242192.


Stats for on the coin/play?

New user to track-o-bot, and can't figure out how to determine my winrate with a deck or over a time period with the coin vs without the coin.  This is a stat I like to know, and wondering if it's been considered in the past.
I have little Ruby experience (mostly Python for web development work) but wouldn't mind taking an attempt at the feature if there's interest.

@jdharms if you query for the cards played in each match, it should be trivial to simply count the number of games where you play the coin or your opponent plays the coin (probably throw out any matches where no coin was played).  AFAIK, there is no way of doing this from the UI and I doubt @stevschmid would be interested in adding that feature.

@theanine our definitions of trivial look like they're very different.  That sounds like quite a bit of work, haha.  It sounds like you're suggesting downloading the json data for a month or whatever and writing a script to tally the games, or manually going through the games in the ui (eeeeeek!).
It's fine if there's not interest in adding the feature, after all, it is something I can do myself with the API that's available.  I just noticed that the API to post games to trackobot and the history API both have a field for "coin" for each game, but it doesn't appear to be displayed/filterable at all in the UI.  This made me think there might be interest in displaying it somewhere.

34 lines of code from cloc's count.  See here
Enjoy.

So yes, different definitions of trivial, I suppose.
For what it's worth, other trackers seem to natively track this stat. So, it seems like something people want, or at least something a few developers thought people might want, and I thought I'd bring it up.
I know I'd prefer having access to the information even when I can't run Python scripts or when I'm on mobile.
Anyway, thanks for your help.

I'm curious what you consider to be trivial?  I would think 34 LoC / 5-10min is certainly within trivial.
Also, just because other trackers have it doesn't mean it's a good feature to have.  @stevschmid's philosophy is to keep ToB very minimal.  And I think there are many more statistically-related features that would be much more useful than this measure that are deliberately not included in ToB (e.g. average time to play each deck type).
The only potentially useful thing about the feature you'd like would be to see which archetypes are more or less reliant on getting coin / going first.  VS has an article on this (from their supply of many many more games than a single player has in their individual history) and that article isn't even very useful, because it basically shows exactly what you'd expect -- Miracle likes coin, aggro/zoo likes going first.

First, please don't assume everyone writes code as fast as you. Second,
your solution doesn't provide the stats for each deck with the coin, or
against each class. Maybe it's my math background poking through, but
trivial means basically no work, not "anyone comfortable with Python can do
it within an hour".
Tristan, are you officially connected to this project in any way? I
understand that Steve's philosophy here is to keep things minimal, I just
don't think that adding one filter option necessarily is in conflict with
that.
I don't know if it's what you're trying to do, but you are thoroughly
discouraging me from working on this feature, if it's even wanted, but also
anything else for this project. And if this is the treatment I'll get when
I open a ticket I will probably not be doing that again either.
Frankly, why are you being so antagonistic here? It doesn't look like you
do this on every issue opened, so I'm wondering what it was about mine that
caught your eye. Why do you presume to know "the only potentially useful
thing" about a stat? Do you really not believe it's possible to improve
your play with this information? What if someone found they had a 20%
winrate difference without the coin? This almost certainly indicates poor
usage of the coin, not just inherent weaknesses of the deck. This
information is useful!
If you are someone who makes decisions about this projects development,
please let me know so I can stop wasting your time. If you are not, please
stop trying to convince me I don't need the feature I want. I'll gladly
respect whatever direction Steve wants to go in with his project, but I
would very much like him to consider it.
On Mon, Feb 13, 2017 at 3:28 AM Tristan Muntsinger<EMAIL_ADDRESS>wrote:
I'm curious what you consider to be trivial? I would think 34 LoC / 5-10min
is certainly within trivial.
Also, just because other trackers have it doesn't mean it's a good feature
to have. @stevschmid https://github.com/stevschmid's philosophy is to
keep ToB very minimal. And I think there are many more
statistically-related features that would be much more useful than this
measure that are deliberately not included in ToB (e.g. average time to
play each deck type).
The only potentially useful thing about the feature you'd like would be to
see which archetypes are more or less reliant on getting coin / going
first. VS has an article on this (from their supply of many many more games
than a single player has in their individual history) and that article
isn't even very useful, because it basically shows exactly what you'd
expect -- Miracle likes coin, aggro/zoo likes going first.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stevschmid/trackobot.com/issues/79#issuecomment-279322383,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA_mJvsv0kbc5agBXd-khEgETJ0fZn7Eks5rcBQegaJpZM4L-ZCM
.
--
--Daniel

I'm a minor contributor to the project and @stevschmid doesn't get a lot of time to work on it, so I try to help where I can (on Reddit and here on GitHub) based on what I know.  I also tried to help you by writing the code I thought trivial (and yes, I do consider "trivial" to be ~5 minutes of work; 0 minutes of work is a completed feature).  I even wrote it in your preferred language of Python.  I'll leave it to @stevschmid to respond to this issue, but as I said the interface is minimal by design as @stevschmid has stated quite a few times on here and on Reddit.

Perfect!  Thanks for your help again.  Appreciate the effort to package
your sample code in a way readily accessible to me.  I'd still much rather
not have to run a script every time I want this information, and I think it
could be useful to others as well.
I'm sorry for snapping at you--no one's getting paid here or owes me
anything.  I just felt like my feedback/suggestion was being discarded
before it could even be considered, and felt frustrated.
On Mon, Feb 13, 2017 at 4:45 PM, Tristan Muntsinger <
<EMAIL_ADDRESS>wrote:

I'm a minor contributor to the project and @stevschmid
https://github.com/stevschmid doesn't get a lot of time to work on it,
so I try to help where I can (on Reddit and here on GitHub) based on what I
know. I also tried to help you by writing the code I thought trivial (and
yes, I do consider "trivial" to be ~5 minutes of work; 0 minutes of work is
a completed feature). I even wrote it in your preferred language of Python.
I'll leave it to @stevschmid https://github.com/stevschmid to respond
to this issue, but as I said the interface is minimal by design as
@stevschmid https://github.com/stevschmid has stated quite a few times
on here and on Reddit.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stevschmid/trackobot.com/issues/79#issuecomment-279532536,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA_mJkF_eExLaSXIHCPdNuVB1WU0R6-tks5rcM7kgaJpZM4L-ZCM
.

  This should have everything you need.  Also, this is another reason why ToB need not have the features baked in: with API access you can just play with the data as you see fit.
Also I'm sure there's a more pythonic way to write the code linked above, but I'm no Python expert.  I'll accept any refactor PR.


[Feature]:  changing database ports

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
make a possible change external port in database image's. to make it the same with Internal port. 3306 for local docker and 3306 for exaternal connection too for example.
Why should this be worked on?
.

I'm closing this issue, because tracking features will be moved to GitHub discussions. Github issues will be only for bugs.
Version 4 is in full focus. v3 related issues will be closed, if it is not a critical bug.
Thanks for your understanding.


No escaping or percentage sign for jsonencode of log processing -> sample log

Describe the bug
When running -export for log processing, if the sample log contains a %{text} for example, terraform expects a %{if or %{else
An error comes out:
Error: Invalid template control keyword
To Reproduce
add a %{variable} to a log sample
use -export for dynatrace_log_processing
Expected behavior
Escaping of the percentage sign by doubling it: %%{variable}

Thanks, David. Will be fixed with the upcoming release.


Scroll in modal

I using modal, but when insert content inside in body, display scroll...
Why? How correct that?

Have you tried Modals with Fixed Footer?
http://materializecss.com/modals.html
And can you give an online example on codepen.io ?

Yes, with fixed footer. I'll show you the code later.


Server, ruby error

Following your installation manual, with a Droplet from https://cloud.digitalocean.com;
I end up with the following error :
ruby ./dnscat2.rb

/usr/lib/ruby/1.9.1/rubygems/dependency.rb:247:in `to_specs': Could not find rexec (~> 1.6.2) amongst [bundler-1.10.6, eventmachine-1.0.3, rubydns-0.7.3, trollop-2.0] (Gem::LoadError)
	from /usr/lib/ruby/1.9.1/rubygems/specification.rb:777:in `block in activate_dependencies'
	from /usr/lib/ruby/1.9.1/rubygems/specification.rb:766:in `each'
	from /usr/lib/ruby/1.9.1/rubygems/specification.rb:766:in `activate_dependencies'
	from /usr/lib/ruby/1.9.1/rubygems/specification.rb:750:in `activate'
	from /usr/lib/ruby/1.9.1/rubygems.rb:212:in `rescue in try_activate'
	from /usr/lib/ruby/1.9.1/rubygems.rb:209:in `try_activate'
	from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:59:in `rescue in require'
	from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:35:in `require'
	from /root/dnscat2/server/driver_dns.rb:11:in `<top (required)>'
	from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:60:in `require'
	from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:60:in `rescue in require'
	from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:35:in `require'
	from ./dnscat2.rb:14:in `<main>'

I think that's somewhere in the install guide.. you have to install
"ruby-dev" or something. I can look it up later. :)
On Wed, Sep 16, 2015 at 3:10 PM, Swiffers<EMAIL_ADDRESS>wrote:

Following your installation manual, with a Droplet from
https://cloud.digitalocean.com;
I end up with the following error :
ruby ./dnscat2.rb
/usr/lib/ruby/1.9.1/rubygems/dependency.rb:247:in to_specs': Could not find rexec (~> 1.6.2) amongst [bundler-1.10.6, eventmachine-1.0.3, rubydns-0.7.3, trollop-2.0] (Gem::LoadError)     from /usr/lib/ruby/1.9.1/rubygems/specification.rb:777:in block in activate_dependencies'
from /usr/lib/ruby/1.9.1/rubygems/specification.rb:766:in each'     from /usr/lib/ruby/1.9.1/rubygems/specification.rb:766:in activate_dependencies'
from /usr/lib/ruby/1.9.1/rubygems/specification.rb:750:in activate'     from /usr/lib/ruby/1.9.1/rubygems.rb:212:in rescue in try_activate'
from /usr/lib/ruby/1.9.1/rubygems.rb:209:in try_activate'     from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:59:in rescue in require'
from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:35:in require'     from /root/dnscat2/server/driver_dns.rb:11:in <top (required)>'
from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:60:in require'     from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:60:in rescue in require'
from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:35:in require'     from ./dnscat2.rb:14:in '
—
Reply to this email directly or view it on GitHub
https://github.com/iagox86/dnscat2/issues/65.

So, that's not the error I was thinking of, the one I noticed looked like:
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- mkmf (LoadError)
And installing the "ruby-dev" package worked. It's possible I just have to add "rexec" to the gemfile. If installing "ruby-dev" doesn't help, can you run "gem install rexec" and see if that helps?

Issue solved !
Installing the rexec gem fixed the problem.
gem install rexec

Great! I added it to the Gemfile explicitly.


fix: Support grouping by resource in multi resource mode

What this PR does / why we need it?
This PR adds support to use --group-by=resource flag in multi resource mode.
In addition, it improves the output by adding newlines in similar to --group-by=namespace output.

PR Checklist

[ ] This PR adds K8s exceptions (false positives)
[x] This PR adds new code
[ ] This PR includes tests for new/existing code
[ ] This PR adds docs

GitHub Issue
Closes #289

Codecov Report
Attention: Patch coverage is 0% with 33 lines in your changes missing coverage. Please review.

Project coverage is 41.30%. Comparing base (ced2ef2) to head (f1984d9).
Report is 2 commits behind head on main.

Files
Patch %
Lines

pkg/kor/multi.go
0.00%
32 Missing :warning:

pkg/kor/formatter.go
0.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #320      +/-   ##
==========================================
- Coverage   41.34%   41.30%   -0.04%
==========================================
  Files          61       61
  Lines        3212     3215       +3
==========================================
  Hits         1328     1328
- Misses       1671     1674       +3
  Partials      213      213

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


not necessarily an issue with this (I think)

basically, let's say that someone is seeding from another torrent app, and i try to download that file with picotorrent, will it work?
if anyone needs a better explanation i could try, i'm not the best at explaining things

yes, all torrent clients will seed and download from each other because they use the same protocol.


Batch mode for addBlock to speed up syncing (avoid repeated chain selection)

cc @dcoutts

The block fetch clients could use the batch API when they are in the bulk sync mode.

I have a plan for batching addBlock that would not require any changes to the BlockFetchClient.
Current situation: the BlockFetchClient calls addBlockWaitWrittenToDisk for each block. This function will add the block to a queue and wait for the block to have been written to disk (VolatileDB). A background thread consumes this queue and calls addBlockSync for them in the order they were added to the queue. I have traced the length of this queue during a bulk chain sync from a local server and it was mostly non-empty.
Plan:

The queue used by addBlock stores ChainFragment blks (or AnchoredFragment if that's useful?) instead of BlockToAdd blk.

When addBlock tries to add a new block to the end of the queue:

If the queue is empty, it should create a new ChainFragment (BlockToAdd blk) for the block.
If the queue is not empty and contains a fragment, we check whether the new block fits onto the fragment. If so, we append it to the fragment. If not, we create a new fragment and append that to the queue.

In the background thread processing the queue, for each fragment, we add all blocks to the VolatileDB and then perform chain selection for the first block in the fragment, not the last!
When performing chain selection for a block, we will compute all candidate chains starting from the block, so all other blocks in the fragment would be found. If we were to perform chain selection only for the last block, we would not find any candidates forking off at a block before the last one. Take the following example:
A--B--C
   \--C'

where the fragment in the queue is A :> B :> C, C' is already in the ChainDB, and A fits onto the current chain. Chain selection for A would find two candidate chains: A :> B :> C, A :> B :> C', but chain selection for C would only find candidate A :> B :> C, but not the other, missing out a chain selection candidate that might have been preferred over the other.
Moreover, the way things are currently set up, chain selection for A would result in the optimal "add to current chain" case and chain selection for C would result in the more expensive "try switch to fork" case.


Close modal on network change

What does this PR do and why?
Subscribe to network change events. Means that the modal closes when user changes network.
Closes https://github.com/jbx-protocol/juice-interface/issues/407
This doesn't add an explicit "Switch Network" button to the modal, but this can be handled in https://github.com/jbx-protocol/juice-interface/issues/290
Screenshots or screen recordings
https://user-images.githubusercontent.com/94939382/167236990-8cf38d1f-9d4a-4a6c-9b00-c01c9a511883.mp4
Acceptance checklist

[x] I have evaluated the Approval Guidelines for this PR.
[x] I have tested this PR in all supported browsers.

@jmilldotdev could you please review this when you get a chance? 🙏

lgtm


How to delete Page Icon ????

Hi gays:
How to delete Page Icon because I have one Page,Thank you very much........

in custom.css you can add:
.pages-menu {
  display: none;
}

Thank You


how to get data puted by clientv3 with curl?

I'm testing clientv3 Go API now. I can put data into key, and get data from that key. But hot to retreive that key and value with curl?

https://github.com/coreos/etcd/blob/master/Documentation/dev-guide/api_grpc_gateway.md


Split checkout, artifact, and plugin phases out of executor.go

The file is too big for me to navigate easily. Splitting it into multiple files will make it easier to work with.
We should split it into multiple packages, but that will require more work.

having not read it yet: shit yes

@benno I was reading the diff and it seems a bit strange. I'll try to make it easier to review.