Adapt Evaluation Plots from lifetimes

lifetimes contains a variety of plotting functions for model evaluation:

[ ] plot_period_transactions
[ ] plot_calibration_purchases_vs_holdout_purchases
[x] plot_frequency_recency_matrix
[x] plot_probability_alive_matrix
[ ] plot_expected_repeat_purchases
[ ] plot_history_alive
[ ] plot_cumulative_transactions
[ ] plot_incremental_transactions
[ ] plot_transaction_rate_heterogeneity
[ ] plot_dropout_rate_heterogeneity

This notebook provides some examples of their use.
@larryshamalama already added the matrix plots, but the others would need to be modified for posterior confidence intervals. Some of them also require utility functions that I'll create PRs for soon. I don't consider plot_history_alive to be all that useful, but if there's interest it's something to consider.
It's also worth reviewing the research papers for additional ideas of plots to add.

I have also found tracking plots incredibly useful for explaining to
stakeholders. CLVTools has some very nice implementations:
https://www.clvtools.com/articles/CLVTools.html (scroll down)
On Thu, Jul 13, 2023 at 3:24 PM Colt Allen @.***> wrote:

lifetimes contains a variety of plotting functions for model evaluation:

plot_period_transactions
plot_calibration_purchases_vs_holdout_purchases
plot_frequency_recency_matrix
plot_probability_alive_matrix
plot_expected_repeat_purchases
plot_history_alive
plot_cumulative_transactions
plot_incremental_transactions
plot_transaction_rate_heterogeneity
plot_dropout_rate_heterogeneity

This notebook
https://github.com/ColtAllen/marketing-case-study/blob/main/case-study.ipynb
provides some examples of their use.
@larryshamalama https://github.com/larryshamalama already added the
matrix plots, but the others would need to be modified for posterior
confidence intervals. Some of them also require utility functions that I'll
create PRs for soon. I don't consider plot_history_alive to be all that
useful, but if there's interest it's something to consider.
It's also worth reviewing the research papers
https://github.com/pymc-labs/pymc-marketing/issues/91 for additional
ideas of plots to add.
—
Reply to this email directly, view it on GitHub
https://github.com/pymc-labs/pymc-marketing/issues/326, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AH3QQV7R2Z43XJLC3ZMCPZDXQBDQVANCNFSM6AAAAAA2JMSLJA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

I have also found tracking plots incredibly useful for explaining to
stakeholders. CLVTools has some very nice implementations:

I think plot_incremental_transactions is the equivalent in lifetimes. Here's an example:

plot_period_transactions is essentially plotting a posterior predictive check for customer purchase frequency:

Unless I'm mistaken, this means we can't use a Potential to define the likelihood. GammaGammaModel is fine because this plotting method doesn't apply, but BetaGeoModel would need its own distribution block per https://github.com/pymc-labs/pymc-marketing/issues/128.

I have also found tracking plots incredibly useful for explaining to
stakeholders. CLVTools has some very nice implementations:

I think plot_incremental_transactions is the equivalent in lifetimes. Here's an example:

I did not realize that these were added/in there. That is very cool. Thanks for pointing them out.

Some WIP ideas for adapting plot_period_transactions in this issue: https://github.com/pymc-labs/pymc-marketing/issues/278

I have a PR to add ax argument to the current CLV plotting functions. Might be a good standard here in order to give additional flexibility to the user

I also want to add a parameter to return the formatted data instead of the plot, in case anyone wants to build the plots in something other than matplotlib (this has come up in my day job).
In the case of the plot_cumulative_transactions and plot_period_transactions functions, returning the data also enables use of the Wasserstein Distance function from scipy for concept drift monitoring, which will be helpful if the model is registered to mlflow.

Love that idea. Separate the data transformation from plotting. Total sense to me

The ArviZ library also has a lot of useful plotting options to compliment whatever we add in pymc-marketing, and examples would be a great addition to the tutorial notebooks. I've played around with a few of them in the dev notebook for ParetoNBDModel:
https://github.com/pymc-labs/pymc-marketing/blob/main/docs/source/notebooks/clv/dev/pareto_nbd.ipynb

Hi all, I was trying to create the histogram comparing observed and expected purchase counts for the BG/NBD model for myself when I came across this issue.  This would be the histogram that a plot_period_transactions function would create for BetaGeoModel.  I put together a function that computes the values based on the Fader, Hardie, and Lee (2007) note, and thought I would share it here in case it's useful.
Based on that note, they calculated that histogram by estimating $Pr(X(t) = x | r, \alpha, a, b)$ for $x = 1, 2, \ldots, 7+$ using each value of $t$ in the original data.  Then they summed over the values of $t$ to get the expected number of repeat purchases (i.e., the expected number of people with $x = 1, 2, \ldots, 7+$) for the original cohort ($E(f_x)$ in their notation).
They calculated this in Excel.  I essentially transcribed their Excel functions into Python, making the following function:
def calculate_pmf(
    t_val: np.array,  # TODO: allow this to be a vector
    max_x: int,
    # defaults are values from lifetimes file
    r: float = 0.242594150135163,
    alpha: float = 4.41359212062015,
    a: float = 0.792919156875463,
    b: float = 2.4258950494563
) -> np.array:

    # First calculate values that don't vary with x or t
    log_beta_ab = betaln(a, b)
    log_gamma_r = gammaln(r)

    # Create an output matrix
    out = np.empty((len(t_val), max_x + 1))

    # Next loop through all the values of t and calculate the pmf for each one
    for idx, t in enumerate(t_val):
        log_alpha_r = r * (np.log(alpha) - np.log(alpha + t))

        # TODO:
        # 1. Figure out how to make this work with a ufunc
        # 2. Calculate the values of x that don't depend on t outside the loop

        # Next create a range of values for x.  This will allow us to use
        # vectorized functions to loop
        x = np.arange(max_x)

        # For each value of the output range, calculate the part that depends on x
        log_term1 = betaln(a, b + x) - log_beta_ab \
            + gammaln(r + x) - log_gamma_r - gammaln(x + 1) \
            + log_alpha_r \
            + (x * (np.log(t) - np.log(alpha + t)))

        log_term2 = np.where(
            x > 0,
            betaln(a + 1, b + x - 1) - log_beta_ab,
            0
        )

        # To calculate the sum, we're going to calculate each term and the apply cumsum
        # Two notes:
        # 1. You need to exponentiate before summing
        # 2. For a given value of x, we need the (x - 1)th term of the sum
        #    We'll take care of that on the fly when creating the output

        individual_sum_terms = np.cumsum(np.exp(
                gammaln(r + x) - gammaln(x + 1) \
                + (x * (np.log(t) - np.log(alpha + t)))
                ))

        # Log and add the terms that you factored out back in
        log_term3 = log_alpha_r - log_gamma_r + np.log(individual_sum_terms)

        # Get the rest of term 3
        term3 = 1 - np.exp(log_term3)

        # Now create the output
        out1 = np.exp(log_term1) + np.where(
            x > 0,
            np.exp(log_term2) * term3[x - 1],
            0
            )

        # Append the remainder as the final term and add it to the output matrix
        out[idx, :] = np.append(out1, 1 - np.sum(out1))

    return out

This version takes just one value for each of the estimated parameters.  So, either you could take the MAP values and plug them in, or calculate it once for each draw from the posterior, which would give you a posterior distribution.  To do the latter, this function would need to be modified to allow the r, alpha, a, and b, arguments to be vectors.
Hopefully this is helpful.  If you want me to create pull request to add this in somewhere, let me know, and I'll try to figure out how to do that.

This version takes just one value for each of the estimated parameters. So, either you could take the MAP values and plug them in, or calculate it once for each draw from the posterior, which would give you a posterior distribution. To do the latter, this function would need to be modified to allow the r, alpha, a, and b, arguments to be vectors.

If you use xarray you shouldn't need much looping or worry about chains/draws. It will take of vectorizing all operations automatically. Something like that is done here: https://github.com/pymc-labs/pymc-marketing/blob/4c9f4ca163f6e34c4b555b51f14f89e8851a0a9b/pymc_marketing/clv/models/beta_geo.py#L223
By the way how does your function differ from that expected_num_purchases or expected_num_purchases_new_customer?

Thank you for taking a look!  I'll see if I can rewrite this to use xarray.  (I am new to pymc and xarray, so that is a helpful pointer.)
Here is my understanding of how this function differs from expected_num_purchases and expected_num_purchases_new_customer.  In short, those functions return the expected number of purchases for a customer, and this function returns the probability that a given customer has 1, 2, 3, ... purchases.

For a given value of $t$, this function returns $Pr(X(t) = x | r, \alpha, a, b)$ for $x = 1, 2, \ldots,$ max_x.  That is, it returns a vector of probabilities for each value of $t$ and $x$.  This is equation (8) from Fader et al. (2005).
expected_num_purchases_new_customer returns the expected number of purchases, which is $E(X(t) | r, \alpha, a, b) = \sum_{x} x * Pr(X(t) = x | r, \alpha, a, b)$.  That is, this function averages over the values of $x$ for each value of $t$ to get the expected purchase count for each time interval $t$.  This is equation (9) from Fader el. al (2005).
expected num_purchases returns the expected number of additional purchases, conditional on the data (a given number of previous purchases) and parameters.   So expected_num_purchases_new_customer covers the interval $(0, T]$, and expected_num_purchases covers the interval $(T, T + t]$.  This is equation (10) from Fader et al. (2005)

The histogram from the paper averages equation (8) over all values of $t$ observed in the original data, to get expected counts of customers who have made 1, 2, 3, ... purchases.  Letting $i$ index customers and $t_i$ represent a customer's age, the histogram calculates $E(f_x) = \sum_i Pr(X(t_i) = x | r, \alpha, a, b)$, which is a function of $x$.  (In the note, they calculate $Pr((X(t) = x)$ for the unique values of $t$ and then multiply by the number of customers who have that value of $t$.)
Regarding using logp, good question.  I think logp is equation (6) in the Fader et al. (2005) paper.  I think the difference between equation (6) and equation (8) is that equation (6) is the joint probability of purchase count $x$, recency $t_x$, and customer age $T$ given the parameters, while equation (8) is just the probability of a purchase count $x$ given the parameters as a function of the length of the time interval.  (That is, conditional on the parameters, equation 8 doesn't use recency or frequency.)  I could definitely be wrong about that though.

Hey @jcfisher,
There is already a PR open for the method you are proposing:
https://github.com/pymc-labs/pymc-marketing/pull/216
It was abandoned by its original author quite some time ago though, so it may be prudent to open another one.
As to using this method to generate plot_period_transactions, I am curious how it would compare to a backend posterior predictive check, but the latter approach is my preference as it can be used across all transaction models.

@ColtAllen Thanks!  Happy to modify that one or create a new one, whichever is easier.  I revised the function above to use xarray, so now it should work with a vector of draws for each parameter.
I tried to find a sample_posterior_predictive function for the BetaGeoModel for comparison, but I'm having trouble getting the one from pymc to work.  Is there are standard way of getting draws from the posterior predictive distribution?  (Apologies if the answer to this question is obvious.)  Without seeing it, I'd assume they would be similar, and agree that if they are, just using a posterior predictive check would be preferable.

I tried to find a sample_posterior_predictive function for the BetaGeoModel for comparison, but I'm having trouble getting the one from pymc to work. Is there are standard way of getting draws from the posterior predictive distribution? (Apologies if the answer to this question is obvious.) Without seeing it, I'd assume they would be similar, and agree that if they are, just using a posterior predictive check would be preferable.

for the BetaGeoModel, the parameters can be sampled like from within this method. You should just be able to pass self.fit_results or model.fit_results into pm.sample_posterior_predictive. If you need access to the pm.Model instance, it'd be in the model attribute of the BetaGeoModel instance

Got it, thanks.  I was having trouble because I wasn't using a with model: block (same as this issue).
Just to double check that I'm understanding this correctly, to generate draws from the posterior predictive distribution, we'd need a random number generator function for the custom likelihood, right?  I'm thinking something like this:
# Random number generator function for BG-NBD model
# Returns the original data sampled from the data generating process, given the parameters
# Original data are:
# 1. Frequency (number of repeat purchases, i.e., number of purchases - 1)
# 2. Recency (time of last purchase relative to now)
# 3. Age (time of first purchase relative to now)
# As inputs, we use:
# * T: "now" time
# * r, alpha, a, b: draw from the posterior parameter values

def beta_geo_rng(T, r, alpha, a, b):

    # Draw from the gamma and beta distributions to get a value of lambda and p
    base_rng = np.random.default_rng()
    lam_draw = base_rng.gamma(shape = r, scale = 1 / alpha)
    p_draw = base_rng.beta(a, b)

    # Time of first purchase
    t = first_purchase_time = np.random.exponential(scale = lam_draw)
    x = 0  # purchases are 0 indexed

    while t < T:
        # Drop out with probability p
        if base_rng.binomial(n = 1, p = p_draw) == 1:
            return (x, T - t, T - first_purchase_time)
        else:
            # If not dropping out, add a new purchase after exponential waiting time
            t += np.random.exponential(scale = lam_draw)
            x += 1

    # Return tuple of frequency, recency, age
    return (x, T - t, T - first_purchase_time)

If this doesn't look right, please let me know.  I'll work on putting together a comparison of this with the $Pr(X(t) = x | \ldots)$ function above.
Also, hope I'm not hijacking the conversation on this issue.  If I should take this discussion somewhere else (or just post a gist with these functions and let you all take it from there), let me know.

Have you encountered memory issue when running plot_frequency_recency_matrix? I am following the https://www.pymc-marketing.io/en/stable/notebooks/clv/clv_quickstart.html and then ran it on my own dataset of 10000 examples and noticed peak memory usage going up to 50-70Gb which then the jupyter kernel eventually gets killed. Haven't investigated in depth but perhaps I can try recreating it with a simple dataset.

Hey @billlyzhaoyh,
When this happens, use model.thin_fit_result. We've been meaning to add an example for this to the Quickstart per https://github.com/pymc-labs/pymc-marketing/issues/448.
Also, this shouldn't ever happen with a MAP fit model unless your data has tens of millions of customers or more.

@ColtAllen Thanks for getting back to me about this and I will certainly try out the thin_fit_result method for all plotting purposes.
The original memory issue was encountered with tens of millions of customers I put together but then I realised the sub-sample of 10000 users I didn't reset their customer_id to index+1 which then solved the problem. Probably worth calling out from the documentation on resetting customer id to index+1 and keep the mapping from customer_id in your custom database to the ones used in modeling purposes.


can't connect to aws cluster

Windows Version
Windows 11 Pro
WSL Version
10.0.22621.1 (WinBuild.160101.0800)
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
5.10.16
Distro Version
20.04.4 LTS (GNU/Linux <IP_ADDRESS>-microsoft-standard-WSL2 x86_64)
Other Software
Docker version 20.10.17, build 100c701
Docker Desktop
Client Version: version.Info{Major:"1", Minor:"23+", GitVersion:"v1.23.15-eks-49d8fe8", GitCommit:"2753aa076f2e0f93ce97ab9c3edc8c244bc1f130", GitTreeState:"clean", BuildDate:"2023-01-09T23:45:22Z", GoVersion:"go1.17.13", Compiler:"gc", Platform:"linux/amd64"}
Repro Steps
I am trying to connect to the cluster while AWS VPN Client is turned on using kubectl and tried many different ways to get to it but I am unable to do so.
Made the dns entries under hostfile. Updated the /etc/resolv.conf with google dns <IP_ADDRESS>. But no luck. On the other hand, the mac just connect to it right off the bat.
kubectl --context arn:xxxx:cluster/xxxxx-eks -n public get pods --v 8
Expected Behavior
connect to the cluster and display list of pods
Actual Behavior
I get this error instead
skipped caching discovery info due to Get "https://D62EBEFA7FCBD91643C7E199343BA7DE.gr7.us-gov-west-1.eks.amazonaws.com/api?timeout=32s": dial tcp: lookup D62EBEFA7FCBD91643C7E199343BA7DE.gr7.us-gov-west-1.eks.amazonaws.com on dns-D62EBEFA7FCBD91643C7E199343BA7DE:53: read udp dns1--D62EBEFA7FCBD91643C7E199343BA7DE:38894->wsl-ip.1:53: i/o timeout
Diagnostic Logs
No response

Hi. Can you please collect networking logs by following the instructions below?
https://github.com/microsoft/WSL/blob/master/CONTRIBUTING.md#collect-wsl-logs-for-networking-issues

thanks for reporting the issue and sorry for the delay following up on this
the description from the "Actual Behavior" section suggests getting DNS errors
I recommend enabling DNS tunneling
If you still hit the issue after enabling DNS tunneling, please collect a new set of networking logs


Error installing: module "toolcli" has no attribute "print"

❯ pip3 --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

❯ python3 --version
Python 3.8.10

❯ pip3 --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

❯ python3 -m ctc setup
ctc initializing...

This process will make sure each of the following is completed:
- setup config path
- setup data directory
- setup networks and providers

Each step can be skipped depending on what you need
- this wizard can be rerun multiple times idempotently
- by default, wizard will leave current settings unchanged

module 'toolcli' has no attribute 'print'

❯ pip install toolcli
Requirement already satisfied: toolcli in ./.local/lib/python3.8/site-packages (0.6.6)
Requirement already satisfied: rich>=12.1.0 in ./.local/lib/python3.8/site-packages (from toolcli) (12.5.1)
Requirement already satisfied: typing_extensions>=0.4.0 in ./.local/lib/python3.8/site-packages (from toolcli) (4.3.0)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in ./.local/lib/python3.8/site-packages (from rich>=12.1.0->toolcli) (2.13.0)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in ./.local/lib/python3.8/site-packages (from rich>=12.1.0->toolcli) (0.9.1)

this was fixed a while ago in the main branch but it hasnt made it's way into a pypi release yet
installing latest commit from main branch should fix it


updated copyright in LICENSE file

@ubukawa, @hfu, @miya0001 , I have updated copyright of LICENSE file now.

Thank you so much! I also think the idea from @hfu  is also great and I support it.

@hfu I think it is good idea to have contributors list in the repository. I will create an issue for this.

@hfu @ubukawa I created this issue https://github.com/unvt/charites/issues/67


Feautre request: batch resize media

I really like this addon, for making new cards, nonetheless I think it would be really useful that the addon could resize previous images from my library.
It would be great if for example I could select some cards, tell the addon which filed to look and resize all the images contained on that field.
Thanks for the hard work.

Hi,
Thanks for this suggestion. But I don't have enough time to develop this feature for now. This addon is only in maintenance currently. Besides images are stored in a folder, you can use other tools to batch resize those images.

The reason I was suggesting this feature was to only resize certain photos contained in a deck but not my entire library. Nonetheless, I will think a workaround.
Ty for the response.


HBASE-25062 The link of "Re:(HBASE-451) Remove HTableDescriptor from …

…HRegionInfo" invalid

Just remove this link, since i think we cannot easily correct it.
 U: .

 U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2455/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
7m 44s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+1 :green_heart:
mvninstall
4m  3s
master passed

+0 :ok:
refguide
5m  7s
branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
3m 30s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+0 :ok:
refguide
5m  8s
patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.

_ Other Tests _

+1 :green_heart:
asflicense
0m 16s
The patch does not generate ASF License warnings.

27m 21s

Subsystem
Report/Notes

Docker
Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2455/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/2455

Optional Tests
dupname asflicense refguide

uname
Linux f86c171828c0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 687e53b7e4

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2455/1/artifact/yetus-general-check/output/branch-site/book.html

refguide
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2455/1/artifact/yetus-general-check/output/patch-site/book.html

Max. process+thread count
78 (vs. ulimit of 30000)

modules
C: . U: .

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2455/1/console

versions
git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Not sure what is the problem...
@saintstack @apurtell , do you still remember what is the problem here?


chore: Add data property in entity explorer as part of "show bindings"

Description

Fixes #12196
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Cypress
Checklist:

[x] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Test coverage results :test_tube:
:red_circle: Total coverage has decreased
// Code coverage diff between base branch:release and head branch: fix/add-data-to-show-bindings

Status
File
% Stmts
% Branch
% Funcs
% Lines

:red_circle:
total
56.29 (-0.01)
37.74 (-0.01)
35.92 (0)
56.52 (0)

:red_circle:
app/client/src/utils/autocomplete/EntityDefinitions.ts
23.81 (-2.51)
0 (0)
12.5 (0)
23.08 (-2.63)

/ok-to-test sha=07fa204

/ok-to-test sha=49e891d

/ok-to-test sha=2591eca

/ok-to-test sha=9bbdbd1


Raise when schemaless struct is used in unsafe_validate_unique/4

I noticed this when looking over this PR: https://github.com/elixir-ecto/ecto/pull/3955.

You can pattern match if the __meta__ field has the Ecto.Schema.Metadata struct. That should be enough and a single case should do it?

Ah thank you. Sorry I had it in my head that metadata was a regular map.


README.md v1.0

updated readme with basic header and sub headers as well as add basic info that will be needed later on

sup.


Remove useless type attribute from snippet

Type attribute on <script> is completely useless in all cases, unless it’s type="module" or you’re using VBScript.

Thank you!


Upgrade to Django 1.11

A new version of Django is out with a new round of updates and deprecations. Good news is that 1.11 is a Long-term stable release so we should have less work now 👍
NOTE: this PR adds a migration so make sure you run ./manage.py migrate. The migration simply adds a database index to some of the fields that we're using for ordering results. There's some other changes there to set default ordering on some tables.
There's still a warning generated which I will deal with in another PR.

Codecov Report

Merging #469 into master will increase coverage by 0.05%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #469      +/-   ##
==========================================
+ Coverage   93.61%   93.67%   +0.05%
==========================================
  Files          24       25       +1
  Lines        1504     1518      +14
==========================================
+ Hits         1408     1422      +14
  Misses         96       96

Impacted Files
Coverage Δ

dplace_app/migrations/0002_updates.py
100% <100%> (ø)

dplace_app/models.py
98.81% <100%> (+0.06%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cbac8dc...203e487. Read the comment docs.


Change how we store state to reduce state growth

Even in absence of transactions the persisted state is growing by approx. 250-300 MB/day. This is because rewards are issued to validators on each block and we produce 86,400 bocks per day and we store state trie updates. We shall consider a more efficient way of storing the state and only construct the trie on the fly to compute its root. Please add your thought @JamesHinshelwood

Simple data compression would have minimal impact:
-rw-r--r-- 1 root root 578M Nov 15 04:02 db.sqlite3
-rw-r--r-- 1 root root 262M Nov 15 04:02 test.gz

This is related to issue #1748

Closing for now, as size of SQLite database has gone down by 20% with the SQLite changes. However, if that is insufficient, then we will need an alternative to having an SQLite database and/or choosing a different window to store things e.g. daily data only.


[WFLY-16833]: Upgrade Apache Artemis to 2.25.0

Upgrade to Artemis 2.24
Return type changed from int to long.

Signed-off-by: Emmanuel Hugonnet<EMAIL_ADDRESS>

Thanks @ehsavoie


Make SemanticVersion compatible with SymVer 2.0

Fix #5004
Fix description
Made compatible with SymVer 2.0 http://semver.org/ (p.10)
Fixed comparisons
Refactored tests
Added new tests
Additional information
Useful sample https://github.com/maxhauser/semver/blob/master/src/Semver/SemVersion.cs
Related #2983

@JamesWTruher @PaulHigin Can you please review this PR?

I've rebased to resolve a merge conflict.

@anmenaga Can you please also review this PR? Thanks!

I still have a concern about the GetHashCode() method. See my new comment at https://github.com/PowerShell/PowerShell/pull/5037#discussion_r149441925

@SteveL-MSFT This PR changes the public APIs, including:

Add a public constructions public SemanticVersion(int major, int minor, int patch, string preLabel, string buildLabel)
Remove public property Label and add two public properties PreReleaseLabel and BuildLabel.
Add a public method public static int Compare(SemanticVersion versionA, SemanticVersion versionB)

Do you think it needs the committee review?

@daxian-dbw the general guidance is that for small api changes, the code review process is sufficient.  However, any maintainer can request Committee review if needed or if the contributors for the code review can't come to an agreement.

SemanticVersion API is new in PowerShell Core so until RTM we can change it free if we need.

I'm fine with the API changes in this PR.

The test failure in AppVeyor is a known issue and will be fixed by #5379
Travis CI passed, but it's not updated here.

😕  Sorry, I forgot to set VS Code case-sensitive flag to search and replace.

The macOS CI was shown as passed yesterday after clicking the Travis CI "Details" link. I have no idea why it's shown as canceled today. Now we seem to have a problem with macOS CI -- it takes forever to start.
Given that I know the Travis CI previously passed, I will merge this PR.


Handle 408 error in client

In the request docstring it mentions 408 explicitly. I think we should check for that here explicitly. Note that in the one test we have in which the server responds with a 408, the client actually gives up before the server sends something back.
Originally posted by @Flix6x in https://github.com/FlexMeasures/flexmeasures-client/pull/7#discussion_r1187445990

In last days also http 502 and 504 (error) messages where received, also handle these?

Comment by Nicolas in PR #28
I believe we should mention that this will re-try until a schedule is ready.
Actually, I'm not so sure this works as expected, as we seem to poll only on failure (408, 503), not on responses that tell us that the results are not ready - I believe those are coded as 400 right now.
@Flix6x should this be resolved in another issue, also in the flexmeasures server?
Or am I interpreting this wrongly?
Originally posted by @nhoening in https://github.com/FlexMeasures/flexmeasures-client/pull/28#discussion_r1195706667

To be cleaer - It seems to me that 400 is the wrong code, so FlexMeasures server might require an issue, as well.


Separate files for split separation of concerns

This PR splits the logic among multiple modules for separation of concerns.

Hi thank you for the PR. I appreciate the work you put in but at this point I don't think that splitting up the crate makes much sense. Should it grow considerably I'll revisit this but for the time being keeping everything in a single file is preferable, I think.


Template responsive is not working

Hello,
In description found, This template is responsive.
But when I tried on my mobile, this template is not responsive.

Good day, @shuklabh. Thank you for your comment. We checked everything carefully and fixed the problem. The problem has been fixed for now. We apologize for the inconvenience caused to you. Have a nice day!)


Adds Deep Dive for Payment request API

cc @zkoch
This adds the updated PaymentRequest API docs to Web Fundamentals.

Change set looks all 👍  and test passing.
Holding merge until go sign on the content.


Rohwer Outpost Newspaper Articles Section

As a developer
I want to make the Rohwer Outpost Newspaper Articles section
so that that section is out of the way so I can worry about other things to work on
Acceptance Criteria

[x] Rohwer Outpost article pages are implemented

Tasks

[x] complete the same steps as the El Joaquin setup

Prerequisites

[x] assets in possession
[x] newspaper prefab completed

done yuuuh full send yuuuh


Add mautic trello plugin

Add documentation for the new Mautic Trello Plugin

Hi @adiwegs thanks for making the PR with the documentation! We don't yet support multilingual but we're hoping that we can at some point in the future.
Can you please take a look at our Style Guide here: https://docs.mautic.org/en/home/contributing#links and update your links to use the code macros? We are trying to move toward using it universally - some pages are still to be updated.
Thanks!

Waiting for https://github.com/mautic/mautic/pull/9086 to be merged.


Third assignment

Pull Request Test Coverage Report for Build 101

113 of 113 (100.0%)  changed or added relevant lines in 16 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 82:
0.0%

Covered Lines:
418

Relevant Lines:
418

💛  - Coveralls


How to add a mobile action for reading NFC tags?

In the ThingsBoard PE Mobile Application there are different mobile actions available like e.g. scan QR code.
https://thingsboard.io/docs/pe/mobile/mobile-actions/
We would like to add a mobile action for reading NFC tags. How should we proceed?

Any input on how we should approach this? If I am not mistaken we also not only need to change the mobile app, but also ThingsBoard itself to add a mobile action for NFC?
Feedback is appreciated.

Hello @technorabilia,
Quick googling leads me to this plugin which may help you. But that's not all and more changes will require definitely. I'm not ready to say what else.
But you may consider ThingsBoard support options https://thingsboard.io/docs/services/support/ to receive a complex answer or a ready solution.
Best Regards.


tiny demo project using ps-lite

is there any project demostrating abilities and best practices of ps-lite ?
mxnet is great, but too heavy for understanding ps-lite.
I am expecting one thousand lines codes which using ps-lite to enable multi-cards training.

If you just want to know more about ps-lite in a tiny example, dist-lr does it. It implements a distributed LR based on ps-lite (currently, only local version, but easy to extend to dist).

how to configure the distributed project. can you give me a example? thanks @ctliu3


The responsive framework is not scaling on a particular device

builder : (context, widget) => ResponsiveWrapper.builder(
          // ? warp all the heights and widths according to screen automatically
          widget,
          defaultScale: true,
          breakpoints: [
            ResponsiveBreakpoint.autoScale(420, name: MOBILE), //  breakpoint is set based on pixel 2 dpi
          ],
          backgroundColor: Theme.of(context).primaryColor,
        ),
// nested widgets ....

flutter mode : release
autoscale should render the same UI on all the devices right ? but why oneplus 6 differs from other devices am i doing anything wrong ?
Images :
UI in oneplus 6 :

UI in rest of the device like pixel2,oneplus 5t , vivo9 , pixel4 .. :

Good question! I have the OnePlus 6 device so lucky us.
Can you share a complete reproduceable code sample?


RSS Feed for the VS Code Blog Has an Invalid Content-Type Charset

Does this issue occur when all extensions are disabled?: Yes

VS Code Version:  latest
OS Version: Win 10

Note: this is an issue with https://code.visualstudio.com, rather than VS Code itself; the report problem link points here. If there is a better place to report issues with the website please make that clear.
Steps to Reproduce:

Use .NET 6's System,.Net.Http.HttpClient to get the contennt of  https://code.visualstudio.com/feed.xml as a string

// http is an instance of HttpClient
using var resp = await http.GetAsync(curUrl);
using var content = resp.Content;
contentText = await content.ReadAsStringAsync(); // ****

At the line marked **** get

System.InvalidOperationException: The character set provided in ContentType is invalid. Cannot read content as string using an invalid character set.
 ---> System.ArgumentException: 'utf8' is not a supported encoding name. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method. (Parameter 'name')

Examining the header one sees:
Content-Type: application/xml; charset=utf8

Problem: RFC 2616 section 3.4 defines that the charset parameter has a case insensitive value from the IANA Character Sets registry. The registry lists "UTF-8", not "utf8".

Fixed


kdenlive: minor refactoring

Description of changes
Follow up PR of https://github.com/NixOS/nixpkgs/pull/209941
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

I'll just do it myself because I'm doing a 24.05.2 update now and things are going to get messy.


Update API to match v3.27.0

Description
This corresponds to running [0] on patchset [1]
from the projectcalico/calico repository
[0] rsync -avz --delete --exclude=.git --exclude=.github . ../../api/
[1] 2e5ac6bd4f39c3905cef8485e145231948d27640
None required

/sem-approve

I think updates need to be done with the Makefile.local update target in this repo.
I've pushed a new release-v3.27 branch to this repository you can use, though!


add drag n drop support to x11

On top of making a merge commit to bring this up to date with main, I made a few fixes:

Properly flush the connection every time we send a XdndStatus message, to make sure the source application receives it and keeps sending us position events.
Also properly flush the connection when making a XConvertSelection request, to make sure the source application receives it and sends us the SelectionNotify reply.
Change the XConvertSelection request logic to only send a request if we haven't requested the data yet, instead of sending a request if we haven't received (or successfully parsed) the data yet. This fixes an issue where multiple requests would be needlessly sent if we received a new position event before the transfer was complete.
Track the timestamp of the XdndPosition event from which we made an XConvertSelection request, and match it in the SelectionNotify reply handler, to make sure we aren't reading stale replies.
Competely clear the cached drag-n-drop state and data when receiving a new XdndEnter event. This fixes some state/caching issues, but it is still a brittle fix that may cause issues if concurrent (or stale) drag sessions' events reach the application.

For good measure, I've also updated the femtovg example, so that the little orange square that follows the mouse also updates drop DragEnter/Position/Drop events. :slightly_smiling_face:

Yeah, I think it would be a good idea to follow how the spec suggests handling it.
I'm okay with you working on it if you're inclined to do so.

I just finished rewriting the logic to use a specific enum to keep track of the XDND session state. :slightly_smiling_face:
Incidentally I ended up completely rewriting the drag_and_drop file, and there isn't a single line of code remaining from winit, so I removed the attribution and license header from that file. (It's still there on the get_property file though, I haven't touched it.)
This commit should fix all of the issues I raised in my previous review. :slightly_smiling_face:


any possibilities of automatic ssl letsencrypt, and used as a reverse proxy to customized address and port e.g.<IP_ADDRESS>:8081 etc?

any possibilities of automatic ssl letsencrypt, and used as a reverse proxy to customized address and port e.g.<IP_ADDRESS>:8081 etc?

You can use any tls.Config you like. Generating this config is none of quic-go’s responsibilities though. I believe there are implementations of the ACME protocol in Go, so this probably won’t be more than a few lines of code.


Async Autocomplete does not show options initially

🐛 Bug Report
When typing a value that is processed async the results do not populate until a user clicks in the box again.
To Reproduce
Steps to reproduce the behavior:
It is currently broken in the showcase
Expected behavior
Results should appear as soon as text is typed.
Link to runnable example or repository (highly encouraged)
showcase
UI Kitten and Eva version

Package
Version

@eva-design/eva

@ui-kitten/components
v5.0.0

@fzf thanks for reporting. I faced the same in the project I'm currently working on.
You may try using the workaround below if you use function components:
  // TODO: UI Kitten issue.
  // Not updates data list visibility, if initially focused with wrong value.
  React.useEffect(() => {
    const shouldBecomeVisible = (autocompleteRef.current?.isFocused() || false) && (data?.length || 0) > 0;
    if (autocompleteRef.current?.state.listVisible !== shouldBecomeVisible) {
      autocompleteRef.current?.setState({ listVisible: shouldBecomeVisible });
    }
  }, [data?.length]);

@artyorsh thanks for the code. Looking forward to a better solution in the future.

@fzf thanks for reporting. I faced the same in the project I'm currently working on.
You may try using the workaround below if you use function components:
  // TODO: UI Kitten issue.
  // Not updates data list visibility, if initially focused with wrong value.
  React.useEffect(() => {
    const shouldBecomeVisible = (autocompleteRef.current?.isFocused() || false) && (data?.length || 0) > 0;
    if (autocompleteRef.current?.state.listVisible !== shouldBecomeVisible) {
      autocompleteRef.current?.setState({ listVisible: shouldBecomeVisible });
    }
  }, [data?.length]);

how to use it? Can you show code ex? please?

@Rizayev https://snack.expo.io/elAoHThIT


Functions v2 to Functions v3

Currently, guide suggested "Azure Functions v2 (.NET Core)", but Microsoft released v3.
Maybe guide could be updated to v3?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 369aefa6-74da-3001-f835-15abdfffd5d6
Version Independent ID: 2cc2a2db-d059-3252-7a1a-4261b7d3c37c
Content: Develop Azure Functions using Visual Studio
Content Source: articles/azure-functions/functions-develop-vs.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

Hi @SychevIgor Thanks for your feedback. We will review and update as appropriate.

@SychevIgor Yes there is a already a work item on this and this will consistent across all Azure function app docs. Thanks for providing feedback and help us in improving the docs. So for now I will proceed with closure of this and If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Database query quota

Goal

Ability to impose query rate limits at database level. This will consider all the queries that are made to the tables present under the database.
Way to specify query quota at database level

Introduce database rate limiter
Pinot decides if the query quota is reached or not with the help of QueryQuotaManager.
QueryQuotaManager interface right now has an acquire(tableName) method which decides if the quota for the provided table is reached or not by returning a boolean.
To allow similar mechanism for database query quota a new interface method acquireDatabase(databaseName) is introduced on QueryQuotaManager interface
Query quota provisioning
Right now as the query and storage quotas on tables are maintained at TableConfig itself, we follow the same approach for database quota configs as well.
We don't have a place to provide database specific configs as it not being a first class entity. But for this usecase and similar future usecases we need to maintain a separate DatabaseConfig znode.
Assumptions on the new DatabaseConfig znode:

This config does not represent the logical database entity.
Absence of this config does not prevent users from creating tables under a database.
Deletion of thus config does not delete the tables under the database.

Provide default database query quota in ClusterConfig as below
"databaseMaxQueriesPerSecond" : "1000"

If we wish to provide an override for a particular database we can do so by creating a znode under PROPERTYSTORE/CONFIGS/DATABASE/<databaseName>
{
  "databaseName" : "db1",
  "quota" : {
    "maxQueriesPerSecond" : "200"
  }
}

Exposed APIs on controller to manage this:

POST /databases/{databaseName}/quotas?maxQueriesPerSecond=<query quota>
GET /databases/{databaseName}/quotas

Query quota update handling
There are 2 source based on which the database query quota is decided

Default query quota provided at cluster config
Override query quota provided at database config

On top of the overall quota provided by above configs, the per broker quota is also affected by the number of live brokers in the cluster. Considering  all the above factors, rate limiter updates are handled as below.

Introduced new user defined message DatabaseConfigRefreshMessage which is sent to brokers upon the database config updates.
Added ClusterConfigChangeListener to ClusterChangeMediator to handle the changes made to database query quota in cluster configs.
Update per broker rate quota upon broker resource EV change
Create database rate limiter upon OFFLINE -> ONLINE transition of tables in  BrokerResourceOnlineOfflineStateModel

We may eventually end up using Database feature. I have a couple of related questions

Doe the DB level quota get even distributed across all tables in that DB ?
On a related note, how does the math work for updating / maintaining and enforcing quotas when there is both table level and DB level quota configured ? Is there a precedent ?
Can we add a new table to an existing DB where that table level quota is exceeding the DB level quota ?

@siddharthteotia

Does the DB level quota get even distributed across all tables in that DB ?
The db level quota is separate from table level quota. Each table has it's separate quota and individually or cumulatively it may exceed the db level quota. But queries will be blocked once the db level quota is reached. Its upto the db admin to ensure proper table level quotas are provided so that tables don’t end up contesting for the db quota share.

On a related note, how does the math work for updating / maintaining and enforcing quotas when there is both table level and DB level quota configured ? Is there a precedent ?
As mentioned above these 2 quotas are separate. For any incoming query both the table quota and db quota is checked and the query will fail if anyone of them is exceeding.

Can we add a new table to an existing DB where that table level quota is exceeding the DB level quota ?
Yes. We don't enforce the rule that sum of all table quotas under a db must not exceed db quota. Cluster/DB maintainer should take the call on what table quota should be and what db quota should be. Not necessary all tables may reach their quota at the same time. DB quota just ensures that at any point the particular db is not running more than x queries and does not affect other DBs in the cluster.

Codecov Report
Attention: Patch coverage is 0% with 65 lines in your changes missing coverage. Please review.

Project coverage is 46.48%. Comparing base (59551e4) to head (0e25959).
Report is 715 commits behind head on master.

Files
Patch %
Lines

...ache/pinot/common/metadata/ZKMetadataProvider.java
0.00%
37 Missing :warning:

.../common/messages/DatabaseConfigRefreshMessage.java
0.00%
11 Missing :warning:

...va/org/apache/pinot/spi/config/DatabaseConfig.java
0.00%
10 Missing :warning:

...a/org/apache/pinot/common/utils/DatabaseUtils.java
0.00%
7 Missing :warning:

:exclamation:  There is a different number of reports uploaded between BASE (59551e4) and HEAD (0e25959). Click for more details.
HEAD has 53 uploads less than BASE

Flag
BASE (59551e4)
HEAD (0e25959)

integration
7
0

integration2
3
0

temurin
12
1

java-21
7
1

skip-bytebuffers-true
3
1

skip-bytebuffers-false
7
0

unittests
5
1

unittests1
2
1

java-11
5
0

unittests2
3
0

integration1
2
0

custom-integration1
2
0

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #13544       +/-   ##
=============================================
- Coverage     61.75%   46.48%   -15.27%
- Complexity      207     1102      +895
=============================================
  Files          2436     1953      -483
  Lines        133233   103996    -29237
  Branches      20636    16789     -3847
=============================================
- Hits          82274    48341    -33933
- Misses        44911    52009     +7098
+ Partials       6048     3646     -2402

Flag
Coverage Δ

custom-integration1
?

integration
?

integration1
?

integration2
?

java-11
?

java-21
46.48% <0.00%> (-15.15%)
:arrow_down:

skip-bytebuffers-false
?

skip-bytebuffers-true
46.48% <0.00%> (+18.75%)
:arrow_up:

temurin
46.48% <0.00%> (-15.27%)
:arrow_down:

unittests
46.48% <0.00%> (-15.27%)
:arrow_down:

unittests1
46.48% <0.00%> (-0.41%)
:arrow_down:

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Are there tests for setting/updating qps due to helix messages for table qps ? Do you plan to tests those out as well ?

@vrajat I am not sure how to specifically test that, will try to add a new integration test for checking query quota. I have done manual testing to ensure all the different flows work. Maybe I can add a section on what all manual tests were performed.

The tests LGTM


Can't publish message (Markdown can't render)

Hello,
I installed isso on my production server, and I got that kind of message in my log:
2016-01-27 14:12:31,052 ERROR: GET /
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/isso/__init__.py", line 136, in dispatch
    response = handler(request.environ, request, **values)
  File "/usr/local/lib/python2.7/dist-packages/isso/views/__init__.py", line 45, in dec
    return func(cls, env, req, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/isso/views/comments.py", line 421, in fetch
    'replies'        : self._process_fetched_list(root_list, plain)
  File "/usr/local/lib/python2.7/dist-packages/isso/views/comments.py", line 464, in _process_fetched_list
    item['text'] = self.isso.render(item['text'])
  File "/usr/local/lib/python2.7/dist-packages/isso/__init__.py", line 114, in render
    return self.markup.render(text)
  File "/usr/local/lib/python2.7/dist-packages/isso/utils/html.py", line 102, in render
    return self._render(text)
  File "/usr/local/lib/python2.7/dist-packages/isso/utils/html.py", line 99, in <lambda>
    self._render = lambda text: sanitize(sanitizer, parser(text))
  File "/usr/local/lib/python2.7/dist-packages/isso/utils/html.py", line 69, in inner
    rv = md.render(text).rstrip("\n")
AttributeError: 'Markdown' object has no attribute 'render'
2016-01-27 14:13:10,464 ERROR: POST /new
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/isso/__init__.py", line 136, in dispatch
    response = handler(request.environ, request, **values)
  File "/usr/local/lib/python2.7/dist-packages/isso/views/comments.py", line 84, in dec
    return func(self, env, req, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/isso/views/__init__.py", line 45, in dec
    return func(cls, env, req, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/isso/views/comments.py", line 221, in new
    rv["text"] = self.isso.render(rv["text"])
  File "/usr/local/lib/python2.7/dist-packages/isso/__init__.py", line 114, in render
    return self.markup.render(text)
  File "/usr/local/lib/python2.7/dist-packages/isso/utils/html.py", line 102, in render
    return self._render(text)
  File "/usr/local/lib/python2.7/dist-packages/isso/utils/html.py", line 99, in <lambda>
    self._render = lambda text: sanitize(sanitizer, parser(text))
  File "/usr/local/lib/python2.7/dist-packages/isso/utils/html.py", line 69, in inner
    rv = md.render(text).rstrip("\n")
AttributeError: 'Markdown' object has no attribute 'render'

I'm using python 2.7 on raspbian.
What can I do to fix this

check the version of misaka, only support 1.0.2

Ok i'll try to downgrade it to version 1.0.2, it's 2.0.0, right now

Problem fixed downgrading misaka version, thanks


As an admin, I want the CSV export to include subtitles and sort titles so I can check imported and cleaned field values.

CSV export now includes subtitle and sort title columns with corresponding data populated. All fields present on the django admin are present in the CSV.


Update doc

Add comments on available_date

Is #301 still relevant, or can we close that PR?

Is #301 still relevant, or can we close that PR?

closed


[stable/docker-registry] remove s3 credential env vars if not specified

What this PR does / why we need it: This PR removed deployment's s3 credential environment variables if no aws credentials provided. Without this PR, this chart will fall into CreateContainerConfigError with s3 storage option and empty s3 credentials because credentials are provided only if they exists.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:

/assign @jpds

/assign @rendhalver

Good plan. I like it.

It seems to be ok to merge. Can you approve this?

/lgtm


Problem with updating settings for a specific group

When I try to update settings for a specific group I get an error:
I'm running the following lines:
$groupID = "df09dd06-e890-498c-ba84-ce7bd99c2894"
$template = Get-AzureADDirectorySettingTemplate | Where-Object {$_.displayname -eq "group.unified.guest"}
$settingsCopy = $template.CreateDirectorySetting()
$settingsCopy["AllowToAddGuests"]=$True
New-AzureADObjectSetting -TargetType Groups -TargetObjectId $groupID -DirectorySetting $settingsCopy
But it gives the following error:
New-AzureADObjectSetting : Error occurred while executing NewObjectSetting
Code: Request_BadRequest
Message: A conflicting object with one or more of the specified property values is present in the directory.
InnerError:
RequestId: a98bb318-736c-4375-9101-528b225847e8
DateTimeStamp: Tue, 16 Oct 2018 14:01:33 GMT
HttpStatusCode: BadRequest
HttpStatusDescription: Bad Request
HttpResponseStatus: Completed
At \de2900\bigdata\VAGT05\JB0251\Powershell\Office365Commands.ps1:101 char:1

New-AzureADObjectSetting -TargetType Groups -TargetObjectId $groupID  ...

  + CategoryInfo          : NotSpecified: (:) [New-AzureADObjectSetting], ApiException
  + FullyQualifiedErrorId : Microsoft.Open.MSGraphBeta.Client.ApiException,Microsoft.Open.MSGraphBeta.PowerShell.NewObjectSetting

What am I doing wrong?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4cd7a703-534a-5fbd-6470-5d29bf512fd2
Version Independent ID: 4f70ed16-6a1c-c0be-3c5e-d56e1b7c7275
Content: Configure group settings using PowerShell in Azure Active Directory
Content Source: articles/active-directory/users-groups-roles/groups-settings-cmdlets.md
Service: active-directory
GitHub Login: @curtand
Microsoft Alias: curtand

@astengaard
Thanks for your feedback! We will investigate and update as appropriate.

@astengaard I don't see an issue with your script. Only thing is as you already have a Group ID, you can try replacing
$template = Get-AzureADDirectorySettingTemplate | Where-Object {$_.displayname -eq "group.unified.guest"}
with
$template = Get-AzureADDirectorySettingTemplate -Id $groupID
Please check if that makes any difference.

@astengaard Please let us know if you have any further query in this regard else we will move ahead to close this issue. Thanks.

Sorry, I didn't get back, I had 2 days off...
I just tried changing the line you suggested, that gave me another error:
Get-AzureADDirectorySettingTemplate : Error occurred while executing GetDirectorySettingTemplate
Code: Request_ResourceNotFound
Message: Resource 'df09dd06-e890-498c-ba84-ce7bd99c2894' does not exist or one of its queried reference-property objects are not present.
InnerError:
RequestId: 8dd62851-7076-4220-abd1-1f1b109aeca4
DateTimeStamp: Fri, 19 Oct 2018 06:22:51 GMT
HttpStatusCode: NotFound
HttpStatusDescription: Not Found
HttpResponseStatus: Completed
When I read the docs for Get-AzureADDirectorySettingTemplate, it seems that the provided id should be an id for the template, not for a group, so it seems ok, that it is failing?
I think my Group already has an attached template, and that's why it is failing, but I dont know how to just edit the attached template for that group without changing it for other groups?
Oh btw. I'm running on Windows 7, does that change anything?

@astengaard We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.

@saurabhsensharma when I try the Remove-AzureADObjectSetting I get
Remove-AzureADObjectSetting -TargetType Groups -TargetObjectId $groupID -Id $settingId
Remove-AzureADObjectSetting : Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
At line:1 char:1

Remove-AzureADObjectSetting -TargetType Groups -TargetObjectId $group ...

  + CategoryInfo          : NotSpecified: (:) [Remove-AzureADObjectSetting], JsonReaderException
  + FullyQualifiedErrorId : Newtonsoft.Json.JsonReaderException,Microsoft.Open.MSGraphBeta.PowerShell.RemoveObjectSetting

Any idea?
Note that this only happens if I try to change the $settingsCopy["AllowToAddGuests"]=$true from $true to $false or $false to $true, here is the script below
$template = Get-AzureADDirectorySettingTemplate | ? {$_.displayname -eq "group.unified.guest"}
$settingsCopy = $template.CreateDirectorySetting()
$settingsCopy["AllowToAddGuests"]=$true
$groupID= (Get-AzureADGroup -SearchString "Test-EXT1").ObjectId
New-AzureADObjectSetting -TargetType Groups -TargetObjectId $groupID -DirectorySetting $settingsCopy
Get-AzureADObjectSetting -TargetObjectId $groupID -TargetType Groups | fl Values

@hhmaity , I was receiving the same error when submitting Remove-AzureADObjectSetting.
I found the problem was that I had set $groupID equal to the group itself rather than to the group's ObjectID.

Basically, there is a template (Setting object) already for the Unified Guest so we need to remove that and then apply the settings.
Step1: get the Id of the template.
Get-AzureADObjectSetting -TargetObjectId $groupID -TargetType Groups
Copy the Id from the output next to the DIsplayName =>  Group.Unified.Guest
Step 2: Remove the setting object from the group.
Remove-AzureADObjectSetting -TargetType Groups -TargetObjectId $groupID -Id "Above GUID"
Here $groupId is the group ID of the team where we are unable to add guest (Unable to change settings for)
and Id will take the value of the if from the output of above command.
Once this is executed successfully, you can change the settings of the group like allowing/blocking guest access etc.


Cannot access compute error when training for object detection

System Information (please complete the following information):

Model Builder Version: 16.2.2047901
Visual Studio Version: 16.8.0 Preview 3.2

Describe the bug
Receive the error "Cannot access compute. If using a low priority compute it may not currently be available. Try again with a dedicated compute or try again later."
To Reproduce
Steps to reproduce the behavior:
Created a new compute for the Object Detection scenario, added the data set, and clicked to train.
Screenshots
Error:

Selected workspace:

Workspace details:

Additional context
I believe it's similar to #991 but I still get the error when using a dedicated compute instance.

Thanks for reporting! we will look into it.

Hi @jwood803, can you provide the log file to see the issue in more detail?

Sure, @zewditu! I uploaded it here.

Thanks @jwood803 , we will look into it.

@jwood803 unfortunately the logs are missing the inner exception that we'd need to get more information. We'll fix that in a future release.
Does it look like the compute was successfully created in the portal? Are there enough compute cores available to start the compute?

@JakeRadMSFT It was successfully created, but it does show 0 nodes and 0 node count. I'm not sure if it only takes nodes when it does a run or if it needs at least one node for it to work.

@JakeRadMSFT I'm getting the same error.
Full log:
37aeb4a9-72d5-4cac-8622-1551e545b7c5.txt


Email Forwarding is really Email Redirecting

You should really change the name from Email Forwarding to Email Redirecting.
It changes the To field to the user's mailbox that it is forwarding to, rather than keeping the original recipient. That is redirecting, your changing the intended recipient. This confused us when we had to trace and investigate emails and hurt our reporting numbers.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5885a586-76b8-83ae-161b-6c9c8793b42d
Version Independent ID: 681fe343-6272-a75a-ed00-dc7ef4355e67
Content: Configure email forwarding - Microsoft 365 admin
Content Source: microsoft-365/admin/email/configure-email-forwarding.md
Service: o365-administration
GitHub Login: @KwekuA
Microsoft Alias: kwekua

@Dburke225  Thank you for your feedback.
If you use redirect, your email will be redirected to another email address specified by the user. This will redirect the emails as if they came directly from the original sender's email address.
If you use forward, your email will be forwarded to another email address, but you will not be able to reply to the original sender. Forwarded emails will arrive as if it was forwarded to you.
Hope this helps!
Thanks
Sri

@msbemba Thank you very much for the contribution and sharing this explanation. @Dburke225 Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


[Tour of Beam] Main page layout

Resolves #22600

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Per Dart package conventions, a package must contain few public libraries that export the implementation files, so that a package hides its directory structure.
See https://dart.dev/tools/pub/package-layout#public-libraries
In our case, we should have playground_components.dart that exports everything the package has, and import it with
import 'package:playground_components/playground_components.dart';

Split view turned very slow.
https://user-images.githubusercontent.com/44893228/188454798-91dc15ad-23ea-4f79-8739-05424b989e73.mp4

Split view turned very slow.
vokoscreen-2022-09-05_16-56-51.mp4

https://user-images.githubusercontent.com/31556582/188841225-0cdd48c2-0044-496b-bdd2-255c2a771c42.mov
Theme mode toggle works very slow too. Is there a quick way to check if it will be smooth in production?

Theme mode toggle works very slowly too.

I think we can ignore this. It's not a daily use case. I have always thought that it's a nice effect, like a transformer robot takes time to transform.


Adapting Pokemon Showdown

This is a list of functions in the two Pokemon Showdown folders this project is adapting (sim and dex) that will, should, and might have its logic completely changed. The functions are also listed for their main reason for changing. To discuss a function in depth, a separate issue can be created directly from this list. I'll be editing this list as ideas change.
Priority 1 (Will Change)

[x] #3
[x] Battle.toJSON/fromJSON [Performance] (copying battle states was resolved by #2)
[x] PRNG.sample [Functionality] (resolved by #2)
[x] #2
[x] Battle.speedSort [Functionality]
[x] #6
[x] Battle.randomizer [Functionality] (resolved by #2)
[x] #7
[x] Battle.go [Functionality] (resolved by #2)
[x] #5
[x] #4

Priority 2 (Should Change)

[ ] BattleQueue.resolveAction [Performance]
[ ] BattleActions.useMove (and helper functions) [Performance][Functionality]
[ ] Dex___.getByID [Performance]
[ ] Pokemon.calculateStat/getStat [Performance]
[ ] Battle.boost [Performance][Functionality]
[ ] Battle.spreadDamage [Performance][Functionality]
[ ] Side.chooseMove [Functionality]
[ ] Side.chooseSwitch [Functionality]
[ ] Side.chooseTeam [Functionality]
[ ] Battle.start [Functionality]

Priority 3 (Might Change)

[ ] ModdedDex.getEffectiveness [Performance][Functionality]
[ ] Pokemon.transformInto [Performance][Functionality]
[ ] Pokemon.setStatus [Performance]
[ ] Pokemon.addVolatile [Performance]
[ ] Pokemon.effectiveWeather [Performance]
[ ] Pokemon.runEffectiveness [Performance]
[ ] Pokemon.runImmunity [Performance]
[ ] Pokemon.runStatusImmunity [Performance]
[ ] Pokemon.transformInto [Functionality]

With all the Priority 1 functions planned out, it's time to start working on the vertical slice!


Cloud add typescript types?

as title, cloud add the types files for typescript development?

Hi @agrozyme
Happy Christmas & New Year!
I will add the typescript files ASAP.


Update LICENSE

add 2019,
PR Checklist
Please check all that apply to this PR using "x":

[x ] I have checked that this PR is not a duplicate of an existing PR (open, closed or merged)
[x ] I have checked that this PR does not introduce a breaking change
[ ] This PR introduces breaking changes and I have provided a detailed explanation below

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Breaking changes (bugfix or feature that introduces breaking changes)
[ ] Code style update (formatting)
[ ] Refactoring (no functional changes)
[x ] Documentation changes
[ ] Other - Please describe:

@ykris45 thank you, but I don't think we need to update all these yearly, see https://softwareengineering.stackexchange.com/questions/130478/what-copyright-date-for-an-update-to-an-open-source-project-from-last-year

@tzarebczan owh.. thanks for add this information to my self.. now... i know more about this information and added more knowledge in my brains

Thanks @ykris45 !!


Docker: can't open file 'gfootball.ppo_train.py': [Errno 2] No such file or directory

Hello I have an issue, where my own python script cannot be found.
The ppo-train.py file is in the same directory as the dockerfile.
My dockerfile:
.
.
RUN pip3 install --use-feature=2020-resolver --no-cache-dir git+https://github.com/openai/baselines.git@master
COPY . /gfootball        # <--- This line here should copy every file in the same directory as the dockerfile to the docker directory gfootball, but in the end, I cant access my own file. But i can access examples!
RUN cd /gfootball && pip3 install --use-feature=2020-resolver .
WORKDIR '/gfootball'

I only use the cpu docker version. So, how can I fix this issue? Thanks in advance!

File (gfootball.ppo_train.py) name is suspicious. Don't you try to load it with -m (in which case you should not provide .py suffix?).


Output problem.

I hope someone will help me solve serious problem with output. I'm using Phi-3.1-mini-4k-instruct-Q4_K_M.gguf and when i ask it same algebra test question it never answers correctly, while in text-gen-webui or kobolt.cpp it always answers correctly. I tried everything, with promt template and without it. (<|system|> {system_prompt}<|end|><|user|> {prompt}<|end|><|assistant|>)
Tried both android version and windows version, both fail.
(i also tried to erase or turn off everything in maid character settings)
Question is: Two years ago, I was three times as old as my brother was. In three years, I will be twice as old as my brother. How old is each of us now?
Answer:  17 and 7
Here is how it answers correctly in text-generation-webui or koboltcpp:
Let's assume my current age is A and my brother's current age is B.
Two years ago, I was (A-2) years old and my brother was (B-2) years old.
According to the first statement, I was three times as old as my brother two years ago:
A - 2 = 3(B - 2)
Now, in three years, I will be (A+3) years old and my brother will be (B+3) years old.
According to the second statement, I will be twice as old as my brother in three years:
A + 3 = 2(B + 3)
Now we have two equations:

A - 2 = 3B - 6
A + 3 = 2B + 6
Let's solve the system of equations:
From equation 1:
A = 3B - 4
Substitute A in equation 2 with (3B - 4):
3B - 4 + 3 = 2B + 6
3B - 1 = 2B + 6
3B - 2B = 6 + 1
B = 7
Now that we have B, we can find A using the first equation:
A = 3B - 4
A = 3(7) - 4
A = 21 - 4
A = 17
So, currently, I am 17 years old and my brother is 7 years old.

Is it possible that the lack of instruct mode may be the problem?

I think the model just cant math

I think the model just cant math

As i said, I'm 100% sure, that using the same gguf file in different apps gives different results. This model is very good at math.
For example here: https://ai.azure.com/explore/models/Phi-3-mini-4k-instruct/version/9/registry/azureml (down on the right is where you enter promt).
Or anywhere you can run the model. Try the question i provided.

I am using the same model as in the screenshots on the main page. Always gettings strange answers.

I am using the same model as in the screenshots on the main page. Always gettings strange answers.

Update:
I just saw there is a newer versions of the app here on github. I installed the app from f-droid first. After installing the version from github it solved the problem.


Multi-threaded inference broken with MKLDNN

Description
I want to do multi-threaded inference with shared model parameters, so I'm testing the MXPredCreateMultiThread API in cpp example. I find that the example is broken with more than 1 threads on MKLDNN build: the output of model inference is not deterministic.
Environment info (Required)
----------Python Info----------
('Version      :', '2.7.16')
('Compiler     :', 'GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1<IP_ADDRESS>)')
('Build        :', ('default', 'Jun 19 2019 07:40:37'))
('Arch         :', ('64bit', ''))
------------Pip Info-----------
('Version      :', '19.1.1')
('Directory    :', '/usr/local/lib/python2.7/site-packages/pip')
----------MXNet Info-----------
('Version      :', '1.5.0')
('Directory    :', '/Users/arcadia/work/mxnet-1.5/python/mxnet')
Commit hash file "/Users/arcadia/work/mxnet-1.5/python/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source.
('Library      :', ['/Users/arcadia/work/mxnet-1.5/python/mxnet/../../lib/libmxnet.so'])
Build features:
✖ CUDA
✖ CUDNN
✖ NCCL
✖ CUDA_RTC
✖ TENSORRT
✔ CPU_SSE
✔ CPU_SSE2
✔ CPU_SSE3
✔ CPU_SSE4_1
✔ CPU_SSE4_2
✖ CPU_SSE4A
✔ CPU_AVX
✖ CPU_AVX2
✖ OPENMP
✖ SSE
✔ F16C
✖ JEMALLOC
✖ BLAS_OPEN
✖ BLAS_ATLAS
✖ BLAS_MKL
✖ BLAS_APPLE
✖ LAPACK
✔ MKLDNN
✔ OPENCV
✖ CAFFE
✖ PROFILER
✖ DIST_KVSTORE
✖ CXX14
✖ INT64_TENSOR_SIZE
✖ SIGNAL_HANDLER
✖ DEBUG
----------System Info----------
('Platform     :', 'Darwin-18.2.0-x86_64-i386-64bit')
('system       :', 'Darwin')
('node         :', 'MacBookPro')
('release      :', '18.2.0')
('version      :', 'Darwin Kernel Version 18.2.0: Fri Oct  5 19:41:49 PDT 2018; root:xnu-4903.221.2~2/RELEASE_X86_64')

Build info (Required if built from source)
clang
MXNet commit hash:
latest commit
4d07d78e1aff36f603231568bfff88242d073756
Minimum reproducible example
I've slightly modified the cpp example to print the first 10 numbers in the output ndarray:
download the code and replace the example/image-classification/predict-cpp folder.
In order to run the example, the changes in PR #15574 is needed to patch the mxnet code.
MXNET_ENGINE_TYPE=NaiveEngine ./image-classification-predict path/to/image [thread number]

The output
openblas 2 threads:
./model/mobilenetv2_0.25-symbol.json ... 146314 bytes
./model/mobilenetv2_0.25-0000.params ... 6135676 bytes
[02:11:17] src/engine/engine.cc:55: MXNet start using engine: NaiveEngine
3.16592 -2.01711 -3.29867 -10.9845 -1.56596 2.30844 -4.57165 -0.211675 0.300732 -4.35327
3.16592 -2.01711 -3.29867 -10.9845 -1.56596 2.30844 -4.57165 -0.211675 0.300732 -4.35327
run successfully

mkldnn 2 threads:
./model/mobilenetv2_0.25-symbol.json ... 146314 bytes
./model/mobilenetv2_0.25-0000.params ... 6135676 bytes
[02:12:40] src/engine/engine.cc:55: MXNet start using engine: NaiveEngine
-4.33902 -8.3967 -3.97791 -2.16607 -7.36041 -7.65456 -6.08846 -3.10948 -6.65086 4.17571
-0.0463818 0.440686 -0.907998 -7.17286 4.8312 -3.03199 4.31585 0.0692099 -0.844623 -8.1773
run successfully

The results change randomly with every execution.

@pengzhao-intel

MXNet does not support multithreading from the interface level. Not even locking based access. The only way to use MXNet in a multi-threaded fashion is by using a jobqueue that is consumed by a sticky thread.

@wuxun-zhang @ZhennanQin please help take a look for this issue, thanks.

@pengzhao-intel @wuxun-zhang @ZhennanQin
The difference between MXPredCreateMultiThread and create executor separately is that the former  shares model parameters. I've tested that if the weight is not shared when creating executors by moving arg_arrays and aux_arrays creation into for loop, then the  result is normal even with MKLDNN.
  for (int i = 0; i < num_threads; i++) {
    std::vector<NDArray> arg_arrays, aux_arrays;
    for (size_t i = 0; i < arg_shapes.size(); ++i) {
      NDArray nd = NDArray(arg_shapes[i], ctx);
      if (arg_params.count(arg_names[i]) != 0) {
        CopyFromTo(arg_params[arg_names[i]], &nd);
      }
      arg_arrays.push_back(nd);
    }
    for (size_t i = 0; i < aux_shapes.size(); ++i) {
      NDArray nd = NDArray(aux_shapes[i], ctx);
      if (aux_params.count(aux_names[i]) != 0) {
        CopyFromTo(aux_params[aux_names[i]], &nd);
      }
      aux_arrays.push_back(nd);
    }

    std::unique_ptr<MXAPIPredictor> ret(new MXAPIPredictor());
    ret->sym = sym;
    ret->ctx = ctx;
    ret->key2arg = key2arg;
    ret->arg_arrays = arg_arrays;
    ret->aux_arrays = aux_arrays;
    ret->out_shapes = out_shapes;

    if (!lazy) {
      std::map<std::string, Context> ctx_map;
      std::vector<NDArray> grad_store(arg_arrays.size());
      std::vector<OpReqType> grad_req(arg_arrays.size(), kNullOp);
      ret->exec.reset(Executor::Bind(sym, ctx, ctx_map,
                                     arg_arrays,
                                     grad_store, grad_req,
                                     aux_arrays));
      ret->out_arrays = ret->exec->outputs();
    }
    out[i] = ret.release();
  }

It's very strange, I think the model parameters are read-only, will it affect MKLDNN calling?

@ZhennanQin I think MKLDNN is OK in multi-threading, the possible reason is that calling mkldnn related method like MKLDNNDataReorder and GetMKLDNNData on ndarray simultaneously messes up the internal mkl_mem_.

Similar race condition on ndarray: #9862

@arcadiaphy did you local solution work for this issue?

@pengzhao-intel My local solution works fine, but it's not suitable for an official PR.


Dask and Distributed Support

(continued from #291)
It would be nice if sentry would support dask and distributed. Namely, the following features should be supported:

sentry sdk initialized on the client should be carried over to the workers
flushing the event queue should work (e.g. on subprocess shutdown for local executors or on worker shutdown for distributed)

feel free to split this ticket into two if required

sentry sdk initialized on the client should be carried over to the workers

We don't support this for any task queue. I'm not sure if it's a good idea because I don't quite know how the implementation would work. We also don't transfer scopes from clients to workers.

We  don't support this for any task queue. I'm not sure if it's a good idea  because I don't quite know how the implementation would work. We also  don't transfer scopes from clients to workers.

OK. Could than at least the integration add some context to the events emitted by the workers? (like computation ID or even which part of the computation graph is currently executed)

Sure. I don't want to discard the idea of carrying over scopes if it makes sense, but carrying over SDK config is probably not expected.
I'm open to all kinds of ideas, but they should be non-invasive in the sense that we shouldn't just add new arguments to a job just because we need them on the other machine.

Currently we do not have plans to support this, Sorry.


Wiki shortcut link for quick start guide is pointing to a 404

Wiki shortcut link for quick start guide is out of date. Its currently pointing to a 404 error

Try clearing your cache. The documentation has been rebranded and the names of the files have changed. Make sure your url matches the below version.
https://github.com/BeyondTrust/pbis-open/wiki/docs/adb-quick-start-guide.pdf

Please see attached image.

Thanks for the clarification. I seemed to have forgotten that link in the update. I've correct it.


Chevron Icons

Describe the requested icon
You'll often see chevron's used for dropdown menu's, or buttons that go back and forward.
I tried to use the Phosphor arrow's, arrow-circle's and caret but these look a bit weird to me for these controls.
Screenshots
This is what we're currently using for a dropdown menu:

And here we are using the Phosphor caret. It looks okay but I suspect chevrons would display nicer:

What is you actual request here?

@rektdeckard i'm requesting basic chevrons for each direction.  I set our default to duotone right off the bat and I just realised the Caret icons are literally chevrons if you disable duotone :P


Add CNEC poster to website

@gtw2  Please submit a pull request that adds your CNEC poster to the website. It should be a pdf and should be located at  /pres/2018-03-08-cnec.pdf within the repository.

closed with #73


Unable to install on Fedora 35

Hello,
I have recently moved computers over and thus installed a fresh Fedora 35. I attempted to install by entering the required commands:
sudo dnf install -y dnf-plugins-core distribution-gpg-keys
sudo dnf copr enable taw/element
sudo dnf install -y element --refresh

However when running the final command to install element, it does no install and outputs the following:
No match for argument: element
Error: Unable to find a match: element

I can confirm the copr repo is there and is enabled.

I did notice the following note on the copr site but not on this git repo

If you require any further details do let me know.

It's annoying that those instructions are still in the README.md and on the element.io website itself.
If it doesn't exist anymore, please consider removing those instructions or mentioning the licencing issues of Electron.

What are the possible solutions for the licensing issue?

Maybe Spectral https://gitlab.com/spectral-im/spectral or Nheko https://github.com/Nheko-Reborn/nheko  is an alternativ?

Spectral is in Alpha and the development seems dead or slow. Nheko seems more active and is available on fedora. Might be a good choice until this issue is fixed.

Is the previous Element fc34 rpm available for download anywhere?
I have been using Nheko as an alternative matrix client for a few weeks and I have had several segfaults. It is bad at handling gifs and mp4, the emoji function is also broken, encryption seems to be buggy as well. The user interface is not finished and several features supported by Element do not work at all.
I also suggest contacting Element over email or their support channel and asking them to help the developer solve the issues or make their own rpm package.


MKDADensity does not work with FDRCorrector or FWECorrector(method='bonferroni')

When trying to correct for multiple comparisons using FDR or FWE (using bonferroni, the montecarlo method works fine), the following error arises.
ValueError: <class 'nimare.correct.FDRCorrector'> requires "p" maps to be present in the MetaResult, but none were found.

psuedo-code
from nimare.correct import FDRCorrector
from nimare.meta import mkda

meta = mkda.MKDADensity()
corr = FDRCorrector()
res = meta.fit(dset)
cres = corr.transform(res)  # this is where the error occurs

Unlike #347, I don't think this is an easy fix. The MKDA methods (minus MKDA Chi2) don't have any significance testing. The core algorithm just calculates the OF maps (essentially the MKDA equivalent to an ALE map). The significance testing happens with max-value Monte Carlo procedure in the FWECorrector, so there's no p-value map to correct for the other Correctors.
I will need to review both the original MKDA papers and possibly the ALE code for deriving its first null distribution to see if there's a native method that I missed or if we can adapt ALE's method to MKDA, respectively.

@tsalo in that case, maybe we should separate permutation generation from monte carlo correction? In principle, I think one should be able to generate permutation-based p-value maps independently of what correction procedure one wants to use. I.e., I don't see a principled reason why one couldn't generate p-values for MKDA via permutation, and then apply bonferroni correction to that.

This is kind of a naive question, but wouldn't the permutation-based p-value maps still be FWE-corrected? The procedure is basically the same as the ALE Monte Carlo FWE-correction procedure, after all.

I'd have to look at the implementation, but the point is just that the generation of permutations is independent of the correction procedure. In the FWE procedure, you generate a bunch of permutations, take the max statistic in each permutation, and then select a threshold based on the Nth percentile of the distribution of max stats. This is the gold standard approach, and should probably remain the default/recommended correction procedure. But you don't have to do it this way; you could, e.g., just construct the null distribution of test statistics at each voxel in the brain, and then refer the observed value to that distribution to get a p-value at each voxel—at which point you can theoretically apply most of the standard MC correction procedures (e.g., pick an arbitrary threshold like p < .0001; use Bonferroni, etc.).
Since the computationally intensive part of the procedure is (I'm assuming) the actual permutation of foci/clusters, it would make sense to abstract that part out of the FWE correction procedure, and have the latter invoke the permutation generation implicitly before taking the max statistic.
If you don't want to mess with the existing FWE correction implementation, that's totally cool, but we would still want to add standalone permutation generation routines, so that we can get a permutation-based p-value at each voxel. At that point, we can address the issue @jdkent flagged—i.e., we would be able to FDR or Bonferroni-correct the permutation-based p-values, just like any other p-values.
Basically what I'm saying is that, for the major estimators, there are in my view really three separate choices the user is making: (1) the estimator, (2) how the p-values are generated (analytically vs. permutation), and (3) how to correct those p-values. It may be that not all possible combinations are going to be valid, but I do think we should be supporting the ones in the title.

In terms of actual usage, would it then look something like this?
from nimare.correct import FWECorrector
from nimare.meta import mkda, ale

meta = mkda.MKDADensity(n_iters=10000, n_cores=2)
corr = FWECorrector(method="bonferroni")  # "montecarlo" no longer available
res = meta.fit(dset)
cres = corr.transform(res)

meta2 = ale.ALE()
corr2 = FWECorrector(method="montecarlo", n_iters=10000, n_cores=2)
res2 = meta2.fit(dset)
cres2 = corr2.transform(res2)  # same basic procedure as MKDADensity.fit()


Include link to Stackblitz starter in the Incorrect Content Report 📄 template?

I'm filing a new issue with Issue: Incorrect Content Report 📄. It asks for a Stackblitz reproduction - but doesn't directly provide a link to it. Mind if I add a link to https://astro.new for convenience? 🙂

Sorry for the wait :sweat_smile:
That sounds like a great addition!


Live photos from the iPhone have a different size in listings compared to when downloaded

Category

[ ] Question
[ ] Documentation issue
[x] Bug

Expected or Desired Behavior
The REST API shows the size of live photos of type HEIC  uploaded with the iPhone to be X bytes.
Observed Behavior
However when the media for these are downloaded they show a different value Y bytes which is much shorter.
Here is the file when listing it via the API
2021/09/16 11:06:17 DEBUG : GET /v1.0/drives/7de8256bfa42446/items/7DE8256BFA42446!103:/big_swan.heic: HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.55.1-DEV
Authorization: XXXX
Accept-Encoding: gzip

And the response (JSON prettified and edited for privacy)
2021/09/16 11:06:17 DEBUG : HTTP RESPONSE (req 0xc000475000)
2021/09/16 11:06:17 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: no-store
Client-Request-Id: 4d33f2d3-99c2-4112-9622-a1f3d3779c15
Content-Type: application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8
Date: Wed, 15 Sep 2021 23:06:17 GMT
Odata-Version: 4.0
Request-Id: 4d33f2d3-99c2-4112-9622-a1f3d3779c15
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"Australia Southeast","Slice":"E","Ring":"4","ScaleUnit":"001","RoleInstance":"ML1PEPF00004AFB"}}

7f7
{
<EMAIL_ADDRESS>"https://graph.microsoft.com/v1.0/$metadata#drives('7de8256bfa42446')/items/$entity",
<EMAIL_ADDRESS>"https://public.dm.files.1drv.com/y4mmCSrkuSjRthyO6ZFDft4_m_LxFe8J7BBWl2ylp6zXYSS-YZBeuQ_U6DF4nV4bv1FDl2DgP_6Fp4FO4OF6CWR6YiDiCnrqZrIzRK25BtSj8etPi5pALEqzzFH2LBsty-hXLK0plO2TohCr9FeRfvZGRJU9cC_j20yB6QXPR_wBxwMvpsKDBH382f10ibeeYt4DbBMNMfVCr2qrmyzkg8HtjRxRT_MG9oUhSws2NQ5POkg-smbi7X86P20FynIs0UA0ieIv3jPL-yt-CSNTVbBR66FwelRlUkPz25sC4QNb88",
  "createdDateTime": "2021-09-13T22:18:06.877Z",
  "cTag": "aYzo3REU4MjU2QkZBNDI0NDYhNDQ3NC4yNTg",
  "eTag": "aN0RFODI1NkJGQTQyNDQ2ITQ0NzQuNg",
  "id": "7DE8256BFA42446!4474",
  "lastModifiedDateTime": "2021-09-13T22:27:05.36Z",
  "name": "big_swan.heic",
  "size": 6352331,
  "webUrl": "XXX",
  "reactions": {
    "commentCount": 0
  },
  "createdBy": {
    "application": {
      "displayName": "OneDrive",
      "id": "480f7c59"
    },
    "user": {
      "displayName": "user",
      "id": "7de8256bfa42446"
    }
  },
  "lastModifiedBy": {
    "user": {
      "displayName": "user",
      "id": "7de8256bfa42446"
    }
  },
  "parentReference": {
    "driveId": "7de8256bfa42446",
    "driveType": "personal",
    "id": "7DE8256BFA42446!103",
    "name": "Pictures",
    "path": "/drive/root:/Pictures"
  },
  "file": {
    "mimeType": "image/heic",
    "hashes": {
      "quickXorHash": "zZ7mPwDhaBHFKsHrBYeywOD0sdM=",
      "sha1Hash": "F7FD365360A267FE2249BCB3F290198F7851AF6D",
      "sha256Hash": "6B876FF49433ACEA015F1C59DA851095192D2EE2F76CD36922EC184DC334D4FF"
    }
  },
  "fileSystemInfo": {
    "createdDateTime": "2021-09-13T22:18:06.877Z",
    "lastModifiedDateTime": "2021-09-13T22:22:35.663Z"
  },
  "image": {
    "height": 3024,
    "width": 4032
  },
  "pendingOperations": {
    "pendingContentUpdate": {
      "queuedDateTime": "2021-09-13T22:22:14.92689Z"
    }
  },
  "photo": {
    "cameraMake": "Apple",
    "cameraModel": "iPhone SE (2nd generation)",
    "exposureDenominator": 60,
    "exposureNumerator": 1,
    "focalLength": 3.99,
    "fNumber": 1.8,
    "iso": 125,
    "orientation": 1,
    "takenDateTime": "2021-09-14T10:17:38Z"
  },
  "shared": {
    "scope": "users",
    "owner": {
      "user": {
        "displayName": "user",
        "id": "7de8256bfa42446"
      }
    }
  }
}
0

We can quite clearly see that the size was reported as 6352331
And here, when the file was downloaded
2021/09/16 11:06:57 DEBUG : HTTP REQUEST (req 0xc000473e00)
2021/09/16 11:06:57 DEBUG : GET /y4mMADT1W1ERPFupg839lgG66eqarRtxUxtzRQhKM05UgHQVC-k5-uMitBX0M_XJwk67x1YoUAgmJxb4QFucfZ6Vl1m5Kpeos0KT8h1tFiqGJGm0DKv5jJjd5IuiJ2BTtFr75bGHoeTZzr9RgegmTWC-okZw6907YRvXfP8AaizQlQJHfq4ubbUwjj8pWT5q9muM8TQYeEfVP23dOXBw1yN8J-4E_8JCx0PTiKTT7EgQDn6DMPy9Y04Zqi4VC8NNFNQ7Nvu7YtaT3J_Yzv2cFw9nUhhiY92-39wcsbqAufXkyo/big_swan.heic HTTP/1.1
Host: public.dm.files.1drv.com
User-Agent: rclone/v1.55.1-DEV
Authorization: XXXX
Referer: https://graph.microsoft.com/v1.0/drives/7de8256bfa42446/items/7DE8256BFA42446!4474/content
Accept-Encoding: gzip

The response is
2021/09/16 11:06:58 DEBUG : HTTP RESPONSE (req 0xc000473e00)
2021/09/16 11:06:58 DEBUG : HTTP/2.0 200 OK
Content-Length: 2247095
Accept-Ranges: bytes
Cache-Control: public
Content-Disposition: attachment; filename="big_swan.heic"
Content-Location: https://public.dm.files.1drv.com/y4mpvYnqMS8mVrBGQyut9YVg5U-AH9S-z_Unc5kYv198tTOhpt22gmzNnzK_ThPuz_oSf10M7ZwY3SGvYuYE159X1MRR5FRrVc8W93j2nnFcsD37ufi03POeDjN8y1ytw7Tpr9xi6QhRXNW5JRqO7PwnsGV17xXUzh52isCt8rqJ88NRrhofNm6VICh9jJw0XE3Y31IYzGwShWEPE08AtvTqfkZ6qwkEty9S9tvnJh9Mze38wJK9tCyAiF0XdE0G9Nmas1U24Q9EPw8rPdrGhAQlCpzZnDJ_NiWjcn6zjevypk
Content-Type: image/heic
Ctag: aYzo3REU4MjU2QkZBNDI0NDYhNDQ3NC4yNTg
Date: Wed, 15 Sep 2021 23:06:57 GMT
Etag: aN0RFODI1NkJGQTQyNDQ2ITQ0NzQuNg
Expires: Tue, 14 Dec 2021 23:06:58 GMT
Last-Modified: Mon, 13 Sep 2021 22:18:07 GMT
Ms-Cv: JxPlQ7cDXkKWfy4GiTfIRA.0
P3p: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
Strict-Transport-Security: max-age=31536000; includeSubDomains
Www-Authenticate: Bearer realm="OneDriveAPI", error="invalid_token", error_description="Invalid auth token"
X-Asmversion: UNKNOWN; 19.758.906.2003
X-Cache: CONFIG_NOCACHE
X-Content-Type-Options: nosniff
X-Msedge-Ref: Ref A: 0AAA19AA453B46FCB7ACF9D4064CFDD9 Ref B: AKL30EDGE0215 Ref C: 2021-09-15T23:06:57Z
X-Msnserver: DM5SCH102211612
X-Preauthinfo: rv;poba;
X-Sqldataorigin: S
X-Streamorigin: X

Which quite clearly shows a Content-Length of 2247095 which is not equal to the 6352331 reported in the listing.
Steps to Reproduce
Uploading a live image from iPhone and examining it via the API.
References
This issue looks very similar to https://github.com/OneDrive/onedrive-api-docs/issues/123 - however that issue is marked as resolved, so I've opened a new issue.
This was originally reported in https://github.com/rclone/rclone/issues/5580 by @kelmes who has example images.

The size includes everything associated with the live photo. Unfortunately, downloading the live photo will only download the image.

That is unfortunate. Is that issue being tracked elsewhere? It'd be nice if Live Images could be downloaded in their entirety. Otherwise, once removed from our iPhones, they end up stuck as only viewable in their original context in OneDrive itself. If OneDrive ever removes support for live photos for one reason or another, users would have no ability to ever retrieve that information again.
Google Photos seems to make both the .heic and .mov parts of Live Photos available separately to get around that issue. It'd be nice if something similar could be made available here. I'm happy to open a separate issue on the topic, if it's worth tracking elsewhere.

Thanks for your feedback.
This is a great request and we recommend submitting this feedback here.
https://feedbackportal.microsoft.com/feedback/forum/e3f04660-2c1c-ec11-b6e7-0022481f8472

This issue is still present and this bug should not have been closed. Can you re-open please @k-tsoi

@k-tsoi @chackman What's the status here? I don't see this issue as a feature request, rather a bug from Onedrive that should be fixed with priority. What is happening is that there is no longer a method to retrieve an uploaded file (after live photo is uploaded, no way to retrieve the mov part of the live photo). Legally I don't think this adheres to Onedrive's agreement with the user as a file storage service


Typo in task name

there is a type in the Task Scheduler the task is RecoveryEnviornment, not Recovery Enviornment

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a83e1751-011c-9add-2c0a-63159fef97da
Version Independent ID: 119b5192-3775-f71e-579d-9ead105ed1b0
Content: Recommended configuration for VDI desktops
Content Source: WindowsServerDocs/remote/remote-desktop-services/rds-vdi-recommendations.md
Product: windows-server-threshold
Technology: remote-desktop-services
GitHub Login: @jaimeo
Microsoft Alias: elizapo

There's a typo in your typo :P
RecoveryEnvironment is correct.

Thanks for your dedication to our documentation. If we have been unable to review your issue in a timely manner, we sincerely apologize for the delayed response. We have captured the informtion you have provided for review. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close


Support netcat like behaviour for az network bastion tunnel

Related command
az network bastion tunnel
Extension name (the extension in question)
ssh-extension
Description of issue (in as much detail as possible)
It would be great if the CLI would have an option to behave like netcat, read from stdin and forward to the tunnel, and read from the tunnel and write to stdout.
This would avoid hardcoding a local port which prevents any useful scripting.
It would also allow it to be used as ProxyCommand within ssh to seamlessly ssh into an azure virtual machine.
As it stands, this requires running a persistent tunnel in one window, and then ssh in another window.  And it's impossible to script it in a unix environment.
And thank you for looking into this.

route to CXP team

+1

I am pretty surprised so few voted/asked for this. Anyway, any feedback on the topic, please? @aznetsuppgithub

whoa, more than a year.  sad, so very sad

@0x416e746f6e  you can wrap it with something like:
#!/usr/bin/env bash

SUBSCRIPTION="$1"
BASTION="$2"
RESOURCE_GROUP="$3"
VM_ID="$4"
TUNNEL_PORT="${5:-50022}"

echo "Opening a proxy tunnel to $VM_ID on port $TUNNEL_PORT and exposing it to stdin/stdout for ssh." > /dev/stderr

az network bastion tunnel \
    --subscription "$SUBSCRIPTION" \
    --name "$BASTION" \
    --resource-group "$RESOURCE_GROUP" \
    --target-resource-id "$VM_ID" \
    --resource-port 22 \
    --port "$TUNNEL_PORT" &
tunnel_pid="$!"

function kill_tunnel() {
    kill $tunnel_pid
    echo "Tunnel Killed $?" > /dev/stderr
}
trap kill_tunnel INT

echo "Tunnel pid: $tunnel_pid" > /dev/stderr
sleep 4
socat "TCP:localhost:$TUNNEL_PORT" -
echo "END" > /dev/stderr

Worked after enabling CRB-repo for CentOS.

@mpcusack-sequelae yeah, I thought of a script like that, but it would be cumbersome to port it across different user machines.  e.g. in non-azure cases it's easy to just manage ssh_config files in a git repo that users would just refer in their respective ssh_config local files with Include directive.  but with the shell-script used as a proxy-command, they would have to clone such repo in a very specific location, which is ugly and error-prone.
all the while the irony is that this bastion command is deployed with a python-plugin sitting in ~/.azure/cliextensions/bastion/azext_bastion.  by the looks of it it would take probably half-day to implement stdin/stdout redirecting wrapper around it, which I would be happy to do, but it's not even open-source!  🤷♂

t.w.i.m.c. I ended up with this atrocity in my ssh_config:
Host name-of-the-instance-in-azure
    HostName      name-of-the-instance-in-azure
    ProxyCommand  bash -c "port=$(( 4096 + $RANDOM %% 1024 )) ; pgid=$( ps -o pgid= -p $$ ) ; az network bastion tunnel --resource-group REDACTED --name REDACTED --resource-port %p --port \$port --target-resource-id /subscriptions/REDACTED/resourceGroups/REDACTED/providers/Microsoft.Compute/virtualMachines/%h & trap \"kill -- -\$pgid\" INT QUIT TERM EXIT ; sleep 2 ; nc <IP_ADDRESS> \$port"
    User          azureuser


Revert "*: update etcd to v3.5.0 (#954)"

This reverts commit 01cc765fe667afa22e2bc903e18e811291aeaf55.
Since dashboard is used by all release branches of PD. We cannot upgrade etcd directly. We will find other approach to work around.

/merge