ENH: add MLE for stats.powerlaw.fit

Reference issue

Part of the gh-11782 issue set.
What does this implement/fix?

This PR adds support for finding the shape, location, and scale of scipy.stats.powerlaw.
Additional information

Although initially looking to this book for the MLE equations, Evans, Hastings, and Peacock (2000), Statistical Distributions, 3rd. Ed., John Wiley and Sons, it was discovered that their equations were incorrect. Thank you to @mdhaber and @WarrenWeckesser for taking a look to resolve this issue. See comments https://github.com/scipy/scipy/issues/11782#issuecomment-721518694 and https://github.com/scipy/scipy/issues/11782#issuecomment-723369745.
TODO
Add constraints on use of formula when certain parameters are fixed:

(If the scale is fixed, we require scale >= max(x) - loc. Otherwise there is a data point that is not in the support of the distribution.)

I wanted to have this uploaded sooner, but I fractured my ankle.
Edit: I expect that there will be a test failure for the case where only the shape is fixed. The solutions are similar and the log likelihood result is slightly above our tolerance level.

Thanks @swallan. Could you update your table in gh-11782?

As discussed, there may be issues when trying to fit all parameters free in this distribution when the shape is less than 1.
As currently implemented, the fit method attempts to calculate the parameters with the MLEs according to each assumption. For simplicity of use I've left it returning both sets (in the order of a > 1 and then a < 1.).
For data generated with a > 1, it is relatively simple to solve, as both methods can return a result and then the one with the lower LL can be selected.
data = stats.powerlaw.rvs(size=1000, a=1.5, loc=0, scale=4)

res_1, res_2 = powerlaw.fit(data)

ll(res_1, data) < ll(res_2, data)
Out[123]: True

res_1, res_2
Out[124]:
((1.4592974592570358, -0.007075673190027264, 4.0065759120495095),
 (1.3691449150824875, 0.0090269475050626, 3.99047329135442))

As expected, the computation done under the assumption that a > 1 has a better log-likelihood result.
However, it becomes more complicated when trying to fit data where the a < 1.
When a < 1, there are analytical solutions for the location and scale, but using them to determine the shape leads to a zero division error with the equation for the shape (as Warren describes:

So, to get around this I just tried multiplying the location by (1-np.finfo(float).eps), which provides result very close to that the data was generated with. The issue is that the LL equation doesn't seem to think it's very close.
def get_a(floc, fscale, data):
    return (- len(data) / (np.sum(np.log((data - floc)/fscale))))

def get_s_a_lt_1(data):
    return np.max(data) - np.min(data)

def get_u_a_lt_1(data):
    return np.min(data)

data = stats.powerlaw.rvs(size=1000, a=.5, loc=0, scale=4)

floc = get_u_a_lt_1(data)
fscale = get_s_a_lt_1(data)

fshape = get_a((1-np.finfo(float).eps)*floc, fscale, data)

fshape, floc, fscale
Out[147]: (0.46708033697945495, 5.052342758533454e-05, 3.9980648093076523)

ll((0.46708033697945495, 5.052342758533454e-05, 3.9980648093076523), data)
Out[148]: 72008.7335533255

super(type(stats.powerlaw), stats.powerlaw).fit(data)
Out[149]: (0.8423636251212886, -0.4166395222333465, 4.414754858126874)

ll((0.8423636251212886, -0.4166395222333465, 4.414754858126874), data)
Out[150]: 1461.5459753196828

What should be done in this situation? It seems to me that the LL is behaving in a way that isn't actually productive to getting results. Graphing the LL function using the previously determined location and scale over many shape values seems to indicate that a should be around our solution of ~.46
# graphed from 0 to 2 for extra context
x = np.linspace(0, 2, num=1000)

y = [ll((shape, 5.052342758533454e-05, 3.9980648093076523), data) for shape in x]

plt.plot(x, y)
Out[155]: [<matplotlib.lines.Line2D at 0x7fcb385f6730>]

This ignores the "solution" of a=1, where the log likelihood is quite small:
ll((1.0, 5.052342758533454e-05, 3.9980648093076523), data)
Out[166]: 1385.8104463789498

Because in this scenario we have decided a < 1. If I include x = 1 as a point being evaluated in the plot:

Thanks for implementing this! We'll talk about it on our call tomorrow.

A short summary of what we discussed:

The images above were generated using _reduce_func, which penalizes being close to the boundary of the domain. The raw LL function agrees that the analytical solution has a higher LL than what the generic optimization is giving you.
In some forms the formula for a blows up in the case that a < 1, but we found how to evaluate the limit it approaches as loc -> np.min(data), so we no longer need to use multiply the location by (1-np.finfo(float).eps).

So we start by assuming a < 1 and calculating the parameters using those equations. We check the results, and if they're inconsistent, we use the formulas (including one call to root_scalar for a > 1.

It is correct that the MLEs appear to work when a=1 as well.
Something else of note that makes testing against the super method problematic is that a runtime error comes up when fitting data where a < 1. I think I have run into this before, would it be advisable to suppress this warning during testing?

would it be advisable to suppress this warning during testing

Yes.

It is correct that the MLEs appear to work when a=1 as well.

I think I spoke too soon on this. Using some hypothesis testing, there is a trend of "failures" when a is approximately 1, though the failures are not consistent and seem to be dependent on the data.
In this context I am defining "failure" as

log likelihood result is not less than or equal
log likelihood results are not allclose
fitted values are not close enough, even to the third decimal.

1. assert (ll_mle_analytical <= ll_numerical_opt)
2. np.testing.assert_almost_equal(ll_mle_analytical, ll_numerical_opt)
3. np.testing.assert_almost_equal(mle_analytical, numerical_opt)

I added tests to discover this in the test suite, and discovered that there are issues the scale and locations are fixed.
The issue that occurs is that the log-likelihood result is positive infinity for the overridden fit method, and the log-likelihood result for the super method is not good but is an actual number. Then try actually comparing the results, and they're usually pretty similar except that the a values differ by tenths.

Marking this as draft due to issues determining bracket analytically (line 6349 in PR)

Marking this as draft due to issues determining bracket analytically (line 6349 in PR)

I don't think I was able to come up with an analytical bound for root_scalar here. I think the path forward here is to start with an assumption and repeatedly double the bound until the sign changes (see _solve here).

I think this is a complete algorithm for powerlaw here.

@mdhaber I'm still working on this, it's not completely fine tuned yet to remove any duplicates so no need to explicitly focus on that right now. (There's a little bit of printing I was using for troubleshooting as well).  I wanted to push to ask about what to do the situation that remains that seems to be causing problems, which is fitting when floc is fixed and the shape used to generate the data is circa 1.
The difference between the -LL functions for our fit and the super fit is remarkably consistent:
In [42]:<PHONE_NUMBER>248629 -<PHONE_NUMBER>502187
Out[42]: 0.0002779746442342912

In [43]: -0.240070883662103 - -0.24034885830650335
Out[43]: 0.0002779746444003528

In [44]: -575.8863441321736  - -575.8866221068179
Out[44]: 0.0002779746442911346

I was wondering if you had any ideas about what to do to fix this.
The computation for the parameters is taking place on lines 6451 to 6453, I'm going to add a comment there with the things I was trying.

I was wondering if you had any ideas about what to do to fix this.

What is the problem? Are you finding that this method is not passing a test because our LL is not better than the super fit method when both the super method and this found the optimum (within numerical error)? Then we change the test. You could also try tightening the tolerances (xtol/rtol) on root_scalar.

I think this PR is ready for review. Common fitting scenarios (where the super method also finds a good fit) seem to work well, as seen by the unit tests. I'll make a graph demonstrating the speed improvements like https://github.com/scipy/scipy/pull/15567#issue-1129071572 soon.
@mdhaber, I was wondering if you had thoughts on this aspect:
When doing hypothesis testing, I found one scenario where it behaves poorly. I am not sure if it is an important failure, because the super method returns location and scale parameters that violate the support of the distribution as well.
The data for powerlaw was generated with the following parameters:
a = 10.171472075514137
location = 0.0
scale = 0.4565302370222894
kwds = {'fscale': .1}

with 100 random variates and np.random.default_rng(79) as the seed. Here are the fitting results:
mle_analytical: (nan, 0.35644311294102454, 0.1)
numerical_opt: (2.363647116816284, 0.24463662465317793, 0.1)
ll_mle_analytical: inf
ll_numerical_opt: 6245854.21541399

The fitting procedure is a two step process for powerlaw, first attemping to find a fit for the data under the assumption that the shape is less than one. This results in a shape of about -10, but the support of the shape is a > 0. Then, under the assumption that the shape is greater than one, we analytically determine the location, but the shape cannot be determined because the location and scale violate the support of the distribution.
This is inside the formula for the shape: np.log((data[mask] - loc)/scale), so when (data - loc / scale) results in a negative number, the result is nan. There is currently a mask to get rid of exactly equal data and loc values, but my first idea is to use the mask to remove any values in data that are equal to or greater than loc. This still doesn't provide a "good" fit, since the log likelihood result is in the hundreds of thousands, but the LL from the super method is in the millions.
Code to run:

import hypothesis as h
from hypothesis import given, strategies as st
import numpy.testing as t
import scipy.stats as stats
import numpy as np

def _assert_less_or_close_loglike(dist, data, func, **kwds):
    """
    This utility function checks that the log-likelihood (computed by
    func) of the result computed using dist.fit() is less than or equal
    to the result computed using the generic fit method.  Because of
    normal numerical imprecision, the "equality" check is made using
    `np.allclose` with a relative tolerance of 1e-15.
    """
    mle_analytical = dist.fit(data, **kwds)
    # return mle_analytical
    numerical_opt = super(type(dist), dist).fit(data, **kwds)
    ll_mle_analytical = func(mle_analytical, data)
    ll_numerical_opt = func(numerical_opt, data)
    print(f"mle_analytical: {mle_analytical}")
    print(f"numerical_opt: {numerical_opt}")
    print(f"ll_mle_analytical: {ll_mle_analytical}")
    print(f"ll_numerical_opt: {ll_numerical_opt}")
    print(ll_numerical_opt - ll_mle_analytical)
    assert (ll_mle_analytical <= ll_numerical_opt or
            np.allclose(ll_mle_analytical, ll_numerical_opt, rtol=1e-15))

a = 10.171472075514137
location = 0.0
scale = 0.4565302370222894
kwds = {'fscale': .1}

data = stats.powerlaw.rvs(a=a,loc=location, scale=scale, size=100,
                          random_state=np.random.default_rng(79))
args = [data, (stats.pareto._fitstart(data), )]

func = stats.powerlaw._reduce_func(args, {})[1]
_assert_less_or_close_loglike(stats.powerlaw, data, func, **kwds)

I am not sure if it is an important failure

Doesn't sound like it. Can you plot the LL function in 3D like I did in a few PRs e.g. gh-10908 to see what's going on?

@swallan The problem here is that the user-provided scale is inconsistent with the data. scale must be greater than np.ptp(data). If not, this should return a FitError. Apparently FitDataError exists for this purpose (in _continuous_distns.py) although I don't think it is appropriate to say that the values of data are invalid. Personally, I would take data as a given and say that fscale is inconsistent.

for data from:
from scipy.stats import powerlaw
a, location, scale = 1.5, 2, .5
data = powerlaw.rvs(a=a, loc=location, scale=scale, size=1000)

W00t! And hypothesis is happy?

MacOS test failure:
scipy/linalg/tests/test_decomp.py ........X............................. [ 20%]
........................................................................ [ 20%]
........................................................................ [ 20%]
........................................................................ [ 21%]
Fatal Python error: Segmentation fault

Current thread 0x000000010364ae00 (most recent call first):
``` seems to be unrelated.

Lint indicates too many blank lines, but the line it talks about is non-empty.

raw results from ASV + code

Seems to "succeed" under the assumption that the shape is less than one, but a much better fit can be found when using the equations under the assumption that the shape is greater than one

OK. How bout that when shape is fixed, we try both assumptions and use the one that results in the better fit (NLLF)?

It's interesting that all-free is not the fastest.

When some are fixed, isn't there just a formula? (Not at a computer.) But all free requires an equation to be solved, right?

It's interesting that all-free is not the fastest.

When some are fixed, isn't there just a formula? (Not at a computer, so not looking at code.) But all free requires an equation to be solved, right?

Oops. I meant slowest.
It is interesting that all free is not the slowest, since all free (and the shape being greater than 1) means root finding.

Seems to "succeed" under the assumption that the shape is less than one, but a much better fit can be found when using the equations under the assumption that the shape is greater than one
OK. How bout that when shape is fixed, we try both assumptions and use the one that results in the better fit (NLLF)?
My guess is that when shape is free we can leave the logic as-is.

It is the scale being fixed causing the problem, but I added a branch that uses the NNLF to fix this. (+ a test with the previously failing data).

Great!

Pretty plots for the speed increase when a is less than and greater than 1.

I discovered a small bug in the fixing of shapes, locations, and scales in the overridden fit method benchmark. I fixed it here, but the issue was that the parameters were in the order loc, scale, shapes... but dist.rvs( accepts them with shapes, then loc and scale. So I flipped that.
The other change I made to the overridden fit method bench mark was to test all of the shape combinations from distr_params.

Great. All set then?

Unfortunately, adding an additional shape to distr_params has caused a runtime warning in the PDF unrelated to the fit method:
___________________ test_cont_basic[500-200-powerlaw-arg82] ____________________
[gw1] linux -- Python 3.9.12 /opt/hostedtoolcache/Python/3.9.12/x64/bin/python
../../build-install/lib/python3.9/site-packages/scipy/stats/tests/test_continuous_basic.py:160: in test_cont_basic
    check_pdf_logpdf_at_endpoints(distfn, arg, distname)
        arg        = (0.6591133289905851,)
        distfn     = <scipy.stats._continuous_distns.powerlaw_gen object at 0x7f7ee62ce940>
        distname   = 'powerlaw'
        m          = array(0.39726842)
        n_fit_samples = 200
        rng        = RandomState(MT19937) at 0x7F7ED9BFED40
        rvs        = array([1.20979855e-02, 8.84328767e-02, 6.15219405e-02, 9.07018255e-01,
       1.21153456e-01, 5.02603195e-01, 7.189236...3.20977866e-01, 2.61[579](https://github.com/scipy/scipy/runs/6118067368?check_suite_focus=true#step:12:579)468e-01, 1.10518691e-01,
       5.55035946e-01, 8.57231798e-01, 1.83167268e-01, 8.02285653e-01])
        sn         = 500
        v          = array(0.09004739)
../../build-install/lib/python3.9/site-packages/scipy/stats/tests/test_continuous_basic.py:552: in check_pdf_logpdf_at_endpoints
    pdf = distfn.pdf(vals, *args)
        args       = (0.659113328990[585](https://github.com/scipy/scipy/runs/6118067368?check_suite_focus=true#step:12:585)1,)
        distfn     = <scipy.stats._continuous_distns.powerlaw_gen object at 0x7f7ee62ce940>
        msg        = 'powerlaw'
        points     = array([0, 1])
        vals       = array([0., 1.])
../../build-install/lib/python3.9/site-packages/scipy/stats/_distn_infrastructure.py:2097: in pdf
    place(output, cond, self._pdf(*goodargs) / scale)
        args       = (array(0.6[591](https://github.com/scipy/scipy/runs/6118067368?check_suite_focus=true#step:12:591)1333),)
        cond       = array([ True,  True])
        cond0      = True
        cond1      = array([ True,  True])
        dtyp       = dtype('float64')
        goodargs   = [array([0., 1.]), array([0.65911333, 0.65911333])]
        kwds       = {}
        loc        = array(0)
        output     = array([0., 0.])
        scale      = array([1, 1])
        self       = <scipy.stats._continuous_distns.powerlaw_gen object at 0x7f7ee62ce940>
        x          = array([0., 1.])
../../build-install/lib/python3.9/site-packages/scipy/stats/_continuous_distns.py:6939: in _pdf
    return a*x**(a-1.0)
E   RuntimeWarning: divide by zero encountered in power
        a          = array([0.65911333, 0.65911333])
        self       = <scipy.stats._continuous_distns.powerlaw_gen object at 0x7f7ee62ce940>
        x          = array([0., 1.])
=========================== short test summary info ============================

@mdhaber I think it should be set for review now.
One thing is that since I corrected the bug of improperly ordered parameters in the MLE benchmark, the data generated will be different now. Is that problematic for the previously stored results that used data generated from parameters in the incorrect order?

Is that problematic for the previously stored results that used data generated from parameters in the incorrect order?

IIUC, there may be a jump in the benchmark results, but that can be explained by this change. No problem there is a blip in the benchmark results.

pep8. For some reason I can't get these identified locally still.

python tools/lint_diff.py seems to have trouble locally when the branch diverged a long time ago. Please do fix them even if you can't reproduce the message locally.

Addressed the requested working changes and test-ran the benchmarks to test the case name change.

Did you write down the derivation of the MLE approach somewhere?

Performed this experiment:

import numpy as np
from scipy import stats
from time import perf_counter
import matplotlib.pyplot as plt

rng = np.random.default_rng(2433006235492611347)
qrng = stats.qmc.Halton(5, seed=rng)

N = 1000
times_pr = np.full(N, np.nan)
times_super = np.full(N, np.nan)
nllfs_super = np.full(N, np.nan)
nllfs_pr = np.full(N, np.nan)

for i in range(N):
    print(i)
    sample = qrng.random()
    sample = stats.qmc.scale(sample, [-3, -3, -3, 1, -1], [2, 3, 3, 5, 1])[0]
    s, loc, scale, size = 10**sample[:4]
    loc_sign = np.sign(sample[4])
    loc *= loc_sign
    size = int(size)

    dist = stats.powerlaw(s, loc, scale)
    rvs = dist.rvs(size=size, random_state=rng)

    try:
        tic = perf_counter()
        s_fit, loc_fit, scale_fit = stats.powerlaw.fit(rvs)
        toc = perf_counter()
        nllf_pr = stats.powerlaw.nnlf((s_fit, loc_fit, scale_fit), rvs)
        nllfs_pr[i] = nllf_pr
        times_pr[i] = toc-tic

    except Exception as e:
        print(f"Failure: {e}")

    try:
        tic = perf_counter()
        s_fit, loc_fit, scale_fit = stats.powerlaw.fit(rvs, superfit=True)
        toc = perf_counter()
        nllf_super = stats.powerlaw.nnlf((s_fit, loc_fit, scale_fit), rvs)
        nllfs_super[i] = nllf_super
        times_super[i] = toc-tic

    except Exception as e:
        print(f"Failure: {e}")

nllf_diff = (nllfs_pr - nllfs_super)/np.abs(nllfs_super)
nllf_diff = nllf_diff[np.isfinite(nllf_diff)]
nllf_diff_good = -nllf_diff[nllf_diff < 0]
nllf_diff_bad = nllf_diff[nllf_diff > 0]

bins = np.arange(-3, 1.5, 0.25)
plt.hist(np.log10(times_pr), bins=bins, alpha=0.5)
plt.hist(np.log10(times_super), bins=bins, alpha=0.5)
plt.legend(('PR', 'main'))
plt.xlabel('log10 of `fit` execution time (s)')
plt.ylabel('Frequency')
plt.title('Histogram of log10 of `fit` execution time (s)')

# plt.hist(np.log10(nllf_diff_good))
# plt.xlabel('log10 of NLLF relative improvements')
# plt.ylabel('Frequency')
# plt.title('Histogram of log10 of NLLF relative improvements')

# plt.hist(np.log10(nllf_diff_bad))
# plt.xlabel('log10 of NLLF relative regression')
# plt.ylabel('Frequency')
# plt.title('Histogram of log10 of NLLF relative regression')

There were 18 cases where the LL was worse. I'm investigating them.

Here's what things look like now across all combinations of fixed and free params.

@chrisb83 Just a quick question, I think - it seemed like we were pretty close to your approval here, but the suggestions weren't implemented immediately, so I wouldn't be surprised if you've forgotten everything and would prefer not to re-review.
If you're comfortable with it, I'd be happy merging this myself at this point. On one hand, I was part of the original development (deriving the equations, interpreting them, and figuring out the strategy), and the author was my student. On the other hand, it's been so long since all of that that my recent review is essentially independent of all that. I reviewed quite thoroughly recently, responding to your comments and making changes as needed. And besides the line-by-line review, the latest results above  speak for themselves.
But if you'd prefer, I can also ask for another technical review from a contributor (of gh-16782 and gh-16839), as they are getting quite familiar with the technical aspects of these fit overrides. Maybe if it looks good to them, too, I can merge then.
Just let me know either way.

@chrisb83 Just a quick question, I think - it seemed like we were pretty close to your approval here, but the suggestions weren't implemented immediately, so I wouldn't be surprised if you've forgotten everything and would prefer not to re-review.

I am fine if you go ahead w/o my review. I remember that the main point was to check the solution for shape < 1 and shape > 1 if shape is free and I see that you implemented that. The rest were minor points.
I prepared a note that I shared with you when I did my review. Maybe we can upload it here in case someone wants to look at the derivation in the future (I could need to read through it one more time and prepare a clean version).

I am fine if you go ahead w/o my review

Does that mean you'd be willing to go through and take a last look?

I'm going to go ahead and merge this. Thanks @swallan @chrisb83.

I come across this distribution when looking into some utility functions, and would like to take the chance to write down the derivation of MLE split into a few comments.
Let $x_1, \cdots, x_n$ be $n\ge 1$ independent observations drawn from the power law distribution with shape parameter $a>0$, scale parameter $b>0$, and location parameter $m \in \mathbb{R}$, such that $m < x_{(1)} \le x_{(n)}\le m+b$, where $x_{(k)}$ is the $k$-th order statistic. That is, all observations have positive likelihood. The density function is
$$
f(x; a,b,m)=\frac{a(x-m)^{a-1}}{b^a}
$$
If $a=1$, this reduces to the uniform distribution over $(m,m+b]$, and MLE is straightforward. Now suppose $a\ne 1$. The log likelihood is
$$
\mathcal{l}(a,b,m)=(a-1)\sum \ln(x_i-m) + n\ln a-na\ln b
$$
First consider the case where the location $m$ is known. In this case, $\mathcal{l}$ is decreasing in $b$, so we want $b$ to be as small as possible (if it is to be estimated). The smallest it can get is
$$
\hat{b} =x_{(n)}-m
$$
With $b$ known or estimated, the MLE for $a$ is found by solving $\partial{\mathcal{l}}/\partial{a}=0$ which yields
$$
a^{-1}=\ln b-\frac{1}{n}\sum\ln(x_i-m)
$$
where $\hat{b}$ is substituted for $b$ if it is estimated. $\hat{a}$ is guaranteed to be positive and finite.

Next, consider the case where location $m$ is unknown but shape $a$ is known. If $a=1$, the distribution reduces to uniform over $(m,m+b]$. In this case, if $b$ is unknown, then $\hat{m}=x_{(1)}$, $\hat{b}=x_{(n)}$. If $b$ is known, then $\hat{m}$ can be any value between $[x_{(n)}-b, x_{(1)}]$, and we may as well choose $\hat{m}=x_{(1)}$. Note that the support of the distribution is extended to include the left end in this case.
If $a>1$, the log likelihood function is decreasing in $m$ and also decreasing in $b$, so we want both to be as small as possible subject to the support constraint $m< x_{(1)}\le x_{(n)}\le m+b$. There are three cases from here. If $m$ is known, then $\hat{b}=x_{(n)}-m$. If $b$ is known, then $\hat{m}=x_{(n)}-b$. If both $m$ and $b$ are unknown, substitute $b=x_{(n)}-m$ into the log likelihood function to obtain
$$
\mathcal{l}(m)=(a-1)\sum\ln(x_i-m)+n\ln a-na\ln(x_{(n)}-m)
$$
The first order condition $\mathcal{l}’(m)=0$ can be rearranged into the equation
$$
\frac{n}{a-1}=\sum\frac{x_{(n)}-x_i}{x_i-m}
$$
The right-hand side is increasing in $m$ and easy to bound, so it’s easy to find the solution $\hat{m}$ by e.g. Brent’s method.


Enable user tagging

Describe the feature you'd like
Hi,
Could you please resolve #560  ? I've got a use case where a lot of users use (with success) Bookstack to exchange informations and that would be a HUGE enhencement and a perfect fit.
Regards,
Describe the benefits this would bring to existing BookStack users
User time won't be wasted.
Can the goal of this request already be achieved via other means?
Yes but this is for power user and not intuitive at all.
Have you searched for an existing open/closed issue?

[X] I have searched for existing issues and none cover my fundamental request

How long have you been using BookStack?
Not using yet, just scoping
Additional context
No response

Please don't purposefully open duplicate issues like this, I'm going to close this off since the request is already covered.
I am moving toward adding user mentions, with elements like the notification system recently being added, but there's still some related bits I want to improve first before specifically tackling user mentions of any kind.

+1 I think that this issue could have been solved in 5 years with the time spent responding to the other issues that the need had been declared in #560 :)

https://www.bookstackapp.com/donate/


MM infinite switch chain prompt

Overview
Currently in develop, connecting MM results in infinite switch chain prompts
References and additional details
N/A
Acceptance Criteria

Find the root cause and fix, or revert offending PR

Need By Date
No response
Screenshots/Mockups
https://jam.dev/c/59860845-5b28-4185-a792-56b540a07156
Estimated effort
No response

GitHub you're drunk


Support to_dom_selector for operation selectors

Resolves #134
FYI, I added to_s onto the final path for the updated selector logic, but that might be a breaking change (since previously a non-String object would be converted via to_json, not to_s). There's also the outstanding question of whether we need to issue a warning or exception if something mangled gets through like an inspect string with class name, etc.
Happy to work on updated docs as well, just let me know.

Thank you, Jared! This is awesome. I cannot fathom why we wouldn't accept this concept for inclusion in v5. We do need to rework this so that it happens inside of dom_id, and not adjacent to it.
dom_id actually gets called both in StimulusReflex (the library) and by users of StimulusReflex creating their own Reflexes. I think it's exciting to consider that people could pass objects to morph that can generate their own DOM selectors.
The other consideration is that the original intention of dom_id was to prepend a # character to the selector, so that it would magically with the ActionView equivalent. If someone's component exposes a to_dom_selector method, someone accessing it with dom_id can take for granted that it will handle the # for them.
I'd like to propose that we add a method to identifiable.rb:
def identifiable?(obj)
  obj.is_a?(ActiveRecord::Relation) || obj.is_a?(ActiveRecord::Base) || obj.respond_to?(:to_dom_selector)
end

This would allow us to change line 35 of operation_builder.rb:
options["selector"] = identifiable?(previous_selector) ? dom_id(previous_selector) : previous_selector

And that would allow us to add the to_dom_selector check to dom_id directly, enabling it across the whole framework:
    def dom_id(record, prefix = nil)
      prefix = prefix.to_s.strip if prefix

      id = if record.respond_to?(:to_dom_selector)
        record.to_dom_selector
      elsif record.is_a?(ActiveRecord::Relation)
        [prefix, record.model_name.plural].compact.join("_")
      elsif record.is_a?(ActiveRecord::Base)
        ActionView::RecordIdentifier.dom_id(record, prefix)
      else
        [prefix, record.to_s.strip].compact.join("_")
      end

      "##{id}".squeeze("#").strip
    end

This also solves the to_s concern with the fall-through case, I think!
What do you think?

I should be fine with the docs (dom_id is already written) but of course I'll pass the section by you for suggestions and improvements.

Makes sense to consolidate inside of dom_id…however I'd advocate for just:
return record.to_dom_selector
in the case of that being available. I don't think assuming it must be an actual id is warranted. Could be any selector. rando-tag[activated] or body > header > nav. That's just my preference anyway.

stares at Jared's comment for 10-12 minutes before replying
I feel like I get where you're coming from, and I thought about ways we could pull this off. For example, what if to_dom_selector methods could return a string or a {selector: "a > b[any-old-thing"}? If it returns a string, it would get a # prepended, and if it came back as a hash, we could pass back the raw selector value from the object. It's a bit of a hack, but it's one I could swallow.
However, the function is dom_id, and the folks calling it expect a fully formed #id to emerge. They are, after all, already able to pass any selector string they like.
That means that instead of making to_dom_selector a special case, instead the developer could just pass my_obj.to_dom_selector as a selector value. I don't explicitly call to_partial_path on my model instances often, but the fact that I can is a cool trick that I have deployed >0 times in my career.
I'm open to exploring either of these paths, or a third!

Would love additional input on this…I guess my dilemma is I've personally used the original Rails' dom_id in an app approximately (checks notes) 0 times, so I have no muscle memory on how that works or what is expected.  I consider using id in the DOM somewhat of an anti-pattern because there's no guarantee of uniqueness whatsoever. Dev A could write something with id="widget" and somebody else could come along and write something with id="widget" and the two have absolutely no commonality whatsoever yet they end up on the same page together and then who knows what will break. Rails' dom_id sort of makes sense as you probably get uniqueness because of the combined AR class name + record id. But if we run with this generic to_dom_selector idea and somebody just returns foo and who knows how many #foo elements are actually on a page, that seems counterproductive.
Anyway, if we do keep using IDs only, I'd recommend changing the method name to to_dom_id. Having string or hash return values with different behavior seems confusing to me. 🤷♂

We're going to have to agree to disagree on a number of points, because we're not looking to move away from using id at this time. CR is a sharp knife and we have four years of experience gathering problem reports; happy to say that non-unique IDs are not something folks are complaining about. We think that id is a standard, expected and powerful tool. CR also has a very different API mouthfeel than anything Turbo Frames is doing.
Similarly, we're not looking to make a break from dom_id. It's short, it's in wide use in production SR projects, and it isn't constrained to folks who are thinking component-first. A small but relevant point is that we use it and like it, or it wouldn't be there. 😉
Good news, though: I have a third path to propose and I think that you'll like it. (I just realized that you probably meant changing the name of to_dom_selector to to_dom_id... not changing dom_id to to_dom_id. 🤦 So forgive me if I'm regurgitating the obvious.)
Why not support both? It's fair to say that we want this to work for as many use cases as possible:
    def dom_id(record, prefix = nil)
      prefix = prefix.to_s.strip if prefix

      id = if record.respond_to?(:to_dom_selector)
        return record.to_dom_selector
      elsif record.respond_to?(:to_dom_id)
        record.to_dom_id
      elsif record.is_a?(ActiveRecord::Relation)
        [prefix, record.model_name.plural].compact.join("_")
      elsif record.is_a?(ActiveRecord::Base)
        ActionView::RecordIdentifier.dom_id(record, prefix)
      else
        [prefix, record.to_s.strip].compact.join("_")
      end

      "##{id}".squeeze("#").strip
    end

I can sell that! What do you think?

Seems perfectly reasonable to me! I'll redo the PR and see what other feedback we can collect.￼

Thanks, Jared... this looks great! I am trying to figure out why the test suite is giving us a void value expression error, though. That's a new one for me.
Do you see anything weird? Did you run standardrb?

Do you see anything weird? Did you run standardrb?

Strange… All seems a-OK on my side

@leastbad Wow, so apparently earlier versions of Ruby wouldn't allow you to return directly within an if statement. That got fixed apparently in 2.7. There's possibly an indication of a backport to 2.6 but as we see here on GH that isn't guaranteed.
https://bugs.ruby-lang.org/issues/11143
Anyway we might want to redo that dom_id logic a bit.

Nice catch! Also: I hate this kind of thing. Also: I can't believe that I've never hit this case before.
Even though I wish we could bump to >=2.7, I don't think that this is a strong enough justification.
In your opinion, what's the least fugly way to set a flag around line 9 and then exit with the value around line 19 if the flag is true?

I switched it out to a guard statement up top. Let me know what you think :)

Duh, of course!
I think it's great. Let me make sure the others agree.

Sorry for being late to the party!
This looks awesome and I love where this is heading!
I have one thought to add: if an object defines both to_dom_selector and to_dom_id shouldn't we then use to_dom_id? From what I can tell we are currently using to_dom_selector in that case.

You're right that in the current implementation, dom selector is plucked first. The fuzzy reasoning behind this was to handle the odd case first, and then treat every other branch in the if as an id.
What are the pros and cons of swapping the priority from selector to id?


Set auto-repeat as soon as the first track in a new station starts to play.

Also:

update test scripts and README
check results of track rating events.

@rectalogic, will you take a quick look at this PR before merging please? Sorry for the large number of commits - I could not manage to squash them for some reason.


Parse Markdown for frontend rendering

Seems Contento theme doesn't do markdown parsing. Or did I miss something?
If Contento uses Markdown editor by default, adding backend markdown parsing or js markdown parsing should be better. :-)
Don't know if Liquid has Markdown support. Or can we use Bootstrap Markdown to only preview post.content.
https://github.com/markedjs/marked

Not sure what the plan is for theming with markdown but you can add extra_filter_modules to the Liquid config:
# Configure Liquid
config :liquid,
  file_system: {Contento.Themer.FileSystem, "priv/themes"},
  extra_filter_modules: [ContentoWeb.ThemeHelpers]

then you can use the helper in liquid templates:
{{ page.content | md_as_html }}

Thanks a lot! Works fine!


.gitignore: exclude *.out only in repo root

The .gitignore rule *.out was sweeping up vendor/github.com/go-errors/errors/cover.out, which made vendoring inconsistent, which was caught by downstream builds (see linked issue).
We should be able to live with only excluding *.out (specifically coverage.out, from make coverage) from the repo root, so do that.
fixes https://github.com/open-cluster-management/backlog/issues/15331

/lgtm

/test e2e

/test e2e-pool

/test e2e


guest to guest chat eDiscovery

Today, guest to guest chat eDiscovery is not supported and the documentation of the feature is not correct.

@nenonix @JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@LolaJacobsen PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


JabberAgent locks up, sends no events, backs up the job queue.

(quoted from chat on gitter.im)
I'm running three DJ workers now, and got another lockup this morning - 3,301 pending jobs. Again, JabberAgent is at the top of the list.  bin/schedule.rb is still in memory and using 1.3% of the CPU it's sitting on.  If I just dump all the pending jobs, that doesn't fix it, I have to kill and restart Huginn to get it going again.
One unusual factoid associated with that: When I bounce Huginn, the number of jobs in the "to run" queue roughly doubles. The workflow is, stop Huginn, start it again, then dump all the pending jobs.
I disabled roughly 110 agents in the past two weeks to see if it was the number of running agents doing it; this does not appear to be the case.
I have five development.log files, one from each of the last four lockups. I've no idea what I'm looking for in there, though.
Now that Huginn has picked up again, the number of jobs is increasing at a rate of approximately 75 jobs every five seconds (if I'm interpreting the output correctly).
Up to 4,583 now and climbing.
Also, and I don't know if this means anything, the background process that's supposed to remove expired events from the database isn't. My database is over 60,000 stored events and climbing every day.  I wish I knew what was going on. If these lockups were consistent... whoa! Huginn just flatlined!  Hang on, let me paste the exception someplace.
http://pastebin.com/Qr9mYhQ3
The Rails server is running, bin/schedule.rb is running (ostensibly), bin/twitter_stream.rb is running, and script/delayed_job 0 and 1 (out of 0, 1, and 2) are running. script/delayed_job #2 is gone.  I've just saved development.log.5.  Trying to load Huginn shows a blank white page.
Restarting Huginn.
22:12:08 web.1 | started with pid 31207
22:12:08 schedule.1 | started with pid 31209
22:12:08 twitter.1 | started with pid 31211
22:12:08 dj0.1 | started with pid 31213
22:12:08 dj1.1 | started with pid 31217
22:12:08 dj2.1 | started with pid 31218

Looks like JabberAgent is hung up again - the icon's blue and it's stuck at the top of /jobs.  There is no red "Delete" button next to it, so I can't terminate it.
Yup, definitely. 28 minutes now.  (Well, it says 28 minutes.)
Wait, what's "Connect to receiver"?

Looks like this a know issue: https://github.com/xmpp4r/xmpp4r/issues/46 https://github.com/xmpp4r/xmpp4r/issues/28
Not sure if one can rescue a deadlock, but you can try to replace with JabberAgent deliver method with this and then check the agent log how often the exception happens:
    def deliver(text)
      client.send Jabber::Message::new(interpolated['jabber_receiver'], text).set_type(:chat)
    rescue Exception => e
      error("#{e.class} #{e.message}")
    end

Hmmmm.... I'll give this a try this afternoon and let you know what happens.

Done.  Monitoring now.

The Huginn lockup I ran into early this morning seems to have dumped a few exceptions into the development.log file.  It'll take me a little while to clean up the text so I can throw it on Pastebin.

http://pastebin.com/6j9MLiVD

I don't know if is related but with the amount of agents you are running I strongly suggest to migrate your setup to run in the production environment. I will definitely be faster and maybe even more stable. When you are running debian or ubuntu the manual installation guide should be straight forward. Copy over your .env file when you are done and huginn will run at least as stable as before.

last_error = 'closed stream Sounds like an issue with your XMPP server as well as the exception backtrace, did something change about the way you authenticate against the server?

No, I haven't reconfigured any instances of the JabberAgent, nor the copy of Prosody running on the same server.  The JabberAgents seem to run decently well most of the time, I get alerts from them from time to time during the day.
Could it be that I have multiple copies of JabberAgent all logging into the Prosody account, sending their messages, and then logging out?  Should I create multiple "service accounts" in prosody for each scenario?  Should I hook the agent networks that use JabberAgent as their means of output into a single JabberAgent for the scenario, and not a bunch of them?

I will give it another try, @dsander, and see if I can make it work this time.

Could it be that I have multiple copies of JabberAgent all logging into the Prosody account, sending their messages, and then logging out?

That sounds like a very good point, when running multiple DJ workers your JabberAgent could be triggered multiple times and run in parallel which might cause the authentication problems.

Especially when taking into account that xmpp4r is multithreaded internally; I've had similar problems with other codebases.
It could also be a particular agent network of mine triggering this problem; I don't know what may or may not have changed in CronAgent, but I need to rework it so that it doesn't trigger as fast as it is.

@dsander To document the discussion in Gitter yesterday, I sat down with some of my co-workers yesterday morning and tried to migrate my environment over to the Production configuration.  The instructions failed at the sudo rake production:export step, we think because runit's scripts weren't finding the RVM environment.  We were able to get something very much like it running which operates noticeably faster.  However, this did not fix the issue with JabberAgent locking up DJ's or threads; just the opposite, in fact.  Huginn is now significantly less stable than it was before.
The patch you suggested for JabberAgent is not reporting any exceptions.  DJ's running Jabber agent do not print anything other than "Agent#186: Sending IM to  with event 900689".
One further thing - when a DJ runs a JabberAgent it stops functioning entirely until it's killed and restarted manually.  Watching the /jobs page after freshly starting up Huginn, one can see three jobs at a time run (and very quickly, at that).  Then a JabberAgent runs, and after that only two jobs at a time run.  Then one.  Then no more until they're killed and started again.  It's a pretty predictable downward slope that occurs inside of eight hours.  The Rails server, scheduler, and Twitter subprocesses do not seem to be affected.

One further point - I've created unique accounts in Prosody for the JabberAgents in each scenario on a 1:1 basis.  I think this rules out all of the JabberAgents all hitting a single XMPP account to send their events in short succession.

I've made one more configuration change - I enabled the legacyauth module in Prosody because it could be that xmpp4r either doesn't do SASL efficiently or at all, which might be part of the problem.  I've turned logging in Prosody all the way up so we can see the server-side of the agent interactions, also.

I have a better idea of what's going on right now.  I just correlated the frozen DJ's output with Prosody's debug logs, and what I see is JabberAgent negotiating a TLS session and authenticating successfully with SASL, logging into the XMPP server (the session is still online), but then the JabberAgent doesn't send its payload.  It does, however, send keepalives to keep the session active:
ov 21 19:54:07 c2sf8d040       debug   Received[c2s]: <iq id='EW9Uc-5292' type='get'>
Nov 21 19:55:09 c2sf8d040       debug   Received[c2s]: <iq id='EW9Uc-5293' type='get'>
Nov 21 19:56:21 c2sf8d040       debug   Received[c2s]: <iq id='EW9Uc-5294' type='get'>

Strangely, I just heard some XMPP messages hit my client; the output of the next DJ over shows the same instance of JabberAgent (agent 62) successfully send some events, and then the DJ ran a few more jobs immediately afterward.  This is weird.

The instructions failed at the sudo rake production:export step, we think because runit's scripts weren't finding the RVM environment.

The documentation recommends not to use rvm or rbenv for a reason, it is nearly impossible to cover every possible configuration. I do not know what you did, I was able to migrate my rvm/capistrano setup to the manual installation guide by backing up the .env file, installing ruby globally (2.2.3 is not needed, any supported version never then 2.0 will work) and following the steps of the guide.

just the opposite, in fact. Huginn is now significantly less stable than it was before.

I really doubt that is related to huginn running in the production environment. For example if you would be using runit the dying DJ workers would be restarted automatically. That does not solve the problem with the JabberAgent, but still would keep your huginn instance functioning.
I can not say much about the JabberAgent failing to communicate properly with your XMPP server because I just use it to receive events from a hipchat server. Maybe the issue needs to be upstreamed to xmpp4r?

That makes sense.
The DJ workers are not dying, they are locking up and have to be killed manually and restarted.  If they were just dying I could write a watchdog for them.  I think the DJs are just locking up faster in production mode than in development mode.
xmpp4r has not been maintained for nearly a year (commit 57e0488e199befde2244e637d39bbacf9d2fe26f was on 4 December 2014).

Are you getting deadlocks in the DB?

Not that I can tell - I was grepping for them in the logs but didn't see any.

So when you say they're locking up, the symptom you're seeing is that no jobs are getting handled, but the processes are still running?  What do the logs say during that time?

Yes to the first question, nothing to the second - the logs just stop updating.

Noticed the following this morning after restarting the threaded job runner: http://pastebin.com/ZGTgvFxa

I wonder how @client can become nil over the runtime, but the fix should be easy, you are using the connect_to_receiver option, right?

I am not using that option.  I am using JabberAgent to emit events over XMPP; there is nothing downstream of them in Huginn.
Should I be?  Are there semantics to the connect_to_receiver option that I'm not understanding (which is likely)?

When one of your jabber agents has the connect_to_receiver option set to true it will connect to the jabber server and emit an event for every XMPP message it receives. Without that I do not see how the code path your execution comes from is executed.

I have configured one of my JabberAgents to have that setting turned on as a test. We shall see what happens.
Why is this an optional setting now? I am not sure I follow the possible use cases.

Actually I just wanted to make sure you don't have the option enabled, because I don't think you need it. It is useful if you want to use the XMPP MUC room as a source of events, it does not send any message back.

In one of my scenarios, I have JabberAgent turned on and connect_to_receiver is set to true.  It is deactivated in my other scenarios.  In the other scenarios it is set to false.

Is this still happening @virtadpt?

I'll be honest with you, I don't know.  I deactivated all of my JabberAgents and set up BeeperAgents and that's been working for me.  I'll give it another try after I dig out from everything that snowed me in while I was on the road last week (probably this weekend) and let you know what I find.

We may as well close out this ticket.  Since I stopped using JabberAgent the problems stopped (I run into different kinds of problems these days) and nobody else has mentioned running into them.

:jhg kjfh

From: The Doctor<EMAIL_ADDRESS>Sent: Wednesday, April 19, 2017 11:25 PM
To: cantino/huginn
Cc: Subscribed
Subject: Re: [cantino/huginn] JabberAgent locks up, sends no events, backs up the job queue. (#1149)
We may as well close out this ticket. Since I stopped using JabberAgent the problems stopped (I run into different kinds of problems these days) and nobody else has mentioned running into them.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/cantino/huginn/issues/1149#issuecomment-295366607, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADe_tecS-_NB2Nmf7osbR7CLQ7324DOlks5rxkp8gaJpZM4Gicj6.

:shrug: okay, works for me. I also don't use it.


Fails on repeating identical deployment - stemcell already exists, not in gzip format

This screenshot is from our pipeline to redeploy concourse. There are no tasks in the pipeline, only resource gets and puts. We have a job to collect versions, so we can be certain that this deployment was using identical resources to the deployment that succeeded less than a minute before-hand. It continues to fail a day later with identical inputs.

Are you using a github release resource with an access token configured, by any chance?

we are

I'm betting it's because of this, then; just ran into a similar issue myself:
https://github.com/google/go-github/issues/246

interesting. hard to explain why it succeeded with the first deploy then failed with what should have been a no-op deploy about one minute later though.

Yep. To confirm you could hijack in and cat the release tarballs.
I'm starting to suspect this was a change rolled out by AWS or some other external service (GitHub?). Or maybe you didn't have an access token configured before?

I'm fairly certain the configuration was identical between those two builds - you can't see it in the screenshot but they occurred within a minute of each other.
And yes, your hunch is correct - the tgz files in the resource directory actually contain error xml

Error>InvalidArgument<Message>Only one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specified</Message><ArgumentName>Authorization</ArgumentName><ArgumentValue>Bearer...

This should be fixed now.

We ran into this same error from the https://github.com/concourse/github-release-resource/ when the repo that had the releases on it was moved to another organization. It seems related but we weren't able to track down the bug if it still exists in the go-github package. We just wanted to mention it somewhere in case someone else stumbles on the same problem. The fix for us was just updating the resource in our pipeline to the new organization name so it didn't hit the redirect anymore.


add some plugins

https://grafana.com/grafana/plugins/grafana-piechart-panel
https://github.com/NatelEnergy/grafana-discrete-panel

the plugins are incompatible with the verson of grafana in use


Configure from Swagger spec

It would be nice if it was possible to configure RestAngular from a Swagger spec. It has most of information you need for configuration.

+1

I'm not familiar with Swagger at all, but if you want to submit a PR with it, sure, go right ahead!


Handle disconnections.

Is there a graph.on('disconnect') event that can be handled if the client loses connection with RedisGraph for any reason?
Also, is there an option to pool connections?

you can either provide a client to the constructor (https://redisgraph.github.io/redisgraph.js/Graph.html) or use graph_client.on('error').
Redis is "single-threaded" (it's not actually single-threaded, but only one thread is handling the commands), so using a connection pool with Redis will not improve bandwidth/performance in most cases.

On the response to the second point, I have the following.

Connection Pools for Serverless Functions and Backend Services which has a Python application using aio-redis for connection pooling.
While the article above also mentions the following, there is an npm package node-redis-connection-pool too.

In the case of JavaScript, the client doesn’t offer a connection pool because of the single-threaded nature of Node.js.

So, I am not sure, what to conclude w.r.t. connection pooling with RedisGraph.


Add some helper convention mappings

In addition to ConventionMappingSettings.Default, add the following:

ConventionMappingSettings.UnderscoredLowerCasedColumns

This will help Postgres users, we could also add

ConventionMappingSettings.UnderscoredPascalCasedColumns

This is a great idea. :+1:

Not only this would be useful for PostgreSQL Server, but also for other DBMSes which are case sensitive such as Oracle and DB2.


2019-02-18 1800 UTC Lineup planning

Hi all!
edit: Schedule so far:

Title
IssueNumber
Order
By

When Runspaces need to have a serious discussion
#130
01
@FriedrichWeinmann

Do The -Split!
#127
02
@IISResetMe

Host mkdocs in an Azure Storage Blob
#122
03
@tomlarse

One approach to testing self-contained scripts with Pester
#114
04
@nohwnd

What's New in the World of Toast?
#113
05
@Windos

We're looking for up to 8 folks for a February 18 @ 1800 UTC PSPowerHour.  We'll be trying a mini podcast / discussion after the demos, for anyone interested in sticking around after their demos (per #124).
ccing folks with open issues:  @mattcushing, @devblackops, @MSAdministrator, @steviecoaster, @vexx32, @nohwnd, @tomlarse, @iisresetme
FYI, we'll also be doing a 2019-02-25 session the next Monday, @ 1800 PST

I won't be able to do this one, but I'm definitely up for doing the following one at 1800 PST. Think I'd love to do #120 at that point. 😄
I should be up for hanging out watching this one, though 👉 👉

I am in with #114

I'd be happy to do #130

18:00 UTC sounds excellent, can do #127!

@RamblingCookieMonster, I can make either date work and can do #113
Feel free to use me to fill numbers if needed.

@RamblingCookieMonster I'm afraid I won't be able to present tonight, as I've contracted a cold 🤧

@IISResetMe that sucks, I was looking forward to your presentation. Get well soon!

Damn @IISResetMe , can only agree with @nohwnd on the sentiment.
Get well and remember to file a bug report - you getting a cold just in time for this definitely is one (in more than one sense ;) )


[NotEmptyCollection] to validate empty list/array

probably by design but would be nice ™

I suggest using a property like [ArraySize] but with no editor behavior. That will be useful in more scenarios
It should be defined like
    [AttributeUsage(AttributeTargets.Field)]
    public class CollectionSize : Attribute
    {
        public int Min = 0;
        public int Max = int.MaxValue;
    }

And used like
    [CollectionSize(Min = 1)] // Equivalent to [NotEmptyCollection]
    public List<Transform> endOfBarrels;

    [CollectionSize(Min = 1, Max = 10)] // Size must be between 1 and 10 inclusive
    public List<Transform> endOfBarrels;

    [CollectionSize(Max = 10)] // Can be empty but size has to be less than or equal 10
    public List<Transform> endOfBarrels;

@AliAlbarrak This is a much better idea.
I think I'll put it into existing API, either ListDrawerSettings or ArraySize

ArraySize with a min max would be nice

ArraySize already exists, it hard sets the array size

It can be like this:

ArraySize(10) is the same as the old one, set the fixed size to 10
ArraySize(5, 10) means min=5, and max=10

Then, ArraySize(min: 1) will do the trick: at least 1 element is expected.
This at least can keep the compatibility with the old API

Hi,
this feature has been added to 3.3.8

ArraySize allows to set range
If you have ListDrawerSettings enabled, the Add and Remove buttons will be disabled/enabled accordingly if you also have ArraySize

For example:

[ArraySize(3)] will make the array size fixed to 3
[ArraySize(1, 5)] will make the array size range to 1-5 (both included)
[ArraySize(min: 1)] will make the array size at least 1 (no max value specific). Useful to require a non-empty array.

[ArraySize(1, 3), ListDrawerSettings] public int[] oneToThreeWithSettings;

That's so nice! Thanks!


Feature/packages page

Description
https://jira.mesosphere.com/browse/DCOS_SITE-180
Urgency

[ ] Blocker
[ ] High
[ ] Medium

Requirements

Test all commands and procedures.
Build content locally and test for formatting/links.
Add redirects to dcos-website/redirect-files.
Change all affected versions (e.g. 1.7, 1.8, and 1.9).
See the contribution guidelines.

Jenkins, Retest this please

[ ] Swap the text and new packages section
[ ] When description text is too long it should truncate and link to the package browser URL for the package (now it just pushes everything else off the card).
[ ] There should be room for at lease one sentence of description (perhaps make font smaller)
[ ] Buttons should always be at the same height
[ ] status tag should not be on the same line or wrap with the title
[ ] Long titles take up almost the whole card

I pushed some of the text info to the branch so please pull it down if you're not sure what I'm trying to describe.

Hi Judith,
The issues are fixed. I've moved the status tag to the top right, otherwise it would conflict with the titles. In my opinion it looks much nicer now.

Looks great! Last two tweaks:

[ ] Can the cta section at the bottom be white instead of gray so it doesn't blend into the new packages cards?
[ ] Could the card carousel start out filling the page instead of with the first card in the middle?

Hi Judith,
I've changed the background color of the cta background. Regarding the card carousel, I think this would be confusing for users :)

@renesijnke what about it do you think would be confusing?

@renesijnke never mind, after looking at it a bit I think I can see what you mean! Thanks!


How to setState in AsyncStorage block with react native?

I want setState in AsyncStorage block but there is a error: undefined is not an object(evaluating 'this.setState').
http://stackoverflow.com/questions/41114460/how-to-setstate-in-asyncstorage-block-with-react-native

please show us your code

I guess you did not use bind, if you invoke a function first you should use bind, in ES6 this does not ref to class instead ref to function, There are two solution to solve this issue.
onPress={()=>{this.onClick();}}
onPress={this.onClick.bind(this)}

I guess you did not use bind, if you invoke a function first you should use bind, in ES6 this does not ref to class instead ref to function, There are two solution to solve this issue.
onPress={()=>{this.onClick();}}
onPress={this.onClick.bind(this)}

It has been solved.http://stackoverflow.com/a/41114902/4256995
But thanks all the same.


[TO ADD] Enhancement: Input recorder

A button that plays the song on the debug menu, but it records the inputs made until you make it stop (maybe put a keybind for it) and puts it on the chart, specifically for making it easier to sync up charts

oh thats neat. sure

Only problem might be the long notes

Done in the latest commit


logging: Use structured logging and simplest log calls

Use more structured logging and don't use the printf-style log calls
unless actually needed.
Signed-off-by: James O. D. Hunt<EMAIL_ADDRESS>

LGTM

LGTM


webkitgtk: Fix webkitgtk for macOS

Motivation for this change
The latest webkitgtk in nixpkgs does not currently build on macOS.  This fixes thew build and webkitgtk seems to work correctly on macOS.
I have included PR https://github.com/NixOS/nixpkgs/pull/33304 (fixes geoclue2 on macOS) in this PR as it is also needed for this to work (since webkitgtk uses geoclue2).
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

Are there any updates on this pull request, please?

Should be resolved by https://github.com/NixOS/nixpkgs/pull/123298 for now.


Add a thing that tells you where the Ansel presets are installed in

i need to find where the ansel presets are but i look in the nvidia corporation folder because everyone says its there but it aint

C:\Program Files\NVIDIA Corporation\Ansel
C:\Program Files\NVIDIA Corporation\Ansel\Custom


UX: TDT ID confusion risk in Mint / Redeem cycle

In current format there is high potential for confusion for users surrounding TDT ID. Consider current behavior:

the minting is completed, user is presented with TDT ID, i.e. : 0x3cAf685693B3e029e53D9C0e5EfE35691Cb5eE47

user holds tBTC until desired redemption time, returns to dAPP clicks redeem bond

Step 1 prompt for TBT ID and BTC address.... this is where user confusion could occur.

Imagine weeks or months have passed since minting, now the user wonders: "TBT ID, hmm, what is that? Is this my ETH wallet address? Wait, was there some number I was supposed to write down and remember it the whole time? Where do I look it up? How do I find that again if I don't remember?"
Suggested improvements:

popup / tooltip / link at Step 1 of Redeem explaining how to find this again if user doesn't remember.

description / link at end of Step 6 of minting, clarifying what the TBT ID is.

Again, think of users who don't really know the technical side. Will you have to remember this number? Is it tied to your ETH address? Can you find it in a contract? is it embedded in the tokens I have? Would it be a good idea to copy it down somewhere, like you might with a password or seed phrase?
I know we want a neat clean interface and don't want to overwhelm with info, but IMHO there has to be at least a link or something available for users who need more clarity on what this address is and what role it plays in the process.

This has less to do with wanting a clean interface and more to do with the dApp being an early stage alpha. We absolutely want to give users better tools for managing their deposits, and phasing the TDT id into the background instead of making it the primary user approach to referencing a deposit is a core part of that. You can see some early work in this department in #222.
Thanks for the detailed thoughts!


Add documentation for experimental lineage schema

We needs docs for the Clone, Tree and Node schemas. I think skeletal docs like we have for Alignment should be fine at this stage:
https://docs.airr-community.org/en/latest/datarep/alignments.html

@javh should this be part of v1.3?

By that I mean it isn't tagged with v1.3 but is a Milestone in AIRR v1.3

Yeah, it should. I just quit using the tags because they were redundant with the milestones.

I just tagged it, as I worry about overlooking things as I am not sure we all look at the Milestone as the mechanism for tracking these things...

Cool. We should probably settle on one mechanism to avoid confusion, at some point. Maybe for the next release?


composite into a 4 vertices polygon

What are you trying to achieve?
Embed small photo1 into large photo2 which has a frame at an angle

Have you searched for similar feature requests?
yes
What would you expect the API to look like?
composite() could take a 4 vertices polygon which represents the frame interior area.
Note that opposite borders don't have to be parallel.
What alternatives have you considered?
don't know other alternatives
Is there a sample image that helps explain?

Any suggestions on how to accomplish this?
Thank you!

3 point affine transformation, yes, please see https://sharp.pixelplumbing.com/api-operation#affine
4 point projection, no. It might be possible to achieve this with some of the new features of SVG, but you'll have to wait for #2884

Thanks for clarification! Now that I know what to search for, I see this already requested in #2388 and #2095
Closing and following those.


Adjust link color and the decorations

⬇

How do you think?

Good looking !

Thank you good looking guy!


False redirect detection in dash parser

Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
2.5.5
Can you reproduce the issue with our latest release version?
Yes
Can you reproduce the issue with the latest code from master?
Yes
Are you using the demo app or your own custom app?
Custom
If custom app, can you reproduce the issue using our demo app?
Yes
What browser and OS are you using?
Chrome/MacOS Catalina
For embedded devices (smart TVs, etc.), what model and firmware version are you using?
What are the manifest and license server URIs?
N/A
What did you do?
Play content with an asset URI supplied by our content auth backend.
What did you expect to happen?
Primary manifest request URL does not change if it is not an actual redirect. The networking engine observes the proper configured number of retries for a manifest request.
What actually happened?
This code in dash_parser does not account for case insensitivity of the hostname in the manifest URL, and can cause false redirect detection (since the host in response.uri will be lowercase):
// For redirections add the response uri to the first entry in the
// Manifest Uris array.
if (response.uri && !this.manifestUris_.includes(response.uri)) {
   this.manifestUris_.unshift(response.uri);
}

Therefore, a request for [Manifest.com/manifest.mpd] becomes [manifest.com/manifest.mpd, Manifest.com/manifest.mpd] on subsequent updates.
This does not necessarily have consequences for playback but may result in extra unexpected networking retries as the networking engine is now given an array of multiple urls, and furthermore, application layer networking filters or proxies that depend on url detection may behave unexpectedly or fail.

Hello!
We faced the same issue. We are trying to play a stream that includes tokens that can change during the playback. In request preprocessors we inject this token into the manifest URL, so each time the manifest URL has changed the new URL will be included to the manifestUris_ array. This behavior will not affect the playback, but the retry mechanism will be not working correctly.
So if the request for the manifest will be failed, the manifest with the previous token will be requested during the retry flow. The token can already be not valid, so the retry mechanism will not work correctly.
This will also fix the behavior when manifest with the original URI will be requested in the retry flow, when the manifest request would be failed.
Using Charles the  302 response was emulated on the demo page for dynamic manifest and then error for the manifest URL. Current approach: the original URI is requested in the retry flow. It is not correct because it would definitely cause redirect one more time:

Expected approach: redirected URL is used in the retry flow.

PR that fixes the problem: https://github.com/google/shaka-player/pull/2763
Best,
Tatsiana


Added describe command to readme

First of all thanks for a great tool I just discovered this morning!
While putting it in place in one of my projects, I was wondering how to understand what the rule will do without diving into the code. I found it after some time but it would have been easier had it been in the readme.
This pull request will make it easier for newcomers to find it.
Note: I didn't regenerate the readme.rst because there are lots of other changes besides mine.

sorry I made a mess with the readme, it's my first time here and still learning.

@julienfalque Thanks for the feedback, I made all requested changes.

@8ctopus can you help me out and rebase and squash your PR? seems the merge tooling has an issue that stops it from squashing automatically

@SpacePossum sorry for the delayed reply, I need to check how to do that without messing the work.

no worries and no rush :)
lemme know how it goes, I can do the squashing manually if you prefer that

Thank you @8ctopus.


Fix IPv4 formatted IP address returning True on ipv6

Fixes #58

Added tests to make sure valid IPv4 addresses return False on IPv6 and vice versa
Made sure any address that lacks a : character returns False on IPv6.

Thanks for the PR! :train:


examples black screen when VR button clicked ~ should be similar to webvr-for-ar

Do you want to request a feature or report a bug?

webvr-for-ar.html works for my stylo-2 ; when I click the VR headsets in the other .html examples the stylo2 goes to a black screen. Should it work similar to the webvr-for-ar?
What is the current behavior?
Older devices Black screen on all AR button click on the bottom right of the screen on all examples except webvr-for-ar.html  (Still seems to pick up on markers and display data, but not augmented)
If the current behavior is a bug, please provide the steps to reproduce.
Use Stylo2 or Galexy SIII and click the AR button on the bottom right of any example

What is the expected behavior?
To function similar to webvr-for-ar.html
If this is a feature request, what is motivation or use case for changing the behavior?
To allow older devices to function in a VR/AR environment (As I am assuming the may be due to the device age)
Please mention other relevant information such as the browser version, Operating System and Device Name
Galexy SIII Google Chrome
Stylo2 Google Chrome
(In Iphone 7 safari no vr button shows up on the screen)
I would be willing to fix this if this is an actual issue; I read over the webvr-for-ar code and should be able to implemented it but was wondering if any major changes might be pushed soon to fix this bug.

Is anyone else having this issue?

Appears this is a duplicate issue from a 2017 mention. If no one is working on this still I will continue to look into it. Have not heard back from Jerome regarding 2.0 release and a possible fix to this issue.


[BUG] Extremely slow downloads on BlobClientBase::download(OutputStream) in unit tests

Describe the bug
In unit tests, BlobClientBase::download(OutputStream) runs extremely slowly, taking up to three minutes for a 256MB file. In release builds, this process takes 10-15 seconds.
Exception or Stack Trace
N/A
To Reproduce
Write a test similar to the followiong in blob service unit tests:
def "Download test"() {
    setup:
    blobac = new BlobClientBuilder().pipeline(blobac.getHttpPipeline()).endpoint("https://xclientdev3.blob.core.windows.net/perftest/4a3ff883-0822-452b-b052-3ee713a497898451525172718223843.txt").buildAsyncClient()

    def start = System.currentTimeMillis()

    blobac.download().then().block()

    def end = System.currentTimeMillis()
    System.out.println(end - start)
}

Code Snippet
Unknown
Expected behavior
A similar runtime to normal builds.
Screenshots
N/A
Setup (please complete the following information):

OS: Windows
IDE : IntelliJ
Version: 12.0.0-preview.4 (additionally, master as of commit 126f19539f0d232ff1d6ce57bb1b34329f069324)

Additional context
None
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

There is a possibility that an exception happened during download resulting in a retry with back-off happening. Does this still occur as there has been a lot of work to harden network failures in the SDKs since this was written.

Closing this as there are performance tests which use download(OutputStream), so if there is a significant regression it'll be caught and a newer tracking issue can be opened if that occurs.


IOC Codes

In sports apps, the IOC country codes are often preferable to the ISO 3166-1 codes, as some of them are more readable to English-speakers:

Country
ISO
IOC

Algeria
DZA
ALG

Aruba
ABW
ARU

Croatia
HRV
CRO

Would adding the IOC codes be a good idea?
http://en.wikipedia.org/wiki/Comparison_of_IOC,_FIFA,_and_ISO_3166_country_codes

Sure, nil for IOC unrecognized countries? Let me know if you find something the IOC recognizes that we don't

@villesundberg @rposborne Has anyone tackled this issue yet?  If not, I'd love to give it a shot!

As far as I can tell, this issue is fixed.
2.3.0 :032 > ISO3166::Country.find_country_by_name("Algeria").ioc
 => "ALG"

@villesundberg Cool.  Just saw the issue and felt it was something I could contribute with.


Fix LIFX lights with disappearing names

Description:
This fixes a recent issue where the friendly light name was momentarily forgotten when a light was turned on at the wall switch.
The fix allows a little cleanup by getting rid of the separate _available variable and its weird setter method.
Checklist:
If the code communicates with devices, web services, or third-party tools:

[X] Local tests with tox run successfully.

I noticed that the commits were in the wrong logical order so I have pushed a rebased branch that is easier to understand. Also updated the PR title to make it clear that this is a bugfix.

Looks good! :dolphin:


progress: add option to remove spinner & steady tick

We use rust-parallel in an airflow job. Because of the way airflow saves the logs of the jobs, it creates a new line every time the spinner is updated, as it doesn't support ANSI I think. This results in a lot of spam, since this crate has a steady tick of 100ms, we get 2.1M useless log lines, which not only makes the logs useless, it also increases the log storage costs.
I've a fork where I've removed both of these, but I would like to raise a PR with an option to disable this at runtime either via a CLI arg or environment variable (such as TERM or NO_COLOR).
What kind of an interface should we have for this.

I agree the progress bar can be much more configurable than current hard-codings, I think at least we want these options:

Ability to disable steady tick
Ability to remove spinner from PROGRESS_STYLE

We may even want to make the entire PROGRESS_STYLE configurable via a command-line option as there are many options that https://github.com/console-rs/indicatif suppports.
Thanks!

I am thinking we can add these 2 command line options easily:
--progress-disable-steady-tick
--progress-disable-spinner
@utkarshgupta137 please let me know if you would like to raise a PR for above, if not I am happy to make this change also.
thanks!

I would personally prefer an environment variable for this, otherwise it would crowd out the command line too much. I feel like this is something that you either always want on or always want off & not toggle it on a per invocation basis. Can have both too.
FYI, there is precedent for this kind of stuff being toggled on or off using environment variables, for example NO_COLOR or CI environment variables.

I pushed a commit here: https://github.com/aaronriekenberg/rust-parallel/commit/24b8257663ad3541ac21ab282342fa4552bcb07c
This disables the steady tick and spinner if stderr is not a terminal, using the builtin std::io::stderr().is_terminal()
@utkarshgupta137 can you please test if this behaves well in your airflow job example?
thanks!

No, doesn't seem to be working.

Thanks for testing @utkarshgupta137
One question to understand:
Is this the way you are running your jobs:

Thanks for testing @utkarshgupta137
One question to understand - is this the way you are running your jobs?
airflow -> ssh -t (force pseudo-terminal) -> rust-parallel (execute commands in parallel)

Yeah, it uses get_pty arg for paramiko.

Learned that tracing subscriber (which rust-parallel uses for logging) uses NO_COLOR environment variable to decide if terminal supports ANSI.  This is a useful thing on its own, if NO_COLOR=1 environment variable is set, rust-parallel logging will not use ANSI color sequences.
References:

https://github.com/tokio-rs/tracing/issues/2388
https://github.com/tokio-rs/tracing/pull/2647

Pushed a commit to make the progress bar check if NO_COLOR environment variable is set to non-empty value.  If yes then we decide the terminal does not support ANSI, so disable steady tick and spinner in progress bar: https://github.com/aaronriekenberg/rust-parallel/commit/10d60198303cc10abd372cf03f823107d1306078
Example usage:
$ export NO_COLOR=1
$ rust-parallel -d all -p sleep ::: 1 2 3

OR
$ NO_COLOR=1 rust-parallel -d all -p sleep ::: 1 2 3

@utkarshgupta137 can you please test this?  Thanks!

In v1.17.0 added PROGRESS_STYLE environment variable to allow for user-configurable progress styles.
See manual for more details.
@utkarshgupta137 I think PROGRESS_STYLE=simple should work well for your jobs - this will disable spinner and steady tick.
Thanks!

Thank you, it is indeed working.


Add Language-Specific Command-Line Flag for Go Binary #830

This pull request introduces a language-specific command-line flag, "--extract", to enhance the functionality of our program. The flag allows users to specify the language they want to extract information from, with options for "go", "rust", and "dotnet". Additionally, the flag supports an "unknown" option.
Usage example; floss -e <language> <binary>
Resolves https://github.com/mandiant/flare-floss/issues/830

good to merge, @Arker123?


test: random change to test presubmit status TGP

Caretaker Note: Please ignore the google-internal-tests status. Still flaky..


Add the Brows app, a file manager with a command palette.

Brows is a file manager for Windows. It'll be cross-platform... eventually. source
Cheers to you for recognizing that typing is more fun than pointing and clicking.

Cool cool. Thanks!


IFTTT just got 2FA!

Good news everyone!
IFTTT just replied to my tweet with the link to their blog post: http://blog.ifttt.com/post/122351757903/two-step-verification-is-now-available-for-your
Starting from today, they support 2FA!

You're number 3. It's been merged already. Thank you for the enthusiasm, though!


ng2-utils dependency

Please move ng2-utils from 'devDependencies' to 'dependencies'. You can read devDependencies. In other words ng2-utils fail to install in my project node_modules.

I think if you use ng2-sticky.umd.js, all these dependencies, which are internally used by ng2-sticky, because all are already inside .umd.js. Thus, it is not necessary to be installed into your module.
I might haven't thought of some other situation. What am I missing?

While that might be true, you should distribute a project that works without the use of pre-bundled files.

K, I will add ng2-utils as dependencies from devDependencies.


HLint hints

Don't conflict with the LTS-8 patch so they can be applied in any order.

I added you to Collaborators.


Hide the navbar when user scrolls down.

I want to add some css styling to make the navbar dissapear when the user scrolls down and appear when the user scrolls up.

@marsianjohncarter sure. You can share a small video related to it before creating pull request. Good luck

@divyanshudhruv here is how it turned out:
https://github.com/user-attachments/assets/0e806299-0ca8-4d9b-9136-f1ec5a64ed4a

@divyanshudhruv Should I open PR or what until 16/10?

You can wait till 16/10 because there are a lot of code that needs to be fixed.Thank you and good luck

You can wait till 16/10 because there are a lot of code that needs to be fixed.Thank you and good luck

For sure. I checked how the project look with all of the additions, and found that it has turned a bit "Bigafolio".
Thank you for your commitment!


[Feat] - CropImageView has no way of setting a CropImageOptions programmatically

Currently, it's constructed via XML attributes, however when using that View programmatically there's no real way to pass in a CropImageOptions to configure it
I wanted to use #390 only to realize that there's no way for me to actually use it.
Maybe the options would also need to be splitted since CropImageView calls into CropOverlayView which calls into CropWindowHandler and they are all sharing the same options. Seems like each should have their own.

I like the idea, let's see how would be implemented on the sample app 👍


what is "lighting" transformation

How were the hard coded values below calculated :
db> coco.py> Line> init()
self._eig_val = np.array([0.2141788, 0.01817699, 0.00341571], dtype=np.float32)
self._eig_vec = np.array([
           [-0.58752847, -0.69563484, 0.41340352],
           [-0.5832747, 0.00994535, -0.81221408],
           [-0.56089297, 0.71832671, 0.41158938]], dtype=np.float32)

These are then used in this function :
utils>image.py> lighting_

def lighting_(data_rng, image, alphastd, eigval, eigvec):
    alpha = data_rng.normal(scale=alphastd, size=(3, ))
    image += np.dot(eigvec, eigval * alpha)

Please explain.

Digging deeper these are values from original coco dataset. And redundant for this paper.


Assigned categories and file types are missing in the JSON response if they were translated

System informations

TYPO Version: 10.4.21
pits_downloadcenter Version: 4.0.3

The download center is available on the website in two different languages: English and German. Therefore, the categories and file types were first created in German and then localized into English.
The categories and file types are assigned to the assets that the download center should display.
In the JSON response, the translation of the category/ file type itself is correctly.
However, the assigned category and file type are missing for the individual files in the JSON response.
Therefore, the filter function by category cannot be used.

I double checkt the assigned categories an reassiged them and now it works again. So it was no issue with the mulitlanguage


Fix #3320 - Prevent select2:open when clearing selection

This pull request includes a

[x ] Bug fix

The following changes were made

Prevent select2:open when clearing selection.
Include originalEvent in clear event, the same as it has been included in unselect event in multiple.js file

Link to the issue ticket: https://github.com/select2/select2/issues/3320

Awesome, thanks.  So, now we need to decide if this is a "bug fix" (and release with 4.0.6), or if this is going to be a significant enough change that we need to put it off to 4.1.

@alexweissman This issue has been existing at least since May 6, 2015 (the creation date of that issue ticket). It has annoyed me probably even longer until now I decided to dig into code and resolve it. I think it is a bug and I suggest to publish the fix as soon as possible. And it doesn't change API, there are no new option params, so nothing will be broken if this would be published as a bugfix...

Seems fair.

With the change in multiple.js, the selections of multiple select cannot be unselected at all.  To resolve the issue, the line evn.stopPropagation() should be moved a bit lower, to after self.trigger('unselect', ...).  Tested with version 4.0.3.

This pull request was never merged and as of 4.0.6rc1 the weird/inconsistent open-dropdown-on-unselect behavior still happens. Was it lost in the shuffle?

Still not merged in v4.0.7. I wrote a test case: https://jsbin.com/sacunojefe/edit?js,output


Workflow engine graceful shutdown

This PR will ensure graceful shutdown of workflow engine.
Please ref https://github.com/StackStorm/st2/discussions/5373 for more details.

Marking the PR as WIP until dependent MR is merged https://github.com/StackStorm/st2/pull/5396

Some work is still needed so the mechanism is more robust. When a workflow engine starts up, does it know to resume workflows that have been paused due to a shutdown?

Added feature to resume.

@khushboobhatia01 The implementation here needs some more rework.
Here're some background on the workflow engine that may help you

a workflow execution can be processed by any workflow engines
the process function to handle workflow execution or handle action execution messages is short lived
the workflow engine is multi-threaded and many process functions could be running at any given time
if the workflow engine needs to pause workflows because there are no other workflow engines running, then the workflow engine should query the database for the list of running workflows

Given the above,

For each workflow engine, we just need to track how many messages it is still processing. We don't need to track the specific workflow execution or task execution. We just need to increment a counter when invoking process and decrement the same counter when exiting process. On shutdown, if the counter is > 0, that means the workflow engine is processing message(s).
Incrementing or decrementing the counter should be thread safe so need to thread lock before updating the counter.
On shutdown, the workflow engine should initiate shutdown and stop receiving any more messages. Then it waits until the counter becomes zero.
After the workflow engine stopped processing, if there are no other workflow engine running, then acquire a distributed lock (use coordinator), query the database for any active/running workflow execution, and pause the workflows. The start up sequence of workflow engine should use the same distributed lock when trying to resume paused workflows. This is to prevent workflow engines stepping over each other.

Please review and let me know if need more clarification.

@m4dcoder Updated the MR with new implementation. One issue with the above flow is during last engine shutdown, workflows which have active tasks could transition only to pausing state instead of paused state. To address this, I've added a delay (https://github.com/StackStorm/st2/pull/5463/files#diff-e4e1e16f3a5aefdf069b0af9bd510c1fde29a1b8df10796b7e74cb183820973cR79) in the engine startup resume sequence to ensure workflows stuck in pausing state transition to paused state before we start resuming.
Please review.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: khushboobhatia01:x: Khushboo BhatiaKhushboo Bhatia seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


When mergeAttrs=true , How to let children always in array, but attribute not ?

When I use :
    var parser = new xml2js.Parser({
        explicitArray: false,
        mergeAttrs: true,
    });

The children are not always  in array.
If I set  explicitArray=true,   the attributes are in array too (I don't want it)

I actually encountered this problem also.  It's a simple way to separate attributes and elements.  Elements are arrays, attributes aren't.  I would be happy to submit a pull request to make this possible, but it seems like a breaking change.


Inspect ORM classes with column types, fixes #441

Description
Inspect ORM classes with column types, fixes #446
Needed for exclude and include in queries.
Questions

[ ] Question1

Status

[x] Ready to go

@doaa-altarawy Do we need any tests here? If not, feel free to merge.


Developer instruction fixes

Adding some troubleshooting steps to Developer instructions

This change is

Thanks for improving this!

/lgtm
/approve


bump jquery dependency up to latest 2.x

Having angular-knob's jquery dependency be "~2.0.3" means that a project (like mine) that uses a newer 2.x version of jquery will get the following warning when doing a bower install:
Please note that,
    angular-knob#0.0.3 depends on jquery#~2.0.3 which resolved to jquery#2.0.3
    bootstrap#3.3.6 depends on jquery#1.9.1 - 2 which resolved to jquery#2.2.4
    jquery-knob#1.2.13 depends on jquery#>=1.7.0 which resolved to jquery#3.1.0
Resort to using jquery#2.2.4 which resolved to jquery#2.2.4
Code incompatibilities may occur.

It might be even better to change this to something similar to jquery-knob, like ">=2.0.3".
This is a breaking change (I think..?), because at the very least this means jquery.min.js will reside in a different location upon installation.

self bump

@yunlzheng can you merge please?


Separate Monitoring Stack

We need to separate out the monitoring stack first which sends out alerts to Slack, Twilio and StatusPage as errors in the backend and performance degradation can affect monitoring

[ ] split out into separate repo / docker build
[ ] host on non-AWS (cloudflare)
[ ] setup hooks into slack, twilio alerts and statuspage

see issue from #229 on what we should cover
combine with 1-month test data in unit tests

#210 on monitoring as well

see https://github.com/opennem/monitoring

Design a new monitoring page that includes status page output and monitoring API output

closed - status page separate issue


Add missing nodes warning

Closes https://github.com/mads-hartmann/bash-language-server/issues/74
Currently, the extension is just silent if a node is missing, but this PR adds an error to the bottom of a file if a node is missing.

Unfortunately, we cannot show the missing node where it occurs... So we should evaluate if this PR gives any additional value. See https://github.com/tree-sitter/tree-sitter-bash/issues/30
Besides, I'm bumping the version number and increase the required server version to the current release (a lot are probably using an old version).

@skovhus I'll merge and release tomorrow 👍

I've sneaked in a small upgrade of the grammar in this PR. 😊


Add repeat_interleave

Is your feature request related to a problem? Please describe.
We need to be able to use repeate_interleave in the ttnn api to build stable diffusion models
Describe the solution you'd like
I need a fallback up to torch from ttnn for now and need subsequent work to actually implement this in ttnn properly
Describe alternatives you've considered
Converting to torch manually with unnecessary code.
Additional context
Please follow the description as defined https://pytorch.org/docs/stable/generated/torch.repeat_interleave.html
tensor_2d = torch.tensor([[1, 2], [3, 4]])
result = torch.repeat_interleave(tensor_2d, 2, dim=0)
print(result)
# Result :
# tensor([[1, 2],
#         [1, 2],
#         [3, 4],
#         [3, 4]])

@arakhmati , this one is done right as we have provided the fall back for Boris.


Aura duration of Buffs arent shown in classic SoD

The targetframe doesnt display the duration spiral even though the duration information is seemingly available

I'm not really touching Buffs (right now), so it just shows like the default UI. You need something like ClassicAuraDurations, but i think its not really updated for SoD 😐
I'll take a look if i have time, i also noticed this while playing SoD.

Ah I see, so I guess it inherits from the default target frames that makes sense!
I'll see if a separate addon like classicAuraDurations would fix this :)
Also big props to you for doing this retail ui addon, I love it! 🥳

I'm also using ClassicAuraDurations now, and it seems to work fine.
If you have other suggestions or bugs, just open another issue. Thanks!


docs($q): Document optional parameters for $q.when

This documents undocumented methods present in the $q.when API.

Closing as per prior discussion & age


EasyScheduler supports LDAP authentication, and Hive supports LDAP authentication.(EasyScheduler支持LDAP认证， Hive等连接支持LDAP认证)

EasyScheduler supports LDAP authentication
Hive and other connections support LDAP authentication
Support for Group Mapping via LDAP

EasyScheduler支持LDAP认证
Hive等连接支持LDAP认证
支持通过LDAP做Group Mapping

good idea！thx！
and welcome to sumit pull request to implement these features!

增加基于消息驱动的流程调度模式

+1 mark

+1

+1

ladp is deeply depended in my bigdata ecosystem

I will implement ldap login authentication later

later
Hi, how is it going?

I will implement ldap login authentication later

Hi, how is it going?

I will implement ldap login authentication later

Hi, how is it going?

sorry，I gave up.....

这个需求2年了，还没考虑支持啊，LDAP在大数据验证领域用的还是非常广泛的，大部分组件都支持，感觉还挺重要的

https://github.com/apache/dolphinscheduler/pull/3743
this issue is closed  in #3743

I will implement ldap login authentication later

Hi, how is it going?

sorry，I gave up.....

haha, me too


Rework _compute_shared_prefix_bits.

What is wrong?
https://github.com/ethereum/trinity/blob/86b1bfb6f5fcc7b943fb408ae206c047aecb828d/p2p/kademlia.py#L373-L388
The current implementation of _compute_shared_prefix_bits uses the string representation of the binary form of the node ids to perform the check in a very indirect manner.  We are also likely to be incurring non-trivial overhead via all the string manipulation that has to occur.
How can it be fixed

This should be validated to match the existing implementation's functionality as well as add tests for the utility function if none exist.

We should be able to apply a binary & to reduce the entire list of node ids to a single value and then count the leading set bits on that value.
The new implementation should 1. be tested against the old one, 2. have tests  and 3. be profiled against the old one so we can have a sense for the speed difference.

Issue should be closed as obsolete.

I believe you are correct @stobiewan. Thank you for the note.


Release #6

Could you kindly create a new release on NPM that includes https://github.com/shuding/next-view-transitions/pull/6?
Thank you!

Done!


The content of the article is broken

https://docs.microsoft.com/en-us/powershell/module/azuread/set-azureadmsgroup?view=azureadps-2.0

{{Fill MailEnabled Description}} など、うまく置換できていません。

Hi @ShintakuNobuhiro
Thank you for your issue submission.  I have created a work item to track this issue and will be sending it to engineering.  At this point, I am going to go ahead and close this issue.  I will post an update as soon as I have any additional information.
Thank you!
Bill


Blazor Server SignalR message size for JS interop

@TanayParikh ... This bit I probably missed, since I didn't understand what that work was about.
https://docs.microsoft.com/en-us/aspnet/core/blazor/javascript-interoperability/call-dotnet-from-javascript?view=aspnetcore-6.0#size-limits-on-javascript-interop-calls
It appears in both JS interop topics from an INCLUDE at ...
https://github.com/dotnet/AspNetCore.Docs/blob/main/aspnetcore/blazor/includes/js-interop/size-limits.md
If the whole thing is no longer relevant for >=6.0, then I can drop the section. If it requires a re-write, then do you want to do it or paste the revised section into an issue comment here?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2964d558-cd6d-7421-f7b9-cb939792c2cf
Version Independent ID: a3517e56-69fc-1ec7-b55a-dbb54d68a807
Content: Call .NET methods from JavaScript functions in ASP.NET Core Blazor
Content Source: aspnetcore/blazor/javascript-interoperability/call-dotnet-from-javascript.md
Product: aspnet-core
Technology: aspnetcore-blazor
GitHub Login: @guardrex
Microsoft Alias: riande

Hey @guardrex, this is mostly still valid, I'd probably add a cross-link for streaming as the recommended alternative (as it really just does what the rest of the recomendations suggest internally (like chunking and avoiding blocking the UI thread). Will have a PR up shortly 😄


Improve error output

The most common error I get is "A script did not complete before its timeout expired."

It's not clear to a newcomer what that means. It makes me think there's some deep problem in the setup, when it just means that waitForVisible wasn't able to find main.ptabs. While I've figured out what it means and trained myself to look at the bottom of the callstack, it would be nice if:

A clearer message is given at top that includes the failing webdriver call.
That message is displayed in the web UI, so that for most errors I can just look at that instead of scrolling up in the log tail. Current message is:
Perhaps there are other error types in which the meat of the error isn't on the first line, and it would be nice to get the relevant line in the web UI

I agree this is a scary stack puke! I've already started on humanifying cucumber errors. I'll add webdriver errors to the mix too and make them a little more focused on the UI

Haha, great to hear, thanks
On Wed, Apr 29, 2015 at 11:33 PM, Sam Hatoum<EMAIL_ADDRESS>wrote:

I agree this is a scary stack puke! I've already started on humanifying
cucumber errors. I'll add webdriver errors to the mix too and make them a
little more focused on the UI
—
Reply to this email directly or view it on GitHub
https://github.com/xolvio/meteor-cucumber/issues/81#issuecomment-97677817
.

Another error case that isn't handled well - malformed Gherkin Syntax. I recently typoed in my tests and was quite confused as to why my tests were running and running.

Fixed the first issue in the browser reporter - will be in 0.7.0. The console reporting is a little more tricky, I'll try to do that for both issues mentioned above

Just want to add my own experience to the discussion:
this.Then(/^I should be on the login page$/, function(callback) {
      this.browser.
        .waitForExist('.email', 1000)
        .waitForVisible('.email', 1000)
        .call(callback);
    });

I forgot to remove a second dot when I was writing the above test (after this.browser) and I had no visible error or warning. My tests would simply not run until I found and removed the dot (which took a good 10 minutes to pinpoint).

I'm on 0.8.0, and getting a Element is not currently interactable and may not be manipulated error. I didn't used to see it on previous versions of this pkg. I was able to fix the error by adding a couple .waitForVisibles, but the problem is that the error doesn't tell me which webdriver call in a long chain is the cause, so I have to do a commenting-out binary search to see at which call the error begins occurring. Would be nice to either print out the call, eg .click '#datetime' or the line number.
   When I login as a@b                                                  # features/parlay.feature:6
[cuke-monkey][hooks] browser error event - capturing screenshot
    When I create a parlay                                               # features/parlay.feature:7
      Element is not currently interactable and may not be manipulated

the last few versions have massively improved error logging