How to interpret LIME results?

I am considering using LIME, and I am having some struggle to understand what exactly it outputs.
I posed a question on stack exchange with a MCVE, but maybe this is more suitable here.
Consider the following code, that uses logistic regression to fit a logistic process, and uses LIME for a new example.
import numpy as np
import lime.lime_tabular
from sklearn.linear_model import LogisticRegression

# generate a logistic latent variable from `a` and `b` with coef. 1, 1
data = []
for t in range(100000):
    a = 1 - 2 * np.random.random()
    b = 1 - 2 * np.random.random()
    noise = np.random.logistic()
    c = int(a + b + noise > 0)  # to predict
    data.append([a, b, c])
data = np.array(data)

x = data[:, :-1]
y = data[:, -1]

# fit Logistic regression without regularization (C=inf)
classifier = LogisticRegression(C=1e10)
classifier.fit(x, y)

print(classifier.coef_)

# "explain" with LIME
explainer = lime.lime_tabular.LimeTabularExplainer(
                x, mode='classification',
                feature_names=['a', 'b'])

explanation = explainer.explain_instance(np.array([1, 1]), classifier.predict_proba, num_samples=100000)
print(explanation.as_list())

output:
[[ 0.9981159   0.99478328]]  # print(classifier.coef_)
[('a > 0.50', 0.219), ('b > 0.50', 0.219)] # print(explanation.as_list())

the ~[[1, 1]] is because we are doing logistic regression to a Logistic process with these coefficients.
What do the values 0.219... mean? Are they relatable to any quantity of this example?

I think this is a bit confusing because you're using numerical data, but the default parameters in TabularExplainer discretize the data into quartiles. It is harder to interpret explanations for numerical features for the following reasons:

The values may be in different ranges. We can always standardize the data, but then the meaning of the coefficients changes
It's hard to think about double negatives (i.e. negative weight for a negative feature = positive contribution)

Anyway, let's consider the meaning of the explanations in the discretized version.
What ('a > 0.50', 0.219) is saying is that on average (considering the training data distribution), having a in this bucket raises the prediction by 0.219. Consider the following:
import itertools
other_values = np.arange(-1, .49, .01)
current_pred = classifier.predict_proba([1, 1])[0, 1]
current_pred - classifier.predict_proba(np.array(list(itertools.product(other_values, [1]))))[:, 1].mean()
# output: (0.21064350778528229)

Roughly, what I'm doing above is integrating over other values of a while keeping b fixed. On average, if we do that, the output moves by 0.211. Think of doing that for both features, while weighting by locality - that is what the coefficients in the explanation are getting at.
You could set discretize_continuous=False in the LimeTabularExplainer constructor. This example would still be a tricky one, because there are many equivalent linear models that fit the data equally well with different intercepts, and LIME will pick an arbitrary one (so the weights are not necessarily going to be the same, even if the approximation is almost perfect)

Thanks Marc for the input. It did help.
As a follow-up, here are the results of the relative error of LIME with increasing number of samples, where relative error means the explanation with a given number of samples x, against sampling from the opposite side a < 0.5 (x samples), like you did.

Would you expect the relative error to go to zero? If not, what variables would I need to increase for the error to go to zero? If none, what approximations explain the discrepancy of ~5%?
Figure generated with the code below:
import numpy as np
import lime.lime_tabular
from sklearn.linear_model import LogisticRegression

data = []
for t in range(1000000):
    a = 1 - 2 * np.random.random()
    b = 1 - 2 * np.random.random()
    noise = np.random.logistic()
    c = int(a + b + noise > 0)  # to predict
    data.append([a, b, c])
data = np.array(data)

x = data[:, :-1]
y = data[:, -1]

classifier = LogisticRegression(C=1e10)
classifier.fit(x, y)

print(classifier.coef_)

explainer = lime.lime_tabular.LimeTabularExplainer(x, mode='classification', feature_names=['a', 'b'])

event = np.array([1, 1])

current_pred = classifier.predict_proba(event)[0, 1]

result = []
for samples in [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]:
    samples = samples * 1000
    print(samples)
    # increase number of samples for the explanation
    explanation = explainer.explain_instance(event, classifier.predict_proba, num_samples=samples).as_list()

    # freeze b and sample `a` from the interval `-1 < a < 0.50`
    import itertools
    other_values = -1 + 1.5 * np.random.random(samples)  # a_i from U(-1,0.5)
    other_values = np.array(list(itertools.product(other_values, [1])))  # as a matrix [[a_1, 1], [a_2, 1], ...]
    residuals = current_pred - classifier.predict_proba(other_values)[:, 1]

    relative_error = (explanation[0][1] - residuals.mean())/residuals.mean()

    result.append([samples, relative_error])
result = np.array(result)

import matplotlib.pyplot as plt
plt.figure()
plt.plot(result[:, 0], result[:, 1])
plt.ylabel('relative error')
plt.xlabel('samples')
plt.xscale('log')
plt.savefig('test.png')

Output to stdout:
[[ 0.99826413  1.00231008]]
1000
2000
4000
8000
...

I would not expect the error to go to zero, because the model is using continuous data while LIME is approximating it with the discretized version. Also, there is the locality weighting, i.e. samples near the point being explained are weighted more heavily than samples far away.
The error would go to zero if the model was actually using discretized data and if you set the kernel width to infinity.

Marcos, thank you for the explanation and for taking the time to read and comment this, much appreciated!
To double-check we understood everything so far, these are the facts so far:

F1: on a simple logistic regression (as defined on the code above), there is a systematic error of ~5% for the event [1,1].

These are the hypothesis on the table:

H1: LIME result of 0.219 in ('a > 0.50', 0.219) above is roughly how much the probability increases when a > 0.5 in comparison to a in other quantiles. [code in your first comment]
H2: the systematic error (F1) is explained by two factors:

LIME is discretizing data
there is a Kernel, so it is not a simple average over the other quartiles

Let's assume that "roughly" in H1 means within 10%. I.e. if LIME's systematic error is within 10% on points on the quartile, then H1 is valid.
To test H1, we can repeat the same experiment as we did for F1 on different events. H1 states that the error remains "roughly" small (10%).
Below I show the same errors as before for different events (in the legend, a single run per point):

(the code I used is at the end of this comment, in case someone wants to double-check)
We see that there are events with errors of 120%, way above the 10% threshold. Only the event (1,1) is below 10%. I conclude from this result that the hypothesis H1 is false. In other words, regardless of H2, the hypothesis H1 that LIME result of 0.219 in ('a > 0.50', 0.219) is how much the probability increases when a > 0.5 is not supported by the results in the figure above.
Maybe the interpretation is different? Or do you think that LIME is not applicable for this case? If not, why would you expect it to be applicable in continuous data? (Logistic regression is the simplest classification example I know of...)
Have you tested LIME on this type of examples? I went through the tests folder and haven't found a test on the actual values. I was also not able to find anything on the arxiv paper.
If you think that we should switch to the non-discretized version, please let me know, I would happily repeat this for the non-discrete (with an equivalent test).

import numpy as np
import lime.lime_tabular
from sklearn.linear_model import LogisticRegression

data = []
for t in range(1000000):
    a = 1 - 2 * np.random.random()
    b = 1 - 2 * np.random.random()
    noise = np.random.logistic()
    c = int(a + b + noise > 0)  # to predict
    data.append([a, b, c])
data = np.array(data)

x = data[:, :-1]
y = data[:, -1]

classifier = LogisticRegression(C=1e10)
classifier.fit(x, y)

explainer = lime.lime_tabular.LimeTabularExplainer(x, mode='classification', feature_names=['a', 'b'])

print(classifier.coef_)

import matplotlib.pyplot as plt
plt.figure()
for i in range(1, 6):
    event = np.array([0.5 + 0.1*i, 0.5 + 0.1*i])

    current_pred = classifier.predict_proba(event)[0, 1]

    print(event, current_pred)

    result = []
    for samples in [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]:
        samples = samples * 1000
        print(samples)
        # increase number of samples for the explanation
        explanation = explainer.explain_instance(event, classifier.predict_proba, num_samples=samples).as_list()

        # freeze b and sample `a` from the interval `-1 < a < 0.50`
        import itertools
        other_values = -1 + 1.5 * np.random.random(samples)  # a_i from U(-1,0.5)
        other_values = np.array(list(itertools.product(other_values, [1])))  # as a matrix [[a_1, b], [a_2, b], ...]
        residuals = current_pred - classifier.predict_proba(other_values)[:, 1]

        print(explanation, residuals.mean())

        relative_error = (explanation[0][1] - residuals.mean())/residuals.mean()

        result.append([samples, relative_error])
    result = np.array(result)

    plt.plot(result[:, 0], result[:, 1], 'o-', label='(%.2f, %.2f)' % tuple(event))
plt.ylabel('relative error')
plt.xlabel('samples')
plt.xscale('log', basex=2)
plt.legend()
plt.savefig('test.png')

Discretization does make everything tricky. In your code, you're computing the residual with respect to the prediction of the event. However, LIME is taking the event to be a > 0.5 and b > 0.5, not two specific values. So, instead of:
current_pred = classifier.predict_proba(event)[0, 1]

We should have
lime_event = (.5 +  .5 * np.random.random(samples * 2)).reshape(-1, 2) # a > .5 and b > .5
current_pred = classifier.predict_proba(lime_event)[:,1].mean()

Also, the value of b for LIME is b > 0.5, so instead of:
other_values = -1 + 1.5 * np.random.random(samples)  # a_i from U(-1,0.5)
other_values = np.array(list(itertools.product(other_values, [1])))  # as a matrix [[a_1, b], [a_2, b], ...]

let's have:
other_values = -1 + 1.5 * np.random.random(samples)  # a_i from U(-1,0.5)
other_b = (.5 +  .5 * np.random.random(samples)) # b_i from U(.5, 1)
other_values = np.vstack((other_values, other_b)).T

These two would explain why your error goes up the further you are from a=1, I think.
Also, explanation.as_list() returns the features in decreasing order of importance, so relative error should be:
relative_error = (dict(explanation)['a > 0.50'] - residuals.mean())/residuals.mean()

Doing these results in a relative error that is ~constant with respect to the events (around 10%).
Anyway, note that the explanation also has an intercept. What I meant by 'roughly' before is that the weight for 'a > 0.50' is going to be close to:
explanation.intercept[1]+ dict(explanation.as_list())['b > 0.50'] - classifier.predict_proba(other_values)[:, 1]

Thank you @marcotcr for the explanation. That does indeed explain the error above:

(code below)
To summarize: the interpretation of

[('a > 0.50', 0.219), ('b > 0.50', 0.219)]

is

the probability of 1 increases by 0.219 when a in [0.5,1] when compared to a in [-1,0.5], averaged over b in [0.5,1].

Doesn't this imply that LIME result only depends on the quartile that the events belong to? For example, isn't it possible for LIME to provide the same explanation for two events whose outcome is opposite? (e.g. the model gives a different prediction on different values of the same quartile).
import numpy as np
import lime.lime_tabular
from sklearn.linear_model import LogisticRegression

data = []
for t in range(1000000):
    a = 1 - 2 * np.random.random()
    b = 1 - 2 * np.random.random()
    noise = np.random.logistic()
    c = int(a + b + noise > 0)  # to predict
    data.append([a, b, c])
data = np.array(data)

x = data[:, :-1]
y = data[:, -1]

classifier = LogisticRegression(C=1e10)
classifier.fit(x, y)

explainer = lime.lime_tabular.LimeTabularExplainer(x, mode='classification', feature_names=['a', 'b'])

import matplotlib.pyplot as plt
plt.figure()
for i in range(1, 6):
    event = np.array([0.5 + 0.1*i, 0.5 + 0.1*i])
    print(event)

    result = []
    for samples in [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]:
        samples = samples * 1000

        lime_events = (.5 + .5 * np.random.random(samples * 2)).reshape(-1, 2)  # a > .5 and b > .5
        current_pred = classifier.predict_proba(lime_events)[:, 1].mean()
        del lime_events

        # increase number of samples for the explanation
        explanation = explainer.explain_instance(event, classifier.predict_proba, num_samples=samples).as_list()

        # freeze b and sample `a` from the interval `-1 < a < 0.50`
        other_values = -1 + 1.5 * np.random.random(samples)  # a_i from U(-1,0.5)
        other_b = (.5 + .5 * np.random.random(samples))  # b_i from U(.5, 1)
        other_values = np.vstack((other_values, other_b)).T
        residuals = current_pred - classifier.predict_proba(other_values)[:, 1]

        relative_error = (dict(explanation)['a > 0.50'] - residuals.mean())/residuals.mean()

        print(samples, relative_error)
        result.append([samples, relative_error])
    result = np.array(result)

    plt.plot(result[:, 0], result[:, 1], 'o-', label='(%.2f, %.2f)' % tuple(event))
plt.ylabel('relative error')
plt.xlabel('samples')
plt.xscale('log', basex=2)
plt.legend()
plt.savefig('test.png')

Yes, that is possible. That is a problem with discretization, we lose the ability to differentiate things within the discretized bins. Obvious solutions to this involve using more bins (deciles, entropy-based discretization) or not discretizing at all. Not discretizing at all has its own drawbacks, which is why I decided to leave discretization in as the default.

@marcotcr. I understand. Regardless of this particular point, LIME remains a useful tool for interpretability. Thank you and the other authors for taking the time to develop and publish it, provide source code to reproduce its results, and thank you especially for clarifying the points raised here. Definitely a great example of how science should be done!
I will close this as resolved.

Thanks for the thoughtful questions!

Ok, I returned to this, now for the non-discretized version. Essentially, trying to do the same for the non-discretized version of LIME. My expectation, based on the results from the discretized version, is that LIME approximates the partial derivative of the function in respect to each input. However, I may be mistaken, because I am getting a 20% systematic error between LIME and the partial derivate.

import numpy as np
import lime.lime_tabular
from sklearn.linear_model import LogisticRegression

data = []
for t in range(1000000):
    a = 1 - 2 * np.random.random()
    b = 1 - 2 * np.random.random()
    noise = np.random.logistic()
    c = int(a + b + noise > 0)  # to predict
    data.append([a, b, c])
data = np.array(data)

x = data[:, :-1]
y = data[:, -1]

classifier = LogisticRegression(C=1e10)
classifier.fit(x, y)

print(classifier.coef_)

explainer = lime.lime_tabular.LimeTabularExplainer(x, mode='classification', feature_names=['a', 'b'], discretize_continuous=False)

event = np.array([0.7, 0.7])

current_pred = classifier.predict_proba(event)[0, 1]

result = []
for samples in [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]:
    samples = samples * 1000
    # increase number of samples for the explanation
    explanation = explainer.explain_instance(event, classifier.predict_proba, num_samples=samples).as_list()

    # freeze b and sample `a_i` (`a` + N(0, 0.001), `b`) to compute partial derivatives
    a_i = event[0] + np.random.normal(scale=0.001, size=samples)

    a_i = np.array([[x, event[1]] for x in a_i])  # as an array of events [[a_1, b], [a_2, b], ...]

    # partial derivatives df/da
    d_a = (current_pred - classifier.predict_proba(a_i)[:, 1])/(event[0] - a_i[:, 0])

    # confirmed that d_a is approximatelly d_a1 below
    # exp = np.exp(np.dot(event, np.array([1, 1])))
    # d_a1 = 1 * exp / (1 + exp)**2

    relative_error = (dict(explanation)['a'] - d_a.mean())/d_a.mean()

    result.append([samples, relative_error])
result = np.array(result)

import matplotlib.pyplot as plt
plt.figure()
plt.plot(result[:, 0], result[:, 1])
plt.ylabel('relative error')
plt.xlabel('samples')
plt.xscale('log')
plt.savefig('test.png')

Maybe we should move away from the partial derivative interpretation, and back to the original meaning of an explanation: a linear model that approximates the black box model locally.
The additional complication is that we scale the data inside explainer if data is not discretized.
Thus, for some x, if you take
scaled_x = (x - explainer.scaler.mean_) / explainer.scaler.scale_
fhat = exp.intercept[1] + dict(exp.as_list())['a'] * scaled_x[:, 0] + dict(exp.as_list())['b'] * scaled_x[:, 1]
f = classifier.predict_proba(x)[:, 1]

We should have that (f - fhat).mean() is small, in particular for x that are close to the original instance.
Does this make sense?

It makes sense.
The reason I approached it from the partial derivatives is that given a point x' = x + h, f(x') - f(x) = (x' - x)^T.Df + O(h^2) (multivariable Taylor series, first order around x). In this view, if the local regressor is a simple linear regression (without lasso), shouldn't the coefficients be equal to the partial derivatives of f?

Let's call f'(x') the gradient of x'.
Let x be your 'event' above. f(x) is then current_prediction
The taylor expansion gives us the following linear approximation
f(x') = f(x) + f'(x').dot(x' - x)
LIME is trying to find w such that (ignoring the local weighting for now):
f(x') = intercept + w.dot(x')
I don't see why w should be equal to f'(x') in this case. The taylor expansion as an approximation requires us to compute f'(x') for every point we're predicting.
Also, I think f'(x) should be x * exp(x * w) / (exp(x * w) + 1) ^2 , you had exp(x * w) / (exp(x * w) + 1) ^2 if I understood the code correctly.

If I want to print the coefficients of the local model which LIME, how can I do that? Thanks in advance if you can reply.

If I want to print the coefficients that the local LIME model learned, is there a way to do that? Thanks in advance.

exp.as_list() or exp.as_map()

I am finding the set of variables list we getting from explainer is keep changing when we run the rf_explainer.explain_instance on the same test data, may I please know do we have to set.seed or something, or why the variable importance changes. Thanks

see #67, #119, #199.

Hi marcotcr,
Thanks for the reply, I am still getting different values,
#My explainer
model_explainer = lime.lime_tabular.LimeTabularExplainer(X_train.values[:,:], mode='classification',verbose=True,                                                 training_labels=data_norm'class'],feature_names=feature_names,random_state=np.random.seed(42))

#My Function
def explain(exp, instance, predict_fn):
  np.random.seed(42)
  exp_data = exp.explain_instance(instance, predict_fn)
  return exp_data.as_list()

#My Call
explain(model_explainer,X_test.values[1], model.predict_proba)

Still i am getting different values
In [146]: explain(model_explainer,X_test.values[1], model.predict_proba)
Intercept 0.9839002793712486
Prediction_local [0.98390028]
Right: 0.9936423124350471
Out[146]:
[('Dec_Reason_CLA_URNED <= 0.00', 0.0),
 ('Property_Acceptable_Y <= 1.00', 0.0),
 ('Trading_State_NT <= 0.00', 0.0),
 ('Decision_Reason_REQUEST_F <= 0.00', 0.0),
 ('Product_Type_P <= 0.00', 0.0),
 ('Product_Type_I <= 0.00', 0.0),
 ('Fax_Number <= 0.00', 0.0),
 ('Product_Type_D <= 0.00', 0.0),
 ('Valuation_Acceptable_Y <= 1.00', 0.0),
 ('Product_Type_B <= 0.00', 0.0)]

In [147]: explain(model_explainer,X_test.values[1], model.predict_proba)
Intercept 0.9849758158162301
Prediction_local [0.98497582]
Right: 0.9936423124350471
Out[147]:
[('Product_Type_B <= 0.00', 0.0),
 ('Product_Type_8 <= 0.00', 0.0),
 ('Valuation_Acceptable_Y <= 1.00', 0.0),
 ('Home_Phone <= 0.00', 0.0),
 ('Permanent_Resident_Y <= 1.00', 0.0),
 ('Product_Type_P <= 0.00', 0.0),
 ('Dec_Reason_PRE_CLAPOL <= 0.00', 0.0),
 ('Dec_Reason_FR_ERT <= 0.00', 0.0),
 ('Product_Type_C <= 0.00', 0.0),
 ('Product_Type_I <= 0.00', 0.0)]

hello
I have a lstm model. it is a classification model.
I want to use explain_instance. what is predict function in explain_predict?
thanks


CLEARTEXT communication not permitted by network security policy

Problem description
When running tabris serve --logging and then attempting to connect with an Android 9 device, the following error is seen on the device:
CLEARTEXT communication to 192.168.1.x not permitted by network security policy

The server's request log shows no requests.
Opening http://192.168.1.x:8080/ in a browser on the same Android device shows:
Tabris.js CLI version 3.0.0-rc1-dev.20190416 is running

Environment

Tabris.js version: 3.0.0-rc1-dev.20190416
Device: Pixel 3
OS: Android 9

The APK was generated from a bare project created with tabris init and then built using the cloud build service without adjusting any of the build settings

I can successfully connect to the aformentioned server with the 3.0.0-beta2 Developer app currently published in the Play Store

You are seeing the expected behavior. Android prohibits clear text communication by default. We did not opt to remove this security feature in tabris. You can use a custom security policy on your app or a "global" flag in your config.xml like:
    <edit-config file="app/src/main/AndroidManifest.xml" target="/manifest/application" mode="merge">
      <application android:usesCleartextTraffic="true" />
    </edit-config>

Hm, this prevents a bit of a dilemma.  The solution for this isn't documented anywhere, which prevents tabris serve from working.
I don't think it would be advisable to silently enable the flag on debug builds, as it would give the impression of non-https calls working when in fact they would fail on production builds.
Adding a cert to the tabris-cli would be difficult due to the number of addresses that could be used (and there usually isn't a domain name available for an IP address).

I agree with the current confusion. tabris ios behaves in the same way. It is does not allow cleartext communication by default. Also the app created by tabris init does not allows cleartext communication as can be seen here.
The ios client however respects the config.xml element access. It would make sense to map this element to the android based <network-security-config>.


Context save/restore between different cores

The context stored on one core cannot always be restored and executed on another core.
The capabilities of vector units even in one system may be different.
It is required to find a method to determine whether a certain context can be loaded onto another kernel or not.
While the supported sets of vector instructions may be defined via CSRs, the “MAXVL” values for same configuration may be different too.

This is a platform/profile issue. It was mentioned in #42.


Lacking a keyword, which returns current data from the clipboard

Now "Copy" simply works: ctrl+c -> ctrl+v
Would be useful to be able to copy fields with some other keyboard combinations also, because random "copy to clipboard" -keyboard shortcuts are used at desktop apps for example.

Pushed to master in commit dbd165f28222c2de968702dc1e5c8cd6636aac8e


Add feature flag to control gix transport impl

Checklist

[x] I have read the Contributor Guide
[x] I have read and agree to the Code of Conduct
[x] I have added a description of my changes and why I'd like them included in the section below

Description of Changes
The crate gix provides two features, blocking-http-transport-reqwest and blocking-http-transport-curl, but they cannot be set simultaneously.
Most of the time, this requirement is easy to meet, unless we depend on multiple crates that have chosen different features.
In deps.rs, we encountered this problem:

deps.rs -> rustsec -> tame-index -> gix/blocking-http-transport-reqwest
deps.rs -> crates-index -> gix/blocking-http-transport-curl

Perhaps we should expose two features: gix-curl and gix-reqwest, so that upper-level users can have more fine-grained control.

It's not allowing me to push to your fork despite it saying so, please fix the issues if you can.


Target certain single img after loaded

Re #21 @lmeurs brought up the idea that there ought be some mechanism to allow the user to know when exact image has loaded

So that I can use $('#list').imagesLoaded() and do something exactly when - and only when - beta.jpg loads.

Maybe the easiest solution is provide an img argument in dfd.progress

As I've posted in #21, this is my new progress arguments proposition:
deferred.notifyWith( $image, [
    isBroken,  // boolean status of $loaded_image
    $images,   // all images in stack
    $proper,   // thus far properly loaded images
    $broken    // thus far broken images
] );

The .progress callback scope (this) will be the jQuery wrapped loaded image that triggered notify, and the arguments are state of the image and thus far available objects. It is simple enough, but might be even more with dropping the isBroken argument in favor of $broken.index( $image[0] ) !== -1, which does make it simpler, but only when you don't need it :)
What's your thought on this?

@desandro so what is your call? I can't decide whether to drop the isBroken argument, move it at the end, or leave as it is in proposition.

Thanks for pinging me on this, didn't know if you had something in the works.
I feel this is good to move forward as is. One consideration...
I imagine the typical use case is that I want to know when my special image, within a collection, has been properly loaded. With that in mind, the pieces of information I need are:

whether or not the image in question is indeed the one I'm looking for. This is resolved by supplying $image
whether or not that image has properly loaded. In your proposal, that looks to be isBroken

With that in mind, I feel that the isBroken argument is good where it is, at the front of arguments.
I am curious as to why the variable denotes a negative response. isBroken = true seems counter-intuitive to isLoaded = true. -- are you implying that the plugin cannot properly detect if image has loaded, but it can detect if it is broken ( similar to a guilty/not guilty response)?  In which case, then its value and name does make sense.

@desandro I've choose isBroken, because isLoaded === true is the default assumption that would be true in 99.9% of cases, thus isBroken is the information I'm really looking for from this attribute.
Plus if someone needs this kind of attribute, then they want to run a specific code when image is broken. With isLoaded, they would have to negate !isLoaded, or compare isLoaded === false. With isBroken it is just a matter of if( isBroken ){ ... }.
Or at least that's what would be more comfortable for me :) But I don't really mind changing it to isLoaded of you want. Or rather isProper, as isLoaded state is automatically assumed by firing progress method (the image has finished with loading = isLoaded, here is the info - isProper/isBroken, $images etc).

Good points about isBroken.  I guess I was over-thinking it :P
In which case, I think this is good as proposed. Gets my thumbs up.


CAD-2166:  delegation map size trace & export (with UTxO size) to prometheus

extend LedgerQueries with delegMapSize -- to query the effective delegation map size
trace delegMapSize with TraceStartLeadershipCheck
export both utxoSize and delegMapSize as metrics
move all metrics to the cardano.node.metrics namespace.  This affects:

cardano.node.Forge.metrics
cardano.node.ChainDB.metrics

WARNING: breaking change -- see point №4.  This also simplifies configuration, making the following unnecessary:
https://github.com/input-output-hk/cardano-node/blob/master/configuration/cardano/mainnet-config.json#L66-L71
cc @disassembler @denisshevchenko

bors r+

bors r+

bors r+


Add Nuget reference to System.Net.Http

Add Nuget reference to System.Net.Http to resolve issue when connecting to a MFA secured O365 instance using an App Password.

Included in https://www.nuget.org/packages/spkl/1.0.212-beta


Error While installing

hey while installing this i got following error do you know why it happens. how can i solve it?
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['C:\\Users\\99559\\AppData\\Local\\Programs\\Python\\Python311\\python.exe', '-m', 'pip', 'install', 'spacy', '--no-deps']' returned non-zero exit status 1.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for crawl4ai
Failed to build crawl4ai
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (crawl4ai)

Hi,
Downgrade your pip version to 22.2 for instance and that should do the job. Seems that pip newer version is not compatible with the setup.py

ok i will try

did you downgrade to pip to 22.1.2 ? python -m pip install pip==22.1.2

Better to install a new clean virtual environment and then reinstall 22.1.2 pip version and from there go with Craw4AI installation. that should do the job.

@Sabakhupenia Sorry for issues you faced within the installation. In the new updat this issue has beem resolvedm I will push it soon. However at the mean time please try this and Let me know.

Virtual Environment:
First, create a new, clean virtual environment. This isolates your project dependencies and can help avoid conflicts.

python -m venv crawl4ai_env
source crawl4ai_env/bin/activate

Downgrade pip: (I don't believe you need this, however you can try)
In the new environment, downgrade pip to version 22.1.2:

python -m pip install pip==22.1.2

Modify setup.py:
The current setup.py file is trying to install spacy without dependencies, which might be causing issues:

from setuptools import setup, find_packages
import os
from pathlib import Path
import subprocess
from setuptools.command.install import install

# Create the .crawl4ai folder in the user's home directory if it doesn't exist
crawl4ai_folder = os.path.join(Path.home(), ".crawl4ai")
os.makedirs(crawl4ai_folder, exist_ok=True)
os.makedirs(f"{crawl4ai_folder}/cache", exist_ok=True)

# Read the requirements from requirements.txt
with open("requirements.txt") as f:
    requirements = f.read().splitlines()

# Define the requirements for different environments
default_requirements = [req for req in requirements if not req.startswith(("torch", "transformers", "onnxruntime", "nltk", "spacy", "tokenizers", "scikit-learn", "numpy"))]
torch_requirements = [req for req in requirements if req.startswith(("torch", "nltk", "spacy", "scikit-learn", "numpy"))]
transformer_requirements = [req for req in requirements if req.startswith(("transformers", "tokenizers", "onnxruntime"))]

setup(
    name="Crawl4AI",
    version="0.2.72",
    description="🔥🕷 Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper",
    long_description=open("README.md").read(),
    long_description_content_type="text/markdown",
    url="https://github.com/unclecode/crawl4ai",
    author="Unclecode",
<EMAIL_ADDRESS>    license="MIT",
    packages=find_packages(),
    install_requires=default_requirements,
    extras_require={
        "torch": torch_requirements,
        "transformer": transformer_requirements,
        "all": requirements,
    },
    entry_points={
        'console_scripts': [
            'crawl4ai-download-models=crawl4ai.model_loader:main',
        ],
    },
    classifiers=[
        "Development Status :: 3 - Alpha",
        "Intended Audience :: Developers",
        "License :: OSI Approved :: Apache Software License",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.7",
        "Programming Language :: Python :: 3.8",
        "Programming Language :: Python :: 3.9",
        "Programming Language :: Python :: 3.10",
    ],
    python_requires=">=3.7",
)

Installation:
Now, try to install your package. The default installation doesn't need "spacy".

pip install -e .

In case you need "spacy" and If you still encounter issues, you might need to install some dependencies manually:
pip install spacy --no-deps
python -m spacy download en_core_web_sm
pip install torch
pip install transformers

Let me know if you encounter any specific errors after trying these steps, and I'll be happy to help further.

hey, this bug is now solved! I installed it successfully! thank you for your kind appreciation! nice!


RENEE errors when attempting to deploy from git clone

When attempting to run RENEE from a git cloned repo, RENEE will not run:
Running with just renee command
[sevillas2@cn4333 RENEE]$ renee --help
bash: renee: command not found

Invoking bash prior leads to a file name too long error and a from command not found error
[sevillas2@cn4333 RENEE]$ bash renee --help
renee: line 12: RENEE: Rna sEquencing aNalysis pipElinE:
An highly reproducible and portable RNA-seq data analysises pipeline
About:
    This is the main entry for the RENEE pipeline.
USAGE:
	$ renee <run|build|unlock|cache> [OPTIONS]
Example:
    $ renee run --input .tests/*.R?.fastq.gz --output /data/sevillas2/RNA_hg38 --genome hg38_30 --mode slurm
: File name too long
renee: line 15: from: command not found
renee: line 16: from: command not found

Using ./renee looks for a directory that doesn't exist
sevillas2@cn4333 RENEE]$ ./renee --help
ERROR: Folder does not exist : /gpfs/gsfs12/users/sevillas2/resources

This also will not work due to #54

This may work now that #54 has been fixed

I'll test once we merge the other changes and see what happens

This is fixed!


Make "Field of Study" optional.

First of all thanks to all the contributors for building and maintaining this amazingly helpful tool.
Issue
Context
There are many developers, whose Field of Study is not technical. Hence, not important when they are applying for tech jobs. And sometimes adding it to the resume can turn out to be a negative point on the resume. Therefore, it is better to keep this field optional.
Current Behaviour
The "Field of Study" is mandatory inside the Education section.

Suggested Behaviour
The "Field of Study" should be optional inside the Education sections.
Assigning
If you also think that this should be done. Then, I will be more than happy to work on this issue.

Closing this in favor of Reactive Resume v3 release.
If you continue to face this issue, please raise a new one and I'll have a look at it as soon as I can.
In the meantime, if you are able to, please consider donating to keep supporting the project: https://www.buymeacoffee.com/AmruthPillai :)
Thank you!


'resolve' command issues

The resolve command in Cmdr has a conditional argument (provided as a function). With the way Nexus-Admin provides the command info, functions within the Args table will never be passed to the client (as per networking limitations). Thus the client always believes the resolve command only has 1 argument.
While there is an exception for the Run/ClientRun functions of commands, this exception is not used for arguments.

Possible (albeit hacky) solution to the problem is including the line
if ExistingCommand then CmdrCommandData.Args = ExistingCommand.Args end

Before the line
self.Cmdr.Registry:RegisterCommandObject(CmdrCommandData)

In the ClientRegistry script

Resolved in #18.


Add support for Python 3.7

Signed-off-by: Andy Dirnberger<EMAIL_ADDRESS>

This will require a release of Doozer 2.0.


What happens to existing connection when scaling in

If we set VMSS to be autoscaled by metric, and a scale in decision is triggered, before the VM got deallocated, what happens to existing connection or calculation on the VM

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b94c5948-acb1-e3aa-7148-6aa2278bcdef
Version Independent ID: 9ada224f-5a76-bcde-2dc0-928a29c4323f
Content: Azure virtual machine scale sets FAQs
Content Source: articles/virtual-machine-scale-sets/virtual-machine-scale-sets-faq.md
Service: virtual-machine-scale-sets
GitHub Login: @mayanknayar
Microsoft Alias: manayar

Thanks for the question. We are currently investigating and will update you shortly.

@CharmanderJieniJieni To understand better, can you please elaborate on your scenario a little more. For example, what do you mean by calculation on the VM. Thanks :)

@Karishma-Tiwari-MSFT , say we have a VMSS, all VMs inside perform a bunch of batch process which consumes high CPU usage, and we monitor by CPU metric to decide if scale out/in is needed.
When scale in condition got triggered and a VM is to be deallocated, what if there is still executing batch process in progress. Will that be lost.
Another case is for web app, if scale in got triggered and there are still http connections to the VM to be deallocated, will front end user see error such as 404?

@CharmanderJieniJieni Please refer to this stack overflow thread along the lines of your question: https://stackoverflow.com/questions/54520232/how-does-azure-vmss-handle-existing-request-while-scaling-down-or-scaling-in
Let me know if you have any other questions. Thanks :)

Thanks a lot!


hierachies of sections and paragraphs

This is a concrete suggestion to address the problem brought up in #83 . I've been running/testing this for about a week without problems. The updated grammar accepts a strict supset of the current one.
Does the project policy require new tests? If this is accepted, should there be a paralleling change in latex-lsp?

@martinra Thanks for the PR!

Does the project policy require new tests?

I think, the example from the issue (https://github.com/latex-lsp/tree-sitter-latex/issues/83#issue-1680004914) would be a good test case.

If this is accepted, should there be a paralleling change in latex-lsp?

Yes, I can make the change in texlab.


Custom tags where count.index is used (iteration)

I'm trying to generate various FQDN names that have different conventions.
I had a highly modified (from previous version) and I used the count and count.index to determine how many iterations to make and created the names accordingly.
Here are the variations:

ui.system.com
db01.ststem.com
db02.system.com
xyz001.xx.system.com
ax01.xyz001.system.com
ax01.xyz001.system.com

For example, the db and the ax are iterated on the host_name when count is greater than 1.
Here's what my tag block looked like inside the instance resource:
resource "aws_instance" "ec2_instance" {
  count = "${var.number_of_instances}"

...
  tags {
    Environment = "${var.track_environment}"
    Name        = "${var.number_of_instances > 1 ? "${var.host_name}${format("%02d", count.index + 1)}.${var.instance_name}" : "${var.instance_name}" }"
    Track       = "${var.track_name}"
    Desc        = "Instance for: ${var.number_of_instances > 1 ? "${var.host_name}${format("%02d", count.index + 1)}.${var.instance_name}" : "${var.instance_name}" }"
    LaunchGroup = "StartEC2Group${upper(substr(var.vpc_name,0,1))}${substr(var.vpc_name,1,-1)}"
    StopGroup   = "StopEC2atNight"
  }

I'm guessing, to keep this functionality, my only option is to modify this module; the main.tf file in resource "aws_instance" "this"
Thanks for any tips,
Chris.

Yes, you need to modify this module, because you need to access count.index which is available only inside resource block.

Thank you, Anton.
FYI, my modification looks like this, in case you or anyone else might have a thought about how to generalize this.
  # modifications
  # tags = "${merge(var.tags, map("Name", format("%s-%d", var.name, count.index+1)))}"  # original tag code

  tags = "${merge(var.tags,
    map("Name", "${var.number_of_instances > 1 ? "${var.host_name}${format("%02d", count.index + 1)}.${var.instance_name}" : "${var.instance_name}" }"),
    map("Desc", "Instance for: ${var.number_of_instances > 1 ? "${var.host_name}${format("%02d", count.index + 1)}.${var.instance_name}" : "${var.instance_name}" }")
    )}"
  volume_tags = "${merge(var.volume_tags,
    map("Name", "${var.number_of_instances > 1 ? "${var.host_name}${format("%02d", count.index + 1)}.${var.instance_name}" : "${var.instance_name}" }"),
    map("Desc", "Volume for: ${var.number_of_instances > 1 ? "${var.host_name}${format("%02d", count.index + 1)}.${var.instance_name}" : "${var.instance_name}" }")
    )}"

Thanks! Looks scary :) I normally (for 99.9% of cases) use auto-scaling groups and set EC2 tags from instance itself if necessary.

Haha.  Sorry.  I'm a nubie so perhaps it's overkill.  I get tired of looking at so many things in AWS and trying to figure our which system they're associated.  So I'm a bit obsessive trying to apply, for example, the actual fqdn of the instance in the Name field.
I am also using the iterative count mechanism to build multiple variants of the same instances.  So not every instance is using it's own unique resource definition.
While I'm developing for AWS now, I'm also trying to keep from going too deep into AWS, because we'll also need t support the same infrastructure on private cloud openstack.  If I can't create it in AWS and Openstack, I need to be careful what I use in AWS so I don't paint us into a corner.
Thanks so much for your help and advice.


helm package overwrites previous

helm version: 2.14
I am attempting to create a CD pipeline for Helm to deploy a private registry to AWS S3.
My directory structure is as follows:
/
├── charts/
│   ├── chart1/
│       ├── ...
|    ├── chart2/
│       ├── ...
├── dist/
│   ├── chart1-0.1.0.tgz
│   └── chart1-0.2.0.tgz
│   └── chart2-0.1.0.tgz
└── index.yaml

I have a deploy script as follows:
echo "Packaging chart updates"
helm package -d dist/ charts/chart1
helm package -d dist/ charts/chart2
#add additional charts here

# Update index
helm repo index .

echo "Syncing to S3"
aws s3 sync . s3://....

Concern: helm package will overwrite existing packaged versions, changing the digest in index.yaml (after helm repo index .).
Is this a bug? Would the digest change have any effect on gitops style orchestrators? And/or my expectation would be if nothing changed in charts/charts1 then the digest also wouldn't change but perhaps the digest is based off a timestamp as well as the contents?
TLDR: Should helm package check for existing?

Working as intended. The user knows best in this situation. If they want to override the packaged version, that's up to them. Other third-party packages like chartmuseum and harbor can take alternative opinions on this approach, as this is just an opinion of helm repo index.
Closing as intended.


upload: Don’t leak an extra body event handler for each edited message

Merged, thanks @andersk! I remember stopping a few of these from being introduced in the last big upload.js PR, but apparently didn't audit carefully enough for existing instances.


[Bug] json-rpc method call get_account_state_with_proof returns server internal error when given an old version

🐛 Bug
json-rpc method call get_account_state_with_proof returns server internal error when given an old version
To reproduce
Code snippet to reproduce
curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"get_account_state_with_proof","params":["00000000000000000000000000000001", 1, 1],"id":1}' https://premainnet.diem.com/v1 | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   356  100   240  100   116   1311    633 --:--:-- --:--:-- --:--:--  1945
{
  "libra_chain_id": 19,
  "libra_ledger_version": 363221945,
  "libra_ledger_timestampusec":<PHONE_NUMBER>074511,
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": -32000,
    "message": "Server error: Missing node at NodeKey { version: 1, nibble_path:  }.",
    "data": null
  }
}

Expected Behavior
DB raises a specific error that JSON-RPC service can check it is data pruned with message explains data is pruned, and response the error message without "server error"

you can downcast the error to see if it is missing_root_error.
https://github.com/diem/diem/pull/6882


Syntax Error in printable-characters.js

Line 12, zeroWidthCharacters, is not in key: value format. What is supposed to be there?

This problem exists in the latest version published on npm: 1.0.28. I'd recommend testing your modules before publishing.

The module is tested before publishing (via a continuous integration service), and the deployment process is fully automated by that service. You can see the test/build log here: https://travis-ci.org/xpl/printable-characters
So a broken build can't get to NPM, really.
The error you experience is caused by an outdated version of JS engine you use, which does not understand a modern JS syntax. In ES2015 we can use shorthand property notation:
var a = 'foo',
    b = 42,
    c = {};

// Shorthand property names (ES2015)
var o = {a, b, c};

See more here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer

If you still need the ECMAScript5 support (e.g. for older browsers), I'd suggest you to use Babel for transpiling your project: https://babeljs.io/

@xpl Hmm, I see. I have an issue with UglifyJS v2.8.26 which is less than 6 months old. You might want to consider waiting to use modern JS syntax until popular tools like that catch up.

@fresheneesz I expect people (who develop browser apps) to use Babel on their end, it's like an industry standard nowadays. Just add it before Uglify in your build script, and you can use all the modern syntax and forget about the compatibility problems.
But if you insist, I can add pre-compile the library for you, putting the pre-built version right in the repository. It will just take some time to configure the CI build script properly...

@xpl I don't want to have you change your package if its only to support someone not using best practices that are easy to do. I was just surprised this came up suddenly when I've never had a js version compatibility issue before. I would imagine that updating to newer versions of JS should start at the application level first, and only be required by things applications depend on (eg modules) after a certain amount of adoption has happened. I don't know the state of adoption of these things at this point.
So ok, but I'm trying to add a babel step to my webpack 3 config and getting this issue: https://github.com/simlrh/babel-webpack-plugin/issues/1 . Maybe I'm just using an old version of node (5.1.0), but its not that old. Iono, I would imagine if I'm getting this issue, others will probably too. I don't wanna insist, but it seems like this could be a major pain for me and people with similar setups. Just updating on my dev machine isn't gonna cut it. Gotta update our production machine too.

@fresheneesz The current (latest) version of Node is 8.5.0, and 6.11.3 is the officially recommended stable build. I would personally recommend 7.6.0+, though (because of the native async/await syntax support, which really saves time for asynchronous code). Try to update and see if it helps... Try also reinstall your node_modules (by deleting that folder and running npm cache clean --force) after the update, if problem persists.

@fresheneesz Ah, and seems also that the babel-webpack-plugin you trying to use is not a very popular tool.
Have you considered using the babel-loader instead (2k stars on github)?

@fresheneesz you can also try to switch from Uglify to a minifier that already supports ECMAScript6 syntax out of the box, like this one: babel-minify or uglify-es

@xpl Thanks for the suggestions. I got the babel-webpack-plugin to work on my local, but it eats up enough memory on the server to be killed by the out-of-memory-killer. : ( So does babel-minify-webpack-plugin. I feel like I've spent all day on this. Just to use your string.ify package..
Ok, I gave up on trying to add a babel step and instead wrote my own stringifier: https://github.com/fresheneesz/beautinator .


merge from arlac77/template-github,arlac77/template-kronos-component

package.json

chore(deps): add ^3.9.0 remove ^3.8.2 (devDependencies.ava)

Codecov Report

Merging #724 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #724   +/-   ##
=======================================
  Coverage   92.94%   92.94%
=======================================
  Files          12       12
  Lines         765      765
  Branches       86       86
=======================================
  Hits          711      711
  Misses         54       54

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95560d1...cdbf557. Read the comment docs.


Remote add -p not work with hub.protocol = https

I can't set ssh protocol for remote even if specifying -p option explicitly with hub.protocol = https.
For me, this is an unexpected. Is this a bug or expected behavior?
% hub --version
git version 1.9.1
hub version 2.2.9

# .gitconfig
[hub]
	protocol = https

% hub remote -v
origin  https://github.com/jsk-ros-pkg/jsk_recognition.git (fetch)
origin  https://github.com/jsk-ros-pkg/jsk_recognition.git (push)

% echo $GITHUB_USER
wkentaro

% hub remote add -p wkentaro

% hub remote -v
origin  https://github.com/jsk-ros-pkg/jsk_recognition.git (fetch)
origin  https://github.com/jsk-ros-pkg/jsk_recognition.git (push)
wkentaro        https://github.com/wkentaro/jsk_recognition.git (fetch)
wkentaro        https://github.com/wkentaro/jsk_recognition.git (push)

Of cource it works if I removed the config of hub.protocol from my .gitconfig, but what I want to do is setting https as the default and ssh with explicit specification.
This is because I want to use ssh my local PCs, but don't want to use ssh on servers or shared PCs because the sshkey is usually not registered to GitHub.

I'm sorry this was unclear. The -p never meant "use SSH protocol". It meant "use a writeable protocol" (as opposed to git: protocol). In the case then HTTPS is the default, -p does nothing and is deprecated. See https://github.com/github/hub/issues/1270#issuecomment-247323843
The solution for you is to do HUB_PROTOCOL=ssh hub remote add wkentaro.

This is unexpected behavior for me as well and the documentation seems to be at odds with what you're saying above. RE docs for git clone [-p]:

Use -p to select the ssh protocol unconditionally.

Is there a way to configure hub to default to the ssh protocol? I've tried...
$ git config --global hub.protocol ssh
...but that still prompts for my GH creds.
I'm not sure what I'm missing here. I've even tried...
$ HUB_PROTOCOL=ssh git clone 'user/repo'
...and this also prompts me for my GH creds. Any pointers would be helpful.

@wpcarro Is this the first time you use a hub command, by any chance? On first use, hub (not git) will ask you for your GitHub username and password so it can use the GitHub API. This happens before the repository has even started to be cloned, so it has nothing to do with git's own transport.
After that, HUB_PROTOCOL=ssh git clone user/repo should definitely use the ssh protocol for clone operation, provided that you run a fairly up-to-date version of hub.


leftwm: 0.2.11 -> 0.3.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/leftwm/versions.
meta.description for leftwm is: "A tiling window manager for the adventurer"
meta.homepage for leftwm is: "https://github.com/leftwm/leftwm"
meta.changelog for leftwm is: "https://github.com/leftwm/leftwm/blob/0.3.0/CHANGELOG"
Updates performed

Rust version update

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.3.0 with grep in /nix/store/1vfh2y8ppk3w5x161g2zzry52ydlpi9k-leftwm-0.3.0
found 0.3.0 in filename of file in /nix/store/1vfh2y8ppk3w5x161g2zzry52ydlpi9k-leftwm-0.3.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
leftwm

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/1vfh2y8ppk3w5x161g2zzry52ydlpi9k-leftwm-0.3.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A leftwm https://github.com/r-ryantm/nixpkgs/archive/35bb4e4accd18cbd62059fa26d12915b28a145bd.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/1vfh2y8ppk3w5x161g2zzry52ydlpi9k-leftwm-0.3.0
ls -la /nix/store/1vfh2y8ppk3w5x161g2zzry52ydlpi9k-leftwm-0.3.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.


Support timeline for Windows 32-bit

Raising to see how long we want to support Windows 32-bit and if there are particular needs motivating its support. Asking as this effectively doubles our CI time. So want to make sure that is an allocation that we are comfortable.
FWIW in conda-forge we did a poll earlier this year to see how many Windows 32-bit users we have and whether it made sense to support the platform further. We learned a very small percentage of users are on Windows 32-bit with most on 64-bit. It goes without saying that we opted to drop Windows 32-bit after learning this.

I'd support dropping 32-bit, especially if conda-forge has dropped it.
Speeding up CI would really benefit the development process.
On Tue, 6 Nov 2018, 18:54 jakirkham<EMAIL_ADDRESS>wrote:

Raising to see how long we want to support Windows 32-bit and if there are
particular needs motivating its support. Asking as this effectively doubles
our CI time. So want to make sure that is an allocation that we are
comfortable.
FWIW in conda-forge we did a poll earlier this year to see how many
Windows 32-bit users we have and whether it made sense to support the
platform further. We learned a very small percentage of users are on
Windows 32-bit with most on 64-bit
https://gitter.im/conda-forge/conda-forge.github.io?at=5ace39431130fe3d36c7da37.
It goes without saying that we opted to drop Windows 32-bit after learning
this.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/zarr-developers/numcodecs/issues/97, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAq8QlGUp7vVTEe5X0GPm1QwIFMv_lbwks5usdrUgaJpZM4YREJk
.

As to speeding up CI, enabling fast-finish for builds should also help. ( https://github.com/zarr-developers/numcodecs/issues/123 )

I'll drop 32-bit support in the next release (whatever comes after 0.6.1) unless anyone objects.

Dropping 32-bit is probably for the best.


gen_syscall_header_py is being run at the wrong time

gen_syscall_header.py is being run at cmake time, not at build time.
There seems to be an assumption that this action has no dependencies. This is not true. The script itself is a dependency.
The build system needs to re-run this script, when it builds the kernel, if the script itself changes. This used to work but broke with the new build system.

Another problem: if the script fails to run, the build doesn't break.


Android WebView: opt-in for handling of iframe urls

Fixes #18442
This is a PR to allow developers to revert back to WebView behavior in 0.48.4 regarding the loading of iframes. Currently Webview no longer detects the loading of iframe urls. And this is needed to detect and bookmark Google AMPs.
With the prop handleIFrameLoadingEvent, we can opt in for the firing of these events.
Test Plan
I tested the changes on WebView

without the parameter,
with the parameter at False,
and with the parameter at True.

It worked as intended. I can now retrieve AMP urls when the parameter is at True.

Related PRs
Related to ffbd3db which removed  the firing of the loading events. for iframes.

Yes. Below the PR for documentation change
PR269
Release Notes

[ANDROID] [BUGFIX] [WEBVIEW] - Added a prop handleIFrameLoadingEvent to Webview to allow opt-in for the firing of loading events for iframes (not only top level pages).

WebView PRs are on hold pending approval of the WebView proposal at https://github.com/react-native-community/discussions-and-proposals/pull/3

@ksegla thank you for this PR! 😄
Unfortunately, we can't land your changes this in repository. We're trying to move WebViews to https://github.com/react-native-community/react-native-webview in an effort to reduce the surface area of React Native. Do you mind re-opening this PR there? They're usually really quick to respond to PRs/issues.


Profile tlsproxy under heavy load

When proxy is under heavy load, you can do the following:

Open a tunnel

ssh -L 4000:localhost:4000<EMAIL_ADDRESS>
Grab the profile

go tool pprof -png http://localhost:4000/debug/pprof/profile > /tmp/profile.png

Awesome thanks @oxtoacart!

@oxtoacart did this.


Feature/289 make table compact

Description
Moves some column elements to the individual row tabs (specifically the 'Additional' column. Modified the node status column and some column headers to make the table more compact. Rearranged some columns as needed for improved U.I flow.
Fixes #289
Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Functional tests both locally and on esgf development server with production build settings. Visual inspection of all UI tweaks made. No changes made to backend or data related calls.

[ x] Local Pre-commit Checks
[x] CI/CD Build

Checklist

[ x] My code follows the style guidelines of this project
[x ] I have performed a self-review of my own code
[x ] My changes generate no new warnings
[x ] If applicable - I have commented my code, particularly in hard-to-understand areas
[x ] If applicable - I have made corresponding changes to the documentation
[ x] If applicable - New and existing unit tests pass locally with my changes
[x ] Any dependent changes have been merged and published in downstream modules

@sashakames reviewed and approved this PR.


Search by opcode

If you know opcode, do you have to search? Check out Emmet plugin for VSCode.

type "0XYZ:"
Ctrl + Space
opcode expands to command name


[PL-49148] Add option to Configure an optional registry mirror for delegate images

Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: Add option to Configure an optional registry mirror for delegate images
Jira/GitHub Issue numbers (if any): PL-49148
Preview links/images (Internal contributors only): https://6621719baf7a7614f8d642df--harness-developer.netlify.app/docs/platform/delegates/install-delegates/delegate-upgrades-and-expiration#configure-an-optional-registry-mirror-for-delegate-images

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[x] Successful preview build.
[x] Code owner review.
[x] No merge conflicts.
[x] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://6621719baf7a7614f8d642df--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66217312e6b63217dcb31f87--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://662173126d58fc15fc842f32--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://6621798e56a9811de89ed215--harness-developer.netlify.app


Correct the link to Homebrew on Linux GitHub issues

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] ~Have you written new tests for your changes?


Populate browser database from mock artifact

Write simple ETL scripts from a mock artifact's TSV files (project.tsv, files.tsv) to populate all tables

@calvinnhieu do you imagine this work will be reviewed in another ticket?

Ah, good point. I suppose this is being reviewed now with Colin's work. Will bring into QA/Review


Did you reproduce the paper result?

From the paper they sample a few good models.
Have you reproduced the similar results?

Thanks for following this repo. As mentioned in the "Result" section, we are currently running this code and waiting for the results, which are expected to be available by July 20 if everything goes well.
We are following configurations from AttentiveNAS. However, since we have reduced the number of running GPUs, the exact results may vary depending on the different hyperparameters. Therefore, we hoped that the changes made would have as little impact as possible on the training process.
Besides, if you use this code to run and get some results, we'd appreciate it if you could provide them via Pull Requests 👏.

Why it takes so long? I am going to train with this repo. I think the performance can be improved a lot if you use Nvidia Dali as the data loader.

Following AttentiveNAS we train this code for 5+360 epochs, and the sandwich rule requires 4 architectures to be sampled in each epoch, thus the total number of epochs reaches 4*(5+360)=1460. Other methods, such as calibrating BN running statistics, also incur time overhead in their computation.
DALI defines & uses its transform functions in the dali.Pipeline type. However, this code uses AutoAugment to build transformations for images, but the transformation functions provided by DALI cannot meet the requirements of AutoAugment.


docs: Add a new policy for bindings support

As discussed on Discord:

I / we are very grateful for the contributions of folks such as @vanillajonathan & @linux-china & @kasvith for adding bindings to PRQL
We're currently in a place where upgrading prql-lib is quite difficult, and @aljazerzen has spent lots of time on this in the last few days, because some of the bindings are initial early efforts
It's great to have these initial early efforts — everything starts out small before it becomes big
But we shouldn't gate the changes to PRQL on the compatibility of these yet, because it slows down PRQL development

So this establishes three tiers of support, and proposes to only gate PRQL changes on bindings that have sufficient support and tooling.
When we have more support & tooling, then we can promote these to "Supported"
Does this make sense to folks?

Sure, as long as there is al least one working binding so that we can can know that it is possible to write a binding against prql-lib. So with the Python and JavaScript bindings listed as supported, then as long as those bindings get updated upon any changes to prql-lib then that is good because then hopefully it should be possible to get the other bindings to work too.

So with the Python and JavaScript bindings listed as supported, then as long as those bindings get updated upon any changes to prql-lib

Good point. These don't rely on prql-lib — they use rust libraries to facilitate the bindings.
Probably we should have a support level for prql-lib there too. It's probably "Unsupported" but fairly close to "Supported"; do you agree @aljazerzen ?

I kind of would like one binding that uses FFI to be supported. Getting FFI done can be tricky, so if I see it has been done for at least one binding then it gives me faith that with enough trial-and-error I might somehow get it to work.

I kind of would like one binding that uses FFI to be supported. Getting FFI done can be tricky, so if I see it has been done for at least one binding then it gives me faith that with enough trial-and-error I might somehow get it to work.

I 100% empathize with this.
I think prql-java is the furthest along. If @zhicwu were up for merging his code, that would get us all the code we needed. Then I'm not sure whether @zhicwu or @linux-china would be up for maintaining it...

prql-lib does not have a build process established - I don't even know how it should be packaged, so I'd leave it at "unsupported".
For FFI example impl, look at php bindings, which are now at the level where all the functionality is implemented. But it also does not have a build process.


Front second sprint

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.996%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
493

Relevant Lines:
498

💛  - Coveralls


INSERT statements with many parameters: 'h' format requires -32768 <= number <= 32767

executing prepared INSERT statements with large numbers of parameters will fail with the following error message
the error message is

'h' format requires -32768 <= number <= 32767

this error is encountered for code that previously worked without issue with psycopg2 as the driver.
@elongl @jaklan and @misteliy surfaced this issue via https://github.com/dbt-labs/dbt-redshift/issues/347
this repo's "cousin" has some more context in this particular issue comments (below). My (limited) understanding is that there's a less fancy type inference happening related to the types of parameter sets passed to .execute()

https://github.com/tlocke/pg8000/issues/120#issuecomment-1288381859
https://github.com/tlocke/pg8000/issues/120#issuecomment-1293946586

possible solution
implement Cursor.setinputsizes(), which is currently stubbed out
https://github.com/aws/amazon-redshift-python-driver/blob/910be71314229e5b2febc60ddf8b8bcc992ea5f0/redshift_connector/cursor.py#L471-L476
perhaps pg8000's implementation could be useful here?
https://github.com/tlocke/pg8000/blob/f751b94a910975fd2e551814b7395336a5153cc8/pg8000/dbapi.py#L616-L629
    def setinputsizes(self, *sizes):
        """This method is part of the `DBAPI 2.0 specification"""
        oids = []
        for size in sizes:
            if isinstance(size, int):
                oid = size
            else:
                try:
                    oid = PY_PG[size]
                except KeyError:
                    oid = UNKNOWN
            oids.append(oid)

        self._input_oids = oids

Hey @dataders , thank you for opening this issue. The team will take a look into this and work on reproducing as well as evaluating how pg8000 addressed this fix.

@dataders Thanks for writing up this issue! Could you provide a reprex for this error?

Is this difference causing a performance impact in dbt?

This is a savvy question to understand the problem holistically @Brooke-white 🧠
I'm not sure the precise performance impact is currently, but I did some quick experiments within https://github.com/dbt-labs/dbt-redshift/pull/468.
It feels like the two most important impacts on dbt users currently are:

That the error is even possible
When it does happen, the error message doesn't help the user understand what went wrong

We've tried to mitigate how commonly it might happen in https://github.com/dbt-labs/dbt-redshift/pull/468, but it's still possible for dbt users to create a seed file that would raise this error.
And it's definitely possible for non-dbt users to raise this same error. I don't know about other folks, but it took me a fair of research to understand what the error message actually meant (and to be honest, I still don't totally understand 😅).
So raising a more understandable error message would be useful.
And figuring out how to make it less likely or impossible*, would be even better.

Thanks for sharing those details @dbeatty10 , 😄 .
Yes, I agree the error should not be possible & the error message should be easier to understand. But considering that I want to be cautious in implementing this method given that it is used in sqlalchemy, as tlocke mentioned in the linked issue by ensuring we have good test coverage in redshift-connector and sqlalchemy-redshift before merging.
I believe the following is a path to fixing 1 (feel free to add anything else you think of)

attempt plug/play with the pg8000 implementation
run redshift-connector's test suite and see what breaks
write tests for setinputsizes (likely can reuse much of pg8000 tests) for every datatype supported by Redshift (this means the datatypes we support via Postgres and the Redshift specific ones like GEOMETRY, GEOGRAPHY, SUPER, VARBYTE. FWIW we have a list of supported datatypes in type_utils file.

For fixing 2 I have a follow up question. As I haven't repro-d this myself, knowing exactly where this is thrown from redshift-connector could help us understand the options. If this is thrown by the driver, it is an easier case to catch it and repackage it with a more readable message than if it's given to us by Redshift server.

Hi @dataders and @dbeatty10,
Thanks to everyone for providing notes from your investigation into this issue—it was very helpful 😄
Here is my investigation so far,
Setinputsizes won't resolve the too many parameters limitation. To test: grabbed the prepared statement and parameters from the reprex usinginsert_data_bulk and the debugging stack right before it generated the error.
Executed the prepared statement against Redshift using pg8000, psycopg2, psycopg3, and redshift-connector.
Ex:
pg8000.exceptions.DatabaseError: {'S': 'ERROR', 'C': '42P02', 'M': 'there is no parameter $-32768', 'F': '../src/pg/src/backend/parser/parse_coerce.c', 'L': '279', 'R': 'coerce_type'}
All drivers generated a similar error except for psycopg2. I attempted to use setinputsizes with a very simple example: Insert 32768 rows of ints to a single column table. 32768 generates the same error with setinputsizes. 32767 executes successfully.
I attribute the root cause to the difference in protocols between psycopg2 and redshift-connector (+other drivers). Redshift-connector exclusively uses extended query protocol while psycopg2 only uses simple query. Pg8000 and Psycopg3 supports both. Psycopg2 merges parameters and prepared statement before it sends it to the database, see here. Data is sent as string literals in the query which is why it never encountered the parameter limitation.
The parameter limitation 32767 is enforced on the client and server-side:

On the server side (for postgres), there is a hardcoded parameter limit for prepared statements. See it here.
On the client side, the error is thrown when the driver attempts to convert the number of parameters to a signed 16-bit integer, causing overflow. So the database never receives the query. See redshift-connector, and similarly, pg8000.

It likely is not just a driver issue as I changed the conversion to an unsigned int16 (pack format=H), hoping that it would allow double as many parameters, however I got the same error for pg8000 to Redshift. I believe the extended protocol for PARSE and BIND limits the number of parameters.
As a baseline and quick fix:

I second @dbeatty10 @Brooke-white to make the error output more meaningful. Perhaps, classify it as DataError.

The fixes that require more time for implementation and testing can be broken down into two categories: low ~3-5 weeks and high time commitment ~2+ months. I still have to do some performance testing and will post my updates later. @jiezhen-chen

Woah! Your investigation yielded amazing details @soksamnanglim 🤩

fixed in 2.0.914

🏆 @Brooke-white
Is sounds like the new error message is something like this?
Prepared statement exceeds bind parameter limit 32767. 987654321 bind parameters were provided. Please retry with fewer bind parameters.

Hey @dbeatty10, yep that's correct! @soksamnanglim addressed this in #187 and we are now raising a DataError with that message.


Migrate all uses of -1 to kDynamicSize

The process for replacing uses of -1 was:

Find all occurrences of -1 (and - 1).
Read the context of the code to determine whether the use of -1 (or - 1) is appropriate.
Replace -1 with ShapedType::kDynamicSize where appropriate.

closes #327

We discussed with @ghpvnist that I will be continuing this work and will make the changes called out in the comments above. I've assigned the ticket to myself and will be closing this PR for now.


Some Demo's not working on datastar site

Hey,
Recently came across datastar, it looks super cool and seems like a fun way to build for the web.
I noticed some of the demos on the site aren't working for me on desktop (windows 11, chrome) or mobile (android, chrome).
https://data-star.dev/guide/getting_started#setall
https://data-star.dev/guide/getting_started#toggleall
https://data-star.dev/examples/scroll_into_view
P.S. very nice:

Thanks for reporting this! setAll() and toggleAll() have been fixed.
I’m seeing this error at https://data-star.dev/examples/scroll_into_view

Uncaught error scrollIntoViewKeyNotAllowed

This turned out to be an issue with the example, fixed in https://github.com/starfederation/datastar/commit/436e4ddbd818cf18034def47d37e9142c1de5b1e.


Improve CI testing and publishing

Testing

Run fuzzing tests in parallel. Will show red light correctly and be faster
Get cargo fuzz from: https://github.com/rust-fuzz/cargo-fuzz/releases/tag/0.11.2
Split up testing step by crates (eventually this should mean that it should only run tests for crates that have changed)
Have some sort of feature matrix for testing

need to check whether caching is working correctly
Publishing

Automatically add ezno-ast-generator if ezno-parser is updated

https://github.com/marketplace/actions/brndnmtthws-rust-action-cargo-binstall / https://github.com/brndnmtthws/rust-action perhaps?


AzureAD integration bypassed with --admin argument

The Azure AD integraiton works fine and we are prompted to login when we run kubectl after "az aks get-credentials" , but when you run "az aks get-credentials" with the --admin argument you can run kubectl commands without the AzureAd authentication prompt, this makes the whole AzureAD integration easily bypassed!  Isn't this a problem?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a47c6df7-5103-1336-eb0b-68f95155e6c8
Version Independent ID: f4420b08-9b57-f581-fcb4-f069afdd5b0c
Content: Integrate Azure Active Directory with Azure Kubernetes Service - Azure Kubernetes Service
Content Source: articles/aks/azure-ad-integration.md
Service: container-service
GitHub Login: @mlearned
Microsoft Alias: mlearned

you can restrict the user from pulling the aks credentials using admin flag by limiting the user to "azure kubernetes service Cluster user role" in IAM

@MicrosoftDocs/aks-pm Please add your comments

@skynetrulz Thanks for adding insights.

@khbafl
In order to successfully run az aks get-credentials with the --admin option, you need to have Cluster Admin Role privileges.
You shouldn't be delegating access to your cluster by granting Cluster Admin Role. Use of admin credentials should be limited to only when necessary.
As a general rule, you should be granting specified access to all AKS users by implementing RBAC using AAD identities, please refer this.
AKS user permissions within the cluster should be entirely defined by Roles and RoleBinding resources.
I hope this help clarify the limited use case for cluster admin credentials.

Ok I see, so to prevent the users from using the "--admin" argument to bypass AD Integration, we have to give them the "Azure Kubernetes Service Cluster User Role" IAM Role on the AKS ressource. Thank you,

Hope the provided information is helpful.
We will now close this issue. If there are further questions regarding this, please tag me in a comment. I will reopen it and we will continue the discussion.


fix: update custom chart data and type only

Description

Please include a summary of the changes and which issue has been fixed. Please also include relevant motivation
and context. List any dependencies that are required for this change.

custom fusion chart is refreshed on change of data to fixed this issue we remove chart config on componentDidUpdate

Fixes #11962
Type of change

Bugfix (non-breaking change which fixes an issue)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Test coverage results :test_tube:
:green_circle: Total coverage has increased
// Code coverage diff between base branch:release and head branch: feature/issue-11962

Status
File
% Stmts
% Branch
% Funcs
% Lines

:green_circle:
total
56.35 (0.01)
37.8 (0)
35.96 (0)
56.58 (0)

:green_circle:
app/client/src/selectors/commentsSelectors.ts
85.25 (1.64)
64.71 (2.95)
73.33 (0)
90.59 (2.35)

:red_circle:
app/client/src/utils/WorkerUtil.ts
89.76 (0)
70.59 (-1.96)
100 (0)
93.33 (0)

:red_circle:
app/client/src/utils/autocomplete/TernServer.ts
52.71 (-0.23)
40.83 (-0.84)
36.21 (0)
56.74 (-0.25)

:green_circle:
app/client/src/widgets/ChartWidget/component/index.tsx
50.9 (0.31)
23.23 (-0.48)
54.84 (1.71)
49.35 (1.25)

/ok-to-test sha=bfe03de

/ok-to-test sha=2a3f996

/ok-to-test sha=5080533

ok-to-test sha=783d399

/ok-to-test sha=602a5eb

/ok-to-test sha=f58a2be

@techbhavin Tested this PR and it is working as expected now. The chart doesn't refresh on choosing different data values on the fusion chart
https://www.loom.com/share/7c5e80c78c0b404ba3e04b88182db780


For each loop mutability

The following code is allowed by the typechecker:
for i in range(10) {
    print(i)
}

Obviously it mutates i but the typechecker still allows this.
We however should requirea mutable variable on the left side of in.

Oh, I thought I made i immutable, so only the loop can internally change its value...

It is true that i is immutable inside the loop, but still it feels like i is declared here and then mutated on each iteration.
This is not how it is implemented but it I thing we should force it to be immutable.
I mean languages that are purely functional don't have for each loops for exactly this reason, because i would get mutated.

It was mutable before... With the last commit I made it immutable. At least this is how Swift handles this


Refactor greatest and least Presto functions using simple function API

This PR is trying to resolve https://github.com/facebookincubator/velox/issues/3728
Refactor the greatest/least functions using simple function API so that it can be automatically vectorized.
The following changes are made:
(1) rename GreatestLeast.cpp to GreatestLeast.h since we are using simple functions API
(2) GreatestLeast Functions will support the following CPP Type: bool int8_t int16_t int32_t int64_t int128_t float double Varchar LongDecimal ShortDecimal Date Timestamp, which is the same as the previous GreatestLeast Functions before refactoring
(3) Now, GreatestLeast Function will also support NaN comparisons for DOUBLE and REAL data types. NaN is considered as the biggest according to https://github.com/prestodb/presto/issues/22391
As https://github.com/facebookincubator/velox/issues/3728#issuecomment-2027075925 points out we don't need benchmark for this change.
Thank you for helping review!

Aware of the compatibility issues. Fix in progress

The error happens during one of the circleci check
The previous signature is least(DECIMAL(precision,scale)...) -> DECIMAL(precision,scale) and greatest(DECIMAL(precision,scale)...) -> DECIMAL(precision,scale)
But current signature changed to least(decimal(i1,i5)...) -> decimal(i1,i5) and greatest(decimal(i1,i5)...) -> decimal(i1,i5)
I just investigated the errors, and I am not sure what's going wrong.
I also check the previous PR https://github.com/facebookincubator/velox/pull/9096, and it seems like the error also happens there in circleci check. @mbasmanova May I know if the error is "normal"? If not, do you have any recommended way for fixing it? Thanks

@Real-Chen-Happy The error you are seeing is a limitation of the CI check: https://github.com/facebookincubator/velox/issues/9240
CC @kgpai

@Real-Chen-Happy Thank you for the refactoring. Looks great overall. Some comments.
In Presto, greatest and least functions allow array and struct inputs as well. It would be nice to add support for these in a follow-up.

Sure! For clarification, do you mean adding these support in this PR or in a separate PR?

@mbasmanova Could you help review the updated code again? For this version, I make the following changes

Resolve nit issues mentioning in the code reviews (removing unused includes, rewriting the comments, etc)
Use call for the function instead of callNullFree
Assert the size of input is greater than 0 by changing registration code
Add support NaN comparisons for DOUBLE and REAL data types
Add unit test for NaN comparisons

@mbasmanova Could you help review the updated code again? For the third version, I make the following changes:

Fix the issues mentioned in the code review
Remove the VELOX_USER_CHECK
Just confim from https://github.com/prestodb/presto/issues/22391, NaN in DOUBLE and REAL will be considered as the biggest element in the greatest/least function.

Thank you!

Circle Check seems to throw another  error. The signature is changed from greatest(type...) -> type to greatest(type type...) -> type
presto_pr_signatures.json
I think it's because the registration code is modified from registerFunction<ParameterBinder<GreatestFunction, T>, T, Variadic<T>>(...) to registerFunction<ParameterBinder<GreatestFunction, T>, T, T, Variadic<T>>(...)
May I know if the compatibility error can be ignored? I am not sure whether the new signature will impact on the existing users. If it needs to be fixed, I think we can

use original registration code
use FOLLY_ALWAYS_INLINE bool call(...), and return false for any 0 size Variadic

Circle Check seems to throw another error. The signature is changed from greatest(type...) -> type to greatest(type type...) -> type presto_pr_signatures.json
I think it's because the registration code is modified from registerFunction<ParameterBinder<GreatestFunction, T>, T, Variadic<T>>(...) to registerFunction<ParameterBinder<GreatestFunction, T>, T, T, Variadic<T>>(...)
May I know if the compatibility error can be ignored? I am not sure whether the new signature will impact on the existing users. If it needs to be fixed, I think we can

use original registration code
use FOLLY_ALWAYS_INLINE bool call(...), and return false for any 0 size Variadic

@mbasmanova Thanks for reviewing. May I know if the compatibility error might be a concern this time? Or it’s fine?

May I know if the compatibility error might be a concern this time? Or it’s fine?

I think this is fine. See my earlier reply:
https://github.com/facebookincubator/velox/pull/9308#issuecomment-2029619013

@mbasmanova Could you help review the updated code again? For the new version, I make the following changes:

Refactor the tests

Thank you!

@mbasmanova Just updated the code to fix the nit issues. Thank you for helping review!
Also, may I know how to get invited to Velox Slack channel? I sent my request a week ago to<EMAIL_ADDRESS>but still not hear back anything yet. Let me know if there is anything I need to do prior to joining. Thanks!

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Thank you for the reminder @mbasmanova . @Real-Chen-Happy  I just sent you an invite. Can you check if you can access slack now?

@mbasmanova has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Thank you for the reminder @mbasmanova . @Real-Chen-Happy I just sent you an invite. Can you check if you can access slack now?

Yes I have the access now! Thank you so much! @sanumandla

@mbasmanova Just updated the code to rename the registration function. Thank you for helping review!

@mbasmanova Thank you again for helping review and fix errors. I am very excited to submit my first commit to Velox!

@Real-Chen-Happy Thank you for the contribution.


[BUG]  upgrade dependency of pulsar to enable continuous index

Describe the bug
Currenttly, denpendencies of pulsar are 2.8.0-rc-202012200040 which do not support continuouse index introduced by https://github.com/apache/pulsar/pull/9039.
To support this feature we should upgrade the dependencies to 2.8.0-SNPASHOT or an alternative release version which contains the commit of https://github.com/apache/pulsar/pull/9039.

2.8.0-rc-202012242230


[SQL Connector] Upsert pulsar testing

Figured out why 2 parallelism would fail in testTemporalJoin
This is because one of the partition is not generating data and doesn't proceed the watermarks.
Basically, we need to make sure  the hash distribution matches the Flink hash distribution in some Exchange.


remove rate limits and link to documentation overview

we should streamline where we mention specific rate limits: https://mapzen.com/documentation/search/api-keys-rate-limits/
and link to https://mapzen.com/documentation/search/api-keys-rate-limits/ instead
cc/ @rmglennon

Guessing you meant this for the 2nd link: https://mapzen.com/documentation/overview/#rate-limits


[] build failure

Host Environment

OS: [e.g. Windows/Linux etc...]
Compiler: revision

To Reproduce
Steps to reproduce the behavior:
./vcpkg install xxxx
Failure logs
-Cut and paste the appropriate build messages from the console output.
-Please attach any additional failure logs mentioned in the console output.
Additional context
Add any other context about the problem here, such as what you have already tried to resolve the issue.

Please provide the error log.

We don't have enough information to solve this issue, unfortunately; if you have any more information to help us solve this, please reopen!


CARDS-2611: Support for enforcing specific subject identifier formats

To test create a new Subject Type with:

pattern ^[A-Z]{2}\d{7}$
pattern hint 2 capital letters followed by 7 digits

Works great according to specs, but now that I see it in action I'm not a fan of the specs. The interaction is awkward with subject type before the identifier. We need to come up with a better flow.

@sdumitriu @marta- I am able to modify ID in composum in an invalid format. Strangely I ca add or remove digits, however removing or adding letters was not allowed; the value would go back to the original once I tried to e.g. add another letter.  Please see the screenshot:

I am still able to modify id in wrong format in the composum.


export_sourcedir plugin

This proposal will supersede #180 .
The plugin will allow the copying or symlinking of directories in the app to the output dir. This will allow the snakemake app to access contained files via relative files, solving the provenance issues caused by reinstalling the app in a different location
def export_sources(
    paths: str | Path | list[str | Path],
    *,
    root: str | Path = ".",
    timestamp: bool = True,
    checksum: bool = False,
    version: str | tuple[str, str, str] | packaging.version.Version | None = None,
    symlink: bool = False,
):
    """Recursively copy given directories and files into the output dir.

    Allows workflow to access files in the directories via relative paths. All exported
    paths must be subdirectories of the root, which defaults to the snakemake_dir given
    to SnakeBidsApp. Paths will be normalized relative to the root, then exported as-is.

    If exported files are already in the output dir, the export will only be performed
    on outdated files. By default, outdated files are determined by comparing the mtime
    of the previously exported file to the file in the app. If a ``version``
    specifier is used, checks for outdated files will only be made when the version
    changes.

    Parameters
    ==========
    paths :
        Paths of directories and files to export. Directories are exported recursively.
    root :
        Root of exported directories. All paths will be exported as relative to this
        root.
    timestamp :
        Allow the default behaviour of updating exports whenever the exported copy is
        older than the file to be exported. Defaults to ``True``.
    checksum :
        Find outdated exports by computing the file checksum. This is the most
        robust solution to deal with snakemake's automatic provenance, but may be slow
        for large files. If set, this takes precendence over ``timestamp``.
    version :
        A version string, tuple, or ``Version`` object to be recorded with the output
        data. Outdated exports will be updated only when the version changes.
    symlink :
        If ``True``, symlinks will be made to exported files, rather than copying. This
        method is faster when large files are being exported, but may not be compatible
        with some distributed modes of snakemake.
    """

I think this feature would be quite useful, but not a priority before 1.0.

See also #109, use of a plugin like this would probably supersede any reference to those variables


⚠ Jenkins Plugins Artifactory has degraded performance

In 17c19c9, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1279 ms

Resolved: Jenkins Plugins Artifactory performance has improved in ae6fa50 after 37 minutes.


Add concept of network units (NU)

Currently we have compute units (CU) and storage units (SU). It is clear however that we also need network units. The lack of these leads to certain primitives currently being free to use, while they obviously should not. Another side effect, as a result of #178, is that these primitives will now essentially live forever, unless manually decomissioned

The network unit concept was briefly discussed a while back. The idea is to be able to track how much network traffic is consumed by a user/workload. The definition of a NU was 1GiB of traffic.
This definition is very vague a requires to be extended. Some open questions:

What is the definition of traffic ?
is it all inbound and outbound traffic.
Is it traffic between nodes ?
is it traffic to public internet ?
What about the gateway ?
What happens when you run out of NU ?

Once we have answered these questions, Then we can start thinking about implementation and how to track the consumption.

@m-motawea please follow up here so we have an overview on what's needed to change in our side


[ITERATION_6] Update R24.5 RC Versions to latest stable versions

As team
I want finally check if we use the latest versions for shared service
so that we are 24.5 ready
Link

[ ] Trace-X https://github.com/eclipse-tractusx/traceability-foss/issues/762
[ ] IRS  https://github.com/eclipse-tractusx/item-relationship-service/issues/359
[ ] relates to https://github.com/eclipse-tractusx/item-relationship-service/issues/358

Hints / Details
Acceptance Criteria
Acceptance Criteria

[ ] IRS uses latest stable version of DiscoveryFinder in R24.5 https://github.com/eclipse-tractusx/sldt-discovery-finder/releases/tag/discoveryfinder-0.2.4
[ ] IRS uses latest stable version of Portal Services in R24.5 https://github.com/eclipse-tractusx/portal/releases/tag/portal-1.X.0
[ ] IRS uses latest stable version of Semantic Hub in R24.5 https://github.com/eclipse-tractusx/sldt-semantic-hub/releases/tag/semantic-hub-0.2.1
[ ] IRS uses latest stable version of EDC DiscoveryService in R24.5 https://github.com/eclipse-tractusx/portal-assets/releases/tag/v1.X.0
[ ]  IRS uses latest stable version of EDC 0.7.0 in R24.5
[ ] Environments are updated with new Versions (DEV, ..)

Out of Scope

Blocked: See ticket #358

Planning 2:

services and IRS implementations should be up to date already (except for EDC)
check if there are newer versions available

Acceptance Criteria

[x] IRS uses latest stable version of DiscoveryFinder in R24.5 https://github.com/eclipse-tractusx/sldt-discovery-finder/releases/tag/discoveryfinder-0.2.5
[ ] IRS uses latest stable version of Portal Services in R24.5 https://github.com/eclipse-tractusx/portal/releases/tag/portal-1.X.0 No other portal service other than DiscoveryService is used.
[x] IRS uses latest stable version of Semantic Hub in R24.5 https://github.com/eclipse-tractusx/sldt-semantic-hub/releases/tag/semantic-hub-0.2.3
[x] IRS uses latest stable version of EDC DiscoveryService in R24.5 https://github.com/eclipse-tractusx/portal-backend/releases/tag/v1.8.1
[x]  IRS uses latest stable version of EDC 0.7.0 in R24.5
[x] Environments are updated with new Versions (DEV, ..)

PO acceptance in behalf of @jzbmw


Update stable version in repo from 2.1.0 to 2.3.0

Think about update stable version to 2.3.0
Version: stable=2.1.0 (188 days old) => latest=2.3.0 (15 days old)
Installs: stable=2957 (57.01%), latest=299 (5.76%), total=5187
Click to edit

@sbormann
According to README.md this adapter need 2.3.0 to work properly with js-controller 5.
Release 2.3.0 is not yet at stable repository. As js-controller 5 will be released in the very near future, please create a stable PR to upgrade stabel release of this adapter to 2.3.0 to provide a js-controler 5 ready version.
If we do neither receive a PR nore any comment is added here stating that an upgrade of stable is not appropiate we might consider upgradung stable  to 2.30 (= current latest) anyway.
Thnaks for your support.

This issue seems to be outdated.
This issue suggests to update the stable version of this adapter to 2.3.0 but the current stable version is already 2.3.0.
So this issue should be closed.


Bugfix/2546 retesteth clock

PR description
Fixed Issue(s)

fixes #2546
expose a protected method to set/update the clock in TimestampBoundedByFutureParameter in order to support the mutable RetestethClock
Changelog

[ ] I thought about the changelog and included a changelog update if required.

Do not merge, there are outstanding issues with this change when running reference tests with retesteth engine

Making this change breaks many existing reference tests.  Closing for now.


feat: require safe-ds version 0.22.0

Summary of Changes

Require safe-ds version 0.22.0.
Handle new TabularDataset (former TaggedTable).

:tada: This PR is included in version 0.14.0 :tada:
The release is available on:

v0.14.0
GitHub release

Your semantic-release bot :package::rocket:


Add a dedicated file for Keras' security policy

Describe the feature and the current behavior/state.
Keras' security policy is currently at the very end of CONTRIBUTING.md. I suggest this information be moved to a dedicated SECURITY.md file.
GitHub treats SECURITY.md as a special file. If it exists in your repository, a new "issue type" is created that redirects users to the policy if they've discovered a vulnerability. The policy also appears in the project's Security Panel.
Alternatively, the policy can also be added to the https://github.com/keras-team/.github repository. This will be treated as the default policy for all projects in the keras-team organization.
Will this change the current api? How?
No.
Who will benefit from this feature?
Security researchers will have an easier time safely reporting vulnerabilities, therefore increasing Keras' overall security.

Do you want to contribute a PR? (yes/no): YES

Briefly describe your candidate solution(if contributing):

I will simply move the security policy to a SECURITY.md file at the root of the repository.
I would have preferred to send this PR to the keras-team/.github repository (therefore applying the policy to all repos), but since the repository is empty, I cannot create a fork and send a PR there.

@pnacht thanks for filing the issue, this sounds reasonable. Just approved the PR.


Config base url change to correct rss feed urls.

Closes #350
Small change to prevent double slashes '//' in RSS feed related urls.

Tested ACK.  Thanks!