Allow listing of inactive offerings

Description
This PR fixes https://github.com/apache/cloudstack/issues/8817 by allowing listing of Inactive offering ids to the user and changing state of service offering & disk offering using the update commands.

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[x] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI

Feature/Enhancement Scale

[ ] Major
[x] Minor

Screenshots (if appropriate):
How Has This Been Tested?

How did you try to break this feature and the system with this change?

@blueorangutan package

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 9007

@blueorangutan test

@vishesh92 a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[SF] Trillian test result (tid-9561)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45672 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8821-t9561-kvm-centos7.zip
Smoke tests completed. 129 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

@vishesh92 Please add the view changes in upgrade path, Rest changes LGTM.
The current listing behavior (with 'Active' state) is maintained in API / UI ?

Views are now updated from their individual view sql files @sureshanaparti . These are processed during upgrade.

@vishesh92 Please add the view changes in upgrade path, Rest changes LGTM.
The current listing behavior (with 'Active' state) is maintained in API / UI ?

Views are now updated from their individual view sql files @sureshanaparti . These are processed during upgrade.

Thanks for confirming @DaanHoogland

for the error message issue see #8882


dashboard/app: don't report crashes with revoked reproducers

We won't add a revoked reproducer (crash.ReproIsRevoked) to the report, but the fact that it's revoked does not affect reporting priority. As the result we can report a crash on very old kernel commit (frequently very negatively perceived).
When we mark crash as revoked reproducer, we should update its reporting priority as if it does not have a repro.

I might be missing the context of the problem here but I see that crash.ReproIsRevoked decreases the priority already:
https://github.com/google/syzkaller/blob/464ac2eda061918b0834afc83052d755176d25a1/dashboard/app/api.go#L910
and crash.ReproIsRevoked is not reported:
https://github.com/google/syzkaller/blob/464ac2eda061918b0834afc83052d755176d25a1/dashboard/app/reporting.go#L578
I've covered crash.UpdateReportingPriority prioritization logic with tests but I doubt that I understood the problem to try to solve it.
https://github.com/novitoll/syzkaller/commit/00b241059fea6dd6da1e9be4139ce129c1749d30

cc @a-nogikh IIRC we discussed this, and, yes, revoked reproducers correctly handled.
Close this?

If there are non-revoked reproducers, we will always prefer them to the revoked ones. And we do update the priority once we have revoked a reproducer. But if the only reproducer was revoked, we still give that crash a higher priority than other crashes, which never had a reproducer attached.
If "repro revoked" should have the same priority as "no repro", then we just need to update Crash.UpdateReportingPriority.


Working with rank-deficient EEGLAB data

So a colleague of mine was handed a set of files that had been pre-processed in EEGLAB. The data was read via mne.read_epochs_eeglab(). When computing the noise covariance via mne.compute_covariance() and visualizing the results via Covariance.plot(), we noticed that the data looked a bit odd. Turns out that during preprocessing, artifact rejection via ICA had been carried out, and some ICs had been removed. So the data was now rank-deficient. However, MNE didn't "pick this up" – in fact, mne.compute_rank() assumed the rank was full even with rank=None to force an actual estimation. And according to raw.info, obviously, the rank was full as well.
So I'm wondering how to best deal with this now. We could certainly pass rank=dict(eeg=true_rank) to all functions that require a rank input. But this seems cumbersome and error-prone, and I'm not sure if there are other places where this situation could lead to errors? I would prefer just being able to do something like, raw.rank = true_rank; or, even better, the EEGLAB import should do something of that sort. A way to detect rank deficiency after ICA is simply by checking if the number of sensors is equal to the number of ICs in the EEG structure:
data = scipy.io.loadmat('myfile.set')
n_comps, n_sensors = data['EEG']['icaweights'][0, 0].shape
if n_comps == n_sensors:
    print('Data has full rank')
else:
    print(f'Data has rank {n_comps}')

(Of course this only works if no (avg) reference has been set)
Any opinions on that?
cc @cbrnr

Also just looking at what MNE's ICA.apply() does, it also appears to me that a potential change of data rank is not "documented" anywhere. Is this not bound to lead to issues?

one intuition is that the EEGLAB data were stored to disk in single
precision and our rank detection code does not use
thresholds for this

one intuition is that the EEGLAB data were stored to disk in single precision and our rank detection code does not use thresholds for this

So you think that, under normal circumstances, rank calculation should work? Because I also remember that in the mne-study-template, we had to switch to using rank='info', because the default didn't work with some MEG datasets…

Can you really determine the rank of the channel data based on the number of ICs? Especially in the case where some ICs are zeroed out and the remaining ICs are back-projected to channel space, I'm not sure if determining the rank is that easy. How does MNE compute the rank of the data matrix? SVD?

yes MNE uses SVD and a clipping see
https://github.com/mne-tools/mne-python/blob/master/mne/rank.py#L21

@cbrnr

Can you really determine the rank of the channel data based on the number of ICs? Especially in the case where some ICs are zeroed out and the remaining ICs are back-projected to channel space, I'm not sure if determining the rank is that easy.

I'll be totally honest with you and say that I have no idea. All I can say is that when we used this approach, we got much more sensible-looking results with our data:
EEGLAB data noise cov estimated with rank=None

EEGLAB data noise cov estimated with rank=n_comps_from_EEGLAB_icaweights

I agree that in this special case it might make sense to infer the rank from the retained components (although strictly speaking, this is probably not the rank from a mathematical perspective). Can you quickly explain the plots to me? What is the noise covariance and what do these plots show?

Im my experience the # of ica components does not give a strong clue on the rank, it cam coincide with it but often it does not.

yes it's more related to the number of PCA dimensions that were used

@cbrnr

I agree that in this special case it might make sense to infer the rank from the retained components (although strictly speaking, this is probably not the rank from a mathematical perspective). Can you quickly explain the plots to me? What is the noise covariance and what do these plots show?

Based on epoched data, the covariance of the pre-stimulus periods was estimated via mne.compute_covariance, with the rank parameter set to either None or the number of retained ICs. The figures I posted are the SVD spectra plots based on these covariance estimates (generated via Covariance.plot()).
My layman's way of reading this plot is that the ordinate shows how much variance is explained by the data, and the abscissa corresponds to the sensors, ordered descending by explained variance. So as you move along the abscissa from left to right, once you reach the number of channels that is equal to the data rank, you will see a steep decline in explained variance.
In the first figure, there is an unusual step in this plot at around sensor ~50; investigation of the data showed that only 48 ICs had been retained, so we then set rank=48, which produced the second figure. And this one looks much more like what I would expect.

what was the PCA dimension used in eeglab?

@agramfort @dengemann
Bummer. So. How to proceed from here, then? @agramfort suggested this might have to do with floating point precision in the EEGLAB data set… how could we handle that?

what was the PCA dimension used in eeglab?

I don't think PCA dimensionality reduction was used, but I would need to check

please check and eventually sahre the eeglab file

I had a quick look at the preprocessing pipeline, which was based on PREP, and I couldn't find any hints that PCA was used.
I've shared the data with @agramfort, but if anyone else would like to take a look too, let me know.
This is a MWE used to derive.
(Plots look a little different because this is a different participant now, but the pattern is the same as what can be seen in the figures I posted above)
import mne
import matplotlib
from scipy.io import loadmat

matplotlib.use('Qt5agg')

fname = 'eeglab_epochs.set'
epochs = mne.read_epochs_eeglab(fname)

# Estimating covariance with rank=None
cov = mne.compute_covariance(epochs=epochs, tmax=0, method='auto', rank=None)
cov.plot(epochs.info)

# Estimating covariance with rank derived from number of retaind ICs
eeglab_data = loadmat(fname)['EEG']
n_components, n_sensors = eeglab_data['icaweights'][0, 0].shape
rank = dict(eeg=n_components)
cov = mne.compute_covariance(epochs=epochs, tmax=0, method='auto', rank=rank)
cov.plot(epochs.info)

Maybe because ICA is usually kickstarted with PCA this often (but not always) has the same effect on rank reduction like plain PCA?

Maybe because ICA is usually kickstarted with PCA this often (but not always) has the same effect on rank reduction like plain PCA?

I don't know… but, remember that 'pca' parameter from our attempts to find the MNE parameter set that's equivalent to EEGLAB's ICA? None of that was done during preprocessing of the data – no explicit dimensionality reduction.

This is a strange case. Those components are definitely lower amplitude than the rest, but being down by just two orders of magnitude does not indicate that they have been (properly) zeroed out numerically. So in some sense, whitening is doing something reasonable by boosting them. But it's understandable that the whitened data then are dominated by these components when you look at it.
On the other hand, if you really want those components gone and you do it by setting the rank (manually or if you have the code detect it automatically somehow), then implicitly you will project all of those lower-dimensional components into the null space of the data when you eventually whiten it or the forward operator with the rank-deficient noise-cov-based whitener. I think this ends up being mathematically equivalent to using those null-space components as SSP / projection vectors in the first place (which could be okay), except that you will have ignored any additional rotation of the space done by ICA. So if the desired outcome is truly zeroing out those N components, it might actually be better to take the components, orthogonalize them, and use them properly as SSP vectors in this case.
I also might be wrong about the interpretation here, hopefully going through the data with @agramfort at some point will be informative. My experience with ICA is very limited...

looking at the file I see:
icaweights: [48x63 double]
so a PCA of dim 48 was done before ICA. This explains cliff in eigen values
you see

looking at the file I see: icaweights: [48x63 double] so a PCA of dim 48 was done before ICA. This explains cliff in eigen values you see

Is that what it means? I thought it implies that 63-48 ICs have been discarded…
But if you're right, this would make things easy, no? Because then I could be certain that the rank actually is 48 in this case?

I just browsed a little through the PREP pipeline code and apparently it does things like:
        interpolatedChannels = ...
            EEG.etc.noiseDetection.reference.interpolatedChannels.all;
        channelsLeft = setdiff(referenceChannels, interpolatedChannels);
        pcaDim = length(channelsLeft) - 1;
        fprintf('%s: pcaDim = %d\n', EEG.setname, pcaDim);
        EEG = pop_runica(EEG, 'icatype', 'cudaica', 'extended', 1, ...
            'chanind', referenceChannels, 'pca', pcaDim);

So there we have it… maybe some channels got interpolated and then PCA was used to reduce data dimensionality before ICA was run.
Which is great in the sense that we may have an explanation of what happened here and how the data got to be.
But it also gets back to my original question somehow: Is there a way to tell MNE, ideally upon data import, about the rank of that data? Otherwise  I would have to manually  keep track of all rank-altering operations to pass the corrct rank=N parameter for covariance estimation and inverse operator assembly.

But it also gets back to my original question somehow: Is there a way to tell MNE, ideally upon data import, about the rank of that data? Otherwise I would have to manually keep track of all rank-altering operations to pass the correct rank parameter for covariance estimation and inverse operator assembly.

I think rank-checking is not an easy one. Relying on automation is never 100% safe. I've seen similar issues also in large-scale datasets such as LEMON from MPI-CBS. In the end manual rank-checking is doable: you compute covs, plot super-imposed all eigenvalue spectra and search for a common rank that make sense as a compromise. It's not too bad. I'd be of course happy if we can do better in terms of automated detection but the manual route is not too annoying in the meantime.

I think rank-checking is not an easy one. Relying on automation is never 100% safe.

I do agree, but in my specific EEGLAB example, it could have worked :) Don't you think we could try to cover such cases, and maybe emit a warning to let the user know that we're doing our best here, but might be wrong? And how would we store that information, anyway?

OR at least print a warning when loading such a dataset?

what do you mean such data? EEGLAB data which contain an ICA solution?
I would suggest to people to look at their covariance

Don't you think we could try to cover such cases, and maybe emit a warning to let the user know that we're doing our best here, but might be wrong? And how would we store that information, anyway?

I'd document how to do this by hand inspecting cov plots. I think a cool addition could be a convenience function making superimposed plots of SVD spectra, suggesting a common rank, or at least facilitating finding one visually. Imagine you plot 100 cov SVD spectra with some transparency to then see where the kink is gonna be in most of the cases, then you go for a lower bound. It's really fast and easy in the end.

I think a cool addition could be a convenience function making superimposed plots of SVD spectra, suggesting a common rank, or at least facilitating finding one visually. Imagine you plot 100 cov SVD spectra with some transparency to then see where the kink is gonna be in most of the cases, then you go for a lower bound. It's really fast and easy in the end.

Please ignore my ignorance here, but: which SVD spectra do you suggest to overlay? Those from multiple recordings / participants? Or those from a single recording, with covariance estimates assuming rank = [1, n_sensors] or something?

I'd document how to do this by hand inspecting cov plots.

Yes that would be helpful. I had no idea how these plots were supposed to look like until @agramfort told me some basics.
I think this is a difficult thing for quite a few users coming from EEG. In my previous lab, we'd use Cartool for source estimation, and it (or the LAURA algorithm it uses) doesn't require you to pass any kind of (noise) covariance matrix. In fact, you just plug in your evoked signals and you're set. So that's quite different (and more complicated) in MNE then… And those users then, like me, would have no idea about how to read these SVD plots.
In my experience, many researchers attempt doing their first steps with MNE by plugging in preprocessed data, because a motivation is to take advantage of MNE's the source reconstruction capabilities. That said, and given the popularity of EEGLAB, I think we should help users more by at least providing a useful hint when they import potentially rank-deficient data.

eeglab_data = loadmat(fname)['EEG']
if 'icaweights' in eeglab_data:
    n_components, n_sensors = eeglab_data['icaweights'][0, 0].shape
    if n_components != n_sensors:
        logger.info(f'It is likely that the rank of your data was reduced to '
                    '{n_comps} via PCA before running ICA in EEGLAB. Please '
                    'read https://mne.tools/wtf-does-that-mean to find more '
                    'information on what that means, and how do handle it.')

Also I'm still concerned about the fact that a user would have to manually keep track of rank-changing operations from that point onward. Can we not offer something like,
raw = raw.reduce_rank_via_pca(rank=123)

and be good?

Please ignore my ignorance here, but: which SVD spectra do you suggest to overlay?

Multiple subjects, such that you get a feeling for what is normal and where to suspect the real common rank.

I think this is a difficult thing for quite a few users coming from EEG. In m

It's a concern that goes far beyond source modeling.
The rank may also be important for certain predictive modeling pipelines which need explicit treatment of the rank, e.g, when using Riemannian embeddings.

providing a useful hint when they import potentially rank-deficient data.

Sure, why not as long as we have a clear criterion for detecting that.

Also I'm still concerned about the fact that a user would have to manually keep track of rank-changing operations from that point onward.

Things can get more complicated and cannot be easily predicted once head movements and other non-stationarities come into play, especially in MEG.
If you want to really be sure what's going on you must do it by hand and document things using plots.

Can we not offer something like (...) keep track of the new rank somewhere, and be good?

I am not sure I understand where you want to go. Why would you offer a function to reduce the rank (who needs to do this anyways knows how to do this)?

I think including a nice message specifically for EEGLAB data would be a great addition. I particularly like the URL - we should put our glossary or FAQ there.
Regarding keeping track of the rank, I think it is always best to just compute the rank yourself. You can use mne.compute_rank and maybe adapt tol to get a good estimate. I haven't used these rank calculations before, but would it be possible to estimate the threshold by detecting a steep drop in the eigenvalues (as opposed to an absolute threshold)? I.e. we look at the derivative and threshold that to get the rank?

I haven't used these rank calculations before, but would it be possible to estimate the threshold by detecting a steep drop in the eigenvalues (as opposed to an absolute threshold)? I.e. we look at the derivative and threshold that to get the rank?

Yes. But sometimes it's not so easy and corner cases are more frequent than it seems.
There can be noise, sometimes double kinks appear etc. or kinks are too smooth, often caused by non-stationarity. Therefore, I'd say a semi-automated approach is good compromise: use compute_rank / the cov plots (which do this internally) and then see if the estimates make sense. Correct if needed.

https://mne.tools/wtf-does-that-mean

@hoechenberger btw. I could not find that URL ... nice idea though.

@dengemann I agree! So this leaves the info message for imported EEGLAB data for this issue.
Is there an MNE tutorial about rank estimation and when rank is important? It sounds like this might be an interesting topic relevant to MNE users.

@dengemann

Please ignore my ignorance here, but: which SVD spectra do you suggest to overlay?

Multiple subjects, such that you get a feeling for what is normal and where to suspect the real common rank.

Ok, could be nice I guess :)
But in my specific use case, this wouldn't have helped, as apparently the rank deficiency I observed was introduced by interpolation of bad channels, followed by a PCA step before doing ICA. Since the number of interpolated channels can vary between participants, I don't think a "common" rank would make sense here.

providing a useful hint when they import potentially rank-deficient data.

Sure, why not as long as we have a clear criterion for detecting that.

I think the code snippet I provided above should be a reliable indicator for this specific use case. But I'd be more comfortable if some EEGLAB experts were to confirm this :)

Also I'm still concerned about the fact that a user would have to manually keep track of rank-changing operations from that point onward.

Things can get more complicated and cannot be easily predicted once head movements and other non-stationarities come into play, especially in MEG.
If you want to really be sure what's going on you must do it by hand and document things using plots.

Can we not offer something like (...) keep track of the new rank somewhere, and be good?

I am not sure I understand where you want to go. Why would you offer a function to reduce the rank (who needs to do this anyways knows how to do this)?

It's not so much about reducing rank per-se, but to inform MNE that the rank of the data has changed. You see, in the mne-study-template, for example, I'm explicitly passing rank='info' when estimating covariance and when creating the inverse operator. We did this after discovering that rank=None, i.e. rank estimation based on the data alone, didn't yield very usable results  with the data we had tested the pipeline on – switching to rank=info'fixed these issues. Now I haven't looked into the code to figure what _exactly_rank='info'` does, but my undersatanding is that takes into account projectors, reference channels etc. It would be good if a rank-reducing operating like PCA could be considered and treated accordingly within this existing framework.
@cbrnr

I particularly like the URL - we should put our glossary or FAQ there.

@dengemann

@hoechenberger btw. I could not find that URL ... nice idea though.

Haha, not sure the others would be too happy about it… 😂 But if there are no objections, I'd be the first to sign up!

don't think a "common" rank would make sense here.

In some contexts a common rank would be the minimum rank, deliberately inducing more rank reduction for some of the datasets.
Anyways, whether comparing subjects or not, cov plots are really amazingly useful.

Is there an MNE tutorial about rank estimation and when rank is important? It sounds like this might be an interesting topic relevant to MNE users.

@cbrnr I think some info can be found here and there. A tutorial on debugging rank issues could be really nice and save some user support time in the future ...

Just for the record, I just pulled @larsoner's #7736, and this is what I get:
>>> print(mne.compute_rank(epochs, tol=1e-6, tol_kind='relative'))

Computing rank from data with rank=None
    Estimated rank (eeg): 48
    EEG: rank 48 computed from 63 data channels with 0 projectors
{'eeg': 48}

… which is good!
So… merely a precision issue, like @agramfort suggested??

I didn't understand how that PR could fix it until I looked back at...

EEGLAB data noise cov estimated with rank=None

... I realized that I probably misunderstood this plot. Did you have some form of regularization on (e.g., shrunk) during estimation? If so it would explain why the first shelf is only 1e-2 or so below the real data (regularization inflated the eigenvalues). Can you do the same plot of the noise covariance estimated with empirical? You will probably see a much larger shelf at 48 (at least 1e-6 below the largest eigenvalue), and hence using a rtol vs atol like in #7736 could indeed help.

@larsoner Will get back to you tomorrow!

Okay, so I asked my colleagues to provide me with more data. And as is sometimes the case, things were a little messy – I couldn't exactly recreate the figure I posted initially, and it was unclear which dataset and version of MNE were used to generate it. Also turned out I got mixed up between data rank and noise covariance matrix rank rank at some point.
So I just started from scratch.
MWE
This is the MWE I came up with. Please find the example dataset here:
https://drive.google.com/open?id=13aiakuwp1HmJt-BYsRISCk6XcdO2INuf
import mne
from scipy.io import loadmat

mne.set_log_level(verbose='error')

# Load data
infile = '05_ICAr_newevents.set'

eeglab_data = loadmat(infile)['EEG']
eeglab_n_comps, eeglab_n_sensors = eeglab_data['icaweights'][0, 0].shape

epochs = mne.read_epochs_eeglab(infile)
epochs.apply_baseline()

tmax = 0

# Rank computations start here.
rank_epo = mne.compute_rank(epochs.copy().crop(tmax=tmax))['eeg']

try:   # Only works when running mne-python from larsoner:rank
    rank_epo_rtol = mne.compute_rank(epochs.copy().crop(tmax=tmax),
                                     tol=1e-6, tol_kind='relative')['eeg']
except TypeError:
    rank_epo_rtol = '—'

cov_empirical = mne.compute_covariance(epochs, tmax=tmax, method='empirical')
cov_shrunk = mne.compute_covariance(epochs, tmax=tmax, method='shrunk')

rank_cov_empirical = mne.compute_rank(cov_empirical, info=epochs.info)['eeg']
rank_cov_shrunk = mne.compute_rank(cov_shrunk, info=epochs.info)['eeg']

print(f'No. of channels:                     {epochs.info["nchan"]}')
print(f'1st dim of EEGLAB icaweights matrix: {eeglab_n_comps}')
print(f'Pre-stimulus Epochs rank (default):  {rank_epo}')
print(f'Pre-stimulus Epochs rank (rtol):     {rank_epo_rtol}')
print(f'Noise cov. (empirical) matrix rank:  {rank_cov_empirical}')
print(f'Noise cov. (SHRUNK) matrix rank:     {rank_cov_shrunk}')

Output on master
No. of channels:                     63
1st dim of EEGLAB icaweights matrix: 48
Pre-stimulus Epochs rank (default):  63
Pre-stimulus Epochs rank (rtol):     —
Noise cov. (empirical) matrix rank:  48
Noise cov. (SHRUNK) matrix rank:     63

Output on larsoner:rank (#7736)
No. of channels:                     63
1st dim of EEGLAB icaweights matrix: 48
Pre-stimulus Epochs rank (default):  63
Pre-stimulus Epochs rank (rtol):     48
Noise cov. (empirical) matrix rank:  48
Noise cov. (SHRUNK) matrix rank:     63

SVD Plots
The SVD plots are identical between master and larsoner:rank, as far as I can tell.
Empirical

SHRUNK

Consistency of results
I've tested this on 24 datasets, and the pattern of results is always the in all but one dataset: cov. rank (SHRUNK-regularized) is reported as full; while the empirical cov. rank is equal to the 1st dimension of the icaweights matrix and to the rtol-based rank. In one dataset, the computed rank deviates by 1 from the first icaweights matrix dimension (50 vs 51).
Additional information
I was also provided with a list of interpolated channels, and the number of interpolated channels is typically much smaller than the difference between the number of total channels and the first dimension of the icaweights matrix. Since, like I said, the PREP pipeline performs interpolation and rank reduction via PCA before ICA (based on the number of interpolated channels), I am inclined to believe that the icaweights matrix contains the ICs that are to be kept after PCA and (manual) IC rejection.

Output on larsoner:rank (#7736)
...
Noise cov. (SHRUNK) matrix rank:     63

This is expected because the rank internally is calculated still using atol not rtol, so this result should not change. Basically if you want rtol behavior or custom tol you should use rank = compute_rank(...) manually, then pass the rank to compute_covariance.
Automatic rank estimation will always have this sort of problem where you might miss the shelf. If we try to make it smarter we will almost certainly break some other things that currently work. So I think we just need to document that people need to examine the SVD spectrum of their empirical noise cov first, then ensure that the rank is preserved if they are using an automatic regularizer, because if the rank is not computed properly that automatic regularizer will inflate the "zero" eigenvalues.

This is expected because the rank internally is calculated still using atol not rtol, so this result should not change.

But this still doesn't explain why I (correctly) get

Noise cov. (empirical) matrix rank:  48

no? Or is the regularization in compute_covariance performed before calling compute_rank?

In the empirical case, it doesn't matter what you pass as rank in compute_covariance(..., rank=...) because it's not used, and the resulting matrix is unregularized. And then when you compute the rank of that matrix using rtol, you get 48.
In the shrunk case, when you do rank=None when computing the covariance, internally we estimate the rank of the covariance matrix using atol, which will mis-estimate the rank to be 63. The covariance is regularized in this dim-63 space, so then the eigenvalues that probably should get zero get inflated by the regularization. At that point it doesn't matter if you compute the rank of the resulting regularized matrix with atol or rtol because the regularization process has (errantly) moved those "should-be-zero" eigenvalues up to just ~1e-2 below "should-be-nonzero" eigenvalue levels -- you'll get 63 either way.
On the other hand, if you used the rtol method before regularizing the matrix by calling rank = compute_rank(..., rank_tol_kind='relative'), you will get back correctlry that rank = dict(eeg=48), and then when you pass this to compute_covariance(..., rank=rank, method='shrunk') it will correctly use the subspace of dimension 48 when regularizing, and then if you compute the rank of that matrix using rtol or atol (shouldn't matter at that point) they should both be 48.

Thanks for the explanation, @larsoner!
So I guess what this boils down to, in the end, is improving the docs?

Yeah I think so


Failed to execute visual testing in mobile native app with "appium:options" caps

Environment (please complete the following information):

Node.js version: v22.12.0
NPM version: 10.9.0
Browser name and version: [e.g. Chrome 68]
Platform name and version: MacOS 15.2
WebdriverIO version: 9.4.5
@wdio/visual-service version: 6.2.5
Config of WebdriverIO + @wdio/visual-service
An example of how you configured the @wdio/visual-service
capabilities: [
{
// capabilities for local Appium web tests on an Android Emulator
platformName: "Android",
//   browserName: "Chrome",
"appium:automationName": "UiAutomator2",
"appium:options": {
udid: "emulator-5554",
app: "app/app.apk",
fullReset: true
},
port: 4724,
},
],

Describe the bug
The visual testing run into an error if user used capabilities that have "appium:options"
To Reproduce
Steps to reproduce the behavior:
use the capabilities like above and run visual testing
Expected behavior
The test should run without any issues
Log
[0-0] unknown method in "Onboarding voice changer app.should onboarding successfully and open home screen"
unknown method: WebDriverError: Method is not implemented when running "execute/sync" with method "POST" and args "function hideScrollBars(hide) {
try {
document.body.style.overflow = hide ? 'hidden' : '';
}
catch (_e) {
// Do nothing
}
}"
at async beforeScreenshot (file:///Users/truongdinh/Appium/AzureAutopilot/node_modules/webdriver-image-comparison/dist/helpers/beforeScreenshot.js:36:9)
at async saveWebElement (file:///Users/truongdinh/Appium/AzureAutopilot/node_modules/webdriver-image-comparison/dist/commands/saveWebElement.js:44:34)
at async checkWebElement (file:///Users/truongdinh/Appium/AzureAutopilot/node_modules/webdriver-image-comparison/dist/commands/checkWebElement.js:22:57)
Additional context
Add any other context about the problem here.

Thanks, this will be fixed with this PR https://github.com/webdriverio/visual-testing/pull/688

@Dinh246
It's fixed in version 6.3.0


[SPARK-41332][CONNECT][PYTHON]  Fix nullOrdering in SortOrder

What changes were proposed in this pull request?
fix the wrong nullOrdering
Why are the changes needed?
existing implementation generate incorrect nullOrdering;
Does this PR introduce any user-facing change?
no
How was this patch tested?
added ut

Merged to master.


.zshrc

# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH

source ~/.bash_profile
#export NODE_ENV=dev
export NODE_ENV=test

# mysql path
export PATH=${PATH}:/usr/local/mysql/bin

# mongodb path
export PATH=${PATH}:/usr/local/mongodb/bin

# Path to your oh-my-zsh installation.
export ZSH=$HOME/.oh-my-zsh

# vscode
alias vsc='/Applications/Visual\ Studio\ Code.app/Contents/Resources/app/bin/code';
VSC_BIN='/Applications/Visual\ Studio\ Code.app/Contents/Resources/app/bin';
PATH=$VSC_BIN:$PATH;
export PATH;

# Set name of the theme to load. Optionally, if you set this to "random"
# it'll load a random theme each time that oh-my-zsh is loaded.
# See https://github.com/robbyrussell/oh-my-zsh/wiki/Themes
ZSH_THEME="robbyrussell"

# Uncomment the following line to use case-sensitive completion.
# CASE_SENSITIVE="true"

# Uncomment the following line to use hyphen-insensitive completion. Case
# sensitive completion must be off. _ and - will be interchangeable.
# HYPHEN_INSENSITIVE="true"

# Uncomment the following line to disable bi-weekly auto-update checks.
# DISABLE_AUTO_UPDATE="true"

# Uncomment the following line to change how often to auto-update (in days).
# export UPDATE_ZSH_DAYS=13

# Uncomment the following line to disable colors in ls.
# DISABLE_LS_COLORS="true"

# Uncomment the following line to disable auto-setting terminal title.
# DISABLE_AUTO_TITLE="true"

# Uncomment the following line to enable command auto-correction.
# ENABLE_CORRECTION="true"

# Uncomment the following line to display red dots whilst waiting for completion.
# COMPLETION_WAITING_DOTS="true"

# Uncomment the following line if you want to disable marking untracked files
# under VCS as dirty. This makes repository status check for large repositories
# much, much faster.
# DISABLE_UNTRACKED_FILES_DIRTY="true"

# Uncomment the following line if you want to change the command execution time
# stamp shown in the history command output.
# The optional three formats: "mm/dd/yyyy"|"dd.mm.yyyy"|"yyyy-mm-dd"
HIST_STAMPS="mm/dd/yyyy"

# Would you like to use another custom folder than $ZSH/custom?
# ZSH_CUSTOM=/path/to/new-custom-folder

# Which plugins would you like to load? (plugins can be found in ~/.oh-my-zsh/plugins/*)
# Custom plugins may be added to ~/.oh-my-zsh/custom/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git)

source $ZSH/oh-my-zsh.sh

# User configuration

# export MANPATH="/usr/local/man:$MANPATH"

# You may need to manually set your language environment
# export LANG=en_US.UTF-8

# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
#   export EDITOR='vim'
# else
#   export EDITOR='mvim'
# fi

# Compilation flags
# export ARCHFLAGS="-arch x86_64"

# ssh
# export SSH_KEY_PATH="~/.ssh/rsa_id"

# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
alias zshconfig="mate ~/.zshrc"
alias ohmyzsh="mate ~/.oh-my-zsh"
alias github_add_ssh="ssh-agent -s && ssh-add ~/.ssh/id_rsa_github"

export PATH="/usr/local/opt/icu4c/bin:$PATH"
export PATH="/usr/local/opt/icu4c/sbin:$PATH"

export PATH="$HOME/.yarn/bin:$PATH"

# nvm
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm

# alias for open app
# alias atom='/Applications/Atom.app/Contents/MacOS/Atom'
alias subl='/Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl'
alias code='/Applications/Visual\ Studio\ Code.app/Contents/Resources/app/bin/code'

C:\Program Files (x86)\Sublime Text3\sublime_text.exe
/mnt/c/Program\ Files\ (x86)/Sublime\ Text3/sublime_text.exe
D:\Program Files (x86)\Microsoft VS Code\bin
/mnt/d/Program Files (x86)/Microsoft VS Code/Code.exe


Trusted Types compatibility: script for identifying violations

As mentioned in the #103699 thread by @jrieken, there is a need for a tool that would help developers engaged in the VSCode project identify the Trusted Types violations. This can be achieved by adding tsec as a devDependency and an appropriate yarn script.

I have also added a task-definition so that tsec errors show in the editor itselfs


format Cargo.toml

Any thoughts on formatting the Cargo.toml with this VS Code extension?
https://marketplace.visualstudio.com/items?itemName=be5invis.toml
It puts groups of values in alphabetical order. Not indenting with make it easier to search.

I agree.


.dcf-form breaks

The "appearance: none" declaration in _components.forms.scss seems to break default style and functionality for range inputs. At minimum, I would recommend adding :not([type="range"]) to the initial css rule in the file. Other css rules may be needed to further support.

Thanks for filing this issue, @ncolgrove


FirefoxDebugSession: Failed stackTraceRequest: TypeError: Cannot read property 'getObjectGripAdapter' of undefined

ERROR|140.098|FirefoxDebugSession: Failed stackTraceRequest: TypeError: Cannot read property 'getObjectGripAdapter' of undefined

This happens 4 times out of 5 when I set a breakpoint in a function, that is executed deep down in an async chain. the funny thing, 1 time out of 5 it does work properly. When I turn "consoleLevel": { "default":  "Warn" } it seem to work more often. It almost looks like the extra time debug option takes to output info is enough for the adapter to load the needed object, that contains this getObjectGripAdapter property.
I probably can provide the test case, but it would require me to transfer like 1/3 of my current project into it, so if you can make sense out of it without - I'd rather not ^^.

Well, I can't make sense of it yet, but I'll try to reproduce it. Do you use Promises or classic callback-based async calls? Do you have any more observations as to when this happens and when it doesn't? Also, a log file (with "fileLevel": { "default": "Debug" }) may be useful.

there is nothing else related in the log. I have a bunch of WARN |004.706|PathConversion mainly regarding scripts, who's url contains query parts, which do not seem to be related. I use jQuery (1.11.2) .then. IIRC at some point jQuery tries to use Promise internally if the browser supports it, but I don't really know. Anyway if you use FF 49 and jQuery 1.11.2 you#ll probably end up with the same internls as me...

So far I have been unable to reproduce this, but I'll do some refactoring soon, especially on one class that I suspect is responsible for this bug (the class coordinates various asynchronous operations and hence is the prime suspect for any bug that is timing-sensitive). So I hope I'll catch this bug then.

We face the same problem in our current setup (Angular 2 application). For us this happens nearby in every interaction with the page, which makes debugging in VS Code via Firefox nearby impossible, whereas debugging in VS Code via Chrome (via Debugger for Chrome) works like a charm.
Let me know if I can provide more information or can do some testing for you to get this issue fixed.

OK, I gave my code another long, hard stare and may have found the issue.
Please update and tell me if it works for you now. If it does, you should see a new warning message in the diagnostic log starting with "Unexpected object grip of type ...". If you see this log message please post it here because it contains an undocumented object from the Firefox debug protocol that I would like to know about.

Well, it seems to work properly now and the debug session remains responsive. With the same breackpoints as before. However I'm not getting this "Unexpected object grip of type ..." anywhere, despite my logfile quickly growing to several MBs size.
Thank you!

Yes, it does work now. For me the debug session remains responsive, too.
I also get the log message that you've described:
INFO |078.991|FirefoxDebugSession: Thread server1.conn3.child1/27 paused , reason: breakpoint
WARN |079.474|VariableAdapter: Unexpected object grip of type symbol: {"type":"symbol","name":"Symbol.iterator"}
WARN |079.477|VariableAdapter: Unexpected object grip of type symbol: {"type":"symbol","name":"Symbol.iterator"}
WARN |079.478|VariableAdapter: Unexpected object grip of type symbol: {"type":"symbol","name":"Symbol.iterator"}

Does that help?

I also get those ERROR|079.760|ObjectGripActorProxy: No such actor. Quite a lot actually. However they seem to appear alongside with the actual script errors and do not hinder debugging process... at least not visibly.

@dotcs : yes, that helps. Now I know how to reproduce this and how to fix it properly.
The "No such actor" messages are actually about a bug in Firefox. I will change the log message to make this clear.

@hbenl : Awesome! Thanks for your support, keep up the good work and let me know in case I can do anything else to support you.


add credentials for pictures on homepage (again)

somehow they disappeared - add them again.
Picture of Maren: Tilman Vogler Photography
who is the photographer of the other one?

Done


Switch Component

Added switch component using Headless UI
Todo:

Need to fix some css
Change name to Switch from Toggle

This pull request is being automatically deployed with Vercel (learn more). To see the status of your deployment, click below or on the icon next to each commit.
🔍 Inspect: https://vercel.com/major-eth/yearn-web-lib/8S9NP8f3Ke7um1aCeofgRr7wdxMi ✅ Preview: Failed

Here is the deploy error :
src/components/Toggle/Toggle.tsx(61,14): error TS2769: No overload matches this call.
--
16:51:12.738 | Overload 1 of 2, '(props: { children?: ReactNode; color?: string \| undefined; slot?: string \| undefined; style?: CSSProperties \| undefined; title?: string \| undefined; ref?: ((instance: HTMLSpanElement \| null) => void) \| RefObject<...> \| null \| undefined; ... 254 more ...; onCheck: (enabled: boolean) => void; } & { ...; } & { ...; }): ReactElement<...>', gave the following error.
16:51:12.738 | Property 'onCheck' is missing in type '{ checked: boolean; "aria-hidden": "true"; height: number; width: number; }' but required in type '{ children?: ReactNode; color?: string \| undefined; slot?: string \| undefined; style?: CSSProperties \| undefined; title?: string \| undefined; ref?: ((instance: HTMLSpanElement \| null) => void) \| RefObject<...> \| null \| undefined; ... 254 more ...; onCheck: (enabled: boolean) => void; }'.
16:51:12.739 | Overload 2 of 2, '(props: StyledComponentPropsWithAs<"span", DefaultTheme, ToggleProps, never, "span", "span">): ReactElement<StyledComponentPropsWithAs<"span", DefaultTheme, ToggleProps, never, "span", "span">, string \| JSXElementConstructor<...>>', gave the following error.
16:51:12.739 | Property 'onCheck' is missing in type '{ checked: boolean; "aria-hidden": "true"; height: number; width: number; }' but required in type '{ children?: ReactNode; color?: string \| undefined; slot?: string \| undefined; style?: CSSProperties \| undefined; title?: string \| undefined; ref?: ((instance: HTMLSpanElement \| null) => void) \| RefObject<...> \| null \| undefined; ... 254 more ...; onCheck: (enabled: boolean) => void; }'.

Hey @phillip055, any updates here? If you don't have the time I can try and wrap this up, please let me know

Hey @phillip055, any updates here? If you don't have the time I can try and wrap this up, please let me know
following this up!


Fix rustic-cargo-new

Make it ask any path, not an existing directory.
Apply path expansion, otherwise "~/blabla" is passed directly to
cargo new, and it silently creates "~" directory in the current one.

I wonder why this actually works(without your changes). The sentinel shouldn't be able to read project-path, at least for me that's the case.

Well, it doesn't :)


async-executor v1.5.2 breaks the build

error: future cannot be sent between threads safely
   --> /home/uk/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rfd-0.11.4/src/backend/xdg_desktop_portal.rs:43:9
    |
43  | /         Box::pin(async {
44  | |             OpenFileRequest::default()
45  | |                 .accept_label("Pick file")
46  | |                 .multiple(false)
...   |
59  | |                 .map(FileHandle::from)
60  | |         })
    | |__________^ future created by async block is not `Send`
    |
    = help: within `[async block@/home/uk/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rfd-0.11.4/src/backend/xdg_desktop_portal.rs:43:18: 60:10]`, the trait `std::marker::Send` is not implemented for `*const async_executor::State`

...and more...

Thanks! The async-executor 1.5.2 got yanked (smol-rs/async-executor/issues/49), so we should be safe for now.
Will keep this open for now until the situation in async-executor issue is more clear.

We could close this issue if the async-executor release contained unintended, breaking changes.


moved bundles configuration into separate bundles.json file

@genadis I'm going to close this PR for now since we had an update to the bundle config. Once you get the CLA signed, I'd be happy to accept an updated PR to accomplish this.


Expressing default calling convention via Python API

What is the feature you'd like to have?
I would like to be able to set the default calling convention in a Python architecture plugin.
Is your feature request related to a problem?
It looks like without being able to set one, the transformation to HLIL gets confused and clobbers registers that shouldn't be.
Additional Information:
Per discussion in Slack: https://binaryninja.slack.com/archives/C0CVALTLN/p1717353782152439?thread_ts=1717172170.298609&cid=C0CVALTLN

Mason
I should also note that your R0 and R1 registers are clobbered, that is why you get a bare return
Looks like the python API can't express a default calling convention, which causes a cascading set of issues in the initial analysis. So no, not really something you did wrong. The analysis is able to pickup your calling convention but I think at a stage after the translation is already done, causing the clobbering.

Added in 4.1.5428-dev

Thanks!

@negasora How do you use this functionality? I don't see it documented...

To use a custom calling convention with a custom arch:
arch = MyArch.register()
cc = MyCallingConvention(arch, "my calling convention")
arch.set_default_calling_convention(cc)

A more in depth example will be done with #5532

Thanks! I'm going to update my architecture and evaluate the HLIL improvement.

Oh wow, the improvement is dramatic! The HLIL is still not perfect but it actually looks generally sensible most of the time, and I suspect I could improve this a lot by playing with the architecture---I might switch back to using subregisters as it seems possible that the issues I was getting were more related to the lack of default calling convention support than to the use of subregisters themselves.
Thanks so much!


User stopped getting unread message badges

A user today suddenly stopped getting unread message badges in excess of the unread messages they already had.

Messages displayed like normal.
Private Messages list would re-arrange like normal, but no badge appears
Mobile app displayed badges and updated like normal
Persisted across chrome and firefox, but only tested on Centos. Did not test desktop app
User had over 4000 unread messages, but reports that they've had over 5000 unread before with no issue
Some of these unread messages were quite old
Many of these unread messages were from muted streams
-zulip server 2.0.2

We resolved the issue by logging out, logging back in, and saying "Yes" to the "mark all messages as read" box. Once they were all marked, Zulip resumed normal badge updates.

This is almost certainly fixed in master.  The issue is that the webapp before 2.1 would fail to count very old unread messages because of how it fetched unread message data from the server, and we changed the webapp to use the mobile apps' unread messages database index since then.

Sounds good, thanks Tim!


HDDS-2381. In ExcludeList, add if not exist only.

What changes were proposed in this pull request?
Add a check if not contains for addDatanode, addPipeline, addContainer.
Think of a case we have one pipeline in the cluster.(3 nodes)
As now when we have a single pipeline, it failed we add to pipelineIds, and we get a new pipeline from SCM, now SCM by excluding it will try and if there are no pipelines, it will return the same pipeline. And after some time in the client, we got an exception, and we try to add this pipeline again. When it is being added again, we should check if it already exists do nothing. To avoid this, added contains check.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2381
How was this patch tested?
The check added will take care, and the log will not print same pipeline.

It seems the acceptance failed as it couldn't find the directory ../ozone/compose.
I will re-trigger the test.

/retest


Adds candidates page for November

First pass page of candidates for November election.

@rentry The search doesn't appear to be working the way it did on the primary candidates page.

@shannonmcharg ✔

The front matter key needs to be set to the the candidate column. It looks like it might have changed from 3 to 4. All good now!


commonmeta doesn't include member ids

Steps to reproduce:
cd /tmp
git clone https://github.com/front-matter/commonmeta.git
cd commonmeta
go build -o cm
./cm con
./cm convert "10.5555/12345678" --from crossref 2> cm-out.json

View 'cm-out.json'

The REST API provides a member key, "7822" in this case: https://api.crossref.org/works/10.5555/12345678

Fixed via 94914e9 and released v0.41 (also upgrading to go 1.23). If you have cloned the github repo, you can do
go run main.go convert "10.5555/12345678" --from crossref

and will see
"publisher": {
    "id": "https://api.crossref.org/members/7822",
    "name": "Test accounts"
  },


Save Makeflow artifacts outside of the container

The current behavior or issue
Unable to leverage the makeflow artifacts preventing detection of "no changes" scenarios and makeflow cleanup.
The steps taken to reproduce the behavior or issue, or specify a location where the steps were recorded
Run the workflow. The makeflow artifacts are not available
Expected behavior
There's a location where makeflow artifacts can be saved from run to run of container
Screenshots
N/A
Add other supporting information that may be useful
N/A
Completion criteria
Makeflow artifacts can be made available from run to run of container for:

detection of previous successful runs (makeflow doesn't cause work to be done)
the clean capabilities of makeflow are used by scif 'clean' app

** Technical notes**

For the "normal" case, another Docker mount at a specific location would allow data persistence. If the expected folder is not available, then it should be created with a warning message; note this results in non-permanent artifact storage.
For the DE environment, a possible approach is to use a subfolder in the working directory

By mounting to the data folder associated with the app that is run, makeflow artifacts are automatically saved. For example ... -v `pwd`/makeflow_artifacts:/scif/data/short_workflow ...

CyVerse: After conversing with @julianpistorius we've determined that we should make the artifacts available during processing and clean them up upon a successful run. This can be accomplished by specifying the working folder to point to /scif/data/<app name>. Specifying a shorter path name doesn't work since the CyVerse mount command replaces the folder and doesn't overlay it

Waiting on PR https://github.com/AgPipeline/drone-makeflow/pull/4 before making a new PR

PR: https://github.com/AgPipeline/drone-makeflow/pull/8


Fix updating user model instance, cant use update on individual object

Quick fix for invalid use of update on the user model. Instead, we iterate the final_map and use setattr() followed by a save().
Fixes #39

Do you know what's up with the tests?

I'm pretty sure GitHub actions were timing out yesterday. I'll rerun.

Ok, it should be gtg now. I had to push another commit to get it to snap out of it though.


[Bug]: Faking custom metadata key-value pairs in responses is not possible

Description
We use custom metadata attributes (e.g. these) extensively to automatically manage resource revisions. For example, we create assistants with a checksum  metadata property to verify a given assistant matches the definition in our source code to detect changes via the playground, and recreate the assistant in these cases, to avoid undefined behaviour in production.
Therefore, in test cases, I would like to be able to ensure a given code path will correctly check for the existence of an assistant; to do so, I fake an AssistantListResponse, and include the metadata key with the test checksum.
The response returned, however, will not include the metadata property. On further inspection, I discovered faking any metadata properties is not possible, due to the way the Fakeable::buildAttributes() function works. When encountering additional properties not defined in response fixtures, it will add them to the final response payload, but only if those properties are numeric. A side effect of that, however, is that metadata cannot be faked at all currently, despite it is supposed to be a dictionary of user-defined key-value pairs.
Considering the aforementioned numeric key check, which probably exists for a reason, should stay in place: Maybe a special exception could be made for metadata? That property exists on pretty much all OpenAI API resources, and is the only dictionary property I'm aware of. This could be as simple as:
  foreach ($override as $key => $value) {
-     if (! is_numeric($key)) {
+     if (! is_numeric($key) && $key !== 'metadata') {
          continue;
      }

      $new[$key] = $value;
  }

Steps To Reproduce

Fake a response, for example:$fakeAssistants = AssistantListResponse::fake([
    'data' => [ [
        'metadata' => [ 'foo' => 'bar' ]
    ] ]
]);

Verify the metadata array is empty:assert(
    !empty($fakeAssistants->data[0]->metadata),
    'Metadata is empty, despite "foo" being included in the fake call above'
);

OpenAI PHP Client Version
v0.9.0
PHP Version
8.3
Notes
No response

Hi @Radiergummi
Thanks for raising the issue and your comprehensive explanation. Issue fixed in https://github.com/openai-php/client/pull/414


[Bug]: LIQA currently crashes with most recent version of lifelines package

Contact Details
<EMAIL_ADDRESS>What happened?
We use LIQA for estimating affected transcript abundances. Currently, LIQA will crash with the python lifelines package > v0.27.0.
If you are experiencing this, please downgrade your lifelines package to 0.26.4
Version
1.0.0 (Default)
What platform are you working on?
Mac Intel
Relevant log output
Traceback (most recent call last):
  File "/panfs/home/yang4414/fryxx094/miniconda3/envs/selr2_test/lib/python3.9/site-packages/lifelines/utils/__init__.py", line 966, in pass_for_numeric_dtypes_or_raise_array
    raise ValueError("Empty array/Series passed in.")
ValueError: Empty array/Series passed in.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/panfs/home/yang4414/fryxx094/miniconda3/envs/selr2_test/lib/python3.9/site-packages/liqa_src/quantify.py", line 419, in <module>
    kmf.fit(Tforkmf, event_observed=Eforkmf)
  File "/panfs/home/yang4414/fryxx094/miniconda3/envs/selr2_test/lib/python3.9/site-packages/lifelines/utils/__init__.py", line 56, in f
    return function(model, *args, **kwargs)
  File "/panfs/home/yang4414/fryxx094/miniconda3/envs/selr2_test/lib/python3.9/site-packages/lifelines/fitters/kaplan_meier_fitter.py", line 127, in fit
    return self._fit(durations, event_observed, timeline, entry, label, alpha, ci_labels, weights)
  File "/panfs/home/yang4414/fryxx094/miniconda3/envs/selr2_test/lib/python3.9/site-packages/lifelines/fitters/kaplan_meier_fitter.py", line 354, in _fit
    (self.durations, self.event_observed, self.timeline, self.entry, self.event_table, self.weights) = _preprocess_inputs(
  File "/panfs/home/yang4414/fryxx094/miniconda3/envs/selr2_test/lib/python3.9/site-packages/lifelines/utils/__init__.py", line 906, in _preprocess_inputs
    durations = np.asarray(pass_for_numeric_dtypes_or_raise_array(durations)).reshape((n,))
  File "/panfs/home/yang4414/fryxx094/miniconda3/envs/selr2_test/lib/python3.9/site-packages/lifelines/utils/__init__.py", line 970, in pass_for_numeric_dtypes_or_raise_array
    raise ValueError("Values must be numeric: no strings, datetimes, objects, etc.")
ValueError: Values must be numeric: no strings, datetimes, objects, etc.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I have fixed the version of lifelines to 0.26.4 so that users do not need to worry about that.
Please sync current changes to pypi. Later, I will create github actions to upload PyPI automatically after we change something.

Changes synced to PyPi


-i and --image don't work?

Neither -i nor --image returns an image. Instead I see the password in plain text.
I'm on commit 1cc6ba2deac3855de32108fca0aa4db99371dc56. I'm using macOS 10.15.7 and the built in terminal.

Hi @OmgImAlexis,
It looks like you have misunderstood how to use the --image flag. In the description for the --help flag, it states that it

must be used along with --qrcode

I understand that this syntax may be confusing as the flags are not that intuitive. We therefore working on making it better in #22 based on this comment https://github.com/sdushantha/wifi-password/pull/22#issuecomment-770908185, where the syntax will look like this:

--show-qr: show qr code on the commandline
--save-qr image.png: save qr code into image file path
``: do neither.
--show-qr --save-qr image.png: do both


feat: use provider gas estimation for arbitrum

update umbrella SDK
update ethers to v5.7.2

Context
Turns out our gas metrics do not works well, so we will be using default metrics from provider.
I will remove alpha from versions before merge.
To-do list

[ ] Added tests
[x] Tested on dev/sbx
[ ] Changelog was updated with current changes
[ ] Documentation or guides were updated (slab, readme)
[ ] Version was updated on package.json (releases/hotfixes)

Checklist before merge

[ ] there are no errors in logs in any workers
[ ] new blocks are being discovered and marked as finalized
[ ] foreign blocks are being dispatched
[ ] squash all commits before merging

looks good, my only grief is why we have 3 styles of naming folders in services. we're using camelcase, pascal case, and lowercase/snakecase. we should standardize on one.

I was trying to do it but for some reason it make few workert to crash. I think it might be because of lower/upper case when dir name changed from SomeName to someName. I reverted change. I will merge as is. I will try to figure out this in the future.

Sounds good


underflow / saturation of unsigned list not working

Description
When subtracting a larger unsigned number from a smaller one the result is correctly 0.0. If both values are part of a list, the result is wrong, if an underflow occurs (seems to be set to the upper limit, instead of the lower one).
Minimal example:
from fxpmath import Fxp

# single unsigned value does work
a = Fxp(10, False, 14, 3)
b = Fxp(15, False, 14, 3)
c = a - b
print(c)  # 0.0 --> correct
c.info(verbose=3)

# unsigned list does not work
d = Fxp([10, 21], False, 14, 3)
e = Fxp([15, 15], False, 14, 3)
f = d - e
print(f)  # [4095.875 6.0] --> 4095.875 is the upper limit
f.info(verbose=3)

c.info(verbose=3) delivers the following output:

dtype           =       fxp-u15/3
Value           =       0.0
underflow       =       True
inaccuracy      =       True
Signed          =       False
Word bits       =       15
Fract bits      =       3
Int bits        =       12
Val data type   =       None
Upper           =       4095.875
Lower           =       0.0
Precision       =       0.125
Overflow        =       saturate
Rounding        =       trunc
Shifting        =       expand

f.info(verbose=3) this output:

dtype           =       fxp-u15/3
Value           =       [4095.875 6.0]
overflow        =       True
inaccuracy      =       True
Signed          =       False
Word bits       =       15
Fract bits      =       3
Int bits        =       12
Val data type   =       None
Upper           =       4095.875
Lower           =       0.0
Precision       =       0.125
Overflow        =       saturate
Rounding        =       trunc
Shifting        =       expand

Version infos

OS:          Windows 10 21H2 (19044.2130)
python:   3.7.6 (miniconda), 3.10.8 (mingw64)
fxpmath: 0.4.8

Hello @AndrzejKowalski9917
I tested your example in a fresh conda environment with python 3.7.6 (conda), fxpmath 0.4.8 (from anaconda) and I couldn't reproduce de error.
I also tested in a conda env with python 3.8.13 and fxpmath installed from Pip, and it's also working well.
Both runs were performed in linux, so I can not tell about relationship with Windows OS yet.
What version of numpy are you using?
Could you run again your example in a fresh conda env?

Hi @francof2a ,
thank you for your answer. I tried now in Linux (Arch) with

python 3.10.8
numpy 1.23.4
0.4.8

and also were not able to reproduce the error.  In Windows I also tried python 3.10.0, installed via the installer from python.org, fxpmath and numpy installed via pip (same versions as on Linux). Here the error occurs.
I can try a fresh conda env in Windows tomorrow at work.

Hello @francof2a,
I just tried at work again (Windows 10) with a clean virtualenv (conda env does not work here for reasons unknown).
Python is still version 3.7.6 and only the following packages are installed:

(test_env) D:\tmp\py>pip list
Package    Version
---------- -------
fxpmath    0.4.8
numpy      1.21.6
pip        22.3.1
setuptools 65.5.1
wheel      0.38.4

The error persists. So it seems to be an error specific to the Windows OS.


Ensure git history has some record of who triggered relevant GitHub Actions

Created from a comment on https://github.com/equinix-labs/metal-go/pull/171#pullrequestreview-1732643199:

Looking at the PR description, it may be helpful to include the user that triggered the action if that is known and can be included in the PR description template (@ tagging that user -- "This API Sync PR was automated through [action link] by @displague on 2023-11-08.").

The workflow_dispatch event payload includes a sender property that has the information we want: https://docs.github.com/en/webhooks/webhook-events-and-payloads#workflow_dispatch
In addition to capturing the sender in the PR description (and/or PR title) for sync PRs, we should also capture the sender for the release job (we could include that in the release description and/or the regenerate code for version <x> commit message).

This has been fixed in equinix/equinix-sdk-go and will not be implemented here.


DAG-DLT workshop preparation

We are organizing a workshop on DAG-based DLTs at https://icbc2024.ieee-icbc.org/workshop/dag-dlt.
### Tasks

Contacted a number of potential interested authors and invited speakers.

Finalized contacted list of potential interested researchers and previous participants.
Invited keynote speaker.

No more tasks related to the workshop for now

Finalized contacted list of potential interested researchers and previous participants.
Invited keynote speaker.

No more tasks related to the workshop for now


Project Default Namespace Regression?

I see this feature was added but it is not working for me: Use Default namespace if defined

Nvm, I see now, there are some options I can specify.


Mark current layout bold (in "View > Window layouts")

Please highlight current layout in the window layout selection menu entry.

@andfoy, please work on this one.
Also, make all strings in the Window layouts menu translatable (e.g. Vertical layout, Matlab layout, etc).

Hello @ccordoba12  and @goanpeca,
I implemented this feature but instead of marking in bold the selected layout, I used a radio button to represent the selection and  keep consistency with similar functionalities in Spyder (e.g. Source > Convert end-of-line characters). See the screenshot below.

However, there is still one issue. The selection does not persist across restarts because Spyder stores the selected layout only temporarily in memory. Do you have any suggestions about the best approach to persist this parameter across restarts?


Evaluate the possible use of Github Community forum for help requests

Feature Request
People are submitting help requests on Github tickets and not following the recommended way. In the meantime, there exists Github Community. Perhaps it could be an occasion to bind both with a redirection to Github Community forums. (I for one rarely hang on Ionic forums).
There already exists the Ionic forums and StackOverflow, so this might add some fragmentation. But perhaps something could be considered otherwise, like using exclusively Github Community pages (which not sure I'd do, though), or perhaps as a third place to ask.

Hi there! Thanks for the suggestion. I don't think we're going to go down that road for feature request. In the future, I think a better option would be to adopt a RFC process, where all feature requests can go through a detailed process of getting verified and all potential issues could be discussed before any code is written.
This will not only give the ionic team a clear idea of what a feature should do and the consequences of adding such a feature, but also act as a "design  doc" for community member who would like to submit a PR.
I think this in the long run would be the better/more maintainable way to discuss new features.
Thanks!

Alright then. The idea was more of having a place on Github to help people, not specially for feature requests, though.
Anyway, thanks for your attention.
Cheers.


Update Member section dividend prerequisites.

Still shows confirmed as a condition.

Cemented confirmed as the only condition.


UI Improvement: Show statistical and historical data on the web GUI

Hi @mhejrati.
I am working on this feature, I gonna create a CSV logger to store all of the objects and distances data in CSV files, then perform some statistical and historical analysis.

Hi @mhejrati.
I have worked on this issue. Some javascript modules are added to your web interface in order to visualize csv files. I will send a PR to you.


Dark Mode Not Available in Mobile View

Description of feature request, idea
Hey, The Dark mode is Not Positioned Correctly  in the Mobile View . How ever we can see it in desktop view , the Button is Set Positioned  in Mobile view .
Thanks :)

Add (multiple) screenshot links (if applicable)
No response
Add any other context about the feature request here
No response

Hey I would love to work on it


[WIP] Randomly generated default conn_seed

closes #356

Changed conn_seed default to None from 3
Updated docstring
Untested (probably will break)

Codecov Report

Merging #492 (89f34d6) into master (bab8196) will decrease coverage by 0.08%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #492      +/-   ##
==========================================
- Coverage   90.51%   90.42%   -0.09%
==========================================
  Files          18       18
  Lines        3415     3415
==========================================
- Hits         3091     3088       -3
- Misses        324      327       +3

Impacted Files
Coverage Δ

hnn_core/network.py
91.93% <ø> (ø)

hnn_core/parallel_backends.py
81.49% <0.00%> (-0.83%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bab8196...89f34d6. Read the comment docs.

looks like tests did not break 😄 You should update whats_new once done. Was this discussed already with @rythorpe and @ntolley ?

looks like tests did not break smile You should update whats_new once done. Was this discussed already with @rythorpe and @ntolley ?

Yeah, I was kinda surprised that all the tests passed. And yes, @mjpelah and I discussed this today! I suspect that none of our tests actually allow the default value (which is now None) to get passed to _drive_cell_event_times() - we'll need to test this and probably adjust some down-stream code.

looks like tests did not break smile You should update whats_new once done. Was this discussed already with @rythorpe and @ntolley ?

Yeah, I was kinda surprised that all the tests passed. And yes, @mjpelah and I discussed this today! I suspect that none of our tests actually allow the default value (which is now None) to get passed to _drive_cell_event_times() - we'll need to test this and probably adjust some down-stream code.

conn-seed is passed into _connection_probability where it is used as a seed to create the rng however, only if probability != 1.0. If not, it is defaulted to 3 (now changed to None)
If the seed is None (quoting from numpy.org): "then fresh, unpredictable entropy will be pulled from the OS," so the generator will produce different random numbers each time it is used.
The only place rng is used is in selecting a random subset of connections to retain:
new_connections = rng.choice(range(len(all_connections)), n_connections, replace=False)
@rythorpe @jasmainak Is this something we want to be different each time it is ran?
In the meantime, I will edit the test, making probability 0.99, and see if it runs.

Changed conn_seed default to None in _connection_probability.
Since _connection_probability is only called in add_connection, it will always already be defaulted. Therefore defaulting conn_seed again in _connection_probability is redundant.
It may be worth keeping though for ease of reading?

Changed conn_seed default to None in _connection_probability.

This looks good. I just noticed that I got conn_seed and event_seed confused in my previous comment - good catch. Unless anyone else objects (cc @jasmainak @ntolley), I don't think we need a unit test for this.

Since _connection_probability is only called in add_connection, it will always already be defaulted. Therefore defaulting conn_seed again in _connection_probability is redundant.
It may be worth keeping though for ease of reading?

I like it the way you have it. Go ahead and append a bullet point item to the "Changelog" section of whats_new.rst and don't forget to add your GH link at the end of the file!

I like it the way you have it. Go ahead and append a bullet point item to the "Changelog" section of whats_new.rst and don't forget to add your GH link at the end of the file!

Done!

This PR turned out to be very straightforward. Any final thoughts or critiques @jasmainak @ntolley @chenghuzi?

also set the PR title to [MRG] when ready from your end

Perfect, thanks @mjpelah ! Congratulations on your first ever contribution to hnn-core :-)


Generation fail when no PK is not found in 0.4.0

Hi,
generation in v0.4.0 with MSSQL driver fails instead of just giving warning stack trace:
<EMAIL_ADDRESS>[7:25:33 PM] Starting creation of model classes.
tedious deprecated The default value for `config.options.enableArithAbort` will change from `false` to `true` in the next major version of `tedious`. Set the value to `true` or `false` explicitly to silence this message. node_modules/typeorm-model-generator/node_modules/mssql/lib/tedious/connection-pool.js:61:23
Table sys_db_update has no PK.
Error occurred in typeorm-model-generator.
<EMAIL_ADDRESS><EMAIL_ADDRESS>If you think this is a bug please open an issue including this log on https://github.com/Kononnable/typeorm-model-generator/issues
(node:16882) UnhandledPromiseRejectionWarning: SyntaxError: Argument expression expected. (115:23)
  113 | toDelete:boolean;
  114 |
> 115 | @ManyToOne(()=>Import,import=>import.importHracs)
      |                       ^
  116 | @JoinColumn([{ name: "importid", referencedColumnName: "importId" },
  117 | ])
  118 |
    at t (project/node_modules/typeorm-model-generator/node_modules/prettier/parser-typescript.js:1:285)
    at Object.parse (project/node_modules/typeorm-model-generator/node_modules/prettier/parser-typescript.js:14:180461)
    at Object.parse (project/node_modules/typeorm-model-generator/node_modules/prettier/index.js:9739:19)
    at coreFormat (project/node_modules/typeorm-model-generator/node_modules/prettier/index.js:13252:23)
    at format (project/node_modules/typeorm-model-generator/node_modules/prettier/index.js:13510:73)
    at formatWithCursor (project/node_modules/typeorm-model-generator/node_modules/prettier/index.js:13526:12)
    at project/node_modules/typeorm-model-generator/node_modules/prettier/index.js:44207:15
    at Object.format (project/node_modules/typeorm-model-generator/node_modules/prettier/index.js:44226:12)
    at databaseModel.forEach.element (project/node_modules/typeorm-model-generator/dist/src/ModelGeneration.js:56:36)
    at Array.forEach (<anonymous>)
(node:16882) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:16882) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

but it runs fine on v0.3.5 just prints out:
<EMAIL_ADDRESS>[7:26:54 PM] Starting creation of model classes.
Table sys_db_update has no PK.
Error occured in typeorm-model-generator.
<EMAIL_ADDRESS><EMAIL_ADDRESS>
and generate all entities.
This probably re-introduce #54

Ahh I've just realized that this is probably caused by table that is named "import" in db schema ... so dumb name


[BUG] App: Can't create/modify entities with umlauts in string

Describe the bug
I can't create/modify entities with umlauts (e.g. ä ö ü) in a name in the app.
To Reproduce
Steps to reproduce the behavior:

Go to parts > actions > new category
try to create the category "Geräte"
See error

Expected behavior
Entity should be created/modified without error
Screenshots

Deployment Method
Docker
Version Information
0.5.0 dev

Thanks for reporting this @dermodmaster
Will have to look into this one for you :)

@dermodmaster are you using the app on iOS or Android?

@dermodmaster I've just uploaded 0.4.7 for release which fixes this issue. Normally takes a day or two to be available for download.
I'll close this issue out for now, if the new version does not address the issue for you please feel free to re-open and we can work on it further.

android and the problem is fixed. Thank you! :)


Unable to mirror local repository to another local repository

Hi,
After a fresh upgrade of my Redmine instance (v2.6.6) with Redmine_Git_Hosting plugin (v1.1.3), everything is working as excepted. There is still an error with the mirroring function (first time I try to use it).
I setup a mirror into the mirroring section of my repository. The mirror is in fact another repository onto the same server. Then I try to force the mirroring by clicking on the "Push" link. Redmine answers me with the following error :
Envoi des changements au miroir<EMAIL_ADDRESS>: Échec

Sortie :
Non-zero exit code pid 6517 exit 128 for `sudo -n -u git -i env GIT_SSH=/data/git/.ssh/run_gitolite_admin_ssh git --git-dir repositories/myrepo.git push --mirror<EMAIL_ADDRESS>
Indeed, I check if the admin key (the same as defined as the mirroring key) is configured on all projects with RW+ access. But it is not the case, only onto the gitolite-admin repository.
I try to add this key into my redmine instance (via my profile view), but Redmine returns me an error: "The key is already used for the gitolite admin key".
What can I do ?

Still have this error with Redmine_Git_Hosting 1.14. I can't make it work.
When I go into the administration panel, into the Test Configuration tab, I can read this :
Gitolite banner
hello redmine_gitolite_admin_id_rsa, this is git@XXXX running gitolite3 v3.6-2-g2d5c690 on git <IP_ADDRESS>

 R W	gitolite-admin
 R W	testing

I check this page : http://redmine-git-hosting.io/troubleshooting/
I try to apply the steps described into the section "Initialization of the repo with README file does not work", but it doesn't work for me. In fact, the change is correclty set manually, but then canceled when a user add a key.
Any idea on how to configure this key to have RW access on all repository ?

Ok, I found it. It seems it is not yet possible via Redmine_Git_Hosting to mirror 2 repository inside Redmine.
I edited lib/redmine_git_hosting/gitolite_wrapper/global.rb and comment code into remove_redmine_key:
81           # Delete the key
82           #repo_conf.permissions[0]['RW+'][''].delete(redmine_gitolite_key)
83
84           # We cannot remove this repository as it may contains other configuration that we didn't check.
85           # Instead add a dummy key so the repo_conf is still valid for Gitolite
86           # RW+ = <empty string> is not valid
87           #repo_conf.permissions[0]['RW+'][''].push('DUMMY_REDMINE_KEY') if repo_conf.permissions[0]['RW+'][''].empty?

Then restart Redmine.
Add the admin git into the @all section of gitolite.conf, and finally resynchronize keys via the admin panel of Redmine_Git_Hosting. Admin key should now have RW access to all.
Could it be possible to dissociate the mirror key from the admin key ? In the settings of each git repository, we could imagine having a checkbox to give write access to the mirror key.

Could it be possible to dissociate the mirror key from the admin key ?

This is already done for the next release.


DBZ-2499 Solve unknown event types

https://issues.redhat.com/browse/DBZ-2499

Unrelated test failure addressed via https://github.com/debezium/debezium/pull/2049

Unrelated test failure addressed via https://github.com/debezium/debezium/pull/2049


#40224

Specify Azure.Identity version as 1.12.1
Updated the Azure.Identity package version from a wildcard (*) to a specific version (1.12.1). This change ensures consistent dependency management and helps avoid potential issues with future versions.

no need to merge this anymore as azure issue fixed


[bug bash] Need more instructions on running and testing the bot with LUIS/QnA

Github issues for C# /JS / Java/ Python should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.
Sample information

Sample type: [\samples\ or \solutions]
Sample language: [dotnetcore or webapi or es6 or nodejs or typescript]
Sample name:

Describe the bug
There are few steps missing and really needed to be done before one could run the bot.  For c# bots, the fields for LUIS/QnA settings in appsettings.json file need to be populated.  For JS/Typescript bots, the same fields in .env file need to be populated.  Furthermore, we need examples for how the fields "QnAEndpointHostName",  "LuisAPIHostName" should look like.  I recall some needs "https://" prefix, some don't for host name, depending how the helper class forms the URL.
Without updating the fields properly, nothing could be tested.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
Give a clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
[bug]

To Reproduce
Steps to reproduce the behavior:
Build the application and then in chat emulator type any message . .
Using debugger , its failing at FlightBookingRecognizer.cs at below line:-
public virtual async Task RecognizeAsync(ITurnContext turnContext, CancellationToken cancellationToken)
where T : IRecognizerConvert, new()
=> await _recognizer.RecognizeAsync(turnContext, cancellationToken);
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

[bug]

Review of this sample and update of readme taking place as part of https://github.com/microsoft/botframework-sdk/issues/5897


Streamline series iterator creation

@juliusv I hope you'll like this. @fabxc In case you find time to have a look.
This will fix issue #1035 and will also help to make issue #1264 less
bad.
The fundamental problem in the current code:
In the preload phase, we quite accurately determine which chunks will
be used for the query being executed. However, in the subsequent step
of creating series iterators, the created iterators are referencing
all in-memory chunks in their series, even the un-pinned ones. In
iterator creation, we copy a pointer to each in-memory chunk of a
series into the iterator. While this creates a certain amount of
allocation churn, the worst thing about it is that copying the chunk
pointer out of the chunkDesc requires a mutex acquisition. (Remember
that the iterator will also reference un-pinned chunks, so we need to
acquire the mutex to protect against concurrent eviction.) The worst
case happens if a series doesn't even contain any relevant samples for
the query time range. We notice that during preloading but then we
will still create a series iterator for it. But even for series that
do contain relevant samples, the overhead is quite bad for instant
queries that retrieve a single sample from each series, but still go
through all the effort of series iterator creation. All of that is
particularly bad if a series has many in-memory chunks.
This commit addresses the problem from two sides:
First, it merges preloading and iterator creation into one step,
i.e. the preload call returns an iterator for exactly the preloaded
chunks.
Second, the required mutex acquisition in chunkDesc has been greatly
reduced. That was enabled by a side effect of the first step, which is
that the iterator is only referencing pinned chunks, so there is no
risk of concurrent eviction anymore, and chunks can be accessed
without mutex acquisition.
To simplify the code changes for the above, the long-planned change of
ValueAtTime to ValueAtOrBefore time was performed at the same
time. (It should have been done first, but it kind of accidentally
happened while I was in the middle of writing the series iterator
changes. Sorry for that.) So far, we actively filtered the up to two
values that were returned by ValueAtTime, i.e. we invested work to
retrieve up to two values, and then we invested more work to throw one
of them away.
The SeriesIterator.BoundaryValues method can be removed once #1401 is
fixed. But I really didn't want to load even more changes into this
PR.
Benchmarks:
The BenchmarkFuzz.* benchmarks run 83% faster (i.e. about six times
faster) and allocate 95% fewer bytes. The reason for that is that the
benchmark reads one sample after another from the time series and
creates a new series iterator for each sample read.
To find out how much these improvements matter in practice, I have
mirrored a beefy Prometheus server at SoundCloud that suffers from
both issues #1035 and #1264. To reach steady state that would be
comparable, the server needs to run for 15d. So far, it has run for
1d. The test server currently has only half as many memory time series
and 60% of the memory chunks the main server has. The 90th percentile
rule evaluation cycle time is ~11s on the main server and only ~3s on
the test server. However, these numbers might get much closer over
time.
In addition to performance improvements, this commit removes about 150
LOC.

Great - I'll try to review this in the next days!

Looks reasonable otherwise.

Thanks for the review. Will check ASAP.

Hope everything is clarified above. Now working on the "one iterator per (FP, offset) tuple" issue.

Thanks, all makes sense.

@juliusv There, I fixed it!

@beorn7 The commit description reads "Add test that exposes the problem.", but I can't see an added test?

Ugh... forget to git add... just a sec...

Cool :+1:

Thanks everybody. @juliusv  now on to https://github.com/prometheus/prometheus/pull/1424 ! :)
I might just merge the already reviewed https://github.com/prometheus/prometheus/pull/1426 once https://github.com/prometheus/prometheus/pull/1424 is reviewed.

#1424 still unreviewed, so I merge now.


serviceEndpoint not set when not providing mediatorConnectionsInvite to InitConfig

This issue is probably related to #381
When I start my agent for the first time and pass the mediatorConnectionsInvite to the InitConfig, it successfully connects to the mediator and everything works fine. However, if I restart my agent with exactly the same InitConfig configuration, my application hangs at the agent.initialize() call.
I can resolve this by removing the mediatorConnectionsInvite param from the InitConfig, this will make the agent initialize successfully and reconnect to the mediator. However, because I'm not passing mediatorConnectionsInvite to the InitConfig, serviceEndpoint in the invitation that is returned from agent.connections.createConnection() is not set to the mediator url.
In other words: I can only get a valid invitation object at the first run. Thereafter any agent I pass the invitation object to won't know how to reach my agent.
In my view the agent should start without problems when mediatorConnectionsInvite is set, regardless of any existing mediator connections. I think @TimoGlastra's suggestion in #381 makes sense. Furthermore, I think the serviceEndpoint value inside the invitation shouldn't depend on the InitConfig, but should instead be based on a MediatorRecord (if any).

@burdettadam could you take a look at this?

Yes, I will take a look at this. I think I see the problem.

Fixed in #419


Language server is not configured for filetype ``

Hi, I'm new to kak and kak-lsp, I'm trying to setup C completion on kak using kak-lsp, however I'm having a few issues :
I tried doing "kak-lsp -vvvv --session session" in a window and then "kak -s session file.c" + lsp-capabilities (in kak)
the kak-lsp window reported me this :
Feb 23 10:19:54.599 DEBG Language server is not configured for filetype ``, module: kak_lsp::session:80

I'm not sure what if I have configured something wrong or if this is a possible bug.
configuration files :
kakrc :
eval %sh{kak-lsp --kakoune -s $kak_session}
hook global WinSetOption filetype=(c|cpp|zig) %{
    lsp-enable-window
    lsp-auto-hover-enable
    lsp-auto-signature-help-mode
    lsp-auto-hover-insert-mode-disable
}

kak-lsp :
snippet_support = false
verbosity = 2

[server]
# exit session if no requests were received during given period in seconds
# works only in unix sockets mode (-s/--session)
# set to 0 to disable
timeout = 1800 # seconds = 30 minutes

[language.c_cpp]
filetypes = ["c", "cpp"]
roots = ["compile_commands.json", ".clangd"]
command = "clangd"

[language.zig]
filetypes = ["zig"]
roots = ["build.zig"]
command = "zls"

my environment :
OS : ArchLinux
kak : Kakoune v2020.09.01
kak-lsp : kak-lsp 9.0.0
clangd : clangd version 11.1.0
Thanks in advance.

Language server is not configured for filetype

Looks like Kakoune fails to detect the file type.
If it does, the first three lines of the -vvv log should be like
Feb 23 18:17:16.271 INFO Starting main event loop, module: kak_lsp::session:29
Feb 23 18:17:16.272 DEBG Searching for vars starting with KAK_LSP_PROJECT_ROOT_C_CPP, module: kak_lsp::project_root:46
Feb 23 18:17:16.273 DEBG Routing editor request to Route { session: "foo", language: "c_cpp", root: "/home/johannes" }, module: kak_lsp::session:95

Can you try launching Kakoune without other configuration - for example
$ HOME=$(mktemp -d) kak -s session file.c -e 'echo %opt{filetype}'

The :echo %opt{filetype} should display "c" in the prompt.
Kakoune's file.kak should be active and set this file type automatically. It requires you to have the file program installed (which is part of base-devel).

Here is the full logs of kak-lsp (-vvv) (it gets printed if I run lsp-capabilities) :
Feb 23 18:30:31.340 DEBG From editor:
session  = "session"
client   = "client0"
buffile  = "/home/infrandomness/Documents/Epitech/tek1/mysokoban/src/main.c"
filetype = ""
version  = 1
method   = "capabilities"
[params]
, module: kak_lsp::editor_transport:122
Feb 23 18:30:31.341 DEBG Language server is not configured for filetype ``, module: kak_lsp::session:80

You should first debug without kak-lsp, because the problem seems to be the wrongly-detected filetype.
Open file.c and run :echo %opt{filetype} - if this does not print c, then we need to fix that.
:set-option window filetype c might fix it for the active buffer, but that should happen automatically.
I believe if you launch kak-lsp manually, it keeps running; I'm not sure.

I believe if you launch kak-lsp manually, it keeps running; I'm not sure.
it exited when I exited kak

so the filetype is detected and correct

OK; then :lsp-capabilities must produce a log statement like filetype = "c" (as opposed to your previous filetype = "")
Also when you open the file you should get a line containing Routing editor request to Route { session: "foo", language: "c_cpp" in the log.
Assuming the above statements hold - which means that kak-lsp works with a temporary home - my guess is that there is something in your other config that disables the filetype.
Are you setting disabled_hooks? Did you override the autoload directory?
Try commenting everything in your kakrc except for the kak-lsp bits to narrow it down.

Are you setting disabled_hooks?

No

Did you override the autoload directory?

All the things you see in the codeblocks of the first message is all of what I've got, and everything looks to be related to kak-lsp, I don't know what to comment

I think I found it - lsp-auto-signature-help-mode is wrong, it should rather be lsp-auto-signature-help-enable
First rule of Kakoune debugging: check the *debug* buffer for errors :face_with_head_bandage:

the buffer was empty, I have checked multiple times

If you run :lsp-auto-signature-help-mode it won't be empty.

I have changed it, but it is still not detecting the filetype

Okay it wasn't the wrong command.
Here is a Dockerfile with your config. This works for me - run kak test.c - the log will be in ~/log.
FROM docker.io/archlinux:latest

RUN pacman -Sy
RUN pacman -S --noconfirm kakoune kak-lsp clang

RUN groupadd -g 1000 user \
  && useradd -p $(openssl passwd -1 fish) -d /home/user -m -u 1000 -g 1000 user

USER user
WORKDIR /home/user

RUN mkdir -p ~/.config/kak/ && printf '%s\n' >~/.config/kak/kakrc \
	'eval %sh{kak-lsp --kakoune -s $kak_session}' \
	'set global lsp_cmd "kak-lsp -s %val{session} -vvvv --log /home/user/log"' \
	'hook global WinSetOption filetype=(c|cpp|zig) %{' \
	'    lsp-enable-window' \
	'    lsp-auto-hover-enable' \
	'    nop TODO the below command is wrong, but it does not seem to matter' \
	'    lsp-auto-signature-help-mode' \
	'    lsp-auto-hover-insert-mode-disable' \
	'}'

RUN mkdir -p ~/.config/kak-lsp && printf '%s\n' >~/.config/kak-lsp/kak-lsp.toml \
	'snippet_support = false' \
	'verbosity = 2' \
	'[server]' \
	'# exit session if no requests were received during given period in seconds' \
	'# works only in unix sockets mode (-s/--session)' \
	'# set to 0 to disable' \
	'timeout = 1800 # seconds = 30 minutes' \
	'[language.c_cpp]' \
	'filetypes = ["c", "cpp"]' \
	'roots = ["compile_commands.json", ".clangd"]' \
	'command = "clangd"' \
	'[language.zig]' \
	'filetypes = ["zig"]' \
	'roots = ["build.zig"]' \
	'command = "zls"'

RUN printf '%s\n' >test.c \
	'void bar() { }' \
	'void foo() { bar(); }'

This works on my side too, logs : https://paste.opensuse.org/5435998

Yes - the point of the container is that it works the same way anywhere.
What you'd need to do know is find out what is different between your system and the container.
Maybe you have overridden the file command or something.
A basic sanity check would be to run kak somefile.c -e 'echo %opt{filetype}' - I'm still not sure if that correctly prints c when you are not using a temporary $HOME.

kak somefile.c -e 'echo %opt{filetype}'

Prints nothing

Ok, I think we're close.
The detection is implemented in file.kak, so what is the output of
file -b --mime-type -L somefile.c

wait, so.. I've uninstalled kakoune and kak-rls using pacman -Rs (delete package, configuration, ..), deleted the config files in my home folder, reinstalled them using pacman -S kakoune kak-rls and then rewrote the configuration files, and it looks like it works now..

file -b --mime-type -L somefile.c

text/x-c

Cool - please reopen if you can reproduce.
(on a second thought, the filetype is actually set by /usr/share/kak/rc/filetype/c-family.kak, not file,kak)

So uh, I re open this issue because this is happening again on my other computer I've just installed fedora (33) on, and I ran all the steps above, doing all the steps above did not resolve the issue, the exact same problem is happening where kak-lsp prints :
Feb 23 10:19:54.599 DEBG Language server is not configured for filetype ``, module: kak_lsp::session:80

I do not understand how to solve this problem once and for all

As said above, this is not an error in kak-lsp - your %opt{filetype} is wrong. Try to narrow down the differences between the non-working setup and a working one.

I'm not sure what's going on here, I have the same configuration files both in a docker container, and it works (I've copied from here : https://github.com/kak-lsp/kak-lsp/issues/462#issuecomment-784518338), but on my host it doesn't

Weirdly I'm facing the same issue too. Same error, but my :echo %opt{filetype} is set correctly, but :lsp-capabilities also doesn't return anything. The debug buffer shows four detected config changes (which did not happen)
My config is located in .config/kak-lsp/kak-lsp.toml and I installed it via the aur, because it failed via plug (127 - whatever that means)
kakrc.txt
kak-lsp.toml.txt
I have absolutely no clue what could be causing this.

@UltraBlackLinux I can't reproduce, your config works for me on a C file.
Consider providing a full reproducer (maybe a docker file).
Also, it's unlikely that the information in this thread will help us here, so it's better to open a separate issue.
One problematic thing about your config is that you call both lsp-enable and lsp-enable-window, which results in redundant diagnostics etc.; you only need one.
I also recommend trying the default kak-lsp.toml (i.e. just delete yours)

I'm experiencing this exact error, and :echo %opt{filetype} (and the visibly working syntax highlighting) indicate that the filetype is detected just fine. I'm open to the possibility that my init.kak is otherwise broken, but kak-lsp is the only thing that's misbehaving in any way I can detect.

Notably, it seems to produce this message on files where the LSP does work as well, so it's not actually correlated with the issue I'm debugging. Specifically, I'm debugging my Haskell LSP, but Go produces the same message despite the LSP working fine there.

After replacing the naked lsp-enable with hook global WinSetOption filetype=* lsp-enable it seems to go away, and everything that worked before still does. Presumably filetype is initialised later, so it's still unset when lsp-enable is called at init time?

@edef1c Does it happen with the default config?
Can you give steps to reproduce? It should be easy to comment out stuff from your init.kak to find a the minimal config that exposes the problem.

hook global WinSetOption filetype=* lsp-enable

This is an odd setup, you should either run lsp-enable once, or lsp-enable-window in all windows you care about.
It still works but every new lsp-enable adds errors to the *debug* buffer.
Also filetype=* looks like a mistake; it will only ever match the empty filetype (e.g. *scratch* buffer, or plain text files).
You want something like filetype=.* (or filetype= if you want to match the empty filetype)

I'm starting to think that  Language server is not configured for filetype ``
is a legit warning. It happens if you run LSP commands in a buffer with the empty filetype.
Since 15d29970 (Show unconfigured filetype errors in editor, 2022-07-22) we show it in the
editor, but only if the request does not come from a hook. This means that you are probably
running some LSP command interactively.

Presumably filetype is initialised later, so it's still unset when lsp-enable is called at init time?

Which file does it happen one and how do you run lsp-enable?
Usually, a BufCreate hook takes care of setting filetype foo, which triggers a WinSetOption=foo hook to enable LSP.
So there should be no ordering issue.
Regardless, this should really not be related to this problem. It should work the same way with lsp-enable.


[Bug] Run analysis and dependency commands in the same container / pod

There are several performance advantages to doing this:

we currently share m2 directory between containers to speed things up, but that doesn't work on mac
when we decompile a JAR, we end up decompiling twice because analyze command updates the base path (to run things faster), but dependency command does not

This was completed in #175


V2 Get-EXOMailboxStatistics seem to have recently broken at the endpoint

Receiving a REST Protocol error. This worked flawlessly since Ignite and only recently broke within a month.
Get-EXOMailboxStatistics -ExchangeGuid <ToString() of GUIDs passed in a foreach loop>
Get-EXOMailboxStatistics : An error occurred while processing this request..
ProtocolError: (:) [Get-EXOMailboxStatistics], Exception
One or more errors occurred., Microsoft.Exchange.Management.RestApiClient.GetExoMailboxStatistics

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: de2836a0-0af3-c147-2d4a-78220ac4e0f1
Version Independent ID: 2cde9735-6281-bb9f-bf07-b30b2f6c7f37
Content: Exchange Online PowerShell V2
Content Source: exchange/docs-conceptual/exchange-online/exchange-online-powershell-v2/exchange-online-powershell-v2.md
Service: exchange-online
GitHub Login: @chrisda
Microsoft Alias: chrisda

@kevinblumenfeld, please ask this question at exocmdletpreview[at]service[dot]microsoft[dot]com (email address intentionally obscured to help prevent spam).
The EXO V2 module is still in Preview, and that email address is the best place to get answers.
Closing

I have done so.  Thank you @chrisda