Better support for DE in scverse

Continuation of a discussion in https://github.com/theislab/single-cell-best-practices/issues/141
Goal
Implementation of useful helper methods for common DE tools. We don't plan to reimplement several DE methods. Other people are trying this already.
Tools
We could implement wrapper functions that make it streamlined to run DE tools within the scverse ecosystem. The output of the tools should be in the right slots of the AnnData object.
Most important

wrap basic methods from scanpy (wilcoxon, t-test, ...)
wrap  pydesq2, see also https://github.com/owkin/PyDESeq2/issues/15

Other options
Disclaimer: while the rpy2 code snippets should work in principle, I haven't used them extensively and am not sure if they follow the best practices of the respective tool.

edgeR (through rpy2) [code snippet]
MAST (through rpy2) [code snippet] -> should additionally support LME models.
Limma (through rpy2)
statsmodels linear model wrapper [code snippet]

Input specification
I envisage something like
def de_analysis(
    adata: AnnData
    groupby: str,
    *,
    formula: Optional[str]
    contrast: Optional[Any]
) -> pd.DataFrame:
    """
    Perform differential expression analysis

    Parameters
    ----------
    adata
        single-cell or pseudobulk AnnData object
    groupby
        column in adata.obs that contains the factor to test, e.g. `treatment`.
        For simple statistical tests (t-test, wilcoxon), it is sufficient to specify groupby. Linear models require to specify
        a formula. In that case, the `groupby` column is used to compute the contrast.
    formula
        model specification for linear models. E.g. `~ treatment + sex + age`.
        MUST contain the factor specified in `groupby`.
    contrast
        TODO
        See e.g. https://www.statsmodels.org/devel/contrasts.html for more information.
    """

Clearly, the challenge here is to find an interface that works with both simple methods like t-test and the more flexible linear models. It will also always be a tradeoff between simplicity and the possibility to use all features of the underlying methods (e.g. do we want to support arbitrary contrasts? Do we want to support multiple contrasts for a single model fit?).
Some of these cases could require splitting the function up into a fit and a compute_contrast step. In that case an OOP interface might be more appropriate.
Output specification
Each DE tool should output a data frame with at least the following columns

gene_id
log2 fold change
mean expression
unadjusted p-value
adjusted p-value

and optionally other columns (may depend on the method).
Plots

Add volcano plots from decoupler/wrap them (plot_volcano)
Note: an (optionally) interactive version based on plotly/altair could be really nice -- hover over the dots to view the gene name.

Add paired or unpaired scatter+boxplots are nice ways of visualizing individual genes (each dot is a pseudobulk-sample)
[code snipped based on seaborn]

Add bar charts. Scatterplot on top for paired observations. Each dot represents the fold change for one paired observation.
[code snipped based on altair]

heatmap of coefficients with FDR correction
[code snippet based on altair]

@grst what else do you think we should add? Feel free to edit my issue (hope you have the rights now - if not please ping me).

Tools
Personally, I never made good experiences with rpy2 -- which is one reason why I stopped using these functions in more recent projects. Instead I performed DE in R and connected it through a workflow manager.
Are you sure you want to support that? It would also entail quite complicated CI setups if we were to test it.
Maybe we start with pydeseq2, t-test, wilcoxon test?
Plots
another nice one I forgot, which is especially useful when comparing between multiple conditions:

Visualization frameworks
My plotting functions are a mix of altair and matplotlib. Do you want to stick to one of them?

Personally, I never made good experiences with rpy2 -- which is one reason why I stopped using these functions in more recent projects. Instead I performed DE in R and connected it through a workflow manager.
Are you sure you want to support that? It would also entail quite complicated CI setups if we were to test it.

rpy2 is at the moment an optional dependency for pertpy because milopy can also work with edgeR. I'd keep stick to this for support for the R options.

Maybe we start with pydeseq2, t-test, wilcoxon test?

Yup. We'd have to discuss what the interface should look like. Would you base it on the current scanpy t-test/wilcoxon test or would you redesign it completely? If yes, what should it look like?

My plotting functions are a mix of altair and matplotlib. Do you want to stick to one of them?

Preferably matplotlib. We replaced altair code in other tools with matplotlib equivalents.

Would you base it on the current scanpy t-test/wilcoxon test or would you redesign it completely? If yes, what should it look like?

I would probably use the scanpy implementation but redesign the output. IMO all tools should return a dataframe with one row per var with p-value and logFC. This dataframe could optionally be saved in adata.varm.
I also have a wrapper for a statsmodels linear model which could be relevant here:
https://github.com/icbi-lab/luca/blob/89c4e6109bc723f6958cae7af791398b28e8e422/lib/scanpy_helper_submodule/scanpy_helpers/compare_groups/lm.py#L24-L151

Preferably matplotlib. We replaced altair code in other tools with matplotlib equivalents.

Fair enough, shouldn't be too hard to rewrite it. Maybe an excuse to play with the new seaborn API.

Another point to consider is how the interface would look like. groupby is nice for a simple pairwise or groupwise comparison, but what about more complicated cases? Do we allow to specify patsy model strings?

I would probably use the scanpy implementation but redesign the output. IMO all tools should return a dataframe with one row per var with p-value and logFC. This dataframe could optionally be saved in adata.varm.

Yes, that makes a lot of sense.

I also have a wrapper for a statsmodels linear model which could be relevant here: https://github.com/icbi-lab/luca/blob/89c4e6109bc723f6958cae7af791398b28e8e422/lib/scanpy_helper_submodule/scanpy_helpers/compare_groups/lm.py#L24-L151

Yeah, why not.

Preferably matplotlib. We replaced altair code in other tools with matplotlib equivalents.

Fair enough, shouldn't be too hard to rewrite it. Maybe an excuse to play with the new seaborn API.

Yup, but we also have manpower for this if you don't want to do this yourself.

Another point to consider is how the interface would look like. groupby is nice for a simple pairwise or groupwise comparison, but what about more complicated cases? Do we allow to specify patsy model strings?

I wouldn't really need them for a quick t-test or Wilcoxon test but when doing serious work with edgeR/deseq2 for complex setups - yes I think that we need to support them.

Yup, but we also have manpower for this if you don't want to do this yourself.

ok, even better. So implementation-wise, what would you need from my side?

If you explicitly designed an interface (parameters that you want and output DF) it would certainly help and guide one of my students. I am confident though that I could also do that if you're busy.
Could you please edit my post and add links to the current implementations to all of your plots?
You are of course welcome to contribute to pertpy, but if you're busy one of my students will help out.

Updated the comment and added some draft specs for further discussion.

Excellent, thank you very much.
So would the de_analysis have an additional parameter called "method" to which this is passed to? Or would you have separate public functions per test?

Good point. I would have said the former. Updated the specs accordingly.

What do we do with https://scanpy.readthedocs.io/en/stable/generated/scanpy.pl.rank_genes_groups.html#scanpy.pl.rank_genes_groups and all of its siblings? Think it expects the old format?

tbh, I'm not a big fan of how scanpy currently stores the DE results. So I wouldn't mind if the scanpy functions were rewritten. But that would be either a breaking change or require a bunch of code handling legacy formats, so not sure if this is going to happen in a reasonable timeframe.

Unlikely.

Alternatively, these functions could be mirrored in pertpy, together with the other plotting functions planned. I have a code snippet for adding a data frame to anndata such that it works with the scanpy plotting functions here.

So the mirroring would work as follows: users calls
pt.pl.rank_genes_groups(adata, key="MYDERESULTSDF")

Under the hood we call your function and then call the scanpy plotting function?
Or how would you design it?

Maybe by using a context manager that temporarily adds the converted data frame to adata.uns?
def rank_genes_groups(adata, key, *args, **kwargs):
   with _add_de_res_for_scanpy(adata, key):
       sc.pl.rank_genes_groups(adata, *args, **kwargs)

Yeah, that's a lovely idea. Do you think that we should expose another "add_de_res_for_pertpy" function that does the same thing but this time load the DE results into adata.varm? We know where to store such dataframes, but newbies are always confused with the slots and varm is rarely used.
But I think that we've got it figured out! Do you want to tackle any of this? If not, I'll find someone as mentioned.

Yeah, that's a lovely idea. Do you think that we should expose another "add_de_res_for_pertpy" function that does the same thing but this time load the DE results into adata.varm

Either that, or allow to pass a data frame directly to the plotting function:
def plot_something(
    adata: AnnData,
    de_res: pd.DataFrame | str,
    p_col: str = "FDR",
    fc_col: str = "log2FC",
):
    """
    de_res:
         This may be either
            * the key under which the results are stored in `adata.varm`, or
            * a pandas data frame with DE results. The data frame must use gene
               identifiers from `adata.var_names` as index.
    """

But I think that we've got it figured out! Do you want to tackle any of this? If not, I'll find someone as mentioned.

If you had someone it would be awesome! I'm happy to review if required.

Closing this now in favor of more specific issues.


[HOLD for payment 2024-11-01] [$125] BUG: SelfDM is marked as unread even though there is no message

BUG: SelfDM is marked as unread even though there is no message

I signed up and wen’t through onboarding modals
When I get to the Concierge chat my SelfDM is unread
But there is no message in the self DM

The first image is when I choose “Manage my team”
The second is the SelfDM when you choose “Track my expenses”
In both cases, I don’t think it should be unread.

We discussed this internally and can't think of any cases where the SelfDM should be unread. So let's please update the flow so that it is marked as read when users come in as new users.
Some background from @trjExpensify:
https://github.com/Expensify/App/pull/43489 - It looks like we did this in this PR.
We need to remove the selfDM from this line: https://github.com/Expensify/App/blob/f180cbb97f7158ee6339215c9d5313b50f232c8c/src/libs/ReportUtils.ts#L6127

Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~021847081858608334041
Upwork Job ID:<PHONE_NUMBER>608334041
Last Price Increase: 2024-10-18
Automatic offers:

mkhutornyi | Reviewer | 104524422
mkzie2 | Contributor | 104524424

Issue OwnerCurrent Issue Owner: @zanyrenney

Proposal
Please re-state the problem that we are trying to solve in this issue.
SelfDM is marked as unread even though there is no message
What is the root cause of that problem?
We force showing unread for new selfDM (no message):
https://github.com/Expensify/App/blob/f180cbb97f7158ee6339215c9d5313b50f232c8c/src/libs/ReportUtils.ts#L6127
What changes do you think we should make in order to solve the problem?

Remove the isSelfDM check from the above.

I'm dropping the price on this as a one-liner we've identified.

@zanyrenney when this is deployed, please make sure we remove this test case from QA: https://github.com/Expensify/Expensify/issues/412760
CC: @isagoico

Confirmed it's working on production as intended!

@mkhutornyi please complete the checklist (you can edit my comment here to do so) so that i can close this out and pay it.

I have DM'd @mkhutornyi

Cool, please can you also complete the checklist? this is past due for payment @mkhutornyi  Thanks!

@isagoico already replied to the comment about updating the test case here.

BZ checklist above!

payment summary
@mkhutornyi requires payment automatic offer (Reviewer) - paid $125
@mkzie2 requires payment automatic offer (Contributor) - paid $125


Displaying or passing the values in Grid from ViewBag not working.

Trying to pass some values from ViewBag.
It gives following error :

An error occurred during the compilation of a resource required to process this request. Please review the following specific error details and modify your source code appropriately.
D:\ErpOneCore\Views\Voucher_index1.cshtml
An expression tree may not contain a dynamic operation
columns.Add(model => Html.ActionLink("Details", "Details1", new { id = model.id, mainid=**ViewBag.data.id** })).Titled("Details");

Is this the error of the grid though? :)

Yes

Sorry, but I can't find the code which checks the expression for dynamic usage and throws An expression tree may not contain a dynamic operation, can you link me in github?


Review: Review the Cursor Pagination Implementation and assess the other DB Providers

The implementation of Cursor Pagination from this code base is quite awesome.
This story will allow us to consider the implementation to be integrated to the core library (RepoDb) or the official extension for SQL Server (RepoDb.SqlServer) if needed/possible. In addition, this User Story is used as a placeholder for Cursor Pagination via other DB Provider (i.e.: MySQL, PostgreSQL and SQLite).
Note: An additional User Stories will be created if needed in order for us to fully integrate the solution to the core library.
Ensure the author of this extension library is notified.

@mikependon this is great, thank you so much for the complement on my repo!
In fact you have already read my mind. I wanted to ping you soon about the inclusion of the cursor pagination into the core of RepoDb!
I was just waiting to iron out a few more issues/concepts before proposing it.
I'd be happy to create a draft PR over the next couple of weeks if you would like that, or whatever next steps you think would be best.

The implementation of Cursor Pagination from this code base is quite awesome.
This story will allow us to consider the implementation to be integrated to the core library (RepoDb) or the official extension for SQL Server (RepoDb.SqlServer) if needed/possible. In addition, this User Story is used as a placeholder for Cursor Pagination via other DB Provider (i.e.: MySQL, PostgreSQL and SQLite).
Note: An additional User Stories will be created if needed in order for us to fully integrate the solution to the core library.
Ensure the author of this extension library is notified.

the link "code base" is broken

The implementation of Cursor Pagination from this code base is quite awesome.
This story will allow us to consider the implementation to be integrated to the core library (RepoDb) or the official extension for SQL Server (RepoDb.SqlServer) if needed/possible. In addition, this User Story is used as a placeholder for Cursor Pagination via other DB Provider (i.e.: MySQL, PostgreSQL and SQLite).
Note: An additional User Stories will be created if needed in order for us to fully integrate the solution to the core library.
Ensure the author of this extension library is notified.

the link "code base" is broken

That is because @cajuncoding, the author of the extension library renamed the repository due to GraphQL request. The new code base is found here.

That is because @cajuncoding, the author of the extension library renamed the repository due to GraphQL request. The new code base is found here.

Sorry about that broken link, I had to rename by request from the HotChocolate core team...they want to keep any other uses of their namespace out of Nuget so renaming GitHub was also needed for consistency.

Sorry about that broken link, I had to rename by request from the HotChocolate core team...they want to keep any other uses of their namespace out of Nuget so renaming GitHub was also needed for consistency.


Support for C++ constexpr (basics)

defining a built-in object as constexpr (static or not)
defining a function as constexpr (ignoring it's content)
defining a user-defined type with only constexpr operations to be used in constexpr context (classic case that needs to be supported)

Not handled (yet):

if constexpr
[] constexpr { ... } lambda specifier

Looks like my changes broke when I rebased over main, will fix.

@serge-sans-paille I found no way to determine if constexpr is implicit or explicit when a constructor is defaulted. Any idea?

otherwise I'll comment this case as the generated code is almost equivalent.

Had some conflicts when rebasing on main today.
Also, I just stopped using Constructor() = default to avoid the issue of distinguishing implicit vs epxlicit constexpr, the problem is still here but it should not be a big issue.
I did spot an inconsistency in object construction syntax generated which is NOT equivaleent however: T object{ blah }; generates T object(blah); which

could trigger the "most vexing parse";
the original doesnt allow implicit conversions, the generated version does;
if T have a constructor which takes an initializer list (for example every standard container does) then that constructor should be used with the original syntax, it will not compile or worse it will compile but with a different consturctor with the generated syntax.
Just noting this here so that you know there is this issue, to fix when it can be.


fix(dateparser): ignore invalid baseDate

Fix issue where the datepicker model value being set to an invalid date will cause all future calls to dateParser.parse() to return an invalid date. The "fields" object just below my change would get all its properties set to NaN, and the resulting date would therefore still contain NaN values, and would therefore be invalid.
This caused an issue where if you use a input[type=text] field for the datepicker, if you enter an invalid date, and then correct it to a valid date (by typing it, not by using the datepicker), the value would never flip back to valid.
See http://plnkr.co/hndJHbOgtPP9eWG9A31v to reproduce easily.

Enter a letter into the end of the date field to make the value invalid. The "Selected date is:" display at the top will show a blank value.

Delete the letter you entered to make the field valid again. The "Selected date is:" display at the top will continue to show a blank value, and the value you entered will not be parsed.

I just realized this is a duplicate of #3708. Sorry. I'll leave it here but feel free to close in favor of that one if you'd prefer.

Closing in favour of #3708. Thanks for reporting as duplicate.


[Question]: overflow and underflow

Feature Request Type
Documentation
Have you searched existing issues?
Yes
Is your feature request related to a problem?

Given a field type of FM64 and a fxp_fraction_bits = 24. How to calculate (roughly) the smallest positive number it can handle without underflow, and the largest positive number it can handle without overflow? I'm using CHEETAH protocol.
What would be the impact if one number is a little bit smaller than "the smallest positive number without underflow" in question 1? Will the number be wrapped around to a very large positive number? or be truncated somehow?
Similarly, what would be the impact if one number is a little bit larger than "the largest positive number without overflow" in question 1?

I have look around in documentation but could not find very relevant information. It would be very helpful if you could help provide some guidance. Thank you in advance!
Describe features you want to add to SPU
--
Describe features you want to add to SPU
--

Given a field type of FM64 and a fxp_fraction_bits = 24. How to calculate (roughly) the smallest positive number it can handle without underflow, and the largest positive number it can handle without overflow? I'm using CHEETAH protocol.

Hi @linzzzzzz

FM64 means width = 64bits, with fxp_fraction_bits = 24, epsilon is 2^-24 = 5.96e-8, that is the smallest representable positive number. Since we already used 24bits for fraction part, we have 40bits for whole numbers. That gives us somewhere around 2^40 - 1 for largest whole numbers.

But be aware of some operation requirements: mul/matmul will double the fraction bits during computation, that means only 16bits for whole numbers. Comparison/Truncation may requires k-2 bit is unused.
2&3. During floating-point to fixed-point encoding, we use (intXX_t)(v * 2^fxp_bits), that means, fraction bits after scaling will be discarded and remaining whole number will be fitted into target integer, if fails, the behavior is undefined. During computation data's are just treated as integers, and follows int or uint overflow behaviors (some of those are UBs).

Hi @linzzzzzz
I'll close this one, feel free to open new issue when other questions pop up.


Internet Explorer 11 issues

The site doesnt look good in IE11 unfortunately.

Flexbox, gradient font methods, etc. is what is making this not IE11 friendly. Unfortunately I am choosing to just support 'modern browsers' with the themes on Start Bootstrap rather than deal with the headache of making everything backward compatible with old browsers. This theme is a bit more experimental as far as my themes go, so it's a take it or leave it sort of thing here I'm afraid.


Alerts

...

I've checked https://github.com/gaearon/react-side-effect.
It's cool but not relevant to this task.

Done


How to bind ShiftPageUp / ShiftPageDown

Description of the problem or steps to reproduce
I am try write next key-bind option to bindings.json file:
     "ShiftPageUp": "SelectPageUp",
     "ShiftPageDown": "SelectPageDown"

But selection don't work. Why?
Specifications
micro v.2.0.3
OS: Debian Linux 9
Terminal: Konsole

Try
"Shift-PageUp": "SelectPageUp",
"Shift-PageDown": "SelectPageDown"

Works for me

Thanks @Productive2 , solved also #1627.

Back in 2004, the key bindings should have worked out of the box on konsole, after this change:
https://invent.kde.org/utilities/konsole/-/commit/541581182
However, the commit contains some typos, and instead of just fixing the typos, the support for shift+cursor was removed for some "compatibility" reason in 2005:
https://invent.kde.org/utilities/konsole/-/commit/5cd61bfb9
https://bugs.kde.org/show_bug.cgi?id=92749#c5
The support for shift+arrow was eventually added back in 2012:
https://invent.kde.org/utilities/konsole/-/commit/ad4314c76
https://bugs.kde.org/show_bug.cgi?id=204388
However, the support for shift+pgup and shift+pgdown is still missing. For now, we can add it back via "Edit Current Profile..." -> "Keyboard" -> "Edit..." -> "Add", with the following entries:

PgUp+Shift+AppScreen : "\E[5;2~"
PgDown+Shift+AppScreen : "\E[6;2~"

After that, micro bindings.json from the issue description should work fine.


Use own build on Azure App Service

To solve my performance problems (see here) I created my own EF build which solved the problem and works fine in my development environment.
But today I tried to publish a new version to my Azure App Service and it failed to start with Strong name signature could not be verified.. I quickly copied over the original 6.2.0 version and the app is now working again, but with the performance problems...
Is there a way to disable the strong-name check on Azure to use my custom build?
Further technical details
EF version: 6.2.0
Database Provider: EntityFramework.SqlServer
Operating system: Azure App Service
IDE: Visual Studio 2017 15.9.9

@chrfin I doubt very much that there is a way to do this, but that would really be a question for the Azure folks.


Make TV power optional

In order to hide it from older TVs with webOS versions that don't have the option to wake on lan.
Added the parameter "powerSwitch" in order to handle this, defaults to true.

What older TV's do you mean? The power switch should work on all TV'which have at least webOS2

My TV from 2014 supports just webOS 1, and I have all the functionality of this plugin except turning the TV on which doesn’t work.
I heard that LG would update 2014 TVs to webOS 2, but sadly that didn’t happen (for my model at least).

Yes, but this plugin only officially supports webOS2 and never TV's. Besides that turning on the TV is the main feature of this plugin so i don't see a reason to make it optional...

I use the plugin to control volume and switch between apps.
Also the parameter is optional, all current configurations would work like nothing had been changed.

Yes, but in that case you remove the switch from your room or hide it from favorites in the Home app or Eve app or any other app which you use, and this is less complicated.
I honestly don't think this is a good idea to make a main feature of this plugin optional.


Added two plugins

Added Two plugins.

Path intellisense which helps in autocompletion of files names in a directory saving lots of time.

Git History which gives detailed history of a line or a file. Extremely useful.

Regards
Ishan Jain

Really like these two and VS Code was missing those for a very long time! Thanks, merging.


Fix how cgroup manager is created based on cgroups path

This aims to fix some issues with how cgroup manager is selected currently, and  ultimately fix #1277 .
The issue with  current implementation is that if we have to use systemd, i.e. when user namespace is set or running rootless, then we simply fallback to systemd manager, which expects the cgroup path to be either absent or of the form [slice]:[scope_prefix]:[name] .
However, the oci spec does not have any such restriction, and for cgroup path it simply states here that :

cgroupsPath (string, OPTIONAL) path to the cgroups. It can be used to either control the cgroups hierarchy for containers or to run a new process in an existing container.

The value of cgroupsPath MUST be either an absolute path or a relative path.

In the case of an absolute path (starting with /), the runtime MUST take the path to be relative to the cgroups mount point.
In the case of a relative path (not starting with /), the runtime MAY interpret the path relative to a runtime-determined location in the cgroups hierarchy.

Now I checked with runc and crun, with the same config as given to TestUserNamespace test, and as the path given is /testing/TestUserNamespaces-WritableRootFS , both of them create the cgroups at path /sys/fs/cgroup//testing/TestUserNamespaces-WritableRootFS , which is to say they use v2 amanger instead of systemd (?)
Thus, in this PR I have added check if the path is absolute and use v2 instead of systemd.
However, this is still not complete, as if the path given is relative, then this will still fail as systemd parsing will fail as before ; in which case the question is what is the correct thing to do in all cases?
Please take a look. Thanks.
Signed-off-by: Yashodhan Joshi<EMAIL_ADDRESS>

The config given by test :

{
    "ociVersion": "1.0.2-dev",
    "process": {
        "user": {
            "uid": 0,
            "gid": 0,
            "additionalGids": [
                1,
                2,
                3,
                4,
                6,
                10,
                11,
                20,
                26,
                27
            ]
        },
        "args": [
            "sh",
            "-c",
            "exit 7"
        ],
        "env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "cwd": "/",
        "capabilities": {
            "bounding": [
                "CAP_CHOWN",
                "CAP_DAC_OVERRIDE",
                "CAP_FSETID",
                "CAP_FOWNER",
                "CAP_MKNOD",
                "CAP_NET_RAW",
                "CAP_SETGID",
                "CAP_SETUID",
                "CAP_SETFCAP",
                "CAP_SETPCAP",
                "CAP_NET_BIND_SERVICE",
                "CAP_SYS_CHROOT",
                "CAP_KILL",
                "CAP_AUDIT_WRITE"
            ],
            "effective": [
                "CAP_CHOWN",
                "CAP_DAC_OVERRIDE",
                "CAP_FSETID",
                "CAP_FOWNER",
                "CAP_MKNOD",
                "CAP_NET_RAW",
                "CAP_SETGID",
                "CAP_SETUID",
                "CAP_SETFCAP",
                "CAP_SETPCAP",
                "CAP_NET_BIND_SERVICE",
                "CAP_SYS_CHROOT",
                "CAP_KILL",
                "CAP_AUDIT_WRITE"
            ],
            "permitted": [
                "CAP_CHOWN",
                "CAP_DAC_OVERRIDE",
                "CAP_FSETID",
                "CAP_FOWNER",
                "CAP_MKNOD",
                "CAP_NET_RAW",
                "CAP_SETGID",
                "CAP_SETUID",
                "CAP_SETFCAP",
                "CAP_SETPCAP",
                "CAP_NET_BIND_SERVICE",
                "CAP_SYS_CHROOT",
                "CAP_KILL",
                "CAP_AUDIT_WRITE"
            ]
        },
        "rlimits": [
            {
                "type": "RLIMIT_NOFILE",
                "hard": 1024,
                "soft": 1024
            }
        ],
        "noNewPrivileges": true
    },
    "root": {
        "path": "rootfs"
    },
    "mounts": [
        {
            "destination": "/proc",
            "type": "proc",
            "source": "proc",
            "options": [
                "nosuid",
                "noexec",
                "nodev"
            ]
        },
        {
            "destination": "/dev",
            "type": "tmpfs",
            "source": "tmpfs",
            "options": [
                "nosuid",
                "strictatime",
                "mode=755",
                "size=65536k"
            ]
        },
        {
            "destination": "/dev/pts",
            "type": "devpts",
            "source": "devpts",
            "options": [
                "nosuid",
                "noexec",
                "newinstance",
                "ptmxmode=0666",
                "mode=0620",
                "gid=5"
            ]
        },
        {
            "destination": "/dev/shm",
            "type": "tmpfs",
            "source": "shm",
            "options": [
                "nosuid",
                "noexec",
                "nodev",
                "mode=1777",
                "size=65536k"
            ]
        },
        {
            "destination": "/dev/mqueue",
            "type": "mqueue",
            "source": "mqueue",
            "options": [
                "nosuid",
                "noexec",
                "nodev"
            ]
        },
        {
            "destination": "/sys",
            "type": "sysfs",
            "source": "sysfs",
            "options": [
                "nosuid",
                "noexec",
                "nodev",
                "ro"
            ]
        },
        {
            "destination": "/run",
            "type": "tmpfs",
            "source": "tmpfs",
            "options": [
                "nosuid",
                "strictatime",
                "mode=755",
                "size=65536k"
            ]
        }
    ],
    "linux": {
        "uidMappings": [
            {
                "containerID": 0,
                "hostID": 1000,
                "size": 10000
            }
        ],
        "gidMappings": [
            {
                "containerID": 0,
                "hostID": 2000,
                "size": 10000
            }
        ],
        "resources": {
            "devices": [
                {
                    "allow": false,
                    "access": "rwm"
                }
            ]
        },
        "cgroupsPath": "/testing/TestUserNamespaces-WritableRootFS",
        "namespaces": [
            {
                "type": "pid"
            },
            {
                "type": "ipc"
            },
            {
                "type": "uts"
            },
            {
                "type": "mount"
            },
            {
                "type": "network"
            },
            {
                "type": "user"
            }
        ],
        "maskedPaths": [
            "/proc/acpi",
            "/proc/asound",
            "/proc/kcore",
            "/proc/keys",
            "/proc/latency_stats",
            "/proc/timer_list",
            "/proc/timer_stats",
            "/proc/sched_debug",
            "/sys/firmware",
            "/proc/scsi"
        ],
        "readonlyPaths": [
            "/proc/bus",
            "/proc/fs",
            "/proc/irq",
            "/proc/sys",
            "/proc/sysrq-trigger"
        ]
    }
}

Codecov Report

Merging #1288 (2d48e16) into main (497f2e6) will increase coverage by 0.01%.
The diff coverage is 0.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1288      +/-   ##
==========================================
+ Coverage   69.21%   69.22%   +0.01%
==========================================
  Files         118      118
  Lines       12514    12513       -1
==========================================
+ Hits         8661     8662       +1
+ Misses       3853     3851       -2


Import XML has Gem with blank nameSpec attribute

I don't know how to reproduce this, and when I import it and immediately export the problem goes away, but this person posted an import string where the active skill's (Shield Charge) nameSpec attribute is an empty string.
https://pastebin.com/m1qZCdrb

I managed to reproduce this; it occurs when blanking a gem slot without actually removing it. That instance is fixed, but note that the nameSpec attribute may be intentionally empty under rare conditions.


I would like to contribute to this project . Kindly assign a feature that I should work upon !

Hey buddy I think this we as a participant should find

Okay sure
I will look into it !!!
On Wed, 2 Mar 2022, 22:04 Rishabh Jain, @.***> wrote:

Yes! I will add the issues! you can comment on them so that I can assign
you those!
—
Reply to this email directly, view it on GitHub
https://github.com/jain-rishabh-21/DoctorVerse/issues/2#issuecomment-1057126211,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQVHO7MGLGOUKHAQUDJO5ULU56KBDANCNFSM5PREOIHQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>


APTx Function: Default value of gamma should be 0.5 when trainable=False

In APTx activation function,  the default value of gamma should be 0.5 when trainable=False. One can verify it in the APTx paper here: https://arxiv.org/pdf/2209.06119

Thank you for this PR


literal_unroll != numba.literal_unroll?

Hi, I'm seeing a strange behaviour, confirmed in two v0.48 environments. The same function behaves differently when using literal_unroll or numba.literal_unroll.
def make_foo():
    x = (1, 2)
    @njit
    def _foo_unrolled_global():
        a = 0
        for val in numba.literal_unroll(x):
            a += val
        return a
    return _foo_unrolled_global

def make_bar():
    x = (1, 2)
    @njit
    def _bar_unrolled_global():
        a = 0
        for val in literal_unroll(x):
            a += val
        return a
    return _bar_unrolled_global

foo = make_foo()
foo()
# OK

bar = make_bar()
bar()
UnsupportedError: Failed in nopython mode pipeline (step: handles literal_unroll)
Failed in literal_unroll_subpipeline mode pipeline (step: switch const list for tuples)
Invalid use of literal_unroll, argument should be a tuple or a list of constant values, found unknown problem

I know the second error is related (and fixed by) #5182, so it should not fail in v0.49, but the puzzle is why the first one does not fail in v0.48.
cc: @stuartarchibald

Thanks for the report. I think part of what you observe is related to the use of UniTuple types, which work in both circumstances regardless of whether the literal_unroll transform takes place, i.e. the literal unroll can fail to run and the code will still compile fine.
def make_baz():
    x = (1, 2)
    @njit
    def impl():
        a = 0
        for val in x:
            a += val
        return a
    return impl

make_baz()() # this is fine

Adjusting the above by making the tuple heterogeneous in type will make it so a literal unroll is required for compilation to succeed. e.g.
def make_success():
    x = (1.2, 3j)
    @njit
    def impl():
        a = 0
        for val in literal_unroll(x):
            a += val
        return a
    return impl

make_success()() # works fine

def make_fail():
    x = (1.2, 3j)
    @njit
    def impl():
        a = 0
        for val in x:
            a += val
        return a
    return impl

make_fail()() # fails in typing with "Invalid use of getiter with parameters (Tuple(float64, complex128))"

Now there's a sample which can only succeed if the literal_unroll works the bug can be pinned down...
def make_bug():
    x = (1.2, 3j)
    @njit
    def impl():
        a = 0
        for val in numba.literal_unroll(x):
            a += val
        return a
    return impl

make_bug()() # same issue as `make_fail`, Invalid use of getiter...

The make_bug fails the same way as make_fail which suggests that invoking literal_unroll in this way is not working correctly. This is confirmed by running:
NUMBA_DEBUG_PRINT_WRAP=literal_unroll python issue5344.py

where issue5344.py contains the make_bug sample above. It can be observed that the IR is not mutated by the transform.
As the author of the transform I'm fairly sure I wrote the use case of from numba import literal_unroll and from numba import literal_unroll as somethingelse but do not remember dealing with it as a getattr on numba, so this is a bug! Work around is trivial, i.e. use a supported spelling, however we ought to fix it!

Just ran into this bug while trying to clean my imports and getting my code more explicit. It has not been fixed in the meantime.
numba version 0.53.1+8.gddf817af0.dirty

It has not been fixed in the meantime.

Indeed. It's also present in 0.54RC2.


Complete IAS reporting

There will be five reports:

Expected annual damage distribution (please see revision in design document)
Expected annual damage by damage category
Performance: distribution of AEP given a user-identified and user-selected threshold
Performance: Long-term risk given a user-identified and user-selected threshold
Performance: Assurance of user-identified and user-selected threshold  (please see revision in design document)

This is completed.


DHParameters constructor checks

I'm trying to connect (code here) to api.telegram.org and the server provides these DH Parameters:

Deep inside .Connect call, a DefaultTlsDHVerifier compares these against a set of 11 predefined parameter groups, and throws an error (code 71 aka insufficient_security) since it's not one of them¹. Okay guess I have to pass these values to a verifier constructor²:
byte[] p = { 0x8b, 0xf9, 0x0d, 0x21, ... };
byte[] g = { 0x02 };
var groups = new List<DHParameters>{ new DHParameters(p, g) };
var verifier = new DefaultTlsDHVerifier(groups, 0);

But these values do not satisfy the requirement of g <= p - 2 in DHParameters constructor because p is a negative number (leftmost bit is set). Why does C# version have this check when Java version doesn't?
Compare:
https://github.com/bcgit/bc-csharp/blob/master/crypto/src/crypto/parameters/DHParameters.cs#L86
https://github.com/bcgit/bc-java/blob/master/core/src/main/java/org/bouncycastle/crypto/params/DHParameters.java#L86
¹Why do it? The way I see it, the server decides whatever parameters to use.
²What's the whole point of Verifier in this case? Can I safely skip it (public override bool Accept(DHParameters _) => true;)?

P is not negative. The encodings are unsigned.


.env file at root of source branch breaks action (still)

Describe the bug
This is related to #210 as we get a Malformed entry entry when trying to run the latest gitHub action.
We also have a root level .env file. The part of the file where it seems to trip up on is:
FILE_DOWNLOAD_RSA_KEY='-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAr9jabmSaRKcaf7KK8qmWWDQdU3+fjVuYNFLGiRXQl+o3HyaG
...

To Reproduce
Steps to reproduce the behavior:

Setup a project using this GitHub action
Add a root level .env file, containing a multi line variable, encompassed by single quotes.

Expected behavior
I think I would not expect a GitHub action to read my .env file at all, to be honest.

Hi @Bertg, could you please provide a GitHub Action logs?

2024-05-27T19:16:39.6686557Z Current runner version: '2.316.1'
2024-05-27T19:16:39.6708918Z ##[group]Operating System
2024-05-27T19:16:39.6709540Z Ubuntu
2024-05-27T19:16:39.6710037Z 22.04.4
2024-05-27T19:16:39.6710331Z LTS
2024-05-27T19:16:39.6710669Z ##[endgroup]
2024-05-27T19:16:39.6711135Z ##[group]Runner Image
2024-05-27T19:16:39.6711562Z Image: ubuntu-22.04
2024-05-27T19:16:39.6711922Z Version: 20240516.1.0
2024-05-27T19:16:39.6712993Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240516.1/images/ubuntu/Ubuntu2204-Readme.md
2024-05-27T19:16:39.6714422Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240516.1
2024-05-27T19:16:39.6715371Z ##[endgroup]
2024-05-27T19:16:39.6715742Z ##[group]Runner Image Provisioner
2024-05-27T19:16:39.6716220Z 2.0.369.1
2024-05-27T19:16:39.6716645Z ##[endgroup]
2024-05-27T19:16:39.6718941Z ##[group]GITHUB_TOKEN Permissions
2024-05-27T19:16:39.6720533Z Actions: write
2024-05-27T19:16:39.6721224Z Attestations: write
2024-05-27T19:16:39.6721793Z Checks: write
2024-05-27T19:16:39.6722169Z Contents: write
2024-05-27T19:16:39.6722577Z Deployments: write
2024-05-27T19:16:39.6723027Z Discussions: write
2024-05-27T19:16:39.6723420Z Issues: write
2024-05-27T19:16:39.6723800Z Metadata: read
2024-05-27T19:16:39.6724252Z Packages: write
2024-05-27T19:16:39.6724597Z Pages: write
2024-05-27T19:16:39.6724983Z PullRequests: write
2024-05-27T19:16:39.6725464Z RepositoryProjects: write
2024-05-27T19:16:39.6725880Z SecurityEvents: write
2024-05-27T19:16:39.6726308Z Statuses: write
2024-05-27T19:16:39.6726745Z ##[endgroup]
2024-05-27T19:16:39.6729654Z Secret source: Actions
2024-05-27T19:16:39.6730191Z Prepare workflow directory
2024-05-27T19:16:39.7341494Z Prepare all required actions
2024-05-27T19:16:39.7495974Z Getting action download info
2024-05-27T19:16:39.8862106Z Download action repository 'actions/checkout@v4' (SHA:a5ac7e51b41094c92402da3b24376905380afc29)
2024-05-27T19:16:40.1774647Z Download action repository 'crowdin/github-action@v1' (SHA:30849777a3cba6ee9a09e24e195272b8287a0a5b)
2024-05-27T19:16:40.5135979Z Complete job name: 🗂 Upload translation sources to Crowdin
2024-05-27T19:16:40.5729883Z ##[group]Build container for action use: '/home/runner/work/_actions/crowdin/github-action/v1/Dockerfile'.
2024-05-27T19:16:40.5776874Z ##[command]/usr/bin/docker build -t 20ad07:5f089c9fabdb44c0be8f78135d8bbd5e -f "/home/runner/work/_actions/crowdin/github-action/v1/Dockerfile" "/home/runner/work/_actions/crowdin/github-action/v1"
2024-05-27T19:16:41.2153660Z #0 building with "default" instance using docker driver
2024-05-27T19:16:41.2154911Z
2024-05-27T19:16:41.2155310Z #1 [internal] load .dockerignore
2024-05-27T19:16:41.2156538Z #1 transferring context: 2B done
2024-05-27T19:16:41.2157680Z #1 DONE 0.0s
2024-05-27T19:16:41.2162779Z
2024-05-27T19:16:41.2179788Z #2 [internal] load build definition from Dockerfile
2024-05-27T19:16:41.2192421Z #2 transferring dockerfile: 189B done
2024-05-27T19:16:41.2194172Z #2 DONE 0.0s
2024-05-27T19:16:41.2194933Z
2024-05-27T19:16:41.2199860Z #3 [auth] crowdin/cli:pull token for registry-1.docker.io
2024-05-27T19:16:41.2203247Z #3 DONE 0.0s
2024-05-27T19:16:41.2204041Z
2024-05-27T19:16:41.2204598Z #4 [internal] load metadata for docker.io/crowdin/cli:3.19.4
2024-05-27T19:16:41.3170094Z #4 DONE 0.3s
2024-05-27T19:16:41.4413410Z
2024-05-27T19:16:41.4419085Z #5 [internal] load build context
2024-05-27T19:16:41.4420302Z #5 transferring context: 67.25kB done
2024-05-27T19:16:41.4421471Z #5 DONE 0.0s
2024-05-27T19:16:41.4422004Z
2024-05-27T19:16:41.4423127Z #6 [1/4] FROM docker.io/crowdin/cli:3.19.4@sha256:d4300f04d4e456a5bf33e473158a052a6929f92375d092db3db79719e18938a0
2024-05-27T19:16:41.4425291Z #6 resolve docker.io/crowdin/cli:3.19.4@sha256:d4300f04d4e456a5bf33e473158a052a6929f92375d092db3db79719e18938a0 done
2024-05-27T19:16:41.4427424Z #6 sha256:3bb54f18b81441d8fc88f4feef46684e8417cf40a9e5d9d8e82e81fd8d9dafc7 2.17kB / 2.17kB done
2024-05-27T19:16:41.4429312Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 2.10MB / 60.78MB 0.1s
2024-05-27T19:16:41.4431715Z #6 sha256:651473210f22d8a5ccfda7b85258273abc82713bc8084e9397b45710e263d3c4 0B / 7.65MB 0.1s
2024-05-27T19:16:41.4433512Z #6 sha256:5426e9aaa67c3490798058591521f8ec7b2374f8f136b2745cfbd2a193658a9c 0B / 7.66MB 0.1s
2024-05-27T19:16:41.4435150Z #6 sha256:d4300f04d4e456a5bf33e473158a052a6929f92375d092db3db79719e18938a0 1.58kB / 1.58kB done
2024-05-27T19:16:41.6223012Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 8.39MB / 60.78MB 0.3s
2024-05-27T19:16:41.6224634Z #6 sha256:651473210f22d8a5ccfda7b85258273abc82713bc8084e9397b45710e263d3c4 6.29MB / 7.65MB 0.3s
2024-05-27T19:16:41.6226657Z #6 sha256:5426e9aaa67c3490798058591521f8ec7b2374f8f136b2745cfbd2a193658a9c 7.66MB / 7.66MB 0.3s done
2024-05-27T19:16:41.6228047Z #6 sha256:889c520d6b47f8815c2586925a1af4ca29dc7af1efcfee9c3ba47ff57a2f4a5c 0B / 209B 0.3s
2024-05-27T19:16:41.7240889Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 17.83MB / 60.78MB 0.4s
2024-05-27T19:16:41.7242711Z #6 sha256:651473210f22d8a5ccfda7b85258273abc82713bc8084e9397b45710e263d3c4 7.65MB / 7.65MB 0.3s done
2024-05-27T19:16:41.7244302Z #6 sha256:889c520d6b47f8815c2586925a1af4ca29dc7af1efcfee9c3ba47ff57a2f4a5c 209B / 209B 0.3s done
2024-05-27T19:16:41.7245919Z #6 sha256:c3e4051bdfada98d4d6171ef51d1b306c27ec538e0775fa93ffd4d24b8314363 125B / 125B 0.4s done
2024-05-27T19:16:41.9211759Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 29.36MB / 60.78MB 0.6s
2024-05-27T19:16:42.1185409Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 38.80MB / 60.78MB 0.8s
2024-05-27T19:16:42.2372847Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 47.19MB / 60.78MB 0.9s
2024-05-27T19:16:42.4180372Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 57.67MB / 60.78MB 1.1s
2024-05-27T19:16:42.5553253Z #6 sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 60.78MB / 60.78MB 1.2s done
2024-05-27T19:16:42.5555197Z #6 extracting sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a
2024-05-27T19:16:43.9712900Z #6 extracting sha256:7472b320d1fe05ed5001ce5b2f11da7e767cd4a35b0c7caeeccca9f6f781fd0a 1.4s done
2024-05-27T19:16:44.6298220Z #6 extracting sha256:651473210f22d8a5ccfda7b85258273abc82713bc8084e9397b45710e263d3c4
2024-05-27T19:16:44.7721790Z #6 extracting sha256:651473210f22d8a5ccfda7b85258273abc82713bc8084e9397b45710e263d3c4 0.1s done
2024-05-27T19:16:44.7723556Z #6 extracting sha256:5426e9aaa67c3490798058591521f8ec7b2374f8f136b2745cfbd2a193658a9c 0.1s done
2024-05-27T19:16:45.0115918Z #6 extracting sha256:889c520d6b47f8815c2586925a1af4ca29dc7af1efcfee9c3ba47ff57a2f4a5c done
2024-05-27T19:16:45.0117975Z #6 extracting sha256:c3e4051bdfada98d4d6171ef51d1b306c27ec538e0775fa93ffd4d24b8314363 done
2024-05-27T19:16:45.0119464Z #6 DONE 3.5s
2024-05-27T19:16:45.0119744Z
2024-05-27T19:16:45.0120471Z #7 [2/4] RUN apk --no-cache add curl git git-lfs jq gnupg;
2024-05-27T19:16:45.0871699Z #7 0.242 fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
2024-05-27T19:16:45.2216287Z #7 0.377 fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
2024-05-27T19:16:45.5423730Z #7 0.697 (1/38) Installing c-ares (1.27.0-r0)
2024-05-27T19:16:45.6553003Z #7 0.708 (2/38) Installing libunistring (1.1-r2)
2024-05-27T19:16:45.6554309Z #7 0.739 (3/38) Installing libidn2 (2.3.4-r4)
2024-05-27T19:16:45.6555374Z #7 0.749 (4/38) Installing nghttp2-libs (1.58.0-r0)
2024-05-27T19:16:45.6556425Z #7 0.757 (5/38) Installing libcurl (8.5.0-r0)
2024-05-27T19:16:45.6557149Z #7 0.774 (6/38) Installing curl (8.5.0-r0)
2024-05-27T19:16:45.6557904Z #7 0.785 (7/38) Installing libexpat (2.6.2-r0)
2024-05-27T19:16:45.6558706Z #7 0.793 (8/38) Installing pcre2 (10.42-r2)
2024-05-27T19:16:45.6559448Z #7 0.810 (9/38) Installing git (2.43.4-r0)
2024-05-27T19:16:45.8319429Z #7 0.902 (10/38) Installing git-lfs (3.4.1-r3)
2024-05-27T19:16:45.8320533Z #7 0.986 Executing git-lfs-3.4.1-r3.post-install
2024-05-27T19:16:45.9356417Z #7 1.002 Git LFS initialized.
2024-05-27T19:16:45.9357221Z #7 1.003 (11/38) Installing libgpg-error (1.47-r2)
2024-05-27T19:16:45.9358161Z #7 1.012 (12/38) Installing libassuan (2.5.6-r1)
2024-05-27T19:16:45.9359144Z #7 1.019 (13/38) Installing ncurses-terminfo-base (6.4_p20231125-r0)
2024-05-27T19:16:45.9360145Z #7 1.030 (14/38) Installing libncursesw (6.4_p20231125-r0)
2024-05-27T19:16:45.9361170Z #7 1.041 (15/38) Installing pinentry (1.2.1-r1)
2024-05-27T19:16:45.9361972Z #7 1.047 Executing pinentry-1.2.1-r1.post-install
2024-05-27T19:16:45.9362789Z #7 1.051 (16/38) Installing libgcrypt (1.10.3-r0)
2024-05-27T19:16:45.9363740Z #7 1.066 (17/38) Installing gnupg-gpgconf (2.4.4-r0)
2024-05-27T19:16:45.9364541Z #7 1.074 (18/38) Installing gmp (6.3.0-r0)
2024-05-27T19:16:45.9365285Z #7 1.090 (19/38) Installing nettle (3.9.1-r0)
2024-05-27T19:16:46.0455830Z #7 1.106 (20/38) Installing gnutls (3.8.4-r0)
2024-05-27T19:16:46.0456682Z #7 1.127 (21/38) Installing libksba (1.6.5-r0)
2024-05-27T19:16:46.0457419Z #7 1.135 (22/38) Installing gdbm (1.23-r1)
2024-05-27T19:16:46.0458369Z #7 1.141 (23/38) Installing libsasl (2.1.28-r5)
2024-05-27T19:16:46.0459227Z #7 1.149 (24/38) Installing libldap (2.6.6-r1)
2024-05-27T19:16:46.0460179Z #7 1.161 (25/38) Installing npth (1.6-r4)
2024-05-27T19:16:46.0460926Z #7 1.167 (26/38) Installing gnupg-dirmngr (2.4.4-r0)
2024-05-27T19:16:46.0461755Z #7 1.182 (27/38) Installing sqlite-libs (3.44.2-r0)
2024-05-27T19:16:46.0462696Z #7 1.200 (28/38) Installing gnupg-keyboxd (2.4.4-r0)
2024-05-27T19:16:46.1554430Z #7 1.209 (29/38) Installing gpg (2.4.4-r0)
2024-05-27T19:16:46.1555520Z #7 1.224 (30/38) Installing gpg-agent (2.4.4-r0)
2024-05-27T19:16:46.1556399Z #7 1.238 (31/38) Installing gpg-wks-server (2.4.4-r0)
2024-05-27T19:16:46.1557004Z #7 1.246 (32/38) Installing gpgsm (2.4.4-r0)
2024-05-27T19:16:46.1557538Z #7 1.257 (33/38) Installing gpgv (2.4.4-r0)
2024-05-27T19:16:46.1558051Z #7 1.269 (34/38) Installing gnupg-utils (2.4.4-r0)
2024-05-27T19:16:46.1558619Z #7 1.283 (35/38) Installing gnupg-wks-client (2.4.4-r0)
2024-05-27T19:16:46.1559509Z #7 1.293 (36/38) Installing gnupg (2.4.4-r0)
2024-05-27T19:16:46.1559981Z #7 1.299 (37/38) Installing oniguruma (6.9.9-r0)
2024-05-27T19:16:46.1560442Z #7 1.310 (38/38) Installing jq (1.7.1-r0)
2024-05-27T19:16:46.3154809Z #7 1.328 Executing busybox-1.36.1-r15.trigger
2024-05-27T19:16:46.3156072Z #7 1.336 OK: 210 MiB in 81 packages
2024-05-27T19:16:46.3156798Z #7 DONE 1.5s
2024-05-27T19:16:46.5169340Z
2024-05-27T19:16:46.5170014Z #8 [3/4] COPY . .
2024-05-27T19:16:46.5170819Z #8 DONE 0.0s
2024-05-27T19:16:46.5171143Z
2024-05-27T19:16:46.5171406Z #9 [4/4] COPY entrypoint.sh /entrypoint.sh
2024-05-27T19:16:46.5172155Z #9 DONE 0.0s
2024-05-27T19:16:46.5172738Z
2024-05-27T19:16:46.5172991Z #10 exporting to image
2024-05-27T19:16:46.5173498Z #10 exporting layers
2024-05-27T19:16:47.0140153Z #10 exporting layers 0.6s done
2024-05-27T19:16:47.0294404Z #10 writing image sha256:0b5a55b99593830c4b7d4c810a5ce6a600efe11887a3e6fea0c8aaf145668f00 done
2024-05-27T19:16:47.0296366Z #10 naming to docker.io/library/20ad07:5f089c9fabdb44c0be8f78135d8bbd5e done
2024-05-27T19:16:47.0297434Z #10 DONE 0.7s
2024-05-27T19:16:47.0349701Z ##[endgroup]
2024-05-27T19:16:47.0741125Z ##[group]Run actions/checkout@v4
2024-05-27T19:16:47.0742408Z with:
2024-05-27T19:16:47.0742764Z   repository: redacted/redacted
2024-05-27T19:16:47.0743908Z   token: ***
2024-05-27T19:16:47.0744419Z   ssh-strict: true
2024-05-27T19:16:47.0744775Z   ssh-user: git
2024-05-27T19:16:47.0746733Z   persist-credentials: true
2024-05-27T19:16:47.0747629Z   clean: true
2024-05-27T19:16:47.0748369Z   sparse-checkout-cone-mode: true
2024-05-27T19:16:47.0749433Z   fetch-depth: 1
2024-05-27T19:16:47.0749885Z   fetch-tags: false
2024-05-27T19:16:47.0750256Z   show-progress: true
2024-05-27T19:16:47.0751518Z   lfs: false
2024-05-27T19:16:47.0752287Z   submodules: false
2024-05-27T19:16:47.0753102Z   set-safe-directory: true
2024-05-27T19:16:47.0754028Z ##[endgroup]
2024-05-27T19:16:47.4125201Z Syncing repository: redacted/redacted
2024-05-27T19:16:47.4127417Z ##[group]Getting Git version info
2024-05-27T19:16:47.4128532Z Working directory is '/home/runner/work/redacted/redacted'
2024-05-27T19:16:47.4129846Z [command]/usr/bin/git version
2024-05-27T19:16:47.4158967Z git version 2.45.1
2024-05-27T19:16:47.4202914Z ##[endgroup]
2024-05-27T19:16:47.4237962Z Temporarily overriding HOME='/home/runner/work/_temp/d7fa7fc3-53a8-4ee0-835e-f954721938a6' before making global git config changes
2024-05-27T19:16:47.4240275Z Adding repository directory to the temporary git global config as a safe directory
2024-05-27T19:16:47.4245058Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/redacted/redacted
2024-05-27T19:16:47.4296466Z Deleting the contents of '/home/runner/work/redacted/redacted'
2024-05-27T19:16:47.4316958Z ##[group]Initializing the repository
2024-05-27T19:16:47.4318265Z [command]/usr/bin/git init /home/runner/work/redacted/redacted
2024-05-27T19:16:47.4394640Z hint: Using 'master' as the name for the initial branch. This default branch name
2024-05-27T19:16:47.4399477Z hint: is subject to change. To configure the initial branch name to use in all
2024-05-27T19:16:47.4403573Z hint: of your new repositories, which will suppress this warning, call:
2024-05-27T19:16:47.4407135Z hint:
2024-05-27T19:16:47.4410446Z hint: 	git config --global init.defaultBranch <name>
2024-05-27T19:16:47.4413823Z hint:
2024-05-27T19:16:47.4417307Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2024-05-27T19:16:47.4421131Z hint: 'development'.
We have however locally never had issues with crowdin push or crowdin pull command. (do note that I can't say that this output never appears, as we have them as part of an automated script, so if the command succeeded and that log entry showed, we would not know.

@Bertg thank you!
Could you please provide the backtrace of this error?
crowdin lint --debug

Please also provide your crowdin.yml configuration file. The error itself looks very strange.

Absolutely:
> crowdin lint --debug
io.github.cdimascio.dotenv.DotenvException: Malformed entry MIIEpQIBAAKCAQEAr9jabmSaRKcaf7KK8qmWWDQdU3+fjVuYNFLGiRXQl+o3HyaG
	at io.github.cdimascio.dotenv.internal.DotenvParser.parse(DotenvParser.java:68)
	at io.github.cdimascio.dotenv.DotenvBuilder.load(DotenvBuilder.java:77)
	at com.crowdin.cli.properties.PropertiesBuilder.getDotenv(PropertiesBuilder.java:241)
	at com.crowdin.cli.properties.PropertiesBuilder.setEnvOrPropertyIfExists(PropertiesBuilder.java:229)
	at com.crowdin.cli.properties.BaseProperties$BasePropertiesConfigurator.populateWithValues(BaseProperties.java:48)
	at com.crowdin.cli.properties.PropertiesBuilderChecker.populateWithConfigFileParams(PropertiesBuilderChecker.java:35)
	at com.crowdin.cli.properties.PropertiesBuilderChecker.populateWithConfigFileParams(PropertiesBuilderChecker.java:10)
	at com.crowdin.cli.properties.PropertiesBuilder.build(PropertiesBuilder.java:166)
	at com.crowdin.cli.properties.PropertiesBuilders.buildChecker(PropertiesBuilders.java:51)
	at com.crowdin.cli.commands.picocli.LintSubcommand.getProperties(LintSubcommand.java:27)
	at com.crowdin.cli.commands.picocli.LintSubcommand.getProperties(LintSubcommand.java:13)
	at com.crowdin.cli.commands.picocli.GenericActCommand.run(GenericActCommand.java:27)
	at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
	at picocli.CommandLine.access$1500(CommandLine.java:148)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
	at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
	at picocli.CommandLine.execute(CommandLine.java:2170)
	at com.crowdin.cli.commands.picocli.PicocliRunner.execute(PicocliRunner.java:39)
	at com.crowdin.cli.Cli.main(Cli.java:16)

It looks like your .env parsing library does not support multi line files. https://github.com/cdimascio/dotenv-java?tab=readme-ov-file (see last FAQ question).
It is still weird to me that the tool reads my .env file, it is not in the docs; it has its own very extensive confirmation file... its just really surprising to me.

@andrii-bodnar I guess the main issue here is the CLI tool, not the action.
I will report the issue there.


Feat nodejs: Added Nodejs Conversion

Nodejs go-conversion:
Added "Nodejs" support to go-convert specific to jenkins migration.
Nodejs Jenkins Trace:
Test_Nodejs_plugin__4.json
go run main.go jenkinsjson --downgrade ~/go/Test_Nodejs_plugin__4.json
Nodejs harness pipeline step:
- step:
    identifier: stagenull99086b
    name: Nodejs
    spec:
      command: |
        npm --version
        npm install
      image: node:latest
      shell: Sh
    timeout: ""
    type: Run

Hello @Ompragash,
Could you please review the nodejs conversion step, and provide your comments on this if there are any?
Thank you.

@Ompragash Could you please review the Nodejs Conversion step and provide your review comments on this?
Thanks.


Fix for loop bug which breaks favicon replacement

When iterating over the list , if a link is removed the list gets one element shorter which was causing things to be skipped over.
Switched to only bumping i if nothing is removed.
This was breaking the favicon replacement as "alternate icon" was getting skipped over.
Tested by building own plugin and testing own environment.
Testing done

### Submitter checklist
- [x] Make sure you are opening from a **topic/feature/bugfix branch** (right side) and not your main branch!
- [x] Ensure that the pull request title represents the desired changelog entry
- [x] Please describe what you did
- [x] Link to relevant issues in GitHub or Jira
- [x] Link to relevant pull requests, esp. upstream and downstream changes
- [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue

Note: the commit message is wrong should be "only bump i when link isnt removed"

@TobiX I took your advice and just created a static array, makes things easier. sorry for all the commits. I can create a new branch with the one commit  and a new PR to keep git cleaner.

All fine, I can simply squash all changes.


Intellisense and AutoComplete

Issue Type: Bug
Auto fix and Ctrl+. and Ctrl + space is not getting me any suggestion for the import statment for example I typed ICollection to declare a variable and as previously behaviour we type CTRL + . this Auto imports the name space of the collevtion generics , but the actual behaviour was "nothing Suggested"
VS Code version: Code 1.25.1 (1dfc5e557209371715f655691b1235b6b26a06be, 2018-07-11T15:43:53.668Z)
OS version: Windows_NT x64 6.3.9600

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz (8 x 2494)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: disabled_softwarevideo_decode: enabledvideo_encode: enabledvpx_decode: unavailable_softwarewebgl: enabledwebgl2: enabled

Memory (System)
15.91GB (4.08GB free)

Process Argv
C:\Program Files\Microsoft VS Code\Code.exe

Screen Reader
no

VM
67%

Extensions (3)

Extension
Author (truncated)
Version

csharpextensions
jch
1.3.0

csharp
ms-
1.15.2

f12-open-file
rex
0.3.1

+1

What programming language are using when this happens. If it is c# then please file the issue against the c# extension.


Table HTML and dataframe need FIX in JupyterLab Light theme

https://leriomaggio.github.io/texbook-jupyter-theme/TexBook_Theme_Sample_default.html#dataframe

Fixed in notebook-magic branch. Needs merge in master before closing.

Merged


Implement rotational-cipher exercise

Went ahead and added a PR to x-common to fix the canonical test data


Save Project Automatically

Describe the bug
We do not want people to loss their work or projects, So we intend to automatically save their projects.
Algo 1:
U can use a time interval to always call the save function
Algo 2 & Better
We only want to save content when the user adds content or when the user is active. So to achieve this, We intend to call the Save function when a new item is added to the development dashboard or after 10 active clicks
File/Files you will work on
dashboard.html if your code looks code, You can make another pull request for the ui_dashboard page
Possible Lines of Code
You should not add above the possible lines of code. (if given)
Result or Final Looks
Projects will be saved automatically.

Can you assign it to me?

I was able to work on this. We can now save content automatically. And the content is recovered when ever the page is loaded. Though I plan on improving the code better.


disable parameter for Precommit.check

Unlike commenting out a check, a disabled check would still show up in the output with a warning.

I don't need this.


Explicitly instantiating WeakPtr does not impl WeakPtrTarget

If I have
#[cxx::bridge]
mod ffi {
    extern "C++" {
        type Foo;
    }
    impl WeakPtr<Foo>;
}

fn foo_thing(foo: WeakPtr<ffi::Foo>) {}

It fails with the trait `WeakPtrTarget` is not implemented for `Foo`.
This also happens with UniquePtr (and I'd assume SharedPtr, but I didn't test that).
It's not clear to me whether it's intended for these impls to generate these internal trait impls, but it would be nice to have some mechanism to do it. As a workaround I have a dummy function taking WeakPtr<Foo> to force generation.

@jsgf Do you happen to know what the issue was? I have roughly the same issue and can't see what to add, despite looking at all the tests.

I don't remember specifically, but I think I may have got my ffi and non-ffi types mixed up:
struct MyThing(ffi::MyThing);

fn foo(foo: WeakPtr<MyThing>) { ... } // WRONG: should be ffi::MyThing


Pass xhr in the condition function

Sometimes, xhr doesn't have url property. (Example: www.rbauction.com is using responseURL) In order for this to be backward compatible, we will pass the xhr in the url condition function and let the user deal with it.
Status: Ready to Merge
Reviewers: @donnielrt @johnboxall
How to test this PR

Pull repo
Run npm install and  bower install
Run grunt test

Changes

Add xhr as an additional parameter to the condition function
Fix test
Add condition parameter test

@Helen-Mobify we will need to update the readme too. Looks like we have another couple of PRs open, we can probably merge them all into a common release branch?

@Helen-Mobify this is looking good - but could you submit a change to the README regarding this change in API? Once we have that we can get this merged in.

@Helen-Mobify reopening this against the version-0.2.0 branch!


xComputerManagement should contain resource for virtual memory management

It would be beneficial if the module would contain a resource to manage the virtual memory settings.

...and I initially wanted to take over the implementation, but I am not able to assign myself to the issue. Please feel free to assign me to it.


Integrating VertexAI / Google LLMs

Should integrate the VertexAI / Google Cloud Platform LLMs (already on LangChain) such as:
https://github.com/hwchase17/langchain/blob/master/langchain/llms/vertexai.py#L72

https://github.com/logspace-ai/langflow/issues/419 will be relevant as well in order to close this issue

I'll see if I can get my hands on it. I'm having trouble getting LangChain to recognize the credentials.

Sounds good, and the credentials for which part? Happy to help

I added the node, but running the gcloud set project ... didn't make the langchain class recognize the credentials for some reason.

I added the node, but running the gcloud set project ... didn't make the langchain class recognize the credentials for some reason.

I have noticed some issues with the langchain integration picking up ADC (specifically the default Project ID) which seem to be resolved by running aiplatform.init(project=PROJECT_ID_HERE). Maybe try that?

wondering if/when we maybe able to expect any progress here in using GCP VertexAI LLM models and services within LangFlow, like we already can in LangChain?

I'm happy to assist from the Google side. I'm a backend troll and can't typescript to save my life, but I'm very familiar with our API platform/auth/etc and the python client libraries side of things. @ogabrielluiz or anyone else familiar with the frontend/langflow codebase side of the house, let me know if you want to set up a pair troubleshooting session to get that auth issue knocked out real quick (or just direct me to the relevant code, whatever's easiest)

Hey @TMRolle.
I've created a PR (#555) end added the Component to it. You should be able to drop it in the canvas and (try to) run it.
Should I set up a function to run google.auth before instantiating it?


PublishTestResults@2 task encapsulates all the test results in a single test run

Hello,
Could anybody please help me with making "PublishTestResults" task upload test results correctly without merging them in a single test run? Some details are below.
I have several test runs which results I need to upload to ADO. They are stored in several *trx files. How can I upload these test results without merging them into one test suite? I need to have the same amount of test suites as a number of *trx files.
There are the tool parameters I applied for uploading:

task: PublishTestResults@2

displayName: 'Publish Test Results **/*.trx'
inputs:
testResultsFormat: VSTest
testResultsFiles: '**/*.trx'
searchFolder: '$(ResultsFileSource)'
mergeTestResults: true
testRunTitle: '$(TestRunTitle)'
condition: succeeded()
When I change "mergeTestResults" parameter value on "false" - it still encapsulates all tests in a single test run.
Please let me know how I can use this task correctly. Feel free to reach me by mail<EMAIL_ADDRESS>or by Microsoft Teams at any time you like.
Thanks!

Hello, I looked through the source code of PublishTestResults@2 task and found a constraint there which encapsulates all runs in one when the amount of .trx files is more than 100. But for the enterprise project I'm working on it is too few. We have about 500 ones.
Is it possible somehow to get around this restriction? I would like to be added as a contributor and make this task better and more flexible.
Thank you.

@V-vltru I am closing this issue as you can use results.publish vso command to publish/create the runs way you need it.

@navin22 how does the invocation of the results.publish vso command work? Similar to these https://github.com/microsoft/azure-pipelines-tasks/blob/master/docs/authoring/commands.md?
I tried it like so: ##vso[results.publish testResultsFormat=VSTest;testRunTitle=projectName]/path/to/project.trx but that didn't work. So the vso command has different Parameters than the Task?


mx.symbol.load.json not exported in R-package NAMESPACE

NAMESPACE file has to be modified manually to include:
export(mx.symbol.load.json)

I have updated docs in #1500
This should be fixed after merging.


Adding a header to all requests

Hi all,
is there a way to add a header  ( such as X-Auth-Token) to all requests? My authorization depends on this header area so it needs to be put in the request.
Thanks in advance.

Hi, yes, you can do that.
At least if you're using admin-on-rest components in custom app. In this case you can add any headers in your rest-client.

Yes you have to implement your own rest client

i have followed the guides, created the react app and the admin panel.
there is a scala rest api server of mine, so i needed to write a custom rest client for that. I did as in the examples and for the sake of testing i put my custom header to the options variable in method convertRESTRequestToHTTP. Like this;
options.headers['X-Auth-Token'] = token;
Then i used this custom rest client on the requests, like this;
<Admin restClient={scalaRestClient}>               <Resource name="agents" list={AgentList}/>   </Admin>
but there weren't this header on my reqeust when i open the chrome dev console and check the request (Also my api server returns unauthorized which should be happen when there were no or invalid header, i am sure token is valid)
I am new to the react so sorry for terms i used above.

Can you put the rest client code on a codepen ?

this is the rest client i am using, exactly same in the examples except the header part

If you look at the fetch code you'll notice that authorization header like this will be handled for you, you just have to pass a user option like this:
const token = "token";

const options = {
    user: {
        authenticated: true,
        token,
    },
};


sails.lift returns undefined

I think sails.lift() should return an object I can use and lower... does that make sense? Should I use sails as a singleton instead? Come to think of it, can I lift multiple apps at once?

First off, funny profile pic. Second, the answers are no, yes and sort of, respectively.
Sails is currently doing global stuff a lot by default... So there are no application instances. I could give you my opinion on the matter but that'd take ages.
Sails does use an instance under water (which you could use, too) but that really isn't being advertised anywhere. You could write up your own app.js (I recommend doing that anyway) where you start these instances yourself. If you do decide to go down this road, don't forget to disable all globals in config/globals.js or you'll get clashes. Come to think of it, chances are that you'll encounter some issues when lifting multiple instances in the same process anyway. Share your findings with us, I'd love to know how it goes. I'm also curious what you're trying to do :)

sails.lift returns an instance of the lifted app in the callback:
require('sails').lift(options, function(err, app) {
   app.lower();
});

You can absolutely raise and lower multiple instances.  You'll want to make sure you 1) use the Sails constructor to get new instances and 2) turn off globals in the apps' settings, as @RWOverdijk intimated:
// Use the .Sails() constructor to create two separate instances
var sails1 = require('sails').Sails();
var sails2 = require('sails').Sails();
sails1.lift({globals: false}, function(err, app1) {...}
sails2.lift({globals: false}, function(err, app2) {...}

If you don't turn globals off in the apps, then you can end up with collisions in the global space with models, controllers, etc.  With globals turned off, you'll still have access to the app's Sails instance in controllers, services, etc. as this.sails, and in controller actions as req._sails.  Models are then available under sails.models[<model identity>].

Thanks @sgress454!


fix ListModelInfoTests for Python2/non postgres

Some of the tests in ListModelInfoTests are broken on master; This PR fixes them.

Coverage increased (+3.3%) to 71.147% when pulling 72c31843867c6a5fdf0f619249cc1a0c39942645 on terencehonles:fix-tests into 85b89d629f8ad17a048eb12baa20fe88b18eb7c1 on django-extensions:master.

Thanks !


Ribbon menu support

What changed?
Describe the changes you have made to improve this project.
Closed issues.

@anandkishore987 The best way to have Ribbon menu support is to use Fluent.Ribbon. MahApps will not continue their own old implementation.


Redis

@dbolson @ottbot
Adds redis period limiter. This relies on redis 2.6.12 for additional options on the set command.

Does this need anything in the readme updated?


Error while running "make"

Need help! Stuck over this issue from last 2 weeks but couldn't get build successfull. Any help would be appreciated.
/cygdrive/c/Users/sudhakar/Downloads/llvm-9.0.1.src/include/llvm/ADT/FoldingSet.h:376: undefined reference to `llvm::FoldingSetNodeID::ComputeHash() const'
collect2: error: ld returned 1 exit status
make[3]: *** [lib/MemoryAnalysisModules/CMakeFiles/MemoryAnalysisModules.dir/build.make:629: lib/MemoryAnalysisModules/cygMemoryAnalysisModules-9.dll] Error 1
make[3]: Leaving directory '/cygdrive/d/P.hD/2021/SCAF-master/scaf-build-debug'
make[2]: *** [CMakeFiles/Makefile2:320: lib/MemoryAnalysisModules/CMakeFiles/MemoryAnalysisModules.dir/all] Error 2
make[2]: Leaving directory '/cygdrive/d/P.hD/2021/SCAF-master/scaf-build-debug'
make[1]: *** [Makefile:150: all] Error 2
make[1]: Leaving directory '/cygdrive/d/P.hD/2021/SCAF-master/scaf-build-debug'
make: *** [Makefile:18: scaf-debug] Error 2

We compiled the LLVM, and installed only LLVM-headers. The path provided by the collaborator is to find from LLVM directory which is in the downloads folder of yours.

Both headers and libraries are needed for compilation. The former for compiling individual object files, the latter for linking. Sorry I don't quite get the second sentence

Can you post lib/MemoryAnalysisModules/CMakeFiles/MemoryAnalysisModules.dir/link.txt to see which LLVM path is used?

/usr/bin/c++.exe -std=c++17 -g   -Wl,-rpath-link,  -Wl,-O3 -Wl,--gc-sections -shared -Wl,--enable-auto-import -o cygMemoryAnalysisModules-9.dll -Wl,--out-implib,libMemoryAnalysisModules.dll.a -Wl,--major-image-version,9,--minor-image-version,0 CMakeFiles/MemoryAnalysisModules.dir/AcyclicAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/AnalysisTimeout.cpp.o CMakeFiles/MemoryAnalysisModules.dir/ArrayOfStructures.cpp.o CMakeFiles/MemoryAnalysisModules.dir/AutoRestrictAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/BasicLoopAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/CallsiteBreadthCombinator.cpp.o CMakeFiles/MemoryAnalysisModules.dir/CallsiteDepthCombinator.cpp.o CMakeFiles/MemoryAnalysisModules.dir/CallsiteSearch.cpp.o CMakeFiles/MemoryAnalysisModules.dir/ClassicLoopAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/DisjointFieldAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/FieldMallocAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/FindSource.cpp.o CMakeFiles/MemoryAnalysisModules.dir/GetCallers.cpp.o CMakeFiles/MemoryAnalysisModules.dir/GlobalMallocAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/IntrinsicAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/Introspection.cpp.o CMakeFiles/MemoryAnalysisModules.dir/KillFlow.cpp.o CMakeFiles/MemoryAnalysisModules.dir/LLVMAAResults.cpp.o CMakeFiles/MemoryAnalysisModules.dir/LoopAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/LoopVariantAllocation.cpp.o CMakeFiles/MemoryAnalysisModules.dir/NoCaptureFcn.cpp.o CMakeFiles/MemoryAnalysisModules.dir/NoCaptureGlobalAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/NoCaptureSrcAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/NoEscapeFieldsAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/PHIMazeAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/PureFunAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/RefineCFG.cpp.o CMakeFiles/MemoryAnalysisModules.dir/SCEVAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/SVFResults.cpp.o CMakeFiles/MemoryAnalysisModules.dir/SemiLocalFunAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/SimpleAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/StdInOutErr.cpp.o CMakeFiles/MemoryAnalysisModules.dir/SubloopCombinator.cpp.o CMakeFiles/MemoryAnalysisModules.dir/TraceData.cpp.o CMakeFiles/MemoryAnalysisModules.dir/TypeAA.cpp.o CMakeFiles/MemoryAnalysisModules.dir/UniquePaths.cpp.o

You don't have any of the LLVM libraries linked (most likely CMake not detecting an LLVM installation). There should be a -Wl, -rpath, PATH_TO_LLVM_INSTALL/llvm at the end of that file if it is being linked correctly. Are you using WSL? Building and running SCAF on Windows has never been tested and we do not support any OS that is not UNIX-like.

You don't have any of the LLVM libraries linked (most likely CMake not detecting an LLVM installation). There should be a -Wl, -rpath, PATH_TO_LLVM_INSTALL/llvm at the end of that file if it is being linked correctly. Are you using WSL? Building and running SCAF on Windows has never been tested and we do not support any OS that is not UNIX-like.


[KYUUBI #2887] Add a POLLING balance policy for spark engine pool

Why are the changes needed?

As described in #2887, random policy may cause task-hot-issues or production accident, so a POLLING balance policy is added to avoid this problem, we can "set kyuubi.engine.pool.balance.policy = POLLING" to use it.
How was this patch tested?

[x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

please run below command to re-gen the docs.
KYUUBI_UPDATE=1 build/mvn clean install -Pflink-provided,spark-provided,hive-provided -DwildcardSuites=org.apache.kyuubi.config.AllKyuubiConfiguration

A high-level comment, I checked your closed PR using o.a.c...DistributedAtomicInteger, I think it's a good idea, to make it decouple w/ zookeeper, we can add a method in
traits DiscoveryClient {
  ...
  def getAndInc(path: String): Int
}

For Zookeeper, DistributedAtomicInteger can be used to implement, and for ETCD, explicitly lock and get then set may be required. We also got reported that some guys want a JDBC-based DiscoveryClient implementation, and implementing the method using SQL is not much complex.

Codecov Report

Merging #3662 (02a8311) into master (a6832b7) will increase coverage by 0.10%.
The diff coverage is 90.47%.

@@             Coverage Diff              @@
##             master    #3662      +/-   ##
============================================
+ Coverage     51.94%   52.04%   +0.10%
  Complexity       13       13
============================================
  Files           484      485       +1
  Lines         27136    27235      +99
  Branches       3784     3796      +12
============================================
+ Hits          14095    14174      +79
- Misses        11675    11686      +11
- Partials       1366     1375       +9

Impacted Files
Coverage Δ

.../org/apache/kyuubi/ha/client/DiscoveryClient.scala
36.36% <ø> (ø)

...ha/client/zookeeper/ZookeeperDiscoveryClient.scala
69.88% <66.66%> (-0.06%)
:arrow_down:

...he/kyuubi/ha/client/etcd/EtcdDiscoveryClient.scala
67.64% <90.00%> (+1.39%)
:arrow_up:

...ain/scala/org/apache/kyuubi/engine/EngineRef.scala
74.24% <90.90%> (+1.13%)
:arrow_up:

...in/scala/org/apache/kyuubi/config/KyuubiConf.scala
97.28% <100.00%> (-0.07%)
:arrow_down:

...yuubi/server/metadata/jdbc/JDBCMetadataStore.scala
89.27% <0.00%> (-0.70%)
:arrow_down:

.../apache/kyuubi/server/api/v1/BatchesResource.scala
68.91% <0.00%> (-0.64%)
:arrow_down:

...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala
80.74% <0.00%> (-0.63%)
:arrow_down:

...a/org/apache/kyuubi/service/TFrontendService.scala
91.17% <0.00%> (-0.30%)
:arrow_down:

... and 5 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

thanks @ChrisYu78, overall lgtm

thanks @ychris78, merging to master

thanks @ychris78, merging to master

thanks, merged to master

The title is a bit misleading, this policy should work for all engine types.

The title is a bit misleading, this policy should work for all engine types.
thanks, corrected


[SPARK-48548][SQL] Perf improvement for escapePathName

What changes were proposed in this pull request?

Add a faster path for storage paths or their parts that do not require escaping to avoid creating a StringBuilder to append per character.
Why are the changes needed?

performance improvement for hotspots
Does this PR introduce any user-facing change?

no
How was this patch tested?

existing tests
Was this patch authored or co-authored using generative AI tooling?

no

cc @cloud-fan

Do we have any unit tests for escapePathName?

https://github.com/apache/spark/blob/f9542d008402f8cef96d5ec347583c7c1d30d840/sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala#L1803

Do we have any unit tests for escapePathName?

https://github.com/apache/spark/blob/f9542d008402f8cef96d5ec347583c7c1d30d840/sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala#L1803

Although it's true that those tests indirectly cover these paths, by "unit test" I was referring to a dedicated test of just escapePathString itself which actually asserts over the form of the output. As it stands now, the test in DDLSuite is asserting over some end-to-end functionality to check that the path escaping logic is actually invoked, but that test would not fail if we made breaking or incompatible changes to the escaping logic itself. For example, if we switched from uppercase to lowercase hexadecimal conversions then that change wouldn't cause the DDLSuite test to fail.
In addition, the more "end-to-end" tests are slow to run.
I think it can still be useful to have relatively more end-to-end tests covering important scenarios (since special characters in paths could cause problems at other layers and we might want more end-to-end coverage for such potential cases), but I think we should augment those tests with dedicated unit tests for the helper functions to assert that their actual output doesn't change.
This will be useful if we choose to optimize the int -> hex conversion (which I'll suggest in another comment).

I add some further optimization now the result become
[info] OpenJDK 64-Bit Server VM 17.0.10+0 on Mac OS X 14.5
[info] Apple M2 Max
[info] Escape Tests:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
[info] ------------------------------------------------------------------------------------------------------------------------
[info] Legacy                                             5557           5613          36          0.2        5557.3       1.0X
[info] Kent                                                693            712          36          1.4         692.8       8.0X
[info] Josh                                                878            889          10          1.1         878.4       6.3X

I will add the uts you suggested

This optimized version looks good to me overall. Could you check in the new unit test suite and update the PR description to note the improved test coverage?
Let's give this a bit to see if anyone else has feedback, then I'll be glad to help merge this either tomorrow night or next week.

Gentle ping @dongjoon-hyun @cloud-fan, can you take a look at this improvement when you have some time? Thanks in advance.

Merged into master for Spark 4.0. Thanks @yaooqinn and @JoshRosen


Missing data in an observed flows will give a nan for the Nash-Sutcliffe

I was testing the calibration of models with the HYSETS_2020_ERA5 dataset and I found out that when a data is missing in the observed streamflow, the calibration fails and the Nash-Sutcliffe is set to nan. It gives back the calibrated parameters but I'm not sure they can be considered valid since the "fitness" is null. I don't know if this behavior is just when a flow is missing or when any data is missing, since the temperature and precipitation are all filled properly.
Finally I don't know if its Ouranos or a backend issue, so I'll open the issue here, but we can move it if its a backend issue.
Here's some image of the issue :
Streamflow missing :

Result :

If you want to reproduce the issue, the watershedId that I took was 23.

The NSE is computed by RavenC itself, not the Python wrapper, so I suspect there isn't much we can do here, except raising the issue with James.


AWS provider V4 compatibility

what

Replace cloudposse/lb-s3-bucket directly with cloudposse/s3-log-storage.
Use the latest version of cloudposse/s3-log-storage for AWS provider V4 compatibility

why

Module doesn't work with AWS provider v4
cloudposse/lb-s3-bucket doesn't seem to be maintained and doesn't add much value

## warning

migration can be destructive. see cloudposse/s3-log-storage warnings.

Thank you for this contribution. We are going to put it on hold for now, since the module works with AWS provider v4.9.0 and the changes here are not backward compatible.

The code in this module itself is compatible, but its submodule cloudposse/lb-s3-bucket isn't.
Therefore, if one tries to use this module in a code with aws provider >4.0, it fails with the following message:
╷
│ Error: Unsupported attribute
│
│   on .terraform/modules/alb.access_logs.s3_bucket/main.tf line 30, in resource "aws_s3_bucket" "default":
│   30:         for_each = var.enable_glacier_transition ? [1] : []
│
│ This object does not have an attribute named "enable_glacier_transition".
╵
╷
│ Error: Unsupported attribute
│
│   on .terraform/modules/alb.access_logs.s3_bucket/main.tf line 44, in resource "aws_s3_bucket" "default":
│   44:         for_each = var.enable_glacier_transition ? [1] : []
│
│ This object does not have an attribute named "enable_glacier_transition".
╵

That happens even if access logging is turned off:
module "alb" {
  source  = "cloudposse/alb/aws"
  version = "0.36.0"

  access_logs_enabled               = false
  access_logs_s3_bucket_id          = null
  # ...

I believe that's why #110 was also created – this module isn't compatible with v4.

@rstml See #113, which is confirmed to work with AWS v4.9.0

Ah, thanks for clarifying. Didn't realise things got better in 4.9.


bg-gradient not working

Background gradients are not supported by React Native so NativeWind cannot support them. I'm not aware of any suitable workaround to resolve this.

For others here looking for an alternative, check out https://docs.expo.dev/versions/latest/sdk/linear-gradient/.


Explore the possibility of Github Copilot integration

Are there any plans for integrating Github Copilot with Spyder? https://copilot.github.com/

That's a shame, but thanks for confirming, @ccordoba12.


Reading an inp mesh file causes a AttributeError: 'numpy.ndarray' object has no attribute 'append'

I try to read an abaqus inp mesh of a submarine that be can be found on Internet, that is about 39Mo.
http://dsk.ippt.pan.pl/docs/abaqus/v6.13/books/eif/undex_subbody_h005.inp
After removing trailing spaces and trailing comma from this file, I try to read it with meshio. However, I encounter an error, that does not seem to be related with the mesh, but more with meshio. Here is the log of the error on the fixed input file (no trailing space, and a removed trailing comma line 818313)
import meshio
meshio.__version__
# '2.0.4'
mesh = meshio.read('undex_subbody_h005.inp')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-efa787a1d89c> in <module>()
----> 1 mesh = meshio.read('undex_subbody_h005.inp')

c:\program files\python37\lib\site-packages\meshio\helpers.py in read(filename,
file_format)
    151     )
    152
--> 153     return format_to_reader[file_format].read(filename)
    154
    155

c:\program files\python37\lib\site-packages\meshio\abaqus_io.py in read(filename
)
     94     """
     95     with open(filename, "r") as f:
---> 96         out = read_buffer(f)
     97     return out
     98

c:\program files\python37\lib\site-packages\meshio\abaqus_io.py in read_buffer(f
)
    124                 points, point_gid = _read_nodes(f, point_gid, points)
    125             elif word.startswith("ELEMENT"):
--> 126                 cells = _read_cells(f, word, cells)
    127             elif word.startswith("NSET"):
    128                 params_map = get_param_map(word, required_keys=["NSET"])

c:\program files\python37\lib\site-packages\meshio\abaqus_io.py in _read_cells(f
, line0, cells)
    181             break
    182         data = [int(k) for k in filter(None, line.split(","))]
--> 183         cells[t].append(data[-num_nodes_per_elem:])
    184
    185     # convert to numpy arrays

AttributeError: 'numpy.ndarray' object has no attribute 'append'

Isn't there a smaller mesh with the same problem?

Also, I'm getting a different error here:
ValueError: invalid literal for int() with base 10: '\n

Read function fails because of multiple identical declarations of *Element. In the file, there are two sections
*Element, type=S3R

The first one is converted to a numpy array, then one can not append to a numpy array.
I am testing this to fix the bug (if one can have multiple sections with identical keys)
def _read_cells(f, line0, cells):
    sline = line0.split(",")[1:]
    etype_sline = sline[0]
    assert "TYPE" in etype_sline, etype_sline
    etype = etype_sline.split("=")[1].strip()
    if etype not in abaqus_to_meshio_type:
        msg = "Element type not available: " + etype
        raise RuntimeError(msg)
    t = abaqus_to_meshio_type[etype]
    num_nodes_per_elem = num_nodes_per_cell[t]
    current_cells = []
    while True:
        last_pos = f.tell()
        line = f.readline()
        if line.startswith("*"):
            break
        data = [int(k) for k in filter(None, line.split(","))]
        current_cells.append(data[-num_nodes_per_elem:])

    # convert to numpy arrays
    # Subtract one to account for the fact that python indices are
    # 0-based.
    current_cells = numpy.array(current_cells, dtype=int)
    if t in cells:
        cells[t] = numpy.concatenate((cells[t], current_cells), axis=0)
    else:
        cells[t] = current_cells

    f.seek(last_pos)
    return cells

I just fixed some things about the abaqus reader in #269; it should now be much faster as well.
One remaining problem is that meshio doesn't support referencing sets yet (like abaqus does); the same goes for multiple meshes per file (or "Instances", as abaqus calls it).

Thanks @nschloe , I have tested the new code with undex_subbody_h005.inp file. It returns the following error: I don't know if the input file does not respect inp format, or if the python script is too strict...
import meshio
mesh = meshio.read('undex_subbody_h005.inp')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-65aa98e1854a> in <module>()
----> 1 mesh = meshio.read('undex_subbody_h005.inp')

~/meshio/meshio/helpers.py in read(filename, file_format)
    151     )
    152
--> 153     return format_to_reader[file_format].read(filename)
    154
    155

~/meshio/meshio/abaqus_io.py in read(filename)
    91     """
    92     with open(filename, "r") as f:
---> 93         out = read_buffer(f)
    94     return out
    95

~/meshio/meshio/abaqus_io.py in read_buffer(f)
    123             elif word.startswith("NSET"):
    124                 params_map = get_param_map(word, required_keys=["NSET"])
--> 125                 setids = read_set(f, params_map)
    126                 name = params_map["NSET"]
    127                 if name not in nsets:

~/meshio/meshio/abaqus_io.py in read_set(f, params_map)
    236         if line.startswith("*"):
    237             break
--> 238         set_ids += [int(k) for k in line.strip().strip(",").split(",")]
    239     f.seek(last_pos)
    240

~/meshio/meshio/abaqus_io.py in <listcomp>(.0)
    236         if line.startswith("*"):
    237             break
--> 238         set_ids += [int(k) for k in line.strip().strip(",").split(",")]
    239     f.seek(last_pos)
    240

ValueError: invalid literal for int() with base 10: 'subbody1'

Ok, sorry for bothering. I close the issue for time being.

Let's leave it open, it's a bug after all! :)


Stripe Refunds: No Button

Expected behavior:
A button is displayed where an admin can refund a Stripe payment
Actual behavior:
No button is displayed
Reproduce steps:
I can't seem to find any button. I looked in an order inside the "Order", "Menu Items", "Status History" and "Payment Attempt" tabs and also in the orders list.
Version:
v3.0.4-beta.28
Additional Information:
(Additional information, if any)

Are these legacy payments? The refund button only appears for payments made after the update. Also only with the stripe payment gateway as things stand.

No, these are payments on the latest version that happened after the update. All on stripe as Live payments.
The only thing that I can think might be different if you're not having the same problem is that I renamed the payment gateway to something like "Secure online Payment" so that it makes more sense to the customer...

Hmm, seems we missed a step - see code changes - https://github.com/tastyigniter/ti-ext-payregister/pull/21

Just put an order through via Stripe's Test API and could not see a refund button still.
Admin Order page loading and Payment Attempts is slightly slower at loading now, which might indicate that this update has allowed for some API communication to work.
I am also using the Stripe Authorize extension with Order Approver, and the payment gateway settings are set to "Authorize Only" if that makes any difference.

Just put an order through via Stripe's Test API and could not see a refund button still.
Admin Order page loading and Payment Attempts is slightly slower at loading now, which might indicate that this update has
allowed for some API communication to work.
I am also using the Stripe Authorize extension  ith Order Approver, and the payment gateway settings are set to "Authorize
Only" if that makes any difference.

There is no API communication in the background -it just happens at the point of order / or refund.
It should mark the is_refundable column in the payment_logs to be 1, and when that happens the refund appears in the "Payment attempts" tab on the order.
I've updated the PR with new code

That fixes it. Thanks.


Endpoint performance benchmarking/profiling

Obviously the time to e.g. retrieve a Node etc is very dependent on aiida-core, plus postgres setup, plus hardware etc.
But... it would be nice if we had some basic feedback on the magnitude of the times related to different endpoints, and flag any that are particularly problematic.
For example, in aiida-core I set up: https://aiidateam.github.io/aiida-core/dev/bench/ubuntu-18.04/django/
Perhaps, there is some pydantic specific tool to achieve this?

Note on this, a think that I have played around with in the past is that on the postgres database you can set it up with the pg_stat_statements module, and then get back statistics on how many queries are being made to the database (see e.g.
https://github.com/chrisjsewell/aiida-profiling/blob/master/aiida_perf/db_stats.py)
I've never quite managed yet how to many this easy to incorporate into e.g. a pytest run (you need to maybe have pgtest setup the test database with the module, then have a fixture that resets the statistics before each test)

Copying some comments by @CasperWA and @flavianojs, from the last aiida meeting here:

As being invited to a meeting by Flaviano, we discussed speed issues concerning queries with the REST API, but it should be solved with the “new” /querybuilder endpoint.
They have a need to be able to have better speed when querying AiiDA.
This is a general issue for any (G)UI front-end for AiiDA and is even an issue for the OPTIMADE servers - the difference in speed is apparent when comparing with Materials Project, who implements a “vanilla” optimade server with a dedicated MongoDB back-end.

Firstly, it would be helpful if you could think of any "metrics" we should be aiming for?
Off the top of my head, something like "retrieving all formulas of StructureData from a database of 1 million structures in under 0.1 second". Obviously this may be difficult in practice to codify exactly in to a test, but it starts us pointing in the right direction.
In terms of the \querybuilder endpoint, yeh err not a big fan (at least in its current form). Just because it is quite "unstructured" at present (difficult to validate) and has no kind of rate limiting etc (exposes the server to intended or unintended DDOS).
It might be helpful to get a sense of what queries you are currently making with this endpoint, and also have a look at https://aiida-restapi.readthedocs.io/en/latest/user_guide/graphql.html and comment if this would achieve your use-cases? (e.g. you can query for multiple things in a single request and also everything goes through the querybuilder so no ORM objects are ever initialised which can be a bottleneck)
(we can spin this off into a separate issue, but just wanted to write this all down before I forget 😄)


Benchmark Documentation between pandas, fugue, and native spark.

I think it would be worthwhile to have some documentation and benchmark numbers on using pandas, Fugue(spark), and native Spark to compare between different scenarios.

In upcoming release #315. Closing.