Conditional branching and bool for arrays

In discussions with @jpivarski and @pfackeldey at pyhep.dev we identified a few aspects of behavior that I believe could be improved.
Short version

bool() on a dask-awkward is True at the moment, which can result in unintended behavior
automatic evaluation of Dask expressions to determine conditional branching is unintuitive and I believe instead raising an exception with dask-awkward might be a better idea

bool() on a dask-awkward array
import dask_awkward as dak

arr = dak.from_lists([[0, 0]])
bool(arr)

evaluates as True. This seems independent of the content of the array. The example above should raise an exception. If it is already known that the bool() does not make sense, such in this case where the argument is not a scalar, I believe an exception such as what numpy provides makes sense:
import dask.array as da
import numpy as np

arr = da.array(np.zeros(2))
bool(arr)

-> ValueError: The truth value of a Array is ambiguous. Use a.any() or a.all().
Automatic computation for branching
As a consequence of the behavior shown above, where the array evaluates as True, branching based on the content of a dask-awkward array is not working correctly.
import uproot
import dask
import awkward as ak

ttbar_file = "https://github.com/scikit-hep/scikit-hep-testdata/"\
    "raw/main/src/skhep_testdata/data/nanoAOD_2015_CMS_Open_Data_ttbar.root"

evts = uproot.dask({ttbar_file: "Events"})
evts["meta"] = ak.zeros_like(evts.run, dtype="float")

def f(evts):
    if ak.sum(evts["meta"]) > 100:
        res = {"num_electrons": ak.sum(ak.num(evts.Electron_pt, axis=1))}
    else:
        res = {"num_muons": ak.sum(ak.num(evts.Muon_pt, axis=1))}
    return res

dask.compute(f(evts))

This returns ({'num_electrons': 69},), as ak.sum(evts["meta"]) > 100 is True, despite
bool((ak.sum(evts["meta"]) > 100).compute())

being False when actually computed. The behavior here departs from what Dask does. Consider the following variation with dask.array:
import uproot
import dask
import awkward as ak
import numpy as np
import dask.array as da

ttbar_file = "https://github.com/scikit-hep/scikit-hep-testdata/"\
    "raw/main/src/skhep_testdata/data/nanoAOD_2015_CMS_Open_Data_ttbar.root"

evts = uproot.dask({ttbar_file: "Events"})
meta_arr = da.array(np.zeros(200))

def f(evts, meta_arr):
    if np.sum(meta_arr) > 100:
        res = {"num_electrons": ak.sum(ak.num(evts.Electron_pt, axis=1))}
    else:
        res = {"num_muons": ak.sum(ak.num(evts.Muon_pt, axis=1))}
    return res

dask.compute(f(evts, meta_arr))

In this case the last line evaluates as ({'num_muons': 41},), which is the correct answer. What happens as far as I understand is that a .compute() is triggered when building the graph and encountering if np.sum(meta_arr) > 100: dask/array/core.py#L1861 (same for scalars dask/array/core.py#L1869).
The behavior of dask.array ultimately causes the expected result to be produced, however the way in which this happens might be surprising. A possibly expensive computation can happen during what otherwise would just be graph building, in a way that is not explicitly communicated to the user.
The automatic .compute() in certain cases is also the reason for why
import dask.array as da
import numpy as np
bool(da.array(np.zeros(1))), bool(da.array(np.ones(1)))

evaluates to (False, True) without any .compute() visible.
Thoughts on expected behavior
I find the automatic .compute() very unintuitive. Users coming from pure Dask might be used to it, so I can see arguments for following the same pattern. I would however propose to raise an exception regardless when a bool() evaluation on a dask-awkward needs to happen during tracing, informing the user that they might want to consider calling .compute() manually. In the first example, this would look like the following:
if (ak.sum(evts["meta"]) > 100).compute():

instead of:
if ak.sum(evts["meta"]) > 100:

In high energy physics applications, typical use cases I have in mind where branching is needed are different processing pipelines for different types of input files. In these cases I believe it is desirable to not evaluate all possible ways to trace through the function, which I think is what a np.where does in contrast (evaluate both and then combine afterwards). This helps prevent potentially expensive calculations. Whenever possible, such branching should probably be made with static or already evaluated metadata as opposed to in-file non-evaluated metadata to prevent complications from the behavior listed above.
This behavior might catch users off guard, especially if they port awkward code over to dask-awkward. I am not sure where to best communicate this as a possible "gotcha" but I think it could be useful to spread awareness.

The implicit .compute()s are extremely unintuitive, we already see this routinely with folks using dak.to_parquet and uproot.dask_write. I would honestly suggest we change everything to be lazy by default as much as possible.
The metadata issue comes down to opinions about organizational practices and how those impact coding and data preparation.
I think some of the opinions are better justified than others, but they heavily depend on how you were taught to do data analysis and assumptions about how metadata is organized and presented to you.
I would argue that statements like:
if (ak.sum(evts["meta"]) > 100).compute():

should be avoided in columnar analysis scenarios since this pattern is the same as doing one step of cheap further processing of metadata.
i.e. using the metadata area available in nanoevents, during dataset preparation:
events.metadata["do_muons"] = (ak.sum(whatever_populates_evts["meta"]).compute() > 100)

This implies that when creating the dataset you get events from some of this information should have been synthesized already. I imagine this is probably actually talking about ATLAS's PHYSLITE MC cross sections varying event by event or something like that. In that case what you really need is a lookup tool from process bar-code to cross section, which correctionlib handily provides you. One just needs to make the appropriate lookup table and centrally provide it.

Some of this is also trying to be compliant with various other standards around. One concrete case: python len() is required to return an int, so there's no choice there.
You get an exception if you try to bool on an array or dataframe (dask or otherwise), although exactly one value is a special case for arrays.

JAX will instead raise a TracerBoolConversionError during trace-time if you try to bool on a jax.Array. To me this feels like the more natural way. In Dask the error message may want to explicitly tell the user that one can do this but then you need to consciously add a .compute() by yourself.


⚠ Processors has degraded performance

In 1a4c201, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 46 ms

Resolved: Processors performance has improved in d2ffa83.


Added Python 3.7 support to tox.ini

I of course tested this locally and worked as expected.

Some other parts that need to be set to Python 3.7. If you can fix them, I'd be happy to merge this PR.

Sorry, I just fixed this in #515 , so I will close this PR.
If you plan to make other contributions, please feel free to initiate PR.


Strange behavior with identifying double quotes

I think I'm using the latest version of the tree-sitter grammar (as included in https://github.com/emacs-tree-sitter/tree-sitter-langs though looking at it, I'm not entirely sure I have a way of telling what variation of the grammar I've got).  I'm using typst-ts-mode for Emacs, however as far as I can tell, the syntax coloration is handled through tree-sitter so I THINK this is the right place to address a potential bug.
I'm seeing some strange marking of double quote marks.  In my current document, it starts when I use the #show regex() command.  The code that produces the error is:
//
// Convenience Formatting
//
// Helps highlight To Do tagging
#show "TODO": it => text(red, strong(it))
// Forces any string matching the expression for hex number to be monospaced
// automatically.  The string splicing is a hack making the renderer avoid
// infinite recursion.  Cannot be used with _ separators as this breaks
// text nodes in Typst.
#show regex("0x[a-fA-F0-9]*"): it => raw(it.text.at(0)) + raw(it.text.slice(1))
// Forces any string matching the expression for a binary number to be
// monospaced automatically.  The string splicing is a hack making the renderer
// avoid infinite recursion.
#show regex("0b[01]*"): it => raw(it.text.at(0)) + raw(it.text.slice(1))

The syntax errors begin at the first regex as shown here:

Note that it does not catch the first " of the regex instruction, but it DOES catch the 2nd " leading to a whole slew of issues throughout the document later.  This Typst code does compile without error and I've checked the syntax here on the online Typst editor and it's working there.  The code is fine, however I think the tree-sitter grammar has an error.
Or if this has been fixed, it's because the bundled libraries I've got are out of date.  That's also a possibility.  I'm in Windows so it's not a particularly easy thing to build ancillary libraries for whatever I want and I lean on the bundled repo.

I cannot reproduce this problem using either of the dynamic library from release 0.12.81 or compiling Typst tree sitter parser from source. I'm on Linux, using Emacs master branch.
Can you check you have latest typst-ts-mode installed?

Okay, I have updated everything.  The tree-sitter grammar is still at 13 and typst-ts-mode is as listed above at 0.9.0 with the last commit of 39a9e63.  However I have discovered something new..
That code snippet placed into a NEW file properly parses!  Whatever has happened, happened earlier to throw off the lexing.  I am now in a process of copying over bits of the document and trying to figure out exactly where the first error appears.  Will update as soon as I can.

That was relatively quick.  I have found the issue though I do not know what the formal rule is.  So I had previously:
#set list ( blah, blah, blah) // One space between list and (

This is correct Typst grammar, at least according to Typst itself both the website app and the standalone build.  HOWEVER, both the website app and the tree-sitter grammar do not lex if there is a space.  It very much wants it to be:
#set list(blah, blah, blah) // No space between list and (

So, I have it fixed.  However the language specification does appear to permit a space between a function name and the parameters.  It might be wise for either Typst to decide this is illegal, or for grammars to show correctly if there is a space.

Here's what I see exactly (I kind of summarized).  Correct version:

Incorrect version (space between item and paren):


lifecyle: Add a forget() API for Link

Default behaviour is that the Drop trait calls detach() when the program is terminated (or panics).
For some types of long running programs you may wish to intentionally forget() the fd wrapped by Link, so you can keep the program running after exit. Query and detach can happen out-of-band for some program types (e.g via netlink, or bpf_prog_query)

I'm not sure it should be called forget(), but then again I am struggling to come up with a better name. Maybe something like keep_program_attached(), disown(), or keep_attached(). Maybe I'll come up with a better name.


Latest version breaks Git Bash integration on Windows 10.

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.65.0
OS Version: Windows 10

Steps to Reproduce:

Update to 1.65.0

Latest update broke Git Bash integration. Git Bash no longer appears in the list of available terminals. "Git Bash" is not longer a valid option for "terminal.integrated.defaultProfile.windows". The Internet suggests adding a terminal by setting the path in "terminal.integrated.shell.windows" but that appears to no longer be an option.

Setting the path in "terminal.integrated.automationProfile.windows" makes bash the default but it still does not appear in the list of options. Also every time I start VSCode I get a new terminal session below the old one which I cannot clear.

Remove terminal.integrated.shell.*** entirely, see the note in the property description.

Remove terminal.integrated.shell.*** entirely, see the note in the property description.

Removing the line does not solve the problem. The property description says:
recommended way to configure your default shell is by creating a terminal profile in #terminal.integrated.profiles.windows#and setting its profile name as the default in#terminal.integrated.defaultProfile.windows#.
I cannot set the name in terminal.integrated.defaultProfile.windows as it only accepts the default values. The note for this property also references deprecated properties: terminal.integrated.shell.windows or terminal.integrated.shellArgs.windows.

The configuration as described in here https://code.visualstudio.com/docs/editor/integrated-terminal does not work.

The only configuration that functions partially is using the deprecated "terminal.integrated.shell.windows". I would still prefer it function as it did prior to the update.

Try to restart vscode or re-install git to the default directory. We're not sure why the configuration you tried above is not working for you.
Does it work if you use a different name, IE Git bash 2?


Asynchronous Tests

It'd be nice to be able to test asynchronous functions (in parallel maybe?) without having to resort to so much boilerplate.
Right now, the state of the art involves HACK_NO_XPCALL and is just that -- a hack! Even if xpcall supported yielding, we don't want to block execution of other tests while we wait for a single async test to complete.

I think we're good on this now that HACK_NO_XPCALL is deprecated!


view what is using the ID

How can I view what Az resources are using a specific user-assigned managed identity?
I'm consolidating user-assigned managed IDs, but it's very difficult to do so because, unlike viewing what the ID gives access to, there seems to not be an easy way to view what resources are using the ID.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0b3b66e0-5c12-7859-4106-faaa5544fac6
Version Independent ID: 63066fab-2b7c-c592-c9a8-738155c6abff
Content: Manage user-assigned managed identities - Azure AD
Content Source: articles/active-directory/managed-identities-azure-resources/how-manage-user-assigned-managed-identities.md
Service: active-directory
Sub-service: msi
GitHub Login: @barclayn
Microsoft Alias: barclayn

@SteveColegrove Thanks for reaching out. We will investigate and update the thread accordingly.


Add chitaranjan to contributors list

@TheCodeTalker, do you wanna reopen this pull request so that I can merge it to this project?
Don't worry about any merge conflict that might occur. We'll sort it out

ya sure


Error in checkGrep(grep(".A.txt", files)) when running maegatk

Hi
I am trying to run maegatk on my dataset. I have installed all the modules required as stated in the tutorial.
java, bwa, bedtools, freebayes, R (4.1.2, with data.table, Matrix, GenomicRanges, SummarizedExperiment). I am running it on python 3.7
I have tried to run the program on both the test dataset, and my own dataset using the commands below:
maegatk bcall --input $bam -o $resul_out -c $ncores -b $barcodes -mr $minReads -z
I keep getting the same error in both instances:
Mon Mar 14 15:46:24 AEST 2022: maegatk v0.1.1
Mon Mar 14 15:46:24 AEST 2022: Found bam file: Data/test_maester.bam for genotyping.
Mon Mar 14 15:46:24 AEST 2022: Will determine barcodes with at least: 100 mitochondrial reads.
Mon Mar 14 15:46:24 AEST 2022: User specified mitochondrial genome matches .bam file
Mon Mar 14 15:46:30 AEST 2022: Finished determining/splitting barcodes for genotyping.
Mon Mar 14 15:46:31 AEST 2022: Genotyping samples with 24 threads
Error in checkGrep(grep(".A.txt", files)) :
Improper folder specification; file missing / extra file present. See documentation
Calls: importMito -> checkGrep
Execution halted
I have attached the a list of all the files generated using (ls -lRh $result_folder), scatter.log, gather.log
test_result_file_list.txt
maegatk.snakemake_scatter.log.txt
maegatk.snakemake_gather.log.txt
Any help would be greatly appreciated.
Thanks

i got the same problem..
expecting resolutions.

luckly , i seem have understand this error，
you need run bwa index with your input genome.fa file

Hi,
I am experiencing the same problem on test data.
Thank you for handling this :)

Also, the definition of -mr (minimum reads) was used incorrectly. The function used to call -C in freebayes is actually minimum alternative read, which is not the same thing as minimum read.

Also, the definition of -mr (minimum reads) was used incorrectly. The function used to call -C in freebayes is actually minimum alternative read, which is not the same thing as minimum read.

Here we use -mr only in the context of the fgbio consensus demultiplexing (https://github.com/caleblareau/maegatk/blob/6662c7c4188c2c254e9416450295553700473794/maegatk/bin/python/oneSample_maegatk.py#L70)

Hi Caleb
The original *.bam was generated using cellranger count to do the mapping of the fastqs
Best regards
Jennifer

From: Caleb Lareau @.>
Sent: Saturday, 9 April 2022 3:59 PM
To: caleblareau/maegatk @.>
Cc: Miss Jennifer Lu @.>; Mention @.>
Subject: Re: [caleblareau/maegatk] Error in checkGrep(grep(".A.txt", files)) when running maegatk (Issue #5)
Hi @uqjlu8https://github.com/uqjlu8, your error stems from an improperly specified bam file header:
Error in the scatter log (line 83):
ERROR::READ_GROUP_NOT_FOUND:Record 1, Read name NS500239:389:H3F5JBGXC:1:12210:7650:20169_GTCACAATCGGATGGA-2+CTGTCACATT, RG ID on SAMRecord not found in header: Gen22_LG_D22:0:1:H3F5JBGXC:1
This biostars post discusses it a bit more: https://www.biostars.org/p/50338/
In short, this is an error upstream of the maegatk took and something that would require you to check with how you processed your single-cell sequencing data. May I ask what pipeline you used to go from fastq to bam files?
—
Reply to this email directly, view it on GitHubhttps://github.com/caleblareau/maegatk/issues/5#issuecomment-1093720510, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AL7N3ZNRQ3FEDSFDR2YEYU3VEEMDBANCNFSM5QUQIDVA.
You are receiving this because you were mentioned.Message ID: @.***>

just to clarify-- did the test data work OK for you?

Hi Caleb, unfortunately not. I was getting the same CheckGrep error. That's why I submitted the original ticket
Regards
Jennifer

From: Caleb Lareau @.>
Sent: Saturday, 23 April 2022 11:39 AM
To: caleblareau/maegatk @.>
Cc: Miss Jennifer Lu @.>; Mention @.>
Subject: Re: [caleblareau/maegatk] Error in checkGrep(grep(".A.txt", files)) when running maegatk (Issue #5)
just to clarify-- did the test data work OK for you?
—
Reply to this email directly, view it on GitHubhttps://github.com/caleblareau/maegatk/issues/5#issuecomment-1107162348, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AL7N3ZLA7LULPEJNS4DIPETVGNIGRANCNFSM5QUQIDVA.
You are receiving this because you were mentioned.Message ID: @.***>

Hello @caleblareau,
I also face same problem.
Test data or my own data cannot proceed at  making ~/final/SAMPLE_NAME.A.txt.gz part.
Files are intact and I am quite sure this(https://www.biostars.org/p/50338/) is not the point.
Hope you provide any response.
Thanks.

I got the same problem with the test data.
I noticed that the version is 0.1.2 in setup.py, and the PyPi version is 0.1.1. Can the difference between the two versions solve this problem?
YB

version 0.1.2 was for indel calling, so I don't think it'll impact anything for you. You can try though-- just clone the repo then install it from the local branch:
``
python3 -m pip install -e maegatk

Hello
I am running into a similar issue. It seems like this might be related to this:
https://github.com/caleblareau/mgatk/issues/22
I ran maegatk with --snake-sdout flag and the error stems from the following:
AttributeError in line 54 of ~/venv3/lib/python3.6/site-packages/maegatk/bin/snake/Snakefile.maegatk.Gather
'InputFiles' object has no attribute 'As'
Any idea on how to work around this?
Stef

Hi @si3 the README file is now slightly expanded and explains what intermediate output files you should expect. If there is no As in Gather, then it means that line 35 here fails and the output of Scatter is incomplete, so I suggest looking into the Scatter log. Scatter should have generated a set of files in temp_bam,ready_bam,sparse_matrices folders. If you are keeping intermediate files, you can check which files are missing, it should also be reflected in Scatter's log.
These are the files that should be present for each CB:

-> l temp_bam/ | head
total 215944
-rw-r--r-- 1 safina vangalenlab  1387286 Jan  6 15:57 CCCGTCGTGGTA-1.temp0.bam
-rw-r--r-- 1 safina vangalenlab  1365836 Jan  6 15:57 CCCGTCGTGGTA-1.temp1.bam
-rw-r--r-- 1 safina vangalenlab 13993678 Jan  6 15:57 CCCGTCGTGGTA-1.temp1.5.sam
-rw-r--r-- 1 safina vangalenlab  1320355 Jan  6 15:57 CCCGTCGTGGTA-1.temp1.5.bam
-rw-r--r-- 1 safina vangalenlab   489313 Jan  6 15:57 CCCGTCGTGGTA-1.temp2.bam
-rw-r--r-- 1 safina vangalenlab  1925330 Jan  6 15:57 CCCGTCGTGGTA-1.temp0.fastq
-rw-r--r-- 1 safina vangalenlab  1123027 Jan  6 15:57 GACCGTGCATTT-1.temp0.bam
-rw-r--r-- 1 safina vangalenlab  1122782 Jan  6 15:57 GACCGTGCATTT-1.temp1.bam
-rw-r--r-- 1 safina vangalenlab 11246101 Jan  6 15:57 GACCGTGCATTT-1.temp1.5.sam
(base)

-> l ready_bam/ | head
total 6728
-rw-r--r-- 1 safina vangalenlab 216594 Jan  6 15:57 CCCGTCGTGGTA-1.qc.bam
-rw-r--r-- 1 safina vangalenlab     96 Jan  6 15:57 CCCGTCGTGGTA-1.qc.bam.bai
-rw-r--r-- 1 safina vangalenlab 219598 Jan  6 15:57 GACCGTGCATTT-1.qc.bam
-rw-r--r-- 1 safina vangalenlab     96 Jan  6 15:57 GACCGTGCATTT-1.qc.bam.bai
-rw-r--r-- 1 safina vangalenlab 189703 Jan  6 15:58 CCACAAAACATG-1.qc.bam
-rw-r--r-- 1 safina vangalenlab     96 Jan  6 15:58 CCACAAAACATG-1.qc.bam.bai
-rw-r--r-- 1 safina vangalenlab 261193 Jan  6 15:58 GGCGCTAATGAA-1.qc.bam
-rw-r--r-- 1 safina vangalenlab     96 Jan  6 15:58 GGCGCTAATGAA-1.qc.bam.bai
-rw-r--r-- 1 safina vangalenlab 219108 Jan  6 15:59 TTCCTACGCAAT-1.qc.bam
(base)

-> l sparse_matrices/ | head
total 27904
-rw-r--r-- 1 safina vangalenlab 159767 Jan  6 15:57 CCCGTCGTGGTA-1.A.txt
-rw-r--r-- 1 safina vangalenlab 159806 Jan  6 15:57 CCCGTCGTGGTA-1.C.txt
-rw-r--r-- 1 safina vangalenlab  77507 Jan  6 15:57 CCCGTCGTGGTA-1.G.txt
-rw-r--r-- 1 safina vangalenlab 125697 Jan  6 15:57 CCCGTCGTGGTA-1.T.txt
-rw-r--r-- 1 safina vangalenlab 308142 Jan  6 15:57 CCCGTCGTGGTA-1.coverage.txt
-rw-r--r-- 1 safina vangalenlab 162181 Jan  6 15:57 GACCGTGCATTT-1.A.txt
-rw-r--r-- 1 safina vangalenlab 160649 Jan  6 15:57 GACCGTGCATTT-1.C.txt
-rw-r--r-- 1 safina vangalenlab  77111 Jan  6 15:57 GACCGTGCATTT-1.G.txt
-rw-r--r-- 1 safina vangalenlab 129431 Jan  6 15:57 GACCGTGCATTT-1.T.txt

Do you have all of these files?

Same error here. Did you have a chance to find a fix?
Thank you.

@noranekonobokkusu ,
Thank you for your reply.
I looked into the temp directory.  There is expected number of bam files in the 'barcoded_bams' folder. However, 'temp_bam' and 'ready_bam' are empty. 'sparse_matrices' is also empty.
Would you have ideas how to proceed?

Yes, try looking into the snakemake-scatter text log file in the logs/ folder, it should have some error messages.
Also note that a recently added option --skip-barcodesplit allows you to skip the barcode splitting step if it already finished successfully. It shouldn't matter if you are running maegatk on a small test dataset.

I actually don't have that file.
Here is the content of the 'logs'
filterlogs
rmdupslogs
base.maegatk.log
maegatk.parameters.txt
maegatk.snakemake_gather.log

@NBurnaevskiy this is new. Are you running it on a test dataset? Are there any errors (or any messages) in the output of this run? (like sh.o or sh.e files, or just stdout/err?)
I also noticed that yesterday you opened and then closed a yaml-related issue. Might it be that you introduced a fix that causes snakemake-scatter not to be executed?

Yes, we had to introduce a fix into yaml command. We followed the instruction from error message and yaml documentation.
We had error message:
"AttributeError:
"dump()" has been removed, use
yaml = YAML(typ='unsafe', pure=True)
yaml.dump(...)
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/cli.py", line 300
yaml.dump(dict1, yaml_file, default_flow_style=False, Dumper=yaml.RoundTripDumper)"
In the cli.py file we changed line 300 from
yaml.dump(dict1, yaml_file, default_flow_style=False, Dumper=yaml.RoundTripDumper)
to
yaml=YAML()
yaml.default_flow_style = False
yaml.dump(dict1, yaml_file)
That allow the script not to crash as that point.
We will now to change to what error message recommended:
yaml = YAML(typ='unsafe', pure=True)
yaml.dump(dict1, yaml_file, default_flow_style=False, Dumper=yaml.RoundTripDumper)
Do you think that could cause current issue with temp files?

I suspect recent modifications @caleblareau introduced on yaml are not present in the current python package on PyPi.
Can you try replacing your entire cli.py with the code from https://github.com/caleblareau/maegatk/blob/master/maegatk/cli.py, and then try suggestions from my previous comment and report results?

I just ran a script on the test file (with our current yaml).
Exactly the same result.

I am waiting for our admin to change yaml file.

You can try installing maegatk locally which would allow you to modify the code yourself. Old yaml commands got outdated with the current version of yaml. I would try updating the maegatk code rather than downgrading yaml version.

Just got response from our admin. They did exactly what you suggested, replaced old yaml file with updated one.
Test run failed with identical symptoms.
Yes, we do have have file snake.scatter.yaml in ./internal/parseltongue folder

Error log from maegatk.snakemake_gather.log
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.gather.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Job stats:
job                           count

all                               1
make_depth_table                  1
make_final_sparse_matrices        1
total                             3
Select jobs to execute...
[Thu Mar 21 10:13:43 2024]
rule make_final_sparse_matrices:
output: /home/nburnaevskiy/maegatk_test/output/final/maegatk.A.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.C.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.G.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.T.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.coverage.txt.gz
jobid: 2
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/final/maegatk.A.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.C.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.coverage.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.G.txt.gz, /home/nburnaevskiy/maegatk_test/output/final/maegatk.T.txt.gz
resources: tmpdir=/tmp
[Thu Mar 21 10:13:43 2024]
rule make_depth_table:
output: /home/nburnaevskiy/maegatk_test/output/final/maegatk.depthTable.txt
jobid: 1
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/final/maegatk.depthTable.txt
resources: tmpdir=/tmp
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.gather.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.gather.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
gzip: /home/nburnaevskiy/maegatk_test/output/final/maegatk.A.txt: No such file or directory
gzip: /home/nburnaevskiy/maegatk_test/output/final/maegatk.C.txt: No such file or directory
gzip: /home/nburnaevskiy/maegatk_test/output/final/maegatk.G.txt: No such file or directory
gzip: /home/nburnaevskiy/maegatk_test/output/final/maegatk.T.txt: No such file or directory
gzip: /home/nburnaevskiy/maegatk_test/output/final/maegatk.coverage.txt: No such file or directory
Waiting at most 5 seconds for missing files.
[Thu Mar 21 10:13:44 2024]
Finished job 1.
1 of 3 steps (33%) done
MissingOutputException in rule make_final_sparse_matrices in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Gather, line 33:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/final/maegatk.A.txt.gz
/home/nburnaevskiy/maegatk_test/output/final/maegatk.C.txt.gz
/home/nburnaevskiy/maegatk_test/output/final/maegatk.G.txt.gz
/home/nburnaevskiy/maegatk_test/output/final/maegatk.T.txt.gz
/home/nburnaevskiy/maegatk_test/output/final/maegatk.coverage.txt.gz
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-03-21T101342.762479.snakemake.log

Now the tool produced slightly more results but crashed anyway.
We finally the file maegatk.snakemake_scatter.log.
Looks like yaml gives a problem in another file.
Its content
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Job stats:
job                   count

all                       1
make_sample_list          1
process_one_sample       10
total                    12
Select jobs to execute...
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/GGGTTCGCCTCC-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGGTTCGCCTCC-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGGTTCGCCTCC-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/GGGTTCGCCTCC-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.coverage.txt
jobid: 8
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/GGGTTCGCCTCC-1.depth.txt
wildcards: sample=GGGTTCGCCTCC-1
resources: tmpdir=/tmp
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/TGGTAGTGAACC-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TGGTAGTGAACC-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TGGTAGTGAACC-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/TGGTAGTGAACC-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.coverage.txt
jobid: 10
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/TGGTAGTGAACC-1.depth.txt
wildcards: sample=TGGTAGTGAACC-1
resources: tmpdir=/tmp
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/ACAATTAGCACT-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/ACAATTAGCACT-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/ACAATTAGCACT-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/ACAATTAGCACT-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.coverage.txt
jobid: 2
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/ACAATTAGCACT-1.depth.txt
wildcards: sample=ACAATTAGCACT-1
resources: tmpdir=/tmp
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/CCACAAAACATG-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCACAAAACATG-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCACAAAACATG-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/CCACAAAACATG-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.coverage.txt
jobid: 3
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/CCACAAAACATG-1.depth.txt
wildcards: sample=CCACAAAACATG-1
resources: tmpdir=/tmp
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/CTAACCCGGAAT-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CTAACCCGGAAT-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CTAACCCGGAAT-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/CTAACCCGGAAT-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.coverage.txt
jobid: 5
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/CTAACCCGGAAT-1.depth.txt
wildcards: sample=CTAACCCGGAAT-1
resources: tmpdir=/tmp
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/GGCGCTAATGAA-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGCGCTAATGAA-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGCGCTAATGAA-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/GGCGCTAATGAA-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.coverage.txt
jobid: 7
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/GGCGCTAATGAA-1.depth.txt
wildcards: sample=GGCGCTAATGAA-1
resources: tmpdir=/tmp
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/GTACCCACAGCC-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GTACCCACAGCC-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GTACCCACAGCC-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/GTACCCACAGCC-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.coverage.txt
jobid: 9
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/GTACCCACAGCC-1.depth.txt
wildcards: sample=GTACCCACAGCC-1
resources: tmpdir=/tmp
[Thu Mar 21 12:11:51 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/TTCCTACGCAAT-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TTCCTACGCAAT-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TTCCTACGCAAT-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/TTCCTACGCAAT-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.coverage.txt
jobid: 11
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/TTCCTACGCAAT-1.depth.txt
wildcards: sample=TTCCTACGCAAT-1
resources: tmpdir=/tmp
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
[Thu Mar 21 12:11:52 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/CCCGTCGTGGTA-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCCGTCGTGGTA-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCCGTCGTGGTA-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/CCCGTCGTGGTA-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.coverage.txt
jobid: 4
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/CCCGTCGTGGTA-1.depth.txt
wildcards: sample=CCCGTCGTGGTA-1
resources: tmpdir=/tmp
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
[Thu Mar 21 12:11:52 2024]
rule process_one_sample:
input: /home/nburnaevskiy/maegatk_test/output/.internal/samples/GACCGTGCATTT-1.bam.txt
output: /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GACCGTGCATTT-1.qc.bam, /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GACCGTGCATTT-1.qc.bam.bai, /home/nburnaevskiy/maegatk_test/output/qc/depth/GACCGTGCATTT-1.depth.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.A.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.C.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.G.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.T.txt, /home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.coverage.txt
jobid: 6
reason: Missing output files: /home/nburnaevskiy/maegatk_test/output/qc/depth/GACCGTGCATTT-1.depth.txt
wildcards: sample=GACCGTGCATTT-1
resources: tmpdir=/tmp
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Waiting at most 5 seconds for missing files.
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Config file /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 10
Rules claiming more threads will be scaled down.
Select jobs to execute...
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
Traceback (most recent call last):
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17, in
config = yaml.load(stream, Loader=yaml.Loader)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1085, in load
error_deprecation('load', 'load', arg=_error_dep_arg, comment=_error_dep_comment)
File "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
raise AttributeError(s, name=None)
AttributeError:
"load()" has been removed, use
yaml = YAML(typ='rt')
yaml.load(...)
and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 17
config = yaml.load(stream, Loader=yaml.Loader)

Waiting at most 5 seconds for missing files.
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGGTTCGCCTCC-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGGTTCGCCTCC-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/GGGTTCGCCTCC-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGGTTCGCCTCC-1.coverage.txt
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TGGTAGTGAACC-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TGGTAGTGAACC-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/TGGTAGTGAACC-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TGGTAGTGAACC-1.coverage.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/ACAATTAGCACT-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/ACAATTAGCACT-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/ACAATTAGCACT-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/ACAATTAGCACT-1.coverage.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCACAAAACATG-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCACAAAACATG-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/CCACAAAACATG-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCACAAAACATG-1.coverage.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/GGGTTCGCCTCC-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGGTTCGCCTCC-1.qc.bam GGGTTCGCCTCC-1
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/TGGTAGTGAACC-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TGGTAGTGAACC-1.qc.bam TGGTAGTGAACC-1
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/ACAATTAGCACT-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/ACAATTAGCACT-1.qc.bam ACAATTAGCACT-1
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGCGCTAATGAA-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGCGCTAATGAA-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/GGCGCTAATGAA-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GGCGCTAATGAA-1.coverage.txt
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GTACCCACAGCC-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GTACCCACAGCC-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/GTACCCACAGCC-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GTACCCACAGCC-1.coverage.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/CCACAAAACATG-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCACAAAACATG-1.qc.bam CCACAAAACATG-1
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CTAACCCGGAAT-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CTAACCCGGAAT-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/CTAACCCGGAAT-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CTAACCCGGAAT-1.coverage.txt
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/GGCGCTAATGAA-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GGCGCTAATGAA-1.qc.bam GGCGCTAATGAA-1
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TTCCTACGCAAT-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TTCCTACGCAAT-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/TTCCTACGCAAT-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/TTCCTACGCAAT-1.coverage.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/GTACCCACAGCC-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GTACCCACAGCC-1.qc.bam GTACCCACAGCC-1
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCCGTCGTGGTA-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCCGTCGTGGTA-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/CCCGTCGTGGTA-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/CCCGTCGTGGTA-1.coverage.txt
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/CTAACCCGGAAT-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CTAACCCGGAAT-1.qc.bam CTAACCCGGAAT-1
MissingOutputException in rule process_one_sample in file /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/snake/Snakefile.maegatk.Scatter, line 21:
Job 0 completed successfully, but some output files are missing. Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GACCGTGCATTT-1.qc.bam
/home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GACCGTGCATTT-1.qc.bam.bai
/home/nburnaevskiy/maegatk_test/output/qc/depth/GACCGTGCATTT-1.depth.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.A.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.C.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.G.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.T.txt
/home/nburnaevskiy/maegatk_test/output/temp/sparse_matrices/GACCGTGCATTT-1.coverage.txt
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/TTCCTACGCAAT-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/TTCCTACGCAAT-1.qc.bam TTCCTACGCAAT-1
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/GACCGTGCATTT-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/GACCGTGCATTT-1.qc.bam GACCGTGCATTT-1
python /net/module/sw/maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py /home/nburnaevskiy/maegatk_test/output/.internal/parseltongue/snake.scatter.yaml /home/nburnaevskiy/maegatk_test/output/temp/barcoded_bams/CCCGTCGTGGTA-1.bam /home/nburnaevskiy/maegatk_test/output/temp/ready_bam/CCCGTCGTGGTA-1.qc.bam CCCGTCGTGGTA-1
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-03-21T121150.050858.snakemake.log

Did they replace the yaml command or the python file in the installation folder (like, in ~/.local/lib/python3.9/site-packages/maegatk/)? snakemake_gather failed because it didn't have required input files. Are you sure you are not getting any other error messages at any stage of running it, and that you replaced cli.py entirely and didn't change anything else? and that you didn't have any existing (even empty) output directories like temp_bam and ready_bam before rerunning the command? I don't see a way to reproduce the absence of snakemake_scatter output.

@NBurnaevskiy just saw your new comment, let me take a look.

Ksenia, thank you for all your responses.

I see, so yaml syntax in oneSample_maegatk.py got outdated, too. I will recreate in on my computer (I currently have an older yaml so everything seems to work) and try fixing it.

we are iteratively working through issues. yaml update creates problems in few places.
first we updated file oneSample_maegatk.py, to follow new yaml syntax.
we updated line 16 and 17 to make this
with open(configFile, 'r') as stream:
yaml = YAML(typ='rt')
config = yaml.load(stream, Loader=yaml.Loader)
Then we updated header to change this
from ruamel import yaml
into this
from ruamel.yaml import YAML
Now we got an error:
File ".../maegatk/0.2.0/lib/python3.10/site-packages/maegatk/bin/python/oneSample_maegatk.py", line 18, in
config = yaml.load(stream, Loader=yaml.Loader)
AttributeError: 'YAML' object has no attribute 'Loader'. Did you mean: 'Reader'?

I am also trying the same - they dropped the load function https://yaml.readthedocs.io/en/latest/, but I am still figuring out what is the correct way to rewrite that

maybe it should be
config = yaml.load(stream, Loader=Loader)
?

It seems dropping the argument altogether allows the pipeline to proceed and start generating files in temp_bam/:
yaml = YAML(typ='rt')
config = yaml.load(stream)

Ok, I can confirm that config = yaml.load(stream) allows the pipeline to produce the final .rds file. @caleblareau can you confirm these changes are enough?
from ruamel.yaml import YAML
...
yaml = YAML(typ='rt')
config = yaml.load(stream)

I can confirm that the script produced final output files.
There were some complains in the log, like this
samtools: /net/module/sw/maegatk/0.2.0/bin/../lib/libtinfow.so.6: no version information available (required by samtools)
but hopefully they are not critical.
I will now try real data and let you know if it completes.
Thank you very much Ksenia. I hope that this tools will useful for us.


Refactor template selection logic for targets.

As title.
Closes #7150

/AzurePipelines run

This PR is triggering the current testing issue with SVML. Am going to keep running it to look for a pattern, am deleting azure-pipelines run confirmation comments.

/AzurePipelines run


Accordion Btn-link center text sometimes.

Just had an issue on a website where I had an FAQ in an accordion.  It was found that on smaller phone screens some of the FAQs were center aligned where as others were left aligned.
Found that btn-link was applying CSS of text-align:center but the link was set to display inline - therefore the text would only center when the FAQ went across multiple lines but would normally be left aligned at other times - problem can be seen in attached image.

Bug reports must include a live demo of the problem. Per our contributing guidelines, please create a reduced test case via CodePen/JS Bin or Stackblitz and report back with your link, Bootstrap version, and specific browser and OS details.
This is an automated reply

A live demo of this is at the following website...
http://carers.axiam-pc.co.uk/join-us
if you look at the website full screen on a big enough monitor you get the following image.

However if you reduce the size of the screen down so that the words on the button start to wrap onto a second line then you get this effect...

Bootstrap version is v4.3.1 Browser is Chrome 78.0.3904.70  and OS is Windows 10, has also been tested on Apple iPad Safari and Android Chrome too with same result.

Yeah, that's normal behaviour for inline elements. You can add .text-left to the buttons to align them to the left. Maybe also add .btn-block to increase the hit area of the buttons.
We should also update this in our docs, feel free to contribute!

ok thanks for that answer.  Didn't actually think about adding text-left to it to fix it (to fix it on the main site I added the following CSS
.accordion .btn-link {
text-align: left !important;
}
, also thanks for the advice about adding btn-block and I've just noticed that adding btn-block to it also fixes the alignment too without extra css/classes.


fix: fixing broken link

The link to CONTRIBUTING.md was broken
This commit changes the link to the version on the branch master of the
project.

Nice catch! Thanks a lot 😊


Add tests to project

The project has no tests yet. Add them using pytest.

Change ci and cd commands to jobs with mutually exclusive options.
Test if indent=2 parameter can be passed to json.dumps function.


Adding Dotnet Sample

Of course the PR might not get accepted, but I've added a dotnet sample for this, based on the node version. More info at https://github.com/OfficeDev/msteams-tabs-sso-sample-nodejs/pull/10

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 52221dd3-95ec-96b9-1da8-9478527c2602
Version Independent ID: 32e9c2d8-733d-0865-2ea3-727d8684a10d
Content: Single Sign-On - Teams
Content Source: msteams-platform/tabs/how-to/authentication/auth-aad-sso.md
Product: msteams
GitHub Login: @laujan
Microsoft Alias: lajanuar

@HiltonGiesenow, we are looking into your code.

ok great, hopefully it's of use


Routes were preprocessed and are ready to use.

Transforming the routes list which is in fact an array of strings, into a dynamic array of pointers to simplify further regex ops over it.
Excluding route IDs from all the routes in the bus routes list using regex ops.
Adding a constant: the regex pattern for the element to be excluded from a bus stops sequence: it is an arbitrary identifier of a route, which is not used in the routes processing anyhow.

She looks transformed but for the great good ! :+1: — Approved. :blue_heart:


[Feat] Modify map zoom in search and location approval.

Adjust Zoom Level During Search:

When a user performs a search, the map should automatically adjust its zoom level to provide an optimal view of the searched location.

Set Zoom Level for Location Approval:

When users are asked to approve or select a location on the map, the zoom level should be set to provide a detailed view that facilitates easy identification and selection of the exact location.

@byakter , @danielsh28 is taking this one.
If you have any insights please add them here
Daniel, there was an initial pr for this issue I reverted: it did solved the zoom adjustments but I there was also a bug related to maps (osm map instead of google map)
the revert is here, it might help
The pr itself is here

Solved in https://github.com/data-for-change/anyway-newsflash-infographics/pull/1162 and https://github.com/data-for-change/anyway-newsflash-infographics/pull/1163


[Feature request] Include/exclude filename pattern

Description

Thanks a lot for this sick library.
I'm using Next.js with CSS Modules to style pages/components, and the pages directory looks like this.
pages
  |- index.tsx
  |- index.module.scss
  |- profile.tsx
  |- profile.module.scss
     ...

The problem here is that pathpida generates page typedefs for *.module.scss files too. It's not something fatal, but could leads to some typing mistakes as the editor actually suggests pagesPath.profile_module (this PAGE doesn't exist!) as well as pagesPath.profile (disired one, apparently) when you type pagesPath.prof....
Describe the solution you'd like

Include/exclude pattern.
yarn pathpida --include "*.tsx"
or
yarn pathpida --exclude "*.module.scss"

Describe alternatives you've considered

Adding underscore _ as prefix to all the *.module.scss files solves the problem itself, but is not very elegant imo.
Additional context

Thank you for posting this.
I've had similar requests from other users, so I'll see what I can do.
I may change it to ignore scss/css extensions by default so that it can be used immediately without any configuration.

Supported in v0.15.0.


HIVE-28328:Hive Compilation Failed Due to License-Maven-Plugin URL Mismatch Error

What changes were proposed in this pull request?
HIVE-28328
as per the above jira, hive compilation is failing on both master and branch-4.0 with the mentioned failures. This PR handles the failures.
Why are the changes needed?

Does this PR introduce any user-facing change?
No
Is the change a dependency upgrade?
No
How was this patch tested?
compiled with the patch, it went well.

@basapuram-kumar, this is not failing for me, I tried with the same command present in the JIRA. Attaching the screenshot

I have seen license related error in my org artifactory but not in opensource.

Hello @Aggarwal-Raghav,
Thank you for looking into this issue. The compilation error I'm encountering is related to the 'APACHE-2.0' license keys. This error occurs consistently across different machines I've tried.
The solution I provided addresses the issue by adding the Apache-2.0 license file URLs in the pom.xml. This change appears to resolve the problem. I don't see any failures related to our organization's Artifactory.

@basapuram-kumar, I can't repro it either

not sure what you are doing here. if master build was broken all PRs build would be failing. Not sure about the Profiles you are putting in, half of them don't exist in hive.
try a simple
mvn clean install -DskipTests -Pdist

and see if it work, if it does, then there ain't any problem, I doubt the use of -pl :hive-packaging in your command as well, try without that


Ondersteun ook Magento zonder inventorymodules

Edit: translated to English.
Subject: Support Magento without Inventory modules enabled
It's not possible to use the Flowbox module when the Inventory modules are disabled. After adding a flowboxwidget to a page and saving that page, the following error is shown:

[2021-03-05 12:51:58] report.CRITICAL: Error: Cannot instantiate interface Magento\InventoryCatalogApi\Model\GetSkusByProductIdsInterface in /home/john/doe/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php:121

In code vendor/itonomy/module-flowbox/Block/Widget/Flow.php one can see that the following interface is called:
\Magento\InventoryCatalogApi\Model\GetSkusByProductIdsInterface
Could this call be rewritten so to support Magento without Inventory modules enabled. This because I don't have an option to enable those inventory modules in my Magento installation.
Tnx.

Hi there,
Thanks for the report.
We'll remove the dependency on the inventory modules in the following release.

@RaphaelBronsveld do you have an ETA when this will be released?

Hi @evs-xsarus ,
The issue has been fixed in commit https://github.com/Itonomy/module-flowbox/commit/6c29bd8cb43fc9e1e2b894a28ed00280b57b44ae. If it's high priority, you can use the composer patches package to update the code without having to rely on the version being available in composer.
We're currently waiting for marketplace approval so we can close this issue :)


add missing endpoints

Nice :)

forgot to actually test .notification :)

This should be ready, just need .chat implemented in #399
I heavily dislike the ad schedule related endpoints, they should never have been released in GA...


yor-tag.sh Applying Tags to Resources without Tags Argument.

Hello,
After running the post-push workflow, there exists a step which adds the yor tags to the resources. However, in my case these tags are getting applied to azurerm_log_analytics_linked_service which doesn't have any tags argument in general thus the following terraform validate step fails.
I am not sure how to get around this.

Thanks @luke-taylor for reporting this issue to us, we'll take a look. Would you please tell me your AzureRM Provider's version? Thanks!

You're welcome @luke-taylor, it's me who owe you a thanks.
I've removed all azurerm resources that don't contain tags anymore from my special personal fork of yor, the latest runner image's version should be v0.27.1. The CI pipeline is building the image, please wait for 1-2 hours and then remove the wrong tags and kick off your pipeline again.

The CI Pipeline now works! Feel free to close this issue, happy to have this one solved so quickly!

The CI Pipeline now works! Feel free to close this issue, happy to have this one solved so quickly!

Thanks for the update @luke-taylor, I'm working with yor team to solve this issue once and for all.


[ScreenShaking] Multi-layer and noise based shaking.

Changes

Embed the Noise extension for a more realistic shake (especially at low frequency)
Configuration per layer
Linear easing
Fix conflicts with camera movements.

Example
It contains 2 scenes:

Screen shake and camera following
Parallax screen shake

Try it:

Project: https://www.dropbox.com/s/nai88cdnjg76ynm/ScreenShakeExamples.zip?dl=1
Build: https://liluo.io/games/759a0e77-b308-421a-a433-028fa2654c90

JavaScript hack
It won't be accepted with this but I share the extension because it still works well and can be useful.
I opened a discussion to find a solution:

https://github.com/4ian/GDevelop/discussions/3762

// It Avoids conflicts with other extensions that also use the camera.
// If something changes in the engine it will no longer do anything.
if (runtimeScene._updateObjectsPreEvents
    && gdjs
    && gdjs.evtsExt__CameraShake__onScenePreEvents
    && gdjs.evtsExt__CameraShake__onScenePreEvents.registeredGdjsCallbacks
    && gdjs.evtsExt__CameraShake__onScenePreEvents.registeredGdjsCallbacks.length
    && gdjs.callbacksRuntimeScenePreEvents) {

    // Force preEvent execution before behavior ones.
    if (!runtimeScene.__camerShake_hasAlreadyAddHook) {
        runtimeScene.__camerShake_hasAlreadyAddHook = true;
        const superUpdateObjectsPreEvents = runtimeScene._updateObjectsPreEvents.bind(runtimeScene);
        runtimeScene._updateObjectsPreEvents = function () {
            gdjs.evtsExt__CameraShake__onScenePreEvents.func(runtimeScene, runtimeScene);
            superUpdateObjectsPreEvents();
        }
    }

    // Avoid to run preEvent a second time.
    const callback = gdjs.evtsExt__CameraShake__onScenePreEvents.registeredGdjsCallbacks[gdjs.evtsExt__CameraShake__onScenePreEvents.registeredGdjsCallbacks.length - 1];
    const callBackIndex = gdjs.callbacksRuntimeScenePreEvents.indexOf(callback);
    if (callBackIndex >= 0) {
        gdjs.callbacksRuntimeScenePreEvents.splice(callBackIndex, 1);
    }
}

This looks interesting and it might be able to replace all the use-cases of my version.
Both versions seem to include:

Variable duration
Variable frequency (I call this "time between shakes" because I feel it is more use friendly, despite 4ian's suggestion to use frequency)
Variable amplitude
Affect position (X and/or Y), rotation, and zoom
Include helper condition ("Is shaking")

Your version has some very nice benefits:

Works on multiple layers
Noise looks more natural than random

Your version has some minor negatives:

More steps to use. Can this be improved?
Can it perform a "shake forever" functionality?
Forced to include a full copy of the noise extension. We need extension dependencies, similar to how behavior dependencies work.
I think you might need a unique timer per layer.  It looks like a single timer is used.

@D8H I would be probably be convinced if we can implement everything in my screenshake example (source code included):
https://victrisgames.itch.io/gdevelop-camera-shake-example

More steps to use. Can this be improved?

It depends what the user do. With the default configuration only 1 action have to be called and it only have 3 parameters.

Can it perform a "shake forever" functionality?

It can. There is a start and a stop action.

Forced to include a full copy of the noise extension. We need extension dependencies, similar to how behavior dependencies work.

It's not really an issue. The user doesn't see it and it's relatively easy to copy/paste it with a text editor if an update is needed.

I think you might need a unique timer per layer.  It looks like a single timer is used.

To shake layers with different delays? What use case do you see for it?

Missing a "single shake" effect that allows a one-time deterministic movement (for a knockback (or similar) effect)

It's not a shake. It should be another extension.

@D8H I would be probably be convinced if we can implement everything in my screenshake example (source code included): https://victrisgames.itch.io/gdevelop-camera-shake-example

There it is: https://www.dropbox.com/s/vkm5lmh20ipbayp/CameraShakeExample_Tristan.zip?dl=1
I fixed a few bugs in the compatibility action.

Hello @D8H 👋 !
A review has started for this extension, but a new option has presented itself in the meantime. We now have a list of unreviewed "Community extensions", and your extension can be added to it, bypassing the review phase.
If you chose to not take a review, you need to keep the following in mind:

As of now, community extensions are not yet shown to users, but they should be until next update
Community extensions will never be shown to GDevelop users directly, they will have to check a checkbox for them to be shown
All community extension have a warning about them being unstable and unreviewed displayed on their store page
Your extension will be added immediately, no question asked, and without you needing to do any changes to your extension
You will still need to make an extension that passes the bot's tests (meaning you still should respect the extension best practices)

tl;dr You will not need to do any more (or only minimal) work on your extension, but it might not meet the standards of all users, and reach less users.
If you chose to continue on with the review:

The reviewing process might require of you to make some changes to your extension until it meets our reviewer's standards
Your extension will reach more users
You will be delivering a better extension to the community
It will take a bit more time for your extension to be added to the store

tl;dr You will need to do a bit more work but deliver a high quality contribution that will reach more people
The choice is yours, please tell us how you would like to continue.

I love that this can cover multiple layers, that's something that's been missed in the original camera shake extension. 👍
I will say though...
The philosophy for extensions that I would like to see implemented is "fewer more powerful extensions". The way that the current object and camera shake extensions work allows for a huge variety of options:
-Single knock back/hit/bounce effects.
-Bouncing screen elements up and down.
-Screen shake.
-Stretch and squash the screen.
and the list goes on.
The only thing it was missing, in my opinion, was the the multi layer functionality. 🤔
If we lose the ability to do directional single shakes, the entire extension will be lesser for it. Adding complexity to the process, by requiring multiple extensions or actions to do similar things, while not adding a lot of benefit to the user. 🤔
Also, having to set your settings at the beginning of a scene instead of within the action itself is really restrictive, can't easily set different shakes for different actions, and adds a lot more clutter to the event sheet. (This part might just be because it's in development, but a finished product should look similar to the current shake action's layout)
I think the noise and the easing are really nice, the example project shows what would be really good for an "earthquake" type shake. 🤔
I would just rather not see this replace the current extension if it's going to exclude some of the existing features, like being able to use it for single bounces/shakes, and simplicity of the original.

I love that this can cover multiple layers, that's something that's been missed in the original camera shake extension. 👍
I will say though... The philosophy for extensions that I would like to see implemented is "fewer more powerful extensions". The way that the current object and camera shake extensions work allows for a huge variety of options:

It's nice when an extension can be used to do more than what is was originally designed for but only if it doesn't make it more complicated than 2 dedicated extensions.

-Single knock back/hit/bounce effects. -Bouncing screen elements up and down. -Stretch and squash the screen.

These one should probably use camera tweens on game without scrolling or a dedicated extension because it's the kind of thing where easing functions is very important and the screen-shake extension can't give these features.

Also, having to set your settings at the beginning of a scene instead of within the action itself is really restrictive, can't easily set different shakes for different actions, and adds a lot more clutter to the event sheet. (This part might just be because it's in development, but a finished product should look similar to the current shake action's layout)

Extensions have properties. It's not yet possible to define one with event-based extensions but it will come and any extension that puts everything in one action will have to do a breaking change to use properties.
It allows cleaner events because if you only change the amplitude of the shake, you don't have to put back the whole configuration.

I would just rather not see this replace the current extension if it's going to exclude some of the existing features, like being able to use it for single bounces/shakes, and simplicity of the original.

We will have a lot of time to discuss it because this extension needs a very bad hack to work so even if the other extension bugs with scrolling games, it won't replace it soon.

There is a test scene variable useless in Shake camera and in Shake camera (deprecated)

OnSceneLoaded have the same actions as in ShakeCamera but with different values.

The condition on this event is "never". It was a workaround for a bug that didn't generate the code of functions that was only called from life-cycle function, but it's no longer needed. I removed them.

I like the shake effect this produces.
I don't like how it's presented in the events.😅
There is a lot of clutter. A lot of these things, like the default settings and start/stop shaking, could be combined in to single events to reduce clutter and simplify the work flow.
Just being able to start and stop the shake during a specific time requires 3 actions, where the existing extension has the duration build in.

If someone wanted two entirely different forms of camera shake for some reason, they would need all of this to accomplished 2 different kinds of shakes within the same project. Where previously they would need 2 actions.

These are a lot of actions that each do only one thing.

I like the shake effect this produces.
I don't like how it's presented in the events.😅
There is a lot of clutter. A lot of these things, like the default settings and start/stop shaking, could be combined in to single events to reduce clutter and simplify the work flow.
Just being able to start and stop the shake during a specific time requires 3 actions, where the existing extension has the duration built in.

There is already an action for it.

If someone wanted two entirely different forms of camera shake for some reason, they would need all of this to accomplished 2 different kinds of shakes within the same project. Where previously they would need 2 actions.
These are a lot of actions that each do only one thing.

There are default values, it's very likely users need rotation and zoom. With this version, you don't need to set them, when the previous version the shake action had 9 parameters (most of them useless 90% of the time).
It also offer more features, for instance you could put a sinusoid in the amplitude to add some effects. Every properties can be eased independently during a shake.

So there are even more actions that do a single thing.
Could this be a toggle on the other "start shaking" action? Instead of further cluttering the action list. 🤔

Being "useless" is a bit subjective, and default values being built in is great, but that still leaves us with a big list of actions that do single things. Increasing clutter in the event sheet and action list.

So there are even more actions that do a single thing.
Could this be a toggle on the other "start shaking" action? Instead of further cluttering the action list. 🤔

In this case, when an hypothetic  "permanent" toggle is enabled, the "duration" and the "easing duration to stop" wouldn't make sense.

Being "useless" is a bit subjective, and default values being built in is great, but that still leaves us with a big list of actions that do single things. Increasing clutter in the event sheet and action list.

Both way works, but I have the impression that configurations are usually done with independent actions in the engine. For instance, the jump of a platformer character can be configured with 3 actions for: the jump speed, the gravity, the maximum and falling speed.

When you only want to change the jump speed, It avoids having to set the other 2 parameters.
It also avoids to break the compatibility when properties are added or removed.

If I had the time I would make a pass on the reviewed extensions to comply this, because all these actions with a lot of parameters are not practical to use.

Those aren't really comparable though I think. 🤔
I set my default controls for my platformer based on how I want it to feel, and then tweak individual variables to achieve some temporary goal. A run, a high jump, some kind of restriction from a thing the player is standing it, etc. But I'll generally want all other variables to stay as they were, and I'll want to go back to that preset after the temporary effect has ended. It's relatively constant.
With camera shakes I could want dramatically different shakes depending on the situation I'm in. If you're making a platformer game and the character is riding in a shaking elevator you could would want something with entirely different settings compared to an earthquake or general rumble of some kind. They probably aren't going back to a default setting. Shakes could be dramatically different depending on the circumstances.

Having a single action that contains all of your default settings, and then having another single action with toggles that could be used to trigger the starting/stopping of the shaking, would minimize the number of actions cluttering the action list and event sheet.
The fact that I didn't originally know(After looking) that there was a third start/stop action, shows that the clutter can hide useful tools that a user would want to use if it were presented in a more minimalist and organized way.

I think the issue here is that my examples are not useful to learn the extension, only to test it.
The 3 actions are pretty clear in my opinion.

Having a single action that contains all of your default settings, and then having another single action with toggles that could be used to trigger the starting/stopping of the shaking, would minimize the number of actions cluttering the action list and event sheet.
The fact that I didn't originally know(After looking) that there was a third start/stop action, shows that the clutter can hide useful tools that a user would want to use if it were presented in a more minimalist and organized way.

I put a green cross on actions that will satisfy 90% of the use cases.
If the layer selection was in the same action that other properties, users would need to copy-paste 4 values for each layers.
Merging the 4 properties would force users to set something for rotation and zoom to save only one action.
So, I think that it wouldn't make it easier to understand and would remove flexibility (like dynamically changing the properties value while the screen is shaking).

I think you just highlighted my point. 🤔
4 out of 12 of those actions are what you think people are going to use, and the rest are cluttering the action list.
Would combining them, and having blank(Or toggled) parameters pick the previously set default settings, be possible? That way you wouldn't need to copy the variables each time you want to change the shake settings. 🤷♂

I think you just highlighted my point. 🤔
4 out of 12 of those actions are what you think people are going to use, and the rest are cluttering the action list.
Would combining them, and having blank(Or toggled) parameters pick the previously set default settings, be possible? That way you wouldn't need to copy the variables each time you want to change the shake settings. 🤷♂

I guess we don't understand each other at all 😅
These 2 event are enough to make a screen shake:

I do not like this much clutter in the event sheet or action list, I personally find it overwhelming and distracting, but the extension functionally works and does what it sets out to do.
I'm not a UX designer so I'll stop arguing that point. I think you can merge this extension, but we'll need documentation showing the two events you show in the above comment making it very clear that those are the actions you need and the rest are only needed if you want a more specific kind of screen shake.


'popular' page takes too long to load

Story: As a user, I want the popular page to load quickly so I can view the popular content quickly and without confusion.
Problem: the page loads pretty quickly, but the content takes a while to load, and the content shifts around as it loads due to masonry.js figuring out the best way to lay it out.
One aspect of the solution is probably to load a thumbnail for big gifs rather than the full gif itsself.  related: https://github.com/lbryio/spee.ch/issues/152
Another related but different issue with the popular page is the thumbnail for the videos not being the actual thumbnail: https://github.com/lbryio/spee.ch/issues/220

I think It would be best to create a card-type component with fixed proportions so that masonry can be removed and the display can count on being able to tile them at fixed proportions.

Perhaps only load 10 at a time, but implement infitine scrolling so it keeps requesting the next 10 automatically

@billbitt if you store the image/thumbnail width and height in the database so that they are available at render time, I bet you can get masonry to render a lot faster.
AFAIK there is no way to get a truly masonry-like layout without javascript.

replaced by #336