ledger: reduce the memory allocation of the Merkle tree generation algorithm

Description
What is the purpose of the change?
减少merkle算法的内存分配
修复前:
BenchmarkNormalMerkle-160<PHONE_NUMBER>6 ns/op         2067500 B/op      30017 allocs/op

修复后：
BenchmarkNormalMerkle-160<PHONE_NUMBER>8 ns/op         1835604 B/op          5 allocs/op

对于一个10000个交易的区块，内存分配次数从30017降到了5
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

Codecov Report

Merging #229 (2812d64) into master (9042b75) will increase coverage by 0.09%.
The diff coverage is 85.71%.

@@            Coverage Diff             @@
##           master     #229      +/-   ##
==========================================
+ Coverage   47.39%   47.48%   +0.09%
==========================================
  Files         133      133
  Lines       11413    11429      +16
==========================================
+ Hits         5409     5427      +18
+ Misses       4869     4865       -4
- Partials     1135     1137       +2

Impacted Files
Coverage Δ

bcs/ledger/xledger/ledger/ledger_hash.go
44.00% <85.71%> (+7.30%)
:arrow_up:

bcs/contract/xvm/code_manager.go
63.63% <0.00%> (-2.73%)
:arrow_down:

bcs/consensus/xpoa/xpoa.go
55.94% <0.00%> (-0.89%)
:arrow_down:

bcs/consensus/tdpos/tdpos.go
40.96% <0.00%> (-0.81%)
:arrow_down:

kernel/consensus/base/driver/chained-bft/smr.go
50.46% <0.00%> (+0.92%)
:arrow_up:

bcs/ledger/xledger/ledger/ledger.go
56.82% <0.00%> (+0.97%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9042b75...2812d64. Read the comment docs.

Codecov Report

Merging #229 (2812d64) into master (9042b75) will increase coverage by 0.18%.
The diff coverage is 85.71%.

@@            Coverage Diff             @@
##           master     #229      +/-   ##
==========================================
+ Coverage   47.39%   47.58%   +0.18%
==========================================
  Files         133      133
  Lines       11413    11429      +16
==========================================
+ Hits         5409     5438      +29
+ Misses       4869     4856      -13
  Partials     1135     1135

Impacted Files
Coverage Δ

bcs/ledger/xledger/ledger/ledger_hash.go
44.00% <85.71%> (+7.30%)
:arrow_up:

bcs/consensus/tdpos/tdpos.go
40.96% <0.00%> (-0.81%)
:arrow_down:

kernel/consensus/base/driver/chained-bft/smr.go
50.46% <0.00%> (+0.92%)
:arrow_up:

bcs/ledger/xledger/ledger/ledger.go
56.82% <0.00%> (+0.97%)
:arrow_up:

bcs/network/p2pv1/conn.go
61.29% <0.00%> (+6.45%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9042b75...2812d64. Read the comment docs.


Skip STDOUT/STDERR capture when reports are not generated

Using the example config.around(:each) behavior listed here has a strange side effect with tools like pry (and possibly others) in which invoking binding.pry seems to simply be ignored.  This is probably to do with pry's expectation of working with a terminal and finding none.
But in any case, this code checks whether the RspecJunitFormatter has been requested and captures output conditionally on that setting.

Would you accept a pull request to package this code into the library itself?  It seems cleaner, especially if you maintain a lot of ruby repositories...

Capturing stdout is nor a core responsibility of this formatter. This example code is provided as a launching point for what are likely to be more specific requirements for stdout capture. For example, another approach would be to add a tag to the each in this configure block then tag each spec unit requiring stdout capture explicitly. But this is an exercise to the user, and will be unique for each use.
It'd be neat if this was extracting into an rspec_stdout_capture gem which uses the right meta-data keys, though!

Your comment about not putting the example code into your library makes sense.
However, I don't see what that has to do with rejecting this update to your documentation.  At the very least, your README.md should acknowledge the potential consequences of the code it is suggesting.  (It took us a quite a long while to figure out why binding.pry wasn't working when debugging an rspec test.)

Are we in disagreement about the need to add any sort of warning to the documentation?

Sorry, you're right that a note in the docs is a good idea, I just hadn't got to this notification yet! I'll update the README now.

I've added a note in 3738c28.

Thanks!


hnsep refers to the vocoder?

Hi,
I wonder if checkpoint/vr refers to the vocoder and its config.yaml file?
If yes, I wonder if there are pretrained vocoder weights and config.yaml that I  can use to train my custom variance/Diffsinger?
All the vocoders I found looking in the repo are not compatible with the DiffSinger(openvpi) format. and I keep getting the following error
RuntimeError: Error(s) in loading state_dict for CascadedNet:         Missing key(s) in state_dict:  ListOfMissingKeys

hnsep is the harmonic-noise separation model. See https://github.com/openvpi/DiffSinger/blob/main/docs/BestPractices.md


Downgrade karma to v6.0.4.

It seems v6.1.0 has some disconnect issues.

I'll merge this and see how it goes. Maybe it's unrelated to karma and it's the usual karma-browserstack issue. In that case, I'll update it again in the future.


Report of MOAs sizes/values per dose analysis (ranking) on consensus datasets

@gwaygenomics  @shntnu

The goal here was to determine the size/value of each MOA (Mechanism of action) for each dose based on taking the median of the correlation values between compounds of the same MOA in the consensus datasets.

MOAs with only one compound were excluded. Out of 601 MOAs, 369 were excluded.
Results: I will be showing only the first 10 MOAs values per dose for all the consensus datasets and the heatmap plots of the 232 MOAs for each of the consensus datasets. Doses 0 (has only dmso) and 7 (has only two MOAs) were excluded from the analysis.
1a. Median Aggregation Consensus dataset - consensus_median (whole plate normalization).

Heatmap plots -- I split the dataframe into 3 parts for easier visualization:

1b. Median Aggregation Consensus dataset - consensus_median_dmso (dmso normalization).

heatmap plots

2a. Modified Z Score Aggregation (MODZ) dataset - consensus_modz (whole plate normalization).

heatmap plots

2b. Modified Z Score Aggregation (MODZ) dataset - consensus_modz_dmso (dmso normalization).

heatmap plots

@gwaygenomics
- MOAS that do not have the same number of compounds in all Doses

@AdeboyeML inspected one of the cases and found that the dose remapping can be flawed. E.g. PYM50028 (BRD-K62277907-001-01-6) has two doses coded as level 1

@AdeboyeML uses corr_val = abs(df_dose_corr.loc[cpds[y], cpds[x]]) but we shouldn't take the absolute value. Now this will likely not change the actual results because it is very unlikely that we will have negative values among replicates, but we should fix.
Also consider computing the correlation matriix of the subsetted dataframe corresponding to the replicates you care about and then take the median of the lower (or upper triangular matrix). This is an implementation detail, but the logic is otherwise correct.

just an update to https://github.com/broadinstitute/lincs-profiling-comparison/issues/2#issuecomment-717303986

Dose 7 was not supposed to be included in the https://github.com/broadinstitute/lincs-profiling-comparison/issues/2#issuecomment-717303986 MOAs  that do not have the same number of compounds in all Doses figure since it has only two MOAs, that was a bug in my code.

- MOAS that do not have the same number of compounds in all Doses

- Results from Null Distribution

Null distribution - is generated by getting the median correlation score of randomly combined compounds that do not share/come from the same MOAs.

In our case, we generated 1000 median correlation scores from randomly combined compounds as the null distribution for each MOA.

A P value can be computed nonparametrically by evaluating the probability of random compounds of different MOAs having greater median similarity value than compounds of the same MOAs.

Looking forward to digging into this!!
On Tue, Nov 3, 2020 at 9:02 PM Adeniyi Adeboye<EMAIL_ADDRESS>wrote:

Results from Null Distribution

Null distribution - is generated by getting the median correlation
score of randomly combined compounds that do not share/come from the same
MOAs.
In our case, we generated 1000 median correlation scores from randomly
combined compounds as the null distribution for each MOA.
A P value can be computed nonparametrically by evaluating the
probability of random compounds of different MOAs having greater median
similarity value than compounds of the same MOAs.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/broadinstitute/lincs-profiling-comparison/issues/2#issuecomment-721469990,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJHQPC57X4UICO5MFUKSITSOCY33ANCNFSM4S52XGNQ
.

--
-Shantanu

- In addition to the figures shown in https://github.com/broadinstitute/lincs-profiling-comparison/issues/2#issuecomment-721469990, I included the density distribution of the p-values vs median scores in the same plots:
- - Median Consensus

- - Median DMSO Consensus

- MODZ Consensus

- MODZ DMSO Consensus

@AdeboyeML quick question:
In this figure (where each data points is an MOA, I assume), if two MOA's have the same number of compounds, and the same x-axis value (median pairwise correlation between compounds), do they also have the same y-axis value?
Or phrased more simply, are you computing the null distribution once for each MOA size (by size I mean number of compounds in the MOA classs)? Or are you doing it once per MOA? Both are fine, but the former is preferred to remove y-axis variance that's not informative.

@AdeboyeML kudos for making the data so easy to peek into. I was curious to see if one could see a dose-response in some of the MOAs. Looks like we do so it.
More during profiling check-in!

Code
moa_consistency <-
  read_csv(
    "https://raw.githubusercontent.com/broadinstitute/lincs-profiling-comparison/9bc5db8167674e2c8bec5cee3fcc043117acfbf6/1.Data-exploration/moa_sizes_consensus_datasets/median_dmso_moa_median_scores.csv"
  )

moa_consistency %<>% rename(moa = X1)

moa_consistency %<>% pivot_longer(-moa, names_to = "dose", values_to = "score")

moa_consistency %<>% mutate(dose = as.integer(str_remove(dose, "dose_")))

moa_consistency %<>%
  inner_join(
    moa_consistency %>%
      group_by(moa) %>%
      summarize(score_median = median(score)) %>%
      filter(score_median > 0.30)
  )

p <-
  ggplot(moa_consistency, aes(dose, score)) +
  geom_line() +
  facet_wrap( ~ round(score_median, 2) ~ moa,
              ncol = 5,
              scales = "free_y")

ggsave("~/Desktop/moa_score_dose_response.png",
       width = 10,
       height = 10)

@shntnu In regards to question in https://github.com/broadinstitute/lincs-profiling-comparison/issues/2#issuecomment-724933716

Or phrased more simply, are you computing the null distribution once for each MOA size (by size I mean number of compounds in the MOA classs)? Or are you doing it once per MOA? Both are fine, but the former is preferred to remove y-axis variance that's not informative.

I computed the null distribution once for each MOA based on its Size (i.e. by size - number of compounds in a MOA class) per Dose, this means I computed 1000 random median scores for each MOA per Dose from which I computed the p-value.

Null Distribution for each MOA means - I selected  a number of random compounds from different MOAs for each MOA, in which this number is the size of the MOA, I did this 1000 times for each MOA per dose.

if two MOA's have the same number of compounds, and the same x-axis value (median pairwise correlation between compounds), do they also have the same y-axis value?

No, the p-value (y-axis) is based on randomly selected 1000 lists of compounds (the length of each list is the size of the MOA) from which I computed the median scores for each MOA, which is unique to each MOA.

@AdeboyeML thanks for clarifying. Everything looks good, but the one change I recommend is to use the same null distribution for all MOAs of the same size.
There is no upside to having different null distributions for each unique MOA, while it has the downside of adding uninformative variance to the p-value estimates.
cc @gwaygenomics

@shntnu  @gwaygenomics
- Results from the Null distribution, based on using the same null distribution for all MOAs of the same size.

There seems to be no relationship between the median pairwise correlation and the obtained p-value generated from null distribution.

- Median Consensus

- Distribution of the median pairwise correlation scores

- P-values distribution across doses
- Increase in MOAs with values below the significant level (0.05) as dose increases

- -MOAs with p-values <0.05 in all doses

- These MOAs dose responses:

- The above results and distributions are similar for the Modz Consensus datasets

There seems to be no relationship between the median pairwise correlation and the obtained p-value generated from null distribution.

That's really strange – sounds like a bug to me

@shntnu @gwaygenomics
Comparing the distribution of median scores between L1000 and Lincs Cell painting Consensus datasets
- Major points

213 MOAs (Mechanism of actions) present in both Cell painting and L1000 Level-5 data are compared based on the distribution of their median scores.

During alignment of MOAs in L1000 with the MOAS in Cell painting, I realized that MOAs found in the same broad sample in both  L1000 & Cell painting data are partly named differently i.e. the naming of same MOAS in both are not consistent.

Results.
Scatter plot btw L1000 vs Lincs Cell Painting median scores per dose

Median scores in cell painting data are more spread out and have more extreme median values than L1000 data.

Distribution of median scores in L1000 and Cell Painting Data per dose

Distribution on a dose-by-dose basis

- I am still trying to figure out the reason behind the relationship between the p-value and median scores (null distribution) in https://github.com/broadinstitute/lincs-profiling-comparison/issues/2#issuecomment-726123235

@shntnu @gwaygenomics
Results from the Null distribution, based on using the same null distribution for all MOAs of the same size. (L1000 & Cell Painting)

I have been able to figure out what was wrong with my code, which resulted in the strange relationship between the p-value and median scores (null distribution) in https://github.com/broadinstitute/lincs-profiling-comparison/issues/2#issuecomment-726123235

Corrected Plots of the Null distribution are below:

Cell painting
MODZ Consensus

L 1000
MODZ Level-5 data

- The above results and distributions are similar for both median and rank level-5 (Consensus data) in cell painting and L1000.


Properly mark include files

Signed-off-by: Olof Kindgren<EMAIL_ADDRESS>

Perfect, thanks!
Will use that for some other core too.


[SPARK-21004]remove the never used declaration in function warnDeprec…

JIRA Issue:https://issues.apache.org/jira/browse/SPARK-21004
the function {warnDeprecatedVersions} of class SparkContext, the declaration of
val javaVersion = System.getProperty("java.version").split("[+.\-]+", 3)
is never used in this function.

Please close this -- I commented in the JIRA already


Will there ever be an alternative to pressing Double R to refresh on Android when developing?

🐛 Bug Report
I know this is probably brought up alot but I feel I need to speak my mind.
It feels almost embarrassing as a developer how much I personally struggle with developing a React Native android app. I have lots of text fields in my apps and I spend so much time every day struggling with trying to trigger a refresh while in development mode and the keyboard coming up. All of my text fields have dozens and dozens of rrrrrrrr in it.
Why can't we support Command/Ctrl R like on the iOS simulator? Is there another way to trigger a refresh I'm not aware of? Surely having to resort to force quitting my simulator and rebuilding the app isn't the only other alternative.

You can use cmd+M or ctrl+M to show developer menu, then reload from there.


Upgrade autovalue to 1.7.4 (latest version).

R: @robinyqiu

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

I found that many GCP dependencies are pinned to auto-value 1.7.2. So we can leave it at 1.7.2 for now.

I found that many GCP dependencies are pinned to auto-value 1.7.2. So we can leave it at 1.7.2 for now.


Strong (Simple?) Request: Separate options for "Clear" and "Undo" for sketching

Is your feature request related to a problem? Please describe.
The sketching option under images in StraboSpot2 is very valuable, and I utilize it often (especially when teaching!). However, I can't count how many times I've accidentally "Cleared" a sketch I've spent a lot of time on instead of "Undo"ing my previous pen-line.
Describe the solution you'd like
I think a simple fix for this would to be move the "Clear" option to the other side, or separate from all other options completely.

@dmongovin I made the change and moved the Clear button to the other side.  We cannot do to much modification since this sketch package is a imported package.  Hope this helps.


feat(pcli): support new ibc query logic

Previously pcli was only capable of looking up info on IBC clients, connections, and channels if the user provided a unique identifier for the resource in question. To facilitate IBC in practice, we want to enable querying for lists of these values, so the end user (or developer) can inspect the current options in a legible format.
Closes #2937.

Requesting feedback from @avahowell. It might be useful to compare results between the current state of this PR and after applying this diff:
diff --git a/crates/bin/pcli/src/command/query/ibc_query.rs b/crates/bin/pcli/src/command/query/ibc_query.rs
index e821b93bf..6de19a816 100644
--- a/crates/bin/pcli/src/command/query/ibc_query.rs
+++ b/crates/bin/pcli/src/command/query/ibc_query.rs
@@ -83,12 +83,7 @@ impl IbcCmd {
                     .await?
                     .into_inner()
                     .client_states;
-                let clients_ids: Vec<String> =
-                    client_states.into_iter().map(|c| c.client_id).collect();
-                // TODO: Resolve formatting for these representations. The key_value lookup
-                // for IbcCmd::Client recasts as a TendermintClientState; the ibc ClientState
-                // is less informative. For now, just emitting the names.
-                let clients_json = serde_json::to_string_pretty(&clients_ids)?;
+                let clients_json = serde_json::to_string_pretty(&client_states)?;
                 println!("{}", clients_json.to_colored_json_auto()?);
             }
             IbcCmd::Connection { connection_id } => {

Wasn't able to drag the channel (singular) logic over to the new format, so I left that as the old k/v lookup.


add check for filename and file_format during read and write

The filename check was made only in read and not write. Also, no check was made to see whether the user-specified file format is supported by the code. This lead to 'KeyError' on the prompt which is not very descriptive for the end user. Also, there was code repetition in 'read' and 'write' modules. I have added a new function 'check_filename' that would check whether the specified file format is supported by the code. This is in regards to Issue #208.

Issue #208 is different, but I still like this PR. Better error message are better. :+1:.

In the meantime, a few asserts with proper error messages on failure have landed in meshio. This obsoletes this PR I think. Feel free to reopen if you think that this is still useful.

Thanks for considering the suggestion. The changes are good.


Investigate possibilities to send attachments to Salesforce

https://app.zenhub.com/files/41149571/3d4f6f49-dd1c-4dab-9235-414d7968f8e6/download


Task hint          : 'macs2 callpeak -t /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz -c /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_17'
Task resources     : 'cpus: -1	mem: -1.0 B	wall-timeout: 8640000'
State              : 'ERROR'
Dependency state   : 'ERROR'
Retries available  : '1'
Input files        : '[/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz, /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz]'
Output files       : '[/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz]'
Script file        : '/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr2.line_83.id_36.sh'
Exit status        : '1'
Program            :
	# SYS command. line 85

	 if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi;  export PATH=/gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/.:/gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/modules:/gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)

	# SYS command. line 86

	 export LC_COLLATE=C

	# SYS command. line 91

	 macs2 callpeak -t /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz -c /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz -f BED -n /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign -g hs -p 0.01 --nomodel --shift 0 --extsize 230 --keep-dup all -B --SPMR

	# SYS command. line 94

	 sort -k 8gr,8gr "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak | awk 'BEGIN{OFS="\t"}{$4="Peak_"NR ; print $0}' | gzip -nc > /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz

	# SYS command. line 97

	 rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.xls \
				"/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak \
				"/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_summits.bed

	# SYS command. line 101

	 if [[ false == "false" ]]; then \
				rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg; \
				TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; \
				exit; \
			fi

	# SYS command. line 111

	 macs2 bdgcmp -t "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -m FE

	# SYS command. line 114

	 slopBed -i "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -g /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph

	# SYS command. line 115

	 rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg

	# SYS command. line 118

	 sort -k1,1 -k2,2n /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph > /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 119

	 bedGraphToBigWig /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bw

	# SYS command. line 120

	 rm -f /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 126

	 chipReads=$(zcat /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}')

	# SYS command. line 128

	 controlReads=$(zcat /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}'); sval=$(echo "${chipReads} ${controlReads}" | awk '$1>$2{printf "%f",$2} $1<=$2{printf "%f",$1}')

	# SYS command. line 130

	 macs2 bdgcmp -t "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -m ppois -S "${sval}"

	# SYS command. line 133

	 slopBed -i "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -g /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph

	# SYS command. line 134

	 rm -rf "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg

	# SYS command. line 137

	 sort -k1,1 -k2,2n /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph > /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 138

	 bedGraphToBigWig /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bw

	# SYS command. line 139

	 rm -f /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 141

	 rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg

	# SYS command. line 143

	 TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."
StdErr (100000000 lines)  :
	Traceback (most recent call last):
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/bin/macs2", line 4, in <module>
	    __import__('pkg_resources').run_script('MACS2==<IP_ADDRESS>50731', 'macs2')
	  File "/home/rdali/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 719, in run_script
	    self.require(requires)[0].run_script(script_name, ns)
	  File "/home/rdali/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1517, in run_script
	    exec(script_code, namespace, namespace)
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 614, in <module>

	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 55, in main

	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/callpeak_cmd.py", line 30, in <module>
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/OptValidator.py", line 27, in <module>
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 7, in <module>
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 6, in __bootstrap__
	  File "__init__.pxd", line 155, in init MACS2.IO.Parser (MACS2/IO/Parser.c:24419)
	ValueError: numpy.dtype has the wrong size, try recompiling

4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr2.line_83.id_36' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_ctl1.line_651.id_30, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_ctl1.line_651.id_30' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_rep1.line_181.id_12, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_rep1.line_181.id_12' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_1_rep1.line_260.id_13, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_1_rep1.line_260.id_13' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_2.line_111.id_23, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_2.line_111.id_23' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_2.line_111.id_11, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_2.line_111.id_11' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.bam_to_bedpe_rep1.line_613.id_17, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.bam_to_bedpe_rep1.line_613.id_17' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bed.spr_PE_rep1.line_180.id_20, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bed.spr_PE_rep1.line_180.id_20' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_rep1.line_323.id_15, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_rep1.line_323.id_15' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr1.line_68.id_33, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr1.line_68.id_33' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_ctl1.line_323.id_27, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_ctl1.line_323.id_27' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr2.line_68.id_34, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr2.line_68.id_34' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_1.line_111.id_22, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_1.line_111.id_22' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_ctl1.line_181.id_24, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_ctl1.line_181.id_24' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_xcor.subsample_bedpe2tag_rep1.line_85.id_19, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_xcor.subsample_bedpe2tag_rep1.line_85.id_19' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_1.line_111.id_10, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_1.line_111.id_10' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_rep1.line_651.id_18, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_rep1.line_651.id_18' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35, state: ERROR
Task failed:
Program & line     : '/gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/modules/callpeak_macs2_chipseq.bds', line 83
Task Name          : 'macs2 n/s rep1-pr1'
Task ID            : 'chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35'
Task PID           : '4610'
Task hint          : 'macs2 callpeak -t /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz -c /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_17'
Task resources     : 'cpus: -1	mem: -1.0 B	wall-timeout: 8640000'
State              : 'ERROR'
Dependency state   : 'ERROR'
Retries available  : '1'
Input files        : '[/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz, /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz]'
Output files       : '[/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz]'
Script file        : '/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35.sh'
Exit status        : '1'
Program            :
	# SYS command. line 85

	 if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi;  export PATH=/gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/.:/gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/modules:/gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)

	# SYS command. line 86

	 export LC_COLLATE=C

	# SYS command. line 91

	 macs2 callpeak -t /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz -c /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz -f BED -n /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign -g hs -p 0.01 --nomodel --shift 0 --extsize 230 --keep-dup all -B --SPMR

	# SYS command. line 94

	 sort -k 8gr,8gr "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak | awk 'BEGIN{OFS="\t"}{$4="Peak_"NR ; print $0}' | gzip -nc > /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz

	# SYS command. line 97

	 rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.xls \
				"/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak \
				"/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_summits.bed

	# SYS command. line 101

	 if [[ false == "false" ]]; then \
				rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg; \
				TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; \
				exit; \
			fi

	# SYS command. line 111

	 macs2 bdgcmp -t "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -m FE

	# SYS command. line 114

	 slopBed -i "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -g /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph

	# SYS command. line 115

	 rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg

	# SYS command. line 118

	 sort -k1,1 -k2,2n /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph > /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 119

	 bedGraphToBigWig /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bw

	# SYS command. line 120

	 rm -f /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 126

	 chipReads=$(zcat /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}')

	# SYS command. line 128

	 controlReads=$(zcat /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}'); sval=$(echo "${chipReads} ${controlReads}" | awk '$1>$2{printf "%f",$2} $1<=$2{printf "%f",$1}')

	# SYS command. line 130

	 macs2 bdgcmp -t "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -m ppois -S "${sval}"

	# SYS command. line 133

	 slopBed -i "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -g /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph

	# SYS command. line 134

	 rm -rf "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg

	# SYS command. line 137

	 sort -k1,1 -k2,2n /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph > /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 138

	 bedGraphToBigWig /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bw

	# SYS command. line 139

	 rm -f /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph /gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 141

	 rm -f "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "/gs/project/yxw-673-aa/rdali/MTaylor/project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg

	# SYS command. line 143

	 TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."
StdErr (100000000 lines)  :
	Traceback (most recent call last):
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/bin/macs2", line 4, in <module>
	    __import__('pkg_resources').run_script('MACS2==<IP_ADDRESS>50731', 'macs2')
	  File "/home/rdali/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 719, in run_script
	    self.require(requires)[0].run_script(script_name, ns)
	  File "/home/rdali/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1517, in run_script
	    exec(script_code, namespace, namespace)
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 614, in <module>

	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 55, in main

	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/callpeak_cmd.py", line 30, in <module>
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/OptValidator.py", line 27, in <module>
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 7, in <module>
	  File "/gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 6, in __bootstrap__
	  File "__init__.pxd", line 155, in init MACS2.IO.Parser (MACS2/IO/Parser.c:24419)
	ValueError: numpy.dtype has the wrong size, try recompiling

4 days 14:16:01.883	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35' finished.
Fatal error: /gs/project/yxw-673-aa/rdali/tools/TF_chipseq_pipeline/chipseq.bds, line 1237, pos 2. Task/s failed.
chipseq.bds, line 76 :	main()
chipseq.bds, line 79 :	void main() { // chipseq pipeline starts here
chipseq.bds, line 91 :		call_peaks() // call peaks in parallel (MACS2,SPP)
chipseq.bds, line 820 :	void call_peaks() {
chipseq.bds, line 1237 :		wait
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
4 days 14:16:02.500	Writing report file 'chipseq.bds.20170604_153842_923.report.html'
4 days 14:16:02.634	Program 'chipseq.bds.20170604_153842_923' finished, exit value: 1, tasks executed: 27, tasks failed: 3, tasks failed names: macs2 n/s rep1 , macs2 n/s rep1-pr2 , macs2 n/s rep1-pr1.
4 days 14:16:02.635	Finished. Exit code: 1
4 days 14:16:02.635	ExecutionerLocal 'Local[41]': Killed
4 days 14:16:02.702	ExecutionerLocal 'Local[41]': Finished running

4 days 14:15:59.701	ExecutionerLocal 'Local[41]': Tasks [Local[41]]		Pending: 0	Running: 1	Done: 26	Failed: 2
| PID  | Task state        | Task name          | Dependencies | Task definition                                                                                |
| ---- | ----------------- | ------------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| 6644 | running (RUNNING) | macs2 n/s rep1-pr2 |              | macs2 callpeak -t project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz -c project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_17 |
Traceback (most recent call last):
File "tools/miniconda3/envs/aquas_chipseq/bin/macs2", line 4, in
import('pkg_resources').run_script('MACS2==<IP_ADDRESS>50731', 'macs2')
File ".local/lib/python2.7/site-packages/pkg_resources/init.py", line 719, in run_script
self.require(requires)[0].run_script(script_name, ns)
File ".local/lib/python2.7/site-packages/pkg_resources/init.py", line 1517, in run_script
exec(script_code, namespace, namespace)
File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 614, in
File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 55, in main
File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/callpeak_cmd.py", line 30, in
File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/OptValidator.py", line 27, in
File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 7, in
File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 6, in bootstrap
File "init.pxd", line 155, in init MACS2.IO.Parser (MACS2/IO/Parser.c:24419)
ValueError: numpy.dtype has the wrong size, try recompiling
Task failed:
Program & line     : 'tools/TF_chipseq_pipeline/modules/callpeak_macs2_chipseq.bds', line 83
Task Name          : 'macs2 n/s rep1-pr2'
Task ID            : 'chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr2.line_83.id_36'
Task PID           : '6644'
Task hint          : 'macs2 callpeak -t project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz -c project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_17'
Task resources     : 'cpus: -1	mem: -1.0 B	wall-timeout: 8640000'
State              : 'ERROR'
Dependency state   : 'ERROR'
Retries available  : '1'
Input files        : '[project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz, project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz]'
Output files       : '[project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz]'
Script file        : 'project/ChIPSeq/chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr2.line_83.id_36.sh'
Exit status        : '1'
Program            :
	# SYS command. line 85

	 if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi;  export PATH=tools/TF_chipseq_pipeline/.:tools/TF_chipseq_pipeline/modules:tools/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)

	# SYS command. line 86

	 export LC_COLLATE=C

	# SYS command. line 91

	 macs2 callpeak -t project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz -c project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz -f BED -n project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign -g hs -p 0.01 --nomodel --shift 0 --extsize 230 --keep-dup all -B --SPMR

	# SYS command. line 94

	 sort -k 8gr,8gr "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak | awk 'BEGIN{OFS="\t"}{$4="Peak_"NR ; print $0}' | gzip -nc > project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz

	# SYS command. line 97

	 rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.xls \
				"project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak \
				"project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_summits.bed

	# SYS command. line 101

	 if [[ false == "false" ]]; then \
				rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg; \
				TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; \
				exit; \
			fi

	# SYS command. line 111

	 macs2 bdgcmp -t "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -m FE

	# SYS command. line 114

	 slopBed -i "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -g tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph

	# SYS command. line 115

	 rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg

	# SYS command. line 118

	 sort -k1,1 -k2,2n project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph > project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 119

	 bedGraphToBigWig project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bw

	# SYS command. line 120

	 rm -f project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 126

	 chipReads=$(zcat project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}')

	# SYS command. line 128

	 controlReads=$(zcat project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}'); sval=$(echo "${chipReads} ${controlReads}" | awk '$1>$2{printf "%f",$2} $1<=$2{printf "%f",$1}')

	# SYS command. line 130

	 macs2 bdgcmp -t "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -m ppois -S "${sval}"

	# SYS command. line 133

	 slopBed -i "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -g tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph

	# SYS command. line 134

	 rm -rf "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg

	# SYS command. line 137

	 sort -k1,1 -k2,2n project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph > project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 138

	 bedGraphToBigWig project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bw

	# SYS command. line 139

	 rm -f project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 141

	 rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr2/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr2.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg

	# SYS command. line 143

	 TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."
StdErr (100000000 lines)  :
	Traceback (most recent call last):
	  File "tools/miniconda3/envs/aquas_chipseq/bin/macs2", line 4, in <module>
	    __import__('pkg_resources').run_script('MACS2==<IP_ADDRESS>50731', 'macs2')
	  File ".local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 719, in run_script
	    self.require(requires)[0].run_script(script_name, ns)
	  File ".local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1517, in run_script
	    exec(script_code, namespace, namespace)
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 614, in <module>

	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 55, in main

	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/callpeak_cmd.py", line 30, in <module>
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/OptValidator.py", line 27, in <module>
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 7, in <module>
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 6, in __bootstrap__
	  File "__init__.pxd", line 155, in init MACS2.IO.Parser (MACS2/IO/Parser.c:24419)
	ValueError: numpy.dtype has the wrong size, try recompiling

4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr2.line_83.id_36' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_ctl1.line_651.id_30, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_ctl1.line_651.id_30' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_rep1.line_181.id_12, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_rep1.line_181.id_12' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_1_rep1.line_260.id_13, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_1_rep1.line_260.id_13' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_2.line_111.id_23, state: FINISHED
4 days 14:16:01.881	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_2.line_111.id_23' finished.
4 days 14:16:01.881	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_2.line_111.id_11, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_2.line_111.id_11' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.bam_to_bedpe_rep1.line_613.id_17, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.bam_to_bedpe_rep1.line_613.id_17' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bed.spr_PE_rep1.line_180.id_20, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bed.spr_PE_rep1.line_180.id_20' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_rep1.line_323.id_15, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_rep1.line_323.id_15' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr1.line_68.id_33, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr1.line_68.id_33' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_ctl1.line_323.id_27, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.dedup_bam_PE_2_ctl1.line_323.id_27' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr2.line_68.id_34, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_spp.spp_rep1_pr2.line_68.id_34' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_1.line_111.id_22, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_ctl1_1.line_111.id_22' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_ctl1.line_181.id_24, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_sam_PE_ctl1.line_181.id_24' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_xcor.subsample_bedpe2tag_rep1.line_85.id_19, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_xcor.subsample_bedpe2tag_rep1.line_85.id_19' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_1.line_111.id_10, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.align_bwa.bwa_aln_rep1_1.line_111.id_10' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_rep1.line_651.id_18, state: FINISHED
4 days 14:16:01.882	Wait: Task 'chipseq.bds.20170604_153842_923/task.postalign_bam.bedpe_to_tag_rep1.line_651.id_18' finished.
4 days 14:16:01.882	Wait: Waiting for task to finish: chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35, state: ERROR
Task failed:
Program & line     : 'tools/TF_chipseq_pipeline/modules/callpeak_macs2_chipseq.bds', line 83
Task Name          : 'macs2 n/s rep1-pr1'
Task ID            : 'chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35'
Task PID           : '4610'
Task hint          : 'macs2 callpeak -t project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz -c project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_17'
Task resources     : 'cpus: -1	mem: -1.0 B	wall-timeout: 8640000'
State              : 'ERROR'
Dependency state   : 'ERROR'
Retries available  : '1'
Input files        : '[project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz, project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz]'
Output files       : '[project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz]'
Script file        : 'project/ChIPSeq/chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35.sh'
Exit status        : '1'
Program            :
	# SYS command. line 85

	 if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi;  export PATH=tools/TF_chipseq_pipeline/.:tools/TF_chipseq_pipeline/modules:tools/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)

	# SYS command. line 86

	 export LC_COLLATE=C

	# SYS command. line 91

	 macs2 callpeak -t project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz -c project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz -f BED -n project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign -g hs -p 0.01 --nomodel --shift 0 --extsize 230 --keep-dup all -B --SPMR

	# SYS command. line 94

	 sort -k 8gr,8gr "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak | awk 'BEGIN{OFS="\t"}{$4="Peak_"NR ; print $0}' | gzip -nc > project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.narrowPeak.gz

	# SYS command. line 97

	 rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.xls \
				"project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_peaks.narrowPeak \
				"project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_summits.bed

	# SYS command. line 101

	 if [[ false == "false" ]]; then \
				rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg; \
				TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; \
				exit; \
			fi

	# SYS command. line 111

	 macs2 bdgcmp -t "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -m FE

	# SYS command. line 114

	 slopBed -i "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg -g tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph

	# SYS command. line 115

	 rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_FE.bdg

	# SYS command. line 118

	 sort -k1,1 -k2,2n project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph > project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 119

	 bedGraphToBigWig project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bw

	# SYS command. line 120

	 rm -f project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.bedgraph project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.fc.signal.srt.bedgraph

	# SYS command. line 126

	 chipReads=$(zcat project/ChIPSeq/160130_CTCF/align/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}')

	# SYS command. line 128

	 controlReads=$(zcat project/ChIPSeq/160130_CTCF/align/ctl1/HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.gz | wc -l | awk '{printf "%f", $1/1000000}'); sval=$(echo "${chipReads} ${controlReads}" | awk '$1>$2{printf "%f",$2} $1<=$2{printf "%f",$1}')

	# SYS command. line 130

	 macs2 bdgcmp -t "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg -c "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg --outdir project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1 -o "HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -m ppois -S "${sval}"

	# SYS command. line 133

	 slopBed -i "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg -g tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes -b 0 |   awk '{if ($3 != -1) print $0}' |  bedClip stdin tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph

	# SYS command. line 134

	 rm -rf "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_ppois.bdg

	# SYS command. line 137

	 sort -k1,1 -k2,2n project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph > project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 138

	 bedGraphToBigWig project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph tools/TF_chipseq_pipeline/Genomes/hg19/hg19.chrom.sizes project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bw

	# SYS command. line 139

	 rm -f project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.bedgraph project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign.pval.signal.srt.bedgraph

	# SYS command. line 141

	 rm -f "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_treat_pileup.bdg "project/ChIPSeq/160130_CTCF/peak/macs2/pseudo_reps/rep1/pr1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.pr1.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign"_control_lambda.bdg

	# SYS command. line 143

	 TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."
StdErr (100000000 lines)  :
	Traceback (most recent call last):
	  File "tools/miniconda3/envs/aquas_chipseq/bin/macs2", line 4, in <module>
	    __import__('pkg_resources').run_script('MACS2==<IP_ADDRESS>50731', 'macs2')
	  File ".local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 719, in run_script
	    self.require(requires)[0].run_script(script_name, ns)
	  File ".local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1517, in run_script
	    exec(script_code, namespace, namespace)
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 614, in <module>

	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 55, in main

	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/callpeak_cmd.py", line 30, in <module>
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/OptValidator.py", line 27, in <module>
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 7, in <module>
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/IO/Parser.py", line 6, in __bootstrap__
	  File "__init__.pxd", line 155, in init MACS2.IO.Parser (MACS2/IO/Parser.c:24419)
	ValueError: numpy.dtype has the wrong size, try recompiling

4 days 14:16:01.883	Wait: Task 'chipseq.bds.20170604_153842_923/task.callpeak_macs2_chipseq.macs2_n_s_rep1_pr1.line_83.id_35' finished.
Fatal error: tools/TF_chipseq_pipeline/chipseq.bds, line 1237, pos 2. Task/s failed.
chipseq.bds, line 76 :	main()
chipseq.bds, line 79 :	void main() { // chipseq pipeline starts here
chipseq.bds, line 91 :		call_peaks() // call peaks in parallel (MACS2,SPP)
chipseq.bds, line 820 :	void call_peaks() {
chipseq.bds, line 1237 :		wait
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
4 days 14:16:02.500	Writing report file 'chipseq.bds.20170604_153842_923.report.html'
4 days 14:16:02.634	Program 'chipseq.bds.20170604_153842_923' finished, exit value: 1, tasks executed: 27, tasks failed: 3, tasks failed names: macs2 n/s rep1 , macs2 n/s rep1-pr2 , macs2 n/s rep1-pr1.
4 days 14:16:02.635	Finished. Exit code: 1
4 days 14:16:02.635	ExecutionerLocal 'Local[41]': Killed
4 days 14:16:02.702	ExecutionerLocal 'Local[41]': Finished running

If a peak-caller is not specified, it defaults to spp. In such case, macs2 is used for generating bigwigs. Yes if you got nodup.tagAlign.filt.regionPeak.gz, then that means spp ran successfully.
About macs2 error, it looks like macs2 is combined with wrong numpy version. Please check the following:
$ source activate aquas_chipseq
$ conda list numpy

packages in environment at /gs/project/yxw-673-aa/rdali/tools/miniconda3/envs/aquas_chipseq:

numpy                     1.10.2                   py27_0

It's a problem related to Conda's virtual environment. https://github.com/conda/conda/issues/448#issuecomment-195848539
Can you add the following to your ~/.bashrc and re-login?
export PYTHONNOUSERSITE=True

Please let me know if this works.

I get a different error now:
	INFO  @ Fri, 09 Jun 2017 15:33:37: #3   Write bedGraph files for control lambda (after scaling if necessary)... project/ChIPSeq/160130_CTCF/peak/macs2/rep1/HI.4031.007.Index_2.160130_CTCF_lib_170103_R1.PE2SE.nodup.tagAlign_x_HI.4031.007.Index_3.160130_inp_lib_170103_R1.PE2SE.nodup.tagAlign_control_lambda.bdg
	INFO  @ Fri, 09 Jun 2017 15:33:37: #3   --SPMR is requested, so pileup will be normalized by sequencing depth in million reads.
	INFO  @ Fri, 09 Jun 2017 15:33:37: #3 Call peaks for each chromosome...
	Traceback (most recent call last):
	  File "tools/miniconda3/envs/aquas_chipseq/bin/macs2", line 4, in <module>
	    __import__('pkg_resources').run_script('MACS2==<IP_ADDRESS>50731', 'macs2')
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 744, in run_script
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 1506, in run_script
	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 614, in <module>

	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/EGG-INFO/scripts/macs2", line 56, in main

	  File "tools/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/MACS2-<IP_ADDRESS>50731-py2.7-linux-x86_64.egg/MACS2/callpeak_cmd.py", line 261, in run
	  File "MACS2/PeakDetect.pyx", line 105, in MACS2.PeakDetect.PeakDetect.call_peaks (MACS2/PeakDetect.c:1632)
	  File "MACS2/PeakDetect.pyx", line 251, in MACS2.PeakDetect.PeakDetect.__call_peaks_w_control (MACS2/PeakDetect.c:3172)
	  File "MACS2/IO/CallPeakUnit.pyx", line 857, in MACS2.IO.CallPeakUnit.CallerFromAlignments.call_peaks (MACS2/IO/CallPeakUnit.c:12297)
	  File "MACS2/IO/CallPeakUnit.pyx", line 911, in MACS2.IO.CallPeakUnit.CallerFromAlignments.call_peaks (MACS2/IO/CallPeakUnit.c:12049)
	  File "MACS2/IO/CallPeakUnit.pyx", line 966, in MACS2.IO.CallPeakUnit.CallerFromAlignments.__chrom_call_peak_using_certain_criteria (MACS2/IO/CallPeakUnit.c:12455)
	  File "MACS2/IO/CallPeakUnit.pyx", line 461, in MACS2.IO.CallPeakUnit.CallerFromAlignments.__pileup_treat_ctrl_a_chromosome (MACS2/IO/CallPeakUnit.c:6457)
	EOFError

00:38:10.137	Wait: Task 'chipseq.bds.20170609_152437_788/task.callpeak_macs2_chipseq.macs2_n_s_rep1.line_83.id_10' finished.
Fatal error: tools/TF_chipseq_pipeline/chipseq.bds, line 1237, pos 2. Task/s failed.
chipseq.bds, line 76 :	main()
chipseq.bds, line 79 :	void main() { // chipseq pipeline starts here
chipseq.bds, line 91 :		call_peaks() // call peaks in parallel (MACS2,SPP)
chipseq.bds, line 820 :	void call_peaks() {
chipseq.bds, line 1237 :		wait
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
00:38:10.377	Writing report file 'chipseq.bds.20170609_152437_788.report.html'
00:38:10.416	Program 'chipseq.bds.20170609_152437_788' finished, exit value: 1, tasks executed: 3, tasks failed: 2, tasks failed names: macs2 n/s rep1-pr1 , macs2 n/s rep1.
00:38:10.417	Finished. Exit code: 1
00:38:10.417	ExecutionerLocal 'Local[38]': Killed

https://github.com/taoliu/MACS/issues/105
Can you check your disk space on $TMP?
$ echo $TMP
$ df -h $TMP

$echo $TMP
$ df -h $TMP
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda6       193G   42G  141G  23% /
tmpfs            32G  149M   32G   1% /dev/shm
/dev/sda2       477M  116M  337M  26% /boot
/dev/sda1       200M  260K  200M   1% /boot/efi
/dev/sda3        30G   20G  8.1G  72% /cvmfs-cache
/dev/gs         3.1P  2.9P  221T  93% /gs
/dev/lb         1.3P  1.1P  145T  89% /lb
/dev/sb         456T  405T   52T  89% /sb
/dev/sf1        349T  229T  120T  66% /sf1
I had set $TMPDIR to the extra 1TB not $TMP:
$echo $TMPDIR
/gs/scratch/
$ df -h $TMPDIR
Filesystem      Size  Used Avail Use% Mounted on
/dev/gs         1.0T  2.0M  1.0T   1% /gs

Sorry. MACS2 does not use $TMP or $TMPDIR. It always writes temporary files on /tmp. Can you check your /tmp too?

df -h /tmp
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda6       217G   19G  188G   9% /

I guess that your /tmp was running out of space at the moment you ran the pipeline. Can you try running pipelines again?

Looks like it ran to completion. Thanks!