[ADAM-1018] Add support for Spark SQL Datasets.

Resolves #1018. Adds the adam-codegen module, which generates classes that:

Implement the Scala Product interface and thus can be read into a Spark SQL Dataset.
Have a complete constructor that is compatible with the constructor that Spark SQL expects to see when exporting a Dataset back to Scala.
And, that have methods for converting to/from the bdg-formats Avro models.

Then, we build these model classes in the org.bdgenomics.adam.sql package, and use them for export from the Avro based GenomicRDDs. With a Dataset, we can then export to a DataFrame, which enables us to expose data through Python via RDD->Dataset->DataFrame. This is important since the Avro classes generated by bdg-formats can't be pickled, and thus we can't do a Java RDD to Python RDD crossing with them.
This is on top of #1387.
To do:

[ ] Add Python/SQL docs
[ ] Add pyadam-shell script (thoughts on name?)
[ ] Finish Dataset bind-on-load work, pending review comments
[ ] PyPI

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1788/Build result: FAILURE[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains f6661e40fb6240282f6b7360de9797983ea202cc # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1391/merge^{commit} # timeout=10Checking out Revision f6661e40fb6240282f6b7360de9797983ea202cc (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f f6661e40fb6240282f6b7360de9797983ea202ccFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

This is awesome, nice work!  A few general questions before I dig in to review:
Are the new java-friendly methods on *RDD (from the first commit) necessary or simply more convenient?
Does the dataset stuff compile against Spark 1.6.x or is there some conditional build logic I'm not seeing?
Is name.capitalize in DumpSchemasToProduct going to work consistently everywhere (in other words, do we have any oddly camel-cased field names in bdg-formats)?

Thanks @heuermh!
The Java friendly methods are for the Python support in #1387. The TL;DR is that Py4j needs "proper" Java methods (no default parameters, value types are java.lang..., etc)
Datasets came in Spark 1.6.0. We'll need to drop Spark 1.5 and earlier, but I think that makes sense generally now that the blessed release of Spark is 2.1.0.
Yup! The Capitalize method just capitalizes the first letter of a string, which is the same behavior Avro uses for making getters/setters.

+1 to dropping support for Spark 1.5, which is really only a todo for docs & Jenkins

Rebased on latest and greatest changes over at #1387, and trimmed Jenkins down to Spark 1.6.2 and 2.0.0.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1804/Build result: FAILURE[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 4556d458547fb0947ceb9894d6912216d912c208 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1391/merge^{commit} # timeout=10Checking out Revision 4556d458547fb0947ceb9894d6912216d912c208 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 4556d458547fb0947ceb9894d6912216d912c208First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1806/
Test PASSed.

Are there currently any tests that roundtrip an RDD (or its ADAM wrapper like VariantRDD) to a dataset and back?

RE: tests, these are a bit thin, but https://github.com/bigdatagenomics/adam/pull/1391/files#diff-306c6dde689c4cbcb6d3e982dc401221R81 forces a round trip RDD->Dataset->RDD.

What about moving from ugly-Java classes with getters and setters to case classes (like they have in https://github.com/hammerlab/spark-genomics )?

@antonkulaga I'd love to do that, but that would require an almost complete rewrite of the core read transformations and format conversion classes, which is pretty hard to justify from the perspective of engineering effort. It's something like a 10k+ LOC rewrite. Realistically, I think we'll move away from the Java Avro models slowly as we rewrite blocks on top of the Dataset APIs over the next year.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1937/Build result: FAILURE[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 74aafd990026fc13a97d878b93e1f945d27c2d49 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1391/merge^{commit} # timeout=10Checking out Revision 74aafd990026fc13a97d878b93e1f945d27c2d49 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 74aafd990026fc13a97d878b93e1f945d27c2d49First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1983/Build result: FAILURE[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 9662833ef2e2d65041d6624c9f662d14572b530c # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1391/merge^{commit} # timeout=10Checking out Revision 9662833ef2e2d65041d6624c9f662d14572b530c (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 9662833ef2e2d65041d6624c9f662d14572b530cFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-8.7%) to 73.199% when pulling 133fcca0ffb3f228a56ad16c56e9c5453e2c661d on fnothaft:issues/1018-dataset-api into ea9ce6c9af6b77943be2fedc01e20e139da74111 on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1984/Build result: FAILURE[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 312c057581461d455ccca66b4a1f00c7dfb01746 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1391/merge^{commit} # timeout=10Checking out Revision 312c057581461d455ccca66b4a1f00c7dfb01746 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 312c057581461d455ccca66b4a1f00c7dfb01746First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-6.7%) to 75.184% when pulling 92db52ae4b9415938f47fa8f5a387a2c4620ba5c on fnothaft:issues/1018-dataset-api into 0dd09b258dda051dea7b10d2e79658fafd235197 on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1987/Build result: FAILURE[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains f02af2af08fa4462f13edd7adf415ae11ed751fd # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1391/merge^{commit} # timeout=10Checking out Revision f02af2af08fa4462f13edd7adf415ae11ed751fd (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f f02af2af08fa4462f13edd7adf415ae11ed751fdFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh et al
I've started a more ambitious push in https://github.com/bigdatagenomics/adam/pull/1391/commits/92db52ae4b9415938f47fa8f5a387a2c4620ba5c to dynamically choose whether to load data using ParquetAvro or Spark SQL by an elaborate set of tweaks to the GenomicRDD class hierarchy. For now, I've only built this out on AlignmentRecordRDD. Anyone have comments on this? I think I'm going to build it out further tonight/tomorrow across the other types, but would love early feedback.

@jpdna just pushed a fix for the k-mer counting issue you were having on #1387.

Coverage decreased (-6.7%) to 75.148% when pulling f60f21a8206dbac2385747a6506d1bb619fa48c6 on fnothaft:issues/1018-dataset-api into 0dd09b258dda051dea7b10d2e79658fafd235197 on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1988/Build result: FAILURE[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 7cef7ddb114714e227a2cca6b00bbc1d9d53c4a9 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1391/merge^{commit} # timeout=10Checking out Revision 7cef7ddb114714e227a2cca6b00bbc1d9d53c4a9 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 7cef7ddb114714e227a2cca6b00bbc1d9d53c4a9First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, retest this please.

Coverage decreased (-6.7%) to 75.148% when pulling f60f21a8206dbac2385747a6506d1bb619fa48c6 on fnothaft:issues/1018-dataset-api into 0dd09b258dda051dea7b10d2e79658fafd235197 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1990/
Test PASSed.

Coverage decreased (-6.8%) to 75.072% when pulling 083f60abddb7361be1f9cc3f03b12c925faed87c on fnothaft:issues/1018-dataset-api into 0dd09b258dda051dea7b10d2e79658fafd235197 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2006/
Test PASSed.

Coverage decreased (-7.0%) to 75.072% when pulling 3523d785338ebf84262262d5d579f6e063f000bb on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2035/
Test PASSed.

Coverage decreased (-6.6%) to 75.48% when pulling c9152d622da62454e3d0b2246f42044ebb61742c on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2037/
Test PASSed.

Coverage decreased (-6.3%) to 75.716% when pulling 5d5f196278203f883735ffa0869d5bc9209ef2aa on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2039/
Test PASSed.

Coverage decreased (-6.3%) to 75.73% when pulling 5c1604fd38564e83993ab5cff962604683fc8416 on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2040/
Test PASSed.

Coverage decreased (-6.3%) to 75.73% when pulling 5c1604fd38564e83993ab5cff962604683fc8416 on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2041/
Test PASSed.

Coverage decreased (-4.5%) to 77.588% when pulling 08eb06b88658bceef4f08e60dda8996c7c41f20b on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2042/
Test PASSed.

Coverage decreased (-4.4%) to 77.624% when pulling 07d40ad9820e3a536fd10ab3b8a03205a29c89ce on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2044/
Test PASSed.

This is good to go, other than adding docs. Can I get a code review?

Coverage decreased (-4.4%) to 77.641% when pulling 503664f8ff77f7caa8e457fadf156f6f30cadf21 on fnothaft:issues/1018-dataset-api into 2820e940c0df57a3a9c6439060ae4e53f8a466b8 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2052/
Test PASSed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2056/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 09049ea92814a31b80c330936c6b1bea1ab9ec24 # timeout=10Checking out Revision 09049ea92814a31b80c330936c6b1bea1ab9ec24 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 09049ea92814a31b80c330936c6b1bea1ab9ec24 > /home/jenkins/git2/bin/git rev-list 345d9003ff11b7c5aeee72eca436cc786400c0f2 # timeout=10Triggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

bdg-formats 0.11.0 has pushed to Maven Central. This is good to merge from my end.

@jpdna just pushed a fix for the k-mer counting issue you were having on #1387.

Yup - thanks @fnothaft worked for me too.
Just to restate as we may want in the python docs, this is what I did after building this branch to test at python REPL
export PYSPARK_SUBMIT_ARGS="--jars $ADAM_HOME/adam-assembly/target/adam_2.10-0.23.0-SNAPSHOT.jar --driver-class-path  $ADAM_HOME/adam-assembly/target/adam_2.10-0.23.0-SNAPSHOT.jar pyspark-shell"

export PYTHONPATH="$SPARK_HOME/python/lib/py4j-0.9-src.zip:$SPARK_HOME/python/lib/pyspark.zip"

#started a python virtualenv

$ADAM_HOME/adam-python/make develop

#In normal python REPL
from pyspark import SparkContext
from bdgenomics.adam.adamContext import ADAMContext
sc = SparkContext("local","jpappname")
ac = ADAMContext(sc)
reads = ac.loadAlignments("/home/paschallj/adam_python/v2/adam/adam-core/src/test/resources/small.sam")

What combinations of Spark and Scala versions is this expected to work on?
I see a failure when running the tests with maven under Spark 2.1.0 and scala 2.10 ( I didn't test Spark 2.1.0 and scala 2.11 yet)
 load parquet to sql, save, re-read from avro *** FAILED ***
  org.apache.spark.sql.AnalysisException: 'save' does not support bucketing right now;
  at org.apache.spark.sql.DataFrameWriter.assertNotBucketed(DataFrameWriter.scala:314)
  at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:207)
  at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:198)
  at org.bdgenomics.adam.rdd.read.DatasetBoundAlignmentRecordRDD.saveAsParquet(AlignmentRecordRDD.scala:180)
  at org.bdgenomics.adam.rdd.AvroGenomicRDD.saveAsParquet(GenomicRDD.scala:1119)
  at org.bdgenomics.adam.rdd.AvroGenomicRDD.saveAsParquet(GenomicRDD.scala:1181)
  at org.bdgenomics.adam.rdd.read.AlignmentRecordRDDSuite$$anonfun$20.apply$mcV$sp(AlignmentRecordRDDSuite.scala:508)
  at org.bdgenomics.utils.misc.SparkFunSuite$$anonfun$sparkTest$1.apply$mcV$sp(SparkFunSuite.scala:102)
  at org.bdgenomics.utils.misc.SparkFunSuite$$anonfun$sparkTest$1.apply(SparkFunSuite.scala:98)
  at org.bdgenomics.utils.misc.SparkFunSuite$$anonfun$sparkTest$1.apply(SparkFunSuite.scala:98)
  at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
  at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
  at org.scalatest.FunSuite.withFixture(FunSuite.scala:1555)
  at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
  at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
  at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
  at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
  at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
  at org.bdgenomics.adam.util.ADAMFunSuite.org$scalatest$BeforeAndAfter$$super$runTest(ADAMFunSuite.scala:24)
  at org.scalatest.BeforeAndAfter$class.runTest(BeforeAndAfter.scala:200)
  at org.bdgenomics.adam.util.ADAMFunSuite.runTest(ADAMFunSuite.scala:24)
  at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
  at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
  at scala.collection.immutable.List.foreach(List.scala:318)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
  at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
  at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
  at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:208)
  at org.scalatest.FunSuite.runTests(FunSuite.scala:1555)
  at org.scalatest.Suite$class.run(Suite.scala:1424)
  at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1555)
  at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
  at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
  at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
  at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:212)
  at org.bdgenomics.adam.util.ADAMFunSuite.org$scalatest$BeforeAndAfter$$super$run(ADAMFunSuite.scala:24)
  at org.scalatest.BeforeAndAfter$class.run(BeforeAndAfter.scala:241)
  at org.bdgenomics.adam.util.ADAMFunSuite.run(ADAMFunSuite.scala:24)
  at org.scalatest.Suite$class.callExecuteOnSuite$1(Suite.scala:1492)
  at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1528)
  at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1526)
  at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
  at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
  at org.scalatest.Suite$class.runNestedSuites(Suite.scala:1526)
  at org.scalatest.tools.DiscoverySuite.runNestedSuites(DiscoverySuite.scala:29)
  at org.scalatest.Suite$class.run(Suite.scala:1421)
  at org.scalatest.tools.DiscoverySuite.run(DiscoverySuite.scala:29)
  at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:55)
  at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2563)
  at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2557)
  at scala.collection.immutable.List.foreach(List.scala:318)
  at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:2557)
  at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1044)
  at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1043)
  at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:2722)
  at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1043)
  at org.scalatest.tools.Runner$.main(Runner.scala:860)
  at org.scalatest.tools.Runner.main(Runner.scala)

*** 1 TEST FAILED ***
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] ADAM_2.10 .......................................... SUCCESS [ 13.800 s]
[INFO] ADAM_2.10: Avro-to-Dataset codegen utils ........... SUCCESS [ 13.719 s]
[INFO] ADAM_2.10: Core .................................... FAILURE [06:57 min]
[INFO] ADAM_2.10: APIs for Java ........................... SKIPPED
[INFO] ADAM_2.10: CLI ..................................... SKIPPED
[INFO] ADAM_2.10: Assembly ................................ SKIPPED
[INFO] ------------------------------------------------------------------------

Possibly related, I also get and error when running this at adam shell:

import org.bdgenomics.adam.rdd.ADAMContext._
import org.bdgenomics.adam.projections.{ AlignmentRecordField, Projection }
val x = sc.loadAlignments("adam-core/src/test/resources/small.sam")
val mydataset = x.dataset

where it seems to work with Spark 1.6.3

@devin-petersohn  and others:
When we write from dataset to parquet like with:
https://github.com/bigdatagenomics/adam/pull/1391/files#diff-610e7c7659ef318ca13f300f14e83307R169
would we expect to write out a  _partitionMap.avro based on the recent merge of Devin's work?
I was recently experimenting with this PR by add a parititionBy like:
    dataset.toDF()
      .write
      .format("parquet")
      .option("spark.sql.parquet.compression.codec", compressCodec.toString.toLowerCase())
      .partitionBy("contigName")
      .save(filePath)
    saveMetadata(filePath)

as a way to use Spark-SQL parquet hive-like dir based partitioning described here (only in Spark 2) https://spark.apache.org/docs/latest/sql-programming-guide.html#partition-discovery, which can be picked up by sql queries, as a way to divide up parquet files into a directory hierarchy of chr or even sub-chr 10MB bins in a way that Spark-SQL can understand and use to optimize range-based genomic queries in a course-grain way.
I am curious to evaluate whether that is useful or if it is redundant to Devine's work, thus question above as to if we should be writing out a
_partitionMap.avro   If there were very little cost to the SQL/Hive type dir based partitioning, maybe we could do both that and the _paritionMap.avro in a complementary way.

For our purposes, a partitionMap has a very strict definition: The data is sorted, and the partitionMap contains the bounds of each partition. There are options in the future to take further advantage of it, but for now that is how we define it.

When trying to run the python apis, all of the GenomicRDD functions seem to be unaccessible after calling _jvmRdd.
For example, reads.toDataFrame() on an AlignmentRecordRDD generates the error:
Py4JError: An error occurred while calling o48.toDataset. Trace:
py4j.Py4JException: Method toDataset([]) does not exist
	at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:335)
	at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:344)
	at py4j.Gateway.invoke(Gateway.java:252)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:209)
	at java.lang.Thread.run(Thread.java:745)

This seems to happen for all functions after calling  _jvmRdd.

@akmorrow13 can you provide the full sequence of calls you made? I'd like to understand better.

Sure @fnothaft!
Here is what I was running in the notebook:
import sys
sys.path.append("/Users/akmorrow/ADAM/adam/adam-python/bdgenomics.adam-0.21.1_SNAPSHOT-py2.7.egg")
from bdgenomics.adam.adamContext import ADAMContext
ac = ADAMContext(sc)

testFile = '/Users/akmorrow/ADAM/mango/example-files/chr17.7500000-7515000.bam.adam'
reads = ac.loadAlignments(testFile)

from pyspark.sql import DataFrame, SQLContext

reads.toDataFrame()

Where I get the error:
Py4JError: An error occurred while calling o48.toDataset. Trace:
py4j.Py4JException: Method toDataset([]) does not exist
	at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:335)
	at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:344)
	at py4j.Gateway.invoke(Gateway.java:252)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:209)
	at java.lang.Thread.run(Thread.java:745)

Just cleared the conflicts.
@akmorrow13 I'm pretty sure you're running on an out of date version of the python bindings? There's no toDataFrame in the current bindings.
@devin-petersohn let's chat about adam-core/src/test/scala/org/bdgenomics/adam/rdd/SortedGenomicRDDSuite.scala. It appears that there was a bug in Scala case classes that was allowing the optPartitionMap field to escape it's protected status. As such, there's no good way to make SortedGenomicRDDSuite compile right now without either removing test code or opening up protections on optPartitionMap. Let me know your thoughts.

Coverage decreased (-5.1%) to 78.214% when pulling fc2077ea5f359d7aa9bf78eff6498cc84825e784 on fnothaft:issues/1018-dataset-api into 152a8ad17509919f25a4f5b7c04161c5b313fc33 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2109/
Test PASSed.

This is an interesting one. I don't recall the optPartitionMap having a protected access modifier. I thought it was package private (private[rdd]), but the git blame doesn't show that. The interesting thing here is that this diff shows that it was private[rdd]. I will do more digging to see where it was changed, but right now even my github branch shows protected, so maybe I am crazy. I'll have to search through the logs to see if I can find it.
I'll make a PR against your branch later this morning for my recommended updates. We can leave this as protected and just create a private class that opens up the protections to the test.

@fnothaft - I tried many different ways to access it, but there is no way to test the partition map. All we are doing generally is making sure the partition map is sorted in most of those tests, so there's no need to keep them. The one test that I do care about is the last one where we test that we have the same partition map in memory that we wrote to disk.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2114/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains d3f6329eeefac2e1c8493a9fdf7c5fd2526de3a9 # timeout=10Checking out Revision d3f6329eeefac2e1c8493a9fdf7c5fd2526de3a9 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f d3f6329eeefac2e1c8493a9fdf7c5fd2526de3a9 > /home/jenkins/git2/bin/git rev-list 69b10f54228ae95f6642d18f02643da3cdbc4b0a # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

TIL: http://docs.scala-lang.org/tutorials/FAQ/initialization-order.html

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2115/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains fe3edc516afe2e86b3597252c2a48b03f70523de # timeout=10Checking out Revision fe3edc516afe2e86b3597252c2a48b03f70523de (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f fe3edc516afe2e86b3597252c2a48b03f70523de > /home/jenkins/git2/bin/git rev-list d3f6329eeefac2e1c8493a9fdf7c5fd2526de3a9 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, retest this please.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2116/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains fe3edc516afe2e86b3597252c2a48b03f70523de # timeout=10Checking out Revision fe3edc516afe2e86b3597252c2a48b03f70523de (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f fe3edc516afe2e86b3597252c2a48b03f70523de > /home/jenkins/git2/bin/git rev-list fe3edc516afe2e86b3597252c2a48b03f70523de # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2117/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 512f5ad1ebee870d9349e90b0552cf6478294286 # timeout=10Checking out Revision 512f5ad1ebee870d9349e90b0552cf6478294286 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 512f5ad1ebee870d9349e90b0552cf6478294286 > /home/jenkins/git2/bin/git rev-list fe3edc516afe2e86b3597252c2a48b03f70523de # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Changes Unknown when pulling 98478e13dfb7470a84c5cd922efb38c7cd78b8a1 on fnothaft:issues/1018-dataset-api into ** on bigdatagenomics:master**.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2118/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ccb3c9634621b0f90c376b90a61112498f4eb5be # timeout=10Checking out Revision ccb3c9634621b0f90c376b90a61112498f4eb5be (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ccb3c9634621b0f90c376b90a61112498f4eb5be > /home/jenkins/git2/bin/git rev-list 512f5ad1ebee870d9349e90b0552cf6478294286 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-4.09%) to 78.955% when pulling 4a5230978e08d26c904e79cbcf54363fd2611400 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2122/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 370a1984247abbef575fa109831fdb7f30a64211 # timeout=10Checking out Revision 370a1984247abbef575fa109831fdb7f30a64211 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 370a1984247abbef575fa109831fdb7f30a64211 > /home/jenkins/git2/bin/git rev-list ccb3c9634621b0f90c376b90a61112498f4eb5be # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.7%) to 82.32% when pulling 4a5230978e08d26c904e79cbcf54363fd2611400 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Coverage decreased (-0.7%) to 82.32% when pulling 4a5230978e08d26c904e79cbcf54363fd2611400 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2123/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 370a1984247abbef575fa109831fdb7f30a64211 # timeout=10Checking out Revision 370a1984247abbef575fa109831fdb7f30a64211 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 370a1984247abbef575fa109831fdb7f30a64211 > /home/jenkins/git2/bin/git rev-list 370a1984247abbef575fa109831fdb7f30a64211 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-0.7%) to 82.32% when pulling 8d38e6acf4b79817b7f766b51ec2b09c3525e3b9 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2124/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains bc50354959a38201c1e097997327d3122f0bd132 # timeout=10Checking out Revision bc50354959a38201c1e097997327d3122f0bd132 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f bc50354959a38201c1e097997327d3122f0bd132 > /home/jenkins/git2/bin/git rev-list 370a1984247abbef575fa109831fdb7f30a64211 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2125/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains bc50354959a38201c1e097997327d3122f0bd132 # timeout=10Checking out Revision bc50354959a38201c1e097997327d3122f0bd132 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f bc50354959a38201c1e097997327d3122f0bd132 > /home/jenkins/git2/bin/git rev-list bc50354959a38201c1e097997327d3122f0bd132 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.439% when pulling 8d38e6acf4b79817b7f766b51ec2b09c3525e3b9 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2126/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains bc50354959a38201c1e097997327d3122f0bd132 # timeout=10Checking out Revision bc50354959a38201c1e097997327d3122f0bd132 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f bc50354959a38201c1e097997327d3122f0bd132 > /home/jenkins/git2/bin/git rev-list bc50354959a38201c1e097997327d3122f0bd132 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-0.6%) to 82.439% when pulling ede80d1079b16a94b7b299b5767332196157f886 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2127/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 5b1f270ea5162a1d8669ac38e97623fb08ba456d # timeout=10Checking out Revision 5b1f270ea5162a1d8669ac38e97623fb08ba456d (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 5b1f270ea5162a1d8669ac38e97623fb08ba456d > /home/jenkins/git2/bin/git rev-list bc50354959a38201c1e097997327d3122f0bd132 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.454% when pulling ede80d1079b16a94b7b299b5767332196157f886 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2128/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 5b1f270ea5162a1d8669ac38e97623fb08ba456d # timeout=10Checking out Revision 5b1f270ea5162a1d8669ac38e97623fb08ba456d (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 5b1f270ea5162a1d8669ac38e97623fb08ba456d > /home/jenkins/git2/bin/git rev-list 5b1f270ea5162a1d8669ac38e97623fb08ba456d # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-0.6%) to 82.439% when pulling a1593837570bd0ed558643379a6f22a8ce63a6eb on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2129/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 76c088c146209eb1a46bb58d59185dcc0d2711b5 # timeout=10Checking out Revision 76c088c146209eb1a46bb58d59185dcc0d2711b5 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 76c088c146209eb1a46bb58d59185dcc0d2711b5 > /home/jenkins/git2/bin/git rev-list 5b1f270ea5162a1d8669ac38e97623fb08ba456d # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, retest this please.

Coverage decreased (-0.6%) to 82.474% when pulling 2c522529425aaa16cfa915d70766a0b2f59b3e5b on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2130/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 # timeout=10Checking out Revision 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 > /home/jenkins/git2/bin/git rev-list 76c088c146209eb1a46bb58d59185dcc0d2711b5 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.489% when pulling 2c522529425aaa16cfa915d70766a0b2f59b3e5b on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2131/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 # timeout=10Checking out Revision 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 > /home/jenkins/git2/bin/git rev-list 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-0.1%) to 82.951% when pulling 92e296eb98c3ec585a4c365c37dd3f07827e9849 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2133/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 3b9453b9955b06f61358e1ee5ad07c0c53fe766e # timeout=10Checking out Revision 3b9453b9955b06f61358e1ee5ad07c0c53fe766e (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3b9453b9955b06f61358e1ee5ad07c0c53fe766e > /home/jenkins/git2/bin/git rev-list 0c20ea4d6cc9e2cb6dab0cd15a7c18a086707719 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.405% when pulling 92e296eb98c3ec585a4c365c37dd3f07827e9849 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2135/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 3b9453b9955b06f61358e1ee5ad07c0c53fe766e # timeout=10Checking out Revision 3b9453b9955b06f61358e1ee5ad07c0c53fe766e (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3b9453b9955b06f61358e1ee5ad07c0c53fe766e > /home/jenkins/git2/bin/git rev-list 3b9453b9955b06f61358e1ee5ad07c0c53fe766e # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.42% when pulling 92e296eb98c3ec585a4c365c37dd3f07827e9849 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2136/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 3b9453b9955b06f61358e1ee5ad07c0c53fe766e # timeout=10Checking out Revision 3b9453b9955b06f61358e1ee5ad07c0c53fe766e (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3b9453b9955b06f61358e1ee5ad07c0c53fe766e > /home/jenkins/git2/bin/git rev-list 3b9453b9955b06f61358e1ee5ad07c0c53fe766e # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-0.6%) to 82.42% when pulling 8f94907d1b9eb947e1d792a1e4088cf5bcfe7b42 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2137/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 333a16fe26338cbd92797e59f6f764fe3be66e29 # timeout=10Checking out Revision 333a16fe26338cbd92797e59f6f764fe3be66e29 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 333a16fe26338cbd92797e59f6f764fe3be66e29 > /home/jenkins/git2/bin/git rev-list 3b9453b9955b06f61358e1ee5ad07c0c53fe766e # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.462% when pulling 8f94907d1b9eb947e1d792a1e4088cf5bcfe7b42 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2138/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 333a16fe26338cbd92797e59f6f764fe3be66e29 # timeout=10Checking out Revision 333a16fe26338cbd92797e59f6f764fe3be66e29 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 333a16fe26338cbd92797e59f6f764fe3be66e29 > /home/jenkins/git2/bin/git rev-list 333a16fe26338cbd92797e59f6f764fe3be66e29 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-0.6%) to 82.462% when pulling 695dab1f4980e05e710d2e607b039390437bca8d on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2140/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains a6041a3b5ff97dcefacaacb10e9cfffd75c26921 # timeout=10Checking out Revision a6041a3b5ff97dcefacaacb10e9cfffd75c26921 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f a6041a3b5ff97dcefacaacb10e9cfffd75c26921 > /home/jenkins/git2/bin/git rev-list 333a16fe26338cbd92797e59f6f764fe3be66e29 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.477% when pulling 695dab1f4980e05e710d2e607b039390437bca8d on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2141/
Test PASSed.

Coverage decreased (-0.6%) to 82.462% when pulling d5ea78a95321f6ca0a72e38c0145652887047c8e on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2143/
Test PASSed.

This is good to go from my end. Pinging for a review.

Coverage decreased (-0.6%) to 82.462% when pulling 06a7701194f6dff6569ac4f22b5dec4c2d3fe274 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2152/
Test PASSed.

I was testing setting up ENV variables and starting up a Python REPL again as I wanted to add docs on how to do that,but find that what I did earlier no longer works.
I try:
export PYSPARK_SUBMIT_ARGS="--jars $ADAM_HOME/adam-assembly/target/adam_2.10-0.23.0-SNAPSHOT.jar --driver-class-path  $ADAM_HOME/adam-assembly/target/adam_2.10-0.23.0-SNAPSHOT.jar pyspark-shell"
export PYTHONPATH="$SPARK_HOME/python/lib/py4j-0.9-src.zip:$SPARK_HOME/python/lib/pyspark.zip"
#started a python virtualenv
$ADAM_HOME/adam-python/make develop

#In normal python REPL
from pyspark import SparkContext
from bdgenomics.adam.adamContext import ADAMContext
sc = SparkContext("local","jpappname")
ac = ADAMContext(sc)
reads = ac.loadAlignments("/home/paschallj/adam_python/v2/adam/adam-core/src/test/resources/small.sam")

But at ac = ADAMContext(sc) I get:
>>> ac = ADAMContext(sc)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/paschalj/test9/adam_frank_dataset/adam/adam-python/src/bdgenomics/adam/adamContext.py", line 44, in __init__
    c = self._jvm.org.bdgenomics.adam.rdd.ADAMContext(sc._jsc.sc())
TypeError: 'JavaPackage' object is not callable

Hmm, that all looks like it should be OK @jpdna. What happens if you launch a Spark shell with
spark-shell \
  --jars $ADAM_HOME/adam-assembly/target/adam_2.10-0.23.0-SNAPSHOT.jar \
  --driver-class-path  $ADAM_HOME/adam-assembly/target/adam_2.10-0.23.0-SNAPSHOT.jar

Can you import org.bdgenomics.adam.rdd.ADAMContext successfully?

Can you import org.bdgenomics.adam.rdd.ADAMContext successfully?

Ah, I just got it to work, the jar file is named now adam-assembly_2.10-0.23.0-SNAPSHOT.jar when I guess it was not before.
so
export PYSPARK_SUBMIT_ARGS="--jars $ADAM_HOME/adam-assembly/target/adam-assembly_2.10-0.23.0-SNAPSHOT.jar --driver-class-path  $ADAM_HOME/adam-assembly/target/adam-assembly_2.10-0.23.0-SNAPSHOT.jar pyspark-shell"
export PYTHONPATH="$SPARK_HOME/python/lib/py4j-0.9-src.zip:$SPARK_HOME/python/lib/pyspark.zip"

#setup ADAM python
$ADAM_HOME/adam-python/make develop

#Start python REPL
from pyspark import SparkContext
from bdgenomics.adam.adamContext import ADAMContext
sc = SparkContext("local","jpappname")
ac = ADAMContext(sc)
reads = ac.loadAlignments("/home/paschallj/adam_python/v2/adam/adam-core/src/test/resources/small.sam")

work for me now.  Do we want to add above or something similar to Docs - I don't see currently this info about how to setup for Python?  I will add if you suggest where.

The info for getting adam-python should be there, but yeah I think you are right that we are missing the ADAMContext entrypoint. Let me add that in.

The info for getting adam-python should be there

I must be missing it - where?  does it include like export PYSPARK_SUBMIT_ARGS... export PYTHONPATH... instructions where needed?

@jpdna python install docs start here

Coverage decreased (-0.4%) to 82.611% when pulling 9ee0375d2c4554b77acd04f9b15b1f060fe5033d on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2165/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 2eead442b33943dbe5d11820532bc58cf33d6865 # timeout=10Checking out Revision 2eead442b33943dbe5d11820532bc58cf33d6865 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 2eead442b33943dbe5d11820532bc58cf33d6865 > /home/jenkins/git2/bin/git rev-list d78d032696e61cf67e06d33ba4a61637775e7f7e # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.1.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.1.0,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.1.0,centosTriggering ADAM-prb ? 2.3.0,2.11,2.1.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,2.1.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.1.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,2.1.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.1.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-0.4%) to 82.611% when pulling 9ee0375d2c4554b77acd04f9b15b1f060fe5033d on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2166/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 2eead442b33943dbe5d11820532bc58cf33d6865 # timeout=10Checking out Revision 2eead442b33943dbe5d11820532bc58cf33d6865 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 2eead442b33943dbe5d11820532bc58cf33d6865 > /home/jenkins/git2/bin/git rev-list 2eead442b33943dbe5d11820532bc58cf33d6865 # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.10,2.1.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.1.0,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.1.0,centosTriggering ADAM-prb ? 2.3.0,2.11,2.1.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,2.1.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.1.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,2.1.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.1.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Jenkins, test this please.

Coverage decreased (-0.6%) to 82.477% when pulling 9ee0375d2c4554b77acd04f9b15b1f060fe5033d on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2168/
Test PASSed.

@heuermh I've added 3 commits that address your comments. LMK if they look good to you and I will squash them into the appropriate commits in the PR.

Coverage decreased (-0.4%) to 82.611% when pulling 54b6ab34c764e2faeee9ecfe3a1194ab94980d98 on fnothaft:issues/1018-dataset-api into 0306717cb952511d48e514475b70ae995a468b5a on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2173/
Test PASSed.

Woot!  The python stuff works for me.  Thanks for the improved docs.

Squashed and rebased.

Coverage decreased (-0.8%) to 82.512% when pulling d9b22cb638f0d634b328054104e6e9a69847c15d on fnothaft:issues/1018-dataset-api into 8572fb7e8f32c78580306c9a685d07b9cca5e54e on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2177/
Test PASSed.

Rebased and addressed review comments.

Coverage decreased (-1.6%) to 82.532% when pulling 95efe05093c0bc4cb1fd73ad1327681c8d1bd46f on fnothaft:issues/1018-dataset-api into acb0c0731da1644c8222fa5af4926d54e93a9885 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2187/
Test PASSed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2188/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains fffb194e87a4c5b2b788cc0fbc6911434ace31bf # timeout=10Checking out Revision fffb194e87a4c5b2b788cc0fbc6911434ace31bf (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f fffb194e87a4c5b2b788cc0fbc6911434ace31bf > /home/jenkins/git2/bin/git rev-list 3b62fe1f15d69e832817f4315cb7e9b540db40fb # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2189/Build result: FAILURE[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1391/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains b286cf604daccc72b7a80738f39fa860941ecbc1 # timeout=10Checking out Revision b286cf604daccc72b7a80738f39fa860941ecbc1 (origin/pr/1391/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f b286cf604daccc72b7a80738f39fa860941ecbc1 > /home/jenkins/git2/bin/git rev-list fffb194e87a4c5b2b788cc0fbc6911434ace31bf # timeout=10Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Coverage decreased (-1.4%) to 82.796% when pulling 4851a77be29a40dfc056efbf1333361bb638eb46 on fnothaft:issues/1018-dataset-api into acb0c0731da1644c8222fa5af4926d54e93a9885 on bigdatagenomics:master.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2190/
Test PASSed.

Thank you, @fnothaft!  Also thanks to @jpdna @devin-petersohn for review!

@fnothaft any idea what the performance implication is with this change?

Depends on the kernel. We've got a smattering of various kernels which are fully/partially ported over to Spark SQL. Here's a (very unscientific) few off of the top of my head:

#1528, which ports the read coverage counter over to Spark SQL leads to a 2x performance improvement on our standard test dataset.
Upstream, https://github.com/bigdatagenomics/avocado/pull/239 moved the genotyper over to Spark SQL which was also about a 2x performance improvement, off of the top of my head. That's a somewhat impure benchmark though, because I just ported the aggregation step that scores the variants over to SQL. The front-end of the genotyper is still RDD based, and the post-aggregation filtration is also still RDD based (it was pending this PR).
I need to look up the benchmarks that I ran for the k-mer counter that I ported over in this PR.
Ditto for various simple queries (e.g., variant count a la #879, and flagstat) that are ripe to move over to Spark SQL.
We have a old branch sitting around that rewrote the duplicate marker on top of Spark SQL; this led to a 25% speedup on Spark 1.5.2. I'm going to clean that up on top of this work, and we'll see what perf it gives us now.

Nice! Great to know that the work we did in the last two years are paying off and helping genomics :)

@rxin Oh yeah! I'm quite excited about the future possibilities we've got on top of Spark SQL. I see this PR as simply the first step towards moving more code over to the SQL APIs, and moving further away from RDDs.


Adds a Delaunay triangulation capability to Quest

Summary

This PR is a feature

It adds a new Delaunay triangulation capability to quest in 2D and 3D.
The triangulation is built incrementally by inserting successive points and updating to mesh to ensure it satisfies the Delaunay condition: No vertices are in the circumsphere of any triangle/tetrahedron in the mesh. For each inserted point, we remove the triangles/tetrahedra in its "Delaunay cavity" and fill this hole with a new set of triangles/tetrahedra containing the new point (the "Delaunay ball").

The underlying triangulation is encoded using the Indexed mesh data structure with Adjacencies (IA), which is implemented using slam's dynamic sets, relations and maps.

This PR contains several other optimizations and improvements throughout axom in support of this new capability, primarily in core, slam and primal.

The main new additions are quest::Delaunay.hpp and slam's IAMesh class as well as optimizations and improvements to some geometric primitives in primal, such as Triangle, Tetrahedron and Sphere.
There is also a driver example delaunay_triangulation which generates a triangle or tetrahedral mesh from a specified number of random points within a rectilinear bounding box.

Credits: This capability was initially implemented by @raineyeh during Summer 2017 and has been sitting in a branch.

Note: This PR got pretty big as I was optimizing the Delaunay triangulation, which depended on large parts of core, slam, primal and spin.  I'll add some comments to highlight the key parts.


Ditch the select from functions in the DB

In the database we have these SelectFrom functions that allow us to do arbitrarily complicated things in choosing a collection of rows from the database.    Much better to just have a SelectALL function that just does the simple thing necessary to get all rows from the DB.  This should be faster for the typical use case, and simplify the code significantly.

Migrated to trello.


T265 Mask API

Adds a new Mask API that allows the user to continually send a tracking mask to inform the T265 that it should avoid creating features on objects that are either static to the device (e.g. the boom on a drone) or not static with respect to the device, but also not static in the world (e.g. controllers in a VR scenario).

Once #5213 has been merged, this will be re-based on top of development and we will consider merging it then.

Any plans to merge this?


Update pipeline for new ws

Enables hermetic builds and unit tests for tekton pipelines in the new ws
Fixes: https://github.com/securesign/scaffolding/issues/79

@lance

/ok-to-test

/retest

Seeing this for the prefetch-dependencies task on ct-server.
STEP-PREFETCH-DEPENDENCIES

2023-10-10 13:41:02,300 ERROR InvalidInput: 1 validation error for user input\npackages -> 1\n  package path does not exist (or is not a directory): hack/build-assets/certificate-transparency-go-1.1.6
Error: InvalidInput: 1 validation error for user input
packages -> 1
  package path does not exist (or is not a directory): hack/build-assets/certificate-transparency-go-1.1.6

Seeing this for the prefetch-dependencies task on ct-server.
STEP-PREFETCH-DEPENDENCIES

2023-10-10 13:41:02,300 ERROR InvalidInput: 1 validation error for user input\npackages -> 1\n  package path does not exist (or is not a directory): hack/build-assets/certificate-transparency-go-1.1.6
Error: InvalidInput: 1 validation error for user input
packages -> 1
  package path does not exist (or is not a directory): hack/build-assets/certificate-transparency-go-1.1.6

Ah i see the issue its not under hack/build-assets/certificate-transparency-go-1.1.6 its its under hack/build-assets, will follow up with a fix

/retest

/retest

/retest

/retest

Still something going on with ct-server it seems.
[build] [1/2] STEP 5/8: COPY hack/build-assets/certificate-transparency-go-1.1.6/certificate-transparency-go-1.1.6.tar.gz /ct_server/
[build] Error: building at STEP "COPY hack/build-assets/certificate-transparency-go-1.1.6/certificate-transparency-go-1.1.6.tar.gz /ct_server/": checking on sources under "/workspace/source": copier: stat: "/hack/build-assets/certificate-transparency-go-1.1.6/certificate-transparency-go-1.1.6.tar.gz": no such file or directory


Added Groovy DSL definition for C++/Java PR builds

Motivation
Added Jenkins DSL definitions for new PR validation jobs.
This will create one job for Java8 tests and one for C++/Python which will be running in parallel.

retest this please


Fixed color combination

Fixes:  #1046
Description
I added the shade of black that IMO looks good for both register and login page .
Type of PR

[ ] Bug fix
[x] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)

Checklist:

[x] I have made this change from my own.
[ ] I have taken help from some online resources.
[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[ ] I have commented my code, particularly in hard-to-understand areas.
[ ] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers and screenshots after making the changes.

It is not looking appealing.


Optionally inject contentscript in iframes

Optionally inject contentscript in iframes
It useful for tweets and other widgets on page
It may be enable by default
Manifest option "all_frames": true

It require method to detect main CS and CS in iframe for same tab.
Primary must execute commands, and sub scripts must only listen and obey.
Also it require synchronization of singleton instances like button for translate selected text which should be shown only in one place - main page or iframe.
p.s. Use match_about_blank for blank iframes: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/content_scripts

If we will enable this feature, we have to:

Listen events for each contentscript instance on tab
Calculate one state for all instances, to send responses on requests (like popup requests "what's page translation state?", "what's page language?")
Exchange data between instances

We have to try find another way, like explore iframe as DOM node if possible, to keep only one contentscript instance

It would be really nice to be able to translate pages in iframes too. Probably easier to explore the iframes as DOM nodes yes