Require Definition conflicts when trying to use this library as a mix in another complicated Require app

@khalilof please provide a better description of the issues you are seeing. Hard to tell what you mean.

at line 5
define(["angular","chartist"], factory);
must be replaced with for example :
define("angular-chartist",["angular","chartist"], factory);
for some reasons I use multiple require modules in one files, and I include this library too,  then as it's stated in RequireJS site, when defining multiple modules , one must pass a module name as first parameter ,  I did it on my local and it worked as above,
so I thought sharing this info, to avoid such conflicts for other people when they come cross such case
I am using
"name": "angular-chartist.js",
"version": "3.3.13",

Can you try npm install angular-chartist.js@next and let me know if that fixes it?
You should be on v4.0.0-beta if you do that correctly. I use Babel now to generate the UMD, so maybe that fixes it.

-_- my project is not using babel at all forget about it just wanted to share the solution for such case, cause it took me a while to find the reason
you can close the issue and thanks for the quick response

@khalilof Nothing about what I said above suggested you need to be using Babel.

there is no Dist compiled file with the provided v4.0.0-beta
and the src is using some es6 features , which I don't support in my project
is it possible like the version 3 to provide a dist folder containing a es5 compiled output?

Can you try running npm install angular-chartist.js@next again. I forgot to publish to npm, it will include the dist folder once you install.

:zap:   you still have dist/  in your .gitignore,  I still don't get the dist folder

Prior to publishing, the dist/ folder gets built and added so when the package is published it is included:
I just installed it and I see it there. Please see the following:
❯ npm i angular-chartist.js@next
<EMAIL_ADDRESS>/Users/will/Projects/app-template
└──<EMAIL_ADDRESS>extraneous

❯ tree node_modules/angular-chartist.js
node_modules/angular-chartist.js
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── dist
│   ├── angular-chartist.js
│   └── angular-chartist.min.js
├── package.json
├── src
│   └── angular-chartist.js
├── test
│   └── index.spec.js
└── webpack.config.js

3 directories, 10 files

How are you installing this?

@khalilof any updates on this?


CMMN property Business Key uses wrong expression

<camunda:in businessKey="#{execution.processBusinessKey}" />
should be <camunda:in businessKey="#{caseExecution.caseBusinessKey}" />
according to: https://docs.camunda.org/manual/latest/reference/cmmn11/tasks/process-task/

Thx!


Update api-reference page to refer to ComponentProp type utility

Description (required)
I have made the text bold as suggested in the PR. Though, it looks like the odd-one-out among other texts. I addressed the same concern in this comment yesterday, but it has yet to receive a reply. If required, I will make the changes as needed.

Related issues & labels (optional)

Closes #9125
Suggested label: improve documentation

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

en
reference/api-reference.mdx
Source changed, localizations will be marked as outdated.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


[BUG] Cannot submit tasks to master

Hi,
I met an issue. Anyone can help? Thanks in advance.
After deploy spark-standalone-cluster-on-docker (images: andreper/spark-master:3.0.0) on a server (192.XX.X.X), I try to test by another PC (192.XX.X.Y).
cmd steps:
$ spark-shell --master spark://192.XX.X.X:7077
val count = sc.parallelize(1 to 1000).filter { _ =>
  val x = math.random
  val y = math.random
  x*x + y*y < 1
}.count()

I got below error.(infinite loop messages)

WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Build Env.

Images: andreper/spark-master:3.0.0
Docker Engine version:  20.10.14
Docker Compose version: Docker Compose version v2.2.3

@rilakgg I am facing the same issue. Did you find a solution for that problem?

@Estreuselito @rilakgg
This is not a bug.
You have another Spark Application running on this cluster already (e.g. a Spark Session created in Jupyter Lab). Close this Session/Spark Application.
In a Standalone Cluster default behavior for a Spark Application is to "grav" all ressources. If this is a permanent Session, it will block any other application from correctly getting any ressource.

@ThomasMannKenbun is there a way around this?

@ThomasMannKenbun I think it would work if jupyter was run on the same container as spark master and starting pyspark with the --master local[*] option. Tested on Iceberg image. The app isn't listed on localhost:7077.


[InterfaceGen] Print bodies of inlinable functions in textual interfaces

This patch adds support to the AST printer for printing the bodies of inlinable functions.
Still left to do:

[x] Add more tests
[x] Get tests passing and ensure NFC to other bits of the printer
[x] Add serialization/deserialization of body text to binary module format
[x] Add parser support for mixed inlinable/non-inlinable accessors

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - d61216717d75a2b8d473b2fa70a8484b0284915a

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - d61216717d75a2b8d473b2fa70a8484b0284915a

Build failed
Swift Test OS X Platform
Git Sha - d61216717d75a2b8d473b2fa70a8484b0284915a

@harlanhaskins Why do we need to serialize the inlinable body text? I thought it was for swiftinterface files only

@slavapestov We need to be able to compile an interface file into a binary swiftmodule, which means recompiling inlinable bodies

Ah, that's not really it: it's when you do a non-whole-module build but still want to generate a textual interface. In that case you pass through the partial module stage.

@swift-ci please test

Build failed
Swift Test OS X Platform
Git Sha - 0c448652b4ee52f0c173675c2eb3478d87f0de8e

Build failed
Swift Test Linux Platform
Git Sha - 0c448652b4ee52f0c173675c2eb3478d87f0de8e


add new plugin eleventy-plugin-atlasicons

@pdehaan when you have a time, would you please review?


Add instructions to readme for AWS console setup of bastion host

Also add blurb to explain how the linux AMI used in the CLI approach is not supported in each region.

Updated readme


Check that TODO and FIXME tags are within comment.

Check that a TODO and FIXME tag is within a comment before including in the tree.

Unless this becomes a requested enhancement, I will not spend time on this issue.


⚠ Samsung has degraded performance

In 90cc4b9, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 1954 ms

Resolved: Samsung performance has improved in 42a6f2c.


Non-Essential GitHub entities?

Question
Are there any entities that we can disable from collection on GitHub for the upcoming demo to make it run faster?

We will need to see the final demo dashboards sql queries first to see which ones we need.

I think we need it all at this time. Users will have to use multiple tokens to make it faster.


(maint) Merge 5.5.x to 6.0.x

CLA signed by all contributors.

Could you squash 96b71fc27ad35d2ab3ebdc2ab6249ee32731ec4f into the merge commit? As it is now rspec fails on the merge commit which can make git bisect later difficult later.


chore: Add pattern check for group in nodeClassRef

Fixes #N/A
Description
If a user creates a nodePool with nodeClassRef as shown below, nodePool fails to watch the nodeClass and nodePool remains stuck in not ready condition if EC2NodeClass is created before nodePool.
nodeClassRef:
  group: karpenter.k8s.aws/v1
  kind: EC2NodeClass
  name: default

A check has been added for the field group such that it will not allow slash.
How was this change tested?
Added test for the change. Tested on local cluster.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>9
Details

0 of 0   changed or added relevant lines in 0 files are covered.
8 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.07%) to 77.872%

Files with Coverage Reduction
New Missed Lines
%

pkg/test/expectations/expectations.go
2
94.64%

pkg/controllers/node/termination/controller.go
2
66.47%

pkg/controllers/disruption/consolidation.go
4
87.25%

Totals

Change from base Build<PHONE_NUMBER>4:
-0.07%

Covered Lines:
8893

Relevant Lines:
11420

💛  - Coveralls


Figure out how to properly set the 'namespace' on a metric

We define our deploy_timestamp metric as:
GaugeMetricFamily('deploy_timestamp', 'Deployment timestamp', labels=['namespace', 'app', 'image_sha'])

However, our metric data actually ends up looking like:
deploy_timestamp{app="basic-nginx-10",endpoint="http",exported_namespace="basic-nginx-prod",image_sha="sha256:a344eb5673e7e3faf871e7375762f1869c823c957aa126ce8f52033395f2b358",instance="<IP_ADDRESS>:8080",job="deploytime-exporter",namespace="pelorus-master",pod="deploytime-exporter-2-sz2bq",service="deploytime-exporter"}

Notice that the namespace field gets moved to a field called exported_namespace, and the actual namespace label gets set based on the namespace that the exporter pod is running in. Why, and how do we fix it?
I've noticed that some of the metrics that get exported by kube-state-metrics are able to set the namespace properly. Perhaps there are some clues there.
kube_namespace_created

kube_namespace_created{endpoint="https-main",federated_job="federated-prometheus-openshift-monitoring",instance="<IP_ADDRESS>:8443",job="kube-state-metrics",namespace="abryson-redhat-sandbox",pod="kube-state-metrics-66dfc9f94f-dclj6",prometheus="openshift-monitoring/k8s",prometheus_replica="prometheus-k8s-0",service="kube-state-metrics"}

It appears that the prometheus operator will rename any labels that conflict with the labels it adds through a ServiceMonitor. Luckily, we can configure HonorLabels to override this replacement.

Propose to rename those to pelorus_namespace so there is less chance of the collision.


[Bug]: file upload fails, sends only a small faction of the file

Is there an existing issue for this?

[X] I have searched the existing issues

Description
customer reported issue, when uploading a file to REST API - "For whatever reason, blob binary files are not injected into workstreams like base64 blobs are".
This works with a workaround, i.e., using a jsobject to convert the blob to binary before sending it for upload.
internal conversation for reference
Steps To Reproduce
similar to #20315 , but I haven't been able to reproduce the issue yet - will update here with more information.
Public Sample App
No response
Environment
Production
Severity
High (Blocker to building or releasing)
Issue video log
No response
Version
Cloud

what's the timeframe on this bug? We have a workaround but it's high memory intensive to workaround.  thx!

@ericambo thank you for the nudge! We will need some time to solve this bug - will update the timelines here when we've prioritised it. Right now, the team is working on fixing a few higher priority bugs.

what is the latest? is there a recommended way to reliably upload files to S3?

Hey, just fyi I was able to circumvent this using custom JS. You basically have to handle the binary string when it’s a blob (over 5mb) and then convert it from the blob to a local string object in js. Then upload using the string.

@ericambo Thanks a lot for sharing this issue with us. I am trying to investigate the issue and the cause of it. So far I have tried using BaseRow's file upload API by passing in binary data more than 5mb and it works for me without any issues and file is also uploaded successfully.
What I want to understand from you is, how your file upload API is configured. In your API, how are we sending the file data in the body and what headers we are setting. It would be great if you can share this information, it will help in accelerating the debugging process.
Furthermore, it will also be a great idea to sync over a quick call to understand the state of application and the APIs. I am sharing calendly link here

Hi @ericambo Please reschedule another meeting at this link

Had a call with @ericambo yesterday to discuss this issue:

We could see that when sending file data in binary format with more than 5mb was only uploading 2b to the Azure blob storage. We changed the file picker format to base64, set body to binary and removed manually added content-type header and tried this method of file upload, With this approach, we were able to upload files successfully on Azure blob storage and it uploaded the full file. But this method has one caveat, it only allowed us to upload upto 14mb and when uploading file more than that we saw an issue of Input byte ending with 23231. This might already be resolved with, will need to confirm once on latest version PR
Another issue mentioned by Eric for S3, not allowing large file uploads has been fixed with this PR

Action items:

On latest version, check the file upload (>5mb) with binary file format
On latest version, check the file upload (>14mb) with base64 format

@ericambo can you please confirm the self hosted version that you have?

@sneha122 we are self hosted on 1.34.1.  Are you saying if we upgrade to latest that we will 100% be able to fix our issues?

@ericambo Let me confirm which version fixes all of the above issues and get back to you, so that you complete version upgrade in one go and all of your issues will be fixed

@ericambo Could you please try one thing? In your application, can you please change the query body from Filepicker1.files[0].data to Filepicker1.files[0], keeping the file picker format as binary? Please let me know if this uploads file correctly or not.
My hunch is that data property is not getting converted from blob url to binary but complete file data gets parsed and by passing files[0] instead of files[0].data, we ensure that blob url gets converted to file.

@ericambo New version v.144 fixes all of the following issues and you can upgrade to this version to get all of your issues fixed, it fixes:

Uploading larger files with REST API
File upload to S3 fails for certain files

Also the issue with Binary file type can be fixed by passing data as FilePicker.files[0] instead of FilePicker[0].files[0].data.
I am closing this issue as all of the file upload issues are fixed. Please feel free to reopen if you think any of the issues are not fixed


this is a new featur

hi guys we have a new requirement form customer for developing admin  panel

#hi

Hi

hi

hi


Introduce healthchecker extension

Signed-off-by: Nikolay Maslarski<EMAIL_ADDRESS>

Coverage remained the same at 87.212% when pulling 984b44fa0c9e200c90e50bccac4b7f78b69f724e on SAP:healthcheck into 4c0ba6d6f9b4e0f38ce444395c2c707ae4d8819b on cloudfoundry-incubator:master.

Coverage remained the same at 87.212% when pulling 984b44fa0c9e200c90e50bccac4b7f78b69f724e on SAP:healthcheck into 4c0ba6d6f9b4e0f38ce444395c2c707ae4d8819b on cloudfoundry-incubator:master.

Coverage remained the same at 87.212% when pulling cf3719cb8741003b7770ca8b47129541625153c8 on SAP:healthcheck into 4c0ba6d6f9b4e0f38ce444395c2c707ae4d8819b on cloudfoundry-incubator:master.


Interpolation doesn't work?

The camera movement is fine but the controller movement is still laggy. Can you please help?
I'm using the latest Unity LTS version (2018.4)

I'm also having such a laggy interpolation. I'm using Unity 2019.3.7f1.

Do both of you have the interpolation controller prefab in the scene?

Another script was cancelling the interpolation prefabs, it works now. Thanks

I am now using other type of FPS controller, but I will test your one again tomorrow, maybe it will be better without interpolation issues

@W4R-The-Hyper Could you tell us how you fixed the interpolation problem? Which script cancelled interpolating?

It's a script of mine, long story :p

A good note that now PlayerMovement is a child of InterpolationTransform and so you should not have both PlayerMovement and InterpolationTransform on the same transform like you did before

@Colanderp Hi! I've created new Unity project, downloaded master branch, imported FPS Movement Only.unitypackage, then moved Player prefab onto the scene, and camera is still jerky.
Here is an Unity project you can check it out:
https://cloud.warp.sh/index.php/s/Ld2MittgHB8mb26
I'm experiencing problems with my current FPS controller so will be more than happy to be able to use your one, as it works ok. The only issue is camera lagging.
Note that I'm using Unity 2019.3.7f1.
Thank you for your help!

Did you place the InterpolationController prefab in the scene too?

Holy moly. I thought the Player prefab already contained all required components. After adding InterpolationController component into the Player object it just starter working okay.
Would be nice to include such information inside README.md. Maybe it is included in tutorial video, but I wasn't so much patient to watch it fully.
Thank you for helping me!
Have a nice day!

There ya go! Each has a README that explains that you need the InterpolationController.prefab


Add missing a11y features, #11

This PR improves the accessibility of the tooltip and its target.

:tada: This PR is included in version 2.4.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


blink

For blinking of a cursor, we want a script to return true/false periodically.
Here is a possible implementation:
/// @param {real}	offTime			time during which false (default: 500 ms)
/// @param {real}	onTime			time during which true (default: same as offTime)
/// @param {real}	phaseShift		offset time (default: 0)
/// @param {real}	clock			which clock to use (default: current_time)
///
/// @description	For a cursor, be able to made it blink

function blink(offTime = 500, onTime = offTime, phaseShift = 0, clock = current_time)
{
	return (clock + phaseShift) mod (offTime + onTime) >= offTime;
}```

I'm not quite sure about using name blink, as it's so short people may use it for other variables. But I think the functionality might be a good addition. In that regard, it's similar to the wave suggestion.

Potential name suggestion: cycle_blink, which might also match cycle_around and cycle_between for waving functions, if those names are accepted.

I agree that the current name is too short/generic. Your suggestion cycle_blink is fine with me


Support for manual management of Commands coming out of Modules

Currently a Bootique app exposes in CLI all commands coming from all its included modules. It works fine in most cases, but has some limitations:

Only one command can be executed at a time. If multiple command flags were supplied, one of them is chosen semi-randomly. Combining multiple commands is not possible (e.g. UpdateCommand from bootique-liquibase followed by ServerCommand from bootique-jetty)
Fine-tuning your user-visible CLI is not possible (e.g. hiding some Module commands from CLI, using Module as a library, and providing your own commands).

We need a straightforward API for more advanced management of the command set. Set<Command> is injected into OptionsProvider and DefaultRunner. Perhaps we should introduce a second Set collection injected into those services, that is a result of filtering the original set with all module contributions. Then we can have a pluggable filtering service that produces the second set from the first. Then we can add some simple API to Bootique class to manage this, e.g. exclude all module-contributed commands, etc.
Also there should be a way to order and chain commands (setting CommandOutcome.shouldExit to false should help with chaining; what about ordering?)

Related: #19

closing. resolved per #19  , #23  and other smaller tasks.


cFE TIME unit tests break when different configuration options are used

When using the unmodified sample version of the {{{cfe_platform_cfg.h}}} file, the test cases defined in {{{time_UT.c}}} all pass.
But if the user makes any time-related modifications to the platform config file, many of the unit test cases break.  In particular, configuring a time server vs. time client or setting SRC_MET == TRUE, etc.
The test cases in "time_UT.c" need to include/accommodate other valid configuration options.

Imported from trac issue 78.  Created by jphickey on 2015-07-09T16:30:14, last modified: 2019-07-03T12:48:08

Trac comment by jphickey on 2017-10-20 11:58:34:
This ticket points to a deeper issue and first requires a clean up of the TIME subsystem in general.  Many of the configuration options trigger calls into functions that do not exist and are not defined in the respective API at all.  Functions like:
OS_SelectTone()
OS_GetLocalMET()
Since these options have been broken in previous versions and this isn't a //new// issue in 6.6, I recommend deferring this to cfe-next.

Trac comment by jphickey on 2017-10-20 11:59:48:
NOTE: for example ticket #123 is one that should be a dependency of this one.

Trac comment by acudmore on 2019-04-03 11:29:42:
As part of tickets 47, 78, and 92, I would like to do a survey of missions for TIME subsystem options.
Depending on the results of the survey, we should:

Implement stub PSP functions to implement any necessary functions (PSP equivalents of OS_Get/SetLocalMET , OS_SelectTone) and update unit tests etc.
Deprecate the un-needed options/features and prepare to remove them in a more comprehensive time subsystem simplification.

Trac comment by jhageman on 2019-04-01 09:41:02:
CCB 3/27/2019 - discussed removing unimplemented configuration options.  Assigning to Alan for review.
Edit - fixed CCB date typo

Trac comment by jphickey on 2019-04-03 11:40:01:
For what its worth, the "TimeBase" feature that was added and implemented in the "ng" OSAL versions may make these functions and special features in CFE_TIME unnecessary.
For the most part, CFE TIME just needs a properly synchronized 1Hz PPS signal from the PSP. Although I've never seen an actual implementation of e.g. OS_SelectTone my guess is that this would ultimately select the synchronization source for the 1Hz signal.  The "MET" is probably analogous to the free run timer in the TimeBase object, which is already implemented (a monotonically incrementing tick counter that is reset when the system boots).
With the TimeBase layer the PSP can implement all this logic on its own and it is totally abstract to applications.  CFE_TIME just gets a 1Hz callback based on the timebase, and the PSP can decide what to actually synchronize that to.
The main question to evaluate is whether any of the CFE_TIME ground commands or telemetry would change or report this status, and if so, we might need some form of PSP call to get the status in an abstract fashion.

Trac comment by jhageman on 2019-07-03 12:48:08:
Moved unfinished 6.6.1 issues to next minor release

Related to #302

@skliper bump
We ran into this problem while building the unit tests for our WFIRST FSW. Could I expect this in an upcoming release?

Refactoring time services is a long term goal and would include fixing the unit tests, so I'd expect it will happen at some point.  If it's a project request that needs to be prioritized you could initiate with whoever is handling project requests at this point.  Maybe @jwilmot or @ejtimmon ?

Settings described in 2072 also fail to build, which is a setup used when there's an app providing the ExternalTime and ExternalTone.


How to set this  in  neither -- kubeconfig nor --master was specified

-bash-4.2$ kubectl logs -f chaos-monitor-6c7694d5b4-gmqr4 -n litmus
W0518 12:55:01.419135       1 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0518 12:55:01.419443       1 main.go:65] Beginning to serve on port :8080

cc: @rahulchheda can you look into this please ?

Hey @ravikumar2000 , this is just a precautionary message, when running inside Kubernetes Cluster, it will auto determine the Kubeconfig. Or else it has to be provided with --kubeconfig flag, while running as binary.
As I can see you are running it on a cluster, so please ignore this.

@rahulchheda can we log this if inClusterConfig also doesn't work? Else, may give out conflicting signals.

We are unable to see any metircs in grafana after we run pod-delete litmuschaos experiment.
We are looking for cluster_overall_experiments_count in prometheus

How to add this to kubeconfig
Running Chaos Exporter on the local Machine
Run the exporter container (litmuschaos/chaos-exporter:ci) on host network. It is necessary to mount the kubeconfig & override entrypoint w/ ./exporter -kubeconfig
Execute curl <IP_ADDRESS>:8080/metrics to view metrics

We are not seeing any data in prometheus.
Appreciate your quick help on this

We are getting like this for chaos monitor
-bash-4.2$ kubectl logs -f chaos-monitor-6c7694d5b4-gmqr4 -n litmus
W0518 12:55:01.419135       1 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0518 12:55:01.419443       1 main.go:65] Beginning to serve on port :8080
I0518 12:55:01.422305       1 controller.go:33] Started creating Metrics
I0518 12:55:01.422325       1 controller.go:39] Registering Fixed Metrics
error: unexpected EOF

@ravikumar2000 , could you try to head out to service created for chaos-exporter in your cluster? And try to curl :8080/metrics. I think you might have tried that already, just check for the service account you are providing.

As for binary, .exporter --kubeconfig=/path/to/your/config is to be provided.

Can you give me the steps on installing chaosexporter.
I just added kubectl apply -f chaosexporter.yaml
https://github.com/litmuschaos/chaos-exporter/tree/master/deploy
PFA metrices that we are getting

@ravikumar2000 some questions:
a) Are you able to see the pod-delete experiment getting executed successfully? i.e., are the target pods being killed ?
b) If (a) is true, can you check is you do have the ChaosResult resource in the namespace you are running chaos from?  It should ideally give you a verdict of "Pass"
(c) If both the above are true, check if the .spec.monitoring  field in the ChaosEngine is set to "true".
These are the pre-requisites for you to see some metrics via the exporter. Also I would recommend running the exporter via deployment rather than as binary

Yes i can see verdict as Pass but timestamp is not changing.
Where to set .spec.monitoring as true ?
-bash-4.2$ kubectl describe chaosresult litmusnginx-pod-delete
Name:         litmusnginx-pod-delete
Namespace:    default
Labels:       chaosUID=ece152bc-9757-11ea-b0fc-fa163eb35c46
type=ChaosResult
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"litmuschaos.io/v1alpha1","kind":"ChaosResult","metadata":{"annotations":{},"labels":{"chaosUID":"ece152bc-9757-11ea-b0fc-fa...
API Version:  litmuschaos.io/v1alpha1
Kind:         ChaosResult
Metadata:
Creation Timestamp:  2020-05-16T09:31:56Z
Generation:          1
Resource Version:    1164679
Self Link:           /apis/litmuschaos.io/v1alpha1/namespaces/default/chaosresults/litmusnginx-pod-delete
UID:                 15e0ebf0-9758-11ea-b0fc-fa163eb35c46
Spec:
Engine:      litmusnginx
Experiment:  pod-delete
Status:
Experimentstatus:
Fail Step:  N/A
Phase:      Completed
Verdict:    Pass
Events:

Thanks @ravikumar2000 for confirming ! The timestamp of the result resource wont change per se (as the verdicts are patch operations - you can observe changes to the resourceVersion though). The monitoring field needs to be enabled here (I am presuming you picked the chaosengine from here:
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: nginx-chaos
  namespace: default
spec:
  appinfo:
    appns: 'default'
    applabel: 'app=nginx'
    appkind: 'deployment'
  # It can be true/false
  annotationCheck: 'true'
  # It can be active/stop
  engineState: 'active'
  #ex. values: ns1:name=percona,ns2:run=nginx
  auxiliaryAppInfo: ''
  chaosServiceAccount: pod-delete-sa
  monitoring: true
  # It can be delete/retain
  jobCleanUpPolicy: 'delete'
  experiments:
    - name: pod-delete
      spec:
        components:
          env:
            # set chaos duration (in sec) as desired
            - name: TOTAL_CHAOS_DURATION
              value: '30'

            # set chaos interval (in sec) as desired
            - name: CHAOS_INTERVAL
              value: '10'

            # pod failures without '--force' & default terminationGracePeriodSeconds
            - name: FORCE
              value: 'false'

Yes
-bash-4.2$ cat pod_delete_engine.yaml
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: litmusnginx
namespace: default
spec:
appinfo:
appns: 'default'
applabel: 'app=litmusnginx'
appkind: 'deployment'
It can be true/false
annotationCheck: 'true'
It can be active/stop
engineState: 'active'
#ex. values: ns1:name=percona,ns2:run=nginx
auxiliaryAppInfo: ''
chaosServiceAccount: pod-delete-sa
monitoring: false
It can be delete/retain
jobCleanUpPolicy: 'delete'
experiments:
- name: pod-delete
spec:
components:
env:
# set chaos duration (in sec) as desired
- name: TOTAL_CHAOS_DURATION
value: '30'
        # set chaos interval (in sec) as desired
        - name: CHAOS_INTERVAL
          value: '10'

        # pod failures without '--force' & default terminationGracePeriodSeconds
        - name: FORCE
          value: 'true'

Let me update  Monitoring as 'true" and try it again

Thanks @ravikumar2000  (note: I have edited your logs a bit to format it - I hope you will not mind)  Can you also suggest what is the nature of the metrics you are looking for? i.e., the kind of visualization around chaos. There is some work ongoing here in this repo: https://github.com/litmuschaos/chaos-observability that can help with that (for ex: there is a way to convert Kubernetes events generated during the experiment into metrics which may be pretty useful)

Hi Karthik,
we are looking for  pod level & node level resources in chaos experiments.
Thanks for sharing this

Yes
-bash-4.2$ cat pod_delete_engine.yaml

apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: litmusnginx
  namespace: default
spec:
  appinfo:
    appns: 'default'
    applabel: 'app=litmusnginx'
    appkind: 'deployment'
  # It can be true/false
  annotationCheck: 'true'
  # It can be active/stop
  engineState: 'active'
  #ex. values: ns1:name=percona,ns2:run=nginx
  auxiliaryAppInfo: ''
  chaosServiceAccount: pod-delete-sa
  monitoring: false
  # It can be delete/retain
  jobCleanUpPolicy: 'delete'
  experiments:
    - name: pod-delete
      spec:
        components:
          env:
            # set chaos duration (in sec) as desired
            - name: TOTAL_CHAOS_DURATION
              value: '30'

            # set chaos interval (in sec) as desired
            - name: CHAOS_INTERVAL
              value: '10'

            # pod failures without '--force' & default terminationGracePeriodSeconds
            - name: FORCE
              value: 'true'

Let me update Monitoring as 'true" and try it again

I executed with pod delete
-bash-4.2$ kubectl describe chaosresult litmusnginx-pod-delete
Name:         litmusnginx-pod-delete
Namespace:    default
Labels:       chaosUID=ece152bc-9757-11ea-b0fc-fa163eb35c46
type=ChaosResult
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"litmuschaos.io/v1alpha1","kind":"ChaosResult","metadata":{"annotations":{},"labels":{"chaosUID":"ece152bc-9757-11ea-b0fc-fa...
  Using the inClusterConfig.  This might not work.
I0519 08:38:34.951466       1 main.go:65] Beginning to serve on port :8080
I0519 08:38:34.955573       1 controller.go:33] Started creating Metrics
I0519 08:38:34.955596       1 controller.go:39] Registering Fixed Metrics
panic: runtime error: index out of range [1] with length 1
goroutine 12 [running]:
github.com/litmuschaos/chaos-exporter/controller.getExperimentMetricsFromEngine(0xc00038bab8, 0xc000160850, 0x0, 0x0, 0x0)
/home/travis/gopath/src/github.com/litmuschaos/chaos-exporter/controller/scrape.go:92 +0x379
github.com/litmuschaos/chaos-exporter/controller.GetLitmusChaosMetrics(0xc000208000, 0xc000000000, 0x139a2a2)
/home/travis/gopath/src/github.com/litmuschaos/chaos-exporter/controller/scrape.go:48 +0x17e
github.com/litmuschaos/chaos-exporter/controller.Exporter(0xc0001cd0e0)
/home/travis/gopath/src/github.com/litmuschaos/chaos-exporter/controller/controller.go:43 +0xef
created by main.main
/home/travis/gopath/src/github.com/litmuschaos/chaos-exporter/cmd/exporter/main.go:62 +0x67

I am trying this on a local cluster & will get back in a bit. Please hold on & sorry for the issues !

No problem

@ravikumar2000 could you please share the output of

kubectl describe chaosengine litmusnginx

What is the version of Litmus are you using ? i.e., Operator & Chaos-Charts

-bash-4.2$ kubectl describe chaosengine litmusnginx
Name:         litmusnginx
Namespace:    default
Labels:
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"litmuschaos.io/v1alpha1","kind":"ChaosEngine","metadata":{"annotations":{},"name":"litmusnginx","namespace":"default"},"spe...
API Version:  litmuschaos.io/v1alpha1
Kind:         ChaosEngine
Metadata:
Creation Timestamp:  2020-05-16T09:30:48Z
Finalizers:
chaosengine.litmuschaos.io/finalizer
Generation:        1
Resource Version:  1176040
Self Link:         /apis/litmuschaos.io/v1alpha1/namespaces/default/chaosengines/litmusnginx
UID:               ece152bc-9757-11ea-b0fc-fa163eb35c46
Spec:
Annotation Check:  true
Appinfo:
Appkind:              deployment
Applabel:             app=litmusnginx
Appns:                default
Chaos Service Account:  pod-delete-sa
Components:
Runner:
Image:     litmuschaos/chaos-runner:1.3.0
Engine State:  stop
Experiments:
Name:  pod-delete
Spec:
Components:
Env:
Name:         TOTAL_CHAOS_DURATION
Value:        30
Name:         CHAOS_INTERVAL
Value:        10
Name:         FORCE
Value:        true
Rank:             0
Job Clean Up Policy:  delete
Monitoring:           true
Status:
Engine Status:  completed
Experiments:
Last Update Time:  2020-05-19T09:07:09Z
Name:              pod-delete-j8lgvl
Status:            Running
Verdict:           Awaited
Last Update Time:  2020-05-19T09:08:40Z
Name:              pod-delete-78pvmw
Status:            Execution Successful
Verdict:           Pass
Events:
Type    Reason                     Age                   From                     Message

Normal  ExperimentDependencyCheck  48m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        48m                   litmusnginx-runner       Experiment Job 'pod-delete-kszcgk' created for Chaos Experiment 'pod-delete'
Normal  ExperimentDependencyCheck  47m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        47m                   litmusnginx-runner       Experiment Job 'pod-delete-bhjh2r' created for Chaos Experiment 'pod-delete'
Normal  ExperimentJobCleanUp       46m                   litmusnginx-runner       Experiment Job 'pod-delete-bhjh2r' is deleted
Normal  ExperimentDependencyCheck  34m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        34m                   litmusnginx-runner       Experiment Job 'pod-delete-1x8tt4' created for Chaos Experiment 'pod-delete'
Normal  ExperimentDependencyCheck  33m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        33m                   litmusnginx-runner       Experiment Job 'pod-delete-lo08a6' created for Chaos Experiment 'pod-delete'
Normal  ExperimentDependencyCheck  32m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        32m                   litmusnginx-runner       Experiment Job 'pod-delete-k9hhox' created for Chaos Experiment 'pod-delete'
Normal  ExperimentDependencyCheck  31m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        31m                   litmusnginx-runner       Experiment Job 'pod-delete-mxylmq' created for Chaos Experiment 'pod-delete'
Normal  ExperimentDependencyCheck  30m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        30m                   litmusnginx-runner       Experiment Job 'pod-delete-wogugu' created for Chaos Experiment 'pod-delete'
Normal  ExperimentDependencyCheck  29m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        29m                   litmusnginx-runner       Experiment Job 'pod-delete-ldojmj' created for Chaos Experiment 'pod-delete'
Normal  ExperimentJobCleanUp       28m                   litmusnginx-runner       Experiment Job 'pod-delete-ldojmj' is deleted
Normal  ExperimentDependencyCheck  22m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        22m                   litmusnginx-runner       Experiment Job 'pod-delete-cbbia5' created for Chaos Experiment 'pod-delete'
Normal  ExperimentJobCleanUp       21m                   litmusnginx-runner       Experiment Job 'pod-delete-cbbia5' is deleted
Normal  RestartInProgress          19m (x24 over 2d23h)  chaos-operator           Chaos Engine restarted, will re-create all chaos-resources
Normal  ExperimentDependencyCheck  19m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        19m                   litmusnginx-runner       Experiment Job 'pod-delete-j8lgvl' created for Chaos Experiment 'pod-delete'
Normal  ExperimentDependencyCheck  18m                   litmusnginx-runner       Experiment resources validated for Chaos Experiment: 'pod-delete'
Normal  ExperimentJobCreate        18m                   litmusnginx-runner       Experiment Job 'pod-delete-78pvmw' created for Chaos Experiment 'pod-delete'
Normal  PreChaosCheck              18m                   pod-delete-78pvmw-mnchf  AUT is Running successfully
Normal  ChaosInject                17m                   pod-delete-78pvmw-mnchf  Injecting pod-delete chaos on litmusnginx-5869bbd965-ghdpk pod
Normal  PostChaosCheck             17m                   pod-delete-78pvmw-mnchf  AUT is Running successfully
Normal  Summary                    17m                   pod-delete-78pvmw-mnchf  pod-delete Experiment Passed!
Normal  ExperimentJobCleanUp       17m                   litmusnginx-runner       Experiment Job 'pod-delete-78pvmw' is deleted
Normal  ChaosEngineCompleted       17m (x23 over 2d23h)  chaos-operator           Chaos Engine completed, will delete or retain the resources according to jobCleanUpPolicy

Can we get a grafana dashboard json file as we are observing only 2 passed & 3 total experiments we ran it only 2 pod-delete experiment.

Yes sure! We will be sharing it shortly cc: @rajdas98 (so, are you able to see metrics now or still see the crash? )

yes i am able to see the metrics
but showing wrong data in grafana as attached snapshot
tootal runs=2
total passed experiments=3
nothing was update in running experiments in grafana

@ravikumar2000 Can you validates these wrong metrics in prometheus metric endpoint also?

yes it should be simillar can you share the json file for grafana dashbaord

I can see only 3 chaos experiments with chaosresults
getting unexpected EOF
-bash-4.2$ kubectl get chaosengine
NAME                        AGE
litmusnginx                 3d
litmusnginx-network-chaos   3d
nginx-network-chaos         3d
-bash-4.2$ kubectl logs -f chaos-monitor-6c7694d5b4-gmqr4 -n litmus      W0519 10:16:23.967197       1 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0519 10:16:23.967549       1 main.go:65] Beginning to serve on port :8080
I0519 10:16:23.970524       1 controller.go:33] Started creating Metrics
I0519 10:16:23.970545       1 controller.go:39] Registering Fixed Metrics
error: unexpected EOF
-bash-4.2$
Please let me know the issues
I have used below nginx file
-bash-4.2$ cat nginx.yaml
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: litmusnginx
spec:
selector:
matchLabels:
app: litmusnginx
replicas: 2 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: litmusnginx
spec:
containers:
- name: litmusnginx
image: nginx:1.14.2
ports:
- containerPort: 80
Let me know if any update is required

If we delete a pod it will create 2 passed experiments
1 for pod delete  & other for network-chaos.
Is it expected behaviour?

If we delete a pod it will create 2 passed experiments
1 for pod delete & other for network-chaos.
Is it expected behaviour?

No it isn't, do you have both experiments listed in the engine?

Hey @ravikumar2000 was wondering, would you be interested in a quick zoom/hangouts call to discuss this issue?

Please share meeting invite. I will join the meeting
Regards,
Ravi
On Tue, 19 May, 2020, 9:38 PM Karthik Satchitanand, <
<EMAIL_ADDRESS>wrote:

Hey @ravikumar2000 https://github.com/ravikumar2000 was wondering,
would you be interested in a quick zoom/hangouts call to discuss this issue?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/litmuschaos/chaos-exporter/issues/64#issuecomment-630921475,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACDBRQ6JP7X5EGC2EBOJPQTRSKVJFANCNFSM4NEYDJMQ
.

Please share meeting invite. I will join the meeting Regards, Ravi
…
On Tue, 19 May, 2020, 9:38 PM Karthik Satchitanand, < @.***> wrote: Hey @ravikumar2000 https://github.com/ravikumar2000 was wondering, would you be interested in a quick zoom/hangouts call to discuss this issue? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#64 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDBRQ6JP7X5EGC2EBOJPQTRSKVJFANCNFSM4NEYDJMQ .

https://zoom.us/j/91358162694 - this is the community sync up link. I have just joined!

Ok let me join
On Tue, 19 May, 2020, 9:51 PM Karthik Satchitanand, <
<EMAIL_ADDRESS>wrote:

Please share meeting invite. I will join the meeting Regards, Ravi
… <#m_6308126105612596520_>
On Tue, 19 May, 2020, 9:38 PM Karthik Satchitanand, < @.***> wrote: Hey
@ravikumar2000 https://github.com/ravikumar2000
https://github.com/ravikumar2000 was wondering, would you be interested
in a quick zoom/hangouts call to discuss this issue? — You are receiving
this because you were mentioned. Reply to this email directly, view it on
GitHub <#64 (comment)
https://github.com/litmuschaos/chaos-exporter/issues/64#issuecomment-630921475>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACDBRQ6JP7X5EGC2EBOJPQTRSKVJFANCNFSM4NEYDJMQ
.
https://zoom.us/j/91358162694 - this is the community sync up link. I
have just joined!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/litmuschaos/chaos-exporter/issues/64#issuecomment-630928245,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACDBRQ7BC7TDQG3LU2D2VF3RSKWXFANCNFSM4NEYDJMQ
.

Ok, so after a discussion with Ravi it turns out the following things needed a bit of change:

ChaosEngine Spec (monitoring, annotationCheck)
PromQL queries on the Grafana dashboard that @ravikumar2000 had set up
Some Panel properties which were showing a cumulative count over actual values

Some findings @rahulchheda you may be interested in is the cluster_overall_running_experiment metrics -- this was showing up as a bit dodgy at times. The others are accurate.
Thanks for your time @ravikumar2000 & @rajdas98 !

@ravikumar2000 please feel free to close this issue if you are OK!

Hi Karthik,
Yes you can close this ticket.
Can we integrate litmushchaos & jenkins ?  for  running chaos experiments with Jenkins?
Please share if you have any documents on it.
Regards,
Ravi

Shifted the question on litmuschaos/litmus
Closing this issue based on recent comments!


java.lang.IllegalStateException: Key END found in ......

I am running RNA-seq pipeline on 12 rat samples. It seems GATK failed in the variant calling. The error information is as below:

03:22:50.041 INFO  GenomicsDBImport - Initializing engine
03:22:50.842 INFO  IntervalArgumentCollection - Processing 282763074 bp from intervals
03:22:50.845 INFO  GenomicsDBImport - Done initializing engine
Created workspace /app/datamart/jieyi_bio/rat_star2/work/joint/gatk-haplotype-joint/1/1/1-1_0_282763074_genomicsdb
03:22:51.698 INFO  GenomicsDBImport - Vid Map JSON file will be written to 1-1_0_282763074_genomicsdb/vidmap.json
03:22:51.698 INFO  GenomicsDBImport - Callset Map JSON file will be written to 1-1_0_282763074_genomicsdb/callset.json
03:22:51.698 INFO  GenomicsDBImport - Complete VCF Header will be written to 1-1_0_282763074_genomicsdb/vcfheader.vcf
03:22:51.698 INFO  GenomicsDBImport - Importing to array - 1-1_0_282763074_genomicsdb/genomicsdb_array
03:22:51.710 INFO  ProgressMeter - Starting traversal
03:22:51.711 INFO  ProgressMeter -        Current Locus  Elapsed Minutes     Batches Processed   Batches/Minute
03:22:51.711 INFO  GenomicsDBImport - Starting batch input file preload
03:22:51.831 INFO  GenomicsDBImport - Finished batch preload
03:22:51.832 INFO  GenomicsDBImport - Importing batch 1 with 6 samples
03:22:52.120 INFO  GenomicsDBImport - Shutting down engine
[April 19, 2018 3:22:52 AM CST] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 0.54 minutes.
Runtime.totalMemory()=506986496
java.lang.IllegalStateException: Key END found in VariantContext field INFO at 1:401700 but this key isn't defined in the VCFHeader.  We require all VCFs to have complete VCF headers by default.
at htsjdk.variant.vcf.VCFEncoder.fieldIsMissingFromHeaderError(VCFEncoder.java:173)
at htsjdk.variant.vcf.VCFEncoder.encode(VCFEncoder.java:112)
at htsjdk.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:224)
at com.intel.genomicsdb.GenomicsDBImporter.add(GenomicsDBImporter.java:1368)
at com.intel.genomicsdb.GenomicsDBImporter.importBatch(GenomicsDBImporter.java:1418)
at org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport.traverse(GenomicsDBImport.java:509)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:893)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)
' returned non-zero exit status 3

Thanks, sorry about that. Could you pass on the VCF file that has the issue so I can take a look at it?

Hi guys - I'm running into this issue now using the RNA-Seq pipeline (see related discussion on #2390).
Here's what my setup looks like:
details:
- algorithm:
    adapters:
    - truseq
    - illumina
    aligner: star
    exclude_regions:
    - polyx
    - lcr
    - highdepth
    - altcontigs
    expression_caller:
    - salmon
    mark_duplicates: true
    strandedness: firststrand
    transcriptome_align: false
    trim_reads: atropos
    variantcaller:
    - gatk-haplotype
  analysis: RNA-seq
  description: MOLM13_rep1
  files:
  - /projects/karsanlab/rdocking/KARSANBIO-1254_pipeline/KARSANBIO-1390_rna_seq_runs/data/A34048_1.fq.gz
  - /projects/karsanlab/rdocking/KARSANBIO-1254_pipeline/KARSANBIO-1390_rna_seq_runs/data/A34048_2.fq.gz
  genome_build: hg19
  metadata:
    batch: MOLM13_rep1_batch

(I don't usually run trimming, but I was trying it to see if it helped with the memory issues from the other ticket). In this case, the run passes through the first-stage of GATK variant calling, then crashes like this:
[May 12, 2018 1:37:16 AM PDT] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 0.02 minutes.
Runtime.totalMemory()=811073536
java.lang.IllegalStateException: Key END found in VariantContext field INFO at chr4:48991 but this key isn't defined in the VCFHeader.  We require all VCFs to have complete VCF headers by default.
        at htsjdk.variant.vcf.VCFEncoder.fieldIsMissingFromHeaderError(VCFEncoder.java:173)
        at htsjdk.variant.vcf.VCFEncoder.encode(VCFEncoder.java:112)
        at htsjdk.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:224)
        at com.intel.genomicsdb.GenomicsDBImporter.add(GenomicsDBImporter.java:1368)
        at com.intel.genomicsdb.GenomicsDBImporter.importBatch(GenomicsDBImporter.java:1418)
        at org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport.traverse(GenomicsDBImport.java:509)
        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:893)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
[

The relevant commands run were:
SplitN:
[2018-05-11T23:00Z] eccfx01n1.hpc.bcgsc.ca: unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && gatk-launch --java-options '-Xms727m -Xmx3272m -XX:+UseSerialGC -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpC8Uo9D' SplitNCigarReads -R /projects/rdocking_prj/software/bcbio-nextgen/data/genomes/Hsapiens/hg19/seq/hg19.fa -I /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/align/MOLM13_rep3/MOLM13_rep3-dedup.bam --output /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpJw9pVK/MOLM13_rep3-dedup.splitN.bam

HaplotypeCaller:
[2018-05-12T02:16Z] eccfx01n2: export SPARK_USER=rdocking && unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && gatk-launch --java-options '-Xms800m -Xmx94349m -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmp617YSx' HaplotypeCallerSpark --reference /projects/rdocking_prj/software/bcbio-nextgen/data/genomes/Hsapiens/hg19/ucsc/hg19.2bit --annotation MappingQualityRankSumTest --annotation MappingQualityZero --annotation QualByDepth --annotation ReadPosRankSumTest --annotation RMSMappingQuality --annotation BaseQualityRankSumTest --annotation FisherStrand --annotation MappingQuality --annotation DepthPerAlleleBySample --annotation Coverage -I /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/align/MOLM13_rep2/MOLM13_rep2-dedup.splitN.bam -L /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bedprep/ref-transcripts-rnaseq_clean-glimit-nolcr-nopolyx.bed --interval-set-rule INTERSECTION --spark-master local[32] --conf spark.local.dir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmp617YSx --annotation ClippingRankSumTest --annotation DepthPerSampleHC --emit-ref-confidence GVCF -GQB 10 -GQB 20 -GQB 30 -GQB 40 -GQB 60 -GQB 80 --output /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmp617YSx/MOLM13_rep2-gatk-haplotype.vcf

GenomicsDBImport:
[2018-05-12T08:43Z] eccfx01n3: unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && gatk-launch --java-options '-Xms800m -Xmx94349m -XX:+UseSerialGC -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/joint/gatk-haplotype-joint/MOLM13_rep1_batch/chrM/bcbiotx/tmpbFjdnm' GenomicsDBImport --reader-threads 32 --genomicsdb-workspace-path MOLM13_rep1_batch-chrM_0_16571_genomicsdb -L chrM:1-16571 --variant /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/MOLM13_rep1-gatk-haplotype.vcf.gz

GenotypeGVCFs:
[2018-05-12T08:43Z] eccfx01n3: unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && gatk-launch --java-options '-Xms800m -Xmx94349m -XX:+UseSerialGC -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpjJJBx4' GenotypeGVCFs --variant gendb:///projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/joint/gatk-haplotype-joint/MOLM13_rep1_batch/chrM/MOLM13_rep1_batch-chrM_0_16571_genomicsdb -R /projects/rdocking_prj/software/bcbio-nextgen/data/genomes/Hsapiens/hg19/seq/hg19.fa --output /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpoTr60e/MOLM13_rep1_batch-chrM_0_16571.vcf.gz -L chrM:1-16571 --use-new-qual-calculator

The actual command that errored out was this one:
[0:apply]: CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && gatk-launch --java-options '-Xms800m -Xmx94349m -XX:+UseSerialGC -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/joint/gatk-haplotype-joint/MOLM13_rep3_batch/chr4/bcbiotx/tmp_GnLP6' GenomicsDBImport --reader-threads 32 --genomicsdb-workspace-path MOLM13_rep3_batch-chr4_0_191154276_genomicsdb -L chr4:1-191154276 --variant /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/MOLM13_rep3-gatk-haplotype.vcf.gz
Using GATK jar /projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/share/gatk4-<IP_ADDRESS>-0/gatk-package-<IP_ADDRESS>-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xms800m -Xmx94349m -XX:+UseSerialGC -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/joint/gatk-haplotype-joint/MOLM13_rep3_batch/chr4/bcbiotx/tmp_GnLP6 -jar /projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/share/gatk4-<IP_ADDRESS>-0/gatk-package-<IP_ADDRESS>-local.jar GenomicsDBImport --reader-threads 32 --genomicsdb-workspace-path MOLM13_rep3_batch-chr4_0_191154276_genomicsdb -L chr4:1-191154276 --variant /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/MOLM13_rep3-gatk-haplotype.vcf.gz

Here are the first few lines of the VCF:
#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  MOLM13_rep3
chr1    11869   .       G       <NON_REF>       .       .       END=14470       GT:DP:GQ:MIN_DP:PL      0/0:0:0:0:0,0,0
chr1    14471   .       C       <NON_REF>       .       .       END=14476       GT:DP:GQ:MIN_DP:PL      0/0:6:15:5:0,15,182
chr1    14477   .       G       <NON_REF>       .       .       END=14477       GT:DP:GQ:MIN_DP:PL      0/0:9:0:9:0,0,280

It does indeed look like GVCF rather than VCF.
Any ideas? I'm not sure that this isn't a GATK issue rather than a bcbio issue.

@rdocking @roryk
This is really a weird problem. I have reproduced this error for quite a few times a month ago. But yesterday I used a subset of the input fastq files to run the pipeline (head 100000 lines), it successfully finished! The output vcf.gz files looks right.
Maybe this is caused by bcbio, not GATK, is it right?

Thanks much for following up on this and for the detailed run information. I haven't been able to reproduce this so appreciate the help trying to determine the cases where it happens to we can push a fix to bcbio or GATK, depending on where the problem lies.
Rod, the workflow you describe and file types are exactly right. We first create gVCFs for each sample, then these get loaded in a GenomicsDB (the failing step), then get joint called.
The error messages indicate that this file:
/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/MOLM13_rep3-gatk-haplotype.vcf.gz

is missing the END line in the header, which should look like:
##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">

Is that correct? I'm not sure why that would happen, but it would be helpful to confirm and then look into the logs to determine the HaplotypeCallerSpark command that generates that file. If we can reproduce this error with a minimal case outside of bcbio then could try to prepare a bug report for the GATK team. Alternatively, if there is something else going on we can try to track it down. Thanks for the help looking into this.

Hi Brad - thanks for looking into this. I re-ran this yesterday using the new split-by-chromosome logic, and got caught up at a different step (but it seems to be a similar root cause):
For the same sample, for chromosome 1:
Run GATK:
[2018-05-15T02:20Z] eccfx01n3: export SPARK_USER=rdocking && unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && gatk-launch --java-options '-Xms800m -Xmx94349m -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmppIToLE'
HaplotypeCallerSpark --reference /projects/rdocking_prj/software/bcbio-nextgen/data/genomes/Hsapiens/hg19/ucsc/hg19.2bit
--annotation MappingQualityRankSumTest --annotation MappingQualityZero --annotation QualByDepth --annotation ReadPosRankSumTest --annotation RMSMappingQuality --annotation BaseQualityRankSumTest --annotation FisherStrand --annotation MappingQuality --annotation DepthPerAlleleBySample --annotation Coverage
-I /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/align/MOLM13_rep3/MOLM13_rep3-dedup.splitN.bam -L /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/regions/MOLM13_rep3-chr1-gatk-haplotype-regions-glimit-nolcr-nopolyx.bed
--interval-set-rule INTERSECTION --spark-master local[32]
--conf spark.local.dir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmppIToLE
--annotation ClippingRankSumTest --annotation DepthPerSampleHC
--emit-ref-confidence GVCF -GQB 10 -GQB 20 -GQB 30 -GQB 40 -GQB 60 -GQB 80 --output /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmppIToLE/MOLM13_rep3-chr1-gatk-haplotype.vcf

Compress and index:
[2018-05-15T02:39Z] eccfx01n3: cat /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmppIToLE/MOLM13_rep3-chr1-gatk-haplotype.vcf  | /projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin/bgzip --threads 32 -c > /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpm7_R_E/MOLM13_rep3-chr1-gatk-haplotype.vcf.gz
[2018-05-15T02:39Z] eccfx01n3: /projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin/tabix -f -p vcf /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpL3qNEh/MOLM13_rep3-chr1-gatk-haplotype.vcf.gz

FixHeader:
[2018-05-15T07:25Z] eccfx01n3: zgrep ^# /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/regions/MOLM13_rep3-chr1-gatk-haplotype.vcf.gz > /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpOscSa1/MOLM13_rep3-chr1-gatk-haplotype-fixheader-header.vcf
[2018-05-15T07:25Z] eccfx01n3: unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && picard FixVcfHeader HEADER=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpOscSa1/MOLM13_rep3-chr1-gatk-haplotype-fixheader-header.vcf INPUT=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/regions/MOLM13_rep3-chr1-gatk-haplotype.vcf.gz OUTPUT=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/MOLM13_rep3-chr1-gatk-haplotype-fixheader.vcf.gz

This time, this step is where it crashes:
[1:apply]: CalledProcessError: Command 'set -o pipefail; unset JAVA_HOME && export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && picard FixVcfHeader HEADER=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpOQ0QCk/MOLM13_rep1-chr1-gatk-haplotype-fixheader-header.vcf INPUT=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/regions/MOLM13_rep1-chr1-gatk-haplotype.vcf.gz OUTPUT=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/MOLM13_rep1-chr1-gatk-haplotype-fixheader.vcf.gz
23:44:43.965 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/share/picard-2.18.4-0/picard.jar!/com/intel/gkl/native/libgkl_compression.so
[Mon May 14 23:44:43 PDT 2018] FixVcfHeader INPUT=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/regions/MOLM13_rep1-chr1-gatk-haplotype.vcf.gz OUTPUT=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/MOLM13_rep1-chr1-gatk-haplotype-fixheader.vcf.gz HEADER=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/bcbiotx/tmpOQ0QCk/MOLM13_rep1-chr1-gatk-haplotype-fixheader-header.vcf    CHECK_FIRST_N_RECORDS=-1 ENFORCE_SAME_SAMPLES=true VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
[Mon May 14 23:44:43 PDT 2018] Executing as rdocking@eccfx01n4 on Linux 2.6.32-573.8.1.el6.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_144-b01; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.18.4-SNAPSHOT
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: AD
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: GQ
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: GT
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: MMQ
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: PGT
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: PID
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: PL
INFO    2018-05-14 23:44:44     FixVcfHeader    FORMAT line found in HEADER will be added to OUTPUT: SB
INFO    2018-05-14 23:44:44     FixVcfHeader    Writing the output VCF.
[Mon May 14 23:44:44 PDT 2018] picard.vcf.FixVcfHeader done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=514850816
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" java.lang.IllegalStateException: Key END found in VariantContext field INFO at chr1:11869 but this key isn't defined in the VCFHeader.  We require all VCFs to have complete VCFheaders by default.
        at htsjdk.variant.vcf.VCFEncoder.fieldIsMissingFromHeaderError(VCFEncoder.java:173)
        at htsjdk.variant.vcf.VCFEncoder.encode(VCFEncoder.java:112)
        at htsjdk.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:224)
        at picard.vcf.FixVcfHeader.doWork(FixVcfHeader.java:194)
        at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:282)
        at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103)
        at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:113)
' returned non-zero exit status 1

In this case, it looks like the intermediate files in bcbiotx aren't there anymore, but the last VCF files that exist are in:
/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/regions/
Which look like, for each chromosome:
> ls -1 | grep 'chr1-' | grep 'rep1'
MOLM13_rep1-chr1-gatk-haplotype-regions.bed
MOLM13_rep1-chr1-gatk-haplotype-regions-glimit.bed
MOLM13_rep1-chr1-gatk-haplotype-regions-glimit-nolcr.bed
MOLM13_rep1-chr1-gatk-haplotype-regions-glimit-nolcr-nopolyx.bed
MOLM13_rep1-chr1-gatk-haplotype.vcf.gz
MOLM13_rep1-chr1-gatk-haplotype.vcf.gz.tbi

These are the INFO fields from those VCF files:
##INFO=<ID=BaseQRankSum,Number=1,Type=Float,Description="Z-score from Wilcoxon rank sum test of Alt Vs. Ref base qualities">
##INFO=<ID=ClippingRankSum,Number=1,Type=Float,Description="Z-score From Wilcoxon rank sum test of Alt vs. Ref number of hard clipped bases">
##INFO=<ID=DP,Number=1,Type=Integer,Description="Approximate read depth; some reads may have been filtered">
##INFO=<ID=DS,Number=0,Type=Flag,Description="Were any of the samples downsampled?">
##INFO=<ID=ExcessHet,Number=1,Type=Float,Description="Phred-scaled p-value for exact test of excess heterozygosity">
##INFO=<ID=InbreedingCoeff,Number=1,Type=Float,Description="Inbreeding coefficient as estimated from the genotype likelihoods per-sample when compared against the Hardy-Weinberg expectation">
##INFO=<ID=MLEAC,Number=A,Type=Integer,Description="Maximum likelihood expectation (MLE) for the allele counts (not necessarily the same as the AC), for each ALT allele, in the same order as listed">
##INFO=<ID=MLEAF,Number=A,Type=Float,Description="Maximum likelihood expectation (MLE) for the allele frequency (not necessarily the same as the AF), for each ALT allele, in the same order as listed">
##INFO=<ID=MQ,Number=1,Type=Float,Description="RMS Mapping Quality">
##INFO=<ID=MQ0,Number=1,Type=Integer,Description="Total Mapping Quality Zero Reads">
##INFO=<ID=MQRankSum,Number=1,Type=Float,Description="Z-score From Wilcoxon rank sum test of Alt vs. Ref read mapping qualities">
##INFO=<ID=RAW_MQ,Number=1,Type=Float,Description="Raw data for RMS Mapping Quality">
##INFO=<ID=ReadPosRankSum,Number=1,Type=Float,Description="Z-score from Wilcoxon rank sum test of Alt vs. Ref read position bias">

Let me know if there's other information I can provide - here's the complete commands log:
bcbio-nextgen-commands.log
The debug log is ~33M - let me know if there's some part of that that would help.
Thanks again for all your help here!

OK - looks like this is actually something quite weird: the END info tag is in most (but not all) of the intermediate VCF files:
[2018-05-15 12:14:10]{rdocking@clingen01}/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicates/work/variation/rnaseq/gatk-haplotype/regions/> find MOLM13_rep1-chr*.vcf.gz | xargs zgrep '^##INFO=<ID=END' {}
gzip: {}.gz: No such file or directory
MOLM13_rep1-chr10-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr11-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr12-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr13-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr14-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr15-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr16-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr17-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr18-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr19-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr20-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr21-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr22-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr4-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr5-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr6-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr7-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr8-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chr9-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">
MOLM13_rep1-chrX-gatk-haplotype.vcf.gz:##INFO=<ID=END,Number=1,Type=Integer,Description="Stop position of the interval">

(i.e., it's missing from the chr1-chr3 VCF files, but there everywhere else)

Re-running the HaplotypeCaller command gives me a VCF with the correct ID=END field in the header - perhaps this is something to do with the cluster jobs running out of memory.

Rod;
Thanks for following up on this and all of the testing. To try and summarize: the region based approach is working through previous issues with runtimes, but we're getting some outputs with partial headers and missing END. If you re-run those with the same command, the headers are correct, indicating the problem is intermittent. Is that all correct?
If so, I think our next step is to get a correct and incorrect output for a region (with and without the END tag) and then inspect the outputs to look at differences. Ideally we'd do this with direct calls to gatk outside of bcbio. We could then summarize and prepare a bug report for the GATK team, as definitely we don't want to have non-deterministic results and would like to get to the root cause of this issue so we can detect and avoid. Would it be possible to generate those two outputs on your working dataset and share? If you have any tips about how best to generate the problem outputs I can also try to reproduce on my side so I can make these.
Thanks again for the help debugging.

@chapmanb @roryk
Hi, I rerun the HaplotypeCallerSpark command to generate the vcf files, but they still have no INFO=<ID=END lines. I have seen that in the log  file reported the warning message like

[2018-05-16T00:39Z] cnode02: 08:39:49.655 WARN  HaplotypeCallerSpark -
[2018-05-16T00:39Z] cnode02:    Warning: HaplotypeCallerSpark is a BETA tool and is not yet ready for use in production
[2018-05-16T00:39Z] cnode02: 08:39:49.655 INFO  HaplotypeCallerSpark - Initializing engine
[2018-05-16T00:39Z] cnode02: 08:39:49.656 INFO  HaplotypeCallerSpark - Done initializing engine
[2018-05-16T00:39Z] cnode02: 18/05/16 08:39:50 INFO SparkContext: Submitted application: HaplotypeCallerSpark
[2018-05-16T00:39Z] cnode02: 08:39:59.224 INFO  HaplotypeCallerSpark - ********************************************************************************
[2018-05-16T00:39Z] cnode02: 08:39:59.224 INFO  HaplotypeCallerSpark - The output of this tool DOES NOT match the output of HaplotypeCaller.
[2018-05-16T00:39Z] cnode02: 08:39:59.224 INFO  HaplotypeCallerSpark - It is under development and should not be used for production work.
[2018-05-16T00:39Z] cnode02: 08:39:59.224 INFO  HaplotypeCallerSpark - For evaluation only.
[2018-05-16T00:39Z] cnode02: 08:39:59.224 INFO  HaplotypeCallerSpark - Use the non-spark HaplotypeCaller if you care about the results.
[2018-05-16T00:39Z] cnode02: 08:39:59.224 INFO  HaplotypeCallerSpark - ********************************************************************************

So, I guess the beta version of HaplotypeCallerSpark caused this problem. Is there any way to use HaplotypeCaller rather than HaplotypeCallerSpark in bcbio?

Rod - thanks so much for looking into this, I appreciate the help.
@hliu2016 -- if you use a single core bcbio will use GATK HaplotypeCaller, but it's quite slow which is why we've been working to incorporate the Spark based caller. The practical speeds of the single caller approach make it not very useful for reasonable sized datasets. Our validations of the Spark caller (on DNA variant calling, where we have truth sets) look great:
https://github.com/bcbio/bcbio_validations/tree/master/gatk4
https://github.com/bcbio/bcbio_validations/tree/master/deepvariant#deepvariant-v06-release-strelka2-stratification-and-initial-gatk-cnn
but clearly we have a few issues to work through in terms of stability which is what we're hoping to tackle here.
Thanks again for all the helpful discussion and feedback.

@chapmanb,
This problem has been reproduced for quite a few times. Could I use gatk 3.8 to work through this problem?

@hliu2016 - you can use GATK 3.8 by using tools_off: gatk4 in your config:
http://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html?highlight=tools_off#changing-bcbio-defaults
For what it's worth, in my experience using GATK 3.8 within the RNA-Seq pipeline here will be very slow.

Updated testing results:

I extracted the HaplotypeCaller commands from the bcbio logs, and ran them directly in the same cluster environment.
I confirmed that the original command that was failing (the chr1 job), sitll gave the improper results when run directly
I confirmed that this issue is still present in GATK <IP_ADDRESS>
I confirmed that the issue seems to relate to the amount of sequence being queried by subsetting the input BED file:

20k intervals, ~31Mb: works
30k intervals, ~51Mb: works
40k intervals, ~71Mb: works
50k intervals, ~89Mb: works
60k intervals, ~109Mb: works
70k intervals (all chr1), ~125Mb: fails

This seems to be related to the amount of sequence, not the number of intervals. Using the input BED file without removal of the low-complexity regions and polyX regions also fails:

3,252 intervals, ~128Mb: fails.

This run is from a subset BAM file that I made to try to reproduce the issue more quickly. The BAM file is ~3G, and is shareable if need be. I'm attaching the cluster log output for the 60k job (that succeeeded) and the 70k job (that failed). I can't see anything notable, but I don't have a ton of experience with GATK.
gatk_debug_60k.txt
gatk_debug_70k.txt
Here's the submission command for the jobs:
export SPARK_USER=rdocking && \
unset JAVA_HOME && \
export PATH=/projects/rdocking_prj/software/bcbio-nextgen/data/anaconda/bin:$PATH && \
gatk-launch \
--java-options '-Xms800m -Xmx94349m -Djava.io.tmpdir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicate_one_small/debug/' \
HaplotypeCallerSpark \
--reference /projects/rdocking_prj/software/bcbio-nextgen/data/genomes/Hsapiens/hg19/ucsc/hg19.2bit \
--annotation MappingQualityRankSumTest --annotation MappingQualityZero \
--annotation QualByDepth --annotation ReadPosRankSumTest \
--annotation RMSMappingQuality --annotation BaseQualityRankSumTest \
--annotation FisherStrand --annotation MappingQuality \
--annotation DepthPerAlleleBySample --annotation Coverage \
-I /projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicate_one_small/work/align/MOLM13_rep1/MOLM13_rep1-dedup.splitN.bam \
-L /projects/karsanlab/rdocking/KARSANBIO-1254_pipeline/KARSANBIO-1390_rna_seq_runs/data/gatk_debug/chr1_70k.bed \
--interval-set-rule INTERSECTION \
--spark-master local[12] \
--conf spark.local.dir=/projects/karsanscratch/rdocking/KARSANBIO-1390_rna_seq_runs/molm13_replicate_one_small/debug \
--conf spark.driver.host=localhost \
--conf spark.network.timeout=800 \
--conf spark.executor.heartbeatInterval=100 \
--annotation ClippingRankSumTest --annotation DepthPerSampleHC \
--emit-ref-confidence GVCF -GQB 10 -GQB 20 -GQB 30 -GQB 40 -GQB 60 -GQB 80 \
--output MOLM13_rep1-chr1-70k-gatk-haplotype.vcf

Thanks!

One more note: I scaled the java memory request up a bit further for the failing job to:
--java-options '-Xms800m -Xmx110g
... and it produced a properly formed header. I'll see if I can get the equivalent memory request into my bcbio config and work past the issue that way.

@rdocking,
Thanks very much for sharing your experience in this problem. I have also tested that more memory allocation is helpful to produce proper header in the vcf.gz files, but are not sure to what extent the Xmx parameter should be set for different size bam files. I look forward to a systematic solution in the next release of gatk.

Submitted an issue on the GATK side - https://github.com/broadinstitute/gatk/issues/4821
Not sure if there's much that can be done on the bcbio side to guard against this until it's fixed upstream.

Rod -- thanks so much for all this incredible debugging and for submitting the issue upstream to the GATK folks. This helps so much, and I'm 100% agreed that fixing in GATK will provide the most benefit to the most people.
As a temporary way to avoid the issue I've tried to improve the parallelization by chromosome to now split into smaller blocks. Based on your analysis, I hope this will avoid large blocks, result in less memory requirements, and remove the incorrect outputs. This is all implemented in the latest development version and you shouldn't need any changes/adjustment to your configuration.
If you have an opportunity to test this, please let us know if it helps. We could tweak splitting more aggressively if needed as I tried to guess some on what the best number of splits would be.
Thank you again for all the work and help on this.

Confirmed that this works with my setup - the jobs are split into smaller blocks, and all the VCF files were produced with propoer header.
Thanks a lot Brad for helping us work through this!

Rod -- nice one, thanks for the positive report. Glad it's now working for you and I'll close this for now. Hopefully it will also get fixed upstream in GATK to avoid any future issues but glad to have analyses going.

@chapmanb
Thanks very much for your nice work. I further hope GATK fix this problem.

@chapmanb,
I have just tested the updated codes, it seems that there are still some bugs for GATK 4.0 in HaplotypeCallerSpark  step. I am running the pipeline on 12 rat samples, and the error is as below:

CalledProcessError: Command 'set -o pipefail; export SPARK_USER=root && unset JAVA_HOME && export PATH=/app/smartngs/bcbio/anaconda/bin:$PATH && gatk-launch --
java-options '-Xms500m -Xmx45864m -Djava.io.tmpdir=/app/datamart/jieyi_bio/sample/work/bcbiotx/tmpf1Mlmd' HaplotypeCallerSpark --reference /app/smartngs/bcbio/genomes/R
norvegicus/rn6/ucsc/rn6.2bit --annotation MappingQualityRankSumTest --annotation MappingQualityZero --annotation QualByDepth --annotation ReadPosRankSumTest --annotatio
n RMSMappingQuality --annotation BaseQualityRankSumTest --annotation FisherStrand --annotation MappingQuality --annotation DepthPerAlleleBySample --annotation Coverage
-I /app/datamart/jieyi_bio/sample/work/align/3_21Y_rep1/3_21Y_rep1-dedup.splitN.bam -L /app/datamart/jieyi_bio/sample/work/variation/rnaseq/gatk-haplotype/regions/3_21Y
rep1-9_57996107_115555261-gatk-haplotype-regions.bed --interval-set-rule INTERSECTION --spark-master local[16] --conf spark.local.dir=/app/datamart/jieyi_bio/sample/wo
rk/bcbiotx/tmpf1Mlmd --conf spark.driver.host=localhost --conf spark.network.timeout=800 --conf spark.executor.heartbeatInterval=100 --annotation ClippingRankSumTest --
annotation DepthPerSampleHC --emit-ref-confidence GVCF -GQB 10 -GQB 20 -GQB 30 -GQB 40 -GQB 60 -GQB 80 --output /app/datamart/jieyi_bio/sample/work/bcbiotx/tmpf1Mlmd/3
21Y_rep1-9_57996107_115555261-gatk-haplotype.vcf
at org.broadinstitute.hellbender.engine.spark.SparkCommandLineProgram.doWork(SparkCommandLineProgram.java:30)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
at org.broadinstitute.hellbender.tools.walkers.genotyper.GenotypeLikelihoodCalculators.calculateGenotypeCountUsingTables(GenotypeLikelihoodCalculators.java:388)
at org.broadinstitute.hellbender.tools.walkers.genotyper.GenotypeLikelihoodCalculators.getInstance(GenotypeLikelihoodCalculators.java:263)
at org.broadinstitute.hellbender.tools.walkers.genotyper.AlleleSubsettingUtils.subsettedPLIndices(AlleleSubsettingUtils.java:279)
at org.broadinstitute.hellbender.tools.walkers.genotyper.AlleleSubsettingUtils.subsetAlleles(AlleleSubsettingUtils.java:61)
at org.broadinstitute.hellbender.tools.walkers.genotyper.GenotypingEngine.calculateGenotypes(GenotypingEngine.java:296)
at org.broadinstitute.hellbender.tools.walkers.genotyper.GenotypingEngine.calculateGenotypes(GenotypingEngine.java:210)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerGenotypingEngine.assignGenotypeLikelihoods(HaplotypeCallerGenotypingEngine.java:15
0)

Sorry about the issue. This is an non-deterministic error that I've also hit, and submitted a bug report: broadinstitute/gatk#4661 I haven't been able to provide a reproducible case because re-running the same command worked. So a couple of suggestions:

Try re-running in place to see if a re-run works on your sample.
Respond to that GATK issue to let them know you're also seeing the problem, just so they know it's affecting multiple people. Maybe someone will be able to give us ideas to work around it.

Sorry to not have a definitely solution but hope this helps some.

Hi,
I am also facing the same issue. I've tried re-running the command a few times by changing the number of threads from 10 to 20 to 1: bcbio_nextgen.py config.yaml -n 1 and also changing the java options but every time I realized that it resumes from where it had left off so it doesn't generate the vcf files again. How do I rerun the specific command where it had generated the vcf files?
Here is my config.yaml:
details:
- algorithm:
    aligner: star
    variantcaller: gatk-haplotype
    # tools_off: gatk4
  analysis: RNA-seq
  description: 'Sample 1: CDL-634-13P'
  files:
  - /mnt/isilon/cbmi/variome/rathik/mendelian_rnaseq/fastq/CDL-634-13P/CDL-634-13P_pe_1.fq.gz
  - /mnt/isilon/cbmi/variome/rathik/mendelian_rnaseq/fastq/CDL-634-13P/CDL-634-13P_pe_2.fq.gz
  genome_build: GRCh37
  metadata:
    batch: CDL-634-13P
fc_name: CDL-634-13P-CDLS
upload:
  dir: /mnt/isilon/cbmi/variome/rathik/mendelian_rnaseq/bcbio_out/final
resources:
  gatk:
    jvm_opts: ["-Xms2g", "-Xmx110g"]

Thanks for the report and all of the work on debugging this. If you'd like to re-run specific commands, the best way is to look them up in log/bcbio-nextgen-commands.log and then you can replicate outside of bcbio itself. If bcbio is picking up after some of the generated VCF files, that indicates it finished cleanly on them and should be working.
Practically, the jvm_opts memory allocation you're providing for GATK looks like it might be problematic. That is the memory allocated per core, so you're providing up to 110 Gb of memory per core used, which will likely overrun available resources when run in parallel with multiple cores supplied. Hopefully scaling this back to the available memory per core on your machine will have better outcomes.
Thanks again for the work debugging this.


CLI & UI: Please add multiple download

Please add possibility to select multiple found subs in UI and download them all.
-naming for subs aftet 1st one can be *.2.srt, *.3.srt etc.
-ignore "cannot download error message" if more than 1 is selected and at least one was downloaded.
-same functionality for CLI: just flag like --dl-all to switch download of 1st found and all found.
Thank you in advance!

I don't think I like this suggestion. It's not very intuitive and I can't think of a good use case. In what order should the subs be named in case of multiple selection?

They should be named in same order they are sorted now )
(Another useful request here, is to add sorting by column by pressing on column header, but I will use cli since it's working now ))
As for the topic: it is very useful: I download many subs for TV series, usually on release day. There are often bad subtitles with different timings or just bad translations. And they can also appear at the top of the list.
If several are downloaded, I just quick-switch in player to next one(single key press) and see how it fits.
It also kinda fixes "cannot download error message", because, well, something will be still downloaded. And this message appears on 1st sub in the list sometimes too. (that's another issue, but i've accidentally deleted TV episodes with reproduce, will report later if reappear).

Yo,
I hope this get implemented :)

I want to keep the GUI as clean and easy to use as possible. Reserving this functionality for command-line only.

Any update on this for the command-line only?

I'll try to have a look at this the coming weekend.

The multi-language download would be a great feature since subtle downloads faster than other software, but sadly replaces the .srt


ERROR - failed to load model from ./models/gpt4all-lora-quantized-ggml.bin

Current Behavior
The default model file (gpt4all-lora-quantized-ggml.bin) already exists. Do you want to replace it? Press B to download it with a browser (faster). [Y,N,B]?N
Skipping download of model file...
Cleaning tmp folder
Virtual environment created and packages installed successfully.
Launching application...
Checking discussions database...
[2023-04-18 10:11:49,423] {model.py:73} INFO - Loading model ...
llama_model_load: loading model from './models/gpt4all-lora-quantized-ggml.bin' - please wait ...
llama_model_load: invalid model file './models/gpt4all-lora-quantized-ggml.bin' (bad magic)
[2023-04-18 10:11:49,424] {model.py:75} ERROR - failed to load model from ./models/gpt4all-lora-quantized-ggml.bin
Steps to Reproduce
run webui.bat
Screenshots

What are your system specs? OS? CPU? RAM? Did you download model correctly? I mean maybe its corrupted, try downloading using browser, and then copy it to /models/ folder.

Win11, 9900K, 32G.
The file was downloaded completely without any issues. I tried downloading it again using the browser, but the file size and error message were the same.

CPU seems to support AVX2..
Im running this in ubuntu VM, seems to load just fine.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Checking discussions database...
llama_model_load: loading model from './models/gpt4all-lora-quantized-ggml.bin' - please wait ...
llama_model_load: n_vocab = 32001
llama_model_load: n_ctx   = 512
llama_model_load: n_embd  = 4096
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 32
llama_model_load: n_layer = 32
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 11008
llama_model_load: n_parts = 1
llama_model_load: type    = 1
llama_model_load: ggml map size = 4017.70 MB
llama_model_load: ggml ctx size =  81.25 KB
llama_model_load: mem required  = 5809.78 MB (+ 2052.00 MB per state)
llama_model_load: loading tensors from './models/gpt4all-lora-quantized-ggml.bin'
llama_model_load: model size =  4017.27 MB / num tensors = 291
llama_init_from_file: kv self size  =  512.00 MB
Chatbot created successfully
 * Serving Flask app 'GPT4All-WebUI'
 * Debug mode: off
[2023-04-18 09:28:50,988] {_internal.py:224} INFO - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (<IP_ADDRESS>)
 * Running on http://<IP_ADDRESS>:9600
 * Running on http://mm.jj.ss:9600
[2023-04-18 09:28:50,988] {_internal.py:224} INFO - Press CTRL+C to quit

Well i did a git pull right now on VM, and i cant load it anymore aswell
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Checking discussions database...
llama_model_load: loading model from './models/gpt4all-lora-quantized-ggml.bin' - please wait ...
./models/gpt4all-lora-quantized-ggml.bin: invalid model file (bad magic [got 0x6e756f46 want 0x67676a74])
        you most likely need to regenerate your ggml files
        the benefit is you'll get 10-100x faster load times
        see https://github.com/ggerganov/llama.cpp/issues/91
        use convert-pth-to-ggml.py to regenerate from original pth
        use migrate-ggml-2023-03-30-pr613.py if you deleted originals
llama_init_from_file: failed to load model
Chatbot created successfully
 * Serving Flask app 'GPT4All-WebUI'
 * Debug mode: off
[2023-04-18 09:35:03,952] {_internal.py:224} INFO - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (<IP_ADDRESS>)
 * Running on http://<IP_ADDRESS>:9600
 * Running on http://<IP_ADDRESS>:9600
[2023-04-18 09:35:03,952] {_internal.py:224} INFO - Press CTRL+C to quit

@ParisNeo something is borked again. Maybe pyllamacpp, ... my hate towards python keeps growing...

You got "bad magic" too.

You got "bad magic" too.

its most likely that the model loading script was updated because this UI relies and depends on other packages/repos, so if they get updated it will take some time for main dev of this repo to look through the code and fix it. He on vacation right now. So just hang tight. it will be fixed eventually.

@Datou Hi, try this model, it works, the original model is messed up, idk why.
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
Checking discussions database...
llama_model_load: loading model from './models/ggml-vicuna-13b-4bit-rev1.bin' - please wait ...
llama_model_load: n_vocab = 32001
llama_model_load: n_ctx   = 512
llama_model_load: n_embd  = 5120
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 40
llama_model_load: n_layer = 40
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 13824
llama_model_load: n_parts = 2
llama_model_load: type    = 2
llama_model_load: ggml map size = 7759.84 MB
llama_model_load: ggml ctx size = 101.25 KB
llama_model_load: mem required  = 9807.93 MB (+ 3216.00 MB per state)
llama_model_load: loading tensors from './models/ggml-vicuna-13b-4bit-rev1.bin'
llama_model_load: model size =  7759.40 MB / num tensors = 363
llama_init_from_file: kv self size  =  800.00 MB
Chatbot created successfully
 * Serving Flask app 'GPT4All-WebUI'
 * Debug mode: off
[2023-04-18 12:36:03,700] {_internal.py:224} INFO - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (<IP_ADDRESS>)
 * Running on http://<IP_ADDRESS>:9600
 * Running on http://mama-mia.juu:9600
[2023-04-18 12:36:03,700] {_internal.py:224} INFO - Press CTRL+C to quit

For me it loaded after i pulled the newest changes from git and redownloaded gpt4all-lora-quantized-ggml.bin model

Sorry guyes I have a very slow connection these days and I lost the connection yesterday. It should work now. Please if the problem is solved make sure to close the issue.


New Tractus-X project contributor

Before requesting the project contributor role, please make sure, that you:

Make yourself familiar with the Eclipse Foundation project roles: https://eclipse-tractusx.github.io/docs/oss/contributor-committer
Make ourselves familiar with prerequisites, like a signed ECA: https://eclipse-tractusx.github.io/docs/oss/getting-started
Check the committer list of our project and highlight at least one of them on this issue, since every committer
can add project contributors: https://projects.eclipse.org/projects/automotive.tractusx/who

Necessary Information
Email address to add<EMAIL_ADDRESS>Committers, that could take over: @borisrizov-zf
Please make sure, that you accept the invitation you receive from GitHub, after you have been added as project contributor.

Hi @RoKrish14, do you know maybe a committer from ZF?

I am only aware of @borisrizov-zf , just from the ZF Id.

to @RoKrish14 just to inform you every committer can add new Contributors to our Eclipse Foundation project.
To @borisrizov-zf, cool then congrats  for your voting and won election to be a committer.
Question from myside could you please take over the onboarding a new Contributor into our Eclipse Project?
If help needed we are here 👍
Thanks in advance.

@FaGru3n sure. Do we have a workflow for this I can follow?

Hi not a place where i am aware of.
but simple for Email address to add<EMAIL_ADDRESS>

Thanks! It's done.


Mention uniqueness check in plugin name method docs

Objective
One of the most important usages of plugin names is currently not mentioned in its documentation: the uniqueness check
Solution

Mention that the plugin name is used to check for uniqueness

bors r+


Fixing "walk multipart email", adding "cc" criteria, adding "get email count" keyword, updating tests

Without decoding the data sent to "message_from_string()" function under the walk_multipart_email keyword will results in the following error:

TypeError: initial_value must be str or None, not bytes

Added .decode('utf-8') to fix it.
Added "get email count" keyword for asserting that the email folder contains X amount of emails that fulfill the set criteria. Added "cc" criteria to filter emails.
Updated library name in tests and removed deprecated tests.
Also removed "test_should_mark_all_emails_as_read" and "test_should_delete_all_emails" as they wont work with "self._get_all_emails()" being called inside the keyword.

I had a private smtp server (SmarterMail) for testing. I've tested with the is_secure-parameter set to true, clear password and default port. I have not tested this library with public servers such as gmail.
I validated that the counting keyword and cc-criteria worked by manually viewing the mailbox contents, after sending different amounts of cc-email to the mailbox.
The walk multipart email keyword crashed right away so just getting the keyword green was my test case. I did also validate that get multipart field names returned the appropriate field names, with the fixed multipart walking.
If you have some specific test cases in mind, I can write those in robot to test this update further.

It's fine. As a long-term goal I would like to get (inspiration to write) real integration tests to run with Gmail/Outlook.
I will try to release a version with these fixes this week.