ansible_rulebook.rule_set_runner - ERROR - Error calling action run_playbook

Please confirm the following

[X] I agree to follow this project's code of conduct.
[X] I have checked the current issues for duplicates.
[X] I understand that ansible-rulebook is open source software provided for free and that I might not receive a timely response.

Bug Summary
I'm using Webhook event source to trigger ansible rulebook from the the command line using another locally installed service. The first time it triggers it almost always works but subsequent triggers I sometimes get an error (below)
Environment
1.0.3
Executable location = /usr/local/bin/ansible-rulebook
Drools_jpy version = 0.3.7
Java home = /usr/lib/jvm/java-17-openjdk-amd64
Java version = <IP_ADDRESS>
Python version = 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
Ubuntu 22.04.3 LTS on ESXi AMD64
Steps to reproduce
Using this rulebook:
---
- name: Run playbook
  hosts: all
  sources:
    - ansible.eda.webhook:
        host: <IP_ADDRESS>
        port: 6000
  rules:
    - name: Webhook called
      condition: event.payload.cmd == 'start'
      action:
        run_playbook:
          name: /home/charlespick/playbook.yml

Actual results
2023-10-29 18:46:29,916 - ansible_rulebook.rule_set_runner - ERROR - Error calling action run_playbook, err [('/home/charlespick/.ansible/cp/47aa5da64f', '/tmp/edach48wb7s/project/.ansible/cp/47aa5da64f', "[Errno 6] No such device or address: '/home/charlespick/.ansible/cp/47aa5da64f'")]

Expected results
Playbook should execute reliably
Additional information

@charlespick Since you are using a local playbook file, it would have to be copied into the project directory for the ansible-runner. Can you add copy_files: True as an option like shown here https://github.com/ansible/ansible-rulebook/blob/7e34383f1e9ba404134de011d2d88af8f768cbd5/tests/rules/test_set_facts.yml#L30

Hi @mkanoor
It doesn't look like that worked

Is it possible that there is a playbook in your collection with the same name?

- name: Listen for events on a webhook
  hosts: all
  ## Define our source for events
  sources:
    - ansible.eda.webhook:
        host: <IP_ADDRESS>
        port: 5000
  ## Define the conditions we are looking for
  rules:
    - name: Say Hello
      condition: event.payload.message == "Ansible is super cool"
  ## Define the action we should take should the condition be met
      action:
        run_playbook:
          name: say-what.yml

say-what.yml
- name: say thanks
  hosts: localhost
  gather_facts: false
  tasks:
    - debug:
        msg: "Thank you, {{ event.sender | default('my friend') }}!"

Try to execute this rulebook
To trigger the rule book use following command.
curl -H 'Content-Type: application/json' -d "{\"message\": \"Ansible is alright\"}" <IP_ADDRESS>:5000/endpoint
curl -H 'Content-Type: application/json' -d "{\"message\": \"Ansible is super cool\"}" <IP_ADDRESS>:5000/endpoint

Hi @nageshredhat We need the output of your ansible-rulebook cmd as well as the output of the ansible-rulebook --version. You can also try with -vv flag for more debug information.

I have same issue, rather than raising new issue, thought to discuss here, here is my rulebook
- name: Listen Kafka Events for BGP Neighbors
  hosts: all
  sources:
  - ansible.eda.kafka:
      host: "{{ hostname }}"
      port: "{{ port }}"
      topic: "{{ topic }}"
      group_id: "{{ group_id }}"
      offset: latest
      verify_mode: CERT_NONE
      security_protocol: SASL_PLAINTEXT
      sasl_mechanism: SCRAM-SHA-512
      sasl_plain_username: "{{ sasl_plain_username }}"
      sasl_plain_password: "{{ sasl_plain_password }}"

  rules:
    - name: Reason for BGP State Down
      condition: events.body.fields.open_check_error_code == "neighbor-down"
      action:
        debug:
          msg: |
            **Device: {{ event.body.tags.source }}
            **BGP Neighbor: {{ event.body.tags.neighbor_address }}
            **Description: {{ event.body.fields.description | default('N/A') }}
            **Remote ASN: {{ event.body.fields.remote_as_number }}
            **Address Family: {{ event.body.fields['af_data/af_name']}}
            **Prefix Limit: {{ event.body.fields['af_data/max_prefix_limit'] }}
            **Prefix Limit Threshold: {{ event.body.fields['af_data/max_prefix_threshold_percent'] }}
            **Reason: {{ event.body.fields.reset_reason }}

command to run this rulebook
ansible-rulebook --rulebook rulebooks/bgp-max-pfx-rulebook.yml -i cml_hosts.yml --verbose --vars .kafka_vars.yml

Here is the error keep repeating
2024-04-25 11:39:33,415 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:34,094 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:34,792 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:35,443 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:36,120 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:36,793 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:37,488 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:38,140 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:38,853 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:39,632 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:40,355 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:41,067 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:41,756 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:42,422 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:43,107 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:43,799 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 11:39:44,482 - ansible_rulebook.rule_set_runner - ERROR -

It started ok and after couple of minutes, I am getting this error and it keeps repeating itself.
Please advise.

Are all those attributes defined in the event payload. You can just use the
action:
debug:
to see what it prints that way we will know if its missing fields in the substitution. In Jinja you can put a default value like
{{ event.body.tags.source |default("missing")}}

Are all those attributes defined in the event payload. You can just use the action: debug:
to see what it prints that way we will know if its missing fields in the substitution. In Jinja you can put a default value like
{{ event.body.tags.source |default("missing")}}

thanks for the response @mkanoor , I will try that out, thanks for the advise, but is this related to the issue I am having ?

@muhammad-rafi When you change it to this
 rules:
    - name: Reason for BGP State Down
      condition: events.body.fields.open_check_error_code == "neighbor-down"
      action:
        debug:

We will at least know if the issue is related to the attribute missing in the Jinja substitution. The default debug action prints the entire payload.

    - name: Reason for BGP State Down
      condition: events.body.fields.open_check_error_code == "neighbor-down"
      action:
        debug:
          msg: |
            **Device: {{ event.body.tags.source | default("missing") }}
            **BGP Neighbor: {{ event.body.tags.neighbor_address | default("missing") }}
            **Description: {{ event.body.fields.description | default("missing") }}
            **Remote ASN: {{ event.body.fields.remote_as_number | default("missing") }}
            **Address Family: {{ event.body.fields['af_data/af_name'] | default("missing") }}
            **Prefix Limit: {{ event.body.fields['af_data/max_prefix_limit'] | default("missing") }}
            **Prefix Limit Threshold: {{ event.body.fields['af_data/max_prefix_threshold_percent'] | default("missing") }}
            **Reason: {{ event.body.fields.reset_reason | default("missing") }}

I have changed it to this as requested by you but I was not getting missing values, the issue is, it starts OK in the beginning and the after couple of I starts getting the following Memory threshold reached issue along with the one I mentioned earlier.
2024-04-25 22:53:24 543 [Thread-0] WARN org.drools.ansible.rulebook.integration.api.rulesengine.AutomaticPseudoClock - Pseudo clock is diverged, the difference is 207 ms. Going to sync with the real clock.
2024-04-25 22:54:57,237 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 93% > 90%
2024-04-25 22:54:57 386 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 93% > 90%. MaxMemory = 536870912, UsedMemory = 503722296
2024-04-25 22:54:57,387 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 93% > 90%
2024-04-25 22:54:57 546 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 93% > 90%. MaxMemory = 536870912, UsedMemory = 503981880
2024-04-25 22:54:57,547 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 93% > 90%
2024-04-25 22:54:57 719 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 93% > 90%. MaxMemory = 536870912, UsedMemory = 504231584
2024-04-25 22:54:57,719 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 93% > 90%
2024-04-25 22:54:57 885 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 93% > 90%. MaxMemory = 536870912, UsedMemory = 504512136
2024-04-25 22:54:57,886 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 93% > 90%
2024-04-25 22:54:58 049 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 93% > 90%. MaxMemory = 536870912, UsedMemory = 504788864
2024-04-25 22:54:58,050 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 93% > 90%
2024-04-25 22:54:58 203 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 93% > 90%. MaxMemory = 536870912, UsedMemory = 505036304
2024-04-25 22:54:58,203 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 93% > 90%
2024-04-25 22:54:58 367 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 94% > 90%. MaxMemory = 536870912, UsedMemory = 505286392
2024-04-25 22:54:58,367 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 94% > 90%
2024-04-25 22:54:58 544 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 94% > 90%. MaxMemory = 536870912, UsedMemory = 505618384
2024-04-25 22:54:58,545 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 94% > 90%
2024-04-25 22:54:58 706 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 94% > 90%. MaxMemory = 536870912, UsedMemory = 505848056
2024-04-25 22:54:58,706 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 94% > 90%
2024-04-25 22:54:58 875 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 94% > 90%. MaxMemory = 536870912, UsedMemory = 506078912
2024-04-25 22:54:58,875 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 94% > 90%
2024-04-25 22:54:59 056 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 94% > 90%. MaxMemory = 536870912, UsedMemory = 506334488
2024-04-25 22:54:59,057 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 94% > 90%
^C2024-04-25 22:54:59 416 [main] ERROR org.drools.ansible.rulebook.integration.api.rulesengine.MemoryMonitorUtil - Memory occupation is above the threshold: 94% > 90%. MaxMemory = 536870912, UsedMemory = 506578240
2024-04-25 22:54:59,417 - ansible_rulebook.rule_set_runner - ERROR - org.drools.ansible.rulebook.integration.api.rulesengine.MemoryThresholdReachedException: Memory threshold reached: 94% > 90%

omitted some output

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Thread-0"
2024-04-25 23:15:29,696 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:30,424 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:31,198 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:31,910 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:32,649 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:33,339 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:34,034 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:34,716 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:35,408 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:36,071 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:36,767 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:37,510 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:38,174 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:38,849 - ansible_rulebook.rule_set_runner - ERROR -
2024-04-25 23:15:39,529 - ansible_rulebook.rule_set_runner - ERROR -

besides this I have another rule for the playbook to run and same issue if I enable that rule too.
    - name: Display Kafka Logs and Run Action Playbook
      # for ioxr BGP neighbor down due to prefix limit exceeded
      condition: events.body.fields.is_neighbor_max_prefix_shutdown == "true" and events.body.fields.reset_reason == "max-prefix-exceeded"
      action:
        run_playbook:
          name: playbooks/bgp-max-pfx-fix.yml
          extra_vars:
            target_host: "{{ event.body.tags.source }}"
            bgp_neighbor: "{{ event.body.tags.neighbor_address }}"
            bgp_remote_asn: "{{ event.body.fields.remote_as_number }}"
            bgp_local_asn: "{{ event.body.fields.local_as }}"
          verbosity: 1
          copy_files: True

please advise.

@muhammad-rafi When you change it to this
 rules:
    - name: Reason for BGP State Down
      condition: events.body.fields.open_check_error_code == "neighbor-down"
      action:
        debug:

We will at least know if the issue is related to the attribute missing in the Jinja substitution. The default debug action prints the entire payload.

I have tried this suggested as well, it does print the entire payload but again same issue happening after couple of minutes

@muhammad-rafi It seems like some sort of a memory leak in the aiokafka or the kafka source plugin. How many events do you think are getting sent across? Is there a lot of events coming along and since they don't get ack'ed is the same event repeating. If you ran ansible-rulebook with the -vv option you will see the event coming in.
Also monitor the memory of the process, the JVM is dying  because its running out of memory because something is not releasing memory.

thanks @mkanoor I have a same doubt, this may be happening, I may be dealing with 1000+ interesting events, I was looking for some work around to slow it down or control the memory option with ansible rulebook. Let me know please if you have any suggestions

just to add, on the kafka we dont see any issue, it must be at my end.

@mkanoor any other thoughts on this one please ? this is still an issue

@muhammad-rafi Do you see the memory increasing for the python process? How many events are being processed?


Exit code 8 when updating from 0.4.3 to 0.4.4 and 0.4.5

Hi, Im getting this error when i update the package from version 0.4.3 to 0.4.4 and 0.4.5
W20150812-10:02:17.051(-5)? (STDERR)
W20150812-10:02:17.053(-5)? (STDERR) /Users/bc/.meteor/packages/meteor-tool/.<IP_ADDRESS>a6rm8b++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:245
W20150812-10:02:17.053(-5)? (STDERR) 						throw(ex);
W20150812-10:02:17.053(-5)? (STDERR) 						      ^
W20150812-10:02:17.053(-5)? (STDERR) Error: Match error: Failed Match.OneOf or Match.Optional validation in field layout
W20150812-10:02:17.053(-5)? (STDERR)     at checkSubtree (packages/check/match.js:244:1)
W20150812-10:02:17.053(-5)? (STDERR)     at packages/check/match.js:296:1
W20150812-10:02:17.053(-5)? (STDERR)     at Function..each..forEach (packages/underscore/underscore.js:113:1)
W20150812-10:02:17.053(-5)? (STDERR)     at checkSubtree (packages/check/match.js:290:1)
W20150812-10:02:17.054(-5)? (STDERR)     at check (packages/check/match.js:32:1)
W20150812-10:02:17.054(-5)? (STDERR)     at compile (packages/lookback:emails/emails.coffee:101:5)
W20150812-10:02:17.054(-5)? (STDERR)     at packages/lookback:emails/emails.coffee:384:9
W20150812-10:02:17.054(-5)? (STDERR)     at Function..each..forEach (packages/underscore/underscore.js:113:1)
W20150812-10:02:17.054(-5)? (STDERR)     at init (packages/lookback:emails/emails.coffee:381:9)
W20150812-10:02:17.054(-5)? (STDERR)     at MailerClass (packages/lookback:emails/emails.coffee:390:3)
=> Exited with code: 8

@vhuerta Hi! Sorry for late reply. Did you call the package with the correct arguments?

Im using this:
´´´´
Mailer.init({
templates: {
resetPassword: {
path: 'email/reset_email/template.html',
layout: 'emailLayout',
route: {
path: '/resetPassword/:name',
data: function() {
return {
background: Accounts.urls.backgroundUrl(),
logo: Accounts.urls.logoUrl(),
icon: Accounts.urls.iconMailUrl(),
user: "Ricardo Martinez Perez de Leon",
resetPasswordUrl: "http://<IP_ADDRESS>:3000/password_reset/123",
year: function() {
var now = new Date();
return now.getFullYear();
}
}
}
}
}
},
layout: {
name: 'emailLayout',
path: 'email/layout.html'
}
});
´´´´

@vhuerta Can you try out the latest version of this package as well?

Still getting this
W20151012-15:27:58.653(-5)? (STDERR) /Users/vhuerta/.meteor/packages/meteor-tool/.<IP_ADDRESS>sd3e7j++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:245
W20151012-15:27:58.653(-5)? (STDERR) 						throw(ex);
W20151012-15:27:58.653(-5)? (STDERR) 						      ^
W20151012-15:27:58.653(-5)? (STDERR) Error: Match error: Failed Match.OneOf or Match.Optional validation in field layout
W20151012-15:27:58.654(-5)? (STDERR)     at checkSubtree (packages/check/match.js:244:1)
W20151012-15:27:58.654(-5)? (STDERR)     at packages/check/match.js:296:1
W20151012-15:27:58.654(-5)? (STDERR)     at Function..each..forEach (packages/underscore/underscore.js:113:1)
W20151012-15:27:58.654(-5)? (STDERR)     at checkSubtree (packages/check/match.js:290:1)
W20151012-15:27:58.654(-5)? (STDERR)     at check (packages/check/match.js:32:1)
W20151012-15:27:58.655(-5)? (STDERR)     at compile (packages/lookback_emails/emails.coffee:107:5)
W20151012-15:27:58.655(-5)? (STDERR)     at packages/lookback_emails/emails.coffee:356:9
W20151012-15:27:58.655(-5)? (STDERR)     at Function..each..forEach (packages/underscore/underscore.js:113:1)
W20151012-15:27:58.655(-5)? (STDERR)     at init (packages/lookback_emails/emails.coffee:353:9)
W20151012-15:27:58.655(-5)? (STDERR)     at MailerClass (packages/lookback_emails/emails.coffee:362:3)
=> Exited with code: 8

@vhuerta The layout field for a template takes an object or boolean, you've provided a string. I've clarified the documentation on this: https://github.com/lookback/meteor-emails#layouts.
Please note that if you set the layout globally, as you've done, there's no need to specify it again for that specific template. If you do, send an object with name and path keys.
Let me know how that works out!
(Please note that latest version is 0.5.1).

Great i remove the layout from the resetPassword template and now its working fine in version 0.5.x thanks for the help


Feature request: replacing variable initializer

TensorFlow Hub seems to fix its initializer and load the values from the checkpoint file.
However, for example, we might need random initializer in the case of training ImageNet in scratch for research.
Do you have any plan to add the feature to replace initializer?

In TF2, a variable gets its variable set right at creation time; there will no longer be an explicit initializer op that gets executed at some time. This carries over to TF Hub modules in TF2 format, and makes your feature request infeasible in the TF2 world. To not complicate the transition from TF1 to TF2, we are not going to add this for TF1.
Unfortunately, that means TF Hub does not (currently) support an option to take module x and then choose between training from scratch vs warmstarting from its pre-trained weights.


Support for Multiple versions for YouVersion Parallel Mode reading

Thank you very much for this great plugin sir!
Is it possible to link two versions for parallel mode in YouVersion? Like:
Parallel Mode Settings version [LANGUAGE:VERSION]:

ENG:ESV, CEB:RCPV

Text:

@John 3:16

Result would be like this?
[John 3:16]
(https://www.bible.com/bible/59/GEN.1.RCPV?parallel=2187)](https://www.bible.com/bible/59/GEN.1.ABCEB?parallel=562)

Hey,
It is possible, but I see some problems:

YouVersion parallel mode works only for whole chapters, so it would work only for Gen 1 not Gen 1:2 (I could work for Gen 1:2 but page shown will be the same)
How to show it in settings? Secend dropdown? Who to pick versions for parallel mode? How to trigger it? As another bible version or via custom trigger char (similar to @)?
Feel free to make fork of this repo and try to make this futhure. I will try my best to merge it if you make it.

PS. As i can see links to you version don't requires version name so the parallel links coudl look like this:
https://www.bible.com/bible/{ version_1 }/{ book }.{ chapter }?parallel={ version_2 } for eg. https://www.bible.com/bible/59/GEN.1?parallel=2187

Hello @jaanonim
Thanks for the reply. I'll consider looking into this in the future maybe when I have time.
One use case I intend for this feature is for teaching and sermon notes when dealing with reading versions in local languages.
Thanks again sir! God bless.