Broadcast join support based on broadcast rule

Motivation
Druid now supports real joins (#8728). Even though this introduced a lot of opportunities to expand the capability of Druid, it is in practice pretty limited yet since we support only joins of a Druid datasource to Lookups or inline datasources. This proposal is to support the broadcast join using the broadcast rule (https://github.com/apache/druid/pull/4077). The goals of the proposed design are:

Easy to update configuration: Druid operators should be able to easily broadcast/unbroadcast datasources.
Support joins to stream: The coordinator should be able to broadcast datasources to realtime tasks.
Memory-efficient handling of giant broadcast datasources: The broadcast datasource can be large. Druid should be able to handle them efficiently.

Proposed changes
The idea is pre-broadcasting a datasource in every node and using it at query time. To do so, a datasource can be created using the regular ingestion spec and indexing service, and then be broadcasted by setting a BroadcastDistributionRule via the coordinator API. The entire datasource (its all segments) will be broadcasted all together to every historical, task (or indexer), and broker. A broadcasted datasource can be switched into a regular datasource by setting a loadRule via the coordinator API. The coordinator will be responsible for broadcasting datasources similar to that it distributes regular datasources based on loadRule.
The BroadcastDistributionRule can be set for any datasource. However, for faster hash join, we can create a pre-built hash table at ingestion time and load it in query nodes. At query time, the join query engine will check whether there is a pre-built hash table or not. If it's there, the engine can use it. Otherwise, it can create one on the fly.
The below subsections desrcibe what should be done to implement the broadcast join with the pre-built hash table.
Pre-built hash table
The pre-built hash table is created by the index task. Since the broadcasted datasource can be large, the query node (historicals, tasks, indexers, and brokers) should be able to read them using mmap as for regular segments to avoid heavy GC overhead.
The format of the hash table is not determined yet. I will update this proposal or write another proposal for it. Whatever the new format would be, a new SegmentizerFactory will be added to handle it properly.
The index task will create pre-built hash tables
To let the task know what kind of segments/indexes it needs to create, we need a new configuration in the ingestion spec. However, I'm not 100% sure what a nice configuration would be for it yet. One of the reasons is the undetermined hash table format. I think the new configuration might be different depending on what we want to do with the hash table format. For example, we need at least the list of columns to build the hash table which should be specified via the user configuration. I'm not sure yet what else we need.
As a result, I would like to suggest to add SegmentizerFactory in the tuningConfig as a meantime solution. The segmentizerFactory is already an extendable interface and JSON-serializable. By doing this, we can test out various segment/index format for hash join. After the dust settles down and we find the best format, we can deprecate the SegmentizerFactory and add a better configuration in the ingestion spec. An example would be
     "tuningConfig" : {
       "type" : "index_parallel",
       "segmentizerFactory": {
           "type": "hashIndexedTable",
           "keyColumns": ["columns", "to", "create", "hash table"]
       },
       ...
    }

If the SegmentizerFactory is missing in the ingestion spec, the MMappedQueryableSegmentizerFactory will be used by default. The index task will write the SegmentizerFactory in the factory.json file when it writes the segment file. See the below "SegmentizerFactory for loading data and pre-built hash table" section for how to read the segment/hashtable file.
For broadcast datasources, it is not allowed to append more data. That means, the whole datasource should be broadcasted all together.
Modifying BroadcastDistributionRule
The BroadcastDistributionRule has a field of colocatedDatasources. This field was for users to specify with what datasources the broadcasted datasource will be colocated. This causes a bug in broadcasting, that is, the broadcasted segment should be moved with the colocated segment atomically which it doesn't do it now. To fix this issue, I think it's better to remove this field because a regular datasource is usually distributed to all historicals if it's large enough and so the colocatedDatasources doesn't seem very useful anyway. Instead, we can add tiers similar to loadRules.
I believe this doesn't cause any compatibility issue as it is not practically in use in any production. I'm pretty sure about it because 1) we didn't have a proper use case for it before, i.e. joins, and 2) it is only partially implemented and doesn't fully work.
Assigning and balancing segments
The coordinator should assign all segments of a broadcast datasource to all historicals, realtime tasks, brokers, and indexers. When assigning segments, replication throttling shouldn't apply for broadcasting. Also, the coordinator should skip balancing for broadcasted datasources.
Segment loading in brokers, realtime tasks, and indexers
All brokers, realtime tasks, and indexers will load assigned segments. Both ZK-based and HTTP-based segment loading should be supported.
SegmentizerFactory for loading data and pre-built hash table
Once a segment file is downloaded in a historical, a task, an indexer, or a broker, the file should be loaded properly including the pre-built hash table. The node should create the right type of SegmentizerFactory from the factory.json file created by the index task. The new SegmentizerFactory should know details of how to load data and the hash table. Once the SegmentizerFactory finishes loading, the node announces the loaded segment.
Querying broadcasted datasources
All broadcast datasources will be registered on the index schema. When you want to join using SQL, you can do like below:
SELECT count(*)
FROM druid.t1, index.t2
WHERE t1.col = t2.col

For joins to broadcast datasources, the broker will find the most recent broadcast segments which are available in all nodes where it wants to send the query. If there is no such broadcast segment, the query will fail. Once it finds such broadcast segments, it will send the join query to all nodes which have the broadcast segments and collect the result from them. Note that the join of a broadcast datasource and another broadcast datasource will not be supported in this proposal.
For directly querying on the broadcast datasources, the broker should pick up only one node among historicals and brokers including itself and send the query to it.
Rationale

Expanding the Lookup APIs to support wide tables. This idea was dropped because the Lookup API is tightly coupled with its data KV data model. It also has its own way to load data which is pretty limited compared to the regular ingestion tasks.

Operational impact
As explained in the "Modifying BroadcastDistributionRule" section, the colocatedDatasources will be removed from BroadcastDistributionRule. However, I don't think this will cause any compatibility issue.
Future work

Tier support for broadcast rule
Stabilizing the Indexer. Since the broadcast datasource will be replicated per task with the middleManager, it can occupy a huge amount of memory. It is more realistic to use the Indexer for broadcast joins to stream.
Support joins of a broadcast datasource to another broadcast datasource.
Broadcast rule and lookup segments. The lookup API can be replaced with the combination of the broadcast rule and lookup segments.
Broadcasting only a part of a datasource. Imagining that you may want to broadcast only the segments of some recent time period since they are frequently joined. In this case, allowing to broadcast only a part of a datasource would be useful. We can similarly allow appending data to broadcast datasources.

Hi @jihoonson, thanks for your great ideas and plans to move forward from inline datasources joining.

The coordinator should assign all segments of a broadcast datasource to all historicals, realtime tasks, brokers, and indexers.

Can we have a fine-grained segment assignment/loading plan with the help of HashedPartitionsSpec?
As we know Druid already enables hash-based secondary partitioning for batch indexing tasks.
Suppose that there are several datasources which are configured with same secondary hash-based partitionsSpec which contains partitionDimensions user intends to joining. Then we introduce a new LoadRule, like ColocateDistributionRule which colocates segments with same chunk id, to load segments among query processing services. And to work with realtime data, Druid has to enable HashedPartitionsSpec in realtime ingestion tasks while currently the historical segments with hashed ShardSpec seems not appendable by default. But still could it be a potential improvement that is compatible with this proposal?

Hey @yuanlihan, that is a great idea! We should definitely support such a partition-aware segment loading in the future. I'm not sure about details of the new segment loading algorithm yet, but it could be compatible with this proposal.


failed to cherry pick PR 3778 from ty-dc, to branch release-v0.8

commits a5a29e1474036ec9bffa91ebea1f3635157be86f of ty-dc conflict when merging to branch release-v0.8, please manually cherry pick it by yourself.
PR https://github.com/spidernet-io/spiderpool/pull/3778 ,  action https://github.com/spidernet-io/spiderpool/actions/runs/10454536885
CONFLICT (modify/delete): docs/usage/install/upgrade-zh_CN.md deleted in HEAD and modified in a5a29e14 (Merge pull request #3778 from ty-dc/fix/error-gc).  Version a5a29e14 (Merge pull request #3778 from ty-dc/fix/error-gc) of docs/usage/install/upgrade-zh_CN.md left in tree.
Auto-merging docs/usage/install/upgrade.md
CONFLICT (content): Merge conflict in docs/usage/install/upgrade.md
Auto-merging go.mod
CONFLICT (content): Merge conflict in go.mod
Auto-merging go.sum
CONFLICT (content): Merge conflict in go.sum
Auto-merging pkg/gcmanager/pod_cache.go
Auto-merging pkg/workloadendpointmanager/workloadendpoint_manager.go
CONFLICT (content): Merge conflict in pkg/workloadendpointmanager/workloadendpoint_manager.go
Auto-merging pkg/workloadendpointmanager/workloadendpoint_manager_test.go
CONFLICT (content): Merge conflict in pkg/workloadendpointmanager/workloadendpoint_manager_test.go
Auto-merging test/doc/reclaim.md
CONFLICT (content): Merge conflict in test/doc/reclaim.md
Auto-merging test/e2e/common/constant.go
Auto-merging test/e2e/common/pod.go
CONFLICT (content): Merge conflict in test/e2e/common/pod.go
Auto-merging test/e2e/common/spiderpool.go
Auto-merging test/e2e/common/tools.go
Auto-merging test/e2e/reclaim/reclaim_test.go
CONFLICT (content): Merge conflict in test/e2e/reclaim/reclaim_test.go
Auto-merging test/e2e/spidermultus/spidermultus_test.go
CONFLICT (content): Merge conflict in test/e2e/spidermultus/spidermultus_test.go
Auto-merging vendor/github.com/spidernet-io/e2eframework/framework/pod.go
Auto-merging vendor/modules.txt
CONFLICT (content): Merge conflict in vendor/modules.txt
error: could not apply a5a29e14... Merge pull request #3778 from ty-dc/fix/error-gc
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config advice.mergeConflict false"

fix #3914


auto mtls might be breaking regular TLS

Configure DR like
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: istiod
  namespace: istio-system
spec:
  host: istiod.istio-system.svc.cluster.local
  subsets:
    # Set up TLS to istiod backends
    - name: tls
      trafficPolicy:
        connectionPool:
          http:
            h2UpgradePolicy: UPGRADE
        tls:
          mode: SIMPLE

config_dump shows both transport_socket and transport_socket_matches and the traffic is wrong - its neither recognized as a TLS handshake nor is it plaintext.
Disabling auto mtls and everything works.
Assigning to myself to investigate further

Pretty sure this is fixed by #25711


[feature request] we need to add falco as an app

we can support Falco the container runtime security tool as an app, they have charts that we can use.
https://github.com/falcosecurity/charts
Expected Behaviour

Current Behaviour

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

What Kubernetes distribution are you using?

kubectl version

Operating System and version (e.g. Linux, Windows, MacOS):

uname -a

cat /etc/os-release

What arkade version is this?

arkade version

I'm a volunteer to do this issue. 👍

@developer-guy please spend more time on filling out this issue template, it seems rushed.

@alexellis I tried to add all the details in a proper way, can you review it, thank you.


v1.23 blog post: Dual-stack IPv4/IPv6 Networking Reaches GA

This is a blog post on behalf of SIG-Network for the v1.23 release, on the topic Dual-stack IPv4/IPv6 Networking (tracking issue, supporting KEP).
Next Steps:

[x] Determine blog release date; adjust this PR as needed (set date per @karenhchu)
[x] SIG-Docs review
[x] SIG-Network tech review

Handy preview link: https://deploy-preview-30538--kubernetes-io-main-staging.netlify.app/blog/2021/12/07/dual-stack-networking-ga/
Signed-off-by: Bridget Kromhout<EMAIL_ADDRESS>hold
/cc  @karenhchu @lachie83 @reylejano @khenidak @aojea @thockin @dcbw @caseydavenport

/assign @nate-double-u

@Priyankasaggu11929 Please add this to 1.23 milestone.
Thanks!

/assign @sftim
we need the other Tim too :)

/retitle v1.23 blog post: Dual-stack IPv4/IPv6 Networking Reaches GA

/retitle v1.23 blog post: Dual-stack IPv4/IPv6 Networking Reaches GA

/retitle v1.23 blog post: Dual-stack IPv4/IPv6 Networking Reaches GA

/lgtm
(i do believe we need to expand on the tech docs. but that is a battle for another day).

We may have further sig-network edits coming from @aojea, but in general I think this is ready for sig-docs comments.

We may have further sig-network edits coming from @aojea, but in general I think this is ready for sig-docs comments.

(i do believe we need to expand on the tech docs. but that is a battle for another day).

lgtm
Thanks

/lgtm

/milestone 1.23

/lgtm

/priority important-soon

/approve


Adds Recovery cones for felinids

meow meow

About the PR
Media

[ ] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl:

add: Added fun!
remove: Removed fun!

...

oh cool a bug that i can't figure out!

it does not appear in game, i might need to cut the radar dish

):

  oh cool a bug that i can't figure out!

im sure you forgot a dash there.

you forgot a dash there.

 ):

Read carefully the error output. It gives you line numbers in the YML file.

you forgot a dash there.

...

interesting...

sprites aren't appearing

strange thing is, i haven't gotten any error except invalid scancode

the cone isn't appearing at all except the icon

the cone isn't appearing at all except the icon

You have the RSI state as equipped-MASK while the item is worn on the neck.

Oh

I thought that the masking part of the sprite, would it be like equipped-NECK?

as of now, i believe it is now commit-worthy

The side profile looks alright on the sprite. The front profile, however:

If you can improve the south-facing part of the sprite just a bit I can fix up the rest of the PR for you

The side profile looks alright on the sprite. The front profile, however:

im gonna bash my skull into my table ifi have to redesign it after this

The icon sprite still looks like a box. The Felinidcone.yml file should at least by felinidcone.yml and preferably felinid_cone.yml. No need to use capital letters unless absolutely required. Not sure what's going on with the shock collar. Could use a better description. Didn't this have a recipe in one of the previous PRs?

It didn't

Closed for now as I'm am busy


VSinder automatically scrolls to the top

In vsinder , whenever we open a chat and then close it , it automatically scrolls to the top.
It takes a lot of time to scroll to the bottom if we have a lot of friends like me. Would be neat if this issue is resolved.

https://user-images.githubusercontent.com/61649925/103970626-b7747280-518e-11eb-8c4e-8401f999e07d.mp4

https://user-images.githubusercontent.com/61649925/103970626-b7747280-518e-11eb-8c4e-8401f999e07d.mp4


HotChocolate should infer [GraphQLIgnore] from interfaces if that is the only way a class is exposed

Is your feature request related to a problem? Please describe.
This is mainly a problem incurred from an assumption on how HotChocolate uses interfaces in a pure-code-first environment which turned out to be incorrect. I had assumed that I could use interfaces to effectively dictate what parts of my implementing classes were exposed externally via GQL, therefore I had thought that the framework would ignore the parts of my classes that weren't covered by interfaces.
When I got to run my project, HotChocolate started complaining about being 'Unable to infer or resolve a schema type from the type reference XXX', where XXX tended to be Framework constructs like IAsyncResult, Type, RuntimeFieldHandle etc.
I now understand that it just looked at my implementing classes to build a schema and ignored that half of the methods weren't actually exposed outside.
Describe the solution you'd like
I'd like HotChocolate to recognise when properties aren't exposed by any queries and automatically ignore them
Describe alternatives you've considered
The alternatives I can see are a liberal application of [GraphQLIgnore] or moving from pure code-first to code-first, which is a shame, as just exposing the queryable surface via interfaces seemed a much cleaner way of doing things.

That would work if you never expose the class itself.

That's what I originally attempted to do, but I could only see the following ways of doing it:
In SchemaBuilder:
Only adding the query => There is no object type implementing interface 'XXX'
Adding the query and interface => There is no object type implementing interface 'XXX'
Adding the query, interface and class => needs GraphQLIgnore on problematic properties on the class
Adding the query and class => needs GraphQLIgnore on problematic properties on the class
The only other way of doing it that I can see is if, rather than just relying on the SchemaBuilder, I need to explicitly state somewhere either in the interface or implementing classes what their relationship is, in addition to the pure c# public class ClassName : InterfaceName.


Allow to provide BroadcastChannel implementation

The suggestion is to allow custom BroadcastChannel implementation, notably https://github.com/pubkey/broadcast-channel

This is not a polyfill. Do not set this module to window.BroadcastChannel. This implementation behaves similiar to the BroadcastChannel-Standard with these limitations:
You can only send data that can be JSON.stringify-ed.
While the offical API emits onmessage-events, this module directly emitts the data which was posted

As mentioned, it's unsafe to polyfill BroadcastChannel with incompliant implementation. It can affect other page scripts that do feature detection.
It could be implemented by adding new options:
BroadcastChannel: BroadcastChannelImpl,
eventTransform: data => ({ data }),

Also related to #2 as BroadcastChannel shim is implemented through localStorage.

I'll check that out. Thanks.

https://github.com/wobsoriano/pinia-shared-state/commit/35ee3cacd3dd535dc54a79025c5627cc27839295


The longer my nodes are synchronized, the fewer my peers will be until one

The longer my nodes are synchronized, the fewer my peers will be until one，Can this situation be improved？

Please attach your logs or we will not be able to investigate this issue. See this comment.

Please attach your logs or we will not be able to investigate this issue. See this comment.

 Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[744059102,499955188]}}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.200080Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"40.014s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.200127Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"40.571s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.200513Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[744059102,499955188]}}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.897980Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.898047Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 13:57:52 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:52.137689Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address.  Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[245557572,999551008]}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.316636Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.374s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.316681Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.012s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.316921Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[245557572,999551008]}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.413226Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.413309Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.582667Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[27437055,151522376]}}
Jan 23 14:00:30 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:00:30 UTC [Info] Updating new available work took 6.066083908081055 ms
Jan 23 14:00:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:00:44 UTC [Info] Updating new available work took 6.68787956237793 ms
Jan 23 14:01:27 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:27.483328Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"41.425s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:01:27 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:27.483369Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"40.975s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:01:27 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:27.483607Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[27437055,151522376]}}
Jan 23 14:01:28 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:28.675850Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:01:28 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:28.675934Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:01:28 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:28.917801Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[401332569,79029017]}}
Jan 23 14:02:05 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:02:05 UTC [Info] verifier successfully got sigkill
Jan 23 14:02:05 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:02:05 UTC [Info] verifier successfully stopped
Jan 23 14:02:05 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:02:05 UTC [Info] Daemon started process of kind "Verifier" with pid 13360
Jan 23 14:02:20 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:20.632077Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.023s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:02:20 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:20.632120Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.191s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:02:20 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:20.632356Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[401332569,79029017]}}
Jan 23 14:02:21 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:21.323044Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:02:21 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:21.323106Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:02:21 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:21.585608Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1001313162,797172075]}}
Jan 23 14:03:11 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:11.868886Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.399s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:03:11 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:11.869025Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.384s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:03:11 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:11.869506Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1001313162,797172075]}}
Jan 23 14:03:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:12.009477Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:03:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:12.009541Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:03:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:12.233369Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[426855443,407406407]}}
Jan 23 14:04:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:01.639633Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.074s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:04:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:01.639673Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"23.832s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:04:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:01.639924Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[426855443,407406407]}}
Jan 23 14:04:02 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:02.343424Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:04:02 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:02.343486Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:04:02 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:02.586383Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address.  Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[955674734,1041699254]}}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.172597Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.049s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.172633Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.827s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.172875Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[955674734,1041699254]}}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.876975Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.877038Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:06:13 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:13.134560Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address.  Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1000839739,490788163]}}
Jan 23 14:08:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:00.536014Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.957s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:00.536055Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.318s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:00.536313Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1000839739,490788163]}}
Jan 23 14:08:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:01.226469Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:01.226542Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:01.471535Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[763909110,812381167]}}
Jan 23 14:08:53 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:53.444435Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.484s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:53 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:53.444475Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.988s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:53 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:53.444715Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[763909110,812381167]}}
Jan 23 14:08:54 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:54.125499Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:54 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:54.125571Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:54 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:54.370994Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[496799275,926167554]}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.008813Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.863s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.008853Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.274s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.009099Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[496799275,926167554]}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.162461Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.162563Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.385342Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[236906308,190797858]}}
Jan 23 14:11:06 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:06.467398Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"39.307s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:11:06 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:06.467437Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"41.275s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:11:06 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:06.467677Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[236906308,190797858]}}
Jan 23 14:11:07 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:07.157937Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:11:07 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:07.158004Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:11:07 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:07.402853Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[355797379,349982937]}}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.116080Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.263s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.116117Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.95s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.116349Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[355797379,349982937]}}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.814884Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.814947Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:01.059620Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[185962089,411663691]}}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.106499Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.438s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.106540Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"27.109s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.106780Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[185962089,411663691]}}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.816830Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.816901Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:56 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:56.065390Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[256686893,41657536]}}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.190425Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"27.417s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.190464Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"27.207s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.190709Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[256686893,41657536]}}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.901716Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.901781Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:13:52 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:52.142678Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[743950700,994019101]}}
Jan 23 14:14:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:14:44 UTC [Info] verifier successfully got sigkill
Jan 23 14:14:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:14:44 UTC [Info] verifier successfully stopped
Jan 23 14:14:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:14:44 UTC [Info] Daemon started process of kind "Verifier" with pid 13476
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.088910Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"42.212s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.088949Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"41.234s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.089190Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[743950700,994019101]}}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.196675Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.196735Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.471390Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address.  Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[744059102,499955188]}}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.200080Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"40.014s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.200127Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"40.571s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.200513Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[744059102,499955188]}}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.897980Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 13:57:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:51.898047Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 13:57:52 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 05:57:52.137689Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address.  Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[245557572,999551008]}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.316636Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.374s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.316681Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.012s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.316921Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[245557572,999551008]}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.413226Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.413309Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:00:04 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:00:04.582667Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[27437055,151522376]}}
Jan 23 14:00:30 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:00:30 UTC [Info] Updating new available work took 6.066083908081055 ms
Jan 23 14:00:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:00:44 UTC [Info] Updating new available work took 6.68787956237793 ms
Jan 23 14:01:27 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:27.483328Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"41.425s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:01:27 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:27.483369Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"40.975s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:01:27 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:27.483607Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[27437055,151522376]}}
Jan 23 14:01:28 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:28.675850Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:01:28 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:28.675934Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:01:28 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:01:28.917801Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[401332569,79029017]}}
Jan 23 14:02:05 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:02:05 UTC [Info] verifier successfully got sigkill
Jan 23 14:02:05 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:02:05 UTC [Info] verifier successfully stopped
Jan 23 14:02:05 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:02:05 UTC [Info] Daemon started process of kind "Verifier" with pid 13360
Jan 23 14:02:20 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:20.632077Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.023s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:02:20 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:20.632120Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.191s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:02:20 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:20.632356Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[401332569,79029017]}}
Jan 23 14:02:21 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:21.323044Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:02:21 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:21.323106Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:02:21 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:02:21.585608Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1001313162,797172075]}}
Jan 23 14:03:11 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:11.868886Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.399s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:03:11 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:11.869025Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.384s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:03:11 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:11.869506Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1001313162,797172075]}}
Jan 23 14:03:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:12.009477Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:03:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:12.009541Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:03:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:03:12.233369Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[426855443,407406407]}}
Jan 23 14:04:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:01.639633Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.074s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:04:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:01.639673Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"23.832s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:04:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:01.639924Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[426855443,407406407]}}
Jan 23 14:04:02 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:02.343424Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:04:02 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:02.343486Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:04:02 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:04:02.586383Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address.  Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[955674734,1041699254]}}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.172597Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.049s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.172633Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.827s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.172875Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[955674734,1041699254]}}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.876975Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:06:12 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:12.877038Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:06:13 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:06:13.134560Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address.  Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1000839739,490788163]}}
Jan 23 14:08:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:00.536014Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.957s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:00.536055Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.318s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:00.536313Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[1000839739,490788163]}}
Jan 23 14:08:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:01.226469Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:01.226542Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:01.471535Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[763909110,812381167]}}
Jan 23 14:08:53 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:53.444435Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.484s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:53 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:53.444475Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.988s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:08:53 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:53.444715Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[763909110,812381167]}}
Jan 23 14:08:54 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:54.125499Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:54 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:54.125571Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:08:54 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:08:54.370994Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[496799275,926167554]}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.008813Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"24.863s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.008853Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.274s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.009099Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[496799275,926167554]}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.162461Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.162563Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:09:45 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:09:45.385342Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[236906308,190797858]}}
Jan 23 14:11:06 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:06.467398Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"39.307s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:11:06 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:06.467437Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"41.275s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:11:06 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:06.467677Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[236906308,190797858]}}
Jan 23 14:11:07 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:07.157937Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:11:07 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:07.158004Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:11:07 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:11:07.402853Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[355797379,349982937]}}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.116080Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"25.263s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.116117Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.95s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.116349Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[355797379,349982937]}}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.814884Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:00 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:00.814947Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:01 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:01.059620Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[185962089,411663691]}}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.106499Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"26.438s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.106540Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"27.109s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.106780Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[185962089,411663691]}}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.816830Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:55 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:55.816901Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:12:56 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:12:56.065390Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[256686893,41657536]}}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.190425Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"27.417s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.190464Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 134, characters 12-27"},"message":"Base SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"27.207s"},"event_id":"885328f9dca84fa1cfb99089080613d6"}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.190709Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[256686893,41657536]}}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.901716Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:13:51 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:51.901781Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:13:52 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:13:52.142678Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[743950700,994019101]}}
Jan 23 14:14:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:14:44 UTC [Info] verifier successfully got sigkill
Jan 23 14:14:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:14:44 UTC [Info] verifier successfully stopped
Jan 23 14:14:44 iZrj9hl2epnkofxolky0ytZ coda[8663]: 2021-01-23 06:14:44 UTC [Info] Daemon started process of kind "Verifier" with pid 13476
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.088910Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"42.212s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.088949Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"41.234s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.089190Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[743950700,994019101]}}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.196675Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.196735Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}
Jan 23 14:15:16 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:15:16.471390Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 201, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[340081660,761005416]}}
Jan 23 14:16:39 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:16:39.218652Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"41.766s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:16:39 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:16:39.218692Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 129, characters 12-27"},"message":"Merge SNARK generated in $time","metadata":{"pid":12619,"process":"Snark Worker","time":"40.481s"},"event_id":"bcc5cf56ab582a129736e0906dea7cfa"}
Jan 23 14:16:39 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:16:39.218934Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 217, characters 14-25"},"message":"Submitted completed SNARK work $work_ids to $address","metadata":{"address":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker","work_ids":[340081660,761005416]}}
Jan 23 14:16:39 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:16:39.920855Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 169, characters 8-20"},"message":"Snark worker working directory $dir","metadata":{"dir":"/home/lilymina","pid":12619,"process":"Snark Worker"}}
Jan 23 14:16:39 iZrj9hl2epnkofxolky0ytZ coda[8663]: {"timestamp":"2021-01-23 06:16:39.920942Z","level":"Debug","source":{"module":"Snark_worker__Functor","location":"File "src/lib/snark_worker/functor.ml", line 181, characters 6-18"},"message":"Snark worker using daemon $addr","metadata":{"addr":"<IP_ADDRESS>:8301","pid":12619,"process":"Snark Worker"}}

These logs don't contain the information that we would need to investigate this.
You can run client export-local-logs (or client export-logs) to get the full log contents as a tar file, which we can look into if you upload it here.

These logs don't contain the information that we would need to investigate this.
You can run client export-local-logs (or client export-logs) to get the full log contents as a tar file, which we can look into if you upload it here.

Closing this issue as it is stale for a long time.


Update the playbook for deploying Static.

The aim here isn't so much to change the procedure, rather to document what's (more or less) already happening. The existing doc was clearly out of date and out of sync with the note in the Release app.

Extract the playbook steps and the background info into separate sections.
Update the playbook steps to reflect the current state of GOV.UK.
Clarify why the clunky manual process is the way it is for now, so that it's easier to figure out when it's ok to change or eliminate it in future.

Please review for accuracy in particular :) It's entirely possible that I've made mistakes based on misinterpreting what I've read in the code or other docs while putting this together.
I'd also be open to deleting this entirely, if people feel it's not useful/discoverable enough to be worth keeping.

Cool, thanks for the quick review @chao-xian!
I'll merge now, but if anyone spots anything that I've got wrong then please still comment and I'll be happy to fix it in a followup.


Extra copy when coercing

f.h
#include <vector>
void f(std::vector<double> const&);

g.pyx
from libcpp.vector cimport vector

cdef extern from "f.h":
    cdef void f(vector[double]) nogil

def g(x):
    f(x)

def h(x):
    cdef vector[double] y = x
    with nogil:
        f(y)

For g, cython generates roughly:
  std::vector<double>  __pyx_t_1;
  __pyx_t_1 = __pyx_convert_vector_from_py_double(__pyx_v_x); if (unlikely(PyErr_Occurred())) __PYX_ERR(0, 7, __pyx_L1_error)
  f(__pyx_t_1);

which seems ok. For h, it gives:
  std::vector<double>  __pyx_v_y;
  std::vector<double>  __pyx_t_1;
  __pyx_t_1 = __pyx_convert_vector_from_py_double(__pyx_v_x); if (unlikely(PyErr_Occurred())) __PYX_ERR(0, 10, __pyx_L1_error)
  __pyx_v_y = __pyx_t_1;
[...]
        f(__pyx_v_y);

That's too much copying. I don't think I can avoid naming the variable y if I want to use nogil. I assume cython doesn't want to write the result of the conversion directly to y in case an error occurs (?). But could it at least generate __pyx_v_y = MOVE(__pyx_t_1); where MOVE is std::move in C++11 or later?
The obvious workaround is to avoid using C++ types in interfaces, stick to double* and size, but we don't always choose what we are interfacing...
Cython version 0.29.14

Is this solved by #3362? Please try 3.0.

Oups, I just tried master (I should have done that first...) and indeed it is fixed, sorry for the noise.


This is a fake bug

I have a problemt that I need to write bots and test issues that are fake bugs

Environment

NetDaemon release with the issue: 0.2.3
Last working NetDaemon release (if known):
Operating environment (Home assistant Add-on/Docker/Dev setup):
Link to integration documentation on our website:

Link to or paste code that causes the issue

Traceback/Error logs

Additional information

NO!
This is not fake...
pls fix!