Lorenzo Alberton

London, UK   ·   Contact me   ·  

« Open Source Projects

Task / worker pool manager in Go

* Start cli tasks automatically * Maintain the desired number of worker processes for each task * Handle automatic restarts when a worker dies or stalls The task manager is be able to start any cli (shell) script from the chosen directory. For tasks that are long-running and meant to be monitored continuously, each worker process should send regular keep-alive messages via a ZeroMQ PUB-SUB channel to communicate its health, and should handle SIGTERM messages when asked to terminate. If the worker doesn't respond to a SIGTERM signal, it will be killed with SIGKILL after a (configurable) grace period. The number of workers stalled/stopped since the task manager was started is reported in the task status.

The main package is the Task Manager (wpmanager), which can load the configuration, start some tasks automatically, and handle signals (CTRL+C) and HTTP requests to control the state of the tasks.
There's also a sample console application (wpconsole) to control the status and the configuration of each task from the command line. It supports controlling task managers running on different hosts/ports, and has a tab-completion interface (for both commands and task names).