Build a Highly Available Distributed Cron

Crons are widely used to set up scheduled jobs and automate certain parts of a system. Setting up Cron for a simple application is very straightforward. All you need to do is configure it in your server using `crontab` to run at scheduled intervals and trigger the job.

But what if the server with this crontab goes down? We need to ensure that the system is resilient to a single node failure. The simplest way to do this is to have 2 (or multiple) servers with crontab running. That, however, might trigger the job multiple times. And we surely don’t want that happen!

What we really need is some sort of a fault tolerant, resilient cron setup. How do we then build a Highly Available Distributed Cron?

Let’s say, an application wants to send out reminder emails for drinking water every 30 minutes.

Using AWS CloudWatch As Cron

CloudWatch is a monitoring service. While it is extremely popular for tracking the status and logging system reports, it’s alarm setting ability can be used to act as a Cron.

  • Expose the Email Sending Service as an HTTP endpoint.
  • Create an AWS SNS topic that will hit the exposed HTTP endpoint for triggering the job.
  • Now set up CloudWatch event to periodically trigger a notification. The SNS topic created above will be the event target.

(CloudWatch by itself cannot talk to HTTP endpoints, thus bringing SNS into the picture.)

Building a highly available distributed cron using cloudwatch

Using AWS Lambda as CloudWatch target

If you don’t want to expose any extra HTTP endpoints, AWS Lambda could be another alternative.

  • Create an email sending service as a Lambda function.
  • Set up CloudWatch event to periodically trigger a notification. Include Lambda as this event’s target.

There are few limitations to using Amazon Lambda. It only allows you 128mb memory for each run and each run cannot be longer than 5 minutes. So if your logic is heavy and consumes more memory/time, you’d be better off exposing the endpoint and using CloudWatch with SNS.

Note: While we have used AWS tools in our solutions, other cloud service providers also have provisions to implement a similar solution. Microsoft Azure has scheduled Azure Functions and Google Cloud provides App Engine Cron.

Enjoyed our content? Subscribe to receive our latest articles right in your inbox:
(no spam, promise!)