Crons are widely used to set up scheduled jobs and automate certain parts of a system. Setting up Cron for a simple application is very straightforward. All you need to do is configure it in your server using `crontab` to run at scheduled intervals and trigger the job.
But what if the server with this crontab goes down? We need to ensure that the system is resilient to a single node failure. The simplest way to do this is to have 2 (or multiple) servers with crontab running. That, however, might trigger the job multiple times. And we surely don’t want that happen!
What we really need is some sort of a fault tolerant, resilient cron setup. How do we then build a Highly Available Distributed Cron?
Let’s say, an application wants to send out reminder emails for drinking water every 30 minutes.
Using AWS CloudWatch As Cron
CloudWatch is a monitoring service. While it is extremely popular for tracking the status and logging system reports, it’s alarm setting ability can be used to act as a Cron.
- Expose the Email Sending Service as an HTTP endpoint.
- Create an AWS SNS topic that will hit the exposed HTTP endpoint for triggering the job.
- Now set up CloudWatch event to periodically trigger a notification. The SNS topic created above will be the event target.
(CloudWatch by itself cannot talk to HTTP endpoints, thus bringing SNS into the picture.)
Using AWS Lambda as CloudWatch target
If you don’t want to expose any extra HTTP endpoints, AWS Lambda could be another alternative.
- Create an email sending service as a Lambda function.
- Set up CloudWatch event to periodically trigger a notification. Include Lambda as this event’s target.
There are few limitations to using Amazon Lambda. It only allows you 128mb memory for each run and each run cannot be longer than 5 minutes. So if your logic is heavy and consumes more memory/time, you’d be better off exposing the endpoint and using CloudWatch with SNS.
Note: While we have used AWS tools in our solutions, other cloud service providers also have provisions to implement a similar solution. Microsoft Azure has scheduled Azure Functions and Google Cloud provides App Engine Cron.
Enjoyed our content? Subscribe to receive our latest articles right in your inbox:
(no spam, promise!)