Regardless of your app’s purpose, it is highly probable that you need to run background jobs at certain intervals to perform tasks such as sending reports, cleaning up the database, or fetching data from a 3rd party API. One of the options you can leverage in Heroku is to utilize custom clock processes.
In this post, we’ll review how clock processes work and compare them to Cron To Go, in order to help you choose the solution that’s right for you.
What are clock processes?
A clock process is a scheduling process that runs within your Heroku application, in the same manner that web and worker processes run, and is defined in your app’s Procfile. For example:
web: gunicorn gettingstarted.wsgi
clock: python clock.pyThe clock process is intended to run as a singleton process (i.e. just a single dyno) or else, it would trigger duplicate jobs. In essence, it is simply another dyno, running code that’s intended to schedule and trigger work that needs to be done, preferably by worker dynos, since doing the actual work within the clock process (the scheduling process) may prevent other jobs from being triggered. The code in clock.py (or any other programming language) should perform the actual scheduling, using libraries such as APScheduler, node-cron, whenever gem, Quartz, and others. Which means that scheduling involves programming.
What is Cron To Go?
Cron To Go is a distributed, scalable, and reliable cloud scheduler that you can add to your Heroku application as an add-on. It exercises an enterprise grade scheduler under the hood and allows you to define flexible schedules through a variety of methods, using the familiar Unix cron format.
Scheduling/Execution separation
It is recommended to separate the executing code from the scheduling code. That is, have the code in your clock process only handle the scheduling, and then queue background jobs to be executed by background processes. If you’re asking yourself: why, the answer is that performing the actual tasks requires time and possibly more resources, but there is only a single process dedicated to scheduling (otherwise jobs would run multiple times). Total separation between the executing code and the scheduling code would allow you to scale out the number of background workers. However, this requires more effort on the part of your developers.
With Cron To Go, scheduling and execution are separated by design - Cron To Go handles the scheduling and when the time comes for a job to be executed, it initiates a one-off dyno to perform said job - no need to handle queues. If you reach your app’s concurrent one-off dynos limit, you can simply reach out to Heroku and request to increase the limit.
Configuration as code
When using a clock process, it is the code that you write which is in charge of job scheduling. It’s not any different from the rest of your application code, and as such, it should be stored in your code repository and pushed to Heroku whenever you make any changes you’d like to deploy within your application.
Cron To Go supports configuration as code in the form of a simple yaml or json file that describes the jobs you run. These include the schedule (cron expression and time zone), the command to execute, the dyno type, and the time-to-live, after which, the dyno will be killed by Heroku (this is beneficial to ensure that your job doesn’t run for too long in case there’s an intermittent issue). You can read more about how to update your Cron To Go jobs as part of your deployment processes here.
User interface
Cron To Go also boasts a graphical user interface that can be leveraged by technical as well as non technical users to create, modify and monitor your app jobs. In addition to the dashboard, you can also install the Cron To Go CLI plugin for the Heroku command line and use your terminal to schedule and monitor jobs. You can read the CLI documentation here.
On the other hand, clock processes don’t have a built-in user interface to help you or your users manage jobs. If you’d like to have one, you will need to implement it yourself.
Complex scheduling requirements
Cron To Go supports cron expressions, which are an incredibly flexible way to define schedules. In addition, it supports time zones other than UTC in case you’d like to make sure a certain report is sent daily by a certain wall-clock time.
Your scheduling options within a clock process depend on the scheduling library you use. Some of them support sub-minute scheduling, which is not available with Cron To Go. However, if you require a job to run every second, you may want to rethink your solution, as it may be better to use an alternative mechanism that doesn’t involve scheduling, but instead, responding to events.
Cost
Clock process based job scheduling requires, at the minimum, 2 dynos working 24/7 - one clock dyno and one background worker. In addition, you will most likely need a messaging queue for the scheduler to communicate with the background worker.
Cron To Go’s pricing is based on the number of jobs you run. Since job execution relies on one-off dynos that start when a job is triggered and terminate when a job completes, your dyno charges will only incur the cost for the actual execution time. Depending on your use case, using Cron To Go may be incredibly cost effective (in addition to other benefits such as history tracking, notifications, and more).
Dyno restarts
All dynos in Heroku are restarted at least once a day. This is also true for the dyno that runs your clock process, and therefore, job executions that are supposed to be triggered during dyno restart may be skipped. You should make sure to implement some logic for dyno restart to ensure that no execution is skipped.
Cron To Go itself doesn’t rely on dynos for scheduling, and therefore, isn’t affected by dyno restarts. The one-off dynos that perform the actual work may be restarted, but the same goes for the worker dynos that are used to run tasks with a clock process.
History, notifications and alerts
Cron To Go tracks job execution and allows you to check your job history when issues arise. Moreover, you can set up email notifications to alert the team when a job fails or use webhook notifications to integrate Cron To Go with other monitoring services.
To sum up this comparison, unless you need sub-minute scheduling, or you insist on writing your own code for the purpose of scheduling, you should give Cron To Go a try. You can even import jobs from other schedulers (such as the Heroku Scheduler) into Cron To Go. Start your 7 day free trial now!
Post image generated by DALL-E 2
