Why My Cron Job Ran at the Wrong Time (and How Timezones Tricked Me)

It was a Tuesday morning when my phone buzzed at 6:47 AM. A Slack message from our client: "Why did the weekly digest email go out at 2:30 AM last night? Our users are complaining."

I stared at the ceiling for a moment, already knowing exactly what had happened even before I opened my laptop. I had written that cron job myself, three weeks prior. I had tested it locally. I had watched it fire right on schedule. And somehow, in production, it had decided that 8:30 PM Eastern Standard Time was actually 2:30 AM.

This is the story of how I spent six hours chasing a ghost, and what I learned that I genuinely wish someone had told me before I shipped that job.

The Setup That Looked Perfectly Fine

The job itself was simple — a Node.js script that pulled weekly analytics data for each user account and sent a formatted digest email. The cron expression I wrote was:

30 20 * * 1

Monday evening, 8:30 PM. That was the intent. Send it when users are wrapping up their workday, maybe sitting with a cup of tea, receptive to a summary of their week. Clean, sensible, human.

On my MacBook, running crontab -l and watching the job fire at exactly 8:30 PM local time felt like proof. I committed the systemd timer configuration, opened a pull request, merged it, and called it done.

What I had not considered — what was somehow not even a flicker in my mind — was that our production server ran on UTC. And I was sitting in New York, which at that time of year is UTC-5.

8:30 PM UTC-5 is 1:30 AM UTC. Except I had used 20:30 in the cron expression, which the server obediently interpreted as 20:30 UTC — which landed at 3:30 PM Eastern. The email went out in the afternoon. Nobody noticed for a week, because the open rates were slightly off but not egregiously so. Then we shifted to EST (UTC-5) for winter, the clocks changed, and suddenly the math produced 2:30 AM for some users and 3:30 AM for others depending on their own timezone assumptions about "when did this arrive."

It was a mess. And it was entirely my fault.

The Debugging Session That Taught Me to Read Logs Differently

My first instinct was to SSH into the server and check the crontab directly. Everything looked correct. The expression matched what I had written. No obvious typo. The job itself was running without errors — the script logs showed a successful completion timestamp that I initially read as 20:30 and thought, wait, that's right?

Then I ran date on the server.

Mon Jun 17 20:31:04 UTC 2024

There it was. The server's clock was 20:31 UTC. Back home, my local machine said 3:31 PM. Five hours of difference. The cron job had run exactly when I told it to. The server had done nothing wrong. I had told it the wrong thing.

I pulled up the email delivery logs next. Every digest had fired at 20:30 UTC since deployment. That 20:30 looked so familiar to me that I had skimmed past it in every log review I had done, assuming it matched my local 8:30 PM intention. Our brains fill in gaps. We see what we expect to see. This is why debugging in production is genuinely dangerous — familiarity breeds blindness.

The Fix, and Why It's Not Just "Convert to UTC"

The naive fix is obvious: just subtract the offset. If I want 8:30 PM EST (UTC-5), I write 30 01 * * 2 — 1:30 AM UTC on Tuesday. Problem solved, right?

Mostly. But this breaks every time DST shifts. In summer, EST becomes EDT (UTC-4), so the job now fires at 9:30 PM local time instead of 8:30 PM. You are back to manual arithmetic. Your cron job has become a seasonal maintenance task.

The cleaner solution, and the one we moved to, was to stop using raw cron and start using a scheduler that understood timezones natively. We were already using Node.js, so we switched to node-cron with explicit timezone support:

const cron = require('node-cron');

cron.schedule('30 20 * * 1', sendWeeklyDigest, {
  timezone: 'America/New_York'
});

That single timezone option does what six hours of debugging and two days of calendar math could not: it lets the scheduler handle DST automatically, converts correctly when clocks change, and makes the intent legible in code review. Three months later, when clocks changed in November, the job still fired at 8:30 PM Eastern. No intervention needed.

If you are using systemd timers rather than classic cron, you have OnCalendar with a timezone suffix:

[Timer]
OnCalendar=Mon 20:30 America/New_York
Persistent=true

Python's APScheduler has a timezone parameter on the scheduler instance. Laravel's task scheduler lets you chain ->timezone('America/New_York') onto any scheduled closure. Most modern scheduling tools have a path out of the UTC-only trap — you just have to know to look for it.

What I Actually Changed in My Workflow

Beyond the code fix, this incident changed three things about how I work.

First, I added a timezone assertion to every job's log output. The first line any scheduled job writes is now: [job: weekly-digest] fired at 2024-06-17T20:30:00Z (America/New_York: 2024-06-17T15:30:00-05:00). Both the UTC timestamp and the intended local time. When I review logs now, I can see immediately whether the UTC-to-local conversion matches my expectation.

Second, I treat all cron expressions as documentation. I add a comment above every cron expression explaining the human-readable intent and the timezone it assumes:

# Weekly digest — Monday 8:30 PM Eastern (America/New_York)
# Uses node-cron timezone option; DO NOT convert to UTC manually
30 20 * * 1

This sounds obvious. It felt obvious to me even before this happened. But I hadn't been doing it, because I was optimizing for brevity. Now the comment is non-negotiable.

Third, I added a one-time firing test to our deployment checklist. Before we shipped the fixed version, I ran the job manually and verified the logged timestamp against the local time on two machines in different timezones. It takes four minutes. It would have saved six hours.

The Broader Thing About Timezone Bugs

What makes timezone bugs particularly nasty compared to other categories of production bugs is that they are often not bugs at all by the system's definition. The system did exactly what you told it to do. Your assumptions were wrong, not the code. That makes them harder to find in code review, harder to catch in testing, and more likely to slip through.

They also tend to have delayed consequences. A timezone offset of five hours might go unnoticed for weeks if the behavior is "close enough" — users in different timezones partially overlap with your wrong time, emails arrive in the early morning rather than at 2 AM, analytics look a little weird but not broken. By the time someone notices clearly enough to file a complaint, you have weeks of logs to sort through and a mental model of "this is working fine" that you have to consciously override.

The tools to catch these problems exist. cronitor.io, healthchecks.io, and similar monitoring services let you set expected execution windows and alert on drift. Running date on a new server before assuming any timing behavior is a thirty-second habit that costs nothing. Libraries like Luxon, date-fns-tz, and Python's pytz or the newer zoneinfo module give you named timezones with DST handling as a first-class feature rather than an afterthought.

The gap is not tool availability. The gap is that we assume things will work as they do locally, right up until they visibly don't.

One Week Later

The fixed digest started going out at 8:30 PM Eastern. Open rates climbed back to normal. The client sent a message saying the feedback from their users had turned positive again — people liked getting the email in the evening, reading it before bed, starting Tuesday with fresh context.

That was actually the part that stuck with me most. The job's value depended entirely on timing. An email at 8:30 PM is a useful summary. An email at 2:30 AM is noise that damages trust. The difference between the two was a timezone assumption I had made silently, without writing it down, and without testing it in an environment that matched production.

UTC is not a mistake. Running servers on UTC is the right default — it avoids a whole other class of problems. The mistake is assuming your local mental model of time applies to a system you haven't explicitly configured to share it. Write down your timezone intent. Use tools that encode it. Log both UTC and local time. And always, always run date on a new machine before you trust it with anything time-sensitive.