If the response is robotic, it isn't a page: alerting an unsupervised daily pipeline

A daily pipeline that runs without supervision changes the alerting problem rather than removing it. The Tatano Energy platform publishes 8 SEO articles every day across 4 separately indexed country domains, with 0 manual intervention required. Nobody is watching it on a Tuesday night. So the only design question left is which conditions should reach a human at all, and which should land quietly on a dashboard.

If the response is robotic, it isn't a page

The SRE guidance is blunt: if a page merely merits a robotic response, it shouldn't be a page. Applied to an unsupervised publisher, that disqualifies most of what tempts you to alert. A single model call failing is robotic, because the next fallback level handles it. A step that re-runs and then succeeds is robotic. Keep the pager for conditions where a person has to make a decision the pipeline cannot make for itself.

Four signals, read at the cadence of the job

The four golden signals of monitoring are latency, traffic, errors, and saturation. A daily job makes them awkward: traffic is one run, and an error rate measured over a single sample is either zero or one. Convert them into counts. Traffic becomes publications completed, errors become steps that exhausted every option, latency becomes the wall-clock span of the run, and saturation becomes how much retry and fallback headroom that run consumed.

Probe frequency deserves the same restraint. For a web service targeting no more than 9 hours aggregate downtime per year, which is 99.9% annual uptime, the SRE book notes that probing for a 200 success status more than once or twice a minute is probably unnecessarily frequent. A publisher that fires once a day needs even less black-box polling than that. What it needs is a dependable record of whether today's run produced its output.

Fallback consumed is a saturation reading

A resilient publisher hides its own trouble. The matthieupesesse.com engine has 3 model fallback levels, and an article that lands via the third level is still an article published daily. White-box monitoring exists for exactly this: it allows detection of imminent problems, failures masked by retries, and so forth. Record which level served each run. Output alone cannot distinguish a healthy day from the last good day before the pipeline stops producing anything.

That makes fallback depth the most useful number on the board. Level one serving every run means there is nothing to do. The last level serving run after run means the headroom is gone, and the next upstream problem publishes nothing at all. Changing provider, prompt or budget is a human decision, so a sustained stay at the final level is a legitimate page, while any single fallback event is not.

Give retries a budget so they cannot organise against you

Retries are where automated recovery turns into the fault. A cascading failure is a failure that grows over time as a result of positive feedback, and unsynchronised retries feed it: if retries aren't randomly distributed over the retry window, a small perturbation such as a network blip can cause retry ripples to schedule at the same time. Always use randomised exponential backoff when scheduling retries, including inside a pipeline that only runs once a day.

Then cap the total. The guidance suggests a server-wide retry budget, for example allowing only 60 retries per minute in a process, and if the retry budget is exceeded, not retrying and simply failing the request. That budget doubles as an alert condition. Exceeding it is not a robotic situation: it means the pipeline has stopped absorbing failure and started amplifying it, and the failures you now see are deliberate rather than incidental.

Two anomalies the schedule produces by itself

Twice a year the clock manufactures work that no error counter will attribute correctly. Per the crontab manual, non-existent times such as the missing hours during the daylight savings time conversion will never match, causing jobs scheduled during those missing times not to be run, while times that occur more than once will cause matching jobs to be run twice. A skipped run and a duplicate run are traffic anomalies, not errors. Set CRON_TZ deliberately, since it specifies the time zone specific for the cron table.

Absence is the condition that absence-based alerting exists for. A run that never starts emits no errors and no logs, and cron will consider a crontab at least partially broken if the last entry is missing a newline and is terminated by EOF. Nothing inside the pipeline can report either case. Expect the day's publications by a deadline and page when the count falls short, because the signal is the missing output rather than a message from the job.

Zero manual intervention describes the normal day, not a guarantee about every day. Tatano Energy requires 0 manual intervention and the personal platform takes around 0 h of weekly content operations, which is precisely why the surviving alerts have to be few and meaningful: output short of its count by the deadline, retry budget exceeded, fallback depth stuck at the last level. Everything else belongs on a dashboard that nobody is obliged to read.

Sources

Google SRE Book — Monitoring Distributed Systems — https://sre.google/sre-book/monitoring-distributed-systems/

Google SRE Book — Addressing Cascading Failures — https://sre.google/sre-book/addressing-cascading-failures/

man7.org (cronie) — crontab(5) Linux manual page — https://man7.org/linux/man-pages/man5/crontab.5.html

Neurolinks case study — Four markets, one codebase — https://neurolinks.be/work/tatano-energy

Neurolinks case study — A personal brand that publishes itself — https://neurolinks.be/work/matthieu-pesesse-media