Eight a day in seven languages: the publishing schedule is your eval distribution

An unsupervised generative pipeline has a task distribution you can read off a schedule. The Tatano Energy platform we built publishes 8 SEO articles every day across 4 country domains in 7 languages, with manual intervention required: 0. An eval set for that pipeline should carry the same shape. Most eval sets carry a different one: whatever was convenient to write, in one language, on the day someone had a spare hour.

The distribution is already in the schedule

Anthropic's evaluation guidance is to design evals that mirror your real-world task distribution. For a daily publisher that distribution is not a research question; it is the plan: how many pieces, in which languages, for which markets. Tatano Energy serves 7 languages and has 4 country domains indexed separately. A suite written entirely in one language measures one slice of the work and then reports it as the whole.

Weighting matters as much as coverage. A set with a single token question per secondary language will pass on volume and tell you nothing about the languages where output is thinnest. Build the set so that the share of questions per language and per market tracks the share of published output, then keep that mapping visible when either the languages or the markets change.

Volume forces automated grading

The same guidance states that more questions with slightly lower-signal automated grading are better than fewer questions with high-quality human hand-graded evals. On the Matthieu Pesesse platform, articles publish autonomously daily, and weekly content operations time is ~0 h. At that cadence the preference stops being a trade-off and becomes the only available option. Hand grading scales with the hours a person has. A daily schedule does not wait for those hours.

Write the questions so a machine can grade them

Structure questions to allow for automated grading: multiple-choice, string match, code-graded or LLM-graded. That choice is per dimension, not per suite. Whether output is in the target language is a code check. Whether the identifiability marker is present is a code check, and under the AI Act, providers of generative AI have to ensure that AI-generated content is identifiable. Whether a claim is grounded in retrieved text is LLM-graded.

A question like 'is this article good' has no grader. It has to be decomposed into assertions something can check: the target language, the required structure, the marker, each factual sentence traceable to a retrieved passage. If a dimension cannot be expressed as one of those four grading shapes, it will quietly become a human review step, and that step is what the schedule removed.

One average hides the failure you care about

Most use cases need multidimensional evaluation along several success criteria. For a multilingual publisher those criteria are at least the target language, grounded claims, required structure and identifiability. Collapsing them into one score lets a strong grounding result cover a run that shipped the wrong language to one domain. Gate each dimension separately, so a failure names itself instead of being averaged into a pass.

With zero manual intervention, the eval is the reviewer

OWASP describes misinformation as false or misleading information that appears credible, and overreliance as users placing excessive trust in generated content without verifying its accuracy. Its mitigations are concrete: use retrieval-augmented generation against trusted external sources, and implement tools and processes to automatically validate key outputs, especially in high-stakes environments. When manual intervention required is 0, that automatic validation is not a safety net behind a reviewer. It is the reviewer.

Before expanding a generative pipeline, write the distribution down: pieces per day, languages, markets. Then decide the grader for each dimension before writing any questions, and keep the dimensions separate in the result. A suite built that way can grow with the output instead of being rewritten each time a language or a market is added, and it tells you which dimension failed rather than that something did.

Sources

Claude Platform Docs (Anthropic) — Define success criteria and build evaluations — https://platform.claude.com/docs/en/test-and-evaluate/develop-tests

OWASP Gen AI Security Project — LLM09:2025 Misinformation — https://genai.owasp.org/llmrisk/llm092025-misinformation/

European Commission (Shaping Europe’s digital future) — AI Act — https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai

Neurolinks case study — Four markets, one codebase — https://neurolinks.be/work/tatano-energy

Neurolinks case study — A personal brand that publishes itself — https://neurolinks.be/work/matthieu-pesesse-media