Broadcast Traffic Scheduling Automation

Configuring Make-Good Triggers Based on Ratings

Post-air telemetry from Nielsen or Comscore routinely exposes soft under-delivery: a spot cleared cleanly, the log reconciled, yet the campaign fell short of its contracted GRP or impression guarantee. Nobody preempted anything — the audience simply did not show up. This guide solves one exact task: turning a post-air ratings payload into a deterministic, auditable make-good trigger that fires only when delivery drops below a contractual floor, respects a compliance boundary, and emits an idempotent order the traffic system can act on without a human in the loop. It is the ratings-armed entry point into Automating Make-Good Routing for Preemptions, itself a bounded subsystem of the broader Spot Scheduling Validation & Rule Engines architecture. Getting the trigger wrong is not cosmetic: a threshold set too tight floods the desk with make-goods on normal measurement noise, and one set too loose silently eats an SLA breach that surfaces later as a billing dispute the audit trail cannot defend.

Unlike a hard preemption — where an Emergency Alert System activation or a sports overrun physically displaces a spot and the make-good debt is instantaneous — a ratings trigger evaluates statistical under-delivery across a multi-day measurement window. That difference drives the design: the trigger is asynchronous (it runs on the T+1/T+2 data drop, not at air), it must model tolerance rather than test equality, and it must distinguish a routine short-fall that warrants a make-good from a catastrophic miss that is a contractual event in its own right. Everything below assumes the delivered record already carries a resolved spot schema and a normalized billing code, so the trigger reasons about delivery, not data hygiene.

Prerequisites

  • Python 3.11+ — required for the model_validator semantics and timezone-aware datetime.now(timezone.utc) comparisons used below.
  • Pinned dependencies — pydantic==2.7.1 for the ratings payload contract. Pin exactly; Pydantic v2 validator signatures shift across minor releases.
  • A ratings feed — authenticated read access to the Nielsen/Comscore post-air drop (or your internal impression warehouse), delivering actual_grps keyed by spot_id and market.
  • Traffic-system write scope — a service token scoped to emit make-good orders, plus read access to the inventory resolver that answers “what open, compliant windows exist in this market?”.
  • A tolerance policy store — a version-controlled YAML file or a policy table holding per-tier thresholds and cure windows, reloadable without a restart.
  • A Redis-backed idempotency store — a key store with TTL matching the contractual cure window, so a retried telemetry drop cannot mint a duplicate make-good.

Step-by-Step Implementation

The trigger is a pure decision function wrapped in an orchestration shell: a validated payload is scored into a delivery ratio, the ratio is classified against a tolerance band, and only a genuine short-fall proceeds to inventory resolution and order emission. Everything else — on-target delivery, missing data, or a compliance-grade miss — exits without routing and is logged for reconciliation.

Ratings-armed make-good trigger decision tree A post-air ratings payload is scored into a delivery ratio. If the ratio meets the tier tolerance, the spot is WITHIN_TOLERANCE and no action is taken. Otherwise, if the ratio is below the 0.50 compliance floor the miss is a COMPLIANCE_VIOLATION that is escalated rather than auto-routed. A ratio between the floor and the tolerance band is TRIGGERED and proceeds to inventory resolution: a compliant window emits an idempotent make-good order, and no window yields ROUTING_FAILED for backoff and human review. Post-air ratings payload Compute delivery ratio actual ÷ contracted ratio ≥ tier tolerance? yes WITHIN_TOLERANCE no action no ratio < floor (0.50)? yes COMPLIANCE_VIOLATION escalate · do not auto-route no TRIGGERED resolve inventory compliant window? yes Emit idempotent make-good order no ROUTING_FAILED backoff + human review

Figure — The ratings trigger classifies each delivery ratio into on-target, compliance-grade under-delivery, or a routable short-fall, and only the last path emits a make-good order.

Step 1 — Structured audit logging and the tolerance policy

Goal: emit machine-parseable audit lines in the traffic-ops timestamp | level | module | spot_id shape, and externalize the thresholds so ratings volatility is absorbed by tolerance bands rather than a single hardcoded cutoff. Hard cutoffs at 100% delivery generate excess make-good volume on normal measurement variance; graduated bands scale the tolerance to market size and demographic granularity.

python 
import hashlib
import logging
import sys
import uuid
from datetime import datetime, timedelta, timezone
from enum import Enum
from typing import Any, Optional

# Audit trail: timestamp | level | module | spot_id-bearing message
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
    handlers=[
        logging.StreamHandler(sys.stdout),
        logging.FileHandler("makegood_trigger_audit.log", mode="a", encoding="utf-8"),
    ],
)
logger = logging.getLogger("makegood_trigger")

# Per-tier delivery thresholds. The band widens as measurement error grows:
# national panels are stable, niche demographics are noisy and need slack.
TOLERANCE_BANDS: dict[str, float] = {
    "NATIONAL": 0.95,   # +/- ~2% panel error
    "REGIONAL": 0.94,
    "DMA": 0.92,        # +/- ~4% error at local grain
    "NICHE": 0.90,      # rolling 7-day averaging recommended upstream
}
# Below this ratio the miss is not a make-good, it is a contractual event.
COMPLIANCE_FLOOR: float = 0.50

Expected log line: 2026-07-03 02:14:07,881 | INFO | makegood_trigger | policy loaded | spot_id=- bands=4 floor=0.50

Step 2 — The Pydantic ratings payload contract

Goal: attach validation to the record itself so a malformed or stale drop fails closed before it can score. A Pydantic validator enforces a positive contracted baseline, a timezone-aware air time, and a staleness guard — a payload whose air window closed more than 48 hours ago is quarantined, never used to route a retroactive make-good that would violate pacing.

python 
from pydantic import BaseModel, Field, model_validator


class RatingsPayload(BaseModel):
    spot_id: str = Field(..., description="Traffic-log identifier of the under-delivered spot")
    market: str = Field(..., description="Market / tier key, e.g. NATIONAL or a DMA code")
    tier: str = Field(..., description="Tolerance tier: NATIONAL | REGIONAL | DMA | NICHE")
    aired_end: datetime = Field(..., description="UTC end of the measured air window")
    contracted_grps: float = Field(..., gt=0, description="Guaranteed GRPs from the order")
    actual_grps: Optional[float] = Field(default=None, description="Measured GRPs; None if not yet reported")
    cure_window_days: int = Field(default=14, ge=1, description="Contractual make-good cure window")
    client_id: str = ""
    trace_id: str = Field(default_factory=lambda: str(uuid.uuid4()))

    @model_validator(mode="after")
    def reject_stale_or_untyped(self) -> "RatingsPayload":
        # Air time must be timezone-aware UTC; a naive datetime is a data fault.
        if self.aired_end.tzinfo is None:
            raise ValueError(f"aired_end must be timezone-aware | spot_id={self.spot_id}")
        age = datetime.now(timezone.utc) - self.aired_end
        if age > timedelta(hours=48):
            raise ValueError(f"stale telemetry {age} | spot_id={self.spot_id}")
        if self.tier not in TOLERANCE_BANDS:
            raise ValueError(f"unknown tolerance tier {self.tier} | spot_id={self.spot_id}")
        return self

Expected log line (on rejection): the raised ValueError is caught by the caller and logged as ... | WARNING | makegood_trigger | quarantine stale | spot_id=NAT-77213 age=51:12:04.

Step 3 — The delivery-ratio classifier

Goal: collapse the decision into a single deterministic function returning a status and the ratio that produced it. The order of the checks is load-bearing: on-target delivery short-circuits first, the compliance floor is tested before the routable band, and missing data defers rather than guessing.

python 
class TriggerStatus(Enum):
    WITHIN_TOLERANCE = "within_tolerance"
    INSUFFICIENT_DATA = "insufficient_data"
    COMPLIANCE_VIOLATION = "compliance_violation"
    TRIGGERED = "triggered"
    ROUTING_FAILED = "routing_failed"


def classify_delivery(payload: RatingsPayload) -> tuple[TriggerStatus, float]:
    """Score a payload into a trigger decision. Pure and side-effect free."""
    if payload.actual_grps is None:
        # No measurement yet: defer to the next drop, never route on a guess.
        return TriggerStatus.INSUFFICIENT_DATA, 0.0

    ratio = payload.actual_grps / payload.contracted_grps
    threshold = TOLERANCE_BANDS[payload.tier]

    if ratio >= threshold:
        return TriggerStatus.WITHIN_TOLERANCE, ratio
    if ratio < COMPLIANCE_FLOOR:
        # A catastrophic miss is a contractual event, not an auto-routed make-good.
        return TriggerStatus.COMPLIANCE_VIOLATION, ratio
    return TriggerStatus.TRIGGERED, ratio

Expected log line: 2026-07-03 02:14:08,004 | INFO | makegood_trigger | classified | spot_id=NAT-77213 status=triggered ratio=0.812

Step 4 — Idempotent inventory resolution and order emission

Goal: for a TRIGGERED payload, ask the inventory resolver for a compliant replacement window, stamp a deterministic idempotency key so a retried drop cannot double-book, and emit the order. A resolver that returns nothing is not an error to swallow — it escalates to ROUTING_FAILED for backoff and human review. The engine holds the policy and resolver as injected collaborators so it stays unit-testable against fixtures.

python 
class RatingsMakeGoodEngine:
    """Evaluates post-air ratings and routes make-goods on genuine short-falls."""

    def __init__(self, inventory_resolver: Any) -> None:
        # inventory_resolver.find_windows(market, deficit_grps, priority) -> list[dict]
        self._resolver = inventory_resolver

    def _idempotency_key(self, payload: RatingsPayload) -> str:
        # Deterministic across telemetry retries: same spot + window + air = same key.
        raw = f"{payload.spot_id}:{payload.market}:{payload.aired_end.isoformat()}"
        return hashlib.sha256(raw.encode("utf-8")).hexdigest()

    def _resolve_window(self, payload: RatingsPayload, deficit_grps: float) -> dict[str, Any]:
        windows = self._resolver.find_windows(
            market=payload.market,
            deficit_grps=deficit_grps,
            priority=1,
        )
        if not windows:
            raise LookupError(f"no compliant window | spot_id={payload.spot_id}")
        # Resolver returns candidates ranked by pacing fit; take the best.
        return windows[0]

    def execute(self, payload: RatingsPayload) -> dict[str, Any]:
        status, ratio = classify_delivery(payload)
        audit = {
            "trace_id": payload.trace_id,
            "spot_id": payload.spot_id,
            "status": status.value,
            "delivery_ratio": round(ratio, 4),
            "decided_at": datetime.now(timezone.utc).isoformat(),
        }

        if status is TriggerStatus.WITHIN_TOLERANCE:
            logger.info("on target | spot_id=%s ratio=%.3f", payload.spot_id, ratio)
            return {"action": "none", "audit": audit}
        if status is TriggerStatus.INSUFFICIENT_DATA:
            logger.info("deferred | spot_id=%s reason=no_actual_grps", payload.spot_id)
            return {"action": "defer", "audit": audit}
        if status is TriggerStatus.COMPLIANCE_VIOLATION:
            logger.error("compliance miss | spot_id=%s ratio=%.3f", payload.spot_id, ratio)
            return {"action": "escalate", "audit": audit}

        deficit = payload.contracted_grps - (payload.actual_grps or 0.0)
        try:
            window = self._resolve_window(payload, deficit)
        except LookupError as exc:
            logger.critical("routing failed | spot_id=%s error=%s", payload.spot_id, exc)
            audit["status"] = TriggerStatus.ROUTING_FAILED.value
            return {"action": "failed", "audit": audit}

        order = {
            "order_id": f"MG-{uuid.uuid4().hex[:8].upper()}",
            "source_spot_id": payload.spot_id,
            "client_id": payload.client_id,
            "market": payload.market,
            "deficit_grps": round(deficit, 3),
            "scheduled_start": window["start"],
            "scheduled_end": window["end"],
            "idempotency_key": self._idempotency_key(payload),
            "trace_id": payload.trace_id,
            "emitted_at": datetime.now(timezone.utc).isoformat(),
        }
        logger.info("make-good emitted | spot_id=%s order=%s deficit=%.2f",
                    payload.spot_id, order["order_id"], deficit)
        audit["order_id"] = order["order_id"]
        return {"action": "routed", "order": order, "audit": audit}

Expected log line: 2026-07-03 02:14:08,120 | INFO | makegood_trigger | make-good emitted | spot_id=NAT-77213 order=MG-9A3F1C02 deficit=28.40

Verification & Testing

Confirm four invariants against fixtures that mirror a real post-air drop: on-target delivery routes nothing, a genuine short-fall emits exactly one order, a catastrophic miss escalates instead of routing, and a missing measurement defers. Drive the engine with a stub resolver so the test is deterministic and offline.

python 
class _StubResolver:
    def __init__(self, windows: list[dict]) -> None:
        self._windows = windows

    def find_windows(self, **_: Any) -> list[dict]:
        return self._windows


def _payload(actual: Optional[float], tier: str = "NATIONAL") -> RatingsPayload:
    return RatingsPayload(
        spot_id="NAT-77213", market="NATIONAL", tier=tier,
        aired_end=datetime.now(timezone.utc) - timedelta(hours=6),
        contracted_grps=150.0, actual_grps=actual, client_id="ACME",
    )


def test_trigger_decisions() -> None:
    windows = [{"start": "2026-07-10T20:00:00Z", "end": "2026-07-10T20:00:30Z"}]
    engine = RatingsMakeGoodEngine(_StubResolver(windows))

    # 146/150 = 0.973 >= 0.95 national threshold: on target, no action.
    assert engine.execute(_payload(146.0))["action"] == "none"

    # 122/150 = 0.813: a routable short-fall emits one idempotent order.
    routed = engine.execute(_payload(122.0))
    assert routed["action"] == "routed"
    assert routed["order"]["order_id"].startswith("MG-")

    # 60/150 = 0.40 < 0.50 floor: escalate, never auto-route.
    assert engine.execute(_payload(60.0))["action"] == "escalate"

    # No measurement yet: defer to the next drop.
    assert engine.execute(_payload(None))["action"] == "defer"


def test_idempotency_key_is_stable() -> None:
    engine = RatingsMakeGoodEngine(_StubResolver([]))
    p = _payload(122.0)
    assert engine._idempotency_key(p) == engine._idempotency_key(p)

The stable-key assertion is the load-bearing one: because the idempotency key is a hash of spot_id, market, and air time, a telemetry drop replayed by the vendor produces the identical key, and the Redis store rejects the duplicate before a second make-good is ever booked.

Edge Cases & Failure Handling

Stale or replayed telemetry. Nielsen/Comscore drops are re-published when a panel correction lands, so the same spot_id can arrive twice with different actual_grps. The 48-hour staleness guard in Step 2 rejects late replays outright, and the deterministic idempotency key blocks a duplicate order for anything inside the window. Never widen the staleness bound to “catch up” on a backlog — a retroactive make-good against a closed pacing window corrupts the very accounting the trigger exists to protect. Route stale records to the same quarantine queue used across make-good routing for preemptions and reconcile by hand.

Inventory deadlock. When the resolver returns zero compliant windows, the engine returns ROUTING_FAILED rather than relaxing constraints to force a placement. Wrap the resolver call in exponential backoff with jitter; after three failed attempts, escalate to a human-in-the-loop dashboard carrying the pre-computed deficit_grps and alternative dayparts. A window that clears the deficit but violates competitive separation is not a valid fallback — that collision is exactly what time slot conflict detection exists to catch, and the resolver must honor it.

Threshold drift and false triggers. A tier whose threshold is set from stale panel assumptions either floods the desk or masks real misses. Treat TOLERANCE_BANDS as live policy, not code: reload it atomically from the policy store so a mid-batch edit never applies half a change, and review the band the same disciplined way you would when adjusting thresholds for automated scheduling — shadow the new value against a historical drop, confirm the make-good rate moves the way you expect, then promote it behind a flag.

FAQ

Why is a 40% delivery ratio escalated instead of auto-routed as a make-good?

Because a miss that deep is rarely a normal short-fall the market can absorb with compensatory inventory — it usually signals a measurement fault, a mis-tagged spot, or a genuine breach that triggers a contractual clause (credit, cash-back, or renegotiation) rather than a routine make-good. Auto-routing it would silently consume scarce inventory to paper over a problem that a human needs to see. The COMPLIANCE_FLOOR sends it to escalation with the ratio attached, and the traffic desk decides.

How does this differ from a make-good triggered by an actual preemption?

A preemption make-good is event-driven and instantaneous: an EAS activation or sports overrun physically displaces a spot, and the debt is known at air. This ratings trigger is asynchronous and statistical — it runs on the T+1/T+2 post-air drop and reasons about a delivery ratio over a measurement window. Both funnel into the same routing engine documented in Automating Make-Good Routing for Preemptions; only the arming signal differs.

Where should the tolerance thresholds actually live?

Not in the source. Externalize TOLERANCE_BANDS and each tier’s cure window to a version-controlled YAML file or a policy table the engine reloads atomically without a restart. That keeps a threshold change auditable, reversible, and out of a deploy — the same separation of policy from mechanism used when tuning thresholds for scheduling accuracy.

What stops a re-published ratings drop from creating a duplicate make-good?

The idempotency key — a SHA-256 of spot_id, market, and air time — is deterministic, so a replayed drop hashes to the identical key. Persist that key in a Redis store with a TTL matching the contractual cure window; the traffic system rejects any order whose key already exists. Combined with the 48-hour staleness guard, a vendor re-publish can never mint a second order for the same air.

Related content

Up one levelAutomating Make-Good Routing for Preemptions