Your next pipeline will be called by an agent

It is 2:40 on a Tuesday morning and the nightly load for the finance mart has failed at step 14 of 22.

In most companies, what happens next depends on who is on call. An engineer gets paged, opens a runbook last updated in 2023, works out from memory that step 14 can be rerun safely but step 9 cannot, checks that the upstream file actually arrived, reruns, and goes back to bed at 4:15. The knowledge that made that safe is in their head.

Now imagine an agent is on call instead. It can read logs. It can call APIs. It is fast and does not get tired. What it cannot do is guess which step is safe to rerun. Unless the pipeline tells it.

That is the whole question of agent-ready data, and most enterprises are about to discover they migrated their pipelines without answering it.

Why are migration programs and agent programs on a collision course?

Most large companies are running two programs that have not met yet.

One is the warehouse migration: off Teradata, SQL Server, SSIS or Hadoop, onto Databricks, Microsoft Fabric or Snowflake. It has a budget, a steering committee and a date.

The other is the agent program: assistants that answer questions over company data, agents that triage incidents, workflows that act on their own and escalate when unsure. It has a different budget, a different committee and, usually, a demo that worked beautifully on a clean dataset.

They collide because every agent that does useful work on enterprise data eventually needs to operate a pipeline: rerun a load, backfill a partition, trace where a number came from, check whether today's data is complete before answering. If the pipelines being migrated this year cannot be operated by an agent, the agent program will pay to rebuild them next year.

What does an agent need from a data pipeline?

An agent-ready pipeline carries all five in a form a machine can read.

What is MCP, and how does it apply to pipelines?

The important line is the description. It carries exactly the knowledge the on-call engineer had at 2:40 in the morning, and it travels with the pipeline. The agent learns what it can do from the tool itself, with no custom integration written for that one pipeline, and the server enforces the same access controls, approvals and audit logging that apply to a person.

MCP does not make a pipeline agent-ready by itself. The operations still have to be exposed deliberately, through a server with authorization and execution controls. That is engineering work, and it is far easier to do while the pipeline is being rebuilt than after.

{
  "name": "rerun_failed_step",
  "description": "Reruns a single failed step of the finance_mart_nightly pipeline. Steps 9 and 17 are not idempotent and require human approval before rerun. Returns the run id and the step's row counts.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "run_id": {"type": "string", "description": "The failed run to resume"},
      "step": {"type": "integer", "description": "Step number to rerun"}
    },
    "required": ["run_id", "step"]
  }
}

Why should this happen during the migration, not after it?

The obvious plan is migrate first, make it agent-ready later. It is the expensive plan, for two reasons.

The knowledge is produced during the migration, then thrown away. A migration is the one moment someone reads every procedure, maps every dependency and rebuilds every job. Which steps are safe to rerun, what each one reads and writes, what breaks downstream: all of it is discovered on the way through. Discard it at cutover and you pay to rediscover it the first time an agent needs it.

Hand-written pipelines do not share a shape. Each one encodes its author's choices about naming, error handling and retries. An agent cannot learn one convention and apply it everywhere, because there is no one convention. Deterministic conversion produces the opposite: the same structure and the same operations across thousands of pipelines, because the same rules produced them. Exposing consistent pipelines as tools is a pattern applied once. Exposing inconsistent ones is a project per pipeline.

What does "agentically primed" mean at Novuz?

We describe it as two separate things, and we keep them apart on purpose.

Agents can run Novuz. The platform is headless: everything the interface does is available through APIs. Your orchestrator, your CI/CD pipeline, a partner's delivery tooling or an AI agent can trigger discovery, conversion, validation and deployment, inspect the results and wait for a human approval before the next step.

Agents can run what Novuz builds. The pipelines we deliver on Databricks, Snowflake or Microsoft Fabric can be exposed as MCP tools your agents discover and call, under your access controls, approvals and logs.

What sits between those two is not an agent. The conversion is deterministic: agents decide what to ask for and when, and the engine gives the same output for the same input, so every result can be tested, audited and repeated.

Isn't determinism less important once agents are involved?

It is more important. An agent that reruns a step expects the same input to produce the same output. If the pipeline underneath was produced by a process that nobody can prove matches the original, the agent is now automating an unverified result at machine speed, and it will do so confidently. Agents raise the stakes on proof; they do not lower them.

Five questions to put to any migration plan

If the answers are no, the migration is building the next migration.

1. Will the migrated pipelines expose their operations to agents, or only to people?
2. Will the dependency map and lineage survive cutover in a form a machine can read?
3. Can the migration itself be driven from your own tooling and approval workflow, or only from a vendor's console?
4. When an agent reruns a step, does the same input give the same output, and can you show the evidence?
5. Will an agent's actions be logged and approved exactly like a person's?