If AI makes the same mistake repeatedly in the same project, the problem is usually not just the model.

The task contract may be unclear. The context package may miss a source file. Stop conditions may be absent. Tests may not cover the behavior. The evidence gate may be too weak. Project rules may exist only in chat.

The final layer of a project-specific AI delivery pipeline is feedback and knowledge capture.

Do not only correct AI in chat

Many people tell the agent, “do not do that next time.”

That is almost no process improvement.

The next session, tool, model, or project directory may not carry that lesson. Even if the model remembers, it may not know which project, scenario, or file scope the lesson belongs to.

Real project memory should enter project mechanisms.

Classify the failure first

After a failure, do not immediately add another rule. Classify the failure.

Common classes include:

  • unclear requirement;
  • wrong context;
  • wrong source of truth;
  • excessive permission;
  • unsafe tool call;
  • missing test;
  • missing evidence;
  • unclear release boundary;
  • source-specific lesson generalized incorrectly;
  • project rule not loaded by the AI.

Once the failure is classified, the project can decide what to fix.

Lessons belong in different layers

Not every lesson should become a skill.

Some lessons belong in the issue template because they affect task intake.

Some belong in AGENTS.md because they are project-wide rules.

Some belong in a skill because they are workflow-specific.

Some belong in tests or fixtures because machines should verify them.

Some belong in an evidence gate because they define completion.

Some belong in scripts because text reminders are not strong enough.

Some should remain project-local and should not be promoted to central knowledge.

Only stable cross-project lessons should become central knowledge.

Skill is not a trash bin

Skill files can become another messy documentation folder.

If every AI mistake adds one more sentence to a skill, the skill becomes longer, more contradictory, and harder to trigger.

Good rules should live in the narrowest verifiable layer.

If it is code behavior, use a test.
If it is execution process, use a script or state machine.
If it is task entry, use an issue template.
If it is a project-wide boundary, use AGENTS.md.
If it is a workflow manual, use a skill.
If it is a cross-project method, use the central knowledge base.

The role of the central knowledge base

The central knowledge base should not take over every project detail.

It should capture stable cross-project patterns: task contracts, evidence contracts, context packages, dynamic skill lifecycle, publication boundaries, release boundaries, and similar methods.

Project repositories should keep project facts, private materials, source-specific interpretation, and concrete implementation details.

That keeps the central knowledge base as the method layer while project pipelines retain execution authority.

Conclusion

An AI delivery pipeline should not only execute tasks. It should learn how the project can use AI better next time.

A repeated failure should not remain a complaint. It should become a clearer contract, better context, stronger evidence, narrower permissions, a better gate, or a more reliable project rule.

That is why a pipeline is stronger than chat: it turns one experience into structure for the next run.