MTMoueen Togarvi5 min read

AI-Native Application Development: Designing Around Uncertainty

Quick overview

Updated

Build AI-native applications properly: designing for probabilistic output, interface patterns that expose uncertainty, evaluation as a build gate, and cost architecture.

AI-native application development means building software whose core behavior depends on model output rather than software with a model bolted on. The distinction matters because the engineering practices differ, particularly around correctness, testing, and interface design.

The defining constraint is that a central component returns a distribution of plausible outputs rather than a determined answer.

Design for a distribution, not an answer

Conventional software also has incomplete requirements and uncertain inputs. A generative-model feature adds another source of variation: plausible outputs can differ across runs, and fluent wording does not establish factual correctness.

That means the question during design is not "does it work" but "what happens on the tail". For every model-driven feature, specify the acceptable error rate, what a wrong output costs, how a user detects one, and how they recover.

Features where a confident wrong answer is expensive and undetectable are poor candidates. Features where the user naturally verifies the output, or where errors are cheap to correct, are good ones.

Interface patterns that work

  • Show the source. Presenting the evidence alongside a generated answer lets users verify without trusting blindly, and it converts an opaque assertion into a checkable one.

  • Make correction cheap. Editing a draft is far better than regenerating it. Users tolerate imperfect output when fixing it is faster than starting over.

  • Expose uncertainty honestly where you can measure it, and decline rather than guess when confidence is low. A system that says it does not know retains more trust than one that fabricates.

  • Prefer suggestion over action for anything consequential, at least until evaluation evidence supports automation.

  • Keep an escape hatch to the deterministic path or a human, always available and obvious.

Evaluation is a build gate, not a QA phase

Use unit tests for permissions, calculations, and state transitions, alongside evaluations for model behaviour. Anthropic's evaluation guide describes defining success criteria and selecting grading methods for them. A valid JSON response, a correct factual answer, and an acceptable customer experience require different checks.

Build the set from permitted real examples and deliberately constructed edge cases: ambiguous requests, missing data, adversarial content, and reported failures. Synthetic examples can extend coverage, but compare their difficulty and distribution with actual use before treating their scores as representative.

Set thresholds that block deployment. Without a gate, quality drifts silently, particularly across model version changes that you did not initiate.

Use production feedback to propose new evaluation cases. Review corrections before treating them as labels: a user may express a style preference, omit context, or introduce another factual error. Keep personal information out of shared test fixtures unless its use is authorised and necessary.

Architect for model change

Assume the model will be replaced, deprecated, repriced, or updated underneath you. Isolate model interaction behind an interface so swapping providers is a contained change rather than a rewrite.

Keep prompts, tool definitions, and evaluation sets in version control alongside code, with the same review process. These are program logic, not configuration.

Pin model versions where the provider allows it, and treat a version change as a deployment requiring evaluation, not an automatic upgrade.

Cost is an architectural concern

Cost depends on request volume, model inputs and outputs, retries, tools, storage, and review work. A pilot's short inputs or heavy manual assistance can make its apparent cost unrepresentative of production.

Model cost per successfully completed user action. Compare smaller models where they meet the required quality. Cache only when permission scope, source freshness, and invalidation are understood, and test context trimming so it does not discard evidence necessary for the task.

Instrument cost per feature and per user from the first release, not after the bill arrives.

Data boundaries

Decide what may enter prompts, logs, traces, and vendor systems before building. Every one of those is a copy of the data with its own access and retention implications.

Minimize what is sent. Redact where possible. Confirm the provider's retention and training terms in writing, and record the decision for audit.

Example: build a service-enquiry assistant with a reviewable result

A hypothetical agency wants to turn a customer's brief into a project-scope draft. Begin with fields such as business goal, required integrations, delivery constraints, and unanswered questions. Retrieve approved service descriptions and label unknown details explicitly. Do not let the model invent a delivery commitment or price from an incomplete brief.

The interface shows the original brief beside the proposed scope, with editable fields and links to the service information used. Saving a draft is distinct from sending a proposal. The send action validates the recipient and uses the user's confirmed version, so a later regeneration cannot silently change an approved proposal.

Evaluate extraction accuracy, unsupported commitments, useful follow-up questions, editing time, and successful saves. Keep permission checks and recipient validation deterministic. NIST's Generative AI Profile includes confabulation and human-AI interaction risks; this example translates those concerns into product behaviour a reviewer can inspect.

Release the smallest useful draft workflow before adding autonomous research or persistent memory. Compare it with the current manual process on a held-out set of briefs. A shorter completion time is useful only if the review shows that scope accuracy and customer handling remain acceptable.

Frequently asked questions

How is this different from adding an AI feature?

Scale and centrality. When the model drives core behavior, evaluation, cost, and failure handling become architectural concerns rather than feature details.

Which release gap should we check first?

Check whether you have a reproducible baseline and evaluation set. Without those, attributing a regression to a code, prompt, corpus, or model change becomes much harder.

Should we fine-tune?

First identify the measured failure. Missing current knowledge may call for retrieval; invalid actions need better controls. Fine-tuning is worth evaluating for a defined behavioural task with suitable examples and a baseline, rather than as a general repair for every quality issue.

How do we handle model deprecation?

Isolate the integration, pin versions, keep an evaluation suite, and budget for periodic migration as a routine cost.

If the product needs continuity between sessions, review agent memory architecture before adding persistent state. Explore our software and web capabilities or discuss your product constraints.

MT

Written by

Moueen Togarvi

Founder & CEO at Voquarn Code, focused on product engineering, search growth, and practical AI systems.

About author
Turn the insight into action

Need a practical plan for your next digital project?

Tell us what you are building. We will help you clarify the scope, technical approach, and highest-value first step.

Discuss your project