AI audits·9 min read

What an AI UX audit actually is (and what it can't catch)

Four very different technologies sell under one label. The difference that matters: does the AI look at your product, or does it use it? Receipts included, ours too.

An AI UX audit uses artificial intelligence to find usability problems in a digital product, in minutes instead of weeks. The term hides four very different methods, and the difference that matters is simple: does the AI look at your product, or does it use it?

What is an AI UX audit?

An AI UX audit is an automated review of a website or app that finds usability problems and ranks them by severity, without recruiting human testers. A useful one behaves like the audit a consultant would run: it checks the product against established rules (Nielsen’s 10 heuristics, WCAG 2.2, the Laws of UX), attaches evidence to every claim, and tells you what to fix first. A bad one prints twenty generic observations about whitespace and calls it insight.

The category is young and the label is doing a lot of work. Four different technologies currently sell under it, and they are not interchangeable.

The four kinds of AI UX audit

Every tool in this space belongs to one of four families, defined by what the AI actually receives as input: an image, a persona, or a browser session.

FamilyWhat the AI doesExample toolsPublished priceHonest use
Attention predictionPredicts where eyes land on a static design, from eye-tracking training dataAttention InsightFrom €29/moPre-launch layout checks; no usability findings
Screenshot reviewAn LLM critiques page captures against guideline librariesUXAudit.Now, uxaudit.appFree tier; Pro $199/moQuick static pass; misses interaction issues (see below)
Synthetic usersLLM personas answer interview and survey questions about your productSynthetic Users$2 to $60 per interviewHypothesis generation; never decisions (NN/g’s line, below)
Agent-based auditA browser agent signs up, clicks, submits, breaks flows, and reports what it hitClapBack, Loop11 agents, UxiaClapBack one-time from $49; Loop11 from $179/moFinding interaction and flow defects with evidence

Academia arrived at the same split, and treats interaction as the point rather than an implementation detail. The papers behind that claim (UXAgent, WebProber, UXBench) are walked through in our comparison of agents, synthetic users, and real users.

Can AI actually do a UX audit?

Yes for finding rule-checkable defects, no for replacing human research, and it depends entirely on which of the four families you picked. The screenshot family has a public report card, and it is rough.

80%
false-positive rate when Baymard Institute tested ChatGPT-4 as a UX auditor across 12 e-commerce pages in their October 2023 study. Six human professionals found 257 verified issues; the model produced 178 suggestions, mostly noise.

The detail that matters more than the headline: the model caught 26% of the issues visible in the screenshot but only 14% of the issues on the live page, because roughly half of real problems are interaction-gated, invisible to any still image. The full autopsy of that study, including why the free audit worked out to a negative hourly wage, is in our ChatGPT UX audit prompts piece.

The roast

A screenshot audit grades your product the way you’d grade a restaurant from a photo of the menu.

The receipt

Jakob Nielsen, reviewing the same data, counted about a dozen invented findings per screenshot and estimated that rejecting them costs more time than the audit saves. Flow problems fared worst: a model that sees one page at a time cannot see what breaks between pages.

That study is from 2023 and models have improved since. What has not changed is the structure of the problem: a model that receives a static capture cannot attempt your signup, trigger your error states, lose its progress in your onboarding, or watch your form silently reject valid input. The 2026 UXBench benchmark is built on exactly this premise: models must collect interaction evidence in a live browser before they are allowed to critique, and even then their reports differ meaningfully in quality. Interaction is the admission ticket, not a guarantee.

What an AI UX audit can’t catch

No AI method, agents included, can tell you why users want your product, whether your pricing feels fair, or what a confused human does at 11pm with a deadline. Those answers only come from real people. Nielsen Norman Group’s position on synthetic research is blunt: “UX without real-user research isn’t UX.” We agree, and we sell one of these tools.

The honest division of labor: an agent testifies, it does not opine. It can verify that your delete button has no undo, that your onboarding loses progress on reload, that your checkout contrast fails WCAG 1.4.3. It cannot feel frustration; it can only document what would cause it, with a named rule attached. For the full method-by-method comparison, including where synthetic users go wrong, see AI agents vs synthetic users vs real users.

AI UX audit vs UX audit vs usability testing

A UX audit is an expert review against rules; usability testing watches real users attempt tasks; an AI UX audit automates the first, never the second. They find different problems, which is why Nielsen recommended pairing expert evaluation with user testing back in 1995.

Human UX auditAI UX audit (agent-based)Usability testing
Who does the workA consultant or your teamA browser agent5+ real users per round
FindsRule violations, flow breaksRule violations, flow breaksBehavior, confusion, motivation
Blind spotsEvaluator fatigue and familiarityTaste, desirability, contextRare paths users don’t hit in-session
Typical timeline1 to 6 weeksMinutes, under an hour1 to 3 weeks per round
Typical cost$1,650 to $15,000 advertised$49 to a few hundredRecruitment alone can pass $100 per participant

The cost column deserves its own receipts; we compiled 20 published rate cards in what a UX audit really costs in 2026. The one-sentence version: the advertised market runs from $35 gig listings to $15,000 agency packages, and the extremes are not buying the same thing.

How an agent-based audit actually runs

The method behind ClapBack, minute by minute: the agent gets a URL and nothing else, then uses the product the way a first-time user with zero patience would. Everything it hits becomes a finding with a severity, a cited rule, a screenshot, and a suggested fix.

agent run · your-app.com
00:00Opens your-app.com in a clean browser profile
00:41Reads the landing page, locates the primary CTA
01:12Starts signup with a fresh email address▸ finding: Form demands company size before showing any product
03:05Submits the form with a deliberately weak password▸ finding: Error banner names no field
04:38Completes signup, lands in onboarding
06:20Abandons onboarding midway, then returns▸ finding: Progress lost, flow restarts from step 1
09:47Runs the core flow end to end, twice
14:31Tests every empty state, error state, and destructive action▸ finding: Delete has no confirm and no undo
19:02Writes findings: severity, rule, screenshot, fix
Report ready · findings sorted by severity, exported as tickets
A representative agent run. Findings appear where the product resisted, not where a checklist said to look.

If you are evaluating any tool in this category, ours included, four questions separate the serious from the decorative:

  • Does it interact? If the input is a screenshot, you are buying the 14% discovery tier.
  • Does every finding cite a checkable rule? A finding you can argue with is worth ten vibes.
  • Does it admit uncertainty? Hallucinated findings cost review time; Nielsen measured the audit going net-negative.
  • Does the output land where work happens? A report you have to transcribe into the backlog is homework; a finding that arrives as a ticket gets fixed. We wrote up the format in turning UX findings into tickets.

Four questions, one shortcut: paste your URL into ClapBack and see which ones we pass.

Receipts for this article
Baymard Institute · Testing ChatGPT-4 for UX Audits (Oct 2023) · Jakob Nielsen · Unreliability of AI in Evaluating UX Screenshots (Oct 2023) · UXBench · Measuring the Actionability of LLM-Generated UX Critiques (arXiv 2606.16262) · NN/g · Synthetic Users: If, When, and How (Jun 2024) · NN/g · Usability Problems Found by Heuristic Evaluation (1995) · NN/g · 10 Usability Heuristics for User Interface Design

Questions people actually ask

Can AI replace a UX researcher?
No. AI audits find rule-checkable defects (broken flows, contrast failures, missing feedback) fast and cheap. Understanding why users behave the way they do still requires research with real people, as NN/g's synthetic-users guidance makes explicit.
How accurate are AI UX audits?
It depends on the method. Baymard measured an 80% false-positive rate for screenshot-based ChatGPT-4 audits in 2023, and found the model caught only 14% of live-page issues because roughly half of real issues require interacting with the page. Agent-based audits that operate the product avoid that structural blind spot, and citing named rules makes each finding checkable.
How long does an AI UX audit take?
Minutes for most products, under an hour for large ones, versus 1 to 6 weeks for a typical human-run audit. Agent-based runs spend most of that time actually using the product: signing up, submitting forms, and breaking flows.
How much does an AI UX audit cost?
Subscription tools run from free tiers to $199 per month depending on the method family. ClapBack is a one-time scan from $49. Human audits advertise anywhere from $35 gig listings to $15,000 agency packages, per our analysis of 20 published rate cards.

Reading about UX debt is the slow way.

Paste your URL and an AI agent will use your product like a real user, find what breaks, and hand you the tickets. From $49, no install, no retainer.

Get my free roast →
Your site or a competitor’s. We won’t tell.
Keep reading
AI audits
AI agents, synthetic users, real users, or your own eyes: four ways to find out your UX sucks
Money
UX audit cost in 2026: the median is $500, not $25,000
AI audits
4 ChatGPT UX audit prompts that work (and the wall they all hit)