Home/Methodology

A reproducible AI recommendation study

See how Agentzia tests what AI agents recommend.

Every selected AI client receives the same natural buying situations in a separate conversation. The client decides what to search, read, compare, and recommend. Agentzia preserves the available evidence and turns the completed paths into a structured report.

Matched requests
Independent client decisions

The test panel

Up to four questions reveal different parts of the buying journey.

Two situations name the product. When Agentzia can derive a useful brand-free product description, two more describe only the category or problem so discovery can happen naturally.

01 / Known product
I am considering this product. Is it any good?

Tests existing perception, trust, and purchase hesitation after the customer names the product.

02 / Category discovery
What are the best products in this category?

Tests whether the product enters an unprompted shortlist.

03 / Problem led
I have this problem. What should I use to solve it?

Tests whether the product is associated with the job it was built to do.

04 / Competitive choice
Which option would you choose for this situation?

Tests the evidence and tradeoffs that shape a final choice.

Evidence layers

Keep the client decision separate from the analysis.

01 / Client

Natural research

Representative ChatGPT, Claude, Gemini, and Grok research profiles answer independently with web access. Consumer apps can evolve separately from these controlled profiles.

02 / Record

Preserved evidence

Agentzia stores the complete answer, citations, reported queries, surfaced pages, alternatives, and provider-observed events whenever the client exposes them.

03 / Analysis

Structured findings

A separate analysis pass extracts discovery, understanding, trust, recommendation, competitors, and page-level actions while preserving the original response underneath.

Controls and interpretation

Know what stays matched and what can move.

LayerHeld constantCan changeHow the report handles it
Customer requestSituation type and prompt suite version.Optional product and competitor context supplied by the user.Every path displays the exact request it received.
ProductSubmitted public URL and captured deployment fingerprint.The live page, public web, and generated discovery context can change between releases.Matched comparisons require the exact prompt roster and compatible test settings. A changed roster starts a new baseline.
ResearchSelected AI client and access to its configured web tools.Search results, source availability, and model behavior.Each answer keeps its citations and available trace fidelity separate.
Decision indexPublished outcome rules for the report version.The observed decisions in each completed path.The index summarizes this panel. It is a run-level signal rather than a market-share forecast.

Repeat after a release

Compare matched decisions with their evidence attached.

Rerun the same client set after a meaningful website or positioning change. Agentzia compares completed paths when the generated prompt roster is identical and shows which outcomes moved. If changed public product language produces different discovery prompts, the new run becomes a fresh baseline.

Search rankings, public sources, and model behavior can vary over time. The report records those conditions so a changed decision is interpreted with its supporting evidence.

Some providers expose exact searches and page visits. Others expose citations or a reported research summary. The report labels this fidelity instead of converting unavailable telemetry into a zero.

Method questions

Understand what each result can support.

Do all AI clients receive the same questions?

Yes. Every selected client receives the same available situation set in separate conversations. A complete set has four situations. Category and problem discovery require a usable brand-free description of the product, so a run can contain two situations when that context cannot be derived. The wording stays natural and does not tell the client which pages, claims, or competitors to inspect.

Does Agentzia control what an AI client searches?

Each research profile decides whether to search, which sources to use, and what to include in its answer. A separate controlled search-results track measures brand and category visibility without changing the client decision.

Can a run be reproduced exactly?

Agentzia preserves the URL, client set, situation suite, deployment fingerprint, timestamps, answers, and available evidence. Exact matched comparisons require the generated prompt roster to remain unchanged. If public product language changes the category or problem prompt, the next run becomes a new baseline. Live web results and model behavior can also change, so the report shows observed decisions rather than claiming permanent outcomes.

What happens when a provider fails?

The report identifies incomplete paths. A retry reuses completed paths and continues the missing work without spending additional credits.

Public evidenceInspect complete reports and responsesCategory guideUnderstand AI visibility and recommendationsData handlingSee what Agentzia stores and why

Start with a public product page

Run the same buying situations through the major AI clients.

Run your first test