Retrieval augmented generation for accuracy and user trust
Saving research hours by enabling instant, accurate and traceable data surfacing.
Situation
Our key USP at Protobrand is using behavioral science to access consumers’ subconscious. We ask respondents to use metaphors to describe, for example, the type of person who loves Nike. These metaphors help researchers understand the why behind consumer decisions better than direct questioning, so they are often included in client reporting.
Finding quality quotes is incredibly time-consuming, with researchers spending hours populating reports. As clients demand faster turnaround at lower cost, this approach was becoming unsustainable.
As a custom research agency, accuracy and detail are paramount. Any AI-driven solution would need to earn researcher trust, so near-100% accuracy and full source traceability were paramount.
Task
Build a function that surfaces high-quality, specifically relevant quotes quickly and easily, earning researcher trust through validation mechanisms and source traceability.
KPIs:
- Time spent finding quotes: rely on qualitative feedback, given it is difficult to isolate individual tasks within analysis and reporting.
- Feature adoption: usage of the function relative to similar features already available on platform.
Action
We built a verbatim search function integrated into the chat-based analysis product on platform, combining vector databases and layered LLM calls:
- Layered LLM calls: a planner routes each request to either semantic search or theme-based search depending on intent.
- LLM-as-a-judge: an evaluation layer on top, refined using real user feedback.
- Source traceability: every result surfaces the respondent’s metadata so researchers can manually verify quotes whenever they want, building trust over time.
Result
Verbatim search landed well. In the 14 days after launch, users sent this query an average of 16 times per project, 63% more than similar functions.
User feedback was positive about time savings and ease of use:
“…indulge with the tools the team are building… it has made super super quick turnaround actually possible and not, like, dreadful.”
“I would use the verbatim search and say ‘Hey, please find the best quotes for outdoorsiness’ and it did find them and they were perfect! … We had a slide ready in a matter of minutes.”