Qlik Answers - Fast Answers

[NAM_climber]

I designed 10 natural language questions across three difficulty levels (simple, intermediate and complex), calculated the precise expected answers from the raw data first, and then fired the same questions at Qlik Answers Fast to see how it held up.

The results? 7/10 and I'm genuinely impressed.

Here's the honest breakdown:

✅ 7 Pass - exact matches across all simple and most complex questions. Total revenue, order counts, year-over-year decline, top salesperson by AOV ,all answered precisely and in under 20 seconds. Every single time. For an AI answering natural language questions against a live data model, that's a seriously strong result.

🟡 2 Partial Pass - the return rate questions identified the correct category and the correct reason, but the values differed from expected. This was entirely by design. Fast Answers mode isn't built for questions that require deep multi-hop reasoning across loosely linked tables. That's what Qlik Answers Full (the complete agentic experience) is for, and those questions would have resolved cleanly there.

🔴 1 Fail - and this one is on me, not Qlik. I deliberately left a synthetic key issue in the data model between FactMarketing and DimCampaign. Qlik correctly couldn't break down marketing ROI by channel because the table links were intentionally broken. The engine behaved exactly as it should. A good reminder that AI is only as good as the data model underneath it.


All response times were under 20 seconds, which for a tool answering complex analytical questions in plain English is outstanding.

The takeaway: Fast Answers is exactly what it says on the tin: quick, accurate, and reliable for the questions your data model is built to support. Know your model, know your mode, and the results speak for themselves.

[Or]

@adithyarpai May well be interested in this. 

[NAM_climber]

Im keen to hear about your experiences and thoughts , personaly i think Fast Answers is fantastic and fills the gap between normal dashboard interaction and the full answers , but the key takeaway is AI will only be as good as your data , so having quality governed data will give a more accurate experience in using AI

[adithyarpai]

Thanks for the rigor here  @NAM_climber — pre-computing expected values from raw data before testing, and publishing the exact question set with a verdict table, is exactly the kind of evaluation we want more of. Really useful.

A couple of things jumped out from the detail table that I'd like to dig into with you, if you're open to it:

On I3 (marketing ROI by channel) — it looks like Fast Answers didn't just decline to answer, it returned the same 1,593% ROI across every channel. If that number is the blended/overall ROI repeated across rows rather than a true error state, that's a more specific bug than "couldn't break down by channel" — it's a silent fallback to an aggregate when the join can't be disambiguated, which is worth us tracking precisely.

On C3/A1 (return rate by category) — the finding notes FactReturns links via OrderID rather than ProductKey. If that's right, this isn't a reasoning-depth limitation that Full Mode would resolve on its own — it's a join grain mismatch that fans returns out across every product in an order. That would produce the same inflated numbers regardless of mode, unless the model itself is fixed. Good catch, and a good reminder that model design is doing a lot of the work here, as you said.

Would you be willing to share the underlying data model (or just the FactReturns/FactMarketing relationships) so we can confirm both of these precisely? This is genuinely useful test data and I'd rather run it down properly than guess.