The Two-Tier Answer System

Why two tiers?

A single-tier system would mean either always doing the expensive forum search (slow, 15-30s for every question) or never doing it (missing crucial edge-case knowledge that only appears in BGG threads).

Two tiers let me be fast on simple questions and thorough on hard ones.

Loading diagram...

Tier 2 adds community knowledge on top of official rules -- it never replaces the rulebook.

Confidence score

After Tier 1 synthesis, the AI model returns a confidence estimate alongside the answer. This is a self-reported score from the model, calibrated against the quality and coverage of the retrieved chunks. If the top-K chunks do not well cover the question, confidence is low.

The threshold is configurable. I tune it per-game based on how well the rulebook is structured (dense rulebooks score higher confidence on the same question than sparse ones).

Auto-escalation is silent

You never see the escalation happen. If your question triggers Tier 2, you just wait a few extra seconds. The response will arrive with community citations ([T1], [T2]) alongside the PDF citations, which tells you it was a Tier 2 answer.

When Tier 2 finds nothing

Tier 2 can fail gracefully. If the BGG community has no threads about the relevant rule (rare for popular games, more common for new releases), I return the Tier 1 answer with a note that no community discussion was found.

Performance breakdown