Testing Phase Guide
How to test and provide feedback
How to test and provide feedback
Thank you for participating! Your feedback directly shapes the quality of this tool. This guide explains how to use the app, what to test, and how to provide feedback effectively. Keep this page open as a reference while you test.
The answer was helpful and correct. One click — takes 1 second. Feedback is saved immediately.
Something was wrong. This automatically opens the detailed feedback form so you can tell us what happened.
Click the comment icon to open the full form anytime — even for good answers where you want to add notes.
Star Rating (1–5)
Overall quality of the answer. 5 = excellent, 1 = completely wrong.
Quick Assessment (3 buttons)
These are the most important fields for measuring quality. Please fill them in whenever possible:
Was it correct?
Yes — factually accurate
Partial — partly right, partly wrong
No — wrong answer
Hallucinated?
No — everything is from the sources
Yes — the AI made up information not found in the cited documents
Sources relevant?
Yes — cited docs match the topic
Partial — some relevant, some not
No — wrong documents cited
Issue Tags (for low ratings)
Select what went wrong: incorrect, incomplete, irrelevant, confusing, outdated, or dangerous.
Comments & Expert Fields
Add any notes. If you're a domain expert, check the expert box to provide the correct answer or note what was missing.
Auto-Captured Context
You don't need to manually record the mode, search tier, confidence score, grounding score, or response time — these are automatically saved with every feedback submission.
Simple factual question from manuals
Ask about a known procedure. Expect: correct answer, Tier 1, high confidence.
Question needing broader sources
Ask about troubleshooting. Click "Search Deeper" if Tier 1 isn't enough.
Question needing ticket history
Ask about a past incident. Go to Tier 3 to find ticket history.
Out-of-scope question
Ask something unrelated. System should refuse or show a disclaimer — not make up an answer.
Fast vs Deep mode comparison
Ask the same question in both modes. Deep mode should be more thorough.
Cross-check cited sources
Click on source citations and verify the answer actually matches what the document says.
Low-confidence edge case
Ask an unusual question with limited KB coverage. Look for orange border and disclaimer text.
No information available
Ask about something not in the knowledge base at all. System should refuse rather than fabricate.
Conversation management
Pin, archive, delete conversations.
Source viewing
Click source cards — PDFs and images should open correctly.
Response time
Target: <30s fast mode, <60s deep mode.
Auto-titling
New conversations should get relevant titles.
Shows which sources were searched. Tier 1 = manuals only, Tier 2 = + forum & tech notes, Tier 3 = + all tickets.
How confident the system is in the retrieved sources. Higher = the sources closely match your question.
How much of the answer is backed by the cited documents. Green (70%+) = well-grounded. Yellow (40–70%) = partially. Red (<40%) = low — check carefully.
Orange left border means the system added a disclaimer — it's not confident in the answer. Read it with caution and verify with the sources.
Appears when more source tiers are available. Click to widen the search to include more document types.
We're measuring these metrics from your feedback. Help us hit these targets:
Answer Correctness
"Was it correct?" = Yes or Partial
Hallucination Rate
"Hallucinated?" = Yes
Source Relevance
"Sources relevant?" = Yes
User Satisfaction
Average star rating
Rate every answer — even a quick thumbs up/down helps. The 3 assessment buttons take just seconds more.
Check the sources — click citation cards to verify the answer actually comes from the cited documents.
Try "Search Deeper" — if Tier 1 doesn't answer fully, click the deeper search button before giving up.
Browse the Sources Catalog — check what documents are available to know what to ask about.
Try both modes — Fast mode for quick answers, Deep mode for comprehensive ones. Compare the results.
Test the limits — ask off-topic or very niche questions to see if the system handles them gracefully.
Need help? Contact the project team.
View real-time results on the Feedback Dashboard.