Skip to main content

Core Concepts

Teckel AI transforms scattered AI interactions into structured documentation insights. Here's how our system works and what makes it powerful for documentation teams.

The Teckel Judge​

Our unified evaluation engine analyzes every AI response in a single, efficient process. The Teckel Judge combines quantitative scoring inspired by RAGAS metrics and LLM generated qualitative analysis to provide comprehensive insights that documentation teams can immediately act upon. See the Teckel Judge section for more detail.

Topic Intelligence: Your Documentation Roadmap​

The real power of Teckel AI comes from understanding patterns across thousands of queries. We automatically organize user needs into actionable insights.

Automatic Topic Discovery​

We use advanced clustering to group similar queries together, to provide at-a-glance insight into what users are asking your AI systems about:

  • Semantic Grouping - Queries with similar meaning cluster together, even with different wording
  • Dynamic Evolution - Topics adapt as user needs change over time
  • Statistical Significance - Only meaningful patterns surface, filtering out noise

Example: These queries would automatically group into a "password reset" topic:

  • "How do I reset my password?"
  • "Forgot password help"
  • "Can't log in, need new password"
  • "Password recovery process"

Documentation Gap Analysis​

For each topic, we identify exactly what's missing:

  • Performance Metrics - Topics with low accuracy scores lack proper documentation
  • Query Volume - See how many users are at risk of misinformation
  • Impact Scoring - Prioritize fixes by potential improvement
  • Trend Tracking - Monitor if knowledge gaps are improving after intervention

How Documentation Teams Use This​

Here's what makes our topic intelligence invaluable for documentation teams:

Direct Feedback Mapping
Every topic shows:

  • Real user queries (what they're actually asking)
  • Current document performance (which docs try to answer these questions)
  • Specific gaps (what information is missing)
  • Fix priority (based on volume and impact)

Topic Relationships
We map connections between topics, revealing:

  • Which topics users explore together
  • Documentation that is or should be linked
  • Content that should be consolidated
  • Navigation improvements needed

Actionable Tasks
Instead of vague "improve documentation" feedback, you get:

  • "Create a password reset guide - 487 users asked, poor accuracy score"
  • "Update API authentication docs - missing OAuth flow details"
  • "Link billing FAQ to subscription management - users ask both together"

Hypothetical Example: E-commerce Documentation​

  1. Topic: "Shipping Rates" (892 queries/month)

    • Accuracy: 0.4 (very low)
    • Missing: International shipping info for Spain, Germany, Poland
    • Action: Create comprehensive shipping guide for Europe

  2. Topic: "Return Process" (634 queries/month)

    • Accuracy: 0.7 (moderate)
    • Missing: Return label generation steps
    • Action: Add step-by-step instructions on where to print a return label

Result: After fixing these major gaps in just two topics, overall AI accuracy across the system improves from 67% to 75%.

Document-Topic Intelligence Network​

Understanding how your documents support different user needs is crucial for maintaining high-quality AI responses. Teckel AI provides sophisticated analysis of the relationships between your documentation and the topics users ask about.

How Documents Support Multiple Topics​

A single document rarely answers just one type of question. Our system tracks how each document performs across different topic clusters:

Multi-Topic Coverage
Your "User Authentication Guide" might support:

  • Login troubleshooting (85% accuracy)
  • Password policies (92% accuracy)
  • Session management (67% accuracy)
  • Security settings (71% accuracy)

This granular view reveals that while the document excels at password policy questions, it needs improvement for session management queries.

The Document Performance Matrix​

We analyze every document through multiple lenses:

Utilization Rate
How often is this document retrieved and actually used to answer questions? A low utilization rate might indicate:

  • Poor chunking or indexing
  • Irrelevant content mixed with valuable information
  • Need for document splitting or reorganization

Topic-Specific Performance
The same document can perform brilliantly for one topic while failing another. We track:

  • Accuracy per topic cluster
  • Which sections support which topics
  • Performance trends over time
  • Impact of document updates on different topics

Cross-Document Dependencies
Modern documentation is interconnected. We identify:

  • Documents that work well together
  • Missing links between related content
  • Redundant information across multiple documents
  • Opportunities for consolidation

Feedback Aggregation & Pattern Recognition​

When hundreds of queries reference the same document, patterns emerge:

Consolidated Feedback Analysis
Instead of reviewing individual query feedback, see aggregated insights:

  • "Users consistently ask about rate limits, but the API guide doesn't mention them"
  • "The setup guide references a config file that no longer exists in v3.0"
  • "Multiple queries show confusion between 'workspace' and 'project' terminology"

Pattern-Based Prioritization
We identify which document issues affect the most users:

  • Critical: Affecting 500+ queries per week with less than 60% accuracy
  • High: Impacting major topic clusters with declining performance
  • Medium: Isolated issues in low-traffic topics

Real-World Impact Example​

Consider a SaaS platform's documentation:

Document: "Billing & Subscriptions FAQ"
Topics Supported:

  • Payment Methods (423 queries/week) - 89% accuracy ✓
  • Plan Upgrades (312 queries/week) - 45% accuracy ✗
  • Invoice Management (198 queries/week) - 78% accuracy ⚠
  • Cancellation Process (156 queries/week) - 91% accuracy ✓

Analysis Reveals:

  • Plan upgrade information is severely lacking
  • The document excels at payment and cancellation topics
  • Invoice management could use minor improvements

Recommended Action: Split the FAQ into focused guides. Create a dedicated "Plan Management Guide" with detailed upgrade/downgrade procedures. This single change would improve accuracy for 312 weekly queries from 45% to an estimated 85%.

The Feedback Loop​

Our system creates an automatic improvement cycle:

  1. Monitor - Track every query and document reference
  2. Analyze - Identify patterns and performance issues
  3. Prioritize - Focus on high-impact improvements

Allowing you to:

  1. Update - Fix documentation based on specific feedback
  2. Validate - Measure improvement in real-time

This systematic approach transforms reactive documentation maintenance into proactive quality management, ensuring your AI always has accurate, relevant information to provide users.

Ground Truth Testing: Automated Knowledge Base Validation​

Teckel AI now includes revolutionary ground truth testing that automatically validates your vector database with custom prompts based on identified weaknesses.

How Ground Truth Testing Works​

Our system proactively tests your knowledge base to distinguish between:

Retrieval Issues

  • Your vector search isn't finding relevant documents
  • Documents exist but aren't being matched to queries
  • Embeddings or chunking strategies need optimization

Content Issues

  • Information genuinely doesn't exist in your knowledge base
  • Documentation has gaps that need to be filled
  • Conflicting information exists across documents

The Testing Process​

  1. Pattern Analysis: We identify recurring weaknesses in AI responses
  2. Custom Query Generation: Precise test prompts are automatically generated these specific weaknesses
  3. Vector Database Testing: We test your RAG search engine with these sample queries
  4. Root Cause Identification: Results reveal whether it's a search or content problem
  5. Targeted Recommendations: You receive specific guidance on what to fix

Example Scenario​

Imagine users frequently ask about "password complexity requirements" but get poor responses:

Ground Truth Test Results:

  • Test Query: "What are the minimum password requirements?"
  • Search Results: Found 0 relevant chunks
  • Diagnosis: Content Gap - No documentation exists
  • Recommendation: "Create password policy documentation including minimum length, character requirements, and expiration rules"

Versus:

  • Test Query: "How do I reset my password?"
  • Search Results: Found 3 conflicting chunks with high similarity scores
  • Diagnosis: Conflicting Documentation - Multiple versions of same guide exist
  • Recommendation: "Verify which authentication guide to use and remove others from vector database"

Benefits for Documentation Teams​

  • Proactive Detection: Find issues before users encounter them
  • Clear Diagnostics: Know exactly what needs fixing
  • Prioritized Actions: Focus on high-impact improvements
  • Continuous Validation: Automatic testing ensures ongoing quality

Processing and Turnaround Times​

Trace Batch Processing (Standard)​

  • Typical turnaround: Within 1 hour
  • Maximum time: 24 hours
  • Cost-effective for most use cases
  • Ideal for production environments

Trace Realtime Processing (Premium)​

  • Immediate results in 20-30 seconds
  • Available for time-sensitive applications
  • Premium pricing applies
  • Contact sales for access

Feedback Processing​

  • Any Topic or document level feedback can be created on demand
  • Automated feedback on low scores sent weekly

Integration with Your Workflow​

Teckel AI fits into existing documentation workflows without disruption:

For Documentation Teams​

  • Weekly reports on document performance
  • Prioritized fix lists based on ground truth testing
  • Clear distinction between search and content issues
  • Direct links to problematic documents
  • Before/after quality tracking

For Product Teams​

  • Feature gap identification from user queries
  • Usage patterns for roadmap planning
  • Quality metrics for OKRs
  • Customer satisfaction correlation

For Engineering Teams​

  • RAG system performance metrics
  • Vector search effectiveness data
  • Model behavior insights
  • Integration health monitoring