Documentation

AI Test Oracle - User Guide & Privacy Information

How to Use the App

Step 1: Enter Your Data

  • Go to the Test Lab page
  • Enter the Original Prompt (optional but recommended for some tests)
  • Paste the AI Response you want to test (50-2,000 characters)
  • Optionally provide your own OpenAI API Key if you want to use a custom key

Step 2: Select Tests

  • Review the available tests and select which ones to run
  • All tests are selected by default
  • Click on test cards to toggle them on/off

Step 3: Run Tests

  • Click "Run Tests" to start the analysis
  • Wait for the tests to complete (usually takes a few seconds)
  • View your results with detailed scores and metrics

Available Tests

  • Response Time Analysis - Analyzes response complexity
  • Toxicity Check - Scans for harmful language
  • Hallucination Detection - Detects fabricated facts
  • Bias Detection - Identifies various types of bias
  • Guardrails Compliance - Verifies safety guidelines
  • Consistency Analysis - Detects contradictions
  • Prompt Injection Detection - Detects injection attempts
  • Relevance Score - Measures relevance to prompt
Data That IS Stored
Test Results: Your test results including scores, metrics, and analysis details are saved to MongoDB for history tracking.
AI Response: The AI response text you tested is stored with the test results.
Original Prompt: If provided, the original prompt is stored with the test results.
Test Metadata: Test ID, timestamp, selected tests, and execution times are stored.
Feedback: If you submit feedback, your rating and comments are stored (email is optional).
Test Count: A global counter of total tests run is maintained.
Data That IS NOT Stored
OpenAI API Keys: Custom API keys you provide are never stored. They are only used for the current test run and immediately discarded.
Session Data: Your session test count is stored locally in your browser only, not on the server.
User Identity: No personal identification information is collected or stored (unless you optionally provide an email in feedback).
IP Addresses: IP addresses are not logged or stored.
Privacy & Security
API Key Security: If you provide a custom OpenAI API key, it is used only for that specific test run. The key is sent securely to the backend, used for API calls, and then immediately discarded. It is never saved to the database, logged, or stored in any way.
Data Access: Stored test results are accessible through the History page. You can view your test history and download results as JSON or PDF.
Data Retention: Test results are stored in MongoDB. You can clear your test history at any time using the clear history function.
Third-Party Services: When using OpenAI API (either default or custom key), your data is sent to OpenAI for analysis. Please review OpenAI's privacy policy for information about their data handling.