Opik deployment

Opik is an open-source platform for evaluating, testing, and monitoring LLM applications, built by Comet.

You can use Opik for:

• Development:
  • Tracing: Track all LLM calls and traces during development and production (Quickstart, Integrations).
  • Annotations: Annotate your LLM calls by logging feedback scores using the Python SDK or the UI.
• Evaluation: Automate the evaluation process of your LLM application:
  • Datasets and Experiments: Store test cases and run Experiments (Datasets, Evaluate your LLM Application).
  • LLM as a judge metrics: Use Opik's LLM as a judge metric for complex issues like hallucination detection, moderation and RAG evaluation (Answer Relevance, Context Precision).
  • CI/CD integration: Run evaluations as part of your CI/CD pipeline using our PyTest integration.
• Production Monitoring: Monitor your LLM application in production and easily close the feedback loop by adding error traces to your evaluation datasets.

Architecture¶

Opik uses the same datastores as Comet EM, along with dedicated frontend and backend pods for Opik and a Clickhouse database which is deployed and managed by the Comet Helm release.

Deployment¶

To proceed with applying the Helm chart, please refer first to our Helm chart documentation, which covers the necessary Helm commands. After that, follow the steps below to enable Opik.

Opik is enabled within Comet, via the following setting in the Helm override-values:

# ...
comet:
  # ...
  opik:
    enabled: true
# ...

Resource Requirements¶

Running Comet with Opik increases the overall median resource requirements. For more details, please refer to the Opik section of the Resource Requirements Page.