Skip to content
To Comet website
Doc version: 3.13.6

Deployment Options

Comet offers a number of deployment options for self-hosted Comet installations. This document outlines the deployment options currently offered. If you have a requirement that is not covered by this document please speak to your sales team.

Basic Requirements

NOTE: The following are the basic requirements for deploying Comet's core Experiment Management and related features. Additional requirements may be necessary for specific features such as Comet's Model Production Monitoring (MPM) or other advanced features.

To deploy Self-Hosted Comet you will need:

  • A valid Comet License Token (provided by Comet)
  • A Linux Server or Kubernetes Cluster meeting the hardware requirements
  • A supported Linux distribution (if not using a Kubernetes Cluster):
    • Ubuntu (Preferred)
      • 18.04 LTS
      • 20.04 LTS
      • 22.04 LTS
    • Amazon Linux
      • 2
      • 2023
    • Red Hat Enterprise Linux
      • 8
      • 9
    • CentOS 8 (deprecated)
  • HTTP/HTTPS network connectivity between where you run the Comet SDK and the Comet server.
  • Familiarity with Linux command line tools is also recommended.

Additionally, while a DNS name is optional, it may be useful for easier access to the application, and is required for using HTTPS.

In this section, we will provide detailed information on these requirements and any additional considerations that should be kept in mind when deploying the application.

Data Layer Dependencies

Comet requires a number of data layer dependencies to function, which can be deployed as part of the Comet installation itself (internal) or separately (external).

  • Object storage: S3 or compatible.
    • For storing experiment data and artifacts.
  • Relational DB: MySQL (5.7+/8) or compatible.
    • For storing metadata, users, and other in-app configuration.
  • Queue/Cache: Redis or compatible.
    • For caching and background job processing.

Managed Solutions

These data layer dependencies can be satisfied in a number of ways. However when possible, we recommend using managed solutions for each:

If you have an internal on-premise managed solution that is not listed above, this is also recommended.

Self-Managed Solutions

If a managed solution is not possible, Comet can also be deployed you can also deploy MySQL/MariaDB, Redis, and MinIO on your infrastructure to satisfy these dependencies:

  • Object storage:
    • MinIO
  • Relational DB:
    • MySQL
    • MariaDB
  • Queue/Cache:
    • Redis

Internal Data Layer Dependencies

All deployments will install and run the dependencies locally on the same compute resource by default.

At any point during or after the Comet deployment you can also elect to change one or more of these dependencies to use a different supported solution. For example, if you started with a Comet deployment that locally stored data using MinIO, you can seamlessly migrate to S3 (and back) at your leisure.

Deployment Types

Comet can be deployed in a number of ways, depending on your requirements. The following are the most common deployment types:

Single Server Installation (All In One)

Our most common deployment type is a single server installation, where all the Comet components are installed on a single machine. This is the simplest deployment type and is recommended for most users.

This server can be bare metal, or a virtual machine, and can be hosted on premises or in a Cloud Provider VPC:

  • AWS EC2
  • Google Compute Engine
  • Azure Virtual Machines
  • Other cloud providers (Digital Ocean, Linode, Rackspace, Softlayer, etc.)

The Data Layer dependencies can be external or internal (All-in-One).

Docker Compose

An alternative approach to a single server installation is to use Docker Compose.

Kubernetes Deployment

For larger installations, those requiring high availability, scalability, those already using Kubernetes, or those who have a requirement to use Kubernetes, Comet can be deployed on a Kubernetes cluster.

We strongly reommend using a managed Kubernetes service such as:

  • AWS Elastic Kubernetes Service (EKS)
  • Google Kubernetes Engine (GKE)
  • Azure Kubernetes Service (AKS)
  • Other managed Kubernetes services (Digital Ocean, Akamai, etc.)
  • On-premises Kubernetes clusters (OpenShift, Rancher, etc.) managed by an internal infrastructure team.

We discourage using self-managed Kubernetes clusters unless you have a dedicated Kubernetes management team, due to the complexity of managing the Kubernetes Control Plane yourself.

Data Layer dependencies can be external or internal to Kubernetes Comet deployment.

Installation

Depending on the installation type chosen, Comet can provide the following ways to install:

  • Single Server:
    • Installation utilities (cometctl) for automatically installing Comet on a single server.
    • Pre-baked golden Amazon Machine Image (AMI)
    • Debian and RPM packages for deployments in RedHat or Debian based Linux distributions.
    • Docker Compose YAML
      • This can be generated for you with our cometctl tool.
  • Kubernetes:
    • Helm Chart
    • Docker images
      • Comet will provide ready to use docker images hosted on our private docker registry. These images are security scanned and updated regularly and based on one of the supported Linux distributions.

All Docker images, packages, and install scripts are hosted privately and can be made available to air-gapped customers through a secure download link.

Other Self-Hosted Considerations

Auth

Comet is configured to use a local User/Password DB (stored in the MySQL DB) by default. Additionally, Comet allows for integration with a number of industry standard authentication methods, including but not limited to the following:

System Health and Logging

Comet is capable of integrating with many of the industry's most popular monitoring/observability solutions, including:

  • Application Performance Monitoring (APM)
    • New Relic
  • Infrastructure
    • Datadog
    • AWS Cloudwatch
    • Prometheus
    • Grafana
  • Business Intelligence (BI)
    • AWS Cloudwatch
  • Log Aggregation
    • Loki
    • Splunk
    • AWS Cloudwatch
    • ELK/EFK

If your preferred monitoring solution is not listed, please get in touch with us.

Upgrades

Comet is in active development and new versions are released on a steady cadence containing feature releases and bug fixes. For further information, please see the deployment documentation specific for your installation type.

Dec. 17, 2024