Aio

Installation Steps ¶

Start by installing CometCtl on the machine on which you want to install Comet.

Then install Comet:

cometctl aio install

As part of the installation process, cometctl will prompt you to set your comet_base_domain, when it does, you can set it to the IP or domain clients will use with the Comet SDK.

After that, you should see a splash screen showing a successful install, with instructions on how to access the Comet WebUI. If you did not set the domain, or would like to change it for any reason, run cometctl aio update-config and set or change the comet_base_domain variable to your new IP or domain.

If you made any changes, you must run cometctl aio update-config

Configuring External Data Layers¶

By default cometctl will install the core comet data-layers locally and configure Comet appropriately to use them. However you can also configure the use of external hosts/services for these data layers, which is the recommended approach for production deployments.

MySQL¶

To configure an externally hosted relational database in place of the on-premises MySQL installation add the following variables by running cometctl aio update-config:

mysql_host: <host>
mysql_user: <username>
mysql_password: <password>

mysql_db: <db_name>

If you do not explicitly specify a name, logger will be used as the application database name.

In case you’re authenticating to RDS using IAM Roles

You’ll need to enable SSL and set some other configuration settings as follows:

mysql_iam_enabled: true
mysql_use_ssl: true
mysql_aws_region_name: us-east-1

Once you've finished configuring, run cometctl aio update-config. If you have configured prior to installation, you may now continue where you left off.

Redis:¶

To configure an externally hosted caching layer in place of the on-premises Redis installation, invoke cometctl aio update-config and add the following variables:

redis_host: <host>

redis_auth_token: <auth token>
redis_port: <redis_port>

Once you've finished configuring, save your file. If you have configured prior to installation, you may now continue where you left off.

S3:¶

To configure an externally hosted datastore in place of the on-premises MinIO installation, invoke cometctl aio update-config and add the following variables:

When using GCP, set the s3_url variable to storage.googleapis.com instead of s3.amazonaws.com

s3_url: s3.amazonaws.com
s3_bucket: <s3 bucket name>
s3_access_key: <s3 access key>
s3_secret_key: <s3 secret key>
s3_https_enabled: true

s3_region: <s3 region>

If you are authenticating using S3 roles rather than Access keys, set both the S3 key variables to the following string value:

s3_access_key: “IAM-ROLE”
s3_secret_key: “IAM-ROLE”

If your deployment is using Kubernetes, set the values to the following instead:

s3_access_key: “IAM-ROLE-SERVICE-ACCOUNT”
s3_secret_key: “IAM-ROLE-SERVICE-ACCOUNT”

Once you've finished configuring, run cometctl aio update-config. If you have configured prior to installation, you may now continue where you left off.

Enabling SSL¶

Run cometctl aio enable-ssl

Setup either a cloud Load Balancer with an SSL Certificate attached or make use of LetsEncrypt.

If using LetsEncrypt, run: cometctl aio enable-ssl --letsencrypt

Special cases¶

If comet is fronted by a load balancer, set the idle timeout to at least 150 seconds, in order to allow uploads of big files.

Auth Setup ¶

Comet Supports various authentication methods. View the method's documentation for setup instructions.

Upgrading Comet¶

The following command will upgrade Comet to the latest version and prompt for an application restart to apply:

cometctl aio upgrade

Updating License Token¶

The following command can be used to update Comet with a new license token:

cometctl license-update -t {provided-token}

Useful Administration Commands¶

Updating config

The following opens the Comet configuration file for editing and validates the changes on saving:

cometctl aio update-config

Restart all Comet services

cometctl aio restart-services

Perform a quick health check to verify the services are running

cometctl aio healthcheck

Enabling SMTP ¶

SMTP is enabled by default as of Comet installer version 2.0.50. The Comet SMTP server is currently supported on a best-effort basis only.

If you want to provide your own SMTP server, invoke cometctl aio update-config and make sure to change the following values:

smtp_enabled: true
smtp_host: <host>
smtp_port: <port>
smtp_user: <user>
smtp_password: <password>
smtp_protocol: <smtp or smtps>
mail_from: <user@yourcompanymail.com>

Update system configuration:

cometctl aio update-config

For additional documentation, please see SMTP Configuration.

Backups ¶

We recommend setting up a policy on your cloud provider to rotate snapshots from your instance volume(s).

• AWS guide
• GCP guide
• Azure guide
• Bare Metal: We recommend you periodically take a backup of the disk or volume containing your experiment data using a method of your choosing.

Server Usage Report and BI Events ¶

In order to manage your team’s Comet subscription, Comet needs to track and report usage statistics from your on-premesis or VPC deployment of the Comet system. This allows Comet to easily track and manage user statistics for true-ups and billing, without any additional overhead for your team’s management of the deployment.

Firewall whitelist information is as follows

Port: 443
Host: stats.comet.com
IP: 3.210.8.13
Proto: TCP

Hostname and IP are statically set and should not change. A complete sample usage report can be found below:

{
    "usage_id": "2022-12",
    "report": [
      {
        "username": "benjamin",
        "email": "z@gmail.com",
        "created_at": 1662014273604,
        "uiUsageCount": 1,
        "uiUsageUpdateTs": 1662014275281,
        "sdkUsageCount": 0,
        "sdkUsageUpdateTs": 1662014275281,
        "suspended": false
      },
    ],
    "number_of_users": 1,
    "licenseKey": "licenseKey1234",
    "domain": "gmail.com",
    "backendVersion": "3.2.74",
    "frontendVersion": "NA"
  }

Below is an anonymized version of an actual BI event report:

{
  "anonymousId": null,
  "context": {
    "library": {
      "name": "analytics-python",
      "version": "2.2.1"
    }
  },
  "event": "new_experiment_be",
  "integrations": {},
  "messageId": "e400c2e6-d161-4225-a992-a3621fff95eb",
  "originalTimestamp": "2023-01-13T14:40:41.337309+00:00",
  "properties": {
    "backendVersion": "3.2.165",
    "client": "my-company",
    "domain": "my-company.com",
    "experimentKey": "c249967859b24153b3c8659aac9g3dab",
    "frontendVersion": "5.20.2",
    "licenseKey": "pffka9493j2bd963do98Tfabccf80997r",
    "on_prem": true,
    "optimizationId": null,
    "organizationId": "PJ9ULeGB7kqQ4G7J08DsrgpoS",
    "organizationName": "defaultOrganization",
    "projectId": "a6e22f54d84a548b3f1760bf6421602b",
    "projectName": "my-recommender-model",
    "pyVersion": "3.8.12",
    "sdkVersion": "3.31.22",
    "status": "true",
    "teamName": "my-team",
    "version": "3.31.22"
  },
  "receivedAt": "2023-01-13T14:40:41.956Z",
  "sentAt": "2023-01-13T14:40:41.714Z",
  "timestamp": "2023-01-13T14:40:41.579Z",
  "type": "track",
  "userId": "my-name",
  "writeKey": "zZVvzqFNeXDt2KSyeGofvrsKY6osd6Ht"
}

If you have any questions regarding Comet Usage and Events Reporting, please contact support@comet.com or your account manager.

Caveats and FAQ¶

Q: I'm seeing a prompt asking me to restart as part of the install A: On certain newer distributions of Linux, you may be prompted as part of cometctl init or cometctl install with an ncurses prompt asking whether you'd like to restart any services. If you do not know what to do here, you may hit enter and bypass this.