How to set up Cody Gateway locally

WARNING: This is a development guide - to use Cody Gateway for Sourcegraph, refer to Sourcegraph Cody Gateway.

This guide documents how to set up Cody Gateway locally for development.

To get started, Cody Gateway is included in the standard dotcom run set. Since Cody Gateway cucrently depends on Sourcegraph.com, there's not much point running any other run set.

sg start dotcom

However, the locally running Cody Gateway is not used by default - dev-private credentials point towards a shared development instance of Cody Gateway. To use the locally running Cody Gateway, follow the steps in use a locally running Cody Gateway

Use a locally running Cody Gateway

To use this locally running Cody Gateway from your local Sourcegraph instance, configure Cody features to talk to your local Cody Gateway in site configuration, similar to what customers do to enable Cody Enterprise:

{
  "completions": {
    "enabled": true,
    "provider": "sourcegraph",
    "endpoint": "http://localhost:9992",
    "chatModel": "anthropic/claude-2",
    "completionModel": "anthropic/claude-instant-1",
    // Create an Enterprise subscription and license key:
    // https://sourcegraph.test:3443/site-admin/dotcom/product/subscriptions
    // Under "Cody services", ensure access is enabled and get the access token
    // to use here.
    // Note that the license and tokens will only work locally.
    "accessToken": "..."
  }
}

Similar values can be configured for embeddings to use embeddings through your local Cody Gateway isntead.

Now, we need to make sure your local Cody Gateway instance can access upstream LLM services. Add the following to your sg.config.overwrite.yaml:

commands:
  cody-gateway:
    env:
      # Access token for accessing Anthropic:
      # https://start.1password.com/open/i?a=HEDEDSLHPBFGRBTKAKJWE23XX4&h=my.1password.com&i=athw572l6xqqvtnbbgadevgbqi&v=dnrhbauihkhjs5ag6vszsme45a
      CODY_GATEWAY_ANTHROPIC_ACCESS_TOKEN: "..."
      # Create a personal access token on https://sourcegraph.test:3443/user/settings/tokens
      # for your local site admin user. This allows your local Cody Gateway to
      # access user information in the Sourcegraph instance.
      CODY_GATEWAY_DOTCOM_ACCESS_TOKEN: "..."
      # Other values, such as CODY_GATEWAY_OPENAI_ACCESS_TOKEN and
      # CODY_GATEWAY_OPENAI_ORG_ID, can be set to access OpenAI services as well.

For more configuration options, refer to the configuration source code.

Then, restart sg start dotcom and try interacting with Cody!

Observing Cody Gateway

You can get full tracing of Cody interactions, including Cody Gateway parts, with sg start monitoring's Jaeger component. Cody Gateway will also emit traces of its background jobs.

All event logging is output as standard logs in development under the cody-gateway.events log scope.

Working with BigQuery for events logging

NOTE: To make authentication work magically with the GCP, you need to install the gcloud CLI and run gcloud auth login --project cody-gateway-dev once.

To send events to BigQuery while developing Cody Gateway locally, add the following to your sg.config.overwrite.yaml:

commands:
  cody-gateway:
    env:
      CODY_GATEWAY_BIGQUERY_PROJECT_ID: cody-gateway-dev

Then to view events statistics on the Enterprise subscriptions page, add the following section in the site configuration, and run the sg start dotcom stack:

{
  "dotcom": {
    "codyGateway": {
      "bigQueryGoogleProjectID": "cody-gateway-dev",
      "bigQueryDataset": "cody_gateway",
      "bigQueryTable": "events"
    }
  }
}