Authorize user access to Azure Databricks with OAuth

This article explains how to authorize user access to Azure Databricks resources when using the Azure Databricks CLI or Azure Databricks REST APIs.

Azure Databricks uses OAuth 2.0 as the preferred protocol for user authorization and authentication outside of the UI. Unified client authentication automates token generation and refresh. After a user signs in and grants consent, OAuth issues an access token for the CLI, SDK, or other tool to use on the user’s behalf. Each access token is valid for one hour, after which a new token is automatically requested.

In this article, authorization refers to using OAuth to grant access to Azure Databricks resources, while authentication refers to validating credentials through access tokens.

For more high-level details, see Authorizing access to Azure Databricks resources.

Ways to authorize access to Azure Databricks resources

Azure Databricks supports two ways to authorize user accounts with OAuth:

• Automatic (recommended): Use unified client authentication if you're working with supported tools and SDKs, such as the Azure Databricks Terraform SDK. This approach handles token generation and refresh automatically.

• Manual: Generate a code verifier and challenge, then exchange them for an OAuth token. Use this method if your tool doesn't support unified client authentication. For details, see Manually generate OAuth U2M access tokens.

Automatic authorization with unified client authentication

To perform OAuth authorization with Azure Databricks SDKs and tools that support unified client authentication, integrate the following within your code:

Environment

To use environment variables for a specific Azure Databricks authentication type with a tool or SDK, see Authorizing access to Azure Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified client authentication and the Default methods for client unified authentication.

For account-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the value of your Azure Databricks account console URL, https://accounts.azuredatabricks.net.
• DATABRICKS_ACCOUNT_ID

For workspace-level operations, set the following environment variables:

• DATABRICKS_HOST, set to the value of your Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net.

Profile

Create or identify an Azure Databricks configuration profile with the following fields in your .databrickscfg file. If you create the profile, replace the placeholders with the appropriate values. To use the profile with a tool or SDK, see Authorizing access to Azure Databricks resources or the tool's or SDK's documentation. See also Environment variables and fields for unified client authentication and the Default methods for client unified authentication.

For account-level operations, set the following values in your .databrickscfg file. In this case, the Azure Databricks account console URL is https://accounts.azuredatabricks.net:

[<some-unique-configuration-profile-name>]
host       = <account-console-url>
account_id = <account-id>

For workspace-level operations, set the following values in your .databrickscfg file. In this case, the host is the Azure Databricks per-workspace URL, for example https://adb-1234567890123456.7.azuredatabricks.net:

[<some-unique-configuration-profile-name>]
host = <workspace-url>

CLI

For the Azure Databricks CLI, run the databricks auth login command with the following options:

• For account-level operations, --host https://accounts.cloud.databricks.com --account-id <account-id>.
• For workspace-level operations, --host <workspace-url>.

Then follow the instructions in your web browser to log in to your Azure Databricks account or workspace.

For more details, see OAuth authorization with the Databricks CLI.

VS Code

For the Databricks extension for Visual Studio Code, follow the steps in Set up authorization for the Databricks extension for Visual Studio Code.

Connect

OAuth U2M authentication is supported in Databricks Connect for Python starting with Databricks Runtime 13.1 and for Scala starting with Databricks Runtime 13.3 LTS.

For Databricks Connect, you can either:

• Use a config profile: Set workspace-level values in your .databrickscfg file as described on the Profile tab. Also set the cluster_id to your workspace instance URL.
• Use environment variables: Set the same values as shown on the Environment tab. Also set the DATABRICKS_CLUSTER_ID to your workspace instance URL.

Values in .databrickscfg take precedence over environment variables.

To initialize Databricks Connect with these settings, see Compute configuration for Databricks Connect.

Terraform

Before you apply your Terraform configuration, you must run one of the databricks auth login commands on the CLI tab depending on whether your configuration uses workspace or account operations. These commands generate and cache the required OAuth token at .databricks/token-cache.json in your user's home folder.

Account-level operations

For default authentication:

provider "databricks" {
  alias = "account"
}

For direct configuration:

provider "databricks" {
  alias      = "account"
  host       = <retrieve-account-console-url>
  account_id = <retrieve-account-id>
}

Replace the retrieve- placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as HashiCorp Vault. See also Vault Provider. In this example, you can set account_id to the Azure Databricks account console URL.

Workspace-level operations

For default authentication:

  provider "databricks" {
  alias = "workspace"
}

For direct configuration:

provider "databricks" {
  alias = "workspace"
  host  = <retrieve-workspace-url>
}

Python

Before you run your code, you must run the databricks auth login command on the CLI tab with either workspace or account operations options. These commands generate and cache the required OAuth token at .databricks/token-cache.json in your user's home folder.

Account-level operations

For default authentication:

from databricks.sdk import AccountClient

a = AccountClient()
# ...

For direct configuration:

from databricks.sdk import AccountClient

a = AccountClient(
  host       = retrieveAccountConsoleUrl(),
  account_id = retrieveAccountId()
)
# ...

Replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as Azure KeyVault.

Workspace-level operations

For default authentication:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient()
# ...

For direct configuration:

from databricks.sdk import WorkspaceClient

w = WorkspaceClient(host = retrieveWorkspaceUrl())
# ...

For more information about authenticating with Azure Databricks tools and SDKs that use Python and that implement Databricks client unified authentication, see:

• Set up the Databricks Connect client for Python
• Set up authorization for the Databricks extension for Visual Studio Code
• Authenticate the Databricks SDK for Python with your Azure Databricks account or workspace

Java

Before you run your code, you must run the databricks auth login command on the CLI tab with either workspace or account operations options. These commands generate and cache the required OAuth token at .databricks/token-cache.json in your user's home folder.

Account-level operations

For default authentication:

import com.databricks.sdk.AccountClient;
// ...
AccountClient a = new AccountClient();
// ...

For direct configuration:

import com.databricks.sdk.AccountClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveAccountConsoleUrl())
  .setAccountId(retrieveAccountId());
AccountClient a = new AccountClient(cfg);
// ...

Replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as Azure KeyVault.

Workspace-level operations

For default authentication:

import com.databricks.sdk.WorkspaceClient;
// ...
WorkspaceClient w = new WorkspaceClient();
// ...

For direct configuration:

import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
// ...
DatabricksConfig cfg = new DatabricksConfig()
  .setHost(retrieveWorkspaceUrl())
WorkspaceClient w = new WorkspaceClient(cfg);
// ...

For more information about authorizing and authenticating with Azure Databricks tools and SDKs that use Java and that implement Databricks client unified authentication, see:

• Set up the Databricks Connect client for Scala (uses the Azure Databricks SDK for Java for authentication)
• Authenticate the Databricks SDK for Java with your Azure Databricks account or workspace

Go

Before you run your code, you must run the databricks auth login command on the CLI tab with either workspace or account operations options. These commands generate and cache the required OAuth token at .databricks/token-cache.json in your user's home folder.

Account-level operations

For default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient())
// ...

For direct configuration:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
a := databricks.Must(databricks.NewAccountClient(&databricks.Config{
  Host:      retrieveAccountConsoleUrl(),
  AccountId: retrieveAccountId(),
}))
// ...

Replace the retrieve placeholders with your own implementation to retrieve the values from the console or some other configuration store, such as Azure KeyVault.

Workspace-level operations

For default authentication:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient())
// ...

For direct configuration:

import (
  "github.com/databricks/databricks-sdk-go"
)
// ...
w := databricks.Must(databricks.NewWorkspaceClient(&databricks.Config{
  Host: retrieveWorkspaceUrl(),
}))
// ...

For more information about authenticating with Databricks tools and SDKs that use Go and that implement Databricks client unified authentication, see Authenticate the Databricks SDK for Go with your Azure Databricks account or workspace.

Manually generate OAuth U2M access tokens

This section is for users working with third-party tools or services that don't support the Databricks unified client authentication standard. If you need to manually generate, refresh, or use Azure Databricks OAuth tokens for OAuth U2M authentication, follow the steps in this section.

Step 1: Generate a code verifier and challenge

To generate OAuth U2M access tokens manually, start by creating a code verifier and a matching code challenge. You'll use the challenge in Step 2 to get an authorization code, and the verifier in Step 3 to exchange that code for an access token.

The following Python script generates a verifier and challenge. Although you can use them multiple times, Azure Databricks recommends generating a new pair each time you manually generate access tokens.

import hashlib, base64, secrets, string

# Allowed characters for the code verifier, per PKCE spec
allowed_chars = string.ascii_letters + string.digits + "-._~"

# Generate a secure code verifier (43–128 characters)
code_verifier = ''.join(secrets.choice(allowed_chars) for _ in range(64))

# Create the SHA256 hash of the code verifier
sha256_hash = hashlib.sha256(code_verifier.encode()).digest()

# Base64-url-encode the hash and strip any trailing '=' padding
code_challenge = base64.urlsafe_b64encode(sha256_hash).decode().rstrip("=")

# Output values
print(f"code_verifier:  {code_verifier}")
print(f"code_challenge: {code_challenge}")

Step 2: Generate an authorization code

To get a Azure Databricks OAuth access token, you must first generate an OAuth authorization code. This code expires immediately after use. You can generate the code at either the account or workspace level:

• Account level: Use to call both account-level and workspace-level REST APIs across all workspaces that your user can access.
• Workspace level: Use to call REST APIs within a single workspace.

Generate an account-level authorization code

1. Locate your account ID.

2. In your browser, navigate to the URL with the following replacements:

   • <account-id>: Your Azure Databricks account ID
   • <redirect-url>: A local redirect URI (for example, http://localhost:8020)
   • <state>: Any plain-text string to validate the response
   • <code-challenge>: The code challenge from Step 1

   https://accounts.azuredatabricks.net/oidc/accounts/<account-id>/v1/authorize
   ?client_id=databricks-cli
   &redirect_uri=<redirect-url>
   &response_type=code
   &state=<state>
   &code_challenge=<code-challenge>
   &code_challenge_method=S256
   &scope=all-apis+offline_access
   

3. Log in when prompted to access your Azure Databricks account.

4. Copy the authorization code from your browser’s address bar. It’s the value after code= and before & in the redirect URL:

   http://localhost:8020/?code=dcod...7fe6&state=<state>
   

   Verify that the state value matches what you originally provided. If it doesn’t, discard the code.

5. Continue to Generate an account-level access token.

Generate a workspace-level authorization code

1. In your browser, navigate to the URL with the following replacements:

   • <databricks-instance>: Your <databricks-instance> with the Azure Databricks workspace instance name, for example adb-1234567890123456.7.azuredatabricks.net
   • <redirect-url>: A local redirect (e.g., http://localhost:8020)
   • <state>: Any plain-text value for response validation
   • <code-challenge>: The challenge string from Step 1

   https://<databricks-instance>/oidc/v1/authorize
   ?client_id=databricks-cli
   &redirect_uri=<redirect-url>
   &response_type=code
   &state=<state>
   &code_challenge=<code-challenge>
   &code_challenge_method=S256
   &scope=all-apis+offline_access
   

2. Log in when prompted to access your Azure Databricks account.

3. Copy the authorization code from your browser’s address bar. It’s the value after code= and before & in the redirect URL:

   http://localhost:8020/?code=dcod...7fe6&state=<state>
   

   Verify that the state value matches what you originally provided. If it doesn’t, discard the code.

4. Continue to Generate a workspace-level access token.

Step 3: Exchange the authorization code for an access token

To exchange the authorization code for a Azure Databricks OAuth access token, choose the appropriate level:

• Account level: Use to call both account-level and workspace-level REST APIs across all workspaces your user can access.
• Workspace level: Use to call REST APIs within a single workspace.

Generate an account-level access token

1. Use curl to exchange the account-level authorization code for an OAuth access token.

   Replace the following in the request:

   • <account-id>: Your Azure Databricks account ID
   • <redirect-url>: The redirect URL from the previous step
   • <code-verifier>: The verifier you generated earlier
   • <authorization-code>: The authorization code from the previous step

   curl --request POST \
   https://accounts.azuredatabricks.net/oidc/accounts/<account-id>/v1/token \
   --data "client_id=databricks-cli" \
   --data "grant_type=authorization_code" \
   --data "scope=all-apis offline_access" \
   --data "redirect_uri=<redirect-url>" \
   --data "code_verifier=<code-verifier>" \
   --data "code=<authorization-code>"
   

2. Copy the access_token value from the response. For example:

   {
     "access_token": "eyJr...Dkag",
     "refresh_token": "doau...f26e",
     "scope": "all-apis offline_access",
     "token_type": "Bearer",
     "expires_in": 3600
   }
   

   The token is valid for one hour.

3. Continue to Step 4: Call a Azure Databricks REST API.

Generate a workspace-level access token

1. Use curl to exchange the workspace-level authorization code for an OAuth access token.

   Replace the following in the request:

   • <databricks-instance>: Your <databricks-instance> with the Azure Databricks workspace instance name, for example adb-1234567890123456.7.azuredatabricks.net
   • <redirect-url>: The redirect URL from the previous step
   • <code-verifier>: The verifier you generated earlier
   • <authorization-code>: The workspace-level authorization code

   curl --request POST \
   https://<databricks-instance>/oidc/v1/token \
   --data "client_id=databricks-cli" \
   --data "grant_type=authorization_code" \
   --data "scope=all-apis offline_access" \
   --data "redirect_uri=<redirect-url>" \
   --data "code_verifier=<code-verifier>" \
   --data "code=<authorization-code>"
   

2. Copy the access_token value from the response. For example:

   {
     "access_token": "eyJr...Dkag",
     "refresh_token": "doau...f26e",
     "scope": "all-apis offline_access",
     "token_type": "Bearer",
     "expires_in": 3600
   }
   

   The token is valid for one hour.

Step 4: Call a Azure Databricks REST API

Use the access token to call account-level or workspace-level REST APIs, depending on its scope. To call account-level APIs, your Azure Databricks user must be an account admin.

Example account-level REST API request

This example uses curl along with Bearer authentication to get a list of all workspaces associated with an account.

• Replace <oauth-access-token> with the account-level OAuth access token.
• Replace <account-id> with your account ID.

export OAUTH_TOKEN=<oauth-access-token>

curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
"https://accounts.azuredatabricks.net/api/2.0/accounts/<account-id>/workspaces"

Example workspace-level REST API request

This example uses curl along with Bearer authentication to list all available clusters in the specified workspace.

• Replace <oauth-access-token> with the account-level or workspace-level OAuth access token.
• Replace <databricks-instance> with the Azure Databricks workspace instance name, for example adb-1234567890123456.7.azuredatabricks.net.

export OAUTH_TOKEN=<oauth-access-token>

curl --request GET --header "Authorization: Bearer $OAUTH_TOKEN" \
"https://<databricks-instance>/api/2.0/clusters/list"