Access Connector for Azure Databricks: Installation and Configuration Guide

To install the Access Connector for Azure Databricks, you'll need to register an Azure Active Directory (AAD) application. This application will serve as the authentication mechanism for your connector.

The Access Connector for Azure Databricks supports authentication using Azure Active Directory (AAD), which allows you to use your existing AAD credentials to authenticate with your Azure Databricks workspace.

First, create an AAD application in the Azure portal. You can do this by navigating to Azure Active Directory in the Azure portal, clicking on "App registrations", and then clicking on "New application."

Once you've created the AAD application, you'll need to configure the necessary permissions and consent for your Azure Databricks workspace.

The Azure CLI is a powerful tool for managing Azure resources, including the Azure Databricks Access Connector. You can use the `az databricks access-connector` command to create, delete, list, and update access connectors.

To increase logging verbosity and show all debug logs, simply use the `az --debug` command. This will provide you with detailed information about the commands you're running.

To delete an access connector, you'll need to provide either the `--ids` argument or one or more resource IDs. This will ensure that you're deleting the correct connector.

One of the most useful features of the `az databricks access-connector` command is its ability to list all access connectors within a subscription. You can also use the `--total-items` argument to specify the total number of items to return in the command's output.

If you need to paginate the output, you can use the `--next-token` argument to specify where to start paginating. This is the token value from a previously truncated response.

Here's a summary of the `az databricks access-connector` command:

To update an access connector, you can use the `--add`, `--remove`, and `--set` arguments to add, remove, or update properties of the connector. This is a powerful feature that allows you to customize your access connectors to meet your specific needs.

In the next section, we'll explore some examples of how to use the `az databricks access-connector` command to create access connectors with different types of identities.

Authentication is supported through token-based authentication. You'll need to generate a personal access token for the service account, which can be done by following the steps in Databricks documentation.

To generate the token, you'll need to create a personal access token for the workspace user. This will be used as the password when configuring the data source connection from Alation.

The personal access token should be saved in a secure location, as it will be used to authenticate the connection.

To authenticate with Databricks, you'll need to generate a personal access token for your service account. This token will serve as your password when configuring the data source connection.

The process involves following the steps in Databricks documentation to create the token and storing it in a secure location. This is crucial for maintaining data security.

You'll use the personal access token as your password when setting up the data source connection from Alation. This ensures that your data remains secure and protected.

For a seamless authentication experience, make sure to save the personal access token in a secure location, as you'll need it for future connections.

To connect to your Azure Databricks data source in Alation, you'll need to populate the data source connection information. This involves specifying the JDBC URI, which requires a specific format.

The JDBC URI should be specified in the required format, and you can find more information on this in the Alation documentation.

For authentication, you'll use a personal access token as your password. This token should be generated for the service account and saved in a secure location.

You'll use the token as the password when configuring your data source connection. This means you'll paste the personal access token for the service account in the Password field.

Here's a summary of the data source connection parameters:

By following these steps, you'll be able to establish a secure connection to your Azure Databricks data source in Alation.

To set up query log ingestion for your Azure Databricks data source, you'll need to configure it in both Databricks and Alation.

You can find the instructions for configuring query log ingestion in Databricks by referring to the Azure Databricks OCF Connector: Query Log Ingestion article.

To start configuring query log ingestion in Alation, you'll need to create and configure a new data source.

Here are the steps to create and configure a new data source in Alation:

Next, you'll need to configure the settings of your data source, including query log ingestion.

Here are the settings you'll need to configure:

To configure query log ingestion, you'll need to populate the data source connection information, including the JDBC URI, Username, and Password.

Here are the details you'll need to enter:

Once you've populated the data source connection information, you can configure the query log ingestion options for your data source, including scheduling the QLI job if necessary.