Development & Ingestion

A client library that lets you connect a local IDE, application, or non-notebook environment to remote Databricks compute, running DataFrame operations on the cluster while you write code locally.

Developing and debugging Spark/PySpark code in a local IDE such as VS Code or PyCharm while it executes against a real Databricks cluster.

DataFrame operations are sent to and run remotely on Databricks compute; only the results are returned to the local client.

The workspace URL, authentication (a personal access token or OAuth), and a target compute — a specific cluster ID or serverless.

Personal access tokens (PAT) and OAuth-based authentication.

When you want full IDE features — code completion, refactoring, unit testing, version control, and step-through debugging — while still running on cluster compute.