This page gives an overview of all public Spark SQL API.

Spark SQL is a Spark module for structured data processing. It provides a programming abstraction called DataFrames and can also act as a distributed SQL query engine. It enables unmodified …

Nov 6, 2025 · Apache Spark is the technology powering compute clusters and SQL warehouses in Azure Databricks. This page provides an overview of the documentation in this section.

Nov 19, 2025 · With Spark DataFrames, you can efficiently read, write, transform, and analyze data using Python and SQL, which means you are always leveraging the full power of Spark.

Dec 1, 2025 · This page provides an overview of reference available for PySpark, a Python API for Spark. For more information about PySpark, see PySpark on Azure Databricks.

2 days ago · Learn Databricks System Design in this guide covering Lakehouse architecture, pipelines, ML workflows, reliability, scaling, and interview prep.

Nov 30, 2025 · PySpark reference This page provides an overview of reference available for PySpark, a Python API for Spark. For more information about PySpark, see PySpark on Databricks.

Nov 19, 2025 · Spark SQL allows you to mix SQL queries with Spark programs. With Spark DataFrames, you can efficiently read, write, transform, and analyze data using Python and SQL, …

This tutorial will familiarize you with essential Spark capabilities to deal with structured data typically often obtained from databases or flat files. We will explore typical ways of querying and aggregating …

Dec 2, 2025 · Use Spark SQL or DataFrames to query data in this location using file paths. To learn more about Databricks-provided sample data, see Sample datasets. Many PySpark operations …