Data Transformation and Modeling

A temporary view (createOrReplaceTempView) is session-scoped — visible only within the current SparkSession and gone when the session ends.

df.createOrReplaceTempView("view_name"); then query it with SQL like SELECT * FROM view_name.

A global temp view (createGlobalTempView) is tied to the cluster/application and shared across all sessions; it lives in the global_temp database.

Reference it with the global_temp prefix: SELECT * FROM global_temp.view_name.

CREATE VIEW view_name AS SELECT ...; it stores the query definition in the metastore and is recomputed when queried.

Use df.select("col1", "col2") or df.select(col("col1"), col("col2")).