Azure Databricks, Pandas, and open datasets (Pragmatic AI Solutions, 2022)
Azure Databricks with Pandas and Open Datasets. Find out how to get a working cluster with Databricks using Azure and then use the full Pandas API operating in the cluster with Open Datasets and a Python Jupyter Notebook. This video will walk you through creating a workspace in Azure to create the Databricks service, then create the cluster that comes with the Pyspark Pandas API, and finally import the open datasets into the cluster. Although straightforward to create a Databricks cluster with Azure, it is a bit more involved to run a Python Jupyter Notebook that has Azure ML Open Datasets installed and availabe in the cluster along with the ability to use the full Pandas API you are used to working with and taking advantage of the clustering capabilities from Databricks.