Overview

A dev instance was created using Azure Portal, following the steps outlined in the Azure quickstart - create a storage account

Find it at Home/Resource groups/TC-DevTest-ArcGIS-RG/egisdevdatalake

Sample data

Flight delay data has been downloaded for each year from 2005 to 2008, and loaded into this data lake using the storage explorer . More years are available, but these 4 years were over 2GB so I stopped there.

Airport lat/lon from OpenFlights have been uploaded to /reference/.

The TC-eGIS group in azure active directory has inherited access as a contributor.

Test job

Following the getting started with Apache Hadoop... tutorial.

On hold Tried to use only the Gen 2 data lake referenced above, but required a managed service identity . I'll need to coordinate with the cloud team to continue down that road. Tried to use a standard storage account and joining the data lake as additional storage to get started, but the deployment failed.

Update Should have credentials now. Am blocked trying to connect as a big data file share, but those credetntials should work here. Putting on the shelf until I can regroup with the team.