Python Function Apps

Owned by Diego Cesa de Meira (Unlicensed)
Last updated: Sept 24, 2019 by Josh Hevenor (Unlicensed)
9 min read

Some of the configuration of eGIS, including post-install site configuration and TC Smart Maps configuration, is achieved through the use of Python scripts. This page describes how to create a development pipeline for a Python script.

Create a Python Azure Function

The first step is to create an Azure Function project.

  1. Ensure that you have Python 3.6 installed on your local machine. Azure Functions currently only supports Python 3.6.

  2. Install the Azure Functions extension for Visual Studio Code.

  3. In Visual Studio Code, click the Azure icon.

  4. Click Create New Project...

  5. Choose a folder in your repository in which to create the new Azure Function, and click Select.

  6. Choose HTTP trigger from the dropdown list.

  7. Specify a name for the HTTP trigger and press 'Enter'.

  8. Choose the Function authorization level for the HTTP trigger. This will ensure that the function can only be invoked if a function key is passed as a parameter.

  9. Code your Python function. Note that the function will be called via the main function in __init__.py.

  10. Add the Python packages on which your code is dependent to the requirements.txt file. See the pip documentation for more information on requirements files.

 

Create a Build Pipeline

Follow the instructions in Development Pipeline to create a development pipeline that's triggered when the Azure Function code is merged into the master branch. The trigger should be filtered on the folder that contains the Azure Function.

 

Create a Function App

  1. In the Azure portal, search for Function App. Click the Function App result.

     

  2. Click Add.

  3. Specify an App name, Subscription, and Resource Group for the Function App. Choose Linux for OS, Code for Publish, Consumption Plan for Hosting Plan, East US for Location, Pythonfor Runtime Stack, and, optionally, use an existing Storage account.

     

  4. Click Create.

  5. In the Azure Dashboard, click Refresh, then click the name of the function app that you just created.

     

Create a Release Pipeline

Deploy the Azure Function App

  1. Follow the instructions in Development Pipeline to create a service connection.

  2. In the Azure DevOps portal, click Pipelines.

  3. Click the context menu next to the most recent successful build, and click Release.

  4. In the Select a template panel, type Function in the Search box.

  5. Click Deploy a function app to Azure Functions.

  6. In the Stage 1 box, click 1 job, 1 task.

  7. Choose the target Azure service connection from the Azure subscription dropdown list.

  8. Choose Function App on Linux from the App type dropdown list (Python Function Apps are only supported on Linux).

  9. Choose the Function App that you created in the previous section from the App Service name dropdown list.

  10. Click Save.

  11. Optionally, specify a folder in which to store the pipeline, and add a comment, then click OK.

  12. Click Create release.

  13. Click Create. Azure DevOps will run the release pipeline, which will deploy the Python Function App.

Invoke the Azure Function

  1. In the Azure dashboard, click Refresh next to the Python Function App, and verify that the function has been deployed.

     

  2. Click the name of the Azure Function.

     

  3. Click </> Get function URL.

     

  4. Click Copy.

  5. In Notepad, paste the URL. The URL should include the function's base URL along with the function key.

    https://arcgispythonfunctionapp.azurewebsites.net/api/HttpTrigger?code=b5gt2BH3wPzWQt25RsBBqZ3Ejpjvmi/wMZ85lcvWo40emAKFbCWKAg==

  6. In Azure DevOps, On the Stage 1 context menu, click Add an agentless job.

     

  7. Click the plus sign to add a new task to the Agentless job.

     

  8. Next to Invoke Azure Function, click Add.

     

  9. Click Azure Function.

     

  10. Type the base URL to the Azure function in Azure function URL.

  11. Type the function key in Function key.

  12. Click Save.

     

  13. Click OK.

You've created a CI / CD pipeline that will automatically deploy and run your Python script when you merge it into the master branch in your Azure DevOps repository.

 

Redeploy Azure Function App

Once Azure Function App is setup, it is in maintenance stage. It might require to do bug fix, or add new functions in. After checking in the code from Visual Studio Code, you need to redeploy the function app. Visual Studio Code cannot be used to deploy Function App, it requires Docker to do so. Once the pipeline is setup, the checked in code should be automatically deployed. Sometimes there are errors with this automation step.

From DevOps, open Release-93, which has the error mark:

Open the Logs, and the error is listed:

There is a restriction though that Azure is expecting 20 seconds interval between the two deployment, e.g., from DEV to Sandbox. Once time out happens on DEV for example, deployment can still work on Sandbox, while Pilot and PROD require manual intervention before doing deployment.

For the possible errors and solutions, it will be added in RDIMS-#15688753.

 

Notes on Setup

Folder Structure

Any resources that are used by the function app must be contained in the function app's folder structure. They cannot be contained in a sibling or parent folder of the function app.

 

Build Configuration

Working Directory

In order to ensure that the Python dependencies are referenced correctly, set the Working Directory for the Install Application Dependencies task to be the function app folder.

 

Selective Archive

To ensure that only the function app's folder is published, and not the entire repository, set the root folder to archive to be the function app folder.

 

Retrieving Secrets from an Azure Key Vault

Some secrets, such as passwords for the eGIS environment, should be maintained in an Azure key vault. To access these from a Azure Python function app, they can be mapped to application settings on the Azure function app, and retrieved as environment variables from within the Python function app.

 

  1. Define an identity for the Azure function app.

    In the Azure portal, navigate to the function app, click the Platform features tab, and click Identity. Click the Status toggle to On, and click Save.

  2. Allow the application's identity to retrive secrets from the Azure key vault.

    In the Azure portal, navigate to the key vault, click Access policies, the click Add new. Click Select principal, and type the name of the app in the Search box. Click the name of the app in the search results (this is the identity created in the previous step), and click Select. IN the Secret permissions dropdown list, click Get and List. Click OK.

    Click Save to save the changes to the access policies.

  3. Map the key vault secrets to function app settings.

    In the Azure portal, navigate to the key vault, click Secrets, then click the secret that you want to use in the Azure app function. Click the current version, then click to copy the Secrete Identifier (a URL) to the clipboard.

    In the Azure portal, navigate to the function app, click Function app settings, then click Manage application settings. Under Application Settings, click New application setting. Type the Name of the new setting, and type a value for the new setting in the format:

    @Microsoft.KeyVault(SecretUri=<secret identifier url>)

    where <secret identifier url> is the URL that you copied above.

    Click OK.

  4. Retrieve the application setting in Python.

    In your Python code, use the following code to retrieve the secret from the application settings:

    import os password = os.getenv('EGIS_PASSWORD')

    In this way, you can securely retrieve secrets from the Azure key vault and use them in your Azure Python function apps.

 

Network Access to Portal Sites

  1. Azure Python function apps are currently in Preview, and thus are not available in all regions (including Canada Central). In order for Azure Python function apps to access the eGIS environments' portal URLs, we set up the following temporary solution:
    Create an app service plan in the East US region. Note that this app service plan has a fixed monthly cost regardless of usage. For production deployment, we will use a consumption plan (also in Preview) that will cost only per usage.

     

  2. Create the Azure function app using the app service plan.

     

  3. Determine the outbound IP addresses used by the Azure function app. Open the 1. Function App in the Azure portal, click on the 2. Platform features tab, and click on 3. Resource Explorer, Find the key named outboundIpAddresses in the configuration, and copy the set of IP addresses there.

4. Configure the eGIS environment's virtual network to allow HTTPS traffic from these IP addresses. In the Azure portal, navigate to the eGIS environment's web VM, click Networking,the modify hte HTTPS rule (port 443) to allow traffic from these IP addresses.

 

When these steps are completed, the Azure function app will have access to the eGIS environment's portal. Note that this is not the preferred production approach, since this approach requires that traffic be sent out to the Internet. When Azure python functions are made generally available, this approach should be changed so that the function app's service plan is integrated with the eGIS environment's virtual network.