Purge Azure CDN with Event Grid and Azure Automation

Azure Automation
Event Grid

In this post, I demonstrate how to automatically purge content from an Azure Content Delivery Network (CDN) using Event Grid and Azure Automation.  I am writing this post with two audiences in mind.  First, if you have a CDN and need to purge old content every time content is updated or removed from a source, this post will show you how.   However, if you don’t have a CDN but are interested in understanding how to configured Event Grid to trigger automation jobs in an Azure, this post is also for you.

Background

A CDN distributes and caches content across the globe, providing lower latency delivery and reducing global bandwidth usage.  It may be necessary to rapidly refresh the cache when new content is added to the source.  This post will demonstrate how to do that using Event Grid and Azure Automation.

If you would like to follow along at home, you will need an Azure CDN Configured with an Azure Storage Account V2 as the source.  I have a post demonstrating how to set that up here. You will also need an Azure Automation account.  If you don’t have an Azure Automation account, I have a whole playlist on setting one up here.  The Azure Automation runbook used in this example can be found here.

Flow Overview

The basic flow of operations is as outlined below:

  1. Content added (or removed) from the Storage Account blob source.
  2. Add or Delete action on the blob storage triggers Event Grid.
  3. Event Grid runs a web hook for the Azure Automation runbook, passing JSON data including file path.
  4. The Automation runbook parses JSON and runs a purge job against the path.
Event Grid Flow

Azure Automation

The web hook is required to set up the Event Grid action, so let’s start there.  Start by going to your Azure Automation account and create a runbook.  

Create Runbook

Create Purge Runbook

For this example, I’m only going to publish a single write-output command as a placeholder.  Remember, for now, I only need a web hook for the runbook.  We will get to the working runbook shortly.  Save and publish the runbook.

PowerShell Runbook
Publish Temp Runbook

Now that we have a published runbook we can create a web hook to trigger it.  From the runbook, go to the Add web hook at the top of the page.

This will take you to the first of two steps to create a web hook.  Give the new web hook a name, make sure it’s set to enabled and set an expiration date.  Note the date, the web hook will stop working after that date.  Lastly, copy and paste the URL someplace secure.  You will not be able to retrieve this URL once the web hook has been created.  If you lose it, a new one will need to be created.

Webhook
Create Web Hook

Click OK to move to the next step.  Specify the run environment and parameters in the Modify run settings section.  If the Runbook required parameters or needs to run on a hybrid worker, that could be set here.  Neither applies to this example, so click OK and Create to add the web hook.

The CDN module is required to run the purge command.  Add that by going into Modules Gallery in the automation account.  Search for CDN.  Add the Az.CDN module. 

Azure Automation Module
Add CDN Module

If you are using the default AzureRM commands in the Automation Account, add the AzureRM.CDN module instead.  Also change the Unpublish-AzCDNEndpointContent command in the script to the AzureRM command: Unpublish-AzureRMCDNEndpointContent.

Event Grid

Now that we have a web hook, let’s move onto setting up Event Grid.  Start by going to the Storage Account used as the CDN source. Look for Events on the sidebar of the Storage Account.

Event Grid
Storage Events

This will open the Events, Get Started page.  Select the More Options option.

Event Grid More Options
Events – More Options

You will see multiple options to choose from.  Select the Web Hook option.

Web Hook Option

Give the Event Subscription a name.  Leave the Event Schema and Topic Details as default.  Notice that the Event Types have the option of Blob Created or Blob Deleted.  Leave both selected.  Under Endpoint Details, select Web Hook.

Event Grid Subscription
Create Event Subscription

Next, under Web Hook, click on Select and endpoint.  This will bring up the Subscriber Endpoint.  Enter the Web Hook URL created in the previous step.  Click Confirm Selection and Create to add the event.

Select Web Hook

After the event is created, go to Event Subscriptions to verify.  Notice you can watch the activity from the Event Subscriptions page.  It is also possible to modify the event by clicking on the event name.

Event Subscriptions

Test the Event

Now that the web hook is in place, let’s test.  The event will trigger when new blobs are uploaded or deleted from the container.  To test, simply upload or remove content from the container.  The example below uses Storage Explorer to download and then upload index.html.  This adds content and triggers the event.

Blob Upload

Events now show activity as shown below.

Event Triggered

Go to the Azure Automation Runbook.  Under Jobs, you will see a job completed.  This is the job Event Grid triggered.

Azure Automation Job Triggered

Click on the Job and go to Output.  You will see that the job output, in this example a simple message.

Runbook Output

The output is not very useful.  More important is the input.  Notice the input WEBHOOKDATA  is JSON Code with details about the event.  Copy the data into an editor, such as Visual Studio Code and save it as a .json file.  I used the Prettier VS Code extension to format the content as JSON.

Runbook Input
Formatted JSON Data

Under RequestBody, there are a couple of values that give path information.  This information is used below to run a purge against a specific file each time it’s uploaded or deleted.  This example will use the value assigned to “url” to build a path for the purge operation.

URL Path

Azure Automation Runbook

Almost there! The Event Trigger is configured to fire a runbook.  The runbook is configured and we have the parameter needed to run the purge command.  The last step is to build a runbook. 

Below I step through each section of the runbook.  If you are unfamiliar with creating runbooks, I have the YouTube Playlist for you! The complete runbook is available on my GitHub site here .

The first section in the script is the comments.  After that is the parameter section.  This adds the JSON input to the $Web hookData variable for use later in identifying the file path.

Azure Automation Runbook
Parameter Section

After the parameter section is the authentication section.  This authenticates the runbook with the Azure Automation Run As account, giving it permissions to run the purge command.

Azure Automation Runbook
Authentication Section

The next section assigns the CDN Profile name, Resource Group and Endpoint Name to variables.  Update these fields with values from your environment.  I also set the default error action to a variable, this is used for error handling. 

Azure Automation Runbook
Set Variables

After that, I manipulate the JSON data to get the correct file path.  The data I need is in the RequestBody section of the JSON, the first step is to assign that data to a new variable, $requestbody

Azure Automation Runbook
Variable requestBody

Once that’s assigned, I assign the value of “url” located in data to the $filePath variable.  The value of url includes the full FQDN and file path.  I don’t need the FQDN for the purge command, only the file path.  The replace command is used to replace the FQDN with nothing, removing the part of the url I don’t need. For example, if this is the full url:

https://cdntestcir.blob.core.windows.net/source/index.html

replacing “https://cdntestcir.blob.core.windows.net” with “nothing” will leave “/source/index.html”, the path needed for the purge command.  Update the url value from your environment.

The full command looks like below. 

Azure Automation Runbook
Convert File Path

The try, catch blocks are used for error handling.  Errors encountered as the runbook runs are written to the Error stream.  For more information on the Try, Catch, Finally, blocks, see “The Big Book of PowerShell Error Handling” available from PowerShell.org https://leanpub.com/u/devopscollective

Last, the unpublish-AzCDNEndpointContent command is used to purge the file.  This command takes a minute or two to run. 

Unpublish-AzCdnEndpointContent Command

Final Testing

Test by updating or deleting a file from the CDN Source Storage Account.  This will trigger Event Grid to send the web hook and cause the Runbook to run.  The Azure Automation runbook will report the completed job once finished.

Completed Job

The Activity Log on the CDN Endpoint will show a successful purge.  Clicking on the drop-down arrow will show details of the job.

Event Grid and Azure Automation
Successful Purge

Summary

Here we are at the end.  This post demonstrated how to use Event Grid and Azure Automation to automate the process of purging content from a CDN when content is deleted from the source or new content is added.  The CDN use case is fairly specific, but Event Grid is integrated into many Azure services and the same workflow could be applied to a multitude of services to automate within Azure.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.