{
"cells": [
{
"cell_type": "markdown",
"id": "9dc63b4a-ebb9-41be-bf43-ff4549f37cbd",
"metadata": {},
"source": [
"# Animating NEXRAD Level II Data\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "79d482d1-ae86-44bc-ae65-7abbc8afe278",
"metadata": {},
"source": [
"## Overview\n",
" \n",
"Within this notebook, we will cover:\n",
"\n",
"1. Exploring the \"guts\" of a NEXRAD radar file \n",
"1. Animating a sequence of AWS-served NEXRAD Level 2 Radar scans\n",
"\n",
"## Prerequisites\n",
"| Concepts | Importance | Notes |\n",
"| --- | --- | --- |\n",
"| [Cartopy Intro](https://foundations.projectpythia.org/core/cartopy/cartopy.html) | Required | Projections and Features |\n",
"| [Matplotlib Basics](https://foundations.projectpythia.org/core/matplotlib/matplotlib-basics.html) | Required | Basic plotting |\n",
"| [Py-ART Basics](../foundations/pyart-basics) | Required | IO/Visualization |\n",
"\n",
"- **Time to learn**: 20 minutes\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "44203db4-00ab-4fad-8b61-ead013282b80",
"metadata": {
"tags": []
},
"source": [
"## Imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "77872ced-4f00-4520-8e4e-651c7aa2961b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import pyart\n",
"import fsspec\n",
"from metpy.plots import USCOUNTIES, ctables\n",
"import matplotlib.pyplot as plt\n",
"import cartopy.crs as ccrs\n",
"import cartopy.feature as cfeature\n",
"import warnings\n",
"from datetime import datetime as dt\n",
"from datetime import timedelta\n",
"\n",
"import matplotlib\n",
"matplotlib.rcParams['animation.html'] = 'html5'\n",
"from matplotlib.animation import ArtistAnimation\n",
"\n",
"warnings.filterwarnings(\"ignore\")"
]
},
{
"cell_type": "markdown",
"id": "87af5f44-55e6-4d23-9523-6e49e2563f88",
"metadata": {},
"source": [
"## Select the time and NEXRAD site"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0e400af6-472d-435e-9a07-2b16978f63f4",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"datTime = dt(2014,5,26,22)\n",
"year = dt.strftime(datTime,format=\"%Y\")\n",
"month = dt.strftime(datTime,format=\"%m\")\n",
"day = dt.strftime(datTime,format=\"%d\")\n",
"hour = dt.strftime(datTime,format=\"%H\")\n",
"timeStr = f'{year}{month}{day}{hour}'\n",
"\n",
"site = 'KSJT'"
]
},
{
"cell_type": "markdown",
"id": "e53147e6-a08a-444b-a7df-9b2a958c4c3b",
"metadata": {},
"source": [
"Point to the [AWS S3 filesystem](https://aws.amazon.com/s3/)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "79521c33-842a-4eb5-b9f7-e57cf420ea96",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"fs = fsspec.filesystem(\"s3\", anon=True)"
]
},
{
"cell_type": "markdown",
"id": "70c7dd2e-893d-475f-8ee1-35571e4125bb",
"metadata": {},
"source": [
"Depending on the year, the radar files will have different naming conventions. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "39b43c6d-6177-439d-ba97-82e2dc619d34",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"pattern1 = f's3://noaa-nexrad-level2/{year}/{month}/{day}/{site}/{site}{year}{month}{day}_{hour}*V06'\n",
"pattern2 = f's3://noaa-nexrad-level2/{year}/{month}/{day}/{site}/{site}{year}{month}{day}_{hour}*V*.gz'\n",
"pattern3 = f's3://noaa-nexrad-level2/{year}/{month}/{day}/{site}/{site}{year}{month}{day}_{hour}*.gz'"
]
},
{
"cell_type": "markdown",
"id": "9fa4912e-9a5b-4972-ad14-6bd0025ca494",
"metadata": {},
"source": [
"Construct the URL pointing to the radar file directory and get a list of matching files."
]
},
{
"cell_type": "markdown",
"id": "fea66cf6-1d9e-4a72-8b89-ecbba7ae5277",
"metadata": {
"tags": []
},
"source": [
"Try each file pattern. Once the list of files is non-empty, we are all set."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4ba12bed-2963-453f-a98b-8b8e1d79f502",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"files = sorted(fs.glob(pattern1))\n",
"\n",
"if (len(files) == 0):\n",
" files = sorted(fs.glob(pattern2)) \n",
"\n",
"if (len(files) == 0):\n",
" files = sorted(fs.glob(pattern3)) "
]
},
{
"cell_type": "markdown",
"id": "29103e21-33df-443f-b9d2-a07f9d479634",
"metadata": {
"tags": []
},
"source": [
"If we still have an empty list, either there are no files available for that site/date, or the file name does not match any of the patterns above. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ca2573bb-f9be-4720-96c4-7f9c9c081c01",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"if (len(files) == 0):\n",
" print (\"There are no files found for this date and location. Either try a different date/site, \\\n",
"or browse the NEXRAD2 archive to see if the file name uses a different pattern.\")\n",
"else:\n",
" print (files)"
]
},
{
"cell_type": "markdown",
"id": "2df4be79-bba7-4d94-a4de-a13e4e2a247a",
"metadata": {},
"source": [
"## Read the Data into PyART\n",
"\n",
"Read in the first radar file in the group and list the available fields."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "01b86b2b-78e5-444b-b23b-a7322e6236c6",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radar = pyart.io.read_nexrad_archive(f's3://{files[0]}')\n",
"list(radar.fields)"
]
},
{
"cell_type": "markdown",
"id": "e5452855-71e1-47af-9aea-c38390191635",
"metadata": {},
"source": [
"The `radar` object has a lot of useful data ... and *metadata*! One way to look at the attributes of any Python object is to use the `vars` function. It returns a Python `dictionary`; each dictionary `key` has an associated value ... which might be a single value ... a list of values ... another dictionary ... and so on."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5cf0b6bf-0a52-4a7d-97e5-b54c8a31b1f3",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict = vars(radar)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7dd8a694-ca11-4fdf-bf39-eb89fd242084",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict"
]
},
{
"cell_type": "markdown",
"id": "ad662746-3852-4ac7-b13c-8f0163df0988",
"metadata": {},
"source": [
"We can get a list of all the individual keys in the dictionary:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba40b88a-1b89-43fd-b292-a8911c325a7e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict.keys()"
]
},
{
"cell_type": "markdown",
"id": "17b21ae3-9aa0-4949-b792-532ef516dd55",
"metadata": {},
"source": [
"The first key is `time`. Let's look at it:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5dfe99e1-a72e-40c3-b1f0-5ebcbffb5522",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict['time']"
]
},
{
"cell_type": "markdown",
"id": "f42b1d9e-4db8-43f6-b75b-10a3a18c952b",
"metadata": {},
"source": [
"It's also a Python dictionary! It's shorter than its parent's ... so we can see that there are six keys. Let's look at the first one, `units`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a479ab5a-23ca-414f-acb9-888a1418e5ef",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict['time']['units']"
]
},
{
"cell_type": "markdown",
"id": "30df8dd1-cf32-4311-934a-c5312824113c",
"metadata": {},
"source": [
"It's a string ... and a useful one ... as it provides a base time that we can use to infer timestamps for the start of each of the individual sweeps that make up the full volume scan of this particular radar file."
]
},
{
"cell_type": "markdown",
"id": "ce1632bd-88c3-4cc9-9550-c5c74c2429fb",
"metadata": {},
"source": [
"Let's look at `time`'s `data` key:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9151c63a-7e98-4b52-be48-7fd2a16a2eb8",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radar.time['data'] # equivalent to: radarDict['time']['data']"
]
},
{
"cell_type": "markdown",
"id": "1a0b2b99-85d6-4f81-86f6-92d358747415",
"metadata": {},
"source": [
"Note: Each key in the top-level radar dictionary can be accessed as an attribute in the radar object itself.
"
]
},
{
"cell_type": "markdown",
"id": "1203facf-52cc-4ecb-b9b6-d8663b7674e2",
"metadata": {},
"source": [
"Question: What does each value represent, in terms of time?
"
]
},
{
"cell_type": "markdown",
"id": "4d9c259b-f0d2-4805-bb6b-f26575163d5b",
"metadata": {},
"source": [
"How many elements are in this array?"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b6c723f-8485-4020-a54f-2f448a2368c5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"len(radar.time['data'])"
]
},
{
"cell_type": "markdown",
"id": "e7cb8e77-9d0b-4d29-87d4-a992dc058fba",
"metadata": {},
"source": [
"Other keys include `longitude` and `latitude`. Let's have a look at them:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5d982863-4b05-42d4-bb67-2ca810135d6c",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict['longitude'], radarDict['latitude']"
]
},
{
"cell_type": "markdown",
"id": "bcb0e298-4507-4b2e-9990-a03977e1cdb2",
"metadata": {},
"source": [
"These are also dictionaries! What do you think the `data` keys represent in both of them?"
]
},
{
"cell_type": "markdown",
"id": "86fabf32-766f-498f-922f-2a6351c13b5f",
"metadata": {},
"source": [
"Exercise: Write code cells to explore the `sweep_number` and `elevation` keys.
"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ea5f4799-8b90-4e8f-a942-17d584fad105",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict['sweep_number']"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4019a6c9-0884-4bae-b187-5d3269394d28",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict['elevation']"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "16dd7700-4c1c-4f83-8c86-87a2d9c96efa",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radarDict['elevation']['data']"
]
},
{
"cell_type": "markdown",
"id": "7f9b0902-b496-478c-bb25-ab948e04dac7",
"metadata": {},
"source": [
"What is the length of the data array associated with elevation?"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a7346813-8c3b-4071-9496-1101b956fc7d",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"len(radarDict['elevation']['data'])"
]
},
{
"cell_type": "markdown",
"id": "34d0109c-4331-40e9-ac09-ce7dbdd08c0e",
"metadata": {},
"source": [
"Consider: How does the array size for elevation and time compare to the array size for sweep index number? What do you think that ratio implies? Is it the same ratio for all sweeps?
"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b73e4762-73d1-40b9-a23e-c9af8baeb67b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"cLon, cLat = radar.longitude['data'], radar.latitude['data']\n",
"cLon, cLat"
]
},
{
"cell_type": "markdown",
"id": "a41d0c7b-52aa-4536-ba7e-e8f4a7304c3c",
"metadata": {},
"source": [
"Specify latitude and longitude bounds for the resulting maps, the resolution of the cartographic shapefiles, and the desired sweep level."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "10c457c5-de71-4cf5-b181-51ef2585f395",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"lonW = cLon - 2\n",
"lonE = cLon + 2\n",
"latS = cLat - 2\n",
"latN = cLat + 2\n",
"domain = lonW, lonE, latS, latN\n",
"\n",
"res = '10m'\n",
"sweep = 0"
]
},
{
"cell_type": "markdown",
"id": "b88c92a1-80bc-482d-8105-a3e22b474717",
"metadata": {},
"source": [
"Define a function that will determine at which ray a particular sweep begins; also define some strings for the figure title."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7a2062b8-35f9-444b-bc4d-b7407da6cf16",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"def nexRadSweepTimeElev (radar, sweep):\n",
" \n",
" sweepRayIndex = radar.sweep_start_ray_index['data'][sweep]\n",
"\n",
" baseTimeStr = radar.time['units'].split()[-1]\n",
" baseTime = dt.strptime(baseTimeStr, \"%Y-%m-%dT%H:%M:%SZ\")\n",
"\n",
" timeSweep = baseTime + timedelta(seconds=radar.time['data'][sweepRayIndex])\n",
"\n",
" timeSweepStr = dt.strftime(timeSweep, format=\"%Y-%m-%d %H:%M:%S UTC\")\n",
"\n",
" elevSweep = radar.fixed_angle['data'][sweep]\n",
" elevSweepStr = f'{elevSweep:.1f}°'\n",
" return timeSweepStr, elevSweepStr"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "509e3137-9773-4d3c-9ce2-e3143abf4fbf",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radar.sweep_start_ray_index"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1625e8a7-6afe-4089-8f45-7602d0ffc304",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"radar.sweep_number"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "23f3764b-1ebc-493a-8f4d-752461127a5b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"field = 'reflectivity'\n",
"shortName = 'REFL'"
]
},
{
"cell_type": "markdown",
"id": "b477f6a0-dd49-4455-aec9-12f0e7a137a2",
"metadata": {},
"source": [
"## Create a single figure of reflectivity, zoomed into the area of interest.\n"
]
},
{
"cell_type": "markdown",
"id": "e8526e21-5c50-4337-bb2c-0ee01d2d3eb4",
"metadata": {},
"source": [
"PPI Display We construct a pseudo Plan Position Indicator (PPI) display. A true PPI display has the radar antenna at its center, with distance from it and the height above ground drawn as concentric circles (Source:
Wikipedia)
"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d9ddb189-ecaa-47d6-8e26-c682be63f1ef",
"metadata": {},
"outputs": [],
"source": [
"# Creating color tables for reflectivity (every 5 dBZ starting with 5 dBZ):\n",
"ref_norm, ref_cmap = ctables.registry.get_with_steps('NWSReflectivity', 5, 5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "260f1f9b-8ad8-44ae-9ae2-76ccd93f5764",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"ref_cmap"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5e8df98-d041-451d-bae2-832bfbf68fad",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Call the function that creates the title string, among other things.\n",
"timeSweepStr, elevSweepStr = nexRadSweepTimeElev (radar, sweep)\n",
"titleStr = f'{site} {shortName} {elevSweepStr} {timeSweepStr}'\n",
"\n",
"# Create our figure\n",
"fig = plt.figure(figsize=[15, 6])\n",
"\n",
"# Set up a single axes and plot reflectivity\n",
"ax = plt.subplot(111, projection=ccrs.PlateCarree())\n",
"ax.set_extent ([lonW, lonE, latS, latN])\n",
"\n",
"display = pyart.graph.RadarMapDisplay(radar)\n",
"ref_map = display.plot_ppi_map(field,sweep=sweep, vmin=20, vmax=80, ax=ax, raster=False, title=titleStr,\n",
" colorbar_label='Equivalent Relectivity ($Z_{e}$) (dBZ)', norm=ref_norm, cmap=ref_cmap, resolution=res)\n",
"# Add counties\n",
"ax.add_feature(USCOUNTIES, linewidth=0.5);"
]
},
{
"cell_type": "markdown",
"id": "c7bb5fb4-beea-45b9-a4b0-7b7d9181baab",
"metadata": {},
"source": [
"Next, let's create a function to create a plot, so we can loop over all the radar files of the specified hour."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b9218c65-bb8d-46ef-89ce-26afa9e14daf",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"def plot_radar_refl (idx, site, radar):\n",
" proj = ccrs.PlateCarree()\n",
" # New axes with the specified projection\n",
" ax = fig.add_subplot(111, projection=proj)\n",
" ax.set_extent(domain)\n",
" ax.add_feature(USCOUNTIES.with_scale('5m'),edgecolor='grey', linewidth=1, zorder = 3 )\n",
" display = pyart.graph.RadarMapDisplay(radar)\n",
" ref_map = display.plot_ppi_map(field,sweep=sweep, cmap=ref_cmap, norm=ref_norm, ax=ax, colorbar_flag = False, \n",
" title_flag = False, colorbar_label='Equivalent Relectivity ($Z_{e}$) (dBZ)', resolution=res)\n",
" return ax "
]
},
{
"cell_type": "markdown",
"id": "147e85e7-cf3b-4554-8a47-7603f5e6a4e2",
"metadata": {},
"source": [
"Color bar omitted Embedding the colorbar into an animation sequence is non-trivial ... so we do not include a colorbar here.
"
]
},
{
"cell_type": "markdown",
"id": "fee4c8fd-e0d4-4dc9-a6ce-35bf2ea89fa9",
"metadata": {},
"source": [
"Loop over the files. Save each image."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5bc36464-9c42-4f89-a442-414b9d462bd8",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"backend_ = matplotlib.get_backend() \n",
"backend_"
]
},
{
"cell_type": "markdown",
"id": "eae9fbd8-4273-45bd-84dc-ed390f4ab2de",
"metadata": {
"tags": []
},
"source": [
"Set the Jupyter matplotlib backend to one that will not show the graphics inline (this will save time in notebook execution)"
]
},
{
"cell_type": "markdown",
"id": "dfa92865-0c3d-4db2-9518-2e580e7a249d",
"metadata": {},
"source": [
"Be patient! For an hour's worth of radar files (which can be 15 or so if the radar is in precip mode) this cell, and the animation creation cell that follows, will take a few minutes!
"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "25ff9711-2d97-4840-acbc-2e4a5bb3b0c5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"matplotlib.use(\"Agg\") # Prevent showing stuff\n",
"\n",
"meshes = []\n",
"fig = plt.figure(figsize=(10,10))\n",
"for n, name in enumerate(files[:]):\n",
" print (n, name, site)\n",
" radar = pyart.io.read_nexrad_archive(f's3://{name}') \n",
" ax1 = plot_radar_refl(n, site, radar)\n",
" timeSweepStr, elevSweepStr = nexRadSweepTimeElev (radar, sweep)\n",
" titleStr = f'{site} {shortName} {elevSweepStr} {timeSweepStr}' \n",
" print (titleStr)\n",
" title = ax1.text(0.5,0.92,titleStr,horizontalalignment='center',\n",
" verticalalignment='center', transform=fig.transFigure, fontsize=15)\n",
" meshes.append((ax1,title))"
]
},
{
"cell_type": "markdown",
"id": "974bb09f-4d70-4d4d-b39b-ffbfadb05db1",
"metadata": {},
"source": [
"Set the Jupyter matplotlib backend to the default value, so we see output once again."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "88291ec2-d401-4cb9-b527-3e3cd0a5f7cc",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"matplotlib.use(backend_)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4f1ac794-8bf1-4bc3-b10b-06cfa192fc47",
"metadata": {},
"outputs": [],
"source": [
"anim = ArtistAnimation(fig, meshes)"
]
},
{
"cell_type": "markdown",
"id": "55f9271d-7c5f-4568-9ec1-0b137085a297",
"metadata": {
"tags": []
},
"source": [
"Display the animation (this may take some time, depending on the number of frames)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4c492e6e-0d68-496e-90ad-d01bc6038fba",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"anim"
]
},
{
"cell_type": "markdown",
"id": "644304a2-6f51-428c-9723-e3041fe45133",
"metadata": {},
"source": [
"Save the animation to the current directory (this may also take some time)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bf82afc2-4040-4559-97b8-cd403549555a",
"metadata": {},
"outputs": [],
"source": [
"anim.save(f'{site}_{timeStr}_refl.mp4')"
]
},
{
"cell_type": "markdown",
"id": "c268e3ac-6532-4d2f-b620-c79d08364cdb",
"metadata": {},
"source": [
"---\n",
"### Things to try\n",
"Create an animation for a time and site of interest.\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 August 2022 Environment",
"language": "python",
"name": "aug22"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}