{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Xarray 8: GRIB Data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Overview\n",
"\n",
"1. Explore the differences between GRIB and NetCDF formats\n",
"1. Work with GRIB output from the HRRR regional model\n",
"1. Visualize data that is non-global in extent"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"| Concepts | Importance | Notes |\n",
"| --- | --- | --- |\n",
"| Xarray Lessons 1-7| Necessary | |\n",
"\n",
"* **Time to learn**: 30 minutes\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Imports"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from datetime import datetime\n",
"import xarray as xr\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"from matplotlib import colors\n",
"import cartopy.crs as ccrs\n",
"import cartopy.feature as cfeat\n",
"from metpy.plots import USCOUNTIES"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## GRIB vs NetCDF formats\n",
"\n",
"Advantages of GRIB:\n",
"1. WMO-certifed format. Almost all NWP output that is generated by a national meteorological service will be in GRIB format.\n",
"2. Compresses well (GRIB version 2 only)\n",
"3. Still used for initial / boundary conditions in WRF.\n",
"4. \"Well-behaved\" GRIB data can be fairly easily translated into self-describing NetCDF format. A THREDDS server performs this translation on demand, thus presenting the user with data that looks to be in NetCDF format even if the actual data is in GRIB.\n",
"\n",
"Disadvantages of GRIB:\n",
"1. Not self-describing (unlike NetCDF). A GRIB data file must be accompanied by a set of tables that decodes fields into their actual physicall-relevant names. \n",
"2. Despte being a WMO standard, there is no central source to maintain such tables. Often times, as new models get released, the existing tables that one might be using may not be updated with new parameters. Best strategy: update your GRIB software libraries often (and hope other things don't break as a result).\n",
"3. As tables change, there is no guarantee that a GRIB file you read one year might decode the same way another year.\n",
"4. Grids are written out sequentially to an output GRIB file. As a result, performing temporal and especially spatial subsetting, such as we can do in Xarray with NetCDF or Zarr data, is very difficult."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Examine GRIB output from the HRRR regional model\n",
"\n",
"We have downloaded a small subset of HRRR model output from the U. of Utah's HRRR Archive and placed it in `/spare11/atm533/data`. The data consists of analysis and 1-6 hour forecast output from the 1600 UTC run of the HRRR on 7 October 2020; a period when eastern New York experienced a derecho-like event, very unusual for the time of year."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Specify the date and time; convert from `datetime` format to `string`."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"datetime.datetime(2020, 10, 7, 16, 0)"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"date = datetime(2020,10,7,16)\n",
"date"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'2020100716'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"YYYYMMDDHH = date.strftime('%Y%m%d%H')\n",
"YYYYMMDDHH"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Set forecast hour and convert to a string; pad string with a leading 0 if < 10 (see https://stackoverflow.com/questions/39402795/how-to-pad-a-string-with-leading-zeros-in-python-3/39402910)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'04'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fhr = 4\n",
"fhrStr = f'{fhr:02}'\n",
"fhrStr"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Open the GRIB2 files containing forecasted composite reflectivity data. We use Xarray's `cfgrib` engine to read in this type of data.\n",
"There is also a file containing u- and v- components of the wind at 10 and 80 m above ground level."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"refcFile = '/spare11/atm533/data/' + YYYYMMDDHH + '_hrrr_REFC.grib2'\n",
"windFile = '/spare11/atm533/data/' + YYYYMMDDHH + '_hrrr_WIND.grib2'"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'/spare11/atm533/data/2020100716_hrrr_REFC.grib2'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"refcFile"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"ds = xr.open_dataset(refcFile,engine='cfgrib')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is another Xarray Dataset, consisting of data variables (one in this case), coordinate variables, dimensions and attributes. \n",
"However, the dimension names and coordinate variables look different than in ERA-5."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
<xarray.Dataset>\n", "Dimensions: (step: 7, y: 1059, x: 1799)\n", "Coordinates:\n", " time datetime64[ns] ...\n", " * step (step) timedelta64[ns] 00:00:00 01:00:00 ... 05:00:00 06:00:00\n", " atmosphere float64 ...\n", " latitude (y, x) float64 ...\n", " longitude (y, x) float64 ...\n", " valid_time (step) datetime64[ns] ...\n", "Dimensions without coordinates: y, x\n", "Data variables:\n", " refc (step, y, x) float32 ...\n", "Attributes:\n", " GRIB_edition: 2\n", " GRIB_centre: kwbc\n", " GRIB_centreDescription: US National Weather Service - NCEP\n", " GRIB_subCentre: 0\n", " Conventions: CF-1.7\n", " institution: US National Weather Service - NCEP\n", " history: 2023-09-26T19:34 GRIB to CDM+CF via cfgrib-0.9.1...
<xarray.DataArray 'refc' (step: 7, y: 1059, x: 1799)>\n", "[13335987 values with dtype=float32]\n", "Coordinates:\n", " time datetime64[ns] ...\n", " * step (step) timedelta64[ns] 00:00:00 01:00:00 ... 05:00:00 06:00:00\n", " atmosphere float64 ...\n", " latitude (y, x) float64 ...\n", " longitude (y, x) float64 ...\n", " valid_time (step) datetime64[ns] ...\n", "Dimensions without coordinates: y, x\n", "Attributes: (12/33)\n", " GRIB_paramId: 260390\n", " GRIB_dataType: fc\n", " GRIB_numberOfPoints: 1905141\n", " GRIB_typeOfLevel: atmosphere\n", " GRIB_stepUnits: 1\n", " GRIB_stepType: instant\n", " ... ...\n", " GRIB_name: Maximum/Composite radar reflect...\n", " GRIB_shortName: refc\n", " GRIB_units: dB\n", " long_name: Maximum/Composite radar reflect...\n", " units: dB\n", " standard_name: unknown
<xarray.DataArray 'step' (step: 7)>\n", "array([ 0, 3600000000000, 7200000000000, 10800000000000,\n", " 14400000000000, 18000000000000, 21600000000000], dtype='timedelta64[ns]')\n", "Coordinates:\n", " time datetime64[ns] ...\n", " * step (step) timedelta64[ns] 00:00:00 01:00:00 ... 05:00:00 06:00:00\n", " atmosphere float64 ...\n", " valid_time (step) datetime64[ns] ...\n", "Attributes:\n", " long_name: time since forecast_reference_time\n", " standard_name: forecast_period
<xarray.DataArray 'y' (y: 1059)>\n", "array([ 0, 1, 2, ..., 1056, 1057, 1058])\n", "Coordinates:\n", " time datetime64[ns] ...\n", " atmosphere float64 ...\n", "Dimensions without coordinates: y
<xarray.DataArray 'x' (x: 1799)>\n", "array([ 0, 1, 2, ..., 1796, 1797, 1798])\n", "Coordinates:\n", " time datetime64[ns] ...\n", " atmosphere float64 ...\n", "Dimensions without coordinates: x