{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Interactive visualization of worldwide METAR data: Geoviews"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Overview:\n",
"1. Create an interactive visualization of worldwide METAR data using the [Holoviz](https://holoviz.org) ecosystem"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"| Concepts | Importance | Notes |\n",
"| --- | --- | --- |\n",
"| Pandas | Necessary | |\n",
"| Contextily | Helpful| |\n",
"\n",
"* **Time to learn**: 30 minutes\n",
"***"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from datetime import datetime\n",
"import numpy as np\n",
"import pandas as pd\n",
"import geoviews as gv\n",
"import geoviews.feature as gf\n",
"from geoviews import opts\n",
"import geoviews.tile_sources as gts\n",
"from cartopy import crs as ccrs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create an interactive visualization of worldwide surface meteorological (METAR) data using the Holoviz ecosystem"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[Holoviz](https://holoviz.org) is a suite of open-source Python libraries designed for interactive data analysis and visualization via the browser (including the Jupyter notebook). In this notebook, we will use the [GeoViews](http://geoviews.org/) package, which is part of Holoviz."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Another part of the Holoviz ecosystem is [bokeh](https://bokeh.org/). Bokeh leverages Javascript in order to accomplish interactivity via the browser. GeoViews makes available Bokeh as well as Matplotlib via an extenstion. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"gv.extension('bokeh', 'matplotlib')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use Pandas to read in the file containing the most recent METAR data. \n",
"Since this file has latitude and longitude, we can pass the dataframe directly to GeoViews (i.e. no need to use Geopandas)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# First define the format and then define the function\n",
"timeFormat = \"%y%m%d/%H%M\"\n",
"# This function will iterate over each string in a 1-d array\n",
"# and use Pandas' implementation of strptime to convert the string into a datetime object.\n",
"parseTime = lambda x: datetime.strptime(x, timeFormat)\n",
"\n",
"df = pd.read_csv('/spare11/atm533/data/world_metar_latest.csv',parse_dates=['YYMMDD/HHMM'], date_parser=parseTime, sep='\\s+')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this dataset, missing values are set to -9999.0. Let's replace any instance of this value with `np.nan` throughout the DataFrame."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df.replace(-9999.0,np.nan,inplace=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"NaN = np.nan"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's also remove any rows whose latitudes or longitudes are missing."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df = df.query('SLAT.notnull() | SLON.notnull()')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df.TMPC.describe()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create a set of GeoViews `Points` objects. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We pass it three arguments: \n",
"1. The Pandas dataframe\n",
"2. A list containing the lons and lats\n",
"3. A list containing the columns we want to include"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df_points = gv.Points(df, ['SLON','SLAT'],['STN','TMPC','PMSL','DWPC','SPED','P01M'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Visualize the GeoViews Points object"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df_points"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Notice that the x and y-axes correspond to the range of lons and lats in the dataframe. Also notice the Toolbar to the right of the plot. You can mouse over each tool to see its function. By default, it is in Pan mode ... click and drag to move around."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, activate the *Box Zoom* tool (the single magnifying glass). **Click and drag** from **upper-left to lower-right** to create a box. The plot will automatically adjust to the new zoomed-in view."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you have a mouse, you can also try the *Wheel Zoom* tool, which is just below the *Box Zoom* tool."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The *Reset* tool (i.e., the bottom-most icon) resets the plot to the full domain."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Geo-reference the image via a background raster image.\n",
"The Holoviz suite uses the * to add layers to the same plot. Here, we first specify the Open Street Maps tile source. Then, we add our Points dataframe to it. We also specify options that accomplish the following:\n",
"1. Specify the width and height of the plot\n",
"1. Set the size of the points\n",
"1. Color each point by a variable: in this case, 2 m temperature in Celsius\n",
"1. Add a colorbar\n",
"1. Add the `hover` tool to the Toolbar"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"(gv.tile_sources.OSM * df_points).opts(\n",
" opts.Points(frame_width=800, frame_height=600, size=8, color='TMPC',tools=['hover']))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that `hover` tool is active and appears as the tool icon just below the `reset` tool. Mouse over any of the points and you will see a readout of the data from all the columns that we included in the creation of the df_points GeoViews object."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Make a `Labels` plot"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, let's plot a map that instead of the point locations, outputs the values of a particular column from the dataset. To do this, we create a GeoViews `Labels` object. It takes arguments similar to `Points`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df_labels = gv.Labels(df, ['SLON','SLAT'],['TMPC'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot just the labels."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df_labels"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now layer on the background map tile, as we did for `Points`. Specify the color and size of the text labels."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"figure = (gv.tile_sources.CartoLight * df_labels).opts(\n",
" opts.Labels(frame_height=800,frame_width=800,text_color='purple',text_font_size='10pt'))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"figure"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" Note: If you use the panning tool such that you go beyond the bounds of the map, the data does not reappear.\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"
Things to try: \n",
"
- Combine the worldwide METAR and NYSM Dataframes and visualize the combined Dataframe.
\n",
" - Plot a variable other than temperature.