{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\"pandas
\n", "\n", "# Pandas 3: Indexing, Subsetting, Conditionals\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overview\n", "In this notebook, we'll work with Pandas `DataFrame` and `Series` objects to do the following:\n", "1. Specify a particular column to use as the row index name\n", "2. Select various rows, columns, or both\n", "3. Filter a dataset based on specified conditions\n", "4. Calculate and output some summary information, using data from multiple files\n", "#### We'll once again use NYS Mesonet data from 2 September 2021 at 0200 UTC." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisites\n", "\n", "| Concepts | Importance | Notes |\n", "| --- | --- | --- |\n", "| Pandas | Necessary | Intro |\n", "\n", "* **Time to learn**: 30 minutes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Imports" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create and inspect the relevant `DataFrame`s." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "nysm_sites = pd.read_csv('/spare11/atm533/data/nysm_sites.csv')\n", "nysm_data = pd.read_csv('/spare11/atm533/data/nysm_data_2021090202.csv')" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
stationtimetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
0ADDI2021-09-02 02:00:00 UTC13.313.492.50.000.000.002.65.8...NaN0.00.00.019.920.319.90.520.440.44
1ANDE2021-09-02 02:00:00 UTC14.013.7100.00.2810.670.001.92.6...NaN0.00.00.019.119.119.30.250.210.14
2BATA2021-09-02 02:00:00 UTC14.816.377.50.000.000.001.92.3...NaN0.00.00.020.521.521.30.250.210.22
3BEAC2021-09-02 02:00:00 UTC16.015.998.61.9337.530.483.26.5...NaN0.00.00.018.419.319.80.510.360.37
4BELD2021-09-02 02:00:00 UTC14.314.594.90.001.160.002.64.8...NaN0.00.00.019.720.120.20.500.430.41
..................................................................
121WFMB2021-09-02 02:00:00 UTC12.913.675.00.000.000.000.81.5...NaN0.00.00.018.819.919.90.240.180.20
122WGAT2021-09-02 02:00:00 UTC13.813.879.50.000.000.001.44.0...NaN0.00.00.018.820.220.70.160.250.08
123WHIT2021-09-02 02:00:00 UTC15.715.895.90.000.000.001.42.6...NaN0.00.00.019.320.320.10.280.470.46
124WOLC2021-09-02 02:00:00 UTC14.016.684.60.000.000.000.40.9...NaN0.00.00.021.923.524.20.180.030.07
125YORK2021-09-02 02:00:00 UTC12.013.996.00.000.000.000.00.3...NaN0.00.00.020.521.821.90.130.240.24
\n", "

126 rows × 30 columns

\n", "
" ], "text/plain": [ " station time temp_2m [degC] temp_9m [degC] \\\n", "0 ADDI 2021-09-02 02:00:00 UTC 13.3 13.4 \n", "1 ANDE 2021-09-02 02:00:00 UTC 14.0 13.7 \n", "2 BATA 2021-09-02 02:00:00 UTC 14.8 16.3 \n", "3 BEAC 2021-09-02 02:00:00 UTC 16.0 15.9 \n", "4 BELD 2021-09-02 02:00:00 UTC 14.3 14.5 \n", ".. ... ... ... ... \n", "121 WFMB 2021-09-02 02:00:00 UTC 12.9 13.6 \n", "122 WGAT 2021-09-02 02:00:00 UTC 13.8 13.8 \n", "123 WHIT 2021-09-02 02:00:00 UTC 15.7 15.8 \n", "124 WOLC 2021-09-02 02:00:00 UTC 14.0 16.6 \n", "125 YORK 2021-09-02 02:00:00 UTC 12.0 13.9 \n", "\n", " relative_humidity [percent] precip_incremental [mm] precip_local [mm] \\\n", "0 92.5 0.00 0.00 \n", "1 100.0 0.28 10.67 \n", "2 77.5 0.00 0.00 \n", "3 98.6 1.93 37.53 \n", "4 94.9 0.00 1.16 \n", ".. ... ... ... \n", "121 75.0 0.00 0.00 \n", "122 79.5 0.00 0.00 \n", "123 95.9 0.00 0.00 \n", "124 84.6 0.00 0.00 \n", "125 96.0 0.00 0.00 \n", "\n", " precip_max_intensity [mm/min] avg_wind_speed_prop [m/s] \\\n", "0 0.00 2.6 \n", "1 0.00 1.9 \n", "2 0.00 1.9 \n", "3 0.48 3.2 \n", "4 0.00 2.6 \n", ".. ... ... \n", "121 0.00 0.8 \n", "122 0.00 1.4 \n", "123 0.00 1.4 \n", "124 0.00 0.4 \n", "125 0.00 0.0 \n", "\n", " max_wind_speed_prop [m/s] ... snow_depth [cm] frozen_soil_05cm [bit] \\\n", "0 5.8 ... NaN 0.0 \n", "1 2.6 ... NaN 0.0 \n", "2 2.3 ... NaN 0.0 \n", "3 6.5 ... NaN 0.0 \n", "4 4.8 ... NaN 0.0 \n", ".. ... ... ... ... \n", "121 1.5 ... NaN 0.0 \n", "122 4.0 ... NaN 0.0 \n", "123 2.6 ... NaN 0.0 \n", "124 0.9 ... NaN 0.0 \n", "125 0.3 ... NaN 0.0 \n", "\n", " frozen_soil_25cm [bit] frozen_soil_50cm [bit] soil_temp_05cm [degC] \\\n", "0 0.0 0.0 19.9 \n", "1 0.0 0.0 19.1 \n", "2 0.0 0.0 20.5 \n", "3 0.0 0.0 18.4 \n", "4 0.0 0.0 19.7 \n", ".. ... ... ... \n", "121 0.0 0.0 18.8 \n", "122 0.0 0.0 18.8 \n", "123 0.0 0.0 19.3 \n", "124 0.0 0.0 21.9 \n", "125 0.0 0.0 20.5 \n", "\n", " soil_temp_25cm [degC] soil_temp_50cm [degC] \\\n", "0 20.3 19.9 \n", "1 19.1 19.3 \n", "2 21.5 21.3 \n", "3 19.3 19.8 \n", "4 20.1 20.2 \n", ".. ... ... \n", "121 19.9 19.9 \n", "122 20.2 20.7 \n", "123 20.3 20.1 \n", "124 23.5 24.2 \n", "125 21.8 21.9 \n", "\n", " soil_moisture_05cm [m^3/m^3] soil_moisture_25cm [m^3/m^3] \\\n", "0 0.52 0.44 \n", "1 0.25 0.21 \n", "2 0.25 0.21 \n", "3 0.51 0.36 \n", "4 0.50 0.43 \n", ".. ... ... \n", "121 0.24 0.18 \n", "122 0.16 0.25 \n", "123 0.28 0.47 \n", "124 0.18 0.03 \n", "125 0.13 0.24 \n", "\n", " soil_moisture_50cm [m^3/m^3] \n", "0 0.44 \n", "1 0.14 \n", "2 0.22 \n", "3 0.37 \n", "4 0.41 \n", ".. ... \n", "121 0.20 \n", "122 0.08 \n", "123 0.46 \n", "124 0.07 \n", "125 0.24 \n", "\n", "[126 rows x 30 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
stidnumbernamelatlonelevationcountynearest_citystatedistance_from_town [km]direction_from_town [degrees]climate_divisionclimate_division_namewfocommissioneddecommissioned
0ADDI107Addison42.040360-77.237260507.6140SteubenAddisonNY6.9S1Western PlateauBGM2016-08-10 18:15:00 UTCNaN
1ANDE111Andes42.182270-74.801390518.2820DelawareAndesNY1.5WSW2Eastern PlateauBGM2016-08-04 15:55:00 UTCNaN
2BATA24Batavia43.019940-78.135660276.1200GeneseeBataviaNY4.9ENE9Great LakesBUF2016-02-18 18:40:00 UTCNaN
3BEAC76Beacon41.528750-73.94527090.1598DutchessBeaconNY3.3NE5Hudson ValleyALY2016-08-22 16:45:00 UTCNaN
4BELD90Belden42.223220-75.668520470.3700BroomeBeldenNY2.2NNE2Eastern PlateauBGM2015-11-30 20:20:00 UTCNaN
...................................................
121WFMB14Whiteface Mountain Base44.393236-73.858829614.5990EssexWilmingtonNY3.5W3Northern PlateauBTV2016-01-29 20:55:00 UTCNaN
122WGAT123Woodgate43.532408-75.158597442.9660OneidaWoodgateNY1.4NNW3Northern PlateauBGM2016-08-29 18:20:00 UTCNaN
123WHIT10Whitehall43.485073-73.42307136.5638WashingtonWhitehallNY8.0S7Champlain ValleyALY2015-08-26 20:30:00 UTCNaN
124WOLC79Wolcott43.228680-76.842610121.2190WayneWolcottNY2.4WNW9Great LakesBUF2016-03-09 18:10:00 UTCNaN
125YORK99York42.855040-77.847760177.9420LivingstonYorkNY3.6ESE10Central LakesBUF2016-08-09 17:55:00 UTCNaN
\n", "

126 rows × 16 columns

\n", "
" ], "text/plain": [ " stid number name lat lon elevation \\\n", "0 ADDI 107 Addison 42.040360 -77.237260 507.6140 \n", "1 ANDE 111 Andes 42.182270 -74.801390 518.2820 \n", "2 BATA 24 Batavia 43.019940 -78.135660 276.1200 \n", "3 BEAC 76 Beacon 41.528750 -73.945270 90.1598 \n", "4 BELD 90 Belden 42.223220 -75.668520 470.3700 \n", ".. ... ... ... ... ... ... \n", "121 WFMB 14 Whiteface Mountain Base 44.393236 -73.858829 614.5990 \n", "122 WGAT 123 Woodgate 43.532408 -75.158597 442.9660 \n", "123 WHIT 10 Whitehall 43.485073 -73.423071 36.5638 \n", "124 WOLC 79 Wolcott 43.228680 -76.842610 121.2190 \n", "125 YORK 99 York 42.855040 -77.847760 177.9420 \n", "\n", " county nearest_city state distance_from_town [km] \\\n", "0 Steuben Addison NY 6.9 \n", "1 Delaware Andes NY 1.5 \n", "2 Genesee Batavia NY 4.9 \n", "3 Dutchess Beacon NY 3.3 \n", "4 Broome Belden NY 2.2 \n", ".. ... ... ... ... \n", "121 Essex Wilmington NY 3.5 \n", "122 Oneida Woodgate NY 1.4 \n", "123 Washington Whitehall NY 8.0 \n", "124 Wayne Wolcott NY 2.4 \n", "125 Livingston York NY 3.6 \n", "\n", " direction_from_town [degrees] climate_division climate_division_name \\\n", "0 S 1 Western Plateau \n", "1 WSW 2 Eastern Plateau \n", "2 ENE 9 Great Lakes \n", "3 NE 5 Hudson Valley \n", "4 NNE 2 Eastern Plateau \n", ".. ... ... ... \n", "121 W 3 Northern Plateau \n", "122 NNW 3 Northern Plateau \n", "123 S 7 Champlain Valley \n", "124 WNW 9 Great Lakes \n", "125 ESE 10 Central Lakes \n", "\n", " wfo commissioned decommissioned \n", "0 BGM 2016-08-10 18:15:00 UTC NaN \n", "1 BGM 2016-08-04 15:55:00 UTC NaN \n", "2 BUF 2016-02-18 18:40:00 UTC NaN \n", "3 ALY 2016-08-22 16:45:00 UTC NaN \n", "4 BGM 2015-11-30 20:20:00 UTC NaN \n", ".. ... ... ... \n", "121 BTV 2016-01-29 20:55:00 UTC NaN \n", "122 BGM 2016-08-29 18:20:00 UTC NaN \n", "123 ALY 2015-08-26 20:30:00 UTC NaN \n", "124 BUF 2016-03-09 18:10:00 UTC NaN \n", "125 BUF 2016-08-09 17:55:00 UTC NaN \n", "\n", "[126 rows x 16 columns]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_sites" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Select a row index name from a particular column" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For the `nysm_data` DataFrame, let's set the *station* column as the `row index`, in place of the default index. The `inplace` argument makes it so the `DataFrame` object is updated with that change." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "nysm_data.set_index('station', inplace=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Question: How do we discover the possible arguments for the set_index function?
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Answer: We can follow any function (aka \"method\") with a `?` to read documentation on it.
" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\u001b[0;31mSignature:\u001b[0m\n", "\u001b[0mnysm_data\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mset_index\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n", "\u001b[0;34m\u001b[0m \u001b[0mkeys\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", "\u001b[0;34m\u001b[0m \u001b[0mdrop\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", "\u001b[0;34m\u001b[0m \u001b[0mappend\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", "\u001b[0;34m\u001b[0m \u001b[0minplace\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", "\u001b[0;34m\u001b[0m \u001b[0mverify_integrity\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n", "\u001b[0;34m\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mDocstring:\u001b[0m\n", "Set the DataFrame index using existing columns.\n", "\n", "Set the DataFrame index (row labels) using one or more existing\n", "columns or arrays (of the correct length). The index can replace the\n", "existing index or expand on it.\n", "\n", "Parameters\n", "----------\n", "keys : label or array-like or list of labels/arrays\n", " This parameter can be either a single column key, a single array of\n", " the same length as the calling DataFrame, or a list containing an\n", " arbitrary combination of column keys and arrays. Here, \"array\"\n", " encompasses :class:`Series`, :class:`Index`, ``np.ndarray``, and\n", " instances of :class:`~collections.abc.Iterator`.\n", "drop : bool, default True\n", " Delete columns to be used as the new index.\n", "append : bool, default False\n", " Whether to append columns to existing index.\n", "inplace : bool, default False\n", " If True, modifies the DataFrame in place (do not create a new object).\n", "verify_integrity : bool, default False\n", " Check the new index for duplicates. Otherwise defer the check until\n", " necessary. Setting to False will improve the performance of this\n", " method.\n", "\n", "Returns\n", "-------\n", "DataFrame or None\n", " Changed row labels or None if ``inplace=True``.\n", "\n", "See Also\n", "--------\n", "DataFrame.reset_index : Opposite of set_index.\n", "DataFrame.reindex : Change to new indices or expand indices.\n", "DataFrame.reindex_like : Change to same indices as other DataFrame.\n", "\n", "Examples\n", "--------\n", ">>> df = pd.DataFrame({'month': [1, 4, 7, 10],\n", "... 'year': [2012, 2014, 2013, 2014],\n", "... 'sale': [55, 40, 84, 31]})\n", ">>> df\n", " month year sale\n", "0 1 2012 55\n", "1 4 2014 40\n", "2 7 2013 84\n", "3 10 2014 31\n", "\n", "Set the index to become the 'month' column:\n", "\n", ">>> df.set_index('month')\n", " year sale\n", "month\n", "1 2012 55\n", "4 2014 40\n", "7 2013 84\n", "10 2014 31\n", "\n", "Create a MultiIndex using columns 'year' and 'month':\n", "\n", ">>> df.set_index(['year', 'month'])\n", " sale\n", "year month\n", "2012 1 55\n", "2014 4 40\n", "2013 7 84\n", "2014 10 31\n", "\n", "Create a MultiIndex using an Index and a column:\n", "\n", ">>> df.set_index([pd.Index([1, 2, 3, 4]), 'year'])\n", " month sale\n", " year\n", "1 2012 1 55\n", "2 2014 4 40\n", "3 2013 7 84\n", "4 2014 10 31\n", "\n", "Create a MultiIndex using two Series:\n", "\n", ">>> s = pd.Series([1, 2, 3, 4])\n", ">>> df.set_index([s, s**2])\n", " month year sale\n", "1 1 1 2012 55\n", "2 4 4 2014 40\n", "3 9 7 2013 84\n", "4 16 10 2014 31\n", "\u001b[0;31mFile:\u001b[0m /knight/anaconda_jan21/envs/aug21/lib/python3.8/site-packages/pandas/core/frame.py\n", "\u001b[0;31mType:\u001b[0m method\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "nysm_data.set_index?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "___\n", "One of the arguments to `set_index` is `drop`. If true, it will drop (i.e., delete) the corresponding column from the `DataFrame`. Its default is **True**, so we don't need to include it if that's what we want." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's look at the first couple rows in the `DataFrame` to verify that the index got changed." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
ADDI2021-09-02 02:00:00 UTC13.313.492.50.000.000.02.65.80.9...NaN0.00.00.019.920.319.90.520.440.44
ANDE2021-09-02 02:00:00 UTC14.013.7100.00.2810.670.01.92.60.4...NaN0.00.00.019.119.119.30.250.210.14
\n", "

2 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "ADDI 2021-09-02 02:00:00 UTC 13.3 13.4 \n", "ANDE 2021-09-02 02:00:00 UTC 14.0 13.7 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "ADDI 92.5 0.00 \n", "ANDE 100.0 0.28 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "ADDI 0.00 0.0 \n", "ANDE 10.67 0.0 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "ADDI 2.6 5.8 \n", "ANDE 1.9 2.6 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "ADDI 0.9 ... NaN \n", "ANDE 0.4 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "ADDI 0.0 0.0 \n", "ANDE 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "ADDI 0.0 19.9 20.3 \n", "ANDE 0.0 19.1 19.1 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "ADDI 19.9 0.52 \n", "ANDE 19.3 0.25 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "ADDI 0.44 0.44 \n", "ANDE 0.21 0.14 \n", "\n", "[2 rows x 29 columns]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data.head(2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Success! Now the station name is the row index." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Selecting rows and/or columns from a `DataFrame` using the `loc` and `iloc` operators (full documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Let's say we just want to select a few of the NYS Mesonet sites for analysis. We can select rows / columns either by name or index number. First, let's select by name.\n", "The `loc` operator can accept a single row/column name, or a list of names. Try a single site first. As `loc` is a type of Python list or array index operator, we place the arguments inside brackets `[]`. " ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "time 2021-09-02 02:00:00 UTC\n", "temp_2m [degC] 18.7\n", "temp_9m [degC] NaN\n", "relative_humidity [percent] 100.0\n", "precip_incremental [mm] 4.64\n", "precip_local [mm] 111.03\n", "precip_max_intensity [mm/min] 1.22\n", "avg_wind_speed_prop [m/s] 5.4\n", "max_wind_speed_prop [m/s] 9.5\n", "wind_speed_stddev_prop [m/s] 2.0\n", "wind_direction_prop [degrees] 352.0\n", "wind_direction_stddev_prop [degrees] 24.0\n", "avg_wind_speed_sonic [m/s] 6.6\n", "max_wind_speed_sonic [m/s] 12.5\n", "wind_speed_stddev_sonic [m/s] 2.2\n", "wind_direction_sonic [degrees] 352.0\n", "wind_direction_stddev_sonic [degrees] 18.0\n", "solar_insolation [W/m^2] 0\n", "station_pressure [mbar] 989.3\n", "snow_depth [cm] NaN\n", "frozen_soil_05cm [bit] NaN\n", "frozen_soil_25cm [bit] NaN\n", "frozen_soil_50cm [bit] NaN\n", "soil_temp_05cm [degC] NaN\n", "soil_temp_25cm [degC] NaN\n", "soil_temp_50cm [degC] NaN\n", "soil_moisture_05cm [m^3/m^3] NaN\n", "soil_moisture_25cm [m^3/m^3] NaN\n", "soil_moisture_50cm [m^3/m^3] NaN\n", "Name: MANH, dtype: object" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Display data for the Manhattan (MANH) site.\n", "nysm_data.loc['MANH']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Multiple sites can be passed as a Python `list` ... with its own set of brackets `[]`! This list can be in any order. Once we select more than one row, the output is now a `DataFrame` ... and will be nicely formatted in the notebook." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
RUSH2021-09-02 02:00:00 UTC11.312.998.30.000.000.000.00.00.0...NaN0.00.00.020.521.321.30.260.250.25
BRON2021-09-02 02:00:00 UTC18.017.899.06.86110.351.846.811.52.0...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
VOOR2021-09-02 02:00:00 UTC14.914.998.90.2611.670.002.23.60.6...NaN0.00.00.020.221.121.10.250.330.19
\n", "

3 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "RUSH 2021-09-02 02:00:00 UTC 11.3 12.9 \n", "BRON 2021-09-02 02:00:00 UTC 18.0 17.8 \n", "VOOR 2021-09-02 02:00:00 UTC 14.9 14.9 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "RUSH 98.3 0.00 \n", "BRON 99.0 6.86 \n", "VOOR 98.9 0.26 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "RUSH 0.00 0.00 \n", "BRON 110.35 1.84 \n", "VOOR 11.67 0.00 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "RUSH 0.0 0.0 \n", "BRON 6.8 11.5 \n", "VOOR 2.2 3.6 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "RUSH 0.0 ... NaN \n", "BRON 2.0 ... NaN \n", "VOOR 0.6 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "RUSH 0.0 0.0 \n", "BRON NaN NaN \n", "VOOR 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "RUSH 0.0 20.5 21.3 \n", "BRON NaN NaN NaN \n", "VOOR 0.0 20.2 21.1 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "RUSH 21.3 0.26 \n", "BRON NaN NaN \n", "VOOR 21.1 0.25 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "RUSH 0.25 0.25 \n", "BRON NaN NaN \n", "VOOR 0.33 0.19 \n", "\n", "[3 rows x 29 columns]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Display data for the Rush (RUSH), Bronx (BRON) and Voorheeseville (VOOR) sites.\n", "nysm_data.loc[['RUSH','BRON','VOOR']]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can also select by row number () using the `iloc` operator. Manhattan is in row 66. Recall that Python uses zero-based indexing, so the first row is element 0." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "time 2021-09-02 02:00:00 UTC\n", "temp_2m [degC] 18.7\n", "temp_9m [degC] NaN\n", "relative_humidity [percent] 100.0\n", "precip_incremental [mm] 4.64\n", "precip_local [mm] 111.03\n", "precip_max_intensity [mm/min] 1.22\n", "avg_wind_speed_prop [m/s] 5.4\n", "max_wind_speed_prop [m/s] 9.5\n", "wind_speed_stddev_prop [m/s] 2.0\n", "wind_direction_prop [degrees] 352.0\n", "wind_direction_stddev_prop [degrees] 24.0\n", "avg_wind_speed_sonic [m/s] 6.6\n", "max_wind_speed_sonic [m/s] 12.5\n", "wind_speed_stddev_sonic [m/s] 2.2\n", "wind_direction_sonic [degrees] 352.0\n", "wind_direction_stddev_sonic [degrees] 18.0\n", "solar_insolation [W/m^2] 0\n", "station_pressure [mbar] 989.3\n", "snow_depth [cm] NaN\n", "frozen_soil_05cm [bit] NaN\n", "frozen_soil_25cm [bit] NaN\n", "frozen_soil_50cm [bit] NaN\n", "soil_temp_05cm [degC] NaN\n", "soil_temp_25cm [degC] NaN\n", "soil_temp_50cm [degC] NaN\n", "soil_moisture_05cm [m^3/m^3] NaN\n", "soil_moisture_25cm [m^3/m^3] NaN\n", "soil_moisture_50cm [m^3/m^3] NaN\n", "Name: MANH, dtype: object" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data.iloc[65]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### We can pass in a list to `iloc` too:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
ADDI2021-09-02 02:00:00 UTC13.313.492.50.00.00.02.65.80.9...NaN0.00.00.019.920.319.90.520.440.44
COPE2021-09-02 02:00:00 UTC12.212.394.40.00.00.00.81.70.4...NaN0.00.0NaN19.219.5NaN0.450.26NaN
REDF2021-09-02 02:00:00 UTC12.412.582.10.00.00.01.83.20.6...NaN0.00.00.018.919.218.90.410.410.29
\n", "

3 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "ADDI 2021-09-02 02:00:00 UTC 13.3 13.4 \n", "COPE 2021-09-02 02:00:00 UTC 12.2 12.3 \n", "REDF 2021-09-02 02:00:00 UTC 12.4 12.5 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "ADDI 92.5 0.0 \n", "COPE 94.4 0.0 \n", "REDF 82.1 0.0 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "ADDI 0.0 0.0 \n", "COPE 0.0 0.0 \n", "REDF 0.0 0.0 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "ADDI 2.6 5.8 \n", "COPE 0.8 1.7 \n", "REDF 1.8 3.2 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "ADDI 0.9 ... NaN \n", "COPE 0.4 ... NaN \n", "REDF 0.6 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "ADDI 0.0 0.0 \n", "COPE 0.0 0.0 \n", "REDF 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "ADDI 0.0 19.9 20.3 \n", "COPE NaN 19.2 19.5 \n", "REDF 0.0 18.9 19.2 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "ADDI 19.9 0.52 \n", "COPE NaN 0.45 \n", "REDF 18.9 0.41 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "ADDI 0.44 0.44 \n", "COPE 0.26 NaN \n", "REDF 0.41 0.29 \n", "\n", "[3 rows x 29 columns]" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Display data for the 1st, 32nd, and 88th row in the DataFrame.\n", "nysm_data.iloc[[0,31,87]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Now let's select columns. To do so, we first select rows (in this case, we'll select all rows, as indicated by `:`), and then continue with a list of columns." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]precip_incremental [mm]
station
ADDI2021-09-02 02:00:00 UTC13.30.00
ANDE2021-09-02 02:00:00 UTC14.00.28
BATA2021-09-02 02:00:00 UTC14.80.00
BEAC2021-09-02 02:00:00 UTC16.01.93
BELD2021-09-02 02:00:00 UTC14.30.00
............
WFMB2021-09-02 02:00:00 UTC12.90.00
WGAT2021-09-02 02:00:00 UTC13.80.00
WHIT2021-09-02 02:00:00 UTC15.70.00
WOLC2021-09-02 02:00:00 UTC14.00.00
YORK2021-09-02 02:00:00 UTC12.00.00
\n", "

126 rows × 3 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] precip_incremental [mm]\n", "station \n", "ADDI 2021-09-02 02:00:00 UTC 13.3 0.00\n", "ANDE 2021-09-02 02:00:00 UTC 14.0 0.28\n", "BATA 2021-09-02 02:00:00 UTC 14.8 0.00\n", "BEAC 2021-09-02 02:00:00 UTC 16.0 1.93\n", "BELD 2021-09-02 02:00:00 UTC 14.3 0.00\n", "... ... ... ...\n", "WFMB 2021-09-02 02:00:00 UTC 12.9 0.00\n", "WGAT 2021-09-02 02:00:00 UTC 13.8 0.00\n", "WHIT 2021-09-02 02:00:00 UTC 15.7 0.00\n", "WOLC 2021-09-02 02:00:00 UTC 14.0 0.00\n", "YORK 2021-09-02 02:00:00 UTC 12.0 0.00\n", "\n", "[126 rows x 3 columns]" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data.loc[:,['time','temp_2m [degC]','precip_incremental [mm]']]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### We can specify specific rows and columns" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]precip_incremental [mm]
station
BATA2021-09-02 02:00:00 UTC14.80.00
COLD2021-09-02 02:00:00 UTC14.00.00
QUEE2021-09-02 02:00:00 UTC19.011.95
\n", "
" ], "text/plain": [ " time temp_2m [degC] precip_incremental [mm]\n", "station \n", "BATA 2021-09-02 02:00:00 UTC 14.8 0.00\n", "COLD 2021-09-02 02:00:00 UTC 14.0 0.00\n", "QUEE 2021-09-02 02:00:00 UTC 19.0 11.95" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data.loc[['BATA','COLD','QUEE'],['time','temp_2m [degC]','precip_incremental [mm]']]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### We can use index slicing with `iloc`; note that the last element is *excluded* ... like typical Python slicing behavior." ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
ADDI2021-09-02 02:00:00 UTC13.313.492.50.000.000.002.65.80.9...NaN0.00.00.019.920.319.90.520.440.44
ANDE2021-09-02 02:00:00 UTC14.013.7100.00.2810.670.001.92.60.4...NaN0.00.00.019.119.119.30.250.210.14
BATA2021-09-02 02:00:00 UTC14.816.377.50.000.000.001.92.30.2...NaN0.00.00.020.521.521.30.250.210.22
BEAC2021-09-02 02:00:00 UTC16.015.998.61.9337.530.483.26.51.4...NaN0.00.00.018.419.319.80.510.360.37
\n", "

4 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "ADDI 2021-09-02 02:00:00 UTC 13.3 13.4 \n", "ANDE 2021-09-02 02:00:00 UTC 14.0 13.7 \n", "BATA 2021-09-02 02:00:00 UTC 14.8 16.3 \n", "BEAC 2021-09-02 02:00:00 UTC 16.0 15.9 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "ADDI 92.5 0.00 \n", "ANDE 100.0 0.28 \n", "BATA 77.5 0.00 \n", "BEAC 98.6 1.93 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "ADDI 0.00 0.00 \n", "ANDE 10.67 0.00 \n", "BATA 0.00 0.00 \n", "BEAC 37.53 0.48 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "ADDI 2.6 5.8 \n", "ANDE 1.9 2.6 \n", "BATA 1.9 2.3 \n", "BEAC 3.2 6.5 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "ADDI 0.9 ... NaN \n", "ANDE 0.4 ... NaN \n", "BATA 0.2 ... NaN \n", "BEAC 1.4 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "ADDI 0.0 0.0 \n", "ANDE 0.0 0.0 \n", "BATA 0.0 0.0 \n", "BEAC 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "ADDI 0.0 19.9 20.3 \n", "ANDE 0.0 19.1 19.1 \n", "BATA 0.0 20.5 21.5 \n", "BEAC 0.0 18.4 19.3 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "ADDI 19.9 0.52 \n", "ANDE 19.3 0.25 \n", "BATA 21.3 0.25 \n", "BEAC 19.8 0.51 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "ADDI 0.44 0.44 \n", "ANDE 0.21 0.14 \n", "BATA 0.21 0.22 \n", "BEAC 0.36 0.37 \n", "\n", "[4 rows x 29 columns]" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data.iloc[0:4] # Will display first four rows of DataFrame" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### As a convenience, one can also display a specified number of rows without using the `loc` or `iloc` operators." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
ADDI2021-09-02 02:00:00 UTC13.313.492.50.000.000.002.65.80.9...NaN0.00.00.019.920.319.90.520.440.44
ANDE2021-09-02 02:00:00 UTC14.013.7100.00.2810.670.001.92.60.4...NaN0.00.00.019.119.119.30.250.210.14
BATA2021-09-02 02:00:00 UTC14.816.377.50.000.000.001.92.30.2...NaN0.00.00.020.521.521.30.250.210.22
BEAC2021-09-02 02:00:00 UTC16.015.998.61.9337.530.483.26.51.4...NaN0.00.00.018.419.319.80.510.360.37
\n", "

4 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "ADDI 2021-09-02 02:00:00 UTC 13.3 13.4 \n", "ANDE 2021-09-02 02:00:00 UTC 14.0 13.7 \n", "BATA 2021-09-02 02:00:00 UTC 14.8 16.3 \n", "BEAC 2021-09-02 02:00:00 UTC 16.0 15.9 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "ADDI 92.5 0.00 \n", "ANDE 100.0 0.28 \n", "BATA 77.5 0.00 \n", "BEAC 98.6 1.93 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "ADDI 0.00 0.00 \n", "ANDE 10.67 0.00 \n", "BATA 0.00 0.00 \n", "BEAC 37.53 0.48 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "ADDI 2.6 5.8 \n", "ANDE 1.9 2.6 \n", "BATA 1.9 2.3 \n", "BEAC 3.2 6.5 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "ADDI 0.9 ... NaN \n", "ANDE 0.4 ... NaN \n", "BATA 0.2 ... NaN \n", "BEAC 1.4 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "ADDI 0.0 0.0 \n", "ANDE 0.0 0.0 \n", "BATA 0.0 0.0 \n", "BEAC 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "ADDI 0.0 19.9 20.3 \n", "ANDE 0.0 19.1 19.1 \n", "BATA 0.0 20.5 21.5 \n", "BEAC 0.0 18.4 19.3 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "ADDI 19.9 0.52 \n", "ANDE 19.3 0.25 \n", "BATA 21.3 0.25 \n", "BEAC 19.8 0.51 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "ADDI 0.44 0.44 \n", "ANDE 0.21 0.14 \n", "BATA 0.21 0.22 \n", "BEAC 0.36 0.37 \n", "\n", "[4 rows x 29 columns]" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Display the first four rows (indexes 0, 1, 2, 3)\n", "nysm_data[:4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Question:This next cell will fail; why?
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# This will fail; uncomment to see the error message\n", "#nysm_data[2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Answer:Because the *row index names* are now the four-character site ids!
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### We can, however, pass in a *slice*: in this case, rows 2 and 3 (remember, slices in Python *exclude* the *stop* value, by default)." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
BATA2021-09-02 02:00:00 UTC14.816.377.50.000.000.001.92.30.2...NaN0.00.00.020.521.521.30.250.210.22
BEAC2021-09-02 02:00:00 UTC16.015.998.61.9337.530.483.26.51.4...NaN0.00.00.018.419.319.80.510.360.37
\n", "

2 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "BATA 2021-09-02 02:00:00 UTC 14.8 16.3 \n", "BEAC 2021-09-02 02:00:00 UTC 16.0 15.9 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "BATA 77.5 0.00 \n", "BEAC 98.6 1.93 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "BATA 0.00 0.00 \n", "BEAC 37.53 0.48 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "BATA 1.9 2.3 \n", "BEAC 3.2 6.5 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "BATA 0.2 ... NaN \n", "BEAC 1.4 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "BATA 0.0 0.0 \n", "BEAC 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "BATA 0.0 20.5 21.5 \n", "BEAC 0.0 18.4 19.3 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "BATA 21.3 0.25 \n", "BEAC 19.8 0.51 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "BATA 0.21 0.22 \n", "BEAC 0.36 0.37 \n", "\n", "[2 rows x 29 columns]" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data[2:4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Tip: In general, rely on loc and iloc when you need to perform selections.
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### When you select a single column from a `DataFrame`, a `Series` is returned. \n", "Select the second column from the `DataFrame` (the first column would be referenced as 0 in `iloc`) and assign the returned `Series` to its own object name. Inspect this `Series` object." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI 13.3\n", "ANDE 14.0\n", "BATA 14.8\n", "BEAC 16.0\n", "BELD 14.3\n", " ... \n", "WFMB 12.9\n", "WGAT 13.8\n", "WHIT 15.7\n", "WOLC 14.0\n", "YORK 12.0\n", "Name: temp_2m [degC], Length: 126, dtype: float64" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpc = nysm_data.iloc[:,1]\n", "tmpc" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### We can employ selection operations, such as `loc` and `iloc` to `Series` as well as `DataFrames`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use `loc` to select a particular row index:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "tmpc.loc['BATA']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This next line is equivalent:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "tmpc['BATA']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use `iloc`:" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "14.8" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpc.iloc[2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Referencing via the array index is equivalent:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "14.8" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpc[2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Slice a `Series` with `iloc`:" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI 13.3\n", "ANDE 14.0\n", "BATA 14.8\n", "BEAC 16.0\n", "Name: temp_2m [degC], dtype: float64" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpc.iloc[0:4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Equivalently, slice on the array indices:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI 13.3\n", "ANDE 14.0\n", "BATA 14.8\n", "BEAC 16.0\n", "Name: temp_2m [degC], dtype: float64" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpc[0:4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Note: In a departure from typical Python behavior regarding slices, slicing index names on a Series does not exclude the stop value:
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here, **'BEAC'** is included in the returned `Series`:" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ANDE 14.0\n", "BATA 14.8\n", "BEAC 16.0\n", "Name: temp_2m [degC], dtype: float64" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpc.loc['ANDE':'BEAC']" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ANDE 14.0\n", "BATA 14.8\n", "BEAC 16.0\n", "Name: temp_2m [degC], dtype: float64" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Equivalent to previous cell\n", "tmpc['ANDE':'BEAC']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "### Select particular rows by expressing **conditions**:\n", "#### Applying a *conditional* to a `Series` produces a series of booleans (i.e., `True`/`False` values ... aka a *truth table*)" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI False\n", "ANDE False\n", "BATA False\n", "BEAC True\n", "BELD False\n", " ... \n", "WFMB False\n", "WGAT False\n", "WHIT True\n", "WOLC False\n", "YORK False\n", "Name: temp_2m [degC], Length: 126, dtype: bool" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# What stations had a temperature of at least a certain threshold value?\n", "thresh = 15\n", "tmpc >= thresh" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We pass in this Boolean series to the series it was derived from, in order to produce another series containing the actual values that meet the specified condition." ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "BEAC 16.0\n", "BERK 15.4\n", "BKLN 19.9\n", "BREW 16.2\n", "BRON 18.0\n", "COPA 15.6\n", "DEPO 16.0\n", "DOVE 16.0\n", "ELMI 15.2\n", "ESSX 15.1\n", "FAYE 15.7\n", "GFAL 16.0\n", "GROT 15.1\n", "HFAL 15.9\n", "JORD 15.7\n", "KIND 16.7\n", "MANH 18.7\n", "ONTA 18.2\n", "OTIS 15.7\n", "PENN 15.2\n", "QUEE 19.0\n", "REDH 16.6\n", "SCHO 16.4\n", "SOME 16.5\n", "SOUT 19.3\n", "SPRA 15.1\n", "STAT 18.5\n", "STEP 15.7\n", "STON 20.3\n", "SUFF 16.5\n", "TICO 15.5\n", "WALL 16.2\n", "WANT 24.1\n", "WARW 16.2\n", "WHIT 15.7\n", "Name: temp_2m [degC], dtype: float64" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpc[tmpc >= thresh]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Tip: Until you get used to it, the way the above two cells are coded is a bit non-intuitive. It may make more sense if we instead set the Boolean series to its own object name:
" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "BEAC 16.0\n", "BERK 15.4\n", "BKLN 19.9\n", "BREW 16.2\n", "BRON 18.0\n", "COPA 15.6\n", "DEPO 16.0\n", "DOVE 16.0\n", "ELMI 15.2\n", "ESSX 15.1\n", "FAYE 15.7\n", "GFAL 16.0\n", "GROT 15.1\n", "HFAL 15.9\n", "JORD 15.7\n", "KIND 16.7\n", "MANH 18.7\n", "ONTA 18.2\n", "OTIS 15.7\n", "PENN 15.2\n", "QUEE 19.0\n", "REDH 16.6\n", "SCHO 16.4\n", "SOME 16.5\n", "SOUT 19.3\n", "SPRA 15.1\n", "STAT 18.5\n", "STEP 15.7\n", "STON 20.3\n", "SUFF 16.5\n", "TICO 15.5\n", "WALL 16.2\n", "WANT 24.1\n", "WARW 16.2\n", "WHIT 15.7\n", "Name: temp_2m [degC], dtype: float64" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "condition = tmpc >= thresh\n", "tmpc[condition]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### We can also apply conditions to the entire `DataFrame`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Show a subset of the original `DataFrame` containing only those sites that recorded precip during the five-minute period between NYSM observations:" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "prcpMeasurable = nysm_data['precip_incremental [mm]' ] > 0.0 " ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI False\n", "ANDE True\n", "BATA False\n", "BEAC True\n", "BELD False\n", " ... \n", "WFMB False\n", "WGAT False\n", "WHIT False\n", "WOLC False\n", "YORK False\n", "Name: precip_incremental [mm], Length: 126, dtype: bool" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "prcpMeasurable" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
ANDE2021-09-02 02:00:00 UTC14.013.7100.00.2810.670.001.92.60.4...NaN0.00.00.019.119.119.30.250.210.14
BEAC2021-09-02 02:00:00 UTC16.015.998.61.9337.530.483.26.51.4...NaN0.00.00.018.419.319.80.510.360.37
BKLN2021-09-02 02:00:00 UTC19.920.097.93.2175.190.933.49.52.3...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
BREW2021-09-02 02:00:00 UTC16.216.599.71.0126.890.274.99.81.7...NaN0.00.00.020.320.320.30.410.330.35
BRON2021-09-02 02:00:00 UTC18.017.899.06.86110.351.846.811.52.0...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
CLAR2021-09-02 02:00:00 UTC13.914.097.90.5213.890.110.82.00.5...NaN0.00.00.017.919.119.30.220.220.15
DEPO2021-09-02 02:00:00 UTC16.016.190.10.053.170.002.55.81.5...NaN0.00.0NaN19.920.7NaN0.170.27NaN
DOVE2021-09-02 02:00:00 UTC16.0NaN95.21.3524.690.362.14.10.8...NaN0.00.00.019.621.521.50.390.220.15
DUAN2021-09-02 02:00:00 UTC13.213.1100.00.366.110.000.92.50.5...NaN0.00.00.019.520.220.10.550.380.42
ELDR2021-09-02 02:00:00 UTC14.9NaN98.60.6416.780.160.92.30.4...NaN0.00.00.019.120.120.70.380.520.51
HARP2021-09-02 02:00:00 UTC13.413.696.60.1810.540.003.15.71.2...NaN0.00.00.018.619.319.00.310.280.52
HFAL2021-09-02 02:00:00 UTC15.915.999.50.8621.890.204.37.51.3...NaN0.00.00.019.721.122.10.470.350.35
KIND2021-09-02 02:00:00 UTC16.716.798.60.739.750.192.35.11.0...NaN0.00.00.021.321.921.70.460.410.44
MANH2021-09-02 02:00:00 UTC18.7NaN100.04.64111.031.225.49.52.0...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
MEDU2021-09-02 02:00:00 UTC14.214.294.40.5610.390.153.15.41.1...NaN0.00.00.017.719.118.80.390.370.38
NBRA2021-09-02 02:00:00 UTC14.814.994.00.278.950.001.43.20.8...NaN0.00.00.018.920.720.50.440.430.50
OTIS2021-09-02 02:00:00 UTC15.715.698.80.8828.490.191.12.60.6...NaN0.00.0NaN18.420.4NaN0.340.28NaN
QUEE2021-09-02 02:00:00 UTC19.0NaN98.011.9563.432.6810.016.52.5...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
REDH2021-09-02 02:00:00 UTC16.616.697.80.8219.730.243.76.61.2...NaN0.00.00.021.923.924.30.280.250.30
ROXB2021-09-02 02:00:00 UTC14.014.098.20.6011.850.143.05.30.9...NaN0.00.0NaN17.217.3NaN0.550.52NaN
SCHA2021-09-02 02:00:00 UTC14.915.198.10.5310.010.120.51.40.4...NaN0.00.00.021.522.122.40.220.140.05
SCHO2021-09-02 02:00:00 UTC16.416.299.90.559.070.181.64.51.0...NaN0.00.00.021.121.921.90.330.190.14
SCHU2021-09-02 02:00:00 UTC14.814.796.30.072.750.001.32.40.4...NaN0.00.00.021.222.422.10.070.120.08
SOME2021-09-02 02:00:00 UTC16.516.498.31.7546.430.442.66.01.4...NaN0.00.00.020.321.721.30.690.420.62
STAT2021-09-02 02:00:00 UTC18.518.597.60.3993.400.006.19.41.6...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
STEP2021-09-02 02:00:00 UTC15.715.693.00.738.540.213.15.91.1...NaN0.00.00.019.419.919.70.320.330.31
SUFF2021-09-02 02:00:00 UTC16.516.298.60.6566.080.164.212.12.0...NaN0.00.00.020.320.121.70.470.340.28
TANN2021-09-02 02:00:00 UTC13.113.498.90.8017.900.170.83.50.7...NaN0.00.00.018.018.418.40.590.490.42
VOOR2021-09-02 02:00:00 UTC14.914.998.90.2611.670.002.23.60.6...NaN0.00.00.020.221.121.10.250.330.19
WALL2021-09-02 02:00:00 UTC16.216.298.41.7724.480.453.35.91.3...NaN0.00.00.019.921.921.50.570.420.44
WANT2021-09-02 02:00:00 UTC24.124.098.70.063.470.008.013.31.9...NaN0.00.00.023.724.424.30.220.050.05
WARW2021-09-02 02:00:00 UTC16.216.2100.01.3546.920.292.74.40.6...NaN0.00.00.018.221.121.80.430.320.31
WBOU2021-09-02 02:00:00 UTC14.414.299.10.5219.270.113.65.80.9...NaN0.00.00.017.520.521.30.510.360.34
\n", "

33 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "ANDE 2021-09-02 02:00:00 UTC 14.0 13.7 \n", "BEAC 2021-09-02 02:00:00 UTC 16.0 15.9 \n", "BKLN 2021-09-02 02:00:00 UTC 19.9 20.0 \n", "BREW 2021-09-02 02:00:00 UTC 16.2 16.5 \n", "BRON 2021-09-02 02:00:00 UTC 18.0 17.8 \n", "CLAR 2021-09-02 02:00:00 UTC 13.9 14.0 \n", "DEPO 2021-09-02 02:00:00 UTC 16.0 16.1 \n", "DOVE 2021-09-02 02:00:00 UTC 16.0 NaN \n", "DUAN 2021-09-02 02:00:00 UTC 13.2 13.1 \n", "ELDR 2021-09-02 02:00:00 UTC 14.9 NaN \n", "HARP 2021-09-02 02:00:00 UTC 13.4 13.6 \n", "HFAL 2021-09-02 02:00:00 UTC 15.9 15.9 \n", "KIND 2021-09-02 02:00:00 UTC 16.7 16.7 \n", "MANH 2021-09-02 02:00:00 UTC 18.7 NaN \n", "MEDU 2021-09-02 02:00:00 UTC 14.2 14.2 \n", "NBRA 2021-09-02 02:00:00 UTC 14.8 14.9 \n", "OTIS 2021-09-02 02:00:00 UTC 15.7 15.6 \n", "QUEE 2021-09-02 02:00:00 UTC 19.0 NaN \n", "REDH 2021-09-02 02:00:00 UTC 16.6 16.6 \n", "ROXB 2021-09-02 02:00:00 UTC 14.0 14.0 \n", "SCHA 2021-09-02 02:00:00 UTC 14.9 15.1 \n", "SCHO 2021-09-02 02:00:00 UTC 16.4 16.2 \n", "SCHU 2021-09-02 02:00:00 UTC 14.8 14.7 \n", "SOME 2021-09-02 02:00:00 UTC 16.5 16.4 \n", "STAT 2021-09-02 02:00:00 UTC 18.5 18.5 \n", "STEP 2021-09-02 02:00:00 UTC 15.7 15.6 \n", "SUFF 2021-09-02 02:00:00 UTC 16.5 16.2 \n", "TANN 2021-09-02 02:00:00 UTC 13.1 13.4 \n", "VOOR 2021-09-02 02:00:00 UTC 14.9 14.9 \n", "WALL 2021-09-02 02:00:00 UTC 16.2 16.2 \n", "WANT 2021-09-02 02:00:00 UTC 24.1 24.0 \n", "WARW 2021-09-02 02:00:00 UTC 16.2 16.2 \n", "WBOU 2021-09-02 02:00:00 UTC 14.4 14.2 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "ANDE 100.0 0.28 \n", "BEAC 98.6 1.93 \n", "BKLN 97.9 3.21 \n", "BREW 99.7 1.01 \n", "BRON 99.0 6.86 \n", "CLAR 97.9 0.52 \n", "DEPO 90.1 0.05 \n", "DOVE 95.2 1.35 \n", "DUAN 100.0 0.36 \n", "ELDR 98.6 0.64 \n", "HARP 96.6 0.18 \n", "HFAL 99.5 0.86 \n", "KIND 98.6 0.73 \n", "MANH 100.0 4.64 \n", "MEDU 94.4 0.56 \n", "NBRA 94.0 0.27 \n", "OTIS 98.8 0.88 \n", "QUEE 98.0 11.95 \n", "REDH 97.8 0.82 \n", "ROXB 98.2 0.60 \n", "SCHA 98.1 0.53 \n", "SCHO 99.9 0.55 \n", "SCHU 96.3 0.07 \n", "SOME 98.3 1.75 \n", "STAT 97.6 0.39 \n", "STEP 93.0 0.73 \n", "SUFF 98.6 0.65 \n", "TANN 98.9 0.80 \n", "VOOR 98.9 0.26 \n", "WALL 98.4 1.77 \n", "WANT 98.7 0.06 \n", "WARW 100.0 1.35 \n", "WBOU 99.1 0.52 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "ANDE 10.67 0.00 \n", "BEAC 37.53 0.48 \n", "BKLN 75.19 0.93 \n", "BREW 26.89 0.27 \n", "BRON 110.35 1.84 \n", "CLAR 13.89 0.11 \n", "DEPO 3.17 0.00 \n", "DOVE 24.69 0.36 \n", "DUAN 6.11 0.00 \n", "ELDR 16.78 0.16 \n", "HARP 10.54 0.00 \n", "HFAL 21.89 0.20 \n", "KIND 9.75 0.19 \n", "MANH 111.03 1.22 \n", "MEDU 10.39 0.15 \n", "NBRA 8.95 0.00 \n", "OTIS 28.49 0.19 \n", "QUEE 63.43 2.68 \n", "REDH 19.73 0.24 \n", "ROXB 11.85 0.14 \n", "SCHA 10.01 0.12 \n", "SCHO 9.07 0.18 \n", "SCHU 2.75 0.00 \n", "SOME 46.43 0.44 \n", "STAT 93.40 0.00 \n", "STEP 8.54 0.21 \n", "SUFF 66.08 0.16 \n", "TANN 17.90 0.17 \n", "VOOR 11.67 0.00 \n", "WALL 24.48 0.45 \n", "WANT 3.47 0.00 \n", "WARW 46.92 0.29 \n", "WBOU 19.27 0.11 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "ANDE 1.9 2.6 \n", "BEAC 3.2 6.5 \n", "BKLN 3.4 9.5 \n", "BREW 4.9 9.8 \n", "BRON 6.8 11.5 \n", "CLAR 0.8 2.0 \n", "DEPO 2.5 5.8 \n", "DOVE 2.1 4.1 \n", "DUAN 0.9 2.5 \n", "ELDR 0.9 2.3 \n", "HARP 3.1 5.7 \n", "HFAL 4.3 7.5 \n", "KIND 2.3 5.1 \n", "MANH 5.4 9.5 \n", "MEDU 3.1 5.4 \n", "NBRA 1.4 3.2 \n", "OTIS 1.1 2.6 \n", "QUEE 10.0 16.5 \n", "REDH 3.7 6.6 \n", "ROXB 3.0 5.3 \n", "SCHA 0.5 1.4 \n", "SCHO 1.6 4.5 \n", "SCHU 1.3 2.4 \n", "SOME 2.6 6.0 \n", "STAT 6.1 9.4 \n", "STEP 3.1 5.9 \n", "SUFF 4.2 12.1 \n", "TANN 0.8 3.5 \n", "VOOR 2.2 3.6 \n", "WALL 3.3 5.9 \n", "WANT 8.0 13.3 \n", "WARW 2.7 4.4 \n", "WBOU 3.6 5.8 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "ANDE 0.4 ... NaN \n", "BEAC 1.4 ... NaN \n", "BKLN 2.3 ... NaN \n", "BREW 1.7 ... NaN \n", "BRON 2.0 ... NaN \n", "CLAR 0.5 ... NaN \n", "DEPO 1.5 ... NaN \n", "DOVE 0.8 ... NaN \n", "DUAN 0.5 ... NaN \n", "ELDR 0.4 ... NaN \n", "HARP 1.2 ... NaN \n", "HFAL 1.3 ... NaN \n", "KIND 1.0 ... NaN \n", "MANH 2.0 ... NaN \n", "MEDU 1.1 ... NaN \n", "NBRA 0.8 ... NaN \n", "OTIS 0.6 ... NaN \n", "QUEE 2.5 ... NaN \n", "REDH 1.2 ... NaN \n", "ROXB 0.9 ... NaN \n", "SCHA 0.4 ... NaN \n", "SCHO 1.0 ... NaN \n", "SCHU 0.4 ... NaN \n", "SOME 1.4 ... NaN \n", "STAT 1.6 ... NaN \n", "STEP 1.1 ... NaN \n", "SUFF 2.0 ... NaN \n", "TANN 0.7 ... NaN \n", "VOOR 0.6 ... NaN \n", "WALL 1.3 ... NaN \n", "WANT 1.9 ... NaN \n", "WARW 0.6 ... NaN \n", "WBOU 0.9 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "ANDE 0.0 0.0 \n", "BEAC 0.0 0.0 \n", "BKLN NaN NaN \n", "BREW 0.0 0.0 \n", "BRON NaN NaN \n", "CLAR 0.0 0.0 \n", "DEPO 0.0 0.0 \n", "DOVE 0.0 0.0 \n", "DUAN 0.0 0.0 \n", "ELDR 0.0 0.0 \n", "HARP 0.0 0.0 \n", "HFAL 0.0 0.0 \n", "KIND 0.0 0.0 \n", "MANH NaN NaN \n", "MEDU 0.0 0.0 \n", "NBRA 0.0 0.0 \n", "OTIS 0.0 0.0 \n", "QUEE NaN NaN \n", "REDH 0.0 0.0 \n", "ROXB 0.0 0.0 \n", "SCHA 0.0 0.0 \n", "SCHO 0.0 0.0 \n", "SCHU 0.0 0.0 \n", "SOME 0.0 0.0 \n", "STAT NaN NaN \n", "STEP 0.0 0.0 \n", "SUFF 0.0 0.0 \n", "TANN 0.0 0.0 \n", "VOOR 0.0 0.0 \n", "WALL 0.0 0.0 \n", "WANT 0.0 0.0 \n", "WARW 0.0 0.0 \n", "WBOU 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "ANDE 0.0 19.1 19.1 \n", "BEAC 0.0 18.4 19.3 \n", "BKLN NaN NaN NaN \n", "BREW 0.0 20.3 20.3 \n", "BRON NaN NaN NaN \n", "CLAR 0.0 17.9 19.1 \n", "DEPO NaN 19.9 20.7 \n", "DOVE 0.0 19.6 21.5 \n", "DUAN 0.0 19.5 20.2 \n", "ELDR 0.0 19.1 20.1 \n", "HARP 0.0 18.6 19.3 \n", "HFAL 0.0 19.7 21.1 \n", "KIND 0.0 21.3 21.9 \n", "MANH NaN NaN NaN \n", "MEDU 0.0 17.7 19.1 \n", "NBRA 0.0 18.9 20.7 \n", "OTIS NaN 18.4 20.4 \n", "QUEE NaN NaN NaN \n", "REDH 0.0 21.9 23.9 \n", "ROXB NaN 17.2 17.3 \n", "SCHA 0.0 21.5 22.1 \n", "SCHO 0.0 21.1 21.9 \n", "SCHU 0.0 21.2 22.4 \n", "SOME 0.0 20.3 21.7 \n", "STAT NaN NaN NaN \n", "STEP 0.0 19.4 19.9 \n", "SUFF 0.0 20.3 20.1 \n", "TANN 0.0 18.0 18.4 \n", "VOOR 0.0 20.2 21.1 \n", "WALL 0.0 19.9 21.9 \n", "WANT 0.0 23.7 24.4 \n", "WARW 0.0 18.2 21.1 \n", "WBOU 0.0 17.5 20.5 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "ANDE 19.3 0.25 \n", "BEAC 19.8 0.51 \n", "BKLN NaN NaN \n", "BREW 20.3 0.41 \n", "BRON NaN NaN \n", "CLAR 19.3 0.22 \n", "DEPO NaN 0.17 \n", "DOVE 21.5 0.39 \n", "DUAN 20.1 0.55 \n", "ELDR 20.7 0.38 \n", "HARP 19.0 0.31 \n", "HFAL 22.1 0.47 \n", "KIND 21.7 0.46 \n", "MANH NaN NaN \n", "MEDU 18.8 0.39 \n", "NBRA 20.5 0.44 \n", "OTIS NaN 0.34 \n", "QUEE NaN NaN \n", "REDH 24.3 0.28 \n", "ROXB NaN 0.55 \n", "SCHA 22.4 0.22 \n", "SCHO 21.9 0.33 \n", "SCHU 22.1 0.07 \n", "SOME 21.3 0.69 \n", "STAT NaN NaN \n", "STEP 19.7 0.32 \n", "SUFF 21.7 0.47 \n", "TANN 18.4 0.59 \n", "VOOR 21.1 0.25 \n", "WALL 21.5 0.57 \n", "WANT 24.3 0.22 \n", "WARW 21.8 0.43 \n", "WBOU 21.3 0.51 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "ANDE 0.21 0.14 \n", "BEAC 0.36 0.37 \n", "BKLN NaN NaN \n", "BREW 0.33 0.35 \n", "BRON NaN NaN \n", "CLAR 0.22 0.15 \n", "DEPO 0.27 NaN \n", "DOVE 0.22 0.15 \n", "DUAN 0.38 0.42 \n", "ELDR 0.52 0.51 \n", "HARP 0.28 0.52 \n", "HFAL 0.35 0.35 \n", "KIND 0.41 0.44 \n", "MANH NaN NaN \n", "MEDU 0.37 0.38 \n", "NBRA 0.43 0.50 \n", "OTIS 0.28 NaN \n", "QUEE NaN NaN \n", "REDH 0.25 0.30 \n", "ROXB 0.52 NaN \n", "SCHA 0.14 0.05 \n", "SCHO 0.19 0.14 \n", "SCHU 0.12 0.08 \n", "SOME 0.42 0.62 \n", "STAT NaN NaN \n", "STEP 0.33 0.31 \n", "SUFF 0.34 0.28 \n", "TANN 0.49 0.42 \n", "VOOR 0.33 0.19 \n", "WALL 0.42 0.44 \n", "WANT 0.05 0.05 \n", "WARW 0.32 0.31 \n", "WBOU 0.36 0.34 \n", "\n", "[33 rows x 29 columns]" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data[prcpMeasurable]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Of those rows that met the conditon, display the 2-m temperatures." ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ANDE 14.0\n", "BEAC 16.0\n", "BKLN 19.9\n", "BREW 16.2\n", "BRON 18.0\n", "CLAR 13.9\n", "DEPO 16.0\n", "DOVE 16.0\n", "DUAN 13.2\n", "ELDR 14.9\n", "HARP 13.4\n", "HFAL 15.9\n", "KIND 16.7\n", "MANH 18.7\n", "MEDU 14.2\n", "NBRA 14.8\n", "OTIS 15.7\n", "QUEE 19.0\n", "REDH 16.6\n", "ROXB 14.0\n", "SCHA 14.9\n", "SCHO 16.4\n", "SCHU 14.8\n", "SOME 16.5\n", "STAT 18.5\n", "STEP 15.7\n", "SUFF 16.5\n", "TANN 13.1\n", "VOOR 14.9\n", "WALL 16.2\n", "WANT 24.1\n", "WARW 16.2\n", "WBOU 14.4\n", "Name: temp_2m [degC], dtype: float64" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data.loc[prcpMeasurable,'temp_2m [degC]']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Now filter based on two conditions:" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [], "source": [ "WarmWet = (nysm_data['precip_incremental [mm]'] > 0.0) & (nysm_data['temp_2m [degC]'] > thresh)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI False\n", "ANDE False\n", "BATA False\n", "BEAC True\n", "BELD False\n", " ... \n", "WFMB False\n", "WGAT False\n", "WHIT False\n", "WOLC False\n", "YORK False\n", "Length: 126, dtype: bool" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "WarmWet" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
timetemp_2m [degC]temp_9m [degC]relative_humidity [percent]precip_incremental [mm]precip_local [mm]precip_max_intensity [mm/min]avg_wind_speed_prop [m/s]max_wind_speed_prop [m/s]wind_speed_stddev_prop [m/s]...snow_depth [cm]frozen_soil_05cm [bit]frozen_soil_25cm [bit]frozen_soil_50cm [bit]soil_temp_05cm [degC]soil_temp_25cm [degC]soil_temp_50cm [degC]soil_moisture_05cm [m^3/m^3]soil_moisture_25cm [m^3/m^3]soil_moisture_50cm [m^3/m^3]
station
BEAC2021-09-02 02:00:00 UTC16.015.998.61.9337.530.483.26.51.4...NaN0.00.00.018.419.319.80.510.360.37
BKLN2021-09-02 02:00:00 UTC19.920.097.93.2175.190.933.49.52.3...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
BREW2021-09-02 02:00:00 UTC16.216.599.71.0126.890.274.99.81.7...NaN0.00.00.020.320.320.30.410.330.35
BRON2021-09-02 02:00:00 UTC18.017.899.06.86110.351.846.811.52.0...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
DEPO2021-09-02 02:00:00 UTC16.016.190.10.053.170.002.55.81.5...NaN0.00.0NaN19.920.7NaN0.170.27NaN
DOVE2021-09-02 02:00:00 UTC16.0NaN95.21.3524.690.362.14.10.8...NaN0.00.00.019.621.521.50.390.220.15
HFAL2021-09-02 02:00:00 UTC15.915.999.50.8621.890.204.37.51.3...NaN0.00.00.019.721.122.10.470.350.35
KIND2021-09-02 02:00:00 UTC16.716.798.60.739.750.192.35.11.0...NaN0.00.00.021.321.921.70.460.410.44
MANH2021-09-02 02:00:00 UTC18.7NaN100.04.64111.031.225.49.52.0...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
OTIS2021-09-02 02:00:00 UTC15.715.698.80.8828.490.191.12.60.6...NaN0.00.0NaN18.420.4NaN0.340.28NaN
QUEE2021-09-02 02:00:00 UTC19.0NaN98.011.9563.432.6810.016.52.5...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
REDH2021-09-02 02:00:00 UTC16.616.697.80.8219.730.243.76.61.2...NaN0.00.00.021.923.924.30.280.250.30
SCHO2021-09-02 02:00:00 UTC16.416.299.90.559.070.181.64.51.0...NaN0.00.00.021.121.921.90.330.190.14
SOME2021-09-02 02:00:00 UTC16.516.498.31.7546.430.442.66.01.4...NaN0.00.00.020.321.721.30.690.420.62
STAT2021-09-02 02:00:00 UTC18.518.597.60.3993.400.006.19.41.6...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
STEP2021-09-02 02:00:00 UTC15.715.693.00.738.540.213.15.91.1...NaN0.00.00.019.419.919.70.320.330.31
SUFF2021-09-02 02:00:00 UTC16.516.298.60.6566.080.164.212.12.0...NaN0.00.00.020.320.121.70.470.340.28
WALL2021-09-02 02:00:00 UTC16.216.298.41.7724.480.453.35.91.3...NaN0.00.00.019.921.921.50.570.420.44
WANT2021-09-02 02:00:00 UTC24.124.098.70.063.470.008.013.31.9...NaN0.00.00.023.724.424.30.220.050.05
WARW2021-09-02 02:00:00 UTC16.216.2100.01.3546.920.292.74.40.6...NaN0.00.00.018.221.121.80.430.320.31
\n", "

20 rows × 29 columns

\n", "
" ], "text/plain": [ " time temp_2m [degC] temp_9m [degC] \\\n", "station \n", "BEAC 2021-09-02 02:00:00 UTC 16.0 15.9 \n", "BKLN 2021-09-02 02:00:00 UTC 19.9 20.0 \n", "BREW 2021-09-02 02:00:00 UTC 16.2 16.5 \n", "BRON 2021-09-02 02:00:00 UTC 18.0 17.8 \n", "DEPO 2021-09-02 02:00:00 UTC 16.0 16.1 \n", "DOVE 2021-09-02 02:00:00 UTC 16.0 NaN \n", "HFAL 2021-09-02 02:00:00 UTC 15.9 15.9 \n", "KIND 2021-09-02 02:00:00 UTC 16.7 16.7 \n", "MANH 2021-09-02 02:00:00 UTC 18.7 NaN \n", "OTIS 2021-09-02 02:00:00 UTC 15.7 15.6 \n", "QUEE 2021-09-02 02:00:00 UTC 19.0 NaN \n", "REDH 2021-09-02 02:00:00 UTC 16.6 16.6 \n", "SCHO 2021-09-02 02:00:00 UTC 16.4 16.2 \n", "SOME 2021-09-02 02:00:00 UTC 16.5 16.4 \n", "STAT 2021-09-02 02:00:00 UTC 18.5 18.5 \n", "STEP 2021-09-02 02:00:00 UTC 15.7 15.6 \n", "SUFF 2021-09-02 02:00:00 UTC 16.5 16.2 \n", "WALL 2021-09-02 02:00:00 UTC 16.2 16.2 \n", "WANT 2021-09-02 02:00:00 UTC 24.1 24.0 \n", "WARW 2021-09-02 02:00:00 UTC 16.2 16.2 \n", "\n", " relative_humidity [percent] precip_incremental [mm] \\\n", "station \n", "BEAC 98.6 1.93 \n", "BKLN 97.9 3.21 \n", "BREW 99.7 1.01 \n", "BRON 99.0 6.86 \n", "DEPO 90.1 0.05 \n", "DOVE 95.2 1.35 \n", "HFAL 99.5 0.86 \n", "KIND 98.6 0.73 \n", "MANH 100.0 4.64 \n", "OTIS 98.8 0.88 \n", "QUEE 98.0 11.95 \n", "REDH 97.8 0.82 \n", "SCHO 99.9 0.55 \n", "SOME 98.3 1.75 \n", "STAT 97.6 0.39 \n", "STEP 93.0 0.73 \n", "SUFF 98.6 0.65 \n", "WALL 98.4 1.77 \n", "WANT 98.7 0.06 \n", "WARW 100.0 1.35 \n", "\n", " precip_local [mm] precip_max_intensity [mm/min] \\\n", "station \n", "BEAC 37.53 0.48 \n", "BKLN 75.19 0.93 \n", "BREW 26.89 0.27 \n", "BRON 110.35 1.84 \n", "DEPO 3.17 0.00 \n", "DOVE 24.69 0.36 \n", "HFAL 21.89 0.20 \n", "KIND 9.75 0.19 \n", "MANH 111.03 1.22 \n", "OTIS 28.49 0.19 \n", "QUEE 63.43 2.68 \n", "REDH 19.73 0.24 \n", "SCHO 9.07 0.18 \n", "SOME 46.43 0.44 \n", "STAT 93.40 0.00 \n", "STEP 8.54 0.21 \n", "SUFF 66.08 0.16 \n", "WALL 24.48 0.45 \n", "WANT 3.47 0.00 \n", "WARW 46.92 0.29 \n", "\n", " avg_wind_speed_prop [m/s] max_wind_speed_prop [m/s] \\\n", "station \n", "BEAC 3.2 6.5 \n", "BKLN 3.4 9.5 \n", "BREW 4.9 9.8 \n", "BRON 6.8 11.5 \n", "DEPO 2.5 5.8 \n", "DOVE 2.1 4.1 \n", "HFAL 4.3 7.5 \n", "KIND 2.3 5.1 \n", "MANH 5.4 9.5 \n", "OTIS 1.1 2.6 \n", "QUEE 10.0 16.5 \n", "REDH 3.7 6.6 \n", "SCHO 1.6 4.5 \n", "SOME 2.6 6.0 \n", "STAT 6.1 9.4 \n", "STEP 3.1 5.9 \n", "SUFF 4.2 12.1 \n", "WALL 3.3 5.9 \n", "WANT 8.0 13.3 \n", "WARW 2.7 4.4 \n", "\n", " wind_speed_stddev_prop [m/s] ... snow_depth [cm] \\\n", "station ... \n", "BEAC 1.4 ... NaN \n", "BKLN 2.3 ... NaN \n", "BREW 1.7 ... NaN \n", "BRON 2.0 ... NaN \n", "DEPO 1.5 ... NaN \n", "DOVE 0.8 ... NaN \n", "HFAL 1.3 ... NaN \n", "KIND 1.0 ... NaN \n", "MANH 2.0 ... NaN \n", "OTIS 0.6 ... NaN \n", "QUEE 2.5 ... NaN \n", "REDH 1.2 ... NaN \n", "SCHO 1.0 ... NaN \n", "SOME 1.4 ... NaN \n", "STAT 1.6 ... NaN \n", "STEP 1.1 ... NaN \n", "SUFF 2.0 ... NaN \n", "WALL 1.3 ... NaN \n", "WANT 1.9 ... NaN \n", "WARW 0.6 ... NaN \n", "\n", " frozen_soil_05cm [bit] frozen_soil_25cm [bit] \\\n", "station \n", "BEAC 0.0 0.0 \n", "BKLN NaN NaN \n", "BREW 0.0 0.0 \n", "BRON NaN NaN \n", "DEPO 0.0 0.0 \n", "DOVE 0.0 0.0 \n", "HFAL 0.0 0.0 \n", "KIND 0.0 0.0 \n", "MANH NaN NaN \n", "OTIS 0.0 0.0 \n", "QUEE NaN NaN \n", "REDH 0.0 0.0 \n", "SCHO 0.0 0.0 \n", "SOME 0.0 0.0 \n", "STAT NaN NaN \n", "STEP 0.0 0.0 \n", "SUFF 0.0 0.0 \n", "WALL 0.0 0.0 \n", "WANT 0.0 0.0 \n", "WARW 0.0 0.0 \n", "\n", " frozen_soil_50cm [bit] soil_temp_05cm [degC] soil_temp_25cm [degC] \\\n", "station \n", "BEAC 0.0 18.4 19.3 \n", "BKLN NaN NaN NaN \n", "BREW 0.0 20.3 20.3 \n", "BRON NaN NaN NaN \n", "DEPO NaN 19.9 20.7 \n", "DOVE 0.0 19.6 21.5 \n", "HFAL 0.0 19.7 21.1 \n", "KIND 0.0 21.3 21.9 \n", "MANH NaN NaN NaN \n", "OTIS NaN 18.4 20.4 \n", "QUEE NaN NaN NaN \n", "REDH 0.0 21.9 23.9 \n", "SCHO 0.0 21.1 21.9 \n", "SOME 0.0 20.3 21.7 \n", "STAT NaN NaN NaN \n", "STEP 0.0 19.4 19.9 \n", "SUFF 0.0 20.3 20.1 \n", "WALL 0.0 19.9 21.9 \n", "WANT 0.0 23.7 24.4 \n", "WARW 0.0 18.2 21.1 \n", "\n", " soil_temp_50cm [degC] soil_moisture_05cm [m^3/m^3] \\\n", "station \n", "BEAC 19.8 0.51 \n", "BKLN NaN NaN \n", "BREW 20.3 0.41 \n", "BRON NaN NaN \n", "DEPO NaN 0.17 \n", "DOVE 21.5 0.39 \n", "HFAL 22.1 0.47 \n", "KIND 21.7 0.46 \n", "MANH NaN NaN \n", "OTIS NaN 0.34 \n", "QUEE NaN NaN \n", "REDH 24.3 0.28 \n", "SCHO 21.9 0.33 \n", "SOME 21.3 0.69 \n", "STAT NaN NaN \n", "STEP 19.7 0.32 \n", "SUFF 21.7 0.47 \n", "WALL 21.5 0.57 \n", "WANT 24.3 0.22 \n", "WARW 21.8 0.43 \n", "\n", " soil_moisture_25cm [m^3/m^3] soil_moisture_50cm [m^3/m^3] \n", "station \n", "BEAC 0.36 0.37 \n", "BKLN NaN NaN \n", "BREW 0.33 0.35 \n", "BRON NaN NaN \n", "DEPO 0.27 NaN \n", "DOVE 0.22 0.15 \n", "HFAL 0.35 0.35 \n", "KIND 0.41 0.44 \n", "MANH NaN NaN \n", "OTIS 0.28 NaN \n", "QUEE NaN NaN \n", "REDH 0.25 0.30 \n", "SCHO 0.19 0.14 \n", "SOME 0.42 0.62 \n", "STAT NaN NaN \n", "STEP 0.33 0.31 \n", "SUFF 0.34 0.28 \n", "WALL 0.42 0.44 \n", "WANT 0.05 0.05 \n", "WARW 0.32 0.31 \n", "\n", "[20 rows x 29 columns]" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nysm_data.loc[WarmWet]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Question: What if you wanted to produce some text output for each time, expressing certain extreme values for a variety of parameters, and reference it to a column in another DataFrame?
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Answer: Use the idxmax / idxmin method to display the row label of the maximum / minimum value in a Series.
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's create `Series` objects from the NYS Mesonet Station Info table, as well as from the data table." ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [], "source": [ "stid = nysm_sites['stid']\n", "lats = nysm_sites['lat']\n", "lons = nysm_sites['lon']\n", "\n", "time = nysm_data['time']\n", "tmpc = nysm_data['temp_2m [degC]']\n", "rh = nysm_data['relative_humidity [percent]']\n", "pres = nysm_data['station_pressure [mbar]']\n", "wspd = nysm_data['max_wind_speed_prop [m/s]']\n", "drct = nysm_data['wind_direction_prop [degrees]']\n", "pinc = nysm_data['precip_incremental [mm]']\n", "ptot = nysm_data['precip_local [mm]']\n", "pint = nysm_data['precip_max_intensity [mm/min]']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What is the maximum 2m temperature of all stations in the `DataFrame`?" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "24.1" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tmpcMax = tmpc.max()\n", "tmpcMax" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In what row # occurs the maximum temperature? We might think to use the `idxmax` method ..." ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'WANT'" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "siteTmpcMax = tmpc.idxmax()\n", "siteTmpcMax" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### That works, but it outputs the row index name (recall that when we read in the data file, we specified that the site ID column woudl be used for the row index name). \n", "Let's use the `argmax` method to get the row index number." ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "114" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "indxTmpcMax = tmpc.argmax()\n", "indxTmpcMax" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can do the same for another extrema ... e.g., minimum 2m temperature." ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "tmpcMin = tmpc.min()\n", "siteTmpcMin = tmpc.idxmin()\n", "indxTmpcMin = tmpc.argmin()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Knowing the row index #, we can get the corresponding full station name from the NYSM station table:" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "siteNameTmpcMax = nysm_sites['name'].iloc[indxTmpcMax]" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [], "source": [ "siteNameTmpcMin = nysm_sites['name'].iloc[indxTmpcMin]" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Warmest NYSM Site: Wantagh (WANT), 24.1\n", "Coldest NYSM Site: Edwards (EDWA), 10.3\n" ] } ], "source": [ "print (\"Warmest NYSM Site: %s (%s), %.1f\" % (siteNameTmpcMax,siteTmpcMax, tmpcMax))\n", "print (\"Coldest NYSM Site: %s (%s), %.1f\" % (siteNameTmpcMin,siteTmpcMin, tmpcMin))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Exercise: find and output the highest and lowest sea-level pressure from the dataset.\n", "

Consider: would you want to just find the max/min station pressure values, and then convert those two values to SLP? Or would it be better to first apply the SLP reduction to the entire station pressure Series?

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Solution: It's possible that the max/min station pressure values might not also correspond to the stations with max/min SLP, since there is elevation dependence.\n", "

So, first create a series of the station pressure values, and then apply the SLP reduction formula. This requires that we retrieve the elevation data from the site file as well.

" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "pres = nysm_data['station_pressure [mbar]']\n", "elev = nysm_sites['elevation']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Now, it should be as straightforward as passing these two `Series` to the SLP reduction formula ... shouldn't it?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Employ the SLP reduction equation that we used in the previous notebook:" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [], "source": [ "sensorHeight = .5\n", "# Reduce station pressure to SLP. Source: https://www.sandhurstweather.org.uk/barometric.pdf \n", "slp = pres/np.exp(-1*(elev+sensorHeight)/((tmpc+273.15) * 29.263))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Looks like it worked! Let's see what we get for max/min values:" ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "nan nan\n" ] } ], "source": [ "print(slp.max(), slp.min())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Uh-oh ... `nan` signifies missing data! Let's look at what our **slp** `Series` object looks like:" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 NaN\n", "1 NaN\n", "2 NaN\n", "3 NaN\n", "4 NaN\n", " ..\n", "WFMB NaN\n", "WGAT NaN\n", "WHIT NaN\n", "WOLC NaN\n", "YORK NaN\n", "Length: 252, dtype: float64" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "slp" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Look at the length of the series! It's 252 ... which is 126 times 2!\n", "Let's just look at all the rows of the series first ..." ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 NaN\n", "1 NaN\n", "2 NaN\n", "3 NaN\n", "4 NaN\n", "5 NaN\n", "6 NaN\n", "7 NaN\n", "8 NaN\n", "9 NaN\n", "10 NaN\n", "11 NaN\n", "12 NaN\n", "13 NaN\n", "14 NaN\n", "15 NaN\n", "16 NaN\n", "17 NaN\n", "18 NaN\n", "19 NaN\n", "20 NaN\n", "21 NaN\n", "22 NaN\n", "23 NaN\n", "24 NaN\n", "25 NaN\n", "26 NaN\n", "27 NaN\n", "28 NaN\n", "29 NaN\n", "30 NaN\n", "31 NaN\n", "32 NaN\n", "33 NaN\n", "34 NaN\n", "35 NaN\n", "36 NaN\n", "37 NaN\n", "38 NaN\n", "39 NaN\n", "40 NaN\n", "41 NaN\n", "42 NaN\n", "43 NaN\n", "44 NaN\n", "45 NaN\n", "46 NaN\n", "47 NaN\n", "48 NaN\n", "49 NaN\n", "50 NaN\n", "51 NaN\n", "52 NaN\n", "53 NaN\n", "54 NaN\n", "55 NaN\n", "56 NaN\n", "57 NaN\n", "58 NaN\n", "59 NaN\n", "60 NaN\n", "61 NaN\n", "62 NaN\n", "63 NaN\n", "64 NaN\n", "65 NaN\n", "66 NaN\n", "67 NaN\n", "68 NaN\n", "69 NaN\n", "70 NaN\n", "71 NaN\n", "72 NaN\n", "73 NaN\n", "74 NaN\n", "75 NaN\n", "76 NaN\n", "77 NaN\n", "78 NaN\n", "79 NaN\n", "80 NaN\n", "81 NaN\n", "82 NaN\n", "83 NaN\n", "84 NaN\n", "85 NaN\n", "86 NaN\n", "87 NaN\n", "88 NaN\n", "89 NaN\n", "90 NaN\n", "91 NaN\n", "92 NaN\n", "93 NaN\n", "94 NaN\n", "95 NaN\n", "96 NaN\n", "97 NaN\n", "98 NaN\n", "99 NaN\n", "100 NaN\n", "101 NaN\n", "102 NaN\n", "103 NaN\n", "104 NaN\n", "105 NaN\n", "106 NaN\n", "107 NaN\n", "108 NaN\n", "109 NaN\n", "110 NaN\n", "111 NaN\n", "112 NaN\n", "113 NaN\n", "114 NaN\n", "115 NaN\n", "116 NaN\n", "117 NaN\n", "118 NaN\n", "119 NaN\n", "120 NaN\n", "121 NaN\n", "122 NaN\n", "123 NaN\n", "124 NaN\n", "125 NaN\n", "ADDI NaN\n", "ANDE NaN\n", "BATA NaN\n", "BEAC NaN\n", "BELD NaN\n", "BELL NaN\n", "BELM NaN\n", "BERK NaN\n", "BING NaN\n", "BKLN NaN\n", "BRAN NaN\n", "BREW NaN\n", "BROC NaN\n", "BRON NaN\n", "BROO NaN\n", "BSPA NaN\n", "BUFF NaN\n", "BURD NaN\n", "BURT NaN\n", "CAMD NaN\n", "CAPE NaN\n", "CHAZ NaN\n", "CHES NaN\n", "CINC NaN\n", "CLAR NaN\n", "CLIF NaN\n", "CLYM NaN\n", "COBL NaN\n", "COHO NaN\n", "COLD NaN\n", "COPA NaN\n", "COPE NaN\n", "CROG NaN\n", "CSQR NaN\n", "DELE NaN\n", "DEPO NaN\n", "DOVE NaN\n", "DUAN NaN\n", "EAUR NaN\n", "EDIN NaN\n", "EDWA NaN\n", "ELDR NaN\n", "ELLE NaN\n", "ELMI NaN\n", "ESSX NaN\n", "FAYE NaN\n", "FRED NaN\n", "GABR NaN\n", "GFAL NaN\n", "GFLD NaN\n", "GROT NaN\n", "GROV NaN\n", "HAMM NaN\n", "HARP NaN\n", "HARR NaN\n", "HART NaN\n", "HERK NaN\n", "HFAL NaN\n", "ILAK NaN\n", "JOHN NaN\n", "JORD NaN\n", "KIND NaN\n", "LAUR NaN\n", "LOUI NaN\n", "MALO NaN\n", "MANH NaN\n", "MEDI NaN\n", "MEDU NaN\n", "MORR NaN\n", "NBRA NaN\n", "NEWC NaN\n", "NHUD NaN\n", "OLDF NaN\n", "OLEA NaN\n", "ONTA NaN\n", "OPPE NaN\n", "OSCE NaN\n", "OSWE NaN\n", "OTIS NaN\n", "OWEG NaN\n", "PENN NaN\n", "PHIL NaN\n", "PISE NaN\n", "POTS NaN\n", "QUEE NaN\n", "RAND NaN\n", "RAQU NaN\n", "REDF NaN\n", "REDH NaN\n", "ROXB NaN\n", "RUSH NaN\n", "SARA NaN\n", "SBRI NaN\n", "SCHA NaN\n", "SCHO NaN\n", "SCHU NaN\n", "SCIP NaN\n", "SHER NaN\n", "SOME NaN\n", "SOUT NaN\n", "SPRA NaN\n", "SPRI NaN\n", "STAT NaN\n", "STEP NaN\n", "STON NaN\n", "SUFF NaN\n", "TANN NaN\n", "TICO NaN\n", "TULL NaN\n", "TUPP NaN\n", "TYRO NaN\n", "VOOR NaN\n", "WALL NaN\n", "WALT NaN\n", "WANT NaN\n", "WARS NaN\n", "WARW NaN\n", "WATE NaN\n", "WBOU NaN\n", "WELL NaN\n", "WEST NaN\n", "WFMB NaN\n", "WGAT NaN\n", "WHIT NaN\n", "WOLC NaN\n", "YORK NaN\n", "dtype: float64" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pd.set_option('display.max_rows',None)\n", "slp" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Missing (i.e. `NaN`) values everywhere, and indeed, we have double the number of rows that we thought we'd have!\n", "Let's take a look at our **elev** and **pres** Series objects." ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [], "source": [ "# go back to default # of rows displayed\n", "pd.set_option('display.max_rows',10)" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0 507.6140\n", "1 518.2820\n", "2 276.1200\n", "3 90.1598\n", "4 470.3700\n", " ... \n", "121 614.5990\n", "122 442.9660\n", "123 36.5638\n", "124 121.2190\n", "125 177.9420\n", "Name: elevation, Length: 126, dtype: float64" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "elev" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI 951.47\n", "ANDE 947.40\n", "BATA 979.93\n", "BEAC 994.69\n", "BELD 954.02\n", " ... \n", "WFMB 941.26\n", "WGAT 959.46\n", "WHIT 1006.29\n", "WOLC 996.90\n", "YORK 991.17\n", "Name: station_pressure [mbar], Length: 126, dtype: float64" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pres" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### The issue is that our **elev** and **pres** series have *different* row indexes! In order for calculations to work when applied to different Series objects, their indexes *must correspond*!\n", "Recall that we set the row index for the `nysm_data` DataFrame to *station id*, but kept the default for `nysm_sites`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Since `tmpc` and `pres` both have their row index set to station id, let's set `elev`'s index to that as well. " ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "station\n", "ADDI 507.6140\n", "ANDE 518.2820\n", "BATA 276.1200\n", "BEAC 90.1598\n", "BELD 470.3700\n", " ... \n", "WFMB 614.5990\n", "WGAT 442.9660\n", "WHIT 36.5638\n", "WOLC 121.2190\n", "YORK 177.9420\n", "Name: elevation, Length: 126, dtype: float64" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "elev.index = pres.index\n", "elev" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Now, repeat the SLP reduction calculation." ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1014.0598722184151 998.940242497739\n" ] } ], "source": [ "slp = pres/np.exp(-1*(elev+sensorHeight)/((tmpc+273.15) * 29.263))\n", "slpMax = slp.max()\n", "slpMin = slp.min()\n", "print (slpMax, slpMin)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### With the row indexes common among all three Series in the equation, it works!\n", "We can now find the corresponding row numbers and summarize as we did for max/min temperature." ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [], "source": [ "siteSlpMax = slp.idxmax()\n", "indxSlpMax = slp.argmax()\n", "siteSlpMin = slp.idxmin()\n", "indxSlpMin = slp.argmin()" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [], "source": [ "siteNameSlpMax = nysm_sites['name'].iloc[indxSlpMax]\n", "siteNameSlpMin = nysm_sites['name'].iloc[indxSlpMin]" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Highest SLP NYSM Site: Clymer (CLYM), 1014.1\n", "Lowest SLP NYSM Site: Brooklyn (BKLN), 998.9\n" ] } ], "source": [ "print (\"Highest SLP NYSM Site: %s (%s), %.1f\" % (siteNameSlpMax,siteSlpMax, slpMax))\n", "print (\"Lowest SLP NYSM Site: %s (%s), %.1f\" % (siteNameSlpMin,siteSlpMin, slpMin))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Tip: Be mindful of how you define row indexes when working with multiple `DataFrame`s!
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## Summary\n", "* Use the `loc` and `iloc` methods to select rows, columns, or combinations of the two in a Pandas `DataFrame`.\n", "* These two methods can also be used for `Series`.\n", "* Setting *conditions* on particular rows or columns produce True/False values, which can then be passed into a `DataFrame` or `Series` for more meaningful output.\n", "* The `idxmax` / `idxmin` and `argmax` / `argmin`functions can be combined to provide meaningful output from multiple `DataFrames`.\n", "\n", "### What's Next?\n", "In the next notebook, we will explore how Pandas lets us efficiently work with data that are date- and time-based.\n", "\n", "## Resources and References\n", "1. [Selecting in Pandas](https://pandas.pydata.org/docs/getting_started/index.html#getting-started)\n", "1. [Pandas User Guide](https://pandas.pydata.org/docs/user_guide/index.html#user-guide)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 August 2022 Environment", "language": "python", "name": "aug22" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.5" } }, "nbformat": 4, "nbformat_minor": 4 }