Pandas Notebook 1, ATM350 Spring 2026

Here, we read in a text file that has climatological data compiled at the National Weather Service in Albany NY for 2025, previously downloaded and reformatted from the xmACIS2 climate data portal.

We will use the Pandas library to read and analyze the data. We will also use the Matplotlib package to visualize it.

Motivating Science Questions:¶

1. How can we analyze and display tabular climate data for a site?

2. What was the yearly trace of max/min temperatures for Albany, NY last year?

3. What was the most common 10-degree maximum temperature range for Albany, NY last year?

Specify the location of the file that contains the climo data. Use the linux ls command to verify it exists.¶

Note that in a Jupyter notebook, we can simply use the ! directive to “call” a Linux command.¶

Also notice how we refer to a Python variable name when passing it to a Linux command line in this way ... we enclose it in braces!¶

Use pandas’ read_csv method to open the file. Specify that the data is to be read in as strings (not integers nor floating points).¶

Once this call succeeds, it returns a Pandas Dataframe object which we reference as df¶

By simply typing the name of the dataframe object, we can get some of its contents to be “pretty-printed” to the notebook!¶

Our dataframe has 365 or 366 rows (corresponding to all the days in the year) and 10 columns that contain data. This is expressed by calling the shape attribute of the dataframe. The first number in the pair is the # of rows, while the second is the # of columns.¶

It will be useful to have a variable (more accurately, an object ) that holds the value of the number of rows, and another for the number of columns.¶

Remember that Python is a language that uses zero-based indexing, so the first value is accessed as element 0, and the second as element 1!¶

Look at the syntax we use below to print out the (integer) value of nRows ... it’s another example of string formating.¶

Let’s do the same for the # of columns.¶

To access the values in a particular column, we reference it with its column name as a string. The next cell pulls in all values of the year-month-date column, and assigns it to an object of the same name. We could have named the object anything we wanted, not just Date ... but on the right side of the assignment statement, we have to use the exact name of the column.¶

Print out what this object looks like.

Each column of a Pandas dataframe is known as a series. It is basically an array of values, each of which has a corresponding row #. By default, row #'s accompanying a Series are numbered consecutively, starting with 0 (since Python’s convention is to use zero-based indexing).¶

We can reference a particular value, or set of values, of a Series by using array-based notation. Below, let’s print out the first 30 rows of the dates.¶

Similarly, let’s print out the last, or 364th row (Why is it 364, not 365???)¶

Note that the selection method does not accept a single negative value.For example, the cell below, which basically requests the last row, won’t work!

However, using a negative value as part of a slice does work! Let’s output the last nine values of the series:

Examine the two newly-defined Series objects.

Let’s now list all the days that the high temperature was >= 90. Note carefully how we express this test. It will fail!¶

Why did it fail? Remember, when we read in the file, we had Pandas assign the type of every column to string! We need to change the type of maxT to a numerical value. Let’s use a 32-bit floating point #, as that will be more than enough precision for this type of measurement. We’ll do the same for the minimum temp.¶

Now, the test works. What does this data series look like? It actually is a table of booleans ... i.e., true/false values.¶

As the default output only includes the first and last 5 rows , let’s slice and pull out a period in the middle of the year, where we might be more likely to get some Trues!¶

hotDays, as defined above, is Pandas series. We can treat it the same as an existing column in the original dataframe. By passing it into the dataframe, it returns a new DataFrame, which contains only those rows that match the criterion.¶

Then, recall that to get a count of the # of rows, we take the first (0th) element of the array returned by a call to the shape method.

Determine the number of days exceeding the specified criterion, and then print out an informative sentence.

Let’s reverse the sense of the test, and get its count. The two counts should add up to the total number of days in the year!¶

We can combine a test of two different thresholds. Let’s get a count of days where the max. temperature was in the 70s or 80s.¶

Let’s show all the climate data for all these “pleasantly warm” days!¶

Notice that after a certain point, not all the rows are displayed to the notebook. We can eliminate the limit of maximum rows and thus show all of the matching days.¶

We can reset this option back to the default.¶

Now let’s visualize the temperature trace over the year! Pandas has a method that directly calls Matplotlib’s plotting package.¶

The data plotted fine, but the look could be better. First, let’s import a package, seaborn, that when imported and set using its own method, makes matplotlib’s graphs look better.¶

Info on seaborn: https://seaborn.pydata.org/index.html

Next, let’s plot the two traces simultaneously on the graph so we can better discern max and min temps (this will also enure a single y-axis that will encompass the range of temperature values). We’ll also add some helpful labels and expand the size of the figure.¶

You might notice that this graphic did not instantly render. Note that the x-axis label is virtually unreadable. This is because every date is being printed along it!¶

We will deal with this by using one of Pandas’ methods that take strings and convert them to a special type of data ... not strings nor numbers, but datetime objects. Note carefully how we do this here ... it is not terribly intuitive, but we’ll explain it more in an upcoming lecture/notebook on datetime. You will see though that the output column now looks a bit more date-like, with a four-digit year followed by two-digit month and date.¶

Matplotlib will recognize this array as being date/time-related, and when we pass it in as the x-axis, the graphic appears faster, and we also have a more meaningful x-axis label.¶

We’ll further refine the look of the plot by adding a legend and have vertical grid lines on a frequency of one month.¶

Let’s save our beautiful graphic to disk.¶

Now, let’s answer the question, “what was the most common range of maximum temperatures last year in Albany?” via a histogram. We use matplotlib’s hist method.¶

Just about perfect ... except it would be nice to label every ten degrees on the x-axis. This will require two tweaks to the code cell.¶

First, we pass an additional argument to ax.hist.

Thus, we’ll add bins=10 to the ax.hist function call.

Secondly, to properly label the x-axis, we’ll add a method from another Matplotlib class ... Locator. From Matplotlib’s documentation,

A useful semi-automatic tick locator is MultipleLocator. It is initialized with a base, e.g., 10,
and it picks axis limits and ticks that are multiples of that base.

So let’s import the MultipleLocator method, and specify that each x-axis tick mark and label should occur with x-axis array values that are multiples of 10.

Save this histogram to disk.¶

Use the describe method on the maximum temperature series to reveal some simple statistical properties.

What’s next?¶

We will further explore the annual Albany climate dataset with Pandas.

References¶

1. Project Pythia: Pandas

2. The Carpentries: Pandas

3. Project Pythia: Matplotlib

4. Matplotlib: Setting x/y-axis tick label properties