Assignment 2¶

Before working on this assignment please read these instructions fully. In the submission area, you will notice that you can click the link to Preview the Grading for each step of the assignment. This is the criteria that will be used for peer grading. Please familiarize yourself with the criteria before beginning the assignment.

An NOAA dataset has been stored in the file data/C2A2_data/BinnedCsvs_d400/fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89.csv. This is the dataset to use for this assignment. Note: The data for this assignment comes from a subset of The National Centers for Environmental Information (NCEI) Daily Global Historical Climatology Network (GHCN-Daily). The GHCN-Daily is comprised of daily climate records from thousands of land surface stations across the globe.

Each row in the assignment datafile corresponds to a single observation.

The following variables are provided to you:

id : station identification code
date : date in YYYY-MM-DD format (e.g. 2012-01-24 = January 24, 2012)
element : indicator of element type
- TMAX : Maximum temperature (tenths of degrees C)
- TMIN : Minimum temperature (tenths of degrees C)
value : data value for element (tenths of degrees C)

For this assignment, you must:

Read the documentation and familiarize yourself with the dataset, then write some python code which returns a line graph of the record high and record low temperatures by day of the year over the period 2005-2014. The area between the record high and record low temperatures for each day should be shaded.
Overlay a scatter of the 2015 data for any points (highs and lows) for which the ten year record (2005-2014) record high or record low was broken in 2015.
Watch out for leap days (i.e. February 29th), it is reasonable to remove these points from the dataset for the purpose of this visualization.
Make the visual nice! Leverage principles from the first module in this course when developing your solution. Consider issues such as legends, labels, and chart junk.

The data you have been given is near Ann Arbor, Michigan, United States, and the stations the data comes from are shown on the map below.

#%matplotlib notebook

import matplotlib.pyplot as plt
import matplotlib as mpl
#mpl.get_backend()
import matplotlib.dates as mdates

import pandas as pd
import numpy as np
import datetime as dt

data_file = "fb441e62df2d58994928907a91895ec62c2c42e6cd075c2700843b89.csv"
raw_data = pd.read_csv(data_file)
raw_data.to_csv("raw_data.csv")

#print(raw_data.describe())
#print(raw_data.head())


# take the inverse of the mask to delete all leap days
raw_data = raw_data[~raw_data['Date'].str.contains("02-29")]

# separate 2005-2014 data and 2015 data
data_5_14 = raw_data[raw_data['Date'] < '2015-01-01' ]
data_2015 = raw_data[raw_data['Date'] > '2014-12-31']
#print(data_2015.head())

# separate TMAX data from TMIN data
max = data_5_14[data_5_14['Element']=='TMAX']
#max.to_csv('max.csv')
min = data_5_14[data_5_14['Element']=='TMIN']

# find max and min values for each date from among the
# measuring stations
max = max.groupby(['Date']).max()
min = min.groupby(['Date']).min()
data_2015_max = data_2015.groupby(['Date']).max()
data_2015_min = data_2015.groupby(['Date']).min()
data_2015 = data_2015_max.append(data_2015_min)
#print('new data_2015: \n', data_2015.describe())
#max.reset_index(inplace=True)
#max.to_csv('max_after_groupby.csv')

# drop unnecessary columns
max = max.drop(columns=['ID','Element'])
min = min.drop(columns=['ID','Element'])
data_2015.drop(columns=['ID','Element'])

# get the absolute highs and lows
max_high_temp = max.max()[0]
min_low_temp = min.min()[0]
#print('max_high_temp: ', max_high_temp)
#print('min_low_temp: ', min_low_temp)

# remove 20115 data between max_high_temp and min_low_temp
data_2015 = data_2015[(data_2015['Data_Value'] > max_high_temp) 
                      | (data_2015['Data_Value'] < min_low_temp)]

#print("for scatter plot: \n",data_2015)


# prepare the dataframes for plotting
max = max.reset_index()
min = min.reset_index()
data_2015 = data_2015.reset_index()

#print('max: \n', max.describe())
#print('min: \n', min.describe())
#print('data_2015: \n', data_2015.describe())

max_values = max['Data_Value'].to_numpy()
max_values = max_values / 10
max_dates = max['Date'].to_numpy()

# convert np array of string dates to list of datetime.dates
max_dates = \
    [dt.datetime.strptime(d,'%Y-%m-%d').date() for d in max_dates]

#print(max_dates)
#print(type(max_dates))
min_values = min['Data_Value'].to_numpy()
min_values = min_values / 10
min_dates = min['Date'].to_numpy()

# convert np array of string dates to list of datetime.dates
min_dates = \
    [dt.datetime.strptime(d,'%Y-%m-%d').date() for d in min_dates]

print("888888888", max_dates[0], max_dates[-1])
data_2015_values = data_2015['Data_Value'].to_numpy()
data_2015_values = data_2015_values / 10
data_2015_dates = data_2015['Date'].to_numpy()

print("******", type(data_2015_dates[0]))


#
# First Axes.....
#

first_axes = plt.gca()
x = first_axes.xaxis

# rotate the tick labels for the x axis
for item in x.get_ticklabels():
    item.set_rotation(-45)

first_axes.set_xlabel('\n Daily Highs and Lows 2004 through 2014')
first_axes.set_ylabel('Temperature (C)')
first_axes.set_title('2015 Temperatures that Exceed Maximum Highs and Lows from \
2004 through 2014 \n')

#first_axes.set_xlim(left=min_dates.min(), right=min_dates.max())

first_axes.xaxis.set_major_formatter(mdates.DateFormatter('%m/%d/%Y'))
first_axes.xaxis.set_major_locator(mdates.AutoDateLocator())
first_axes.set_ylim(top=70, bottom=-40)

plt.subplots_adjust(bottom=0.25, top=0.7)
f_axes_max_line, = first_axes.plot(max_dates, 
                                  max_values, 
                                  'r', 
                                  label='Maximum', 
                                  alpha=0.5, 
                                  linewidth=0.5)


f_axes_min_line, = first_axes.plot(min_dates, 
                                  min_values, 
                                  'blue', 
                                  label='Minimum', 
                                  alpha=0.5, 
                                  linewidth=0.5)

first_axes.fill_between(max_dates, max_values, min_values, facecolor='blue', alpha=0.25)

plt.subplots_adjust(bottom=0.25, top=0.55)
#
# build second axes
#


second_axes = first_axes.twiny() 
second_axes.set_xlabel("2015 Temps Exceeding 2004 to 2014 Highs and Lows\n")

second_axes.xaxis.set_major_formatter(mdates.DateFormatter('%m/%d/%Y'))
second_axes.xaxis.set_major_locator(mdates.DayLocator())

left = mpl.dates.datestr2num('2015-02-01')
right = mpl.dates.datestr2num('2015-02-28')
#left = mpl.dates.datestr2num('2015-01-01')
#right = mpl.dates.datestr2num('2015-12-31')
second_axes.set_xlim(left=left, right=right)
# rotate the tick labels for the upper x axis

# rotate the axis labels
top_x = second_axes.xaxis
for item in top_x.get_ticklabels():
    item.set_rotation(315)

# reduce the number of xticks to every other xtick in February
xticks = second_axes.get_xticks()
xticks = xticks[0::2]
second_axes.set_xticks(xticks)

#print(data_2015_dates)
#print(type(data_2015_dates[0]))
data_2015_dates_num = mpl.dates.datestr2num(data_2015_dates)
print(data_2015_dates[0])
print(type(data_2015_dates[0]))

'''
second_axes.annotate(s='some string', 
                     xy=(data_2015_dates[0],data_2015_values[0]), 
                     xytext=(-10.0,0.0),
                     textcoords='offset points',
                     ha='right')
'''
s_axis_scatter = second_axes.scatter(
    data_2015_dates_num, 
    data_2015_values) 

plt.legend([f_axes_max_line,f_axes_min_line, s_axis_scatter],
           ("2004/14 Max Temps",'2004/14 Min Temps','2015 Extremes'),
           loc='upper left',
           fontsize='small')

fig = plt.gcf()
fig.set_size_inches(10, 10)
#plt.gcf().autofmt_xdate()

#host.yaxis.get_label().set_color(p1.get_color())
#leg.texts[0].set_color(p1.get_color())

#par.yaxis.get_label().set_color(p2.get_color())
#leg.texts[1].set_color(p2.get_color())

888888888 2005-01-01 2014-12-31
****** <class 'str'>
2015-02-20
<class 'str'>

	First you prepare the aromatics: finely sliced leeks chopped cilantro ginger paste
	Sautee the tilapia in canola oil.
	Then you sautee them in the oil from the tilapia, letting them steam. After a minute or so, add soy sauce and quar gum to thicken.
	While the aromatics cook, gently steam some bok choy.
	Plate the dinner, placing the aromatics on the tilapia.

Research on Exercise and Nutrition

Pre-Run

21st Century Genocide

Pandas Min Max

Assignment 2¶

Fascism

Fear

Chinese Tilapia

Sugar Thoughts

Will, Motivation

Comfort

Comfort

44th Marine Corps Marathon

2019 Training Journal

American Greatness

Communicate, Educate, Elevate