Category Archives: Data Analysis

Collecting Energy Price Data

I’m not sure why exactly, but I am particularly interested in fluctuating energy costs, particularly the costs associated with putting gas in my car.  I remember that Regular Unleaded Gas (or Petrol for you Europeans) was $.99 a gallon when I started driving around 1985 in the arid South West of San Antonio, TX.  Of course gas is more expensive today, but I’m often surprised how relatively cheap gas prices remain for a gallon of Regular Unleaded.  I feel fairly confident that it’s just a matter of time before all energy costs, Regular Unleaded Gas not excluded, will rapidly increase.  Energy is a limited resource yet the global population continues to grow.

Anyway, I wrote a python script (adapted from a Perl script I wrote to do the same a few years ago) to grab the current National Average Price for Regular Unleaded Gas.  My script runs automatically each morning to collect the daily price of Regular Unleaded Gas and dumps it into a MySQL database I have running on AWS.

Here’s a graph snapshot of the data I’ve collected so far (click to enlarge).

Graph the GasHere’s the table of the data I’ve collected so far:

+—-+———————+————+——–+——+——-+——+
| id | rec_create_dt | price_date | price | year | month | day |
+—-+———————+————+——–+——+——-+——+
| 1 | 2016-07-25 01:53:08 | 2016-07-24 | $2.165 | 2016 | 7 | 24 |
| 2 | 2016-07-25 09:46:16 | 2016-07-25 | $2.161 | 2016 | 7 | 25 |
| 7 | 2016-07-26 11:11:50 | 2016-07-26 | $2.154 | 2016 | 7 | 26 |
| 8 | 2016-07-27 06:00:16 | 2016-07-27 | $2.154 | 2016 | 7 | 27 |
| 9 | 2016-07-28 06:00:22 | 2016-07-28 | $2.148 | 2016 | 7 | 28 |
| 10 | 2016-07-29 06:00:21 | 2016-07-29 | $2.142 | 2016 | 7 | 29 |
| 11 | 2016-07-30 06:00:22 | 2016-07-30 | $2.139 | 2016 | 7 | 30 |
| 12 | 2016-07-31 09:30:22 | 2016-07-31 | $2.135 | 2016 | 7 | 31 |
| 13 | 2016-08-01 09:30:22 | 2016-08-01 | $2.132 | 2016 | 8 | 1 |
| 14 | 2016-08-02 09:30:23 | 2016-08-02 | $2.126 | 2016 | 8 | 2 |
| 15 | 2016-08-03 09:30:23 | 2016-08-03 | $2.120 | 2016 | 8 | 3 |
| 16 | 2016-08-04 09:30:23 | 2016-08-04 | $2.116 | 2016 | 8 | 4 |
| 17 | 2016-08-05 09:30:23 | 2016-08-05 | $2.120 | 2016 | 8 | 5 |
| 18 | 2016-08-06 09:30:23 | 2016-08-06 | $2.124 | 2016 | 8 | 6 |
| 19 | 2016-08-07 09:30:24 | 2016-08-07 | $2.123 | 2016 | 8 | 7 |
| 20 | 2016-08-08 09:30:23 | 2016-08-08 | $2.123 | 2016 | 8 | 8 |
| 21 | 2016-08-09 09:30:24 | 2016-08-09 | $2.124 | 2016 | 8 | 9 |
| 22 | 2016-08-10 09:30:24 | 2016-08-10 | $2.127 | 2016 | 8 | 10 |
| 23 | 2016-08-11 09:30:25 | 2016-08-11 | $2.130 | 2016 | 8 | 11 |
| 24 | 2016-08-12 09:30:25 | 2016-08-12 | $2.129 | 2016 | 8 | 12 |
| 25 | 2016-08-13 09:30:21 | 2016-08-13 | $2.127 | 2016 | 8 | 13 |
| 26 | 2016-08-14 09:30:26 | 2016-08-14 | $2.125 | 2016 | 8 | 14 |
| 27 | 2016-08-15 09:30:26 | 2016-08-15 | $2.124 | 2016 | 8 | 15 |
| 28 | 2016-08-16 09:30:26 | 2016-08-16 | $2.125 | 2016 | 8 | 16 |
| 29 | 2016-08-17 09:30:27 | 2016-08-17 | $2.132 | 2016 | 8 | 17 |
| 30 | 2016-08-18 09:30:26 | 2016-08-18 | $2.135 | 2016 | 8 | 18 |
| 31 | 2016-08-19 09:30:27 | 2016-08-19 | $2.141 | 2016 | 8 | 19 |
| 32 | 2016-08-20 09:30:23 | 2016-08-20 | $2.152 | 2016 | 8 | 20 |
| 33 | 2016-08-21 09:30:28 | 2016-08-21 | $2.158 | 2016 | 8 | 21 |
+—-+———————+————+——–+——+——-+——+