In this post we are going to talk about how to quickly and easily analyze data with Python. Specifically this will show you how to import and export data from CSV’s (comma seperated variables) into Python. Note that this post is a work in progress and information will continue to be updated.

Click here to view the presentation slides. They have some useful links to documentation that can help you in your Python endeavours!

Basic Python

Here is a 5 minute refresher on some basic Python syntax and conventions.

a = 1
b = 2

a + b

3

a - b

-1

a * b

2

b / a

2.0

b % a

0

a = 2
b = 3

b**a

9

Lets try some harder things!

for i in range(0, 10, 1):
    print(i)

0
1
2
3
4
5
6
7
8
9

NOTE THAT THE LOOP DOES NOT PRINT 10

Importing Libraries

Some functionality isnt supported by Python by default. Common libraries include:

• os
• pathlib
• math
• Numpy
• Pandas
• Scipy
• Matplotlib
• Sympy
• Thermo

Read more about Python internal libraries here

You can import an entire library under an shortened alias. The following are the typical conventions:

import numpy as np
import pandas as pd
import matplotlib as plt

Lists

Say I have a list of items in Python. The following are helpful indexing conventions. Read more about lists here

sample_list = [1, 2, 3, 4, 5]

sample_list[0]

1

sample_list[-1]

5

sample_list.pop(0)

1

sample_list

[2, 3, 4, 5]

for item in sample_list:
    print("The item {} multiplied by 2 is {}".format(item, item*2))

The item 2 multiplied by 2 is 4
The item 3 multiplied by 2 is 6
The item 4 multiplied by 2 is 8
The item 5 multiplied by 2 is 10

sample_list.append(0)
print(sample_list)

[2, 3, 4, 5, 0, 0, 0]

Try not to do this with big data. There are better functions for iterating over large datasets.

Dictionaries

You can store lists, arrays, and data in dictionaries and get them with keys

sample_dict = {
    'small': 1,
    'medium': 2.5,
    'large':5
}

sample_dict

{‘small’: 1, ‘medium’: 2.5, ‘large’: 5}

sample_dict = {
    'small': sample_list,
    'large': sample_list*2
}

sample_dict

{‘small’: [2, 3, 4, 5], ‘large’: [2, 3, 4, 5, 2, 3, 4, 5]}

Functions

If you have a commonly repeated block of code, make it a function!

def do_a_thing():
    print("Do a thing!")

do_a_thing()

Do a thing!

def celsius_to_fahrenheit(celsius):
    fahrenheit = 9/5*celsius + 32
    return fahrenheit

celsius_to_fahrenheit(21)

69.80000000000001

celsius_to_fahrenheit(23)

73.4

for item in sample_list:
    print(celsius_to_fahrenheit(item))

35.6
37.4
39.2
41.0

Try This

Take 2 lists of data of the following:

list_1 = [21, 23, 65, 23, 65, 12]
list_2 = [34, 12, 54, 54, 12, 54]

Create a THIRD list where the contents of this list is the results of list_1 divide by list_2

What is Numpy?

Numpy is a library that basically turns Python into a free version of Matlab.

Step 1: Importing libraries

import numpy as np

This imports all of numpy as a library and accessing any aspect of Numpy can be done by:

np.array([0, 1, 2, 3])

array([0, 1, 2, 3])

np.linspace(0, 1, 101)

array([ 0. , 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08,
0.09, 0.1 , 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17,
0.18, 0.19, 0.2 , 0.21, 0.22, 0.23, 0.24, 0.25, 0.26,
0.27, 0.28, 0.29, 0.3 , 0.31, 0.32, 0.33, 0.34, 0.35,
0.36, 0.37, 0.38, 0.39, 0.4 , 0.41, 0.42, 0.43, 0.44,
0.45, 0.46, 0.47, 0.48, 0.49, 0.5 , 0.51, 0.52, 0.53,
0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.6 , 0.61, 0.62,
0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.7 , 0.71,
0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79, 0.8 ,
0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87, 0.88, 0.89,
0.9 , 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98,
0.99, 1. ])

What is Pandas??

Pandas is a library that is going to solve all your engineering problems.

pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python. Additionally, it has the broader goal of becoming the most powerful and flexible open source data analysis / manipulation tool available in any language. It is already well on its way toward this goal.

import pandas as pd

data = pd.read_csv('HW1-2 Data.csv')

data.describe()

Lets make these columns easier to use!

data.rename(index=str, columns={"Volume [mm^3]]":"volume",
                                "Mass [mg]": "mass",
                                "Feret Diameter 1 [mm]":"feret_diam_1",
                                "Feret Diameter 2 [mm]":"feret_diam_2"
                               }
           )

data['volume'].mean()

0.029487654130769765

data['mass'].mean()

0.20721328766934172

data['mass'].quantile(.65)

0.05503852664868761

import numpy as np
data = data.apply(np.sqrt)

Challenge time

Convert the units of each column into units without prefixes (ie: metres, meters cubed)

data.assign(density = lambda x: x['mass']/x['volume'])

import matplotlib.pyplot as plt
plt.style.use('ggplot')

plt.figure()
data.plot.hist(alpha=1, stacked=True, bins=20)
plt.show()

data['volume'].plot.hist(bins=10)
plt.show()

data.plot.box()
plt.show()

data['volume'].plot.box()
plt.show()

There are MANY tools I did not cover in this tutorial but this should show the basic building blocks of data analysis with Python. Here is an extensive cheat sheet from Hitesh Jethva of PCWDLD to aid you in the learning process. Feel free to comment below in case of any questions!