Introduction to numpy and matplotlib

Introduction to numpy and matplotlib#

Note

This material is mostly adapted from the following resources:

This lecture will introduce NumPy and Matplotlib. Numpy and Matplotlib are two of the most fundamental parts of the scientific python ecosystem. Most of everything else is built on top of them.

Numpy: The fundamental package for scientific computing with Python. NumPy is the standard Python library used for working with arrays (i.e., vectors & matrices), linear algebra, and other numerical computations.

Note

Documentation for this package is available at https://numpy.org/doc/stable/index.html.

Matplotlib: Matplotlib is a comprehensive library for creating static and animated visualizations in Python.

Note

Documentation for this package is available at https://matplotlib.org/stable/index.html.

Note

If you have not yet set up Python on your computer, you can execute this tutorial in your browser via Google Colab. Click on the rocket in the top right corner and launch “Colab”. If that doesn’t work download the .ipynb file and import it in Google Colab

Then install numpy and matplotlib by executing the following command in a Jupyter cell at the top of the notebook.

!pip install matplotlib numpy

Importing a Package#

This will be our first experience with importing a package.

Usually we import numpy with the alias np.

import numpy as np

NDArrays#

NDarrays (short for n-dimensional arrays) are a key data structure in numpy. NDarrays are similar to Python lists, but they allow for fast, efficient computations on large arrays and matrices of numerical data. NDarrays can have any number of dimensions, and are used for a wide range of numerical and scientific computing tasks, including linear algebra, statistical analysis, and image processing.

Thus, the main differences between a numpy array and a list are the following:

  • numpy arrays can have N dimensions (while lists only have 1)

  • numpy arrays hold values of the same datatype (e.g. int, float), while lists can contain anything.

  • numpy optimizes numerical operations on arrays. Numpy is fast!

# create an array from a list
a = np.array([9, 0, 2, 1, 0])

Note

If you’re in Jupyter, you can use <shift> + <tab> to inspect a function.

# find out the datatype
a.dtype

dtype('int64')

# find out the shape
a.shape

(5,)

# another array with a different datatype and shape
b = np.array([[5, 3, 1, 9], [9, 2, 3, 0]], dtype=np.float64)

# check dtype
b.dtype

dtype('float64')

# check shape
b.shape

(2, 4)

Array Creation#

There are lots of ways to create arrays.

np.zeros((4, 4))

array([[0., 0., 0., 0.],
       [0., 0., 0., 0.],
       [0., 0., 0., 0.],
       [0., 0., 0., 0.]])

np.ones((2, 2, 3))

array([[[1., 1., 1.],
        [1., 1., 1.]],

       [[1., 1., 1.],
        [1., 1., 1.]]])

np.full((3, 2), np.pi)

array([[3.14159265, 3.14159265],
       [3.14159265, 3.14159265],
       [3.14159265, 3.14159265]])

np.random.rand(5, 2)

array([[0.82131636, 0.59695767],
       [0.73446255, 0.54464715],
       [0.63324613, 0.57096081],
       [0.42692265, 0.85000261],
       [0.24299411, 0.36162389]])

np.arange(10)

array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

np.arange(2, 4, 0.25)

array([2.  , 2.25, 2.5 , 2.75, 3.  , 3.25, 3.5 , 3.75])

A frequent need is to generate an array of N numbers, evenly spaced between two values. That is what linspace is for.

np.linspace(2, 4, 20)

array([2.        , 2.10526316, 2.21052632, 2.31578947, 2.42105263,
       2.52631579, 2.63157895, 2.73684211, 2.84210526, 2.94736842,
       3.05263158, 3.15789474, 3.26315789, 3.36842105, 3.47368421,
       3.57894737, 3.68421053, 3.78947368, 3.89473684, 4.        ])

Numpy also has some utilities for helping us generate multi-dimensional arrays. For instance, meshgrid creates 2D arrays out of a combination of 1D arrays. It takes two arrays representing x and y values and creates two 2D grids: one for all the x coordinates and another for all the y coordinates, which can be used to evaluate functions on a complete grid covering the x-y plane.

x = np.linspace(-2 * np.pi, 2 * np.pi, 5)
y = np.linspace(-np.pi, np.pi, 4)
xx, yy = np.meshgrid(x, y)
xx.shape, yy.shape

((4, 5), (4, 5))

yy

array([[-3.14159265, -3.14159265, -3.14159265, -3.14159265, -3.14159265],
       [-1.04719755, -1.04719755, -1.04719755, -1.04719755, -1.04719755],
       [ 1.04719755,  1.04719755,  1.04719755,  1.04719755,  1.04719755],
       [ 3.14159265,  3.14159265,  3.14159265,  3.14159265,  3.14159265]])

xx

array([[-6.28318531, -3.14159265,  0.        ,  3.14159265,  6.28318531],
       [-6.28318531, -3.14159265,  0.        ,  3.14159265,  6.28318531],
       [-6.28318531, -3.14159265,  0.        ,  3.14159265,  6.28318531],
       [-6.28318531, -3.14159265,  0.        ,  3.14159265,  6.28318531]])

Indexing#

Basic indexing in numpy is similar to lists.

# get some individual elements of xx
xx[3, 4]

6.283185307179586

# get some whole rows
xx[0]

array([-6.28318531, -3.14159265,  0.        ,  3.14159265,  6.28318531])

# get some whole columns
xx[:, -1]

array([6.28318531, 6.28318531, 6.28318531, 6.28318531])

# get some ranges (also called slicing)
xx[0:2, 3:5]

array([[3.14159265, 6.28318531],
       [3.14159265, 6.28318531]])

Visualizing Arrays with Matplotlib#

Let’s create a slightly bigger array:

x = np.linspace(-2 * np.pi, 2 * np.pi, 100)
y = np.linspace(-np.pi, np.pi, 50)
xx, yy = np.meshgrid(x, y)
xx.shape, yy.shape

((50, 100), (50, 100))

For plotting a 1D array as a line, we use the plot command.

To use this function, we first need to import it from the matplotlib library.

from matplotlib import pyplot as plt

The line imports the visualization module pyplot from the matplotlib library and nicknames it as plt for brevity in the code.

plt.plot(x);
_images/59f37f71746a2f034d66257691efddeacbe2c60c4e72043581349e45b72c17dd.png

There are many ways to visualize 2D data. He we use pcolormesh.

plt.pcolormesh(xx);
_images/b735547287136418f6877bb19716e101e7886f81366e1ea5f2766f490013c340.png

Array Operations#

There is a huge number of operations available on arrays.

All the familiar arithemtic operators are applied on an element-by-element basis.

Basic Math#

f = np.sin(xx) * np.cos(0.5 * yy)

plt.pcolormesh(f)

<matplotlib.collections.QuadMesh at 0x7f986c3060d0>
_images/9a9c28f60d761ff2969689d770ace939066e805219cf1f548bd20a8d9a6c8231.png 
xx == yy

array([[False, False, False, ..., False, False, False],
       [False, False, False, ..., False, False, False],
       [False, False, False, ..., False, False, False],
       ...,
       [False, False, False, ..., False, False, False],
       [False, False, False, ..., False, False, False],
       [False, False, False, ..., False, False, False]])

np.any(xx == yy)

False

Broadcasting#

Not all the arrays we want to work with will have the same size!

Broadcasting is a powerful feature in numpy that allows you to perform operations on arrays of different shapes and sizes. It automatically expands the smaller array to match the dimensions of the larger array, without actually making copies of the data, so that element-wise operations can be performed. This is done by following a set of rules that determine how the shapes of the arrays align.

Broadcasting allows you to vectorize operations and avoid explicit loops, leading to more concise and efficient code. It’s particularly useful when working with large data sets, as it helps optimize memory usage and computational speed.

Dimensions are automatically aligned starting with the last dimension. If the last two dimensions have the same length, then the two arrays can be broadcast.

print(f.shape, x.shape)

(50, 100) (100,)

g = f * x

print(g.shape)

(50, 100)

plt.pcolormesh(g)

<matplotlib.collections.QuadMesh at 0x7f9847f35f50>
_images/ae7cdc8c06b520ea3522a36dfbe962ab8532c062d1003a7ff891de8a530fead6.png

Reduction Operations#

In data science, we usually start with a lot of data and want to reduce it down in order to make plots of summary tables.

There are many different reduction operations. The table below lists the most common functions:

Reduction Operation

Description

numpy.sum()

Computes the sum of array elements over a given axis.

numpy.mean()

Computes the arithmetic mean along a specified axis.

numpy.min()

Computes the minimum value along a specified axis.

numpy.max()

Computes the maximum value along a specified axis.

numpy.prod()

Computes the product of array elements over a given axis.

numpy.std()

Computes the standard deviation along a specified axis.

numpy.var()

Computes the variance along a specified axis.

 
# sum
g.sum()

-3083.038387807155

# mean
g.mean()

-0.616607677561431

# standard deviation
g.std()

1.6402280119141424

A key property of numpy reductions is the ability to operate on just one axis.

# apply on just one axis
g_ymean = g.mean(axis=0)
g_xmean = g.mean(axis=1)

plt.plot(x, g_ymean)

[<matplotlib.lines.Line2D at 0x7f9847f78550>]
_images/9a153da544f113ca7467ca1f4ab1195f8686681a71ad9f6c52b295f7ed001d55.png 
plt.plot(g_xmean, y)

[<matplotlib.lines.Line2D at 0x7f9845e07350>]
_images/1fded03b32e495737d509594c246a8f0c01bf79f19ad849acc95b266f4c9ff57.png

Figures and Axes#

The figure is the highest level of organization of matplotlib objects.

fig = plt.figure()

<Figure size 640x480 with 0 Axes>

fig = plt.figure(figsize=(13, 5))

<Figure size 1300x500 with 0 Axes>

fig = plt.figure()
ax = fig.add_axes([0, 0, 1, 1])
_images/08ecbafa3ab09bac3c2447803d9c77aa09f63d1658cf685672f85371574751df.png

Subplots#

Subplot syntax is a more convenient way to specify the creation of multiple axes.

fig, ax = plt.subplots()
_images/45816d171484af6f5b3de210bb3ae3d5c2ef9ecbcb6905647d8f0d99ce65e1a7.png 
ax

<Axes: >

fig, axes = plt.subplots(ncols=2, figsize=(8, 4), subplot_kw={"facecolor": "blue"})
_images/d13d7bcd4027f26608a4fa5b18749d5fb3b1bf1f7c2a5168a329fc7a8568e170.png 
axes

array([<Axes: >, <Axes: >], dtype=object)

Drawing into Axes#

All plots are drawn into axes.

# create some data to plot
import numpy as np

x = np.linspace(-np.pi, np.pi, 100)
y = np.cos(x)
z = np.sin(6 * x)

fig, ax = plt.subplots()
ax.plot(x, y)

[<matplotlib.lines.Line2D at 0x7f986cb703d0>]
_images/26123995c5a7e6fd324c4795873fb9408c4c05a2bfad9a6250e928685fabf7fe.png

This does the same thing as

plt.plot(x, y)

[<matplotlib.lines.Line2D at 0x7f9845c35a10>]
_images/26123995c5a7e6fd324c4795873fb9408c4c05a2bfad9a6250e928685fabf7fe.png

This starts to matter when we have multiple axes to manage.

fig, axes = plt.subplots(figsize=(8, 4), ncols=2)
ax0, ax1 = axes
ax0.plot(x, y)
ax1.plot(x, z)

[<matplotlib.lines.Line2D at 0x7f9845b0e390>]
_images/1f3d81c63cbac35e97b0a4646fb98dc43d1edd925b37da64ea7e97348b7a5894.png

Labeling Plots#

Labeling plots is very important! We want to know what data is shown and what the units are. matplotlib offers some functions to label graphics.

fig, ax = plt.subplots(figsize=(4, 4))

ax.plot(x, y)
ax.set_xlabel("x")
ax.set_ylabel("y")
ax.set_title("x vs. y")

# squeeze everything in
plt.tight_layout()
_images/013412ef4aec4968b9c53f539730f0c74d14b971781abd3f96f69b37e16b82bd.png

Customizing Plots#

fig, ax = plt.subplots()
ax.plot(x, y, x, z)

[<matplotlib.lines.Line2D at 0x7f9847f07390>,
 <matplotlib.lines.Line2D at 0x7f9845d93090>]
_images/53daff352457e204d31551780d6a2be0feeb84cf3ad3b32b2395e0a7169251e6.png

It’s simple to switch axes

fig, ax = plt.subplots()
ax.plot(y, x, z, x)

[<matplotlib.lines.Line2D at 0x7f9845bce690>,
 <matplotlib.lines.Line2D at 0x7f9845b2a990>]
_images/5d373d037cc35549347ed7313c4eaf2de195894c6be2724c8241c1540865740a.png

Line Styles#

fig, ax = plt.subplots()
ax.plot(x, y, linestyle="--")
ax.plot(x, z, linestyle=":")

[<matplotlib.lines.Line2D at 0x7f9845a6fa10>]
_images/8f149f7e2a01b1a9d45325bb7d841beaa0acc52bab96c365213e4a94df3196dd.png

Colors#

As described in the colors documentation, there are some special codes for commonly used colors.

fig, ax = plt.subplots()
ax.plot(x, y, color="black")
ax.plot(x, z, color="red")

[<matplotlib.lines.Line2D at 0x7f98448fa490>]
_images/fcf4117463dd9ddbbcabd403cc9440556ed34076aa818bb415c9c749f9805648.png

Markers#

There are lots of different markers availabile in matplotlib!

fig, ax = plt.subplots()
ax.plot(x[:20], y[:20], marker="o", markerfacecolor="red", markeredgecolor="black")
ax.plot(x[:20], z[:20], marker="^", markersize=10)

[<matplotlib.lines.Line2D at 0x7f9844972f50>]
_images/0ce16396bb75dff3356909be408359cd19cab1833a8d72006aa8a91dc6d3522a.png

Axis Limits#

fig, ax = plt.subplots()
ax.plot(x, y, x, z)
ax.set_xlim(-5, 5)
ax.set_ylim(-3, 3)

(-3.0, 3.0)
_images/184c83bd6283960b8346bfc92a84c4030d3b2276f3187013bfbbd8bac11b495f.png

Scatter Plots#

fig, ax = plt.subplots()

splot = ax.scatter(y, z, c=x, s=(100 * z**2 + 5), cmap="viridis")
fig.colorbar(splot)

<matplotlib.colorbar.Colorbar at 0x7f9845abded0>
_images/f367543495bcb4b2143627f1249b6f0c8cba1135ea5b1abe2ebb3c7382f32637.png

There are many different colormaps available in matplotlib: https://matplotlib.org/stable/tutorials/colors/colormaps.html

Bar Plots#

labels = ["Reuter West", "Mitte", "Lichterfelde"]
values = [600, 400, 450]

fig, ax = plt.subplots(figsize=(5, 5))
ax.bar(labels, values)

ax.set_ylabel("MW")

plt.tight_layout()
_images/671f0ca595f102ab30b66ce7c4d6229e00352b2998107d2da4c75256444f57d0.png

Exercises#

Import numpy under the alias np.

Hide code cell content 
import numpy as np

Create the following arrays:

  1. Create an array of 5 zeros.

  2. Create an array of 10 ones.

  3. Create an array of 5 \(\pi\) values.

  4. Create an array of the integers 1 to 20.

  5. Create a 5 x 5 matrix of ones with a dtype int.

Hide code cell content 
np.zeros(5)

array([0., 0., 0., 0., 0.])
Hide code cell content 
np.ones(10)

array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])
Hide code cell content 
np.full(5, np.pi)

array([3.14159265, 3.14159265, 3.14159265, 3.14159265, 3.14159265])
Hide code cell content 
np.arange(1, 21)

array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
       18, 19, 20])
Hide code cell content 
np.ones((5, 5), dtype=np.int8)

array([[1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1],
       [1, 1, 1, 1, 1]], dtype=int8)

Create a 3D matrix of 3 x 3 x 3 full of random numbers drawn from a standard normal distribution (hint: np.random.randn())

Hide code cell content 
np.random.randn(3, 3, 3)

array([[[-0.91365383, -1.32954173, -0.70789895],
        [ 0.2806742 ,  0.38628899, -0.38847005],
        [-0.13454887, -1.72060064, -1.40172658]],

       [[ 0.47753016,  0.23655442,  0.30673732],
        [-1.1715514 , -1.37647537,  0.51464247],
        [-1.1680009 , -1.25798903, -1.50993686]],

       [[ 1.47934732, -0.48221132,  0.63640246],
        [ 0.56549916, -0.71877611,  0.92913435],
        [ 1.18279236,  1.33350704,  0.00275824]]])

Create an array of 20 linearly spaced numbers between 1 and 10.

Hide code cell content 
np.linspace(1, 10, 20)

array([ 1.        ,  1.47368421,  1.94736842,  2.42105263,  2.89473684,
        3.36842105,  3.84210526,  4.31578947,  4.78947368,  5.26315789,
        5.73684211,  6.21052632,  6.68421053,  7.15789474,  7.63157895,
        8.10526316,  8.57894737,  9.05263158,  9.52631579, 10.        ])

Below I’ve defined an array of shape 4 x 4. Use indexing to procude the given outputs.

Hide code cell content 
a = np.arange(1, 26).reshape(5, -1)
a

array([[ 1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10],
       [11, 12, 13, 14, 15],
       [16, 17, 18, 19, 20],
       [21, 22, 23, 24, 25]])
Hide code cell content 
a[1:, 3:]

array([[ 9, 10],
       [14, 15],
       [19, 20],
       [24, 25]])

array([[ 9, 10],
       [14, 15],
       [19, 20],
       [24, 25]])
Hide code cell content 
a[1]

array([ 6,  7,  8,  9, 10])

array([ 6,  7,  8,  9, 10])
Hide code cell content 
a[2:4]

array([[11, 12, 13, 14, 15],
       [16, 17, 18, 19, 20]])

array([[11, 12, 13, 14, 15],
       [16, 17, 18, 19, 20]])
Hide code cell content 
a[1:3, 2:4]

array([[ 8,  9],
       [13, 14]])

array([[ 8,  9],
       [13, 14]])

Calculate the sum of all the numbers in a.

Hide code cell content 
a.sum()

325

Calculate the sum of each row in a.

Hide code cell content 
a.sum(axis=1)

array([ 15,  40,  65,  90, 115])
Hide code cell content 
a.sum(axis=0)

array([55, 60, 65, 70, 75])

Extract all values of a greater than the mean of a (hint: use a boolean mask).

Hide code cell content 
a[a > a.mean()]

array([14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25])
Contents