Introduction to pypsa

Introduction to `pypsa`#

PyPSA stands for Python for Power System Analysis.

PyPSA is an open source Python package for simulating and optimising modern energy systems that include features such as

conventional generators with unit commitment (ramp-up, ramp-down, start-up, shut-down),
time-varying wind and solar generation,
energy storage with efficiency losses and inflow/spillage for hydroelectricity
coupling to other energy sectors (electricity, transport, heat, industry),
conversion between energy carriers (e.g. electricity to hydrogen),
transmission networks (AC, DC, other fuels)

PyPSA can be used for a variety of problem types (e.g. electricity market modelling, long-term investment planning, transmission network expansion planning), and is designed to scale well with large networks and long time series.

Compared to building power system by hand in pyomo, PyPSA does the following things for you:

manage data inputs
build optimisation problem
communicate with the solver
retrieve and process optimisation results
manage data outputs

Dependencies#

pandas for storing data about network components and time series
numpy and scipy for linear algebra and sparse matrix calculations
matplotlib and cartopy for plotting on a map
networkx for network calculations
pyomo for handling optimisation problems

Note

Documentation for this package is available at https://pypsa.readthedocs.io.

Note

If you have not yet set up Python on your computer, you can execute this tutorial in your browser via Google Colab. Click on the rocket in the top right corner and launch “Colab”. If that doesn’t work download the .ipynb file and import it in Google Colab.

Then install the following packages by executing the following command in a Jupyter cell at the top of the notebook.

!pip install pypsa matplotlib cartopy highspy

Basic Structure#

Component	Description
Network	Container for all components.
Bus	Node where components attach.
Carrier	Energy carrier or technology (e.g. electricity, hydrogen, gas, coal, oil, biomass, on-/offshore wind, solar). Can track properties such as specific carbon dioxide emissions or nice names and colors for plots.
Load	Energy consumer (e.g. electricity demand).
Generator	Generator (e.g. power plant, wind turbine, PV panel).
Line	Power distribution and transmission lines (overhead and cables).
Link	Links connect two buses with controllable energy flow, direction-control and losses. They can be used to model: HVDC links HVAC lines (neglecting KVL, only net transfer capacities (NTCs)) conversion between carriers (e.g. electricity to hydrogen in electrolysis)
StorageUnit	Storage with fixed nominal energy-to-power ratio.
GlobalConstraint	Constraints affecting many components at once, such as emission limits.
Store	Storage with separately extendable energy capacity.
	not used in this course
LineType	Standard line types.
Transformer	2-winding transformer.
TransformerType	Standard types of 2-winding transformer.
ShuntImpedance	Shunt.

Note

Links in the table lead to documentation for each component.

Warning

Per unit values of voltage and impedance are used internally for network calculations. It is assumed internally that the base power is 1 MW.

From structured data to optimisation#

The design principle of PyPSA is that basically each component is associated with a set of variables and constraints that will be added to the optimisation model based on the input data stored for the components.

For an hourly electricity market simulation, PyPSA will solve an optimisation problem that looks like this

(8)#\[\begin{equation} \min_{g_{i,s,t}; f_{\ell,t}; g_{i,r,t,\text{charge}}; g_{i,r,t,\text{discharge}}; e_{i,r,t}} \sum_s o_{s} g_{i,s,t} \end{equation}\]

such that

(9)#\[\begin{align} 0 & \leq g_{i,s,t} \leq \hat{g}_{i,s,t} G_{i,s} & \text{generation limits : generator} \\ -F_\ell &\leq f_{\ell,t} \leq F_\ell & \text{transmission limits : line} \\ d_{i,t} &= \sum_s g_{i,s,t} + \sum_r g_{i,r,t,\text{discharge}} - \sum_r g_{i,r,t,\text{charge}} - \sum_\ell K_{i\ell} f_{\ell,t} & \text{KCL : bus} \\ 0 &=\sum_\ell C_{\ell c} x_\ell f_{\ell,t} & \text{KVL : cycles} \\ 0 & \leq g_{i,r,t,\text{discharge}} \leq G_{i,r,\text{discharge}}& \text{discharge limits : storage unit} \\ 0 & \leq g_{i,r,t,\text{charge}} \leq G_{i,r,\text{charge}} & \text{charge limits : storage unit} \\ 0 & \leq e_{i,r,t} \leq E_{i,r} & \text{energy limits : storage unit} \\ e_{i,r,t} &= \eta^0_{i,r,t} e_{i,r,t-1} + \eta^1_{i,r,t}g_{i,r,t,\text{charge}} - \frac{1}{\eta^2_{i,r,t}} g_{i,r,t,\text{discharge}} & \text{consistency : storage unit} \\ e_{i,r,0} & = e_{i,r,|T|-1} & \text{cyclicity : storage unit} \end{align}\]

Decision variables:

\(g_{i,s,t}\) is the generator dispatch at bus \(i\), technology \(s\), time step \(t\),
\(f_{\ell,t}\) is the power flow in line \(\ell\),
\(g_{i,r,t,\text{dis-/charge}}\) denotes the charge and discharge of storage unit \(r\) at bus \(i\) and time step \(t\),
\(e_{i,r,t}\) is the state of charge of storage \(r\) at bus \(i\) and time step \(t\).

Parameters:

\(o_{i,s}\) is the marginal generation cost of technology \(s\) at bus \(i\),
\(x_\ell\) is the reactance of transmission line \(\ell\),
\(K_{i\ell}\) is the incidence matrix,
\(C_{\ell c}\) is the cycle matrix,
\(G_{i,s}\) is the nominal capacity of the generator of technology \(s\) at bus \(i\),
\(F_{\ell}\) is the rating of the transmission line \(\ell\),
\(E_{i,r}\) is the energy capacity of storage \(r\) at bus \(i\),
\(\eta^{0/1/2}_{i,r,t}\) denote the standing (0), charging (1), and discharging (2) efficiencies.

Note

For a full reference to the optimisation problem description, see https://pypsa.readthedocs.io/en/latest/optimal_power_flow.html

Simple electricity market example#

Let’s get acquainted with PyPSA to build a variant of one of the simple electricity market models we previously built in pyomo. Hopefully, it can convince you that it will be easier to work with PyPSA than with pyomo.

We have the following data:

fuel costs in € / MWh\(_{th}\)

fuel_cost = dict(
    coal=8,
    gas=100,
    oil=48,
)

efficiencies of thermal power plants in MWh\(_{el}\) / MWh\(_{th}\)

efficiency = dict(
    coal=0.33,
    gas=0.58,
    oil=0.35,
)

specific emissions in t\(_{CO_2}\) / MWh\(_{th}\)

# t/MWh thermal
emissions = dict(
    coal=0.34,
    gas=0.2,
    oil=0.26,
    hydro=0,
)

power plant capacities in MW

power_plants = {
    "SA": {"coal": 35000, "wind": 3000, "gas": 8000, "oil": 2000},
    "MZ": {"hydro": 1200},
}

electrical load in MW

loads = {
    "SA": 42000,
    "MZ": 650,
}

Building a basic network#

By convention, PyPSA is imported without an alias:

import pypsa

First, we create a new network object which serves as the overall container for all components.

n = pypsa.Network()

The second component we need are buses. Buses are the fundamental nodes of the network, to which all other components like loads, generators and transmission lines attach. They enforce energy conservation for all elements feeding in and out of it (i.e. Kirchhoff’s Current Law).

Components can be added to the network n using the n.add() function. It takes the component name as a first argument, the name of the component as a second argument and possibly further parameters as keyword arguments. Let’s use this function, to add buses for each country to our network:

n.add("Bus", "SA", y=-30.5, x=25, v_nom=400, carrier="AC")
n.add("Bus", "MZ", y=-18.5, x=35.5, v_nom=400, carrier="AC")

For each class of components, the data describing the components is stored in a pandas.DataFrame. For example, all static data for buses is stored in n.buses

n.buses

attribute	v_nom	x	y	carrier	v_mag_pu_set	v_mag_pu_min	v_mag_pu_max	control
Bus
SA	400.0	25.0	-30.5	AC	1.0	0.0	inf	PQ
MZ	400.0	35.5	-18.5	AC	1.0	0.0	inf	PQ

You see there are many more attributes than we specified while adding the buses; many of them are filled with default parameters which were added. You can look up the field description, defaults and status (required input, optional input, output) for buses here https://pypsa.readthedocs.io/en/latest/components.html#bus, and analogous for all other components.

There’s a variant of n.add() called n.madd() which allows you to add multiple components at once. For instance, multiple carriers for the fuels with information on specific carbon dioxide emissions, a nice name, and colors for plotting.

The function n.madd() again takes the component name as the first argument and then a list of component names and then optional arguments for the parameters. Here, scalar values, lists, dictionary or pandas.Series are allowed. The latter two needs keys or indices with the component names.

emissions

{'coal': 0.34, 'gas': 0.2, 'oil': 0.26, 'hydro': 0}

n.madd(
    "Carrier",
    ["coal", "gas", "oil", "hydro", "wind"],
    co2_emissions=emissions,
    nice_name=["Coal", "Gas", "Oil", "Hydro", "Onshore Wind"],
    color=["grey", "indianred", "black", "aquamarine", "dodgerblue"],
)

Index(['coal', 'gas', 'oil', 'hydro', 'wind'], dtype='object')

The n.add() function is very general. It lets you add any component to the network object n. For instance, in the next step we add generators for all the different power plants.

In Mozambique:

n.add(
    "Generator",
    "MZ hydro",
    bus="MZ",
    carrier="hydro",
    p_nom=1200,  # MW
    marginal_cost=0,  # default
)

In South Africa (in a loop):

for tech, p_nom in power_plants["SA"].items():
    n.add(
        "Generator",
        f"SA {tech}",
        bus="SA",
        carrier=tech,
        efficiency=efficiency.get(tech, 1),
        p_nom=p_nom,
        marginal_cost=fuel_cost.get(tech, 0) / efficiency.get(tech, 1),
    )

As a result, the n.generators DataFrame looks like this:

n.generators

attribute	bus	control	p_nom	p_nom_mod	p_nom_extendable	p_nom_min	p_nom_max	p_min_pu	p_max_pu	...	min_up_time	min_down_time	up_time_before	down_time_before	ramp_limit_up	ramp_limit_down	ramp_limit_start_up	ramp_limit_shut_down	weight	p_nom_opt
Generator
MZ hydro	MZ	PQ	1200.0	0.0	False	0.0	inf	0.0	1.0	...	0	0	1	0	NaN	NaN	1.0	1.0	1.0	0.0
SA coal	SA	PQ	35000.0	0.0	False	0.0	inf	0.0	1.0	...	0	0	1	0	NaN	NaN	1.0	1.0	1.0	0.0
SA wind	SA	PQ	3000.0	0.0	False	0.0	inf	0.0	1.0	...	0	0	1	0	NaN	NaN	1.0	1.0	1.0	0.0
SA gas	SA	PQ	8000.0	0.0	False	0.0	inf	0.0	1.0	...	0	0	1	0	NaN	NaN	1.0	1.0	1.0	0.0
SA oil	SA	PQ	2000.0	0.0	False	0.0	inf	0.0	1.0	...	0	0	1	0	NaN	NaN	1.0	1.0	1.0	0.0

5 rows × 34 columns

Next, we’re going to add the electricity demand.

A positive value for p_set means consumption of power from the bus.

n.add(
    "Load",
    "SA electricity demand",
    bus="SA",
    p_set=loads["SA"],
    carrier="electricity",
)

n.add(
    "Load",
    "MZ electricity demand",
    bus="MZ",
    p_set=loads["MZ"],
    carrier="electricity",
)

n.loads

attribute	bus	carrier	p_set	q_set	sign
Load
SA electricity demand	SA	electricity	42000.0	0.0	-1.0
MZ electricity demand	MZ	electricity	650.0	0.0	-1.0

Finally, we add the connection between Mozambique and South Africa with a 500 MW line:

n.add(
    "Line",
    "SA-MZ",
    bus0="SA",
    bus1="MZ",
    s_nom=500,
    x=1,
    r=1,
)

n.lines

attribute	bus0	bus1	type	x	r	g	b	s_nom	s_nom_mod	s_nom_extendable	...	v_ang_min	v_ang_max	sub_network	x_pu	r_pu	g_pu	b_pu	x_pu_eff	r_pu_eff	s_nom_opt
Line
SA-MZ	SA	MZ		1.0	1.0	0.0	0.0	500.0	0.0	False	...	-inf	inf		0.0	0.0	0.0	0.0	0.0	0.0	0.0

1 rows × 30 columns

We can have a sneak peek at the network we built with the n.plot() function. More details on this in a bit.

n.plot(bus_sizes=1, margin=1);

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

_images/2532539e329ef939241750e04e586610fe63bbdc3ad58741e219bcac03b35067.png

Optimisation#

With all input data transferred into PyPSA’s data structure, we can now build and run the resulting optimisation problem. In PyPSA, building, solving and retrieving results from the optimisation model is contained in a single function call n.optimize(). This function optimizes dispatch and investment decisions for least cost.

The n.optimize() function can take a variety of arguments. The most relevant for the moment is the choice of the solver. We already know the different solvers from the introduction to pyomo (e.g. “cbc”, “glpk”, “gurobi” etc.). They need to be installed on your computer, to use them here!

n.optimize(solver_name="highs")

Let’s have a look at the results.

Since the power flow and dispatch are generally time-varying quantities, these are stored in a different location than e.g. n.generators. They are stored in n.generators_t. Thus, to find out the dispatch of the generators, run

n.generators_t.p

Generator	MZ hydro	SA coal	SA wind	SA gas	SA oil
snapshot
now	1150.0	35000.0	3000.0	1500.0	2000.0

or if you prefer it in relation to the generators nominal capacity

n.generators_t.p / n.generators.p_nom

Generator	MZ hydro	SA coal	SA wind	SA gas	SA oil
snapshot
now	0.958333	1.0	1.0	0.1875	1.0

You see that the time index has the value ‘now’. This is the default index when no time series data has been specified and the network only covers a single state (e.g. a particular hour).

Similarly you will find the power flow in transmission lines at

n.lines_t.p0

Line	SA-MZ
snapshot
now	-500.0

n.lines_t.p1

Line	SA-MZ
snapshot
now	500.0

The p0 will tell you the flow from bus0 to bus1. p1 will tell you the flow from bus1 to bus0.

What about the shadow prices?

n.buses_t.marginal_price

Bus	SA	MZ
snapshot
now	172.413793	-0.0

Basic network plotting#

For plotting PyPSA network, we’re going to need the help of some old friends:

import matplotlib.pyplot as plt
import cartopy.crs as ccrs

PyPSA has a built-in plotting function based on matplotlib, ….

n.plot(margin=1, bus_sizes=2)

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

(<matplotlib.collections.PatchCollection at 0x7f017d7d0090>,
 <matplotlib.collections.LineCollection at 0x7f017d7ddb90>)

_images/7d532d8cbfbee8c0f8a7d34b0971231ad6591f5380d90b85140658c5098dfa6d.png

Since we have provided x and y coordinates for our buses, n.plot() will try to plot the network on a map by default. Of course, there’s an option to deactivate this behaviour:

n.plot(geomap=False);

_images/3bf38f2ddc3ebeba67e650c67e5e73ecbf77aceaf7bdf5b992b4eefef3902ece.png

The n.plot() function has a variety of styling arguments to tweak the appearance of the buses, the lines and the map in the background:

n.plot(
    margin=1,
    bus_sizes=2,
    bus_colors="orange",
    bus_alpha=0.7,
    color_geomap=True,
    line_colors="orchid",
    line_widths=3,
    title="Test",
);

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

_images/d4493529ec3350e5f69daf194e89158675ef852ca28da70d9de543e7a3f90da2.png

Just like with geopandas we can also control the projection of the network plot:

fig = plt.figure(figsize=(5, 5))
ax = plt.axes(projection=ccrs.EqualEarth())

n.plot(ax=ax, margin=1, bus_sizes=2);

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

_images/a09bbc400da69c5eeeac5832c5654dd3bfff43c0465cee86f2b5f2013b1f6420.png

We can use the bus_sizes argument of n.plot() to display the regional distribution of load. First, we calculate the total load per bus:

s = n.loads.groupby("bus").p_set.sum() / 1e4

bus
MZ    0.065
SA    4.200
Name: p_set, dtype: float64

The resulting pandas.Series we can pass to n.plot(bus_sizes=...):

n.plot(margin=1, bus_sizes=s);

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

_images/7cfa59fd3011f43492a4a54768ce5258fece3a73517b974c7715b52cafe389d6.png

The important point here is, that s needs to have entries for all buses, i.e. its index needs to match n.buses.index.

The bus_sizes argument of n.plot() can be even more powerful. It can produce pie charts, e.g. for the mix of electricity generation at each bus.

The dispatch of each generator, we can find at:

n.generators_t.p.loc["now"]

Generator
MZ hydro     1150.0
SA coal     35000.0
SA wind      3000.0
SA gas       1500.0
SA oil       2000.0
Name: now, dtype: float64

If we group this by the bus and carrier…

s = n.generators_t.p.loc["now"].groupby([n.generators.bus, n.generators.carrier]).sum()

… we get a multi-indexed pandas.Series …

bus  carrier
MZ   hydro       1150.0
SA   coal       35000.0
     gas         1500.0
     oil         2000.0
     wind        3000.0
Name: now, dtype: float64

… which we can pass to n.plot(bus_sizes=...):

n.plot(margin=1, bus_sizes=s / 3000);

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

_images/77e4fadf3f1d723df382b221295e7eef29599af8eb0aa34acea19864135d8859.png

How does this magic work? The plotting function will look up the colors specified in n.carriers for each carrier and match it with the second index-level of s.

Modifying networks#

Modifying data of components in an existing PyPSA network is as easy as modifying the entries of a pandas.DataFrame. For instance, if we want to reduce the cross-border transmission capacity between South Africa and Mozambique, we’d run:

n.lines.loc["SA-MZ", "s_nom"] = 400

n.lines

attribute	bus0	bus1	type	x	r	g	b	s_nom	s_nom_mod	s_nom_extendable	...	v_ang_max	sub_network	x_pu	r_pu	g_pu	b_pu	x_pu_eff	r_pu_eff	s_nom_opt	v_nom
Line
SA-MZ	SA	MZ		1.0	1.0	0.0	0.0	400.0	0.0	False	...	inf	0	0.000006	0.000006	0.0	0.0	0.000006	0.000006	500.0	400.0

1 rows × 31 columns

n.optimize(solver_name="highs")

You can see that the production of the hydro power plant was reduced and that of the gas power plant increased owing to the reduced transmission capacity.

n.generators_t.p

Generator	MZ hydro	SA coal	SA wind	SA gas	SA oil
snapshot
now	1050.0	35000.0	3000.0	1600.0	2000.0

Global constraints for emission limits#

In the example above, we happen to have some spare gas capacity with lower carbon intensity than the coal and oil generators. We could use this to lower the emissions of the system, but it will be more expensive. We can implement the limit of carbon dioxide emissions as a constraint.

This is achieved in PyPSA through Global Constraints which add constraints that apply to many components at once.

But first, we need to calculate the current level of emissions to set a sensible limit.

We can compute the emissions per generator (in tonnes of CO\(_2\)) in the following way.

\[\frac{g_{i,s,t} \cdot \rho_{i,s}}{\eta_{i,s}}\]

where \( \rho\) is the specific emissions (tonnes/MWh thermal) and \(\eta\) is the conversion efficiency (MWh electric / MWh thermal) of the generator with dispatch \(g\) (MWh electric):

e = (
    n.generators_t.p
    / n.generators.efficiency
    * n.generators.carrier.map(n.carriers.co2_emissions)
)
e

Generator	MZ hydro	SA coal	SA wind	SA gas	SA oil
snapshot
now	0.0	36060.606061	NaN	551.724138	1485.714286

Summed up, we get total emissions in tonnes:

e.sum().sum()

38098.044484251375

So, let’s say we want to reduce emissions by 10%:

n.add(
    "GlobalConstraint",
    "emission_limit",
    carrier_attribute="co2_emissions",
    sense="<=",
    constant=e.sum().sum() * 0.9,
)

n.optimize(solver_name="highs")

n.generators_t.p

Generator	MZ hydro	SA coal	SA wind	SA gas	SA oil
snapshot
now	1050.0	30603.423345	3000.0	7996.576655	-0.0

n.generators_t.p / n.generators.p_nom

Generator	MZ hydro	SA coal	SA wind	SA gas	SA oil
snapshot
now	0.875	0.874384	1.0	0.999572	-0.0

n.global_constraints.mu

GlobalConstraint
emission_limit   -216.158537
Name: mu, dtype: float64

Can we lower emissions even further? Say by another 5% points?

n.global_constraints.loc["emission_limit", "constant"] = 0.85

n.optimize(solver_name="highs")

No! Without any additional capacities, we have exhausted our options to reduce CO2 in that hour. The optimiser tells us that the problem is infeasible.

Data import and export#

Note

Documentation: https://pypsa.readthedocs.io/en/latest/import_export.html.

You may find yourself in a need to store PyPSA networks for later use. Or, maybe you want to import the genius PyPSA example that someone else uploaded to the web to explore.

PyPSA can be stored as netCDF (.nc) file or as a folder of CSV files.

netCDF files have the advantage that they take up less space than CSV files and are faster to load.
CSV might be easier to use with Excel.

n.export_to_csv_folder("tmp")

WARNING:pypsa.io:Directory tmp does not exist, creating it

INFO:pypsa.io:Exported network tmp has lines, loads, carriers, buses, generators, global_constraints

n_csv = pypsa.Network("tmp")

INFO:pypsa.io:Imported network tmp has buses, carriers, generators, global_constraints, lines, loads

n.export_to_netcdf("tmp.nc");

INFO:pypsa.io:Exported network tmp.nc has lines, loads, carriers, buses, generators, global_constraints

n_nc = pypsa.Network("tmp.nc")

INFO:pypsa.io:Imported network tmp.nc has buses, carriers, generators, global_constraints, lines, loads

A slightly more realistic example#

Dispatch problem with German SciGRID network

SciGRID is a project that provides an open reference model of the European transmission network. The network comprises time series for loads and the availability of renewable generation at an hourly resolution for January 1, 2011 as well as approximate generation capacities in 2014. This dataset is a little out of date and only intended to demonstrate the capabilities of PyPSA.

n = pypsa.examples.scigrid_de(from_master=True)

INFO:pypsa.examples:Retrieving network data from https://github.com/PyPSA/PyPSA/raw/master/examples/scigrid-de/scigrid-with-load-gen-trafos.nc

WARNING:pypsa.io:Importing network from PyPSA version v0.17.1 while current version is v0.27.1. Read the release notes at https://pypsa.readthedocs.io/en/latest/release_notes.html to prepare your network for import.

INFO:pypsa.io:Imported network scigrid-de.nc has buses, generators, lines, loads, storage_units, transformers

There are some infeasibilities without allowing extension. Moreover, to approximate so-called \(N-1\) security, we don’t allow any line to be loaded above 70% of their thermal rating. \(N-1\) security is a constraint that states that no single transmission line may be overloaded by the failure of another transmission line (e.g. through a tripped connection).

n.lines.s_max_pu = 0.7
n.lines.loc[["316", "527", "602"], "s_nom"] = 1715

Because this network includes time-varying data, now is the time to look at another attribute of n: n.snapshots. Snapshots is the PyPSA terminology for time steps. In most cases, they represent a particular hour. They can be a pandas.DatetimeIndex or any other list-like attributes.

n.snapshots

DatetimeIndex(['2011-01-01 00:00:00', '2011-01-01 01:00:00',
               '2011-01-01 02:00:00', '2011-01-01 03:00:00',
               '2011-01-01 04:00:00', '2011-01-01 05:00:00',
               '2011-01-01 06:00:00', '2011-01-01 07:00:00',
               '2011-01-01 08:00:00', '2011-01-01 09:00:00',
               '2011-01-01 10:00:00', '2011-01-01 11:00:00',
               '2011-01-01 12:00:00', '2011-01-01 13:00:00',
               '2011-01-01 14:00:00', '2011-01-01 15:00:00',
               '2011-01-01 16:00:00', '2011-01-01 17:00:00',
               '2011-01-01 18:00:00', '2011-01-01 19:00:00',
               '2011-01-01 20:00:00', '2011-01-01 21:00:00',
               '2011-01-01 22:00:00', '2011-01-01 23:00:00'],
              dtype='datetime64[ns]', name='snapshot', freq=None)

This index will match with any time-varying attributes of components:

n.loads_t.p_set.head(3)

Load	1	3	4	6	7	8	9	11	14	16	...	382_220kV	384_220kV	385_220kV	391_220kV	403_220kV	404_220kV	413_220kV	421_220kV	450_220kV	458_220kV
snapshot
2011-01-01 00:00:00	231.716206	40.613618	66.790442	196.124424	147.804142	123.671946	83.637404	73.280624	175.260369	298.900165	...	202.010114	222.695091	212.621816	77.570241	16.148970	0.092794	58.427056	67.013686	38.449243	66.752618
2011-01-01 01:00:00	221.822547	38.879526	63.938670	187.750439	141.493303	118.391487	80.066312	70.151738	167.777223	286.137932	...	193.384825	213.186609	203.543436	74.258201	15.459452	0.088831	55.932378	64.152382	36.807564	63.902460
2011-01-01 02:00:00	213.127360	37.355494	61.432348	180.390839	135.946929	113.750678	76.927805	67.401871	161.200550	274.921657	...	185.804364	204.829941	195.564769	71.347365	14.853460	0.085349	53.739893	61.637683	35.364750	61.397558

3 rows × 485 columns

We can use simple pandas syntax, to create an overview of the load time series…

n.loads_t.p_set.sum(axis=1).div(1e3).plot(ylim=[0, 60], ylabel="MW")

<Axes: xlabel='snapshot', ylabel='MW'>

_images/dceb87b0be0c819b4e039839460239433b36ef6188ba2d44ec8d360108bc63fc.png

… and the capacity factor time series:

n.generators_t.p_max_pu.T.groupby(n.generators.carrier).mean().T.plot(ylabel="p.u.")

<Axes: xlabel='snapshot', ylabel='p.u.'>

_images/c4197232af01c27368b51412018f019997ac16f5123e4b9309238a3f39547a1c.png

We can also inspect the total power plant capacities per technology…

n.generators.groupby("carrier").p_nom.sum().div(1e3).plot.barh()
plt.xlabel("GW")

Text(0.5, 0, 'GW')

_images/ef8795800170f837dc7fb20e0ffdb2019df3ddf7dcd8641450f289f4ed2804d7.png

… and plot the regional distribution of loads…

load = n.loads_t.p_set.sum(axis=0).groupby(n.loads.bus).sum()

fig = plt.figure()
ax = plt.axes(projection=ccrs.EqualEarth())

n.plot(
    ax=ax,
    bus_sizes=load / 2e5,
);

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

_images/a0f7abad333fcce1bae2cf688237d29b7b9966322efc89b9398c6e22b2f4683f.png

… and power plant capacities:

capacities = n.generators.groupby(["bus", "carrier"]).p_nom.sum()

For plotting we need to assign some colors to the technologies.

import random

carriers = n.generators.carrier.unique()
colors = ["#%06x" % random.randint(0, 0xFFFFFF) for _ in carriers]
n.madd("Carrier", carriers, color=colors)

Index(['Gas', 'Hard Coal', 'Run of River', 'Waste', 'Brown Coal', 'Oil',
       'Storage Hydro', 'Other', 'Multiple', 'Nuclear', 'Geothermal',
       'Wind Offshore', 'Wind Onshore', 'Solar'],
      dtype='object')

Because we want to see which color represents which technology, we cann add a legend using the add_legend_patches function of PyPSA.

from pypsa.plot import add_legend_patches

fig = plt.figure()
ax = plt.axes(projection=ccrs.EqualEarth())

n.plot(
    ax=ax,
    bus_sizes=capacities / 2e4,
)

add_legend_patches(
    ax, colors, carriers, legend_kw=dict(frameon=False, bbox_to_anchor=(0, 1))
)

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

_images/b132905473153e68cc4e178229ed2fef63b69b2612ff313823b43b28796cea43.png

This dataset also includes a few hydro storage units:

n.storage_units.head(3)

	bus	p_nom	carrier	marginal_cost	max_hours	efficiency_store	efficiency_dispatch	control	p_nom_mod	...	build_year	lifetime	state_of_charge_initial	state_of_charge_initial_per_period	state_of_charge_set	cyclic_state_of_charge	cyclic_state_of_charge_per_period	standing_loss	inflow	p_nom_opt
StorageUnit
100_220kV Pumped Hydro	100_220kV	144.5	Pumped Hydro	3.0	6.0	0.95	0.95	PQ	0.0	...	0	inf	0.0	False	NaN	False	True	0.0	0.0	0.0
114 Pumped Hydro	114	138.0	Pumped Hydro	3.0	6.0	0.95	0.95	PQ	0.0	...	0	inf	0.0	False	NaN	False	True	0.0	0.0	0.0
121 Pumped Hydro	121	238.0	Pumped Hydro	3.0	6.0	0.95	0.95	PQ	0.0	...	0	inf	0.0	False	NaN	False	True	0.0	0.0	0.0

3 rows × 30 columns

So let’s solve the electricity market simulation for January 1, 2011. It’ll take a short moment.

n.optimize(solver_name="highs")

Show code cell output Hide code cell output

WARNING:pypsa.components:The following transformers have zero r, which could break the linear load flow:
Index(['2', '5', '10', '12', '13', '15', '18', '20', '22', '24', '26', '30',
       '32', '37', '42', '46', '52', '56', '61', '68', '69', '74', '78', '86',
       '87', '94', '95', '96', '99', '100', '104', '105', '106', '107', '117',
       '120', '123', '124', '125', '128', '129', '138', '143', '156', '157',
       '159', '160', '165', '184', '191', '195', '201', '220', '231', '232',
       '233', '236', '247', '248', '250', '251', '252', '261', '263', '264',
       '267', '272', '279', '281', '282', '292', '303', '307', '308', '312',
       '315', '317', '322', '332', '334', '336', '338', '351', '353', '360',
       '362', '382', '384', '385', '391', '403', '404', '413', '421', '450',
       '458'],
      dtype='object', name='Transformer')

WARNING:pypsa.components:The following transformers have zero r, which could break the linear load flow:
Index(['2', '5', '10', '12', '13', '15', '18', '20', '22', '24', '26', '30',
       '32', '37', '42', '46', '52', '56', '61', '68', '69', '74', '78', '86',
       '87', '94', '95', '96', '99', '100', '104', '105', '106', '107', '117',
       '120', '123', '124', '125', '128', '129', '138', '143', '156', '157',
       '159', '160', '165', '184', '191', '195', '201', '220', '231', '232',
       '233', '236', '247', '248', '250', '251', '252', '261', '263', '264',
       '267', '272', '279', '281', '282', '292', '303', '307', '308', '312',
       '315', '317', '322', '332', '334', '336', '338', '351', '353', '360',
       '362', '382', '384', '385', '391', '403', '404', '413', '421', '450',
       '458'],
      dtype='object', name='Transformer')

INFO:linopy.model: Solve problem using Highs solver

INFO:linopy.io:Writing objective.

Writing constraints.:   0%|          | 0/15 [00:00<?, ?it/s]

Writing constraints.:  13%|█▎        | 2/15 [00:00<00:01, 12.86it/s]

Writing constraints.:  27%|██▋       | 4/15 [00:00<00:00, 16.18it/s]

Writing constraints.:  87%|████████▋ | 13/15 [00:00<00:00, 33.97it/s]

Writing constraints.: 100%|██████████| 15/15 [00:00<00:00, 26.69it/s]

Writing continuous variables.:   0%|          | 0/6 [00:00<?, ?it/s]

Writing continuous variables.: 100%|██████████| 6/6 [00:00<00:00, 63.79it/s]

INFO:linopy.io: Writing time: 0.68s

INFO:linopy.solvers:Log file at /tmp/highs.log.

Running HiGHS 1.5.3 [date: 2023-05-16, git hash: 594fa5a9d-dirty]
Copyright (c) 2023 HiGHS under MIT licence terms
Presolving model
18737 rows, 44750 cols, 115742 nonzeros
13209 rows, 39028 cols, 107511 nonzeros
12712 rows, 31688 cols, 99693 nonzeros
Presolve : Reductions: rows 12712(-130256); columns 31688(-27952); elements 99693(-163933)
Solving the presolved LP
Using EKK dual simplex solver - serial
  Iteration        Objective     Infeasibilities num(sum)
          0     0.0000000000e+00 Ph1: 0(0) 0s
      17079     9.1992880405e+06 Pr: 0(0); Du: 0(3.02869e-13) 4s
Solving the original LP from the solution after postsolve
Model   status      : Optimal
Simplex   iterations: 17079
Objective value     :  9.1992880405e+06
HiGHS run time      :          4.28

INFO:linopy.constants: Optimization successful: 
Status: ok
Termination condition: optimal
Solution: 59640 primals, 142968 duals
Objective: 9.20e+06
Solver model: available
Solver message: optimal

INFO:pypsa.optimization.optimize:The shadow-prices of the constraints Generator-fix-p-lower, Generator-fix-p-upper, Line-fix-s-lower, Line-fix-s-upper, Transformer-fix-s-lower, Transformer-fix-s-upper, StorageUnit-fix-p_dispatch-lower, StorageUnit-fix-p_dispatch-upper, StorageUnit-fix-p_store-lower, StorageUnit-fix-p_store-upper, StorageUnit-fix-state_of_charge-lower, StorageUnit-fix-state_of_charge-upper, Kirchhoff-Voltage-Law, StorageUnit-energy_balance were not assigned to the network.

('ok', 'optimal')

Now, we can also plot model outputs, like the calculated power flows on the network map.

line_loading = n.lines_t.p0.iloc[0].abs() / n.lines.s_nom / n.lines.s_max_pu * 100  # %

norm = plt.Normalize(vmin=0, vmax=100)

fig = plt.figure(figsize=(7, 7))
ax = plt.axes(projection=ccrs.EqualEarth())

n.plot(
    ax=ax,
    bus_sizes=0,
    line_colors=line_loading,
    line_norm=norm,
    line_cmap="plasma",
    line_widths=n.lines.s_nom / 1000,
)

plt.colorbar(
    plt.cm.ScalarMappable(cmap="plasma", norm=norm),
    label="Relative line loading [%]",
    shrink=0.6,
)

/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/cartopy/mpl/feature_artist.py:144: UserWarning:

facecolor will have no effect as it has been defined as "never".

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[72], line 13
ax = plt.axes(projection=ccrs.EqualEarth())
n.plot(
   ax=ax,
   bus_sizes=0,
   (...)
   line_widths=n.lines.s_nom / 1000,
)
---> 13 plt.colorbar(
   plt.cm.ScalarMappable(cmap="plasma", norm=norm),
   label="Relative line loading [%]",
   shrink=0.6,
)

File /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/matplotlib/pyplot.py:2341, in colorbar(mappable, cax, ax, **kwargs)
   if mappable is None:
       raise RuntimeError('No mappable was found to use for colorbar '
                          'creation. First define a mappable such as '
                          'an image (with imshow) or a contour set ('
                          'with contourf).')
-> 2341 ret = gcf().colorbar(mappable, cax=cax, ax=ax, **kwargs)
return ret

File /opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/matplotlib/figure.py:1285, in FigureBase.colorbar(self, mappable, cax, ax, use_gridspec, **kwargs)
if cax is None:
   if ax is None:
-> 1285         raise ValueError(
           'Unable to determine Axes to steal space for Colorbar. '
           'Either provide the *cax* argument to use as the Axes for '
           'the Colorbar, provide the *ax* argument to steal space '
           'from it, or add *mappable* to an Axes.')
   fig = (  # Figure of first axes; logic copied from make_axes.
       [*ax.flat] if isinstance(ax, np.ndarray)
       else [*ax] if np.iterable(ax)
       else [ax])[0].figure
   current_ax = fig.gca()

ValueError: Unable to determine Axes to steal space for Colorbar. Either provide the *cax* argument to use as the Axes for the Colorbar, provide the *ax* argument to steal space from it, or add *mappable* to an Axes.

_images/5b4c07d1b733a8e925fa0530f0eaf6bc1e4bc2049eb4da984250646791a0357b.png

Or plot the hourly dispatch grouped by carrier:

p_by_carrier = n.generators_t.p.groupby(n.generators.carrier, axis=1).sum().div(1e3)

fig, ax = plt.subplots(figsize=(11, 4))

p_by_carrier.plot(
    kind="area",
    ax=ax,
    linewidth=0,
    cmap="tab20b",
)

ax.legend(ncol=5, loc="upper left", frameon=False)

ax.set_ylabel("GW")

ax.set_ylim(0, 80);

Or plot the aggregate dispatch of the pumped hydro storage units and the state of charge throughout the day:

fig, ax = plt.subplots()

p_storage = n.storage_units_t.p.sum(axis=1).div(1e3)
state_of_charge = n.storage_units_t.state_of_charge.sum(axis=1).div(1e3)

p_storage.plot(label="Pumped hydro dispatch [GW]", ax=ax)
state_of_charge.plot(label="State of charge [GWh]", ax=ax)

ax.grid()
ax.legend()
ax.set_ylabel("MWh or MW")

Or plot the locational marginal prices (LMPs):

fig = plt.figure(figsize=(7, 7))
ax = plt.axes(projection=ccrs.EqualEarth())

norm = plt.Normalize(vmin=0, vmax=100)  # €/MWh

n.plot(
    ax=ax,
    bus_colors=n.buses_t.marginal_price.mean(),
    bus_cmap="plasma",
    bus_norm=norm,
)

plt.colorbar(
    plt.cm.ScalarMappable(cmap="plasma", norm=norm), label="LMP [€/MWh]", shrink=0.6
)

Introduction to pypsa

Contents

Introduction to pypsa#

Dependencies#

Basic Structure#

From structured data to optimisation#

Simple electricity market example#

Building a basic network#

Optimisation#

Basic network plotting#

Modifying networks#

Global constraints for emission limits#

Data import and export#

A slightly more realistic example#

Introduction to `pypsa`#