PyPSA: Introduction

PyPSA stands for Python for Power System Analysis.

PyPSA is an open source Python package for simulating and optimising modern energy systems that include features such as

• conventional generators with unit commitment (ramp-up, ramp-down, start-up, shut-down),

• time-varying wind and solar generation,

• energy storage with efficiency losses and inflow/spillage for hydroelectricity

• coupling to other energy sectors (electricity, transport, heat, industry),

• conversion between energy carriers (e.g. electricity to hydrogen),

• transmission networks (AC, DC, other fuels)

PyPSA can be used for a variety of problem types (e.g. electricity market modelling, long-term investment planning, transmission network expansion planning), and is designed to scale well with large networks and long time series.

Compared to building power system by hand in linopy, PyPSA does the following things for you:

• manage data inputs in standardised format

• build standardised optimisation problem

• communicate with the solver(s)

• retrieve and process optimisation results

• manage data outputs in standardised format

The documentation describes it as follows:

PyPSA is an open-source Python framework for optimising and simulating modern power and energy systems that include features such as conventional generators with unit commitment, variable wind and solar generation, hydro-electricity, inter-temporal storage, coupling to other energy sectors, elastic demands, and linearised power flow with loss approximations in DC and AC networks. PyPSA is designed to scale well with large networks and long time series. It is made for researchers, planners and utilities with basic coding aptitude who need a fast, easy-to-use and transparent tool for power and energy system analysis.

Key Dependencies¶

• pandas for storing data about network components and time series

• numpy and scipy for linear algebra and sparse matrix calculations

• matplotlib and cartopy for plotting on a map

• networkx for network calculations

• linopy for handling optimisation problems

Basic Structure¶

From structured data to optimisation¶

The design principle of PyPSA is that each component is associated with a set of variables and constraints that will be added to the optimisation model based on the input data stored for the components.

For an hourly electricity market simulation, PyPSA will solve an optimisation problem that looks like this

such that

Decision variables:

• $g_{i,s,t}$ is the generator dispatch at bus $i$, technology $s$, time step $t$,

• $f_{\ell,t}$ is the power flow in line $\ell$,

• $g_{i,r,t,\text{dis-/charge}}$ denotes the charge and discharge of storage unit $r$ at bus $i$ and time step $t$,

• $e_{i,r,t}$ is the state of charge of storage $r$ at bus $i$ and time step $t$.

Parameters:

• $o_{i,s}$ is the marginal generation cost of technology $s$ at bus $i$,

• $x_\ell$ is the reactance of transmission line $\ell$,

• $K_{i\ell}$ is the incidence matrix,

• $C_{\ell c}$ is the cycle matrix,

• $G_{i,s}$ is the nominal capacity of the generator of technology $s$ at bus $i$,

• $F_{\ell}$ is the rating of the transmission line $\ell$,

• $E_{i,r}$ is the energy capacity of storage $r$ at bus $i$,

• $\eta^{0/1/2}_{i,r,t}$ denote the standing (0), charging (1), and discharging (2) efficiencies.

Simple electricity market example¶

Let’s get acquainted with PyPSA to build a variant of one of the simple electricity market models we previously built in linopy.

We have the following data:

• fuel costs in € / MWh$_{th}$

• efficiencies of thermal power plants in MWh$_{el}$ / MWh$_{th}$

• specific emissions in t$_{CO_2}$ / MWh$_{th}$

• power plant capacities in MW

• electrical load in MW

Building a basic network¶

By convention, PyPSA is imported without an alias:

First, we create a new network object which serves as the overall container for all components.

The second component we need are buses. Buses are the fundamental nodes of the network, to which all other components like loads, generators and transmission lines attach. They enforce energy conservation for all elements feeding in and out of it (i.e. Kirchhoff’s Current Law).

Components can be added to the network n using the n.add() function. It takes the component name as a first argument, the name of the component as a second argument and possibly further parameters as keyword arguments. Let’s use this function, to add buses for each country to our network:

For each class of components, the data describing the components is stored in a pandas.DataFrame. For example, all static data for buses is stored in n.buses

You see there are many more attributes than we specified while adding the buses; many of them are filled with default parameters which were added. You can look up the field description, defaults and status (required input, optional input, output) for buses here https://docs.pypsa.org/latest/user-guide/components/buses/, and analogously for all other components.

The method n.add() also allows you to add multiple components at once. For instance, multiple carriers for the fuels with information on specific carbon dioxide emissions, a nice name, and colors for plotting. For this, the function takes the component name as the first argument and then a list of component names and then optional arguments for the parameters. Here, scalar values, lists, dictionary or pandas.Series are allowed. The latter two needs keys or indices with the component names.

The n.add() function is very general. It lets you add any component to the network object n. For instance, in the next step we add generators for all the different power plants.

In Mozambique:

In South Africa (in a loop):

As a result, the n.generators DataFrame looks like this:

Next, we’re going to add the electricity demand. This time, without a loop.

A positive value for p_set means consumption of power from the bus (in MW).

Finally, we add the connection between Mozambique and South Africa with a 500 MW line:

We can have a sneak peek at the network we built with the n.plot() function.

Or with its interactive sibling n.explore().

Optimisation¶

With all input data transferred into PyPSA’s data structure, we can now build and run the resulting optimisation problem. In PyPSA, building, solving and retrieving results from the optimisation model is contained in a single function call n.optimize(). This function optimizes the operational dispatch decisions for least cost, as well as investment decisions for components marked as extendable (more on this next time).

The n.optimize() function can take many arguments. The most relevant for the moment is the choice of the solver. We already know some solvers from the introduction to linopy (e.g. “highs” and “gurobi”). They need to be installed in your environment (e.g. conda), to use them here:

Let’s have a look at the results.

Since the power flow and dispatch are generally time-varying quantities, these are stored in a different location than e.g. n.generators. They are stored in n.generators_t. Thus, to find out the dispatch of the generators, run

or if you prefer it in relation to the generators nominal capacity

You see that the time index has the value ‘now’. This is the default index when no time series data has been specified and the network only covers a single state (e.g. a particular hour).

Similarly you will find the power flow in transmission lines at

The p0 will tell you the flow from bus0 to bus1. p1 will tell you the flow from bus1 to bus0.

What about the shadow prices?

Explore the optimisation model¶

Under the hood, a call to n.optimize() builds a linopy optimisation model, solves it with the specified solver, and then retrieves the results back into the PyPSA data structure. We can access this intermediate linopy model via n.model. This allows us to explore the optimisation model in more detail, for instance to see all variables and constraints that were added to the optimisation problem.

Basic network plotting¶

The results can also be visualised with PyPSA’s built-in plotting functions. The n.plot() function uses matplotlib to create static plots, while n.explore() creates interactive plots based on pydeck. In the following, we will focus on the interactive plotting with n.explore().

The n.explore() function has a variety of styling arguments to tweak the appearance of the buses, the lines and the map in the background. For example, we can size the buses according to their load:

The bus_size argument of n.explore() can be even more powerful.

The dispatch of each generator, we can find at:

If we group this by the bus and carrier and use it as bus_size, we can create pie charts at each bus showing the generation mix.

Additionally, we can show the line flows by specifying the line_flow argument.

In the backend, the plotting function will look up the colors specified in n.carriers for each carrier and match it with the second index-level of s.

Modifying networks¶

Modifying data of components in an existing PyPSA network is as easy as modifying the entries of a pandas.DataFrame. For instance, if we want to reduce the cross-border transmission capacity between South Africa and Mozambique, we’d run:

You can see that the production of the hydro power plant was reduced and that of the gas power plant increased owing to the reduced transmission capacity.

Global constraints for emission limits¶

In the example above, we happen to have some spare gas capacity with lower carbon intensity than the coal and oil generators. We could use this to lower the emissions of the system, but it will be more expensive. We can implement the limit of carbon dioxide emissions as a constraint.

This is achieved in PyPSA through Global Constraints which add constraints that apply to many components at once.

But first, we need to calculate the current level of emissions to set a sensible limit.

We can compute the emissions per generator (in tonnes of CO$_2$) in the following way.

where $\rho$ is the specific emissions (tonnes/MWh thermal) and $\eta$ is the conversion efficiency (MWh electric / MWh thermal) of the generator with dispatch $g$ (MWh electric):

Summed up, we get total emissions in tonnes:

So, let’s say we want to reduce emissions by 10%:

Can we lower emissions even further? Say by another 5% points?

No! Without any additional capacities, we have exhausted our options to reduce emissions in that hour. The solver tells us that the problem is infeasible, i.e. there is no solution that satisfies all constraints. Better revert this change:

Data import and export¶

You may find yourself in a need to store PyPSA networks for later use. Or, maybe you want to import the genius PyPSA example that someone else uploaded to the web to explore.

Among other file formats, PyPSA can be stored as netCDF (.nc) file or as an Excel file (.xlsx). The approach is similar for all formats:

A slightly more realistic example¶

Dispatch problem with German SciGRID network

SciGRID is a project that provides an open reference model of the European transmission network. The network comprises time series for loads and the availability of renewable generation at an hourly resolution for January 1, 2011 as well as approximate generation capacities in 2014. This dataset is a little out of date and only intended to demonstrate the capabilities of PyPSA.

There are some infeasibilities without allowing extension. Also, to approximate so-called $N-1$ security, we don’t allow any line to be loaded above 70% of their thermal rating. $N-1$ security is a constraint that states that no single transmission line may be overloaded by the failure of another transmission line (e.g. through a tripped connection).

Because this network includes time-varying data, now is the time to look at another attribute of n: n.snapshots. Snapshots is the PyPSA terminology for time steps. In most cases, they represent a particular hour. They can be a pandas.DatetimeIndex or any other list-like attributes.

This index will match with any time-varying attributes of components:

We can use simple pandas syntax, to create an overview of the load time series...

... and the capacity factor time series:

We can also inspect the total power plant capacities per technology...

... and plot the regional distribution of loads...

... and power plant capacities...

... for which we need to assign some colors to the technologies first:

So let’s solve the electricity market simulation for January 1, 2011:

Now, we can also plot model outputs, like the calculated power flows on the network map.

Or plot the hourly dispatch using PyPSA’s built-in statistics functionality:

Statistics¶

The PyPSA n.statistics module provides a variety of pre-defined tables and plots to analyse optimisation results at different aggregation levels. For instance, the energy balance, the operational costs (OPEX), curtailment, prices and market value. We will use this module in more detail in later workshops on sector coupling and investment optimisation.

Energy balance:

Operational costs (OPEX):

There is much more to explore in PyPSA. If you are hooked, have a look at the documentation and the examples section.

Exercises¶

Modify some of the input data of the South Africa - Mozambique network n, first removing the emission limit global constraint:

Task 1: Model an outage of the transmission line by removing it. How does the model compensate for the lack of transmission?

Task 2: Double the wind capacity. How is the electricity price in South Africa affected? What generator is price-setting?

Task 3: South Africa brings a new 1000 MW nuclear power plant into operation with an estimated marginal electricity generation cost of 20 €/MWh. Will that affect the electricity price?