Polars Series shift() Function with Examples

  • Post author:
  • Post category:Polars
  • Post last modified:May 8, 2025
  • Reading time:14 mins read
You are currently viewing Polars Series shift() Function with Examples

In Polars, the shift() function is used to shift the values of a Series by a specified number of periods (i.e., move the values up or down along the index). This is useful when you want to compare the current value with previous or future values in time series data. You can also specify a fill value (like None or NaN) to populate the positions left empty by the shift.

Advertisements

In this article, I will explain the shift() method in Polars Series, covering its syntax, parameters, and practical usage, demonstrating how it returns a new Series with values shifted and any resulting gaps filled with a specified fill_value or None.

Key Points –

  • The shift() function moves the values of a Series by a specified number of positions.
  • You can shift the Series either up or down by providing a positive or negative integer to the n parameter.
  • Positive periods shift the data down, introducing nulls or a fill value at the top
  • Negative periods shift the data up, introducing nulls or a fill value at the bottom.
  • Shifting does not modify the original Series; it returns a new shifted Series.
  • It works with different data types, including numeric, string, and Boolean Series.
  • The fill_value parameter can accept constants or expressions (e.g., mean, literal values).
  • By default, the fill_value is None, which results in nulls filling the shifted-out positions.

Polars Series shift() Introduction

Let’s know the syntax of the shift() function.


# Syntax of series shift()
Series.shift(n: int = 1, *, fill_value: IntoExpr | None = None) → Series

Parameters of the Series shift()

Following are the parameters of series shift() method.

  • n (int, default = 1) –
    • The number of periods to shift the values of the Series.
    • Positive n will shift the values downwards (move the values to the next index), while negative n will shift the values upwards (move the values to the previous index).
  • fill_value (IntoExpr | None, optional)
    • The value to fill the gaps created by the shift.
    • If not provided, the gaps are filled with None (i.e., null).
    • You can provide a constant value or an expression, such as the mean of the Series or any other valid expression.

Return Value

This function returns a new Series with the values shifted and the gaps filled with the specified fill_value or None.

Usage of Polars Series shift() Function

The shift() function in Polars shifts the values in a Series by a specified number of positions. You can shift the data forward or backward, and the empty positions (caused by the shift) are filled with a specified value (by default, None).

Now, let’s create a Polars Series.


import polars as pl

# Sample Series
ser = pl.Series("integers", [5, 10, 15, 20, 25, 30])
print("Original Series:\n", ser)

Yields below output.

polars series shift

You can shift the Series down by 1 using the shift() function, and it will automatically fill the resulting empty space with None (which is the default behavior).


# Shift down by 1 (default behavior)
ser2 = ser.shift(1)
print("Shifted Series (down by 1):\n", ser2)

In the above example, the first element is shifted out, and None (null) is inserted at the beginning, with the other values shifted down.

polars series shift

Shift Down by 2 Positions

To shift the Series down by 2 positions, you can use the shift() method with n=2. This will move the elements in the Series down by two positions, and the first two positions will be filled with null (or a custom fill_value if provided).


# Shift down by 2 positions
ser2 = ser.shift(2)
print("Shifted Series (down by 2 positions):\n", ser2)

# Output:
# Shifted Series (down by 2 positions):
# shape: (6,)
# Series: 'integers' [i64]
# [
#	null
#	null
#	5
#	10
#	15
#	20
# ]

Here,

  • The first two elements (5 and 10) are moved down by 2 positions.
  • The first two positions are filled with null values.
  • The rest of the elements shift down by 2 positions, with 15 being in the first position, 20 in the second, and so on.

Shift Up by 1 Position (N = -1)

To shift the Series up by 1 position (using n = -1), you can specify a negative value for n. This will move the elements in the Series up by one position, and the last position will be filled with null (or a custom fill_value if provided).


# Shift up by 1 position
ser2 = ser.shift(-1)
print("Shifted Series (up by 1 position):\n", ser2)

# Output:
# Shifted Series (up by 1 position):
# shape: (6,)
# Series: 'integers' [i64]
# [
#	10
#	15
#	20
#	25
#	30
#	null
# ]

Here,

  • The first element (5) is removed and shifted up, and the rest of the elements move up by one position.
  • The last position (which previously contained 30) is now filled with null (default fill_value).
  • This type of shifting is often used when working with time-series data to create lag variables, or for computing differences between current and previous values.

Shift with a Custom Fill Value

To shift a Polars Series and use a custom fill value like 100, you can pass the desired value to the fill_value parameter in the shift() method. This lets you replace the default null values with any custom value of your choice, such as 100, "Unknown", or any other value that fits your context.


# Shift down by 1 position with a custom fill value (100)
ser2 = ser.shift(1, fill_value=100)
print("Shifted Series (down by 1 with custom fill value 100):\n", ser2)

# Output:
# Shifted Series (down by 1 with custom fill value 100):
# shape: (6,)
# Series: 'integers' [i64]
# [
#	100
#	5
#	10
#	15
#	20
#	25
# ]

Here,

  • The series is shifted down by 1 position.
  • Instead of filling the first position with null (the default behavior), the first position is filled with 100 because of the custom fill_value.
  • The rest of the elements shift down by 1 position, with 5 moving to the first position, 10 to the second, and so on.

Shift by -2 with a Fill Value

To shift a Polars Series up by 2 positions (using n=-2) with a custom fill value, use the shift() method while specifying both n and fill_value. This shifts the Series up by two positions, and the last two positions, left empty by the shift, will be filled with the given fill_value instead of null.


# Shift up by 2 positions with a custom fill value (999)
ser2 = ser.shift(-2, fill_value=999)
print("Shifted Series (up by 2 with custom fill value 999):\n", ser2)

# Output:
# Shifted Series (up by 2 with custom fill value 999):
# shape: (6,)
# Series: 'integers' [i64]
# [
#	15
#	20
#	25
#	30
#	999
#	999
# ]

Here,

  • -2 means shift up by 2 positions.
  • The first 4 values will move up by 2 places.
  • The last 2 vacated positions will be filled with 999 (the fill_value).

Shift a Boolean Series

You can shift a Boolean Series in Polars using the shift() method, just like with numeric Series. It supports shifting up or down and also allows specifying a custom fill_value if needed.


import polars as pl

# Sample Boolean Series
ser = pl.Series("bools", [True, False, True, True, False])

# Shift down by 1
ser2 = ser.shift(1)
print("Shifted Boolean Series (down by 1):\n", ser2)

# Output:
# Shifted Boolean Series (down by 1):
# shape: (5,)
# Series: 'bools' [bool]
# [
#	null
#	true
#	false
#	true
#	true
# ]

Here,

  • The first value becomes null (default fill).
  • The remaining values are shifted down by one.

Conclusion

In conclusion, the shift() method in Polars is a versatile function that allows you to shift the values of a Series either up or down by a specified number of positions. It works seamlessly with various data types, including numeric and Boolean Series. You can also customize how the resulting gaps are filled by specifying a fill_value. This makes it a valuable tool for time-series analysis, feature engineering, and other data manipulation tasks, offering both flexibility and efficiency in data processing workflows.

Happy Learning!!

Reference

Tags: