Polars Tutorial: Faster DataFrame Analysis in Python

By Mike Levy · May 7, 2026

If you have worked with pandas on a large dataset, you have probably stared at a progress bar and wondered if there was a faster way. There is. It is called Polars, and it has quietly become one of the most talked-about libraries in the Python data ecosystem.

Polars is a DataFrame library built from the ground up for speed. It is written in Rust, uses Apache Arrow as its memory model, and runs operations across all your CPU cores by default. The result is a library that often outperforms pandas by 5–10x on real-world workloads — sometimes much more.

This tutorial introduces Polars from scratch. You do not need to know Rust, and you do not need to abandon pandas. By the end, you will know how to read data, filter and transform it, group and aggregate it, and understand when Polars is the right tool for the job.

What’s in this tutorial

• What makes Polars different
• Installation
• Reading data
• Selecting columns
• Filtering rows
• Adding and transforming columns
• GroupBy and aggregation
• Sorting
• Lazy evaluation
• Polars vs pandas: a side-by-side comparison
• When to use Polars vs pandas

What makes Polars different

Pandas was built in 2008, before multi-core processors were the norm and before datasets routinely hit hundreds of millions of rows. It does a remarkable job, but it was not designed for the scale of data we work with today.

Polars was designed with modern hardware in mind. Three things make it fast:

Apache Arrow memory format. Polars stores data in columnar memory using the Arrow standard. This makes column operations extremely cache-efficient and enables zero-copy data sharing with other Arrow-compatible tools like DuckDB and Apache Spark.

Parallelism by default. Every Polars operation runs across all available CPU cores automatically. You do not need to configure anything. If your machine has 16 cores, Polars uses all 16.

Query optimization. When you use Polars in lazy mode (more on this below), it analyses your entire chain of operations before running any of them. It can reorder, combine, and eliminate steps to do the minimum amount of work necessary.

The benchmark numbers are striking. On typical data tasks — groupby, filter, join, sort — Polars is consistently faster than pandas, often by a wide margin. On very large datasets, the gap grows.

Installation

Install Polars with pip:

pip install polars

That is all. Polars has no dependency on pandas, NumPy, or anything else you might already have installed. It is self-contained.

Reading data

For this tutorial, we will use a dataset of the world’s top 1,000 movies from IMDb. You can download it from Kaggle. Save it as imdb_top_1000.csv.

Reading a CSV in Polars looks familiar:

import polars as pl

df = pl.read_csv("imdb_top_1000.csv")
print(df.head())

shape: (5, 16)
┌────────────────┬────────────────────────────────┬───────┬───────────┬───┬────────┬────────┬────────┐
│ Poster_Link    ┆ Series_Title                   ┆ Released_Year ┆ Certificate ┆ … ┆ Meta_score ┆ Director ┆ No_of_Votes │
│ ---            ┆ ---                            ┆ ---   ┆ ---         ┆   ┆ ---        ┆ ---      ┆ ---         │
│ str            ┆ str                            ┆ str   ┆ str         ┆   ┆ i64        ┆ str      ┆ i64         │
╞════════════════╪════════════════════════════════╪═══════╪═════════════╪═══╪════════════╪══════════╪═════════════╡
│ https://m.med… ┆ The Shawshank Redemption       ┆ 1994  ┆ A           ┆ … ┆ 80         ┆ Frank Darabont ┆ 2343110 │

A few things to notice right away. Polars prints the shape of the DataFrame at the top (shape: (5, 16)). It also shows you the data type of each column beneath the column name. This is a small thing, but it saves you from running df.dtypes constantly.

You can also read Parquet, JSON, and many other formats:

df = pl.read_parquet("data.parquet")
df = pl.read_ndjson("data.jsonl")

Selecting columns

To select one or more columns, use the select method with pl.col():

df.select(pl.col("Series_Title", "Released_Year", "IMDB_Rating"))

You can also select by data type, which is handy when you have many columns:

# Select all string columns
df.select(pl.col(pl.Utf8))

# Select all numeric columns
df.select(pl.col(pl.NUMERIC_DTYPES))

This is already a step up from pandas, where selecting by dtype requires a separate method (select_dtypes).

Filtering rows

Filtering in Polars uses the filter method. The syntax uses pl.col() to reference columns:

# Movies rated higher than 8.5
high_rated = df.filter(pl.col("IMDB_Rating") > 8.5)
print(high_rated.shape)

(48, 16)

You can combine conditions with & (and) and | (or):

# High-rated movies released after 2000
modern_classics = df.filter(
    (pl.col("IMDB_Rating") > 8.5) & (pl.col("Released_Year") > "2000")
)

String filtering works cleanly too:

# Nolan films only
nolan = df.filter(pl.col("Director") == "Christopher Nolan")

If you are coming from pandas, the main adjustment is using pl.col() instead of just the column name as a string. It feels slightly more verbose at first, but it makes complex expressions much more readable and enables Polars to optimize them.

Adding and transforming columns

To add or modify columns, use with_columns. This is one of the most important methods in Polars:

# Convert Gross revenue (stored as a string like "28,341,469") to an integer
df = df.with_columns(
    pl.col("Gross")
    .str.replace_all(",", "")
    .cast(pl.Int64)
    .alias("Gross_Int")
)

You can add multiple columns in a single with_columns call, which is more efficient than chaining multiple operations:

df = df.with_columns([
    pl.col("IMDB_Rating").round(0).alias("Rating_Rounded"),
    (pl.col("Runtime").str.extract(r"(\d+)").cast(pl.Int64)).alias("Runtime_Mins"),
])

This is where Polars starts to feel genuinely different. In pandas, you typically mutate the DataFrame with df["new_col"] = .... In Polars, with_columns returns a new DataFrame. This immutability makes it easier to chain operations and reason about what your code is doing.

GroupBy and aggregation

GroupBy in Polars is fast and expressive. Here is the average IMDB rating by director, for directors with at least 3 films in the dataset:

by_director = (
    df.group_by("Director")
    .agg([
        pl.col("IMDB_Rating").mean().alias("Avg_Rating"),
        pl.col("IMDB_Rating").count().alias("Film_Count"),
    ])
    .filter(pl.col("Film_Count") >= 3)
    .sort("Avg_Rating", descending=True)
)

print(by_director.head(10))

shape: (10, 3)
┌───────────────────────┬────────────┬────────────┐
│ Director              ┆ Avg_Rating ┆ Film_Count │
│ ---                   ┆ ---        ┆ ---        │
│ str                   ┆ f64        ┆ u32        │
╞═══════════════════════╪════════════╪════════════╡
│ Christopher Nolan     ┆ 8.575      ┆ 8          │
│ Stanley Kubrick       ┆ 8.433333   ┆ 6          │
│ Akira Kurosawa        ┆ 8.4        ┆ 5          │

The agg method accepts a list of expressions, each of which can apply a different aggregation to a different column. You can calculate the mean of one column, the sum of another, and the max of a third — all in a single group_by().agg() call.

Compare this to the equivalent in pandas:

# pandas
by_director = (
    df.groupby("Director")["IMDB_Rating"]
    .agg(["mean", "count"])
    .rename(columns={"mean": "Avg_Rating", "count": "Film_Count"})
    .query("Film_Count >= 3")
    .sort_values("Avg_Rating", ascending=False)
)

Both get the job done, but the Polars version keeps everything inside a single expression chain, and it will run significantly faster on large datasets because the group_by and the subsequent filter can be optimized together.

Sorting

Sorting is straightforward:

# Sort by rating, highest first
df.sort("IMDB_Rating", descending=True).head(5)

# Sort by multiple columns
df.sort(["Released_Year", "IMDB_Rating"], descending=[False, True])

Lazy evaluation

This is the feature that separates Polars from most other DataFrame libraries.

In eager mode (everything we have done so far), each method runs immediately and returns a result. This is simple and intuitive, but it means Polars cannot see ahead to optimize your query.

In lazy mode, you build up a query plan and Polars executes it all at once when you call .collect(). Before executing, Polars applies a query optimizer that can:

• Push filters earlier in the pipeline to reduce the data being processed
• Eliminate columns you select but never use
• Combine multiple operations into a single pass over the data

Switching to lazy mode requires just one change — replace pl.read_csv() with pl.scan_csv(), and call .collect() at the end:

result = (
    pl.scan_csv("imdb_top_1000.csv")
    .filter(pl.col("IMDB_Rating") > 8.0)
    .group_by("Director")
    .agg(pl.col("IMDB_Rating").mean().alias("Avg_Rating"))
    .sort("Avg_Rating", descending=True)
    .collect()  # Execute the optimized query
)

You can inspect the query plan before running it:

query = (
    pl.scan_csv("imdb_top_1000.csv")
    .filter(pl.col("IMDB_Rating") > 8.0)
    .group_by("Director")
    .agg(pl.col("IMDB_Rating").mean())
)

print(query.explain())

For small datasets, the difference is marginal. For large datasets — tens of millions of rows or more — lazy mode can cut execution time dramatically because the filter might eliminate 90% of the data before the groupby ever runs.

As a rule of thumb: use eager mode for exploration and interactive analysis, and lazy mode for production pipelines.

Polars vs pandas: a side-by-side comparison

Here is a quick reference for common operations in both libraries:

The APIs are similar enough that pandas users can get productive quickly. The main conceptual shift is moving from mutable operations (df["col"] = ...) to method chaining with immutable returns.

When to use Polars vs pandas

Use Polars when:

• Your dataset is large (millions of rows or more) and pandas is slow or running out of memory
• You are building a data pipeline that runs on a schedule and performance matters
• You want to write expressive, chainable transformation code
• You are working with Parquet files — Polars handles them extremely efficiently

Stick with pandas when:

• You are working interactively with small datasets where speed is not a concern
• You rely on libraries that expect pandas DataFrames (many ML libraries, for example, have not yet added native Polars support)
• Your team is more comfortable with pandas and the switching cost outweighs the performance gain

The good news is that converting between the two is easy when you need to:

# Polars → pandas
pandas_df = polars_df.to_pandas()

# pandas → Polars
polars_df = pl.from_pandas(pandas_df)

So you can use Polars for the heavy lifting and hand off to pandas or scikit-learn when needed.

Next steps

Polars has strong documentation at pola.rs and an active community on GitHub. Once you are comfortable with the basics covered here, the natural next step is exploring:

• Joins — Polars supports all standard join types with a clean API
• Window functions — for running totals, rankings, and other calculations over groups
• Streaming mode — for datasets too large to fit in memory

If you are working with large datasets and Python, Polars is worth learning. The API is well-designed, the performance improvement is real, and it fits naturally alongside the rest of the Python data stack.

To sharpen your Python data skills further, check out our Python for Data Science path, or dive deeper into data manipulation with our pandas course.