First, congratulations on your selection as a 2025 BigDATAwire Person to Watch! What inspired you to create Polars?

I was a fan of the Rust language and observed that there was no proper DataFrame library in the community. When I started to do some research on the algorithms that are needed for typical data processing (Joins, Group By’s etc.), I learned that the whole concept of query planning and execution engines was ignored mostly in Python single machine DataFrame libraries, which just evaluated the operations ad-hoc. This convinced me that merging query optimization with DataFrame libraries was something that was missing for single node OLAP and motivated me to create Polars.
 

Polars is 7X faster than PySpark, and 9X faster than Dask. How did you make it so flipping fast?

This isn’t something you can pinpoint to a single thing. When I started it was mostly because of efficient algorithms, columnar memory, query planning, effective parallelism and the fact that I wrote it in Rust. However, Polars is getting a lot faster as we speak. We wrote a new query engine that does morsel driven parallelism and has much better cache behavior than the current in-memory engine has. I would say, Polars is fast because we put a lot of effort in optimal performance design on that goal from scratch with control over everything that is important for performance.
 

Can you tell us about your startup, Polars Inc? What are its products and who should consider using them?

Polars the company is building a managed cloud platform that allows you to run Polars queries remotely and scale Polars beyond a single machine.

It is currently very common that data scientist write DataFrame related code in pandas and that a production team rewrites that to pyspark to scale. This is something that should not be needed. We offer Polars as an API that is optimal on small scale and large scale. Just as SQL can run on SQLLite or Snowflake.
 

What does the future hold for Polars the library?

There is a lot of cool stuff in the works. Extensibility is very important to us, so we want to extend our Plugin support from IO sources to IO sinks. We also want to create extension types, which allows you to design your own domain logic in the Polars type model. This would for instance open up fields like Geo.

The biggest feature that we are working on now, is our new streaming engine which is ~3.5 faster than Polars is today! Other than that, it can run on much larger datasets (The data doesn’t have to fit in RAM). This is the future of Polars execution and will empower users to process larger datasets with Polars versatile API.
 

What can you tell us about yourself outside of the professional sphere – unique hobbies, favorite places, etc.? Is there anything about you that your colleagues might be surprised to learn?

To be honest, between being a father of two young kids and running a company, I don’t have much time left anymore for hobbies. I find I can relax a bit during the time-off I get when doing the dishes whilst watching a lot of obscure YouTube videos.