09-20, 13:25–14:00 (Europe/Amsterdam), Escher
Python is beloved for its simplicity and versatility, but it can struggle with performance in compute-intensive tasks. Rust, on the other hand, offers high performance and memory safety. This talk will explain how you can harness the power of Rust to enhance Python modules using the PyO3 library.
We will explore this through a practical example: a pure Python payment handler and an optimized version where its functionality is abstracted away using Rust. This approach will demonstrate how to overcome performance bottlenecks while retaining the ease of use and flexibility that Python offers. However, like any tool, it comes with its own considerations and trade-offs.
This talk is particularly interesting for Machine Learning Engineers and Python developers seeking to boost the performance of their applications.
Python's simplicity and extensive libraries make it a go-to language for many data scientists and machine learning engineers, but performance can become a limiting factor in compute-intensive scenarios. Rust, a language known for its speed and memory safety, can help address these issues. This talk will guide attendees on how to integrate Rust into Python projects using the PyO3 library, focusing on a practical example.
Breakdown:
- Minutes 0-10: Characteristics of Python vs. Rust
- Introduction to Python’s strengths and limitations (and why they are as they are)
- Introduction to Rust's advantages in performance and safety vs. its relative disadvantages
-
Examples of well-known Python packages using Rust (e.g., Pydantic, Ruff, Polars).
-
Minutes 10-15: Introduction to PyO3
- The ways to connect from Python to Rust (sole Rust binary, PyO3)
- What is PyO3 and how it bridges Python and Rust.
-
Basic usage and setup.
-
Minutes 15-25: Case Study – Optimizing a Payment Handler
- Initial implementation in Python and identification of performance bottlenecks.
-
Rewriting critical parts in Rust using PyO3.
-
Minutes 25-28: Results and Alternatives
- Demonstration of performance results / benchmarking
-
Pros and cons of the PyO3 approach + the alternatives out there
-
Minutes 28-30: Conclusion
-
Summary of key takeaways.
-
Minutes >30: Q&A
- Open floor for questions and discussion.
Takeaway
Attendees will leave with a clear understanding of the strengths and limitations of Python vs those of Rust. They will see how one can use Rust to optimize Python modules, leveraging the PyO3 library, exemplified by optimizing a dummy payment handler.
Prior Knowledge Expected
Basic understanding of Python programming and familiarity with performance bottlenecks in Python applications. Familiarity with Rust is a plus but not required.
Type / tone
This is an informative talk that includes practical examples.
I am a hybrid machine learning engineer, data scientist and cloud infrastructure engineer whose professional track record covers many aspects of applied data science, MLOps and (big) data engineering. I am passionate about developing scalable, production-ready, and efficient ML applications / infrastructure.
I relate to the everyday struggles that data scientists and machine learning engineers encounter in their workflows, whether that be reproducible experimentation, feature engineering, model training, or inference. Being able to come up with solutions for impediments in these areas and enabling data science teams to be as productive as possible is what drives me.