If you’ve worked in computer vision or robotics for more than a week, you know the pain. That deep, soul-crushing pain of trying to figure out why your model is failing. You’re staring at a wall of text logs, maybe a few static images saved to a folder, and you’re trying to reconstruct a dynamic, multi-sensory event in your head. It’s like trying to understand a movie by looking at three random screenshots and a page from the script. It’s a nightmare.
I’ve spent more nights than I care to admit `print()`-ing the shape of a tensor, or trying to overlay bounding boxes with OpenCV’s clunky `cv2.imshow()`, only to have the window freeze. We've all been there. It’s the dirty little secret of our field: our debugging tools often feel like they’re from a different century.
So, when I come across a tool that doesn't just promise to ease that pain but actually seems to enjoy it, I sit up and pay attention. And lately, that tool has been Rerun. It’s not just another plotter; it feels more like a time-traveling debugger for your AI's senses. And honestly? It’s been a game-changer for my workflow.
So, What Exactly Is Rerun Anyway?
At its core, Rerun is a combination of an SDK (Software Development Kit) and a visualizer. Think of it this way: your Python, C++, or Rust code is running your complex robotics simulation or CV algorithm. You use the Rerun SDK to send little messages—or logs—about what's happening. “Here’s the camera feed at this timestamp.” “Here’s the 3D point cloud from the LiDAR.” “Here’s the trajectory the robot arm is planning.”
Then, the Rerun viewer picks up all those messages and organizes them on an interactive timeline. You can scrub back and forth in time, zoom in on a 3D model, inspect a specific data point, and see how all your different data streams line up. It’s a full, multimodal, temporal debugging experience. You're not just seeing screenshots; you're re-living the data stream. For anyone working on what the industry is starting to call Physical AI—AI that interacts with the real world—this is huge.
Visit Rerun
Getting Your Hands Dirty with Rerun
The part that really hooked me was how simple it is to get started. There wasn't a three-day setup process or a configuration file from the seventh circle of hell. For Python, it's literally:
pip install rerun-sdk
And you're pretty much ready to go. The fundamental idea is logging data with a simple API. You initialize it, and then you just… log stuff. It looks something like this:
import rerun as rrrr.init("my_awesome_project", spawn=True)# Your code here...position = [1.0, 2.0, 3.0]color = [255, 0, 0]rr.log("my_point", rr.Points3D([position], colors=[color]))That’s it. That one `rr.log()` call sends a 3D point to the Rerun viewer. Now imagine doing that for thousands of points, for images, for tensors, for text annotations, all within your main application loop. It just works. The viewer pops up and you have an interactive window into your code’s soul. This low barrier to entry is, in my book, a massive win.
The Features That Actually Matter
A long feature list is one thing, but tools are defined by the features you actually use. With Rerun, a few things really stand out in day-to-day work.
Blazingly Fast Performance Thanks to Rust
Let's talk about Rust. Rerun is built on it, and you can feel it. The viewer is snappy and responsive, even when you're throwing a decent amount of data at it. This isn’t a small detail. A debugging tool that lags or crashes is worse than no tool at all. Because Rerun's core is so performant, it doesn’t get in the way. It feels light, which is the highest compliment you can pay to any developer tool.
See Everything with Multimodal Visualization
This is the star of the show. The real world isn't just a single data type. A self-driving car perceives the world through cameras (images), LiDAR (3D point clouds), and RADAR (more sensor data), all while tracking its own position (transforms) and making decisions (text logs). Trying to debug that with separate tools is a recipe for a headache.
Rerun lets you log all of it to the same timeline. You can see the camera image perfectly synchronized with the 3D point cloud that was captured at the exact same moment. It's like getting a director's cut of your AI's perception, complete with multiple camera angles and developer commentary. It makes spotting misalignments and weird correlations so much easier.
The Joy of an Open-Source Core
A huge part of the appeal for me, and for many in the community, is that Rerun is open source. You can check out their code on GitHub, contribute, and be confident that the core tool isn’t going to be locked behind a paywall. This fosters trust and encourages adoption. For developers, researchers, and students, this means you can pick it up and use it on your projects today, for free.
Open Source vs The Commercial Cloud
Rerun has a smart model that separates the local, open-source tool from its upcoming commercial offering. This isn't a bait-and-switch; it's a logical division of labor.
| Rerun Open Source | Rerun Commercial (in development) |
|---|---|
| Primarily for local debugging and visualization. Runs on your machine. | Managed data platform for large-scale data. |
| Free and available on GitHub. | Aimed at teams and enterprises. |
| Powerful visualization for multimodal logs on web and native. | Adds data ingestion, storage, querying, and streaming at scale. |
My take? The open-source tool is perfect for the individual developer, the researcher, or the small team trying to build and debug a system. It solves the immediate, painful problem of seeing your data. The commercial platform is the answer to the next question: “Okay, now I have terabytes of this data from all my test runs… what do I do with it?” It's for managing, querying, and collaborating on that data at an organizational level. The fact that their commercial side is still in development with design partners tells me they're taking the time to get it right, which is respectable.
Where It Shines and Where It's Still Growing
No tool is perfect, especially not a relatively new one. But Rerun gets so much right. Its strength is its laser focus on making the developer's life easier. The speed, the simplicity of the API, the powerful, integrated visualizations—it all just clicks. I was working on a personal project involving 3D pose estimation, and being able to visualize the 3D skeletons overlaid on the 2D video feed in real-time saved me, and I'm not exaggerating, days of frustrating guesswork.
On the flip side, as they are open about, the solution for managing data at a massive scale is still being built out in their commercial platform. If you're a giant corporation looking for a petabyte-scale, fully-managed data warehouse for robotics data today, you'll have to join their waitlist. But that’s hardly a criticism of the open-source tool, which is fantastic at what it's designed to do. It’s a young project in a fast-moving space, and it’s growing in the right direction.
Is Rerun Worth Your Time? (Spoiler: Yes)
Absolutely. If you are a student, researcher, or engineer working with any kind of sensor data that has a time component—especially if it involves 2D images or 3D space—you should download Rerun right now. It will improve your workflow.
The core SDK and viewer are free and open source, so the cost of trying it is basically zero. For larger teams, Rerun is a project you should be watching very, very closely. The foundation they've built with the open-source tool is so strong that their commercial platform has the potential to become an industry standard for the Physical AI data stack.
We've been making do with subpar tools for too long. We've been cobbling together solutions with Matplotlib, print statements, and sheer force of will. Rerun feels like a tool built by people who have actually experienced that pain and decided to build the solution they always wanted. And for that, I'm a huge fan.
Frequently Asked Questions about Rerun
- What languages does Rerun support?
- Rerun has primary SDKs for Python, Rust, and C++. This covers the vast majority of use cases in the robotics and computer vision communities.
- Is Rerun free to use?
- Yes, the core Rerun SDK and viewer are completely free and open source under a permissive license. The company plans to offer a commercial cloud platform for large-scale data management in the future.
- What kind of data can I visualize with Rerun?
- A whole lot! It's built for multimodal data, including 2D images, 3D point clouds, bounding boxes (2D and 3D), tensors, scalars for plotting, text logs, and various geometric primitives like arrows and lines. It's all synchronized on a timeline.
- How is Rerun different from something like TensorBoard?
- It's a great question. While TensorBoard is excellent for plotting scalars, histograms, and viewing a few images from machine learning training runs, it's not really designed for time-series or spatial data. Rerun's strength is its interactive, time-synchronized visualization of complex, multimodal data like 3D scenes, which is something TensorBoard doesn't focus on.
- Can I use Rerun in a Jupyter Notebook?
- Yes, Rerun is designed to be embeddable and works well within environments like Jupyter notebooks and web apps, making it a flexible tool for both research and development.
- What is "Physical AI"?
- Physical AI is a term for artificial intelligence systems that perceive and interact with the physical world. This includes fields like robotics, autonomous vehicles, augmented reality, and drone technology. Rerun is positioned as a key tool for developers in this space.
