Using Odyssey to Analyze Floating Point Expressions

For the last 8 months, I’ve been working on Odyssey, a tool for analyzing floating point expressions by integrating floating point tools into one dynamic workbench. One of our big milestones is approaching, as we’ll present Odyssey at the Supercomputing Conference (SC 2024) in less than a week.

This blog serves as an overview of recent updates to Odyssey, including feature improvements, UI updates, and the process of preparing for the upcoming demo.

Introduction: Floating Point Error and Odyssey

Ensuring numerical accuracy is essential in computing, particularly when working with floating-point values. Even small inaccuracies can compound into system-breaking errors across scientific simulations and financial calculations.

There are many tools out there, each with their own unique approach. For example, Herbie generates floating-point rewrites to improve accuracy in expressions, and FPTaylor uses Taylor expansions to create tight error bounds.

However, these tools often operate independently. Switching between them to find the best approach can be inconvenient and time-consuming, not to mention the steep learning curve with new tools; downloading binaries, working with command-line interfaces, and finding documentation can be challenging. Odyssey steps in to address these problems by integrating multiple floating-point tools into a single, user-friendly workbench. Odyssey makes it easy to analyze and rewrite expressions in one place and includes visualizations that provide insights into accuracy, speed, and error. While Odyssey might not be the end-all-be-all on floating-point error analysis, we foresee it as a useful tool to guide users through identifying errors, finding alternative expressions, and making informed choices for their own situations.

Example: Analyzing sqrt(x + 1) - sqrt(x) with Odyssey

Let’s walk through how Odyssey tackles a classic expression that contains floating point error: sqrt(x + 1) - sqrt(x).

Expression Input: From Odyssey’s homepage, we input sqrt(x + 1) - sqrt(x) and set a range of values for x. We’ll go with the default bounds for now, but they can be adjusted as needed.

Visual Feedback: A few indicators immediately give us insights into the expression. The right hand side of the screen displays the accuracy of the expression, which is around 53.3%, and a default speedup of 1.0x. The left side shows an error plot, illustrating how the expression’s accuracy varies with x. Everything looks smooth until we hit a sharp dip around x = 1, a classic red flag for floating-point issues.

Local Error Analysis: Clicking any point on the error plot opens a local error graph. Here, we see the expression’s abstract syntax tree (AST), which breaks down each operation. The subtraction node is highlighted in red, indicating that it’s the primary source of error. When we hover over this node, Odyssey shows the real and approximate values and explains that this is a cancellation error.

Understanding the Cause: Why is this happening? As x gets large, adding 1 barely affects the value due to the finite precision of floating points—it just rounds away. Essentially, we end up with sqrt(x) - sqrt(x) = 0. That small difference we expected is gone, and Odyssey shows us the resulting error.

Improving the Expression: Users can click the “Improve” button to ask Herbie to find a better expression, which generates alternatives. After a few moments, Odyssey displays the new options with metrics for accuracy and speed. In this case, suppose the user values accuracy the most. After considering all the alternatives, the user selects 1.0 / (sqrt(x) + sqrt(1.0 + x)). While it’s slightly slower than the original, its accuracy jumps from the original 53.3% to 99.8%, making it a much better choice.

This example showcases the iterative process that Odyssey promotes. Odyssey helps users understand where and why their expression has error, and find alternatives without needing to jump between different tools or guess where the problems might lie.

Recent Changes to Odyssey

The past few months have seen a lot of updates for Odyssey as we prepare for SC. Here are some highlights regarding new features and UI changes:

Pareto Plot: Odyssey now offers a Pareto plot to visualize trade-offs between accuracy and speedup. This feature is especially useful when users need to balance precision against performance. With the Pareto plot, users can visually identify the sweet spot that best fits their needs.

Expression Export: Users can now export optimized expressions to several programming languages. This feature bridges Odyssey’s capabilities with real-world applications, making it easy for developers to apply optimized expressions directly to their projects without additional adaptation.

Concrete Values and Error Explanation: Users can view concrete values for each operation node in the local error graph. Users can also see detailed explanations for errors, helping them understand why certain calculations fail. One of Odyssey’s goals is to provide users the tools to reason about their expression’s floating point behavior. Through these features, Odyssey acts as an analysis and teaching tool all in one.

User Interface Updates:I The Odyssey UI has come a long way! We’ve worked closely with UX designer Natalie Clark to ensure Odyssey’s layout is engaging and accessible for everyone, even those less familiar with floating-point analysis. Over the past few months, we’ve standardized fonts, sizes, colors, and spacing to create a clean, cohesive look. We’ve also simplified the navigation to make switching between MathJS and FPCore formats one click, emphasized important buttons/graphs, and improved visuals to give Odyssey a more professional, polished feel.

Preparing for SC 2024

With SC approaching soon, we’ve been hard at work iterating on our presentation and practicing.

Selecting Expressions for the Demo We’ve chosen expressions that highlight typical floating-point pitfalls, like the quadratic formula, which often falls victim to rounding errors when numbers differ greatly in magnitude. Odyssey’s ability to rewrite this formula to minimize error is one of the main points we’ll showcase, as it resonates with many who work in scientific computing. To see the rest of our expressions, come visit us in Atlanta!

Practicing the Presentation I’ve been building my communication skills in various ways: I obtained feedback on the delivery of my talk and further refined its content after a PLSE lunch talk, I practiced elevator-style pitches at the Allen School’s Industrial Affiliates Research Showcase and Open House, and wrote the blog post you’re reading right now!

Objectives

At SC, we’re aiming to:

Engage a Broader Audience: We want Odyssey to appeal to more than just floating point experts. By demonstrating how Odyssey simplifies error analysis, we hope to show its value for anyone across computing fields.

Gather Usability Feedback: SC is a great opportunity to learn from attendees about how Odyssey could fit into their workflows. Their insights will help us refine the tool and guide future development.

Explore Potential Integrations: With so many tool developers at SC, we’re excited to integrate other tools into Odyssey. Whether through direct integration or collaborative projects, we want to make Odyssey’s collection of tools more diverse.

Looking Ahead and Final Thoughts

SC is a big milestone, but it’s just the start. We want Odyssey become a standard tool across researchers and developers. Future plans include integrating even more tools, like Numerical Fuzz and GPU-FPX, conducting user studies, and potentially introducing Odyssey in a classroom setting.

For me, working on Odyssey has been both challenging and rewarding. I’ve learned a ton about floating point, UI design, and presenting research to different audiences. I’m excited to demo Odyssey at SC, see how it’s received, and incorporate any feedback that we get.

A huge thanks to Edward Misback, Zach Tatlock, Pavel Panchekha, and the Odyssey/Herbie team for all their help and guidance!