Loading page…
Its November 13 write-up extends the picture beyond an October paper’s H100 results. TurboPairFormer targets costly molecular calculations while improving numerical repeatability.
Listen to this story
TurboPairFormer’s gains come from changing how triangle attention is computed during training: it reuses intermediate work in the backward pass and compensates for information lost in BF16 rounding. Lambda’s November 13 H200 results add a hardware datapoint for OpenFold3 fine-tuning, but the reported speed is for training steps, not end-to-end elapsed time. The software supports H100 and H200 GPUs; comparisons against cuEquivariance, not only Triton, are important when assessing its practical advantage.
At crop size 768 on H200, TurboPairFormer took 14.67 seconds per step versus 22.53 seconds for OpenFold3 Triton, a 1.54× speedup.
In the October H100 paper, TurboPairFormer was 1.73× faster than Triton but only 1.13× faster than NVIDIA cuEquivariance at crop size 768.
Those H100 timings cover GPU computation on 16 GPUs and exclude data loading, transfers, and EMA updates.
Lambda’s November 13 write-up reports that TurboPairFormer ran OpenFold3 training steps 1.54× faster than its Triton backend on NVIDIA H200 GPUs. Built with Stevens Institute of Technology and the OpenFold team, the open-source software targets expensive molecular-structure calculations. The H200 figures complement H100 benchmarks in the team’s October research paper.
TurboPairFormer implements triangle attention and triangle multiplication, building blocks for predicting molecular structures. Triangle attention connects information about two molecular components through a third. Its computing cost grows cubically: doubling the training crop size makes this operation about eight times as expensive.
The implementation changes the backward pass, which calculates gradients used in training. Rather than repeatedly recomputing attention probabilities for different gradients, it recomputes each small block once and reuses it for all four gradient types: query, key, value, and pair bias.
A second change preserves information normally lost to rounding. Converting outputs from the higher-precision FP32 number format to lower-precision BF16 discards detail. TurboPairFormer saves an approximation of that remainder and uses it to improve gradient calculations, without changing the BF16 output itself.
The team’s research paper, submitted October 5, reported the lowest GPU computation time among eight OpenFold3 backend configurations on 16 H100 GPUs. At crop size 768, TurboPairFormer was 1.73× faster than OpenFold3 Triton, but a smaller 1.13× faster than NVIDIA cuEquivariance.
Those H100 timings measure GPU computation, not complete elapsed training time. They cover optimizer steps 21 through 200 and exclude data loading, transfers to GPUs, and EMA model-averaging updates.
Against a high-precision FP64 reference, the authors reported the lowest mean query, key, and pair-bias gradient errors among tested implementations. Controlled experiments found that retaining the rounding remainder reduced those errors by 28–47%. Value-gradient accuracy was essentially tied with cuEquivariance.
Repeatability comes from adding gradient contributions in a fixed order. On a single H100, the team repeated 600 fixed-input cases five times each. All four gradients matched bit for bit in every case under fixed execution conditions.
TurboPairFormer currently supports H100 and H200 GPUs. Researchers can install it from PyPI, with an OpenFold3 quickstart guide supplied by Lambda:
pip install turbopairformerLoading discussion...
Join the conversation
Explain whether isolated GPU speed is enough to judge a training tool.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.