The Triangle Inequality in the Evoformer is a Misnomer

---

The triangle inequality is a fundamental geometric identity. Add two sides of a triangle and the resulting sum is greater than the third side. Or more generally, given any three points in a plane, the sum of the distances of two different pairs is greater than or equal to the distance between the third pair.

In the Evoformer; a module present in some protein structure models including the Nobel Prize winning AlphaFold, a technique or submodule called the triangle inequality is used to incorporate geometrical laws into the model to make the model learn more efficiently.

Now, given the name of the technique, “Triangle Inequality”, your first thought might be the plugging the famous inequality into the model somehow.

$$a_1 + a_2 > a_3$$

where $a_1, a_2, a_3$ are the lengths of a triangle.

But no, it doesn’t work that way. It’s engineered using the same principle many post-transformer models use - Learn it!

$$z'_{ij} = \sum\limits_k a_{ik} \odot b_{jk}$$

where

$$a_{ik}=W_a z_{ik},\qquad b_{jk}=W_b z_{jk}.$$

with $z_{ij}$ being the pair representation and $W_a$, $W_b$, learned weights which makes the operation assymetric and therefore we have two passes for both directions(inward and outward). The architecture is such that the relationship between 3 points are learned during training so that we can’t confidently say it’s using the triangle inequality as is. We, however, could definitely assume that it’s hidden somewhat within the learned principle but figuring out the learned principle isn’t trivial. I haven’t looked into this yet but we could possibly observe new or derive existing physical rules of protein or chemical geometry if we could somehow convert the vectors into pure math.

Feel free to make a comment or star the article if you enjoyed it.