The Lie derivative for measuring learned equivariance

Properties
authors	Nate Gruver, Marc Finzi, Micah Goldblum, Andrew Gordon Wilson
year	2022
url	https://arxiv.org/abs/2210.02984

Abstract

The Lie derivative is introduced, a method for measuring equivariance with strong mathematical foundations and minimal hyperparameters that finds that transformers can be more equivariant than convolutional neural networks after training, and that as models get larger and more accurate they tend to display more equivariance, regardless of architecture.

Notes