Skip to content

Community/Ecosystem

Einops tutorials cover multiple einops usages (and you'd better first follow tutorials), but it can also help to see einops in action.

Selected projects

Here are some open-source projects that can teach how to leverage einops for your problems

  • @lucidrains has a dramatic collection of vision transformers

    • there is a plenty of good examples how to use einops efficiently in your projects

  • lambda networks (non-conventional architecture) implemented by @lucidrains

    • nice demonstration how clearer code can be with einops, even compared to description in the paper
    • implementation and video

  • capsule networks (aka capsnets) implemented in einops

    • blazingly fast, concise (3-10 times less code), and memory efficient (3 times lower memory consumption) capsule networks, written with einops

  • NuX — normalizing flows in Jax

    • different rearrangement patterns in normalizing flows have nice mapping to einops

  • For video recognition, look at MotionFormer and TimeSFormer implementations

  • For protein folding, see implementation of invariant point attention from alphafold 2

Community introductions to einops

Tutorial in the AI summer about einops and einsum: https://theaisummer.com/einsum-attention/

Introduction to einops by Kapil Sachdeva https://www.youtube.com/watch?v=xGy75Pjsqzo

Implementing visual transformer in pytorch: https://towardsdatascience.com/implementing-visualttransformer-in-pytorch-184f9f16f632

Refactoring machine learning code, one of posts in a series is devoted to einops: https://www.paepper.com/blog/posts/refactoring-machine-learning-code-einops/

ML TLDR thread on einops: https://twitter.com/mlsummaries/status/1400505282543955970

Book "Deep Reinforcement Learning in Action" by Brandon Brown & Alexander Zai contains an introduction into einops in chapter 10.

Other einops-based projects worth looking at:

(ordered randomly)