The Future of Artificial Intelligence

My final project report analyzing and evaluating the paper "Low Rank Sinkhorn Factorization" [ICML 2021] by Meyer Scetbon, Marco Cuturi and Gabriel Peyré. Conducted as part of the ENS MVA Mater of Science in the computational optimal transport course by Gabriel Peyré.

Very interesting class to discover optimal transport which can be interpreted as sorting in high dimensions but most concretely as a way to measure the difference between two probability distributions. The mathematical problem of transporting all the information from one distribution to another with minimal cost dates back to the 18th century and today helps design better deep learning models! For example, GANs were made more stable by utilizing transport distances in the adversarial models. There's more direct applications found in NLP for document similarity, shape matching through point cloud distributions and many others in biology. 

Specifically, the paper I review presents a new algorithm to efficiently solve a relaxed version of the original optimal transport problem. The authors assume transport between distributions in high dimensions can be broken into independent transports between smaller subsets of each distribution. Key advantages: works with any cost, more interpretable, breaks distributions into low ranks [which could potentially have applications beyond transport solving?]

Driven by the success of AlphaGo in defeating world champion Go players, Deep Mind researchers redesigned matrix multiplication as a board game, learnable through reinforcement learning. They trained AlphaTensor on this innovative state space. It successfully learned past matrix multiplication techniques and even designed its own, faster by 2 operations. This represents a huge breakthrough for artificial intelligence. It promises first acceleration of deep learning training as even a two step acceleration could lead to hundreds and even thousands less operations on large datasets. On a higher level, these results also demonstrate the capacity of machines to innovate beyond human learning. AlphaGo identified game playing strategies unknown to experts and for the first time GoogleAI was able to generalize this new learning to another domain. This innovation by AlphaTensor later improved on by Austrian researchers promises a future where machine and humans learn from each other.

Link to deep mind article discussing novel matrix multiplication optimization algorithm (AlphaTensor): https://www.deepmind.com/blog/discovering-novel-algorithms-with-alphatensor