I am an ML practitioner, and I have always been drawn to the question of what happens under the hood inside a deep neural network. To explore that question hands-on, I started working through ARENA’s mechanistic interpretability exercises. I am writing this post to share my understanding as I develop it, beginning with one of the simplest interesting mechanisms found in transformers: induction circuits.

I watched the virtual car in a synthetic town rendered by a video game engine follow a predetermined path to reach its destination. Then I watched it again, and again and again. I wrote an eval script which counted exactly 6 successes out of 10. My unsophisticated, RL based model has finally learned to drive!

An important facet of science is replication, that is, being able to reproduce the results claimed in any scientific work. Independent testing, challenging and verification of published results not only add to the crediblity of the research, but also helps in democratizing scientific knowledge. Research based on machine learning should therefore be put up against the same verification process in order to be called science.

Recently, the researchers at Zalando, an e-commerce company, introduced Fashion MNIST as a drop-in replacement for the original MNIST dataset. Like MNIST, Fashion MNIST consists of a training set consisting of 60,000 examples belonging to 10 different classes and a test set of 10,000 examples. Each training example is a gray-scale image, 28x28 in size. The authors of the work further claim that the Fashion MNIST should actually replace MNIST dataset for benchmarking of new Machine Learning or Computer Vision models.