I watched the virtual car in a synthetic town rendered by a video game engine follow a predetermined path to reach its destination. Then I watched it again, and again and again. I wrote an eval script which counted exactly 6 successes out of 10. My unsophisticated, RL based model has finally learned to drive!

An important facet of science is replication, that is, being able to reproduce the results claimed in any scientific work. Independent testing, challenging and verification of published results not only add to the crediblity of the research, but also helps in democratizing scientific knowledge. Research based on machine learning should therefore be put up against the same verification process in order to be called science.

Recently, the researchers at Zalando, an e-commerce company, introduced Fashion MNIST as a drop-in replacement for the original MNIST dataset. Like MNIST, Fashion MNIST consists of a training set consisting of 60,000 examples belonging to 10 different classes and a test set of 10,000 examples. Each training example is a gray-scale image, 28x28 in size. The authors of the work further claim that the Fashion MNIST should actually replace MNIST dataset for benchmarking of new Machine Learning or Computer Vision models.