ImageNet Classification with Deep Convolutional Neural Networks

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012.

Models

Using Dataset

Implementation	Accuracy	Weights	Memory	Conv Ops	etc	link
Keras		35,659,688	35,659,688 * 4bytes			Keras
Tensorflow Slim		62,378,344	62,378,344 * 4bytes			Slim
Pytorch	90.1% (CIFAR 10, added BN)	58,323,690	58,323,690 * 4bytes			Pytorch

Tip & Trick

name	for What	reference
Relu	faster train / training deeper network	Paper
LRN	better generalization (replaced with BN later)	LRN in caffe
Overlapping Pooling	to avoid overfitting	-
Drop Out	to avoid overfitting	Paper
Color PCA augmentation	to avoid overfitting, data augmentation	-

Error of paper

• 224x224 is actually 227x227