Does anyone have results for CIFAR-100 or Imagenet using fully connected (standard) neural networks instead of convolutional neural networks. I realize that this would generalize poorly and overfit because of the large number of parameters, but I'm somewhat curious about just how much worse they do relative to convnets and how that changes as the size of the images and the number of data points increases.
On MNIST fully connected neural networks are still somewhat close to convnets, but this is probably because the images have fairly consistent positions and are always at the center of the frame. The Deepface paper only used a single layer of convolutions and used locally connected layers after the max-pooling stage as the faces were pre-aligned to a canonical position.
[link][comment]