Library for out-of-the-box (Deep) Learning on Images?

I've got a set of images, each with an output label.

What libraries are out there that will let me load in my images and do classification? I mean libraries that automatically handle extracting features like color histogram, or SIFT/SURF, etc.?

Additionally, any libraries like this that do deep learning on images out of the box?