I need some help finding features (xpost from /r/computervision)

I posted this on /r/computervision, but thought I might get some help here also.

Here are samples of my images: http://imgur.com/a/FDw3A#0

All the images look like that. It is a high speed industrial packaging printer, and I've processed the raw images to get what you see above.

I now need to identify the character. Just turning the images into a binary array representing the pixels and passing the data to a Random Forrest classifier is getting me about 49% recognition right now. As I understand it, it will be a better use of my time to do feature engineering instead of changing classification algorithms.

I've tried Haralick, Local Binary Patterns, Threshold Adjancency Statistics, and Zernike all without increased performance. SURF and SIFT are really out due to patents (also I believe the main advantage of these are scaling and rotation invariance) which I don't realy need.

EDIT:Confusion Matrix