Svm only returns one type of category on prediction?

I am working on a bag of words, svm classifcation pipeline, and have two categories, the first with 6000 images and the second with 70. The test set is 30% of my data, and the training is 70%. When I try predicting with my test set, the svm believes everything is part of one category. My histograms contain 1000 bins. Am I doing something wrong with how I am approaching the problem? Thanks!