Question about test error in k-fold cross validation

For my particular data set, I am running k-fold cv for k = 3, 5, and 10. Most of the time, the error decreases as I progress from 3 to 5 and then to 10, but sometimes it decreases from 3 to 5 and then increases from 5 to 10. Does this mean anything? I would expect error to decrease. I have 28 subjects, if that makes any difference.

I'm currently an undergraduate, so any insight would be really helpful.