Will expectation maximization always improve upon the last iteration until convergence?

I am trying to implement an expectation maximization algorithm, and my implementation tends towards the optimal solution, but the tedancy is not consistent (calculated from the kullback leibler divergence). Is that evidence that my algorithm is not correctly implemented?