Adding captions to images?

I obviously have an opinion about the right way to do this given the availability of the figure element in HTML5, but using span elements in conjunction with the aria-labelledby attribute will produce accessible, valid HTML.

So I think you are on the right track.