Ultimately, it depends on which ereader devices you want to support.
Unless you are making a fixed layout book, the only method that will work 100% of the time is to make the caption a part of the image. Since epub and mobi are reflowable document formats, there is always a possibility that the device being used to view the document will push a text caption on the next page. However, devices do not usually split images across more than one page.
While I have used the text-in-image solution in the past, I don't personally recommend it, since images can be displayed at many different sizes and resolutions depending on the device or screen. It may be difficult for readers to read the caption while viewing on their phone, but the caption may look unusually large compared to body text on a computer screen. People using an e-ink reader may find the image text grainy or difficult to read. I agree that it is undesirable for the caption to get orphaned from its image, but I think that most readers of ebooks are accustomed to these kinds of text-flow failures, and would prefer the text being on the next page over not being able to read the text at all.
If you would like to do the best you probably can, you could try wrapping your image and caption in a <div>
with page-break-inside: avoid;
styling, but support for this CSS property is not universal, so you are still at the mercy of the device.
Some people prefer to use page-break-before: always;
before the image, but this also has drawbacks. (You can end up with only one or two lines of text on a page and then a forced page-break for the image.)
In the event that all of the devices you plan to support can handle SVG tags, you could wrap the image in <svg>
and include the text caption inside the SVG tag with the image. I have heard this works very well (but it requires SVG support).
If you're still interested in other more complicated ideas, you can check this thread on the MobileRead forums.