I think that embedding the font you want to use is your best option.
Many fonts have a set of greek (and/or non latin) letters, so you should be safe by just typing the letters, and they should be correctly displayed regardless of the chosen font.
But if, like you said, you want to include other special characters, embedding the full font could be the best solution (just be sure that the license of that font allows you to do it); the only drawback that I can think about is a slightly larger final file-size, but realistically speaking, a handful of kilobytes more should not be a problem for anyone.
Another thing that you should consider is that many ereaders have some degree of font customization settings; i.e. with my Kobo Glo, I can select from a list of installed fonts and this setting will override what you would have specified inside your ebook; then I could have some problems to visualize correctly your text, if the font I select doesn't support the characters you have used.
For this reason, whenever you are including a font that is required to correctly display some characters (and not just for aesthetic reasons, maybe to render a fancy title), you could add to your ebook something like a disclaimer page where you inform your readers that if they want to be sure to correctly visualize it, they should not use font customization with it.