Apart from the consideration what format the most people have, you should consider your wish to control over the layout of the text and how well the format can be converted to other formats by your readers.
Some formats like TXT allow very little control over the layout. In PDF you have much control over the layout, as you have in EPUB and MOBI.
Conversion of some formats is easier than others, the calibre FAQ gives a list of best formats to convert in decreasing preference: LIT, MOBI, AZW, EPUB, AZW3, FB2, DOCX, HTML, PRC, ODT, RTF, PDB, TXT, PDF. So if you would provide only PDF and TXT readers cannot easily provide good readable EPUBs for their device. In my experience the difference between the first 5 formats for input is minimal.
That leads me to recommend EPUB or MOBI as the formats if you target only EBooks and one of those and TXT if you want to capture the computer audience as well, but don't care about layout. Replace TXT with PDF (combined with EPUB or MOBI) if you care about the layout.
And then of course the software you use to generate the format has some influence on what you can do, but I assumed that you can generate the formats you indicated in your question in some way.