There are actually lots of sources for books that are DRM-free.
Some provide free books (i.e. books that you do not have to pay for),
and others are just publishers who choose to sell books without DRM,
and trust you for not giving away copies, since they need to sell the
books to make a living. It may also be authors who sell or
give their books directly on the Internet.
For example, many retailers like Kobo, Smashwords or Amazon have most
books for sale, but also have books available for free. Both kinds may
be with or without DRM. Amazon uses private standards which, though
not DRM stricto-sensu, may be very inconvenient for some users (such
as myself, using Linux and Android) unless you run Windows or a Mac.
Where to look depends very much on the kind of books you are looking
for. For example, I am interested in science fiction in English and
classical literature in French.
Sites I would recommend for a start are Project Gutenberg and Internet
Archive (see sites: Project Gutenberg - Internet
Archive)
Macmillan is a major publisher who sells a lot of books
without DRM. A lot of smaller publishers do that too (such as Baen,
Phoenix Pick, Wildside Press and others in Science Fiction and
Fantasy). Most occasionally give books away for free.
Book bundles sellers, such as HumbleBundle and StoryBundle, sell bundles of books that are usually DRM-free (in several years I encountered only one bundle that was under some kind of DRM control, and the seller actually accepted to reimburse me with no fuss). These can be very interesting sources of good books that are otherwise sold only with DRM. And they are usually rather cheap, compared to their standard price.
Most books in the public domain can be found for free and/or without DRM if you look hard
enough.
Note that when books have no hard DRM (the kind that may be bothering
you), they may still have watermarks, i.e. unique identifiable
characteristics, so that the origin can be traced when those books are
not supposed to be redistributed by the purchaser.