4 of 7 deleted 179 characters in body

Global assessment

Note: I am assuming that the poster of the question calls "unrestricted books" the books that have no DRM, i.e., no technical devices intended tp prevent some uses, such as copy, search, or adaptation to new reading devices. All books are born restricted by copyright, at least today, and very few fully remove copyright restrictions by providing adequate public licences.

The production of unrestricted books by Project Gutenberg and similar projects is tiny compared to raw digitization and OCR, as donne by Google and large public and academic projects, for example by national libraries. It is on the order of 100 000 books, a magnitude order of 1/1000th of the total number of books, that seems to range somewhere between 50 and 130 millions, depending on how books are counted. Large digitization projects like those led by Google, National Libraries and large institutions are about two orders of magnitude more productive, some producing several millions of digitized books, though exact numbers are sometimes hard to assess.

However, if you consider quality digitization with proofread text and structured formatting like the epub standard, efforts like the Project Gutenberg seem significantly more productive, as large digitization projects produce only a few thousands, unrestricted, quality e-books (from what I have been able to check).

This is not surprising since large digitization projects are intended mostly for preservation purposes, and for accessibility by the research community (or for indexation purposes in the case of Google), while Project Gutenberg and its likes are intended to make the literature available to a wider reading audience. Raw digitization for conservation performed by large project is fairly cheap and fast, while quality digitization require long and costly human intervention (about one order of magnitude higher), that can be well covered by crowd-proofreading and editing.

Another aspect is that, while the total number of books is very large, only a small fraction is of interest to a wide audience. So the relevant corpus for Project Gutenberg is probably much smaller than the relevant corpus for large digitization project that have to deal with all kinds of archives, inventories and the like. This is compounded by the fact that the books that are of interest to a wide audience are to a large extent still under copyright, hence much less likely to be made available legally without restrictions, by any digitization project.

This actually stresses the importance in the book count of orphan books, and more generally of books not cared for by rightholders, and whether they can, or cannot, be made legally available, with or without restrictions. This may represent on the order of half the books in copyright. In the USA, it is an important aspect of the legal battle around the Google Book Search Settlement Agreement, and it is the object of a specific law in France.

So, I would tend to believe that cooperative projects like the Project Gutenberg are the major source of unrestricted ebooks, if you consider reading resources for a wide public. But, it makes little sense to compare the output of two types of projects that have different purposes, at least in the medium term.

Still, it should be noted that preservation is probably a more urgent task than high quality digitization. It is rather sound that preservation projects will avoid diverting much of their limited resources on costly high quality output, other than for public relation purposes. Also, the mass of documents digitized by large projects provide a good source of raw material for smaller projects (in size) that are aiming for quality output that has a chance of attracting some readership.

In regard to the question of a reference source, I am leaving below my first attempt at analyzing the question, which shows that there are many sources of information, but too heterogeneous to be easily compiled into a single view with precise figures.

We should also remember that digitization is not the only source of ebooks. There are authors, publishers and institutions that produce new books without DRM restrictions, though there may be legal restrictions on use not enforced with DRM, as was the case before with classical paper books.

You may stop reading here

The complement below tells you more about my sources, and my analysis. Given the complexity of the question and the lack of standardized criteria, concepts and figures, and of uniform resources, some of my statements below may be disputable, or appear to be inconsistent: corrections are welcome. Please, keep this difficulty in mind, and don't shoot the pianist, he is doing the best he can.*

About digitization projects

This answer is based on my memories from work I did about 3 to 5 years ago. I do not really have time to do precise research on this right now or to retrieve the documents, but here are, from memory, some figures that should gives you some basis to assess the size of projects, and what can be expected from the overall body of written literature worldwide. Take these figures only as orders of magnitude. The way the size of book corpora is measured varies widely, depending on the sources, depending on how you define book, and on how you decide that two books are distinct. In many cases, this is not even made clear. Also, in some legal contexts, a document may be considered a book only if it has been commercialized through official publishing channels.

For the work I was doing, I could more or less maintain some consistency between the figures that were relevant to that work. But I am considering here a wider context, for which I do not have the time and resources to work this out. So please take this answer as an approximate state of things.

Digitization project vary widely in size (number of books digitized) and in quality. The quality can range from digital images of the pages to OCRed pages (OCR = optical character recognition) without proofreading, sometimes combined with the original image, to various ebook formats where the text has been proofread and its presentation carefully designed and reviewed. The cost of digitizing varies widely depending on the quality from a few tens of dollars to $500 for an average book, though it can go far beyond that price for some books (these are, from memory, figures I gathered a few years ago by asking various professionals).

A very large corpus of digitized books (8.8 millions) can be accessed on Internet Archive, which probably covers everything that is made public in the USA and much of the rest of the world. But it is not obvious to me how one can extract statistics about the projects that digitized these books, or about the quality of these books: they can all be downloaded in epub, but often not in any epub you care to have on your eReader (though I am impressed by what can be achieved purely mechanically).

It is actually hard to gather reliable information, as digitization is still a good business for some, and people are reluctant to publicize they real costs, or to be honest about quality. You often see documents, including official ones from government bodies, that maintain a deliberate confusion on those issues, in order to favor some policies or some interest groups.

Human intervention (proofreading, presentation) is the major source of cost to my knowledge, which is no surprise.

A project like Project Gutenberg is actually a rather small project in quantity. Their home page announces over 50 000 books of the world literature (100 000 with partner projects), and they actually have nearly 51 000 books. This is about 1/10th of the number of books currently in print (i.e. that you should be able to order in paper form) in a country like France (somewhere around 600 000, give or take 20% ... my figures a a few years old, and they different ways of counting). This number is also on the same order as the books still under copyright but no longer available in print (still for France). My guess is that should be more of less the same in other countries, with some variations due to country size and linguistic communities.

I have no reliable idea what is the number of books in the public domain, but it seems to be larger, given the number of public domain documents that are claimed to be digitized by some projects. But again, what is a book.

Overall, worldwide, I thought the total number of book is somewhere near the fifty millions mark, give or take some 30%. But then, it is not clear what counts as a book. Does a 10 pages novelette count as one? Does an international treaty, or a constitution count as one? That seems to often be the case in the Gutenberg project.

Looking on the web, 129 Million Different Books Have Been Published, according to Google. But what counts as two different books? (read the Google page on this). Two editions of the same book count as different. Then, how many Oliver Twist count as distinct books?

So assessing the Gutenberg project with these figures, we can estimate that it digitized on the order of 1/1000th of the existing books (and twice as much when adding partner projects), which is an achievement, given the quality attained.

There are other digitization projects similar to the Gutenberg project, and often cooperating with it, with the same size limitation (usually smaller).

The Google Books Library Project does digitization on a much wider scale, but its purpose seems to be foremost to make existing literature searchable, not to make it available in digital form, though they will do that when it is legal, i.e. for books in the public domain, or when the copyright owners will allow it. In order to be (more or less) searchable, the books only have to be OCRed, which does not means that the resulting files are convenient for ereading on usual eReaders.

Public domain books that have been digitized by Google are usually accessible either through Google, or through the libraries who collaborated with the project, or with Google under some other contractual form. That may still leave some hurdles when downloading books, as Google wants to benefit from the digitization (done at its own expense) as a competitive advantage when operating its search engine. So you should expect downloading to be at least protected by legal notes and technical means like CAPTCHA (or some other means to ensure it is only done by human beings), particularly so that mass downloading of these digital works is prevented, whether accessed from Google or from some of the collaborating institutions. I did not actually check experimentally on the web how this work, as my knowledge comes from discussions with some of the people involved, and from knowledge of some contractual clauses.

I am not sure how it is to be reconciled with the fact that all books on Internet Archive can apparently be freely downloaded in bulk. It may be that only a limited fraction of the books digitized are made available on Internet Archive.

An important issue, not definitely resolved, is whether orphan books (i.e. books with unreachable rightholders) may also be made available like public domain books. This is a messy issue, but it may concern a sizable part of in-copyright books (precise figures vary significantly), which are the more recent ones, but probably the part least sought for by the public.

Google dominates the digitization market because they can offer the service for free (up to a quality point) since they have their own advantage (for competing on the search engine market) in doing it.

According to wikipedia:

By the end of 2008, Google had reportedly digitized over seven million books, of which only about one million were works in the public domain. Of the rest, one million were in copyright and in print, and five million were in copyright but out of print.

Note that these figures do not match the proportions I gave for France (see above), and I do not know whether the difference comes from the way the Google books were selected (in library collections). There is also the fact that the Google figures probably include lots of duplication for the same title. This was propably reinforced by the fact that the Google library project was moderately good on metadata.

I did not find information regarding the current size (in 2015) of the Google digitization effort.

Some major digitization effort are conducted by publishers (sometimes in cooperation with Google), but only for books under copyright they control.

"The mass of unrestricted ebooks" is to be expected to come mostly from books that are in the public domain (and possibly from orphan books depending on legislation and court decisions). To my knowledge, the other existing major efforts are conducted by national libraries (like the Library of Congress in the USA or the BNF (Bibliothèque Nationale de France) in France, or sometimes by lesser (in size only) public libraries, administrations or academic bodies.

In the case of the French National Library, BNF, they have a large digitization project called Gallica, in collaboration with many other institutions, museums or libraries in particular.

Their home page boasts, among others, 630 000 books, 1.5 million periodicals, 40 000 musical scores, 66 000 manuscripts, 95 000 maps. However, they claim only "several thousands" of books in EPUB, which they chose (for structured representations) as it is an official international standard.

The site gives you access to ¨several millions documents", that you can download (use the search engine at the top of the page). These may be PDF, JPEG and/or EPUB, mainly.

But the selection of EPUB documents actually seems to claim only about 3000 EPUB books, of which about half are literary works.

Gallica itself contributes to Europeana, which is a similar project at the European level.

Thus you should expect to find very large digitization projects run by various governments to preserve the cultural heritage of their country, and smaller ones, often associated, run by various institutions. But this goes much beyond books and literature (it may even include architectural monuments) on the one hand, and is not necessarily intended for convenient use by the public (such as epub format for books) on the other hand.

This gives you an idea of the masses of books being digitized, but I have to leave it up to readers to explore the web for figures regarding specific institutions and projects.

This is also to be confronted with complex economic and policy issues. For example, some publishers may that their interest is to keep masses of books, no longer available in print, in a digitized format that discourages readership, so as to avoid market disruptions (they may also simply keep them unaccessible, if allowed by law). And governments may choose to support such policies. But whether this is in the public interest, or whether the underlying market analyses are sound, is beyond the scope of the question.

To summarize in a sentence, the production of unrestricted books by Project Gutenberg and similar projects is tiny if you consider raw digitization and OCR. It seems to be by far the largest part of the unrestricted ebooks if you want better quality digitization with structured formatting like the epub standard.

Beyond digitization

As aptly remarked in a comment by Ben Crowell, unrestricted ebooks do not necessarily arise from digitization. Some authors (e.g. Cory Doctorow) and some publishers (e.g. O'Reilly) do publish unrestricted ebooks. Here I am assuming that you actually mean unrestricted by DRM, that prevent various manipulations such as copying or other text-based manipulations, as opposed to legal restrictions imposed by copyright.

The books sold by O'Reilly are under copyright, and some uses are not permitted, even though this is not enforced mechanically. Similarly, Doctorow's book are under copyright and available for sale. But they are delivered with a Creative Common licence that allows to do about anything you want with them, including copies, as long as it is not commercial. You can get them for free from the author.

Many books are now made available without restrictions by DRM, and with very permissive licences, especially in the academic world, see for example "The Assayer" site.

This should not be confused with gratis ebooks that are available free of charge, but may be technically restricted by DRM. Many publishers do have some such books, used as bait to attract readership to their collections. Sometimes they are also without technical restrictions by DRM, though the use may be limited legally by the licence, which is always required since all books are born in-copyright.

2 related questions: