Categories
Books Gadgets Media Technology

eBooks, filetype, and DRM

This morning I got a tweet from Bobbi Newman that said:

librarianbyday

Can someone explain to me the tech reasons Kindle doesn’t work with library ebooks, know its DRM, want more specific plz & thnx @griffey

More than you ever wanted to know about filetypes, DRM, and eBooks…here we go.

There are two different things going on when someone tries to open an eBook file on an eReader. One is filetype…how the file itself is organized internally, how the information contained within is encoded. This is analogous to the difference between a Word file saved as a .doc file, a Word file saved as a .docx file, and an Powerpoint file (.ppt). All are different filetypes…the program involved in the creation, editing, and display of those files describes the information contained inside. Right now, there are two main filetypes being used to describe eBook files: the Amazon eBook standard, or .amz file, and the ePub file (.epub) that is used by just about every other eBook vendor.

Amazon  purchased Mobipocket (an early ebook vendor/distributor) way back in 2005, and used their format as the basis for their current proprietary .amz filetype. ePub, on the other hand, is an open, XML based eBook standard, and is used by a huge number of eBook vendors…indeed, it’s easily the standard for current ebook publishing.

But filetype is only half the battle. In addition to the way the file is organized/structured internally, there is also Digital Rights Management to deal with. Think of DRM on an eBook as a lock, with your eReader having the key to open the lock and display the file. Without the lock, the eReader can’t open the file at all…can’t even see what it is. And if it has the key, but can’t read the filetype, that’s no good either…in that case, you can view the contents of the file, but will have no idea how to render it on the screen properly.

Amazon, in addition to using a proprietary filetype, also uses a proprietary DRM mechanism. This means in order to read an Amazon-purchased eBook, you have to have an eReader with the right key, as well as the right interpreter for the file. So far, that means that you have to be using a Kindle, or alternatively, using the Kindle software provided for any number of other devices (Windows, Mac, iOS devices, Android devices). This doesn’t mean that’s the way it has to be. Amazon could choose, tomorrow, to remove all DRM from their files. This would mean that you’d still need a program to interpret the .amz, but you wouldn’t need the key anymore. Conversely, Amazon could license their DRM to other eReaders, in effect handing them the key…but it would still be up to the eReader itself to be able to display the .amz file.

Vendors that use the ePub format have chosen different sorts of DRM to lock up their content. Apple and their iBook app use the ePub format, but wrap it up with their Apple-specific Fairplay DRM. This means that while the file itself would be readable by any device that can interpret an .epub file, without that particular key on their keyring, the eReader can’t do anything. Sony, Barnes & Noble, Overdrive, and other eBook vendors have chosen a shared DRM solution. They license their DRM from Adobe, and run Adobe Content servers that provide the keys to epub files that they sell. This means that if an eReader has the key to one of those stores, it has the key to all of them…think of it as a shared master key for any Adobe DRM’d file.

This illustrates why, although both Apple and B&N use epub as their filetype, you can’t buy a book from the B&N store and then move it over to your iBook app on your iPad. Conversely, you can’t buy something on the iBook store, and then move it to your Nook. Same filetype, different lock.

Overdrive, in supporting Adobe DRM’d epub files, work with Sony eReaders as well as the B&N Nook…same filetype, same DRM key to unlock them.

With all that said: any eReader that will read a given filetype will read said filetype if the file doesn’t have any DRM. So if you convert an existing document to an epub using software like Calibre, Sigil, or InDesign, that file will able to be read on a Nook, Sony Reader, AND the Apple iPad/iPhone/iPod Touch. If you have some text and you convert it to, say, a Mobipocket file (.mobi or .pdb) then it would be readable on the Kindle AND the Apple iBooks app…but not on the Nook. For a complete list of eReaders and their corresponding filetypes, there is no better place than Wikipedia’s Comparison of eBook Formats article.

While a DRM free eBook ecosystem would clearly be the best for the consumer (choice of device, free movement of files from device to device, etc), the second best option is an ecosystem where the DRM is ubiquitous and the patron doesn’t even realize it’s there. This was the case with Apple and the early battles for music sales on the ‘net…they had the store and the distribution network (iTunes) as well as the device used to access the content (iPod). All of the content was, originally, DRM’d, but largely no one noticed since it was completely invisible for the average user.

The biggest issue with eReaders and library patrons is that this chain isn’t seamless. The content providers and their DRM servers are huge headaches for the average eReader user. My hope is that publishing goes the same way that music did, we we find both a common filetype and lose the DRM. But it took digital music years and years to get there…so I’m not holding my breath.

I hope that helped, but if it didn’t and you still have specific questions about your situation with eReaders/eBooks, ask away in the comments and I’ll do my best to answer them.

By griffey

Jason Griffey is the Director of Strategic Initiatives at NISO, where he works to identify new areas of the information ecosystem where standards expertise is useful and needed. Prior to joining NISO in 2019, Jason ran his own technology consulting company for libraries, has been both an Affiliate at metaLAB and a Fellow and Affiliate at the Berkman Klein Center for Internet & Society at Harvard University, and was an academic librarian in roles ranging from reference and instruction to Head of IT at the University of TN at Chattanooga.

Jason has written extensively on technology and libraries, including multiple books and a series of full-periodical issues on technology topics, most recently AI & Machine Learning in Libraries and Library Spaces and Smart Buildings: Technology, Metrics, and Iterative Design from 2018. His newest book, co-authored with Jeffery Pomerantz, will be published by MIT Press in 2024.

He has spoken internationally on topics such as artificial intelligence & machine learning, the future of technology and libraries, decentralization and the Blockchain, privacy, copyright, and intellectual property. A full list of his publications and presentations can be found on his CV.
He is one of eight winners of the Knight Foundation News Challenge for Libraries for the Measure the Future project (http://measurethefuture.net), an open hardware project designed to provide actionable use metrics for library spaces. He is also the creator and director of The LibraryBox Project (http://librarybox.us), an open source portable digital file distribution system.

Jason can be stalked obsessively online, and spends his free time with his daughter Eliza, reading, obsessing over gadgets, and preparing for the inevitable zombie uprising.

48 replies on “eBooks, filetype, and DRM”

“My hope is that publishing goes the same way that music did, we we find both a common filetype and lose the DRM. But it took digital music years and years to get there…so I’m not holding my breath.”

I would hope that the learning curve would be faster since the music industry did some of the heavy lifting for digital distribution and format. If they want to blaze a new trail, they are certainly welcome to; but I think there is a consumer expectation that will ferment over time to bring it in line with music formats.

Actually I do have another question 🙂 So why can’t I put OverDrive epub files on my Kindle? It sounds like I *should* be able to load those files types. I know I can load PDFs. When I open Adobe Digital Editions that gives me the DRM and allows me to transfer the file to my device. Is it because Adobe Digital Editions doesn’t recognize the Kindle when its connected to the PC?

*Great* explanation, Jason. Until someone convinces the publishing industry that an easy and ubiquitous DRM solution is a better alternative than piracy, I don’t think we’ll see much movement. It was Amazon and the other non-DRM vendors that forced the change with music; it’s too bad Amazon isn’t leading the way with books, too.

Bobbi, you can’t use epub files on your Kindle because the Kindle just doesn’t include the software needed to read that format. See http://en.wikipedia.org/wiki/Comparison_of_e-book_formats#Supporting_Hardware for a table of what formats the Kindle can support. It’s not really to do with DRM or Adobe, it’s the decision of the people who made the Kindle.

The ‘why’ behind that decision is probably: if they allow people to load epub-format books on their Kindles, then Kindle-owners might buy those instead of buying .amz-format books from Amazon.

Bobbi, Deborah nailed it. You can’t use Overdrive on the Kindle because not only can the Kindle NOT read ePub files, it also can’t open the Adobe DRM.

This has nothing to do with the device, necessarily…Amazon could issue a software update tomorrow that changes both of those things. But for now, you can’t. I mean, unless you break the DRM and convert the file to something that the Kindle will read.

As long as Amazon has 70% of ebook sales, their format and DRM is a de facto standard and publishers cannot afford to pick sides in the DRM wars. Also there’s no incentive for Amazon to abandon their format/DRM for one controlled by Adobe.

There probably is incentive for OverDrive and Amazon to add Amazon format/DRM to OD’s offerings (OD does support Mobipocket format/DRM, which Amazon owns and which is virtually the same as one of the Kindle’s 2 formats – but the selection is very limited).

It’s not clear how this will work out, but I don’t expect DRM to go away until print edition sales erode significantly, and until then we’ll have 3 DRM islands: Adobe, Amazon, Apple. Of the 3, Apple’s seems the smallest and poorest (both in device support and content selection) and is unlikely to win much market share.

Finally I would note that Adobe DRM has different versions, books that use the latest version (the one B&N is using) cannot be read on devices that use an older version. Some of those devices will be updated, but some undoubtedly won’t. Unless and until Adobe stops tweaking the DRM, this will be an ongoing issue.

Excellent post.

Its a shame there is no way of currently getting a Library ebook service onto a Kindle or an iOS device, although there is the Overdrive app for that platform of course, which gives it a huge boost and no such joy on the Kindle.

So thankful you posted this. I am working on an article on e-books for schools and this was very helpful.

It also made me realize if we have a choice of devices for a school library, the Nook would provide more options than the Kindle. (now we can hope that B/N holds out!)

I do hope that book publishers are quicker to catch onto this sea change than the music business was. To add to the mess, so many school publishers are rolling out their own e-book platforms. Your explanation will help many of us sort out the appropriate options. Thanks again.

Excellent review of the current ebook environment. I’ve passed your post on to many in my library. Thanks!

Hi! I am an intern at the Brooklyn Public Library and am working on their e-book offerings. This post and others linked here have been incredibly helpful. Will be digging into it more over the semester, but thanks for the headstart.

“This was the case with Apple and the early battles for music sales on the ‘net…they had the store and the distribution network (iTunes) as well as the device used to access the content (iPod). All of the content was, originally, DRM’d, but largely no one noticed since it was completely invisible for the average user.”

You can give this same argument for Amazon and its Kindle ecosystem. For the average user of the Kindle that buys books from Amazon’s store there is a seamless connection to the Kindle. You click the “1-click” button and 60 seconds later it shows up on your Kindle. That is why Amazon rose to the top of the eReader and eBook market. Add in the fact that you can send the same book to a variety of devices (iPhone, Android, Blackberry, PC, Mac, iPad), Amazon has created its own device agnostic ecosystem with a DRM that will be largely invisible to its users.

So, can you use library epubs with Nook for iPad? The Overdive (used by library) seems to only offer options for audiobooks. How about iBooks?

Brian: That’s a great question! Not currently, as the Nook for iPad doesn’t give you any way to move epub files into the program besides directly purchasing them from B&N…you can’t drop epub’s into the program in any way.

In order to use a Nook to read library-provided ebooks from the Overdrive service, you have to be using the actual hardware Nook, and not just the app on the iPad.

Thanks for an informative and well written post on this rather important topic.

The excessive number of ebook formats and the hundreds of e-reading devices, all doing their own thing, is stopping the more careful buyer from fully embracing ebooks and the technologies involved.

Hoping change takes place, and soon.

As I understand it, most if not all of these DRM schemes have been compromised. Thus, the argument that DRM is needed to prevent organized criminals from engaging in massive infringement does not ring true. Organized crime already has the wherewithal to strip the DRM from any eBook and make as many copies as they care to.

Perhaps the real reasons behind DRM are not as socially acceptable as combatting organized crime.

Our Library has very cautiously entered the eBook fray with a small collection from NetLibrary (now owned by Ebsco). The titles are in PDF, which we thought the Kobo reader (reasonably popular in Canada) could read, as well as the Sony Reader. Turns out Kobo can’t read PDF very well. But Ebsco is promising more titles in epub in January, so…

Does this mean that any Adobe PDFs I have purchased independent of Amazon, will need to be converted somehow, so that I can load & read them on my Kindle?

PDFs that do not have Adobe DRM on them can be read without any conversion on the Kindle. If you have purchased PDFs from a site that uses Adobe DRM, the Kindle won’t allow those to be displayed….you’ll have to find a way to remove the DRM before the Kindle will be able to display them.

Nice write-up.

A couple of typos.

Amazon’s file extension is .azw (not .amz).

Mobipocket comes as .mobi or .prc (not pdb).

Note that prc was also used by Sony earlier. It is a container format that could be any number of things but is now so associated with mobi as to be considered interchangeable with it now.

Jeff…thanks for the correction on the .azw thing. You’re right, .azw is the eBook filetype, and .amz is the file that Amazon uses for controlling music downloads. I just missed that completely.

As to .pdb, it’s an older ebook format that is similar to mobi, and I still had a few hanging out. I just chose the wrong file to check the extension when I was writing. .prc is what I meant.

Thanks again!

[…] Digital Rights Management—which are access control technologies that are used by hardware manufacturers, publishers, copyright holders, and individuals to limit the use of digital content and devices. **This is why Amazon’s Kindle does not work well if at all with libraries digital software (mainly Overdrive), why an ebook you want is not available or the waiting list is so long for it (libraries can buy titles, but they only come with so many licenses for viewing at the same time—or certain books are not yet available in digital format and sometimes never will be) More on explaining DRM can be found here  […]

[…] The first question is harder, but the answer is that ePub on Kindle doesn’t mean anything to libraries or to patrons. Why? Our good friend DRM. It is extraordinarily unlikely that Amazon will replace their own DRM with that used by the library industry (and by others in the eBook world), Adobe Digital Editions. If you aren’t familiar with the various DRM used in the industry, I’ll refer you to my previous post on eBooks, Filetype, and DRM. […]

I was considering a purchase of an ebook reader to read some of the hundreds of books in PDF and other file forms but after seeing these problems, it seems to much of a gamble to spend money for something that has an even chance of not working for me. I found an earlier Sony 300cs model for a fair price that was refurbed but even that likely would not work for me. I do have the Kindle PC software and do not like it and prefer to convert back to PDF to read on my computer. Thanks for the heads up, too much market control for me to get in yet.

Leave a Reply

Your email address will not be published. Required fields are marked *