eBooks, filetype, and DRM

This morning I got a tweet from Bobbi Newman that said:

librarianbyday

Can someone explain to me the tech reasons Kindle doesn’t work with library ebooks, know its DRM, want more specific plz & thnx @griffey

More than you ever wanted to know about filetypes, DRM, and eBooks…here we go.

There are two different things going on when someone tries to open an eBook file on an eReader. One is filetype…how the file itself is organized internally, how the information contained within is encoded. This is analogous to the difference between a Word file saved as a .doc file, a Word file saved as a .docx file, and an Powerpoint file (.ppt). All are different filetypes…the program involved in the creation, editing, and display of those files describes the information contained inside. Right now, there are two main filetypes being used to describe eBook files: the Amazon eBook standard, or .amz file, and the ePub file (.epub) that is used by just about every other eBook vendor.

Amazon  purchased Mobipocket (an early ebook vendor/distributor) way back in 2005, and used their format as the basis for their current proprietary .amz filetype. ePub, on the other hand, is an open, XML based eBook standard, and is used by a huge number of eBook vendors…indeed, it’s easily the standard for current ebook publishing.

But filetype is only half the battle. In addition to the way the file is organized/structured internally, there is also Digital Rights Management to deal with. Think of DRM on an eBook as a lock, with your eReader having the key to open the lock and display the file. Without the lock, the eReader can’t open the file at all…can’t even see what it is. And if it has the key, but can’t read the filetype, that’s no good either…in that case, you can view the contents of the file, but will have no idea how to render it on the screen properly.

Amazon, in addition to using a proprietary filetype, also uses a proprietary DRM mechanism. This means in order to read an Amazon-purchased eBook, you have to have an eReader with the right key, as well as the right interpreter for the file. So far, that means that you have to be using a Kindle, or alternatively, using the Kindle software provided for any number of other devices (Windows, Mac, iOS devices, Android devices). This doesn’t mean that’s the way it has to be. Amazon could choose, tomorrow, to remove all DRM from their files. This would mean that you’d still need a program to interpret the .amz, but you wouldn’t need the key anymore. Conversely, Amazon could license their DRM to other eReaders, in effect handing them the key…but it would still be up to the eReader itself to be able to display the .amz file.

Vendors that use the ePub format have chosen different sorts of DRM to lock up their content. Apple and their iBook app use the ePub format, but wrap it up with their Apple-specific Fairplay DRM. This means that while the file itself would be readable by any device that can interpret an .epub file, without that particular key on their keyring, the eReader can’t do anything. Sony, Barnes & Noble, Overdrive, and other eBook vendors have chosen a shared DRM solution. They license their DRM from Adobe, and run Adobe Content servers that provide the keys to epub files that they sell. This means that if an eReader has the key to one of those stores, it has the key to all of them…think of it as a shared master key for any Adobe DRM’d file.

This illustrates why, although both Apple and B&N use epub as their filetype, you can’t buy a book from the B&N store and then move it over to your iBook app on your iPad. Conversely, you can’t buy something on the iBook store, and then move it to your Nook. Same filetype, different lock.

Overdrive, in supporting Adobe DRM’d epub files, work with Sony eReaders as well as the B&N Nook…same filetype, same DRM key to unlock them.

With all that said: any eReader that will read a given filetype will read said filetype if the file doesn’t have any DRM. So if you convert an existing document to an epub using software like Calibre, Sigil, or InDesign, that file will able to be read on a Nook, Sony Reader, AND the Apple iPad/iPhone/iPod Touch. If you have some text and you convert it to, say, a Mobipocket file (.mobi or .pdb) then it would be readable on the Kindle AND the Apple iBooks app…but not on the Nook. For a complete list of eReaders and their corresponding filetypes, there is no better place than Wikipedia’s Comparison of eBook Formats article.

While a DRM free eBook ecosystem would clearly be the best for the consumer (choice of device, free movement of files from device to device, etc), the second best option is an ecosystem where the DRM is ubiquitous and the patron doesn’t even realize it’s there. This was the case with Apple and the early battles for music sales on the ‘net…they had the store and the distribution network (iTunes) as well as the device used to access the content (iPod). All of the content was, originally, DRM’d, but largely no one noticed since it was completely invisible for the average user.

The biggest issue with eReaders and library patrons is that this chain isn’t seamless. The content providers and their DRM servers are huge headaches for the average eReader user. My hope is that publishing goes the same way that music did, we we find both a common filetype and lose the DRM. But it took digital music years and years to get there…so I’m not holding my breath.

I hope that helped, but if it didn’t and you still have specific questions about your situation with eReaders/eBooks, ask away in the comments and I’ll do my best to answer them.

48 thoughts on “eBooks, filetype, and DRM

  1. “My hope is that publishing goes the same way that music did, we we find both a common filetype and lose the DRM. But it took digital music years and years to get there…so I’m not holding my breath.”

    I would hope that the learning curve would be faster since the music industry did some of the heavy lifting for digital distribution and format. If they want to blaze a new trail, they are certainly welcome to; but I think there is a consumer expectation that will ferment over time to bring it in line with music formats.

  2. Actually I do have another question :-) So why can’t I put OverDrive epub files on my Kindle? It sounds like I *should* be able to load those files types. I know I can load PDFs. When I open Adobe Digital Editions that gives me the DRM and allows me to transfer the file to my device. Is it because Adobe Digital Editions doesn’t recognize the Kindle when its connected to the PC?

  3. *Great* explanation, Jason. Until someone convinces the publishing industry that an easy and ubiquitous DRM solution is a better alternative than piracy, I don’t think we’ll see much movement. It was Amazon and the other non-DRM vendors that forced the change with music; it’s too bad Amazon isn’t leading the way with books, too.

  4. Bobbi, you can’t use epub files on your Kindle because the Kindle just doesn’t include the software needed to read that format. See http://en.wikipedia.org/wiki/Comparison_of_e-book_formats#Supporting_Hardware for a table of what formats the Kindle can support. It’s not really to do with DRM or Adobe, it’s the decision of the people who made the Kindle.

    The ‘why’ behind that decision is probably: if they allow people to load epub-format books on their Kindles, then Kindle-owners might buy those instead of buying .amz-format books from Amazon.

  5. Bobbi, Deborah nailed it. You can’t use Overdrive on the Kindle because not only can the Kindle NOT read ePub files, it also can’t open the Adobe DRM.

    This has nothing to do with the device, necessarily…Amazon could issue a software update tomorrow that changes both of those things. But for now, you can’t. I mean, unless you break the DRM and convert the file to something that the Kindle will read.

  6. As long as Amazon has 70% of ebook sales, their format and DRM is a de facto standard and publishers cannot afford to pick sides in the DRM wars. Also there’s no incentive for Amazon to abandon their format/DRM for one controlled by Adobe.

    There probably is incentive for OverDrive and Amazon to add Amazon format/DRM to OD’s offerings (OD does support Mobipocket format/DRM, which Amazon owns and which is virtually the same as one of the Kindle’s 2 formats – but the selection is very limited).

    It’s not clear how this will work out, but I don’t expect DRM to go away until print edition sales erode significantly, and until then we’ll have 3 DRM islands: Adobe, Amazon, Apple. Of the 3, Apple’s seems the smallest and poorest (both in device support and content selection) and is unlikely to win much market share.

    Finally I would note that Adobe DRM has different versions, books that use the latest version (the one B&N is using) cannot be read on devices that use an older version. Some of those devices will be updated, but some undoubtedly won’t. Unless and until Adobe stops tweaking the DRM, this will be an ongoing issue.

  7. Wonderful stuff Jason. I just shared this with the eBook committee at my library. I hope it will help us answer the confusing questions we have about file types and DRM.

  8. Excellent post.

    Its a shame there is no way of currently getting a Library ebook service onto a Kindle or an iOS device, although there is the Overdrive app for that platform of course, which gives it a huge boost and no such joy on the Kindle.

  9. So thankful you posted this. I am working on an article on e-books for schools and this was very helpful.

    It also made me realize if we have a choice of devices for a school library, the Nook would provide more options than the Kindle. (now we can hope that B/N holds out!)

    I do hope that book publishers are quicker to catch onto this sea change than the music business was. To add to the mess, so many school publishers are rolling out their own e-book platforms. Your explanation will help many of us sort out the appropriate options. Thanks again.

  10. Excellent review of the current ebook environment. I’ve passed your post on to many in my library. Thanks!

  11. Hi! I am an intern at the Brooklyn Public Library and am working on their e-book offerings. This post and others linked here have been incredibly helpful. Will be digging into it more over the semester, but thanks for the headstart.

  12. “This was the case with Apple and the early battles for music sales on the ‘net…they had the store and the distribution network (iTunes) as well as the device used to access the content (iPod). All of the content was, originally, DRM’d, but largely no one noticed since it was completely invisible for the average user.”

    You can give this same argument for Amazon and its Kindle ecosystem. For the average user of the Kindle that buys books from Amazon’s store there is a seamless connection to the Kindle. You click the “1-click” button and 60 seconds later it shows up on your Kindle. That is why Amazon rose to the top of the eReader and eBook market. Add in the fact that you can send the same book to a variety of devices (iPhone, Android, Blackberry, PC, Mac, iPad), Amazon has created its own device agnostic ecosystem with a DRM that will be largely invisible to its users.

  13. So, can you use library epubs with Nook for iPad? The Overdive (used by library) seems to only offer options for audiobooks. How about iBooks?

  14. Brian: That’s a great question! Not currently, as the Nook for iPad doesn’t give you any way to move epub files into the program besides directly purchasing them from B&N…you can’t drop epub’s into the program in any way.

    In order to use a Nook to read library-provided ebooks from the Overdrive service, you have to be using the actual hardware Nook, and not just the app on the iPad.

  15. Thanks for an informative and well written post on this rather important topic.

    The excessive number of ebook formats and the hundreds of e-reading devices, all doing their own thing, is stopping the more careful buyer from fully embracing ebooks and the technologies involved.

    Hoping change takes place, and soon.

  16. As I understand it, most if not all of these DRM schemes have been compromised. Thus, the argument that DRM is needed to prevent organized criminals from engaging in massive infringement does not ring true. Organized crime already has the wherewithal to strip the DRM from any eBook and make as many copies as they care to.

    Perhaps the real reasons behind DRM are not as socially acceptable as combatting organized crime.

  17. I oversee the creation & uploading to Amazon/Apple et al. of our ebook offerings, and this is a fabulous explanation – new required reading for our ePub staff!

  18. Our Library has very cautiously entered the eBook fray with a small collection from NetLibrary (now owned by Ebsco). The titles are in PDF, which we thought the Kobo reader (reasonably popular in Canada) could read, as well as the Sony Reader. Turns out Kobo can’t read PDF very well. But Ebsco is promising more titles in epub in January, so…

  19. Thank you for the clear explanation. I hope it is ok if I quote you in an eBook handout I’m making for the staff at my library.

  20. Does this mean that any Adobe PDFs I have purchased independent of Amazon, will need to be converted somehow, so that I can load & read them on my Kindle?

    1. PDFs that do not have Adobe DRM on them can be read without any conversion on the Kindle. If you have purchased PDFs from a site that uses Adobe DRM, the Kindle won’t allow those to be displayed….you’ll have to find a way to remove the DRM before the Kindle will be able to display them.

  21. Nice write-up.

    A couple of typos.

    Amazon’s file extension is .azw (not .amz).

    Mobipocket comes as .mobi or .prc (not pdb).

    Note that prc was also used by Sony earlier. It is a container format that could be any number of things but is now so associated with mobi as to be considered interchangeable with it now.

  22. Jeff…thanks for the correction on the .azw thing. You’re right, .azw is the eBook filetype, and .amz is the file that Amazon uses for controlling music downloads. I just missed that completely.

    As to .pdb, it’s an older ebook format that is similar to mobi, and I still had a few hanging out. I just chose the wrong file to check the extension when I was writing. .prc is what I meant.

    Thanks again!

  23. I was considering a purchase of an ebook reader to read some of the hundreds of books in PDF and other file forms but after seeing these problems, it seems to much of a gamble to spend money for something that has an even chance of not working for me. I found an earlier Sony 300cs model for a fair price that was refurbed but even that likely would not work for me. I do have the Kindle PC software and do not like it and prefer to convert back to PDF to read on my computer. Thanks for the heads up, too much market control for me to get in yet.

Comments are closed.