Thoughts on Reference Management Software
Reference management software is an incredibly useful tool for researchers and academics. It solves two problems that are otherwise very frustrating: (1) organizing and searching PDFs of journal articles, and (2) properly formatting a bibliography when writing a paper with citations.1
I wrote the first version of this post back in 2015, and I’ve periodically updated it since then (see below).
The biggest change since 2015 is that some reference managers will now directly interface with Google Docs – if you were using Microsoft Word and “track changes” to collaborate on manuscripts, you should strongly consider switching to a Google Docs-based workflow.
The other big change since 2015 is that it’s feasible now to use a web-based reference manager without a native macOS/Windows app. This is due to improvements in web technology, like pdf.js.
I’ve added a summary of my current (April 2020) recommendations below. Under that are a list of all the updates I’ve made to the post since 2015 in reverse chronological order. The original post is preserved at the bottom, though it’s likely not very useful given the amount of time that has passed since it was written.
Last updated: May 19, 2020
Recommendations for most people
If you want a free application, Zotero is the way to go.
If you want really good Google Docs integration and don’t mind using Chrome for everything (their web app is Chrome-only), Paperpile is the way to go. As of April 2020 there are private betas of Microsoft Word plug-ins (Windows and macOS), and iOS and Android apps. Based on their maturity I expect they will be publicly available sometime in 2020.
$3/month for academics.
If you want native desktop and mobile apps, but don’t care about Google Docs integration, look at Papers.
This app started out as an indie Mac-only app, changed hands several times, and eventually landed at Digital Science, who also own Altmetric and figshare. It’s always had a great interface, though it has had stability problems in the past. I assume the stability issues have been worked out since I last used it extensively several years ago, but let me know if there are still problems.
$5/month for academics.
Avoid the following
- Any software that’s not a subscription. You want your reference management software to be maintained long-term, and a subscription is the best business model for ensuring financial stability. Free software is usually a bad idea because it’s not sustainable (except for Zotero, which is a non-profit and does have a subscription offering for expanded storage). Paid-up-front software must be comparatively expensive (see: EndNote at $250).
- Any software from a startup. Again, stability is key for reference management software, and startups are by definition not stable. I would ignore anything that hasn’t been around for at least 3 years.
- Any software that doesn’t have good shared library support. Research is collaborative, and it’s crazy to not be able to share references (and PDFs) easily. Zotero and Paperpile both support shared libraries.
- Mendeley. They are locking in users by restricting exports from their database, and are acting like jerks on Twitter. See the June 12, 2018 update for details.
Update: April 12, 2020
For the last few months I’ve been using Paperpile for all my reference management. This was driven in part by getting access to their beta Word plug-in and iOS app. With these, PaperPile now has these key features for my workflow:
- Google Docs integration for inserting citations.
- Seamless iOS sync of PDFs and annotations (via their beta app).
- Microsoft Word plug-in (via their beta app).
The iOS app is especially helpful for me as I like reading and annotating PDFs on my iPad. I find it much easier to use than the F1000 iPad app.
I don’t love the idea of relying primarily on a web application for reference management — what if their server goes down right before a deadline, or the company shuts down entirely? However, I’m willing to live with this risk given how convenient PaperPile’s platform is. I am using the feature to automatically back up the PDFs in PaperPile to Google Drive to give myself some peace of mind.
Update: November 18, 2019
- CiteAs is a simple tool for getting a properly-formatted citation for any DOI or – more importantly – open source software. Here’s an example for the
stringrR package. The URLs for CiteAs are nicely formatted (
stringrexample), making it easy to create a “Cite this page” bookmarklet.
Update: August 11, 2019
- I’ve been experimenting with F1000 Workspace and Paperpile again. F1000 has a decent (but not great) iOS application. Paperpile currently has an iOS application in private beta. I find the Paperpile web interface much more streamlined than F1000’s, and the F1000 iOS application is also clunky due to not being fully native. Once Paperpile’s iOS application is out, I expect I’ll be switching back.
- I recently found this post by Richard Gao from a year ago, which has a good overview of several popular reference managers. A lot of what he says still holds true today.
- Citationsy is a free service for easily building bibliographies, similar to ZoteroBib.
Update: January 28, 2019
I found two reference management services I’ve never heard of before:
I haven’t tried these out yet – please let me know if you have any experience with them.
I also ran across POLAR, which is not a reference manager per se but does have some nice functionality for working with a library of PDFs. From their website:
POLAR makes it easy to manage your reading.
A powerful document manager for Mac, Windows, and Linux for managing web content, books, and notes and supports tagging, annotation, highlighting and keeps track of your reading progress.
If you don’t care about inserting citations and automatically generating a bibliography, POLAR might be worth a look.
Update: January 23, 2019
- Added link to ZotFile, a nice third-party plugin for Zotero’s desktop app.
- I briefly tested out Zotero’s Google Docs integration and it seems to work fine.
- Thanks to Hacker News, I discovered a few new things:
Update: January 14, 2019
To make this page more user-friendly, I added a “Recommendations for most people” section that I will update separately from these updates.
My current application of choice for my personal use is Bookends, primarily because it has a decent iOS application and it keeps all the PDFs in a folder on iCloud, rather than hiding them in a proprietary database. I can even move PDFs into subfolders and the corresponding reference maintain its link.
I’d like to try Papers again, but I don’t have the bandwidth right now and no Google Docs integration is a deal-breaker for me. It looks like its landed on its feet at Digital Science, which is great – it’s the only reference manager I’ve seen that has native apps on iOS, macOS, and Windows!
Update: June 12, 2018
Mendeley appears to be doing some sketchy stuff to prevent exporting data. According to Zotero:
Mendeley 1.19 and later have begun encrypting the local database, making it unreadable by Zotero and other standard database tools. Mendeley made this change a few months after Zotero publicly announced work on an importer, despite having long touted the openness of its database format as a guarantee against lock-in. At the same time, Mendeley continues to import data from Zotero’s own open database.
(Zotero is a direct competitor to Mendeley (though Zotero is a non-profit), so take this with a grain of salt.)
In any case, this will lock users into Mendeley. This is a bad thing – this webpage is a testament to the volatility of reference management software, so it’s critical users can get their data out in the (quite likely) event a given software vendor goes belly up.
And I don’t buy Mendeley’s security argument. Most users are not sharing computers with untrustworthy people, and the operating system provides protections that seems sufficient for non-sensitive data like references.
This makes Mendeley a hard no for me until they provide a way to export all data to another reference management application.
Update: May 24, 2018
Zotero recently released ZoteroBib, which is a tool for quickly generating bibliographies from DOIs, PubMed IDs, etc. This is a great tool for generating a bibliography for a short article or getting references in the correct format for your CV, but it’s not a replacement for a full-fledged reference manager.
Update: January 13, 2018
Here’s what my current workflow looks like.
Update: January 2, 2018
It looks like Sente (discussed below) may have been abandoned. The developer’s website appears to be down and their Twitter account has been inactive since 2016.
Update: December 7, 2017
An anonymous reader pointed me to colwiz, a reference manager that was apparently started at Oxford. I haven’t tried it, but the marketing page makes it look like a viable alternative to something like Zotero.
Update: May 22, 2016
ReadCube is another reference manager worth looking at. They have web, mobile, and desktop apps, and also recently acquired Papers, which I cover in detail below. The consumer offerings from ReadCube are funded from their publisher services, so this is not a venture-funded consumer app company without a viable business model that is likely to disappear.
Update: September 14, 2015
I just heard about Flow, from the Refworks people (thanks Peyton!). Like Paperpile, it is web-based and integrates with Google Docs. I have not tried it yet myself, so if you use it let me know what you think.
Update: September 10, 2015
I have used Paperpile now, and it works pretty well. They have a beta Google Docs plugin that I use (rather than their Chrome extension; the Google Docs plugin is compatible with Safari, and I don’t like the security implications of browser extensions).
I’ve completely given up on organizing and searching my references in the same software I use to do citations. I now use DEVONthink for organizing and searching. DEVONthink has very advanced search capabilities, which solves my biggest reference management problem: finding papers that I remember reading, but do not remember the title or authors.
So I can’t speak to the reference organization and searching features of Paperpile, but inserting citations and building a list of references in a Google Docs file works just fine.
Paperpile’s killer feature is that it is built around Google Docs, which makes collaborative writing so much easier: everyone has the most current version of the document, and there is no manual merging of changes from multiple authors. The “suggesting mode” feature in Google Docs is now fairly comparable to track changes in Word, the bread and butter of collaborative academic writing.
Regardless of how you like to organize your references, I definitely think Parepile is worth a look just so you can use Google Docs. (Here’s how I use Google Docs with collaborators to avoid any hiccups with people who don’t regularly use Google accounts.)
Update: July 10, 2015
I’ve found two new reference management options since this post was originally published.
The first is Paperpile. This is Chrome extension that integrates reference management with Google Docs. I haven’t had a chance to try it in-depth, but if it works as advertised, it may be the holy grail of collaborative reference management.
The second is Cite This For Me. This is a tool for generating references section entries automatically from an article title or a DOI. If you end up having to make a references section manually, this is a very easy way to do it.
Original post (written in 2015)
I’ve preserved the original post from 2015 for posterity below, but the content is likely very out of date. Please see the updates above.
I’ve been using Zotero to organize and search my references for more than a year, so this is the application I have the most to say about.
- Free, cross-platform. Additional cloud storage is very cheap, and you probably won’t need it.
- Open source, maintained by a group at George Mason in a sustainable manner.
- Really good cloud sync (important for backups even if you don’t use Zotero on multiple computers).
- Good multi-user sharing features, including sharing of PDFs.
- Integration with Microsoft Word seems to work well (I haven’t used it a ton though).
- The PaperShip iOS and Mac app.
- The browser plugin will import citations and their accompanying PDFs from PubMed and many journal websites.
- Really fast full search of all PDFs in your library.
- You can add plugins like
- No good way I’ve found to track which articles need to be read, or mark articles for follow-up.
- Somewhat confusing reference organization capabilities (definitely read the documentation page on this before you try Zotero).
- Annotations in PDFs are kept separately from notes on a reference written in Zotero: you have to open the PDF to see them.
- Desktop application is based off of Firefox, so it’s more clunky than it would be as a fully native application. (Long-term, I think this reliance on XULRunner is a big liability. Users will increasingly expect native-feeling applications, which I doubt will be possible without a complete rewrite.)
- Web interface is mediocre.
- No internal PubMed search,
or way to add references without using the browser extension. Update: references can be quickly added by entering an ISBN, PMID, or DOI. This feature works great.
- Initial setup process can be very confusing for new users.
- No built-in PDF viewer/annotation. This works fine for me because I prefer to use the great PDF support built into OS X, but would definitely be a downside on Windows or for Mac users unfamiliar with Preview.app’s annotation support.
Bottom line: I think Zotero would be great if your reference management needs are lightweight, or you’re willing to put up with a few annoyances to use open source software. If you use notes and annotations heavily, or want read/unread/flagged status for references, look elsewhere.
Mendeley is probably the most widely-used modern reference manager. “Modern” in this context basically means that they have modern interfaces, take the web seriously, and have social features.
- Best web interface of any reference manager out there.
- Free account tier (limited storage).
- Widely used, especially among younger scientists.
- Sync seems to work well in my limited testing.
- Desktop interface is generally good.
- PDF notes displayed alongside notes taken in Mendeley, but viewing annotations still requires opening the PDF.
- Built-in PDF viewer has good annotation functions.
- First-time setup is easy.
- The PaperShip iOS and Mac app also works with Mendeley.
- Social features may be useful.
- Owned by Elsevier, which has a terrible reputation for supporting open reserach. This is a dealbreaker for me.
- No easy way to import PDFs (i.e. no feature like Zotero’s browser plugin that will automatically import PDFs along with the citation). This is also a dealbreaker for me, because importing PDFs quickly saves a huge amount of time.
- Additional storage is relatively expensive ($5/month for just 5gb of space; for comparison, Apple provides 20gb for $1/month).
- Mac app feels non-native in the same way Zotero does. It does not follow standard Mac application best practices. For example, the font size in the main view is smaller than the system font and can’t be adjusted.
- Social features may not be what you want in your reference managers.
Bottom line: If you can look past Elsevier’s reputation and find a PDF import workflow that works for you, this is a good option.
EndNote is probably the most widely used mature reference manager. By “mature,” I mean that it’s been around forever. I used to use EndNote back in college, but it’s changed substantially since then.
- Cross platform.
- Widely used, so there are citation formats for essentially every journal.
- Automatic PDF downloading. This feature is amazing when it works. You can select one or more references in EndNote, click a button, and it will automatically download the PDF into your library if you have access.
- Mac application interface is atrocious in recent versions. Watch the first 30 seconds of this video.
- Costs $220 with education pricing (!!!).
- Web sync is very difficult to set up.
- Web interface is also atrocious.
- Software updates only go one version at a time, so I had to run six upgrades to get from version 7.0 to 7.2. I’m sure there’s a work-around, but this is inexcusable default behavior in modern software.
Bottom line: Zotero and Mendeley both have much better interfaces for much less money. I would try one of these first before trying EndNote. The only stand-out feature in EndNote is automatic PDF downloads, but Zotero’s browser plugin is as good as this in my experience.
Papers used to be Mac-only, but recently released new comparable versions for both Mac and Windows. They also have an iOS app and a beta web interface. I used to use Papers prior to the latest major release and liked it, but had a ton of trouble upgrading to the new version so I switched to Zotero.
Unfortunately, I still hit numerous stability and performance issues even just using Papers for a few hours. Considering this is their flagship app on their original platform, I can’t recommend any of their apps.
- Hands down best desktop interface for both Windows and Mac.
- The Mac app is native and follows good Mac application design practices.
- Good built-in PubMed search interface, which can easily import PDFs.
- Still lots of bugs, performance issues, and crashes with v3 of the Mac app.
This is a web application site-licensed by my institution.
- Web-only, and the website feels dated by a decade or more.
Bottom line: Any application in this post is a better choice.
Sente (appears to be abandoned – see update above) are also options if you use a Mac.
Bookends has a ton of customization options, I found that I needed to do a fair amount of customization to use the bibliography builder, and that the process for doing this was not very intuitive. (I used it because my word processor integrates tightly with it.) It also is clunky in a number of other ways, including a lack of cloud sync. I don’t think there are any big advantages to using it over Zotero or Mendeley.
I really like Sente. It has my favorite sync system of any of the applications I tried2 and it is the only application I found that combines notes and PDF annotations in the main view. The UI is not as modern as Papers, but comparable in terms of usability. It has a steep learning curve for tagging and was difficult to set up with my library’s proxy server, but it has great documentation. If you can live without sync with Windows users,
I think Sente is a great choice (appears to be abandoned – see update above).
I think Zotero is the best general purpose choice. But if you can live with Mac and iOS only,
check out Sente (appears to be abandoned – see update above).
If you can ignore Elsevier’s bad reputation and don’t mind slower PDF import, Mendeley is also a good option.
Note that Mendeley has an option to stay in continuous sync with Zotero. You can use this feature to try out both applications without having to import references twice.
- There are a surprising number of academics who do not use a reference manager. I honestly don’t understand how they get anything done. Any of the applications I mention here is vastly better than keeping PDFs in folders (except maybe RefWorks). If you are one of these people, please try a reference manager. [return]
- Check out this video. This is an absolutely brilliant power user UI feature. [return]