The explanation below may only make sense to you if you read this from an RSS reader. If you don’t please skip it.

Have you ever sent an email to the wrong person? Did wish you could pull it back? I just did this, but for our blog (!).

I was feeding a WP install of what will be our company blog, and then I did something nasty…

I was about to post from my desktop tool (word 2010, used to be live writer), and I hit ‘post’ on the wrong doc, and ‘about’ page, ‘google-translated’ from German; prose as horrible as one can get. The blog selected was academicproductivity.com, not our own. I immediately went to the admin page and removed it, so ap.com’s readers won’t see it. But the RSS feed… is another story. For a blog that was dead for a year, we still have >4000 followers. This was the first post to break a long silent stretch. I’d hate if you, the reader, think we resurrected, just to find out a nonsensical blog post.

Since we use feedburner, removing the post locally didn’t help. I had to log in at feedburner, and try to remove it from there. They have a ‘nuke’ option that should force a refresh. But it just didn’t work. I tried a few times, the nonsensical blog post was still there. The only option I could think of was to delete the feed from feedburner, in the hope that they do not broadcast it. But it was too late; all people who subscribe to ap.com’s RSS feed have received the post.

I apologize for kidnapping your attention without a good reason. I’m sorry you got involved in this, but silly mistakes do occur. I will be more careful in the future.

Since the feedburner feed is now gone, if you want to continue receiving updates from ap.com you’d need to resubscribe. Simply click again on the RSS icon on the address bar, and follow the steps there. In any case, we are not dead, and will continue writing for ap.com whenever we find something worth writing about.

Readers of this blog are not new to my ramblings on soft peer review, social metrics and post-publication impact measures:

• can we measure the impact of scientific research based on usage data from collaborative annotation systems, social bookmarking services and social media?
• should we expect major discrepancies between citation-based and readership-based impact measures?
• are online reference management systems more robust a data source to measure scholarly readership than traditional usage factors (e.g. downloads, clickthrough rates etc.)?

These are some of the questions addressed in my COOP ’08 paper. Jason Priem also discusses the prospects of what he calls “scientometrics 2.0″ in a recent First Monday article and it is really exciting to see a growing interest in these ideas from both the scientific and the STM publishing community.

We now need to think of ways of putting these ideas into practice. Science Online London 2010 earlier this month offered a great chance to test a real-world application of these ideas in front of a tech-friendly audience and this post is meant as its official announcement.

ReaderMeter is a proof-of-concept application showcasing the potential of readership data obtained from reference management tools. Following the announcement of the Mendeley API, I decided to see what could be built on top of the data exposed by Mendeley and the first idea was to write a mashup aggregating author-level readership statistics based on the number of bookmarks scored by each of one’s publications. ReaderMeter queries the data provider’s API for articles matching a given author string. It parses the response and generates a report with several metrics that attempt to quantify the relative impact of an author’s scientific production based on its consumption by a population of readers (in this case the 500K-strong Mendeley user base):

The figure above shows a screenshot of ReaderMeter’s results for social scientist Duncan J Watts, displaying global bookmark statistics, the breakdown of readers by publication as well as two indices (the H_R index and the G_R index) which I compute using bookmarks as a variable by analogy to the two popular citation-based metrics. Clicking on a reference allows you to drill down to display readership statistics for a given publication, including the scientific discipline, academic status and geographic location of readers of an individual document:

A handy permanent URL is generated to link to ReaderMeter’s author reports (using the scheme: [SURNAME].[FORENAME+INITIALS]), e.g.:

http://readermeter.org/Watts.Duncan_J

I also included a JSON interface to render statistics in a machine-readable format, e.g.:

http://readermeter.org/Watts.Duncan_J/json

Below is a sample of the JSON output:

{
	"author": "Duncan J Watts",
	"author_metrics":
	{
		"hr_index": "15",
		"gr_index": "26",
		"single_most_read": "140",
		"publication_count": "57",
		"bookmark_count": "760",
		"data_source": "mendeley"
	},
	"source": "http://readermeter.org/Watts.Duncan_J",
	"timestamp": "2010-09-02T15:41:08+01:00"
}

Despite being just a proof of concept (it was hacked in a couple of nights!), ReaderMeter attracted a number of early testers who gave a try to its first release. Its goal is not to redefine the concept of research impact as we know it, but to complement this notion with usage data from new sources and help identify aspects of impact that may go unnoticed when we only focus on traditional, citation-based metrics. Before a mature version of ReaderMeter is available for public consumption and for integration with other services, though, several issues will need to be addressed.

1. Author name normalisation

The first issue to be tackled is the fact the same individual author may be mentioned in a bibliographic record under a variety of spelling alternates: Rod Page was among the first to spot and extensively discuss this issue, which will hopefully be addressed in the next major upgrade (unless a provision to fix this problem is directly offered by Mendeley in a future upgrade of their API).

2. Article deduplication

A similar issue affects individual bibliographic entries, as noted by Egon Willighagen among others. Given that publication metadata in reference management services can be extracted by a variety of sources, the uniqueness of a bibliographic record is far from given. As a matter of fact, several instances of the same publication can show up as distinct items, with the result of generating flawed statistics when individual publications and their relative impact need to be considered (as is the case when calculating the H- and G-index). To what extent crowdsourced bibliographic databases (such as those of Mendeley, CiteULike, Zotero, Connotea, and similar distributed reference management tools) can tackle the problem of article duplication as effectively as manually curated bibliographic databases, is an interesting issue that sparked a heated debate (see this post by Duncan Hull and the ensuing discussion).

3. Author disambiguation

A way more challenging problem consists in disambiguating real homonyms. At the moment, ReaderMeter is unable to tell the difference between two authors with an identical name. Considering that surnames like Wang appear to be shared by about 100M people on the planet, the problem of how to disambiguate authors with a common surname is not something that can be easily sorted out by a consumer service such as ReaderMeter. Global initiatives with a broad institutional support such as the ORCID project are trying to fix this problem for good by introducing a unique author identifier system, but precisely because of their scale and ambitious goal they are unlikely to provide a viable solution in the short run.

4. Reader segmentation and selection biases

You may wonder: how genuine is data extracted from Mendeley as an indicator of an author’s actual readership? Calculating author impact metrics based on the user population of a specific service will always by definition result in skewed results due to different adoption rates by different scientific communities or demographic segments (e.g. by academic status, language, gender) within the same community. And how about readers who just don’t use any reference management tools? Björn Brembs posted some thoughtful considerations on why any such attempt at measuring impact based on the specific user population of a given platform/service is doomed to fail. His proposed solution, however – a universal outlet where all scientific content consumption should happen–sounds not only like an unlikely scenario, but also in many ways an undesirable one. Diversity is one of the key features of the open source ecosystem, for one, and as long as interoperability is achieved (witness the example of the OAI protocol and its multiple software implementation), there is certainly no need for a single service to monopolise the research community’s attention for projects such as ReaderMeter to be realistically implemented. The next step on ReaderMeter’s roadmap will be to integrate data from a variety of content providers (such as CiteULike or Bibsonomy) that provide free access to article readership information: although not the ultimate solution to the enormous problem of user segmentation, data integration from multiple sources should hopefully help reduce biases introduced by the population of a specific service.

What’s next

I will be working in the coming days on an upgrade to address some of the most urgent issues, in the meantime feel free to test ReaderMeter, send me your feedback and feature requests, follow the latest news on the project via Twitter or just help spread the word!

This is a quick note that may not surprise most people. Amy Bishop, at University of Alabama  Huntsville, just killed three colleagues and injured some more. It seems that this act may be related to having been denied tenure.  A PhD from Harvard, Amy Bishop had grants, and sat in a startup board, which are traces of a successful academic career. She was also a mother of four. Can your academic job environment be so toxic as to motivate murder? She was possibly suffering major depression at the time of the incident, and other mental health issues.

The evidence that an academic career is too stressing is piling up. An academic deals with rejection very often, from both peers and students, gets paid like a boy scout, and works every waking hour. This should be a waking call to all academics that feel tenure is the center of their lives.

A Previous Shooting Death at the Hands of Alabama Suspect – NYTimes.com

UPDATE: removed wrong photo.

Peter Sefton wrote a series of posts on wave. He has published on Scholarly HTML so I read attentively what he has to say. What follows is some highlights of his posts, and my thinking about where things are going. There are at least four things that bother me about wave –as it is today:

1- It’s not really HTML

I thought that waves being XML documents would be a good thing because it’d separate content and formatting. But it seems that they made some strange decisions about how to represent formatting with “very tenuous relationship to HTML”. For example

While there is talk of ‘XML documents’ in the whitepapers etc, a wave document in the current implementation is apparently a series of lines of text. All formatting and what you might think of as structure, such as whether something is a heading or not, is considered an annotation.

This is important right now because the only way to get the resulting doc is to dump the html to a file, or ‘copy-paste’. So, in a way, losing formatting like this would completely incapacitate wave for serious paper writing. I have had some success just copy-pasting and keeping most formatting, but I cannot risk to write a long paper and lose all formatting. Which won’t happen, because…

2- It doesn’t work too well for long, structured documents

Having a large blip for the entire paper with many people editing it seems to perform poorly. And having each person write their own blip-per-paragraph is not very pretty. It’s in fact distracting. I don’t discard wave’s usability to go through the roof once people start making things with it (robots). But it all depends on how well the API is designed. For example, following the mailing list, it seems that there’s no easy way to reorder blips programatically. This sounds like bad design to me.

3- It doesn’t integrate well with citation tools

It may be a matter of time before all the bibliographic tools we like get integrated. For example, Igor does offer some basic integration but this is far from satisfactory.

4-Formatting is simplistic

Wave has no table support. Figures are also not what you would expect, even for a notetaker. Captions are not implemented, nor footnotes. finally, LaTeXy is not the most convenient way to get equations done, I’m afraid, and it doesn’t go both ways.

Clearly, the content/form separation in wave is not designed for academic collaboration, and it shows. The questions is whether we can make this happen by writing robots[1]. Whether wave is the open platform that would make academic writing 2.0 happen.

Wave is just a tool. Why does this matter so much?

You may think that thinking too much about tools is counterproductive. But the way things are, it looks like tools are more and more important. Right now, we are stuck with the paper metaphor. Authors can produce pdfs, and publishers too. A publisher may make a prettier one, but that adds little value. We are equaled in terms of tools. However, publishers such as Elsevier want to get away from the paper metaphor (which is a good thing). As a consequence, authors will not be able to produce HTML as rich as the publisher’s. Here, the difference in tools matter.

In this case, Wave does look like an easy way to craft an interactive experience with little effort. So, even if you discard wave’s usefulness as a collaboration tool, it has quite a lot of value.

But it could very well be that wave doesn’t fulfill its promise. Microsoft Office 2010 offers similar functionality (close to real time edits). And of course, word has unparalleled features such as track changes, integration with endNote, etc. It could be that people adopt this new way of collaborative writing in real time without using wave. What worries me is that openOffice looks seriously left behind now. If it looks like a half-assed implementation of word 2003 features now, imagine when real-time hits mainstream. You need a serious server infrastructure to support that, which is possible for Google or MS, but not for a –smallish- open source foundation. I hope they find a way to jump in the train before it’s too late. Wave has a lot potential, because it is open. If openOffice could support that wave protocol, it could be a big achievement for open source.

If you have had any experience drafting a long doc in wave, please post it in the comments.

[1] For more on this, see my Wave intro for RuPy 2009.

After 100,000 invites went out yesterday, the web is boiling with reviews. The best no-nonsense explanation I found is a chapter of a forthcoming O’reilly book.

If you got an account, I’ve been on the dev preview (intended for developers to build on but otherwise identical), my user is quesada@wavesandbox.com

Looks like Elsevier experiments on how to present scientific papers are starting to get coverage  (on RWW no less).

The basic novelty here is real time search, but everything is peppered with other webby things like comments and AJAX.

The key features of the concept are here, and one can play with working prototypes. They are asking for feedback. I must say this is head and shoulders over reading a pdf on the screen. As highlights: (1) A figure that contains clickable areas so that it can be used as a navigation mechanism to directly access specific sub-sections of the results and figures, (2) references are clustered by sections of the paper they appeared in, and hot-linked.

However, it’s not clear if this kind of effort is just cosmetic or actually an important change. From the RWW article:

Some parts of the available prototypes are interesting but opinion in the scientific community seems split. Is this ground-breaking stuff or yesterday’s news repackaged by another industry threatened by the web? That depends on who you ask.

Let’s see. A publisher is, by definition, someone that helps your ideas reach more people and influence them (or was that a marketer? ). How good are current academic publishers achieving this? Not very well I would say, although it’s not the publisher’s fault only: most academics write in a way that repels readers. The fact is even that a blogger living in the long tail handles audiences orders of magnitudes larger than most academics get with their scholar papers. So one interesting question for academic publishers is how to get our stuff in front of the eyes of the general public, with their ever-decreasing attention span.

In this sense, I think this ‘new way of presenting an article makes it a lot easier to read. So, kudos to Elsevier.

On the other hand, if we want to compete for the attention of the general public, this is woefully inadequate. The Author Interview is a nice touch, but would people care enough to listen to it? Would this interview be helpful towards understanding the article without reading it in detail? Only server stats can answer these questions.

And there are many new ways of gather opinions and solve questions out there. Apart from wikis and forums, there is a new contender: stackOverflow. The basic idea is that the best solutions to a question get voted up. This saves one the effort to go through a thread of  partial answers. It has taken the programming community by storm. Partly because it actually works better than previous solutions (do you remember question answering sites?) and partly because Joel Spolsky and Jeff Altwood backed it and pimped it to their huge audiences. Of course, it wasn’t long before someone (mendicantbug.com) proposed The Stack Overflow of Academia  solution. This is by no means perfect:

The benefits of peer review by the herd are great, but not without pitfalls. First of all, you can be herd-reviewed by morons. Moron 1 might think everything Researcher A publishes is GOLD and gives the thumbs-up no matter how badly the research was done. Ditto on the flipside, with Moron 2 hating everything Researcher A does.

In any case, it’s clearly different to the ‘snail mail’ model of peer review that we still use. The post at stackOveflow is a declaration of principles.

As a commenter notes, the (bigger) problem that remains is that publishers want to be closed and charge for content:

This would be great if it was Google or some other company that doesn’t already charge exorbitant fees. Elsevier is well known to charge some of the highest prices—$300-400 for technical books! If electronic journals with multimedia etc are to take off, it will be because they democratize access to cutting edge science by removing the fees currently charged by academic journals, not b/c Elsevier hires some AJAX programmers. I can only imagine that this will be used as an excuse to charge more for access to Elsevier journals.

Very interesting times. I sense a huge opportunity here somewhere. Michael Nielsen also agrees that Scientific publishing is about to be disrupted. It just not clear to me whether facelifts like what Elsevier proposes are enough, or we need a more radical change.

We want anyone to be able to contribute to ap.com. One way to do this is to leave blog posts open (but with a review queue). We proposed this method here, but not many people seem to be making use of it.

Maybe writing a blog post is too time consuming, and the barrier of entry is too high. An easy solution is microblogging: services like twitter let you share a tiny bit of something interesting you found (with a link), and anyone following you will receive it.

The thing with microblogging is that it doesn’t take much effort to share. Many people (including me) thought it was silly at first, but now it’s mainstream.

Since twitter provides real-time search you can find what people talk about right now. If you want to monitor a special topic, chances are someone came up with a unique way of identify the topic. A spontaneous way of organizing information outside the ‘follows’ structure emerged: the hashtag. These are terms that start with #, example: #iranelection. We have set up #AcaProd for academicproductivity. If you have an idea, or read something outstanding that you would like to share with us, just tweet about it and add #AcaProd somewhere in the 140 characters. Your tweet then is easily found by anyone interested in the topic. We will display all tweets in our front page too.

I found myself sharing a lot of interesting stuff over twitter, and much more often than through a blog, so I have a good feeling about this.

Of course, you should keep sending ideas/suggestions/complaints using our email, blog@academicproductivity.com

May, 2007, we had a post on using an autocompleter to improve writing speed/save keystrokes. I recommended intellicomplete, even though it was abandonware even then.

Moving to more modern OSs (I’m running windows server 2008 64-bit), it simply didn’t work anymore. Plus it never worked well with Firefox or thunderbird. Anyway, I’ve found a much better replacement: Comfort type. It was hard to find, because reading the site, it’s not clear that it can do this kind of job. Looks like this feature is just one more for the author… which is quite impressive since it costs $20, half of what intellicomplete costed.

The main advantages over intellicomplete are:

• Multi-language support. This is killer, you can switch quickly with a shortcut, and as it learns it places words in different dictionaries
• It works on More OSs and applications.
• It also gives typewriter-like auditive feedback, which somehow makes me write more. Subconsciously, my body knows that if you don’t get the typing sound, you must be doing something passive. It’s also helpful to let people on the phone with you know when you are typing (so they don’t think an intimidating silence just started)
• It detects caret on more applications. Not very well on Tunderbird/Frefox, but better than anything else I’ve tried

The problems I can see right now are:

• It doesn’t complete with tab; only numbers, or enter. Enter can be a bit of a pain. This doesn’t bother me much because my tab finger was getting too much work anyway.
• Sometimes is not as fast as I’d like. When it completes a word, you can see it going back and retype what you wanted.
• Sometimes it hangs, but resetting it works.

What is interesting about this application is that it replaces quite a few others. For me, it replaces: Keepass (or any Password Manager) because it has encryption, and Autohotkey (for some basic templates).

Important: it doesn’t work with java applications. Could be a deal breaker. It even works when I use skype.

Nature has published a paper that advocates the use of cognitive-enhancing drugs: "Towards responsible use of cognitive-enhancing drugs by the healthy"

The comments on Nature’s opinion forum are varied, but in general people seems to be against:

"How dare they have the audacity to suggest such a thing?! The solution to the increasing “unnaturalness” in human society is not to simply shrug, say “why stop at homes, clothes, etc” as the authors suggest, and start drugging up otherwise healthy adults! "

As a scientist I do not relish my peers or younger colleagues taking such drugs for the extra edge in career success. I do not relish getting “confidential” advice from a tenure review committee member that next time I should try taking a daily dose of “X”.

We have talked about this before, and I thought that seeing such a strong, influential outlet taking positions on it (by publishing a paper; one could argue that the publisher doesn’t really need to share views with what it published, but still) is important enough to post.

If you are like most people in the academia, you place a lot of value on security and benefits. You also have great plans for that day when you retire and have time to… well, have a life .

Jack Cheng has a superb post on how much you can expect to live and use that free time you have earned:

Picture an average American who decides to stop working at the age of 65. Got it? Now guess how many years he’ll have to enjoy his post-retirement before he passes away.

I’ve asked this to a bunch of friends and coworkers over the last two weeks. I’ve heard answers like “15-20 years” or at the very least, 10 years. But none of those is even close.

The actual answer? 18 months.

Scary. Being a workoholic doesn’t sound that good. Sorry to post this in a productivity blog