The Undercover Economist, Tim Harford, has a good article in today’s Financial Times about the stages in life when different professions are most productive. For example, I did a quick Google/calculation: the average median age of a Nobel Prize winner in physics or chemistry is 55; in the literature and peace prizes, it’s 64. (Sorry, not going to do the full test for statistical difference today). This distinction makes some sense, as the great discoveries in the two scientific subjects are marked by innovation (something that may become replaced by habit with age) and excellence in literature and statesmanship benefits from vast amounts of experience.

But, in keeping with our recent discussions about reform in academia, perhaps the bigger question is whether or not we should be actively targeting funding to match these periods of productivity? A quote from the FT article:

Two of my favourite writers, Malcolm Gladwell and Jonah Lehrer, are worried about this – but from different perspectives. Gladwell, a Galenson fan, worries that our obsession with youthful genius will cause us to reject future late bloomers.

Lehrer has the opposite concern: that funding goes to scientists past their prime. He says the US’s National Institutes of Health (NIH) has been funding ever-older scientists. Thirty years ago, researchers in their early thirties used to receive 10 per cent of NIH grants; by 2006 the figure had fallen to 1 per cent.

From my experience in the UK, I think both groups have good, but different, funding opportunities. Established researchers are well-versed in applying for traditional call-based research grants, whereas young researchers are catered for by a number of fellowship schemes. I haven’t seen much evidence of disciplinary-based bias and to be honest, I think anti-discrimination laws would make it difficult to explicitly exclude a group of talented researchers just because they’ve reached an arbitrary age barrier. Think of Andrew Wiles, who found a proof of Fermat’s Last Theorem but just over the Fields Medal’s age limit of 40.

Ultimately the top performers in these disciplines are so unique that it doesn’t make sense to design generalized development or funding programmes for the rest of us. However we can at least take comfort that our best days may be ahead of us!

Cambridge zoologist Peter A. Lawrence has published a thoughtful piece on the frustration of scientists (whether young or not so young) facing the ruthlessness of the research granting system (Real Lives and White Lies in the Funding of Scientific Research). He suggests how a “drastic simplification of this grant-writing process would help scientists return to the business of doing science” and quotes a passage from a recent NYT column by Stephen Quake, who asks what sounds to me like a challenging question:

Could we stimulate more discovery and creativity if more scientists had…security of…research support? Would this encourage risk-taking and lead to an overall improvement in the quality of science?

I take this as a genuine question in search of a convincing empirical answer.

• The full article is available in PLoS Biology.
• CC-licensed photo courtesy of A. Kuzminski.

Nick Cohen has a provocatively titled piece in today’s Observer, “No one wins in modern-day academia”, examining the shortcomings of the Research Assessment Exercise.

If you’re a UK academic, you’ll know all about the RAE which, as it says on the tin, is an exercise to assess the quality of a department’s research and consequently determine future levels of government funding. My experience with this so far has been quite limited. In my old department, we measured success through policy influence, not journal publications, and my current work is just starting to yield results. But even if you are an active participant in the RAE, the question still remains: is this really the best way to assess our overall effectiveness and success? Academic life is about more than just research: we are also teachers, administrators and professional community members.

So setting aside the official funding role of assessment for the moment, I’d like to ask an open question. How do you evaluate your own success as an academic? (And as a relevant corollary, how does this affect how you choose to spend your time?)

In the last month or so (sorry, we haven’t posted in a month!) I’ve been reading on and thinking about attention economy. I think it is the right paradigm to connect the different bits and pieces of productivity knowledge (we could call them hacks) floating around on the ‘net.

I could write a long intro to the attention economy ideas and how they affect the way we process information AND make decisions… but I have written a series of 4 posts on attention economy and I’d better redirect you there. So, ideally, before you continue reading this post you should have at least skimmed that series, and you should be comfortable with it.

The question I want to address on this post is this: Are we rational about how we allocate attention? This is an important topic because attention allocation to different scientific topics can make or break your career.

Economics uses the assumption that Agents are rational and, thus, markets are efficient.

But we dedicate disproportionate attention to small decisions compared to large ones, and this  does not fit with the rationality assumption:

I have long taken it as ‘given’ that we typically spend too much time and trouble over trivial monetary decisions (such as, whether to get a 4Gb or 2Gb iPod) but not enough on momentous ones (stocks or shares?  this house or that one?). Then when I come to actually make this claim in a written document I realise I have no evidence other than my own convictions, and after some time spent with Google scholar I have not come up with any.  If anyone knows of any relevant research please let me know.

(Daniel Read on the judgment and decision making [JDM] mailing list, Vol 97, issue 5)

There’s very little empirical work studying both psychologically and economically relevant variables simultaneously. Attention is a psychological variable, and return of investment is a concept from economy… and

If, as Goldhaber proposes, we can expect attention to work as currency, then we can calculate a Return of Investment (ROI) for attention.

Assume that attention can be converted to value, and that this happens over time. This is a good assumption: if we pay attention to fitness, our health improves; if we pay attention to [insert hot topic in your area here] you get publications and prestige (that may attract attention to you!).

ROI can be used to select a field of study. Everyone has a ‘hunch’ on what field will be the most productive in the short and long term. But are these intuitions accurate? Do people really switch fields according to their estimated ROI? The answer is… absolutely not. Most people I know picked their field either by whim or by accident, not by design. There are those ‘careerists’ who have the talent of picking only the hot topics (and they thrive in the current ecosystem!). But even those tend to focus on the short term (pressured by tenure-track demands etc) and their production, although impressive on paper, may have questionable long-term impact.

Here we see the effect that Daniel Read pointed out on the JDM mailing list: We don’t really spend much time pondering which line of work we’ll follow; everything feels much more like serendipity. However, small decisions, like which iPod to buy, may take a good chunk of our time.

Prestige, again, may in a way work like printed money does in our current ‘academic’ economy -which is, indeed, an attention economy!-. In previous posts, I have defended the idea that academics work for, and get paid in, prestige (not money). The role of prestige is to certify that the carrier has value, that paying attention to her will give us a good ROI. This explains why Google puts more weight on academic sites (i.e., getting one link from an academic site will do more for your pagerank than a link from a site with the same visits, but from the non-academic world).

The problem is that prestige doesn’t have all the same properties as money. It cannot be quantified, for once. This causes bizarre situations when committees have to evaluate candidates for a faculty position, or when they have to decide who gets the grant.

Online, the amount of attention each page gets is easily quantified, though. You can display google pageRank for each page you visit (that little green bar in the google toolbar, if you installed it in your browser). Google pageRank is an approximation to how ‘worthy of attention’ a page is.  Our previous post on soft peer review capitalizes on this idea: one can measure how much attention an idea is getting (in the form of a paper) by using citations (classical, slow) or bookmarks in sites such as citeUlike or Connoctea (new, yet to be fully used, fast, dynamic).

But if attention works as currency, and attention can be measured -if only approximately-… How can we calculate a Return of Investment (ROI) for our attention (or an approximation thereof)? I don’t think I have a good answer; citations and google pagerank are the most obvious answers, but I think you will agree that there’s more to it that just those.

A rational agent should maximize the amount of attention that her work would get. That is, for each hour of your work time, how much attention are you getting?

Then, the economics concept of investment works too: there are some activities that can lead to quick publications, while other are more long-term. Publish-or-die policy enforces results, and then many researchers are discouraged to pursue ‘big ideas’ (that are a long-term investment). They must be riskier too, so the changes of getting no ROI (no publication after a lot of work) are high. What is the right balance between high-risk, high-yield activities (you know which ones those are in your field!) and the off-the-mill paper-churning ideas that get you by? What’s your ‘saving power’? Can you afford to ‘spend’ attention on big ideas? Are you starved, and you really need to pump up the short-term?

These are interesting questions that came out of the initial assumptions: (1) Attention is the scarce resource. (2) The academia’s currency, prestige, is attention (fossilized attention according to Graham), (3) Attention can be measured, if only approximatedly, and then (4) you can calculate and ROI for your attention investments.

Let me know if you find this approach useful.

Thanks for your attention

-Jose Quesada

Good writing is a skill. I’m not saying I have it, and remember, this is a blog post, maybe the fastest form of writing and reading ). As a skill, it requires practice. And, as Graham says, “Writing doesn’t just communicate ideas; it generates them. If you’re bad at writing and don’t like to do it, you’ll miss out on most of the ideas writing would have generated.”In fact, writing posts like this one is helping me to review and polish ideas I didn’t know I had about writing till this very moment. I’ll start with the most obvious, and will get more subtle/interesting as the list grows.

1. Get your relevant Manual of Style. (e.g., Chicago/APA). APA wants you to buy it in book form, but I think this is one of the resources that should be online.
2. Get Oliver Strunk’s elements of style. It’ll recommend some rules of thumb that may well be obvious (e.g., avoid passive voice. Reduce the use of adverbs to a minimum) but overlooked. There have been several editions, and the older ones can even be found online.

1. Read Graham’s essay On how to write well. This concise piece packs a punch. It’s about essay writing so its advice may not be ideal for a journals paper but it’s fantastic for all other chunks of text we end up producing at the end of the day. Particularly research grant applications, where the reviewers are not experts on your topics and want some clear writing.
2. Get a competent professional editor, even if your English is very good. This can work wonders, if only because a professional editor will write down comments about your style that you should use for future papers. Professional proofreading may accelerate the process and wipe out some easy criticisms. Your university may have a writing center where these services are available. If yours doesn’t, you can hire someone at rentacoder or elancer.
3. Get as many people as possible to read your draft. There are some communities focusing on improving your writing, and giving detailed feedback; I couldn’t find one that’s about scientific writing, drop a comment if you know one. If such a thing doesn’t exists, we definitely need to create one.
4. Read it outloud backwards. I heard about this in the chronicle forums (sorry, I can’t find the link). It really works, even though it may be boring after a while. It’s the closest that an author can be to reading with the reader mindset. If you are too busy or feel ashamed reading stuff outloud, you can use a program to convert text to voice. The impersonal voice may actually help. Then, you can play the resulting audio file at a speed faster than normal. You can also use the player to move back and forth between problematic sections. This way, you can annotate where you need to do edits. If you want a free program that converts text to voice and works well, try Dspeech. It even lets you output an ogg file, my preferred music format. You’ll need a decent SAPI5 voice (you usually need to pay for those).
5. Know who you are writing for. If you are writing an off-the-mill journal paper, that’s the editor and the reviewers first, -you do want to get it accepted-. If you think what you have on your hands has the potential of being a high impact article, do not write for the reviewers, but for your field or even maybe the entire research community that may benefit from what you say.

   I find the honesty of the following devastating. Types of reviewers according to Scott Adams (Note: in case you don’t know, Scott is a famous cartoonist; no, I couldn’t find a more authoritative source):
   

   Assuming scientists are human beings, it seems to me that most peer reviewers would fall into one of these categories:

   1. Asshole

   2. Biased egomaniac

   3. Nice person who doesn’t want to make people feel bad

   4. Too busy to put any quality thought into it

   5. Person with low self-esteem who doesn’t want others to succeed in his or her field

   6. Coward who doesn’t want to rock the boat

   I suppose some scientists have plenty of free time, no biases, and would be happy to see colleagues succeed beyond their own careers. But seriously, how many of those scientists could there be? I don’t know any non-scientists who could fit that description.

6. Avoid distractions. That may mean no internet, no email notifications (gasp!)… ideally, you could try switching workspaces completely. You could move to an isolated cubicle at the library, where alternative activities to writing are close to zero. For maximum effect, you can use a program that lets you type, but little else. That is, all taskbars, gizmos etc in you OS are ignored: just a blank screen and your blinking cursor. Insert references, formatting, etc in your standard word processor later, on a second pass.

   Music listening may count as a distraction, if you have to DJ for yourself, or adjust the volume often. If you must have background music, make it so all tracks have replaygain values, and resist the urge to alter the playlist (or change CD if you are still not enjoying your music collection from your computer).

Disclaimer: this post may be relevant only for social sciences/psychology people. I found a nice thread on the Judgment and decision making (JDM) mailing list on the need for replications.

Lots of good posts on an interesting discussion. The mainstream view is that we simply don’t run enough replications because they are harder to get published. This leads to studies showing that replications are actually very hard, with only a small percentage (about 40% in the social sciences) being successful.  Robyn Dawes seems to thing that replications are overrated:

the “real” scientists do is to futch around until they get it “right.” The multiple study requirement just adds “first and second and third” studies, thereby wasting space and time.

There are comments on Increasing the Percentage of Papers Replicated, and some nice book recommendations on experimenter bias.

Today, while googling for “tenure rat race”, I found Jonathan I. Katz’s page: “Don’t Become a Scientist!“. I find his honesty devastating:

Are you thinking of becoming a scientist? Do you want to uncover the mysteries of nature, perform experiments or carry out calculations to learn how the world works? Forget it!

Science is fun and exciting. The thrill of discovery is unique. If you are smart, ambitious and hard working you should major in science as an undergraduate. But that is as far as you should take it. After graduation, you will have to deal with the real world. That means that you should not even consider going to graduate school in science. Do something else instead: medical school, law school, computers or engineering, or something else which appeals to you.

Why am I (a tenured professor of physics) trying to discourage you from following a career path which was successful for me? Because times have changed (I received my Ph.D. in 1973, and tenure in 1976). American science no longer offers a reasonable career path. If you go to graduate school in science it is in the expectation of spending your working life doing scientific research, using your ingenuity and curiosity to solve important and interesting problems. You will almost certainly be disappointed, probably when it is too late to choose another career.

I think he is right in many levels. But let’s concentrate just on the simplest, easiest to measure: money.

If we academics do the computations proposed in Figuring Out Exactly How Much Your Time Is Worth [The Simple Dollar], we may be in for a surprise.

Basically, you determine your true hourly wage by subtracting all of your work-related expenses from your salary, then calculating the hours you devote to work each year (including commute and other time-sinks) and dividing your remaining salary by your total hours.

Since we work silly hours, the actual pay is quite ridiculous. Of course, one has to factor in the liberty to think, flexible hours etc.

The November 2006 issue of PLoS Computational Biology has a short editorial with ten rules for evaluating postdoc opportunities. An interesting — albeit commonsensical — collection of hints, if you’re approaching the end of your PhD and looking for job opportunities after your defense.

Ten Simple Rules for Selecting a Postdoctoral Position

Thanks Benoît for the pointer.

Internet Marketers (IMs) have an advantage over other professions: they have pretty detailed statistics to use as feedback. For example, they have as indicators hits, time between buys, length of their customer lists, and ultimately… the money they make! They check these statistics daily.

Musicians are punished horribly when they fail performing a passage, not only by their peers but when practicing alone, by their own musical sense jumping in disgust!

In other professions, for example academics, we don’t get such a direct feedback. We may get feedback by how many papers we get published a year, but this is too coarse of a measure, and it only comes in yearly.

We may also consider our rate of success getting funding, but this is again a coarse measure, since we apply to at most dozens of grants in a lifetime.

In teaching, we may get a more direct feedback in that students are normally very expressive and their faces reflect how well our current lecture is doing. Yearly evaluations are also evidence of our performance. But nothing this immediate and direct is available when, say, you are writing a paper.

Is immediate, direct feedback important to achieve top performance? Most expertise theories say so. According to Ericsson the definition of expertise relies on deliberate practice. Deliberate practice entails more than simply repeating a task. Rather, it involves setting specific goals, obtaining immediate feedback and concentrating as much on technique as on outcome.

Also, general common sense says that “that which can be measured, improves.”. This implicitly assumes that the measurement is accurate and immediate.

For this reason, it seems obvious that some direct, accurate, and immediate feedback on how productive we are should be put in place. Now the problem is which productivity measures to use. We’ll cover this topic in a future post.

Technorati Tags:time-management – productivity – measuring

Technorati Tags: time estimation, deadlines, logging, project management

This is one more post on the Hamming series about how to select your research career topics.

It takes courage to think about important unsolved problems. (Excepting of course the officially canonized problems, such as Hilbert’s, Fermat’s Last Theorem, P = NP, …). But the solutions that made a difference were to problems that were not even recognized as such!

The initial cost of tackling an important problem is high; if it is a hard, important problem, chances are that you will not solve it at the first go, and that publication will not happen as fast as tacking smaller, but more constrained, problems. Young researchers may think that this is the only way to start an academic career. And advisors would of course discourage their students if they propose a problem that is somehow ‘non-mainstream’ (Science is a social activity after all).

There are careers built on a single experimental effect. Not because it is particularly relevant, but because it is extremely solid and running more experiments demonstrating that it appears in different circumstances will produce more publications.

Hamming mentions how the first success often changes a socially-handicapped researcher with self-esteem problems into a courageous one. Maybe having a solid effect may act as a catalyst, but it is always important to see it as  a stepping stone, not a solid rock to build a home and not to move around much.

Hamming reflects about the lack of courage he perceives in the generation after him, compared to that of his own. He ascribes the greater amount of courage to the enormous confidence gained from the experience of emerging from a tight spot in the second world war to a glorious victory, and working hard at it. He doesn’t blame the present generation, though, and acknowledges that their experience is very different.

Technorati Tags: philosophy – of – science, – planning, – sociology, – courage, – goals