Archive for the 'Evaluation' Category

Randy Pausch passed away, but left great advice on time management (on top of his motivational tips)

Friday, July 25th, 2008

Just learned that Randy Pausch has passed away. Most of you will have been exposed to his extremely moving talk on ‘how to get the things you want’ at CMU, his last lecture, when he was diagnosed with cancer and give only a few months to live. What I didn’t know is that he also gave a talk on time management.

What I liked the most is his first slide: The goal is FUN. This one, Tim Ferris got right, and he seems very successful at indoctrinating people with this idea. I really hope it works out, because I’d love to see more people having fun around.

However, I challenge you to find a single academic that can live the life that Tim is proposing! But that a different conversation.

Another crucial point is his understanding of the famous 2 x 2 table in , Stephen F. Covey’s 7 Habits of Highly Effective People. Key: the 2 and 3 are often misplaced; people tend to do important, non-urgent last (i.e. after non-important, urgent). So the best order for the four cells is:

Urgent

Not urgent

Important

1

2

Not important

3

4

Also: a recipe for having short meetings: stand up. That limits the time a person can talk to you (you both get tired).

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

How do you evaluate your success?

Friday, June 13th, 2008

Nick Cohen has a provocatively titled piece in today’s Observer, “No one wins in modern-day academia”, examining the shortcomings of the Research Assessment Exercise.

If you’re a UK academic, you’ll know all about the RAE which, as it says on the tin, is an exercise to assess the quality of a department’s research and consequently determine future levels of government funding. My experience with this so far has been quite limited. In my old department, we measured success through policy influence, not journal publications, and my current work is just starting to yield results. But even if you are an active participant in the RAE, the question still remains: is this really the best way to assess our overall effectiveness and success? Academic life is about more than just research: we are also teachers, administrators and professional community members.

So setting aside the official funding role of assessment for the moment, I’d like to ask an open question. How do you evaluate your own success as an academic? (And as a relevant corollary, how does this affect how you choose to spend your time?)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

A solution to wasting time online: set up one computer for web access only, away from your work computer

Monday, May 19th, 2008

Paul Graham does it again. It’s great when you have a mind used to solve problems with simple solutions applied to time management. The more everyday problems hackers manage to solve (this time without use of fancy technology), the better for all of us.

The basic idea is simple: just make your body aware that you are wasting time. It’s even more radical than my interruptron idea of having a time counter growing in size as a function of wasting time.

I can find at least two problems with this idea. Or maybe I’m just trying to rationalize that I don’t want them to pry ‘my precious’ internet from my tired fingers :)

  1. Software updates. I find that I quite often have to do a ’sudo aptitude install’ or some such. At least a couple of times a day. Same for programming languages’ packages. It’d be a pain to switch on the internet for that, plus it’d be tempting to leave it on.
  2. Mail. I often have a quick idea and fire off an email to someone who may need to know or do something about it. My mail reader is integrated with my browser and sending a new mail is just one shortcut away. Again, a bit of a pain to move to a different computer to send a mail.

Still, the advantages are huge. Having no interruptions whatever online? Sounds great. In fact, when I’m really feeling like I need to get something done, I retire to a library with no wireless (and hopefully a comics section to fill scheduled rest time).

I’m interested in this method (2nd computer for internet only) enough to give it a serious try, say a full month.

It’d be great if more people wanted to join a trial, so we can do some n>1 testing on whether this works overall or not. It’d be also great to have some accountability (i.e., people knowing that you are doing this trial, so you feel ashamed if you are not following the rules). The problem is that productivity measures will have to be subjective: i.e., at the end of the month, do you feel you have gotten a lot done? More than any other month while you were online at all times?

What do you think?

PS: there’s a full thread commenting the article here.

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Rethinking life hacks

Friday, March 28th, 2008

“Math is hard; let’s go shopping!”

-hacked Barbie

Summary: It looks like the difficulties of measuring  productivity make people use common sense to give advice on how to improve it instead of actually attacking productivity as a hard problem that needs empirical study. But people do follow barely tested advice on productivity. They are either too busy to afford dismissing it, or too pragmatic to believe that we can reach systematic, scientific productivity techniques.

There is a current craze about productivity in many forms (sometimes disguised as personal development). At least 4 of the top 100 blogs in the blogosphere are about productivity (according 3038597_e5f95e2017_mto technorati’s authority: lifehacker #6; Zen Habits #41; lifehack.org #66 43 Folders #73). There’s a current craze about personal productivity and personal development. The best treatment I have read recently is Cal Newport’s Flak magazine article

In fact, lifehacking is a trend of the 21st century. The idea is to reduce the things that bother you in your life (or reduce the time it takes to complete them) while increasing the quality and quality of the experiences that you like. This is pretty intuitive, but is this a working definition of whatever personal productivity is? Hardly. Today, anything that solves an everyday problem in a clever or non-obvious way might be called a life hack.

Hacks are by definition, unsystematic. Everything goes, as long as it works. This is the contrary to the incremental evolution of scientific thinking. Even though sometimes there are large changes in the form of paradigm shifts, most of the time progress is incremental and lineal.

The advantages are clear: one can build on the knowledge acquired by the previous generation.

But do we have the same incremental progress in personal productivity theories? If there anything remotely similar to a science of productivity? Should people follow only empirically tested advice about productivity?

(more…)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Synchronous lecture materials. How?

Saturday, February 23rd, 2008

The efficient academic google group has a thread on a really interesting problem. Any hack addressing this has a high chance of saving several hours per week for those of you who teach.

Given lecture material has three components:

  1. Slides for digital projection (preferable PDFs rather than PowerPoint or Keynote)
  2. Lecture notes to support what I need to say and remember
  3. Lecture handout

I regularly update all three, but I am finding keeping all three in sync to be a bit tedious.

I’m not sure what the solution is, but what I am visualising is some sort of single document, where you  write the lecture handout. I could then update this with new information between presenting the lecture.

If you have a solution, drop by and post it there (or here!).

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Harvard new policy: make your scholarly articles available free online

Thursday, February 14th, 2008

This is big. Harvard University has a new policy: make your scholarly articles available free online.

As Slashdot commenter hawk describes it:

The academic publishing industry is a dinosaur in desperate need of elimination. It charges tens of thousands of dollars per school for journals that would be more useful as web sites–, not and available several months earlier. As it exists, journals are for the benefit of the publishing companies, not the world at large, academia, or the authors. The economic model is that the faculty write, are paid nothing, and the libraries pay huge fees to the publishing houses.
Will the publishers react to open up? I doubt it; they can’t.
The *real* result of this will be top articles going to online journals, which will first rival and then displace the printed journals. This is a good thing for everyone except the publishing houses.

But what’s in it for me, the end user of the paper? First, faster review cycles. Second, my ideas will reach a wider target (those who are not affiliated with a powerful library and cannot access them otherwise). Third, the ideas will get there faster. Forget about the close to a year delay between accepted and printed. Seeing “In press” in the reference list may be a thing of the pass soon. Fourth, if everything is online (imagine a journal article with a ‘comments’ section, open to anyone), then soft peer review is even easier and more transparent.

Nothing of this should be news, most people have their articles online anyway… but it sometimes breaks the agreement you have with the paper journal. Now that a large university makes this practice a policy, we’ll see other universities follow up soon.

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Cognitive doping for intellectually demanding tasks: worth it?

Friday, February 8th, 2008

This comes at a time when I’m very concerned about what people can do under pressure and how mpusheruch they are willing to sacrifice for their careers. A friend in the tenure track (or the equivalent in the country she lives in) has lost two babies (natural abortion), probably due to stress. There are entire sections in the Chronicle sections describing the super-human efforts people make to achieve a small increase in academic performance. Having a decent social or family life seems like a luxury for more and more academics. Most people invest money and time in this endeavor in ways that are difficult to justify rationally (and we are talking about arguably the smartest sector of the population!).

Would you risk your health as well? Are you prepared to take mind-altering drugs?

Nature has an article on cognitive doping (here’s the direct link if you don’t want to jump through hoops to get it from your library). The topics has been covered in the blogosphere in different places: Shelley Batts, from the point of view of a grad student, says that taking cognitive-enhancing drugs is a no brainer.

(more…)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Iprocrastinate podcast: there are people out there studying procrastination seriously

Thursday, October 18th, 2007

Dr. Pychyl has an interesting podcast series on procrastination. Is procrastination related to certain personality traits?logo Taxes and other aversive stuff: Why do we put it off? Give them a try if you are interested in those questions.

The main difference between these podcasts and the heaps of information that are available on the web about procrastination is that this comes out of a psychology lab.

If you find a good resource (even peer review ed ones :) ) on procrastination, please post it here.

 

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Book Review: The Myths Of Innovation by Scott Berkun

Sunday, October 7th, 2007

 

NOTE: this post and the following are part of an experiement I’m conducting with dictation and delegation. You may find that my writing style is different (more conversational?). I want to know if that bothers you, if you perceive it as a quality dropping. Let me know in the comments!

-Jose

This is my review of ‘The Myths of Innovation’ by Scott Berkun. If you have been following academicproductivity.com  for any amount of time, you know that we really like Scott Berkun’s books. We have reviewed his former book called, The Art of Project Management  (TAPM). I think this book is important for anyone who is aiming for a good output in academic productivity because the book tries to

answer a very important question, which is ‘how can I be more innovative?’ Since innovation is an important part of science, I think this is an important book to read.

(more…)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Productivity tips for students: meet Calvin at productivityhacks.com

Friday, September 14th, 2007

I have recently found that Calvin has moved his email newsletter into a new blog format. Calving is an accomplished MIT student who has published two books (!) on productivity for students: How to Become a Straight-A Student and How to Win at College. His blog has categories such as  Student Productivity and Study Tips with good advice for undergrads and grad students, although honestly, I think even higher-ups in the academic food chain could benefit from these tips.

Example posts:

Monday Master Class: Downgrade the Importance of Writing in Paper Writing

Dangerous Ideas: Sorry Paul Graham, I Think it Does Matter Where You Went to College (Watch out for his “dangerous ideas series”! He is trying to be provocative, and doing it well!)

A highlight of this blog is the educated comments it gets:

There is a world of difference between the questions that are thought out by someone else (the teacher), for the purpose of measuring someone other’s (the student’s) understading of a subject, and the questions that someone (the enterpreneur) has to first figure out are meaningful and then answer him/herself.

We haven’t talked about productivityhacks before because it was more oriented to undergrads, but this is not a good enough reason to deprive ap.com readers from excellent content. I think the actual social divide in the academic world is more like those who worry about getting grades, and those who don’t. This make a huge difference in how your life is organized. Grade-seeking people have their schedule done for them (they know for sure when they’ll need to study like crazy and when they can relax). They normally have lots of social support, since classmates have exactly the same schedule -and they are a legion-! The other side of the divide is for people who people who have to make their own schedule (sometimes, imposing it on others), and can suffer social isolation since their peers do not have the same time constraints, and there are few of them.

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Paul Graham: It doesn’t really matter where you went to college, measured by startup success ratio

Thursday, September 6th, 2007

If you have followed ap.com for any amount of time, you have realized that I really like Paul Graham’s ideas. He has just published an essay where he uses data from his Y! combinator to prove that having a degree from an elite university does not increase your chances of being successful when going for a startup. He argues that the ’startup test’ is a lot more useful to infer a person’s value than the tests that she needs to pass to e.g., getting good grades during high school or getting accepted in a prestigious college: “a high school record that’s largely an index of obedience”. I agree. I think his logic is impeccable.

This new essay just adds on his idea of “prestige is just fossilized brilliance”. Elite institutions capitalize on prestige, but it is not clear -at least from Paul’s analysis- that prestige converts well into real-life success, which I guess is what companies try to hire for. I think academics fight for prestige (clearly, money is not the currency they fight for!). And I think prestige is the wrong thing to look for!

 

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

The definitive hack for your music collection and how to use it to help you reach productivity nirvana: MusicIP review

Sunday, September 2nd, 2007

How can a music playing program be a time saver? What does this have to do with productivity? Well, background music prevents me from getting bored and drift into distractions. Music may shield you from noises and attention-grabbing events logo-glass-blue-home around you. I think music helps me reaching flow when writing/programming.

I will assume that at some point you have taken the time to rip your music collection into your HD, and that you have decent tags. Changing CDs or vinyl is just too distracting. If your tags are a mess, there are lots of tutorials on the web to get them under control. It’ll be worth the effort. At the end of this post, I’ll show you what could be the fastest method with the least human intervention.

(Note: I have talked about how managing music and academic paper collections are similar here; See also ‘noise for academics‘ by Shane).

The problem is that having background music has a cost.

(more…)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Book Review: How to write a lot (Paul Silvia)

Wednesday, August 22nd, 2007

This is a short book that one can read on a flight (I did), but it packs a lot of good advice. Silvia is sick of ’self-help-like’ books on writers block, etc., and it shows. He writes from the point of view of a seasoned author, and shows quite a lot of ‘being there, done that’ advice in both publishing papers and books. If you are planning a book, the last chapters are the best advice I could find on book planning; he even discusses how to negotiate with your publisher.

clip_image00111

So what are the basic recommendations?

- Writing is hard, so there isn’t much of a point trying to find a method to ‘make writing easy’.

- Write on schedule, and in a very consistent manner. Try to create a habit.

- Track your productivity (He uses a SPSS file where he inputs number of words typed per day, and whether he achieved the goal for the day). Tracking is a favorite of mine: it’s surprising how worrying little you get done on a day that feels like you have worked a lot. Tracking will tell you this. It’ll help you planning better: how many words can you write a day, on average? Well, if you have stats, you can say.

- don’t let yourself fall into what he calls ’specious barriers’. For example: “I’m waiting until I feel inspired” or “I need to do some more analysis first.”

- Use social pressure: create an agraphia group with friends/peers, and get together so you can feel ashamed if you didn’t write what you said you would.

The book clearly lives out to its ambitious title. It demystifies productivity (which is always a good thing), and it’ll probably make you laugh outloud… which is more than what I can say about most books that try to do that for you.

Of course, there are chapters that deal with style. Don’t skip those; they are probably the funniest.

This is probably the best book on the topic I have found. Highly recommended.

PS: Paul cites a whole lot of books that look really interesting.

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

12 tips on how to review journal articles

Wednesday, April 25th, 2007

Psychologist Henry L. Roediger III gives some excellent advice for reviewing journals papers (two word summary: be nice!). Though a psychologist, his twelve tips should have applicability for all academic disciplines. An excerpt from his introduction follows:

Many critical skills needed for becoming a successful academic are typically not taught in graduate school, at least not in any formal way. One of these is how to review journal articles. Few students coming out of graduate school have much experience reviewing papers, and yet, at least for those students continuing on in research, reviewing is a skill that will be increasingly critical as their careers develop. In fact, being a good reviewer can greatly help a career. If a young psychologist becomes known as an excellent reviewer, he or she may be selected as consulting editor, then associate editor, and then perhaps the primary editor of a journal.

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Soft peer review? Social software and distributed scientific evaluation

Wednesday, February 21st, 2007
For an extended version of this post, see also:
D. Taraborelli (2008), Soft peer review. Social software and distributed scientific evaluation, Proceedings of the 8th International Conference on the Design of Cooperative Systems (COOP 08), Carry-Le-Rouet, France, May 20-23, 2008

Online reference managers are extraordinary productivity tools, but it would be a mistake to take this as their primary interest for the academic community. As it is often the case for social software services, online reference managers are becoming powerful and costless solutions to collect large sets of metadata, in this case collaborative metadata on scientific literature. connotea popular tagsTaken at the individual level, such metadata (i.e. tags and ratings added by individual users) are hardly of interest, but on a large scale I suspect they will provide information capable of outperforming more traditional evaluation processes in terms of coverage, speed and efficiency. Collaborative metadata cannot offer the same guarantees as standard selection processes (insofar as they do not rely on experts’ reviews and are less immune to biases and manipulations). However, they are an interesting solution for producing evaluative representations of scientific content on a large scale.

(more…)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Noah Coad’s Code : Task Management Software (To Do Lists)

Tuesday, January 30th, 2007

Noan has reviewed 23 todo list applications. This seems to be an ‘open’ problem for many people (How do you pick the right to do list program?). With hundreds of online and stand-alone applications, some people still prefer paper. Looks like there is a big cake waiting for the first software company that gets this right.

 Link to Noah Coad’s Code : Task Management Software (To Do Lists)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!

Measuring performance and immediate feedback

Saturday, October 21st, 2006

Internet Marketers (IMs) have an advantage over other professions: they have pretty detailed statistics to use as feedback. For example, they have as indicators hits, time between buys, length of their customer lists, and ultimately… the money they make! They check these statistics daily.

Musicians are punished horribly when they fail performing a passage, not only by their peers but when practicing alone, by their own musical sense jumping in disgust!

In other professions, for example academics, we don’t get such a direct feedback. We may get feedback by how many papers we get published a year, but this is too coarse of a measure, and it only comes in yearly.

We may also consider our rate of success getting funding, but this is again a coarse measure, since we apply to at most dozens of grants in a lifetime.

In teaching, we may get a more direct feedback in that students are normally very expressive and their faces reflect how well our current lecture is doing. Yearly evaluations are also evidence of our performance. But nothing this immediate and direct is available when, say, you are writing a paper.

(more…)

If you enjoyed this post, make sure you sign up for our mailing list or subscribe to our RSS feed!