views:

124

answers:

6

How do you evaluate publications? Im currently searching for a CS research topic and reading various papers. My dilemma on reading a paper usually is - is it really worthwhile continuing research in this topic?

what are the indicators of impact of research?

btw, im currently interested in - Liveness analysis. what do you think of it?

A: 

Perhaps comparing this topic with others you have expertise in with http://www.google.com/trends will help show which has a larger audience.

Mark L
+1  A: 

You should only target publications in very well established journals that are peer-reviewed such as IEEE. Some publications will include a section about possible future work in the Summary and Conclusion section. You could start by seeing if any of their suggestions have been taken in future publications.

Kevin Crowell
Shouldn't it be ACM for CS?
Pranav
I would sugest both (ACM and IEEE). Looking at some Springer publications can also be a good idea.
Alceu Costa
+5  A: 

The highest impact papers are those that are cited most. Citeseer and the ACM will show you how often a paper is cited. Really influential papers are cited long after they cease to actually be useful. Everyone cites papers they haven't read because they are certain that the paper is the definitive reference.

The definitive way to know the good papers is to have looked at everything in the area, but the question really is where to start.

A good strategy I've found is to start in textbooks, as they will sometimes cite the most important work at the time they are written. Obviously, use a recent text. Liveness comes under compilers, so try Cooper/Torczon, Muchnick, or Appel. Look at the end of the chapters, where there are often mini-literature surveys. (I don't usually recommend the Dragon Book. I just checked it though, and there's nothing useful.)

Finally, look for others to do the work for you. Look at the comments on the top of source files in gcc or LLVM. Look for survey papers. Look for papers that you already know the content of who touched on the topic, and follow the citation trail.

Example: Lets take a quick example. I remember a few papers that use liveness. One is Sam Guyer's 2006 PLDI paper, "Free Me". And I did a bit of work on SSA recently, and people use liveness a lot with SSA. I don't remember a specific recent paper, but I expect that Briggs' semi-pruned SSA probably talks about liveness, so that's somewhere to go second.

So looking at Guyer's paper, I went to the bibliography, and there were maybe two papers that mentioned liveness:

  • M. Hirzel, A. Diwan, and J. Henkel. On the usefulness of type and liveness accuracy for garbage collection and leak detection. ACM TOPLAS
  • H. Inoue, D. Stefanovi´c, and S. Forrest. Object lifetime prediction in Java. Technical Report TR-CS-2003-28, University of New Mexico, May 2003.

TOPLAS is a quality journal, so I'd look there first. And so on...

Paul Biggar
+1  A: 

You cannot have the answer of "is it really worthwhile continuing research in this topic?" just by reading one paper.

First, you need to choose the most important papers. As already said, the most important metric is the number of citations of one paper (Google Scholar or CiteSeerX gives you good indicators).

For a specific topic, you can try to see, in ACM or IEEE if there is a dedicated journal for your topic. Then, in this journal, you will have a good overview of the topic and you will be able to answer your question "is it really worthwhile continuing research in this topic?"

ThibThib
+1  A: 

If you're looking for a good topic to research yourself, the most important factor is do you find it interesting? If not, stop and look elsewhere. In fact, that's how I rate publications: interesting and well written are much more important to me that impact factor. In fact, impact factor isn't necessarily a great way to find what to research next.

Liveness doesn't strike me as hugely interesting, but it may lead to other interesting areas.

Paul Biggar
+1  A: 

Following "veins" of publications is often useful. A metric on individual researchers such as the H-index (http://en.wikipedia.org/wiki/H-index) can be helpful in identifying consistently good authors in a field. From there, following forward and back links (in terms of citations) can usually help you to identify high-impact publications that are consistently cited by high-impact-factor researchers.

Gian