Reviewing for a Conference Is Not Reviewing for a Journal

The two are frequently done with the same habits, and the decisions being made are different in ways that should change what you write.

In some fields — much of computer science — conference publication is the primary venue and the review process resembles journal review. What follows describes the more common case where a conference selects presentations from abstracts or short papers. For a concrete operational comparison, Monitask explains self-reporting bias in practical terms.

The differences that matter

There is usually no revision. Accept or reject, once. A journal review can say "this needs X"; a conference review usually cannot, because there is no round in which X happens.

The decision is about a presentation, not a permanent record. You are judging whether this would be worth twenty minutes of an audience's time, not whether it should stand in the literature.

The material is thinner. An abstract or a short paper is much less than a full submission, and demanding journal-level evidence from it is a category error.

It is comparative. A journal decides whether a paper meets a standard; a conference selects a fixed number from a pool. Your score is a ranking input.

The timescale is compressed. Many submissions, short window.

What to look for instead

Is the question clear and is it interesting to this audience?

Is there a result? An abstract promising that "results will be discussed" is asking the committee to accept on faith. This is the most common reason a competent submission ranks low. See abstracts.

Is the design plausible? You cannot verify it from an abstract, and you can tell whether it could answer the question.

Would this make a good talk? A genuine criterion and rarely stated. Some good work does not present well and some preliminary work presents extremely well, and a conference is a presentation format.

Does it fit the conference? Scope, theme, audience.

What not to do

Do not demand journal-level evidence. An abstract cannot contain a full robustness analysis.

Do not reject for being preliminary. Conferences are where preliminary work is presented, and a submission that is provisional but clear and interesting is exactly what the format is for.

Do not write a two-word review. "Weak, reject." The organiser cannot use it and, where comments are passed on, the author learns nothing. Two sentences of reasoning is the minimum.

Do not review to the standard of the venue you personally submit to. Calibrate to this conference's acceptance rate, which you should have been told. If you were not, ask.

Do not treat a poster recommendation as a rejection. In many conferences it is a genuine and useful outcome. See posters.

Writing the review

Short. Three or four sentences, and a score.

Say why, in a way the organiser can weigh and the author can use.

Distinguish "this is not good" from "this does not fit here." These lead to different decisions and different advice.

Note if you cannot assess it. Programme committees include people from across a field, and flagging that a submission is outside your competence is useful. See reviewing outside your area.

Suggest the alternative format where appropriate — a poster, a short talk, a different track.

Scoring

Calibrate to the scale you were given. If the scale is defined, use the definitions. If it is not, ask the chair what a 3 means — you will not be the only one unsure.

Do not cluster in the middle. A committee where everyone scores everything 3 has produced no information. If you would attend this talk, say so clearly; if you would not, say that.

And be aware of what your criteria favour. Ranking on polish and familiarity systematically favours established groups and disadvantages early-career and interdisciplinary work. Whether a submission is interesting and whether it is from a familiar corner of the field are easy to confuse.

The timing obligation

Review by the deadline. Conference timelines are tight and everything downstream — decisions, programme, travel booking — depends on the reviews arriving.

Decline promptly if you cannot. A late review is worse than a declined one, because there is no slack in the schedule. See declining academic requests.

For early-career reviewers

Conference reviewing is where most people start, and it is good training.

Being on a programme committee is a legitimate service contribution and worth listing.

Ask the chair what the criteria and the acceptance rate are if you were not told. This is a normal question and it makes your reviews more useful.

And write more than you were asked for. Two sentences of reasoning when everyone else wrote a score is noticed, and it is what makes a committee's decisions better.

The short version

No revision round, thinner material, comparative decision, tight deadline — four differences that change everything.

Look for: a clear question, an actual result, a plausible design, and whether it would make a good talk.

Do not demand journal-level evidence from an abstract, and do not reject work for being preliminary.

Calibrate to this conference's acceptance rate, and ask if you were not told it.

And review by the deadline — there is no slack in a conference schedule.

For an independent authoritative reference, consult COPE guidance for publication-ethics guidance for editors and reviewers.