title: [D] First time reviewer. I got assigned 9 papers. I'm so nervous. What if I mess up. Any advice?
author: u/rjmessibarca
contenttype: redditpost
publication: r/MachineLearning
published: 2026-02-26T20:28:09+00:00
sourceurl: https://www.reddit.com/r/MachineLearning/comments/1rfle5p/dfirsttimereviewerigotassigned9papersim/
word_count: 588
I've been working on tech industry for about 7ish year and this is my first time ever reviewing. I looked at my open review tasks and see I have 9 papers assigned to me.
Sorry for noob questions
- What is acceptable? Am I allowed to use ai to help me review or not
- Since it is my first time reviewing i have no priors. What if my review quality is super bad. How do I even make sure it is bad?
- Can I ask the committee to give me fewer papers to review because it's my first time
Overall I'm super nervous and am facing massive imposter syndrome 😭😭😭
Any and every advice would be really helpful
Score: 79 | Comments: 46 | Subreddit: r/MachineLearning
Top Comments
u/mcmcmcmcmcmcmcmcmc_ (47 pts):
If you know ahead of time you won't be able to manage the reviewer load (9 is a lot, especially for the first time), I would message the area chair/senior area chairs asap and let them know. This is a much better outcome for everyone than getting a low quality review or no review at all and having to scramble for emergency reviews.
Just explain the circumstances and let them know which of your batch you feel you are most qualified to review. I think 3-4 papers is what you should shoot for.
As for priors, you should read through old open review conference reviews (especially the one that you are currently reviewing for, if available). This will give you a sense of what reviews cover, what a good review looks like, and, more important, how absolutely terrible many reviews are. As long as you are better than these awful reviews, you will be a net positive imo.
As for AI, most conferences have an AI usage policy these days, and typically it is not allowed except for grammar/spelling/fluency fixes. That isn't necessarily to say you can't use AI to summarize and understand some of the papers they cite, but you can't use it to directly review the paper you have been assigned.
If you get caught, you will typically be desk rejected and banned from submitting again for a while.
u/qalis (18 pts):
I mean, if you are about review quality at all, it probably puts you ahead over at least 30-50% of reviewers out there.
9 papers is A LOT, even for short conference papers, so this will take time. My advice is to look through papers and identify things that look obviously bad / LLM-generated / nonsensical to you. Start with reviews for those, and it will go quickly.
No, don't use AI. Your English may not be perfect, you may make some mistakes - this is ok.
You basically need to summarize good points of the paper, bad points, and questions/points to clarify. Just make sure the things you write about are actually in the paper. Just being factual also puts you ahead of a lot of reviewers.
I would definitely ask for that, yes, particularly since you have no experience.
Additional advice - look primarily for things that make practical sense, are interesting, and are well-evaluated. If you think the main idea is shallow, incremental, makes no sense, evaluation is bad or superficial (e.g. very few datasets, no statistical tests), just write it explicitly. Absolute majority of submitted papers is total crap.
u/unholy_sanchit (78 pts):
Do your best original work on all. I can guarantee they will be better than 90% of LLM generated slop
u/Squirreline_hoppl (9 pts):
I have been chosen within the top 10% of reviewers regularly during my PhD for all conferences. The way I did it was actually by following the advice I have gotten here on reddit long time ago: if you don't understand something, write it down and potentially ask it as question. I feel like the quality of the papers has reduced dramatically over the years. You have to identify the correct baselines they should check against and make sure they report them. Do they have the correct comparisons, ie what's the state of the art on the relevant benchmarks. I have invested about 4h into each paper when I was reviewing. I felt like my fellow reviewers did not invest half of that.
On llms for help with reviewing. You can definitely use them to ask about what the relevant baselines and benchmarks are. But I have recently used chatgpt to help me debug a paper and it did make mistakes, in the sense of what the common sense approach would be. I asked it something about the method in the paper and chatgpt hallucinated the most standard approach instead of giving me what the authors actually did. When I pointed it out with a direct quote, it was apologetic and very sycophantic on my brilliance of reading the paper lol. So be a bit careful.
To be honest, I am afraid most of the other reviews will be Ai generated. I turned down the opportunity of being an AC for the first time this year because I didn't want to dig through Ai generated reviews.
u/KeyApplication859 (8 pts):
It must be ICML, since you mention it's an A* conference and that's the one going on right now. 9 is a lot and does not look acceptable unless you submitted >3 papers. Try to give a fair assessment, look at OpenReview previous year and what score is typically given to get an idea.
u/ScientiaEtVeritas (5 pts):
These are way too many papers, in general but especially for your first time. How should high-quality reviews be possible like that?
u/ThinConnection8191 (6 pts):
I got asigned 6 ICML paper and I have to return one paper to the AC as I dont have any expertise in this field
u/Felix-ML (10 pts):
Assigning 9 papers is begging that guy to use llms in my opinion.