Dear MICCAI Reviewer:
Thank you for volunteering to serve as reviewer for this year’s MICCAI. The statistics from previous years demonstrate that the role of reviewers is decisive in the selection of the accepted papers and that the role of Program Chairs is, in comparison, modest. Therefore, your role will be crucial to shape a great MICCAI 2019 program, identify genuinely innovative and translational ideas, and provide balanced and constructive feedback to researchers of our community.
Although you have probably have provided many conference paper reviews in your career and you may have reviewed for MICCAI before, we believe it is important to summarize what makes a good MICCAI review and some of the expectations from you as a reviewer. We also include the rules that MICCAI 2019 adopts for paper anonymization as part of its double-blind peer review process. Please read these guidelines as part of the overall MICCAI 2018 review process document.
What Makes a Good Review
The role of a reviewer is to identify excellent papers that the MICCAI community must hear about. It is not to reward authors for their hard work and dedication. As such, the review should tell the program committee which papers are exciting and could have a great impact on the field. A good review expresses an opinion about the paper and backs it up with details on strengths and weaknesses of the paper. It is not sufficient to simply summarize the paper and add a couple of questions about low-level details in the paper, nor it is acceptable to express an opinion without backing it up with specifics. A good review is polite. Just like in a conversation, being rude is typically ineffective if one wants to be heard.
Given the page limit, it is unfair (and generally pointless) to ask the authors to substantially expand their paper. Consequentially, it is not useful to recommend acceptance conditional on substantial revisions of the paper; nor to ask for more substantial patient or clinical data. The paper should be evaluated as submitted since the conference has no mechanism to ensure that any proposed changes will be carried out; moreover, the authors are unlikely to have room to add any further derivations, plots, or text.
While the review format might vary, a good review typically includes the following components:
- A quick summary of the paper, which can be as short as a couple of sentences. This part tells the program committee what the major contributions are, what the authors did, how they did it, and what were the results. This part is also helpful for the authors to verify that the reviewer understood their approach and interpretation of the results.
- The opinion of the reviewer about the paper overall. Is it an interesting contribution? Should this paper be presented at the conference? Should it be known to a larger group of people? Is it a significant advance for the field? Is this paper exciting enough for an oral presentation?
- Important: Please remember that a novel algorithm is only one of many ways to contribute meaningfully. A novel interventional system, a validation study, a comparison study, an application of existing analysis methods to a novel problem are just a few more examples. A paper would make a good contribution if you think that others in the MICCAI community would want to know about what authors have done and can learn from their experience.
- The opinion of the reviewer about the clarity of presentation, paper organization and other stylistic aspects of the paper. It is important to know whether the paper is very clear and a pleasure to read, or whether it is hard to understand.
- The opinion of the reviewer about the major strengths of the paper. However tempting it is to immediately point out the problems, a reviewer should also write about a novel formulation, a principled derivation, an original way to use data, a novel application, or anything else that is a strong aspect of this work.
- The major criticisms that a reviewer has about the paper. An effective way to deliver this critique is to summarize it briefly (at most two-three sentences), and then provide detailed arguments so that the program committee and the authors can understand the reviewer’s concerns about particular aspects of the paper.
- A detailed list of comments, to help the authors to revise a weak paper or to expand into journal version a strong paper.
- A list of minor problems, such as grammatical errors, typos, and other problems that can be easily fixed by carefully editing the text of the paper.
- If the reviewer’s expertise is limited to a particular aspect of the paper, a confidential note to the program committee that describes his/her relevant expertise. The review is more likely to be taken seriously if the limitations of the reviewer’s understanding are clearly acknowledged.
- Before submitting a finished report, a wise referee asks, “Would I be embarrassed if this were to appear in print with my name on it?”
Formatting: As part of the review process, please ensure that the paper adheres to the submission guidelines described in http://www.miccai2018.org instructions for authors.
Confidentiality: As a reviewer for MICCAI, you have the responsibility to protect the confidentiality of the ideas represented in the papers you review. MICCAI submissions are by their very nature not published documents. The work is considered new or proprietary by the authors; otherwise they would not have submitted it. Authors are allowed to submit a novel research manuscript that has been archived for future dissemination (e.g. on the arXiv or BioRxiv platform). Sometimes the submitted material is still considered confidential by the authors’ employers. These organizations do not consider sending a paper to MICCAI for review to constitute a public disclosure. Therefore, it is required that you strictly follow the following recommendations:
- Do not show the paper to anyone else, including colleagues or students, unless you have asked them to write a review, or to help with your review. These colleagues and students will also be subject to the same confidentiality.
- Do not show any results or videos/images or any of the supplementary material to non-reviewers.
- Do not use ideas from a paper that you review to develop new ones of your own before its publication.
- After the review process, destroy all copies of papers and supplementary material associated with the submission.
Conflict of Interest: The blind reviewing process will help hide the authorship of papers. If you recognize the work or the author and feel it could present a conflict of interest, decline the review to the Area Chair and inform the Program Chairs. You have a conflict of interest if any of the following is true:
- you belong to the same institution or have been at the same institution in the past 5 years,
- you co-authored together in the past five years,
- you hold or have applied for a grant together also in the past 5 years,
- you currently collaborate or plan to collaborate,
- you have a business partnership,
- you are relatives or have a close personal relationship.
MICCAI 2018 follows a double-blinded reviewing process, according to which anonymity should be preserved for both sides, i.e., reviewers and submitting authors. Accordingly, authors are asked to take reasonable efforts to preserve their anonymity during the reviewing process, including not listing their names, affiliations, websites and omitting acknowledgments. However, all this information will be included in the camera-ready and published version. Please see the Author Guidelines for additional details on how authors have been instructed to act in order to preserve their anonymity, including guidelines for referencing their own prior or concurrently submitted work. Authors violating the guidelines for anonymity will have their papers rejected without further consideration.
Reviewers should also make all efforts to keep their identity invisible to the authors. Don’t give away your identity by asking the authors to cite several of (or only) your own papers. Reviews that reveal the reviewer’s identity are likely to have lower impact in the PC’s decision process. Reviewers are asked to report potential breach of anonymization rules in their reviews. It is emphasized that anonymity should be kept in mind not only during the paper submission and review, but importantly during the rebuttal process.
We realize that with the increase in popularity of publishing technical reports and arXiv papers, sometimes the authors of a paper may be known to the reviewer. The authors are strongly discouraged to make arXiv submissions of their MICCAI papers prior to MICCAI paper acceptance decisions. Conversely, reviewers are not allowed to attempt to identify reviewers based on arxiv submissions or other publicly available technical reports. ArXiv papers are not considered prior work since they have not been peer-reviewed. Therefore, citations to these papers are not required and reviewers should not penalize a paper that fails to cite an arXiv submission. If the review process reveals that breaching of anonymity resulted from existence of an arXiv submission, the PC is likely to reject the paper on this ground.
Reviewers should make every effort to treat each paper fairly, even if they accidentally discover the identity of the authors of the paper. For example: It is Not OK for a reviewer to read a paper, think “I know who wrote this; it’s on arXiv; they’re usually quite good” and accept paper based on that reasoning. Conversely, it is also Not OK for a reviewer to read a paper, think “I know who wrote this; it’s on arXiv; they’re no good” and reject paper based on that reasoning.
Thank you, in advance, for your efforts and contributions toward yet another successful MICCAI Conference,
MICCAI 2018 Organizing Committee