Reviewing Guide

Dear MICCAI Reviewer:

Thank you for volunteering to serve as reviewer for this year’s MICCAI conference. The statistics from previous years demonstrate that the role of reviewers is decisive in the selection of the accepted papers and that the role of Program Chairs is, in comparison, modest. Therefore, your role will be crucial to shape a great MICCAI 2019 program, identify genuinely innovative and translational ideas, and provide balanced and constructive feedback to researchers of our community.

Although you probably have provided many conference paper reviews in your career and you may have reviewed for MICCAI before, we believe it is important to summarize what makes a good MICCAI review and some of the expectations from you as a reviewer. We also include the rules that MICCAI 2019 adopts for paper anonymization as part of its double-blind peer review process. Please read these guidelines as part of the overall MICCAI 2019 review process document.

What Makes a Good Review

The role of a reviewer is to identify excellent papers that the MICCAI community must hear about. It is not to reward authors for their hard work and dedication. As such, the review should tell the program committee which papers are exciting and could have a great impact on the field. A good review expresses an opinion about the paper and backs it up with details on strengths and weaknesses of the paper. It is not sufficient to simply summarize the paper and add a couple of questions about low-level details in the paper, nor it is acceptable to express an opinion without backing it up with specifics. A good review is polite. Just like in a conversation, being rude is typically ineffective if one wants to be heard.

Given the page limit, it is unfair (and generally pointless) to ask the authors to substantially expand their paper. It is counterproductive to recommend acceptance conditional on substantial revisions of the paper, or to ask for more substantial patient or clinical data. The paper should be evaluated as submitted, since the conference has no mechanism to ensure that any proposed changes would be carried out; moreover, the authors are unlikely to have room to add any further derivations, plots, or text.

While the review format might vary, a good review typically includes the following components:

  • A quick summary of the paper, which can be as short as a couple of sentences. This part tells the program committee what the major contributions are, what the authors did, how they did it, and what were the results. This part is also helpful for the authors to verify that the reviewer understood their approach and interpretation of the results.
  • The opinion of the reviewer about the paper overall. Is it an interesting contribution? Should this paper be presented at the conference? Should it be known to a larger group of people? Is it a significant advance for the field? Is this paper exciting enough for an oral presentation?
  • Important: Please remember that a novel algorithm is only one of many ways to contribute meaningfully. A novel interventional system, a validation study, a comparison study, an application of existing analysis methods to a novel problem are just a few examples of meaningful contributions. A paper would make a good contribution if you think that others in the MICCAI community would want to know about what authors have done, and can learn from their experience.
  • The opinion of the reviewer about the clarity of presentation, paper organization and other stylistic aspects of the paper. It is important to know whether the paper is very clear and a pleasure to read, or whether it is hard to understand.
  • The opinion of the reviewer about the major strengths of the paper. However tempting it is to immediately point out the problems, a reviewer should also write about a novel formulation, a principled derivation, an original way to use data, a novel application, or anything else that is a strong aspect of this work.
  • The major criticisms that a reviewer has about the paper. An effective way to deliver this critique is to summarize it briefly (at most two to three sentences), and then provide detailed arguments so the program committee and the authors can understand the reviewer’s concerns about particular aspects of the paper.
  • A detailed list of comments, to help the authors to revise a weak paper or to expand into a journal version of a strong paper.
  • A list of minor problems, such as grammatical errors, typos, and other problems that can be easily fixed by carefully editing the text of the paper.
  • If the reviewer’s expertise is limited to a particular aspect of the paper, a confidential note to the program committee that describes his/her relevant expertise. The review is more likely to be taken seriously if the limitations of the reviewer’s understanding are clearly acknowledged.
  • Before submitting a finished report, a wise referee asks, “Would I be embarrassed if this were to appear in print with my name on it?”

Specific Reviewing Notes.

This year we have a very large numbers of papers in Medical Image Computing but not so many dealing with Computer-assisted interventions. To ensure that we only accept the best MIC papers, and also select an appropriate spectrum of CAI papers in the mix, please keep the following points in mind while reviewing.

MIC-based papers

When reviewing MIC based MICCAI papers, we would like to see
1) whether the proposed methods are innovative or
2) whether the application is innovative.
These are the two important criteria for accepting or rejected those submitted papers.

In particular the following questions should be asked when evaluating MIC-based papers:
1.    Is the topic of paper clinically significant?
2.    Do the authors clearly explain data collection, processing, and division methods?
(It is important to check whether the cross-validation is performed from the beginning of  pre-processing steps, instead of the final stage of decision making.)
3.    Do the data appropriately represent the range of possible patients and disease manifestations?
4.    Are the data labels (if applicable) of sufficient quality to support the claimed performance of the algorithms?
5.    Do the authors report a sufficient number and type of performance measures to accurately represent strengths and weaknesses of the algorithms? Are performance measures reported with confidence intervals?
6.    Does the work make a significant contribution to the field, or is it just incremental over previous work?
7.    Do the authors discuss limitations of their methods and directions for future research?

CAI-based papers.

This year we have particularly encouraged submissions to MICCAI of papers relating to both the implementation of, and training for, computer-assisted Intervention approaches. In particular, we wish to highlight the use of Medical Image Computing techniques that have become integral components of Computer-Assisted Intervention. Areas that are considered significant in a CAI paper include.
1.    Presentation of a device or technology that has potential clinical significance.
2.    Demonstration of clinical feasibility, even on a single subject/animal/phantom.
3.    Demonstration of robust system integration and phantom validation.
4.    A novel MIC approach to solve an unmet CAI need.
5.    Proposal of a cost-effective (frugal technology) approach to implementing an otherwise expensive CAI solution.
6.    Description of a system or device that is robustly validated against appropriate performance metrics
7.    Psycho-physical/Human factors evaluations of CAI systems are presented.
8.    All papers should discuss imitations of proposed systems and provide a clear description of how the data used for the study were acquired.

Formal Rules

Formatting: As part of the review process, please ensure that the paper adheres to the submission guidelines described in instructions for authors.

Confidentiality: As a reviewer for MICCAI, you have the responsibility to protect the confidentiality of the ideas represented in the papers you review. MICCAI submissions are by their very nature not published documents. The work is considered new or proprietary by the authors; otherwise they would not have submitted it. Authors are allowed to submit a novel research manuscript that has been archived for future dissemination (e.g. on the arXiv or BioRxiv platforms). Sometimes the submitted material is still considered confidential by the authors’ employers. These organizations do not consider sending a paper to MICCAI for review to constitute a public disclosure. Therefore, it is required that you strictly follow the following recommendations:

  • Do not show the paper to anyone else, including colleagues or students, unless you have asked them to write a review, or to help with your review. These colleagues and students will also be subject to the same confidentiality.
  • Do not show any results or videos/images or any of the supplementary material to non-reviewers.
  • Do not use ideas from a paper that you review to develop new ones of your own before its publication.
  • After the review process, destroy all copies of papers and supplementary material associated with the submission.

Conflict of Interest: The blind reviewing process will help hide the authorship of papers. If you recognize the work or the author and feel it could present a conflict of interest, decline the review to the Area Chair and inform the Program Chairs. You have a conflict of interest if any of the following is true:

  • you belong to the same institution or have been at the same institution in the past 5 years,
  • you co-authored together in the past five years,
  • you hold or have applied for a grant together also in the past 5 years,
  • you currently collaborate or plan to collaborate,
  • you have a business partnership,
  • you are relatives or have a close personal relationship.

Anonymization Rules

MICCAI 2019 follows a double-blinded reviewing process, according to which anonymity should be preserved for both sides, i.e. reviewers and submitting authors. Accordingly, authors are asked to take reasonable efforts to preserve their anonymity during the reviewing process, including not listing their names, affiliations, websites and omitting acknowledgments. However, all this information will be included in the camera-ready and published version.

Please see the Author Guidelines for additional details on how authors have been instructed to act in order to preserve their anonymity, including guidelines for referencing their own prior or concurrently submitted work. Authors violating the guidelines for anonymity will have their papers rejected without further consideration.

Reviewers also should make all efforts to keep their identity invisible to the authors. Do not give away your identity by asking the authors to cite several of (or only) your own papers. Reviews that reveal the reviewer’s identity are likely to have lower impact in the PC’s decision process. Reviewers are asked to report potential breach of anonymization rules in their reviews. It is emphasized that anonymity should be kept in mind, not only during the paper submission and review, but importantly during the rebuttal process.

We realize that with the increase in popularity of publishing technical reports and arXiv papers, sometimes the authors of a paper may be known to the reviewer. The authors are strongly discouraged to make arXiv submissions of their MICCAI papers prior to MICCAI paper acceptance decisions. Conversely, reviewers are not allowed to attempt to identify reviewers based on arXiv submissions or other publicly available technical reports.  ArXiv papers are not considered prior work since they have not been peer-reviewed. Therefore, citations to these papers are not required and reviewers should not penalize a paper that fails to cite an arXiv submission. If the review process reveals that breaching of anonymity resulted from existence of an arXiv submission, the PC is likely to reject the paper on this ground.

Reviewers should make every effort to treat each paper fairly, even if they accidentally discover the identity of the authors of the paper. For example: It is NOT acceptable for a reviewer to read a paper, think “I know who wrote this; it’s on arXiv; they’re usually quite good” and accept paper based on that reasoning. Conversely, it is also NOT acceptable for a reviewer to read a paper, think “I know who wrote this; it’s on arXiv; they’re no good”, and reject paper based on that reasoning.

Thank you, in advance, for your efforts and contributions toward yet another successful MICCAI Conference,


MICCAI 2019 Organizing Committee

General Chairs:  

Dinggang Shen, University of North Carolina at Chapel Hill, USA

Tianming Liu, University of Georgia, USA

Program Executive:   

Terry Peters, Robarts/Western University, Canada

Larry Staib, Yale University, USA

Sean Zhou, United Imaging Intelligence, China

Caroline Essert, University of Strasbourg, France

Pew-Thian Yap, University of North Carolina at Chapel Hill, USA

Ali Khan, Robarts/Western University, Canada

Submission Platform Manager:

Kitty Wong, Robarts/Western University, Canada


Pin It on Pinterest