This document provides detailed guidelines to Area Chairs (AC) for MICCAI 2019. Please read the instructions carefully if this is your first time as AC for MICCAI, and just as carefully if you have been on the committee before, as there are some changes in the review process compared to previous MICCAIs. Specifically, we adopted the model of the AC, where each AC will have specific expertise and more responsibilities for the papers assigned to him/her. Being an AC is a demanding task that involves managing the review process for a set of papers, selecting papers for orals and awards, interacting with the Organizing Committee (OC) and other ACs.
Having successfully moved to a double-blinded peer-review process at MICCAI 2017, there will be no face-to-face Program Committee meeting for MICCAI 2019, but a day will be set aside to hold teleconferences in three separate time zones (GMT±8h) where the MICCAI 2019 OC will communicate the overall results and statistics of the review process and any arising issues to the ACs, while at the same time collecting feedback from the ACs.
The role of AC therefore continues to entail a significant commitment.
Important Dates for Area Chairs is available from this page.
1. Responsibilities of the Area Chairs
ACs are expected to play an active role in ensuring high quality, timely, fair and constructive reviews. ACs also help the OC select the best papers to be presented at MICCAI 2019. The ACs are involved in the following steps of the review process
Paper Submission and Assignment to Area Chairs
(Deadline for paper submissions: April 2nd, midnight, Pacific Time)
Authors will submit their papers using Microsoft’s Conference Managing Toolkit (CMT). The MICCAI Conference review process will be double-blind, i.e. the ACs and reviewers won’t know the authors, and the authors won’t know the reviewers or ACs. Paper assignment to ACs will be completed automatically using CMT and the Toronto Paper Matching System (TPMS), which has been successfully used for MICCAI 2017 and 2018. The system is complex, considering paper matching scores, subject areas, conflicts and global load balancing – but remember, it is automatic and not necessarily optimal for each individual AC. If you feel a paper has been incorrectly assigned to you, please let email@example.com know; we will re-assign the paper to another AC and you will get a replacement paper that has been manually selected. We anticipate that such reassignments will be rare.
Reviewer Assignment – Phase 1
(Deadline for reviewer assignment: April 20th)
Note that MICCAI 2019 will be running double-blind – neither reviewers nor ACs will see the author list, nor will the authors see the reviewers or ACs. Primary ACs can only see the identity of initially selected reviewers – the intention is to blind them to the final list after reviewer paper bidding and load balancing. Reviewers will not see the ACs, and the ACs will not see one another. The CMT system will provide a list of reviewers for each paper to the AC. This ordered list is generated based on subject areas (relevance) and TPMS. Based on their expertise and judgment, the AC will select at least 9 and a maximum of 15 reviewers. You can filter the list by researcher type (e.g. undergraduate, MSc, PhD, Post-Doc, Faculty) or subject areas and sort by relevance according to keyword matching [score 0-1.0, 1.0=best match] or TPMS rank [1.512,1=best match] to help you choose reviewers. Select your reviewers and save for each paper.
While possible, it is difficult to add a reviewer who is currently not in the reviewer database because we cannot automatically identify conflicts and maintain load balancing through CMT and TPMS. If you wish to add a reviewer not in the database, you must (1) ensure that the person is willing to review, and then (2) contact firstname.lastname@example.org and we will include the reviewer manually within CMT for that paper during Phase 2. Given the overhead involved in this process, it is expected that it will be a relatively infrequent exception.
Reviewer Assignment – Phase 2
(Deadline for Phase 2: April 27th)
Once all ACs have selected the initial set of reviewers, a reviewer paper bidding process will start, after which the CMT system will re-optimize matching of reviewers to papers based on a large weighting placed on the AC’s suggested ranked list of reviewers and the reviewers ranking of paper preferences. This final optimization will consider the TPMS score, relevance, conflicts identified in Researcher.CC and conflict domains from CMT while performing load balancing across all papers, reviewers and ACs. The highest scoring three reviewers on the revised list will be sent the paper for review automatically.
(Deadline for reviews: May 20th)
Each AC will be responsible for shepherding the review process for a maximum of 20 papers. The AC duties in this process include:
- reading the articles;
- following up on reviewers to ensure timeliness and quality in their reviews.
During reviewing, the AC will login frequently and react to the reviewers’ actions; remind reviewers to login and download the papers; and especially ask for more details in the review, or to request a fairer review. This is critical. Should the AC be unsatisfied with the quality of a review, and fail to get further feedback from the reviewer, then the AC will be able to ask (email@example.com) for additional reviewer(s)’ input on the paper, beyond the original three reviewers. Any reviewer who does not provide a quality review will be identified in the reviewer database.
(Deadline for submission of rebuttal: May 28th)
The rebuttal process provides a way for authors to correct possible misinterpretations in the reviewers’ findings and inform the AC’s recommendation for scoring. Once the reviews of the papers have been sent to the authors, authors have one week to submit their rebuttal. Important: Only the ACs will read the rebuttal.
(Deadline: June 4th)
- Identify an initial decision of clear accepts and clear rejects papers.
- Papers with 3 accepts go into early accept. It is expected that around 20% of papers will fall into that category.
- Papers without a single reviewer accept or borderline accept will go into early reject. It is expected that around 50% will fall into that category.
- For papers with other recommendations, the primary AC and two secondary ACs will finalize decisions based on the reviews and the authors’ rebuttal. It is expected that from these remaining 30% of papers, half will be accepted and half will be rejected
The 20% accept, 50% reject and 30% borderline are rough estimates, and may vary slightly between the AC paper pools. As three ACs (one primary, two secondary ACs) will each recommend to either accept or reject the borderline papers, Step 3. will complete the paper selection. As a default position, the AC cannot overrule the consensus of 3 reviewers unless in exceptional circumstances and prior to discussing the circumstances with the Program Chairs.
Program Committee Teleconferences
(June 24-29 Tentative)
We invite and strongly encourage all ACs to participate in one of the three teleconferences, scheduled in separate time zones (GMT±8h) on a single day, where the MICCAI 2019 OC will communicate the overall results and statistics of the review process and present any arising issues to the ACs. In these teleconferences, ACs also will be asked to provide feedback on the overall review process and formulate recommendations on any need for adjustments or improvements. Specific details about the program and responsibilities will be announced prior these teleconferences.
It is expected that savings made by the ACs from not having to travel to a face-to-face meeting will free up financial resources to support their students and early career researchers to attend MICCAI 2019.
2. Best Practices of Being an Area Chair
This paragraph should remind all ACs of best practices of being an AC.
- Written report: One of the most crucial duties of ACs is the preparation of a written report (“meta-review”) on each paper. This is where the AC justifies his/her recommendation to accept/reject a paper. The reports represent a good quality review process and should highlight clearly why the decision was reached. If all reviewers agree on a paper, the report can be simple, while trying to encourage authors and provide constructive feedback. If there is even a slight disagreement on the reviews, it is your job to clarify why and how the disagreement was resolved. Overruling all three reviewers to decide on a paper is not acceptable. If you strongly feel that the paper deserves that, discuss it in detail with the OCs.
- Conflicts of interest: If you identify any paper with which you might have a conflict of interest (e.g. it was written by a current or former student, post-doc or supervisor, or by a collaborator in the last two years), please notify the ACs immediately so the paper can be re-assigned. DO NOT talk to any other AC about papers assigned to you without prior approval from the OCs, as there may be several other ACs conflicted with the paper. Also, DO NOT talk to any other AC about your own paper(s) (the paper(s) on which you are an author) or a paper with which you have conflict, during this whole process. Please remember that it is unacceptable to include anyone as a co-author who has been directly or indirectly involved with the manuscript at any stage on the decision process either as a Reviewer, Area Chair or as part of the OC; this includes any direct follow- on publications (e.g. MICCAI special journal issues).
- Attitude: Be aware that you have a strong influence on the decision for a paper. Take your job very seriously and be fair. Be professional and willing to listen to other reviewers and ACs. Do not give in to undue influence from anyone.
- At the conference: Please keep track of your accepted papers; for talks, have a question or two prepared for the speakers to stimulate discussion. For posters, go see the poster and ask questions.
Acknowledgements: These guidelines were inspired by CVPR 2014 guidelines: http://www.pamitc.org/cvpr14/ac_guidelines.php