Two of our national co-chairs will be attending CIHR's working meeting on Wednesday, July 13, 2016. In advance of the meeting, we prepared a report of early career investigator (ECI) concerns and recommendations regarding CIHR peer review and funding.

We (ACECHR national officers Kristin Connor, Michael Hendricks & Holly Witteman) had a collegial and productive call with CIHR representatives Michel Perron and Jane Aubin on July 4. CIHR will be releasing more information in the coming days and weeks, but we are able to share the following information. All paragraphs that begin with "CIHR’s response:" have been reviewed and, if necessary, edited by CIHR representatives. We are posting their edited version almost exactly as written by them, having corrected only a few typos.
 
What we asked for
 
Urgent action. We continue to emphasize to CIHR that early career investigators (ECIs)--along with mid-career researchers and senior researchers facing funding gaps--cannot wait for the results of a pilot to be analyzed and plans made for adjustments to the process. People’s careers, laboratories, staff, and trainees are in jeopardy now.
 
CIHR’s response: They are planning to use the additional $30M to help address critical gaps in funding. They indicated they too will monitor for any  potential systemic bias in peer review (including potential bias against early career investigators, women, etc.) once the data are in from the current competitions.  They indicated that there was no formal structural interventions in the past for these groups and that the competitions ‘self regulated’.  They indicated that there is mention of this in their peer review manual and a new training module on gender bias. They also indicated that addressing all discrepancies through funding adjustments has the potential to become very complicated. There are many groups that may require attention, e.g., Aboriginal research, linguistic minorities, etc.
 
Clarity about use of $30M. We asked (a) whether they planned to use it all this year or spread it over competitions, (b) whether it will be allocated only to ECIs (as per wording “focus on” ECIs) or will also include gapped mid-career investigators (as per our recommendation), whether they (c) if they have a target for a ratio of full grants to bridge grants.
 
CIHR’s response: Overall, they restated the commitment of allocating the $30M towards early career investigators but to also implement these funds in order to allow for a sustainable competition funding over time (in other words not all at once as that would disadvantage ECI’s in future competitions since the $30M would be fully assigned immediately). As a result any available funds could be used to address any imbalances this competition which wont be known until final results are in and they don’t yet know what that might require, if it’s necessary. They did say that  t (b) it will include other gapped investigators, and (c) there are no specific targets for full:bridge.
 
Access to data & algorithms. We raised the issue that when applicants don’t trust the system, it creates many problems. We asked for more responsiveness to researcher concerns and more transparency. Regarding the latter point, we are specifically asking for (a) access to raw, anonymized data to conduct analyses and (b) transparency around how funding decisions are made, currently including the mathematical formulae and/or normalization algorithm detailing how consolidated ranks are calculated when different reviewers have different numbers of applications in their piles.
 
CIHR’s responses: (a) They are committed to open data – and as soon as practically possible - but also need to comply with privacy legislation and ensure that data are appropriately anonymized. They are working on coming up with a timeline for making data freely available. (b) They will provide this algorithm (but did not specify a time frame.)
 
Career development opportunities for ECIs. Participating in or viewing peer review panels has traditionally been recommended to ECIs as a method to learn about peer review. What sorts of opportunities will ECIs be offered going forward to learn more about peer review?
 
CIHR’s response: They do not currently have a plan for allowing ECIs to observe peer review, but are considering avenues like filming a mock review.
 
What is the purpose of peer review? To clarify possible sources of disagreement, we asked them, in their view, what is the purpose of peer review? We raised this in the context of career development activities like sitting on peer review panels and receiving expert feedback.
 
CIHR’s response: As with other funding agencies, the fundamental purpose of peer review is to rank applications for funding. The other aspects are secondary. CIHR hopes that applicants are getting expert feedback prior to submission. (We did not have a chance to respond to this, though we would argue that while this is certainly very important and of course ECIs and others do this, colleagues and others can only offer so much, and even the most senior colleagues cannot always specify what might tip an application over the funding line in one direction or another, particularly in a changing funding environment.)
 
We also sent a follow-up email suggesting that CIHR include the following 2 questions in the project scheme survey:
 
To add to surveys
 
1.[for applicants] Based on the reviews I received, I believe that all my assigned reviewers had sufficient expertise to review my application.
[for reviewers] I believe I had sufficient expertise to review all the applications I was assigned.
[for virtual chairs] I believe all the reviewers within my virtual panel had sufficient expertise to review the applications they were assigned.
[Likert, anchors: strongly disagree, strongly agree]
(optional) Comments:


2.I believe the Project Scheme First Live Pilot funded the best applications.
[Likert, anchors: strongly disagree, strongly agree]
(optional) Comments:
 
We expect that those whose applications are funded are more likely to respond positively; however, applicants' views will still offer useful insights one way or the other and reviewers’ views and virtual chairs’ views are likely to be very informative.