Volume LI, Number 1
Measuring the Startup Journey and Academic Productivity of New Research Faculty through Systems Engagement, Project Efficiency, and Scientific Publication
Holly R. Zink
Children’s Mercy Hospital
Jack D. Curran
Children’s Mercy Hospital
Little is known about the process of tracking the activity and days-to-productivity of new research faculty in pediatric academic medical centers in the United States. The purpose of this study was to design a quantitative technique for measuring the startup journey and academic productivity of new research faculty at an established research academic medical center. Three measures, (1) engagement, (2) efficiency, and (3) publication, were used to identify the total number of days that it takes for a new research faculty member to move from the discovery phase (value-consumed) into the engagement phase (value-created) after accepting a new position. General findings were that the typical research faculty member hired from 2014 to 2018 at Children’s Mercy was male, submitted his first research project within the first three months of employment and averaged one new research submission per year. He would collaborate on a publication as a co-author within the first six months of employment, his first primary author publication would be published near his first employment anniversary, and he would average 2.9 publications per year in the first few years. The current study hopes to fill a gap in existing literature regarding the best practices for tracking, reporting and comparing the startup journey and academic productivity of new research faculty in pediatric academic medical centers.
Research faculty represent a core resource for research academic medical centers (AMCs). The intellectual capital incurred in education and research training, heightened reputation from sustained scholarship and impact of discovery, and the opportunities related to multidisciplinary activities and research are invaluable. Little is known about the process of tracking the activity and days-to-productivity of new research faculty. The onboarding process at each AMC is often a long-established tradition that combines elements from the faculty recruitment office, the academic department or division, and the central research office; but whether these traditional approaches are efficient, or if they achieve optimal results, is far from certain.
It is in this broader context that Children’s Mercy (CM), wishing to facilitate the vitality and productivity of its research faculty members, conducted a study to measure the startup journey from the discovery phase (value-consumed) into the engagement phase (value-created) and academic success of new research faculty in academic medicine. The purpose of this study was to design a quantitative technique for measuring the startup journey and academic productivity of new research faculty at an established AMC.
Contributions and Research Questions of the Present Study
Facilitating the success of promising new research faculty can have a significant effect on an institution's future. Nurturing new research faculty through organized faculty development may be necessary, but critically evaluating the benefits of an onboarding program is difficult because of the absence of well-defined methods for quantifying new faculty success for the variety of faculty research job descriptions in academic medicine.
There is a large volume of published studies describing the methods for measuring faculty research productivity (Bland et al., 2005; Bland et al., 2002; Creswell, 1985; Finkelstein, 1984; Teodorescu, 2000). The predominant methodology for research productivity among faculty in academic medical centers is quantitative in nature. There are several core productivity models presented in the literature. Among the first, Finkelsteinn (1984) presented seven variables to predict faculty productivity: faculty researchers having a research orientation, the highest terminal degree within a field, early publication habits, previous publication activity, communication with disciplinary colleagues, subscriptions to a large number of journals, and sufficient time allocated to research. Later, Creswell's (1985) model includes institutional factors in assessing faculty research productivity. Successful researchers hold a senior professorial rank, spend at least one-third of their time on research activities, publish early in their careers, receive positive feedback from peers for research efforts, and maintain regular and close contact with colleagues on and off campus who conduct research on similar topics.
Dundar and Lewis (1998) proposed a model where productivity is associated with individual attributes such as personal traits and environmental experiences, and institutional and departmental characteristics such as leadership, culture, structure, and policies. Just a few years later, Teodorescu (2000) proposed an international model where individual achievement variables and institutional characteristic variables predict faculty research productivity across national boundaries. A model by Brocato (2005) proposed that faculty research productivity is related primarily to factors of early research collaboration, personal demographic characteristics, and institutional research environmental factors.
Finally, Bland's (2002) model asserts that high research productivity is strongly associated with eight individual characteristics, fifteen institutional characteristics, and four leadership characteristics. Faculty research productivity is highest when a faculty member has specific individual qualities, works in an institution that is highly conducive to research and is led by someone who possesses essential leadership qualities and uses an assertive–participatory management approach. In 2005, Bland et al. noted that nothing substitutes for recruiting faculty with a passion for research, providing them with formal mentoring programs, facilitating their networks, and providing time for them to do research.
Traditionally, it has been argued that the impact and relevance of research output can be quantified using bibliometric data (Garfield, 2006; Hirsch, 2005; Hutchins et al., 2016); however, it has also been reported that publication productivity often declines during faculty transition (Bland et al., 2005; Lowenstein et al., 2007; Perry et al., 2000; Ries et al., 2012; Wingard et al., 2004). There has been little quantitative analysis of the research productivity of newly hired faculty using metrics beyond bibliographic data. Therefore, for this study, the additional metrics of research engagement (Katz & Martin, 1997; Lee & Bozeman, 2005; Ponomariov & Boardman, 2010), information technology and processes efficiency (Green & Gilbert, 1995; Lowe & Gonzalez-Brambila, 2007) were used to explore the subsurface of new faculty research productivity. This study used a quantitative research design to explore and observe the relationship between new research faculty and variables related to research productivity, namely: engagement, efficiency, and publication. Specifically, this study addressed three questions:
- What is the average number of days between the employment start date and a research faculty member’s first engagement with the research administration system?
- What is the average number of new projects started per year for a new research faculty member?
- What is the average number of days between the employment start date and research faculty member’s first publication and the average number of publications per year for a new research faculty member?
Towards a Model of New Faculty Research Productivity
As described above, the data used in this study came from Children’s Mercy (CM), a pediatric academic medical center located in Kansas City, Missouri. Beginning in the 1990s, CM invested primarily in stand-alone research programs; however, over the last ten years, research began to grow organically into subspecialties across the hospital. More departments and individual divisions were starting to recruit physicians with protected research time and research startup packages. A generational shift also made room for younger faculty that wanted to do a broad range of activities to achieve professional satisfaction.
Going back ten years, CM did not have a process in place to evaluate a research project. If a faculty member wanted to start a project, they would call the Institutional Review Board (IRB) or Institutional Biosafety Committee (IBC) for oversight and then begin the project once they received approval. In 2014, CM implemented a research administration system that operates as the “front door” for all research activities and starts with fundamental questions about the research project principal investigator (PI), participating staff, any expected external funds, and a project budget. Primarily, the system communicates what the project is and what resources the research faculty member will need to be successful to division administrators. The system also helps submit grant applications and research proposals to the sponsor.
One of the first things that the institution saw when the system was first implemented was that it provided all division directors full visibility into what was going on in their areas. The faculty leader and all administrative leaders are informed when a project is first entered, allowing conversations about resources to take place early in the project planning process. The administrative system was a big step forward in transparency, efficiency, and communication regarding research administration processes. Now that it has been active for five years, it is possible to get a more accurate view of new research faculty efficiency from an information systems perspective, in addition to traditional bibliometric data.
Quantitative methods offer an effective way of evaluating the baseline metric to measure the startup journey and academic success of new research faculty. This retrospective study involved secondary data collection from eight different systems, including faculty information, research operations and projects, research integrity, research effort, and publications databases. The purposive convenience sampling included all CM employees with faculty appointments with an employment start date between January 1, 2014, and December 31, 2018, with evidence of planned and reported research participation in their first year of employment.
Specific guidelines were developed for reporting effort at CM to clarify the portion of effort that individual faculty members devote to administrative, research, teaching, and service activities. All duties are assigned by the Division Director and/or Section Chief. It is expected that each division or section implement equitable standard processes and expectations for individual assignments, depending on the specific needs of the division, the career development goals for the individual, and consideration of approved protected time for administrative, research, teaching, and service activities.
Expectations for research productivity are stratified according to the percent effort allocated for research in CM. The Department Chair and institutional policy established the guidelines for research effort and expected productivity. Those faculty with 0.01 to 0.05 FTE protected research time were required to demonstrate some research-related activity (e.g., participating in clinical trials, participating in an investigator-initiated study, mentoring research activity of trainee, coordinating division quality activity with the intent to publish, etc.). Those faculty with 0.06-0.20 FTE protected research time were expected to publish an average of one or more peer-reviewed manuscripts yearly over a three-year period. Individuals with 0.21-0.50 FTE protected research time were expected to publish an average of one or more peer-reviewed manuscripts yearly and receive one or more external grants over a three-year rolling time period. Finally, those individuals with >0.50 FTE protected research time were defined as a researcher being their primary role and hold the expectation of independence as a principal investigator. For established investigators, the research program was expected to be supported by external funding (federal grants, foundations, philanthropy, etc.). For new investigators, the research program would be largely self-supported by external funding after an agreed upon period of startup time (typically three years). Effort allocation and productivity are reviewed on an annual basis through the department's annual assessment form. In all cases, failure to meet expectations would require a reassessment of the amount of protected research time by the Division Director and the Department Chair.
Following the research expectations set forth by the institution, only faculty members with 20% or more research effort were included in this study to ensure that the expectation of both publications and external grant activity were present during the startup period. Therefore, a research faculty member was defined as faculty with planned and reported research effort of 20% or more in their first year of employment, with an employment start date between January 1, 2014, and December 31, 2018, who was hired under the assumption that research would be a regular part of their workload, and therefore, eventual research productivity was expected.
Data from multiple systems were pulled on December 31, 2018, and integrated into a single dataset with the following columns: faculty name, degree, division, department, birthdate, race, gender, employment start date, faculty rank at start date, research effort, assigned space (yes/no), startup total years, startup total award, first research project created date, total number of research projects since start date, first IRB protocol created date, total number of IRB since start date, IBC approval date, date of first publication since hire date, and total number of publications since start date.
Data was collected and combined from several databases to compile a complete dataset of faculty demographics, research effort, research project (IRB and IBC) applications, and publications. Reports pulled on December 31, 2018, from multiple faculty databases, provided a total of 1,070 faculty in the original database with start dates ranging from July 1, 1971, to September 30, 2019. This original report included ten departments across the hospital: Anesthesiology with 44 faculty, Dentistry with ten faculty, Graduate Medical Education with nine faculty, Heart Center with 44 faculty, OB/GYN/Fetal Health with four faculty, Pathology/Laboratory Medicine with 30 faculty, Pediatrics with 772 faculty, Pharmacy with one faculty, Radiology with 31 faculty, and Surgery with 125 faculty listed as of December 31, 2018. A total of 736 faculty with start dates before January 1, 2014, and after December 31, 2018, were removed from the dataset. At this time, 347 faculty remained with start dates between January 1, 2014, and December 31, 2018.
Three measures were used to identify the total number of days that it takes for a new research faculty member to move from the discovery phase (value-consumed) into the engagement phase (value-created). These three variables were engagement, efficiency, and publication. Additional variables, including faculty demographics, the receipt of startup research funding, and the assignment of research space, were also included in the final analysis.
All research projects (animal, human subjects, or non-human subjects) at CM must first submit a new project application in the research system. Therefore, the first measure, engagement, was defined as the day faculty created a new project application as the principal investigator in the research system. It is important to note that this research used the date the application was created within the project application system, not the date the project was submitted or approved. There are specific problems with the use of the submission or approval date when defining faculty engagement, mainly system-approval delays. It was decided that the first moment of faculty research engagement at CM was best represented in the research project application created date. Any research project applications that were created during residency or previous non-faculty employment with CM were removed and not included in the analysis.
The second measure, efficiency, was defined as the average number of total projects that were submitted per year by the faculty member in the research systems. Engagement in multiple research projects at the same time, or planning for future research funding, is often a hallmark of a successful researcher. The efficiency measure helps to frame how many projects a new researcher proposes each year.
The third and final measure, publication, was defined as the publication date of the first research article published by the faculty member with research that was conducted primarily at CM. As it often takes several months for a publication to move through the peer-review process, it was essential to double-check all new publications to ensure that the research was conducted at CM for a valid measurement.
The total population included 59 research faculty (n=59). The results from the preliminary analysis show that 36 respondents (61%) have an M.D., 18 respondents (31%) have a Ph.D., 4 respondents (7%) have both an M.D. and Ph.D., and one respondent (2%) registered other doctoral education. Most research faculty (44, 75%) are assigned within the Department of Pediatrics. This result is not surprising as the Research Institute is housed within the Department of Pediatrics. The average age of research faculty hired from 2014 to 2018 was 41 years, with the majority (34, 58%) of the population ranging between 30 to 39 years old. The youngest range, 30 to 34 years old, accounted for 29% (17 respondents) of the population, matched by the 35-39 range with 29% (17 respondents). The remainder were 40-44 at 10% (6 respondents), 45-49 at 16% (9 respondents), 55-59 at 10% (6 respondents) and 60+ at 2% (1 respondent). The research faculty population was primarily white at 66% (39 respondents), followed by Asian at 14% (8 respondents), Black/African American at 7% (4 respondents), Asian/Indian at 5% (3 respondents), Korean and Hispanic/Latino each at 3% (2 respondents each), and lastly Chinese/Filipino at 2% (1 respondent). Research faculty hired within the study period were predominately male (35, 59%), followed closely by female research faculty (24, 41%). Interestingly, further analysis showed that while male new hires increased on average over the study period, female new hires decreased (see Figure 1).
CM hired an average of 11.8 faculty members per year, with marked increased recruitment in 2016, accounting for 34% (20 respondents) of the total study population. In 2014, 2015, and 2017 each, CM hired 19% (11 respondents each) of the population. In 2016, there was a marked increase in hiring accounting for 34% (20 respondents) of the population. Lastly, in 2018, only 10% (6 respondents) of the research population was hired. The majority of academic faculty start employment in July (24, 41%) as it is the start of the organization’s new fiscal year. Close behind the July month start date is September (9, 15%) and August (8, 14%), also corresponding with the academic year. Most research faculty are hired into the Assistant Professor (35, 59%) academic rank, followed most closely by Associate Professor (14, 24%) and finally Professor (10, 17%).
The first question in this study sought to determine the average time between the employment start date and a research faculty member’s first engagement with the research administration system. Figure 2 is quite revealing in several ways. First, it shows that the majority of new research faculty are accessing the research systems within their first three months (17, 29%), with a significant portion (14, 24%) registering a new project within their first 30 days on campus. A total of 39 out of 59 new research faculty (67%) are likely to enter a new project within their first six months. Finally, it shows that 10% (6 respondents) of our research faculty have not entered a project into the system. Further analysis shows that those research faculty who have not entered a project into the research system were all provided with startup research funding.
The second question in this study sought to determine the average number of new projects started per year for a new research faculty member. In Figure 3, it is apparent that the majority of research faculty (39 respondents, 66%) are creating more than one new research project per year.
The third question in this study sought to determine both the average time between the employment start date and research faculty member’s first publication and the average number of publications per year for a new research faculty member. From the data in Figure 4, it is apparent that the length of time between a research faculty member’s employment start date and the first publication is often greater than one year. There was a significant portion of faculty who were able to produce a publication between 1 to 6 months after their employment start date (25, 36%). Further analysis revealed that most of these earlier publications were a co-author status and not as the primary principal investigator on the publication. Analysis of faculty publication rates showed that, on average, new research faculty produced 2.9 publications per year during the study period (see Figure 5).
The purpose of this study was to introduce a quantitative technique for measuring the startup journey and academic productivity of new research faculty at an established research academic medical center (AMC). Previous literature (Bland et al., 2005; Lowenstein et al., 2007; Perry et al., 2000; Ries et al., 2012; Wingard et al., 2004) has reported that academic productivity often declines during faculty transition; however, these studies were often based on publication productivity alone. The results of this study offer a more comprehensive approach to quantifying the research productivity of newly hired faculty beyond the use of bibliometric data.
By leveraging the data from the research administration systems, this study was able to measure the startup journey of new research faculty members from the last five years. General findings were that the typical research faculty member hired from 2014 to 2018 at CM was a 35-year-old white male with an M.D. hired in July into the Department of Pediatrics as an Assistant Professor. He submitted his first research project within the first three months of employment and averaged one new research submission per year. He would collaborate on a publication as a co-author within the first six months of employment, his first primary author publication would be published near his first employment anniversary, and he would average 2.9 publications per year in the first few years.
Overall results on new research faculty engagement were positive, showing that most new research faculty are entering a new research project into the system within the first 30 days to 3 months on the job. However, further analysis showed that 10% (6 respondents) of research faculty had not entered a project into the research system even though all were provided with startup research funding. This suggests that new faculty members with readily available institutional funds to provide for all initial startup needs may not engage with the research administration systems as quickly as others without internal funds.
While the majority of research faculty (39 respondents, 66%) are creating more than one new research project per year, a large section of the population is still creating less than one new research project per year. There are several possibilities reflected here, and most are encouraging. First is the possibility of multi-year funded projects, which is the best possible scenario. In other words, a faculty member enters a new research project into the research system that is a 5-year fully-funded study. That faculty member then has little reason to start a new project in the research system for several years. Likewise, a faculty member who has received startup funds is also likely not to start multiple projects in the first few years. Faculty with less than one new research project per year were, on average, with CM for 2.3 years as of December 31, 2018. This finding might also provide some insight. Most faculty are given three years to achieve full external funding at CM and therefore may not feel pressured to find funding in their first year.
Findings suggest that a significant portion of faculty (25, 36%) were able to produce a publication between 1 to 6 months after their employment start date through a co-author status on the publication. This information is crucial because it shows collaboration and engagement with fellow research colleagues within the new organization and shows an active and welcoming research community.
Research does not always involve data collection directly from the participants. The information used in this study was collected through routine management information systems and other administrative research activities. Existing data were analyzed to avoid repetition of research and survey fatigue of institutional research faculty members. However, specific ethical considerations of this study about secondary data analysis and data confidentiality were considered (Tripathy, 2013). This study was reviewed and approved by the IRB before execution, and it was determined that the proposed activity did not involve research as defined by U.S. Department of Health and Human Services regulations.
Limits to the generalizability of the work include the personalized nature of the research system, faculty management data systems, research and faculty onboarding practices, and current research support located at CM. Factors that might have limited internal validity in the design, methods, or analysis include publications not currently indexed within the PubMed database or inaccuracies in self-reported data from the participating research faculty members. Additionally, this study was not able to collect publication information for a full five years on the entire population. Additional analysis will need to be completed on the final question regarding the average number of publications per year for a new research faculty member. Efforts made to minimize and adjust for limitations include a rigorous cross-checking of data points across eight different data sets.
It can be challenging to track and monitor a new research faculty member’s progress as they transition into a new role in a new organization. The current study hopes to fill a gap in existing literature regarding the best practices for tracking, reporting, and comparing the startup journey and academic productivity of new research faculty in pediatric AMCs. This research also provides a quantitative method to measure the entire startup journey from the discovery phase (value-consumed) into the engagement phase (value-created) and academic success of new research faculty in academic medicine. Most administrators and faculty members would agree that more rigorous and comprehensive benchmarks are necessary to track the startup journey of research faculty in pediatric AMCs. This research provides a framework to track the research onboarding experience through quantitative measures of engagement, efficiency, and publication. However, more research on this topic needs to be undertaken before the association of the engagement, efficiency, and publication measures, and the research startup journey, are more clearly understood.
Holly R. Zink
Faculty Onboarding Coordinator
Medical Administration, Children’s Mercy Hospital
2401 Gillham Road, Kansas City, Missouri, U.S.A.
Jack D. Curran
Department of Pediatrics, Children’s Mercy Hospital
Kansas City, Missouri, U.S.A.
Correspondence should be addressed to Holly R. Zink, Children’s Mercy Hospital, Medical Administration, 2401 Gillham Road, Kansas City, Missouri; Telephone: (913) 945-6026; E-mail: firstname.lastname@example.org.
Bland, C. J., Center, B. A., Finstad, D. A., Risbey, K. R., & Staples, J. G. (2005). A theoretical, practical, predictive model of faculty and department research productivity. Academic Medicine, 80(3), 225–237. https://doi.org/10.1097/00001888-200503000-00006
Bland, C. J., Seaquist, E., Pacala, J. T., Center, B. A., & Finstad, D. A. (2002). One school’s strategy to assess and improve the vitality of its faculty. Academic Medicine, 77(5), 368–376. https://doi.org/10.1097/00001888-200205000-00004
Brocato, J. J., & Mavis, B. (2005). The research productivity of faculty in family medicine departments at U.S. medical schools: a national study. Academic Medicine, 80(3), 244–252. https://doi.org/10.1097/00001888-200503000-00008
Creswell, J. W. (1985). Faculty research performance: Lessons from the Sciences and the social sciences. ASHE-ERIC Higher Education Report No. 4, 1985. https://eric.ed.gov/?id=ED267677
Dundar, H., & Lewis, D. R. (1998). Determinates of research productivity in higher education. Research in Higher Education, 39(6), 607–631. https://doi.org/10.1023/A:1018705823763
Finkelstein, M. J. (1984). The American academic profession: A synthesis of social scientific inquiry since World War II. Ohio State University Press.
Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 90–93. https://doi.org/10.1001/jama.295.1.90
Green, K. C., & Gilbert, S. W. (1995). Academic productivity and technology. Academe, 81(1), 19–25. https://doi.org/10.2307/40250722
Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569–16572. https://doi.org/10.1073/pnas.0507655102
Hutchins, B. I., Yuan, X., Anderson, J. M., & Santangelo, G. M. (2016). Relative Citation Ratio (RCR): A new metric that uses citation rates to measure influence at the article level. PLOS Biology, 14(9), 1–25. https://doi.org/10.1371/journal.pbio.1002541
Katz, J. S., & Martin, B. R. (1997). What is research collaboration? Research Policy, 26(1), 1–18. https://doi.org/10.1016/S0048-7333(96)00917-1
Lee, S., & Bozeman, B. (2005). The impact of research collaboration on scientific productivity. Social Studies of Science, 35(5), 673–702. https://doi.org/10.1177/0306312705052359
Lowe, R. A., & Gonzalez-Brambila, C. (2007). Faculty entrepreneurs and research productivity. The Journal of Technology Transfer, 32(3), 173–194. https://doi.org/10.1007/s10961-006-9014-y
Lowenstein, S., Genaro, F., & Crane, L. (2007). Medical school faculty discontent: Prevalence and predictors of intent to leave academic careers. BMC Medical Education, 7(1), 37. https://doi.org/10.1186/1472-6920-7-37
Perry, R. P., Clifton, R. A., Menec, V. H., Struthers, C. W., & Menges, R. J. (2000). Faculty in transition: A longitudinal analysis of perceived control and type of institution in the research productivity of newly hired faculty. Research in Higher Education, 41(2), 165–194. https://doi.org/10.1023/A:1007091104399
Ponomariov, B. L., & Boardman, P. C. (2010). Influencing scientists’ collaboration and productivity patterns through new institutions: University research centers and scientific and technical human capital. Research Policy, 39(5), 613–624. https://doi.org/10.1016/j.respol.2010.02.013
Ries, A., Wingard, D., Gamst, A., Larsen, C., Farrell, E., & Reznik, V. (2012). Measuring faculty retention and success in academic medicine. Academic Medicine, 87(8), 1046–1051. https://doi.org/10.1097/ACM.0b013e31825d0d31
Teodorescu, D. (2000). Correlates of faculty publication productivity: A cross-national analysis. Higher Education, 39(2), 201–222. https://doi.org/10.1023/A:1003901018634
Tripathy, J. P. (2013). Secondary data analysis: Ethical issues and challenges. Iranian Journal of Public Health, 42(12), 1478–1479.
Wingard, D., Garman, K. A., & Reznik, V. (2004). Facilitating faculty success: Outcomes and cost benefit of the UCSD National Center of Leadership in Academic Medicine. Academic Medicine, 79(10), S9–S11