Enhancing Data Collection and Utilization of HERD: Survey Results

By SRAI News posted 04-09-2025 04:21 PM

Recommend

Grant Development & Strategy

Enhancing Data Collection and Utilization of HERD: Survey Results

Researchers at the University of Kentucky conducted a comprehensive research study of the National Center for Science and Engineering Statistics (NCSES) Higher Education Research and Development (HERD) Survey, supported by the National Science Foundation (NSF) through Grant #2215223. One part of the study was to assess the perceived levels of difficulty around collecting and reporting the requisite responses for the HERD survey, as well as the utilization of HERD survey results and the effectiveness of current data collection practices among research administration professionals.

The HERD survey is an annual census of United States (U.S.) colleges and universities, which collects data based on expenditures to support research and development activities at the institution during the fiscal period. The data collected from the HERD survey provide important insight into the trends related to research at both the institutional and national levels. Although participation in the survey is voluntary, the overall response rate is consistently high. The NCSES also provides a variety of cumulative data tables, which are used by institutions and federal agencies to help inform decision-making.

The Higher Education Research & Development (HERD) Results Utilization and Data Collection Survey Full Implementation Study was conducted in June 2023. The HERD Survey study was administered via an anonymous online link and used a snowball sampling methodology. A primary goal of this element of the overall study was to better understand the roles and responsibilities of various positions within the institution related to the HERD survey, as well as the perceptions of those tasked with preparing versus applying the results of the survey. Additional observations were made to identify potential clarifications that could improve the data collection and reporting process. This HERD Survey study highlights the critical role this annual, national data collection plays in shaping the research landscape across U.S. higher education and policy development.

Roles and Experience of Respondents

Respondents come from various roles within their institution, but they are primarily concentrated in analytics, finance, and management within research administration. For example, some of the common job titles observed in the data include (Assistant) Director, Assistant or Associate Vice President, Business Analyst, Research Administrator, and Senior Data Analyst.

Of the 159 HERD survey respondents, 106 (67%) respondents were directly involved in HERD survey-related data collection as part of their job responsibilities. Forty-four percent of respondents had five years or less experience specifically in HERD survey data collection, and 55% had six or more years of experience completing the data collection. On average, respondents reported spending 1.96 months collecting the data required for survey completion.

Perceived Difficulty of Specific Questions

Examining the perceived ease of completing each question on the HERD survey, three questions were identified as the most difficult to complete:

Question 15: Demographic and educational categories of R&D personnel.
Question 16: The number of full-time equivalents (FTEs) devoted to R&D.
Question 6: The R&D expenditures from basic research, applied research, and experimental development categories.

Conversely, the preparers reported these three questions as the easiest to complete:

Question 13: Institutional capitalization thresholds for software and equipment.
Question 4: The R&D expenditures from medical school.
Question 12: Direct costs and indirect costs.

Utility and Trustworthiness of HERD Survey Results

Senior leadership plays a critical role in leveraging HERD survey results for institutional strategy. Leaders and decision-makers reported use of HERD data for three primary purposes: a) to identify growth areas in specific disciplines (i.e., engineering or natural sciences); b) to benchmark against peer institutions to evaluate competitiveness in securing federal funding; and, c) to inform decisions about resource allocation to underfunded research areas.

Regarding trustworthiness, respondents generally held a neutral attitude towards the trustworthiness of HERD survey results. Some respondents have concerns over the consistency of data across institutions, such as varying interpretations of HERD survey definitions and methodologies.

Value placed on the HERD Survey

On average, respondents agreed that senior leadership places a high priority on the compilation of accurate responses for the survey at their institution. Respondents also generally agreed that the work of the HERD survey data collection personnel is valued by their institution.

Possible Improvements to Data Collection Process and Reporting

While the survey is widely considered a valuable tool by survey respondents, some improvements may be worth consideration. Based on feedback from respondents, areas for potentially improving the data collection process and resulting reporting of the HERD survey are summarized as follows:

Refine instructions and definitions
Respondents recommended clear and consistent definitions as highly beneficial. For example, they suggested the creation of a comprehensive glossary of key terms and definitions to standardize interpretations across institutions. Additionally, more detailed guidance for subjective questions may be needed, such as those requiring classifications of research expenditures and personnel categories.
Build Automatic and Integrated Data Collection Systems
Many respondents emphasized the need for automation and integration to reduce the manual burden of data collection. They suggested incorporating the HERD survey into existing enterprise resource planning (ERP) systems to streamline data extraction and developing standardized templates or software tools to automate calculations and categorizations.
Training and Support
They further emphasized the value of training sessions and ongoing support from NCSES, such as hosting webinars and workshops helping survey preparers address common challenges and solutions, as well as offering one-on-one consultation sessions or office hours to address specific questions.
Accelerate Reporting
Respondents highlighted the importance of faster reporting, such as reducing the lag time between survey submission and the release of results. This would ensure the relevance, timeliness, and useability of HERD survey results for strategic decision-making — rather than lagging measures of research and development spending.
Tailor the Support for Under-Resourced Institutions
Among participating institutions, we noted that there was disparity in resources and systems. Smaller and less research-intensive institutions indicated they often face challenges in allocating the limited resources to complete the survey, such as staff and technology resources or expertise. Respondents suggested that NCSES provide additional support specifically tailored to smaller institutions to simplify reporting options. Meanwhile, the desire for encouraging collaboration between institutions to share information, best practices, and resources regarding HERD data collection was also expressed. Respondents also emphasized more funding opportunities to support infrastructure improvements for under-resourced institutions to assist with these efforts.

If NCSES administrators could continue their efforts to improve and simplify the HERD survey process for institutions of all sizes, such as easing data collection, refining survey questions, increasing training and documentation to assist with accurate and consistent interpretation and responses, all would benefit. These improvements will not only reduce the burden on preparers but also ensure trustworthiness and enhance utility of the survey results for institutional and national decision-making, eventually maximizing its impact.

This article is based upon work supported by the U.S. National Science Foundation under grant number: 2215223. Any opinions, findings, and conclusions or recommendations expressed in the materials are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

The full report and analysis of the HERD survey section of this study is available online and can be accessed here: https://doi.org/10.13023/cshw-5s09.