Blog Viewer

Volume XLVIII, Number 2

Authors

Xanthe E. Wells
CSIRO

Nigel Foster
CSIRO

Adam Finch
CSIRO

Ian Elsum
Australian National University

Problem Statement

Methods to ensure capital resources are optimally allocated are of interest to Research and Development (R&D) organizations. In large organizations that are scientifically diverse, resource allocation is further complicated because there are differences in capital resource requirements between science areas. Allocation approaches used by institutions are often limited because they are largely retrospective and are based on current operating performance and/or what was allocated in previous years. In addition they do not examine trends in expenditure levels across different disciplines.

Funding, buildings, researchers and support staff, equipment, consumables and operating items are examples of R&D inputs and these inputs are suitable for benchmarking against other organizations. Research equipment expenditures are studied in this paper as they are more suited to benchmarking than some other inputs because similar items are purchased by research organizations worldwide and the equipment market is international (Georghiou, Halfpenny, and Flanagan 2001). In addition, laboratory equipment expenditure levels across different research areas are available for academic institutions in the USA from the National Science Board (2012, 2014). Examination of these data sets shows that expenditure levels differ between disciplines. For example, in the Biological Sciences equipment expenditure is 3% of total R&D expenditure, whereas in Physics it is 3-fold higher. Georghiou et al (2001) also highlights that different science areas will have different levels of infrastructure, including equipment, and labour for optimal outputs.

Georghiou and Halfpenny (1996) noted that despite equipment being a critical component to research progress, there is little information available on how to fund and manage it. We have also noted that there is very little information available on this topic. This study describes an approach to allow more optimal allocation of research equipment and is useful to large diverse organizations because it considers the equipment needs across different disciplines. It is based on benchmarking our organization, CSIRO, against US academic institutions and is the foundation of the allocation process described and discussed.

Observations

R&D Expenditure Levels

R&D expenditure levels are considered to be indicative of innovative capacity because technological progress is dependent upon R&D effort. Expenditure levels are widely reported as a science and technology indicator and a common example is a country’s gross domestic expenditures on R&D as a percentage of total gross domestic product. This is considered to be an indicator of a country’s degree of R&D intensity and is often used as a summary statistic for international comparisons.

There are several types of input required for successful R&D outputs and outcomes and these include funding, buildings, researchers and support staff, equipment, consumables and operating items. The allocation of these resources at optimal levels, particularly capital resources, is obviously crucial to R&D organizations. The allocation process should ideally consider the needs of different organizational units and consider future requirements which may not necessarily reflect those of the past. The extent of change within the different units may also be dissimilar. Resource allocations should ideally anticipate these changes to some degree or at the very least reflect them as they occur.

Resource allocation in large diversified companies and R&D organizations is often based on current operating performance and/or allocations in previous years (Bower, 1986; Scholefield, 1994). Allocating resources for research equipment using these approaches is not optimal. Ideally, resources would be allocated based on the potential for creating value in the future rather than past or current needs. This, however, can be difficult to assess and requires a good understanding of both the relative potential for value creation across all application areas and the organization’s competitive position in each area, as well as anticipating future changes in requirements.

The study outlined in this paper describes an alternative approach to research equipment expenditure allocations. It describes a method that is applicable to diverse R&D organizations that span many disciplines and application areas. Levels of equipment expenditure in other organizations are also examined as they provide a competitive benchmark for comparison. Whilst the method does not consider the future potential for creating value and anticipating future needs, it does take into account differences in the types and equipment requirements across an organization.

R&D Equipment Expenditures across Disciplines

The availability of current and sufficient equipment is a key component to a successful R&D organization. Current and state-of-the-art equipment is necessary to maintain competitiveness and to attract high calibre researchers into the organization. The relative importance of R&D equipment as an input is highlighted in the recommendations for external peer review of research organizations. The National Research Council (2012) recommends that equipment and facilities be considered as part of an assessment of ongoing research, together with evaluating the technical projects and the quality of research staff and management.

Equipment is more suited to benchmarking than some other R&D inputs because similar items are purchased by research organizations worldwide and the equipment market is international (Georghiou, Halfpenny, and Flanagan 2001). Collins, Couper and Record (1990) examined research expenditure in UK institutions across three sectors – industry, research council institutes and higher education institutions. Four disciplines were studied – biochemistry/pharmacology, chemistry, electrical engineering and electronics, and plant sciences. Whilst R&D equipment was not specifically examined, expenditure per researcher on non-pay items, including equipment, was measured. Overheads were excluded from this category. In universities, expenditure on non-pay items was the same for the three science disciplines and 2.6-fold higher for electrical engineering and electronics. Industry non-pay item expenditure levels on biochemistry/pharmacology were 3.2 times greater than electrical engineering and electronics levels. This study highlighted that R&D expenditure on non-pay items, including equipment, is dependent on the field of research and the sector in which the research is being conducted. Other studies in the open literature on equipment management, including expenditures by research field, are scarce.

Comprehensive R&D equipment expenditure data by research field is readily available from the National Science Foundation (NSF), which conducts an annual survey of US academic institutions on R&D funding sources and expenditures. Aggregated results from this survey are summarized in the Science and Engineering Indicators (2012, 2014) and provides a high level analysis of equipment expenditure trends in the US. The data behind this analysis is publicly available making it possible to analyze equipment expenditure as a proportion of total R&D expenditure by research field. This enables an organization or country to apply its research profile according to research field and compare itself to expenditure levels in US academic institutions. This comparison was undertaken for our organization, CSIRO, to provide a greater appreciation of its current equipment situation and needs.

CSIRO and its Research Equipment Funding Allocations

CSIRO (Commonwealth Scientific and Industrial Research Organization), is Australia’s national science agency and is one of the world’s largest and most diverse Research and Development (R&D) organizations. It employs approximately 5,300 people and has an annual budget of AUD1.2 billion. CSIRO conducts research in a wide range of disciplines within the following science and engineering areas – Energy; Environment; Manufacturing, Materials and Mining; Information Sciences; Food, Health and Life Sciences Industries. Its research results are used in a wide range of market sectors, including food and agribusiness, energy sector (solar, oil and gas), mining equipment technology and services, advanced manufacturing and medical devices, and materials. In 2012-13, CSIRO was broadly organized into 5 Groups and its constituent 12 Divisions are allocated to one of these.

CSIRO’s annual capital budgeting process, which includes allocations for research equipment, is run centrally in two stages. Firstly a decision on the total amount for research equipment for the whole organization is made; followed by decisions on the allocation for each Division. Decision- making is a collective process involving the heads of the 5 Groups and the corporate executive responsible for science strategy. The heads of the 5 Groups know their Divisions within their area of responsibility but they have much less knowledge of Divisions in other Groups.

This capital budgeting process shares a number of features with resource allocation in large diversified companies, with similar tendencies to allocate resources based on current operating performance and/or allocations in previous years (Bower, 1986; Scholefield, 1994). As discussed above, allocating resources for research equipment based on this approach is not optimum.

Evaluation and Analysis of Concept

Calculation of R&D Equipment Expenditure Benchmark Values by Research Field

Table 1 shows the research equipment expenditures by research fields in US academic institutions over a five-year time period. These results were calculated from the National Science Foundation (NSF) survey described in the previous section, which provides aggregated data for R&D equipment expenditure and total R&D expenditure by research field. This allows the calculation of the proportion of expenditure on R&D equipment by research field, as well as an overall dollar value. Fields in the Physical Sciences have the highest level of equipment expenditure and overall it is 3-fold higher than equipment expenditure in the Life Sciences. Equipment expenditure levels are lowest in the Social Sciences.

Table 1. Average Expenditures of Funds for Research Equipment at US Academic Institutions as Percentage of Total Academic R&D Expenditure by Field 2005-2009.

The survey includes expenditure for indirect costs. The National Science Foundation describes research equipment as ‘movable items necessary for the conduct of organized research projects’. It is also noted that ‘generally, university equipment that cost less than $5,000 would be classified under the cost category of ‘supplies’. However, private communication with the National Science Foundation indicates that institutions set their own thresholds and this may be as low as $1,000 in some institutions. The overall value of equipment purchased below these thresholds is not known and it is therefore not possible to account for such equipment in this analysis. The survey results do not include data on research facilities and cyber infrastructure as this information is collected in a separate mechanism, the Survey of Science and Engineering Research Facilities. In this survey, facilities are assessed by using metrics on research space, repairs and renovation; and cyber infrastructure is assessed by networking, high-performance computing and data storage indicators.

CSIRO’s Research Profile

CSIRO is a large multidisciplinary national science agency with total R&D expenditure of approximately AUD1.2 billion. During 2008-12, CSIRO produced 12,360 research publications that were captured in the Thomson-Reuters Web of Science (WoS) publication data base and the overwhelming majority of these (10,951, 89%) were journal articles. In addition, there were 751 conference proceedings papers and 657 reviews. The Web of Science database classifies research journals into approximately 250 subjects and the diversity of CSIRO’s research profile is evident by its publication profile according to Web of Science subjects. The two largest subjects for CSIRO are Environmental Sciences and Ecology, between them accounting for 14% of publications. Though slightly smaller in output, CSIRO also focuses heavily on the subjects Materials Science (Multidisciplinary), Plant Sciences and Astronomy & Astrophysics, each of which represent around 6% of CSIRO’s output. Overall, there are over 80 subjects in which CSIRO produced 40 or more publications during this 5-year period.

In 2013, CSIRO was comprised of 12 Divisions, each with a unique publication output profile. WoS subject area profiles have also been generated for each of these and Table 2 lists the largest subject areas for each Division. The multidisciplinary research profile of CSIRO is evident from this Table.

The next step in this study was to compare levels of equipment expenditure in CSIRO to levels in US academic institutions and to achieve this Web of Science subjects were mapped to the research fields used by the NSF. The resulting translation table is shown in the Appendix. A research profile for CSIRO and individual Divisions by research fields used by the NSF was then produced and the profile for CSIRO is shown in Figure 1. There are six fields each comprising more than 5% of CSIRO output. The largest is Biological Sciences, followed by Environmental Sciences, Earth Sciences, Chemistry, Physics and, lastly, Metallurgical and Materials Engineering.

Table 2. CSIRO Divisions and the Main Web of Science Subjects for their Research Publications produced between 2008-12.

Figure 1. CSIRO Publication Output by National Science Foundation Science and Engineering Fields, 2008-12.

CSIRO’s R&D Equipment Expenditure and Comparison to US Academic Institutions

The following steps were taken to calculate benchmark values for CSIRO and its Divisions and are summarised in Figure 2. The research profile for each entity based on research publications was determined as described in section 3 above. The benchmark equipment expenditure percentages for each contributing research field (see Table 1) was then multiplied by the percentage of publications in the research field and expressed as a percentage. These values were then totalled to provide an overall benchmark value for CSIRO and each Divisions and this was then compared to the actual expenditure as a percentage of total R&D expenditure calculated (Table 3). CSIRO’s average actual expenditure for a four-year period was also calculated from internal finance records.

Figure 2. Flow Diagram of Benchmark and Actual Equipment Expenditures. NSF = National Science Foundation

CSIRO defines its assets as items over AUD3,000 (approximately USD2,250) with an enduring value in use. They are categorised into several classes including Land, Buildings, Vehicles, Office, Scientific and IT. Only items in the Scientific category were included in this study. The AUD3,000 threshold value is relatively high in the Australian public sector, however, it has been nominally constant for at least the last 15 years and therefore has decreased in real terms. These values do not include major research facilities such as the Australian Square Kilometre Array Pathfinder, although some smaller items associated with this may be included in R&D equipment expenditure. CSIRO also includes overheads as part of its overall R&D expenditure, as does the NSF.

In 2012-13 financial year, CSIRO’s total R&D expenditure was AUD1.16 billion. At the whole- of-organisation level, actual R&D equipment expenditure was one-third lower than the calculated benchmark value. Whilst this was recognised as a serious issue, overall budget constraints did not allow the organisation to meet the benchmark expenditure level. One Division is above the benchmark value and another is close to the value. Of the remaining Divisions, all were 85% or lower, with 3 Divisions being less than half of its benchmark level.

Table 3. CSIRO and its Divisions - Total R&D Expenditure, Equipment Benchmark Percentage and Actual Equipment Expenditure Percentage

Equipment Expenditure Trends in US Academic Institutions

The benchmark data source for this study is from the NSF which collects data from US academic institutions. It was chosen because it is readily available and covers many institutions over a long time period. Data is also classified on a research field basis and no other data set as complete as this has been identified. This is the main reason why this dataset was chosen for this study. The US data set is comprehensive, covers a wide range of academic institutions and it is generally considered that US R&D is of good quality globally. In addition, the survey is updated every second year and this allows for ongoing evaluation.

The proportion of total R&D equipment expenditure in US academic institutions has declined from 4.1% in 2005 to 3.2% in 2012, whereas actual total expenditure has increased (Figure 3). The Science and Technology Indicators (2014) indicate that FY 2012 equipment expenditure ‘fell to the lowest levels measured in constant dollars since FY 2001’. There is no information available to indicate whether or not these levels of equipment expenditure are adequate but it is likely that they are not in some US institutions, especially given the drastic downturn in the US economy following the Global Financial Crisis. Witonsky (2011) noted in a report covering all types of US laboratories, including academic, hospital and biotechnology laboratories, that the economic downturn had greater impact on equipment budgets than any other laboratory expenditure category. In addition, there may have been other external factors contributing to this decline.

Figure 3.Trends in R&D Equipment and Total Expenditures in US Academic Institutions, 2005- 12. DATA SOURCE: Science and Technology Indicators, National Science Board, 2014.

Reflect and Recommend Solutions

This paper explores a new approach to inform the allocation of R&D equipment expenditures and outlines the implications of the findings to our organization, CSIRO. The method could also be applied to other large multidisciplinary organizations because it considers the variation in equipment needs across a range of disciplines.

There were several implications of this study to CSIRO. The study confirmed the anecdotal evidence that the organization was spending less on its R&D equipment than the benchmark figure suggested to be optimal and provided an approximate quantification on the extent of this gap. It was not, however, possible to allocate additional funds to make up this short fall. The study also highlighted to the organization the differences in capital intensity of different research fields and how this applied to the organization. The process provided a clear and easily understood basis for allocations for R&D equipment. Consequently, the initial notional allocations, resulting from simply applying the ‘algorithm’ to each Division, were perceived as a fair and equitable starting point for making final allocations. Discussion among the decision-makers was then restricted to whether there were any exceptional circumstances that would warrant changing the allocations at the Group level. Generally there were not; the allocations at the Group level were accepted and then small adjustments were made within each Group to take account of the circumstances of each Division in that particular year, for example, the need to purchase an unusually expensive piece of equipment. More recently, the method has assisted managers to understand the equipment needs across the organization following a significant restructure and continues to guide discussions on allocations.

The study used an organizational research profile based on the Web of Science publication classification but it could readily be replaced with other publication classification schemes or other parameters, such as financial or human resources data, provided it was allocated on a field of research basis. Not only was the approach applicable to an entire organization, but it was possible to examine smaller parts of the organization in more detail.

Whilst benchmarking CSIRO would ideally involve gathering data from peer national research agencies, this was not attempted. The benchmark data source for this study is from the NSF which collects data from US academic institutions. It was chosen because it is readily available and covers many institutions over a long time period. Data is also classified on a research field basis and no other data set as complete as this has been identified. This is the main reason why this dataset was chosen for this study. The US data set is comprehensive, covers a wide range of academic institutions and it is generally considered that US R&D is of good quality globally. In addition, the survey is updated every second year and this allows for ongoing evaluation. The Main Science and Technology Indicators (2014) from the OECD provides data on two capital expenditure categories – ‘land and buildings’ and ‘instruments and equipment’ – by research sector for its member countries. This was also examined but not used because information on expenditure per research field is only provided for the combined capital expenditure categories and, therefore, is not at the required level of detail for the purposes of this study. Ideally it would be more optimal for CSIRO to benchmark against other government R&D organizations but this information is not available. In addition, CSIRO does have a reasonably high level of publication output and it is generally considered within the organization that the publication research profile is a sound representation of its overall research profile.

It is important to note the approach described in this study should be used to provide guidance rather than absolute answers because of some limitations as outlined below:

1. The method uses a research publication output to define the research profile of the organization. This would not be applicable to an organization whose publication profile does not reasonably reflect its research areas. If this is the case, financial or human resources data would be a better choice. This could include private R&D companies or applied R&D organizations that do not publish to the same extent as academic organizations.
2. It is likely that the levels of R&D equipment in US academic institutions are probably not optimal, especially in recent years, due economic conditions. This should be kept in mind if an organization is using this approach to determine optimal R&D equipment expenditure levels.
3. The definition of ‘R&D equipment’ varies between organizations as well as within different parts of an organization and it is not possible to readily account for this variation. At the lower end of the monetary scale, the threshold cost values of equipment and supplies varies between organizations. CSIRO classed equipment in this study as equipment assets valued at AUD3,000 (approximately USD2,250) and above. This was slight different to the US equipment data as this was collected from multiple institutions and the threshold varied from USD1,000 to 5,000. Both sources did not include major research infrastructure. Geoghiou (2001) also highlights that there are ‘methodological, conceptual and practical challenges involved in any attempt to systematically benchmark research equipment.’
4. The National Science Foundation data provides a level of R&D equipment expenditure for academic institutions. It is likely that this level would also apply to organizations for which producing new knowledge is important. This level, however, may not be appropriate for other types of institutions such as industrial ones.

A survey of UK universities (Georghiou and Halfpenny, 1996) showed in 1994/95 they spent 13% of their research income on research equipment. This is higher than levels reported in US academic institutions and there are likely to be differences in survey methodologies, as well as different funding and reporting approaches. The higher UK level does not appear to be due to adequate or more equipment being available in UK universities because 79% of UK university departments indicated that they were unable to perform critical experiments due to inadequate equipment. It was also found that 38% of researchers surveyed indicated that their equipment was older than their international peers and it had a higher cost for repairs and maintenance. In addition, it was found that the research equipment required to remain competitive is becoming more expensive. Values of these increased costs were provided as follows:

1. Replacement of equipment with equivalent functionality averaged 1.37 times the original cost of current equipment.
2. The cost of new equipment with a level of functionality similar to the current state of the field as the original item was when it was purchased averaged 1.56 times the original cost.
3. Replacement with state-of-the-art equipment averaged 2.67 times the original cost.

Whilst this report was produced some time ago and it is likely that the actual increases reported may have changed; it highlights that for an organization to maintain its relative competitiveness it needs allow additional funds to replace out-of-date equipment.

Our study does not consider the age and condition current R&D equipment and an understanding of this in CSIRO would require further investigation. If, however, an organization is not sufficiently spending on its R&D equipment, it is very likely that it is not replacing equipment as it ages. It is therefore likely to have a significant amount of out of date equipment, as well as equipment requiring repair, in its laboratories.

R&D equipment undoubtedly has a key role in producing research outputs. Whether or not the equipment is being used optimally in terms of output is difficult to assess between types of institutions and different research fields. A broad assumption would be that academic institutions aim to produce more publications in research journals whereas industrial institutions would produce more intellectual property. There are also differences in publication and intellectual property output levels between research fields. CSIRO researchers in the environmental and space sciences produce relatively high levels of publications and low levels of patents, whereas its researchers in engineering fields tended to have higher patenting rates and biotechnology- related CSIRO researchers produced reasonable amounts of both publications and patents. To determine benchmark levels of output relative to equipment expenditure would therefore need to take into account the research field and whether or not the institute has an industry or academic focus. Georghiou, Halfpenny, and Flanagan (2001) also highlights that different areas of science will have different levels of infrastructure, including equipment, and labour for optimal outputs.

Conclusions

This study describes an alternative approach to inform the more optimal allocation of R&D equipment resources because it considers the variation in equipment needs across different disciplines. The approach would be useful to large diverse organizations, such as CSIRO, that span many disciplines and application areas. Levels of equipment expenditure in other organizations are also examined because it provides a competitive benchmark for comparison.

The study provided CSIRO an assessment of the level of equipment expenditure at the organizational level compared to other institutions. It also identified units within CSIRO with the lowest levels of equipment expenditure. This information is useful in both strategic and operational contexts.

This work could be further expanded by investigating the age and condition of current equipment and quantifying this at the organizational level, as well as further understanding optimal levels of equipment in relation to research outputs, such as intellectual property and publications. It would also be more relevant to benchmark CSIRO against national research agency peers rather than academic institutions. The study also noted how little information on R&D equipment management and expenditure is available in the literature and it is hope that this work will stimulate further discussion on the topic.

Xanthe E. Wells
Executive Manager CSIRO
5 Julius Ave., North Ryde, NSW, 2113, Australia
+61 2 9490 8194
Email: xanthe.wells@csiro.au

Nigel Foster
Project Manager CSIRO
5 Julius Ave., North Ryde, NSW, 2113, Australia
+61 2 9490 8366
Email: nigel.foster@csiro.au

Adam Finch
Analyst CSIRO
Private Mail Bag 2, Glen Osmond, SA, 5064, Australia
+61 8 8273 8105
Email: adam.finch@csiro.au

Ian Elsum
Formerly at CSIRO
Currently Visiting Fellow
Research School of Management
The Australian National University
Canberra, ACT, 0200, Australia
Email: ian.elsum@anu.edu.au

Correspondence concerning this article should be addressed to Xanthe Wells, Executive Manager, CSIRO, 5 Julius Ave., North Ryde, NSW, 2113, Australia, xanthe.wells@csiro.au.

Appendix

Table 4. Translation of Web of Science Subjects to NSF Science and Engineering Fields.

References

Bower, J. L. (1986). Managing the resource allocation process: A study of corporate planning and investment (rev. ed.). Boston, MA: Harvard Business School Press.

Collins, P. M. D., Couper, C. J., & Record, G. C. (1990) The structure of research expenditure. London: Science and Engineering Policy Studies Unit, The Royal Society.

Georghiou, L. G., & Halfpenny, P. (1996) Equipping researchers for the future. Nature, 383, 663-664. doi:10.1038/383663a0

Georghiou, L. G., Halfpenny, P., & Flanagan, K. (2001). Benchmarking the provision of scientific equipment. Science and Policy, 28(4), 303-311. doi:10.3152/147154301781781435

Organisation for Economic Co-operation and Development. (2014, July 14). Main Science and Technology Indicators, 2014(1). doi:http://dx.doi.org/10.1787/msti-v2014-1-en

National Research Council. (2012). Best practices in assessment of research and development organizations. Washington, D.C.: National Academies Press. doi:https://doi.org/10.17226/13529

National Science Board. (2012). Science and engineering indicators 2012. Arlington VA: National Science Foundation (NSB 12-01). Retrieved from https://www.nsf.gov/statistics/seind12/pdf/seind12.pdf

National Science Board. (2014). Science and engineering indicators 2014. Arlington VA: National Science Foundation (NSB 14-01). Retrieved from https://www.nsf.gov/statistics/seind14/ content/etc/nsb1401.pdf

Scholefield, J. H. (1994). The allocation of R&D resource. R&D Management, 24(1), 91-97. doi:10.1111/j.1467-9310.1994.tb00850.x

Witonsky, J. (2011, November 4). Laboratory spending trends. Lab Manager. Retrieved from http://www.labmanager.com/surveys/2011/11/laboratory-spending-trends#.WOSDwGct6cI

Keywords

Research and development expenditures, laboratory equipment, research equipment, benchmarking, organization, management