POLICY EVALUATION: A CASE STUDY OF GENOME CANADA PROGRAMMING 2000-2011
Zhang, Lucy Chen, Master of Public Policy, Johnson-Shoyama Graduate School of Public Policy, University of Saskatchewan, Canada, 2013. Policy Evaluation: A Case Study of Genome Canada Programming, 2000-2011. Supervisor: Dr. Peter W. B. Phillips The policy evaluation literature on research programing generally focuses on the cost-benefit of different choices in research systems. This thesis applies evaluation tools to assess the fit between project allocations and the strategic goals of Genome Canada, a major research funding organization in Canada. Genome Canada (GC) was established in April, 2000, to provide funding and information resources related to genomics research. The research targets many key areas, such as health, agriculture, environment, forestry, energy, mining and fisheries. Since then the scientific community has partnered with government, the private sector, and international organizations to fund research projects on genomics related subjects. Four open competitions (I, II, III and Applied Genomics in Bio-products and Crops or ABC), combined with a wide array of more targeted projects, have collectively been allocated more than C$2 billion in total investment for the 2000-2014 period. This study assesses how well these research projects fit the stated goals of Genome Canada. The study assesses the fit between the goals and research investment decisions of GC. As a first step in this research, we conducted a review of Genome Canada operations to develop the background understanding of the system and its structure. After reviewing the goals, structure, selection processes and progress reports, we found that there was no explicit assessment of the fit between the stated goals and resource allocation decisions. This study targets to fill this area. Second, we investigated the methods used by GC to develop and implement their goals. Once we understood these methods, we developed a research approach to assess the fit between the goals and the outputs. The model was built to test each project against the stated overall program objectives, namely to: develop and implement a coordinated strategy for the technology in Canada; bring together industry, governments, universities, research hospitals and the public to support large-scale genomics and proteomics research projects; provide accessibility to science & technology platforms to researchers; and assist in attracting co-funding for projects from both domestic and international investors. Third, we determined that the review processes contain scientific, financial and management criteria. By using the STATA tool, we tested the relationship between the stated goals of the organization and the share of funds allocated to specific projects both in the total pool of investments and the open competitions. The analysis revealed that the overall fit for the entire investment program between 2001 and 2011 was about 35%, which is quite reasonable for such an analysis. We found the most important variable affecting resource allocation was the quality of the principal investigator. Other stated goals of GC were either less important or insignificant. By segmenting the analysis into the open-competition investments alone, we discovered the fit deteriorated (R2 of 34% dropped to 22%), which suggests the directed investments are a stronger fit with the goals. While we could not conclusively determine the cause, it might be attributed to either weaknesses in the competitive process or a particularly effective and strategic effort by Genome Canada staff. Further analysis would be needed to determine this.
DegreeMaster of Public Policy (M.P.P.)
DepartmentJohnson-Shoyama Graduate School of Public Policy
SupervisorPhillips, Peter W. B.
CommitteeCoates, Ken; Evans, Samantha; Mou, Haizhen
Copyright DateJanuary 2014
KEY WORDS: evaluation