A model for assessing human performace systems: an application in vocational training
Dent, T. B.
The purpose of this thesis was to derive from the adult education, training, and human resource development literature a model to assess human performance systems in a wide variety of situations, and to apply the model in one situation to test its efficacy. After an extensive review of the literature, a seven element performance model was developed. This model suggests the questions which must be asked and the areas to be investigated in assessing a performance system. Woodlands Campus was selected as the site for the case study because of its unique features including an individualized competency-based learning system and computer managed testing. A multi-method approach to data collection was utilized to gather data on objectives and instructions to students and instructors, the tests and test items used, incentives to students and instructors, procedures and processes used in testing students, and the experience and qualifications of instructors. The efficacy of the model should be even more apparent when it is useful in evaluating unique performance situations. The computerized item test bank and test generation system were systematically sampled to assess the quality of the items and the tests generated. Since performance is the goal of competencybased learning, the skills testing component of Woodlands student assessment was systematically sampled to assess quality. A sample of students was interviewed to determine how the assessment system works from the students' perspective and a sample of instructors was interviewed to assess the qualifications and procedures used by instructors in their assessment of students. The study found that the computerized testing system, although large compared to most systems, was a simple system containing a large proportion of poor quality items most of which test only the lowest levels of knowledge. The large number of items, 73,000, provides an inadequate basis for proper testing since it only results in a ratio of 2:1 for total items from which to draw, whereas, experts suggest a 10:1 ratio is necessary. The performance testing ranged from sophisticated to simple with the largest proportion of tests being simple, low-fidelity tests. Recommendations for improvements were made on the basis that three stages were necessary commencing with improvement of the organizational climate, followed by the improvement of staff knowledge and skills which are necessary to improve the student evaluation system. Although improvement of the student evaluation system is the ultimate goal, it is not likely to be achieved by instructors lacking the necessary educational knowledge, skills, and motivation to acquire them.