Review of Researches on Criterion Referenced Tests
welcome to my site teachingonline.net
After having gone onto the detailed analysis of the nature of the problem under study, it becomes more pertinent to undertake review of the studies which directly or indirectly may be related to the problem under study. In fact review of related literature provides the spectrum of the researches which includes the research studies conducted till date. It not only opens the doors to the sources of significant problems but also provides an opportunity to the investigator to ascertain the general trend in the field of investigation. However, every research study equips the investigator with new concept, new knowledge and new ideas so that researcher can meaningfully design the study. Review is important not only from the theoretical point of view, but it also provides insight into the relevance, scope and theoretical basis of the problem, guidelines for choosing an appropriate procedure and tools for the study to be under taken. Realizing the importance of review best (1963) writes: “A familiarity with the literature in any problem area helps the students to discover what is already known, what others have attempted to find out, what methods of attack have been promising and disappointing and what problems remain to be solved.”
The present investigation is the identification of terminal competencies in home science at class XII and evaluation of achievement of these, by students of Gorakhpur district in relation to several back-ground factors.
The researches evidences have demonstrated that number of researches have been conducted in the related areas of the present problem in abroad. However, the investigator has tried to analyze and integrate the available up to-date researches conducted so for in aboard. These studies have definitely thrown some light on the meaningfulness of present research work for convenience the reviewed studies have been classified into two categories:
- Studies regarding mastery tests.
- Studies regarding criterion-referenced tests.
Studies Regarding Mastery Tests.
Carmichael, Dennis. (1973) discussed in his study on two premises. The first is that there are five conditions of readiness which determine the success or failure of educational innovation such as that of mastery learning. These are (I) the desire to change the status quo, (2) a systematic management process, (3) effective leadership, (4) a receptive teaching staff, and (5) financial resourcefulness. The second premise is that instructional innovations, including mastery learning will be successfully implemented and will persist only when the teaching staff, administration board, students and patrons work together to (1) assess student learning needs: (2) analyze existing educational goals, objectives and instructional programmers: (3) drive new goals, objectives and programmes based on needs assessment and problem analysis: (4) implement and monitor revised programmers, including instructional innovations. The assertion is also made that the lack of or the weakening of, any of these conditions will lessen the chances of successful and lasting educational innovation.
Wenteing, Tim L. (1973) found that outcomes of the mastery instruction approach compared to a monastery approach of instruction with varying amounts of feedback being given to students with regard to their responses to multiple choice test items on mastery tests. The mastery strategy was found to be superior in terms of immediate and delayed achievements, but students required 50 percent more time to complete instructions with no difference in attitude. A mastery strategy with partial item feedback appears most desirable when the time trade-off in justifiable.
Block, James H. (1973) presented a group based strategy for mastery learning, exemplified by the work of Banjamin S.bloom, with a detailed discussion of the accompanying operating procedures, and its implications for the classroom teacher.
Morgan, Robert .M (1973) described the implementation of a mastery learning strategy and programmed instruction techniques in Korea (North). It is suggested that these concepts would also facilitate the educational improvement efforts of other developing countries.
Goldner, Ralph H (1973) found that mastery learning can have several advantages for students. It gives a student the feeling of control over his environment; the student acquires self confidence and the student develops an interest in further learning. Bloom’s strategy for mastery learning includes several variables that must be considered: the student’s aptitude, the quality of instruction, the student’s perseverance, and the time allowed for instruction.
Bloom, Benjamin S. (1974) described that a through understanding of time and its use in school learning may help improve the schools, and reviewed the research pertaining to model of school learning, mastery learning and the parameter of time in the learning process.
Brennan, Robert L. (1974) discussed a comprehensive report on mastery tests items. An extensive, critical review of the literature with regard to selected aspect of the criterion referenced and mastery testing fields. Major topics treated include: (a definition, distinctions and background, (b the relevance of classical test theory,(c) validity and procedures for test construction, and (d) test reliability proceeding chapter deals with a treatment of criterion-referenced and mastery item analysis and revision procedures when items are scored in the classical correct/wrong manner. Classical procedures for administrating and scoring items are used. This procedure employs the subjective probabilities typically associated with confidence testing in order to obtain pseudo-classical scores. These scores which have not been considered elsewhere appear to be very useful for item analysis purposes since they have must of the advantages and few of the disadvantages of both classical scores and subjective probabilities.
Garner, Raymond. (1974) answered the question: Is mastery learning an effective strategy in the teaching online of vocational agriculture? Instructional management Problems the individualized instruction approach, and performance based curriculum are topics investigated in answering this question.
Honeycutt, Joan K. (1974) compared the effectiveness of computer-managed instruction (CMI) as an aid to the mastery of factual content, as compared to the method of frequent, preannounce quizzes. Results indicated that CMI group achieved significantly greater mastery of the factual content.
James, F. Geoffrey. (19 74) compared self instructional mastery and non mastery treatments of a specially developed geography unit to determine if there were difference s in learning, retention, and time to testing. The results indicated that high and middle aptitude self instructional mastery treatment students retained more than high and middle aptitude non mastery students. There was no difference between learning and retention for low aptitude students across treatments. Therefore, the differences between aptitudes levels were self instructional mastery units were used. The educational implications were that the mastery treatment facilitated superior achievement at the price of less work covered in the time period, and that mastery treatment for low aptitude students could be effective only with close and careful teacher feedback instead of self instruction.
Okey, James R. (1974) studied the effects of mastery teaching online would have on pupil achievement. A total of 21 teachers participated. Results are reported for the subgroup of third and fourth grade teachers (n=5). Achievement of pupils favored the mastery group for each of the teacher.
Weiser, Jhon R. and others (1974) a model of instruction combining the concepts of learning for master, manicures and multimedia instruction was tested in a comparative veterinary anatomy course. The course, subdivided into twelve manicures, was taught utilizing multimedia instruction. Student feasibility was permitted in choice learning experiences and time of evaluation. To complete each manicures, the student had to attain a mastery level of ninety percent on objectives selected at random. Significant improvements were made in the affective domain with no loss in the cognitive level. This developed model brought about and motivation. The instruction and student felt learning for mastery and minicourses resulted in more accurate certification of the student’s ability, while multimedia instruction permitted greater flexibility in learning.
Breland, Nacy S, (1975) found that PSI students, who are not exposed to a programme in a form classroom situation, show no effective disadvantage when compared to traditionally taught students.
Carlson, John G. (1975) discussed that mastery criterion manipulation may produce behavioral effects. Results of a study investigating three undergraduate classes indicate that the 90 percent criterion class was inferior to both the 80 percent and the varied 60-90 percent criterion class on several measures, including degree of accuracy and final grade.
Mueller, Daniel J. (1975) discussed that educational institutions have at least two major functions: education and certification of competency, and examined the educational strengths and limitations of the mastery learning instruction model with respect to fulfilling these functions. The components of the mastery model are identified as (i) formal specification of a comprehensive set of cognitive objectives; (ii) instruction; (iii) frequent formative diagnostic evaluation; (iv) corrective or remedial instruction measures to remedy learning deficiencies identified in formative evaluation; and (v) criterion-referenced summative evaluation. He holds advantage of mastery instruction is primarily its effectiveness for teaching online basic skill and knowledge to slow learners and students who have not learned how to learn. Consequently, its optimal usefulness is in the elementary grades, especially the primary grades. The model reduces competition among students and reduces student failure and frustration. It is also effective with disadvantage students and slow learners at all educational levels. The models do not do well when implemented in traditionally organized schools with time fixed instructional units. It does not maximize learning for all students. However, the inclusion of an idea of mastery speed along with mastery certification would make mastery grades useful for educational and vocational decision-making.
Shaalvik, Einar M (1975) discussed that, mastery learning is a teaching online strategy that originates from behavior-oriented educational technology and objectives are specified in behavioral terms. It presupposes a hierarchically structured subject matter and a quantitative definition of mental capacity. Further, it is based on the assumption that aptitude-treatment-interactions can be found and that it is possible to base on educational system upon such interactions. These assumptions are found not to be valid, and behavioral criteria of mastery are not consistent with important attitude formation-both mastery learning and educational technology in general are therefore discussed on the basis of a cognitive psychology.
Annarino, Anthony (1976) discussed the design of a physical education curriculum to be used in advanced secondary physical education programmes and in university basic instructional programmes; the design is based on the premise of mastery learning and employs programmed instruction. Include are two programme designs which would permit the implementation of mastery learning instructional strategies.
Reichman, Susan L. (1976) reviewed various procedures and guidelines for the development and construction of criterion referenced tests. A comprehensive model which allows the user to identify and relate specific components which affect the optimal construction and implementation strategies of criterion-referenced tests was proposed. Furthem1ore, it established parameter values which will allow the classification of individuals into mastery or monastery states with prespecified levels of confidence. Although the discussed model incorporates binomial expansions, it uses parameters of selected item for establishing baseline probabilities instead of true scores derived from an assumed population of items.
Harris and Subkoviak (1986) suggested as short-cut statistic for selecting items for mastery tests. They examined three statistical methods for selecting items: (a) the pretest posttest method due to Vox and Vargas; (b) a latent trait or IRT method; and (c) the agreement statistic. A number of distinct data sets were stimulated; and the three item selection methods were applied to each data set for the purpose of studying relationships among the methods. The correlation between the IRT method and the one proposed therein agreement statistic was substantial, suggesting that the later might be recommended as a practical alternative to the former. The results for the pretest-posttest method tended to confirm its well-known limitations.
From the above-mentioned studies on CRM item analysis, the following conclusions can be drawn:
- There have been emerged four types of item statistics for CRM item analysis: (a) difficulty and discrimination values based on classical test theory; (b) pretest-posttest coefficients; (c) item parameters based on IRT; and (d) agreement statistics of Harris and Subkoviak (1986)
- Each type of item statistics has its own limitations. Difficulty and discrimination values based on classical test theory and pre-posttest coefficients are psychometrically weak statistics. The error component of the measurement based on these statistics remains more unexplained. Item statistics based on IRT models are very complex. Agreement statistics can be used only up to the classroom level and they require further research.
- Seemingly trivial variables such as an item format have significant effect on the item parameters.
In short, CRM item analysis has still to get its own Aristotle Gruijter, Dato. ( 1987) discussed about test length in domain referenced test suggests that the average test length in the satisfied sampling can be smaller than the length in random sampling. A further drop in test length can be achieved with special arrangements for the sequencing of items from different strata.