Assessment is a meant to an end and not an end in itself (Miller, Linn and Gronlund 2009:3)
Young academics versus experienced academics
During first few years of teaching, assessment of student understanding is simply a chore with a purpose no more important than to produce a reasonable distribution of final grades (Rebeck and Asarta 2012:177). After a few semesters of experience, new instructors begin to look more deeply into test results. They start asking questions such as: Why does half of the class not understand the difference between demand and quantity demanded? Is it the presentation of the material? Is it student effort? Is it a problematic question I continue to use?
Thoughtful instructors might then question how important it that the students understand certain concepts; whether or not understanding of the concept is critical, o even consistent with the goals of the course (Rebeck and Asarta 2012:177). Then assessment becomes more than a tool used to assign grades, it becomes a process by which assessors answer these questions (Rebeck and Asarta 2012:177).
What is assessment?
Assessment involves the gathering of data, often from exams, to be used to grade students. But, assessment limited to this task leads an instructor to miss tremendous opportunities to improve the course and ultimately advance student achievement over time (Rebeck and Asarta 2012:178). A sound assessment strategy in an economics course consists of both a) gathering data to summarize the level of understanding of students and b) using this data to evaluate current effectiveness of the course and make changes to the course to increase student understanding (Rebeck and Asarta 2012:178). The second component involves identifying areas where the students are thriving, where it only modestly influence student learning, where it might have a weak or no effect, and areas where the effect it does have does not further the objectives of the course (Rebeck and Asarta 2012:178). In economics, the assessment strategy can be described in terms of summative and formative assessment.
Summative assessment measures and describes the level of achievement of the students. Torrance and Pryor (1998) defines summative assessment as assessment that occurs at the end of a course to measure and communicate pupil performance for purposes of certification and (latterly) accountability. It is designed to determine assignment and course grades or for certifying student mastery of intended learning outcomes (Miller et al 2009:39). If a student receives a score of 86 out of 100, it provides an indication of the student’s level of achievement. The scores can be used to assign grades, and it provide feedback to the student with regard to how much they understand relative to how much he or she was expected from them to understand. For the instructor, summative assessment provides information used to assign grades, but most importantly information on the effectiveness of the course as it has been implemented and as it relates to course objectives (Rebeck and Asarta 2012:178). Traditionally, most instruction include lecture and exams, where exams are a form of summative assessment (Faulk 2007:1). Most recently, educators have become more cognizant of formative assessment techniques and the role such techniques can play in facilitating student learning (Faulk 2007:1).
Formative assessment occurs during the course with the express purpose of improving pupil learning. Formative assessment is used to produce information that provides feedback to students to help them to gauge their performance and to improve their learning process (Rebeck and Asarta 2012:178). The assessment techniques provide information to students and instructors about student understanding and course topics before summative assessment (such as examinations) takes place (Faulk 2007:1). It is an ongoing process implemented through the duration of the course. This information is used to monitor learning progress during instruction (Miller et al 2009), identify misunderstanding and confusion about topics, and to correct these misunderstandings before summative assessment takes place (Rebeck and Asarta 2012:178). The instructor can modify the course, and if the feedback is immediate, these changes can take place quickly (Rebeck and Asarta 2012:178).
Functions of formative assessment
According to Wininger and Norman (2005:24) formative assessment have three key functions, namely:
- informing instructors about student learning during instruction with the purpose of guiding and modifying instruction
- providing corrective feedback to students about learning progress for the purpose of guding and improving learning, and
- enhancing student motivation.
Some issues involving these functions are (Faulk 2008:3):
- getting students to respond to feedback
- presenting feedback in such a way that students understand what they need to do to improve
- getting students to understand that feedback is being given if a non-traditional approach is used.
The role of feedback during formative assessment
It is important to note that the literature is unclear about whether feedback or motivation is the dominenat effect resulting from formative assessment (Faulk 2008:4). According to Sadler (1989:122) formative assessment includes both feedback and motivation, and the goal of instruction is to facilitate the transition from feedback to self-monitoring, where feedback is information from an external source, and self-monitoring is information from the learner.
A common problem with feedback is that students receive valid judgements (feedback) on the quality of their work, but they do not necessarily improve (Faulk 2008:4). Therefore, Sadler (1989:121) identified three necessary conditions for effective feedback, namely:
- the learner must know what the goal (ideal standard) is
- the learner’s actual performance must be compared to the ideal performance and
- the learner must engage in actions to close the gap between actual and ideal performance.
Activities constituting formative assessment
One of the issues not explicitly addressed in the literature is exactly what types of activities constitute formative assessment (Faulk 2008:4). According to Faulk (2008:4) formative assessment can take many forms such as:
- teacher’s comments on protfolios or assignments
- peer evaluation
Black and William (1998) note that feedback need to be provided in order for assessments to be formative. It is however not clear which activities are the most effective way to provide feedback for the purpose of formative assessment (Faulk 2008:4).
Formative assessment in higher education
A limited number of studies have focused on formative assessment in higher education.
According to Black and William (1988) formative assessment is effective in a variety of educational settings, circumstances and disciplines and that the quality of feedback is crucial. They pointed out that formative assessment is not used widely.
Slater, Ryan and Samson (1997) focused on portfolio assessment versus traditional objective examination during the semester to compare final exam scores. They did not find a significant difference in scores, but the portfolio group were less anxious, they sped more time reading and studying outside the class and they have enjoyed the learning experience.
Johnson, James, Lye and McDonald (2000) did not focus explicitly on formative assessment, but they have evaluated collaborative problem solving as a method for learning economics, The students discussed issues and problems relevant to intermediate macroeconomics theory during the small group sessions outside the classroom. This might be considered as a type of formative assessment since the students received immediate feedback from group members and the tutor (Faulk 2008:5). The method was particularly beneficial (in terms of a higher course grade) for high attending, below-average international students.
Lan, Bradley, and Parr (1993) examined self monitoring in a graduate-level introductory statistics course. Students in the treatment group filled out a self-monitoring protocol as they proceeded through the course. Another treatment group monitored intructor’s teaching activities using a similar protocol. Results of the study indicate that students participating in the self-monitoring protocol found it an efficient way to gain information about their learning and that these students scored higher on exams.
Stefani (1994) focuses on the reliability of peer and self assessment (in terms of assigning grades) relative to assessment by a professor. She finds that students participating in peer and self assessment give grades similar to the professor and that peer and self assessment motivates students to reach higher levels of achievement.
Keefe and Eplion (2007) investigate the effects of online formative assessment using pretests in an upper level undergraduate business management class and find that formative assessment serves to motivate students to read more chapters and attend more classes which lead to higher course grades. This motivational effect prior to feedback accounts for nearly all of the impact of formative assessment on student achievement.
Finally, a study by Evensky et. al. (2007) examines the relationship between academic performance in principles of economics and feedback on course engagement measured through an educational software called SAGE (Self-Assess, Grow, Educate). Randomly selected students use the SAGE software to complete a weekly on-line survey that documents the student‟s effort in the course – class attendance, hours studying, assignments completed, etc. The goal is to help students develop the metacognitive skills needed to succeed in college. The software provides feedback on the individual‟s effort relative to the class average. A control group filled out a weekly survey on their social life. Students in the treatment group (SAGE participants) scored 20 percent higher in the course than students in the control group (social life survey).
The literature on formative assessment in higher education offers limited guidance for economics principles courses. The types of formative assessment and methods for measuring student success have varied in the few studies examining formative assessment in higher education. A general conclusion that can be drawn is that formative assessment has various positive impacts on a variety of measures of student learning and methods of learning.
While the literature that explicitly focuses on formative assessment in higher education is limited, some formative assessment techniques can be viewed within the broader context of teaching with active approaches. Becker (1997) and Becker and Watts (1998) note that the most prevalent teaching method in economics is the lecture or “chalk and talk” and call for the integration of alternative methods including (but not limited to) cooperative learning and economics experiments. Cooperative learning methods and economics experiments have been shown to have positive effects on student achievement. See Bartlett (1998) for a brief review of the literature on cooperative learning. Emerson and Taylor (2004) provide an analysis of a well designed study showing the positive effects of economics experiments on student achievement. As described below formative assessment techniques may be designed to involve active learning.
Suskie (2009:4) discuss four steps requires of an effective assessment program, namely:
- Establish clear, measurable expected outcomes of student learning
- Ensure that students have sufficient opportunities to achieve these outcomes
- Systematically gather, anayze and interpret evidence to determine how well student learning matches expectations
- Use the resulting information to understand and improve student learning.
When the fourth step is included, assessment becomes a process used to consider and revise the approach to teaching certain concepts with the goal to improve student learning (Rebeck and Asarta 2012:179). The process starts with stating the learning outcomes that a student has to achieve in a given setting (Rebeck and Asarta 2012:179). According to Allen (2004) departmental goals are usually too general to guide assessment and planning. Therefore, faculty need to develop more specific learning objectives to operationalize these program goals (Rebeck and Asarta 2012:179). The learning objectives must describe observable behaviours that faculty can measure (Rebeck and Asarta 2012:179).
Saunders (1998) suggests that instructors begin with a few general outcomes such as:
- Knows basic terms
- Applies economic principles to new situations.
Within these general objectives, Saunders (1998) recommends listing specific illustrative behaviors such as :
- Uses terms correctly
- Predicts the probable outcome
When applied to specific concepts covered, he argues that these objectives must encompass the conditions in which the student will be able to accomplish the objective, and the criteria used to judge his or her achievement (Saunders 1998).
Hansen (1986, 2001) provides a framework for what should be included in these objectives. His ‘proficiencies approach’ identifies what students should be able to do at the end of a course, and can help instructors better focus on what specific outcomes are important and how to achieve these outcomes. The six proficiencies are:
- Access existing knowledge
- Display command of existing knowledge
- Interpret existing knowledge
- Interpret and manipulate economic data
- Apply existing knowledge
- Create new knowledge
Evidence of student understanding can be generated with the use of a variety of tools (Rebeck and Asarta 2012:180):
- Formal examinations in the form of instructor-created exams are used extensively
- norm-referenced standarised test instruments such as the Test of Understanding College Economics (TUCE) are available to assess student knowledge and compare it to the achievement of groups of similar students (Rebeck and Asarta 2012:179).
- Non-examination written work such as essays, journals
- Classroom simulations and experiments
- Group work
- Immediate feedback techniques
For Suskie (2009) changes to teaching is the last of the four ongoing steps. For economics instructors, these changes can be as simple as finding new alternatives to define a concept or seeking new real world examples of economic relationships, or they can be elaborate changes to approaches to helping students learning topics (Rebeck and Asarta 2012:180). The importance of finding and employing alternatives to traditional lectures (chalk-and-talk) has been stressed by Becker and Watts (2001). Numerous teaching techniques are available that economics instructors can employ to improve student learning of economics (Rebeck and Asarta 2012:180).
Assessment is not a panacea; it is a complex activity with many facets, and it is part of the contemporary higher education scene, with all its problems and possibilities (Walvoord 2004:10).
Becker, W.E. and Watts, M. 2001. Teaching economics at the start of the 21st century: Still-chalk-and-talk. American Economic Review, 91(2):446-51.
Black, Paul and Wiliam, Dylan. “Assessment and Classroom Learning.” Assessment in Education: Principles, Policy & Practice 5(1), pp. 7-75.
Faulk, D. 2008. Formative and summative assessment in economics principles courses: Are applied exercises effective? Annual meeting of the American Economic Association, January 4-6 2008. New Orleans, L.A..
Hansen, W.L. 1986. What knowledge is most worth knowing for economics majors. American Economic Review, 76 (2), 149-153.
Hansen, W.L. 2001. Expected proficiencies for undergraduate economics majors. Journal of Economics Education, 32(3), 231-42.
Johnston, C. G. James, R.H., Lye, J. N. and McDonald, I.M. “An Evaluation of Collaborative Problem Solving for Learning Economics.” Journal of Economic Education 31.1 (Winter 2000): 13-29.
Miller, M.D., Linn, R.L. and Gronlund, N.E. 2009. Measurement and assessing in teaching. (10th ed). Pearson Education Inc,Upper Saddle River, NJ
Rebeck, K. and Asarta, C. 2012. Methods of assessment in the college economics. In: Hoyt, G.M. and McGoldrick, K (ed). International handbook on teaching and learning economics. pp. 177-186. Cheltenham, Glos, UK:Edward Elgar Publishing Limited.
Sadler, D. Royce. Formative Assessment and the Design of Instructional Systems. Instructional Science 18.2 (June 1989): 119-144.
Saunders, P. 1998. Learning theory and instructional objectives. In W.B. Walstad and P. Saunders. (eds), Teaching undergraduate economics: A Handbook for Instructors, pp.85-108. Irwin/McGraw-Hill: New York.
Slater, Timothy F., Ryan Joseph M., and Samson, Sara L. 1997. Impact and Dynamics of Portfolio Assessment and the Traditional Assessment in a College Physics Course.” Journal of Research in Science Teaching 34(3) pp. 255-271.
Suskie, L.A. 2009. Assessing student learning: A common sense guide. (2nd ed.). San Fransisco: Jossey-Bass.
Torrance, H. and Pryor, J. 1998. Investigating Formative Assessment Teaching, learning and assessment in the classroom. Philadelphia: Open UP.
Walvoord, B.E. 2004. Assessment clear and simple: a practical guide for institutions, departments and general education. San Fransisco: Jossey-Bass.
Wininger, S. R. and Norman, A. D. 2005. Teacher Candidates’ Exposure to Formative Assessment in Educational Psychology Textbooks: A Content Analysis.” Educational Assessment 10.1 (2005): 19-37.