|
|
 |
|
Assessment in the History Major
Michael J. Galgano, Editor Assessment of academic programs has increased in visibility in recent years and shows every sign of becoming a permanent fixture on campuses across the country. Many institutions have already established assessment offices and departments everywhere are evaluating programs seeking to measure their quality and effectiveness. The present issue treats several dimensions of assessment as they relate to history departments and their chairs. Each contributor is participating to some degree in undergraduate assessment. In the opening essay, Professor Jo Ann Rayfield addresses value-added assessment, defining it and detailing some of its implica- tions for history departments. She provides a useful model for departments to consider along with valuable advice for department chairs. Specifically, she suggests some of the ``pedagogical, psychological, and managerial benefits'' which may accrue to a department through a thorough and meaningful assessment program. In so doing, she underscores the key to assessment: helping the department to help itself. This same message permeates the contributions of Professors John Glen and Frank Gerome, who offer two perspectives on the topic. Ball State started to assess its program last year and Professor Glen shares his thoughts on the pragmatic approach adopted by his colleagues. He identifies some of the pitfalls to be avoided in beginning an assessment project and the need to include all faculty. James Madison has recently completed its initial assessment and Professor Gerome reflects on what his department has learned. While the findings are hardly revolutionary, they are based upon hard data and scrupulously avoid anecdotal evidence; therefore, they are likely to be taken more seriously by the university's administration and governing authorities. Finally, Professor Henry Steffens of the University of Vermont discusses student writing, a vital component in any assessment program. His insights into the value of informal writing exercises in facilitating student learning raise significant points for departmental consideration. The overall issue raises several important ideas for department chairs as they ponder assessment. Professors Glen and Rayfield suggest ways to begin an assessment program. The faculty member asked to head up the project should be interested in assessment, or at least not overtly hostile to the idea. From the outset, the department chair should provide appropriate support, both substantive and moral. For example, as Professor Rayfield indicates, assessment should be an assigned responsibility for which a faculty member is fully recognized and compensated. Second, assessment should involve all faculty in the deliberations about goals, methods of measurement, assessment instruments, and application of results. Committees may report their findings to the larger group or discussions may proceed in full meetings; however, all should participate. Third, the department should discuss what it plans to do with the results. Will they be used to buttress arguments for more resources? Will they help to bring about program adjustments? Will they be used to plan? Professor Gerome has given our department a lengthy list of strengths and weaknesses as well as a blueprint to implement changes. Fourth, in the area of writing, to what extent does the department support ``writing across the curriculum?'' How does the department value writing? What is the department willing to contribute to help foster better student writing? As history chairs grope through the complexities of assessment, they should not be sidetracked by its jargon. Rather, they should recognize the genuine opportunity to strengthen their own program in a unique way and tailor the process accordingly. Value-Added Assessment in History Undergraduate Programs Jo Ann Rayfield The assessment of value-added or summative undergraduate learning, whether overall or in a discipline, has become a topic of hot debate in higher education. Efforts to measure the impact of college are by no means new. Most of these efforts have focused on psychological or behavioral changes in undergraduates. What is new in recent years is the effort to measure cognitive growth added knowledge or enhanced intellectual skills as a result of undergraduate study. In some cases the focus is the entire four-year experience; in other cases the focus is on the major. The movement toward measurement of cognitive change over the course of the four undergraduate years (whether it is called summative undergraduate learning or value-added) is exciting and appropriate for higher education, but it is fraught with challenges and dangers. It should not be undertaken lightly or hastily. In the summer of 1987 I undertook a research project for Illinois State University on the development of indicators of summative undergraduate learning or value-added assessment in history. The observations in this essay were developed in that process. If we are to undertake value-added assessment, and I believe we should, we must start with an intensive period of self-assessment. A well constructed assessment program offers pedagogical, psychological, and managerial benefits to a department. It is a good way to find out how well we serve our students over a longer term than a quarter or semester. With that knowledge we can adjust curriculum or content. It is important for students to see their course of study as a whole which is greater than the list of individual courses. Also, the very process of self-assessment and planning which precedes articulation of instructional goals can enhance departmental cohesiveness and sense of direction. Most importantly, it causes us to decide what it means to have a bachelor's degree in history as opposed to any liberal arts major. For history departments which traditionally bear a heavy service course burden this sense of self is an important ingredient in recruitment, advisement and building esprit de corps among the young professionals we are teaching. The ``managerial'' benefits are not to be disparaged in these times. Information gathered through an assessment program can provide a context within which to make decisions when resources are not adequate or to discuss such curriculum issues as required courses or distribution requirements, and even class size or courses for majors. Such information can be used to persuade administrators and others of our need for resources. Departments in other disciplines have obtained additional resources by documenting a need through the use of test scores, other performance indicators, and employer feedback. History departments should try this strategy. Indeed, we should go beyond assessment of summative undergraduate learning to undertake research on the teaching/learning process in our classes and consistently collect information on the work performance and careers of our majors. That information also should be used in curriculum and instructional decisions. An effective system of measurement of summative undergraduate learning or value-added in a discipline must be predicated on a clear articulation of the intellectual characteristics and instructional goals of the discipline, the institution, and the individual department. A departmental faculty committee should be given assigned time to prepare a detailed statement of instructional goals and to develop and validate indicators which would give baseline and graduation data. The following general guidelines should be used:
History is a discipline with a large factual content component, but it also has a significant methodological or skill component. To measure summative undergraduate learning in history, we must use indicators which measure both. The examinations to evaluate mastery of factual content might be much like the SAT, ACT, or GRE in form. The scope of these examinations should reflect the nature of the curriculum and department goals. The advantage of using available, nationally standardized tests is that the questions have themselves been validated and national norms exist. The dangers are in comparing our students to norms which were based on a different population and in creating a situation which lends itself to concentration on test scores for institutional comparisons rather than for diagnostic or instructional purposes. In history the level of expectation in terms of content mastery or skill demonstration is not as agreed upon as it appears to be in some science disciplines. This reflects the sheer breadth of the discipline and the reluctance of history departments to require a core curriculum or prescribe content. There is an understood core of subject areas to be studied, but not an agreed upon body of specific factual content to be learned. Despite distribution requirements it is possible for an undergraduate history major to select nearly seventy-five percent of the courses in his or her major in either United States or European history to the exclusion of other areas. A department would have to consider the difference between the catalog curriculum and the courses actually selected by its students. Also, not all courses with the same title cover the same set of facts. Some instructors emphasize political history and wars while others emphasize social and economic themes and domestic history. Any effort to measure added knowledge of history from admission to graduation must take these realities into consideration or contribute to a reassessment of these characteristics of the history curriculum. Some institutions use the ACT-Social Studies Reading Test as an initial measurement and a retired form of the ACT COMP to test rising juniors. This is a reasonably effective way to measure value-added in a general education but not to measure progress along a continuum toward mastery of an individual discipline. An exit exam such as the GRE history test can be cobbled on to that, but it does not fit with the first two in such a fashion as to enable one to measure value-added. It would be far more appropriate for historians to work with testing services to generate tests designed specifically for this purpose. The skill component of the discipline is no easier to define or to test. History teachers do teach the skills and methods used in collecting, analyzing, synthesizing, and presenting information. Many would argue that they do so in some degree in all courses, or at least in enough courses that skill oriented or methodology courses are not necessary at the undergraduate level. It is a generally shared (although not clearly-articulated) assumption that the undergraduate major learns a large body of factual content material and that in graduate school he or she develops the interpretive and analytical skills of a historian (research, interpretation, etc.). (``The students need to know more facts before they can interpret the facts in a meaningful way.'') Examinations to assess growth in ability to think historically, analyze historical interpretations, synthesize information, and make generalizations could be developed and validated and graders trained. Alternatively, skill evaluation might be carried out through freshman and senior seminars. The question of skill assessment has implications for history instruction as well as for assessment of value-added. Teaching and evaluating the use of these skills is time-consuming and requires more time for faculty/student interaction than is available to most history faculty. Skills are best taught and measured through assignments (essay questions, term papers, or senior thesis) which require students to use them. Faculty are not able to give adequate emphasis to teaching historical methods/skills to undergraduate majors because of heavy student loads or are reluctant to do so because of the negative student reaction to research paper requirements. Nonetheless, our students need these experiences more than we realize until we take a systematic look at the performance of all our majors. Some problems must be addressed. We should monitor the entire undergraduate experience for each student to ascertain what variables (high school and socio-economic background, other courses, minor fields, internships, extracurricular activities, em- ployment, and the like) show a high correlation with changes in student performance. Also, the baseline point must be defined. Should initial measurement be at freshman admission, rising junior level, transfer admission, declaration of major, or before the student enrolls in advanced courses? The very flexibility (in changing majors for example) of which many institutions are justifiably proud creates some administrative difficulties. The use of results has especially explosive possibilities. It must be clear, for example, whether poor performance is more likely to obtain more resources (for remediation purposes) or whether high performance is more likely to obtain more resources (for rewards). Caution must be exercised so that the tests and the testing process do not distort the curriculum and instructional process. There is no ideal program in place. Some institutions, notably Alverno College and Northeast Missouri State University, have taken hard looks at their curricula, instructional goals and ways to evaluate their students on operant as well as respondent behaviors. The programs at these institutions may not be reproducible everywhere, but their experiences do provide a foundation for interinstitutional dialogue at state and national levels. Once history departments (and their institutions) articulate clear instructional goals and areas of knowledge and skills to be measured, we can expect the testing services to respond by generating entrance and exit examinations. It is essential for history faculty to be active participants in the process of establishing the standards by which history departments and history graduates will be measured. Evaluation criteria for accountability are being established and often such criteria are imposed by bureaucratic managers who have cost per credit hour or credit hour generation as high priorities. If we want to take a stand for the quality of instruction in our discipline, we must seize the debate over assessment as an opportunity to shape standards and articulate an agenda. Otherwise, we risk losing control of our future in the academy. Getting Started: Assessment at Ball State University John Glen The development of an assessment program by the Department of History at Ball State University is not a particularly remarkable story, but the progress that has been achieved thus far makes it an instructive one. Through a combination of foresight, a few well-organized meetings, memoranda that summarized issues and offered concrete alternatives, and sheer good fortune, the department has accomplished the difficult, sensitive, yet essential first step of establishing the criteria by which the academic performance of students majoring in history can be evaluated. In the fall of 1987, department chair John Weakland suggested to me that, as chair of the undergraduate programs committee, I present to my colleagues the task of devising an assessment ``instrument.'' There was no pressure to prepare such a device. Yet we agreed that the department should take the initiative, define its own goals, and have an assessment procedure ready to use in the event the university administration was caught up in the current demand for accountability in higher education. Assessment seemed to be a worthwhile idea anyway. The department, already regarded as one of the strongest at Ball State, was one of six in the College of Sciences and Humanities working with the university's new Center for Academic Assessment, and the development of a set of departmental objectives could become a positive step toward giving greater coherence and direction to the curriculum. To begin the process, members of the undergraduate programs committee received a memorandum outlining the basic issues involved in assessment: What should be assessed, and how do we assess it? I reported that William S. Johnson, director of the Center for Academic Assessment, had offered for consideration a Major Field Achievement Test in History produced by Educational Testing Services, and that senior majors were going to take the test as an experiment in the spring of 1988. But there were considerable doubts about the adequacy of the test (an impression later confirmed when faculty members examined it), particularly since assessment presumably would need to cover research skills, writing ability, and areas like computer competency. This problem carried over into larger, even more complex issues concerning the structure of the department's undergraduate curriculum and its relationship to college and university-wide programs, such as ``writing across the curriculum,'' computer literacy, and general, interdisciplinary, and cross-cultural studies. Assessment had become very complicated very quickly. Committee members spent the next several months searching for an understanding of assessment. Ray White became coordinator of assessment for the history department, charged with the responsibility of recruiting senior history majors to participate in the assessment process, organizing a departmental faculty task force to determine the skills and content a graduating history major should possess, and developing a capstone course for senior history majors in preparation for assessment. Ken Winkle, a former colleague of mine now at the University of Nebraska, forwarded a syllabus for a quantitative history course emphasizing the application of computers to the research and writing of history. These steps and a few informal conversations kept the assessment issue alive, but there was an understandable reluctance among committee members to attend any formal meeting that promised to be long, unfocused, and inconclusive. The breakthrough came with the appearance of Michael J. Galgano's ``Tilting at Windmills or Harnessing Their Power More Efficiently: Assessment in the History Major,'' in the March 1988 issue of Perspectives, the American Historical Association newsletter. This article is a thoroughly useful account of how the issue of assessment has been handled by the history department at James Madison University, where the degree program, number of undergraduate majors, and number of faculty members are only slightly smaller than that found in the history department at Ball State. Even more important, Professor Galgano had furnished a list of departmental goals for majors which could serve as a model for other departments: ``a knowledge of American and world history and geography; a knowledge of historical thinking, interpretations, and processes; an awareness of enduring values and ethics; a love for reading; an ability to do historical research; an ability to think critically with historical perspective and insight; an ability to communicate effectively, both verbally and in writing; and a proficiency in computer use.'' I circulated copies of Professor Galgano's article among members of the undergraduate programs committee, suggesting that a great deal of time and effort could be saved by drawing on the James Madison example instead of trying to reinvent the assessment wheel. The committee was certainly receptive to the idea of using an existing model of assessment. At the same time, members wanted to make sure that the history department's evaluation process was comprehensive enough to reflect the relative flexible course of study pursued by its majors, who are required only to complete a set number of hours in United States and world history. When the committee met formally in mid-March to analyze the ramifications of assessment, I reported on my follow-up investigation of the James Madison program and then turned the meeting over to my colleague John Barber, who has extensively studied course design problems and currently works with the Center for Teaching and Learning at Ball State. Stressing the importance of being clear about the goals or competencies to be measured, Professor Barber reviewed two sets of assessment goals. One set, developed by Alverno College, sought to judge the following: effective communications ability; analytical capability; problem solving ability; facility in forming value judgments within the decision-making process; effective social interaction; understanding of individual/environmental relationships; understanding of the contemporary world; and educated responsiveness to the arts and humanities. The other set, evaluating the ``Qualities of the Liberally Education Person'' at Rutgers University-Newark, covered higher-order cognitive skills; an active awareness of one's natural environment; an active awareness of oneself; and an awareness of and ef- fective action in one's social and cultural environment. At the committee's request Professor Barber provided copies of syllabi that indicated the objectives of his own history courses: these included improvement in thinking ability and intellectual creativeness; progress in the ability to communicate; expansion of knowledge; cultivation of sensitivity and related creative abilities; and development of the ability to form values, reach decisions on social issues, and solve human affairs problems. The committee now had a total of four models from which to select, revise, and synthesize into a set of goals for assessment in the Department of History at Ball State. To prevent any loss of momentum and keep the focus on the definition of goals, I prepared a working paper that compared the models offered by John Barber and the faculty at James Madison University, Alverno College, and Rutgers University-Newark. This simple yet timely act helped crystallize the thinking of the undergraduate programs committee at its next meeting. After carefully analyzing the lists of goals and objectives and their capacity to test for knowledge, values, and skills, the committee decided to adopt James Madison's goals with certain revisions. The final version set forth the following goals for undergraduate history majors at Ball State: a knowledge of American and world history; a knowledge of historical thinking, interpretations, and processes; an awareness of enduring values and ethics; a sensitivity to the historical background of cultural issues that have concerned societies past and present; an ability to do historical research; an ability to think critically with historical perspective and insight; an ability to communicate effectively, both verbally and in writing; and a familiarity with computers and their uses. It was now time to engage the entire history faculty in the assessment effort. Copies of the proposed set of departmental goals were distributed with a request for written comments. I summarized the background of the assessment issue, gave assurances that the undergraduate programs committee had no intention of imposing a specific set of goals on faculty members, and explained that the committee's list represented an attempt to codify the overall criteria the department had already been using to assess academic achievement. There were a handful of suggestions clarifying, elaborating, or underscoring a particular goal. But by the end of the 1987-88 academic year, the faculty had reached a rough consensus on a set of guidelines by which to evaluate not only the abilities of the department's majors but the effectiveness of its teaching as well. The task of developing an instrument to measure academic achievement remains, of course, and there is no guarantee that the assessment process will continue to proceed as smoothly as it has to this point. Nonetheless, the primary lesson to be gained from the experience of Ball State's Department of History is that the complexities and vagaries of assessment can be made manageable. By exerting control over the process early, cooperating with or benefiting from those who have grappled with the problems of quality in higher education, and maintaining a clear sense of direction, it may be possible for departments to have an assessment instrument in place by the time there is a demand for one. What We Have Learned: James Madison University and Assessment Frank A. Gerome Professional historians are constantly evaluating students, administrators, each other, and societies in general; i.e., other people's goals and their methods of achieving them. Not often are we asked to examine our own and attempt to quantify things that are difficult at times to measure. In the past several years however, the department of history at James Madison University has been intensively involved in a self-evaluation of our history program and has tried to do just that. Our recent efforts began with an external evaluation by an OAH/FIPSE Project Team of historians who visited the campus in 1986 and submitted a thorough report on their findings. During this past year, as part of a University-wide assessment effort, I served as coordinator for the undergraduate evaluation of the major in history. It was my responsibility to encourage members of the department to re-examine our goals, determine the nature and methods of testing, plan and direct surveys, and attempt to make some sense of the information we gathered. I will try to summarize briefly some of the key things we have learned. Our initial assessment discussions in department meetings began last September. Our first task was to reach consensus with regard to our undergraduate objectives for the history major. As readers surely know, this is not an easy task especially in dealing with a large department. Nevertheless, after considerable discussion, we finally agreed that the student majoring in history should have: a knowledge of American and World history and geography; a knowledge of historical thinking, interpretations, and processes; the ability to think critically with historical perspective and insight; the ability to do historical research; the ability to communicate effectively, both verbally and in writing; an awareness of enduring values and ethics; and a proficiency in computer use. Last, but hardly least, the student should have developed a love for reading. Then, after developing a more detailed statement for each of these goals, came the daunting task of devising methods of evaluation and testing to determine whether or not our students were attaining them. These methods of evaluation included senior essay examinations, research papers, exit interviews, an alumni survey, tracking students to determine their employment and admissions to graduate and professional schools, and an external peer review of both methods and goals of assessment. Results indicated that our student majors had reached a competent level of writing and analytical ability, knew how to do historical research, and could evaluate and organize information effectively. Recent graduates and current majors have been well pre- pared to enter and pursue a great variety of careers and/or continue their education for an advanced degree. It also became clear that an important departmental strength consisted of a professional, diverse faculty who take their teaching and advising responsibilities seriously and are accessible to students. Departmental strengths also included a great variety of course offerings, developing analytical and writing skills, encouraging effective use of the library, and offering computer training. Students also appreciated the department's efforts to provide career guidance and the opportunity to become involved with a variety of internships. The latter was extremely valuable in developing skills and identifying potential career opportunities. Areas of weakness and suggestions for improving the history major included: increasing the frequency of course offerings at the upper level; requiring more seminar courses which demanded more research and writing; making the department's methodology course more easily available to first and second year majors; integrating more geography into courses; strengthening career planning; and working to improve the University's library collections. Some students indicated they would benefit from more informal as well as structured contact with faculty in the form of discussions related to research projects, books, films, etc. outside of class. Some also suggested that many history majors did not know each other and thought that it might help to distribute the names and addresses of their peers. Also, circulating a departmental newsletter might improve communication between students, faculty, and alumni. During our discussions concerning assessment over the past year, it became apparent that several aspects of the history program might benefit from some changes and restructuring of the undergraduate major. We have already begun to initiate some of these changes and plan to continue our efforts next year. Several of these include considering increasing requirements for courses outside of the field of United States history and the development of a senior seminar taught by various members of the department on a rotating basis. As the department considers and implements some of the suggestions which have evolved from our assessment efforts this year, we expect that the strength and quality of the undergraduate history major at James Madison University will be improved and that our students will come closer to attaining the objectives we have outlined for our program. A Response to the OAH Report on Minority Historians Sucheng Chan Members of the OAH Committee on the Status of Minority Historians and Minority History wish to respond to two points made in the 1986 Report of the Ad Hoc Committee on Minority Historians that was published by the Organization in pamphlet form last year and discussed at the AHA meeting of Department Chairs during the 1987 AHA meeting in Washington, DC. According to survey questions carried in the Report (page 3, Section I, Part A, question #4: ``Are you satisfied by the contribution made to your department by minority historians?'' and page 3, Section I, Part A, question #5: ``Would you say that your university and community are supportive of minority faculty?''), department chairs perceived several problems with respect to minority faculty, two of which, in our view, contradict each other. On the one hand, minority historians are criticized for concentrating ``too heavily'' in the fields of minority history in our teaching and on the other hand, we are faulted for our ``low scholarly productivity.'' Increasing specialization is evident in all fields of scholarship and we all know that writing that path-breaking monograph is one way to gain recognition as a scholar. So why is it considered a ``problem'' when minority historians also try to specialize? Is such criticism expressed because specialization per se is regarded adversely or because some branches of history such as the study of peoples of color are deemed less important than others? If the latter is the case, then such an attitude on the part of our colleagues, especially those who serve as gatekeepers in our career advancement, is a fundamental obstacle hindering our ability to be productive members of the profession. One reason that many minority historians feel that campus environments are not ``hospitable'' is that even when no overt discrimination is directed at our persons, the denigration of our work of our areas of expertise is a constant reminder that we are not really appreciated and that our presence is, at best, tolerated for political reasons that arise out of increasingly widespread student demands for a less Euro-American-centric curriculum taught by a more diversified faculty. When such demands become too loud to be ignored by campus administrators, who is asked to take responsibility for redesigning existing courses or for introducing new ones? More often than not, it is minority faculty, or faculty specializing in the study of non-Western societies and peoples, or women faculty who are pressed into service. Thus, we are placed in the unenviable (and often untenable) position of being called upon time after time by one segment of the university to deal with the ``hot'' issues of our times while another part of the academy takes us to task for not teaching in broad enough areas or publishing a sufficient amount. There are indeed too many competing demands on our time (and, less often acknowledged, on our talents). Many of us have risen to the occasion because we have felt compelled to do so, usually at a considerable cost to our careers and our family lives. Under such circumstances, is it so surprisingly that our ranks are not increasing and many minority students are choosing other professions? For those of us who have chosen the academic life, our burdens would be lighter and we would bear them more willingly if more of our colleagues would join us in dealing with the challenges of our age instead of simply complaining about what we have or have not done. |
||