|
|
 |
|
The Departmental Self-Study Process, Part I
Michael J. Galgano, Editor As mentioned in the last issue, the next two newsletters discuss the forthcoming OAH self-study guide for departments of history. The guide, an outgrowth of the federally-funded OAH/FIPSE Project directed by Bill Williams, is designed to offer guidance to academic departments as they review their programs and plan for the future. This issue will consider sections covering governance, advising, curriculum, and assessment. In each instance, a draft copy of the specific section was sent to a reviewer who was asked to comment on its strengths and limitations. The drafts range from outlines of possible questions to fairly detailed accounts. The section examining governance is evaluated by Professor Jim Crooks of the University of North Florida. Governance is defined in the draft as ``the structure and procedures, formal and informal, that constitute the organization of the department and its ability to determine policy, make decisions and carry out all of the administrative duties required to achieve its missions and goals.'' The section focuses on four principal areas: com- munications, decision making, leadership, and colleagueship. It asks departments to consider the particular roles of the chair, other departmental administrators, committees, the faculty meeting, individual faculty, clerical staff, and students within each area. Each component has its own questionnaire and each asks for descriptive narratives to complement data. Professor Crooks isolates five themes for consideration: the department's relationship to the institution and community, the distinctions between large and small departments, the changing goals of particular departments, the roles of junior and senior faculty, and the potential differences of private and public institutions. Curriculum is reviewed by Donna Spindel, Chair at Marshall University. This section concentrates on the history of the curriculum, its present structure, and the process involved in shifting direction. Further, it calls for curriculum charts summarizing the type of courses, the geographical areas, time periods, topics, approaches, and methods of assessment. A second chart illustrates enrollment, student/faculty ratio, student evaluations, attrition rates, and grades. Finally, the section asks for a comprehensive analysis of the undergraduate and graduate programs covering topics from institutional requirements to honors programs and internships. Professor Spindel suggests dividing the packet according to university type and raises important questions about institutional mission. She also underscores the enormous resource effort involved in completing a detailed departmental self-study. Chris Arndt, Advising Liaison at James Madison University, examines the section on academic advising. The self-study document invites departments to describe the structure of the formal academic advising system for both undergraduates and graduates. Specific questions explore how advisors are selected, student access, and a brief treatment of performance evaluation. There is a questionnaire for students seeking information about the availability of the advisor and the effectiveness of the advice offered. According to Professor Arndt, the section asks the right questions for gathering information about past and present practices; however, it is less helpful in offering direction for improvement. He raises thoughtful points regarding training and evaluation. Finally, Tom Morrissey, Chair at SUCNY, Fredonia, details a model assessment program, also funded by FIPSE, which promises useful insights for the self-study guide. Bill Williams has not yet fully developed the section dealing with assessment. He calls for a narrative description of the assessment instruments used by departments to determine what majors have learned. Departments are asked to compare their practices to others within their institution and to discuss how assessment is used to promote learning. After reflecting on the broad philosophy of assessment, Professor Morrissey describes the present status of five SUNY campuses as they work out a method of assessing an undergraduate history program. It is my hope that these four essays will foster a dialogue among department chairs on self-evaluation and that many of you will offer suggestions and ideas through the Newsletter as we work to assist Bill Williams in creating a common manual for departments of history across the country. Please address your comments to the OAH or to me directly. Governance James B. Crooks The Williams' draft on history department academic governance makes a good beginning with its discussion of personnel, colleagueship, communications, leadership styles, and decision-making. What it omits, however, is the reality that departments rarely govern or function autonomously. Instead, they operate within the context of division, college and/or university, and possibly state and/or board of trustees. In effect, they are subject to the whims, rules and policies of outside agencies. For example, funding for the department comes from the outside and determines the number of positions, telephones, postage stamps, xerox copies, and professional trips faculty may take. Decisions about faculty positions, either to replace departing members or adding for growth, ultimately are in the hands of deans, provosts or others, also affecting faculty governance. In effect, departmental governance may operate within the context of relative autonomy in mature institutions, or in relative power- lessness in institutions where higher administrations maintain greater control. These factors affect decision-making, leadership, colleagueship, and communications, all of which may be restricted within a department because of outside forces. Thus, an early question in evaluating departmental governance would ask to what extent does a department have autonomy to govern itself, or to what extent are departments victims of higher adminis- trative decisions and policies? A second factor to consider in governance is the size of the department. Does a four- to six-person department function differently from a forty- person department? One excluded or disruptive faculty member in a small department can undercut colleagueship much more than one person in a larger group. In addition, the larger department may choose to delegate much of the decision-making to a promotion and tenure committee, un- dergraduate curriculum committee, or graduate program committee, and there may be relatively little collegiality for the whole. Thus the individual professor active in but one committee may have little communication about the others. In contrast, the small department may operate as a committee of the whole for promotions, curriculum, etc., and collegiality becomes essential. In a larger department, the head or chair may have greater auton- omy on budget allocations or scheduling due simply to size. In the more informal atmosphere of the smaller department, all faculty will play substantial roles in all decisions. Thus the question arises: to what extent does departmental size affect governance? Does the size affect colleagueship, decisions or the potential for academic leadership? In the area of decision-making, the Williams' draft assumes collegiality, and of course that is the academic ideal. On the other hand, in many institutions decisions are defined by the institution as being the legal responsibility of the department head, dean, provost, president or trustee/regent. Hopefully, de facto practice is one of collegiality, but a questionnaire evaluating academic governance should ask about the degree to which it exists. Are merit raises decided collegially? Do departmental faculty decide who travels and the amount of reimbursement when funds are limited? Who decides who teaches early morning, evening or Saturday classes? These questions do not suggest that all departmental decisions should be made in town-meeting style, but they suggest that departments do establish policies about who makes what decisions where. For a large department particularly, it may be sensible for the faculty to delegate to the chair day-to-day decision-making in a way that may not appear to match the overall style of governance. The section on governance rightly asks faculty to identify the ``power structure'' and criteria for departmental leadership, but it also might ask about the quality of that leadership. Departments change. They expand curricula into new areas such as public history or graduate programs. They cultivate alumni for fund raising. They seek endowed chairs, or new career opportunities for students. Faculty should evaluate both the quality and direction of change to enable the department to meet the needs of both faculty and students in the near, and perhaps even distant future. In effect, departmental leadership requires the chairperson or others in that position to keep abreast of changes in higher education with particular emphasis on history enrollments, curriculum, and technology, and have a vision of what one's particular department should become. The smaller school seeks to add a public historian or a third world component to its curriculum. The medium-sized school plans summer institutes to better serve community teachers, or for greater research effort and support for faculty. The large school seeks to become one of the nation's ten best. All should have a vision of what they might become which would fit under the category of leadership. Finally, in terms of colleagueship, which the Williams Report rightly sees at the heart of effective academic governance, one looks for a concern for the opportunities and responsibilities of junior versus senior faculty. Is there any remnant of a caste system? To what extent are junior (defined usually as non-tenured) faculty included in major decisions? Further, what roles, if any, do adjuncts play? Have graduate assistants any role more than as underpaid laborers in the classroom? In summary, the section on governance needs to expand its purview to include the relationship of the department to its larger institutional community/hierarchy. It needs to consider the differences between the larger and smaller departments, and what that means for governance. It needs to look at departmental goals for both the present and the future. It needs to look at the roles of junior faculty, adjuncts and graduate assistants to determine their place in the system. Perhaps too, it should ask whether private and public institutions function differently, and to what extent does community college governance differ from that of four-year institutions? Yet having raised these questions, there remains the sense that the Departmental Self-Study Guide is on the right path in its examination of leadership, decision-making, communications, colleagueship and the roles of differing personnel within history departments. Curriculum Donna J. Spindel Our eight-member department at Marshall University was visited by an OAH/FIPSE Project team in October 1986. For at least a month before the visit, we spent a lot of time preparing the previsitation materials requested by the team. As we learned in the process, this was no idle exercise. It had been a long time since the department had last subjected itself to such microscopic scrutiny. In fact, we learned the embarrassing lesson that we are historians who keep poor records. After scurrying around searching through university archives and old storage closets, we came up with a pile of material that answered important questions about our program we had never bothered to ask. I believe my colleagues would agree that this proved to be for us the most valuable part of the total OAH/FIPSE Project evaluation. It is with these thoughts in mind that I offer comments on the proposed curriculum materials request packet prepared by the OAH. Because there is such a diversity in colleges and universities around the country, much of the requested material would be irrelevant to some institutions. For example, our small department with its heavy emphasis in the survey areas, lacks the resources and staff to provide undergraduates with ``program'' choices. It might be worthwhile to develop a separate packet, at least, for four-year schools without graduate programs, with master's programs, and with doctoral programs in history. Rather than a brief statement on the history of the curriculum, evaluators could request a copy of the most recent internal evaluation of the department. Here at Marshall each department prepares a five-year self-study which would answer the team's specific questions. The proposed curriculum chart would contain very specific enrollment histories of courses. Why not include as well information about whether or not each course is an elective or required, and if required, for which segments of the student body? The mission of the institution ought to be a crucial part of the general evaluation. At Marshall a great deal of what we do and how we do it is governed by institutional objectives. In the area of undergraduate programs, departments should supply numbers of majors over the past few years and provide some comparison with other departments in the institution. This is an excellent measure of the extent to which a department can tailor its curriculum for majors. Much of the requested information on graduate programs would not be relevant to Marshall, a master's level institution. For us, enrollment histories would surely be of interest in an evaluation, where those students come from, the number of MA students who are accepted in doctoral level programs and where those programs are. The number of graduate fellowships a department can offer and the amount of each fellowship can also provide some insight into what a department is capable of achieving. The section on administration of degree programs should require departments to explain the amount of faculty time that is allocated to advising and tracking students and the extent to which clerical staff is available for this purpose. We discovered that in areas where resources were in short supply (and there were several), it helped us to have the OAH/FIPSE Project team state this in its final report for the M.U. administration. Based on our experience, I know it would be helpful for departments to have the requests for curriculum information put in numbered form in separate categories. At the very least, this will make it easier for the chair to allocate sections of the report to different faculty, depending on their knowledge or expertise. Also, curriculum information supplied in the evaluation ought to include a brief description of the institution. In this way evaluators can put the mass of information they will be receiving into some kind of context. The curriculum request packet asks for so much detailed information that it would probably be ``humane'' to inform departments which are preparing for an evaluation that it will entail a substantial commitment of time. Our experience at Marshall was certainly a positive one. The OAH/FIPSE Project team's ultimate plea to the university to allocate more resources to history got us nothing (at least they tried!). But examining our program under a microscope led us to make some long overdue changes in the way we operate in the department and in the way we teach. Academic Advising J. Chris Arndt The departmental questionnaire included in the draft contains some helpful questions for a rudimentary evaluation of a history department's academic advising program, but lacks the sufficient depth to provide the type of analysis needed to improve academic advising. The first question could be improved if it asked for a description of the academic advising system (rather than formal advising) since many such programs may not be formalized. In addition, there should be some sort of question as to the intent, purpose and focus of advising. In short, is the advising purely academic, or does it take into consideration other elements? (This is important since the increase of computerized registration systems will ease course selection and make it possible for advisors to provide different types of services.) It would also be helpful to get opinions on academic advising from those providing the advice. If the majority do not take it seriously or are uninterested, the program will be ineffective. This questionnaire should also examine the extent to which an institution provides training for its academic advisors and whether this training is appropriate for the type of advising provided. It might also be helpful to get advisor feedback as to whether advisors feel competent to be providing advice. Finally, it is essential in evaluating a program to ask if there is an advising hierarchy within the institution to which advisees with special problems may be referred. The document currently asks who advisors are, how they are assigned and what their duties are. While these questions are important, it might be helpful to discern how advising is evaluated and whether it has any effect on raises, promotion and the tenure process. The document might also deal with the extent to which informal advising occurs and attempt to ascertain its effectiveness. The questionnaire for students, while again providing a good starting point, also needs to be more comprehensive. It seems that this section of the questionnaire is preoccupied with whether a student can see an advisor. While this may shed some light on advising, it speaks little to the quality of the advising which occurs. An important point which is dealt with in the last question (rate effectiveness of advice received) might be better worded as, ``what value do you place on academic advising in general?'' We might discover (though I doubt it) that we are providing an unnecessary service. It might be appropriate to ask students what types of advice they routinely expect from their advisors and if there were (or were not) specific problems for which they would seek advice. Finally, both student and advisor questionnaires should comment upon the aspects of the advising process which they see as most (and least) beneficial and perhaps a suggestion box of sorts could be included to suggest how academic advising might be improved. Assessment Thomas E. Morrissey It is most important that some form of separation/distinction be made between assessment and determination of grades. If not, both students and faculty will see it as a threat to both of them and the purposes and value of assessment will be lost. Assessment must be used to get an overview of a program, how well it works, what it intends to do and what it actually achieves, where the strengths are and where weaknesses are revealed. It therefore must be distinct from (a) student evaluations of individual instructors and courses, and (b) the grades achieved by indi- vidual students in separate courses or in some form of comprehensive examination. Any instrument(s) of assessment must contain therefore an element of value-neutrality in regard to individual students and faculty members; it must also have a quality that renders it impossible for anyone to teach for the ``test'' lest the results be made useless. A program of assessment then could have a number of forms and components. On the lowest and most basic level these would be a check list by which students and advisors alike could record the requirements of the program with its different parts and stages to mark the progress a student makes toward completion of the degree requirements. A second stage might be several special courses aimed at and required of majors to ensure that at each step they have moved beyond a certain level of achievement and sophistication in the discipline. Thus while freshmen might typically take introductory courses in the American and Western Civilization surveys, there might be at the start of the sophomore year a required course in Introduction to History Methodology and Historiography, which would prepare them for more advanced courses in various areas and time periods during the rest of their college/graduate years. This could then be eventually followed by a senior seminar in which they would be expected to do original research in some phase of historical investigation under intense support and direction of a faculty member. If we are going to assess on a level beyond this and reality seems to demand (in the name of accountability) that we do so, then a further instrument would seem to be in order. It would be intended to measure several things. [I am expressing these ideas on the basis of intensive and extended discussion among a group of ten historians from five campuses who have worked together now for some months on ways to address this problem. The ideas I believe are not my own but express the consensus of the group even if the infelicity of phrasing this is my own.] We would want to determine whether there is a certain body of knowledge that anyone completing a history degree should possess. Now given the fact that many history programs are quite diverse because of the survival of certain members and the loss of others during the crisis years across the country of the 1970s, this requires some degree of flexibility and so our group opted for a series of sub-sections from which an individual student would choose to be tested. For example:
Originally the group opted for three sections but now it has moved because of the inherent problems in having three parts which would require at least ninety minutes to settle for two parts which, however, may not both be from the same geographical area, i.e., a student may not do both in American or both in Western Civilization. In connection with this there are also ten more general questions requiring use of charts, graphs or historiographical knowledge but which all together can be completed in an hour. Then there would be several questions of a general nature from which the student would choose one to write an essay for which they had an hour to work and in which they are expected to illustrate their thesis by a number (at least three) of examples drawn from history. This instrument of assessment is attempting to determine both knowledge (content) and skills (analysis, synthesis, writing, logic, awareness of connections, similarities and differences, etc.). The second part is fairly clear and non-controversial even if the difficulties in choosing essay questions and grading the results are not easy. The first part presents difficulties and disputes and yet some degree of sentiment is usually obvious. There is usually wide agreement that certain people, documents, events, etc., are and were of major importance so that for that chosen subsection a student who has majored in history should be able to recognize, identify, or answer questions on these with some degree of regularity. The results desired are not how any individual student did on a particular subset but how well a whole group did. Of course this raises questions about expectations, i.e., if a fairly large number of students took a particular subset [and in order to evaluate our instrument we offered it to enough students to have some validity as well as comparing the results with some control groups, e.g., incoming freshmen and graduating seniors who have not taken a number of history tests], can we say that programs are fairly successful and attaining their goals if from this group of students of disparate natural ability and from somewhat diverse programs on various campuses, the overall median score is X-number of correct answers out of Y-number of questions? Here there is an obvious difficulty: time constrictions require that there only be Y-number of questions. We will need to compare the results from the control groups to select questions and some hard decisions will be involved. Should there be any questions that all who take the subset should be able to answer? Should there be questions that only a very few even of the history majors would answer correctly? What have we learned about a program if we determine that for some subsets there are questions that almost none of the incoming students and very few of the non-history graduating seniors can answer correctly, but that a majority of the history majors do answer correctly? Do we have reason to be satisfied that at least in area of content, most graduating majors have acquired certain body of knowledge? Do we know why and how they arrived at the correct answer? What can we do with the information that we acquire from this assessment? What could others do with this information? This latter question always lurks in the background as a pervasive atmosphere of threat and foreboding to make many very anxious about assessment. Since many history faculty have experienced years of threats, cuts and insecurity, the last thing they would want is to participate in a program by which some outside force would seize on a series of numerical results, decree that a department has been mathematically determined to be a weak or inferior program and so could or should be cut, retrenched or eliminated. The need to have results that are useful but not dangerous is a key consideration in all of this. If it became clear that for a given department the outcomes by their students were not what they hoped for, a number of reasons for this result could be investigated. Is it that in this department the quality of students compared with others was lower and so what the assessment shows is not that the program but that the student body does not measure up? Or could it be that compared with other departments, this group of faculty, for its own reasons, places certain emphases and stresses certain topics that are different from what many other departments do? (For example, one group might be from a school with a large performing arts program and so this department has configured the content of its courses to fit this clientele and environment, while another department might be from a campus with a large business and economics enrollment and so these will be the major component and stresses that this department will bring into its courses.) Of course the department might also want to review and perhaps even revamp both its offerings and how they are presented and configured in its program, if they determined that they were not satisfied with the results of assessment. Then again a department might be pleasantly surprised to discover that the results of a somewhat elaborate process of assessment were pretty much what they had anticipated based on their own anecdotal conjectures. All of these variables will need to be kept in mind once we have been able to achieve agreement on what are the suitable questions for the multiple choice content section of this assessment. On my own campus we have also considered two other lines of assessment. The first might be some form of portfolio in which the majors would compile their written work and in addition append to these some personal comments and evaluations of how they see and examine their own development in historical thinking skills. The other might be the handbook which the campus FIPSE committee has compiled. This group is examining our campus general college program and is in the process of developing a guide to evaluate and assess certain skills called for in this general program. One of these skills is identified as historical awareness and it might well be of some use to our own search for means and instruments of assessment. Any investigation or experimentation in the area of assessment will then be a very tentative proposition, but we will have to get going. We are going to have to live with it and the more active we are in the process the more valuable the results will be for us, our programs, and the students we serve. American Council of Learned Societies Announces Task Force The American Council of Learned Societies has organized a task force on scholarship and the public humanities. The task force, whose objective is to find means and ways to reinforce the relationship between academic and public humanities plans to focus on a number of specific issues, including the incentive system of higher education and the cultural responsibilities of universities to the communities in which they are located. For more information, write to the ACLS, 228 East 45th Street, New York, New York 10017-3398. |
||