Outcomes Assessment Resource Guide for
Schools and Colleges of Optometry

Developed by the ASCO Task Force on Outcomes Assessment

Kent M. Daum, O.D., Ph.D. (Chair)
Associate Professor of Optometry
University of Alabama at Birmingham
School of Optometry
Birmingham, Alabama

Morris S. Berman, O.D., M.S.
Professor of Optometry and Dean of Academic Affairs
Southern California College of Optometry
Fullerton, California

Roger L. Boltz, O.D., Ph.D.
Associate Professor of Optometry and
Associate Dean for Professional Studies
University of Houston College of Optometry
Houston, Texas

Presented at the Annual Meeting of the Chief Academic Officers, June 24-26, 2001
Accepted by ASCO's Board of Directors, November 16, 2002

Abstract
The Outcomes Assessment Resource Guide for Schools and Colleges of Optometry is designed to provide a basic orientation to outcomes assessment. The Guide includes a brief discussion of some important issues and provides resource tools that may be useful in an assessment program.

Keywords: Outcomes, assessment, program, performance, competency, educational research

Introduction

What Is Outcomes Assessment?
Outcomes assessment is the primary evaluative portion of the strategic management of a student, course, program or institution. An outcome is the result of a particular activity or program¹. Strategic management is a process designed to achieve the results (or outcomes) defined in mission, goals and objectives statements.

The institutional effectiveness paradigm² demonstrates the nature of the management process (Figure 1). Due to its circular nature, each portion of the paradigm is essential. Mission, goals and objectives provide direction for management and allow the application of necessary resources to achieve the intended outcomes. Assessment activities demonstrate the extent to which the outcomes are achieved. Finally, using the assessment results is crucial in ensuring that the intended outcomes are achieved. If modification is appropriate, assessment results provide direction to formulating improved strategies to achieve the mission, goals or objectives at the next iteration. The failure to execute any aspect of the paradigm or failing to maintain the connections of the different parts threatens the success of the entire program. Good management knows what it is trying to do (Mission, Goals and Objectives); does it (Outcomes); understands the output of its process (Assessment Activities); and uses its knowledge of the process to achieve an ever more successful result (Use of Results).

The institutional effectiveness paradigm can be applied at many levels. If applied to an institution (or program), a number of related versions of the paradigm should be aimed at the various goals and objectives for each part of the mission statement. Similarly, the effectiveness paradigm can be aimed at learning within an individual course, and as such, the paradigm may be simpler than that applied to programs or institutions. Ideally, an institution will have interlocking effectiveness paradigms that connect the institution to programs to individual students within and across all courses in the curriculum. Although any single paradigm can be straightforward, connections among effectiveness paradigms across programs and courses and students are challenging to develop, to manage and to keep connected.

Outcomes assessment, then, is a part of the process of managing an activity or program so that:

• The mission, goals and objectives clearly state the intended outcomes (or products or results) of the activity or program.
• The outcomes of the program or activity are assessed (measured or examined) using appropriate tools designed for the task.
• The assessment data are used to modify the program or activity to obtain or maintain the desired outcomes of the program or activity.

The assessment portion of this process provides feedback about the effectiveness of the program or activity.

What Drives Assessment?
Two major issues drive assessment, and it may be difficult to untangle them or to parse their relative importance. One obvious driver for assessment is the various oversight agencies (government {state and/or federal}, accreditation agencies, or institutional management). These agencies have a common desire to certify the extent to which an institution or program is achieving its mission, goals and objectives.

The other major driver of assessment is educational (management) theory and practice. Achieving the best results necessarily discovers a role for assessment (as described in Figure 1 above). Achieving intended results requires goals and objectives; intended outcomes; assessment activities; and includes the use of the results. Faculty should play the most significant role in each of these assessment processes.

In summary, an effective assessment process plays an important role in demonstrating the achievement of an institution, a program, a course or a student. In addition, however, an effective assessment process provides data to guide and improve achievement. The most important driver of assessment should be the desire to make good institutions, programs, courses and students better.

Outcomes Assessment in the Accreditation Process
The Accreditation Council on Optometric Education (ACOE) requires appropriate outcomes assessments for optometric educational programs as part of the accreditation process³. The ACOE "recognizes the importance of identifying and assessing educational and programmatic outcomes as a means to define and measure the quality of educational programs. It has woven outcomes assessment throughout its Standards of Accreditation⁴." Crucial to the understanding of assessment required by the ACOE is that programs are required to select assessment tools that are appropriate for each program. Schools and Colleges, therefore, should tailor an assessment program for their own particular mission, goals and objectives considering the resources available and the needs of the program.

Why is Outcomes Assessment Important?
Assessment is simply feedback on the outcome of a process. Without appropriate feedback an open-loop system is created. Open-loop systems (an engineering term) generally are not self-sustaining and do not usually have the ability to achieve equilibrium. In fact, since there is no feedback, open-loop systems may not even incorporate a goal.

A thermostat controlling temperature is an example of a useful feedback system. When the temperature gets too high, cold air is blown in (or the furnace is turned down). If the temperature sinks too low, the temperature is allowed to rise (or the heat is turned on). Assessment is like the thermostat of an educational process and setting it properly provides a way to adjust the process so that the outcome (i.e., like the temperature) is appropriate.

A characteristic of our current age (post-modernism) is the availability of a vast amount of information that does not provide true knowledge and understanding. An assessment program should provide information organized to efficiently evaluate the effectiveness of the entity being examined. Either too much, too little or inappropriate information is a problem. Since assessment gives insight into the integrity of the mechanism producing the outcome, it is impossible to critically adjust the mechanism without knowledge of the outcomes (sort of like tightening a bolt without being able to feel it tightening).

Ineffective assessment programs have several characteristics in common. They may collect information that does not truly reflect the achievement of the mission, goal or objective. Although this may be good information, it represents wasted time, effort and resources. An ineffective assessment program may collect valuable but disorganized information so that its use may be difficult or impossible. Also, the information may be collected and organized effectively but not provided to the appropriate users of the data. Or, the assessment may be completed on an erratic basis that is too widely spaced to effectively control the process. The most frequent ineffective assessment program is one that collects only subjective data from limited sources and therefore is unable to provide a comprehensive view of the program's achievements or its progress.

A good example of a potentially ineffective assessment program would be a curriculum committee attempting to manage the curriculum using only student evaluation of teaching (SET) feedback. Of course, SET feedback can be critically useful in gaining student perspectives on the effectiveness of the subject being taught. SET feedback does not evaluate the direct outcome of a course or program like certain performance measures (national or state boards or course-embedded performance tests (i.e., practical examinations)). In addition, the SET does not usually allow in-depth consideration of the causes of issues it may identify. A good assessment program is multi-factorial and uses different sources to consider the many facets of a desired outcome.

Good assessment programs manifest the following characteristics⁵ as their philosophical foundation (Table 1). A weakness or absence of any of these essentials places the entire program at risk.

Who Should Be Involved in Outcomes Assessment?
The best assessment programs gain strength from diversity in the groups of individuals surveyed. Programs should identify individuals from on- and off-campus that they would like to involve in their outcomes assessment.

Opportunities for Outcomes Assessment in Schools and Colleges of Optometry
There are many opportunities for the application of outcomes assessment in Schools and Colleges of Optometry. Assessment techniques can focus on growth and on participation and do not have to look only at the magnitude of certain fundamental aspects such as research dollar support. An effective assessment program may not require elaborate resource allocation. Depending on the needs of the program, assessment programs can be geared to obtain the most critical measures of effectiveness without an elaborate administrative apparatus.

Outcomes Statements
Achieving good statements to define the outcome of a process is a critical step in the successful management of that institution, program, course or student. Basic competency statements are statements of the outcomes of a professional program.

Good outcomes data are:

• Clear and understandable;
• Direct and explicit in meaning;
• Reflective of current philosophies, actions and intentions;
• Written in short, simple sentences that state only one thought; and,
• Quantifiable and measurable⁴.

Questions to be used in deriving outcomes statements are⁴:

• What do we want to accomplish?
• What is it we say we do?
• What is it that we want our graduates to be able to do?

Good examples of outcomes statements are:

• The entry-level Optometrist must understand and have skill in the prevention, diagnosis, treatment and management of systemic conditions and processes that relate to vision.
• The entry-level optometrist must demonstrate appropriate personal, professional and ethical values.
• The student must demonstrate competency in performing Goldmann tonometry.

Outcomes are the objectives of an institution, program or course. They are the desired endpoints or achievements of the activity.

Characteristics of Assessment Measures
Assessment activities can be broadly classified as either perception about the outcomes or measures of the performance of the outcomes. Perceptions are subjective assessments and can be powerful measures of the opinions of the various participants or observers about various items within a program. Performance measures are designed to directly examine the outcome of items within a program (or course, etc.). Frequently, perceptions are most useful in helping to explain the performance of some aspect of the program.

Many other features of assessment activities should be considered when developing or using assessment tools. Some of these are described in Table 2. A brief examination of these characteristics demonstrates the challenges involved in understanding the ramifications of the assessment tools. Of course, a given tool can be tightly controlled or relatively uncontrolled as long as the tool meets the needs of the program. Of the many characteristics listed below, the priority and cost as well as considerations regarding validity, accuracy, strengths and weaknesses are especially critical for the successful selection of a tool.

Program Assessment vs. Learning Assessment
Much of the literature on outcomes assessment today is aimed at the assessment of learning. An outstanding example of a program using assessment to guide student learning is Alverno College. Alverno College has developed its entire curriculum to incorporate carefully defined objectives for student learning^6-8. The Alverno College faculty has defined eight expected outcomes for their liberal education process (Table 3). Each of these outcomes is a goal of the educational process at Alverno College. Competencies are developed for specific tasks that make up each of the outcomes.

Judging each competency that makes up an ability incorporates a circular process designed to provide appropriate feedback to all of the stakeholders in the educational process but particularly the student and instructor. These components are listed in Table 4. Each of the components is linked to the others.

Assessment and Basic Competency
After immense work, the optometric education community recently approved a document describing basic competencies for entry-level optometrists⁹. This may be the first successful description of the knowledge, skills and values that graduating optometry students should demonstrate. Methods must be developed to confirm these competencies of graduating optometry students. The development of assessment methodology provides Schools and Colleges a way to accurately know what their products, the optometric graduates, know and can do. This allows the educational institutions to take steps to document the achievement of these competencies and to manage any challenging areas.

Programs may find it helpful to re-organize their basic competency statement into broad categories with appropriate knowledge, skills and values for each. The resulting fewer outcomes may allow a more manageable assessment task.

Basic Methodologies for Conducting Outcomes Assessment

Reliability, Validity and Accuracy
A reliable assessment tool provides data where the variance in the findings is primarily a result of the differences in the sample with respect to what is being measured, rather than bias or other knowledge or skill⁵. Clarity in the statements, adequate time to complete the instrument and good training for the assessors are vital components of achieving a reliable tool⁵.

Validity implies that the tool measures the desired characteristic. This is a critical attribute for an assessment tool. Essentially, validity represents the honesty of a tool, i.e., the tool actually measures what it is designed to measure and this is known⁵.

Accuracy describes whether a tool is a true measure of an outcome. Written tests may be measures related to a clinical skill. Often, however, the true measure of a clinical skill is a demonstration of that skill in an applied state, such as performing the skill on a patient.

Ideally, assessment instruments should provide evidence of their reliability, validity and accuracy. The NBEO examination provides data to support these factors in its reports. Many local instrument test packages also provide standard error of measurement and other data to assist in judging reliability, validity and accuracy. Unfortunately, many locally developed assessment tools must operate without substantial measures of these characteristics. Such tools should have face validity, i.e., the tool is judged to be valid by an examination of its nature.

Surveys
Surveys are a common method to obtain perceptions about particular aspects of an institution, program or course. Generally, survey questions could consider some or all of the following attributes of a given aspect (Table 5). Frequently, as a matter of practicality, surveys must be limited to the most important questions for an outcomes assessment plan. It is important to remember that good outcome assessment plays a critical role in the ongoing management of a program.

Assessment Tools
The Outcomes Assessment Survey of Schools and Colleges of Optometry¹⁰ contains a variety of assessment tools. These tools include copies of surveys of students, faculty, alumni, residents, peer institution faculty, exit surveys, exit interviews, mid-curriculum surveys, and a number of other related tools.

An assessment grid may be a useful method of tracking the objectives for each assessment tool. This grid has the assessment tools along one axis and the competencies or standards being assessed along the other. This can be a useful method assuring that all competencies are assessed.

Similarly, individual courses may find it helpful to make an outcome vs. course objectives grid. This may assist in tracking the methodology to assess each course objective. Individual course learning objectives should exist for each course to guide students in approaching the material that is being presented.

A good goal for a curriculum is to have coordinated syllabi. Coordinated syllabi may be useful in tracking curricular contents across courses. This provides the faculty and the curriculum committee a method to understand what is being taught and where. This process may also be helpful in educating faculty about the curriculum.

Selecting or developing an assessment tool should not be taken casually. Assessment tools should span the program and the nine-accreditation standards. Surveys are good methods of obtaining data and can be relatively inexpensive. Data collection should be undertaken for areas other than the curriculum on a judicious basis. The curriculum demands an ongoing, multi-factorial process. Unfortunately, there are few, if any, commercially available assessment tools designed for professional optometric programs outside of NBEO and OAT examinations.

Assessment Programs
Selecting appropriate assessment tools for a School or College of Optometry can sometimes be daunting in view of the large number of possible tools for the nine standards of the ACOE accreditation requirements. The Outcomes Assessment Survey of Schools and Colleges of Optometry provides many possible tools for the nine standards. This can be a good source to review in initially constructing an assessment program.

Programs may find it useful to select a peer group of optometric institutions that have a similar mission. Within the overall group of optometry schools and colleges, there may be at least two natural peer groups, the public and the private schools and colleges. A peer group allows appropriate comparison of data. Comparing the total research budget of a large, public institution to the budget of a small private institution may be inappropriate and misleading.

Designing & implementing an outcomes assessment plan is obviously a substantial task and requires the support of the administration of the School or College. Certainly, the task should be assigned to someone or some group. Often this could be either the curriculum committee or an assessment committee. One of the important features of assessment is that to be effective, it must be ongoing and regular. Assessment must not be limited to accreditation self-studies.

Assessment programs have many limitations. As we all know, "garbage in, garbage out." Starting slowly and carefully is important in assessment. Keeping the program small and efficient is important, particularly in the beginning. The individuals collecting the data should be constantly asking the receivers and users of the data, "Is this useful?" Or, "Does this information help in managing the program?" If the answers are not strongly "yes," then the assessment effort should be revised. Managing a program or course or institution requires an immersion of the manager into the environment. When this occurs, assessment data provides data that substantiates impressions. The assessment data must be properly interpreted and, of course, most issues in academia are complex and challenge even bright, dedicated people.

Since assessment is an integral part of the strategic management of a program, good assessment requires the integration of the assessment into the program and its management. There are many challenges to outcomes assessment for a program. Some of these are listed in Table 6.

Assessment of Standards for Accreditation for Professional Optometric Degree Programs 2000
A brief discussion of an assessment approach for each of the nine standards for accreditation by the ACOE is contained below. Decisions regarding assessment of the standards should consider issues and the assessment approach and provide a variety of assessments for each standard.

Standard I. Mission, Goals and Objectives
The assessment approach for the area of mission, goals and objectives is twofold:

1. An examination of the mission, goals and objectives statement for clarity, completeness, appropriateness and accuracy.
2. Documentation of opportunities for examination and revision of the mission, goals and objectives.

The assessment also may include an examination of the style of these statements. Generally, evidence of revision after discussion in new versions of the School catalog and surveys of the stakeholders on a periodic basis is sufficient to achieve a good assessment outcome for this area. In addition, a close look at these statements by the faculty during accreditations may be a helpful anchor point. A comparison to other program's statements may assist in this process. The assessment process for the other eight standards is probably the truest indicator of the practical meaning of mission and goals of the institution. Therefore, in some sense, an assessment of the mission, goals and objectives is also included in assessments of the other standards.

Standard II. Governance, Regional Accreditation, Administration, and Finances
Assessing some portions of this standard is a matter of confirming that certain authority, policies, credentials or accreditation exists. As such the best assessment tool is a verification process that assures that these required elements are in place on a periodic basis. One important element of this assessment is to examine the source documents for each. For example instead of using a curriculum vita as evidence of the CEO's Doctor of Optometry degree, a transcript or a diploma should be used. Certain assessment tools are designed to provide opportunities for comment, discussion and revision of the structures used for governance and administration. Of particular importance is evidence of the use of these data by the institution as opposed to simply gathering the data. Evidence of use may be provided in minutes of faculty or executive meetings or in changes in survey data over time or other linkages between the tools and the daily governance or administration of the program.

Because of the scope and complexity of the area of finance, a simple assessment approach may prove useful unless there are particular areas that seem questionable and are worthy of close examination. Probably a good approach is to assemble data of three basic types:

1. Financial performance data. This data should include a review of the basic financial trends for income and expense categories over time, at least five years. This area can include a more intensive review of certain areas such as faculty or staff salaries and benefits.
2. Perceptions of stakeholders. Surveys of various sorts can be designed to examine the impressions of faculty, staff, students and other interested groups about the financial management of the institution.
3. Evidence of discussion, revision and use of assessment data. A very important aspect is to demonstrate that the appropriate financial data is collected on an ongoing basis, that appropriate individuals including faculty examine and discuss the data; and, that the process is arranged so that appropriate adjustments are made on a periodic basis.

Standard III. Facilities, Equipment and Resources
As a result of the standard's wording, most of the assessment probably should involve an examination of perceptions of the parties involved in the teaching and patient care (faculty, students, alumni, patients and staff). Certain accreditation agencies for patient care (e.g., Joint Commission on Accreditation of Health Care Organizations, http://www.jcaho.org/index.html) exist and these agencies potentially may be useful sources of benchmark data regarding patient care. Comparing teaching facilities to other programs may provide a useful subjective benchmark for faculty assessors. The primary benefit of comparisons like these is not so much to provide an "OK" or "not OK" outcome for the assessment but rather to provide ideas for the discussion of desired improvements to these areas. Finally, the assessment program should provide assurance that there are regular opportunities for interested individuals to discuss and recommend improvements to these areas.

The approach to assessing the library falls into a tri-partite pattern:

1. An assessment of the magnitude and scope of several library resources such as the number of volumes or the number of serials available or the number and scope of electronic databases available.
2. A survey of the various users of the library resources to determine their satisfaction with and their perceptions of the library.
3. Documentation of an ongoing, continual cycle of discussion, formation of objectives, assessment of outcomes and revision as applied to the library.

It is important to ensure that this critical area is considered on a periodic basis and that planning occurs to ensure its continual availability and competence.

The approach required for the informational services area involves the following items:

1. An assessment of the presence of appropriate goals and objectives for the area to guide the process.
2. A comparison of resources provided and practices used by IS at other institutions may be a useful guide in formulating opinions about the IS at a particular institution.
   
3. An examination of user satisfaction with the area and perceptions of strengths and weaknesses.
4. Documentation of an ongoing process of goals, outcomes, assessment and revision by the appropriate persons such as IS team, administration, faculty, staff, students and alumni.
   

Standard IV. Faculty
The approach for assessing faculty includes four basic parts:

1. Assembling goals and objectives for individual faculty and for the faculty as a whole allows an examination of the progress of a faculty.
2. Using survey data to understand the perceptions of faculty about the many aspects of their positions.
3. Using cross-program compilations to indicate a faculty's performance.
   
4. Documenting the examination of goals and objectives and assessment of outcomes and then using that information to modify the outcomes.

Standard V. Students
The approach for an examination of students is greatly assisted by survey data provided by ASCO that provides such data as entering GPAs, OAT scores and related data for all institutions. This simplifies cross-program comparisons of entering student characteristics. Perceptions of student needs and perceptions of program strengths and weaknesses can be accomplished by means of survey instruments. Program outcome data such as job success and satisfaction require a survey of alumni. As with each standard, perhaps the most important feature of an assessment program is to ensure that continual examination and modification of program characteristics that affect students goes on periodically and regularly.

Standard VI. Curriculum
The assessment of a complex organ such as the curriculum must have many approaches and include measures of program outcomes. The most basic measures can include the following:

1. NBEO pass rates
2. State Board of Optometry pass rates
3. Individual course basic competency examinations

Particularly if data is examined over time, these can be very effective measures of program success. NBEO scores also are commonly used to consider the effectiveness of individual courses within the curriculum. Individual course competency exams can be designed to ensure the presence of critical skills.

Perceptions of the curriculum can be collected from student evaluation of teaching at the time of the course or at specified times within the curriculum such as mid-curriculum or from student exit surveys. Student exit interviews are useful ways to help the curriculum committee (or others) gain a deep, first hand understanding of the issues facing students regarding the program and the curriculum. The first hand experience can be an effective tool for motivating curriculum members to make changes.

Finally, probably the most important feature of a curriculum assessment process is to document occasions for thoughtful and regular examination of the state of the curriculum, its outcomes and modification as necessary and appropriate.

Standard VII. Clinic Management and Patient Care Policies
This is an area where the most important features of the assessment process should be:

1. Documenting the presence and viability of the required policies, manual, etc.
2. Documenting the presence of appropriate review, discussion and modification of the portions of the program in this area.

Standard VIII. Research and Scholarship
The assessment of research and scholarship for a program or an individual revolves around three aspects:

1. Documentation of various indicators of research and scholarship productivity including research dollars, number of publications and number of citations.
2. Compilation of perceptions of appropriate individuals regarding the status of research and scholarship and the identification of strengths and weaknesses.
3. Determination that an ongoing cycle of review and modification exists to improve research and scholarship.

Standard IX. Residency Education
We recommend that an assessment of residency programs include the following aspects:

1. The number and variety of residency programs.
2. The accreditation of the residency programs offered by an institution.
3. Perceptions of appropriate individuals about the residency program.
4. Documentation that an appropriate cycle of strategic planning is ongoing within the institution.

Finally
Managing an educational program in the Schools and Colleges of Optometry continues to be a challenging task. Fully utilizing the power and clarity provided by an effective assessment program can provide significant managerial assistance while at the same time providing confidence in the outcomes of the program; demonstrating evidence of success; and illuminating opportunities for improvement. Assessment, like a good case history, is never truly finished; each outcome and assessment leads to improved ways to achieve the mission of the program.

Acknowledgments
We gratefully acknowledge the support of the CIBA Vision/ ASCO Total Quality Education Grant Program. This resource was presented at the annual meeting of the Academic Officers of the Schools and Colleges of Optometry, Boston, MA, June 24-26, 2001. We extend thanks to Dr. Diane Beck, professor, School of Pharmacy, Auburn University, Auburn, Alabama. Dr. Beck, a widely published expert on outcomes assessment, reviewed the paper prior to publication.

References

1. Hollenbeck RG, et al. Chair Report of the 1998-99 Academic Affairs Committee, American Association of Colleges of Pharmacy.
2. Nichols JO. Implementing Institutional Effectiveness in Educational Programs to Meet Southern Association of Colleges and Schools' Accreditation Requirements. Presentation at Auburn University, August 29-30, 2000.
3. Council on Optometric Education. Accreditation Manual: Professional Optometric Degree Program Standards 2000. Council on Optometric Education, St. Louis, MO. 42 pp. (Professional Optometric Degree Program Standards 2000, Standards 1.3, 5.1, 6.3, 6.4, 6.8, 6.9 and 7.6).
4. Council on Optometric Education. Assessing Outcomes in Optometric Education, A commentary by the Council on Optometric Education. Optometric Education 23(2): 54-56, 1998.
5. Palomba CA, Banta TW. Assessment Essentials, Planning, Implementing and Improving Assessment in Higher Education. Jossey-Bass Publishers, San Francisco, 1999. 405 pp.
6. Alverno College Faculty. Student Assessment as Learning at Alverno College. Alverno College Institute, Milwaukee, Wisconsin, 1994. 148 pp.
7. Alverno College Faculty. Ability-Based Learning Program Nursing Education. Alverno College Institute, Milwaukee, Wisconsin, 1999. 12 pp.
8. Alverno College Faculty. Ability-Based Learning Program. Alverno College Institute, Milwaukee, Wisconsin, 2000. 4 pp.
9. Heath DA, Daum KM, DiStefano AF, Haine CL, Schwartz SH. Attributes of Students Graduating from Schools and Colleges of Optometry. An Association of Schools and Colleges of Optometry Report, 20 June 2000; Journal of Optometric Education Fall 2000; 26(1): 15-18.
10. Daum KM, Berman MS, Boltz RL, Beck D. Outcomes Assessment Survey of Schools and Colleges of Optometry. J Optom Ed 2003; 28(2):51-63.

Additional Resources

• American Association for Higher Education. Catalog of Publications, 2000. AAHE, Washington DC. 13 pp.
• American Association of Colleges of Pharmacy. Handbook on Outcomes Assessment. AACP, Alexandria, VA, 1995. 38 pp.
• Anderson RS, Speck BW, eds. Changing the Way We Grade Student Performance: Classroom Assessment and the New Learning Paradigm. Jossey-Bass Publishers, San Francisco, Number 74, Summer 1998. 111 pp.
• Angelo T, ed. Classroom Assessment and Research: An Update on Uses, Approaches and Research Findings. Jossey-Bass Publishers, San Francisco, Number 75, Fall 1998. 116 pp.
• Banta TW, ed. Making a Difference, Outcomes of a Decade of Assessment in Higher Education. Jossey-Bass Publishers, San Francisco, 1993. 388 pp.
• Banta TW, Lund JP, Black KE, Oblander FW. Assessment in Practice, Putting Principles to Work on College Campuses. Jossey-Bass Publishers, San Francisco, 1996. 387 pp.
• Beck DE. Module 5: Assessment of Instructional Effectiveness. Health Profesions Education Scholar, Teaching Excellence and Scholarship Development Resources for Health Professions Educators. http://www.educationscholar.org/
• Gaither GH, ed. Assessing Performance in an Age of Accountability: Case Studies. Jossey-Bass Publishers, San Francisco, Number 91, Fall 1995. 107 pp.
• Hagedorn LS, ed. What Contributes to Job Satisfaction Among Faculty and Staff? Jossey-Bass Publishers, San Francisco, Number 105, Spring 2000. 117 pp.
• Loacker G, ed. Self Assessment at Alverno College. Alverno College Institute, Milwaukee, Wisconsin, 2000. 162 pp.
• Loacker G, ed. Student Assessment as Learning at Alverno College. Alverno College Institute, Milwaukee, Wisconsin, 1985. 148 pp.
• Loacker G, Cromwell L, Fey J, Rutherford D. Analysis and Communication at Alverno: An Approach to Critical Thinking. Alverno College Institute, Milwaukee, Wisconsin, 1984. 189 pp.
• Malaney GD, ed. Student Affairs Research, Evaluation and Assessment: Structure and Practice in an Era of Change. Jossey-Bass Publishers, San Francisco, Number 85, Spring 1999. 100 pp.
• Mentkowski M, ed. Learning that Lasts, Integrating Learning, Development and Performance in College and Beyond. Jossey-Bass Publishers, San Francisco, 2000. 536 pp.
• Ratcliff, ed. Assessment and Curriculum Reform. Jossey-Bass Publishers, San Francisco, Number 80, Spring 1992. 108 pp.
• Riordan T, ed. Valuing in Decision-making, Theory and Practice at Alverno College. Alverno College Institute, Milwaukee, Wisconsin, 1992. 18 pp.
• Schuh JH, Upcraft ML. Assessment Practice in Student Affairs, An Applications Manual. Jossey-Bass Publishers, San Francisco, 2001. 510 pp.
• Social Interaction Department. Teaching Social Interaction at Alverno College. Alverno College Institute, Milwaukee, Wisconsin, 1994. about 100 pp.
• Sullins WD. Mission, Goals and Objectives, Memorandum, Council on Optometric Education, October 13, 1999.

Tables

Table 1. Essentials for Assessment Programs (modified⁵)

Table 2. Characteristics of Outcome Assessment Measures

Table 3. Expected Outcomes for Education at Alverno College⁷.

Table 4. Design and Implementation of Assessment as Learning Process at Alverno College⁷.

Table 5. Common Areas Addressed by Assessment Survey Questions

Table 6. Challenges to Outcomes Assessment

Figures

Figure 1. Institutional Effectiveness Paradigm².