Examples Of Assessment Methods For Administrative Program Outcomes Management Essay
Purpose: The purpose of this Institutional Effectiveness Manual (“Manual”) is to provide a formal statement and to describe the processes and quality assurance mechanisms pertaining to institutional effectiveness at the Abu Dhabi School of Management (“ADSM” or “School”).
Policy: It is the policy of ADSM to develop and maintain an institutional effectiveness process for systematically and continuously collecting, analyzing, and acting upon data related to the goals and outcomes that further the School’s mission.
Procedure: Under the authority of the Board of Trustees, the faculty, administration, and staff of ADSM will participate in an ongoing planning-assessment-improvement cycle for both student-learning outcomes and administrative-services outcomes. The Director of Institutional Research will coordinate the process and will provide consultation to all School personnel regarding research design, survey development, measurement pedagogy, quantitative analysis, and related topics that support planning, assessment, and decision-making at the unit and institutional levels.
I. Institutional Effectiveness 3
II. Procedures for Institutional Effectiveness Planning 4
III. ADSM Institutional Effectiveness Framework 5
Integration of Institutional Effectiveness Assessment 5
Abu Dhabi School of Management Vision, Mission & Strategic Objectives 5
Expected Outcomes 7
IV. ACADEMIC Program Guidelines 9
Student Learning Outcomes 10
Methods of Assessing Student Learning Outcomes 13
Examples of Methods of Assessing Student Learning Outcomes 13
Academic Program Outcomes 14
Examples of Program Outcomes for Academic Departments 14
Methods of Assessing Academic Program Outcomes 14
Examples of Program Outcomes Assessment Methods for Academic Departments 14
Performance Targets 15
Examples of Performance Targets 15
Reporting and Using Results 16
Examples of Assessment Results and Their Use 16
VI. Administrative Program Outcomes 18
Examples of Administrative Program Outcomes 18
Examples of Assessment Methods for Administrative Program Outcomes 18
Performance Target 19
Examples of Performance Targets 19
Reporting and Using Actual Results 19
Examples of Assessment Results and Their Use 20
I. Institutional Effectiveness
According to the Southern Association of Colleges and Schools, Institutional Effectiveness means that the “institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on the analysis of those results. [] “
Institutional Effectiveness, therefore, is the process by which the Abu Dhabi School of Management is able to meet its stated mission and objectives. The process allows each unit of the School to define its expected outcomes which are directly derived from and aligned with its stated mission and objectives. A mission statement is formulated by faculty and administration for each academic program and administrative unit stemming from the School’s mission statement. In this way, each School unit will achieve strategic alignment with the overall strategic objectives of ADSM. Program outcomes become the subject of an annual Institutional Effectiveness assessment cycle. The IE assessment Cycle will evaluate whether or not pre-defined outcomes are successfully acheived within the year. Finally, the results of IE assessment cycle are used to identify any gaps and to recommend any necessary modifications for continuous improvements within academic and administrative programs.
Assessment
Assessment is the process by which evidence is collected, evaluated, and reviewed to determine the effectiveness of the School’s academic and administrative programs and services. The main goal of assessment is to improve student learning and the delivery of services to the School community. Specifically, assessment performs two main functions:
Provide information for student learning program improvements-this is accomplished by providing feedback and identifying areas of strengths and weaknesses in student academic performance. Improvement plans to address academic weaknesses are developed and areas of academic strength are highlighted and strengthened.
Provide information about School programs accountability-ADSM has both internal and external accountability. External accountability involves providing evidence to ADEC, CAA, and AACSB for meeting accreditation requirements. Internal accountability involves evaluating the School’s programs and services to help in academic and administrative program evaluation.
II. Procedures for Institutional Effectiveness Planning
An Institutional Effectiveness Plan answers four key questions:
What do we want to accomplish?
Are we achieving what we want to accomplish?
How well are we achieving the results we want?
What are the means we are using to improve what we are doing?
In order for an Institutional Effectiveness plan to be realized, it must possess certain characteristics. These may be summarized as follows:
Defined Outcomes- outcomes must be clearly stated.
Defined Measurements- each outcome must have its associated measurement.
Defined Target- each outcome must have a clearly stated target to achieve.
Assessment Method-there should be defined methods to determine to what degree a desired target outcome has been met using the measurement tool.
Continuous Improvement- the assessment method should be used to improve the performance of the School’s academic programs and administrative units.
III. ADSM Institutional Effectiveness Framework
Integration of Institutional Effectiveness Assessment
The Abu Dhabi School of Management will formally track and assesses its progress toward strategic goal achievements for several years through the development and implementation of a 5-year Strategic Plan and its associated Annual Action Plans.
The strategic management process ensures that resources, decision-making, initiatives, and activities are all aligned with the strategies of ADSM and are executed and monitored correctly. ADSM has developed a conceptual framework for institutional effectiveness (see Figure 1). The framework is a 3-tier model which outlines the relationship between the strategic planning process and outcomes assessment, starting from the School’s level down to the Department, Unit, and Program levels.
Abu Dhabi School of Management Vision, Mission & Strategic Objectives
Vision
The Abu Dhabi School of Management’s vision is to be a center of excellence for entrepreneurship, leadership, and management through the discovery and dissemination of knowledge.
Mission
The Abu Dhabi School of Management’s mission is to develop entrepreneurial managers with the knowledge and skills to contribute to sustainable socio-economic development in the knowledge economy. The School aims to create an enriching and rewarding environment which promotes entrepreneurialism, scholarly inquiry, research, and UAE cultural heritage while fostering diversity, understanding and tolerance.
Strategic Objectives
To create a new cadre of highly talented and diverse entrepreneurs equipped in the science of management in the knowledge economy with practical competencies to take the lead in innovating, improving, and enhancing their environments;
To benchmark its programs and business operations to international standards of excellence;
To establish strategic partnerships with quality international colleges and universities that possess a solid record of academic excellence and achievements;
To recruit and retain faculty and staff with international qualifications and a proven record of accomplishments;
To integrate entrepreneurial values in all of its curricula and business culture;
To promote UAE culture and heritage,
To advocate cultural diversity, understanding and tolerance; and
To increase the intellectual capital of the UAE and the region through original and applied research in the management discipline.
ADSM’s Mission Statement describes the School’s aspiration and educational philosophy. In addition, the School has identified eight strategic objectives in order to fulfill its mission and future vision. All ADSM department and administrative mission statements should be derived from the School’s mission statement and one or more of the eight strategic objectives. The IE assessment plan links each department’s or unit’s mission statement and goals to the School’s mission statement and objectives.
Expected Outcomes
Expected outcomes can be classified as eitherStudent Learning Outcomes or Program Outcomes. The former involves the knowledge, competencies, behaviors, and values expected of the student after program completion. The latter, on the hand, shift away from student learning and is premised on particular operational or programmatic objectives that academic departments and administrative units seek to achieve. It is imperative for academic and administrative units to align their outcomes with ADSM’s outcomes and directly contribute to achieving ADSM’s performance targets as detailed in ADSM’s Business Plan.
ADSM Strategic Outcomes
ADSM has identified strategic outcomes as a measure of achieving its mission and strategic objectives, these are:
Students at ADSM will be engaged in high-quality, entrepreneurial learning experiences and will consistently report high levels of satisfaction with the overall quality of their learning experiences. ADSM will also experience improvements in measures of student success.
ADSM will meet undergraduate and graduate enrollment targets.
Recruitment and retention programs will succeed in attracting and retaining high caliber faculty and staff.
ADSM will be internationally recognized as a top research School through achievements made within the Business Gaming Center, collaborative research initiatives, and through research activities of individual faculty.
Increase in annual research production and transfer of research knowledge through an increase in publications, symposia, conferences, etc.
Through the School’s efforts, and through an increased investment from the government and its partners, ADSM will be able to execute effective long-term planning based on sustainable, unrestricted funding.
Business structure, process and practices will support ADSM’s academic priorities.
Increase in the number of partnerships and collaborative arrangements.
Increased revenue from research activity, Business Gaming Center, Center for Innovation & Entrepreneurship, and from advancement activities such as grants & strategic partnerships.
IV. ACADEMIC Program Guidelines
“The trouble with having no goals, it has been said, is that you may achieve them.
…The best way to begin developing a curriculum is to consider not what courses universities should require, but what qualities they should seek to nurture in students.”
Frank H. T. Rhodes, President Emeritus, Cornell University
The Chronicle of Higher Education, Sept. 14, 2001, B-8-9.
The academic department’s stated goals and outcomes form the foundation for assessing the quality of its academic programs. The process of assessment starts with the faculty. The faculty in each academic department must identify the department’s goals and the student learning outcomes for each degree or support program. The academic goals and objectives should identify the program’s student learning outcomes. The central question which must be answered is: What knowledge, skills, and values should a graduate of our program have?
Fundamentally, assessment enables the department to determine how well students are meeting the academic goals and outcomes determined by the faculty. In this way, program strengths and weaknesses can be identified. The information is used by the faculty to make the necessary modifications to address the areas of weaknesses and accentuate the areas of strengths. The overall effect is continuous quality program improvements.
Academic program assessment is a collective effort which should involve the entire department’s faculty. If the School is to have an effective assessment program that is focused on quality improvement, then the entire faculty must be involved in the assessment process. Student learning outcomes should be determined by the faculty with the desired levels of performance. The learning outcomes are periodically assessed to determine if they are being achieved at desired levels. If the outcomes are not being achieved at the desired levels, then an intervention mechanism is developed by the faculty which begins by determining the causes of the problem and the improvements which need to be implemented in order to achieve the desired level of student performance.
Student Learning Outcomes
“By establishing outcomes, every undergraduate unit on campus creates for itself a benchmark for everything that it does educationally.”
Chris Anson, Professor of English and Director, Campus Writing and Speaking Program, University of North Carolina
It is important to distinguish student learning outcomes from course learning outcomes. While the course outcomes focus on an individual course within a program and state their specific learning objectives, learning outcomes focus on a program objective comprised of a set of courses. The convention is for student learning outcomes to use action verbs for illustrating a student’s expected knowedge and competency levels upon finishing a certain program. To achieve strategic alignment, these outcomes should be directly related to at least one of the above enumerated strategic objectives of the Abu Dhabi School of Management. Action verbs associated with student learning outcomes are verbs such as “explain,” “analyze” and “evaluate” and should be used instead of “be exposed to,” “have an opportunity to,” or “be familiar with.” The outcome should be stated in terms such as: “Students will be able to, analyze ____ to describe knowledge, skills, or attitudes….”
Action Verbs and Types of Learning
Cognitive Learning
Examples of Action Verbs
Knowledge-to recall or remember facts without necessarily understanding them
articulate, define, indicate, name, order, recognize, recall, reproduce, list, tell, describe, identify, show, label, tabulate, quote
Comprehension-to understand and interpret learned information
classify, describe, discuss, explain, express, interpret, contrast, associate, differentiate, extend, translate, review, suggest, restate
Application-to put ideas and concepts to work in solving problems
apply, compute, give examples, investigate,
experiment, solve, choose, predict, translate,
employ, operate, practice, schedule
Analysis-to break information into its component to see interrelationships
analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, distinguish, examine, investigate, interpret
Cognitive Learning
Examples of Action Verbs
Synthesis-to use creativity to compose and design something original
arrange, assemble, collect, compose, construct, create, design, formulate, manage, organize, plan, prepare, propose, set up
Evaluation-to judge the value of information based on established criteria
appraise, assess, defend, judge, predict, rate,
support, evaluate, recommend, convince, conclude, compare, summarize
Affective Learning
appreciate, accept, attempt, challenge, defend,
dispute, join, judge, praise, question, share, support
Source: Institutional Effectiveness Planning Workbook 2004-2005, Texas A&M University
Methods of Assessing Student Learning Outcomes
An assessment method is the means by which a department or unit can measure the degree of success in meeting a student learning or program outcome. The department or unit may use more than one assessment method. These methods may be direct or indirect.
Direct methods– measure what was learned.
Indirect methods– measure perceptions of learning.
In general, both indirect and direct methods should be used to pinpoint specific learning problems and determine the necessary improvements.
Examples of Methods of Assessing Student Learning Outcomes
Direct Assessment Methods
Indirect Assessment Methods
ETS Major Field Tests
Capstone Courses
Grading Using Scoring Rubrics
Case Studies
Licensing or Certification Exams
Student Portfolios
Senior Research Projects
Senior Recitals
Locally Developed Tests
Course Embedded Assessment
Evaluations of Interns
External Examiners or Reviewers
GRE Subject Tests
Student presentations of research to professional organizations
Student Perception of Learning Surveys (e.g. IDEA, NSSE)
Exit Interviews
Alumni Surveys
Analysis of Course Syllabi
Focus Groups
Employer Surveys
Academic Program Outcomes
Objectives set forth by academic or administrative units constitute program outcomes. These particular outcomes identify what will be achieved by the unit or program, and also include perceptions about what was achieved.
Examples of Program Outcomes for Academic Departments
Graduates will be successful in gaining admissions to graduate school or professional programs.
Focused recruitment to increase the number of students in the Foundation Program.
BSc graduates from the Business Management Program-Green Business Management concentration will obtain employment in green business companies.
Graduates will be competitive in obtaining admission to post-baccalaureate programs at highly-ranked international universities.
Faculty research productivity as revealed by papers published or presented will increase by 5 percent annually.
Methods of Assessing Academic Program Outcomes
Program outcomes must be quantifiable. In particular, they must identify desired outcomes that will provide evidence of program quality and whether administrative goals have achieved.
Examples of Program Outcomes Assessment Methods for Academic Departments
Success in student recruitment
Student satisfaction surveys
Student Opinion Survey
Alumni surveys
Projects are on-time and on-budget
Student engagement in the learning process
Student graduation rates
Student retention rates
Student employment rates
Tracking program participation by desired demographics
Faculty publications and presentations rates
Complaint tracking and resolution
Acceptance rates to graduate and professional schools
Monitoring service usage
Performance Targets
Once the student learning and academic program outcomes are identified and their assessment methods have been determined, the next step is to establish their performance targets. The performance target determines how well a student or program performs. Performance targets should be quantitative and easy to measure.
Examples of Performance Targets
At least 80% of students completing this program will achieve a score on the GRE above the 50th percentile of the national distribution.
The department faculty will increase the number of annual publications by at least five percent.
Achieve 85% satisfaction rate of the users of IT services.
The department will achieve 90% retention rate of newly recruited faculty.
In all of the examples above, the exact quantitative targets are set and data is collected relative to the targets. Avoid vague targets such as “The department will increase user satisfaction rate during this academic year.” This implies that any increase, even of just one user being satisfied, is acceptable. When considering a specific target, make sure that the performance target is realistic and achievable. Avoid setting unrealistically high targets since achieving them may be too difficult. Similarly, avoid setting unrealistically low targets to assure success. Remember that not reaching a performance target does not always indicate that there is a problem with the program.– Performance improvement is a process and it may take several attempts to reach the desired performance level. What is important is having the will to improve, instituting, and following the mechanisms for improvements.
Reporting and Using Results
It is the policy of ADSM that actual results must be reported for each student learning and program outcome by the end of the Spring Semester. The results must show whether or not the performance targets of the student learning and program outcomes were met. Specifically, the results must identify both the strengths and weaknesses of each academic or administrative program. A plan to address the weaknesses must also be submitted. This will be the ultimate objective of the continuous quality improvement process of Institutional Effectiveness.
Examples of Assessment Results and Their Use
Emirati freshman enrollment increased by 5% and exceeded the performance target of an increase of 2% over the previous year. The target increase for next year will be set at 6%.
The percentage of graduates of the Foundation Program who scored 550 of the paper base TOFEL exam this year was 80%. This was the first year that that the average score has exceeded the benchmark set by the department of 70%.
VI. Administrative Program Outcomes
Similar to academic program outcomes, administrative program outcomes also identify desired outcomes which measure program quality and the degree to which the unit’s goals are achieved.
Examples of Administrative Program Outcomes
Build and maintain IT infrastructure and instructional support services for faculty and students.
Student degree planning by the academic advisors will be accomplished accurately and in a timely fashion.
Students advising services will provide sound academic advising.
The Foundation Program will help students decide upon an academic major.
Students will learn how to use online library resources effectively to conduct research and complete their assignments.
Examples of Assessment Methods for Administrative Program Outcomes
Success in student recruitment
Student satisfaction surveys
Student Opinion Survey
Alumni surveys
Projects are on-time and on-budget
Student graduation rates
Student retention rates
Student employment rates
Tracking program participation by desired demographics
Faculty publications and presentations rates
Complaint tracking and resolution
Acceptance rates to graduate and professional schools
Monitoring service usage
Organizational excellence standards
Federal and Emirate levels benchmarks
External quality measures
Performance Target
Once a program outcome is identified and its associated method of assessment is determined, the next step is to establish the desired performance target. A criterion for success is determined as it identifies a program’s expected performance in the assessment. The performance target determines how well an administrative program performs. Performance targets should be quantitative and easy to measure.
Examples of Performance Targets
To attain ninety-five percent completion rate of degree plans
Students successfully completing the Foundation Program will have chosen a business management major at ADSM within the first year of admission into the academic program.
Sixty percent of the students who use the Learning Resource Center will express high satisfaction levels of the services provided services
Eighty percent of students using the Student Advising Center will express high satisfaction levels of the services provided services.
Reporting and Using Actual Results
It is the policy of ADSM that actual results must be reported for each student learning and program outcome by the end of the Spring Semester. The results must show whether or not the performance targets of the student learning and program outcomes were met. Specifically, the results must identify both the strengths and weaknesses of each academic or administrative program. A plan to address the weaknesses must also be submitted. This will be the ultimate objective of the continuous quality improvement process of Institutional Effectiveness.
Examples of Assessment Results and Their Use
Emirati freshman enrollment increased by 5% and exceeded the performance target of an increase of 2% over the previous year. The target increase for next year will be set at 6%.
Student surveys indicate that their ability to find an available computer to use is 84%. With the completion of an additional computer lab next year, we anticipate that student satisfaction levels will increase to the target of 85 percent.
Order Now