Three Stages of Academic Program Review

1. Self-Study

The Self-Study is an opportunity for program faculty to examine and analyze the educational effectiveness of the program through data and evidence.  The Self-Study should identify strengths and limitations and make recommendations to address identified problems, challenges and additional resources.

Model Table of Contents for the Self-Study Report

  1. Introduction/Context
    1. Internal and External Context
    2. History of the Program (with an emphasis on recent history)
    3. Mission Statement
  2. Disposition of Last Self-Study Prioritized Recommendations and Plans
  3. Evidence-Based Analysis of Program Quality
    1. Student Profile
    2. Curriculum and Learning Environment
    3. Student Learning, Assessment and Success
    4. Faculty
  4. Evidence-Based Analysis of Program Viability
    1. Demand for the Program
    2. Allocation of Resources and need for additional resources
  5. Summary & Plans
    1. Summary of Analysis
    2. Plans including multi-year student learning assessment plan
  6. Required Appendices
    1. Curriculum Map
    2. List of Faculty Research and Scholarship
    3. Program Strategic Plan Milestones Update
    4. Syllabi for previous two years
    5. Links to on-line archive–where appropriate–for Faculty Meeting Minutes; Annual Student Learning Assessment; Course Descriptions; Degree Requirements; Student Handbook, Application, Admission, Retention Statistics; Entrance Score and Grade Trends; Student Demographics; Student Exit, Alumni and Employer Survey Results; Capstones or other Student Work Portfolios; Research Center Reports

Upon completion, the Program Chair submits the Self-Study Report with Transmittal Memo (see templates online) to the Office of the Provost.

 

2. External Review Report

External Review

The External Review  reviews the Self-Study and all of the relevant documentation and prepares a report.  The goal of the External Review is to ensure objectivity of the program review process, determine how the program compares to other similarly situated programs, and identify commendations as well as make recommendations for improvement.

  1. Goals
    1. Review the Self-Study for accuracy and completeness
    2. Note any additional issues, recommendations, taking into account their special expertise (e.g., disciplinary, student learning assessment)
  2. Process
    1. Program Chair submits completed Self-Study to the Office of the Provost, who reviews it for structural completeness (not substantive adequacy, just verifying that all sections are filled in and all appendices are included). If there are omissions, the report goes back to the Program for completion and resubmission within 14 days.
    2. The Review Committee should include at least two internal and at least two external members.  The Review Committee Chair is generally a university faculty member experienced in AUA review processes, appointed by the Provost. .  The Dean, Program Chair and program faculty compile an annotated list of suggestions of external-to-the-university Reviewers with discipline-specific expertise.  The Provost selects from the list and invites the external-to-the-university reviewers.  (See External-to-the-University Instructions below.)  An annotated list should include a brief explanation of qualification for participation and any past affiliations with AUA.  If none of the names suggested are acceptable to the Provost, s/he provides a brief explanation why the candidates are not acceptable and tasks the Dean, Program Chair, and program faculty with expanding the list until a mutually acceptable external-to-the-university reviewers are selected.  The Provost may chose an external reviewer not from the list if a mutually acceptable name is not identified in a timely manner.  The university faculty members of the Review Committee are appointed by the Provost and typically one faculty member experienced in program review.  The qualifications for participation on the Review Committee include:
      1. Senior academic leadership and/or highest degree in the relevant discipline (for external-to-the-university reviewers),
      2. Experience in conducting academic program reviews (for external-to-the-university reviewers),
      3. No conflict of interest (This is not intended to be an exhaustive list of possible conflicts of interest but provides guidance in this regard)
        1. No direct reporting to the program chair or dean of the program being reviewed
        2. No recent (within the period being reviewed) connection to the unit under review (i.e. former students, instructors, chair, dean, researchers) or the executive leadership or board of the university
        3. No close personal (i.e. close friend, family member,)or professional (i.e. co-authorship of research, co-applicant for a grant proposal or other funding, either way direct reporting, either way dissertation supervision) ties with any of the faculty in the program being reviewed, executive leadership or board of the university
        4. When impossible to avoid a conflict, the program in review should disclose any potential conflicts of interest of suggested reviewers when submitting names
      4. Whenever possible, external-to-the-university reviewers should be from similarly situated institutions (i.e. size, experience)
    1. The Provost forwards the Self-Study to the Review Committee members who review the contents and identify any questions.
    2. Review Committee holds meeting/conference calls with IRO to review Guidelines, Self-Study, outline any questions related to the tasks, outline time frame in line with the timeline below (see page 9 of these Guidelines)
    3. Review committee meets with stakeholders (students, faculty, alumni, employers, program leadership) as needed to clarify issues that arose in the “document review.” External reviewers should have opportunities to meet separately with internal AUA constituencies (students, faculty, alumni, employers, etc.)
    4. The review committee maintains a record of all meetings related to the review.
    5. All discussions related to the program review must be conducted during meetings scheduled and documented in the review report.
    6. Review committee prepares Review Report (see outline below). The Review Report should reflect the opinions of all reviewers.
    7. Review Committee Chair delivers draft review report to the Program Chair, Dean, and Provost who review for completeness
      1. If there are omissions, the document is sent  back to the Review Committee (with copy to Dean, Program Chair, and Provost) for completion and resubmission within 14 days
      2. Program Chair circulates draft Review Report among current core faculty for comments and corrections and sends factual corrections within 14 days
      3. A meeting/Conference call may be scheduled by the Program Chair and Review Committee, if necessary, to review findings/factual clarifications  (Review Committee makes corrections at its discretion)
    1. Once complete, the Review Committee Chair submits the Final Review Report with Transmittal Letter to the Program Chair, Dean, and Provost.
    2. The provost  schedules the Wrap-Up Meeting (see Wrap Up Meeting description below)
    3. Review Progress Check: The Review Committee Chair is tasked with notifying the Provost if the timeline for completion is unexpectedly altered
    4. Review Process Check: At any point in the Review, the Program Chair, Dean, Review Committee Chair or Provost may request a process check to address questions or concerns about the review process.   In rare circumstances of material irregularities, the process may be suspended until the irregularities have been addressed.
  1. Outline of Review Report – The Review report is an audit of the Self-Study. It generally follows and reflects on the adequacy of the data and analysis presented in the Self-Study and is coupled with data and information gathered during the review process. The report is prepared by the Review Committee with the participation of all members including the external-to-the-university reviewer.   The external-to-the-university member may submit a separate report to be included in the record if s/he is unable to take part in the report drafting or has additional comments.    Recommendations, where appropriate, should be made at the end of each section.
  1. Introduction/Context

Is it complete and accurate?  Has the mission statement been reviewed and updated, if necessary.

  1. Disposition of Last Self-Study Prioritized Recommendations and Plans

Have all of the recommendations from the last Self-Study been thoroughly addressed?

  • Evidence-Based Analysis of the Program’s Quality

Is this section complete and accurate? Was appropriate data and analysis used?  Did the program draw appropriate conclusions from student profile data? Were student learning assessments properly and timely completed? Did the program close the loop on student learning assessment recommendations? Did stakeholder meetings confirm the Self-Study’s analysis?

  1. Evidence-Based Analysis of Program Viability

Is this section complete and accurate?   Has the program developed an appropriate recruitment plan based on trends, demographics, competitor analysis?   Are resources allocated effectively?

  1. Summary & Plans

Are strengths, weaknesses and opportunities appropriately and thoroughly identified with supporting evidence?  Did the program make appropriate recommendations for addressing weaknesses, building on strengths, responding to opportunities, and (re)allocating resources based on the evidence and analysis presented?  Does the Self-Study include at least an outline for a multi-year (minimum 2 years) student learning assessment plan?

  1. Review of Self-Study Appendices:   Are these complete and accurate?
  • Appendices to Review Report:  External-to-the-university reviewer CV; list of meetings (stakeholders, dates) conducted by the review committee during the review.

 

Additional Instructions for External-to-the-University Reviewer

  1. Review the Academic Program Self-Study Report (15-20 page document), paying special attention to the discipline-specific parts, i.e., curriculum, instruction, faculty qualifications, and plans (especially Sections III and V), as well as appropriate international trends and best practices.
  1. External-to-the-university reviewers should participate in the drafting of the Review Committee Report.   In the event they are unable to participate or  have additional comments,  external-to-the-university members submit a separate report to be included in the record. In the case of a site visit, the external-to-the-university draft report should be submitted before departure from Armenia.
  2. Provide CV.

 

3. Wrap-Up Meeting with Administration

    1. Wrap-Up Meeting with Administration – Discuss and Prioritize Recommendations
    2. Participants: President, Provost, Dean, Program Chair, Review Committee Chair.
    3. Goal: Prioritized List of Recommendations and Plans, jointly approved by Administration and Program.
    4. Preparation: Program Chair may draft a list of Prioritized Recommendations based on the Self-Study and Review Reports.  Participants should review the Self-Study and Review Reports and list of Prioritized Recommendations (if available) ahead of time and be prepared to discuss and agree upon recommendations on priorities and plans for the program for the coming 3-5 years.
    5. Outcome: Prioritized Recommendations and Multi-year Plan, including budgetary recommendations and Wrap-Up Memo, written by the Program Chair based on Wrap-Up Meeting, which is reviewed and finalized by the Dean and becomes the baseline for the next self-study.