Academic Program Review Procedures

ECU Faculty Manual, Part VI.VII. Curriculum Procedures, Academic Program Development, and the Academic Calendar

I. Introduction and Purpose of Academic Program Review

The ECU Academic Program Review Procedures are and must remain consistent with The Code and UNC Policy Manual 400.1 Policy on Academic Program Planning (specifically, Section V. Review and Evaluation of Existing Academic Programs), The Code and UNC Policy Manual 400.1.1[R] Regulation for Academic Program Planning and Evaluation, ECU University Policy Manual POL02.07.05 Unit Academic Program Review, and ECU Faculty Manual, Part VI, Section VII. Curriculum Procedures, Academic Program Development, and the Academic Calendar.

The purpose of the seven-year Academic Program Review (APR) of all undergraduate and graduate programs is to engage program faculty in a reflective evaluation of program quality and alignment to the pedagogical standards within their discipline as well as ECU’s mission and vision priorities. Program review is an integral part of the university’s ongoing assessment and strategic planning processes, designed to enhance and maintain the quality and rigor of all educational programs. Programs that are formally reviewed by an external accrediting body are not included in the APR process described in these guidelines; however they do respond to quality, efficiency, and productivity measures as defined by Section 400.1. of the UNC Policy Manual. Self-study reports are reviewed and archived by the Institutional Assessment (IA) staff and the institutional accreditation liaison in Institutional Planning Assessment, and Research (IPAR). Programs housed in the same department (or in some cases the same school or college) are described in a single written self-study report.

The comprehensive review of programs, certificates, and concentrations without external accreditation is intended to help faculty and administrators gain a better understanding of the:

• Purpose and outcomes for each degree program, certificate, and concentration associated with a program being reviewed;
• Each program, certificate, or concentration’s effectiveness in achieving its purpose and outcomes, along with overall program quality;
• The faculty’s vision for the program(s) and potential improvements that can be made based on the results of institutional and assessment data; and
• Future programmatic improvements to the recruitment and advancement of students, curriculum, pedagogy, and/or operational functions of the program.

ECU’s APR process consists of a program self-study, a virtual (or in rare instances an on-site) peer review, and action plan development that occurs every seven years, informed in part by the annual student learning outcomes assessment. A program progress report typically occurs at Year Three in the seven-year cycle. An APR is not to be considered a departmental, school, or college review. The focus of an APR is on the specific program(s) being reviewed. Therefore, only the data about the faculty who are closely affiliated with the program should be included in a program’s APR.

II. Academic Program Review Process

The APR process is conducted in three primary phases:

1. An internal self-study of the program(s) by faculty
2. Virtual (or in rare instances an on-site) review conducted by an External Review Committee (ERC), and
3. A final action plan (FAP) produced by faculty and supported by the relevant dean and provost.

The major steps in planning and conducting a formal review are outlined below:

1. Orientation to Academic Program Review
   1. One year prior to the review, the unit administrator (i.e., the department chair, school director, or other immediate administrator of the program) and program faculty are notified of the upcoming review, and an overview meeting is conducted.
   2. One semester prior to the scheduled APR, the unit administrator and program faculty attend an orientation led by the Director of Institutional Assessment (DIA) and/or the APR project manager to go through the review processes and resources.
   3. The unit administrator consults with faculty on three potential dates for the virtual on-site review and proposes names of external and internal reviewers.
   4. One external reviewer must be faculty from ECU’s official peer institutions who are familiar with the discipline; another external reviewer could either be from an official peer institution or from an institution with a similar program; internal reviewers are ECU faculty from a related campus-based discipline who are external to the program under review.
   5. The DIA and APR project manager, in consultation with the Internal Review Committee (IRC) (see 3 below), select two external and one internal reviewer and invites them to serve on the upcoming ERC. The DIA and project manager work with the unit administrator to develop the 2-3 day itinerary for the virtual review, which include meetings of the ERC with the unit administrators, program faculty, students, relevant university administrators, relevant university and community constituents, and the provost.
2. Program Self-Study
   The unit administrator and faculty prepare the self-study according to the APR Guidelines. It is important to have broad-based input from the program faculty. An electronic copy should be sent to the DIA and project manager for distribution to the IRC eight weeks before the on-site review.
3. Internal Review Committee
   The IRC reviews the self-study for accuracy, clarity, consistency, and completeness. The DIA chairs the IRC; members include dean of the home college or school of the program(s) under review, a representative of the Educational Policies and Planning Committee (EPPC) of the Faculty Senate, and the dean of the Graduate School if graduate programs are under review. A liaison to APR from Institutional Research (IR) also reviews the self-study for data accuracy.
4. Revision of Self-Study
   Program faculty revise the self-study based on input from the IRC. The project manager distributes the revised self-study and supporting documents to the ERC (one month prior to on-site review).
5. External Review Committee
   The ERC conducts its review of the undergraduate and graduate programs. A summary of major findings is presented to the program faculty, unit administrator, dean, and the provost on the second day of the review. Within 30 days of completion of the review, the ERC sends an electronic copy of the final Review Committee Report to the project manager, who then distributes to the unit administrator, the IRC, and the provost.
6. Program Response Report
   In a Program Response Report (PRR), the unit administrator and faculty respond to each of the recommendations in the ERC, describing actions they will/will not take to implement the recommendations, who is responsible for the actions, and when they will occur. Program faculty also prioritize the resource needs that emerge from the recommendations.
7. Review of Program Response Report
   Unit administrator and college/school administrators meet to review the PRR and discuss the program’s top priorities, needs that can be addressed by the college or school, and issues for discussion with the provost. After this meeting, the unit administrator and faculty revise the PRR to reflect actions to be taken by the program, unit administrator, college/school, and those needing institutional support.
8. Program Response Report to EPPC
   The PRR is sent to the EPPC representative for their review. Then the self-study, ERC report, and PRR are sent to the chair of EPPC and the review will be placed on an EPPC agenda. The unit administrator attends the EPPC meeting to answer any questions and hear the committee’s decision on whether the program response is approved or not. If the PRR is not approved, the EPPC chair will write a memo with concrete recommendations for improvement within ten days. The program response is to be edited and resubmitted to the chair of EPPC for the next committee meeting. Programs should consult the EPPC “Criteria for Reviewing Academic Program Reviews” document (located under EPPC “Committee Resources” on the Faculty Senate website) as the unit response is written.
9. Final Action Plan with Provost
   The provost leads a Final Action Plan meeting with the unit administrator and the Internal Review Committee. In this meeting, the unit administrator summarizes the program faculty’s responses and action plan; the college/school dean summarizes actions to be taken by the college/school; and the provost provides further recommendations on the actions planned. The DIA records major decisions made at the meeting, to include revisions made or new actions added to the Program Response Report. The provost distributes the major decisions in the form of a memorandum to the unit administrator, dean, and Internal Review Committee. All APR-related documentations are maintained by IPAR.
10. Ongoing Program Review and Enhancement
    The unit administrator and program faculty report on progress three years after the Final Action Plan meeting and summarize the status of the action plan. This progress report will be sent to the DIA, APR project manager, college dean, and the dean of the Graduate School if graduate programs are involved. As designated in the 7-year APR cycle, programs will gain complete a comprehensive periodic review. IA staff will review and monitor recommendations related to assessment, curriculum, and student learning. Faculty are encouraged to report on progress through their annual assessment reports.

III. Roles and Responsibilities

1. Program Faculty (including program director/coordinator) in conjunction with Unit Administrator
   1. Propose dates for the virtual visit, names of internal and external reviewers, and a list of participants for the meetings
   2. Collaborate in writing the self-study, analyzing data, and reflecting on the strengths and weaknesses of the program(s)
   3. Revise the self-study after feedback from the IRC and IR liaison is received
   4. Address each recommendation in the ERC report and develop PRR with an action plan
   5. Work with the unit administrator and dean to refine and finalize the action plan, implement the plan, and report progress 3 years after the Final Action Plan is approved
2. Unit Administrator (i.e., the chair of the home department, school director or the relevant administrator of the program)
   1. Coordinate the activities of the program faculty
   2. Assist with data collection and writing of the self-study
   3. Participate in relevant meetings with ERC
   4. Craft the PRR to the ERC report
   5. Participate in an EPPC meeting when the APR is on the agenda
   6. Participate in the Final Action Plan meeting
3. Dean of the College
   1. Serve on the IRC
   2. During on-site review:
      1. Participate in meetings with the ERC
      2. Participate in faculty/staff debriefing with ERC
      3. Participate in Exit Meeting with ERC and provost
   3. Lead meting of college/school and unit administrator to review PRR to identify actions to be taken at the college level
   4. Participate in the Final Action Plan meeting
4. Internal Review Committee
   1. Includes the following people:
      1. DIA (chair) and APR project manager
      2. Dean of the college that houses the program under review
      3. Dean of the graduate school if graduate programs are under review
      4. EPPC representative
   2. Select the ERC members
   3. Review the self-study and appendices for accuracy, clarity, consistency, and completeness
   4. Participate in Exit meeting with the ERC
   5. Participate in the Final Action Plan meeting with the unit administrator and the provost to finalize action plans and resource priorities
5. External Review Committee
   1. Review the self-study prior to review
   2. Meet with program faculty, staff, students, and other constituencies
   3. Lead the Exit Meetings with faculty, staff and administrators and share preliminary strengths, areas for improvement, and recommendations
   4. Prepare a written report within 30 days of the review which is then shared with the college/school, program faculty, graduate school (if applicable), and division administrators
6. The Provost
   1. Meet with ERC on the final day of their virtual visit to review major findings
   2. Lead the Final Action Plan meeting that includes the unit administrator and Internal Review Committee
7. The Educational Policies and Planning Committee Representative
   1. Serve on the IRC
   2. Provide EPPC with a timely update regarding the quality of self-study and major recommendations found in the ERC Report
   3. Provide feedback to the program administrator and dean on the PRR
   4. Report the final EPPC recommendation decision to the APR project manager
      Note: if the Program Response is not accepted by EPPC, the Chair of EPPC will provide concrete recommendations for improvement to the unit administrator within 10 days.
8. Institutional Research
   1. Maintain the Business Intelligence App as the data source for APR
   2. Meet with program faculty to review data and resources during the orientation meeting
   3. Serve on the IRC in order to review the self-study data for accuracy
9. Director, Institutional Assessment
   1. Chair the IRC
   2. Attend overview and orientation meetings
   3. Conduct welcome meetings with ERC
   4. Attend the exit meeting
   5. Provide an annual update to the provost and Executive Council on all APRs and specialized accreditation visits
   6. Record the Final Action Plan and monitor the three-year progress reports
10. APR Project Manager
    1. Coordinate the review process, establish the review schedule, and facilitate all logistical arrangements
       1. Facilitate overview and orientation meetings
       2. Liaise among the unit administrator and Institutional Research liaison to APR as needed for development of the self-study
       3. Liaise among the unit administrator and the Internal Review Committee members for the review, revision, and distribution of the APR self-study
       4. Collaborate with the unit administrator and the APR executive assistant regarding planning of the virtual visit
       5. Facilitate ERC nomination and selection process in collaboration with the unit administrator and the Internal Review Committee members
    2. Receive and distribute all documents
       1. Facilitate distribution of all documents to the unit administrator, Internal Review Committee and their respective administrative support personnel, External Review Committee, and the provost in preparation for the virtual visit
       2. Distribute the ERC report to the unit administrator upon receipt by the ERC
       3. Facilitate distribution of the response report submitted by the unit administrator to the relevant college dean and EPPC representative/chair
    3. Provide a repository for self-studies, external review reports, program response plans, final action plan memoranda, and progress reports

IV. Components of the Self Study (Limit to 50 pages, excluding appendixes)

Executive summary (3-5 pages): Based on the information presented in the self-study, prepare an executive summary describing:

1. the overall quality of each program that is included in the review and the indicators used to assess the quality;
2. strengths and weaknesses of the program (e.g., How effectively do faculty contribute to teaching and student mentoring, scholarship and creative activity, service mission, and clinical activities of the program? What are the demographics of faculty, students, and staff? Does curricular content represent a variety of cultural and other diverse perspectives as evidenced by curricular content and/or the authors of texts and other curricular resources? How effective are the support staff?);
3. major findings that resulted from the self-study; and
4. significant actions or changes that have been planned as a result of the self-study.

1. Program Purpose
   For each program included in the review:
   • 1.1 Provide a clear and concise statement of the program’s purpose;
   • 1.2 Describe how the program’s purpose aligns to its unit’s and the University’s mission (Student Success, Public Service, and Regional Transformation) and vision priorities (Social and Economic Mobility, Workforce Success, and Rural Health and Well-Being);
   • 1.3 Articulate any specific or unique features of the program(s); and
   • 1.4 Evaluate the current and projected student demand, as measured by enrollments in the majors and degrees produced;
   • 1.5 Evaluate the current and projected workforce demand, as measured by projected job growth and existing data on student employment outcomes;
   • 1.6 Evaluate student outcomes, including persistence, graduation, time to degree, and where possible, post-graduation success;
   • 1.7 Evaluate program costs and productivity, including research, scholarship, and creative activity and student credit hours produced compared to the number and cost of faculty and staff;
   • 1.8 Evaluate the contribution of the program to professions that are critical to the health, educational attainment, and quality of life of North Carolinians; and
   • 1.9 Evaluate any other considerations identified by the chancellor or by the UNC System President
2. Enrollment Degrees and Student Success
   IPAR has provided a Business Intelligence App with information for each degree/certificate program on the Academic Program Inventory. Review the data, collect additional data/information, and respond to the following questions for each program.
   Note: Programs may need to collect additional data on job placement. Licensure exam pass rates can be found.
   • Enrollment and Degrees Analysis
     • 2.1 Describe the program’s enrollment trend over the last seven years to include:
       • headcount enrollment (FT/PT ratio),
       • student demographics
       • characteristics of graduate students (number of complete applications, selectivity, and yield rates; demographics; undergraduate GPAs, admission test scores; and
       • characteristics of undergraduate students (number of complete applications, selectivity, and yield rates; demographics; high school GPAs, SAT/ACT scores, undergraduate GPAs);
     • 2.2 For undergraduate programs, describe the trend regarding the number of degrees awarded each year and time to degree. What actions have been taken to improve degree completion and time to degree?
     • 2.3 For graduate programs, describe the trend regarding the number of degrees awarded each year and time-to-degree of the students. What actions have been taken to improve degree completion and time-to-degree?; and
     • 2.4 Describe actions taken that implement the University’s/College’s strategic plan priority regarding enrollment management such as program expansion or contraction.
       Note: For certificate programs degrees awarded, rather than enrollment, may be more accurate and can be used for this section.
   • Student Success
     • 2.5 What is the 3-year trend regarding D/F/W rates in courses? Where appropriate, how do the D/F/W rates in face-to-face courses compare to those in online courses? What has the program done to address the courses with high D/F/W rates?
     • 2.6 What are the job placements or higher education enrollment of recent program graduates? Does it meet faculty expectations?
       Note: for some programs many students are currently employed so discuss their employment status.
     • 2.7 If applicable, what is the licensure pass rate of the graduates? Does it meet faculty expectations?
     • 2.8 What actions has the program taken over the past seven years to improve student success?
   • Action Plans
     • 2.9 What actions does the program plan to take in the next seven years to increase enrollment and student success? What resources are needed to implement these plans?
3. Curriculum Learning Outcomes and Student Satisfaction:
   Provide an interpretation of assessment findings and other relevant data about the curriculum and quality of student learning in each program being reviewed. Focus on interpretation of data, use of results, and program improvements.
   • Curriculum Analysis
     To support this section, a link to the degree requirements as published in the Catalog should be provided. Also include in an Appendix an updated curriculum map from Nuventive Improve that illustrates the alignment of student learning outcomes to courses in the curriculum.
     • 3.1 Based on degree requirements and the updated curriculum map, describe how course sequences, including prerequisites, are used to introduce and reinforce student learning prior to students being assessed.
     • 3.2 Describe the process the program uses to ensure the curriculum is up-to-date. Describe any innovated approaches in the curriculum, including innovations in equitable teaching and assessment.
   • Student Learning Outcomes Assessment
     To support this section, review program assessment reports from Nuventive Improve as well as other relevant data obtained since the last program review.
     • 3.3 Based on learning outcomes assessment reports/data, what are the identified strengths and weaknesses in student learning outcomes? Does curricular content align to assist graduates to engage a global society?
     • 3.4 Where applicable, are there any significant differences in student outcomes in face-to-face and online programs?
     • 3.5 What decisions have been made and what changes have been instituted on the basis of on-going assessments (e.g., curricular or pedagogical changes, faculty, instructional facilities, student support, funding priorities, the assessment procedure — including student learning outcomes and methods of gathering and analyzing data, etc.)?
     • 3.6 How effective were the changes?
   • Student Satisfaction
     To support this section, review the student survey data such as the Graduating Senior Survey, Graduate Student Exit Survey, and program-level employer/alumni surveys.
     Note: The Graduate Student Exit Survey is not administered to certificate students so the program can use their own data or consider this section as optional for certificate reviews.
     • 3.7 How satisfied are graduating students with the program? Are there significant variations in student satisfaction by student demographics, geographic region, or delivery method?
     • 3.8 How do graduating students and program alumni evaluate the knowledge and skills they have acquired in the program?
     • 3.9 How do employers evaluate the graduates’ knowledge and skills?
     • 3.10 What actions has the program taken to improve student support, services, and satisfaction?
   • Action Plans
     • 3.11 Are there new curricular and pedagogical changes that the program plans to implement in the next seven years to improve student learning?
     • 3.12 What will the program do to improve students’ educational experience and overall satisfaction?
     • 3.13 What resources are needed to implement these plans?
4. Strength of Faculty: Teaching, Research and Scholarship
   To support this section, include faculty bio sketches in an Appendix (1-2 pages per faculty).
   • Faculty Resources
     Review program faculty data provided by IPAR and respond to the following:
     • 4.1 Faculty Profile: Describe the current faculty affiliated with the program (e.g., percent full- versus part-time, demographics, percent with terminal degree, tenure status, etc.)
     • 4.2 Faculty Resources: Does the program have the number and type of faculty to achieve its goals?
     • 4.3 What actions has the program administrator taken to recruit, retain, and advance highly qualified faculty?
   • Analysis of Teaching Productivity
     • 4.4 Describe the trend in student credit hour production in the department over the past seven years, for both Distance Education and campus courses, highlighting the program’s contribution to the General Education program and other degree programs. Considering your department’s instructional faculty FTE, what is the average credit hour production?
     • 4.5 Based on the most recent data (external and/or internal data), what is the general teaching load of the program faculty? Is the teaching load equitably distributed among faculty? What has the program administrator done to adjust teaching load for faculty members of this program?
     • 4.6 Describe the direct contributions (course sections taught) and indirect contributions (grading, tutoring, etc.) of graduate teaching assistants to the program’s teaching mission?
     • 4.7 What are the major achievements of program faculty regarding teaching? What has the program administrator done to support faculty teaching?
   • Analysis of Research, Scholarship and Creative Activities
     Note: The Graduate School will provide links to graduate program theses and dissertations to reviewers.
     • 4.8 What are the major achievements of the program faculty and students regarding research, scholarship and creative activities?
     • 4.9 Describe the strengths and weaknesses regarding research, scholarship, and creative activities.
     • 4.10 what has the unit administrator done to support the research, scholarship and creative activities of program faculty and students?
   • Analysis of Service and Outreach activities
     • 4.11 What major service and outreach initiatives have program faculty and students engaged in? What has the program administrator done to support program faculty and student service/outreach activities?
   • Action Plans:
     • 4.12 What does the unit administrator plan to do to support the teaching, research, and service activities of program faculty and students? What resources are needed to implement these plans?
5. Regional Transformation – Economic Development/Public Service
   • 5.1 As applicable provide a summary of major activities the program faculty and students have participated in to support regional transformation over the last seven years.
   • 5.2 As applicable, what does the program plan to do to support regional transformation? What resources will it need to implement these plans?
6. Resources
   • 6.1 Based on analysis of the unit’s operating budget and revenue sources supporting the academic program as well as annual expenditures, discuss the adequacy of the resources provided and required for maintaining program quality.
   • 6.2 Describe the quality, scope, and projected needs for space to support the program.
7. Other Operational or Programmatic Outcomes
   • 7.1 Describe other assessed outcomes that enable the program to achieve its objectives, e.g., academic advising, number of faculty, graduate student support, operational efficiency, structural re-organization, etc. Summarize strengths and weaknesses identified in the assessment and actions taken to improve these outcomes.
   • 7.2 Action Plans: What does the program plan to do to improve these outcomes? What resources are needed to implement these plans?

Signature Page

External Review Committee Report on the [Program Name(s) of the [Department Name]

Prepared for the
Office of Institutional Planning, Assessment and Research
Division of Academic Affairs
East Carolina University

By

_____________________________ _________________________________
[External Review Name] [External Reviewer Name]
[External Reviewer Institution] [External Reviewer Institution]

__________________________________________
[Internal Reviewer Name], East Carolina University

V. Selecting the External Review Team

An important task is for the program to develop a list of eight potential reviewers from ECU peer institutions, three from national peer institutions (optional) and three internal reviewers. These external reviewers are to be nominated from institutions identified as official peers of East Carolina University and should be professionally prominent individuals, usually nationally recognized in their discipline or field. The potential internal reviewers are ECU faculty from a related campus-based discipline or field. Faculty rank, area of expertise, and leadership experience should be considered as the program seeks potential reviewers. The DIA can assist in identifying internal reviewers. The program should forward the list of potential reviewers to the Director of Institutional Assessment, and then the APR project manager will contact each reviewer to ascertain availability and interest in serving as an academic program reviewer.

The list of potential reviewers is submitted to the APR project manager containing the following information:

• Name of reviewer
• Name of university
• Complete job title/rank and name of a reviewer’s program
• Primary area of scholarly activity (related to the program being reviewed)
• Rationale for selection
• Contact information (full mailing address, e-mail, and telephone number)

Nominees from the list provided by the program will be discussed by the IRC and the official team members will be selected according to the following criteria:

• Two reviewers external to East Carolina University with preference being at least one serving at an ECU official peer institution;
• One internal reviewer from a related campus-based discipline outside of the program;
• External reviewers must be part of a program that is recognized for excellence in the discipline and able to benchmark the programs based on discipline-specific rankings and other publicly available comparisons;
• ERC is a diverse group with experience in both undergraduate and graduate programs as well as with the appropriate teaching, research and service components of the discipline; and
• Reviewers must affirm that there exists no conflict of interest related to the program under review.

VI. Charges to the External Team

The purpose of Academic Program Review (APR) at East Carolina University is to engage faculty in a reflective process of thoughtful study and evaluation of program quality and alignment to East Carolina University’s value, mission, and commitments in support of our students and the region. APR is an integral part of the university’s on-going assessment and strategic planning processes designed to enhance the quality of all educational programs and we sincerely thank you for assisting us. This letter provides you with the charge to the external review team.

External Review Committee Charge
Please make an objective evaluation of the program’s efficiency and effectiveness in achieving its programs’ purpose, and make recommendations that will help in planning improvements. Your resources are the APR Guidelines, a self-study report prepared by the program, copies of the Final Action Plan and Progress Reports from the previous review (if applicable), information you gain through interactions while onsite at ECU, and any additional information requested by you.

Within the broad charge of recommending ways that the program can continue to improve, here are some overarching questions that we would like you to address:

• Based on the information/data provided in the self-study or gathered by the external review committee, what are the program’s overall strengths and weaknesses?
• What major improvements (including student learning and faculty development) has the program made since the previous program review or within the last seven years?
• What is the professional benchmark and how does this program compare?
• What specific recommendations could improve the program’s performance?
• In addition, you may be asked to focus on program-specific questions during your on-site review of the program.

We look forward to meeting you during your time on campus. If you have any questions or require additional information prior to your visit, contact the DIA or APR project manager or the executive assistant to IPAR.

VII. ECU Peer Institutions

Approved by the UNC-BoG (https://ipar.ecu.edu/research/rsrch-peer-institutions/)

Faculty Senate Resolution #17-39, May 22, 2017
Faculty Senate Resolution #21-02, February 2021
Faculty Senate Resolution #24-66, April 2025