CAEP Annual Reporting Measures

The information on this page presents the annual reporting measures required by the Council for Accreditation of Education Preparation (CAEP).

CAEP Measure 1: Completer Effectiveness and Impact on P-12 Learning and Development (R4.1)

UMF’s College of Education utilizes two methods to investigate what impacts program completers have on P-12 student learning and development and gauge completer satisfaction with the effectiveness of their preparation. Because the Maine does not provide data related to P-12 student learning outcomes, the Teacher Education Unit (TEU) utilizes a case study model to examine cohorts of recent graduates across all programs to investigate and ascertain their impact on student growth. A biennial Alumni Survey seeks feedback regarding program strengths and areas for improvement from recent program completers (1-3 years since graduation) and provide opportunities for respondents to discuss their impacts on student learning.

2022-23 Case Study

Our case study is designed to be aligned with work the teachers are already doing. A representative sample of completers who are in their first two years in the field are identified by UMF field supervisor faculty who then personally invite them to participate. UMF chose a case study model because it aligns with its goals of producing competent educators who are reflective, life-long learners. All participating program completers produced pre- and post-assessment data aligned with a unit of study. While it is unable to produce generalizable findings for our programs, the model does provide an opportunity to determine completer impact on student learning as a process, allowing for context, and creating opportunities for reflection on instructional practice and assessment through the act of observation and dialogue. Based on the provided assessment data, all completers participating in the case study demonstrated that their students made positive gains in understanding the learning objectives of their respective units of study.

Components of the Current Case Study Design

  • Evaluation of a unit of study determined by the participating teacher. The unit of study must include pre- and post-assessment measures.
  • A meeting with a UMF field supervisor to discuss the school and classroom context, and the instructional planning and assessment of student learning relative to the unit they are teaching.
  • Observation of a lesson by the field supervisor observation and completion of the TEU’s Essential Areas of Teaching (EAT) instrument, an EPP-designed observation tool used to assess aspects of planning, instruction, management, assessment, and technology.
  • A follow-up meeting with the field supervisor to discuss the lesson observation and EAT assessment. The dialogue addresses completer’s perceptions of student learning, how assessment was used in instruction, strategies for determining student needs, and any ongoing challenges related to assessment they may have.
  • Completion of the Panorama Student Survey, a valid and reliable instrument completed by students in the observed classroom to gain their perspective on the completer’s skill at instruction, establishing expectations, and developing relationships.

2022 Alumni Survey

Assessment Description: UMF consistently seeks input from recent completers regarding their satisfaction with both the effectiveness and relevance of their preparation, how that preparation has impacted their current career, and their educational goals and plans. The alumni survey is sent out to recent program completers (one to three years since graduation) biennially in a digital format. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect feedback about new initiatives and community needs. Alumni responses are anonymous and all questions are optional, providing alumni the opportunity to opt out of any questions they do not feel comfortable answering. Questions identifying the alumni’s program, license, and graduation year allow for disaggregation and comparison. Likert-style scaled questions are identical to questions included in both the employer survey and the end-of-program survey which is completed by student teachers, and evaluates individual indicators related to different aspects of preparation. Short-answer questions inquire about program strengths, weaknesses, and feedback regarding additional preparation needs.

  • More than 91% of respondents (n = 105) are currently employed in the field, with two-thirds in both their preferred career path (i.e. teacher) and desired position (i.e. age group/grade level, content area, location, etc.).
  • 78% of respondents were hired within 4 months of graduation, and nearly 87% had been hired within a year of graduation.
  • In open response questions, alumni highlighted field experiences, faculty and staff, and skills regarding lesson planning and instructional methods aligned with state teaching standards as strengths of their preparation at UMF.
  • UMF seeks to graduate caring teachers, competent educators, and collaborative professional leaders. In this regard, alumni felt TEU’s teacher preparation programs were quite successful with 88% of graduates indicating that UMF was Very Effective or Effective in preparing them to be caring, competent, and collaborative professionals. Areas of strength include:
    • building respectful relationships;
    • supporting and encouraging successful learning in their classrooms;
    • honoring and responding to student differences;
    • effective communication skills;
    • collaborating with colleagues;
    • and, demonstrating a commitment to the ethical and legal responsibilities of the profession.
  • A comment from an alum of the Elementary Education program sums up well the overall sentiment conveyed by respondents to the open response questions:

“After leaving UMF, I left feeling confident and ready to enter a school
system and begin teaching. UMF prepared me with incredible classroom
management skills, communication [and] collaboration skills with
colleagues, students, and families, and content knowledge before
graduating. I felt confident in creating student centered lessons and
activities that would meet the needs of each of my future students as
individuals.”

 

CAEP Measure 2: Satisfaction of Employers and Stakeholder Involvement (R4.2, R5.3)

Stakeholder Involvement

The University of Maine at Farmington’s Teacher Education Unit (TEU) regularly engages with practicing educators and school administrators through a variety of venues to ascertain local and state areas of concern as well as emerging hiring and professional development needs. UMF’s Office of Graduate and Continuing Education’s strong Outreach Office engages in ongoing communication with regional school districts and other stakeholders throughout the state to develop relevant and timely course content and degree programs. The TEU’s Field Services Director regularly attends regional superintendent association meetings to remain abreast of local and state needs. Field Services also collects survey data from mentor teachers every semester to improve and strengthen clinical experiences for both the students and the cooperating schools. An Education Advisory Council facilitated by the Dean of the College of Education and populated with TEU faculty and staff, teachers across various grade levels and content areas, and regional administrators gathers each semester to discuss ongoing concerns and trends within their schools and districts to map out potential areas for collaboration and support. Additionally, the TEU and its faculty regularly apply for and participate in grants that allow for collaboration with school partners to address recruitment, mentoring, and retention of new teachers. Recent examples include a grant to recruit and support adult learners interested in becoming educators and another to study the teacher shortage in rural Maine school districts to determine the factors impacting hiring and retention.

 

Satisfaction of Employers – 2022 Employer Survey

A biennial survey of employers who have hired recent TEU graduates provides additional opportunities for the College of Education to connect with districts across the state. Findings are summarized and shared with TEU administration and faculty to determine the strengths of its preparation programs and clarify potential areas for improvement.

Assessment Description: The Employer Survey is sent every two years to employers of recent alumni (one to three years since graduation) requesting feedback about their satisfaction with both the effectiveness and relevance of the preparation their new hires have received from UMF. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect employer feedback about new initiatives and community needs. Employer responses are anonymous and are not linked to specific alumni; accordingly, data from the survey cannot be directly linked to or disaggregated by a specific TEU program or licensure group. However, the data can be disaggregated by grade level: Early (Infants and toddlers, PK, and K); Elementary (Grades 1 to 5); Middle (Grades 6 to 8); and Secondary (Grades 9 to 12). The first section of the survey includes Likert-style scaled questions that are identical to questions included in both the Alumni Survey and the end-of-program survey. These questions evaluate individual indicators related to different aspects of educator preparation. The second section includes open response questions about perceived program strengths, areas for improvement, and identification of additional preparation needs.

The Employer Survey summarizes unit-wide findings but is also disaggregated by grade level: Early, Elementary, Middle, and Secondary grade levels. Some unit-wide findings include:

  • Just over 90% of employers were Very Satisfied or Satisfied with UMF graduates’ abilities to carry out the assigned responsibilities of their grade level.
  • More than 96% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be caring teachers, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • create positive classroom communities
    • support and encourage successful learning
    • honor and respond to learner differences
    • utilize knowledge of human development
  • More than 86% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be competent educators, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • assess students in order to monitor proficiency and/or growth
    • effectively use instructional technology and facilitate student use of available technology.
  • More than 90% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be collaborative professional leaders, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
    • collaborate effectively with colleagues
    • demonstrate a commitment to ethical and legal responsibilities of the profession
  • Finally open response questions highlighted that recent completers are strong in content knowledge, lesson planning, technology skills, inclusive practices, collaboration skills, and their overall preparedness.

 

CAEP Measure 3: Candidate Competency at Program Completion (R3.3)

Program Benchmark and Completion Requirements

Education is a demanding profession. The TEU is committed to ensuring that UMF graduates are of the highest caliber, ready to teach, and meet state certification requirements. Established benchmarks are designed to monitor student progress through their programs and ensure that students are meeting Maine’s certification requirements at the time of graduation.

Candidacy: Upon completion of their initial practicum experience, a student must apply for and attain formal candidacy in their programs. Only students who are approved for candidacy may continue their programs, including enrollment in upper level courses and advance practicum field experiences. Requirements for candidacy are outlined below:

  • Cumulative GPA of 2.75 or better.
  • Grades of B- or better in all professional education courses. (A student may request one exemption; practicum courses are not eligible for exemption.)
  • A grade of C or better in English 100 or an equivalent transferred English course with a grade of C or higher.
  • Successful completion of practicum/advanced practicum with a grade of B- or better.
  • Satisfactory assessments of candidate habits and ways of being utilizing the Teacher Candidate Dispositions and Professional Expectations assessment in both a classroom and field setting. Assessments are completed by faculty, field supervisors, and mentor teachers. Candidates complete self-evaluations in their field experiences.

Program Completion: Teacher candidates seeking a degree leading to certification are required to:

  • Successfully complete all clinical experiences, including earning a B- or better in practicum and advanced practicum experiences and a 16-week student teaching/internship placement.
  • Submit and present a Student Teaching/Internship Professional Portfolio. Portfolios are cross-rated by at least two faculty evaluators and require pre-service teachers to provide evidence demonstrating proficiency with Maine’s CCTS standards. (CCTS Standards Rationale and Artifact)
  • Submit evidence of proficiency with skills related to lesson and unit planning, instruction, assessment, and use of technology. (Teacher Work Sample)
  • Submit an analysis of the context of their school placement. (Contextual Factors Analysis)
  • Satisfactory assessment of professional dispositions. (Teacher Candidate Dispositions and Professional Expectations)
  • Satisfactory assessment of skills related to proficiency in planning, instruction, classroom management, assessment, and use of technology. (Essential Areas of Teaching; Classroom Management Observation Checklist)
  • A cumulative GPA of 2.75 or better and course grades of B- or higher in all professional education courses.

Assessment

UMF’s College of Education utilizes several assessment tools to ensure its teacher candidates possess the competencies they will need to teach effectively and to positively impact student learning and development.

2022-23 Essential Areas of Teaching Summary

Assessment Description: The Essential Areas of Teaching is a unit-wide assessment conducted by mentor teachers and field supervisors during selected field experiences (Advanced Practicum and Student Teaching). The assessment is conducted at mid-term and end-of-term. The Essential Areas instrument assesses growth over the course of semester-long field experiences on indicators within five key categories: Planning, Instruction, Management, Assessment, and Technology.

For summary purposes, a strength is identified as an indicator with more than 95% of student teachers rated as Meets Expectations by both their field supervisor and their mentor teacher by the end of the placement.

Strengths of TEU student teachers include:

Planning

  • Well-planned for each lesson
  • Including all elements of the lesson plan
  • Addressing modification for individual needs of students

Instruction

  • Using a variety of approaches and strategies
  • Demonstrating subject matter knowledge and building on prior knowledge
  • Providing appropriate feedback
  • Involving all students in lessons
  • Modeling what needs to be learned

Management

  • Demonstrating personal regard for each student

Technology

  • Facilitating and inspiring student learning and creativity
  • Demonstrating ethical and legal use of technology
  • Integrating students use of available technology into instruction

 

2022-23 CCTS Standards Rationale and Artifacts Summary

Assessment Description: The purpose of the CCTS Rationales and Artifacts assessment is to ensure student familiarity and proficiency with the Common Core Teacher Standards (CCTS) and the International Society for Technology in Education (ISTE) standards. Candidates select artifacts that demonstrate proficiency with at least one specific indicator for a given standard and articulate how the artifact serves as evidence of proficiency. The assessment has two rubrics – the CCTS Standards Rationales and Artifacts rubric and the Progress on Standard rubric.

The Rationales and Artifacts rubric is structured into three sections: Introduction of the Standard and Indicator, Introduction to the Artifact, and the Rationale. Candidates are evaluated on a 4-point Likert scale (Proficient, Developing, Beginning, No Evidence) on their ability to (1) communicate the meaning of the standard, (2) align evidence with indicators, (3) justify and articulate the relationship between evidence and proficiency with the standard, (4) clearly communicate through writing, and (5) demonstrate proficiency through evidence.

The Progress on Standard rubric provides an overall assessment of the candidate’s proficiency with the standard and identified indicator based on the outcome of the Rationale assessment and the quality of the artifact. The same Likert scale is utilized.

Course-Based Assessment: Candidates have several opportunities to complete the CCTS Rationales and Artifacts assessment as part of their required coursework in their respective programs. By the time candidates reach their culminating student teaching/internship experiences, they will have addressed at least one standard and indicator from each of the four CCTS/InTASC categories (The Learner & Learning, Content, Instructional Practice, Professional Responsibility) and one or more of the seven ISTE technology standards.

Student Teaching/Internship Professional Portfolio: The capstone project of the student teaching/internship clinical experience contains several assessment instruments, one of which being the Rationale and Artifact assessment. Each teacher candidate provides two Rationales per InTASC/CCTS standard and one Rationale each for two of the ISTE standards. Students may use the same artifact for multiple standards, but each rationale must be unique to the standard and indicator. Portfolios are cross-rated by two field supervisors and signed off on by the Director of Field Services. A successfully completed Portfolio is evidence that teacher candidates have proficiency with the CCTS standards and is required for degree completion.

2022-23 Teacher Work Sample and Contextual Factors Analysis Summary

Assessment Description: The Teacher Work Sample (TWS) and Contextual Factors Analysis (CFA) is a unit-analysis project designed and vetted by the Renaissance Project. This project is a performance-based assessment tool for evaluating student teaching field experiences and evaluates proficiency at unit planning, implementation, assessment, and instructional decision-making. Field supervisors utilize rubrics closely aligned with the CCTS/InTASC Standards. The TWS rubric contains eight categories encompassing 28 indicators while the CFA rubric consists of nine indicators. Summaries are disaggregate by program.

 

2022-23 Teacher Candidate Dispositions & Professional Expectations Summary

Assessment Description: The TEU assesses teacher candidate dispositions multiple times throughout a student’s program, both in the college classroom and in the field. The purpose of the dispositions assessment is to provide early and frequent feedback to students regarding the professional expectations that are embedded in the education field. Two versions of the assessment exist. A classroom version is utilized in methods and non-field-based courses and is completed by course faculty. A field version is utilized in practicum, advanced practicum, internships, and student teaching placements and is completed by both mentor teachers and field supervisors. Students in practicum and advanced practicum placements also complete self-evaluations using the instrument. Goals of the assessment include helping students understand the professional expectations of the field, providing actionable feedback to students regarding those expectations, serving as a trigger to provide support to struggling students, and adding a level of selectivity to ensure teacher candidate quality. Satisfactory outcomes on the assessment are required for both program candidacy and successful completion of student teaching.

 

2022-23 Classroom Management Observation Checklist Summary

Assessment Description: The Classroom Management Observation Checklist (CMOC) is an EPP-developed observation instrument utilized during a teacher candidate’s student teaching placement. The tool includes indicators of both teacher and student actions in the school setting that are integral to an effective and safe classroom community. Indicators are intended to be actionable to provide useful feedback to candidates about strategies to improve their classroom management. The rubric consists of 23 indicators within three categories: Student Behavior Observed (referring to students in the student teachers’ classroom), Teacher Behavior Observed (referring to the student teachers themselves), and Misbehavior. A broad assortment of validated indicators are included which vary in complexity and difficulty in order to provide a comprehensive view of candidate proficiency. Summaries are disaggregated by program.

 

CAEP Measure 4: Ability of Completers to be Hired

Where UMF Graduates Teach

Becoming a graduate of a UMF Education program opens the door for employment around Maine and New England, across the United States, and even internationally. While the State of Maine does not collect data tracking teacher candidate hiring, findings from the TEU’s 2022 biennial survey of recent graduates found more than 91% of respondents are currently employed in the field, with two-thirds in both their preferred career path (i.e. teacher) and desired position (i.e. age group/grade level, content area, location, etc.). More than three quarters (78%) of respondents were hired within 4 months of graduation, and nearly 87% had been hired within a year of program completion.

Title II Reports

Title II is a federal policy tasked with assisting states in “preparing, training, and recruiting high quality teachers and principals” to improve educator and administrator quality. For more information about Title II, please visit their site at https://title2.ed.gov/Public/Home.aspx

UMF 2023 Title II Report