CAEP Accountability Measures

CAEP Annual Reporting Measures

The information on this page presents the annual reporting measures required by the Council for Accreditation of Education Preparation (CAEP).

CAEP Measure 1: Completer Effectiveness and Impact on P-12 Learning and Development (R4.1)

UMF’s College of Education utilizes two methods to investigate what impacts program completers have on P-12 student learning and development and gauge completer satisfaction with the effectiveness of their preparation. Because the Maine does not provide data related to P-12 student learning outcomes, the Teacher Education Unit (TEU) utilizes a case study model to examine cohorts of recent graduates across all programs to investigate and ascertain their impact on student growth. A biennial Alumni Survey seeks feedback regarding program strengths and areas for improvement from recent program completers (1-3 years since graduation) and provide opportunities for respondents to discuss their impacts on student learning.

2023-24 Case Study

Assessment Description: In the absence of state data measuring student growth, UMF chose a case study model in large part because they align with its goal of producing competent educators who are reflective, life-long learners. A representative sample of completers who are in their first two years in the field are identified by UMF field supervisor faculty who then personally invite them to participate. All participating program completers produce pre- and post-assessment data aligned with a unit of study. While it is unable to produce generalizable findings for our programs, the model does provide an opportunity to determine completer impact on student learning as a process, allowing for context. It creates opportunities for reflection on instructional practice and assessment through the act of observation and dialogue. 

Components of the Current Case Study Design

• Evaluation of a unit of study determined by the participating teacher. The unit of study must include pre- and post-assessment measures.
• A meeting with a UMF field supervisor to discuss the school and classroom context, and the instructional planning and assessment of student learning relative to the unit they are teaching.
• Observation of a lesson by the field supervisor observation and completion of the TEU’s Essential Areas of Teaching (EAT) instrument, an EPP-designed observation tool used to assess aspects of planning, instruction, management, assessment, and technology.
• A follow-up meeting with the field supervisor to discuss the lesson observation and EAT assessment. The dialogue addresses completer’s perceptions of student learning, how assessment was used in instruction, strategies for determining student needs, and any ongoing challenges related to assessment they may have.

 

Five cases studies were completed in Fall 2023. All five cases involved recent Elementary Education completers. 

1. Participants apply skills acquired from their preparation.

• • Student-centered practice
  • Individualized and small group instruction
  • Structured guided practice
  • Clear lesson targets and learning objectives
  • Positive, constructive, and ongoing feedback
  • Established routines and procedures
  • Modeling what is to be learned

 

2. Participants employ several methods to learn their students’ needs.

• • Relationship-building, informal observations, and student interactions
  • Outreach and communication with professional staff, team members, and families
  • Regular use of formative assessment to identify learning gaps, strengths, and areas for development

 

3. Lesson observations noted a number of effective strategies to maintain and encourage student engagement.

• • Short “brain breaks” to help students stay focused and re-engage with lessons
  • Effective questioning
  • Interactive activities that utilize available technology
  • Breaking content in manageable chunks to ensure better understanding

 

2024 Alumni Survey

Assessment Description: UMF consistently seeks input from recent completers regarding their satisfaction with both the effectiveness and relevance of their preparation, how that preparation has impacted their current career, and their educational goals and plans. The alumni survey is sent out to recent program completers (one to three years since graduation) biennially in a digital format. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect feedback about new initiatives and community needs. Alumni responses are anonymous and all questions are optional, providing alumni the opportunity to opt out of any questions they do not feel comfortable answering. Questions identifying the alumni’s program, license, and graduation year allow for disaggregation and comparison. Likert-style scaled questions are identical to questions included in both the employer survey and the end-of-program survey which is completed by student teachers, and evaluates individual indicators related to different aspects of preparation. Short-answer questions inquire about program strengths, weaknesses, and feedback regarding additional preparation needs.

• Nearly 87% of respondents (n = 38) are currently employed in the field in both their preferred career path (i.e. teacher) and desired position (i.e. age group/grade level, content area, location, etc.).
• Approximately 89% of respondents were hired within 4 months of graduation, and 100% of respondents had been hired within 8 months of graduation.
• In open response questions, alumni highlighted the various field experiences, preparation to create positive and inclusive learning environments, development of skills to build relationships and collaborate effectively with colleagues, and skills related lesson planning and instructional methods aligned with state teaching standards as strengths of their preparation at UMF.
• UMF seeks to graduate caring teachers, competent educators, and collaborative professional leaders. In this regard, alumni felt TEU’s teacher preparation programs were quite successful, with nearly 70% of graduates indicating that UMF was Very Effective or Effective in preparing them to be caring, competent, and collaborative professionals. Areas of strength include:
  • building respectful relationships;
  • creating positive classroom communities;
  • supporting and encouraging successful learning in their classrooms;
  • using best practices for instruction;
  • designing and implementing instruction linked to appropriate standards;
  • communicating clearly and effectively;
  • practicing reflective, lifelong learning;
  • and, demonstrating a commitment to the ethical and legal responsibilities of the profession.
• A comment from an alum of the Elementary Education program sums up well the overall sentiment conveyed by respondents to the open response questions:

“After leaving UMF, I left feeling confident and ready to enter a school
system and begin teaching. UMF prepared me with incredible classroom
management skills, communication [and] collaboration skills with
colleagues, students, and families, and content knowledge before
graduating. I felt confident in creating student centered lessons and
activities that would meet the needs of each of my future students as
individuals.”

 

CAEP Measure 2: Satisfaction of Employers and Stakeholder Involvement (R4.2, R5.3)

Stakeholder Involvement

The University of Maine at Farmington’s Teacher Education Unit (TEU) regularly engages with practicing educators and school administrators through a variety of venues to ascertain local and state areas of concern as well as emerging hiring and professional development needs. UMF’s Office of Graduate and Continuing Education’s strong Outreach Office engages in ongoing communication with regional school districts and other stakeholders throughout the state to develop relevant and timely course content and degree programs. The TEU’s Field Services Director regularly attends regional superintendent association meetings to remain abreast of local and state needs. An Education Advisory Council facilitated by the Dean of the College of Education and populated with TEU faculty and staff, teachers across various grade levels and content areas, and regional administrators gathers each semester to discuss ongoing concerns and trends within their schools and districts to map out potential areas for collaboration and support. Additionally, the TEU and its faculty regularly apply for and participate in grants that allow for collaboration with school partners to address recruitment, mentoring, and retention of new teachers. Recent examples include a grant to recruit and support adult learners interested in becoming educators and another to study the teacher shortage in rural Maine school districts to determine the factors impacting hiring and retention.

 

Satisfaction of Employers – 2024 Employer Survey

A biennial survey of employers who have hired recent TEU graduates provides additional opportunities for the College of Education to connect with districts across the state. Findings are summarized and shared with TEU administration and faculty to determine the strengths of its preparation programs and clarify potential areas for improvement.

Assessment Description: The Employer Survey is sent every two years to employers (principals and directors of special education services) of recent alumni (one to three years since graduation) requesting feedback about their satisfaction with both the effectiveness and relevance of the preparation their new hires have received from UMF. In order to compare trends over time, most questions are maintained from year-to-year, however, new questions are added when necessary in order to collect employer feedback about new initiatives and community needs. Employer responses are anonymous and are not linked to specific alumni; accordingly, data from the survey cannot be directly linked to or disaggregated by a specific TEU program or licensure group. However, data are disaggregated by grade levels aligned with state certification regulations: Early (Infants, toddlers, and PK); Elementary (K to Grade 6); and Secondary (Grades 7 to 12). The first section of the survey includes Likert-style scaled questions that are identical to questions included in both the Alumni Survey and the end-of-program survey. These questions evaluate individual indicators related to different aspects of educator preparation. The second section includes open response questions about perceived program strengths, areas for improvement, and identification of additional preparation needs.

The Employer Survey summarizes unit-wide findings but is also disaggregated by grade level: Early, Elementary, and Secondary grade levels.  Some unit-wide findings include:

• Just over 92% of employers (n = 49) were Very Satisfied or Satisfied with UMF graduates’ abilities to carry out the assigned responsibilities of their grade level.
• More than 94% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be caring teachers, and more than 90% were Very Satisfied or Satisfied with graduates’ abilities to:
  • build respectful relationships
  • create positive classroom communities
  • honor and respond to learner differences
• Approximately 84% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be competent educators, and more than 85% were Very Satisfied or Satisfied with graduates’ abilities to:
  • know discipline specific content
  • use best practices for instruction
  • effectively use instructional technology and facilitate student use of available technology.
• More than 87% of employers indicated they were Very Satisfied or Satisfied with UMF graduates’ overall preparation to be collaborative professional leaders, and more than 85% were Very Satisfied or Satisfied with graduates’ abilities to:
  • collaborate effectively with colleagues
  • demonstrate a commitment to ethical and legal responsibilities of the profession
• Open response questions highlighted that recent completers are strong in curriculum and instruction, professionalism and preparedness, differentiation and inclusion, and collaboration and relationship-building.

 

CAEP Measure 3: Candidate Competency at Program Completion (R3.3)

State certification regulations require that EPPs ensure “initial and in-service teachers… demonstrate proficiencies associated with” Maine’s Common Core Teaching Standards (CCTS). The CCTS encompasses both InTASC (CCTS 1-10) and ISTE technology standards (CCTS 11). When candidates complete their preparation program, the TEU is confident that not only have they met state requirements for licensure, they are prepared to be caring, competent, and collaborative professional educators and leaders.

Program Benchmark and Completion Requirements

Education is a demanding profession. The TEU is committed to ensuring that UMF graduates are of the highest caliber, ready to teach, and meet state certification requirements. Established benchmarks are designed to monitor student progress through their programs and ensure that students are meeting Maine’s certification requirements at the time of graduation.

Candidacy: Upon completion of their initial practicum experience, a student must apply for and attain formal candidacy in their programs. Only students who are approved for candidacy may continue their programs, including enrollment in upper level courses and advance practicum field experiences. Requirements for candidacy are outlined below:

• Cumulative GPA of 2.75 or better.
• Grades of B- or better in all professional education courses. (A student may request one exemption; practicum courses are not eligible for exemption.)
• Grades of B- or better in all required content courses. For Middle and Secondary Education concentrations, a B- average or better for all required content courses.
• A grade of C or better in English 100 or an equivalent transferred English course with a grade of C or higher.
• Successful completion of practicum/advanced practicum with a grade of B- or better.
• Satisfactory assessments of candidate professional attitudes utilizing the Teacher Candidate Dispositions and Professional Expectations assessment in both classroom and field settings. Assessments are completed by faculty, field supervisors, and mentor teachers. Candidates complete self-evaluations in their field experiences.

 

Program Completion: Teacher candidates seeking a degree leading to certification are required to:

• Successfully complete all clinical experiences, including earning a B- or better in practicum and advanced practicum experiences and a 15-week student teaching/internship placement.
• Grades of B- or better in all professional education courses (one, non-field exemption).
• Grades of B- or better in all required content courses. For Middle and Secondary Education concentrations, a B- average or better for all required content courses.
• Submit and present a Student Teaching/Internship Professional Portfolio. Portfolios are cross-rated by at least two faculty evaluators and require pre-service teachers to provide evidence demonstrating proficiency with Maine’s CCTS standards. (CCTS Standards Rationale and Artifact)
• Submit evidence of proficiency with skills related to lesson and unit planning, instruction, assessment, and use of technology. (Teacher Work Sample)
• Submit an analysis of the context of their school placement. (Contextual Factors Analysis)
• Satisfactory assessment of professional dispositions. (Teacher Candidate Dispositions and Professional Expectations)
• Satisfactory assessment of skills related to proficiency in planning, instruction, classroom management, assessment, and use of technology. (Essential Areas of Teaching assessment; Classroom Management Observation Checklist)
• A cumulative GPA of 2.75 or better. 

 

Assessment

UMF’s College of Education utilizes several assessment tools to ensure its teacher candidates possess the competencies they will need to teach effectively and to positively impact student learning and development. Assessments utilized by the TEU to ensure preservice teachers have proficiency with the CCTS standards, are skilled in planning, instruction, management and assessment, and have the professional dispositions to be successful in the classroom include the Student Teaching Portfolio, the Essential Areas of Teaching instrument, the TEU Lesson Plan and Teacher Work Sample assignments, the Classroom Management Observation Checklist, and the Teacher Candidate Dispositions and Professional Expectations assessment.

2023-24 Student Teaching Portfolio

To complete their programs, candidates submit a Student Teaching Portfolio. Using webtools (eg. Google Sites, Weebly), the Portfolio creatively showcases their student teaching experience, their Philosophy of Education, and includes evidence of proficiency with all eleven CCTS standards (two artifacts and rationales per CCTS standard). Proficiency with standards is determined by the Progress on Standard rubric utilized for the CCTS Rationales and Artifacts assessment tool. The rubric uses a four-point Likert scale, from Proficient (3) to No Evidence (0). Candidates upload their portfolio to SLL, where it is evaluated by their supervisor. Prior to the Portfolio Presentation Day that concludes student teaching, candidates share their digital portfolio with a second supervisor who serves as a cross-rater. On their presentation day, candidates share their portfolio with guests. Candidates meet one-on-one with their cross-rating supervisor to discuss the portfolio and each of the CCTS standards individually. If both evaluators determine that the candidate has met requirements for proficiency, the Director of Field Services submits a final overall evaluation of the portfolio in SLL indicating the candidate has met all requirements. Because the evaluation of the Portfolio is completed in SLL prior to the presentation of their evidence to their cross-rater, mean scores for each of the CCTS standards imply a small percentage of candidates have not achieved proficiency at the conclusion of student teaching. If either or both of the evaluators determine a candidate’s portfolio is missing evidence, or if the evidence is considered incomplete or insufficient, it must be revised and re-evaluated by both supervisors. Once deemed proficient by both evaluators, the Portfolio is then sent forward to the Director.

Portfolio Progress on Standard	AY 21-22 n = 81	AY 22-23 n = 101	AY 23-24 n = 77	TEU n = 259
CCTS 1 Mean Ratings	2.93	2.91	2.95	2.93
CCTS 2 Mean Ratings	2.88	2.99	2.89	2.93
CCTS 3 Mean Ratings	2.89	2.99	2.88	2.93
CCTS 4 Mean Ratings	2.89	2.94	2.87	2.90
CCTS 5 Mean Ratings	2.81	3.01	2.86	2.90
CCTS 6 Mean Ratings	2.89	2.99	2.84	2.92
CCTS 7 Mean Ratings	2.89	2.92	2.82	2.88
CCTS 8 Mean Ratings	2.85	2.95	2.84	2.89
CCTS 9 Mean Ratings	2.85	2.96	2.83	2.89
CCTS 10 Mean Ratings	2.78	2.90	2.84	2.84
CCTS 11.1-11.3  Mean Ratings	2.53	2.78	2.82	2.72
CCTS 11.4-11.7  Mean Ratings	2.85	2.91	2.83	2.87

 

2023-24 Essential Areas of Teaching Summary

Assessment Description: The Essential Areas of Teaching is an assessment conducted by mentor teachers and field supervisors during selected field experiences (Advanced Practicum and Student Teaching). The assessment is conducted at mid-term and end-of-term. The Essential Areas instrument assesses growth over the course of semester-long field experiences on performance indicators within five key categories: Planning, Instruction, Management, Assessment, and Technology.

For summary purposes, a strength is identified as an indicator with more than 95% of student teachers rated as Meets Expectations by both their field supervisor and more than 90% of student teachers rated as Meets Expectations by their mentor teacher by the end of the placement.

Strengths of TEU student teachers include:

Planning

• Addressing modification for individual needs of students

Instruction

• Reviews relevant prerequisites
• Shares objective/target with students
• Uses a variety of approaches and strategies
• Demonstrating subject matter knowledge and building on prior knowledge
• Uses proximity to increase engagement
• Provides appropriate feedback
• Involves all students in lessons
• Models what needs to be learned

Management

• Has procedures/routines well-established
• Reinforces classroom behavior expectations
• Uses proximity to redirect student attention
• Demonstrating personal regard for each student

Assessment

• Gives meaningful feedback to students
• Uses results of previous assessments to drive lessons
• Checks for understanding throughout the lesson
• Modifies instruction during the lesson based on student progress

Technology

• Facilitates and inspires student learning and creativity using technology
• Collaborates/communicates with stakeholders using digital tools
• Demonstrates ethical and legal use of technology
• Uses technology to engage in professional growth and leadership
• Uses available technology to design and plan instruction

 

2023-24 TEU Lesson Plan and Teacher Work Sample

The Lesson Plan (LP) and Teacher Work Sample (TWS) ask candidates to consider key aspects of planning, instruction (including technology), and assessment. Both assessments require candidates to

• identify measurable and observable learning objectives;
• align objectives with appropriate content standards;
• plan, use, and adapt assessments to allow students to demonstrate their knowledge and skills;
• use outcomes to inform instructional decisions and modifications;
• incorporate accessible resources and technology;
• effectively sequence instruction;
• apply strategies that allow for a variety of experiences;
• scaffold content to meet the identified needs of all students.

The LP rubric encompasses six categories rated on a four-point Likert-scale and within each category, performance indicators increase in developmental complexity from Beginning (1) to Proficient (3). The TWS serves as a significant extension of the LP and incorporates a rubric encompassing seven categories scored on a five-point Likert scale from Exceeds (5) to Does Not Meet Expectations (1). Candidates organize and share a cohesive unit of study from their experience, including 5-7 lesson plans, samples of student work, and documentation of assessment. The TWS tasks candidates with presenting a comprehensive look at how they devise and evaluate pre-, formative, and summative assessments; design their instruction to achieve learning targets; and analyze outcomes and modify instruction to address gaps and meet students’ needs

Findings:

For 2023-24, in categories related to planning and instruction, candidates consistently meet, if not exceed, expectations.

• TWS – Learning Goals: Means for the two indicators in this category range from 4.23 to 4.34
• TWS – Design for Instruction: Means in 2023-24 range from 4.27 to 4.33
• LP – Instructional Methods (Teaching and Learning Sequence); Percentages of student teachers rated at Proficient levels on the four relevant indicators range from 88.4% to 93.5%.  This includes detailing the instructional sequence, specific strategies, engagement strategies, incorporation of technology, and pacing that is appropriate for the content and responsive to student needs.

In categories related to assessment and differentiation, candidates are consistently meet expectations in the TWS. The LP rubric indicates this could be an area for growth.

• TWS – Assessment Plan: Means for the two indicators in this category ranged from 4.28 to 4.40.
• TWS – Analysis of Student Learning: Means for the four indicators in this category range from 4.30 to 4.39.
• LP – Assessment: Adapts classroom assessments appropriately to enable all learners to demonstrate their knowledge and skills (78.3% of student teachers at the Proficient level) and Incorporates required accommodations/modifications/extensions in assessments for all identified students (75.7%). 
• LP – Meeting Student’s Needs – Identifies common misconception and plans appropriate scaffolds and/or differentiated instruction (74.9% at the Proficient level). 

 

2023-24 Teacher Candidate Dispositions & Professional Expectations Summary

Assessment Description: The TEU assesses teacher candidate dispositions multiple times throughout a student’s program, both in the college classroom and in the field. The purpose of the dispositions assessment is to provide early and frequent feedback to students regarding the professional expectations that are embedded in the education field. Two versions of the assessment exist. A classroom version is utilized in methods and non-field-based courses and is completed by course faculty. A field version is utilized in practicum, advanced practicum, internships, and student teaching placements and is completed by both mentor teachers and field supervisors. Students in practicum and advanced practicum placements also complete self-evaluations using the instrument. Goals of the assessment include helping students understand the professional expectations of the field, providing actionable feedback to students regarding those expectations, serving as a trigger to provide support to struggling students, and adding a level of selectivity to ensure teacher candidate quality. Satisfactory outcomes on the assessment are required for both program candidacy and successful completion of student teaching.

 

2023-24 Classroom Management Observation Checklist Summary

Assessment Description: The Classroom Management Observation Checklist (CMOC) is an EPP-developed observation instrument utilized during a teacher candidate’s student teaching placement. The tool includes indicators of both teacher and student actions in the school setting that are integral to an effective and safe classroom community. Indicators are intended to be actionable to provide useful feedback to candidates about strategies to improve their classroom management. The rubric consists of 17 indicators within two categories: Student Behavior Observed (referring to students in the student teachers’ classroom) and Teacher Behavior Observed (referring to the student teachers themselves). A broad assortment of validated indicators are included to provide a comprehensive view of candidate proficiency. Summaries are disaggregated by program.

 

CAEP Measure 4: Ability of Completers to be Hired

Where UMF Graduates Teach

Becoming a graduate of a UMF Education program opens the door for employment around Maine and New England, across the United States, and even internationally. While the State of Maine does not collect data tracking teacher candidate hiring, findings from the TEU’s 2022 biennial survey of recent graduates found more than 91% of respondents are currently employed in the field, with two-thirds in both their preferred career path (i.e. teacher) and desired position (i.e. age group/grade level, content area, location, etc.). More than three quarters (78%) of respondents were hired within 4 months of graduation, and nearly 87% had been hired within a year of program completion.

Title II Reports

Title II is a federal policy tasked with assisting states in “preparing, training, and recruiting high quality teachers and principals” to improve educator and administrator quality. For more information about Title II, please visit their site at https://title2.ed.gov/Public/Home.aspx

UMF 2024 Title II Annual Report