CSU Long Beach College of Education Accreditation
The Conceptual Framework and mission statement provide the foundation for program development, implementation, and ongoing review in the unit. Programs leading to initial licensure and programs for other professional school personnel are based on professional, state, and institutional standards, which provide the context for developing programs and assessing candidate knowledge, skills and dispositions. For initial and advanced licensure programs, the California Standards of Program Quality and Effectiveness provide the basis for program development. Masters degree programs for other school professionals adhere to professional and institutional standards (and state licensing standards where applicable).
Information and data used in this section of the Institutional Report derive from a variety of sources. In particular, reference is frequently made to three sources. One source of data is the self-study developed by each program. For programs leading to a credential, self-studies have been written to align with Commission on Teacher Credentialing standards, while programs leading to the Master’s degree have been written to align with campus internal review reporting standards and standards in the respective program fields.
A second data source is the California State University System-wide Evaluation of Professional Teacher Preparation project. This project administers standardized surveys to all California State University initial teacher preparation candidates as they exit their programs (CSU Exit Survey; Exhibit 6), to graduates of initial programs at the conclusion of their first year of teaching (CSU Survey of Graduates; Exhibit 7), and to the employment supervisors of graduates at the end of their first year of teaching (CSU Survey of Graduates’ Employment Supervisors; Exhibit 8). Data from these surveys are made available to the unit annually for program improvement purposes. The structure of the CSU evaluation project is more fully described in Standard 2.
The third primary source of data on candidates and programs is the Program Assessment and Evaluation System (PAES), which serves as the foundation of the Unit Assessment System (UAS) and is described in detail in Standard 2. Using a common unit-wide framework (Exhibit 9), all programs have developed a PAES that specifies candidate assessment and program evaluation activities that are consistent with the Conceptual Framework (Exhibit 5), national and state accrediting agency standards, and standards unique to the professional community of each program. Each program keeps hard copies of its current assessment and evaluation efforts in a PAES binder, available in the Exhibits Room with the program’s self-study document and supporting materials. A full discussion of the PAES occurs in Standard 2. In Standard 1 we provide exemplary data displays drawn from selected program PAES materials, and refer reviewers to the complete binders for a fuller picture of each program.
Table 1.01 (Unit Programs by 3 NCATE Categories) displays unit programs by three categories: initial teacher preparation, advanced (continuing) teacher preparation, and programs for other school personnel. The discussion that follows throughout Standard 1 utilizes these three categories as sub-headings. Our emphasis is on initial teacher preparation programs, and we always provide examples from each of the three initial teacher preparation programs throughout Standard 1. Since we have many programs for advanced (continuing) teacher preparation and for other school personnel, we provide examples from selected programs. Links throughout the text lead the reader to documents for programs not explicitly discussed here. Hard copies of program self-studies, PAES binders, student work samples, and other supporting materials unique to programs reside in the Exhibits Room.
Content, experiences, and assessments for initial teacher preparation programs, advanced teacher preparation programs, and programs for other school personnel are based on the unit’s Conceptual Framework, Commission on Teacher Credentialing professional program standards, national standards and institutional standards. All programs have four transition points for candidates, as summarized in Table 1.02 (Transition Points for Candidates in Unit Programs). A detailed description of transition points is in the supplemental document titled Transition Points in Initial Teacher Preparation Programs, Advanced (Continuing) Teacher Preparation Programs, and Programs for Other Professional School Personnel; Exhibit 10).
Candidates in initial credential programs demonstrate content knowledge in one of three ways: completion of a state-approved subject matter program, passage of the appropriate version of the California Subject Examinations for Teachers (CSET), or by review of a transcript in which the transcript is compared to a state-approved subject matter program (Table 1.03: Candidate Demonstration of Content Knowledge in Initial Teacher Preparation Programs).
In compliance with No Child Left Behind, California requires that candidates for the Multiple Subject and Education Specialist credentials pass the California Subject Examination for Teachers: Multiple Subjects (CSET: Multiple Subjects). Therefore, the pass rate for all program completers is 100 percent. (Table 1.04: CSET Multiple Subjects Pass Rates, 2004-05)
Demonstration of knowledge and skill for teaching reading as measured by passing the Reading Instruction Competence Assessment (RICA) is required of all Multiple Subject and Education Specialist credential candidates at the time of application for the credential. Passage of the RICA is not a program completion requirement per se, but a requirement for the credential. Table 1.05 (Reading Instruction Competence Assessment [RICA] Passing Rates, 2004-05) displays the pass rates on the RICA for CSULB credential candidates for 2004-2005.
Single Subject Program candidates demonstrate content competence through completion of a CCTC-approved program of study in the subject of their teaching authorization, or its equivalent (transcript review), or by passing a CCTC-approved exam (i.e., CSET in the content area). The unit has CCTC-approved programs in 10 content areas (Table 1.06: Single Subject Credential Programs). Course selection, content, and sequence in each subject matter program are developed by arts and sciences faculty in the 10 disciplines represented in the Single Subject Program. In most instances, these subject matter programs also function as undergraduate majors. The exceptions are History/Social Science and Science, which are interdisciplinary subject matter programs (refer to subject matter program documents in the Single Subject Program section of the Exhibits Room). Data show that a high percentage of Single Subject candidates demonstrate content competence by course work, rather than by examination. In 2003-2004, approximately 67% of program graduates demonstrated subject matter competence by completion of a Commission-approved subject matter program or by equivalent course work (transcript review). Of that number, approximately 77% completed their course work at CSULB (CSU Survey of Graduates, 2005 Report, Table 3-B). Single Subject Program candidates must demonstrate subject matter competence in order to advance to student teaching. Therefore, the success rate for all program completers, whether they demonstrate content knowledge by examination or by course work, is 100 percent.
Another important source of information on candidate content knowledge is the CSU Exit Survey (Exhibit 6) completed by candidates at the conclusion of student teaching (the survey is discussed in Standard 2). As Table 1.08 (Perceptions of Their Content Preparation by Initial Program Candidates at Program Exit, 2004-05) displays, exiting candidates in the three initial preparation programs overwhelmingly perceived that their program prepared them well or adequately to know and understand the subjects they teach.
A final source of information on initial credential program candidate content knowledge preparation is the CSU Survey of Graduates and the CSU Survey of Graduates’ Employment Supervisors conducted as part of the CSU System-wide Evaluation of Professional Teacher Preparation (see Standard 2). These surveys are follow-up studies of graduates of the three initial credential programs and their employment supervisors (usually the building principal). Program graduates report high satisfaction with their preparation to teach in their content areas (Table 1.09: Initial Credential Program Graduates’ Perceptions and Their Employment Supervisors’ Ratings of Graduates’ Content Preparation, 2003-2006). Graduates consistently report above the 75% percent level that they were well or adequately prepared as a new teacher to know and understand the subjects of their curriculum. Employment supervisors of these graduates report above the 80 percent level that they were well or adequately prepared as a new teacher to know and understand the subjects of their curriculum during that same period of time. These data provide compelling evidence that graduates of our initial preparation programs are very well regarded in the professional community. The data show that supervisors tend to rate graduates as better prepared to teach content than do graduates themselves.
Advanced (continuing) teacher preparation programs require that candidates hold valid teaching credentials and/or a specified amount of teaching experience or educational background for admissions consideration. This requirement ensures initial level content knowledge pertinent to the candidate’s teaching authorization. Candidates are usually practicing teachers who apply knowledge of their subject regularly in individual educational settings. Professional content knowledge gained through participation in advanced programs is demonstrated through successful completion of program courses, field experiences, comprehensive examinations, theses, exit surveys of candidates, and follow-up studies of program graduates.
Table 1.10 (Content Knowledge in Advanced (Continuing) Programs) displays key content knowledge in programs for advanced (continuing) teacher preparation. This information is more richly captured in each program’s PAES binder, section I.A, along with Student Learning Outcomes and descriptions of benchmark assessments. As described in Standard 2, advanced (continuing) teacher preparation programs participate in the campus-based, WASC-driven annual assessment of Student Learning Outcomes (SLOs). SLOs represented in Table 1.10 will be assessed beginning in the 2006-07 academic year. Each program will assess 1-2 SLOs per year and submit assessment data, results, and analyses in a program report due June 1. The expectation is that each campus program, undergraduate and graduate, will complete a full round of assessing its SLOs by the time of WASC review in 2009.
Exit surveys provide information about candidates’ perceptions about their content knowledge in programs for advanced (continuing) teacher preparation. Although candidate perceptions are no substitute for objective measures of candidates’ content knowledge, these survey data corroborate what our other assessment data show—that candidates are well prepared in their programs. Tables 1.11 and 1.12 show results from exit survey items from two example programs (Early Childhood Education and Curriculum and Instruction) that indicate the extent to which candidates perceived they have learned the content knowledge, skills and dispositions addressed in the program. Exit survey results for the Early Childhood Education Master’s degree program revealed that graduates perceived that the program prepared them well in content knowledge in their field. (Table 1.11: Early Childhood Education Program Candidates’ Perceptions at Program Exit of the Extent to Which They Developed Their Content Knowledge While in the Program, 2004-2005). Curriculum and Instruction program candidates (elementary and secondary combined) reported that they felt well prepared on a number of dimensions related to program goals, content, and Student Learning Outcomes (SLOs). As Table 1.12 (Candidate Perceptions at Program Exit of Their Preparation Related to Content Knowledge and Student Learning Outcomes (SLOs) in Curriculum and Instruction Elementary and Secondary, Sp05-Sp06) displays, on all items except one, well over 90 percent of the candidates strongly agree or agree that the program prepared them to understand research related to the field, write literature reviews, and conduct inquiry and action research. In addition, candidates reported that the program enhanced their understanding and practice of curriculum, instruction and assessment, as well as their ability to bring about positive change.
Professional content knowledge of candidates in programs for other school personnel is demonstrated through successful completion of program courses, field experiences, comprehensive examinations, and theses. Exit surveys of candidates and follow-up studies of program graduates provide further evidence.
Each program has a set of Student Learning Outcomes (SLOs) that reflect the key knowledge, skills, and dispositions candidates are expected to acquire and demonstrate as they progress through the program. The Program Assessment and Evaluation System (PAES) binder for each program contains the SLOs, the benchmark assessments, the program improvement calendar, and data from recent benchmark assessments
Examples of how candidates demonstrate professional content knowledge in programs for other school personnel are shown in Table 1.13 (Professional Content Knowledge in Programs for Other School Personnel).
As a condition for exiting Master’s programs, all candidates must pass a comprehensive exam or complete a thesis or project that allows them to demonstrate in a summative way the extent to which they have met the learning goals of the program. The Graduate Studies and Research Office, under the direction of the Associate Dean for Graduate Programs, oversees policies for graduate programs, administers the comprehensive examinations, and coordinates completion of theses.
Exit surveys provide information about candidates’ perceptions of their professional content knowledge in programs for other school personnel. Tables 1.14 and 1.15 show results from exit survey items from two example programs that indicate the extent to which candidates perceive they have learned the content knowledge, skills and dispositions addressed in the program.
Candidates in the Reading and Language Arts program, which includes a Master’s degree, reading certificate, and advanced Reading Credential, reported significant learning in two key program areas: English language “content” (e.g., structure of the language) and issues around reading and writing instruction, and a knowledge of the development of reading comprehension skills. Both of these are key elements of the knowledge base that candidates will draw on in their work as reading specialists (Table 1.14: Reading Program Candidates’ Perceptions of the Extent to Which They Developed Their Content Knowledge While in the Program, 2000-2005).
The majority of School Psychology program candidates reported feeling prepared in content knowledge in their field of school psychology as well as in prevention, crisis, and mental health upon exiting the program. The survey results displayed in Table 1.15 (School Psychology Program Candidates’ Perceptions at Program Exit of the Extent to Which They Developed Their Content Knowledge While in the Program, 2002-2004)indicate that candidates’ perceptions of their preparedness in their content knowledge increased substantially over a three-year period.
Subject-specific methods courses introduce candidates in initial programs to pedagogical knowledge and skills specific to each content area. All methods courses have stipulated fieldwork hours so that candidates have the opportunity to put into practice the ideas, concepts, and skills they are learning through coursework. In student teaching, candidates demonstrate their abilities to apply these skills in classrooms over the duration of a semester. In this culminating experience, candidates diagnose students’ needs and abilities using assessment instruments studied in class, develop unit plans, assume full responsibility for a public school classroom, and assess student learning, thereby synthesizing the essential knowledge and practice learned in methods courses.
In the Multiple Subject Program, candidates take 5 subject-specific pedagogy courses (Language Arts, Reading, Mathematics, History-Social Science, and Science). In addition, candidates in the Spanish BCLAD track take a course in teaching reading in Spanish and English, while candidates in the Asian BCLAD track take a biliteracy teaching course in the target language (Cantonese & Mandarin, Khmer, Korean, Vietnamese). Candidates in the Education Specialist Program take two subject-specific pedagogy courses in common with Multiple Subject candidates (Reading, Math). In addition, they take courses in teaching literacy and math with a specific emphasis on special needs students.
Candidates in the Single Subject Program take a subject-specific pedagogy course in their discipline (Curriculum and Methods in Teaching in the content area, e.g. English, Math, Art) during the semester prior to student teaching. These courses cover disciplinary curriculum and pedagogy for delivering the curriculum in secondary English, Math, History, Science, etc. classrooms. A 15-hour fieldwork requirement gives candidates opportunities to practice and reflect on their learning in authentic settings.
One source of data on candidate preparation in the area of pedagogical content knowledge is candidate performance on Task 1 of the California Teaching Performance Assessment (CA-TPA; Exhibit 11), which assesses the candidate’s ability to use content-specific and developmentally appropriate pedagogy. (See Standard 2 for a description of the CA-TPA and its use in initial teacher preparation programs.) Both the Multiple Subject and Single Subject programs embed CA-TPA Task 1 in core courses (Table 1.16: CA-TPA Task 1 in Initial Teacher Preparation Programs).
Early results from the performance assessments in the Multiple Subject Program show that, overall, candidates performed well on classroom assessments on 9 important teaching dimensions in 2005-06 (Table 1.17: Multiple Subject Program Candidate Assessment on 9 Key Teaching Performance Expectations Related to Pedagogical Content Knowledge, 2005-06). Mean scores on each of the 9 items were above 3.5 on a 4-point scale. The relatively low mean score on item 9, instructional planning, led the program to provide additional support for candidates by uploading lesson plan formats to TaskStream beginning fall 2006, thus making available to instructors and candidates alternative ways of planning lessons as a resource for candidates during course work, fieldwork and student teaching.
The Multiple Subject Program also collects and analyzes data from Student Teaching Evaluations. Table 1.18 (Multiple Subject Student Teacher Performance on 10 Pedagogical Content Knowledge Items on the Student Teaching Formative Evaluation Form, Spring 2006) displays data on 10 pedagogical content knowledge measures reported by cooperating teachers and university supervisors on the formative (mid-term) evaluations of student teachers.
The Single Subject Program has data on candidate performance yielded from two major assessment instruments. The first dataset involves quantifying assessor scoring on the Student Teaching Evaluation form, which has 56 response items based on specific performance indicators of the Teaching Performance Expectations (TPEs; Exhibit 12). The program is in the process of piloting a database from spring 2006 Student Teaching Evaluations by university supervisors and master teachers. When complete, this database will permit statistical analyses on a wide variety of teaching performance behaviors from two independent observers. Table 1.19 (Single Subject Student Teacher Performance on 12 Pedagogical Content Knowledge Items on the Student Teaching Summative Evaluation Form, Spring 2006) shows data from the pilot study on 12 items on the Student Teaching Summative Evaluation addressing pedagogical content knowledge.
The second project involves building a database for the CA-TPA, Tasks 1-4 (Exhibit 11). As discussed in Standard 2, each of the four TPA Tasks is an assignment in a program course that is assessed using a state-validated scoring rubric. When complete, this database will enable analyses of student performance on each of the four TPA Tasks. We will be able to do comparative analyses across time, subject matter program, and course section. In Table 1.20 (Single Subject Program Aggregate Mean Scores on TPA Tasks 1-4, Spring 2006 [pilot])we provide data from the spring 2006 pilot data collection.
The Education Specialist Program collects and analyzes data from the Candidate Competency Checklist during the student teaching semester. Table 1.21 (Education Specialist Student Teacher Performance on 12 Pedagogical Content Knowledge Items on the Candidate Competency Checklist, Spring 2006)displays data on 12 pedagogical content knowledge measures reported by field supervisors during the culminating field experience.
Data on candidates’ preparation of their preparation in the area of pedagogical content knowledge come from the CSU Exit Survey administered to candidates at the end of student teaching. This survey asks student teachers to respond to a series of items on their content preparation as well as their preparation to develop curriculum and teach their subject.
Table 1.22 (Pedagogical Content Knowledge of Single Subject Candidates at Exit from Program) displays data from the CSU Exit Survey related to the Single Subject Program. The data indicate that candidates perceive their development of pedagogical content knowledge in their teaching subject as an area of strength. Above 90 percent reported they were well or adequately prepared to teach their subject according to state standards and to use textbooks aligned with those standards.
Candidates in the Multiple Subject Program also perceive that they exit the program with strong pedagogical content skills. Over 90 percent of Exit Survey respondents in the two year period 2004-2006 reported that they were well or adequately prepared to teach to state standards in four content areas: reading/language arts, mathematics, science, and history/social studies (Table 1.23: Pedagogical Content Knowledge of Multiple Subject and Education Specialist Candidates at Program Exit in 4 Content Areas, 2004-06).
Candidates in the Education Specialist Program reported that their preparation in pedagogical content knowledge was strongest in teaching reading/language arts and mathematics, about 90 percent perceiving that they were well or adequately prepared in these two subjects (Table 1.23). They reported feeling less well prepared in science and history/social studies.
Two sources of follow-up data on candidate preparation in the area of pedagogical content knowledge are the annual CSU Survey of Graduates and the Survey of Graduates’ Employment Supervisors. Table 1.25 (Multiple Subject Program Graduates’ Perceptions and Their Employment Supervisors’ Ratings of Graduates’ Preparation in Pedagogical Content Knowledge, 2003-2005), Table 1.26 (Education Specialist Program Graduates’ Perceptions and Their Employment Supervisors’ Ratings of Graduates’ Preparation in Pedagogical Content Knowledge, 2003-2005), and Table 1.27 (Single Subject Program Graduates’ Perceptions and Their Employment Supervisors’ Ratings of Graduates’ Preparation in Pedagogical Content Knowledge, 2003-2005) display data on graduates’ perceptions and their employment supervisors’ ratings of how well they were prepared to teach their content. The data show that program graduates perceived their preparation to know and understand their subject content was strong in most of the areas during each of the three survey periods. The data also show that graduates of the three initial credential programs were rated by their employment supervisors (principals, vice-principals, or department chairs) to be well or adequately prepared to know the content of their curriculum and to use a variety of teaching strategies effectively. In both instances, graduates’ perceptions and employment supervisors’ ratings, our graduates compare favorably to the CSU system on most dimensions for most data collection periods.
Each of the three initial teacher preparation programs provides candidates with preparation in the uses of digital technology for professional purposes in different ways. The Multiple Subject Program infuses the elements of the California Level I Technology Standard throughout program courses (see Standard 9, MSCP self-study). The Single Subject Program requires that candidates complete a state-approved Technology Standard Level I course prior to advancing to student teaching. Each of the 10 content areas has its own subject-specific Level I technology course, or draws on the generic course ETEC 444, offered by the EdPAC department. The lone exception is the Science Education program, which embeds Level I technology elements in three courses: EDSS 300S, EDSS 450S, and student teaching (see Standard 9, SSCP self-study). The Education Specialist Program also uses ETEC 444 for all candidates (see ESCP self-study). In addition, instructors in all the programs expect candidates to utilize computer technology for informational, communication, administrative and document production purposes. Almost all faculty use BeachBoard (the campus version of BlackBoard) as a course administration tool. Refer to Standard 3, Standard 5 and Standard 6 for more on technology in the unit.
Exit Surveys of initial program completers provide information about candidates’ preparation for integrating technology in their teaching. For the two years for which we have reliable data (i.e., 2004-2006) over 80 percent of candidates across the three initial credential programs reported that they felt well or adequately prepared to use computer-based technology in their teaching as well as for research and administrative purposes (Table 1.28: Initial Teacher Preparation Program Candidates’ Perceptions at Program Exit of Their Preparation to Use Technology in Their Teaching, 2004-2006).
The CSU Survey of Graduates and Survey of Graduates’ Employment Supervisors provide data on candidate preparation to use digital technology professionally on two dimensions: to help students use computer applications as part of the curriculum and to utilize computer technology as a teaching tool and as an administrative tool (Table 1.29: Initial Credential Program Graduates’ Perceptions and Their Employment Supervisors’ Ratings of Graduates’ Preparation to Use Technology in Their Teaching, 2003-2006). The general pattern is that employment supervisors rate graduates higher on both items than graduates rate themselves in each credential program; this pattern mirrors the system as a whole.
Since candidates for advanced teacher preparation programs already possess a teaching credential in most cases, they have had extensive study of pedagogical content knowledge (PCK) and opportunities to practice their skills both in student teaching and their subsequent induction years. Pedagogical content knowledge is implied, rather than explicitly stated, in program Student Learning Outcomes. Nevertheless, PCK is an important element in advanced programs. Table 1.30 (Location of Pedagogical Content Knowledge Study in Advanced (Continuing) Programs) displays specific courses where PCK is addressed. PCK is demonstrated through completion of program courses, course-based assessment, field experiences, comprehensive exams, and theses. Program descriptions and course syllabi demonstrate that experiences and advanced level coursework are aligned with institutional, state and national standards and frameworks where applicable.
Each initial credential program has a set of courses that addresses professional and pedagogical knowledge to prepare candidates to work successfully in classrooms with students from diverse cultural, linguistic, and social backgrounds. Course goals, objectives, and content are consistent with the unit’s mission statement and Conceptual Framework and have been developed in alignment with state standards.
Table 1.31 (Key Professional and Pedagogical Knowledge in Initial Teacher Preparation Program Courses) shows how initial teacher preparation program courses align with key foundational knowledge and skills required of beginning teachers. Professional and pedagogical knowledge and skills for teacher candidates seeking preliminary credentials in initial programs are most explicitly addressed through learning experiences in targeted courses. These courses provide candidates with broad understanding of the historical, philosophical, and cultural aspects of schooling. Within these courses, candidates learn the principles of child development, learning and motivation, and educational strategies that promote equity for students of all cultural, linguistic, and social backgrounds. These courses have fieldwork requirements (in the form of specific assignments or a set number of hours, or both) that include observation and participation in public school classrooms through which the candidate gains increasing experience with classroom dynamics, including the interactions of diverse learners among themselves, with teachers, and with the curriculum.
The CSU Exit Surveys provide information on preparation in professional and pedagogical knowledge and skills for initial credential program candidates (Table 1.32: Initial Teacher Preparation Program Candidates’ Perceptions at Program Exit of Their Preparation on 6 Dimensions of Professional and Pedagogical Knowledge and Skills, 2004-2006). These data indicate that candidates perceive they are well or adequately prepared on 6 dimensions of professional and pedagogical knowledge and skills by the time they leave their program.
Follow-up surveys of initial credential program graduates and their employment supervisors administered through the CSU System-wide Evaluation project assess the effectiveness of the Multiple Subject, Single Subject, and Education Specialist Credential programs on measures of professional and pedagogical knowledge and skills (Table 1.33: Initial Teacher Preparation Program Graduates’ Perceptions and Their Employment Supervisors’ Ratings of Graduates’ Preparation on 5 Dimensions of Professional and Pedagogical Knowledge and Skills, 2003-2005). Survey results demonstrate strengths and weaknesses of our initial teacher preparation programs. Graduates and their employment supervisors rate as program strengths preparation to understand child/adolescent development and learning needs, preparation in principles of educational equity, preparation to work with students from diverse cultural backgrounds, and preparation to understand the connections among personal, family and community conditions that affect learning. CSU Long Beach programs compare favorably to the CSU system as a whole on nearly every dimension.
Candidates in advanced (continuing) teacher preparation programs learn about professional organizations and apply institutional, state and national standards, assessment techniques, use of technology and other teaching methods throughout course work in their programs (Table 1.34: Location of Key Professional and Pedagogical Knowledge in Advanced (Continuing) Teacher Preparation Programs). Course syllabi document that professional knowledge and skills are included in these courses and that content are related to the Conceptual Framework. Culminating projects, theses, and examinations are ways by which candidates demonstrate they have acquired the requisite knowledge, skills, and dispositions to complete the program.
As an example, candidates in the graduate degree program in Early Childhood Education demonstrate that they have broad professional knowledge and skills including the use of research in teaching and applying appropriate strategies for involving parents in the educative process. Exit Survey results displayed in Table 1.35 (Early Childhood Education Program Candidates’ Perceptions of Their Preparation on 4 Dimensions of Professional and Pedagogical Knowledge and Skills, 2004-2005)demonstrate that program completers are prepared well to understand and evaluate research studies in their field, involve parents in children’s education, and use appropriate curriculum with young children.
Candidates in programs for other school personnel such as school counseling, school psychology, and educational administration learn about and apply institutional, state and national standards, integrate theory, practice, and research, evaluate the needs of diverse and urban populations, collect and analyze data related to their professional work, reflect on their practice, and use technology as a professional tool. Table 1.36 (Key Professional and Pedagogical Knowledge in Programs for Other School Personnel)identifies the professional coursework for each program. Course syllabi document that professional knowledge and skills are included in these courses and that their content is related to the Conceptual Framework and institutional, state and national standards. Candidates use case studies, professional work samples, portfolios, notebooks, and journals to collect and analyze data and reflect on their practice. Course assignments routinely require critical analysis of the extant knowledge base in the field of study through literature reviews and short-term research projects, as can be noted in course syllabi. Program self-study documents and supporting material in the Exhibits Room fill out this picture.
Examples of program data collected on candidates’ development of professional knowledge and skills are displayed in Table 1.37 (Reading Program Candidates’ Development of Professional Knowledge and Skills, 2002-2005), Table 1.38 (Administrative Services Tier I Credential Program Candidates’ Development of Professional Knowledge and Skills, 2004-2005),Table 1.39 (Administrative Services Tier II Credential Program Candidates’ Development of Professional Knowledge and Skills, 2004-2005), and Table 1.40 (School Psychology Credential Program Candidates’ Development of Professional Knowledge and Skills, 2002-2004). In each instance, candidates report that they are highly successful learning the knowledge and skills unique to their field.
Central to the mission of the unit is the preparation of educators who are socially responsible leaders for a rapidly changing, technologically-rich world who value diversity, service and collaboration, and lifelong learning and professional growth. The key principles of the Conceptual Framework imply a set of dispositions that cuts across program and discipline boundaries. In addition, initial teacher preparation programs, advanced (continuing) teacher preparation programs, and programs for other school personnel all have program-specific sets of professional dispositions expected of their candidates related to their respective fields.
In fall 2006 we piloted a Dispositions Survey (Exhibit 13) in selected initial and advanced programs as well as in programs for other school personnel. We identified 12 dispositions that were explicitly stated or implied in the Conceptual Framework (Exhibit 5), such as “promoting intellectual growth for all students,” “valuing diversity among your students,” being a socially responsible leader,” “using research to help inform practice,” and “engaging in ongoing evaluation of my practice.” Candidates responded in two ways. First, we asked them to rank the importance to them of each professional disposition on a 4-point scale ranging from “not that important” to “very important.” Then we asked them to indicate how well they felt their program prepared them on the same 12 items using a 4-point scale ranging from “not at all prepared” to “well prepared.”
Preliminary analyses of the Dispositions Survey data provided some intriguing insights into how candidates perceived the importance of key dispositions and how well their program prepared them. The survey was sent to a sampling of candidates in selected programs who were near program completion. On all of the 12 survey items, over 92% of candidates in all three types of programs—initial credential programs, advanced (continuing) teacher preparation programs, programs for other school personnel—responded that the disposition was “important” or “very important” to them professionally. However, when asked to rate how well their program prepared them, responses were 91% or lower for all items. Fewer than 80% of respondents reported feeling “adequately” or “well” prepared on three items—“collaborate with the community” (64%), “promote school improvement for all students” (70%), “prepared for a rapidly changing, technologically-rich world” (78%). There is clearly a discrepancy between perceived importance and perceived preparation that we’ll need to address. We plan to conduct further data analyses and to initiate discussions at the unit, department, and program levels.
Dispositions for elementary teacher candidates are emphasized throughout the Multiple Subject Program and assessed in various program phases. Upon entry, candidates submit a report by the cooperating teacher regarding their 45-hour fieldwork assignment, pass an oral interview with a program faculty, and submit a statement of personal goals and professional philosophy of education. After admission to the program, candidates attend a mandatory advising meeting at which time they are introduced to and discuss dispositions shared by effective teachers. They complete a self-assessment of dispositions (see Multiple Subject Program self-study in the Exhibits Room) as a means of reflecting on their own attitudes and habits. At this time they establish goals for altering their dispositions as needed in order to develop into a caring, concerned, and responsible educator.
During the program, candidates demonstrate professional dispositions in course activities, assessments, and field experiences. Teaching Performance Expectation 12 (Professional, Legal, and Ethical Obligations) is assessed in the course EDEL 472 on a signature assignment based on CA-TPA Task 1, Scenario 4 (Exhibit 11). Teaching Performance Expectation 13 (Professional Growth) is assessed in EDEL 442 in a signature assignment called Reflective Essay, which is based on a prompt in CA-TPA Task 2 (Exhibit 11). Table 1.41 (Multiple Subject Program Candidates’ Aggregated Scores on TPEs 12 and 13, 2005-06)displays data collected on these two signature assignments in 2005-06.
During the student teaching phase of the program candidates undergo both a mid-point formative assessment and a final summative assessment. The assessment form contains several items that assess candidate dispositions. Table 1.42 (Master Teacher and University Supervisor Ratings of Multiple Subject Program Candidates During Student Teaching on 6 Measures of Dispositions on the Formative Assessment Form, 2005-06)indicates that candidates score strongly on 6 dimensions of dispositional practice. Candidates who do not score well are identified and their university supervisor works with them on areas to improve.
At the summative evaluation at the conclusion of student teaching, these 6 dispositions are collapsed into a single indicator labeled Developing as a Professional Educator. Data from the 2005-2006 year indicate that candidates were rated highly on this disposition indicator by their university supervisors and master teachers when they completed the program (Table 1.43: Combined University Supervisor and Master Teacher Ratings of Multiple Subject Program Candidates at the Conclusion of Student Teaching on the Disposition “Developing as a Professional Educator,” 2005-06).
Candidates in the Single Subject Program are assessed on specific professional dispositions. Based on Danielson’s (1996) performance assessment rubric, we have identified five dispositions that we are assessing in our teacher candidates: caring for students, sensitivity to diversity, sense of fairness, personal reflection, and sense of professionalism. These dispositions are demonstrated throughout our program through what our candidates write, say and do. We access these dispositions at program entry through our EDSS 300 course (instructor assessment, candidate interview, fieldwork evaluation, educational philosophy paper), throughout the program in coursework (various assignments, CA-TPA) and in field experiences (fieldwork evaluations, written assignments) and at during the program culminating experience, student teaching and student teaching seminar (CA-TPA, mid-term and final evaluation). Refer to Table 1.44 (Candidate Dispositions in the Single Subject Program) for a display of where, how, when, and by what vehicle dispositions are assessed.
Candidates in the Education Specialist Program are assessed on specific dispositions that are aligned with the Conceptual Framework and professional standards. The importance of professional dispositions is discussed during the initial group advisement meeting upon program admission, and then further elaborated upon in EDSP 480. Ethical guidelines, effective collaboration and communication skills, maintaining a reflective approach and other key dispositions are discussed at length in several of the program courses, and candidates are given opportunities to develop and reflect upon these developing dispositions via in-class activities, analyses of case studies, and collaborative group projects in core program courses. Table 1.45 (Candidate Self-Ratings and Supervisor Ratings on 3 Key Professional Dispositions in the Education Specialist Program at Program Entry and Exit, 2006-2007)displays aggregated candidate self-assessment and their fieldwork supervisors’ assessment for 3 key professional dispositions.
The CSU Exit Survey (Exhibit 6) of initial program candidates contains several measures of candidate perceptions of their growth in the area of dispositions. Table 1.46 (Perceptions of Initial Credential Program Candidates of Their Preparation on 6 Professional Dispositions, 2005-2006)displays aggregated data from candidates who completed their programs in 2005-2006. Candidates generally indicate they felt well or adequately prepared by their program on 6 measures of professional dispositions. Using 75% as a benchmark, one area of concern involved Single Subject Program candidates, of whom only 69% reported they were well or adequately prepared to know about resources in the school and community for at-risk students and families. Since this item functions as a proxy for the disposition required of a socially responsible educator, it raises some concern. The program will look carefully at 06-07 data on this item to see if 05-06 is an aberration or a trend.
Follow-up survey data available through the CSU system-wide evaluation project allow for comparisons among program candidates (Table 1.46) and graduates and their employment supervisors (Table 1.47: Perceptions of Preparation on 6 Professional Dispositions by Initial Credential Program Graduates and Their Employment Supervisors, 2003-2005) on 5 key dispositions. In general, exiting program candidates feel better prepared in these areas than do graduates looking back on their preparation at the end of their first year of teaching. Similarly, employment supervisors of graduates rate them better prepared in these dispositions than graduates rate themselves.
Professional dispositions for candidates in advanced (continuing) teacher preparation programs and programs for other school personnel are based on the guiding principles in the unit Conceptual Framework (Exhibit 5) and are informed by state and professional organization guidelines in the fields of the various programs. For example, candidates in the School Psychology program are expected to demonstrate positive dispositions toward work persistence, work organization, use of time, collaboration, openness to new ideas, and resilience (capacity to cope with adversity). School Counseling candidates should possess dispositions for openness to the ideas, opinions, beliefs, and lifestyles of others; maturity, self-confidence and a willingness to assume responsibility; and flexibility and resilience. Designated Subjects candidates should embrace expertise, demonstrate enthusiasm for their work, be empathic and strive for clarity. Candidates in both the Tier I and Tier II Educational Administration credential programs are expected to develop a personal code of ethics and model leadership capacity.
Candidate professional dispositions are assessed through multiple processes such as admission interviews, faculty/advisor relationships, self-assessments, university and/or school site supervisors’ field experience assessments, reflections in professional notebooks and portfolios, and exit surveys. For example, in the School Psychology program, candidates complete a Personal Competence Evaluation that includes items on the expectations for dispositions set by the program. Practicum supervisors also complete a fieldwork assessment that rates candidates on punctuality and effective use of time, reliability, and receptivity to new ideas. Candidates in the Dual Language Development program complete signature assignments that have embedded assessments of program expectations for candidate dispositions (curriculum audit, home and school events report, literacy assessment portfolio). Candidates in the Tier I and Tier II Educational Administration credential programs complete an exit survey that includes an item on professional dispositions (Table 1.48: Tier I & Tier II Administrative Services Credential Program Candidate Development on Two Dispositions).
During course work and fieldwork throughout their program, candidates in the initial and advanced teaching programs demonstrate their knowledge and skills related to planning for instruction, monitoring student learning, assessing student learning, and having a positive effect on learning for all students. Candidates are assessed through course work and field experiences utilizing case studies, student work samples or signature assignments, portfolios, and during culminating experiences (e.g., student teaching in initial teacher preparation programs, comprehensive examination or thesis in advanced [continuing] teacher preparation programs).
Candidates have numerous opportunities to study and implement essential teaching practices and behaviors that impact student learning through program course work and field experiences. These essential teaching practices include: planning for instruction; assessing, analyzing, and monitoring student learning; adjusting instruction to meet the needs of English learners, special needs, and otherwise challenging students; and supporting learning for all students. Data on candidate performance are culled from signature assignments in courses, Teaching Performance Assessment Tasks 1-4 (Exhibit 11), CSU Exit Surveys (Exhibit 6), and student teaching evaluations.
In the Multiple Subject Program, each of the five subject-specific methods courses incorporates instruction and practice in key teaching behaviors within the context of learning to teach the specific subject. Candidates thus have multiple study and practice experiences with a set of essential teaching behaviors that impact student learning by the time they begin student teaching. Faculty-developed assignments give candidates opportunities to practice tasks similar to those found in the Teaching Performance Assessment, and the program is in an implementation phase whereby all the CA-TPA Tasks (Exhibit 11) will become verbatim course-embedded activities.
Single Subject Program candidates engage in essential teaching practices and behaviors that impact student learning in four key courses during which they learn to monitor, assess and analyze student learning, plan instruction for all students, and adjust instruction for English learners, special needs, and otherwise challenging students. As embedded course assignments, each of the four Tasks of the Teaching Performance Assessment (Exhibit 11) provides both a teaching and an assessment tool for faculty in these courses.
Candidate performance on the four TPA Tasks is one measure of impact on student learning. Faculty assess candidate work on each of the four Tasks using the state-developed 4-point scoring rubric. Table 1.49 (Single Subject Program Aggregate Mean Scores on TPA Task 1-4, Spring 2006 [pilot])displays data on Single Subject program candidate performance from spring 2006, the pilot semester for data collection. Candidates scored a 3 or 4 at a 90 percent rate, indicating a substantial impact on student learning in those areas related to TPA activities. When the TPA becomes a high-stakes assessment for credentialing purposes in 2008, candidates will need to score 3 or above to pass each Task.
Another measure of candidate impact on student learning during student teaching comes from the formative and summative assessments on the student teaching evaluation form completed independently by university supervisors and cooperating teachers. Table 1.50 (University Supervisor Assessment of Multiple Subject Program Candidate Impact on Student Learning During Student Teaching in 5 Teaching Domains, 2005-2006)displays Multiple Subject Program candidate performance at the conclusion of student teaching in 2005-2006 in 5 teaching domains. The data indicate that student teachers make a strong impact on student learning, as perceived by their student teaching supervisors.
Table 1.51 (University Supervisor Assessment of Single Subject Program Candidate Impact on Student Learning During Student Teaching in 5 Teaching Domains, Spring 2006)displays Single Subject candidate performance during student teaching in spring 2006 on 5 performance indicators related to impact on student learning. Data show that student teaching supervisors perceive that candidates have a positive effect on their students during student teaching.
The initial credential programs each piloted projects during fall 2006 student teaching that had candidates conduct a unit of study in which they did a pre- and post-assessment of student learning. These pilot projects had a twofold purpose: (1) to provide the program with data on candidate impact on student learning during student teaching, and also (2) to provide the candidate with firsthand experience using assessment to improve instruction.
Prior to implementing their unit of study, candidates in the Multiple Subject Program did a pre-assessment of student knowledge. At the end of the unit of study, candidates conducted a post-assessment. Table 1.52 (Multiple Subject Program Student Teacher Impact on Student Learning During Student Teaching, Spring 2006)displays aggregated data of the pre- and post-assessment, and the aggregated gain pre to post. The students of 70 student teachers had an average score of 50% on the unit pre-assessment, and an average score on a post-unit assessment of 82%. The pre to post score gain of 32% suggested that these student teachers had a positive effect on their students’ learning through the teaching unit. Examples of candidate work are in the Exhibits Room.
The Single Subject Program piloted an assignment in student teaching in which candidates conducted a formal pre-assessment, taught a unit, and followed-up with a post-assessment. Actual assignments completed by candidates varied depending on the subject program, the content of the lesson, and the student teacher’s site context. Overall, there was clear evidence that students gained from the pre- to post-assessment. However, because the Single Subject Program comprises 10 different subject area programs, a common metric does not exist that would permit aggregating student scores. The Single Subject program will work with the unit's new assessment coordinator, beginning in fall, 2007, to develop a system for converting assessments to a common metric that will permit aggregation across all program areas. Examples of candidate work on this assignment are available in the Exhibits Room.
In 2006-2007, the Education Specialist Program is piloting a project to investigate candidate impact on students (N = 7 interns). In advanced fieldwork (EDSP 587/588), candidates are required to write Individualized Education Plans (IEPs) with a focus on student goals and objectives, and based on students’ current levels of performance. Candidates are also required to teach lessons to meet the goals and collect data to monitor student progress toward IEP objectives. At the end of the fall 2006 semester faculty dipsticked the data to monitor the project. Copies of candidate-written IEPs from two of their students (youngest and oldest) and the data to demonstrate student progress toward the IEP objectives in three areas—literacy, math, and behavior—were collected. Of the total 22 objectives across all areas, 64% were met. Candidates met 100% of their behavior objectives, 83% of their math objectives, and 53% of their literacy objectives. At the conclusion of the 06-07 pilot year, faculty will analyze the full dataset, with two primary guiding questions: How well do candidates write IEP goals? And, What directions are indicated for adjustments to content coverage in program courses?
Another source of data on candidate impact on student learning is the CSU Exit Survey (Exhibit 6). Data reported in Table 1.53 (Initial Program Candidate Perceptions of How Well the Program Prepared Them to Promote Learning for All Students on 11 Teaching Performance Indicators, 2004-2006)show that most candidates across the three initial programs felt adequately or well prepared to use informal and formal assessment methods, monitor student progress, and support student learning by the time they completed their preparation programs.
Follow-up surveys from the CSU Survey of Graduates (Exhibit 7) and Survey of Graduates’ Employment Supervisors (Exhibit 8) provide data on initial program graduates’ performance on a variety of indicators. Table 1.54 (Perceptions of Preparation on 6 Measures of Impact on Student Learning by Initial Credential Program Graduates and Ratings on the Same 6 Measures by Their Employment Supervisors, 2003-2005)shows data on 6 teaching performance indicators related to impact on student learning. In general, the data indicate that employment supervisors rated impact on student learning on these six dimensions greater than did graduates themselves and that graduates did somewhat better in their own eyes and in those of their supervisors compared to graduates of the CSU system.
An interesting comparison can be made between candidates’ perceptions of their impact on student learning and those of graduates looking back at their time in the preparation programs. Table 1.55 (Comparison of Initial Program Candidate and Program Graduates’ Perceptions of How Well the Program Prepared Them to Promote Learning for All Students on 5 Teaching Performance Indicators) compares candidates doing their student teaching during 2004-05 with graduates in their first year of teaching during that same period on five dimensions of teaching. For all three initial preparation programs on each dimension, candidates perceived they were better prepared on these dimensions than did graduates at the end of their first year of teaching.
Candidates in advanced (continuing) teacher preparation programs demonstrate the ability to assess and analyze student learning, monitor student learning, adjust instruction and support learning for all students through coursework and practicum experiences utilizing case studies, projects, signature assignments, and professional portfolios.
One of the more exciting activities recently is the action research project in the Curriculum and Instruction Master’s degree program (elementary and secondary). Utilizing the School Improvement Plan (SIP), candidates meet with their site administrators to outline primary needs across the school and district. Candidates select one of the designated needs to address with an action research project. Candidates work in teams under the supervision of a faculty member to improve student learning in their schools. Examples of action research projects include developing or improving parent involvement programs and integrating math problem solving across the curriculum. Faculty expertise is matched to candidate interests and action project topics. The projects are written and submitted as a culminating educational experience. A colloquium at the end of the academic year provides a means by which candidates share their projects with the broader educational community. In addition, candidates share the results of their action research projects at their school sites with their colleagues and supervisors. For example, candidates may share their results with teachers at their grade level meetings and/or at school wide faculty meetings.
The Dual Language Development program also uses an action research project as its culminating experience. Many candidates are interested in documenting how the primary language (L1) can be effectively used within an English-only program or how programs designed for native speakers of English function for English learners. Candidates have designed quasi-experimental studies in which student progress under different conditions (with and without the designed treatment) was carefully documented. In 12 of the 15 completed studies, increased student performance was documented. For example, one participant investigated the impact of preview/review in L1 on students’ reading comprehension in English in her third grade classroom. In the remaining 3 studies, the research question did not call for an intervention to be measured; however, in these studies, participants designed data collection measures or materials that have the potential to impact student performance.
Beginning spring 2007, Early Childhood Education candidates will learn to use a variety of assessment instruments with children and use at least two different tools to assess children’s development and learning in infant/toddler and preschool programs through EDEC 520. Also in spring 2007, as part of its evaluation plan, the program will conduct a parent evaluation of a randomly selected group of graduates. The survey with parents will include items that refer to graduates’ impact on children’s academic learning and socio-emotional skills as well as on building home-school relationships.
The knowledge, skills, and dispositions to support the learning of all students that are embedded in the Conceptual Framework (Exhibit 5) form the basis for learning experiences and assessment in programs for the preparation of other professional school personnel.Candidates in programs that prepare other professional school personnel such as reading specialists, educational administrators, school counselors, school psychologists, speech therapists and audiologists, and librarians engage in coursework and field experiences that emphasize establishing educational environments that support student learning. They demonstrate an understanding of the developmental levels of P-12 students, are knowledgeable of the diversity of students, families, and communities in today’s schools, are familiar with the social, cultural, and policy contexts in which they practice, and show the ability to collaborate with teachers and other school personnel in the creation of positive learning environments that lead to success for all students.
All course work and fieldwork experiences in the Educational Administration Tier I and Tier II programs are guided by state program standards which have embedded in them the knowledge, skills, and dispositions that provide candidates with the developmental and conceptual knowledge needed to make a positive impact on the learning environment of the schools they’ll lead as administrators. Candidates in the Tier I program complete significant field experiences through ten hours of field experience in each methods course and through two field experience courses (100 hours in EDAD 580 and two full weeks in EDAD 680), during which they help operationalize pre-existing school programs and plans. With the consent of the site supervisor, a candidate may enhance an existing program or develop a project to address an identified need at the site or in the school district. Evidence of achievement is documented by the candidate through artifacts, self-reflection, and feedback from the district and university supervisor. Additionally, through the culminating research study or thesis, a candidate examines, researches, and makes recommendations to address school or systemic questions and concerns.
Examples of data reporting impact on student/client learning from exiting candidates is displayed in Table 1.56 (Tier II Professional Administrative Services Credential Exiting Candidates’ Perceptions of Their Preparation to Impact Student Learning), Table 1.57 (Impact on Student Learning by Candidates in the Reading/Language Arts Program, 2000-2005), and Table 1.58 (School Psychology Credential Program Candidate Perceptions of Their Preparation to Impact Student Learning, 2002-2004).
The School Psychology program has gone further than any advanced program in the unit to measure candidate impact on student learning, and may provide a model for other programs if their methodology is transferable to other contexts. To measure the impact of School Psychology program candidate skills and knowledge on the academic and behavioral outcomes of school-age children, three case study assignments from three different required courses in the school psychology program (EDP 527, 579, and 560) were examined. All three assignments required the candidate to collect baseline data, implement an intervention, collect and graph weekly progress monitoring data, and make data-based decisions regarding the efficacy of the implemented intervention.
Case study data were entered into an Excel spreadsheet programmed to compute the effect size of each single-subject case. The following formula was used to calculate the effect size: mean of the baseline data subtracted from the mean of the intervention data, divided by the standard deviation of the baseline data. Data are depicted in Table 1.59 (Effect Size Calculations of School Psychology Program Candidates Impact on School-Age Children). Effect sizes were calculated for each target behavior in a case study. Seven effect size calculations indicated a large effect (>.80), three revealed small to moderate effects (<.39), and three were calculated in the negative range (<-0.2).
These and future results will be used to examine differential intervention effects between second-year (e.g., EDP 560 and 579) and third-year (e.g., EDP 527) candidates, and efficacy differences in implementing academic vs. behavioral interventions. Missing data from EDP 560 and EDP 579 case studies precludes answering these questions. Thus far, course changes based on these results include requiring candidates to submit raw data in table format, in addition to a graph, at the end of each semester, and instructors submitting case study data to the program coordinator at the end of each semester.
Programs for other school personnel prepare a variety of school professionals, who play a variety of roles in schools and school districts once they leave our programs. Establishing valid measures of graduates’ impact on P-12 student learning is a challenging task that programs approach in different ways. Table 1.60 (Impact on Student Learning by Graduates of Programs for Other Professional School Personnel) illustrates how programs collect evidence of the impact their graduates have on P-12 students. We continue to address this area through meetings of the Graduate Programs Committee. In addition, in spring 2007 we are having a series of presentations at monthly unit meetings on the topic of candidate and graduate impact on students and clients during which programs present their projects to their colleagues. Still, it remains one of our greatest challenges to develop or utilize existing valid tools for measuring impact.