Section 1: Description of the Model
Model Practices Description. Teachers can gauge student learning through the questions asked during formative assessments and the responses given by the students. We call this process the Formative Questioning Practice. Generally speaking, we have found that spending time to pre-write questions and answers focused on word choice and question sequencing will help improve the rigor of the questions, and better help teachers monitor student responses to determine whether they are on track to meet the lesson objective and expectations of the standards. By keeping these practices in mind, the teachers at Alliance Marc and Eva Stern Math and Science School (Stern MASS), have adopted a school-wide questioning practice using the following steps:
Addressing the Need. Stern MASS became an early adopter of the Common Core State Standards (CCSS) in the 2011-12 school year. As school leaders and lead teachers analyzed the standards and reflected on classroom instruction, we realized that classroom practices needed to change in order for students to master CCSS. State testing data based on the Early Assessment Program (EAP) 2013 English/Language Arts (ELA) indicated 69% of students were not college-ready. Students taking the SAT writing had an average scaled score of 441, which was 47 points lower than the 488 national average. From 2012-16, professional development workshops focused on a school-wide writing practice through the Claim-Evidence-Link (CEL) paragraph structure. Through our focus, CAASPP results demonstrated an increase of 8% in ELA from the 2015 assessment (71%) to 2016 (79%) assessment.
As a school, we realized that in addition to strong writing skills, we needed to find opportunities for students to conduct more of the cognitive lift within classroom instruction so that when they had to write their CELs, they already had formative practice opportunities. CAASPP data already proved that students were performing well in ELA but we needed strategies that could also be easily applied to math classrooms since CAASPP Math data was lower than our ELA performance at 31% (2015) and 34% (2016). We also wanted to take this opportunity to find instructional strategies that could apply to all teachers in various content areas and varying levels of experience.
Essential Resources. Beginning in Spring 2016, the school principal was introduced to instructional practices used by Uncommon Schools, a network of charter schools in New Jersey, New York and Massachusetts. The network analyzed and created strong instructional practices focused around classroom management and instructional rigor that could be coached to beginning teachers and help them become better teachers within a short period of time. Stern MASS administrators and lead instructional teachers began training in 2016-17 and would coach teachers individually based on informal and formal classroom observations. Beginning in 2017-18, a Professional Development (PD) Team with a representative from each department was formed and charged with providing monthly professional development to the teachers. The team learned the instructional practices through monthly, full day train-the-trainer workshops provided by the school principal. The team would then lead a 2-hour school-wide PD that was followed by a 2-hour content area department PD facilitated by the Principal and Assistant Principals for teachers to practice the learned strategies later that month. Materials used during the PDs include video clips, articles, and presentations produced by Relay Graduate School of Education, Alliance College-Ready Public Schools, and ASCD.
Teacher and Administrator Goals. To implement this practice, there were four main goals:
Students, Parents, and Community Goals. By focusing on increasing the instructional practices around questioning in the classroom, stakeholders should be able to notice the following:
Benefiting All Students. By having teachers focus on the formative questioning practices asked in class, teachers are better able to pre-plan “standards met” responses and gauge whether students are on track to meet the learning objective based on their answers. Teachers are then better able to provide better targeted feedback and address any misconceptions because they have already planned for error. Teachers can then readdress or reteach the learned misconception in the moment instead of waiting for the next class period. This helps students not to hold onto learned misconceptions but to correct their knowledge immediately and retain correct information.
Benefiting Unique Student Populations. The Formative Questioning Practice benefits all student subgroups because it holds students accountable to do the heavy lift within the learning process. Since teachers are writing questions based on the text, students do not need to have prior knowledge or the text or background information in order to be able to answer the questions. By having pre-planned answers, teachers can then frame their expectations of student responses while they are monitoring student learning throughout the class period. Teachers are purposefully monitoring all students by walking around the classroom listening to conversations and checking student work. A monitoring pathway allows teachers to check-in quickly with students who are performing well in class so that they can spend more time with struggling or at-risk students. Based on the monitoring, teachers can redirect the learning as needed.
Social-emotional and Behavioral Needs. This practice allows all students to be successful in their learning process because teachers can gauge student progress towards the learning objective. Teachers can provide support through the learning to build student self-confidence and strategically call on students based on their responses to the questions. By having students conduct more of the cognitive lift within the classroom, students are busy throughout the class period which will minimize off task behaviors.
LCAP. The incorporation of Formative Questioning Practices encompasses all of the state priorities addressed in the LCAP. The practice is incorporated within LCAP Goal 3: Increase student achievement through content area curriculum and instructional supports. Professional development was provided to all teachers and administrators (Priority 1) to provide training on how to effectively assess student learning (Priority 2) and provide a school-wide strategy students could use across the content areas and in post-secondary (Priorities 5, 7, and 8). Parent and community stakeholders had opportunities to provide input regarding the questioning practices through Meet Your Teacher and Counselor NIghts, School Coordinating Council, School Board of Directors Meetings, and Parent Newsletters (Priority 3). Student results from the CAASPP SBAC 2017 and 2018 results measured the effectiveness of this practice (Priorities 4 and 5).
Chronic Absenteeism and Suspensions. Stern MASS has been fortunate not to have a high suspension rate (1.5% reported on the California School Dashboard 2018) and a low chronic absenteeism rate. With the questioning process and integration of text-dependent questions, all students can be successful and be able to answer questions from the text despite being absent from school.
Section 2: Implementation and Monitoring of Model Program/Practice
Stakeholder Engagement. When the administrative team decided to focus on a school-wide professional development approach towards formative questioning practices beginning the 2017-18 school year, it was important to get buy-in from the teachers. Teacher practice would be shifting from the teachers being facilitators of knowledge to becoming better monitors of student learning. In Spring 2017, the principal invited one teacher from the core content areas to be part of the PD Team and then invited a teacher representative from the remaining content area departments based on the suggestions of the team. This group collectively brainstormed areas of growth based on their individual practices and by looking at teacher evaluation data around questioning and feedback practices. In Summer 2017, a scope and sequence was created to address certain instructional strategies throughout the year. This was revisited each month during PD Team Planning Sessions. Teachers had also opportunities to provide feedback at the end of each semester to ensure the learning was applicable and being integrated into their instructional practice. The same feedback and monitor process was conducted for the planning and execution of the 2018-19 school year.
Parents have opportunities to be introduced to and learn more about the Formative Questioning Practices during our three annual Meet Your Teacher and Counselor Nights. These practices have also been discussed with parents during School Coordinating Council Meetings or in Parent Newsletters. Members of the School Board of Directors have engaged in discussions regarding these practices through quarterly LCAP discussions.
Methods to Communicate Practice. The PD Team meets monthly to plan a full-staff professional development workshop for our Friday PD that explore one or two specific Formative Questioning Practices. Topics have been selected based on classroom observations and have included topics such as standards-objective alignment, planning with the end in mind, Intellectual Prep Protocols, turn and talks, purposeful monitoring, Read-Write-Discuss(+Revise), Show Call, not Rounding Up, and text-dependent questions. Each month, members of the PD Team work to understand and internalize the area of focus, then design an interactive professional development session for all teachers using the train-the-trainer model. During these PD sessions, the PD Team members introduce and model the instructional strategy and provide teachers with the opportunity to learn about, clarify their understanding, and begin thinking about how the strategy fits into their own practice. Teachers then meet in content area department PD to practice the learned topic, watch video exemplars, practice the strategies, and provide colleague feedback for continual growth. When teachers implement learned practices in their classroom, supporting administrators and instructional coaches provide individualized feedback through observation debrief coaching sessions of the implementation and discuss additional steps to strengthen the teaching practice. Throughout the semester, teachers provide feedback about the efficacy of the PD Team-led sessions through surveys, discussions with the Instructional Leads, discussions with the principal or their supporting administrator, and department meetings with their colleagues.
Monitoring and Assessment Methods of Parents. To ensure that parents are kept informed of our school practices, the school is able to evaluate our effectiveness through informal feedback conversations after parent workshops. Parents also complete an end of year School Satisfaction Survey. One of the survey questions asked whether “this school meets the individual academic needs of my child.” Since the school has focused on the Formative Questioning Practices, the average score has increased from 3.44 (2017) to 3.51 (2018) on a Likert scale of 1-4 with 4 being “strongly agree”.
Capacity Building through Professional Learning. Since the school has focused on Formative Questioning Practices, this has become an opportunity for teachers to become instructional leaders. The PD Team is comprised of one teacher from each of the following content areas: ELA, Math, Science, Social Science, Spanish, Electives (representing Physical Education and Visual and Performing Arts), and Special Education. These teacher leaders work with school administrators to plan monthly professional development sessions for staff-wide PD. This helps to build capacity within the teacher community for exemplifying and modeling these practices, and also leverages the train-the-trainer model, which maximizes an opportunity to further build capacity within the staff. In addition to monthly PD Team Professional Development sessions, our Principal, with the support of the Assistant Principals, facilitates a professional development session for each individual department, wherein teachers further explore and apply the Formative Questioning Practices to their specific content area and lesson plans.
Monitoring Instructional Learning Activities. Members of the PD team also serve as Instructional Leads. As Instructional Leads, teachers work with administrators to review and analyze interim assessment data, which measures student growth toward mastering content area standards. This data informs our school-wide Data Days, pupil-free days where teachers have the opportunity to analyze student achievement data and plan for how to address gaps in student progress or understanding. These teacher leaders can thus bridge the work that is done in professional development with the PD Team with ongoing opportunities for data analysis and reflection during Data Days.
Section 3: Results of the Model Program/Practice
Evaluating Practices Based on Student Outcomes. By focusing on Formative Questioning Strategies, growth has been observed and measured by teacher and student performance metrics. Highly Effective and Master Teachers were able to demonstrate instructional leadership opportunities through planning and facilitating professional development. Their practices were shown during PD since these workshops were held as classroom demonstration lessons. Teachers also become more comfortable with showing their teaching practices to colleagues through a Practice Perfect demonstration lessons and received feedback for growth through sentence frames of “It was effective when...” and “Next time try....” Teachers then had an opportunity to redo specific parts of the lesson based on the feedback given. In addition, teacher evaluation levels increased from Fall 2017 to Fall 2018. The average for specific indicators on the Grow Teacher Evaluation Rubric related to the Formative Questioning Practices increased to above a level 3 (Effective) practice and average increased ranged from 0.27-0.47 points on a 4-point rubric. Fall 2018 observation indicator averages are 3.23 (challenging questions), 3.23 (scholar-to-scholar academic discourse), 3.03 (feedback for scholar growth), and 3.37 (assess and respond to scholar’s thinking).
The positive teacher growth directly affects student achievement levels. Internal grade level interim assessment results using SBAC-aligned questions have demonstrated positive growth school-wide. The percent of questions correct have increased by comparing ELA and Math performance from the 2017-18 to 2018-19 demonstrate positive growth school-wide. The percent of questions correct showed up to 8% growth in ELA and 18% growth in math when comparing grade level results from Interim Assessments #1 (percent correct ranged from 40-59% in ELA and 47-50% in Math) and Interim Assessments #2 (percent correct ranged from 47-54% in ELA and 38-49% in Math). ICA data from grade 11 administrations also showed positive results from the 2017-18 to 2018-19 school years. Both years showed 75% of students meeting or exceeding standards in ELA. ICA Math results showed a 8% growth from 42% in 2017-18 to 50% in 2018-19. Additionally, overall CAASPP results had demonstrated positive trajectory from prior to starting our instructional focus on questioning practices. Before we started our individual and school-wide focus on questioning strategies, CAASPP 2015 results were 71% in ELA and 31% in Math. Based on the coaching from the school administrators and the school-wide professional development, CAASPP 2018 results showed positive increases by 9% in ELA (80% of students meeting or exceeding standards) and 10% in Math (41% of students meeting or exceeding standards). This helped result in the first year of release in ELA and Math performance indicators on the California School Dashboard 2018 of Blue and Green, respectively.
Non-Targeted Student Groups. Internal interim assessment results have also demonstrated positive growth for our English Learners and Students with Disabilities. When comparing the percent of questions correct on the interims for English Learners, there was up to a 31% growth in ELA and 42% growth in math when comparing grade level results from Interim Assessments #1 (percent correct ranged from 28-42% in ELA and 33-42% in Math) and Interim Assessments #2 (percent correct ranged from 39-40% in ELA and 33-40% in Math). When comparing the percent of questions correct on the interims for Students with Disabilities, there was up to a 23% growth in ELA and 38% growth in math when comparing grade level results from Interim Assessments #1 (percent correct ranged from 27-43% in ELA and 33-38% in Math) and Interim Assessments #2 (percent correct ranged from 37-39% in ELA and 28-33% in Math). The population of English Learners and Students with Disabilities who tested in the grade 11 CAASPP 2017 and 2018 administrations were less than 10 students so data was not released due to student privacy.
Continuous Program Improvement. With a mission to ensure all Stern MASS students have access to a rigorous STEM and college-preparatory education, we are constantly looking to build on our successes and refine our practices to further support student achievement. So far, our focus on Formative Questioning Practices has primarily focused around the alignment between lesson objectives and content area standards. This has helped with internal interim data and CAASPP performance demonstrating positive performance increases. However, we will continue to use ongoing student achievement data to inform how we will modify and refine our questioning practices from year to year to target the specific needs of our students in all subgroups. We aspire to have at least 90% of students meeting college-ready benchmarks based on the CAASPP, SAT, ACT, or AP exam metrics by the 2027-28 school year in both ELA and Math. With this in mind, the PD Team will continue to collaborate with school administrators to reflect and create a long term plan for how to move forward to best increase student achievement.