Understanding and Assessing College Readiness Shannon Wells, Ph.D. Mitchell Aulakh
College Readiness • The problem: – Students are sent off to college but must take several remedial courses in order to prepare themselves for rigorous college level work
• What is college readiness? – “When students have the academic and interpersonal knowledge, skills, and behaviors to enter and complete a college course of study without remediation.” 2
Measuring “Readiness” • What truly constitutes “ready”? – Some colleges differ • CSUs have a standard set, but UC and private colleges differ, and out-of-state colleges even more so
• What is appropriate to measure/does it actually indicate readiness? – Are CSTs usable? SAT, AP, ACT, EAP? – What data sources are available? • Are subgroups available?
– College going and success rates probably best • Not collected anymore 3
What is Appropriate to Measure? • Criteria to consider – Readily available • College success rates would be great to have!
– Easily understandable • No complicated index
– Corresponds to college readiness • Is the measure rigorous enough? – For example, is passing CAHSEE a measure of college readiness?
4
Utilizing CSTs • Pros – More familiar – Is correlated to EAP • Can derive specific cutpoints
– Readily available
• Cons – Alignment between standards and college preparedness is off • Addressed with Common Core
– Not used by colleges
Other considerations: • Proficient isn’t enough, should look at advanced as a minimum
5
What is the EAP? • EAP: Early Assessment Program – Partnership between CSU, CDE, and SBE to establish a voluntary measure of college readiness for 11th graders – Goal: To have California high school graduates enter CSU and CCC fully prepared to begin college-level English and math and to reduce the remediation rates
The Need for the EAP: A Disconnect • Discrepancy between what is taught in the public school system versus the expectations of colleges – Apply and accepted to colleges, but enter without the necessary skills to be successful • Greater costs associated with remediation both student and colleges
– Lower chances for graduating college on-time
• Common Core – EAP should continue in some form
The Benefits of EAP • Provides an early signal It tells students, families, and high schools whether student is prepared for college-level English and math • Alerts students about needed actions Informs students about what they can do to achieve readiness in English and/or math before entering CSU • Enables exemptions It allows students to earn an exemption from CSU English Placement Test (EPT) and/or Entry Level Mathematics (ELM)
Who Accepts the EAP? • All CSUs • California Community Colleges – On a voluntary basis for each community college – Dependent upon each community college which subject areas are accepted – Full list of CCCs participating available at: http://extranet.cccco.edu/Divisions/StudentServices/EAP/Accepting EAP.aspx
• Bubble in option to share results with CSU and CCCs
Community College Participation English EnglishConditional
Yes
No
Under Review
65
6
7
10
CCC Accepting EAP Results
A mechanism for accepting this result is under development. Local campuses will make the determination.
Mathematics
58
7
12
MathematicsConditional
12
40
23
Current as of 8/7/2012. Full list available at: http://extranet.cccco.edu/Divisions/StudentServices/EAP/AcceptingEAP.aspx
EAP Test Overview EAP is a free test administered with the Math and English California Standards Test (CST) in the junior year. MATHEMATICS
ENGLISH
75% of CST (45 questions)
75% of CST (45 questions)
15
15
None
Essay Prompt (March 131)
Eligibility
Algebra II class or higher
None
CSU Exemption for
Entry Level Mathematics (ELM)
English Placement Test (EPT)
CST Questions Multiple Choice Quest. (CST in April/May)
Additional assessment
EAP Possible Student Results English
Mathematics
Ready for CSU and Community College Level English
Ready for CSU and Community College Level Math
Ready for CSU and Community College Level English-Conditional
Ready for CSU and Community College Level Math-Conditional
Not Yet Demonstrating CSU and Community College Level English
Not Yet Demonstrating CSU and Community College Level Math
Ready for CSU college-level Mathematic coursesConditional (Conditionally Exempt) Take a fourth year of math with an Algebra II prerequisite Any course accepted by the CSU Mathematics Facility Validation Committee Trigonometry & Math Analysis Pre-Calculus Calculus AP Calculus AB or BC AP Statistics AP Physics Supervised e-learning Course Community College Course
Student must receive a grade of C or better, submit transcript to CSU to get exempt from ELM
Ready for CSU and CCC college-level English courses-Conditional (Conditionally Exempt) Take any of the following in senior year: Expository Reading and Writing Course (ERWC) Advanced Placement (AP) English International Baccalaureate (IB) English English courses approved for extra honors credit by the University of California (so indicated on the UC Doorways “a-g” course list with a star) Student must receive a grade of C or better, submit transcript to CSU to get exempt from EPT
Still not Ready? • Early Start is required for incoming students who have not fulfilled the Entry Level Math and/or English Placement Test proficiency requirements. • The program takes place the summer before the freshman year. • Upon admission, CSU campuses will inform students how and where to sign up for Early Start. • Early Start math and English courses are available at every CSU campus, and online.
Apples to Oranges: The CST and EPT/ELM • What are some of the differences between the expectations/goals of the EAP and CST? • EAP as an indicator for the EPT/ELM – Used for placement, whereas CST is not – Foundation skills are an area of focus, whereas earlier grade level CSTs cover foundation skills Test 11th ELA CST EPT Alg II CST Summ CST ELM
Item Count 75 90 65 65 50
Essay No Yes N/A N/A N/A
Placement? No Yes No No Yes
Focus on English •
Students will be asked to analyze the ideas presented in brief passages (typically 100-150 words). Students should be able to read critically in order to: – identify important ideas – understand direct statements – draw inferences and conclusions – detect underlying assumptions – recognize word meanings in context – respond to tone and connotation Students may be asked to select the answer choice that best summarizes a passage, explains the purpose of a passage, focuses on a specific detail, explains a word in context, compares/contrasts two aspects of a passage, explains the implications or suggestions made in a passage, identifies causal relationships, etc.
Appendix A
Released Test Questions for CSTs 8RW1.3 Vocabulary and Concept Development: Use word meanings within the appropriate context and show ability to verify those meanings by definition, restatement, example, comparison, or contrast.
Focus on Math
ELM Sample Questions
Comparing ELM Samples to CST RTQs Alg5.0 Students solve multistep problems, including word problems, involving linear equations and linear inequalities in one variable and provide justification for each step.
AlgII2.0 Students solve systems of linear equations and inequalities (in two or three variables) by substitution, with graphs, or with matrices.
GE15.0 Students use the Pythagorean theorem to determine distance and find missing lengths of sides of right triangles.
ELM Sample ELM Sample
ELM Sample
Resources for EAP • CSU EAP Page: http://www.calstate.edu/eap/ • CCC Chancellor’s Office: http://extranet.cccco.edu/Divisions/StudentServices/EA P.aspx • Joint CSU & CCC: http://www.collegeeap.org • ETS site for EPT/ELM (EPT/ELM Resources): http://www.ets.org/csu/about • The English Success Website: www.csuenglishsuccess.org • The Math Success Website: www.csumathsuccess.org • 11th Grade Portal: www.csusuccess.org/juniors • ETS Data Site: http://eap2012.ets.org
Resources for EAP Continued • Focus on English Website: www.calstate.edu/sas/documents/focusonenglish.pdf • Focus on Mathematics Website: www.castate.edu/sas/documents/focusonmathematics.pdf • ERWC On-Line Community: http://writing.csusuccess.org • ERWC On-Line Community Registration: http://writing.csusuccess.org/community • CSU Proficiency: http://www.asd.calstate.edu/proformancy/profiency.shtml • Educational Results Partnership: http://edresults.org/
Historical trends and connections
EXAMINING EAP DATA
26
Notes about EAP Data • “Voluntary Measure” – Students take the EAP if they choose • Can release scores
– Plenty of room for influence
• Rates are always calculated based off eligibility or participation – Participation for math is based on only students that are eligible for the program – Ready & conditional rates are calculated based on the pool of EAP test takers
EAP English Statewide Test Results Year
Participation
College Ready
College ReadyConditional
2006 312,167 (75%) 48,072 (15%) 2007 342,348 (78%) 55,206 (16%)
2008 352,943 (79%) 60,392 (17%) 2009 366,925 (82%) 59,367 (16%) 2010 378,870 (84%) 77,826 (21%)
2011 383,060 (86%) 85,506 (23%) 2012 383,565 (87%) 87,086 (23%)
58,507 (15%) 28
EAP Mathematics Statewide Test Results Year
Participation
College ReadyCollege Ready Conditional
2006 137,067 (74%) 16,120 (12%)
58,822 (43%)
2007 141,648 (70%) 17,173 (12%)
60,697 (43%)
2008 147,885 (70%) 19,442 (13%)
62,660 (42%)
2009 169,473 (77%) 22,246 (13%)
74,464 (44%)
2010 178,667 (77%) 26,056 (15%)
75,502 (42%)
2011 190,946 (80%) 29,524 (15%)
81,849 (43%)
2012 203,972 (83%) 30,430 (15%)
92,850 (46%) 29
EAP to CST Results – Cutpoints
What do students need to score on the CST in order to score “ready” on the EAP? EAP Ready
Advanced
Proficient
Basic
Below Basic
RS
%
SS
RS
%
SS
RS
%
SS
RS
%
SS
RS
%
SS
66
88%
420
62
83%
398
52
69%
351
39
52%
303
27
36%
260
Algebra II
54
83%
420
53
82%
419
41
63%
350
31
48%
303
22
34%
259
Summative
60
92%
445
59
91%
430
49
75%
352
39
60%
304
23
35%
235
11th
ELA
EAP Continues with CCSS & SBAC
31
• If tighter alignment is achieved, it is possible that students will no longer need to volunteer to complete supplemental items. Instead, CSU will be able to determine the level of college readiness by consideration of scores on the SBAC assessment without augmented items. • The EAP will continue to provide an early signal of college readiness to rising high school seniors, allowing their better and more productive use of their senior year to gain proficiency if needed.
Diagnose, Prescribe, Treat, Follow Up
EAP DIAGNOSTIC PROCESS
Diagnose Use test to determine weaknesses
Follow Up Use Diagnostic
Prescribe Create data-driven lessons
Treatment Teach data-driven lessons
Start with the Key Questions • What do we want them to learn/know? • How will we know if they learned it? – If they did not learn it, why?
• What will we do if they did not learn it?
What do We Want them to Learn? EAP Standards and Skills indicated on EAP Blueprints and EPT/ELM
• What standards are tested for the CST? – What standards/skills are needed for the EAP based on EPT and ELM?
• What standards are not tested? • Use Released Test Questions to gauge rigor. – CST, EPT, ELM
How will We Know if They Learned It? • EAP Diagnostic in 11th grade before CST – Determine readiness of 11th graders 6 to 8 weeks out from testing – Use for remediation to better prepare students for the augmented standards – Continually monitor student progress towards mastery
EAP Diagnostic Overview Test Grade 11 ELA EAP Diagnostic Algebra II EAP Diagnostic Summative Math EAP Diagnostic
CST % of # of Items Items CST Standards 42
75
56%
12
48
65
74%
17
47
65
72%
32
*Note: Essay for EAP English not included.
If They Did not Learn it, Why? • Results from assessments to determine gaps in student understanding – Heavily-weighted, heavily-missed standards level analysis
– “Error analysis”
UTILIZING EAP DIAGNOSTICS
39
Select List Assessments
41
Filter for Test
Select Assessment
Print Test Booklet
Print Answer Sheets
Accessing Results Through Illuminate
46
Accessing Results Through Illuminate
Filter for Assessment
Select Assessment
Download to PDF, Excel, or View in Browser
View Results – Sort by Standard
View Results – Sort by % Correct
52
53
54
55
Questions?
[email protected] [email protected]
56
