On Track for a Bureaucratic World Beyond the Documentation
On Track for a Bureaucratic World
Welcome
Find the latest version of this presentation, plus related materials, at https://goo.gl/ppZJHO.
Your presenter: Casey Mirch, some data geek at Leadership Public Schools, a charter school management organization with 4 high schools in the San Francisco Bay Area.
[email protected]
On Track for a Bureaucratic World
Welcome
Who is the intended audience? § If you haven’t read the help or attended an introductory On Track session, it’s probably best to do that first. § It’s assumed you can create the Custom Reports necessary for your On Track, or you know someone who can. § If you’ve tried to create an On Track and had trouble translating what you need into On Track, or you got wacky results, I hope this might clarify things somewhat.
Principles What is On Track, Essentially?
What is On Track, Essentially?
Use Cases On Track is a way to convert categorical data about students into scores on interval scales of discrete values, nightly or on demand, all within Illuminate.
§ Categorical: Informational content bottlenecks at whether a student is or is not returned by a contributing Custom Report. § Evaluation can be restricted to a student cohort definable with the student visibility form.
What is On Track, Essentially?
Use Cases On Track is a way to convert categorical data about students into scores on interval scales of discrete values, nightly or on demand, all within Illuminate.
§ Category scores, which may be useful in their own right, are combined for an overall score. § Only interval scales with a modestly small number of values are practical to set up. § A yes/no binary scale is also an interval scale.
What is On Track, Essentially?
Use Cases On Track is a way to convert categorical data about students into scores on interval scales of discrete values, nightly or on demand, all within Illuminate.
§ If you want a snapshot (like with a Student Group), that’s the default. § You can’t get up-to-the-moment status as in most Custom Reports, but you can schedule nightly updates. § Only provides state at last execution; there’s no history.
What is On Track, Essentially?
Use Cases On Track is a way to convert categorical data about students into scores on interval scales of discrete values, nightly or on demand, all within Illuminate.
§ It’s not much use if you don’t have the necessary data in Illuminate. § You can always put arbitrary data into a Summary Assessment. § On Track scores are available in Custom Reports. (Ergo: scores from one On Track can contribute to another On Track.)
Principles Scoring
Principles: Scoring
Scoring is Subtractive § Unless you subtract points for students not in some report, students in the evaluation cohort who don’t appear in any report get full points. § You can set up your Performance Bands so that mastery or desirable bands have the low cut points. § If you want the default score to be zero, you can create a measure that subtracts 100% of the category points for all students in some Custom Report that has no filters. § Order matters. Measures get evaluated in the order listed. Scoring stops at 0% and 100%, even for intermediate evaluation steps.
Principles: Scoring
Scoring is Subtractive Applicable In the Custom Calculations for a Report Measure Not what it seems! + Some %
- Some %
Not in the Custom Report Operates as represented, for +100%. But if there’s no other measure in the category, students in the report also have 100%. Partial percentages only apply to students who are affected by subtractive measures in the category.
Unless subtracted by another measure, students not in the report effectively get +100%. Consider subtracting points for students not in a report as an alternative. This option can add back points subtracted by prior measures. Operates as represented. Operates as represented.
Principles: Scoring
Category Scores § Percentage weights of your Categories don’t have to add up to 100% § For example, alongside other Categories, you can create a Category worth 100% for a factor that automatically results in a top score. § Category weights that add up to less than 100% effectively leave unused “scoring space” at the top of your Performance Band Set.
Principles: Scoring
Performance Bands § The overall maximum score of an On Track doesn’t have to be 100, but… § Performance Bands used in an On Track need to have percentage-oriented cut points. § If your On Track has a maximum score of 4, straightforward cut points for your Performance Bands would be 0, 25, 50, 75, and 100. § That’s different from how Performance Bands work in Custom Reports.
§ Scores are only calculated to integer precision.
Principles: Scoring
Using Scores § To get On Track scores in Custom Reports, use the Metadata link on the Nav Bar. § There are 5 fields overall, and for each Category:
Mastered? [T/F], Max Score [possible], Performance Band [name], Performance Band Level [numeric], and Score.
§ Decide which fields provide meaningful data and how to name them. Consider disabling the rest. § Enable, disable, and rename On Track fields via Reports > Manage Custom Reporting Data. § The 3 Prebuilt reports for On Track are useful. On-Track Student List, On-Track Multiple Configuration Report, On-Track Student Detail Report
Examples Putting it all together
Examples
Student Attributes § This provides one-to-one student data not otherwise available in Custom Reports. § Nightly calculation automatically keeps data reasonably up-to-date. § The overall score isn’t meaningful in this On Track, just the Category scores. § FRL status is calculated as a score of 0 (free), 1 (reduced), or 2 (full price). § CAHSEE pass status is calculated in two other Categories.
Examples
Student Attributes
Examples
Student Attributes
Examples
Title I Part A § This calculates student eligibility for Title I Part A Basic Targeted funding. § Higher academic achievement is reflected as higher scores; program/funding eligibility is reflected by lower scores. § A Category worth 100% enforces age limits by giving the full score to students outside ages 5-17. § Academic measures and cut points are locally defined. § Categories return one point each where student isn’t eligible based on that measure.
Examples
Title I Part A
Examples
Title I Part A
Examples
EPT/ELM for CSU/CCC § This calculates students’ status (Ready, Conditionally Ready, or Not Demonstrating Readiness) with respect to California State University System’s English Placement Test or Entry Level Math. § Uses scores of CAASPP EAP, ACT, SAT, and AP tests in appropriate ranges. § Score in one On Track is used as a measure on a second On Track to verify that Conditionally Ready 12th graders are enrolled in the eligible courses they need.
Examples
EPT/ELM for CSU/CCC
Examples
EPT/ELM for CSU/CCC
Examples
State Seal of Biliteracy § The Seal of Biliteracy recognizes California high school graduates who demonstrate fluency in English and at least one other language. § English fluency must be demonstrated by GPA in ELA courses and by ELA CST (or interim equivalent). § CELDT-score or FEP requirements apply to nonEO students. § Non-English fluency can be demonstrated by 4year language course GPA or by scores on relevant AP, IB, or SAT Subject tests.
Examples
State Seal of Biliteracy
Examples
State Seal of Biliteracy
On Track for a Bureaucratic World
Thank You! Find the latest version of this presentation, plus sample macros and related materials, at https://goo.gl/ppZJHO. If you found this helpful, advocate with your Implementation Manager to make On Track even better, with things like: • Improved documentation about how On Track performs calculations • Long-form description fields for all levels of an On Track configuration • Support for submitting On Track configurations to, and installing them from, ZE Library Or, suggest other improvements by submitting a Feature Request.
