Understanding Your Core Data

---

Data Integration

Overview •  How does data get into Illuminate: §  Tab-delimited text files in required datasets are imported into Illuminate via manual upload or nightly automation. • 

URL of Data Specifications: www.illuminateed.com/downloads

•  Responsibility of our clients §  Produce and send required datasets §  Analyze and Review “Core Data Import Logs” • 

URL of client roles and responsibilities: www.illuminateed.com/roles.html

Data Integration

Processes

Client SIS

Option I (Automation)

Option II (Manual Upload)

NonScheduled/Scheduled

Via UI

SFTP (File Transfer)

Illuminate

File Extract

Illuminate

What’sCoreData Core data is specific datasets required in order for clients to have a full functioning DnA system that allows the ability to create assessments and reports. These files are processed nightly via Illuminate Ed automation feature:

• 

Sites §  This is a list of all school buildings/ sites.

• 

Courses §  Includes entire course catalog. Course ID should be unique

• 

Terms

• 

Mastschd

§  Contains start and end dates per academic year/session/site

• 

• 

Studemo §  List of individual students and their “fixed” demographic fields (ie: Primary Race, Date of Birth, etc) Users §  Dataset usually contains only the teacher records.

§  Contains sections per academic year/ site/session. Please note: If elementary classes do not have sections, the record is not required • 

Roster §  Includes class schedules per student/ academic year/site/session.

CoreDataHierarchy

CoreDataImportLog

●  ●  ●  ● 

As the Illuminate team works through your data imports we will review logs, make tweaks, and help the data load. Ultimately, we need your input, your help, your direction when things are not perfect You need to be reviewing the logs. Please follow up with us!!

RosterImportLog

●  ●  ● 

Some errors are OK. Don’t worry about it. Some errors are horrible. Worry about it. Warnings are just warnings. We don’t care if you don’t.

BestPractices •  Save usernames / passwords and data extraction procedures in a form accessible to all staff involved §  Save form in the cloud (ie: GoogleDrive, DropBox, etc.) • 

URL: https://docs.google.com/a/illuminateed.net/forms/d/ 12G7DEkV1RRHSfRntLUfs7lmKT_5j_d8xTRYJ3luCRK8/edit

•  Monitor “Core Data Import Logs” Have queries or extract tools run from a server or centralized machine, rather than a personal computer •  and latest dates of uploads

RolloverProcess •  Update Queries or Extracts •  Use Rollover Tool §  Explanation of the Rollover Tool •  Instructions to Rollover of Aeries and PowerSchool Extract Tools §  Please refer to the following Resource URL: • 

https://illuminateed.com/rollover.html

DataResources • 

Community: §  https://illuminate.zendesk.com/hc/communities/

• 

Data Specifications: §  https://www.illuminateed.com/downloads/

• 

Courtesy Scripts from other SIS §  https://www.illuminateed.com/courtesyscripts.html

• 

Data Tools & How they Work §  https://www.illuminateed.com/datatools.html

• 

Rollover §  https://illuminateed.com/rollover.html

CommonProblems •  Student does not display: Student must pass through 3 files (Studemo, Enrollment, and Roster) in order for teachers to access. Usually the student enrollment date is after the class date •  Entire class does not display: There maybe a missing section, user (teacher) or course •  Students have multiple teachers assigned: Occurs when teachers share the same Section Number/Site •  Students are displaying under a teacher after they have exited or transferred: The roster file must contain a record with an exit date. If not, the student can be exited in DnA

•  Student does not display as Special Ed: Student must have a value populated in the Primary Disability field of studemo.txt. Tip: Familiarize yourself with the studemo layout •  Student does not display as GATE: Record is provided in the programs.txt and must contain the following: •  - Valid program code •  - Start date •  Tips: Familiarize yourself with the studemo and program file layouts and content

#1RuleOf DataProcess

G.I.G.O. Garbage In Garbage Out

We’reGood ●  But we’re not that good ●  If you give us bad data, you’ll see bad data ●  We CAN transform your data, we can subtly modify your data ●  We CAN’T make good data out of garbage

CascadingDataErrors When just one (valid) course ID does not import: ●  master schedule records for that course ID will fail. ●  roster records for the failed master schedule record will fail ●  Students will be missing classes The question to Illuminate: Why is Johnny missing his 4th period Algebra class? The answer from Illuminate: Have you checked your import logs?

ItJustKeepsCascading When just one (valid) course ID does not import: ●  master schedule records for that course ID will fail. ●  roster records for the failed master schedule record will fail ●  Attendance records for that student and section will fail The question to Illuminate: Why is Johnny missing his 4th period attendance? The answer from Illuminate: Have you checked your import logs?

DataKeepsMakingItRain! When that same (valid) course ID does not import: ●  Transcript records will fail in import ●  Students will be missing grades The question to Illuminate: Why is Johnny missing his Spanish 2 grade? The answer from Illuminate: Have you checked your import logs?

It’sAWaterfallOfPain! When just one (valid) user ID does not import: ●  master schedule records for that user ID will fail. ●  roster records for the failed master schedule record will fail ●  Students will be missing classes The question to Illuminate: Why is Johnny missing his 4th period Algebra class? The answer from Illuminate: Have you checked your import logs?

DataIntegration For Illuminate Education ISI

DataPrep • 

• 

Step 1 §  Determine how many years of data will be converted Step 2 §  Examine and review Illuminate Data Specs • 

• 

• 

Note: If desired fields are not listed in Illuminate Data Spec, then client will need to confirm if field exist in the Illuminate Database.

Step 4 §  Examine and review Illuminate Schema • 

Step 5 §  Move through the ISI Data • 

Import the following data specs into Sandbox, have integrity confirmed, then import onto Live/ Production instance: § 

Illuminate Core Data Spec (minimum required datasets)

§ 

Illuminate Extra Data Spec (language, special ed, attendance, health, discipline, etc.)

§ 

Illuminate Health Data Spec (health, immunizations, cautions, etc.)

https://www.illuminateed.com/downloads/

Step 3 §  Decipher if fields to be converted are not listed in Data Specs • 

• 

https://static.illuminateed.com/schema_spy/

Moving Through

ISIDATA PHASE I (Option I) •  sites •  terms •  courses •  studemo •  users •  enrollment •  mastschd •  roster •  programs •  contacts

PHASE II (Option II) •  transcripts •  stugrades •  transfers •  discipline •  attendance •  specialed •  behavior

PHASE III (Option II) •  health •  health_immunization_ summary •  health_vaccinations •  health_tb •  health_office_visits •  health_cautions •  health_medications •  health_procedures •  health_screaning_bmi •  health_screaning_ dental •  health_screaning_ hearing •  health_screaning_ scoliosis •  health_screaning_ vision

PHASE IV (*) •  Supplemental (Additional required data)