STATE OF ALASKA DEPARTMENT OF EDUCATION & EARLY DEVELOPMENT DIVISION OF EDUCATION SUPPORT SERVICES 801 W. 10TH ST. STE 200 PO BOX 110500 JUNEAU, AK 99811-0500
Request For Proposals RFP Number 2014-0500-2154 Date of Issue: August 30, 2013 Alaska English Language Arts and Mathematics Assessments This RFP addresses Alaska’s transition to new grade level summative assessments in spring 2015 that are aligned to the Alaska English Language Arts and Mathematics Standards. Offerors must submit proposals for services to support assessments and tools being developed and provided by the Smarter SBAC. As a separate cost option the proposal must include the delivery requirements for the SBAC interim assessments, which will operate on the same scoring scale(s) as the summative assessments. Interim assessments will be utilized to determine student progress in relation to summative assessments. Required components are as follows: • • • •
Summative assessments in grades 3-8 and 11 in English language arts and mathematics to be delivered electronically, and if necessary via traditional paper and pencil Interim assessments in grades 3-8 and 11 to be delivered in same delivery formats as summative assessments Summative assessments in grades 9 and 10 in English language arts and mathematics to be delivered electronically, and if necessary via traditional paper and pencil Interim assessments in grades 9 and 10 to be delivered in same delivery formats as summative assessments
As a second cost option offerers are heavily encouraged to provide proposals for assessment programs that are not utilizing SBAC provided assessments and tools. Proposals must include plans to demonstrate Rev. 09/10
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
complete alignment of all assessments and related tools to the Alaska English language arts and Mathematics Standards. Proposals must include one or both of the following types of assessments: • • • •
Custom-developed assessments utilizing all Alaska-owned test items in grades 3-8 and 11 in English language arts and mathematics Custom-developed assessments utilizing all Alaska-owned test items in grades 9 and 10 in English language arts and mathematics Commercially available, published or vendor-owned assessments in grades 3-8 and 11 in English language arts and mathematics Commercially available, published or vendor-owned assessments in grades 9 and 10 in English language arts and mathematics
As a third cost option offerers are encouraged to provide proposals for formative resources to support teachers and educators with classroom instruction.
Offerors Are Not Required To Return This Form. Important Notice: If you received this solicitation from the State of Alaska’s “Online Public Notice” web site, you must register with the procurement officer listed in this document to receive subsequent amendments. Failure to contact the procurement officer may result in the rejection of your offer. All times referenced in this RFP are Alaska Time.
Rob Roys Procurement Officer Department of Education and Early Development
Page 2 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Acronyms Used: AMO Annual Measureable Objective ASPI Alaska School Performance Index CAT Computer Adaptive Test CBA Computer Based Assessment CSSA Comprehensive System of Student Assessment EED Alaska Department of Education and Early Development ELA English Language Assessment ESEA Elementary and Secondary Education Act FY State of Alaska Fiscal Year NCEO National Center for Educational Outcomes at the University of Minnesota PDF Portable Document Format P/P Paper/Pencil QA Quality Assurance QC Quality Control RFP Request for Proposals SBAC Smarter Balanced Assessment Consortium TAC Technical Advisory Committee TLS Alaska Department of Education and Early Development, Division of Teaching and Learning Support
Page 3 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
TABLE OF CONTENTS 1.
SECTION ONE INTRODUCTION AND INSTRUCTIONS ..................................................................... 6 1.01 RETURN MAILING ADDRESS, CONTACT PERSON, TELEPHONE, FAX NUMBERS AND DEADLINE FOR RECEIPT OF PROPOSALS ............................................................................................................................................................................... 6 1.02 CONTRACT TERM AND WORK SCHEDULE ..................................................................................................................... 6 1.03 PURPOSE OF THE RFP ................................................................................................................................................... 8 1.04 BUDGET ...................................................................................................................................................................... 10 1.05 LOCATION OF WORK .................................................................................................................................................. 11 1.06 HUMAN TRAFFICKING ................................................................................................................................................. 11 1.07 ASSISTANCE TO OFFERORS WITH A DISABILITY............................................................................................................ 11 1.08 REQUIRED REVIEW ..................................................................................................................................................... 12 1.09 QUESTIONS RECEIVED PRIOR TO OPENING OF PROPOSALS ......................................................................................... 12 1.10 AMENDMENTS ............................................................................................................................................................. 12 1.11 ALTERNATE PROPOSALS .............................................................................................................................................. 12 1.12 RIGHT OF REJECTION .................................................................................................................................................. 12 1.13 STATE NOT RESPONSIBLE FOR PREPARATION COSTS ................................................................................................... 13 1.14 DISCLOSURE OF PROPOSAL CONTENTS ....................................................................................................................... 13 1.15 SUBCONTRACTORS ..................................................................................................................................................... 14 1.16 JOINT VENTURES......................................................................................................................................................... 14 1.17 OFFEROR'S CERTIFICATION ......................................................................................................................................... 14 1.18 CONFLICT OF INTEREST .............................................................................................................................................. 15 1.19 RIGHT TO INSPECT PLACE OF BUSINESS ...................................................................................................................... 15 1.20 SOLICITATION ADVERTISING....................................................................................................................................... 15 1.21 NEWS RELEASES .......................................................................................................................................................... 15 1.22 ASSIGNMENT ............................................................................................................................................................... 15 1.23 DISPUTES .................................................................................................................................................................... 15 1.24 SEVERABILITY.............................................................................................................................................................. 16 1.25 FEDERAL REQUIREMENTS............................................................................................................................................ 16 1.26 FEDERAL DEBARMENT, SUSPENSION, INELIGIBILITY AND VOLUNTARY EXCLUSION ..................................................... 16
2.
SECTION TWO STANDARD PROPOSAL INFORMATION ................................................................. 17 2.01 2.02 2.03 2.04 2.05 2.06 2.07 2.08 2.09 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18 2.19
AUTHORIZED SIGNATURE ........................................................................................................................................... 17 PRE-PROPOSAL CONFERENCE ...................................................................................................................................... 17 SITE INSPECTION ......................................................................................................................................................... 17 AMENDMENTS TO PROPOSALS ..................................................................................................................................... 17 SUPPLEMENTAL TERMS AND CONDITIONS ................................................................................................................... 17 CLARIFICATION OF OFFERS ......................................................................................................................................... 18 DISCUSSIONS WITH OFFERORS .................................................................................................................................... 18 PRIOR EXPERIENCE ..................................................................................................................................................... 18 EVALUATION OF PROPOSALS ....................................................................................................................................... 20 VENDOR TAX ID ......................................................................................................................................................... 20 F.O.B. POINT .............................................................................................................................................................. 20 ALASKA BUSINESS LICENSE AND OTHER REQUIRED LICENSES..................................................................................... 20 APPLICATION OF PREFERENCES ................................................................................................................................... 21 5 PERCENT ALASKA BIDDER PREFERENCE AS 36.30.170 & 2 AAC 12.260 .................................................................. 22 5 PERCENT ALASKA VETERAN PREFERENCE AS 36.30.175........................................................................................... 22 FORMULA USED TO CONVERT COST TO POINTS AS 36.30.250 & 2 AAC 12.260........................................................ 23 ALASKA OFFEROR PREFERENCE AS 36.30.250 & 2 AAC 12.260 ................................................................................ 24 CONTRACT NEGOTIATIONS 2 AAC 12.315 ................................................................................................................. 25 FAILURE TO NEGOTIATE ............................................................................................................................................. 25
Page 4 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments 2.20 2.21
3.
NOTICE OF INTENT TO AWARD (NIA): OFFEROR NOTIFICATION OF SELECTION ......................................................... 25 PROTEST ..................................................................................................................................................................... 25
SECTION THREE STANDARD CONTRACT INFORMATION ............................................................ 27 3.01 3.02 3.03 3.04 3.05 3.06 3.07 3.08 3.09 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18
4.
RFP 2014-0500-2154
CONTRACT TYPE ......................................................................................................................................................... 27 CONTRACT APPROVAL ................................................................................................................................................ 27 STANDARD CONTRACT PROVISIONS ........................................................................................................................... 27 PROPOSAL AS A PART OF THE CONTRACT .................................................................................................................... 27 ADDITIONAL TERMS AND CONDITIONS ....................................................................................................................... 27 INSURANCE REQUIREMENTS ....................................................................................................................................... 27 BID BOND - PERFORMANCE BOND - SURETY DEPOSIT .................................................................................................. 28 CONTRACT FUNDING ................................................................................................................................................. 28 PROPOSED PAYMENT PROCEDURES ............................................................................................................................. 28 CONTRACT PAYMENT.................................................................................................................................................. 28 INFORMAL DEBRIEFING ............................................................................................................................................... 28 CONTRACT PERSONNEL .............................................................................................................................................. 28 INSPECTION & MODIFICATION - REIMBURSEMENT FOR UNACCEPTABLE DELIVERABLES ............................................. 28 TERMINATION FOR DEFAULT ...................................................................................................................................... 29 LIQUIDATED DAMAGES ............................................................................................................................................... 29 CONTRACT CHANGES - UNANTICIPATED AMENDMENTS ............................................................................................. 29 CONTRACT INVALIDATION.......................................................................................................................................... 29 NONDISCLOSURE AND CONFIDENTIALITY ................................................................................................................... 29
SECTION FOUR BACKGROUND INFORMATION ............................................................................. 31 4.01 BACKGROUND INFORMATION ..................................................................................................................................... 31 ASSESSMENT BACKGROUND.................................................................................................................................................... 31
5.
SECTION FIVE SCOPE OF WORK ......................................................................................................... 34 5.01 5.02
6.
SECTION SIX PROPOSAL FORMAT AND CONTENT ......................................................................... 56 6.01 6.02 6.03 6.04 6.05 6.06 6.07 6.08
7.
PROPOSAL FORMAT AND CONTENT............................................................................................................................. 56 INTRODUCTION .......................................................................................................................................................... 56 UNDERSTANDING OF THE PROJECT ............................................................................................................................. 56 METHODOLOGY USED FOR THE PROJECT.................................................................................................................... 56 MANAGEMENT PLAN FOR THE PROJECT....................................................................................................................... 57 EXPERIENCE, QUALIFICATIONS, AND CORPORATE CAPABILITY ................................................................................... 57 COST PROPOSAL ......................................................................................................................................................... 58 EVALUATION CRITERIA ............................................................................................................................................... 58
SECTION SEVEN EVALUATION CRITERIA AND CONTRACTOR SELECTION .............................. 59 7.01 7.02 7.03 7.04 7.05 7.06
8.
SCOPE OF WORK ......................................................................................................................................................... 34 DELIVERABLES ............................................................................................................................................................ 55
UNDERSTANDING OF THE PROJECT (5 PERCENT) ......................................................................................................... 59 METHODOLOGY USED FOR THE PROJECT (10 PERCENT) ............................................................................................. 59 MANAGEMENT PLAN FOR THE PROJECT (20 PERCENT) ................................................................................................ 59 EXPERIENCE, QUALIFICATIONS, AND CORPORATE CAPABILITY (15 PERCENT) ............................................................. 60 CONTRACT COST (40 PERCENT) ................................................................................................................................. 60 ALASKA OFFEROR PREFERENCE (10 PERCENT) ............................................................................................................ 60
SECTION EIGHT ATTACHMENTS ........................................................................................................ 61 8.01
ATTACHMENTS ........................................................................................................................................................... 61
Page 5 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
1. SECTION ONE INTRODUCTION AND INSTRUCTIONS 1.01 RETURN MAILING ADDRESS, CONTACT PERSON, TELEPHONE, FAX NUMBERS AND DEADLINE FOR RECEIPT OF PROPOSALS Offerors must submit an original and 6 (six) copies of their proposal, in writing, to the procurement officer in a sealed package. Submit only one Cost Proposal in a separate, sealed envelope. No portion of the cost proposal shall be included within the body of the proposal. Proposal package must include one CD with pdf copies of all submitted documents including the proposal and cost proposal. Submissions must be addressed as follows: DEPARTMENT OF EDUCATION AND EARLY DEVELOPMENT DIVISION OF ADMINISTRATIVE SERVICES-PROCUREMENT ATTENTION ROB ROYS RFP NUMBER 2014-0500-2154 801 W TENTH STREET SUITE 200 PO BOX 110500 JUNEAU, ALASKA 99811-0500 Proposals must be received no later than 4:00 PM on Friday, October 4, 2014. Fax proposals are not acceptable. Oral proposals are not acceptable. Email proposals are not acceptable Important Note: There is no overnight express delivery to Juneau, Alaska. Expedited delivery service takes at least two nights. An offeror’s failure to submit a proposal prior to the deadline will cause the proposal to be disqualified. Late proposals or amendments will not be opened or accepted for evaluation. PROCUREMENT OFFICER: Rob Roys– PHONE 907-465-8654 - FAX 907-465-3452 – TDD 907-465-2815
1.02 CONTRACT TERM AND WORK SCHEDULE The contract term and work schedule set out herein represents the State of Alaska's best estimate of the schedule that will be followed. If a component of this schedule, such as the opening date, is delayed, the rest of the schedule will be shifted by the same number of days. The length of the contract will be from the date of award, approximately January 1, 2014, for approximately (545) calendar days until completion, approximately June 30, 2015 and to be renewed annually at the state’s option for six one-year periods, through June 30, 2020. Unless otherwise provided in this RFP, the State and the successful offeror/contractor agree: (1) that any holding over of the contract excluding any exercised renewal options, will be considered as a month-toPage 6 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
month extension, and all other terms and conditions shall remain in full force and effect and (2) to provide written notice to the other party of the intent to cancel such month-to-month extension at least 30-days before the desired date of cancellation. The approximate contract schedule is as follows: • • • • • • • • •
Issue RFP Friday, August 30, 2013; Pre-proposal Teleconference (see §2.02) 1:00 to 3:00 PM on Thursday, September 12, 2013; Deadline for questions 5:00 PM on Friday, September 20, 2013; Deadline for Receipt of Proposals 4:00 PM on Friday, October 4, 2014; Proposal Evaluation Committee completes evaluation by Wednesday, November 13, 2013; State of Alaska issues Notice of Intent to Award a Contract by Friday, November 15, 2013; State of Alaska issues contract December 13, 2013 Contract start January 1, 2014; March 1, 2014 through June 30, 2014, First Work Period o Project planning meeting o Review list of schools to identify technology capability to determine CBA readiness and paper and pencil appropriateness o Attend spring TAC meeting (April 16-17, 2014) o Proposed methodology and timeline for the interim assessments o May rollout of information regarding fall interim assessment o Establish local caching or similar system for districts with potential internet connectivity challenges For custom-developed assessments utilizing all Alaska-owned test items: o Review existing items to align and/or recode to Alaska Standards
•
July 1, 2014 through June 30, 2015, Second Work Period o Project planning meeting o Attend TAC meetings with appropriate staff as needed (two per year, one in the fall and one in the spring) o Fall rollout of summative with online tutorials and practice tests o Development of grade 9 & 10 summative assessment CBA and P/P forms o Rollout of accessibility and accommodations features for CBA o Complete any proposed accommodated test items for paper and pencil spring testing (i.e., CDs and DVDs) o Fall and winter 2014-2105 administration of grade 3-11 interim assessment system (exact dates and future timeline TBD) o April/May 2015 first administration of grades 3-11 summative assessment Spring CBA window TBD within the last 12 weeks of school year; Spring summative assessments are scheduled by districts within the spring assessment window Spring paper and pencil window will be selected to coordinate with the CBA window o May 2015 report results
Page 7 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
For custom-developed assessments utilizing all Alaska-owned test items: o Propose methodology and field testing plan o Content and bias review of field test items o Conduct post field test item reviews and any necessary standard setting or validation activities •
Renewal option one of six July 1, 2014 to June 30, 2015, Third Work Period o Project planning meeting o Attend TAC meetings (two per year – spring and fall) o Fall - deliver the technical report from previously administered spring tests o Fall and winter 2014-2105 administration of grade 3-11 interim assessment system o April/May 2016, second spring assessment administration o May 2016 report all spring assessment results For custom-developed assessments utilizing all Alaska-owned test items: o Item development and committee review for field testing
•
Renewal option two of six July 1, 2015 to June 30, 2016, Fourth Work Period o Activities as outlined in prior years, and other activities TBP
•
Renewal option three of six July 1, 2016 to June 30, 2017, Fifth Work Period o Activities as outlined in prior years, and other activities TBP
•
Renewal option four of six July 1, 2017 to June 30, 2018, Sixth Work Period o Activities as outlined in prior years, and other activities TBP
•
Renewal option five of six July 1, 2018 to June 30, 2019, Seventh Work Period o Activities as outlined in prior years, and other activities TBP
•
Renewal option six of six July 1, 2019 to June 30, 2020, Eighth Work Period o Complete transition document preparation.
1.03 PURPOSE OF THE RFP The Department of Education and Early Development is soliciting proposals for summative assessments used to measure student proficiency and growth that are aligned to the Alaska English language arts and Mathematics Standards. The solicitation also calls for a proposal regarding interim assessments to be designed operating on the same vertical scale(s) as the summative assessments. Alaska is an advisory member of SBAC and plans to identify a contractor to administer, score, and report SBAC-developed assessments. The department also has a desire to consider the possibility of implementing contractor-provided assessments aligned to the Alaska Standards in lieu of SBAC assessments in English language arts and mathematics. The scope of work defined in the RFP includes administration, scoring, and reporting of the SBAC-developed assessments and, as a cost option, contractor-provided assessments in English language arts and mathematics in grades 3 through 11. Page 8 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
The contractor shall either: (1) administer, score, and report on SBAC-developed computer adaptive assessments in English language arts and mathematics content or (2) administer, score, and report on a technically-sound contractor-provided fixed form or computer adaptive assessment aligned to the Alaska English language arts and mathematics standards. The contractor-provided assessment may be a contractor-owned assessment, a custom-made Alaska assessment, or an existing assessment from another entity that the contractor licenses or leases on Alaska’s behalf. If the offeror proposes custom-developed assessments that utilize Alaska–owned items then the offeror must describe any potential challenges that may exist with meeting the 2015 spring timeline. Alternative solutions to meeting the 2015 timeline must include suggestions related to how the current assessment could be altered to provide college and career ready information to students in a manner that would demonstrate increased rigor, as well as maintain the ESEA waiver obligations. The proposal must include provisions for equivalent paper/pencil forms for schools that are unable to administer the computer-based assessment in the first three years. The contractor shall provide the assessments for all Alaska public school students enrolled in grades 3 through 11 beginning in the 2014-2015 academic year (spring 2015 test administration). The contractor-provided assessments must include a combination of selectedresponse items, constructed-response items, and performance task items; however, the state agency will not specify the number of each type of item that must be included in the test blueprint. The assessment must include technology-enhanced items as long as such items are administered using the minimum technology requirements defined herein. At a minimum, the contractor must provide two operational forms of the assessment and a third equivalent form to be used in the event of a breach of test security. The state reserves the right to renegotiate the summative assessments costs, depending on district usage or a change in direction. Renegotiations may be due to, but not limited to, changes in the federal law, changes in the state law, or changes to the content standards. Also included in this solicitation are interim assessments in English language arts and mathematics for grades 3-11. The interim assessments will be used to inform educators on the student achievement and progress toward mastery of the Alaska state English language arts and mathematics standards in grade 311. This assessment system must provide immediate feedback for data analysis at the school and classroom level and must use the same scoring scale(s) as the summative assessments. • •
At a minimum, the mathematics assessments must include two assessments for each domain (grades K-8) and conceptual category (grades 9-11) and embed the Standards for Mathematical Practice, as appropriate. At a minimum, the English language arts assessment must include two assessments for each area: reading, writing, speaking and listening, language, and literacy in history/social studies, science and technical subjects (grades 6-11).
The contractor shall provide an independent study to document the degree to which the contractorprovided assessments demonstrate full alignment of the Alaska ELA and math standards. Such a study shall be completed by using non-contractor consultants or a non-contractor organization for the purpose of objectively determining that the assessments adequately reflect the content and complexity of the Page 9 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Alaska ELA and math standards. For grade-level ELA assessments, the alignment study must demonstrate that the assessments include adequate quality and distribution of reading passages as demonstrated by the quantitative and qualitative readability indices, as well as scoring criteria for performance items that are reflective of the Alaska standards. For grade-level mathematics assessments, the alignment study must demonstrate that the assessments demonstrate appropriate intersections of mathematics content and practices as reflected in the Alaska standards. The grade-level ELA and math assessments must, at a minimum, meet the alignment and content coverage requirements of ESEA.
1.04 BUDGET The Alaska Department of Education & Early Development, Division of Teaching and Learning Support, intends to pay SBAC directly for services that are provided and managed by the consortium. The remaining services, in support of SBAC, will be provided by the offeror and identified in this proposal. EED estimates a budget of $10,000 for the first contract period (which include project planning, participation in TAC meetings and set up) between March 1, 2014 and June 30, 2014. The department estimates a budget of $2,600,000 for the second contract period in, fiscal year 2015 (July 1, 2014 -June 30, 2015). The department estimates a budget of $2,500,000 for the third contract period in FY2016. Increases in FY2017 to FY2019 shall not exceed 3% based on what the development work required is in each individual year. The department estimates the budget for the final contract period, which is a partial fiscal year within FY2020, to be not more than $250,000. Proposals priced at more than $15,000,000 for all contract periods will be considered non-responsive. If the proposal includes custom developed assessments EED expects to consider costs that are consistent with those of the current CSSA contract. EED estimates a budget of $100,000 for the first contract period (which include project planning, item development, participation in TAC meetings and set up) between March 1, 2014 and June 30, 2014. The department estimates a budget of $4,300,000 for the second contract period in, fiscal year 2015 (July 1, 2014 -June 30, 2015). The department estimates a budget of $4,500,000 for the third contract period in FY2016. Increases in FY2017 to FY2019 shall not exceed 3% based on what the development work required is in each individual year. The department estimates the budget for the final contract period, which is a partial fiscal year within FY2020, to be not more than $250,000. Proposals priced at more than $25,000,000 for all contract periods will be considered nonresponsive. If the proposal includes commercially available, published or vendor-owned assessments EED also expects to consider costs that are consistent with or less than those of the current CSSA contract. Proposals priced at more than $25,000,000 for all contract periods will be considered non-responsive. The offeror’s response must include a narrative describing the basis for costs in major task areas. To allow comparisons across contractors and for an effective evaluation, budget information for major task areas must be outlined on the forms provided in Appendix D. Submit only one Cost Proposal in a separate, sealed envelope. No portion of the cost proposal shall be included within the body of the proposal. Page 10 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
1.05 LOCATION OF WORK The state WILL NOT provide workspace for the contractor. The contractor must provide its own workspace. The contractor should include in their price proposal: transportation, lodging, and per diem costs sufficient to pay for person(s) to make trip(s) called for in the RFP. Travel to Alaska for the purpose of this contract in relation to technical advisory meetings, policy advisory meetings, planning meetings, test coordinator meetings, and other large group meetings will most frequently occur in Anchorage, with some meetings in Juneau, however meetings could take place in other locations. By signature on their proposal, the offeror certifies that all services provided under this contract by the contractor and all subcontractors shall be performed in the United States. If the offeror cannot certify that all work will be performed in the United States, the offeror must contact the procurement officer in writing to request a waiver at least 10 days prior to the deadline for receipt of proposals. The request must include a detailed description of the portion of work that will be performed outside the United States, where, by whom, and the reason the waiver is necessary. Failure to comply with this requirement or to obtain a waiver may cause the state to reject the proposal as non-responsive, or cancel the contract.
1.06 HUMAN TRAFFICKING By signature on their proposal, the offeror certifies that the offeror is not established and headquartered or incorporated and headquartered in a country recognized as Tier 3 in the most recent United States Department of State’s Trafficking in Persons Report. The most recent United States Department of State’s Trafficking in Persons Report can be found at the following website: http://www.state.gov/g/tip/ Failure to comply with this requirement will cause the state to reject the proposal as non-responsive, or cancel the contract.
1.07 ASSISTANCE TO OFFERORS WITH A DISABILITY Offerors with a disability may receive accommodation regarding the means of communicating this RFP or participating in the procurement process. For more information, contact the procurement officer no later than ten days prior to the deadline for receipt of proposals.
Page 11 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
1.08 REQUIRED REVIEW Offerors should carefully review this solicitation for defects and questionable or objectionable material. Comments concerning defects and objectionable material must be made in writing and received by the procurement officer at least ten days before the proposal opening. This will allow issuance of any necessary amendments. It will also help prevent the opening of a defective solicitation and exposure of offeror's proposals upon which award could not be made. Protests based on any omission or error, or on the content of the solicitation, will be disallowed if these faults have not been brought to the attention of the procurement officer, in writing, at least ten days before the time set for opening.
1.09 QUESTIONS RECEIVED PRIOR TO OPENING OF PROPOSALS All questions must be in writing and directed
to the issuing office, addressed to the procurement officer. The interested party must confirm telephone conversations in writing. No further questions will be allowed after 5:00 PM on Friday, September 20, 2013. Send questions to:
[email protected] Two types of questions generally arise. One may be answered by directing the questioner to a specific section of the RFP. These questions may be answered over the telephone. Other questions may be more complex and may require a written amendment to the RFP. The procurement officer will make that decision.
1.10 AMENDMENTS If an amendment is issued, it will be provided to all who were mailed a copy of the RFP and to those who have registered with the procurement officer as having downloaded the RFP from the State of Alaska Online Public Notice web site.
1.11 ALTERNATE PROPOSALS Offerors may only submit one proposal for evaluation. In accordance with 2 AAC 12.830 alternate proposals (proposals that offer something different than what is asked for) will be rejected.
1.12 RIGHT OF REJECTION Offerors must comply with all of the terms of the RFP, the State Procurement Code (AS 36.30), and all applicable local, state, and federal laws, codes, and regulations. The procurement officer may reject any proposal that does not comply with all of the material and substantial terms, conditions, and performance requirements of the RFP. Offerors may not qualify the proposal nor restrict the rights of the state. If an offeror does so, the Page 12 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
procurement officer may determine the proposal to be a non-responsive counter-offer and the proposal may be rejected. Minor informalities that: • • • • • • •
do not affect responsiveness; are merely a matter of form or format; do not change the relative standing or otherwise prejudice other offers; do not change the meaning or scope of the RFP; are trivial, negligible, or immaterial in nature; do not reflect a material change in the work; or do not constitute a substantial reservation against a requirement or provision;
may be waived by the procurement officer. The state reserves the right to refrain from making an award if it determines that to be in its best interest. A proposal from a debarred or suspended offeror shall be rejected.
1.13 STATE NOT RESPONSIBLE FOR PREPARATION COSTS The state will not pay any cost associated with the preparation, submittal, presentation, or evaluation of any proposal.
1.14 DISCLOSURE OF PROPOSAL CONTENTS All proposals and other material submitted become the property of the State of Alaska and may be returned only at the state's option. AS 40.25.110 requires public records to be open to reasonable inspection. All proposal information, including detailed price and cost information, will be held in confidence during the evaluation process and prior to the time a Notice of Intent to Award is issued. Thereafter, proposals will become public information. Trade secrets and other proprietary data contained in proposals may be held confidential if the offeror requests, in writing, that the procurement officer does so, and if the procurement officer agrees, in writing, to do so. Material considered confidential by the offeror must be clearly identified and the offeror must include a brief statement that sets out the reasons for confidentiality. Confidential material must be clearly identified and segregated from non-confidential materials within the offeror’s proposal to enable easy redaction for public review.
Page 13 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
1.15 SUBCONTRACTORS Subcontractors may be used to perform work under this contract. If an offeror intends to use subcontractors, the offeror must identify in the proposal the names of the subcontractors and the portions of the work the subcontractors will perform. If a proposal with subcontractors is selected, the offeror must provide the following information concerning each prospective subcontractor within five working days from the date of the state's request: a) b) c) d) e) f)
complete name of the subcontractor; complete address of the subcontractor; type of work the subcontractor will be performing; percentage of work the subcontractor will be providing; evidence that the subcontractor holds a valid Alaska business license; and a written statement, signed by each proposed subcontractor that clearly verifies that the subcontractor is committed to render the services required by the contract.
An offeror's failure to provide this information, within the time set, may cause the state to consider their proposal non-responsive and reject it. The substitution of one subcontractor for another may be made only at the discretion and prior written approval of the project director.
1.16 JOINT VENTURES Joint ventures are acceptable. If submitting a proposal as a joint venture, the offeror must submit a copy of the joint venture agreement which identifies the principals involved and their rights and responsibilities regarding performance and payment.
1.17 OFFEROR'S CERTIFICATION By signature on the proposal, offerors certify that they comply with the following: [a] the laws of the State of Alaska; [b] the applicable portion of the Federal Civil Rights Act of 1964; [c] the Equal Employment Opportunity Act and the regulations issued thereunder by the federal government; [d] the Americans with Disabilities Act of 1990 and the regulations issued thereunder by the federal government; [e] all terms and conditions set out in this RFP; [f] a condition that the proposal submitted was independently arrived at, without collusion, under penalty of perjury; [g] that the offers will remain open and valid for at least 90 days; and [h] that programs, services, and activities provided to the general public under the resulting contract conform with the Americans with Disabilities Act of 1990, and the regulations issued thereunder by the federal government. Page 14 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
If any offeror fails to comply with [a] through [h] of this paragraph, the state reserves the right to disregard the proposal, terminate the contract, or consider the contractor in default.
1.18 CONFLICT OF INTEREST Each proposal shall include a statement indicating whether or not the firm or any individuals working on the contract has a possible conflict of interest (e.g., currently employed by the State of Alaska or formerly employed by the State of Alaska within the past two years) and, if so, the nature of that conflict. The Commissioner, Department of Education and Early Development, reserves the right to consider a proposal non-responsive and reject it or cancel the award if any interest disclosed from any source could either give the appearance of a conflict or cause speculation as to the objectivity of the program to be developed by the offeror. The Commissioner's determination regarding any questions of conflict of interest shall be final.
1.19 RIGHT TO INSPECT PLACE OF BUSINESS At reasonable times, the state may inspect those areas of the contractor's place of business that are related to the performance of a contract. If the state makes such an inspection, the contractor must provide reasonable assistance.
1.20 SOLICITATION ADVERTISING Public notice has been provided in accordance with 2 AAC 12.220.
1.21 NEWS RELEASES News releases related to this RFP will not be made without prior approval of the project director.
1.22 ASSIGNMENT Per 2 AAC 12.480, the contractor may not transfer or assign any portion of the contract without prior written approval from the procurement officer.
1.23 DISPUTES Any dispute arising out of this agreement will be resolved under the laws of the State of Alaska. Any appeal of an administrative order or any original action to enforce any provision of this agreement or to obtain relief from or remedy in connection with this agreement may be brought only in the Superior Court for the State of Alaska.
Page 15 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
1.24 SEVERABILITY If any provision of the contract or agreement is declared by a court to be illegal or in conflict with any law, the validity of the remaining terms and provisions will not be affected; and, the rights and obligations of the parties will be construed and enforced as if the contract did not contain the particular provision held to be invalid.
1.25 FEDERAL REQUIREMENTS The offeror must identify all known federal requirements that apply to the proposal, the evaluation, or the contract.
1.26 FEDERAL DEBARMENT, SUSPENSION, INELIGIBILITY AND VOLUNTARY EXCLUSION Expenditures from this contract may involve federal funds. The U.S. Department of Labor requires all state agencies that are expending federal funds to have a certification filed in the bid (by the bidder) that they have not been debarred or suspended from doing business with the federal government. Certification regarding Debarment, Suspension, Ineligibility and Voluntary Exclusion Lower Tier Covered Transactions (§8.00 Attachment 11) must be completed and submitted with your bid.
Page 16 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
2. SECTION TWO STANDARD PROPOSAL INFORMATION 2.01 AUTHORIZED SIGNATURE All proposals must be signed by an individual authorized to bind the offeror to the provisions of the RFP. Proposals must remain open and valid for at least 90-days from the opening date.
2.02 PRE-PROPOSAL CONFERENCE There will be a pre-bid teleconference held at 1:00 PM on September 12, 2013. Participants should read the RFP and attend the meeting prepared to discuss any concerns. This will be a teleconference and potential bidders are invited to attend the teleconference by calling 1-800-315-6338, code 8654#. The purpose of the conference is to discuss the work to be performed with the prospective offerors and allow them to ask questions concerning the RFP. Questions and answers will be transcribed and sent to prospective offerors as soon as possible after the meeting. Questions should be submitted in writing at least 24 hours in advance to the procurement officer at:
[email protected] . Offerors with a disability needing accommodation should contact the procurement officer prior to the date set for the pre-proposal conference so that reasonable accommodation can be made.
2.03 SITE INSPECTION The state may conduct on-site visits to evaluate the offeror's capacity to perform the contract. An offeror must agree, at risk of being found non-responsive and having its proposal rejected, to provide the state reasonable access to relevant portions of its work sites. Individuals designated by the procurement officer at the state’s expense will make site inspection.
2.04 AMENDMENTS TO PROPOSALS Amendments to or withdrawals of proposals will only be allowed if acceptable requests are received prior to the deadline that is set for receipt of proposals. No amendments or withdrawals will be accepted after the deadline unless they are in response to the state's request in accordance with 2 AAC 12.290.
2.05 SUPPLEMENTAL TERMS AND CONDITIONS Proposals must comply with Section 1.11 Right of Rejection. However, if the state fails to identify or detect supplemental terms or conditions that conflict with those contained in this RFP or that diminish the state's rights under any contract resulting from the RFP, the term(s) or condition(s) will be considered null and void. After award of contract: a) if conflict arises between a supplemental term or condition included in the proposal and a term or condition of the RFP, the term or condition of the RFP will prevail; and Page 17 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
b) if the state's rights would be diminished as a result of application of a supplemental term or condition included in the proposal, the supplemental term or condition will be considered null and void.
2.06 CLARIFICATION OF OFFERS In order to determine if a proposal is reasonably susceptible for award, communications by the procurement officer or the proposal evaluation committee are permitted with an offeror to clarify uncertainties or eliminate confusion concerning the contents of a proposal. Clarifications may not result in a material or substantive change to the proposal. The evaluation by the procurement officer or the proposal evaluation committee may be adjusted as a result of a clarification under this section.
2.07 DISCUSSIONS WITH OFFERORS The state may conduct discussions with offerors in accordance with AS 36.30.240 and 2 AAC 12.290. The purpose of these discussions will be to ensure full understanding of the requirements of the RFP and proposal. Discussions will be limited to specific sections of the RFP or proposal identified by the procurement officer. Discussions will only be held with offerors who have submitted a proposal deemed reasonably susceptible for award by the procurement officer. Discussions, if held, will be after initial evaluation of proposals by the PEC. If modifications are made as a result of these discussions they will be put in writing. Following discussions, the procurement officer may set a time for best and final proposal submissions from those offerors with whom discussions were held. Proposals may be reevaluated after receipt of best and final proposal submissions. If an offeror does not submit a best and final proposal or a notice of withdrawal, the offeror’s immediate previous proposal is considered the offeror’s best and final proposal. Offerors with a disability needing accommodation should contact the procurement officer prior to the date set for discussions so that reasonable accommodation can be made. Any oral modification of a proposal must be reduced to writing by the offeror.
2.08 PRIOR EXPERIENCE In order for offers to be considered responsive offerors must meet these minimum prior experience requirements. a) Project Staffing The offeror’s response will include a staffing plan that includes the allocation of persons and/or departments by FTE across the major tasks to be completed. Any individual assigned to the project 0.5 FTE or more must be named in the offeror’s response and his/her vita must be submitted with this proposal. The information will be provided on the task allocation forms provided in Appendix C.
Page 18 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
The offeror’s response will include descriptions of experience and vitae for all individuals proposed to fill key functions within this project. The contractor’s appointed Project Director/Manager who is responsible for overseeing the management of the project must have a minimum of two years experience in a comparable role in working with a state’s statewide assessment program. Throughout the performance of the contract, the state retains the right of approval of individuals assigned to key management and test development positions within this project, whether the positions be employed by the primary contractor or by a subcontracted firm. The contractor will take action as requested by the state and in the timeframe requested by the state should EED determine the personnel assigned are not qualified for the position they have been assigned. During the proposal evaluation the state will review the personnel listed in the proposal's staffing plan against the minimum prior experience requirements and minimum qualifications listed in the RFP. Additionally, during the contractual period (including all fiscal year renewals), should the contractor or subcontractor have a change in personnel as initially identified in the proposal, the contractor must submit to the state’s project director the vitae for the position replacement for EED to review and approve. b) Corporate Capability The offeror’s response must include a description of the corporate capability of the prime contractor and all proposed subcontractors that will be performing key functions on this project. Subcontractors include individuals and organizations performing tasks directly related to educational testing/measurement such as item development or standard setting, as well as individuals and organizations performing tasks such as printing and shipping. Each corporate capability statement must address the offeror’s qualifications in customized test development along with background, experience, and capacity to perform the tasks required for the successful completion of this project. The response should include evidence of success in working with other large-scale, statewide, assessment contracts that have employed approaches similar to those proposed in the Scope of Work by the offeror (e.g., psychometric expertise in a variety of measurement models, methodologies, and settings in social science measurement; test developer expertise and knowledge in specific content areas possessing degrees in mathematics, social studies, science, reading, English language arts, education, and curriculum; publishing expertise in desktop publishing, graphic design, and editing.) The offeror’s proposal must include a list of all large-scale assessment projects in which the organization is currently, or has been involved, as a prime contractor or subcontractor since 2004. The list should include a short description of the responsibilities and outcomes, dates engaged, and total dollar amount of the contract as well as contact information for each project (i.e., contact name, affiliation, phone number, and email address). An offeror's failure to meet these minimum prior experience requirements will cause their proposal to be considered non-responsive and their proposal will be rejected. Page 19 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
2.09 EVALUATION OF PROPOSALS The procurement officer, or an evaluation committee made up of at least three state employees or public officials, will evaluate proposals. The evaluation will be based solely on the evaluation factors set out in Section SEVEN of this RFP. After receipt of proposals, if there is a need for any substantial clarification or material change in the RFP, an amendment will be issued. The amendment will incorporate the clarification or change, and a new date and time established for new or amended proposals. Evaluations may be adjusted as a result of receiving new or amended proposals.
2.10 VENDOR TAX ID A valid Vendor Tax ID must be submitted to the issuing office with the proposal or within five days of the state's request.
2.11 F.O.B. POINT All goods purchased through this contract will be F.O.B. final destination. Unless specifically stated otherwise, all prices offered must include the delivery costs to any location within the State of Alaska.
2.12 ALASKA BUSINESS LICENSE AND OTHER REQUIRED LICENSES At the time the proposals are opened, all offerors must hold a valid Alaska business license and any necessary applicable professional licenses required by Alaska Statute. Proposals must be submitted under the name as appearing on the person’s current Alaska business license in order to be considered responsive. Offerors should contact the Department of Commerce, Community and Economic Development, Division of Corporations, Business, and Professional Licensing, P. O. Box 110806, Juneau, Alaska 99811-0806, for information on these licenses. Offerors must submit evidence of a valid Alaska business license with the proposal. An offeror's failure to submit this evidence with the proposal will cause their proposal to be determined non-responsive. Acceptable evidence that the offeror possesses a valid Alaska business license may consist of any one of the following: a) b) c) d) e)
copy of an Alaska business license with the correct NAICS code; certification on the proposal that the offeror has a valid Alaska business license and has included the license number in the proposal; a canceled check for the Alaska business license fee; a copy of the Alaska business license application with a receipt stamp from the state's occupational licensing office; or a sworn and notarized affidavit that the offeror has applied and paid for the Alaska business license.
Page 20 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
You are not required to hold a valid Alaska business license at the time proposals are opened if you possess one of the following licenses and are offering services or supplies under that specific line of business: • • • •
Fisheries business licenses issued by Alaska Department of Revenue or Alaska Department of Fish and Game. Liquor licenses issued by Alaska Department of Revenue for alcohol sales only. Insurance licenses issued by Alaska Department of Commerce, Community and Economic Development, Division of Insurance. Mining licenses issued by Alaska Department of Revenue.
2.13 APPLICATION OF PREFERENCES Certain preferences apply to all contracts for professional services, regardless of their dollar value. The Alaska bidder, Alaska veteran, and Alaska Offeror Preferences are the most common preferences involved in the RFP process. Additional preferences that may apply to this procurement are listed below. Guides that contain excerpts from the relevant statutes and codes, explain when the preferences apply and provide examples of how to calculate the preferences are available at the Department of Administration, Division of General Services’ web site: http://doa.alaska.gov/dgs/policy.html Alaska Products Preference - AS 36.30.332 Recycled Products Preference - AS 36.30.337 Local Agriculture and Fisheries Products Preference - AS 36.15.050 Employment Program Preference - AS 36.30.170(c) Alaskans with Disability Preference - AS 36.30.170 (e) Employers of People with Disabilities Preference - AS 36.30.170 (f) Alaska Veteran’s Preference - AS 36.30.175 The Division of Vocational Rehabilitation in the Department of Labor and Workforce Development keeps a list of qualified employment programs; a list of individuals who qualify as persons with a disability; and a list of persons who qualify as employers with 50 percent or more of their employees being disabled. A person must be on this list at the time the bid is opened in order to qualify for a preference under this section. As evidence of an individual's or a business' right to a certain preference, the Division of Vocational Rehabilitation will issue a certification letter. To take advantage of the employment program preference, Alaskans with Disability Preference or Employers of People with Disabilities Preference described above, an individual or business must be on the appropriate Division of Vocational Rehabilitation list at the time the proposal is opened, and must provide the procurement officer a copy of their certification letter. Offerors must attach a copy of their certification letter to the proposal. The offeror's failure to provide the certification letter mentioned above with the proposal will cause the state to disallow the preference.
Page 21 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
2.14 5 PERCENT ALASKA BIDDER PREFERENCE AS 36.30.170 & 2 AAC 12.260 An Alaska Bidder Preference of five percent will be applied prior to evaluation. The preference will be given to an offeror who: (1) holds a current Alaska business license; (2) submits a proposal for goods or services under the name on the Alaska business license; (3) has maintained a place of business within the state staffed by the offeror, or an employee of the offeror, for a period of six months immediately preceding the date of the proposal; (4) is incorporated or qualified to do business under the laws of the state, is a sole proprietorship and the proprietor is a resident of the state, is a limited liability company (LLC) organized under AS 10.50 and all members are residents of the state, or is a partnership under AS 32.06 or AS 32.11 and all partners are residents of the state; and (5) if a joint venture, is composed entirely of entities that qualify under (1)-(4) of this subsection. Alaska Bidder Preference Affidavit In order to receive the Alaska Bidder Preference, the proposal must include a statement certifying that the offeror is eligible to receive the Alaska Bidder Preference. If the offeror is a LLC or partnership as identified in (4) of this subsection, the affidavit must also identify each member or partner and include a statement certifying that all members or partners are residents of the state. If the offeror is a joint venture which includes a LLC or partnership as identified in (4) of this subsection, the affidavit must also identify each member or partner of each LLC or partnership that is included in the joint venture and include a statement certifying that all of those members or partners are residents of the state.
2.15 5 PERCENT ALASKA VETERAN PREFERENCE AS 36.30.175 An Alaska Veteran Preference of five percent will be applied prior to evaluation. The preference will be given to an offeror who qualifies under AS 36.30.170 (b) as an Alaska bidder and is a: a) sole proprietorship owned by an Alaska veteran; b) partnership under AS 32.06 or AS 32.11 if a majority of the partners are Alaska veterans; c) limited liability company organized under AS 10.50 if a majority of the members are Alaska veterans; or d) corporation that is wholly owned by individuals and a majority of the individuals are Alaska veterans. Alaska Veteran Preference Affidavit In order to receive the Alaska Veteran Preference, proposals must include a statement certifying that the offeror is eligible to receive the Alaska Veteran Preference.
Page 22 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
2.16 FORMULA USED TO CONVERT COST TO POINTS
RFP 2014-0500-2154
AS 36.30.250 & 2 AAC 12.260
The distribution of points based on cost will be determined as set out in 2 AAC 12.260 (c). The lowest cost proposal will receive the maximum number of points allocated to cost. The point allocations for cost on the other proposals will be determined through the method set out below. In the generic example below, cost is weighted as 40% of the overall total score. The weighting of cost may be different in your particular RFP. See section SEVEN to determine the value, or weight of cost for this RFP.
EXAMPLE Formula Used to Convert Cost to Points
[STEP 1] List all proposal prices, adjusted where appropriate by the application of all applicable preferences. Offeror #1 Offeror #2 Offeror #3
-Non-Alaskan Offeror -Alaskan Offeror -Alaskan Offeror
$40,000 $42,750 $47,500
[STEP 2] Convert cost to points using this formula. [(Price of Lowest Cost Proposal) x (Maximum Points for Cost)] __________________________________________________ = (Cost of Each Higher Priced Proposal)
POINTS
The RFP allotted 40% (40 points) of the total of 100 points for cost. Offeror #1 receives 40 points. The reason they receive that amount is because the lowest cost proposal, in this case $40,000, receives the maximum number of points allocated to cost, 40 points. Offeror #2 receives 37.4 points. $40,000 x 40 Lowest Max Cost Points
=
Offeror #3 receives 33.7 points.
Page 23 of 61 Version: Original
$42,750 = Offeror #2 Adjusted By The Application Of All Applicable Preferences
1,600,000 ÷
37.4 Points
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
$40,000 x 40 = Lowest Max Cost Points
1,600,000 ÷
$47,500 = Offeror #3 Adjusted By The Application Of All Applicable Preferences
RFP 2014-0500-2154
33.7 Points
2.17 ALASKA OFFEROR PREFERENCE AS 36.30.250 & 2 AAC 12.260 2 AAC 12.260(e) provides Alaska offerors a 10 percent overall evaluation point preference. Alaska bidders, as defined in AS 36.30.170(b), are eligible for the preference. This preference will be added to the overall evaluation score of each Alaskan offeror. Each Alaskan offeror will receive 10 percent of the total available points added to their evaluation score as a preference.
EXAMPLE Alaska Offeror Preference [STEP 1] Determine the number of points available to Alaskan offerors under the preference. Total number of points available - 100 Points 100 x 10% = 10 Total Points Alaskan Offerors Number of Points Available Percentage Preference Given to Alaskan Offerors Under the Preference
[STEP 2] Add the preference points to the Alaskan offers. There are three offerors: Offeror #1, Offeror #2, and Offeror #3. Offeror #2 and Offeror #3 are eligible for the Alaska Offeror Preference. For the purpose of this example presume that all of the proposals have been completely evaluated based on the evaluation criteria in the RFP. Their scores at this point are: Offeror #1 - 89 points Offeror #2 - 80 points Offeror #3 - 88 points Offeror #2 and Offeror #3 each receive 10 additional points. The final scores for all of the offers are: Offeror #1 - 89 points Offeror #2 - 90 points Offeror #3 - 98 points
Page 24 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Offeror #3 is awarded the contract.
2.18 CONTRACT NEGOTIATIONS 2 AAC 12.315 After final evaluation, the procurement officer may negotiate with the offeror of the highest-ranked proposal. Negotiations, if held, shall be within the scope of the request for proposals and limited to those items which would not have an effect on the ranking of proposals. If the highest-ranked offeror fails to provide necessary information for negotiations in a timely manner, or fails to negotiate in good faith, the state may terminate negotiations and negotiate with the offeror of the next highest-ranked proposal. If contract negotiations are commenced, they may be held in the School Finance conference room on the 2nd floor of the Goldbelt Building at 801 W 10th Street in Juneau, Alaska. If the contract negotiations take place in Juneau, Alaska, the offeror will be responsible for their travel and per diem expenses.
2.19 FAILURE TO NEGOTIATE If the selected offeror • • • •
fails to provide the information required to begin negotiations in a timely manner; or fails to negotiate in good faith; or indicates they cannot perform the contract within the budgeted funds available for the project; or if the offeror and the state, after a good faith effort, simply cannot come to terms,
the state may terminate negotiations with the offeror initially selected and commence negotiations with the next highest ranked offeror.
2.20 NOTICE OF INTENT TO AWARD (NIA): OFFEROR NOTIFICATION OF SELECTION After the completion of contract negotiation the procurement officer will issue a written Notice of Intent to Award (NIA) and send copies to all offerors. The NIA will set out the names of all offerors and identify the proposal selected for award. A sample NIA is Attachment 12 to this RFP.
2.21 PROTEST AS 36.30.560 provides that an interested party may protest the content of the RFP. An interested party is defined in 2 AAC 12.990(a) (7) as "an actual or prospective bidder or offeror whose economic interest might be affected substantially and directly by the issuance of a contract solicitation, the award of a contract, or the failure to award a contract." If an interested party wishes to protest the content of a solicitation, the protest must be received, in writing, by the procurement officer at least ten days prior to the deadline for receipt of proposals.
Page 25 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
AS 36.30.560 also provides that an interested party may protest the award of a contract or the proposed award of a contract. If an offeror wishes to protest the award of a contract or the proposed award of a contract, the protest must be received, in writing by the procurement officer within ten days after the date the Notice of Intent to Award the contract is issued. A protester must have submitted a proposal in order to have sufficient standing to protest the proposed award of a contract. Protests must include the following information: a) b) c) d)
the name, address, and telephone number of the protester; the signature of the protester or the protester's representative; identification of the contracting agency and the solicitation or contract at issue; a detailed statement of the legal and factual grounds of the protest including copies of relevant documents; and e) the form of relief requested.
Protests filed by telex or telegram are not acceptable because they do not contain a signature. Fax copies containing a signature are acceptable. The procurement officer will issue a written response to the protest. The response will set out the procurement officer's decision and contain the basis of the decision within the statutory time limit in AS 36.30.580. A copy of the decision will be furnished to the protester by certified mail, fax or another method that provides evidence of receipt. All offerors will be notified of any protest. The review of protests, decisions of the procurement officer, appeals, and hearings, will be conducted in accordance with the State Procurement Code (AS 36.30), Article 8 "Legal and Contractual Remedies.”
Page 26 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
3. SECTION THREE STANDARD CONTRACT INFORMATION 3.01 CONTRACT TYPE This contract is a Firm Fixed Price contract.
3.02 CONTRACT APPROVAL This RFP does not, by itself, obligate the state. The state's obligation will commence when the contract is approved by the Commissioner of the Department of Education and Early Development, or the Commissioner's designee. Upon written notice to the contractor, the state may set a different starting date for the contract. The state will not be responsible for any work done by the contractor, even work done in good faith, if it occurs prior to the contract start date set by the state.
3.03 STANDARD CONTRACT PROVISIONS The contractor will be required to sign and submit the attached State's Standard Agreement Form for Professional Services Contracts (form 02-093/Appendix A, Attachment 02 to this RFP). The contractor must comply with the contract provisions set out in this attachment. No alteration of these provisions will be permitted without prior written approval from the Department of Law. Objections to any of the provisions in Appendix A must be set out in the offeror’s proposal.
3.04 PROPOSAL AS A PART OF THE CONTRACT Part or all of this RFP and the successful proposal may be incorporated into the contract.
3.05 ADDITIONAL TERMS AND CONDITIONS The state reserves the right to add terms and conditions during contract negotiations. These terms and conditions will be within the scope of the RFP and will not affect the proposal evaluations.
3.06 INSURANCE REQUIREMENTS The successful offeror must provide proof of workers' compensation insurance prior to contract approval. The successful offeror must secure the insurance coverage required by the state. The coverage must be satisfactory to the Department of Administration Division of Risk Management. An offeror's failure to provide evidence of such insurance coverage is a material breach and grounds for withdrawal of the award or termination of the contract. Offerors must review form APPENDIX B1, Attachment 03 to this RFP, for details on required coverage. No alteration of these requirements will be permitted without prior written approval from the Page 27 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Department of Administration, Division of Risk Management. Objections to any of the requirements in APPENDIX B1 must be set out in the offeror’s proposal.
3.07 BID BOND - PERFORMANCE BOND - SURETY DEPOSIT Not applicable to this RFP.
3.08 CONTRACT FUNDING Approval or continuation of a contract resulting from this is contingent upon legislative appropriation.
3.09 PROPOSED PAYMENT PROCEDURES The state will make payments based on a negotiated payment schedule. Each billing must consist of an invoice and progress report. No payment will be made until the progress report and invoice has been approved by the project director.
3.10 CONTRACT PAYMENT No payment will be made until the contract is approved by the Commissioner of the Department of Education and Early Development or the Commissioner's designee. Under no conditions will the state be liable for the payment of any interest charges associated with the cost of the contract. The state is not responsible for and will not pay local, state, or federal taxes. All costs associated with the contract must be stated in U.S. currency.
3.11 INFORMAL DEBRIEFING When the contract is completed, an informal debriefing may be performed at the discretion of the project director. If performed, the scope of the debriefing will be limited to the work performed by the contractor.
3.12 CONTRACT PERSONNEL Any change of the project team members named in the proposal must be approved, in advance and in writing, by the project director. Personnel changes that are not approved by the state may be grounds for the state to terminate the contract.
3.13 INSPECTION & MODIFICATION - REIMBURSEMENT FOR UNACCEPTABLE DELIVERABLES The contractor is responsible for the completion of all work set out in the contract. All work is subject to inspection, evaluation, and approval by the project director. The state may employ all reasonable means to ensure that the work is progressing and being performed in compliance with the contract. The project Page 28 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
director may instruct the contractor to make corrections or modifications if needed in order to accomplish the contract’s intent. The contractor will not unreasonably withhold such changes. Substantial failure of the contractor to perform the contract may cause the state to terminate the contract. In this event, the state may require the contractor to reimburse monies paid (based on the identified portion of unacceptable work received) and may seek associated damages.
3.14 TERMINATION FOR DEFAULT If the project director determines that the contractor has refused to perform the work or has failed to perform the work with such diligence as to ensure its timely and accurate completion, the state may, by providing written notice to the contractor, terminate the contractor's right to proceed with part or all of the remaining work. This clause does not restrict the state's termination rights under the contract provisions of Appendix A, attached.
3.15 LIQUIDATED DAMAGES Not applicable to this RFP.
3.16 CONTRACT CHANGES - UNANTICIPATED AMENDMENTS During the course of this contract, the contractor may be required to perform additional work. That work will be within the general scope of the initial contract. When additional work is required, the project director will provide the contractor a written description of the additional work and request the contractor to submit a firm time schedule for accomplishing the additional work and a firm price for the additional work. Cost and pricing data must be provided to justify the cost of such amendments per AS 36.30.400. The contractor will not commence additional work until the project director has secured any required state approvals necessary for the amendment and issued a written contract amendment, approved by the Commissioner of the Department of Education and Early Development or the Commissioner’s designee.
3.17 CONTRACT INVALIDATION If any provision of this contract is found to be invalid, such invalidation will not be construed to invalidate the entire contract.
3.18 NONDISCLOSURE AND CONFIDENTIALITY Contractor agrees that all confidential information shall be used only for purposes of providing the deliverables and performing the services specified herein and shall not disseminate or allow dissemination of confidential information except as provided for in this section. The contractor shall hold as confidential and will use reasonable care (including both facility physical security and electronic security) Page 29 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
to prevent unauthorized access by, storage, disclosure, publication, dissemination to and/or use by third parties of, the confidential information. “Reasonable care” means compliance by the contractor with all applicable federal and state law, including the Social Security Act and HIPAA. The contractor must promptly notify the state in writing if it becomes aware of any storage, disclosure, loss, unauthorized access to or use of the confidential information. Confidential information, as used herein, means any data, files, software, information or materials (whether prepared by the state or its agents or advisors) in oral, electronic, tangible or intangible form and however stored, compiled or memorialized that is classified confidential as defined by State of Alaska classification and categorization guidelines (i) provided by the state to the contractor or a contractor agent or otherwise made available to the contractor or a contractor agent in connection with this contract, or (ii) acquired, obtained or learned by the contractor or a contractor agent in the performance of this contract. Examples of confidential information include, but are not limited to: technology infrastructure, architecture, financial data, trade secrets, equipment specifications, user lists, passwords, research data, and technology data (infrastructure, architecture, operating systems, security tools, IP addresses, etc). Additional information that the contractor shall hold as confidential during the performance of services under this contract include: • • • •
Student names; Students’ state school identification numbers; Students’ test scores or grades; and Any other student personal information, such as address, birth date, school name, health or disciplinary information.
If confidential information is requested to be disclosed by the contractor pursuant to a request received by a third party and such disclosure of the confidential information is required under applicable state or federal law, regulation, governmental or regulatory authority, the contractor may disclose the confidential information after providing the state with written notice of the requested disclosure ( to the extent such notice to the state is permitted by applicable law) and giving the state opportunity to review the request. If the contractor receives no objection from the state, it may release the confidential information within 30 days. Notice of the requested disclosure of confidential information by the contractor must be provided to the state within a reasonable time after the contractor’s receipt of notice of the requested disclosure and, upon request of the state, shall seek to obtain legal protection from the release of the confidential information. The following information shall not be considered confidential information: information previously known to be public information when received from the other party; information freely available to the general public; information which now is or hereafter becomes publicly known by other than a breach of confidentiality hereof; or information which is disclosed by a party pursuant to subpoena or other legal process and which as a result becomes lawfully obtainable by the general public.
Page 30 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
4. SECTION FOUR BACKGROUND INFORMATION 4.01 BACKGROUND INFORMATION The State of Alaska requests offerors to submit proposals to the Department of Education and Early Development for assessments used to measure student proficiency and growth that are aligned to the Alaska English language arts and Mathematics Standards. The solicitation also calls for a proposal regarding interim assessments to be used for formative purposes. The requests are for the administration, scoring and reporting of either contractor-provided grade-level assessments or SBAC developed assessments that are aligned to the Alaska Standards for grades 3-11 in English language arts and math. Assessment Background Alaska currently administers custom assessments in grades 3 through 10 with the SBA in reading, writing and mathematics. The state expects that with the contract resulting from this solicitation, the current SBA will cease to continue following the 2014 spring administration. Alaska adopted the new Alaska English Language Arts and Mathematics Standards in 2012. Alaskans wanted to increase the rigor of our standards so that Alaska’s high school graduates would be better prepared for college education, technical training, and careers after high school. Alaska would like, through this RFP, to have a contractor provide services for a coherent set of assessments that are aligned with the Alaska English Language Arts and Mathematics Standards. The core set of assessments will consist of custom assessments in English language arts and mathematics in grades 3-11 that will be suitable for reporting student achievement and student progress in relation to the Alaska standards, and for inclusion in state and federal school/district accountability programs. In May 2013 Alaska was granted a waiver from ESEA accountability requirements. The new assessments will be utilized to meet the student achievement and progress requirements of ESEA. Alaska joined the SBAC as an advisory state in April 2013. SBAC will design summative assessments to be used late in the school year for grades 3-8 and 11 for English language arts and mathematics. The assessments will be designed to include electronic delivery through CAT and performance tasks. SBAC will also provide paper and pencil versions of the summative assessments during a three year transition period. SBAC is designing assessment items for grades 9 and 10 and Alaska desires to have a contractor develop tests for grades 9 and 10 to ensure grade continuity for student progress reporting capabilities. Alaska is seeking the following services in support of participation in the SBAC developed assessments: 1) Administration services support, scoring, printing, shipping, and other related services in support of administering paper and pencil versions of the summative assessment to be used by schools that demonstrate an inability to utilize computerized versions of the test; 2) A testing engine that will be compatible with SBAC computer adaptive testing, including local caching server capabilities; 3) CAT and paper and pencil versions of summative assessments in grades 9 and 10 for English language arts and mathematics; Page 31 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
4) Psychometric services to ensure validity, reliability and comparability of test results based on mastery of the Alaska standards; 5) Score reporting services for summative assessments including state-level, district-level and schoollevel summary reports including subgroup reporting, individual student reports (ISRs) and interpretative materials for understanding results; 6) Data warehouse access for district and department staff to utilize for analysis of score data at district, school, classroom and individual levels; 7) Hand-scoring of constructed response items, if necessary; and 8) Test security services Alaska currently utilizes erasure analysis for the paper and pencil summative assessments and desires to continue that capability. Test security protocols must include corresponding security measures appropriate for computer-administered assessments. Security documents in support of the test administrations and data audit capabilities are also strongly desired. Further, the state is also requesting proposals for an interim assessment system in English language arts and mathematics to support instruction and is aligned with the scoring scales for the summative assessments administered in the spring for grades 3-11. The interim assessment system shall include periodic assessments that can be administered at multiple times and scored electronically or locally during the school year. The offeror shall demonstrate or describe how the interim assessments will be aligned or linked to the Alaska state standards as explicated on the summative assessment. For additional costs, SBAC plans to provide formative tools to be used in classrooms by students and teachers to diagnose instructional needs and provide progress toward specific individual learning goals. Alaska encourages proposals to include formative resources to support instruction in the classroom. Assessment Schedule The Assessment System contract described in this RFP will be awarded on December 13, 2013, with first operational testing starting in spring 2015 and continuing through the spring 2019 administration and score reporting. The state will renew the contract each state fiscal year based on satisfactory performance and continuation of funding. The computerized assessments in grades 3-11 will be administered in a test administration window beginning on the first Monday or Tuesday in April. Assessment results are to be returned electronically to the districts by the end of May and hard copy reports by the middle of June. A data file suitable for the ASPI and AMO analyses must be delivered by the contractor to EED by May 31 each year; though earlier would be preferable. Offerors should consider that districts will have until the Thursday following the close of the test window to collect, audit, prepare, and box the assessment materials for any paper and pencil assessments. Offerors must include in their proposals a method for the contractor to be responsible for picking up and securing the assessment materials from each district for delivery to the contractor. Offerors are reminded that many sites in Alaska are not served by traditional commercial courier services. Logistics and costs for delivery and retrieval of all assessments and materials for the grades 3-11 state assessments shall be the responsibility of the contractor and included in the firm, fixed fee cost proposal.
Page 32 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Offerors are invited to view the state’s current Statewide Testing Dates Calendar in Appendix F. Offerors are encouraged to discuss challenges and propose solutions in meeting the timelines outlined in the calendar. All tests developed will be based on the Alaska English language arts and Mathematics Standards and Test Specifications. Refer to the Technical Reports on the department’s web site to review the Test Specifications. Project Management The state will assemble an assessment project management team to manage the daily operation of activities under the assessment contract. The management team will include a single state project director (e.g., state assessment director) and support staff as needed. The state project director will serve as the state’s primary point of contact with the contractor. The successful contractor will propose a project management structure and team that will be responsive to Alaska’s needs and circumstances. In particular, the project management should consider the limited staffing available in the department.
Page 33 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
5. SECTION FIVE SCOPE OF WORK 5.01 SCOPE OF WORK A) Introduction This section of the RFP contains a description of the major tasks required of the contractor for the successful completion of this project and provides information on contract deliverables. The offeror’s response must directly reference and address each of the tasks contained in this section. In addition, the offeror’s response must identify any additional tasks not included in this RFP determined necessary for the successful completion of this project. Offerors shall propose alternative or additional tasks that they feel would improve the efficiency of the project and/or quality of the materials produced for the project. The quality of all work and materials produced by the contractor is critical to the successful implementation of Alaska’s statewide assessment and accountability systems. Throughout their proposal, offerors must provide evidence and descriptions of the methods and procedures they use to ensure the quality of their work. Where the State of Alaska participates in the creation or development of an item or materials, the state must have freedom to use the items and materials at will in any way within the state, now and in the future, at no additional cost to the state. The state prefers that copyright be issued in the name of the state. However, offerors are encouraged to propose other ownership arrangements that provide the state what it needs and reduce costs of the contract. The contractor shall either: (1) administer, score, and report on SBAC-developed computer adaptive assessments in English/Language Arts and Mathematics content, or (2) administer, score, and report on a technically-sound contractor-provided fixed form or computer adaptive assessment aligned to Alaska Standards in English language arts and mathematics content. 1) The state intends to utilize the Smarter Balanced Consortium (SBAC) computer adaptive assessments in spring 2015 for grades 3-11 in English language arts and Mathematics aligned to the Alaska English language arts and Mathematics Standards. The following SBAC assessment support services are to be included in responses by offerors: a) CBA/CAT implementation support, including test administration, scoring, and reporting; b) Paper/pencil (P/P) summative assessment (to be used by schools that demonstrate an inability to utilize computerized versions of the test) implementation support, including printing, shipping, administration, scoring, and other related services; c) CBA/CAT and P/P summative assessments development for grades 9 and 10 for English/language arts and mathematics using SBAC items to ensure continuity of successive grades for reporting year-to-year progress in the state’s accountability system;
Page 34 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
d) A testing engine that will be compatible with SBAC computer adaptive testing, including local caching server capabilities; e) Psychometric services to ensure validity, reliability and comparability of test results based on mastery of the Alaska standards; f) score reporting services for summative assessments including state-level, district-level and school-level summary reports including subgroup reporting, individual student reports (ISRs) and interpretative materials for understanding results; g) Data warehouse access for district and department staff to utilize for analysis of score data at district, school, classroom and individual levels; h) Hand-scoring of constructed response items, if necessary; i) Test security services – Alaska currently utilizes erasure analysis for the paper and pencil summative assessments and desires to continue that capability. Test security protocols must include corresponding security measures appropriate for computer-administered assessments. Security documents in support of the test administrations and data audit capabilities are also strongly desired. 2) In addition, the state asks vendors to bid a cost option for contractor-provided fixed form or computer adaptive grade level summative assessments for grades 3-11 that are aligned to the Alaska English language arts and Mathematics Standards. The proposal could include one or both of the following types of assessments: a) Custom-developed assessments designed for Alaska b) Commercially available, published or contractor-owned assessments Proposals should include implementation support, including administration, scoring and reporting. The custom-developed assessments should include a suggested approach for improving the current assessment by 2015 to be more closely aligned to the new Alaska standards. An approach should also be outlined to potentially adjust the scoring and scoring scales to ensure that Alaska meets the requirements of the ESEA Accountability Waiver. Alaska will own all items and all rights to the items utilized for the custom-developed assessments. In proposing this solution potential problems should be identified and outlined, including alternative timelines for developing two unique forms to be used. Alaska also desires a third form to be developed for use in case of security breaches. With this approach an offeror must include a plan to compare the performance of Alaska students relative to students in other jurisdictions within the United States. Commercially available, published or contractor-owned assessments must be available for implementation in spring 2015. Assessment proposals included under this approach must include a description of how the assessments will align to the Alaska Standards. With this approach an offeror must include a plan to compare the performance of Alaskan students relative to students in other jurisdictions within the United States. The state invites proposals that will include cost options for the support services for SBACdeveloped interim assessments or the provision of contractor-provided interim assessments for grades 3-11. Page 35 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Following each operational administration of the English language arts and mathematics grade level assessments, the contractor shall provide scoring and reporting in accordance with the requirements herein. For SBAC-developed assessments, the contractor shall provide reports using scale scores, cut points, and achievement level descriptors provided by SBAC. For contractorprovided assessments, the contractor shall provide any applicable norm-referenced scores as well as cut points and achievement levels relative to the Alaska ELA and math standards that describe student progress toward “college and career ready.” If the cut points and achievement level descriptors have been previously determined for the contractor-provided assessment, the contractor shall provide a cut point validation including a review by Alaskan educators in accordance with the requirements herein. If cut points and achievement level descriptors have not been previously determined for the contractor-provided assessment, the contractor shall complete an achievement level setting with Alaskan educators in accordance with the requirements herein. B) Project Management and Planning 1) Management Team a) Project Director/Manager The contractor will appoint a single project director who oversees the management of the project. The offeror shall specify the percent of this person’s time devoted to this project. The offeror shall also address how continuity in project management will be maintained over the life of this contract. b) Support Staff The contractor will appoint support staff as appropriate to the successful completion of contract deliverables. 2) Management Meetings The contractor will schedule and support regular management meetings with the state’s assessment unit. Regular project management meetings will be held via telephone or web meetings as well as on-site meetings. The contractor should budget for four one-day (remote) meetings and two twoday on-site meetings in Juneau, Alaska in the initial full year (FY15) with a slightly reduced schedule in subsequent years. Additionally, the offeror shall propose weekly status updates by the contractor’s project manager to the state assessment staff. The agenda for these meetings will be reviewed and approved by the state, and the contractor will be responsible for taking and distributing meeting minutes. 3) Management Reports In addition to management meetings, the contractor will provide the following reports: a) Monthly Budget and management reports b) Annual project plan and schedule (statement of work for contact, due no later than June 1, or earlier, of each year) c) Immediate (within 24 hours) reports of any unusual activities/events Page 36 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
4) QA/QC The quality of all work and materials produced by the contractor is critical to the successful completion of the design, development, and implementation of the statewide assessment system. While this section addresses ‘quality control,’ the offeror must also provide evidence and descriptions of the methods and procedures used to ensure the quality of their work throughout their proposal in all relevant sections. For the purposes of this section, the state requests that offerors provide an overview of the company’s QA/QC and general procedures. Examples include Functional Specification documents which assist in the development and testing of the deliverables and processes between the state and the offeror. Further, the offeror shall discuss how the company’s efforts in reviewing and improving its QA/QC procedures during the past three years have led to both improvements in the QA/QC systems and in the products and services of the company. Finally, the offeror shall name the person and level of authority within the company who will be responsible for ensuring that these QA/QC systems and procedures are implemented appropriately and maintained for the Alaska contract. C) Inclusiveness of Assessments The State of Alaska is committed to the principle that common statewide assessments must be accessible to virtually all students. Therefore, the contractor’s response must reflect an understanding of, and commitment to this principle throughout the proposal. In particular, the contractor’s response for a contractor-provided assessment must address the principles of Universal Design as articulated in materials developed by the NCEO and available at http://cehd.umn.edu/nceo/UDmanual/default.html. The offeror will produce a document outlining its incorporation of accessibility at all stages of the test construction and administration, including, but not limited to, physical test design, item development, field-testing, and administration and reporting. State policies regarding the inclusiveness of the assessments also apply to any statewide field testing. Statewide field testing will include students with disabilities and English language learners using the accommodations they would normally use in the large-scale assessment. There is a need for over sampling of these populations to ensure a valid field test. The offeror will address these issues in their description of the field test, and will describe how the field test will guarantee validity for the sample of students with disabilities and English language learners. The offeror will identify accommodations that do not threaten the validity of the assessment, drawing from the lists of acceptable accommodations either outlined by SBAC or that currently exist in the state. When there are accommodations that specifically do threaten validity, these will be identified, and the contractor will work with the state to determine if changes in policy or regulations are necessary.
Page 37 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
D) Use of Technology Alaska is committed to the use of technology to improve the efficiency and accessibility of the assessment program. Throughout the response, the offeror should provide specific examples of how technology will be applied to support the assessment program. It is essential that all technology utilized by school districts be MAC platform compatible as PCs are not frequently found in Alaska school districts. The state requires responses that utilize CATs. Offerors’ responses that include the use of computeradministered tests at any time during the life of the contract as an option for districts and/or schools, must include a detailed description of the methods that will be used to ensure the comparability of results from paper-and-pencil and computer-administered tests sufficient for use in ESEA accountability purposes for all tests. The state requests the proposals include evidence of demonstrated use or clear examples of how technology would effectively work in the diverse state which is made up of urban, rural, and remote rural areas. Some rural/remote areas of the state do not have reliable and consistent connectivity; therefore offerors must address options for addressing this issue in their response. Proposals must include plans for delivering computerized testing including services similar to, but not limited to, local caching systems. Offerors shall include alternative options to online testing for sites that do not have the capacity to offer testing through an online system. The offeror shall specify the system requirements for adequate and fair usage of CBA at the school level. The system requirements shall be stated in sufficient detail for the department to determine the feasibility of the approach and to make comparisons between proposals. This should include all relevant specifications, including for hardware, software, operating system, secure servers, connectivity requirements, network or Internet connections, wiring, power consumption, and security. The specifications should be for “real world use” commensurate with the proposal, and shall address issues related to security, bandwidth, required time to administer to all eligible students in a school, software installation requirements and on-call technical support for schools. Where proposed, offerors’ responses must describe if and how technology will be integrated into the scoring of responses of constructed-response items in paper-pencil testing. This would include optical scanning of student response booklets or other uses of technology to enhance the quality of assessment scoring, improve project times, facilitate project management, or other uses. The state expects proposals using technology for the scoring of individual student responses to constructedresponse items and direct writing assessments to include evidence of reliability, validity and cost effectiveness. The state anticipates that advances in technology and the increased availability of technology will allow for possible substantive changes to the assessment program prior to the ending date of the contract resulting from this RFP. If such changes are made, cost and schedule adjustments related to such changes will be negotiated as needed. The state currently has reports delivered to the districts electronically in a research file format, as well as PDF copies of reports that are later delivered to the district in hard copy. The two most recent administrations of report data should be maintained on the offeror’s secured website. When one Page 38 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
administration is removed from the website the offeror will supply the state with copies of all reports on a secure server. This practice, which allows for quick delivery of reports for those schools that end the school year prior to delivery of hard copy reports, as well as provides hard copies for later delivery, is a practice the state expects to continue. The state currently has the contractor provide the data to a subcontracted firm and the data is loaded into an analysis system for the state and districts to use. Once data is verified by the state and districts through a state process, the updated data replaces the previously loaded data in the analysis tool. Offerors’ responses must address how they, or a subcontractor, will deliver a comparable analysis tool for the state and school districts to use. The current system allows for longitudinal data analysis, and includes assessment data from 2004 to the most recent administration. E) Item Development For contractor-provided assessment proposals, offerors’ response must address the item development requirements as well as the specific tasks included in this section of the RFP. All items used in the development of new Alaskan assessments will be the property of the state. However, offerors are encouraged to propose other ownership arrangements that provide the state what it needs and reduce costs of the contract. In particular, the offerors’ response must include a description of the procedures that will be used to facilitate the work of item review committees consisting of members from across the state, ensure that all items are aligned with the Alaska Standards, and to ensure that all test items are free from bias and accessible to the widest possible range of students. The offerors’ response must address any concerns with particular requirements included in the Test Specifications. 1) Item Development Team The contractor will appoint an item development team responsible for the development of items for the state assessments at grades 3 through 11. The team will include a lead content person in each content area – English language arts and mathematics. The team will include a specialist in students with disabilities and English language learner issues. The team will include sufficient staff to develop the required test items and represent the contractor with the state at committee reviews. 2) Item Review Committees The contractor will support item review committees consisting primarily of grade-appropriate teachers selected from across the state. Additional committee members may include local curriculum coordinators, content specialists, English as second language specialists, and special education specialists. The state will determine the composition of committees. The offeror should plan on supporting the item review committees with the following specifications each for English language arts and mathematics: a) b) c) d)
An elementary level group (grades 3-5) with a minimum of 12 members, A middle school level group (grades 6-8) with a minimum of 12 members, A high school level group (grades 9-11) with a minimum of 12 members, Grade-level subcommittees formed from each committee, or cross grade reviews within the grade cluster committees.
Page 39 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
e) Selected by the state through a recruitment/application process to be highly qualified, motivated to serve on the committees, and representative of regions and backgrounds in the state. f) Meet as needed to review items necessary to fit within the new or field test item development cycle based on the RFP and the proposal to meet the needs outlined in the RFP. g) The meetings will last one or two days and will generally occur on work days. h) Participants’ districts will be paid a substitute reimbursement for meetings. The contractor should budget for approximately $150 per committee member per school day to reimburse districts for the costs of substitute teachers. If teachers are not on contract at the time with a district or being paid by another entity, the contractor will provide a stipend to the committee members of at least $100 per day. i) For meetings that meet the hourly requirements, and involve instruction to committee members, the contractor will work with the state to design professional credit for participants. j) Members’ travel, lodging, and meals will be arranged and paid by the contractor. k) Meetings will be held at a hotel, conference center, or similar suitable location identified by the state in conjunction with the contractor. Most meetings of this type have been held in Anchorage because of its centrality for transportation and the availability of meeting space. The offeror will propose the time of meetings, and should consider that hotel prices are more expensive during the tourist season, or during major events. Additionally, offerors need to understand that while it is not impossible, it is challenging to recruit teachers to serve on committees from May through August. Other times of the year are much more suitable. The offeror shall propose different schedules if necessary to meet the offeror’s proposed timelines for development. 3) Bias/Sensitivity Review Committees The contractor will support a bias review committee consisting of external educators selected by the state to review the content of passages and test items for potential bias. a) The Bias Committee will consist at a minimum of 12 members. b) Members’ travel, lodging, and meals will be arranged and paid by the contractor. c) Members’ districts will be paid a substitute reimbursement by the contractor for meeting attendance approximately $150 per day. If teachers are not on contract with a district or being paid by another entity, the contractor will provide a stipend to each committee member of at least $100 per day. d) The Bias Committee will meet as often as necessary based on test development schedules to: (i) Review passages recommended for the reading test, (ii) Review items recommended for field test, and (iii) Review items recommended for item bank that have been flagged for DIF. e) The Bias/Sensitivity Review Committees will meet in conjunction with the item review committee meetings.
Page 40 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
f) The offeror should propose a process that will help avoid the issue of having assessment items go through development and be flagged for bias/sensitivity (content review, not empirical DIF analyses) only after substantial investment in development effort. For example, sensitivity review may be done in an explicit step in the item development process by having representation on the development teams (e.g., SES, ELL, other subgroups). It could also be done in conjunction with cognitive labs. 4) Number of Items The contractor will develop sufficient adequate items so that item review committees and the State test development team can have a choice of items to include on the operational forms of the tests. The offeror’s response must include a discussion of the expected yield of items at each stage of the item development process (e.g. initial development, after item committee review, after field testing). The offeror’s proposals should include detail about the numbers of items to be developed under each year of the contract, reflective of the offeror’s proposal for form design and equating. 5) Item Bank The contractor shall develop an item bank that includes a database that provides pertinent information for each item, including answer key/rubric and history (placement, item statistics for all administrations of the item, editing and context). The item bank shall be made available to the state annually at the end of each contract year. F) Test Construction 1) Content of Test Forms For contractor-provided, custom summative assessments, the contractor will coordinate and facilitate support meetings of the state’s test development team and the contractor’s test development team to select items to be included on the operational test forms. The offeror’s proposal must include a description of the proposed process for item selection for the operational and final test forms. The selection and ordering of items on the test forms will be based on appropriate psychometric procedures and will meet the requirements of the Test Specifications. The offeror should propose and discuss psychometric procedures and criteria for item selection and test form development. The state will have final approval of the selection of items, content, and test forms. The state will have final approval of items included on test forms. The state reserves the right to have each test form read by two EED assessment staff to ensure that quality assurance has been addressed. The offeror’s response must include a discussion of an efficient procedure for cycles of item review between the contractor and state assessment staff. 2) Test Booklets The state plans to test all students in grades 3-11 via computer delivery, but will consider utilizing paper and pencil assessments for schools that do not possess the capability to test electronically. The offeror should estimate quantities of test booklets required for students enrolled at each grade using the state’s Technology Readiness Tool (TRT); results will be available to offerors at a date Page 41 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
TBD. Estimates should include at least a 5% overage for districts. Offerors shall propose a method for handling late changes in school or district enrollment. Paper and pencil summative and interim assessment items are being developed by SBAC for grades 3-8 and 11 in English language arts and Mathematics. For proposals supporting SBAC services offerors must produce a plan for distributing, administering, scoring and reporting paper and pencil administrations in grades 3-8, 11 as well as coordinating with SBAC to produce paper pencil assessments for grades 9 and 10. The contractor will produce collated, folded, and saddle-stitched booklets for each grade level. The offeror should review the practice tests on the state’s website for style guidance at http://www.eed.state.ak.us/tls/assessment/SBA_ItemSamplers.html. The format and layout of the test booklet will meet the requirements of a style guide agreed to by the state and contractor during the initial months of the contract. A central component of the style guide will be the application of Universal Design principles and procedures in areas such as the design and layout of the booklet, use of graphics, content, and format of directions to ensure access by the broadest possible population of students. The offeror’s proposal must address methods and procedures used to inform test booklet design and how the booklet design will provide a comparable experience for students as the CBA. 3) Answer Documents The offeror should estimate quantities of answer documents required for students enrolled at each grade using the state’s statistics website http://www.eed.state.ak.us/stats/ . Estimates should include at least a 5% overage of materials. All answer documents will be custom-designed for the state assessment program. The answer document will be consistent with that outlined in the prior section. The offeror should review the practice tests the state’s website for style guidance at: http://www.eed.state.ak.us/tls/assessment/SBA_ItemSamplers.html . A light background grid will be printed on the constructed response areas for all mathematics tests. Light background lines will be printed on the constructed response areas for all reading tests. The offeror’s response should suggest ways to help students navigate through the test and answer document to reduce errors in where students put their answers (e.g., periodic icons to link the item with a position in the answer booklet, reducing or eliminate blank pages, or other suggestions). The offeror’s proposal should include methods and procedures for collecting information for pre-printing student information (e.g., contractor gathering information directly from districts electronically, districts generating preprinted labels according to contractor specifications). The offeror will propose a methodology to work with the state to obtain the best pre-printing information when a district does not complete the pre-printing requirements for student information due to limited capacity. The offeror’s response must discuss options for efficiently and accurately collecting all types of student test information necessary for producing the required analyses for ESEA and state accountability. The offeror’s proposal must discuss the issues and recommend a procedure for Page 42 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
reducing the effort and likelihood of error for capturing student-level demographic and other information such as test form (e.g., avoid having students and/or school personnel fill out information on the English language arts and math demographic page of answer documents). 4) Accommodations The contractor will prepare all Braille test booklets and related test materials. In addition, proposal should include both CBA and P/P large print options for students. The offeror’s response should comment on the research and best practice for providing accommodations for visually impaired students, particularly the issue of multiple sizes of large-print versions. Note that the state has had fewer than 50 per grade-level or 400 total large-print versions and 5 per grade-level or 80 total Braille during recent past test administrations. The offeror will propose the creation, production and distribution of the CBA read aloud accommodations and P/P read aloud accommodations. The proposal should include methods to control for over-orders by districts. The state prefers the P/P accommodation be produced on CDs. Offerors should address issues of test design and presentation, including discussion of accessibility and accommodation principles and procedures for contractor-provided CBA. 5) Ancillary Materials Copies of most ancillary materials are posted on the assessment link of the department website http://www.eed.state.ak.us/tls/assessment/sba.html for the offeror’s review. The state desires a seamless transition between the current contractor and a new contractor, and therefore expects offerors to propose how to ensure this occurs. The following ancillary materials will be maintained and/or produced by the contractor for the state assessments. The offeror’s response should discuss the type of information included in manuals and the type and use of shipping labels and control forms for P/P. Offerors should base estimates for quantities of materials on information provided on the website for number of students, schools, districts, etc. in the state and the Technology Readiness Tool. Estimates should include a 5% overage for all materials. The state will receive a pre-print mock-up/proof for review and approval. The state has final authority to approve all language, content and format of ancillary test materials. a) Test coordinator manual A procedural manual for the district test coordinator will be created, produced, and distributed by the contractor. Two manuals will be produced, one for CBA and one for P/P. An electronic version of the manual in an agreed upon format (e.g., Microsoft Word and Adobe PDF) will be made available to the state for posting. The manual should include information about overall responsibilities of district test coordinators, including, but not limited to: testing accommodations, test security, checklists, preparing computers, and downloading secure servers; P/P manuals should also include content such as
Page 43 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
b)
c)
d)
e)
f)
g) h)
RFP 2014-0500-2154
shipping/receiving of test materials, procedures for ordering additional materials, and P/P specific accommodations information. Test administrator manual A procedural manual for the test administrator will be created, produced, and distributed by the contractor for each grade level. Two manuals will be produced, one for CBA and one for P/P. A single copy of the relevant manual will be provided for every 12 students enrolled. An electronic version of each manual will be available for posting on the state’s website. The manual should include information about overall responsibilities of test administrators, including, but not limited to: testing accommodations, test security, checklists, preparing computers, and downloading secure servers; P/P manuals should also include content such as shipping/receiving of test materials, procedures for ordering additional materials, and P/P specific accommodations information. Mathematics reference sheets (P/P only) The contractor will produce a sheet containing mathematics formulas, etc. These reference sheets will be related to the content assessed at each grade level. One copy will be provided to each student as part of the mathematics assessment materials. These sheets will conform to Universal Design principles as described in Test Construction 2(c). The offeror’s response must address methods and procedures used to inform design. The reference sheet is to be separate from the test booklet. Writing reference sheets (P/P only) The contractor will produce a sheet containing a writing skills checklist. This reference sheet will be related to the assessment rubric for improving student compositions. One copy will be provided to each student as part of the writing assessment materials. These sheets will conform to Universal Design principles as described in Test Construction 2(c). The offeror’s response must address methods and procedures used to inform design. The reference sheet is to be separate from the test booklet. Mathematic tools (P/P only) The offeror will propose a method for providing students with standard mathematics tools such as rulers and protractors when required by the assessment. The tools should be disposable after testing and not returned to the contractor. Shipping labels/forms (P/P only) Shipping labels on cartons and testing materials must be scannable for security and tracking purposes. The offeror shall suggest a labeling system which supports test security regulation and assists districts in shipping and distributing materials at the time of testing to avoid lost or missing materials. Accommodated materials Refer to the Test Construction of the RFP for details. Security Checklists The contractor will provide a method for each school to account for test security practices for CBA. In addition, for P/P administration, the contractor will provide Excel spreadsheets for each district specifying all test/answer book security numbers assigned to districts/sites. This checklist will support the district test coordinator in accounting for all testing documentation. This checklist must be available in hard copy with the testing materials and electronically on the contractor’s secure website.
Page 44 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
The contractor will provide the department two copies of all P/P testing materials that are provided to the districts at the time of administration and secure access to the CBA for each grade. A sample will be sufficient for materials that are individualized for each district/school, such as shipping labels. The department will receive its materials at the shipping date scheduled for districts/schools. Offeror’s should provide samples of such materials with the proposal where possible, to indicate their capacity to design and produce supportive materials that are clear and effective. G) Administration 1) Online Enrollment/Pre-coding a) Pre-code System For all operational assessments, the contractor must provide a pre-code system for the state agency to supply information about all students to be tested at the district level. The contractor’s pre-code system must accommodate the electronic transferability of student-level data using standard data formats, i.e., ASCII, tab-delimited or comma-delimited file types. The pre-code system should include a secure server for receiving file submissions. (i) For the grade level assessments, the state agency will provide the contractor with a pre-code file at a time TBD for the spring test administration. (ii) For all assessments, the contractor shall allow school districts the opportunity to pre-code after the final pre-code deadline upon school district request and at the school district’s expense. The process for pre-coding after the deadline shall run through the state agency’s established pre-code selection process, and the state agency shall submit the final pre-code file to the contractor. The contractor’s pre-code systems must provide the state agency and school districts with the ability to generate pre-code rosters sorted by school district, building, grade level, or classroom teacher. (iii) At a minimum, the contractor’s pre-code system must accommodate data disaggregated for the following subgroups: (a) (b) (c) (d) (e) (f)
Page 45 of 61 Version: Original
Gender Race/ethnicity English Language Learner Economically disadvantaged Students with disabilities Specific testing accommodations
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
(iv) If necessary to meet state or federal requirements, the contractor shall provide the state agency with the ability to add additional subgroups for disaggregation, at no additional cost to the state agency. 2) Web-Based Enrollment and Order Collection System For each student tested, the contractor shall collect demographic data and other pertinent information in order to meet or exceed the requirements of the Alaska’s ESEA flexibility waiver. The contractor must collect data via a web-based enrollment and order collection system that meets the requirements of the state agency’s Technical Requirements document as described herein. a) The contractor’s enrollment and order collection system must provide the following: (i) School and school district access via a secure login ID and password with an industrystandard web browser (ii) Ability for schools and school districts to submit anticipated testing dates (iii) Industry-standard secure data transmission, storage, and retrieva (iv) Ability for schools to submit and update the following data for each test administration: (a) (b) (c) (d) (e) (f)
Contact information for school staff Critical school calendar dates Anticipated testing start and end dates Shipping address for testing materials Mailing address for reports and student data Billing address and process for districts to submit purchase orders
(v) Ability for schools and school districts to verify and/or update their previously entered participation data up to one month prior to testing (vi) Ability for state agency personnel to securely access (view, query, and export) all data entered by schools and school districts through the enrollment collection periods (vii) Ability to interface with packing and shipping programs and systems used for shipping and distribution of Braille and Large Print testing materials. (viii) Ability for schools to enroll students until time of testing. 3) Online Administration Requirements -- General Student Population The contractor shall provide a secure web-based assessment delivery and return system for schools to administer assessments online to the general student population, with the exception of those requiring accommodations as described herein. The contractor shall provide all necessary processes and platforms for online assessment administration and submission of testing materials. The contractor’s online administration system must either include an adaptive platform capable of deploying SBAC-developed assessments and obtaining the SBAC platform certification, or must have the capability to deploy SBAC’s open source adaptive engine for administration of SBACdeveloped assessments. (SBAC’s Architecture Review Board is currently considering methods to establish comparability among test administration platforms. Additional information is available Page 46 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
on the SBAC website at http://www.smarterbalanced.org/smarter-balancedassessments/technology/. By July 1, 2014, the contractor’s adaptive platform must be certified in accordance with SBAC’s interoperability requirements. a) The administration window shall be determined by the state agency. For grade level assessments the state agency anticipates a four (4) week administration window from the first week in April through the end of April. b) The state agency estimates, but does not guarantee that grade level assessments will be administered annually to approximately 10,000 students per grade level. 4) Identifying districts and schools and grade level counts a) The state assessment staff will provide the contractor with a database of districts and schools participating in the assessment by a date agreed upon by the contractor and the state. b) The offeror shall propose a method for pre-coding (pre-slugging) student response documents and for ensuring the accuracy of this pre-coded information. The state prefers a color coding process for labels to avoid district and school error when preparing student answer documents. c) The offeror shall propose a method for obtaining from districts accurate statewide projected grade-level enrollments by a date agreed upon by the contractor and the state. The time period should allow for the state to review the numbers to determine if they are outside the numbers that would be expected in an order in the first year. After the first year of the contract, the contractor should have a methodology to determine if orders are significantly outside what would be expected based on prior year orders. d) The offeror shall include a web-based system for districts to enter information such as enrollment by grade/school, and to confirm information provided by the state such as contact information, and grade configurations as well as pre-coding (pre-slugging) information. The system should have appropriate levels for viewing and changing information, and have appropriate security. Changes should have confirmation notices sent to the responsible party in the school/district, and a summary accessible by the state. The system should allow for larger districts to electronically submit enrollment and pre-code information via a commaseparate values (csv) file. The system should also provide larger districts with more than 10,000 students additional time for submission to reduce the number of changes due to transfers within the district. 5) Security of all testing materials The contractor will maintain and monitor security procedures to ensure test items are not revealed to anyone other than the student on testing day. The offeror will provide assurances that the state’s security regulations are followed. Refer to the state’s website for testing regulation information http://www.law.state.ak.us/doclibrary/doclib.html . 6) Shipping Requirements (P/P only) a) The contractor will ship test materials directly to district offices care of district appointed test coordinators. Page 47 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
b) Test materials will arrive in districts in a 4-day window at least 4 weeks before the first day of the testing window. c) Notification/Tracking: The contractor will notify districts via e-mail of all shipments. Districts will be able to track shipments online. d) The contractor will notify the state of shipment/delivery of all materials and the status of undelivered materials. e) The contractor shall keep a log of complaints and issues, how they were resolved, and an indication of customer satisfaction. The log shall be submitted to the state for review weekly at the start of the shipping and terminating after return of materials. The offeror’s proposal shall include a description of options that include the use of technology. f) Districts will gather from schools and will ship all secure materials directly to the contractor following testing. The offeror’s response must describe the proposed method of shipping that includes an awareness of the unique needs of the logistics in Alaska. g) The offeror will propose a method to account for the return of all secure testing materials. The offeror’s response must include a description of methods and procedures used to track shipments from districts and follow-up with schools/districts that have not returned materials. h) The contractor will notify the state of the status of the return of all secure test materials. The offeror’s response must include a description of the procedures used to gather information and anticipated timeline for providing the information. i) The state has worked with RAMIS, a service of Data Collection Partners, to pilot a barcode scanning system for the past two years. The intent of this type of system is to ensure that tests remain secure and accountable at all times during the testing administration. Districts will decide if they are going to contract work with a scanning service. The state desires the contractor propose a methodology to exchange the barcode data on a file format specified by the scanning vendor(s), will take place before the arrival of testing materials in the district for the spring and fall administrations. (Date TBD). H) Support 1) Workshops and Trainings The contractor will support administration workshops in the state approximately one to two months, or 8 to 10 school weeks, prior to administration. Department staff will conduct the workshops with the contractor’s assistance and support. The contractor should budget for at least 2 two-day workshops per year in Anchorage, AK. The state will pay for the attendance of district and state staff, but the contractor will pay for all contractor costs and preparation and shipping of materials. The contractor should budget for up to four additional focused trainings using technology, e.g., training modules, and/or video/audio/WebEx conferencing technology. The additional trainings would be for issues that lend themselves to focused training. 2) Phone support The contractor will provide toll-free telephone support to districts/schools from the time of the start of CBA procedures or first shipment of test materials in the fall (whichever comes first) through the end of the testing window. For four weeks prior to the test window, during the test window, and until shipping is complete the telephone support must be available until at least Page 48 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
8:00am - 4:00pm A.S.T. on all work days. The contractor shall have an email support system that is available throughout the year. The offeror’s response shall discuss options for staffing the support center, training support personnel, and duration of support. The toll-free telephone support must be in addition to any other proposed support system, e.g., web-based or email-based. The contractor will provide ongoing support throughout the year to answer questions related to testing, procedures, and work with individual sites as needed on testing and issues related to the return of assessment results. I) Scanning/Imaging and Scoring (P/P) 1) Scanning/Imaging a) The contractor is responsible for the efficient, accurate, and reliable scanning and/or imaging of all student responses on P/P assessments, including student responses to multiple-choice test and questionnaire items, student responses to constructed-response items, student identification information, and student demographic information provided by the student and/or school principal. The contractor must provide details regarding the accuracy and reliability of the scanning technology/system. In addition, the contractor is responsible for scanning or imaging all ancillary materials, as appropriate. b) The contractor will capture images of all student responses to constructed-response items and store those images so that they can be efficiently linked to and retrieved on the basis of student and school identification information, scores, and item information. c) The contractor will demonstrate that programs have been prepared to accurately scan and image all test materials. d) The contractor will ensure that the scanning database is error-free and contains valid responses in all fields. e) The contractor will provide the state project management team with a detailed report describing any materials that could not be scanned due to damage caused by the school or contractor or other reasons. 2) Multiple-choice items a) All multiple-choice items are machine-scored. b) Special reporting requirements The contractor will provide the state with reports of excessive blank answer documents, item non-response, and multiple marks at the school and district levels. c) In order to investigate erasures on student answer documents for the multiple-choice portions of the summative assessment, the state and contractor will develop procedures. In addition to the procedures the offeror will complete an erasure analysis of the summative assessment for each administration. Based on the criteria for excessive erasures (i.e., a rate of 3 times or 4 times the state standard deviation), the contractor will produce the following reports:
Page 49 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
(i) District/School Erasure Analysis Report (sorted by district). This report contains summary data with analysis of wrong-to-right erasures, right-to-wrong and wrong-to-wrong and standard deviation criterion. (ii) Student Erasure Analysis Report for students identified as having excessive erasures. This report contains student demographic information, an item-by item analysis of wrongto-right erasures, right-to-wrong and wrong-to-wrong and standard deviation criterion. 3) Constructed-response items a) Qualifications of scorers The offeror’s response will include a description of the qualification and experience of the scorers proposed for the state’s assessment program. b) The offeror’s response must include a description of the scoring procedures and development of scoring materials at all stages of the scoring process including, but not limited to, the selection of anchor papers, the development of qualifying and training materials, the “tuning” of scoring rubrics, and the development of calibration sets. c) The state assessment staff will have final approval of all materials prepared for scoring. d) Training/Qualification of scorers The offeror’s proposal will provide details on the processes and procedures used to train scorers and qualify scorers for participation in state scoring. e) Monitoring scorers The offeror’s response will provide details on the quality control processes used to monitor scoring rates and accuracy. The response will also provide details on processes used to identify scorers for retraining or removal and processes used to invalidate scores from particular scorers. This response should include a 10% rate of double-scoring, selection of responses for double scoring, etc. f) Information available from scoring The contractor will propose policies for the type and frequency of information provided from the scoring process (within and across scoring sites) to the state. g) The state Assessment Director shall have the right to request, “on-demand” within four hours any regular report, and to perform unannounced site visits to scoring centers. h) The contractor will provide for representatives or agents from the state to be present at the scoring site(s) during selection of anchors, scoring qualification, training, and initial scoring. The offeror’s response must discuss the issues of making it possible for state oversight with a very limited staff. J) Analysis The contractor is responsible for conducting all analyses necessary to report student, school, district, and state results from the state assessment program and to ensure that tests meet the standards of technical quality. During each year of the contract, based upon negotiations with the state project director and the state technical advisory committee, the contractor will conduct analyses necessary to support (when applicable) test development, test construction, scoring, as well as standard-setting and validation activities as might be necessary. In addition, the contractor will conduct secondary analyses related to security, data interpretation, policy formation, and administrative planning.
Page 50 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
1) Calibration and Scaling a) The contractor will calibrate test items using an appropriate item-response theory model(s). The contractor’s response must include a discussion of the benefits of the proposed IRT model and indicate which software will be used. b) The contractor will translate student scores to a reporting scale developed for the state assessments. The offeror’s response must propose alternative methods for creating a score scale consistent with the reporting requirement contained in the Test Specifications. c) The offeror’s response should propose a method for maintaining the vertically aligned standards that exist in the SBAC assessments in English language arts and mathematics. 2) Equating a) The contractor will design and conduct analyses required to equate the results of the state tests from year to year. The offeror’s response must describe and provide a rationale for the proposed method for equating the tests. Further, the contractor will be required to use a representative sample of the test items for the equating analyses or the contractor must provide a scientific justification for proposing an alternative model. b) If the offeror proposes using multiple forms within a year, the offeror’s proposal should address how the offeror will design and conduct analyses required to calibrate and link test items across test forms within a single year. The offeror’s response must demonstrate an understanding of the test design and describe the method proposed for accomplishing this task. 3) Item Evaluation a) Multiple-choice Items The contractor will produce item statistics for all field test items. The offeror’s response must include a description of the item statistics that should be generated to assist in the evaluation of field test items and how these item statistics will be used to either make or assist in the decisions about items. b) Constructed-response Items The contractor will produce item statistics for all field test items. The offeror’s response must include a description of the item statistics that should be generated to assist in the evaluation of field test items. 4) Test Construction a) The contractor will conduct analyses to support the construction of technically sound test forms. The offeror’s response must include a description of the types of analyses that will be conducted and how the results of those analyses will be used. b) All offerors shall provide evidence to support the assertion that the proposed system will meet the requirements for standards-assessment alignment as specified by the United States Page 51 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Department of Education peer review guidance 5) Reporting The contractor will design and conduct all analyses necessary to produce student, school, district, and state results and other information included in published reports of results. Reporting requirements are described in the Technical Reports on the department’s web site. 6) Additional Analyses a) If any changes are made to the test forms or program, the contractor will conduct a one-time analysis that relates the results from this assessment program with the results of the state’s previous statewide assessment program. The offeror’s response must include a description of the methods proposed for accomplishing this task. b) The contractor will document aspects of development, administration, and reporting that enhance the validity of the assessments for their intended uses. The offeror’s response must include a description of the methods proposed to document validity. c) The contractor will support external validity studies commissioned by the state. The contractor should budget at least $50,000 per year for this task. The offeror’s response must include a discussion of the types of studies that should be conducted to establish the validity and credibility of the assessment program. Issues of validity in relation to accessibility and/or accommodations should be addressed in the offeror’s response. K) Reporting The contractor is responsible for the reporting of results of the state assessments according to the schedule specified in Section 1(c) of this RFP. Specific reporting dates that meet those guidelines for each administration will be agreed upon prior to the beginning of each fiscal year. The establishment of final reporting dates each year will allow the establishment of intermediate milestones that include, but are not limited to, a) the completion of scoring and processing, b) the development, review, and approval of reporting specifications, and report shells, c) the review and approval of equating analyses, d) the delivery, review, and approval of preliminary data files, and e) the delivery, review, and approval of sample reports. Penalties for failing to meet final reporting dates and intermediate milestones will be negotiated prior to the signing of a contract. 1) Reporting of Results a) The contractor will provide reporting of common assessment results based solely on information collected during the assessment including demographic information collected prior to the assessment through a pre-coding procedure. b) The contractor will provide the following reports in paper and/or electronic format as indicated below: (i) Parent/Guardian reports for individual students containing performance level results, scaled score, and guides for interpretation, etc. (2 paper copies per student) (ii) Classroom summary and roster (1 paper copy per school and database) Page 52 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
(iii) School summary report containing whole school performance level results, sub-score results as specified in the Test Specifications, and guides for interpretation for educators (1 paper copy for school and 1 for district, and database) (iv) School subgroup report that includes performance of all required subgroups (1 paper copy for school and 1 for district, and database) (v) District summary report containing whole district performance level results, sub-score results as specified in the Test Specifications (1 paper copy for district, and database) (vi) District subgroup report that includes performance of all required subgroups (1 paper copy for district, and database) (vii) State summary report containing statewide performance level results, sub-score results as specified in the Test Specifications (electronic report, and database) (viii) State subgroup report that includes performance of all required subgroups (electronic report, and database). c) Above mentioned reports will be maintained on a password protected, secure reporting website for posting reports that is accessible by both state and district employees for at least the two most recent test administrations. d) The contractor shall provide the department two copies of what was sent to schools/districts using PDF files of each report sent to schools burned onto a CD. The copy will be provided to the state before the reports that are provided to districts electronically/online are removed from the online system, therefore allowing the state to always have on line backup copies to provide districts as necessary. e) The contractor will provide a student-level database containing complete student records (as described below in “Providing Data Files”) for all students participating in the assessment system. f) The contractor will support reporting and analysis workshops in the state. The contractor should budget for five to six audio and web enhanced workshops to be held at times agreed to by the contractor and state if necessary. Estimates of total participants should be based on one participant per district, and therefore a minimum of 8 participating districts per audio/web enhanced meeting. g) The contractor will develop and produce interpretive materials for parents and schools/districts. The interpretive materials will be provided in electronic format for posting on the state website. The offeror’s response must include a description of the type of information to be included in such materials and methods to increase the usefulness of such materials. h) The offeror shall propose producing data files in a format suitable for secondary analysis to school districts. 2) Providing Data Files and Technical Reports The successful offeror shall provide the state and each district with the following electronic files. The data file should be offered in both a complete and an abbreviated format: a) Student data files, which must include the following information:
Page 53 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
(i) Demographic data elements required to meet ESEA report card requirements. Elements should match EED Online Alaska Student Information System (OASIS) codes. (ii) Summative assessments should have a process for identifying and combining multiple student records that exist for individuals that have multiple test booklets. (iii) Mechanism, when needed, to provide the districts with a system of tracking individual paper booklets to student names to allow for the resolution of mismatched test results for the same individual. (iv) Research file must accommodate use of a 10-digit code for the Alaska Student Identifier Number. It must also accommodate the use of a separate code for the number that a district uses to identify students within the district. b) Group level data files if applicable (district and school) by assessment type, and c) Research File Layout file by assessment. Each assessment file shall be submitted in a tabdelimited format. Data transmission of this file must be delivered through a secure FTP site or a secure on-line system proposed by the contractor. As much as possible the Research File Layout should be consistent to allow the least amount of disruption to the districts. The Research File Layout shall include at a minimum: (i) Variable name (ii) Start count (iii) End count (iv) Length of field (v) Detailed definition of the variable d) Abbreviated file will be in .csv format and will contain headers, complete data file will be in fixed width format. Review updates of the Research Tape specifications and layout shall occur yearly by the contractor and the state. Any modified Research File Layouts will be made available to the districts with enough time to accommodate changes in their systems. The contractor shall provide (1) hard copy and one (1) electronic of a Technical Report annually to the department. The Technical Report shall provide documentation of contract activities and results sufficient to inform inquiries by a technical audience. The majority of the reports outlined in this section are online at the state’s assessment link on the department website. Because the state expects a seamless transition to the new contractor from the current contractor the state encourages offerors to view the information online. 3) Defense of Technical Quality The contractor shall be available to testify and defend the technical quality of the assessments if necessary in a civil or administrative action filed against the state.
Page 54 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
4) Providing Data Analysis Software and Training to Districts/Schools The offeror shall propose, as outlined in a previous section, a system that allows for districts to perform additional analysis. The state currently uses a system provided through a subcontractor and expects any new system to be equal or greater to what is allowed within that system. 5) Standard Setting/standard verification a) Standard setting/standards verification will be conducted within two months following any operational administration of the state assessments that is significantly different than the current program. b) The offeror shall propose a standard setting or standards verification approach with a clear justification of the particular method(s). The offeror must consider the need for standard setting or verification in the development of new forms, or change to current forms, consistent with any new laws, such as reauthorization of ESEA. The offeror shall, in addition to discussing the merits of the particular approach, discuss the potential shortcomings of the proposed method. Further, the offeror’s approach shall take into account that the state‘s existing assessment system has been classifying students into performance levels for several years. c) The offeror shall present a plan for evaluating the validity of the standard setting method in subsequent years. d) The offeror’s response must include a comprehensive description of the proposed standard setting method that includes procedures to occur before, during, and following the formal standard setting process. e) The contractor: will support all standard setting activities including, but not limited to, providing any stipends and covering expenses for participants in the standard-setting process.
5.02 DELIVERABLES Deliverables are outlined in the Scope of Work, State Test Specifications, and deliverable dates will be determined during contract negotiations and the contractual period of performance. Refer to section 1.02 for deliverable time frames. All materials must be reviewed and approved in advanced as outlined in the Scope of Work section 5.01, as well as other sections of the RFP. Refer to the Technical Reports that include the Alaska General Test Specification for information and deliverable timelines on the test forms. Find in the test specifications the operational test windows.
Page 55 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
6. SECTION SIX PROPOSAL FORMAT AND CONTENT 6.01 PROPOSAL FORMAT AND CONTENT The state discourages overly lengthy and costly proposals, however, in order for the state to evaluate proposals fairly and completely, offerors must follow the format set out in this RFP and provide all information requested. The proposal must include: A. Introduction (See §6.02) B. Understanding of the Project (See §6.03) C. Methodology Used for the Project (See §6.04) D. Management Plan for the Project (See §6.05) E. Experience, Qualifications, and Corporate Capability (See §6.06) F. Cost Proposal (submitted separately using budget forms in Appendix D) (See §6.07) G. Required Appendices and Cover Materials (Must include budget form & task allocation forms from appendices D and C.) (See §8.0)
6.02 INTRODUCTION Proposals must include the complete name and address of offeror’s firm and the name, mailing address, telephone number, and email address of the person the state should contact regarding the proposal. The offeror’s proposal shall include an introduction that briefly describes the contractor’s approach for completing the tasks required for this project and should demonstrate the contractor’s overall understanding of the required tasks and the needs of Alaska. This section of the contractor’s response should also introduce any alternative methods or additional tasks that the contractor plans to propose to successfully complete the project. Proposals must confirm that the offeror will comply with all provisions in this RFP; and, if applicable, provide notice that the firm qualifies as an Alaskan offeror. Proposals must be signed by a company officer empowered to bind the company. An offeror's failure to include these items in the proposals may cause the proposal to be determined to be non-responsive and the proposal may be rejected.
6.03 UNDERSTANDING OF THE PROJECT Offerors must provide comprehensive narrative statements that illustrate their understanding of the requirements of the project and the project schedule.
6.04 METHODOLOGY USED FOR THE PROJECT
Page 56 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
Offerors must provide comprehensive narrative statements that set out the methodology they intend to employ and illustrate how the methodology will serve to accomplish the work and meet the state’s project schedule. The offeror’s proposal will describe the specific approach and plans for accomplishing the scope of work called for in the RFP. The response must provide sufficient detail to allow the state to evaluate the proposed methods. The proposal regarding support services for the SBAC assessments must identify separate costs and tasks required for delivering the grade 9 and grade 10 summative and interim assessments. The offeror must respond to each required task described in the scope of work as well as provide descriptions of additional tasks that the contractor determines are necessary for the successful completion of the project. If the contractor proposes alternative or modifications to the specific tasks described in this RFP, each task must be fully described and clearly identified.
6.05 MANAGEMENT PLAN FOR THE PROJECT Offerors must provide comprehensive narrative statements that set out the management plan they intend to follow and illustrate how the plan will serve to accomplish the work and meet the state's project schedule.
6.06 EXPERIENCE, QUALIFICATIONS, AND CORPORATE CAPABILITY Offerors must provide an organizational chart specific to the personnel assigned to accomplish the work called for in this RFP; illustrate the lines of authority; designate the individual responsible and accountable for the completion of each component and deliverable of the RFP. Offerors must outline a plan that includes staffing, the allocation of persons and departments by FTE across the major tasks to be completed. Any individual assigned to the project 0.5 FTE or more must be named in the offeror’s response and his/her vita must be submitted with this proposal. The information will be provided on the task allocation forms provided in Appendix C. The offeror’s response will include descriptions of experience and vitae for all individuals to fill key functions within this project. Throughout the course of the project, the state retains the right of approval of individuals assigned to key management and test development positions within this project. The state also reserves the right to request a change in staff when performance is determined by the project director to be inadequate and the errors in judgment and/or performance may jeopardize district and state relations or state and federal government relations. The response must include a description of the corporate capability of the prime contractor and all proposed subcontractors that will perform key functions of this project. Subcontractors include individuals and organizations performing tasks directly related to the educational testing/measurement Page 57 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
such as item development, standards setting or validation, as well as individuals and organizations performing tasks such as printing and shipping, creation of CDs and DVDs for student accommodations, or preparing Braille or large print versions of the assessment. Each corporate capability statement must address the contractor’s qualifications, background, experience, and capacity to perform the tasks required for the successful completing of this project. The response should include evidence of success in working with other large-scale assessment contracts that have employed approaches similar to those proposed by the offeror in their Scope of Work. The offeror’s proposal must include a list of all large-scale assessment projects the organization is currently or has been involved as a prime contractor or subcontractor since July 1, 2004. The list should include a short description of the responsibilities, dates engaged, and total dollar amount of the contract as well as contact information for each contract (i.e., contact name, affiliation, phone number, email address).
6.07 COST PROPOSAL C Cost proposals must include an itemized list of all direct and indirect costs associated with the performance of the contract, including, but not limited to, total number of hours at various hourly rates, direct expenses, payroll, supplies, overhead assigned to each person working on the project, percentage of each person's time devoted to the project, and profit. The proposal must include narrative describing the basis for costs in each of the major task areas. To allow comparison across contractors, budget information must be provided in the forms provided in Appendix D. Contractors must propose the budget in such a manner that allows the state to determine those costs dedicated to each of the following programs separately: support services for SBAC summative assessments in grades 3-8 and 11, support services for SBAC interim assessments in grades 3-8 and 11, support services for SBAC summative assessments in grades 9 and 10, support services for SBAC interim assessments in grades 9 and 10; and interim assessment. Proposals for custom-developed assessments or commercially available assessments must also include the costs broken out separately for summative and interim assessments for grades 3-8 and 11 as well as for grades 9 and 10.
6.08 EVALUATION CRITERIA All proposals will be reviewed to determine if they are responsive. They will then be evaluated using the criterion that is set out in Section SEVEN. An evaluation may not be based on discrimination due to the race, religion, color, national origin, sex, age, marital status, pregnancy, parenthood, disability, or political affiliation of the offeror. A proposal shall be evaluated to determine whether the offeror responds to the provisions, including goals and financial incentives, established in the request for proposals in order to eliminate and prevent discrimination in state contracting because of race, religion, color, national origin, sex, age, marital status, pregnancy, parenthood, or disability. Page 58 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
7. SECTION SEVEN EVALUATION CRITERIA AND CONTRACTOR SELECTION THE TOTAL NUMBER OF POINTS USED TO SCORE THIS PROPOSAL IS 100
7.01 UNDERSTANDING OF THE PROJECT (5 PERCENT) Proposals will be evaluated against the questions set out below: [a] How well has the offeror demonstrated a thorough understanding of the purpose and scope of the project? [b] How well has the offeror identified pertinent issues and potential problems related to the project? [c] To what degree has the offeror demonstrated an understanding of the deliverables the state expects it to provide? [d] Has the offeror demonstrated an understanding of the state's time schedule and can meet it?
7.02 METHODOLOGY USED FOR THE PROJECT (10 PERCENT) Proposals will be evaluated against the questions set out below: [a] How comprehensive is the methodology and does it depict a logical approach to fulfilling the requirements of the RFP? [b] How well does the methodology match and achieve the objectives set out in the RFP? [c] How well does the methodology ensure security for the assessment system? [d] How well does the methodology address technical soundness necessary to implement an effective, large scale assessment? [e] Does the methodology interface with the time schedule in the RFP?
7.03 MANAGEMENT PLAN FOR THE PROJECT (20 PERCENT) Proposals will be evaluated against the questions set out below: [a] How well does the management plan support all of the project requirements and logically lead to the deliverables required in the RFP? [b] Is the organization of the project team clear? [c] How well does the management plan illustrate the lines of authority and communication? [d] To what extent does the offeror already have the hardware, software, programming needs, equipment, and licenses necessary to perform the contract? [e] Does it appear that the offeror can meet the schedule set out in the RFP? [f] Has the offeror gone beyond the minimum tasks necessary to meet the objectives of the RFP in that has the offer provided more systematic approaches to enhance the RFP objectives or improve the efficiency of the project and/or quality of the materials produced? [g] To what extent has the offeror identified potential problems? Page 59 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
RFP 2014-0500-2154
7.04 EXPERIENCE, QUALIFICATIONS, AND CORPORATE CAPABILITY (15 PERCENT) Proposals will be evaluated against the questions set out below: Questions regarding the personnel: [a] Do the individuals assigned to the project have experience on similar projects? [b] Are vitae complete and do they demonstrate backgrounds that would be desirable for individuals engaged in the work the project requires? [c] How extensive is the applicable education and experience of the personnel designated to work on the project? Questions regarding the firm: [d] How well has the firm demonstrated experience in completing similar projects on time and within budget? [e] How adequate are the physical resources of the firm? [f] How successful is the general history of the firm regarding timely and successful completion of projects? [g] If a subcontractor will perform work on the contract, how well do they measure up to the evaluation used for the offeror? (Note: offerors that do not utilize subcontractors will receive a score equal to 80% of the maximum value assigned to this question.)
7.05 CONTRACT COST (40 PERCENT) Overall, a maximum of 40% of the total evaluation points will be assigned to cost. The cost amount used for evaluation may be affected by one or more of the preferences referenced under Section 2.13.
Converting Cost to Points The lowest cost proposal will receive the maximum number of points allocated to cost. The point allocations for cost on the other proposals will be determined through the method set out in Section 2.15.
7.06 ALASKA OFFEROR PREFERENCE (10 PERCENT) If an offeror qualifies for the Alaska Bidder Preference, the offeror will receive an Alaska Offeror Preference. The preference will be 10 percent of the total available points. This amount will be added to the overall evaluation score of each Alaskan offeror.
Page 60 of 61 Version: Original
STATE OF ALASKA Alaska English Language Arts and Mathematics Assessments
8. SECTION EIGHT ATTACHMENTS 8.01 ATTACHMENTS Attachment 1. Attachment 2. Attachment 3. Attachment 4. Attachment 5. Attachment 6. Attachment 7. Attachment 8. Attachment 9. Attachment 10. Attachment 11. Attachment 12. Attachment 13. Attachment 14. Attachment 15. Attachment 16.
Page 61 of 61 Version: Original
Proposal Evaluation Form Appendix A: Standard Agreement Form Appendix B1: Indemnity and Insurance Requirements Appendix C: Task Allocation Form Appendix D: Budget Forms Appendix E: Alaska Standards English Language Arts Appendix F: Alaska Standards Mathematics Appendix G: Districts & Schools Appendix H: 2012-13 Statewide Enrollment by Grade Federal Debarment Certification Form Sample Notice of Intent to Award Proposal Checklist & Document Order Proposal Cover Sheet Offeror's Certification Preference Worksheet Conflict of Interest
RFP 2014-0500-2154
