Boot Camp To Improve Student Perception and ... - ACS Publications

---

Article pubs.acs.org/jchemeduc

Boot Camp To Improve Student Perception and Performance in Sophomore Organic Chemistry? Hoorah! Matthew R. Siebert,*,† Todd E. Daniel,‡ and Brian D. High† †

Department of Chemistry, Missouri State University, 901 S National Avenue, Springfield, Missouri 65897, United States RStats Institute, Missouri State University, 901 S National Avenue, Springfield, Missouri 65897, United States

‡

S Supporting Information *

ABSTRACT: The sophomore organic chemistry sequence (Organic I and II) has a reputation: one that has prospective students worried. Their anxiety is well-founded, given estimates of the national success rate (defined as a grade of “C” or better) of 50−75%. In an attempt to improve both student success and student opinions about chemistry, we implemented a one-week “boot camp” (preparatory) course that we called Preparing for Organic Chemistry. This course reviewed the concepts of general chemistry required for success in Organic Chemistry I and II. We found improvements in student opinion of Organic Chemistry I (both before starting and after taking the course). Further, we have found that the boot camp had a measurable and statistically significant impact on student success rates. KEYWORDS: Second-Year Undergraduate, Curriculum, Organic Chemistry, Analogies/Transfer, Multimedia-Based Learning, Testing/Assessment, Enrichment/Review Materials

T

POC was conceived as an intersession course (also known as a mini-course or interim course), curricula for which have appeared in the pages of this Journal for many topics.26−30 Many such courses in chemistry are devoted to advanced topics with activities requiring small class sizes. Others appear to have implemented the concept of an Organic Chemistry I boot camp before us, including at the University of Massachusetts Amherst and Eastern Kentucky University.31,32 However, the impact an Organic Chemistry I boot camp course has on student success has never been reported in the primary literature. Similar boot camp style courses have been reported for general chemistry. A recent commentary detailed plans of the Howard Hughes Medical Institute and the Posse Foundation to incorporate a two-week boot camp style program for incoming freshman aimed at recruitment and retention of underrepresented minority students.33 In that commentary, it was noted that there is a disparity between a student’s expectations of difficulty and the realized rigor of university science. The Massachusetts Institute of Technology has also realized the potential of boot camp general chemistry experiences. They offered a three-week boot camp for their general chemistry laboratory class to create a 14-episode reality video series. The video series was aimed at recruiting under-represented minority students with the ultimate “prize” (as described by Cooper) of placement in a campus research laboratory.34

he one-year (“sophomore”) organic chemistry course is historically viewed as very difficult. Estimates of student success approximate that 25−50% of the students who take first-semester organic chemistry (Organic Chemistry I; OC-1) do not pass the course.1 Some instructors have attempted to address this deficit by implementing pedagogical approaches such as flipped2−11 classes that reverse the traditional roles of in-class and out-of-class work, peer-led instruction that creates an environment for small groups of students to work under the guidance of a peer,12−19 or POGIL that offers students learning through guided inquiry.20−24 In 2003, Paul G. Jasien published an analysis of data collected at California State UniversitySan Marcos (CSUSM) over the course of five years.25 Pre- and postcourse surveys were administered to first-semester organic students. Myriad categories were tested for correlation to success (therein defined as a grade of “C” or better, i.e., passing the course at CSUSM). Of note are the following conclusions: (1) entering first-semester organic students are more likely to succeed if they perceived their general chemistry preparation as “exceptional”, and (2) entering first-semester organic students are more likely to succeed if they defined success in the course as making an “A” (compared to “B”). Our conjecture is that student performance in OC-1 at Missouri State University can be affected by introducing a “boot camp” or “preparatory” course, which was titled “Preparing for Organic Chemistry” (POC) and offered during the intersession period. Drawing parallels to “boot camp”, this preparatory course should be an intense experience that lays a strong foundation for success in upcoming engagements (here OC-1). © XXXX American Chemical Society and Division of Chemical Education, Inc.

Received: December 13, 2016 Revised: August 3, 2017

A

DOI: 10.1021/acs.jchemed.6b00963 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

paper on the basis of their personal experiences teaching general and organic chemistry as well as by a survey of general chemistry review topics present in the early chapters of organic chemistry textbooks. The topics selected for POC match both informal and formal surveys of topics reviewed in organic chemistry.37−39 Class met every weekday from 9:00 AM to 12:30 PM, and the focus was on subject mastery. There was no textbook required, and grading was biased toward participation, discussion, and demonstration of knowledge. The overall grade in POC was determined by classroom responses (via “clickers”; 70%) and by online worksheets (administered via learning management software; 30%). For those questions posed in class, students earned partial credit for any response (80%) while those students that came up with the correct response received full credit. Each substantive topic (see Table 1) had an accompanying online worksheet with 10−20 questions. The worksheets were to be completed after class, and multiple attempts were allowed so that students could reflect on their response and incorporate feedback with many opportunities to prove their mastery of the subject. Given the fact that the grading schema are very forgiving to the students, no attempts at POC have been unsuccessful (a few students had dropped after 1 or 2 days). Perhaps if a student, in the future, struggles substantially with the topics in POC (or is unsuccessful), advisement could intervene with recommendations against proceeding into OC-1 in the subsequent semester.

An intervention program for incoming freshmen to address preconceptions of difficulty and effort in university science has also been reported in the field of biology. The Biology Intensive Orientation for Students (BIOS) program at Louisiana State University is a boot camp course aimed at incoming freshmen who will be taking Introductory Biology for science majors.35,36 Reports show that students perform significantly better on their first exam (mean approximately 10% higher), a trend that is also reflected in the students’ final percentage in the class (mean approximately 6% higher).36 Although the original intent of the BIOS program was to impact success in Introductory Biology for Science Majors, Pomarico, Wischusen, and Wischusen note long-term effects on retention within the biology major (as projected based on student performance at the end of the fourth semester).35 Recruitment and retention of under-represented minority students, increased success in OC-1, as well as long-term success of STEM students who take OC-1 are all worthwhile reasons to create the boot camp style course POC. Herein we present our concept of POC, including the intent, curriculum, and outcomes of the course at Missouri State University.

■

COURSE STRUCTURE AND ADMINISTRATION POC was offered as a 1 credit-hour intersession class in the fall of 2014 and 2015. Students registered and paid for the course, which was administered the week immediately prior to the fall OC-1 course. The intersession course is considered, at this university, to occur during fall semester. This means that the additional hour from POC and associated fees are incorporated into the fall term. Despite the fact that this course does not contribute directly to any major or to general education requirements, the course appears to be very popular [greater than 25% of those attempting OC-1 (45−46 students) took POC the preceding intersession for both 2014 and 2015]. Given the schedule of topics for POC (see Table 1), a grade of

■

DEFINITION OF SUCCESS, QUALIFICATION FOR STUDY, AND CLASS MAKEUP Our definition of success in OC-1 is when a student has obtained a grade sufficient to move on to Organic Chemistry II (“C−” or better). However, since students who are ultimately unsuccessful tend to drop a class prior to completing all of the course material (and before receiving a final grade), prediction of success based on criteria other than the final course grade was investigated. The data collected included the following: • Graded course materials, including scores on each of the four exams, one final exam, class attendance, laboratory report scores, laboratory exam scores. • Data obtained directly from the university, including gender, major, class standing (freshman, sophomore, etc.), grade point average (GPA), and ACT test score. Table 2 shows the college and class makeup. The class makeup is corrected by removing any students who did not take exam one in OC-1 and any students known to be repeating OC-1 (four total). Of note in Table 2 is the apparently large disparity in OC-1 success rates of the 2015 cohort when comparing those that took POC (88.9% success) with those that did not take the boot camp course (74.1% success).

Table 1. POC Intersession Course Topics Day

Topics

1

Course Introduction How General Chemistry Differs from Organic Chemistry Organic Functional Groups Trends in the Periodic Table Electron Configurations Orbital Diagrams Bond Types and Polarity Molarity, Concentration, and Dilution Stoichiometry Bonding Theories Intermolecular Forces Solutions Chemical Equilibria Acid/Base Chemistry Rate Laws

2

3

4

5

■

STATISTICAL ANALYSIS OF STUDENT SUCCESS As a preliminary screen, we sought to determine if there existed a statistically significant relationship between completion of POC and success in OC-1. In the 2014 cohort, there was no statistically significant relationship between POC completion and success; 80.4% of the students who completed POC were successful, while 83.5% of those who did not take POC were successful (Pearson χ2 = 0.21, p = 0.64, ns). In the 2015 cohort, however, completion of POC was associated with higher rates of success; 88.9% of the students who completed POC were

“C−” or better for General Chemistry II (or concurrent enrollment) is required. Although in its current construct (as an intersession class) there is no way for a student to be concurrently enrolled in General Chemistry II and POC, future endeavors may include deployment of POC during the regular semester (perhaps as a class that meets only in the second half of the semester) preceding OC-1. Topics were chosen collaboratively by the chemistry instructor authors of this B

DOI: 10.1021/acs.jchemed.6b00963 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 2. Comparative Success in OC-1 for 2014 and 2015 Cohorts of POC and OC-1 Classes Total College Student Characteristic a

N Gender Cumulative GPA before Fall Term (mean ± SD) Success in OC-1, %b

Male, % Female, %

Total OC-1

Completed POC

Did Not Attempt POC

2014

2015

2014

2015

2014

2015

2014

2015

2222 56.0 44.0

2234 55.9 44.1

167 49.1 50.9 3.47 (0.46)

157 39.5 60.5 3.45 (0.48)

46 45.7 54.3 3.45 (0.44)

45 40.0 60.0 3.57 (0.44)

121 50.4 49.6 3.47 (0.48)

112 39.3 60.7 3.42 (0.49)

82.6%

78.3%

80.4%

88.9%

83.5%

74.1%

a

The N values reflect the omission of students who did not take exam one in OC-1 and students known to be repeating OC-1 (four total). bDefined as a grade of “C” or better.

and motivation.40 Given that E1S explicitly tests general chemistry concepts (while later exams do not), the correlation coefficients between GPA and students’ final percentage (FP) in OC-1 were computed both incorporating E1S and excluding E1S. For the 2014 cohort, the correlation coefficient is 0.570 with E1S incorporated, while it is 0.545 without E1S. In 2015, the correlation coefficient is 0.680 with E1S incorporated, while it is 0.672 without E1S. These data suggest that the correlation of GPA to FP is not significantly affected by E1S. Additionally, they reveal a stronger correlation of GPA to FP in 2015 in contrast to 2014. There are two variables with significant correlation to success in OC-1: one is GPA, and the other is completion of POC. We aim to develop a model that predicts a student’s success in the 2015 cohort of OC-1 that incorporates both of these effects using E1S as a predictor of success. A mediated regression analysis was used to predict the mediating effect of GPA on FP in OC-1 and E1S. Using the SPSS add-in PROCESS written by Andrew Hayes,41 a mediated multiple regression procedure41,42 was used to regress the predictor (E1S) and mediator (GPA) on the dependent variable in a forced entry regression analysis. The 2015 cohort provided a sample size of 143 participants for analysis (14 of the 157 were excluded due to missing data). We used 1000 bootstrap samples for bias corrected bootstrap confidence intervals and a 95% level of confidence for all confidence intervals. The mediated regression is a two-step process in which E1S was first regressed on student FP, measuring the direct effect. Then E1S was regressed on FP along with GPA to measure the indirect effect. When GPA is mediating the relationship between FP and E1S, the direct effect will weaken. The mediation model was significant (F(1,140) = 164.54, p < 0.001). There was a significant indirect effect of E1S on a student’s FP through GPA, b = 0.17, bootstrapped confidence intervals (BCa CI) [0.11, 0.25]. The effect size of this indirect effect (academic ability and motivation; GPA) was low moderate or “fair”,43 κ2 = 0.26, 95% BCa CI [0.18, 0.35]. Table 3 contains the multiple regression results (for the 2015 cohort) incorporating E1S, FP, and GPA showing that both steps of the model were statistically significant. Figure 1 illustrates the mediation of E1S on FP in OC-1 through GPA. The weakening of the relationship between FP scores (i.e., dropping from b = 0.54 to b = 0.17) on E1S in the course by the inclusion of GPA shows that while GPA accounts for a significant portion of student classroom performance, completing the preparation course explains a statistically significant amount of student success.

successful compared to 74.1% of those that did not take POC who were successful (Pearson χ2 = 4.13, p = 0.04). The difference in effect between the 2014 and 2015 cohorts can be many-fold. First, the 2014 cohort of OC-1 students was large enough that it was split into one of two different lecture sections, each with a different instructor. This may not entirely explain a lack of statistical significance in 2014 since statistical analysis of student success split by instructor (i.e., considering them to be independent data sets) leads to similarly nonsignificant results. Next, data on student success were analyzed between the 2014 and 2015 academic years, allowing the instructor to reflect and make changes in the fall OC-1 course. The introductory OC-1 chapter, a review of general chemistry, was covered to a lesser extent in the 2015 OC-1 course as compared to the 2014 OC-1 course. This truncated review of the introductory OC-1 chapter de-emphasized explanation of general chemistry topics in favor of reminding students of these concepts. Although this shift may be best handled by those that took POC, it is not clear that this would affect success in OC-1. Given that the 2015 cohort exhibited a statistically significant link between POC and success in OC-1, further statistical analysis was completed (see below). Correlation of Metrics to Percentage in Organic Chemistry I

Of the myriad variables compiled (vide supra), few showed significant correlation to success for the 2015 cohort in OC-1; exam one score (E1S) and GPA both proved to be exceptions, however. Completion of POC was associated with statistically significantly higher E1S (M = 89.4, SD = 8.94) when compared to students who did not complete POC (M = 79.4, SD = 16.0). The Levene’s test for equality of variance showed that the variability in scores among those who did not complete POC was not homogeneous with variability in scores of students that did complete POC (F = 5.72, p = 0.018); therefore, the independent sample t test was calculated assuming that the variances were not equal, [t(138.9) = 10.04, p < 0.001, d = 0.70]. This effect size (0.70) can be interpreted that 75.8% of the students who did not take POC would be considered below average when compared to students that did take POC. GPA was also associated with a statistically significantly higher E1S. Students who completed POC had higher GPAs (M = 3.58, SD = 0.43) than students who did not (M = 3.39, SD = 0.49, t(154) = 2.16, p = 0.032, d = 0.38). The Levene’s test for equality of variance was not significant (F = 2.16, p = 0.144), indicating that the assumption of homogeneity of variance had not been violated. With an effect size of 0.38, 64.8% of students who did not take POC would be considered below average when compared to students who did take POC. Herein, we interpret GPA to be indicative of academic ability C

DOI: 10.1021/acs.jchemed.6b00963 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 3. Multiple Regression Results for Preparation Scores, Final Grade, and GPA Variable Step 1 Constant E1S Step 2 Constant GPA E1S a

b Values

Standard Error

t Valuea

Significanceb

1.62 0.02

0.24 0.003

6.68 7.74