Overview

During the Fall 2003, Spring 2004, Fall 2004, Spring 2005 and the Fall 2005 semesters, students in my Elementary Algebra classes participated in an interactive classroom to learn mathematics. Addison-Wesley's MyMathLab and McGraw-Hill's ALEKS software were used to create this learning environment. The software was supplemented by short presentations by the instructor. A detailed description of the software and the interactive classroom is given below.

Over 200 students enrolled in my MATH 0600 Elementary Algebra sections during those five semesters. Approximately 29\% of those students successfully completed the course. By successful completion, we mean that the student received either an A, B, or C as a letter grade.

During the Fall 2003 semester, two sections of Elementary Algebra were offered that allowed students to test the interactive environment. One section used a McGraw-Hill software package, ALEKS (Assessment and LEarning in Knowledge Spaces), but not a McGraw-Hill textbook. Elementary Algebra by Marvin Bittinger, an Addison-Wesley textbook, was used since that is the mathematics department-approved text for the lecture sections of the Elementary Algebra course. The other section used Addison-Wesley's software, MyMathLab, and the associated Elementary Algebra text by Bittinger. The fully integrated software, textbook, interactive exercises and videos represent the publishing community's response to the need for a variety of delivery systems that, hopefully, will meet the needs of our diverse student population.

The approach for this semester was less presentation and more software interaction in the classroom. Since this was the first semester for MyMathLab, and the second for ALEKS, I decided to test the user-friendly and motivational features of the software packages. As a result, my presentations were kept to a minimum, but the videos were used as supplementary lectures on certain topics. A heavy emphasis was placed on the in-class use of software. Individual exercises were presented on the white board in the classroom whenever students encountered difficulties with certain topics.

In Spring 2004, two day-time sections and one evening section of Elementary Algebra were offered. All three sections used the integrated MyMathLab/textbook package. This time, I increased the instructor presentation time at the beginning of the semester. The objective was to gradually lead students to the more independent study requirements for learning mathematics (see My Educational Philosophy).

By 2005, a pedagogical method was being refined to ensure maximum participation by students. The incorporation of technology into the pedagogy had a significant impact on student “time on task”. Students demonstrated a greater willingness to try to learn mathematics, The following chart indicates that the percentage of students willing to stay with the course through the final exam (letter grades A, B, C, D, F) is large in comparison to those who decided to withdraw.

Figure 1 Grade Distribution: All Students

In the next section I will describe the technology-enhanced approach the mathematics education that emerged from this study.

Pedagogy

By including computer technology in the classroom, we anticipate greater participation by students in the learning process. Since the only way to learn mathematics is to do mathematics, students are likely to be more successful if they are actively involved in the classroom than if they are simply scribing notes. (“I understood it when you explained it in class, but when I got home, I couldn’t do the homework”). This is particularly critical for developmental mathematics students.

By definition, pedagogy is the art and science of teaching. The primary objective of effective teaching is student learning. But what is the student expected to learn? And what does it mean to teach a student?

The current emphasis on student outcomes makes it clear that the general public is interested in the specific concepts and skills that need to be mastered. The outcome encompasses both the learning objectives and the expected assessment results. Therefore, any educational pedagogy must include both an instruction/learning component and an assessment component.

The following paragraphs provide an overview of the pedagogical requirements which technology-enhanced mathematics education must address.

Instruction/Learning

As stated above, students learn mathematics by doing mathematics. By doing

mathematics, I mean

1. Reading the textbook, previewing material before class, and reviewing it after class.

2. Taking notes and listing questions about textbook examples as they read the book.

3. Asking questions during class or during office hours or via email

4. Doing assigned exercises. Students must develop the discipline to solve problems without referring to textbook examples.

5. Reflecting on solutions to problems and summarizing the solution method

6. Taking a “practice test” (e.g. in textbook), grading the test and reviewing the problems they solved incorrectly.

Assessment

A single test is not sufficient to determine a student’s mastery level in any subject, but least of all in mathematics. Several assessments are needed to determine the student’s “working knowledge” of mathematics. By “working knowledge” I mean real time problem solving skills that can be applied at the next academic level or on the job with minimal use of reference material.

Based on the need for working knowledge, cumulative assessments are necessary. The only effective way to measure working knowledge is to test students on a regular basis. Since a key element in all assessments is the time required to accurately solve problems, timed assessments are essential. (For higher level mathematics courses, developing mathematical models for applications is another primary area for assessment)

The benefits of continuous assessment pedagogy are well-documented. The primary benefits to the student are mastery of material and confidence in using mathematics. Benefits to the teacher and student include knowledge of the student’s strengths and weaknesses and development of individualized study plans for achieving mastery.

Technology: A Key Element of this Pedagogy

The use of software like CourseCompass/MyMathLab or MathZone/ALEKS proved to be useful for implementing an educational pedagogy with the above-mentioned requirements.

1. Students received immediate feedback on each problem they attempted during the learning process. In addition, a host of on-line supplements were available for each problem (i.e. guided solutions, links to the textbook, videos, etc.)

2. On-line testing allowed greater frequency of feedback than possible if an instructors was required to correct and grade tests and quizzes.

3. Summaries of student activity made it possible to guide students to their areas of greatest need. The software created study plans tailored to each student. This individual study plan is not possible when many students are assigned to one instructor.

4. When students were unable to attend class, they were still able to work on the scheduled topics at home (most students do have computers). I was able to monitor the student’s progress and provide assistance on-line. (This “distance-learning” component was instrumental in increasing student “time on task”)

5. Videos provided mini-lectures for students who did not understand the explanation in the textbook (but these did not replace the required reading).

6. Students practiced a larger number of exercises since the software regenerates the numerical values for a problem. As a result, particularly challenging exercises were done repeatedly. In this way, students reviewed the solution process (i.e. reflected on the solution process) until they understood its application to that type of problem.

Software Used

Educational software for mathematics should enhance the learning process. Students should be engaged in doing mathematics when they are using the software. The above-mentioned pedagogy demanded a dynamic, learning and assessment environment. The two software packages used in this study were MathZone/ALEKS (McGraw-Hill) and CourseCompass/MyMathLab (Addison-Wesley)

Features

A. For Instruction/Learning

When students were doing practice exercises, software features that were useful included:

Feature	MathZone/ALEKS	CourseCompass/MyMathLab
Solve similar problem	Hint	View an Example
Solve the problem	Show Me	Help Me Solve This
Step through solution	Guided Solution	Help Me Solve This
Print displayed page	Print	Print
Send email to instructor	Ask My Instructor	Ask My Instructor
Opens textbook in pdf-format	Link to Text	Textbook Pages
Video presentation by instructor	Video Lecture	Video
Audio presentation	e-Professor	Animation
Individualized study plan	ALEKS assessment	Study Plan
Interactive exercises in on-line version of textbook		You Try It

B. For Assessment

When developing assessments to determine a student’s learning progress, the ideal is to test core knowledge and skills as well as the student’s ability to apply that knowledge to new problems. Software packages excel at the skill level testing, but not necessarily at the new problem assessments. To provide for this testing requirement, both software packages allowed me to design and upload tests written by me.

It should be noted that each software package partitions exercises by the learning objectives for the sections of the chapter. This was an important feature for the students and for me. Students focused attention on specific problem types they had not mastered and I focused my presentations on difficulties the class was experiencing.

C. For Administration

Both software packages provide grade book capabilities. I could examine the records of individual students or the entire class. Performance on specific exercises enables me to prepare focused presentations on topics the class was struggling to master.

Graded assignments were exported to Excel and combined with the results of written tests. Students who stop attending class were removed from the active roster.

Analysis

Grade Distribution

A total of 210 student records were analyzed for this study. Of these, 107

used ALEKS and 103 used MyMathLab. The grade distribution is contained in

the following table:

Grades	Students
	ALEKS	MyMathLab	All
A	10	4	14
B	15	12	27
C	11	9	20
D	18	24	42
F	23	22	45
NA	3	1	4
W	6	11	17
WF	12	13	25
WP	10	7	17

Table 1: Student Grade Distribution

As indicated in Table 1, very few students successfully complete this algebra

course. A total of 61 received a letter grade of either A, B, or C.

A more visual presentation of this data is contained in the pie charts in Figures 1, 2, and 3. These charts show the percentage breakdown for the data in Table 1. Note: Figure 1 (page 2) is displayed again for the reader’s convenience.

Figure 1 Grade Distribution: All Students

Figure 2 Grade Distribution: ALEKS

Figure 3 Grade Distribution: MyMathLab

Note that the graphs group the students into three populations:

1) A_B_C represents the students who have successfully completed the course and are ready to take a program-required math course.

2) D_F represents those students who are not quite ready for the next math course, but demonstrated the

fortitude to work their way through the complete elementary algebra course.

3) NA_W_WF_WP represents students who received administrative grades based on the following

a) NA: the student attended at least one class but never took a test

b) W: the student withdrew from the course after the change of course period but before the withdraw without academic penalty date

c) WF: the student withdrew from the course after the withdraw without academic penalty date and was failing the course at that time

d) WP: the student withdrew from the course after the withdraw without academic penalty date and was passing the course at that time

The bar graphs of Figures 4 and 5 show the letter by letter grade distribution.

Figure 4 Grade Distribution: All Students

Figure 4 shows a large number of D and F grades. The observation that students “demonstrated a greater willingness to try to learn mathematics” is based on the fact that 41% of the students “stayed to the end” and took the final exam even though they did not successfully complete the course (see Figure 1 for percentages). Since 29% of the students successfully completed, the retention rate was actually 70%. If the success rate is calculated using the number of students retained, 41.2% (61/148) of students completing the course were successful.

Figure 5 Grade Distribution: ALEKS and MyMathLab

Figure 5 displays a comparison between ALEKS and MyMathLab. Overall, the performance of students is similar. Although a greater number of students received A, B and C grades using ALEKS, the number of factors influencing success make it impossible to claim that ALEKS outperformed MyMathLab.

Placement

Since the grades are the "bottom line" as a measure of student success, it is essential that we begin to identify students who may not be successful before the course begins. To this end we began by examining student ACCUPLACER scores as a potential predictor of success and as an indicator of potential failure.

ACCUPLACER

Students entering CCRI are currently required to take ACCUPLACER Math and English tests to determine whether they need developmental courses prior to taking their required program courses. Part one of the Math test (AMAT) is a basic skills and knowledge assessment. The English test contains a reading comprehension component

(ARDG). Both test scores were available for a large segment of the students in this study. The following analysis is designed to determine if there is any useful information in these test scores.

The ACCUPLACER analysis is based on 153 student records. These students took both the AMAT and the ARDG tests. Nine students took only the AMAT so their scores were excluded from the analysis. The other 48 students have no ACCUPLACER scores because they either took COMPASS tests or they started at CCRI before mandatory placement testing was implemented.

Table 2 contains a breakdown of the average AMAT scores for students in each letter grade group. Figure 6 contains a line graph representation of this data for all students. Figure 7 contains a line graph representation of this data for all students.

Grades	Students
	ALEKS	MyMathLab	All
A	69.3	87.7	70.4
B	61.4	71.4	65.0
C	57.9	65.6	59.8
D	58.6	54.3	56.4
F	46.9	39.3	43.6
NA	54.0	44.0	49.0
W	75.3	32.8	46.9
WF	39.6	61.9	50.0
WP	62.0	52.0	60.3

Table 2 Average AMAT Scores

Figure 6 Average AMAT Scores: All Students

Figure 7 Average AMAT Scores: ALEKS/MyMathLab

The decline in average AMAT scores from grades A through F implies there is some reasonable information in this measure of mathematics background. The decline in scores for all students indicates that, independent of the software package used, students with weak math backgrounds earned D's and F's similar to the traditional pure lecture format.

It is interesting to note that there is a sharp difference in average AMAT scores among successful students between the ALEKS group and the MyMathLab group. For instance, there is an 18 point difference between the groups' A students (ALEKS 61.4, MyMathLab 87.7). Ten points separate the B students and 7 points separate the C students. Based on this, it would appear that ALEKS was able to assist less prepared students reach success in elementary algebra. Based on the software's emphasis on repetition of all algebra skills and the continuous review of prerequisite arithmetic skills, this seems reasonable.

The separation in ARDG scores is similar to the pattern in the AMAT scores for A and C students. The MyMathLab students have much higher reading scores in the MyMathLab group. However, the pattern is reversed for B students.

Tables 3 contains a breakdown of the average ARDG scores for students in each letter grade group. Figure 8 contains a line graph representation of this data for all students. Figure 9 contains a line graph representation of this data for all students.

Grades	Students
	ALEKS	MyMathLab	All
A	74.6	90.5	78.1
B	83.1	73.4	79.9
C	64.1	89.5	75.1
D	72.7	64.9	68.8
F	56.9	59.8	58.4
NA	78.0	71.0	74.5
W	93.3	60.7	70.5
WF	60.3	72.0	67.1
WP	66.0	62.7	65.6

Table 3 Average ARDG Scores

Figure 8 Average ARDG Scores: All Students

Figure 9 Average ARDG Scores: ALEKS/MyMathLab

The separability in group scores indicates that AMAT and ARDG scores will be useful for classifying students prior to the start of the course.

The next set of Tables and associated graphs will display the class statistics and distributions for the three groups mentioned above.

Statistic	A_B_C	D_F	NA_W_WF_WP
Median	64.5	42.0	53.0
Max	116	93	93
Min	24	20	20
Range	92	73	73
Mean	64.6	49.7	51.1
Std Dev	26.7	22.6	21.0

Table 4 AMAT Statistics for Student Groups

The associated frequency distribution for AMAT scores is contained in the next table. Note that the Classes in the left column are the range of AMAT scores in which student scores fall. The data values represent the number of students who's scores are in that range.

Class	A_B_C	D_F	NA_W_WF_WP
20-39	9	31	10
40-59	9	14	11
60-79	16	12	11
80-99	7	11	6
100-119	7	0	0

Table 5 AMAT Frequency Distribution

See Figure 10 for a graphical display of the distribution for A_B_C & D_F. As seen in the table and the graph, there is a distinct difference in the statistical distributions for these two groups. Besides the 15 point difference in their sample means, the 22 point difference in their medians is reflected in the skewed shape of the D_F distribution and the more Gaussian shape of the A_B_C distribution. This separability implies

that the AMAT scores should be useful for classifier design.

Figure 10 AMAT Frequency Distribution A_B_C & D_F

Now let's examine the ARDG statistics and distributions.

Statistic	A_B_C	D_F	NA_W_WF_WP
Median	82.5	64	68.5
Max	112	109	111
Min	32	27