Thesis Sample III

Helpful or Harmful: The Impact of Standardized Achievement Testing on 5^th Grade Gifted Students

_______________________________________________________________

A Thesis

Presented to the Education Department of

Albertson College of Idaho

_______________________________________________________________

 

In Partial Fulfillment of the

Requirements for the Degree

Master of Arts in Teaching

_______________________________________________________________

 

by

Elsje M. Taggart

August 2007

 

 

 

The Education Department

upon the recommendation of the research project committee

and the chair of the Master of Arts in Teaching program

hereby accepts this project in partial fulfillment of the requirements for the

Master of Arts in Teaching

 

___________________________________________________

Committee member

 

___________________________________________________

Committee member

 

___________________________________________________

Committee member

 

___________________________________________________

Director, Master of Arts in Teaching Program

 

 

 

 

___________________________________________________

Date

 

 

 

 

 

Abstract

With the increased use of achievement testing to assess school and student performance comes added pressure on students to perform well. While previous research has focused on how academic self-concept affects standardized testing scores, few studies have investigated the impact of achievement testing on the self-concept of those students taking the test. This study surveyed a small population of gifted students to determine if a relationship exists between standardized testing and the development of their academic self-concept over the course of a school year. The results indicated that the students were able to differentiate between academic and nonacademic self-concept, and that academic self-concept was content specific for the domains of reading and math. However, no positive relationship was found between standardized test scores and changes in self-concept.  Recommendations were made for practice and for further study.

 

           

             

 

           

           

 

 

 

 

 

 

 

 

 

Acknowledgements

To my family, without whose loving support I would have never even attempted this endeavor.

To Vicki Matthews-Burwell, my most influential teacher, who constantly inspires me both personally and professionally.

To my husband Chris. My Shoulder. My Rock. My Love. My Life.

 

 

 

 

Table of Contents

Abstract iii

Acknowledgements. iv

List of Tables. vii

Chapter One: Introduction. 1

Background of the Problem.. 1

Statement of the Problem and Purpose. 11

Significance. 11

Delimitations. 12

Definitions. 12

Summary. 13

Chapter Two: Review of the Literature. 15

Introduction. 15

Academic Self- Concept: Internal Development and Differentiation. 16

Academic Self- Concept: External Influences. 18

Academic Self-Concept: Two Theoretical Frameworks. 19

Academic Self- Concept: Influence on Gifted Students. 22

Academic Self-Concept: Subject Specificity and Implications for Achievement 24

Standardized Achievement Testing. 25

Response of Self Concept to Standardized Testing. 30

Validity of the Self-Descriptive Questionnaire. 31

Chapter Three: Research Methodology. 34

Introduction. 34

Study Design. 34

Participants and Setting. 35

Instrument 37

Procedures. 38

Analysis of the Data. 40

Summary. 40

Chapter Four: Findings. 42

Introduction. 42

Descriptive Statistics and Data. 44

Findings from Open-Ended Questions. 52

Summary. 57

Chapter Five: Summary, Conclusions and Recommendations. 60

Summary. 60

Conclusions. 62

Recommendations for Practice. 64

Recommendations for Future Study. 65

References. 67

Appendices. 77

 

  List of Tables

Table 1: Comparison Raw Scores in Reading and Math Scales…………………………… 44

Table 2: Comparison Percentiles Between Total Non-Academic, Total Academic, and Total Self Self-Concept Scores…………………………………………………………………………………………………….. 46

Table 3: Changes in Reading ISAT Scores Compared to Changes in Reading Self-Concept Scores       48

Table 4: Changes in Math ISAT Scores Compared to Changes in Math Self-Concept Scores    49

 

 Chapter One: Introduction

                                             

Background of the Problem

Although issues of achievement testing appear frequently in the media, the impact of standardized testing on students is rarely documented, even though almost all students being educated in America will be given at least one form of a standardized achievement test before they exit school. Yet the idea of a standardized test is not a new one. Some of the earliest testing began in New York State, where the first achievement tests for elementary and junior high students were given in 1925 under the direction of Superintendent of Schools, William O’Shea, and included examinations in composition, arithmetic, spelling, reading and vocabulary (Cuban, 1993). Prior to this time, the New York State Reagents’ Academic Examination had been administered to all high school students in New York since 1878.  Even in their infancy, the tests were not popular. By the 1930’s, almost half of high school teachers and department chairmen in New York saw these annual exams as reinforcing drill and rote memorization and generally having a negative impact on instruction in the classroom (O’Shea, 1927).

Although achievement tests are not a new phenomenon, standardized achievement testing did not become widely used until the 1965 Elementary and Secondary Education Act “stimulated a vastly increased reliance on standard achievement tests as a way of judging educational success” (Popham, 2001, p. 10). These achievement tests are “standardized in the sense that they are administered and scored in a uniform, predetermined manner” (Popham, p. 11). Currently, the five most widely used national standardized achievement tests are used in the United States: the Iowa Test of Basic Skills (ITBS), the California Achievement Test (CAT), the Comprehensive Test of Basic Skills (CTBS), the Stanford Achievement Test, and the Metropolitan Achievement Test. While some states used these national tests, others utilized “customized tests built exclusively for [that] particular state, and some [use] a combination of national and customized items” (Popham, p. 11).

In March of 2001, the Idaho Legislature approved a State Board of Education assessment program, which came to be known as the Idaho Standards Achievement Test, or ISAT (Idaho Board of Education, 2002b; Idaho Board of Education, 2003a). During the previous year, the Assessment and Accountability Commission of the Idaho Board of Education had recommended the adoption of a comprehensive assessment system, which included tests at each grade level K-12 with a concentration in reading, language and mathematics.  In the spring of that same year, the State Department of Education contracted with the Northwest Evaluation Association (NWEA), a private vendor, to create customized tests to measure proficiency in Reading, Language, and Mathematics. These tests were developed to correspond to Idaho’s state educational content standards (Idaho Board of Education, 2002b). The assessments were then administered in both a paper and computerized version, as a pilot to all ninth grade students in Idaho before June of 2002.

The initial summary data and scores from the pilot test were compiled. They indicated that the average student responded correctly to 60-64% of the questions, a figure above the estimated 50% correct response rate (Idaho Board of Education, 2002a). The test was purposefully designed for student scores to fall within this range, as it was theorized that “a great deal more information about a student’s breadth of knowledge on a subject matter can be gleaned from a test that is challenging for all students, rather than just for some” (Idaho Board of Education, 2002a, ¶ 2). It was also determined that “no substantive difference in scores was shown from students who took the paper version versus those who took the computer-delivered version of the test” (Idaho Board of Education, 2002a, ¶ 3). Following these initial observations, teachers from around the state met with representatives from the Board of Education and the testing contractor to review test questions for fairness, reliability, bias, and to determine if they accurately reflected the state’s educational content standards. 

Following the analysis of the test in the summer of 2002, the computerized version of the ISAT was piloted to students in second to tenth grade beginning in September 2002 (Idaho Board of Education, 2002c, ¶4). The computerized version of the test was selected by the State Board of Education based on the “computer’s capacity to quickly adapt or adjust questions based on the student’s response and its power to provide scores… immediately” (Idaho Board of Education, 2002c, ¶6). Following these pilot tests, the ISAT was revised to reflect federal, as well as state, educational requirements, resulting in a “blended” test that would be phased in over the next three years (Idaho Board of Education, 2002b, ¶8). Currently in the state of Idaho, all students in grades 2-10 are given an ISAT test in reading, language and mathematics at least once a year (Idaho State Department of Education, n.d. a, ¶ 4)

However, there is a growing concern among those in the educational field about the validity of standardized achievement tests, especially when proficiency scores have student consequences attached to them (Paris, Lawton, Turner, & Roth 1991). The validity of standardized tests has been questioned on several grounds. Resnick and Resnick (1989) found that tests are based on outmoded theories which assume that knowledge can be decomposed into isolated skills and decontextualized by removing the situation in which it can be applied. Others have argued that the tests lack validity because they do not reflect the curricular content taught in the classroom nor the methods through which it is taught (Linn, 1987; Wiggins, 1989). As well, the credibility of test results is weakened by the variability in the preparation for, and administration of, the standardized tests (Haladyna, Nolen, & Haas, 1990).

Students, however, also develop an increasing dissatisfaction with standardized testing throughout their school career (Paris et al., 1991). Surveys of students in grades 2-10 in Michigan, California, Arizona, and Florida found increasing dissatisfaction with testing, including test validity, as well as decreasing motivation to excel on tests. The results revealed “growing disillusionment about tests, decreasing motivation to give genuine effort, and increasing use of inappropriate strategies” (p.14).

Currently, under “No Child Left Behind” legislation, “states must show ‘adequate yearly progress’ through three indicators. All states must show that 95 percent of students took a standards-based exam and met state established proficiency standards in math and reading. States are then offered a choice for the third indicator to measure student progress” (Idaho Board of Education, 2004b, ¶ 5). In Idaho, the state where this study was conducted, the third indicator, approved by the Idaho Board of Education, is for the students to demonstrate “a year’s worth of academic growth through computer based programs” (Idaho Board of Education, 2004b, ¶ 3).

These annual academic performance goals have been termed Annual Yearly Progress (AYP). To meet these requirements, all students in the school district studied take the Reading ISAT, the Language ISAT, and the Mathematics ISAT at least twice a year. Testing “windows” are set up by the State Department of Education as to when the tests can be taken by students- normally these dates fall in September, January, and April (Idaho State Department of Education, n.d. b). In the school utilized for this study, the administration worked with the teachers to develop a testing schedule so that each grade level tested in only one subject a day, and that there was a scheduled time to take tests if students were absent on the date their class took the test. In order for a student’s score to be counted in state and federal assessment data, every student enrolled in the school needed to take the ISAT tests within the dates of the testing window.

The tests were taken on a computer in the school’s computer lab. For each test, multiple choice-style questions were selected from a database of over 40,000 questions located on a server maintained by the NWEA. The tests were scored as follows:

The test uses a RIT score, or Raush Unit, and the ISAT tests are on a continuous scale from 140 to 300 RIT points. RIT points are applied to measure the level of mastery the student demonstrates on the test and is unrelated to the age and gender of the student. Higher RIT scores indicate a higher level of concept mastery by the student. For example, a score of 160 may correspond with the ability to add and subtract simple numbers, while a score of 245 may indicate mastery of algebra. (Idaho Board of Education, 2004a, ¶ 4)         

When taking the ISAT, the students all started with the same question, but then the questions were differentiated based on student responses. If a student answered incorrectly, the testing program selected a question with a lower RIT level from the database, or it selected a question with a higher RIT level if the correct response was chosen (Idaho Board of Education, 2003a). Each test was designed to be completed in about an hour, yet the students identified as gifted have taken up to three hours to complete a single test; as each question became progressively harder, they spent more and more time per question. This was especially evident in math, where they began to encounter statistical, geometric and algebraic questions on materials that were not introduced until after their grade level.

After each test was completed, the testing program immediately calculated the median RIT level of all the questions presented to the student, creating an overall proficiency score on the test, which was then displayed on the computer screen (Idaho Board of Education, 2003a). When the student responds to the last question there was a small pause as the testing program calculated their median RIT level, then the screen clears and a blue box appeared containing the name of the student, the name of the test just completed, and their proficiency score on the test. By looking at the score, the student knows immediately if his/her performance on the test met proficiency requirements for the grade level. In my experience, almost as soon as the class leaves the testing room students begin talking about the test and start to compare scores.

In my own classroom, I have seen a pattern of behavior among gifted students and their performance on the three ISAT tests. In order to qualify for the Gifted and Talented program, students must perform in the 95^th percentile of their grade level on either the Reading or Math ISAT.  I will refer to the students who qualify for this Gifted and Talented program as gifted students. 

After the initial round of testing in September, most gifted students in my class receive an advanced score and display confidence in their abilities. They proudly share their scores with each other, myself, and any other students or teachers with whom they have contact. They also begin to compare who scored the highest in our class, in the fifth grade as a whole, and in the school as a whole.

However, in January the students begin the second round of testing and the results are much different. Because each student knows his or her scores from the Fall ISAT tests, they are able to immediately compare their fall scores to their winter scores for each of the three ISAT tests. Yet, as the Winter ISAT tests have more questions, and therefore more opportunity for errors to occur, the students’ scores begin to stagnate or even drop. Again, as in September, as soon as the class leaves the testing room, the students begin talking about the test and start to compare their proficiency scores to the scores of others. These conversations also include the new dimension of how many more points each student scored on their Winter ISAT compared to the tests taken in fall.

The conversations students have about their Winter ISAT scores are now not just about their proficiency scores, but also about how many points they earned toward their “growth score,” the score that they should reach by spring in order to demonstrate a year’s worth of academic growth. This “growth score,” calculated by the testing company, reflects what each student’s projected score in spring should be on the reading, language, and mathematics ISAT tests, based on the average growth shown by students across the country with similar RIT scores (Idaho Board of Education, 2005). This score is given to teachers and students after the initial round of tests in fall. Often these scores are only several points above a student’s initial test score from September, however the extended length of each ISAT test and the difficulty of the leveled questions makes gaining these points extremely difficult.

After each of the three ISAT tests in January, the gifted students in my classroom soon begin to compete fiercely as to who had the highest score, or who made the most growth. They begin to calculate and compare how many more points each person scored than the other gifted students, or how many more growth points they earned, even comparing how much farther they are toward reaching their individual growth goals than other students are. This occurs not only among the gifted students in my class, but among all the gifted students in the fifth grade of our school. The gifted students even compare proficiency and growth scores among other gifted students in lower grades, even though each grade level is assigned different proficiency scores. The conversation continues in the classrooms, hallways, in the cafeteria, and on the playground. For students whose scores remained the same or fell from fall to winter, their responses to these conversations are usually critical, analyzing what questions posed them the most problems, where they made mistakes, and how they could remedy the situation during the final ISAT tests in April. Suddenly the students seem to be more apprehensive and appear to question their abilities.

The anecdotal evidence I see in my classroom was supported by several studies. In 1987, Olszewski, Kulienke, and Willis found that in early adolescence gifted students begin to question the validity of their acknowledged abilities. Furthermore, the researchers identified an “imposter syndrome” among older gifted students, which included patterns of doubt and disbelief about the legitimacy of their abilities, and a lack of self-esteem. That same year, Delisle and Galbraith (1987) found that students whose abilities were recognized at early ages held doubts about the accuracy of identification and the objectivity of those individuals who identified them.

In conversations with my students, several indicated that prior academic experiences influenced their anticipation of, and response to, the ISAT tests they had taken. The conversations I have observed among gifted students following the Fall and Winter ISAT tests indicated a connection to research done on the development of academic self-concept, and how it is influenced by school and personal experiences (Arsenault, 2001; Byer, 1999; Williams & Montgomery, 1994).

Academic self-concept is difficult to define, as Strein’s (1993) review of related literature found at least 15 different “self” terms used by various authors. Terms such as “self-concept,” “self-esteem,” “self-worth,” and “self-acceptance,” are often used interchangeably and inconsistently, when they may relate to different ideas about how people view themselves. Strein (1995) explains:

Perhaps the most important distinction that differentiates various conceptualizations is whether self-concept is viewed as an overarching, global characteristic of the person, or as a set of self-evaluations specific to different domains of behavior. The global view, sometimes conceptualized as ‘self-esteem’ or ‘general self-concept,’ is the older and probably the more common view among counselors and therapists (Strein, 1993). …In contrast to the traditional model of global self-concept, multifaceted models stress self-evaluations of specific competencies or attributes, for example, academic self-concept, physical self-concept, and so on. Although some theoretical models are hierarchical, with global self-concept at the apex, most of these models stress the distinctiveness of various self-concept facets. Extensive empirical research in developmental and educational psychology over the past 15 years has strongly supported the multifaceted view. Consistent with research findings, most published self-concept measures now emphasize domain-specific self-concepts. (Global versus Domain- Specific Model, ¶ 2-3)

For the last few decades, researchers have been studying academic self-concept in children, trying to determine how it is developed and how it correlates with a student’s experiences in school (Arsenault, 2001; Byer, 1999; Harter, Whitesell, & Kowalski, 1992; Marsh, Chessor, Craven, & Roche, 1995; Marsh & Hau, 2003; Plecha, 2002; Van Boxtel & Monks, 1992; Williams & Montgomery, 1994). In addition, many researchers have been studying the link between academic self-concept and academic achievement, which has appeared to have a reciprocal relationship (Anderman, Anderman, and Griesinger, 1999; Guay, Marsh, & Boivin, 2003; Marsh, 1992b; Van Damme & Mertens, 2000; Yeung & Lau, 1998). Several studies have indicated that academic self-concept is also subject-specific, especially as the students’ ages increase (Marsh, 1990, 1992a, 1992b; Marsh, Craven, & Debus, 1999). 

The results of these studies, detailed further in Chapter Two, appeared to relate to the experiences of the gifted fifth grade students with whom I work- it is for that reason that I chose this topic of study for the purpose of my research. The style of this case study, detailed further in Chapter Three, was a mixed-method, non-experimental study, following the framework outlined by Merriam (1998), utilizing both the Self-Descriptive Questionnaire-I  (SDQI) survey (Marsh & Yeung, 1999) with Likert-type questions, as well as semi-structured interviews, to assess if a relationship exists between standardized testing and academic self-concept. 

Statement of the Problem and Purpose

According to research done in the last two decades, and as outlined further in Chapter Two, academic self-concept is a multifaceted, content-specific context for the self (Marsh, 1990, 1992a, 1992b; Marsh et al., 1999). It also appears to have a reciprocal relationship with academic achievement, and seems to be heavily influenced by a student’s academic experiences  (Arsenault, 2001; Byer, 1999; Harter et al., 1992). To further this research, this study was designed to investigate if a relationship exists between standardized testing and the academic self-concept of gifted fifth grade students.

 

Research Questions

The following questions guided this study:

• 1. Is self-concept content specific, especially in the subjects of reading and mathematics?
• 2. Can students differentiate between academic self-concept and global (total) self-concept?
• 3. What is the relationship between standardized achievement testing and academic self-concept?

 

Significance

This study will have significance, as it will provide an additional case-study analysis to the current literature on academic self-concept carried out by Marsh et al. (1995) and others. This study looks specifically at the effects, if any, standardized testing has on the academic self-concept of gifted students, and might provide further insights into how academic self-concept is developed in this student population.

 

Delimitations

One delimiting factor for this study was that only fifth grade students from a single elementary school, identified for the Gifted and Talented program in fourth grade or before, were surveyed in the interest of keeping the sample size small. A second  delimitation was that surveys were only administered twice during the school year. Also, instead of fully structured interviews, the researcher conducted semi-structured interviews to allow for variations in student responses. Interviews were conducted only once, after the final round of testing was completed in the spring of 2006.

The cultural and socioeconomic status of those participating in the surveys and interviews might have influenced their responses. A final delimitation was the relationship between the sample population and the researcher.

Definitions

• 1. Self- Concept: for the purpose of this study, self-concept will be defined as an individual’s complex, organized, and dynamic system of learned beliefs, incorporating both cognitive and affective responses toward the self (Bong & Clark, 1999; Purkey, 1988).
• 2. Standardized Achievement test: for the purpose of this study, standardized achievement test is defined as a standard or uniform test given to all students in a geographic location such as a state. The criterion-based test administered to the students participating in this study is the Idaho Standardized Achievement Test (ISAT).
• 3. Gifted students: for the purpose of this study, Gifted students are defined as students who have ranked in the 95^th percentile for their grade level on the ISAT tests in Reading and/or Math.

 

Summary

Chapter One presented the background for this case study. The problem posed in this case study is whether standardized achievement testing has an impact on the academic self-concept of gifted fifth grade students. This research has significance in that while there has been research done on the effects of achievement on academic self-concept, little research has been done on the impact of standardized testing on the self-concept of gifted students.

A review of the research in the next chapter indicated that academic self-concept develops in children as early kindergarten, and that even young students have the ability to differentiate self-concept into multiple dimensions (Craven, McInerney, & Marsh, 2000). The review also looked at the opposing results of several studies on standardized testing and its effect on student’s self-esteem, motivation, and attitudes (Paris et al., 1991; Roderick & Engel, 2001).

In Chapter Three, the case study style of this research is reviewed, following the framework outlined by Merriam (1998), utilizing both a survey and conducting semi-structured interviews, to assess if a relationship exists between standardized testing and academic self-concept. Chapter Four presents the results of survey and interview data, while Chapter Five provides the summary of this case study, the conclusions drawn from its results, and recommendations for practice and further study.

Chapter Two: Review of the Literature                                                                                                                                           

Introduction

Research being done on self-concept in the last two decades has focused on how self-concept is constructed, and at what age it begins to develop in children; if self-concept is content or subject-specific; if self-concept correlates with academic achievement; and if students’ peer relations, age, gender, personal, and school experiences affect the development of self-concept (Anderman et al., 1999; Arsenault, 2001; Byer, 1999; Guay et al., 2003; Harter et al., 1992; Marsh, 1990, 1992a, 1992b, 1993, 2003;  Marsh et al., 1995; Marsh et al., 1999; Plecha, 2002).

In the research on the relationship between self-concept and its effects on academic achievement, several studies indicated a reciprocal relationship. This relationship supports the theory that academic achievement affects self-concept and that academic self-concept has an effect on achievement. However, the studies that have been conducted had mixed results when attempting to find a causal relationship (Guay et al., 2003; Marsh, 2003; Strein, Cain, Edelman, & Schwalb, 1989; Van Damme & Mertens, 2000; Anderman et al., 1999).

Similarly, standardized testing has been researched more frequently in recent decades, but the findings of these studies are mixed as to the impact of achievement testing on student attitudes and motivation, especially when student consequences are attached to the proficiency scores of those tests (Paris et al., 1991; Roderick & Engel, 2001).

The methods of surveying students on their self-concept have even come into question, with some studies pointing toward larger environmental influences than previously considered (Stader & Licht, 1992; Yeung, 2003).

 

Academic Self- Concept: Internal Development and Differentiation

Much research has been done on academic self-concept, the factors that influence its development, and the impact it has on academic achievement. A recent study done by Craven et al. (2000) examined the structure and development of multidimensional self-concept, the stability of self-concept responses over time in young children, and the relationship between prior academic self-concept and its impact on academic growth. This longitudinal study indicated that children begin to develop academic self-concept as early as kindergarten. In this study, 993 students in kindergarten through second grade were given both the Self Description Questionnaire for Infants and the Wide Range Achievement Test 3, in two waves over four years. This investigation extended previous research done by Craven, Marsh, and Debus in 1991 and also Marsh and Craven in 1997, in which young children showed the ability to differentiate between multiple dimensions of self-concept. The results demonstrated modest correlation between achievement scores and cognitive (academic) self-concept scores, but smaller correlation between achievement and affective (overall) self-concept scores. This supported earlier findings by the authors that young children are able to differentiate between cognitive and affective aspects of self-concept. Also, the structure of the children’s self-concept remained stable over a 6-month time span in the two cohorts studied.

Arsenault (2001) also studied self-concept in children, as well as factors that may influence its development.  In a longitudinal study conducted over sixteen years, Arsenault followed the development of self-concept in 104 children from infancy through adolescence. In this investigation, participants were studied through a variety of behavioral measures including the Self-Description Questionnaire II (SDQII), the Middle Childhood Temperament Questionnaire, and the Child Behavior Checklist, while academic achievement and intelligence was recorded by the Woodcock Johnson Psycho-Educational Battery and the Wechsler Intelligence Scale for Children-Revised. 

The model of self-concept development under investigation was that Academic Self-Concept = Constant + Behavior + Social + Cognitive (Arsenault, 2001). The participants were studied between the ages of 18 months to 17 years, which were classified into three distinct periods of development: ages 6 to 9 (Middle Childhood), 10 to12 (Pre-Adolescence), and 13 to15 (Adolescence).

The results indicated that while behavior remained a stable predictor, and temperament contributed throughout childhood and pre-adolescence, academic achievement increasingly influenced academic self-concept over all three developmental periods (Arsenault, 2001). In the same study, gender also seemed to play an increasing factor in academic self-concept, although results were mixed at the end of the study. While males placed more emphasis on behavior and academic achievement, females wholly attributed their development of self-concept to the cognitive factors of intelligence and academic achievement.

 

Academic Self- Concept: External Influences

Other factors have been found to influence the construction of self-concept. Byer (1999, 2001) measured the effects of students’ perceptions of classroom social climate on academic self-concept in middle and high schools. In his research on 185 middle school students, Byer (1999) found evidence of consistent positive relationships between students’ perceptions of social climate and their academic self-concept.  In his 2001 study of 392 rural and urban high school students in 11^th grade, utilizing the Classroom Environment Scale and the Academic Self-Descriptive Questionnaire II (SDQII) as measures, Byer found that perceived classroom involvement positively impacted academic self-concept.

Byer was not the only researcher to find a link between classroom experiences and the development of academic self-concept. Harter et al. (1992) conducted two studies of 801 middle school students, examining the effects of changing educational environments on students’ academic self-concept and motivation. The results indicated that changing emphases in the educational environment does effect academic self-concept, but the significance of the effect was influenced by the students’ perception of competence and their motivational orientation.

Plecha (2002) conducted a study to research the impact of student-faculty and student-peer interaction on academic self-concept. Utilizing 7,440 responses from the Cooperative Institutional Research Program (CIRP) database on the 1996 Student Information Form and the 2000 College Student Survey, Plecha combined similar variables from the two measures to form nine blocks representing environmental input, and then entered them into a stepwise regression. Results indicated that college student-faculty contact had a positive impact on academic self-concept, as did diverse peer interactions.

Williams and Montgomery (1994) found dual influence of both internal and external processes on the development of self-concept. In a study of 103 high school students, Williams and Montgomery looked for “internal ability comparisons across academic domains and external ability comparisons relative to peers in determining academic self-concept” (Abstract, ¶ 1). The study found male students tend to make external comparisons with their peers, while internal comparison may be stronger for females.

Yeung and Lau (1998) also found a correlation of prior academic achievement on internal and external ability comparisons in Chinese students. The study found positive results that indicate prior achievement affects self-concept in matching curriculum domains, demonstrating a strong external social comparison. Additional results were found between prior achievement and non-matching curriculum domains, indicating internal comparison of competence in different curricular areas.

 

Academic Self-Concept: Two Theoretical Frameworks

A significant amount of research conducted in the past two decades suggests that academic self-concept can be correlated with levels of academic achievement, and can also be influenced by age, gender, and school experiences (Anderman et al., 1999; Arsenault, 2001; Byer, 1999; Guay et al., 2003; Harter et al., 1992; Craven et al., 2000; Plecha, 2002, Williams & Montgomery, 1994; Yeung & Lau, 1998). The suggestion that academic self-concept reflects more than just academic achievement can be attributed to the different frame of reference that each individual possesses. Two theoretical frameworks guide this assumption. The “Big Fish Little Pond Effect” hypothesizes that academic self-concept is substantially influenced not just by an individual’s own academic abilities, but by the abilities of other individuals in the same environment (Marsh, 1984a, 1984b; Marsh & Parker, 1984). The Internal/External Frame Model (I/E) argues that students create their academic self-concepts based on a comparison of their abilities in different academic subjects (internal) as well as on a comparison of their abilities with those of others around them (external) (Marsh, 1986).

The “Big Fish Little Pond Effect” (BFLPE) occurs when students of equal ability levels have a higher academic self-concept when comparing themselves to less able students, but a lower academic self-concept when comparing themselves to more able students (Marsh 1984a, 1984b; Marsh & Parker, 1984). This model hypothesizes that a student’s academic self-concept is based on his/her perceived academic abilities when compared with his/her perception of the abilities of others in a reference group (such as a classroom). For example, if a student has an average level of academic ability, but attends a low-ability school, their academic abilities would be higher than the average abilities of other students in the same school, so the student’s academic self-concept would be above average. However, if the student attended a school where the average ability level was higher than that of the individual, the student’s academic self-concept would be lower than average. Therefore, according to this model, a student’s academic self-concept would be positively correlated with individual academic achievement, but negatively correlated with the school-average achievement.

The Internal/External Frame Model (I/E), the theoretical base for this study, describes the phenomenon in which Math and Reading self-concepts have been found to be non-correlated, eventually leading to an alternative to the “Big Fish Little Pond” theoretical model (Marsh, 1986). The research also led to a revision of the Shavelson model, in which self-concept in different subject areas are conceived to create both verbal and mathematical self-concepts (Marsh & Shavelson, 1985; Shavelson & Marsh, 1986). Instead of the external causes associated with the BFLPE, the I/E model hypothesizes that Reading and Math self-concepts are formed in relation to both internal and external frames of reference (Marsh, 1986). Like the BFLPE, it assumes that students compare their perception of their own reading and math abilities to the reading and math abilities of others within the same environment. However, I/E also assumes an internal comparison of abilities in reading and math occurs, creating a second basis for self-concept in both areas. For example, a student may perceive himself as below average in both math and reading, but as slightly higher in math than other academic subjects. The student may have lower math abilities in relation to other students in his class (external comparison), but higher abilities in math compared to his abilities in reading (internal comparison). So, even though his math skill level may be lower than that of his classmates, the student may have an average or above-average mathematical self-concept, depending to the weight he puts on the internal versus external comparisons he made about math ability. Therefore, according to this model, the external comparison should create a positive correlation between Reading and Math self-concepts, while an internal comparison should lead to a negative correlation between Reading and Math self-concepts, and the overall correlation will be near-zero based on the combination of both processes.

In his 1986 study, Marsh reviewed the data of eleven other studies utilizing the SDQI, SDQII, and SDQIII on over 5,200 participants to determine if a correlation exists between Reading and Math self-concepts. The results indicated that there is virtually no correlation between reading and Math self-concepts, refuting that Reading and Math self-concepts combine to form a single academic self-concept. The findings are significant in that although reading and math achievement indicators are highly correlated to their respective self-concepts, they are not correlated with each other. In fact, reading achievement negatively correlated with Math self-concept, and math achievement negatively correlated with Reading self-concept, consistent with the predictions of the I/E model. Thus, when math achievement is high (external) and when math achievement is perceived as higher than reading achievement (internal) a high Math self-concept is to be expected. Therefore, it is in the best interest of the researcher to consider the individual self-concept scores in reading and math as a more accurate representation of a student’s academic self-concept, as a general academic self-concept cannot adequately reflect the different factors that contribute to its formation.

 

Academic Self- Concept: Influence on Gifted Students

Several studies have researched the possible influences of student participation in academically rigorous programs on academic self-concept. In a study of gifted Chinese students, Chan (2001) surveyed 205 students using the seven scales of the Chinese version of the Self-Perception Profile for Adolescents. The purpose of the study was to explore any impact on self-concept due to the fact gifted students are set apart form their peers through admission to special gifted programs. The author administered both the Chinese version of the Self-Perception Profile for Adolescents, as well as the Standard Progressive Matrices and the Roets Rating Scale for Leadership to measure the student’s IQ and leadership skills respectively. The students who took part in the survey demonstrated above average IQ and slightly above average leadership skills. The Self-Perception Profile yielded information on six dimensions of self-concept in gifted students, including scholastic competence, social acceptance, athletic competence, physical appearance, behavioral conduct, and close friendship. The findings clearly indicated that gifted students were able to differentiate between the domains of self-concept, and demonstrated the need for multidimensional assessment of self concept, as was also indicated in the earlier study of Craven et al. (2000).

The academic program also seems to affect the academic self-concept of students, especially when they are placed in Gifted and Talented programs or attend an academically selective school (Marsh et al., 1995; Marsh & Hau, 2003). In a study of elementary school students participating in Gifted and Talented programs, Marsh et al.  (1995) found “systematic declines in three components of academic self-concept… but not in four components of nonacademic self-concept” (Abstract, ¶1) These results were replicated in a cross-cultural study of 103,557 high school students in twenty-six countries, which confirmed that students in an academically selective school or program will have a lower academic self-concept in than in a non-selective school, a phenomenon coined the “Big-Fish-Little-Pond Effect” (Marsh & Hau, 2003). Both of these studies indicate that students participating in a selective or Gifted program are able to differentiate between a more global self-concept and an academic self-concept, which is negatively influenced by their participation in the program or school. Strein et al. (1989) also discussed this “Big-Fish-Little-Pond” effect, as well as possible implications for the impact on academic self-concept created by placement into accelerated or highly competitive academic programs. In another study assessing self-concept to academic performance congruence, the majority of responses from ninth-grade students enrolled in honors courses fell outside the congruence range for both verbal and math abilities, indicating a risk for unrealistic self-concept perceptions relative to their performance (Williams, 1996).  

While Marsh et al. (1995) and Marsh and Hau (2003) found academically gifted students could differentiate between academic and non-academic self-concept, a study by Van Boxtel and Monks (1992) had dissimilar results. Van Boxtel and Monks assessed 168 adolescents, including gifted achievers and underachievers, as well as 74 control subjects. The results suggested that general self-concept, including the self-concept of gifted students, is strongly correlated with academic self-concept. The work of these researchers is relevant to this study, as it attempts to answer if gifted students differentiate between academic and non-academic self-concept, and how they weigh academic and nonacademic components of self-concept when creating a global (total) self-concept.

 

Academic Self-Concept: Subject Specificity and Implications for Achievement

Many research studies have focused on academic self-concept and its relationship to academic achievement. Guay et al. (2003) and Marsh (2003) found support for a reciprocal-effects relationship, indicating academic achievement influences academic self-concept and visa versa. Van Damme and Mertens (2000) established a bi-directional causal relationship, finding “subsequent academic self-concept is based on prior achievement… [in addition] prior academic self-concept forms subsequent achievement” (p. 1) Results also supported a motivational influence of academic self-concept on student effort and student perceptions of school relevance. Through two surveys of sixth through eighth graders, Anderman et al. (1999) found that a positive present academic self-concept is related to positive changes in grade point average, academic performance, and mastery of goals.

Several studies indicate that academic self-concept is subject specific. In 1990, Marsh assessed 759 students and found that academic self-concept is specific to curricular subjects. In an extension of that research, a set of fourteen academic self-concept scales related to performance in eight school subjects was given to a sample of 507 Australian students (Marsh, 1992a, 1992b). Correlations matching areas of self-concept and academic achievement were very specific to school subjects, indicating that “components of academic self-concept are more differential… and that relations between academic self-concepts and academic achievements are more content specific than has been previously assumed” (Marsh, 1992b, Abstract, ¶ 1). 

 

Standardized Achievement Testing

Today, educational achievement is often couched within the confines of standardized testing. This type of testing is often referred to as “high stakes” due to the profound consequences on the students who take them, and the institutions in which they study. The results of these measurements can determine promotion or graduation for students, the level of funding a school or district receives, and in some cases, the promotion or salaries of teachers. In the state of Idaho, for example, the Idaho Board of Education voted to phase in the ISAT as a graduation requirement over a period of three years beginning in 2006. According to State Board of Education rule, 10th graders must score at least “proficient” on the ISAT as one of their graduation requirements. Students can take the test up to nine times and the scores students must receive for the graduation requirements were phased in over three years (Idaho Board of Education, 2003b).

 In their investigation, Paris et al. (1991) utilized multiple surveys to determine the impact of standardized testing on students’ attitudes and motivation. In the first survey, nearly 1,000 students in grades 2-11 in Michigan, California, Arizona, and Florida participated. Significant discrepancies were revealed among different age groups when responses were compared, indicating increasing dissatisfaction with testing over time, including attitudes toward test validity, as well as decreasing motivation to excel on tests. By middle school, students reported feelings of inadequacy in employing appropriate test-taking strategies during testing. In response, Paris et al. administered a second survey to the same 1,000 students on the subject of appropriate testing strategies. Teachers often rely on teaching testing strategies to help students increase their scores on standardized tests. Over time it could be expected that students would become more proficient in utilizing these skills, however survey results did not indicate significant differences in strategy utilization by age. Students in all grades between 2 and 11 reported recognition of testing strategies and older students did not report using those strategies more often than younger students.

In the third survey, Paris et al. (1991) surveyed 250 Michigan students in grades 4, 7, and 10 about the new standardized test in reading administered to Michigan students. Older students were found to be less concerned about cheating on the test, felt less prepared to take the test, were less likely to utilize positive testing strategies and were more likely to exhibit poor test taking skills. As well, the middle and high school students cared less about the results of the test than younger students, and expected their teachers and parents to feel the same lack of concern.

Maehr (1979) had found similar results when investigating the effects of sociocultural and motivation variables on achievement. Using a sample of 7,000 school-aged children, Maehr assessed achievement “conceptualized as the students’ sense of competence” and motivation goals using standardized test instruments (Abstract, ¶1). Results indicated that sociocultural background directly affects motivation and performance, that motivational effects reinforce effects of sociocultural background on performance, and both objective performance and student interpretation of performance affect motivation.

These two studies are especially relevant to this research, as they indicate growing student apathy toward standardized tests, and suggest negative implications for students’ academic self-concepts. In the research studies conducted by Marsh (1992a, 1992b), Guay et al. (2003), Van Damme and Mertens (2000) and Anderman et al. (1999), academic self-concept was directly linked to academic achievement. Therefore, if students are increasingly dissatisfied with testing, do not see standardized tests as valid assessments of their knowledge, and display decreasing motivation to excel on tests, they will be less likely to perform well, potentially harming their sense of academic self concept. If this trend were to continue, their academic self-concepts would diminish.

However, in 2001 (Roderick & Engel, 2001) conducted a year-long qualitative study of 102 low achieving sixth and eighth grade students from five Chicago elementary schools, who were participating in test prep courses. Results revealed contradictory evidence to that found by Paris et al (1991). The researchers utilized student interviews and records, as well as teacher assessments, to gauge the effect of a controversial initiative of the Chicago Public Schools to end social promotion without attainment of a minimum score on the Iowa Test of Basic Skills (ITBS) in grades 3, 6, and 8.

All of the students were questioned in five areas: valuation of outcome on the ITBS and a sense of confidence in their ability to raise their scores, overall engagement in schoolwork, levels of support and academic press (high expectations from teachers), strategies or programs students were using to prepare for the test, and barriers to learning in and out of school. All participants were aware of the social promotion policy and the potential consequences to students if they were unable to pass with a minimum score. Student responses were broken into three categories: Working in School, Preparing but Not Through Classwork, and Worrying but not Working. The first group reported the most altered academic attitudes, with 53% of respondents indicating increased work effort. The students also noted more motivational support from teachers and increased effort on classwork. In the second group in which test prep was received outside of the classroom by after school programs or tutoring, only 9% of respondents noted increased efforts, but did indicate higher levels of parental support and involvement in academic activities. The third group comprised 34% of respondents, who indicated no change in academic attitudes or work effort, and who did not receive assistance in class, did not seek out any additional help before or after school, and noted no change in academic press (high expectations) from teachers or parents. The results of Roderick and Engel’s research indicated that even traditionally low-achieving students, although many opponents of achievement testing argue to the contrary, attempt to rise to meet the challenge when the academic achievement tests carry “high stakes” consequences. 

A study by Rosenshine (2003) comparing National Assessment of Educational Progress (NAEP) scores indicated similar results. Rosenshine’s research came as a response to an earlier study by Amrein and Berliner (2002) which had found states that introduced consequences (high-stakes) to their statewide tests did not show significant gains in statewide NAEP scores. However, Rosenshine included a comparison group formed of states that did not attach consequences to their statewide tests, and found very different results. Rosenshine’s analysis indicated that states that attached consequences outperformed the comparison group of states on each of three NAEP tests in a four year period, demonstrating “meaningful carryover from attaching consequences on statewide tests to statewide NAEP scores” (Abstract, ¶ 1). 

The implications of the studies by Maehr (1979), Paris et al. (1991), Roderick and Engel (2001), Rosenshine (2003), are relevant to the research conducted in this case study, as it addresses the effects of statewide ISAT testing on the academic self-concept of gifted students. These studies are significant in that they mirror the educational environment of the students in the sample population of this research. ISAT testing in the school district being studied does hold consequences for students who do not meet proficiency. The consequences for students not meeting proficiency in sixth grade or above include the requirement of special test-preparation classes and the inability to take electives classes such as band, orchestra, or choir. Additionally, students would no longer qualify for the Gifted and Talented program if they did not continuously score in the 95^th percentile in reading or math on the ISAT.

 

Response of Self Concept to Standardized Testing

The relationship between the “high stakes” of standardized achievement testing and academic self-concept is still unclear. In a study of 700 third, seventh and eleventh grade students, Byrne (1998) set out to determine the relationship between self-concept and academic achievement, if there is evidence of causal predominance between them, and what the social referents are that underlie the formation of self-concept in relation to academic achievement. Collecting data in both fall and spring, she utilized the Self-Descriptive Questionnaires I, II, and III, within appropriate grade levels. Academic achievement in English and math was measured in grades 3 and 7 by the Multilevel Academic Survey Test, and in grade 11 by the Canadian Adult Achievement Test. 

In the third grade sample, overall academic achievement and academic achievement in math both were revealed to be causal predominant over academic self-concept, while no causal predominance was established by either path for English. The opposite was true for the students in seventh grade. No causal predominance could be established between overall or math achievement and self-concept, but English self-concept did appear to have a causal predominance over achievement. In eleventh grade overall and math academic achievement were found to have causal predominance over self-concept, while the opposite was true for English.

In a study of 103 ninth grade honors students who attended public schools, Williams (1996) found differing information on the congruence between achievement and self-concept in the verbal and mathematics domains. Using the ME: Self-Concept Scale for Gifted Children, which had been adapted to assess both verbal and math self-concept, Williams indicated finding scores between math achievement and math self-concept to have more congruence than verbal scores of the same measure.

In the studies done by Byrne and Williams in 1998 and 1996 respectively, mixed results were found between any possible causal relationship between academic achievement and academic self-concept. This might be attributed to the differences in grade level, size of population surveyed, the measure utilized to survey students, or even the environment in which the measure is given.

 

Validity of the Self-Descriptive Questionnaire

In 1993, Marsh reviewed data from the normative archives of the Self-Descriptive Questionnaire in eight random samples of 4,000 students. His findings support a gender-invariant model of relations between math, verbal, academic, and general self-concept. In 1999, Marsh et al. examined the components of self-concept in 7- to 13- year olds. They found that the SDQ-I “factor loadings [are] invariant over both longitudinal and cross-sectional age comparisons… correlations among reading, math, and school self-concepts decreased with age, [while] correlations between competency and affect within domains remained large” (Abstract, ¶1). This indicated students more clearly distinguish between different academic self-concept domains as they age and develop, and that the different self-concept domains are affected by academic achievement.

In an extension of the research conducted by Marsh and Yeung in 1999, Yeung (2003) utilized two confirmatory factor analytic studies in the academic context to investigate the relationship between self-esteem, general self-concept and academic self-concept. Utilizing 474 students aged 11 to 15, in grades 7, 8, and 9, from a state school in China, Yeung created a survey of questions adapted from the SDQII, which utilizes a 6 point Likert-style scale. Analysis of the results showed that while a high correlation between self-esteem and school self-concept existed, the participants did not distinguish between esteem items of an academic or nonacademic nature.

The results of the studies completed by Marsh and Yeung (1999) and Yeung (2003) indicated esteem survey questions, while not academic in nature themselves, can be easily related to academic characteristics when presented in an academic context. This phenomenon, called the “chameleon effect,” occurs when the meaning of self-esteem items were qualitatively changed by the context in which they were presented. The implications of these studies call for further attention to the contextual effect on self-esteem related responses. 

The Self-Descriptive Questionnaire was chosen to be administered for the purposes of this study as it has shown to be gender and age invariant (Marsh 1993; Marsh et al., 1999), as well as indicative of student differentiation between academic subjects (Marsh et al., 1999).  The “chameleon effect” described in the research of Marsh and Yeung (1999) and Yeung (2003) will be applicable to this study, as the Self-Descriptive Questionnaire will be administered to students after the completion of the ISAT tests. If students can easily relate esteem or general self-concept questions to academic characteristics, results may indicate an overall influence of ISAT testing on student academic self-concept. Also, the work of Stader and Licht (1992) indicates that “causal attributions, academic expectancies, and self-evaluation …varied as a function of whether the questionnaire was administered individually or in a group” (Abstract, ¶ 1). Children expressed lower and less accurate predictions individually than in a group. This research also had implications for how the questionnaire was administered, as detailed in Chapter Three.

 

 

Chapter Three: Research Methodology                                                                                              

Introduction

Patton (1985) explained qualitative research as an “effort to understand situations in their uniqueness as part of a particular context…This understanding is an end in itself, so that it is not attempting to predict what may happen in the future…but to understand the nature of that setting” (p.1). Following these principles, the methodology of this case study utilized both a survey and semi-structured interviews. As Merriam (1998) pointed out, “case study does not claim any particular methods for data collection or data analysis. Any and all methods of gathering data, from testing to interviewing, can be used in a case study…this design is chosen precisely because researchers are interested in insight, discovery, and interpretation rather than hypothesis testing” (p. 28).

 

Study Design

This case study was designed to be a mixed method, non-experimental study to evaluate if standardized achievement testing affects the academic self-concept of gifted fifth grade students. It attempted to answer whether self-concept is content specific, especially in the subjects of reading and mathematics; if students differentiate between academic self-concept and global self-concept; and what is the relationship between standardized achievement testing and academic self-concept, if one exists? The cohort studied was a small sample of gifted students enrolled at a single elementary school. Merriam (1998) found “the single most defining characteristic of case study research lies in delimiting the object of the study, the case” (p. 27). Stake (1995) clarified that the case “could be a child. It could be a classroom of children… an innovative program may be a case… the case is a specific, a complex, functioning thing” (p. 2). In this case the researcher attempted to discover insights into a possible relationship between standardized testing and the academic self-concept of a small band of gifted fifth graders. 

 

Participants and Setting

The sample population was a convenience sample, which consisted of seven fifth grade students, previously identified as academically gifted. In addition, an interview was conducted with the teacher of the Gifted and Talented program, who interacted with the participants through pullout sessions on a weekly basis.

The participants included two Caucasian males between the ages of ten and twelve, who chose the names Theirry and Bob as their pseudonyms. Theirry is a very outgoing individual, who has a tendency to become overly exited when talking about something that interests him. He is active in both youth soccer and orchestra, and plays the guitar in his spare time. He is very independent and does not appear overly concerned with what other students think about him. Bob is a very vocal student, often in trouble for blurting out answers in class. He enjoys participating in class and group discussions, and is very interested in reading. While he is a good athlete he does not participate in sports outside of school.

The other five participants are females, also between the ages of ten and twelve. While Luna is Hispanic, the other four girls are all Caucasian. Luna is a quiet girl who is very close to her friends, but appears shy when around people outside of her social group. She is the daughter of migrant farm workers and has spent her summers picking corn and fruit with them in the fields. Her family is very supportive of her schoolwork, and is encouraging of her successes. Mia, too, is also a quiet girl, often overshadowed by her more vocal contemporaries. She is friendly toward everyone, works well with others in groups, and plays well on her soccer team. Mia is not as competitive as some of the other girls in her social group, but she works hard to excel in all she tries. Hamm, Fredet and Miranda are more outspoken than Luna or Mia, but they are all very different from one another. Hamm is the most competitive of the group of participants, both in the classroom and on the soccer field. She plays soccer on several youth teams, and is very outgoing. Not as quiet as the other girls, she does not enjoy reading or sitting still for long, but instead enjoys activities such as orchestra that require movement, concentration and skill. Fredet is a very analytical girl who likes to be correct and precise in all she does. While she always has an opinion on the topic at hand, she enjoys stopping to reflect on what others have said and has a tendency to be overly introspective. Not active in sports, she is an avid reader and has been caught reading novels instead of completing her classwork. Miranda is an opinionated young lady who takes nothing at face value, instead looking deeper at the meaning behind a question or situation. She is an independent thinker who enjoys working on her own, finding that working in groups can sometimes slow her down.   

These students and their Gifted and Talented program teacher were all from the same suburban school in a city of 37,056, located in the southwest part of the state of Idaho. The median household income of the community is $35,600. The ethnic makeup of the town is 68.3% White non-Hispanic, 28.1% Hispanic, 1.8% American Indian, and 19.7% other ethnicity. Resident education for the population aged 25 years and older is 68.8% high school or higher, 11.7% Bachelor’s or higher, 4.2% graduate or professional degree (City Data, n.d.). School demographic data for the 2005-2006 school year included fall enrollment at 506 students, 86% regular education, 14% special education, 6% gifted, 6% limited English proficiency (LEP), and 73% Free and Reduced meals. At the time of the study, the ethnic makeup of the school was similar to that of the city, with 59% of students White non-Hispanic, 40% Hispanic, .2% Native American, and .4% other. In the school, gender was fairly balanced, with males comprising 52%, and females comprising 48% (Washington SIP, n.d.).

 

 Instrument

The survey instrument administered to the students was the Self-Descriptive Questionnaire I (SDQI), developed by H.W. Marsh. The 76-item SDQI evaluates the three areas of academic self-concept derived from the Shavelson model (Marsh & Shavelson, 1985; Shavelson & Marsh, 1986), four areas of non-academic self-concept, and a general-self concept scale derived from Rosenberg self-esteem scale (Rosenberg 1965, 1979). Together, the eight scales reflect a student’s self-rating in each area of self-concept. The SDQI was originally intended for use in fourth through sixth grade, although it has shown reliability in grades 2 and 7+, making it an ideal instrument for the students participating in this study. The surveys of 3,562 students in grades 2 through 6 were used to norm responses to all academic, all non-academic, and all items. As responses varied systematically between gender and grade-levels, separate norms were also created for grades 2 through 4 and for grades 5 and 6. Internal checks for random or biased responses were also developed (Marsh, Barnes, Cairns, & Tidman, 1984).

The initial focus of SDQI research was to identify the major components of self-concept and how they related to each other. In these studies, the internal consistency reliability for the different scales and total scores were all in the .80’s and .90’s, while the correlation between the different self-concept scales was low (Marsh, Smith, Barnes, & Butler, 1983). Although most research was done on samples of Australian students, the generalizability of SDQI results for structure, gender differences in the individual self-concept scales, and mean responses was shown in a cross-national study (Smith & Marsh, 1985; Marsh & Smith, 1987). Validity research also found SDQI responses were systematically related to gender, age, socioeconomic status, academic achievement, and student perceptions of teacher and peer ratings of inferred self-concept, thus supporting the construct validity of the SDQI (Marsh, Smith, & Barnes 1985; Marsh, Relich, & Smith 1983; Marsh, Cairns, Relich, Barnes, & Debus, 1984; Marsh, Barnes, et al., 1984).

For this research study, the students and the Gifted and Talented program teacher also participated in individual semi-structured interviews with the researcher. The researcher prepared a list of 17 potential interview questions for the interview with the Gifted and Talented program teacher, and 20 potential interview questions for the student interviews, which were peer reviewed for bias prior to the interview process.

 

Procedures

The students were administered the SDQI survey twice over the course of the 2005-2006 school year, and interviewed in the spring of 2006. Pseudonyms were chosen by the participants to be used on surveys and interviews to ensure anonymity.

The initial surveys were completed at the close of the Fall ISAT testing window in September of 2005. An additional survey was administered at the end of April 2006, when the final Spring ISAT tests were taken. Participants were given the surveys in a classroom setting at the same time, seated in desks spaced around the room where they were unable to see each other’s answers. Each student, under the direction of the researcher, filled out the identifying and background information section. Instructions were read aloud to the students as they read along on their survey. Sample items were completed together, and time was given for students to ask any questions for clarification of how to fill out answers on the surveys. The survey items were then read aloud twice to the students, with a brief pause between items so students could mark their answers. Questions and any other form of talking were not permitted during the administration of the survey. If students did not understand a term or question, they were instructed to leave it blank and return to it at the end of the survey, when the researcher could come and rephrase the item if necessary. All administration instructions given in the SDQI manual were followed explicitly. The surveys required no more than thirty minutes to complete. The completed surveys were then sealed in an envelope, to assure participants of confidentiality, until the end of April of 2006. The same process was repeated in April of 2006, utilizing the same guidelines to complete the second set of surveys, which were added to the envelope containing the original set of surveys.

Semi-structured interviews were conducted with the participants in the third week of April 2006. These videotaped interviews were completed with each participant individually. After the interviews were finished in April, the videos were transcribed and the participants were given time to read over them to complete a member check. The transcripts were then read, themed and coded. An additional, structured interview was conducted in late February with the Gifted Talented teacher who taught the participants in a pull-out program in the school. Again, the participant read the transcript before it was coded and themed.

 

Analysis of the Data

Analysis of the survey responses took place after the final survey period in late April of 2006. Survey results were entered into a database, and results were tabulated and compared. Interview coding was completed by June of 2006 to be included in the data analysis.

 

Summary        

            The methodology of this case study utilized both a survey and semi-structured interviews, in an attempt to create a descriptive, heuristic look at the particular contextual experience of gifted students and their academic self-concept. The sample population studied in this case was a group of seven (7) fifth grade gifted students from a single suburban school. The participants were administered the Self-Descriptive Questionnaire I (SDQI) survey twice during the 2005-2006 school year, immediately following the completing of their ISAT tests in fall and spring. Videotaped, semi-structured interviews were also conducted with students in spring to provide an in-depth understanding of their experiences.

Responses to the surveys were entered into a database, then were tabulated and compared. Interview data was utilized to add a richer context to survey results. Chapter Four presents the results of this analysis, in an attempt to discover any possible relationships between the different domains and scales of self-concept, and also between the students’ Spring ISAT scores and the academic domains of Reading and Math self-concept.

Chapter Four: Findings

Introduction

Research conducted by Maehr (1979), Arsenault (2001), Guay et al. (2003) and Marsh (1990, 2003) focused on the development of academic self-concept, and the impact age, gender, socioeconomic status, and personal experiences has on its formation. They found that students are better able to differentiate between the different domains of self-concept as they age, and that prior academic achievement impacts the formation of self-concept in both males and females. Additionally, Williams (1996) and Byrne (1998) attempted to find the causal predominance between academic achievement scores and academic self-concept, but had mixed results.

Studies done on the impact of standardized testing have had conflicting results, with Paris et al. (1991) finding that students are increasingly dissatisfied with testing over time, and have a decreasing motivation to excel on tests. Yet, Roderick and Engel’s (2001) research indicated that even traditionally low-achieving students attempt to rise to meet the challenge when the academic achievement tests carry “high stakes” consequences. Rosenshine’s (2003) research supports Roderick and Engel’s conclusions, finding states that attached consequences outperformed a comparison group of states on NAEP tests.

The purpose of this case study was to enhance the current literature on academic self-concept, investigating if standardized testing has an impact on the academic self-concept of gifted fifth grade students. It attempted to determine if self-concept is content specific in the domains of reading and mathematics, if students differentiate between academic self-concept and global self-concept, and if a relationship exists between standardized achievement testing and academic self-concept.

This study was designed to utilize the Self-Descriptive Questionnaire-I (SDQI) as a tool for measuring the self-concept of gifted students in the fall and spring of their fifth grade year. As the research of Yeung (2003) supports a “chameleon effect,” the SDQI was administered to students after the completion of the ISAT tests in both September and April. If students can easily relate esteem or general self-concept questions to academic characteristics, results may indicate an overall influence of ISAT testing on student academic self-concept. In addition, Stader and Licht (1992) found that the administration of questionnaires in a group setting produced higher and more accurate predictions of academic expectations and self-evaluation than questionnaires administered to children individually. This research indicates that the procedures outlined in Chapter Three resulted in SDQI data that is an accurate representation of the participants’ self-concept. 

For this case study, data from the SDQI surveys administered after the fall and spring ISAT tests was analyzed, and information gathered in semi-structured interviews was themed and coded to provide additional material for ­­­interpretation. Self-concept scores were compared to determine students could differentiate between global self-concept and academic domains, and if changes in the different self-concept domains occurred during the course of the 2005-2006 school year. Additionally, changes in ISAT scores were compared to the academic self-concept scores in the domains of reading and math to determine if a relationship existed between standardized test scores and self-concept.

 

Descriptive Statistics and Data

In this study, responses to the SDQI were scored using the instructions from the SDQI Manual.  First, individual scale raw scores were used to calculate the Total Academic, Total Nonacademic, and Total Self raw scores, as well as optional control raw scores. The information from each SDQI survey was entered into an Excel spreadsheet, with items grouped according to the different self-concept scales: Physical Abilities, Physical Appearance, Peer Relations, Parent Relations, Reading, Math, General-School, and General-Self, in addition to the control items. Each response was converted to a numerical score in which False=1, Mostly False = 2, Sometimes False/Sometimes True = 3, Mostly True = 4, and True = 5. The raw scores for each self-concept scale were determined by adding all scores for that scale. If a student omitted three or fewer responses, the mean response for the missing item (provided in the SDQI Manual) was substituted for the missing item’s score, as per scoring instructions. No student omitted more than two responses on any of the surveys. Caution was taken when entering scores into the table, and raw scores were double-checked for accuracy. Total Academic scores were calculated by adding the raw scores of the individual self-concept scales for Reading, Math, and General School, then dividing by three to find the average. Total Nonacademic scores were calculated by adding the raw scores of the individual self-concept scales for Physical Abilities, Physical Appearance, Peer Relations and Parent Relations, then dividing by four to find the average. A Total Self score was created for each individual by adding the Total Nonacademic score to the Total Academic score and dividing by two. For both fall and spring issued surveys, the raw scores for Reading and Math scales were calculated, as well as the raw scores for Total Self, Total Academic, and Total Nonacademic scales.

Norms for the SDQI individual and total scales were presented as mid-interval percentiles. However, Marsh, Barnes, et al. (1984) indicated the distribution of scores on the individual scales was skewed in the direction of higher self-concept scores. There are three approaches for making comparisons among data: the comparison of raw scale scores; the comparison of percentiles; and the comparison of standard scores (T scores). For the purpose of this study, the only comparison of raw scores was between the individual self-concept scales for Reading and Math; all other comparisons were generated from percentile rankings.

Subject Specificity of Self-Concept Scores

The comparison of raw scores is not advised by Marsh, as minor wording changes in a few items on even one scale could raise or lower all scores on that scale relative to the other scales. For this reason, raw scores should not be compared to try and find underlying differences in self-concept. However, because the wording of items in the three academic scales is strictly parallel, changing only the terms “Reading,” “Mathematics,” and “All School Subjects,” these three scales may be compared using raw scores.

 The differences between raw scores in three academic domains can be considered real or “absolute,” as a review of the norms indicates that the Math self-concept scores are systematically lower than Reading or General School scores, especially for females. Therefore, if scores are standardized, the mean for each domain is set at a constant value, and the absolute difference between Reading and Math self-concepts is lost.

 In this study, a comparison of raw scores in Reading and Math self-concept domains demonstrated that students could differentiate between Reading and Math self-concept domains. Analysis of data presented in Table 1 indicated Reading self-concept is higher than Math self-concept, with five out of seven students in fall, and six out of seven students in spring, reporting higher Reading self-concept scores. The average Reading self-concept score was at least five points above the average Math self-concept score in both fall and spring. As well, Reading scores remained more stable over the school year, with only four students demonstrating a change in Reading self-concept scores, while six out of seven students demonstrated a change in Math self-concept scores.

Table 1 Comparison Raw Scores in Reading and Math Scales

	Reading- Raw FALL	Reading- Raw SPRING	Difference Reading Fall to Spring	Math- Raw FALL	Math- Raw SPRING	Difference Math Fall to Spring
Theirry	35	35	0	39	31	-8
Mia	34	34	0	31	24	-7
Hamm	33	24	-9	23	20	-3
Bob	38	40	2	32	26	-6
Fredet	40	40	0	24	27	3
Luna	39	35	-4	32	26	-6
Miranda	40	29	-11	40	40	0
Average	37	34	-3	32	28	-4

 

 

Academic vs. Global Self-Concept

            A comparison of the three Total self-concept scales (Total Non-Academic, Total Academic, and Total Self) utilized mid-interval percentiles, as wording is not parallel between the survey items in the different domains compiled to reach these three scales. Marsh indicated that individual interpretation of minor wording changes in survey items may systematically raise or lower all scores in any of the domains used to calculate the three Total self-concept scales. In order to compensate for this, raw scores of the normative sample for the SDQ-I were converted into percentiles so a student’s relative position in the standardization sample could be identified. Percentiles will indicate the percentage of children in the standardization sample whose scores fell below each student’s score.

As half of the standardization sample of 3,562 utilized by Marsh et al. (1984) had self-concept scores that fell between the 25^th and 75^th percentiles, scores in this range are neither particularly high nor low, indicating and average self-concept. Also, scores within this range do not result in large percentile differences as they do in the extreme upper portion of the scale, therefore they can be considered to be readily interpretable and diagnostically meaningful. Only one student had scores consistently above the 75^th percentile, but due to the small sample size her scores were included for analysis.

In Table 2 the data indicated four out of seven students reported a decrease in Total Non-Academic self-concept (Physical Abilities, Physical Appearance, Peer Relations and Parent Relations), while two reported an increase, and one student reported no change between fall and spring. However, all but one student reported a decrease in Total Academic self-concept (Reading, Math, and General School) by the end of the school year. Total Self scores paralleled Total Academic scores, with six out of seven students reporting a decrease in Total Self self-concept, and one student reporting an increase over the school year, however they were not equivalent.  This implied that the students weighed Academic self-concept more heavily than Non-Academic self-concept when creating an overall Total Self self-concept, but that students differentiated between academic and global self-concept, even in an academic setting.

 

Table 2 Comparison Percentiles Between Total Non-Academic, Total Academic, and Total Self Self-Concept Scores

	Total Non- Academic  Percentile Fall	Total Non- Academic Percentile Spring	Total Academic Percentile Fall	Total Academic Percentile Spring	Total Self Percentile Fall	Total Self Percentile Spring
Theirry	50	50	70	59	64	55
Mia	86	73	58	45	80	64
Hamm	66	80	21	10	37	30
Bob	66	58	70	59	72	64
Fredet	52	73	64	71	55	72
Luna	24	14	77	39	55	23
Miranda	98	86	99	77	99	86

 

 

Impact of Standardized testing on Academic Self-Concept in Reading and Math

The ISAT test generates students’ scores based on the responses given; correct answers result in the presentation of a more difficult question, while incorrect answers results in the presentation of a question that is easier for the student to answer. Because of this leveling system, the ISAT test is not the same for every child; while the RIT level of the questions may be the same, the questions are not. ISAT scores are compared based on the rationale that the RIT levels of the questions are the same, so students who receive similar RIT scores have a similar level of competence in the subject.

While the SDQI questions for the Reading, Math, and General School domains have parallel wording, the ISAT test does not. Therefore, when comparing changes in student scores on the ISAT to changes in self-concept, percentile rankings were utilized in order to make the scales more equitable.

The data presented in Table 3 showed that on the Reading ISAT six out of seven scores went up, but only one out of seven indicated a higher Reading self-concept at the end of the school year. While only one student’s Reading ISAT scores fell between fall and spring, four of seven students’ Reading self-concept scores went down. Because three were above the 75^th percentile, interpretation of this drop needed to cautious, as scores above the 25^th to 75^th percentile range result in larger differences between percentile rankings. However, in the case of Hamm, whose percentile score fell in the average self-concept range, a drop of 32% was significant. Bob, who demonstrated the most improvement between fall and spring on the Math ISAT, reported the only increase in Reading self-concept. In contrast, while Mia was the only student whose Reading ISAT score dropped, it was Hamm and Fredet who reported the largest decreases in Math self-concept, even though they both improved their Reading ISAT scores. In fact, Fredet scored the highest of all seven students on the Reading ISAT, but was one of two who made the most significant drop in self-concept. These contradictory findings indicate that changes in Reading self-concept are not parallel with changes in achievement on the Reading ISAT.

Table 3 Changes in Reading ISAT Scores Compared to Changes in Reading Self-Concept Scores

	Reading ISAT Score Fall	Reading ISAT Score Spring	Difference Reading ISAT Fall to Spring	Reading Self-Concept Percentile Fall	Reading Self-Concept Percentile Spring	Difference Reading Self-Concept Percentile
Theirry	220	234	14	68	68	0
Mia	230	224	-6	51	51	0
Hamm	230	234	4	46	14	-32
Bob	205	231	26	83	95	12
Fredet	233	238	5	93	61	-32
Luna	223	228	5	82	56	-26
Miranda	217	218	1	93	29	-64

 

            In Table 4 the data showed that all seven students improved their score on the Math ISAT, but five of the seven students reported a decrease in their Math self-concept scores. Logically, the student who had the highest score on the Math ISAT of all seven students was the only student to report an increase in Math self-concept. But while the student who made the smallest amount of growth had the largest decline in Math self-concept, his Math ISAT score was still higher than that of four others. Conversely, the two students who made the largest amount of growth on their Math ISAT showed the second and third largest drops in Math self-concept. These contradictory findings indicate that changes in Math self-concept are not parallel to achievement on the Math ISAT.

Table 4 Changes in Math ISAT Scores Compared to Changes in Math Self-Concept Scores

	Math ISAT Score Fall	Math ISAT Score Spring	Difference Math ISAT Fall to Spring	Math Self-Concept Percentile Fall	Math Self-Concept Percentile Spring	Difference Math Self-Concept Percentile
Theirry	238	239	1	88	49	-39
Mia	221	237	16	62	34	-28
Hamm	240	246	6	29	21	-8
Bob	221	237	16	54	29	-25
Fredet	228	241	13	34	47	13
Luna	235	238	3	66	43	-23
Miranda	225	238	13	97	97	0

 

Findings from Open-Ended Questions

I employed a multi-layered approach in analyzing the data.  I first coded interviews to identify common themes using the constant comparative method (Strauss & Corbin, 1990). The codes that emerged from interviews included: Response to the ISAT format, Perceived Fairness of Differentiated Test Questions, Evidence of Curriculum Representation on the ISAT, Accurate Representations of Achievement, Competition Between Self and Others, Helpfulness of Providing Growth Scores, Emotional Response to ISAT/ Pressure and Nervousness, and Consequences for Not Meeting Proficiency.

I then created data matrices (Miles & Huberman, 1994) to identify the following constructs: What were the categories for analysis, When does it occur, Where does it  occur, Why does it occur, and How does it occur.

 While the students responded to all of these themes, some were much more effusive about their opinions. However, all participants appeared to respond candidly, some even humorously, to the interview questions, and none demonstrated any verbal or physical signs of nervousness during the interview process.

Students were first asked to respond to the format of the test, and explain how the test questions were selected. Fredet was the only student who responded she liked the paper and pencil version of the ISAT better than the computerized version, stating that on the computerized version, “if you figured something out you can’t go back,” and that looking at the computer monitor caused her to have headaches during the test.  However Miranda and Bob were representative of the other students, stating that “it doesn’t wear your hand out,” and that responding on the computers is easier. All students seemed to understand that the computerized tests allowed for the presentation of leveled questions, although there were some discrepancies among the students as to how the test began to level the questions. One student stated that the test was based on “the [previous] score and the level of questions,” while another student responded that questions were based on “what grade you’re in, and your past ISAT scores, like what you’ve gotten in the past…how smart you are.”

Despite a lack of firm understanding of how the test questions were leveled, the students had contradictory responses as to the fairness of leveled tests. Fredet’s response represented what several of the female students felt- that the “scores don’t make any sense because we all get different questions. So you might know this one question so they’ll bump you up, but if you don’t then they’ll bump you down. But you might know everything else…they might not know if you really know it or not.” Hamm was another student who agreed that the test questions should be the same for every student, but she also expressed the opinion that the test questions should get more difficult as the year progressed. On the other hand the boys expressed the opinion that they enjoyed the challenge of the leveled questions, and saw the opportunity for teachers to use the scores for diagnostic purposes. Bob’s reflection on the matter was well stated: 

I think it’s better if we all have different questions because we all learn at different rates. So we all- some people know more than others. So it wouldn’t be fair to someone who has a harder time learning stuff. It wouldn’t be fair to them if they got like a really hard question and then the people who are really smart, it’s really easy for them, but the people who learned at a slower rate would have a harder time.

Yet there was consensus in the belief that the ISAT represented what was taught by the curriculum over the school year, and that it was an accurate measure of comprehension. Thierry expressed that he felt more confident on the Spring ISAT tests and had earned a higher score because he had recognized the content, even though the work was more challenging than normal. Miranda expressed that much of what she saw on the test was not yet covered in the classroom curriculum, but revealed utilizing test-taking strategies learned in class to deduce the correct answers. Hamm, however, responded that while she felt the curriculum was well represented on the Reading and Math ISAT tests, the curriculum did not prepare the students effectively for their Language ISAT. 

Despite their responses that indicated the curriculum prepared students for the ISAT tests, all students reported that they felt it was not the most accurate representation of student achievement. All students responded that quarter and semester grades were more reflective of achievement as they represented what was done in class every day instead of just focusing on specific subject for a short time. Two students reflected that  “…they [shouldn't] send only one test to the state. Because what if you forget something that day, or you’re really tired…then you do bad [sic] on your ISAT. And just because you do bad…doesn’t mean that you didn’t [sic] know anything,” while grades are “over a longer period of time and you’re not going to be tired like every single day of the entire year.” Bob also expressed displeasure that not every grade had to take a Science ISAT, and that some subjects were not tested at all, indicating that the ISAT tests did not represent everything the students were learning in class. 

Although students disagreed on the fairness of scoring tests that contained different questions for each student, all students expressed an awareness of the competition that occurs after the test scores are calculated and given to students. Responses indicated that the competition has always been present, that everyone competes against one another, that they enjoy the competition, and they compete not just academically but in sports as well. Several students indicated that it was a “friendly” competition, however one student informed me that it added pressure to know that his friends would be asking him what his score was and comparing it to theirs. One student responded that scores were compared among GT students across grade levels, although proficiency scores are different for each grade. However, Miranda and Thierry both expressed that it doesn’t mater if someone scores higher than they do as long as they are not at the bottom of their classes. In addition, all students indicated that they competed with themselves to beat their previous scores, and two students expressed feeling more internal pressure to do better than external pressure from friends, parents or teachers.

Students also indicated that their knowledge of their growth score influenced how much pressure they put on themselves. One student indicated that she felt pressured by the knowledge of what her growth score was, nervous that she might not achieve it on the Spring ISAT tests. On the other hand, three students reported that growth scores were helpful in setting goals for the score they wanted to achieve in spring, and acted as a motivator to work harder. Hamm agreed with this concept, but added that growth scores were probably more effective for more advanced students as it might make the kids who struggled academically “feel bad- worse than the…top kids.” 

Despite reporting the competition among gifted students was friendly, and the pressure to do well on the ISAT tests was predominantly internal, several students expressed feelings of anxiety about taking the test. One responded she was always concerned her score would not be “advanced enough,” although she could not expand on consequences that might occur if she received a lower score. She expressed feeling the scores were arbitrary numbers, as students did not have any concept about what the numbers represented. Another student reported feeling nervous when the majority of the material presented in the test was above the level he had been instructed in, and therefore he did not feel confident in his answers. Conversely, one female student reported feeling more confident as she encountered increasingly difficult problems on the ISAT, as it indicated she had responded correctly to the previous questions.  Another student expressed that the Spring ISAT in 5^th grade was the first time he felt nervous, as he knew that there were consequences for students who did not meet proficiency. Prior to this he said he had felt confident that he would do well as he had always received advanced scores on previous ISAT tests, and he knew he was academically advanced, as he received good grades and read difficult books.

All students had been informed by their teachers of the possible consequences for not meeting proficiency on the three ISAT tests. Several of the participants were already in orchestra, and had firm ideas as to the fairness of these consequences. Two participants responded that other students who did not meet proficiency in fifth grade should not be able to participate in extra-curricular classes such as orchestra or band. Their rationale was that students who did not meet proficiency needed to focus more on academic achievement and wouldn’t have time for the extra classes, or that they might not want to participate anyway. However, Thierry, who plays both the upright bass and the guitar, hotly argued that “if they want to play [they] should be able to play,” as those students could receive additional academic support after school. Another student expressed that a sliding scale would be more appropriate, as it would be unfair to limit a student who may have missed a proficient score by only one or two points.

 

Summary

By utilizing both the Self-Description Questionnaire-I and semi-structured student interviews, the research questions guiding this case study have been analyzed both quantitatively and qualitatively. Survey results demonstrated some alarming changes in academic self-concept domains among the participants, although the comparison of some self-concept scales parallels the findings of previous research done by Marsh (1990, 2003), Williams (1996) and Byrne (1998). Collecting the individual responses of the participants in interviews enriched this data by providing specific thoughts participants were unable to share in the survey. 

A comparison of raw Reading and Math self-concept scores was used to find an “absolute” difference, as Marsh et al. (1984) found Math self-concept scores to systematically lower than Reading self-concept scores. Marsh et al. argued that the real difference between Reading and Math self-concept would not emerge if scores are standardized and the mean for each scale is set at a constant value. This case study supported the findings of Marsh et al., with survey data indicating a higher and more stable self-concept in Reading, and interview data indicating several students feeling higher levels of anxiety when taking the Math ISAT. This confirmed that students at this age can differentiate between the domains of Reading and Math self-concepts.

This case study also found that students differentiate between Total Self (global) and Total Academic (academic) self-concept. A comparison of Total Non-Academic, Total Academic, and Total Self scales revealed that the students associated their global self-concept more closely to their academic self-concept than to non-academic domains. Interview data indicated that several participants placed more significance on non-academic domains than other students in the sample, however all students but one admitted that they competed more rigorously in academics than they did in other domains.

However, when survey data was analyzed to ascertain if a positive relationship between changes in ISAT scores and changes in academic self-concept domains existed, the scores did not indicate that such a relationship existed. In reading, the student with the highest ISAT scores also had the largest drop in academic self-concept, and although all students but one showed improvement in ISAT scores, four of the seven students had decreasing self-concept scores. Contradictory findings in Math also indicated that while some students made the largest gains in ISAT test scores, they also reported some of the largest decreases in Math self-concept. Interview data suggested that while several students were confidently bolstered by prior academic achievement, the majority of the students expressed nervousness when taking the ISAT due to new and challenging content, as well as internal and external pressure to do well.  This emotional response to the test may have been a factor contributing to dropping self-concept scores in spring, as those are the ISAT tests that hold potential consequences for students.

The findings of this study have confirmed what several studies have shown on the differentiation between self-concept domains and the formation of global self-concept. Although the scope of this case study was limited, it demonstrated that negative changes in self concept can occur in even the brightest and most confident students, and can provide educators information on how students create and differentiate their self-concept at this age.

Chapter Five: Summary, Conclusions and Recommendations

 

Summary

Although achievement tests have been utilized for almost a century, standardized achievement testing did not become widespread until the 1965 Elementary and Secondary Education Act necessitated standardized achievement tests as a measurement of educational success. Currently, states must indicate “adequate yearly progress” by showing that 95 percent of students took a standards-based exam and met state established proficiency standards in math and reading. In the state of Idaho, students must also demonstrate a year’s worth of academic growth through the ISAT tests.

In Chapter One, I described the pattern of behavior concerning gifted students and their response to their performance on the ISAT tests which I have observed in my own classroom. In the fall the gifted students are excited about their proficiency scores, however that excitement soon turns to anxiety after the Winter ISAT test, as the students realize how difficult it is to maintain or raise their scores. By spring, the students struggle to improve their scores when testing on subject matter that will not be introduced until after the fifth grade. Still, a fierce competition continues among the gifted students as to who earned the highest score and who made the most growth, adding additional pressure to increase their proficiency scores in spring.

For the last few decades, researchers have studied how a student’s experiences in school correlate with the development academic self-concept in children (Arsenault, 2001; Byer, 1999; Harter et al., 1992; Marsh et al., 1995; Marsh & Hau, 2003; Plecha, 2002; Van Boxtel & Monks, 1992; Williams & Montgomery, 1994). In addition, many researchers have been studying the link between academic self-concept and academic achievement, which has appeared to have a reciprocal relationship (Anderman et al., 1999; Guay et al., 2003; Marsh, 1992b; Van Damme & Mertens, 2000; Yeung & Lau, 1998). That academic self-concept is also subject-specific, especially as the students’ ages increase, has also been indicated by several studies (Marsh, 1990, 1992a, 1992b, Marsh et al., 1999).  The results of these studies, detailed in Chapter Two, appear to relate to the experiences of the gifted fifth grade students with whom I work- it is for that reason that I chose the topic of academic self-concept for the purpose of my research.

The style of this case study, detailed in Chapter Three, was a mixed-method, non-experimental study, following the framework outlined by Merriam (1998), utilizing both the Self-Descriptive Questionnaire I survey (Marsh & Yeung, 1999) with 76 Likert-type questions, as well as semi-structured interviews. Surveys were administered twice during the 2005-2006 school year, once following the Fall ISAT tests and another following the Spring ISAT tests. This was to allow students to relate their experiences taking the ISAT tests to their self-concept, a phenomenon described as the “chameleon effect” in the research of Yeung (2003). Surveys were completed together following the instructions of the SDQI manual, as the work of Stader and Licht (1992) indicated that students responded more accurately when surveys were administered in a group setting. After administration of the surveys was complete, individual semi-structured interviews were videotaped and transcribed. Pseudonyms were selected by students in order to protect anonymity, and member checks of interview transcripts were completed to ensure accuracy. The data was then compiled and analyzed in an attempt to answer the three questions guiding this case study, as detailed in Chapter Four.

 

Conclusions

This case study attempted to answer if self-concept is content specific in the subjects of Reading and Mathematics, if students can differentiate between academic self-concept and global self-concept, and if there is a positive relationship between standardized achievement testing and academic self-concept. Utilizing SDQI data, raw self-concept scores were compared to determine if students could differentiate between the different domains of academic self-concept, and percentile rankings were used to compare academic and global self-concept, as well as compare ISAT proficiency scores to academic self-concept scores in Reading and Math. Interview responses were included to add deeper contextual meaning to the survey results.

In order to answer the first question, the results of the SDQI were tabulated and compared, utilizing raw data to compare Reading and Math self-concept. The comparison of Reading and Math self-concept scores indicated that students do differentiate between the two academic domains, as Reading self-concept scores were higher and more stable than Math self-concept scores. Five out of seven students in fall, and six out of seven students in spring, reported higher Reading self-concept scores than Math self concept scores. As well, Reading scores remained more stable over the school year, with only four students demonstrating a change in Reading self-concept scores, while six out of seven students demonstrated a change in Math self-concept scores. These results mirrored the results found by Marsh et al. in 1984, which indicated that Math self-concept is systematically lower than Reading self-concept. Interview responses confirmed these findings, indicating that several students felt more confident about their performance on the Reading ISAT than on the Math ISAT.

The comparison of percentile rankings in Total Non-Academic, Total Academic, and Total Self scales indicated that students did differentiate between academic and global self-concept. While four of seven participants reported decreasing self-concept scores on the Non-Academic scale, six out of seven reported decreased self-concept on the Academic scale. Total Self scores paralleled Total Academic scores, with six out of seven students reporting a decrease in Total Self self-concept. The data suggests that the students who participated in this study weighted academic self-concept more heavily than non-academic self-concept when creating their global self-concept. Student responses in interviews confirmed this analysis, with only two students mentioning the importance of achievement in non-academic areas such as sports.

No positive relationship could be found between achievement on the Reading and Math ISAT tests and the formation of academic self-concept. The data showed that on the Reading ISAT six out of seven scores went up, but only one out of seven indicated a higher Reading self-concept at the end of the school year; meanwhile only one student’s Reading ISAT scores fell between fall and spring, but four of seven students’ self-concept scores in Reading went down. In Math the results were even more contradictory, with the data showing that all seven students improved their score on the Math ISAT, but five of the seven students reported a decrease in their Math self-concept scores.  In the semi-structured interviews, several students explained that confidence was a large factor in how they viewed their achievement on the ISAT tests. Additionally, comments were made about the difficulty of completing problems on the Math ISAT that represented materials not taught until after fifth grade. These two factors may explain why self-concept scores in math dropped, even though all students improved their proficiency scores on the Math ISAT.  Therefore, it is reasonable to conclude that student perception of achievement may actually be a larger influence on academic self-concept than actual achievement on standardized tests.

 

Recommendations for Practice

The results of this case study have potential implications on the practices of educators concerning how achievement in different academic areas, as well as on standardized tests, is measured and addressed in the classroom. It appears that the academic self-concept of gifted students is falling at a disturbing rate. If the brightest students in a class, accustomed to academic achievement and confident in their own abilities, demonstrate consistent decreases in their self-concept, it begs the questions as to the response of the academic self-concepts of average or struggling students.

Teachers, administrators, and policy makers need to analyze their dependence on standardized tests, and look for alternative methods for assessing student achievement. As well, educators need to create a forum to openly communicate their observations of student responses to different types of assessments. Additionally, teachers need to establish an open dialogue with students concerning the function of assessments, student perceptions of achievement, and their role in the academic environment.

Previous research has found that Math self-concept scores are systematically lower than Reading self-concept scores, and has also provided evidence that academic self-concept is correlated to academic achievement.  Therefore, students need to be given a forum to express why they feel less confident in their math abilities compared to their abilities in other subject areas, and how achievement in math effects their perceptions of competence.

In addition to creating an open dialog about student achievement on assessments, teachers also need to provide students with an opportunity to review curriculum and problem solving methods. As the spring ISAT assesses an entire year of curriculum, students should be provided with a review of materials to help boost their confidence when presented with test questions on subjects taught in the first months of the school year. Moreover, problem solving skills needs to be systematically taught to students in order to increase their ability to reason through test questions on content that may not have been addressed by the curriculum for that grade level.

 

Recommendations for Future Study

The limited nature of this case study, based on its size and scope, necessitates further investigation. Future studies may want to incorporate more participants, in multiple grade levels, in an attempt to make the study more generalizable. The study could also be replicated utilizing different student populations, surveying participants at different school locations or in different districts across that state. This would also provided researchers with data on how socioeconomic status may influence academic self-concept, as well as further the research done by Marsh and Parker (1984) which has indicated the overall academic performance level of a school impacts the academic self-concept of its students.

Altering the methodology of the case study to include additional interviews, or fully- structured interviews, could also result in data that might clarify student responses to standardized testing more fully. It could also provide researchers with data on student perceptions of different types of assessment, such as portfolios of student work or classroom tests given in schools that utilize a district-wide comprehensive curriculum. 

Moreover, further research on the impact of standardized testing on the academic self-concept of all students is recommended. This research could potentially focus on the effects of achievement testing on particular sub-groups such as Limited English Proficiency, as well as students enrolled in Title I programs, Special Education programs, and general education classrooms. Additional analysis of data should also occur to determine if females have continue to have consistently lower Math self-concept scores than males, as first indicated by Marsh et al. (1984).

Future research may also reveal how the interactions of different self-concept domains impact the formation of a global self-concept, and how the formation of those domains are influenced by parent, teacher, and peer interactions. Additionally, as the ISAT was revised in the spring of 2007 to assess only the content taught in a student’s specific grade level, research on causal relationships between academic achievement and the formation of self-concept needs to continue.

 

 

References

Amrein, and Berliner. (2002, March). High stakes testing and student learning [Electronic Version]. Education Policy Analysis Archives, 10, No 18.

Anderman, E.M., Anderman, L.H., & Griesinger, T. (1999). The relation of present and possible academic selves during early adolescence to grade point average and achievement goals. Elementary School Journal, 100(1), 3-17.

 Arsenault, D. J. (2001, April 19-22). Behavioral, social, and cognitive predictors of adolescent academic self-concept: A longitudinal investigation. Paper presented at the annual meeting of the Society for Research in Child Development, Minneapolis, MN.

Bong, M., & Clark, R. E. (1999). Comparison between self- concept and self- efficacy in academic motivation research. Educational Psychologist 34(3), 139-153. 

Byer, J. L. (1999, February 17-19). Measuring the effects of students’ perceptions of classroom social climate on academic self-concept. Paper presented at the annual meeting of the Louisiana Education Research Association, New Orleans, LA.

Byer, J.L. (2001). The consistency of correlation between students’ perceptions of classroom involvement and academic self-concept in secondary social studies classes. Livingston, AL: University of West Alabama. (ERIC Document Reproduction Service No. ED457207)

Byrne, B. M. (1998, April 13-17). Testing for causal predominance between academic self-concept and academic achievement: A developmental perspective. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.

Chan, D. W. (2001). Global and specific self-concepts of gifted adolescents in Hong Kong. Journal for the Education of the Gifted, 24(4), 344-364.

City Data. (n.d.) Retrieved July 7, 2007 from http://www.city-data.com/city/Caldwell-Idaho.html

Craven, R. G., Marsh, H.W., & Debus, R. (1991). Effects of internally focused feedback and attributional feedback on the enhancement of academic self-concept. Journal of Educational Psychology, 83, 17-26.

Craven, R. G., McInerney, V., & Marsh, H. W. (2000, April 24-28). The structure and development of young children’s self-concepts and relation to academic achievement. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Cuban, L. (1993). How teacher’s taught (2nd ed.). New York: Teachers College Press.

Delisle, J., & Galbraith, J. (1987). The gifted kids survival guide, II. Minneapolis: Free Spirit.

Guay, F., Marsh, H.W., & Boivin, M. (2003). Academic self-concept and academic achievement: Developmental perspectives on their causal ordering. Journal of Educational Psychology, 95(1), 124-136.

Haladyna, T.M., Nolen, S.B., & Haas, N.S. (1990). Raising standardize achievement test scores and the origins of test score pollution. Educational Researcher, 20(5), 2-7.

Harter, S., Whitesell, N.R., & Kowalski, P. (1992). Individual differences in the effects of educational transitions on young adolescents’ perspectives of competence and motivational orientation. American Educational Research Journal, 29(4), 777-807.

Idaho Board of Education. (2002a, June 17). Press Release: ISAT pilot data summary review. Retrieved July 7, 2007, from http://www.boardofed.idaho.gov/press_releases/

Idaho Board of Education. (2002b, September 10). Press Release: State launches new test on Monday. Retrieved July 7, 2007, from http://www.boardofed.idaho.gov/press_releases/

Idaho Board of Education. (2002c, September 13). Press Release: Idaho one of two states to use computers for testing. Retrieved July 7, 2007, from  http://www.boardofed.idaho.gov/press_releases/

Idaho Board of Education. (2003a, February 19). Press Release: Idaho capitalizing on new testing program that will assist in providing quality education to all. Retrieved July 7, 2007, from http://www.boardofed.idaho.gov/press_releases/

Idaho Board of Education. (2003b, November 5). Press Release: State board of education votes on rules for charter schools, graduation requirements. Retrieved July 7, 2007, from  http://www.boardofed.idaho.gov/press_releases/

Idaho Board of Education. (2004a, January 15). Press Release: Board releases Fall ISAT scores. Retrieved July 7, 2007, from http://www.boardofed.idaho.gov/press_releases/

Idaho Board of Education. (2004b, January 26). Press Release: State board approves third academic indicator. Retrieved July 7, 2007, from http://www.boardofed.idaho.gov/press_releases/

Idaho Board of Education. (2005, January 11). Press Release: State board releases Fall ISAT scores. Retrieved July 7, 2007, from http://www.boardofed.idaho.gov/press_releases/

Idaho State Department of Education. (n.d.a). Assessment and accountability reports. Retrieved July 7, 2007, from http://www.sde.idaho.gov/dept/testreports.asp

Idaho State Department of Education. (n.d.b). State report cards by district and school. Retrieved July 7, 2007, from http://www.sde.state.id.us/ipd/reportcard/SchoolReportCard.asp

Linn, R.L. (1987). Accountability: The comparison of educational systems and the quality of test results. Educational Policy, 1, 181-198.

Maehr, M. L. (1979, September 1-5).  Sociocultural and motivational considerations in the assessment of educational achievement: A theoretical overview. Paper presented to annual convention of the American Psychological Association, New York NY.

Marsh, H.W. (1984a). Self-concept, social comparison and ability grouping: A reply to Kulik and Kulik. American Educational Research Journal, 21, 799-806.

Marsh, H.W. (1984b). Self-concept: The application of a frame of reference model to explain paradoxical results. Australian Journal of Psychology, 28, 165-181.

Marsh, H.W. (1986). Verbal and math self-concepts: An internal/external frame of reference model. American Educational Research Journal, 23, 129-149.

Marsh, H.W. (1990). The structure of academic self-concept: The Marsh/Shavelson Model. Journal of Educational Psychology, 82(4), 623-636.   

Marsh, H.W. (1992a). Content specificity of relations between academic achievement and academic self-concept. Journal of Educational Psychology, 84(1), 35-42.  

Marsh, H.W. (1992b). The content specificity of relations between academic achievement and academic self-concept: An extension of the Marsh/Shavelson Model. Paper presented at the annual meeting of the American Educational Research Association, San Francisco CA.

Marsh, H.W. (1993). The multidimensional structure of academic self-concept: Invariance over gender and age. American Educational Research Journal, 30(4), 841-860.

Marsh, H.W. (2003, November). A reciprocal effects model of the causal ordering of academic self-concept and achievement. Paper presented to the NZARE AARE, Auckland, New Zealand.

Marsh, H.W., Barnes, J., Cairns, L., & Tidman, M. (1984) The Self -Descriptive Questionnaire (SDQ): Age effects in the structure and level of self-concept for preadolescent children. Journal of Educational Psychology, 76, 940-956.

Marsh, H.W., Cairns, L., Relich, J.D., Barnes, J., & Debus, R. (1984). The relationship between dimensions of self-attribution and dimensions of self-concept. Journal of Educational Psychology, 76, 3-32.

Marsh, H.W., Chessor, D., Craven, R., & Roche, L. (1995). The effects of gifted and talented programs on academic self-concept: The big fish strikes again. American Educational Research Journal, 32(2), 285-319.

Marsh, H.W., & R.G. (1997). Academic self-concept: Beyond the dustbowl. In G. Phye (Ed.), Handbook of Classroom Assessment: Learning, achievement, and adjustment (pp. 131-198). Orlando, FL: Academic Press.

Marsh, H.W., Craven, R., & Debus, R. (1999). Separation of competency and affect components of multiple dimensions of academic self-concept: A developmental perspective. Merrill-Palmer Quarterly, 45(4), 567-601.

Marsh, H.W., & Hau, K-T. (2003). Big-Fish-Little-Pond Effect on academic self-concept: A cross-cultural (26 country) test of the negative effects of academically selective schools. American Psychologist, 58(5), 364-376.

Marsh, H.W., & Parker, J.W. (1984). Detriments of self-concept: Is it better to be a relatively large fish in a small pond even if you don’t learn to swim as well? Journal of Personality and Social Psychology, 47, 213-231.

Marsh, H.W., Relich, J.D., & Smith, I.D. (1983). Self-concept: The construct validity of interpretations based upon the SDQ. Journal of Personality and Social Psychology, 45, 173-187.

Marsh, H.W., & Shavelson, R.J. (1985). Self-concept: Its multifaceted, hierarchical structure. Educational Psychologist, 20, 107-125.

Marsh, H.W., & Smith, I.D. (1987). A cross-national study of the structure and level of multidimensional self-concepts: An application of confirmatory factor analysis. Australian Journal of Psychology, 39, 61-77.

Marsh, H.W., Smith, I.D., & Barnes, J. (1985). Multidimensional self-concepts: Relationships with sex and academic achievement. Journal of Educational Psychology, 77, 591-596.

Marsh, H. W., Smith, I.D., Barnes, J., & Butler, S. (1983). Self-concept: Reliability, dimensionality, validity, and the measurement of change. Journal of Educational Psychology, 75, 772-790.

Marsh, H.W., & Yeung, A.S. (1999). The lability of psychological ratings: The chameleon effect in global self-esteem. Personality and Social Psychology Bulletin, 25, 49-64.

Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass Publishers.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks: Sage Publications.

Olszewski, P., Kulienke, M., & Willis, G. (1987). Changes in the self-concept of gifted students who participate in rigorous academic programs. Journal for the Education of the Gifted, 19(4), 287-304.

O’Shea, W. (1927). What are the progressive steps of the New York city schools? Educational Review, 74(1), 99-103.

Paris, S., Lawton, T., Turner, J., & Roth, J. (1991). A developmental perspective on standardized achievement testing. Educational Researcher, 20(5), 12-20.

Patton, M.Q. (1985, April). Quality in quantitative research: Methodological principles and recent developments. Invited address to Division J of the American Educational Research Association, Chicago, IL.

Plecha, M. (2002, April 1-5). The impact of motivation, student-peer, and student-faculty interaction on academic self-confidence. Paper presented at the annual meeting of the American Educational Research Association, New Orleans LA.

Popham, W. J. (2001). The truth about testing: An educators call to action. Alexandria, VA: Association for Supervision and Curriculum Development.

Purkey, W. (1988). An overview of self-concept theory for counselors. ERIC Clearinghouse on Counseling and Personnel Services, Ann Arbor MI. (An ERIC/CAPS Digest: ED304630)

Resnick, L.B., & Resnick, D.P. (1989). Tests as standards of achievement in school. In Proceedings of the 1989 ETS Invitational Conference: The uses of standardized tests in American education (pp. 63-80). Princeton, NJ: Educational Testing Service.  

Roderick, M., & Engel, M. (2001). The grasshopper and the ant: Motivational responses of low-achieving students to high-stakes testing. Educational Evaluation and Policy Analysis, 23(3), 197-227.

Rosenberg, M. (1965). Society and the adolescent child. Princeton, NJ: Princeton University Press.

Rosenberg, M. (1979). Conceiving the self. New York: Basic Books.

Rosenshine, B. (2003). High stakes testing: Another analysis. Education Policy Analysis Archives, 11(4).

Shavelson, R.J., & Marsh, H.W. (1986). On the structure of self-concept. In R. Schwarzer (Ed.), Anxiety and cognitions. Hillsdale, NJ: Lawrence Erlbaum.

Smith, I.D., & Marsh, H.W. (1985). A cross-cultural study of self-concept in primary school children. Collected Papers of the 1985 Annual Conference of the Australian Association for Research in Education, pp. 350-353.

Stader, S.R., & Licht, B.G. (1992). Effects of questionnaire administration condition on children’s achievement-related beliefs. Journal of Educational Psychology 84(1), 28-43.

Stake, R.E. (1995). The art of case study research. Thousand Oaks, CA: Sage.

Strauss, A. L., & Corbin, J. M. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, Calif.: Sage Publications.

Strein, W. (1993). Advances in research on academic self-concept: Implications for school psychology. School Psychology Review, 22, 273-284.

Strein, William (1995). Assessment of self-concept. ERIC Digests, 073, Retrieved July 7, 2007, from http://ericae.net/db/edo/ED389962.htm

Strein, W., Cain, P., Edelman, E., & Schwalb, S. (1989). Big-Fish-Little-Pond, and other aspects of student’s academic self-concepts. Paper presented at the annual meeting of the National Association of School Psychologists, Boston MA.

Van Boxtel, H.W. & Monks, F.J. (1992). General, social and academic self-concepts of gifted adolescents. Journal of Youth and Adolescents 21(2), 169-186.

Van Damme, J., & Mertens, W. (2000). Academic self-concept and academic achievement: cause and effect. Paper presented at the annual meeting of the American Research Association, New Orleans, LA. 

Washington Elementary School Improvement Plan (SIP) for 2005-2006. (n.d.) Retrieved June 15, 2006 from

http://www.caldwellschools.org/Resources/SIP/Washington/Washinton.pdf

Wiggins, G. (1989). A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, 70, 73-713.

Williams, J. E. (1996, April 8-12). Academic the self-concept to performance congruence among able adolescents. Paper presented at annual meeting of the American Educational Research Association, New York NY.

Williams, J.E., & Montgomery, D. (1994). Frame of reference theory of self-concept formation with academically-able students. Paper presented at the annual meeting of the American Educational Research Association, New Orleans LA.

Yeung, A. S. (2003). Global self-esteem: Cognitive interpretation in an academic setting. Paper presented at the International Conference on Cognitive Science of the Australian Society for Cognitive Science, Sydney, Australia.

Yeung, A.S., & Lau, I.C.-y. (1998). The Internal/external frame of reference in the self-concept development of higher education students. Paper presented at the Conference of the Higher Education Research and Development Society of Australasia, Auckland New Zealand.

 

 

 

 

Appendices

Permission slips for Surveys

Permission Slips for Interviews

Survey Instrument: Self-Descriptive Questionnaire II (Marsh)

List of Interview Questions- Teacher

List of Interview Questions- Student

Don- do I need to label these Appendix A, Appendix B, etc? Should I include transcripts of student interviews, and if so should I number each line on the transcript in order to list the line number when I quote from the transcript?