Developed by Drs. Julie Libarkin and Steve Anderson

What is the Geoscience Concept Inventory (GCI)?

The Geoscience Concept Inventory (GCI) is a multiple-choice assessment instrument for use in the Earth sciences classroom.  We developed a pool of 73 questions that could be selected by an instructor to create a customized 15-question GCI subtest for use in their course.  These test items cover topics related to general physical geology concepts, as well as underlying fundamental ideas in physics and chemistry, such as gravity and radioactivity, that are integral to understanding the conceptual Earth.  Each question has gone through rigorous reliability and validation studies (Libarkin et al., 2005; Libarkin and Anderson, 2005a,b). 

We built the the GCI using the most rigorous methodologies available, including scale development theory, grounded theory, and item response theory (IRT).   To ensure inventory validity we incorporated a mixed methods approach using advanced psychometric techniques not commonly used in developing content-specific assessment instruments (Libarkin et al., 2002). We conducted ~75 interviews with college students, collected nearly 1000 open-ended questionnaires, grounded test content in these qualitative data, and piloted test items at over 40 institutions nationwide, with ~5000 student participants. 

In brief, the development of the GCI involved interviewing students, collecting open-ended questionnaires, generating test items based upon student responses, soliciting external review of items by both scientists and educators, pilot testing of items, analysis of items via standard factor analysis and item response theory, "Think Aloud" interviews with students during test piloting, revision, re-piloting, and re-analysis of items iteratively. Although time consuming, the resulting statistical rigor of the items on an IRT scale is suggestive of the reliability of this method for assessment test development.

What is the philosophy behind the Geoscience Concept Inventory (GCI)?

We were interested in creating an instrument that could be used to assess learning in classes, and also provide data for research on learning in the geosciences.  Therefore, we need an exam that is easy to deliver to a very large number of students, and easily graded.  Obviously, a multiple-choice format is optimal, provided it is valid and reliable.

However, how do we know if a multiple choice exam is really testing learning???  The main weakness with most multiple choice tests is that it is possible for a student to know very little about a concept, yet pick the correct answer on the exam because the other choices (wrong answers) were not attractive.  Obviously, a test designed in this way is not valid or reliable.

We have eliminated this problem with the GCI by picking incorrect answers that are attractive alternatives to students who do not fully understand the concept.  We interviewed nearly 250 college students about Earth science ideas, and used their most common misconceptions as wrong answers.  Therefore, we know that students are choosing the correct answer because they understand the material. Through our other test design procedures (such as expert review, student think-alouds, pilot testing, IRT and factor analysis, etc.) we have designed test that should yield results that educators and researchers can be confident in.

Example GCI Questions

Sample GCI Question #1. Which answer best describes what the surface of the Earth would be like if you could travel back to the time when the Earth first formed as a planet?

(A) The Earth was about the same temperature as today, and covered with jungles at the surface

(B) The Earth was about the same temperature as today, and covered with water at the surface

(C) The Earth's surface and temperature were similar to today, although no cities existed yet

(D) The Earth's surface was very hot and covered with melted rock

(E) The Earth's surface was very cold and covered with ice

All of the incorrect answers in the question above were actual responses from a number of interviews with college students.  In the example below, all of the incorrect diagrams were also from multiple interviews.

Sample GCI Question #2. The following maps show the position of the Earth's continents and oceans. The o's on each map mark the locations where volcanic eruptions occur on land.  Which map do you think most closely represents the places where these volcanoes are typically observed?

Circle one:        A         B         C         D         E      

   A. Mostly along the margins of the Pacific and Atlantic Oceans

    B. Mostly along the margins of the Pacific Ocean

  C. Mostly in warm climates

D. Mostly on continents

  E. Mostly on islands

Obviously, we were quite surprised when many students in interviews draw volcanoes in a band around the equator (the inspiration for answer C above).  When asked why, students commonly replied that volcanoes exist near the equator "because it's warmer there".  Understanding scales of time, temperature and pressure is clearly one area where college students struggle the most.

Some Common Questions and Comments About the GCI

1.  The test seems too "easy"?

We hear that comment quite frequently from instructors.  During pilot testing of the GCI in 2002, 18 of the 21 instructors who took the exam received perfect scores, and 2 others only had one incorrect answer. We've had some faculty go so far as to call the GCI "dumb"! In fact, during the early stages of GCI development, we had one instructor who asked us "Is there any rational person who doesn't believe the Earth has a magnetic field?" in response to one of the choices on the GCI.  As it turned out, a number of students in this instructor's course who also took the GCI didn't believe the Earth had a magnetic field.

Students, however, seem to find the course quite difficult, with average scores of 7.2 to 7.3 (out of 20) during the pilot phase of testing.  This disconnect between how the instructors feel about the test and how students perform on it is one of the more surprising aspects of our study.

2. The wording on some of the questions and answers seems a bit odd, certainly different than I would have asked it.

In most cases, the questions and the answers (both the right answer and the incorrect answers) are actually a fair bit different than when we first put the test together.  Through our validity and reliability studies (which includes expert analysis, student "think-alouds" (where students take the exam orally and talk out what they are thinking as they read the question) and item response theory) many of our original questions required modification to meet not only the demands of the experts, but to eliminate wording that was confusing to the students.  Again, one of the more important things we've found is that students and faculty view the exam questions and answers very differently, requiring questions/answers that pass muster by both experts and students.

3. Can I use the GCI for research?

It has always been our goal to produce an exam that would serve as both an effective assessment instrument for geoscience courses, but also a mechanism for studying learning.  We urge you to read the new papers we've published on the GCI (below) and contact us if you have an questions.  If you are using the GCI for research, please feel free to let us know since it enables us to make a stronger case for continued funding to expand and modify the test.

Important Documents

If you are using the GCI, please email us at steveanderson@bhsu.edu or libarkin@ohio.edu and let us know.  It's important for future funding that we document its use.  

    The current GCI (all 73 questions)

    Instructions for creating a 15 question subtest

    A Rasch scale that shows the relative difficulty of each question

    Instructions, explanations and the scaling equation for the GCI Rasch scale

    Answers are available by mailing steveanderson@bhsu.edu (we screen answer requests and only provide the answers to faculty/researchers)

Participating Institutions

It is not possible to statistically validate questions without a large sample size, and the following institutions have willingly participated in either the pilot testing of  the questions contained on the GCI, or have subsequently used it to assess learning in the classroom. We are greatly appreciative of the time and effort that the faculty at these institutions have committed to this project.  If I have inadvertently left any institution off this list, please let me know.

Publications

Libarkin, J.C., and Anderson, S.W., 2006a. The Geoscience Concept Inventory: Application of Rasch analysis to concept inventory development in higher education; Rasch Applications in Science Education, ed. X. Liu, JAM Publishers, in press. Click here for a PDF version (Adobe Acrobat) of this paper.

Libarkin, J.C., and Anderson, S.W., 2006b. Science concept inventory development in higher education: A mixed-methods approach in the geosciences; Journal of Research in Science Teaching, in prep. (Note!! This is in preparation, but does contain a more detailed description of the steps involved in the GCI development than in the other papers.  Please note that this has not been submitted yet, so reference accordingly) Click here for a PDF version (Adobe Acrobat) of this paper.

Libarkin, J.C., Kurdziel, J.P., and Anderson, S.W., 2006. College student conceptions of geologic time and the disconnect between ordering and scaling; Journal of Geoscience Education, in review (we will post a pre-print once the paper is accepted)

Libarkin, J.C., Anderson, S.W., *Dahl, J., *Beilfuss, M., Boone, W., and Kurdziel, J.P., 2005. Qualitative analysis of college students' ideas about the Earth: Interviews and open-ended questionnaires; Journal of Geoscience Education, v. 53, p. 17-26.  Click here for a PDF version (Adobe Acrobat) of this paper.

Dahl, J., Anderson, S.W., and Libarkin, J., 2005; Digging into Earth science: Alternative conceptions held by K-12 teachers; Journal of Science Education,  v. 12, p. 65-68. Click here for a PDF version (Adobe Acrobat) of this paper.

Libarkin, J.C., and Anderson, S.W., 2005. Assessment of Learning in Entry-Level Geoscience Courses: Results from the Geoscience Concept Inventory; Journal of Geoscience Education; v. 53. p. 394-401. Click here for a PDF version (Adobe Acrobat) of this paper.

Abstracts

Libarkin, J.C., and Anderson S.W., 2006. Applications of Rasch measurement to assess conceptual understanding in science; NARST 2006 Annual Meeting Abstracts, in press

Libarkin, J.C., and Anderson S.W., 2005. Assessment of diverse geoscience courses through Rasch development of geoscience concept inventory sub-tests: GSA Abstracts with Programs, v. 37.

Libarkin, J.C., and Anderson S.W., 2004, A Tale of Three Theories: Developing and Implementing the Geoscience Concept Inventory: National Science Foundation-Course, Curriculum, and Laboratory Improvement Principal Investigator's meeting (Washington, D.C.: April 16-17, 2004).

Anderson, S.W., and Libarkin, J., 2003. The retention of geologic misconceptions: Alternative ideas that persist after instruction; EOS, Transactions of the American Geophysical Union, v. 84, Abstract ED22E-07.

Dahl, J., Anderson, S.W., and Libarkin, J., 2003. Digging into earth science: Teachers’ alternative conceptions in the geosciences: EOS, Transactions of the American Geophysical Union, v. 84, Abstract ED31B-1166.

Libarkin, J., and Anderson, S.W., 2003. The Geoscience Concept Test: Linking grounded theory, scale development, and item response theory; EOS, Transactions of the American Geophysical Union, v. 84, Abstract ED22E-06.

Boone, W., Libarkin, J.C., Anderson, S., and Beilfuss, M., 2003. Exploring Earth science misconceptions of introductory and non-science majors through the development of the GCT (Geoscience Concept Test), National Association for Research in Science Teaching, 3/23-3/26/2003.

Boone, W.J., Libarkin, J.C., Beilfuss, M., Anderson, S.W., 2003. Exploring Earth-science misconceptions; Hasti--Hoosier Association of Science Teachers Conference. 

Lockhart, K., and Anderson, S.W., 2003. Misleading ideas in geology among second and sixth grade students; Black Hills State University Research Symposium, April, 2003.

Anderson, S.W., Libarkin, J.C., Beilfuss, M., Dahl, J., and Boone, W.J., 2002. Analysis of college students’ ideas about geologic time: Questionnaires and interviews from four institutions; EOS, Transactions of the American Geophysical Union, v.83, p. S115.

Libarkin, J.C., Anderson, S.W., Boone, W.J., Beilfuss, M., and Dahl, J., 2002, The Geoscience Concept Test: A new assessment tool based on student misconceptions; EOS, Transactions of the American Geophysical Union, v 83, Abstract ED11B-0047.

Libarkin, J.C, and Anderson, S., 2001. Developing an Assessment Tool for Geology: Stage One, Student Interviews; EOS, Transactions of the American Geophysical Union, v. 82, p. F250.

Presentations

Libarkin and Anderson, 2005.  GSA presentation on Rasch Analysis

Funding

This project was made possible through a grant from the National Science Foundation's Assessment of Student Achievement program ("Conceptual Understanding of Three Dimensions of Earth Processes in General Education and Introductory Courses: Test Development and Validation", NSF-DUE 0127765 to Julie Libarkin and Steve Anderson)

Contact Information