GOV 1010: Survey Research Methods

 

GOV 1010: Survey Research Methods

Harvard College/GSAS: 119479

Fall 2019

Meeting Time: Monday, Wednesday 12:00 pm - 1:15 pm

Meeting Location: Northwest Science Center B-103. 

Note on Location: The Northwest Science Center B-103. 

 

Note: Northwest Science Center B-103 and B-100 are in the lower (below-ground) level of the Northwest Science Center.  The center can be reached from Oxford St. or via Divinity Ave. We will meet in B-103 for lectures, but have reserved access to the Atrium (B-100) for group activities.

 

 

QRD: This course has been approved to meet the Quantitative Reasoning with Data requirement for Harvard College. 

 

Instructor and Teaching Fellow Contact Information 

Instructor: Teaching Fellows:

Chase H. Harrison

Senior Preceptor in Survey Research,

Department of Government, FAS

Associate Director, Program on Survey Research, IQSS

E-mail: charrison@gov.harvard.edu

Office: CGIS, S-221 (1730 Cambridge St.)

Office hours: Wednesdays, 2:00 - 4:00 pm and by appointment

Mark Hill 

PhD Candidate,

Department of Government, 

Harvard University

Email: markhill@g.harvard.edu

Office hours: TBD and by appointment

 

Riley Carney (rcarney@g.harvard.edu)

Olivia Woldemikael (woldemikael@g.harvard.edu)

Ze Fu (zefu@g.harvard.edu)

 

 

Course Description

Surveys and polls provide much of the data we use to understand the world around us. They are fundamental to the study of public opinion and behavior, and are used a diverse variety of fields, including those related to business, economics, education, public health, international development, and others. 

This course is designed to teach students the skills necessary to implement, use, and interpret survey data. By the end of the course, students should be able to critically assess the quality of survey data, as well as be able to  design and conduct an original survey.

This course does not assume any prior knowledge of statistical methods.

This course incorporates both reading material and hands-on practice in survey design. The course format will include lectures, discussions, hands-on workshops, and student presentations. For the survey project, students will develop, implement, and analyze a survey on a topic of their own choosing.

 

Readings

*The following books have been ordered for your purchase in the Harvard Coop.

**Online versions available via Hollis.

Required: 

Dillman, Don A. Jolene Smyth and Leah Melani Christian. Internet, Phone, Mail, and Mixed Mode Surveys; The Tailored Design Method 4th Edition (Wiley 2014).**

Groves, Robert M. et. al. Survey Methodology, Hoboken, NJ.: Wiley (2006).**

Blair, Edward and Johnny Blair. Applied Survey Sampling. Los Angeles: Sage (2015)

 

 *Most other readings will be available through the course web page or on reserve in Lamont library. 

Supplementary Reading:

There are many other books about survey design that may be useful to you as you design and implement your research project. The most essential of these have been placed on reserve in the Lamont Library. 

Books of particular interest may include:

Couper, Mick P. Designing Effective Web Surveys. Cambridge: Cambridge University Press. (2008). This is a handbook for designing web surveys that many students find helpful.

Fowler, Floyd J. Jr., Improving Survey Questions; Design and Evaluation. Thousand Oaks, CA: Sage Publications. (1995) This is a handbook for writing survey questions that many students find helpful.

 

 

Course Structure

Survey research is a collaborative process. The best survey research almost always happens when people work together, sharing some tasks and delegating others. Survey research done by soloists usually falls flat. In that spirit, students will work in groups to design and conduct surveys for the course.

In addition to attending classroom meetings, students will be expected to meet regularly to design and implement their group project.

Each group assignment will ask students to list the group participants and the work each contributed to the assignment. At the end of the semester, students will be asked to independently write a short assessment of the roles each student played in the group, which will be handed in along with the final paper. The final paper for the class will be based on your group project. Although students are encouraged to share ideas during the survey process, each student must write their final paper individually.

 

Grades

Grades will be calculated as follows:

Requirement: %

Description:

Class Participation  10% 

 

Students are expected to attend lectures, participate in activities as appropriate, and to be able to report on their group projects during each class. Class participation includes attending class, participating in class as appropriate, and participating in short pre and post class assignments. 

 

Group Participation 10%

Students are expected make valuable contributions to the final group project, and to attend a total of 2-3 (depending on progress) mandatory group meetings scheduled when it is convenient for the group and the TF to meet throughout the semester.

Class Assignments 30%

 

Assignment 1: Preliminary Research Question (5%) Due Sept. 16 before class

Assignment 2: Group Research Proposal (10%) Due Oct. 7 before class

Assignment 3: Questionnaire (Group) (5%) Due Oct. 23 before class

Assignment 4: Pre-Test report (Group) (10%) Due Nov. 6 before class

 

Survey Critique 10%

You will get more out of this course, and enjoy it more if you are actively searching for and critiquing actual survey results.  Each student will be responsible for writing two short (2 – 4 page) critiques of a survey, survey question, poll, or interpretation of a survey.  The critique should focus on one particular aspect or issue of the report and should be some aspect of survey methodology covered in the course.  You should provide a copy of (or link to) to the relevant survey report or release you discuss, although any relevant parts for your discussion should be presented in your paper. 

 

The first critique must be received by the start of class October 16, and the second is due by the start of class on November 25.  Critiques handed in after these deadlines will be penalized ½ letter grade (5 / 100 points) for each day or portion thereof ( 1 – 24 hours) it is late.  Students who complete two critiques by November 13 will have the option of writing a third critique by November 25.  In this case, the critique with the lowest grade will be dropped. 

In-Class Examination 10% November 20, 2019
Survey Research Project/Paper 30%

 

DUE: Friday, December 13, at 9:00 AM

For the survey project, students will develop, implement, and analyze their own survey on any topic of their choosing. The final paper (12-15 pages, double-spaced, 12-point, 1-inch margins) is due Friday, December 13, at 9:00 am.

Class assignments will be geared toward the final project, and preliminary versions will be presented during class sessions at the end of the term.

 

TOTAL: 100% 

 

*Note: Failure to complete any course requirement may result in failure of the entire course.

 

Course Practices and Policies

Policy on Collaboration: Collaboration is Required

Participation in group projects is required as part of this course unless a specific exception is granted by the instructor. Each group assignment will ask students to list the group participants and the work each contributed to the assignment. At the end of the semester, students will be asked to independently write a short assessment of the roles each student played in the group, which will be handed in along with the final paper.

The final paper for the class will be based on your group project. Although students are encouraged to share ideas during the survey process, each student must write their final paper individually. The final paper should reflect your own presentation and interpretation of the group project. Final papers should properly cite any books, articles, websites, lectures, etc. that have contributed to the work. Any outside help with the paper should be fully acknowledged.

The single classroom exam (10% of grade) is to be completed by the student alone, without any assistance, and without access to any outside materials. 

Accessibility, Services, and Support. Any student needing academic adjustments or accommodations is requested to present their letter from the Accessible Education Office (AEO) and speak with me by the end of the second week of the term. Failure to do so may result in my inability to respond in a timely manner. All discussions will remain confidential, although AEO may be consulted to discuss appropriate implementation. For more information, please contact me or AEO at 617-496-8707 (V), 617-496-3720 (V/TTY), 617-495-0815 (fax), aeo@fas.harvard.edu or www.aeo.fas.harvard.edu.

Late Assignments. Students should take careful note of the dates requirements are due. All assignments are due on the designated day. Late assignments will be accepted at the discretion of the instructor, and will be marked down a full letter grade for each class meeting late. For the final paper assignment, late papers will NOT be accepted.

Absences. If any student must miss a class or due date owing to a religious observance, please inform me during the first weeks of class so that we can make appropriate arrangements. All students are permitted ONE no-questions-asked absence from lecture.

Availability. I will make every effort to be available to students outside of class. My regular office hours are listed on the first page of the syllabus. If you wish to come at a different time, please don’t hesitate to contact me.

 

Tentative Schedule

Note: In order to best accommodate student projects, scheduling -- including due-dates of assignments -- is subject to change.  (Group assignments will not be due earlier than dates listed on this syllabus).  

Week Dates Topic & Readings
Week 1  

September 4;

 

Course Introduction on Survey Research and Survey Errors

 

Readings: Readings can be found in "Files" tab

  • Groves: Chapter 1
  • Dillman: pp 2 – 10
  • Newport, Frank. Polling Matters: Why Leaders Must Listen to the Wisdom of the People. New York: Time Warner (2004). Chapter 1 and Chapter 7.
Week 2

September 9; September 11

Surveys and the Research Process 

*Assignment 1: Preliminary Research Question, Due TBD

 

Readings: Readings can be found in "Files" tab

  • Groves, et. al: pp 39 – 48
  • Fowler: Chapter 11
  • Oppenheim, A. N. Questionnaire Design, Interviewing, and Attitude Measurement. London: Pinter (1992). Pp. 7-18; 21-29.
Week 3 September 16; September 18

Assignment 1 Due on September 16 by 12 pm on Canvas. 

 

Modes of Data Collection

Readings: Readings can be found in "Files" tab

  • Groves: Chapter 5
  • Dillman: pp 10 – 17; Chapter 2; pp 258-264 ; p298 ; pp 301-306 ; pp 351-353 ; pp 398-403; Chapter 12
Week 4 September 23; September 25

Sampling and Populations

 Readings: 

  • Groves, et. al. Chapter 3
  • Blair and Blair: Chapters 1 - 3 (pp 3 - 88); Chapter 8 (pp 179-219) 
  • Dillman, Smith, Christian Chapter 3, pp 56-75 
Week 5

 

September 30,

October 2;

 Designing Sampling Protocols

Readings: 

  • Blair and Blair: Chapters 4 - 6 (pp 91=154)
  • Dillman, Smith, Christian: Chapter 3, pp 75-93
  • Optional: Groves Chapter 4  

For Reference:

  • Brick, J. Michael (2011) “The Future of Survey Sampling” POQ: 75 (5): 872-888 
  • Yeager, David S., et. al. (2011) "Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples." POQ: 75 (4): 709- 747
  • Iannacchione, Vincent G. (2011) "The Changing Role of Address-Based Sampling in Survey Research." POQ: 75 (3): 556-575
Week 6 October 7; October 9

Conceptualizing Ideas and Operationalizing Questions 

*Assignment 2: Group Research Design, Due October 7 by 12 pm on Canvas

Readings:

  • Dillman, Smith and Christian: Chapters 4 - 5
  • King, Gary, Robert O. Keohane and Sidney Verba. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton University Press (1994). pp 55-63.
  • Oppenheim, A. N. Questionnaire Design, Interviewing, and Attitude Measurement. London: Pinter (1992). pp. 150-162
  • Hox, Joop J. 1997. “From Theoretical Concept to Survey Question.” Pp. 47-70 in Survey Measurement and Process Quality, edited by L. Lyberg et al. New York: Wiley.

Recommended:

Fowler, Floyd J. Jr., Improving Survey Questions; Design and Evaluation. Thousand Oaks, CA: Sage Publications. (1995)

 

Week 7

 

 

October 16;

Note: October 14 no class due to Columbus Day

 

Thinking About Answers & Questions 

*First survey critique due October 16 by 12 pm on Canvas

Readings:

  • Dillman, Smith and Christian: Chapters 6 - 7
  • Groves: Chapter 7
  • Schuman: Chapter 4
  • Schaeffer, Nora Cate and Jennifer Dykema (2011) "Questions for Surveys: Current Trends and Future Directions." POQ: 75 (5): 909-961
  • Krosnick, Jon A. and Leandre R. Fabrigar. 1997. “Designing Rating Scales for Effective Measurement in Surveys.” pp. 141-164 in Survey Measurement and Process Quality, edited by L. Lyberg et al. New York: Wiley.

Optional Readings:

Students designing surveys to be implemented by interviewers might refer to the following as a reference:

  • Groves et. al., Chapter 9
  • Weisberg Chapter 4
  • Fowler, Floyd J. 2014. Survey Research Methods, Fifth Edition. (Loos Angeles: Sage. Chapter 8;
Week 8  October 21; October 23

Pre-Testing Questions

*Assignment 3: Group Questionnaire, Due October 23 by 12 pm on Canvas

Readings:

  • Groves et. al., Chapter 8
  • Fowler, Floyd J. 2014. Survey Research Methods, Fifth Edition. (Los Angeles: Sage. Chapter 7
  • Presser et. al (2004) "Methods for Testing and Evaluating Survey Questions" POQ 68: 109-130
  • Willis, Gordon B. 1999. "Cognitive Interviewing: A 'How To' Guide."
  • Beatty, Paul C. and Gordon B. Willis. 2007. “Research Synthesis: The Practice of Cognitive Interviewing.” Public Opinion Quarterly 71(2): 287-311.

Week 9 October 28; October 30

The Impact of Nonresponse 

Readings: 

  • Groves et. al. Chapter 6
  • Groves et. al. Chapter 10  pp 347 - 359
  • Wagner, James. (2012) "A Comparison of Alternative Indicators for the Risk of Nonresponse Bias." POQ: (2012) 76 (3): 555-575

 

 

Week 10

 

November 4; November 6

Interpreting Survey Findings 

Readings: 

  • Schuman Chapter 5
  • Groves, Robert M. and Lars Lyberg (2010) "Total Survey Error: Past, Present, and Future." POQ: 74 (5): 849-879

 

 

Week 11 November 11; November 13

Special Topics: Pre-election polling

*Assignment 4: Final Questionnaire & Pre-test Report (Group) Due November 11 by 12 pm on Canvas

Readings to be assigned.

*Second survey critique due November 13 by 12 pm on Canvas if interested in submitting 3 and dropping the critique with the lowest grade.

Week 12

November 18

November 20

Readings to be assigned

* In-class examination November 20, 2019  

 

November 25
*Second Survey Critique due November 25 by 12 pm on Canvas  

 

November 27 Thanksgiving Recess: No Class

 

 

 

Week 13

November 27;

November 29

 

 

Week 14 December 2

Course Wrap-Up

 Final Presentations & Preliminary Results

December 4; December 9

 Reading Period 

December 13 

Final Paper, Due at  9:00 AM 

*Extended office hours & tutorials available

    

Please contact Instructor or TF with any additional questions.

 

Course Summary:

Date Details Due