Quantitative Ethnography


Contributor: David Williamson Shaffer

Educational games and simulations, MOOCs, intelligent tutoring systems, and other learning environments provide large quantities of rich data on learning processes and outcomes. We now have more information than ever about what learners are doing and how they are thinking. However, the sheer volume of this data can lead to statistically significant but meaningless quantitative results, while at the same time rendering qualitative methods unfeasible. Quantitative Ethnography is a methodology that blends qualitative and quantitative approaches into a solution for overcoming the weaknesses of traditional methods when applied to big data. Quantitative Ethnography views big data—and data more generally—as evidence about the discourse of particular cultures. To make meaning from this evidence, and thus gain some understanding of the culture, researchers attempt to achieve what Geertz called a qualitatively “thick” description of the data. However, the more data we have, the more difficult this process is. Quantitative Ethnography addresses this problem by using statistical techniques to warrant claims about the quality of thick description. The result is a more unified mixed methods approach that uniquely links the evidence we collect to cultural phenomena of interest, such as learning.

These abstracts are living documents provided and modified by NAPLeS members. The more contributions we have towards refining the documents the more representative they will be of the community as a whole. If you would like to suggest revisions, please contact [email protected].

Video Resources

Listen to the Automated Collaborative Process Analysis webinar


Basic Reading:
  • Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., … & Mislevy, R. (2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media 1(2), 33-53. [Access Online]
Additional Reading:
  • For the most comprehensive coverage of quantitative ethnography, see: Shaffer, D.W. (2017). Quantitative Ethnography. Madison, WI: Cathcart Press.
  • Shaffer, D.W. (in press). Big data for thick description of deep learning. In K. Millis, D. Long, J. Magliano, and K. Weimer (Eds.), Deep learning: Multi-disciplinary approaches. NY, NY: Routledge/Taylor Francis. [Access Online]
  • Shaffer, D.W. (2018). Transforming big data into meaningful insights: Introducing quantitative ethnography. Scientia. [Access Online]
  • Shaffer, D.W., Borden, F., Srinivasan, A., Saucerman, J., Arastoopour, G., Collier, W., Ruis, A.R., & Frank, K. (unpublished). The nCoder: A Technique for Improving the Utility of Inter-Rater Reliability Statistics.
  • Shaffer, D.W., Collier, W., & Ruis, A.R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. [Access Online]
  • Shaffer, D.W. & Ruis, A.R. (2017). Epistemic network analysis: A worked example of theory-based learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Grasevic (Eds.), Handbook of Learning Analytics (pp. 175–187). Society for Learning Analytics Research. [Access Online]
Additional Links & Resources:

Learning Scientists Who Have Researched This Topic

  • Benjamin Allsopp
  • Amanda Barany
  • Matthew Berland
  • Sagit Betser
  • Edith Bouton
  • Kritsa Chindanon
  • Allan Collins
  • Rianne Conjin
  • Charles Crook
  • Andras Csanadi
  • Chris Dede
  • Deepa Deshpande
  • Gideon Dishon
  • Nadav Ehrenfeld
  • James Folkestad
  • Aroutis Foster
  • Ken Frank
  • Judi Fusco
  • Dragan Gasevic
  • James Paul Gee
  • Art Graesser
  • Seung Yon Ha
  • Eric Hamilton
  • Leslie Herrenkohl
  • Srecko Joksimovic
  • David Joyner
  • Yanghee Kim
  • Simon Knight
  • Seung Bok Lee
  • Adam Lefstein
  • Chi Un Lei
  • Kristine Lund
  • Leanne MaBeth MacDonald
  • Hiroyuki Masukawa
  • Wannisa Matcha
  • Camilla Matuk
  • Morten Misfeldt
  • Kamila Misiejuk
  • Toshio Mochizuki
  • Alison Mroczkowski
  • Larry Nucci
  • Diler Oner
  • Jun Oshima
  • Ritsuko Oshima
  • Natasa Pantic
  • Melanie Peffer
  • Anthony Pellicone
  • Carla M. Pugh
  • Peter Reimann
  • Emmy Royse
  • David Williamson Shaffer
  • Elnara Shakarova
  • Simon Buckingham Shum
  • Jin Micheal Splichal
  • Sarah Sullivan
  • Yi-Shan Tsai
  • Nora’Ayu Ahmad Uzir
  • Sally Wai-Yan Wan
  • Nicole Werner
  • Bian Wu
  • Szilvia Zörgő