Summary

Educational games and simulations, MOOCs, intelligent tutoring systems, and other learning environments provide large quantities of rich data on learning processes and outcomes. We now have more information than ever about what learners are doing and how they are thinking. However, the sheer volume of this data can lead to statistically significant but meaningless quantitative results, while at the same time rendering qualitative methods unfeasible. Quantitative Ethnography is a methodology that blends qualitative and quantitative approaches into a solution for overcoming the weaknesses of traditional methods when applied to big data. Quantitative Ethnography views big data—and data more generally—as evidence about the discourse of particular cultures. To make meaning from this evidence, and thus gain some understanding of the culture, researchers attempt to achieve what Geertz called a qualitatively "thick" description of the data. However, the more data we have, the more difficult this process is. Quantitative Ethnography addresses this problem by using statistical techniques to warrant claims about the quality of thick description. The result is a more unified mixed methods approach that uniquely links the evidence we collect to cultural phenomena of interest, such as learning.

Video Resources

Watch the full webinar on Tools of Quantitative Ethnography: Epistemic Network Analysis and nCoder featuring David Williamson Shaffer:

Comprehensive Tutorial on Epistemic Network Analysis Part #1 by David Williamson Shaffer:

Comprehensive Tutorial on Epistemic Network Analysis Part #2 by David Williamson Shaffer:

Reading:

  • Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., ... & Mislevy, R. (2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media 1(2), 33-53. [Access Online]

Additional Reading:

  • For the most comprehensive coverage of quantitative ethnography, see: Shaffer, D.W. (2017). Quantitative Ethnography. Madison, WI: Cathcart Press.
  • Shaffer, D.W. (in press). Big data for thick description of deep learning. In K. Millis, D. Long, J. Magliano, and K. Weimer (Eds.), Deep learning: Multi-disciplinary approaches. NY, NY: Routledge/Taylor Francis. [Access Online]
  • Shaffer, D.W. (2018). Transforming big data into meaningful insights: Introducing quantitative ethnography. Scientia. [Access Online]
  • Shaffer, D.W., Borden, F., Srinivasan, A., Saucerman, J., Arastoopour, G., Collier, W., Ruis, A.R., & Frank, K. (unpublished). The nCoder: A Technique for Improving the Utility of Inter-Rater Reliability Statistics.
  • Shaffer, D.W., Collier, W., & Ruis, A.R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. [Access Online]
  • Shaffer, D.W. & Ruis, A.R. (2017). Epistemic network analysis: A worked example of theory-based learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Grasevic (Eds.), Handbook of Learning Analytics (pp. 175–187). Society for Learning Analytics Research. [Access Online]

Other links & resources:

Learning Scientists Who Have Researched This Topic:

  • Benjamin Allsopp
  • Amanda Barany
  • Matthew Berland
  • Sagit Betser
  • Edith Bouton
  • Kritsa Chindanon
  • Allan Collins
  • Rianne Conjin
  • Charles Crook
  • Andras Csanadi
  • Chris Dede
  • Deepa Deshpande
  • Gideon Dishon
  • Nadav Ehrenfeld
  • James Folkestad
  • Aroutis Foster
  • Ken Frank
  • Judi Fusco
  • Dragan Gasevic
  • James Paul Gee
  • Art Graesser
  • Seung Yon Ha
  • Eric Hamilton
  • Leslie Herrenkohl
  • Srecko Joksimovic
  • David Joyner
  • Yanghee Kim
  • Simon Knight
  • Seung Bok Lee
  • Adam Lefstein
  • Chi Un Lei
  • Kristine Lund
  • Leanne Ma
  • Beth MacDonald
  • Hiroyuki Masukawa
  • Wannisa Matcha
  • Camilla Matuk
  • Morten Misfeldt
  • Kamila Misiejuk
  • Toshio Mochizuki
  • Alison Mroczkowski
  • Larry Nucci
  • Diler Oner
  • Jun Oshima
  • Ritsuko Oshima
  • Natasa Pantic
  • Melanie Peffer
  • Anthony Pellicone
  • Carla M. Pugh
  • Peter Reimann
  • Emmy Royse
  • David Williamson Shaffer
  • Elnara Shakarova
  • Simon Buckingham Shum
  • Jin Micheal Splichal
  • Sarah Sullivan
  • Yi-Shan Tsai
  • Nora'Ayu Ahmad Uzir
  • Sally Wai-Yan Wan
  • Nicole Werner
  • Bian Wu
  • Szilvia Zörgő

 

Member Log In

Please note that we have updated our website & all prior ISLS members must create a new profile

For information regarding membership renewal or changing membership plans click here

Join ISLS

Please click here to become a member

NEW!! NAPLeS Resources


For any changes or additions to NAPLeS content please email us at naples@isls.org