Below is a reading list, which serves as a reference guide about human-in-the-loop systems and highlights various topics that are covered in the course.
A. Interactive Machine Learning
-
Fails, J. A., and Olsen Jr, D. R. 2003. Interactive machine learning. In Proceedings of the 8th International Conference on Intelligent User Interfaces, pg 39-45.
-
Fiebrink, R., D. Trueman, and P. R. Cook. A meta-instrument for interactive, on-the-fly machine learning. Proceedings of New Interfaces for Musical Expression (NIME), 2009.
- Power to the People: The Role of Humans in Interactive Machine Learning" by Saleema Amershi, Maya Cakmak, W. Bradley Knox, Todd Kulesza. AAAI Magazine, 2013.
-
Kapoor, A., Lee, B., Tan, D., and Horvitz, E. 2010. Interactive optimization for steering machine classification. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pg. 1343-1352.
B. Human-in-the-Loop Supervised Learning
- "Minimally-Supervised Learning" by Sanjoy Dasgupta
- "Active Learning"by Burr Settles, Morgan & Claypool, 2012.
- Maya Cakmak, Crystal Chao, and Andrea L. Thomaz. Designing Interactions for Robot Active Learners. IEEE Transactions ON Autonomous Mental Development, 2(2), 2010.
-
S. Vijayanarasimhan and K. Grauman. What's it going to cost you? Predicting effort vs. informativeness for multi-label image annotations. In CVPR, 2009.
-
B. Settles, M. Craven and L. Friedland. Active Learning with Real Annotation Costs. In Proceedings of NIPS Workshop on Cost-Sensitive Learning, 2008.
-
Branson S., Wah C., Babenko B., Schroff F., Welinder P., Perona P., Belongie S., Visual Recognition with Humans in the Loop. European Conference on Computer Vision (ECCV), 2010.
C. Human-Robot Interaction
D. Human Computation and Crowdsourcing
- "Human Computation" by Edith Law and Luis von Ahn, Morgan & Claypool 2011.
- Greg Little, Lydia B. Chilton, Max Goldman, and Robert C. Miller. Exploring Iterative and Parallel Human Computation Processes. In HCOMP 2010.
- E. Law and L. von Ahn. Input-agreement: A New Mechanism for Data Collection using Human Computation Games. In CHI 2009.
- H. Zhang, E. Law, K. Gajos, E. Horvitz, R. C. Miller and D. Parkes. Human Computation Tasks with Global Constraints: A Case Study. In CHI 2012.
- Rajan Vaish, Keith Wyngarden, Jingshu Chen, Brandon Cheung, Michael Bernstein. Twitch Crowdsourcing: Crowd Contributions in Short Bursts of Time. In CHI 2014
-
Omer Tamuz, Ce Liu, Serge Belongie, Ohad Shamir, and Adam Kalai. Adaptively Learning the Crowd Kernel. In ICML, 2011.
-
Panos Ipeirotis and Evgeniy Gabrilovich. Quizz: Targeted Crowdsourcing with a Billion (Potential) Users. In WWW 2014.
-
Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatrowicz, A., White, B., White, S., & Yeh, T. VizWiz: nearly real-time answers to visual questions. In UIST 2010.
-
Daniel S. Weld., Mausam, and Peng Dai. Human Intelligence Needs Artificial Intelligence. In HCOMP 2011.
-
Courses at Other Universities:
E. Mixed-Initiative Systems
- Hoffmann, R., Amershi, S., Patel, K., Wu, F., Fogarty, J., & Weld, D. S. (2009). Amplifying community content creation with mixed initiative information extraction. Proceedings of the 27th International Conference on Human Factors in Computing Systems, 1849-1858.
- "Principles of Mixed-Initiative User Interfaces" by Eric Horvitz, CHI 1999.
- A Framework for Mixed-Initiative Clustering. Yifen Huang and Tom Mitchell. NESCAI 2007, Ithaca, NY, April 2007.
- Shilman, M., Tan, D., & Simard, P. (2006). CueTIP: a mixed-initiative interface for correcting handwriting errors. UIST '06: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology.
F. Brain-Human Interfaces
- Shenoy, P., Tan, D. S., & Tan, D. (2008). Human-aided computing: utilizing implicit human processing to classify images. CHI '08: Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems.
-
E.T. Solovey, P. Schermerhorn, M. Scheutz, A. Sassaroli, S. Fantini, R.J.K. Jacob. Brainput: Enhancing Interactive Systems with Streaming fNIRS Brain Input. In CHI 2012.
-
Szafir, D. and Mutlu, B. Pay attention! Designing Adaptive Agents that Monitor and Improve User Engagement. In CHI 2012.
G. Human-Agent Teamwork
H. Human-in-the-loop Security
- A Framework for Reasoning About Human in the Loop by Lori Cranor.
-
I. Flechais, J. Riegelsberger and A. Sasse. Divide and Conquer: The Role of Trust and Assurance in the Design of Secure Socio-Technical Systems. In 2005 Workshop on New Security Paradigm 2005.
-
E. Choe, S. Consolvo, J. Jung, B. Harrison, S. Patel and J. Kientz. Investigating Receptiveness to Sensing and Inference in the Home using Sensor Proxies. In Ubicomp 2012.
-
S. Uellenbeck, M. Durmuth, C. Wolf, and T. Holz. Quantifying the Security of Graphical Passwords: The Case of Android Unlock Patterns. In Proceedings of the 2013 ACM SIGSAC conference on Computer & Communications Security, 2013. (CCS '13)
-
C. Bravo-Lillo, L. Faith Cranor, J. Downs, S. Komanduri, R. W. Reeder, S. Schechter, and M. Sleeper. Your Attention Please: Designing security-decision UIs to make genuine risks harder to ignore. In Proceedings of the Ninth Symposium on Usable Privacy and Security, 2013. (SOUPS '13)
I. Research Methods in Human-in-the-loop Systems
- Gergle, D., & Tan, D. (2014). Experimental Research in HCI. In Ways of Knowing in HCI. Eds: J.S. Olson and W. Kellogg. Springer.
J. Human-Automation Interaction
-
V. Riley. A General Model of Mixed-Initiative Human-Machine Systems. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting October 1989 vol. 33 no. 2 124-128.
-
R. Parasuraman and V. Riley. Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors: The Journal of the Human Factors and Ergonomics Society 1997 39: 230.
-
N.B. Sarter, D. D. Woods, and C.E. Billings. Automation Surprises. Handbook of Human Factors & Ergonomics, second edition, G. Salvendy (Ed.), Wiley, 1997.
-
R. Parasuraman, T. B. Sheridan and C. D. Wickens. A Model for Types and Levels of Human Interaction with Automation. In IEEE Transactions On Systems, Man and Cybernetics - Part A: Systems and Humans, Vol 30, No. 3, May 2000.
-
C. D. Wickens, H. Li, A. Santamaria, A. Sebok and N. B. Sarter. Stages and Levels of Automation: An Integrated Meta-analysis. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2010 54: 389.
-
D. Kaber, J. Riley, K. Tan and M. Endsley. On the Design of Adaptive Automation for Complex Systems. International Journal of Cognitive Ergonomics, 2001, 5(1), 37--57.
-
H. Furukawa and R. Parasuraman. Supporting System-Centered View of Operators Through Ecological Interface Design: Two Experiments on Human-Centered Automation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2003 47: 567.
K. Ethics