Artificial intelligence (AI) systems are becoming more prevalent, with important applications in business, education, law, and medicine. Intelligent learning systems of the future are likely to require ongoing, human-in-the-loop teaching. Our research aims to create new technologies that allow people to teach machines implicitly/offline or explicitly/online, and in such a way that is mutually beneficial to both humans and machines. In the Teachable Robot Project, we developed a machine teaching platform, called the Curiosity Notebook, that allows groups of students to read educational texts related to a topic, and teach the concepts (e.g., how to classify rocks/minerals) to a conversational agent acting as a less knowledgeable peer.